Case study
How I built this
I'm Nick Castro. The Other Data is a fully automated pipeline I designed and shipped, working with AI assistants, that turns free public data into plain-English stock signals. Here's the problem, how it works, and the tradeoffs I made.
- 35curated stocks
- 111live signals
- 14data sources
- 4hrefresh cadence
- 1,051backtested calls
- 49%1-month hit rate (honest)
- 77automated tests
The problem
Alternative data (airport traffic, shipping, app installs, job postings) is powerful, but it's opaque and expensive. Funds pay big money for it, and regular people rarely see it explained.
Who it's for
Curious retail investors and learners who want to understand why a number might matter to a company, in plain English, without hype.
How it works
- Scheduled pipeline. A GitHub Actions workflow runs every 4 hours. It pulls FRED, EIA, SEC EDGAR, TSA, openFDA, NHTSA, Wikipedia pageviews, pypistats and npm downloads, Hacker News, and public job boards, plus Finnhub news and Tiingo prices. Then it commits the results as static JSON.
- Leans. Each signal is compared with its own recent past or last year. Big enough moves become a tailwind or headwind, and the signals roll up into one lean per stock.
- AI summaries with guardrails. Gemini writes a short plain-English summary from the numbers on the page. An advice filter rejects buy/sell language, and a consistency validator rejects any summary that disagrees with the page's lean.
- Correlation scout. A monthly job tests offbeat public series against stock prices and writes up the most promising ideas, with clear caveats.
- Share cards. At build time, every stock and signal gets its own 1200×630 social card and its own link, so shares unfurl nicely on X, LinkedIn, and Facebook.
- Weekly digest. A Vercel cron job builds a weekly email and sends it through Resend to confirmed subscribers only (double opt-in, one-click unsubscribe).
- Hosting. Static site plus a few serverless functions on Vercel. Every data commit redeploys automatically.
Key decisions and tradeoffs
- Leans, not buy/sell. The site explains which way a clue points. It never tells anyone what to trade. It's education only.
- Publish the honest backtest. Across past checkpoints the leans called the 1-month direction right about as often as a coin flip. I show that openly instead of cherry-picking winners.
- Test improvements out of sample, and say when they fail. I tried weighting the signals that had worked, muting the ones that hadn’t, a conviction threshold, and five new free data sources, choosing everything for each year using only earlier years. None beat a coin flip by more than chance, so the live method stayed the same and the full test is on the track record page.
- Drop Wikipedia bot spikes. One day, "Roblox" got about 4.8 million views versus roughly 7,000 normally. Isolated one-day bursts are now ignored so bots can't fake a surge.
- Prices vs volume. Cardboard box prices can rise because costs rose, not because more boxes shipped. So Amazon now leans on Federal Reserve paperboard output, and box prices are shown only as context.
- Static and free. Precomputed JSON, build-time images, and free API tiers keep it fast, cheap, and hard to break.