Read, Compute, Watch: What a Chat Window Can't Do

You brought your biases. The LLM brought its own.

The first time I used ChatGPT a few years ago, I was blown away and marvelled at its certainty. It reminded me of my daughters when they were teenagers.

The human-like confidence LLMs show can fool you. AI researcher Andrej Karpathy popularized the term “Jagged Intelligence”: they do some things extremely well, and others not so much. The not-so-much part is what can hurt you when it comes to investing.

Human certainty usually has a bias behind it: recency, anchoring, availability, or confirmation. In my own experience, I’d give a different answer on the same stock depending on which one was loudest that day.

The LLM has biases too, just different ones. It writes most fluently about the companies most written about, so its musings skew to what you already know. It leans toward whatever your question implied. It over-weights whatever is loud online this week.

You went to the chat window to escape your own biases. You found a different set.

A rules-based strategy has biases too. Whoever wrote it picked the thresholds. The difference is that a strategy's biases are written down, fixed, repeatable, and testable. It’s not vibes; it’s logic you can backtest. You can't backtest your mood, and you cannot backtest an LLM prompt.

Stock investing asks three things of any tool: 

• Read everything that matters, every day
• Compute it the same way every time
• Watch for the moment the rule fires, and signal
  
  

While an LLM is incredible at language processing, by itself it isn’t the right tool for repeatable data collection, processing, and signaling. 

How an LLM thinks

It generates. It doesn't compute. An LLM produces the next most plausible word, then the next. A paragraph about revenue growth comes out of the same process as a paragraph about a recipe. Nothing inside it adds two numbers unless it stops and runs a program.

It reads the web now, and reading isn't screening. The current models fetch filings and today's prices. That kills the old complaint that they're out of date. But it reads what it fetched the way you skim an article, then writes about it. It can read twenty filings in a session. It cannot rank 5,000 companies on the same definition as of the same date.

It wants to please you. It's trained to be helpful and agreeable. Ask whether a stock is good, and it finds the bull case. Ask whether it's bad, and it finds the bear case. It sounds equally sure both times, and it has no idea how sure it should be.

Read

LLMs are fantastic at reading. A 200-page 10-K, an earnings transcript, a footnote on segment revenue. Extracting, classifying, summarizing. 

Stockyard’s sector taxonomy was built by having AI read SEC filings and pull out what each company actually sells, and how much of its revenue comes from each line. That's a reading job, and the LLM does it well. 

Syd, our AI research assistant, works the same way: it narrates numbers the rules engine produced. Ask it to analyze a stock, and it will narrate based on the computed numbers to help you make sense of a lot of data without inventing numbers. Ask it to create a Smartfolio strategy in plain English and it will.

Use the LLM where the input is language and the output is understanding. That's a big job. 

Compute

Bottom-up investing is simple to describe. Look at every company's fundamentals as they stood on the day, and keep the ones that clear your bar. Investment clubs have done this for decades with the SSG. It's a predictable, analytical method, and it has three requirements a chat window can't meet.

A defined universe. Stockyard’s Water sector, Pure-play tier, is a defined list: the companies whose revenue actually comes from water. Ask a chat window for water companies and you get the ones it has heard of, with an exposure threshold it chose for you and didn’t mention.

Comparable numbers. "Operating margin above the 50th percentile" only means something if every company's margin was computed the same way, from the same point-in-time data, and ranked against the same set. Percentiles are a property of a universe. No universe, no percentile.

Persistence. One good quarter is noise. A Stockyard Discovery can require the hurdle to hold in each of the last several quarters, for every filter, all at once. Run that on the Pure-play water list, with margin and growth both above the median for four straight quarters, and only a handful survive. Run it tomorrow: the same handful. The only thing that changes the answer is a new earnings report that changes growth or margin rankings, and you’ll know which one.

Can an LLM do this? Not alone. With a code sandbox and a data feed, it can write a screener and run it, but then the program did the screening, not the LLM, and the result is only as good as the data and code you can't see from the chat. An LLM can build you a screener. It can't be one.

Watch

This is the part that matters most, and the part a chat window can't do at all.

A chat window does nothing between prompts. Nothing. It doesn't notice that a company reported Tuesday night. It doesn't re-rank that company against 5,000 others before Wednesday's open. It doesn't check whether a stock you hold just broke below its trailing stop. It doesn't notice that the market gate you set three months ago finally opened. Every question is the first question, and every answer forgets the last one.

A Smartfolio is a rule you write once that runs every day. It reads every earnings report as it lands, across 5,000-plus US stocks, and re-screens. It checks price action daily. It holds the screen, the entry rule and the exit rule in one object: which companies qualify, what price action confirms a buy, and what gets you out. When the conditions you told it to watch for actually occur, it generates a buy or sell signal. When they don't, it generates nothing.

That last part is the one investors underrate. One of my published Smartfolios sat idle for 4 months this year because its S&P 500 RSI gate never opened. No signals, no positions, no commentary. Then the gate opened, and the Smartfolio generated its first signals. A chat window can't do that. It can't decline to act on your behalf, because it can't do anything on your behalf between prompts. Disciplined inaction is a feature, and it only comes from a rule that's running when you aren't looking.

And the rule doesn't drift. Published Smartfolio configurations are frozen. What I ran in January is what I'm running in October. That's the only way a track record means anything, and it's the opposite of a tool that gives a slightly different answer every time you ask.

Which tool for which job

Read with the LLM. Compute with rules on data you trust. Watch with automation that runs whether or not you do. The mistake isn't using AI. It's letting the chat window do the second and third jobs because it's willing to.

If you want to see what a rule would have done with your own idea, write it once in Stockyard, look at the backtest, and let it run and watch the market while you don't. See how Smartfolios work here.

Stockyard is in beta. I'm inviting early readers to become founding members — locked-in pricing for life, a direct say in what gets built next, and a badge that says you were here first. Sign up for free at stockyardinvest.com. Founding members get Pro until we launch.