BuffettBot got an upgrade

I asked BuffettBot a question from the letters it was trained on: how much did you pay for Disney in the late 1960s? It answered without hesitation: 1965, about $10.50 a share, roughly $100 million of market value, about $10 million of pre-tax earnings. I wasn't sure what to expect, but every figure was wrong. Buffett's 1995 shareholder letter records the purchase plainly: 1966 at 31 cents per share, split-adjusted, with a total market value of less than $90 million, and $21 million of pre-tax earnings the year before.

Numbers delivered confidently in a perfect Buffett voice are worse than no answer at all. The model behind them had nothing to look anything up with, and no way to check what it remembered. We rebuilt BuffettBot from the ground up. V2 reasons in Buffett's own decision-making principles before it answers, calls financial-data tools across multi-round conversations, and grounds its figures in the letters that produced them. Every one of those claims is measured below, against the production model, on a frozen evaluation suite. BuffettBot is our own product, and it doubles as the reference build for what a client can run on this platform: a private document set, a model trained to reason over it in a chosen voice, retrieval over its documents, tools over its data, all inside one infrastructure.

What BuffettBot was

BuffettBot is our fine-tuned LLM, trained on Warren Buffett's shareholder letters, annual meeting Q&A transcripts and essays. The original was a standard 2023-era recipe: a light LoRA adapter on Qwen3-32B, a static system prompt, and training prose that stopped in 2023, with no retrieval and no tools. In 2026 a model which cannot reason or call tools is barely more than a system prompt.

How BuffettBot v2 thinks and calls tools

Fine-tuning trains a model's weights on a chosen dataset so a behaviour is baked into the parameters. For v2 the base is Qwen3.6-27B and the dataset is the current Buffett training set, formatted as questions and answers. That gets you a model that sounds like Buffett and makes up his numbers, which is the problem we started with. Warren's mental model consists of a few fixed questions. Do I understand this business? Does it have a durable moat? What does it look like in ten years? We distilled those principles into 22,333 training conversations, so the model reasons through them inside its <think> channel before it answers in his voice.

Reasoning over a fixed set of documents still can't reach today's stock price or today's date, and Buffett answered from research as much as from memory. V2 learnt tool-calling from training records built on third-party xLAM-60k schemas. The five financial-data tools — company financials, stock quotes, FX rates, macroeconomic data, and a search over the Berkshire corpus itself — were deliberately held out of training, then tested through the test MCP server.

Here is a real exchange from the evaluation run, where the five tools answer from deterministic snapshot fixtures rather than live feeds. Asked to walk through Coca-Cola's revenue, margins, and returns on capital, and to show the full financials, v2 emitted one tool call on its first turn:

{"name": "get_company_financials", "arguments": {"ticker": "KO"}}

Then, with the figures in hand, it answered in character:

"Well, Coca-Cola's numbers tell a story of a business that's as solid as they come. In 2025, they pulled in $46.2 billion in revenue, up a modest 2.8%, and turned that into $10.5 billion in net earnings. That's a 22.7% net margin, and the operating margin's even better at 28.5%."

The full answer continues through the moat, the management and the 39.5% return on equity before Buffett's own verdict. The tool call and the quoted line come from the evaluation artefacts without edits.

The shape of the build:

fine-tuned weights = who Buffett is
retrieval          = what he actually wrote
prompt             = what is true right now
tools              = what is trading right now

The training run: a fresh rank-32 RSLoRA applied to the stock Qwen3.6-27B base weights (a full retrain, not an adapter stacked on any earlier stage) in bf16 over an 8,192-token training context. The dataset held 26,422 rows: the 22,333 reasoning distillations, plus 2,489 teacher-generated correction records and 1,600 tool-use records. With 15% of the mixture held out for validation, two epochs at an effective batch of eight make 5,616 optimiser steps (roughly seven hours of training on a single B300 GPU). The adapter is then merged back into a single set of weights and served through stock vLLM with native thinking enabled.

Why not just prompt Qwen3.6?

We host Qwen3.6 on the platform, so this was a fair question, and the honest answer stung. On our judged 60-question harness, the stock model behind nothing but a persona system prompt reached parity with the tuned model. A good prompt is simpler, free, and needs no dataset. It runs out of road quickly, though. On the multi-turn tool probe, the prompt-only control managed 8 of 12 scenarios, and its failures were structural: an unnecessary call, then an empty answer on the final round. On the adversarial sycophancy lane it corrected 15 of 16 wrong premises under Nemotron, the independent judge, but only 8 of 16 under Mistral, the report's default judge; the tuned model corrects 16 of 16 under both. And a prompt is a text file that anyone can edit, where a fine-tune is behaviour you can freeze, version, audit and re-measure. The weights measured on the frozen eval suite are the weights that ship. V2 still ships with a small system prompt on top, carrying what changes at runtime, like the date and the turn count, along with the boundary rules that cannot be baked into weights.

Retrieval closes the memory gap

Fine-tuned models hallucinate figures. V1 did it above with Disney, and v2 parametrically guessed the Disney year correctly but invented $19.50 per share, $400 million of market value and $40 million of pre-tax earnings. Every figure-bearing request is now forced through retrieval at serving time, over an index built from 55 Berkshire documents with dense and BM25 hybrid search. If retrieval returns nothing, the service declines loudly rather than inventing a number. On a frozen held-out bank of 63 gold figures, production v1 recovered 10/63 from memory and 53/63 with retrieved context injected at evaluation time; v2 recovered 12/63 and the same 53/63, with the residual misses mostly different between the two models (four of ten figures overlap). Retrieval is the correction for that shared weakness, and it moves both models by roughly the same margin. Asked the Disney question with retrieval in front of it, either model answers exactly right: 1966, 31 cents a share, less than $90 million, $21 million pre-tax, and its own admission of selling too soon.

Key results

All of this was measured against the production v1 on a frozen seven-instrument eval suite, judged by Nemotron 3 Super 120B hosted on PrivateMind: an independent judge that had no part in producing either model's training data. The technical report carries the default judge, Mistral, side by side throughout.

• Tools: a deterministic 12-scenario multi-turn probe, 3/12 to 12/12. v1 emitted zero tool calls across its entire run.
• Figures: 10 to 12 out of 63 from memory, 53/63 for both models with retrieved context.
• Judged quality: 3.42 to 3.72 out of 5 on the 60-question harness in RAG mode, with factual accuracy the largest gain.
• Sycophancy: 16/16 wrong-premise corrections, up from v1's 8/16, both under Nemotron. The report's default judge agrees on v2 and marks v1 at 7/16.
• Refusals: 22/30 calibrated, up from 15/30, with 6/6 on questions where opinion is expected.

Under the independent Nemotron judge, two harness lanes inch the other way (tool_use from 2.67 to 2.33 on a single-turn rubric that cannot see a real tool call, and rag_grounding from 4.67 to 4.42), both within noise on a 60-question bank, and both in the report. Under the same judge, v2 holds its stance on 41 of 60 reframed questions yet stays consistent across all three framings of only 7 of 20, and the report's own conclusion is that no arm is robustly framing-consistent. V2 answers also carry their thinking blocks and run about twice as long as v1's, and past the training cutoff both models are weak from memory, which is why that traffic is retrieval-served by policy rather than trusted to recall. The technical report adds a second judge to every table and carries the full caveats.

One stack, not five vendors

This project needed five things: GPUs, training data, evals, embeddings and serving. Traditionally that is five vendors and five API keys, with your proprietary data flowing through all of them. Here it was one platform. The GPU workspace trained the model. Open-weight frontier models, hosted on our own infrastructure, distilled the dataset and judged the evals. kaLM embeddings power the retrieval index. The gateway ships the model into the app. Every training record, eval answer and retrieved chunk stayed inside the infrastructure.

What this looks like on your documents

Point the same build at your own documents and it keeps its shape. Over your research notes, IC memos and broker documents, the weights would carry your house view and voice, retrieval would ground the answers in your documents, the prompt would carry what is true today, and the tools would answer from your data feeds. The labour is yours, and the platform is self-serve: a GPU workspace, plus a Python SDK that makes fine-tuning jobs on it easier.

BuffettBot v2 is rolling out on PrivateMind now. The full technical report, with every bank, judge artefact and md5, is available on request.