LLMs have Personalities - Interviewed seven models across eight lanes.

By Steve Miles, software & hardware engineer in Perth, Australia.

LLM Personalities, part 1 of 3: numbers from an internal agent bake-off, July 2026.

Last week we ran what amounted to a hiring round. The role: knowledge-base agent. Answer engineering questions by searching our internal knowledge store, cite sources, never make things up. We tested seven models across eight lanes; one model ran both locally and through a hosted provider, giving us a serving-stack control as well as a candidate.

Every candidate got the same take-home test: ten real questions from our codebase, three attempts each, answers marked by an independent examiner (a different model, pinned for the whole round). The knowledge base stayed frozen, so nobody could blame the reference material.

We expected a capability ranking and got a lesson in management.

The scoreboard lied to us twice

The first, permissive-briefing round looked like a straightforward talent hierarchy:

Read it like a manager reads a stack ranking and you'd conclude: cloud models good, local models bad, buy the expensive one. That table was a diagnostic snapshot.

After we fixed the serving configuration, tightened the grounding contract, and corrected gates that penalised valid answers, the production-contract rerun looked like this:

With ten tasks, differences inside roughly ±17 percentage points are statistical ties. The useful result was never a neat ranking; it was seeing which controls moved a lane and how each model failed.

Both early conclusions were wrong, and finding out why took most of the round.

Lie #1: sometimes it's the employee's laptop

Local Gemma's first run produced gibberish: degenerate token loops, leaked formatting tags, zero tool calls. The score alone (0%) said the model was broken.

A diagnostic probe found the real problem. A one-sentence question was reaching the model as three tokens. Our inference server was applying a chat template built for an older generation of the model family, silently deleting the message content, and the model was answering an empty prompt. The fix was one launcher flag: drop the --chat-template override.

A twin control saved us. We had entered the same model weights twice, once on our hardware and once via a hosted provider. When the local twin scores 0% and the hosted twin scores 60%, you don't fire the employee. You fix their laptop.

If you benchmark local models without a hosted twin of the same weights, you are measuring your ops, with extra steps.

Lie #2: the brief is part of the hire

With the server fixed, local Gemma still scored half its hosted twin, 33% against 63%. We chased quantization theories, sampling parameters, decoding differences, and hit dead ends: when Gemma searched the knowledge base, its answers matched the hosted version fine.

The real pattern was stranger. It skipped the search tool on the same three questions, in nine attempts out of nine. Deterministic refusal. It read our instruction, "Use the knowledge base when available", and on questions it felt confident about, treated "when available" as optional.

So we ran the cheapest experiment of the whole round. Same model, same server, same questions, one sentence changed in the brief:

"Use the knowledge base when available." "You MUST search the knowledge base before answering — never answer from memory, even when you think you know the answer."

Result: 33% became 63%. It searched on every attempt and matched its hosted twin exactly.

The gap between a badly-briefed Gemma and a well-briefed Gemma (30 points) was larger than the gap between the best and worst models in the entire top cluster (7 points). The briefing was the dominant variable.

Engineering managers know this pattern. Hand a junior developer a ticket that says "improve the auth flow" and you get whatever they imagined. Hand them acceptance criteria and you get the feature. The seniors don't need the acceptance criteria; that judgment is much of what you're paying for.

The personas

With the environment fixed and the briefs tuned, the seven candidates resolved into people you have worked with.

Claude Opus: the staff engineer. Give them a vague one-liner and they do the right thing anyway: research first, cite everything, escalate when unsure. Across thirty answers, zero fabricated claims and 100% source support. Their early "failures" came from an efficiency budget incorrectly vetoing otherwise perfect answers; once quality and efficiency were separated, Claude reached 80%. The production brief also reduced tool use from 1.63 to 1.2 calls per task without reducing grounding.

GLM-5.2: the disciplined senior. Follows the process almost perfectly (3 unsupported claims in 30 answers), knows when to dig deeper (2 to 4 searches on the hard questions), and costs about a tenth of staff rates. If someone has written the process down, and after this exercise ours is, this is who does the bulk of the work.

laguna-xs: the quick junior. Same pass rate as the senior, five times faster, free. Citations get sloppy under speed (14 unsupported claims). Perfect for interactive work where a human reads the answer with a critical eye; wrong for anything that ships unreviewed.

Gemma 26B: the literal contractor. Capable, and does exactly what the statement of work says. "When helpful" means "at my discretion." "You MUST" means always. Once we understood that, they performed at hosted-twin level from our office GPU, for free. The contract travels with them, though: every integration that uses this model has to carry the imperative brief, forever.

Ornith 9B: the eager grad student. Never skipped the research once, the most conscientious searcher of the whole field. The trouble started when the research came up empty: they would write around the gap with full confidence. A strict grounding brief ("write only what the sources say; if the sources don't say it, say what's missing") halved the fabrications and produced the field's third-best quality score, from a 9-billion-parameter model, tied on pass rate with the staff engineer within the confidence interval. One boundary never moved: when retrieval misses, they still improvise rather than admit it. That is seniority, and no briefing fixes it. Keep the grad student away from any seat where a confident wrong answer costs more than no answer.

gpt-oss-120b: the one who looks busy. Most tool calls of any candidate, 1.47 searches per question, more than the staff engineer. Also the most fabricated claims by far: 78 in 30 answers. They do the research and then ignore it. In a dashboard of activity metrics this employee looks like your hardest worker; in output review, nothing survives. We disqualified this candidate, and the diligent-looking logs were the reason it took so long to see why we had to.

Choose by the failure you can afford

Once the briefing is fixed, the models in the viable cluster pass at roughly the same rate. They differ in how they fail, so pick by which failure mode your use case tolerates:

What we actually changed

The lasting change was where the winning words ended up. The two sentences that moved the numbers ("you MUST search before answering"; "write only what the sources say, and say what's missing rather than filling the gap") no longer live as advice in a wiki. We baked them into the knowledge base's own tool descriptions and agent skills, the surface every future model sees, whichever one we hire next. Then we re-ran the whole round against the production wording, with acceptance gates, to prove the words work as defaults and don't make the frontier model over-search. They work, and it doesn't.

Three takeaways if you're running agents against your own knowledge store:

1. Run a hosted twin of any local model before judging it. Otherwise you're benchmarking your serving stack.
2. Treat the prompt as part of the deployment artifact, versioned, tested, and shipped with the model. At current capability margins, briefing quality beats model choice inside the viable tier.
3. Rank by failure mode, not pass rate. Small eval suites produce statistical ties at the top; the differences that matter in production show up when things go slightly wrong.

Or, in management terms: past a bar that most current models clear, you're not choosing talent. You're choosing which weaknesses you're prepared to manage, and writing a brief good enough that the affordable hire performs like the expensive one.

Method notes: 10 retrieval-and-cite tasks over a frozen production snapshot, 3 repeats per lane, an identical tool loop, and a judge pinned to deepseek-v4-flash. We treated results within roughly ±17 points as ties and scored quality separately from efficiency. Broken source gates were corrected before the production-contract rerun. Local models ran Q4_K_M GGUFs on llama.cpp; identical local and hosted Gemma content results exonerated quantisation for this workload.

LLM Personalities: Part 1: The briefing mattered more than the hire · Part 2: One knowledge base, two AI jobs · Part 3: The evaluation harness was wrong too

Related reading: The story of making local agents usable · Hardware, engines, and benchmark results