The Read Tool

how our read tool saves billions of tokens vs claude code

command code is purpose-built for open models, so we optimize things other coding agents get to ignore. for the v1 release i rebuilt the read tool from scratch. it's now one of the most complicated, carefully engineered pieces of the system, and it saves billions of tokens a month. here's what we learned.

context: we wanted the read_file in command code to be the best among coding agents, then benchmarked it capability-by-capability against the nine other common harnesses: claude code, opencode, cline, kilo, codex, grok, hermes, pi, openclaw. most were open-source; claude code ships none, so its column came from feeding the live tool crafted files and watching what came back.

count the reads in any agent session. every edit starts with a read. every grep hit becomes a read. a plan step opens 3 files. a few hundred reads per session, ~50 million a month across command code.

you've seen the failure modes. it reads a file, learns nothing, reads it again. it reads a 5MB lockfile straight into context. it reads a minified bundle once and that junk sits in the window for every turn after.

napkin math:

i think the read_file tool is like a compiler that turns your filesystem into the model's context. every decision inside it is a token budget decision multiplied by fifty million times it's used every month.

and that's why coding agents feel expensive: the bill is mostly reads building context.

what "saves billions of tokens" means here: cost per successful read. claude code's read tool succeeds by spending more: more tokens per call, more turns per miss, and a model smart enough to fish the signal out of the noise. ours had to succeed by spending less, because our models can't paper over a sloppy read and our users care about the token bill.

ask claude code to read a 3,000-line file and it hands the model all 3,000 lines. ask it for a file with a 3,900-character minified line and it hands over the whole line. no window, no byte ceiling, no per-line clamp. i ran the probe twice because i didn't believe it the first time.

everybody ships a read tool in week one. readFile, slice by offset, return the string. first tool you write, last one you think about. ours ended up as dozens of modules with 98 tests, and it was the highest-leverage thing in v1.

a naive read and a harness engineered read are both "correct". they both work. the difference is that one of them quietly spends a fifth of your context window on bytes the model never needed, and occasionally deadlocks against your own write tool.

a few things i learned worth sharing:

1/ you need three ceilings, not one

and it's always the third one people skip.

every codebase keeps a small zoo of hostile files: the 80,000-line lockfile, the minified bundle that's technically one line, the log that never stops growing. each ceiling handles one animal file if you will.

the line window bounds an ordinary large file. the byte budget bounds a file whose lines are wide rather than many. the per-line clamp catches the case the other two miss: one minified line that sits comfortably inside the 2,000-line window and, on its own, eats the entire byte budget. you get back a single unusable mega-string that displaced everything the model actually needed to see.

drop any one ceiling and there's a shape of file that costs you the whole read. no log will ever show it, just a turn where the model got nothing and paid full price for it.

2/ the most expensive thing a tool can return is silence

the costliest failure is an ambiguous non-answer. an empty result string is indistinguishable, from inside the model, from a broken tool. so it re-reads. widens the window. tries a different path. burns three turns learning what one sentence could have told it.

so every dead end names its own recovery:

two details carry most of the value here. the resume offsets are precomputed, so the model never does pagination arithmetic (which it does in reasoning tokens you pay for, and gets wrong often enough to cost another round trip). and none of these carry an Error: prefix, so the tui doesn't paint them red and the model doesn't treat a fact about the world as a failure worth apologizing for.

(the byte-truncated case deliberately resumes ON the last line shown rather than the line after it, because that line got cut mid-content. an off-by-one in a resume hint is a silently corrupted read, which is the one bug class here that's worse than a wasted turn.)

returns silence

1. 01

   the read comes back empty.

2. 02

   the model can't tell an empty file from a bad offset from a broken tool, so it guesses and retries.

3 turns, no new information

returns the recovery

1. 01

   Note: offset 900 is beyond the end of the file (412 lines scanned). Retry with a smaller offset.

2. 02

   the model knows what happened and what to send instead.

1 turn, correct next call

3/ the bug that taught us the most was relational

and no input validation could ever have caught it.

read_file records what the model has SEEN of each file into a ledger: the content, the mtime at read time, and a flag for whether the view was partial. write_file consults it and refuses to overwrite a file you've only partly seen, because you'd silently destroy the part it never saw.

now compose that with the per-line clamp:

we hit it in the wild, on plan files during plan reviews and refinement. every field in every call was valid. the invariant that broke lived in the relationship between three tools that never call each other.

the fix was three-sided:

• write_file now accepts an overwrite whenever the ledger's recorded content matches the bytes on disk, even when the view was flagged partial. a clamped full read still records the exact raw bytes.
• a genuinely partial view gets its own accurate error, Only part of this file has been read, instead of the misleading has not been read yet, which had been sending models into tiny-window re-read loops.
• the unchanged-read dedup no longer stubs a from-line-1 re-read unless the ledger already holds a full view. the dedup check runs after that guard, because a dedup hit consumes its record.

shape invariants are checkable per field, and every schema you write already checks them. relational invariants across stateful tools are where the real bugs live, and you only find them by watching production traffic.

4/ a cache whose stale hit is catastrophic should expire itself on use

re-reading the same window of an unchanged file is pure waste: the content is already sitting in the conversation. so we return a short stub. fires only when mtime, size, and the exact (offset, limit) window all match.

but that stub points at an earlier tool result. what if compaction ate it? now the model has been told to refer to something it can no longer see. forever.

a dedup hit consumes its record. worst case is one wasted turn instead of an unbounded loop. cheap miss, catastrophic stale hit → self-expiring cache. that shape shows up all over a harness once you look for it. i settled with this design as it was a good enough tradeoff between complexity and risk, and it was the only one that did well in our benchmark.

sticky cache

1. 01

   the record survives every hit. if the referenced result is gone from context, the model is pointed at nothing, forever, and no retry escapes it.

unbounded loss

consume on hit

1. 01

   the record is dropped when it fires, so the natural retry gets real content and the loop resolves itself.

one turn, worst case

5/ filenames are adversarial and the model can't see why

macos names screenshots with a NARROW NO-BREAK SPACE before AM/PM. it stores filenames NFD-decomposed. finder renames turn ' into ’.

different byte strings. in a terminal, the same picture. the model reads the path off the screen, retypes it faithfully, gets "file not found", and no amount of reasoning recovers because the difference isn't rendered. you can burn an entire session on this and never learn anything.

so before failing we retry 7 candidate spellings: narrow space ↔ regular, NFD, NFC, straight ↔ curly quote, NFD+curly. each one re-checked against the workspace boundary, because a repair must never quietly become an escape hatch. then, and only then, "did you mean?": substring match plus a bounded levenshtein of 2, which is what catches AGENT.md → AGENTS.md where substring matching finds nothing. these are the most common super cheap open model problems we now repair, saving more tokens than silly token compression tricks.

when a failure is invisible to the model, retrying is the tool's job. the model would retry the same wrong bytes forever. this is what harness engineering is about: finding the invisible failure modes and fixing them in the tool so the model can focus on reasoning.

6/ my favourite bug lives at a chunk boundary

reads stream chunk by chunk instead of loading the file, so a 400MB line sitting BEFORE your window never accumulates. fine. but it turns out that if the line limit is hit EXACTLY at a chunk boundary, you're standing in a spot where the answer to "is there more file?" doesn't exist yet.

saying "more of the file remains" at that moment is a lie roughly half the time, and it's a lie that costs a turn every time it fires. so defer the decision to the next chunk instead of guessing. when you can't know yet, say nothing yet.

(also: don't break out of the for-await. it calls the iterator's return() and destroys the stream underneath you.)

7/ images attach for real

vision models get the actual image, compressed down a jpeg quality ladder (95 → 80 → 60 → 40 → 20). a 4K screenshot degrades instead of failing to attach. format detection sniffs magic bytes, never the extension: garbage in a .png must never reach the api, real webp must pass. we also gave vision to non-vision models using a VISION tool. so fun.

8/ downscaled images disclose their scale factor

without that line, every click coordinate computed off a screenshot is confidently wrong. nothing in the image says it was resized on the way in. and at the end of the day you're saving token costs.

9/ notebooks render as documents

raw .ipynb is json soup: base64 blobs, per-character source arrays. we return tagged cells, plots attached as images. any cell output over 10,000 chars becomes a jq pointer, so one dataframe dump can't eat the read budget. if you do a lot of data work in notebooks, you can now read the notebook without reading the entire dataframe. the model can still reason about the data, but it doesn't have to pay for it in tokens. major time savings for the user, and a major token savings for the model.

10/ boring formats get one-line answers

svg is text (it's xml, the model can edit it). binary returns its mime type, never garbage bytes. pdf gets a pdftotext hint for now; inline is on the list. and we have tools to read and parse different formats as needed, loaded on demand. the model can reason about the file without reading it all, and the user doesn't pay for it in tokens.

11/ numbering matches cat -n

1-indexed, prefix on every line. the model, your editor, and your stack traces agree on what "line 412" means. every resume offset and edit target depends on that.

12/ repair inputs, don't bounce them

10 aliases for file_path (filePath, absolutePath, target_file...) get repaired through the repair layer. numeric strings coerce via Number(), never parseInt: "2abc" is rejected, never silently read as 2. fractional offsets are rejected, never floored. a silently wrong window is worse than an error. our repair harness engineering shows up everywhere.

13/ some paths must never be opened

/dev/zero, /dev/urandom, /dev/stdin, /proc/<pid>/fd/* are refused by name before any i/o. no extension to check, and the workspace boundary won't save you when cwd is /. a read tool that hangs on /dev/zero is a denial of service you shipped yourself.

14/ hygiene you only notice when it bites

bom stripped. crlf normalized. byte-cap truncation binary-searches a utf-8 prefix so it never splits a codepoint. the dedup ships with a kill-switch env var, because every cache needs one.

so where does everyone else land?

the top of the table is basically solved. eight of ten harnesses have a line window (500 to 2,000) and a second ceiling (25K tokens to 128 KB). that part is common knowledge now.

Changelog

the bottom of the table is empty almost everywhere. across all ten:

the pattern: none of those rows show up in a demo. every one of them starts costing you in hour nine of a long session (remembering what the model has already seen, giving it a way back when a read misses, refusing to read /dev/zero). teams build them only after production forces it.

no one nowadays is sitting watching their agents run and fixing the coding agent's bad behaviors. most developers just select a harness on some random vibe in the first week and never look back. this harness is minimal, it must be the best, this harness is from the model maker, this must be the best. wrong! 🤦‍♂️

… whatever happened to the engineer in us?

claude code is the interesting column precisely because it's the incumbent: ledger, notebooks, vision, empty-file note, and then no window, no byte cap, no clamp, no resume offset, no streaming, no suggestion on a miss. that team just hasn't been forced yet, and it runs on models forgiving enough to absorb the waste.

we were forced. we run open models where a wasted turn is visible in the eval score the same day, which is the entire reason any of the above got built. constraint is a feature - it forces you to engineer the right solution instead of hoping the model will figure it out.

i hope this post helps you see the difference between a harness that just works and one that was engineered to work well.

you can try all of this yourself in Command Code (we're also going open source soon so you'll be able to read the code yourself). i'd love to share more deep dives on harness engineering of command code, let me know what y'all wanna read about.

Related