In this article: why a firm would care · what we built · why an old card can do it · what it cost · what it cannot do · the three builds · what surprised us · the lesson about checking · who this suits · where the engineering is
Why a firm would care
Most firms that use AI send their text to a company that runs the model for them. For most work that is a reasonable arrangement. For some files it is not one the firm, or the client, is comfortable with. The alternative is an open model: one whose makers publish it for anyone to download and run on a machine they own.
The usual objection is cost. The machines built for these models are priced for technology companies. Our question was narrower: can a small firm run a serious open model on hardware it can afford, fast enough to work with, and can the result be measured honestly? The answer is yes, with limits that we state below.
Running a model in your own office removes one outside party from the handling of a file. It does not settle a firm’s confidentiality or privilege duties, and the firm still has to secure its own machine.
What we built
The cards are Nvidia Tesla V100s, sold to data centers from 2017. Each has 32 GB of very fast memory. They are mounted four to a board. We started with two boards (eight cards) and now run three (twelve cards).
The model is DeepSeek V4.1 Flash, released in 2026. It is far too large for one card, so it is cut into sections and each board runs one section. To fit eight cards, most of the model is stored in a compressed form.
Speed is counted in tokens. A token is roughly three quarters of an English word. These are our measurements of November 5, 2026, on eight cards, one request at a time:
| What it is doing | Measured | In plain terms |
|---|---|---|
| Writing legal text | 118 tokens a second | about 90 words a second |
| Writing computer code | 215 tokens a second | code is more predictable, so it goes faster |
| Reading a 128,000-token file before it answers | 59 seconds | roughly 96,000 words read in a minute |
| Longest request it accepts | 524,000 tokens | roughly 390,000 words in one request |
On a standard set of legal reasoning questions (LegalBench, 22 tasks) the same build scored 0.716 in balanced accuracy, where 1.0 is a perfect score. The measurements, the method and the test questions’ sources are public, so anyone can check them or run them again.
Why an eight-year-old card can still do this
When a model writes, it produces one word at a time, and for each word it has to read through a large part of itself. The machine is not waiting on arithmetic. It is waiting on how fast it can fetch its own numbers from memory. On that one measure these cards were built generously, and they still hold up: each fetches data more than three times as fast as an entire current desktop AI machine from the same maker.
What the old cards lack is everything added since: newer cards do arithmetic directly on compressed numbers, and these cannot. Software has to unpack the numbers first. That is slower when the machine reads a long document, which is where old cards fall furthest behind new ones.
The cards also come in boards of four, and the connection between boards is much slower than the connection inside one. So a large model is divided so that each board does its part and hands the result to the next, like stations on a line. Adding a board adds room for a bigger or more precise model. It does not make a single answer arrive sooner, because a request still has to pass through every section in turn.
What it cost
- The cards. When we bought them, a board of four cards cost about $2,400, as Tom’s Hardware reported. The same article found the cards listed for more than double that price by November 2026, and rising. The case for old cards depends on the used market, and the market has moved.
- Electricity. While it works, the eight-card machine draws about 1.1 kilowatts for the cards and processor, measured. Idle, the cards alone draw about 460 watts. It runs on two ordinary office circuits.
- Time. The cards date from February 2026, and months of running smaller models on them came first. Then under three weeks of daily engineering, done with AI coding assistants, took this model from “it runs” to the speeds above. That work is now published, so the next person does not have to repeat it.
What it cannot do
- It does not replace the best cloud models. In our own practice about half of the model work still goes to them. The local model does first-pass work: reading a folder of records, pulling out facts and dates, summarizing, a first draft of something routine. A lawyer reviews everything before it leaves the office.
- It serves one person, or a few. The eight-card machine handles one request at a time. The twelve-card machine keeps four conversations ready and works on three at once. It is not a system for a hundred users.
- A long file takes a minute or two before the first word. After that, a follow-up question on the same file is answered in about a second, because the machine keeps what it has read.
- The compressed model is not the original. On eight cards, the compressed model chooses the same next word as the original about nine times in ten. We built two further versions that store more of the model at its released precision, described below.
- The cards are old. One board was pulled for a burnt power connector. The machine reset twice under load before we limited the cards’ speed. Nvidia’s current driver line is the last that supports them. Anyone who buys them is buying a few more years, not a platform.
- It is not quiet or cool. A kilowatt of electricity becomes a kilowatt of heat, and data-center cards need strong fans.
Three builds, each for a purpose
A model’s numbers can be stored the way its makers released them, or compressed further so that more of the model fits. We built the same model three ways. Each is published with its own measurements.
Volta sPLIt runs on eight cards. To fit, most of the model is compressed throughout. It is the build whose speeds are in the table above. Measured against the original model, it chooses the same next word about nine times in ten.
Volta Max uPLIft also runs on eight cards. It keeps the parts of the model that do the most work at the precision of the version Nvidia publishes, and compresses the rest. In the release we tested for accuracy, it chose the same next word as Nvidia’s version 95 times in 100.
Volta 12X AmPLIfy runs on twelve cards. With a third board, every part of the model fits at that precision. It writes at about 128 tokens a second for one request, reads a 93,000-token file at about 3,800 tokens a second, and keeps four conversations ready.
One caution applies to all three: how closely a build follows a reference version is a measurement about the model, not a promise about the quality of any particular answer.
What surprised us
Most of the speed was other people’s work. The software that lets these old cards run current models is maintained by volunteers and small teams who publish it openly. We credit each of them by name in the research. Our own contribution was a set of improvements on top, and we sent those back: twenty-seven proposed changes to the shared project, fifteen of them accepted so far.
The biggest gains were not exotic. The largest single improvement in day-to-day use was making the machine keep what it had already read. In a long working session, the wait before an answer fell from 31 seconds to under one. No new hardware was involved.
More than thirty ideas failed. Some made the machine faster and changed its answers, and we refused them. All of them are published with their numbers, because a failed experiment that is written down saves the next person a week.
The part that applies to any firm using AI
The lesson we would pass to a firm that never buys a graphics card is about checking, not about hardware.
Every change we tested had its pass-or-fail rule written down before the test ran, so that a result could not be explained into a success afterwards. A test that could not decide was recorded as undecided, never as a pass. That discipline caught a great deal. It did not catch everything.
Our own checker was wrong. We asked a second AI system to review the whole record, with no ability to change anything. It found that one of our checks, the one that confirms two answers are identical, had been comparing them only as far as the shorter one. An answer cut off halfway still passed. We fixed the check and re-examined the earlier results on complete answers. They held, but we had not known that they would.
A lawyer will recognize the pattern. The review step is itself work product, and it needs its own review. A firm that adopts AI should ask of every safeguard: has anyone tested whether this check can fail?
Who this suits
A firm with files it has decided should stay in the building, a small number of people using the model at once, someone willing to keep an unusual machine running, and realistic expectations: first-pass work that a lawyer reviews. It does not suit a firm that needs many people served at once, a quiet room, or a vendor to call when something breaks. For most firms the better first step is to find the one task where a model would save the most time, and solve that with whatever tools fit.
Where the engineering is
- The Volta research site: the long article, every build, every improvement that worked or failed, the testing method, and the source research (source on GitHub).
- Volta sPLIt, Volta Max uPLIft and Volta 12X AmPLIfy: the builds themselves, each with its measurements.
- The hardware list: every part, what was paid for it, and what went wrong.
- PLI Volta Builder: the same method packaged so that an AI coding assistant can repeat it.
- Tom’s Hardware coverage of the eight-card machine, November 9, 2026.
- Local models: what this looks like as a service for a firm.
- Back to the research index