Proprietary Legal Intelligence
  1. Home
  2. Research
  3. Owned hardware

Owned hardware

Volta 12X AmPLIfy: a current AI model, at full published precision, on twelve graphics cards from 2018.

Our twelve-card build serves DeepSeek V4.1 Flash with every one of its 15,360 experts held on the cards at the precision Nvidia publishes, keeps four conversations ready, and accepts requests of 524,288 tokens. This article explains what the build is, how fast it reads and writes at every context length we measured, and what did not work. Every figure carries its conditions.

In this article: what the build is · why twelve cards · what had to be built · how fast it is · several requests at once · documents it has already read · the hardware · what did not work · what is next · where the engineering is

A board of four Tesla V100 SXM2 cards under tall finned heatsinks, on a conference table in front of shelves of law reporters
One of the three boards. Four Tesla V100 cards, 32 GB each, under the heatsinks they came with. The build runs three of these.

What the build is

The cards are Nvidia Tesla V100s, sold to data centers from 2017 and retired from them years ago. Each has 32 GB of very fast memory and they come four to a board, joined inside the board by a fast link. The model is DeepSeek V4.1 Flash, released in October 2026. It is a mixture of experts: for every word it writes, the model consults a handful of small specialist networks, the experts, chosen from a pool of 15,360. Only a few run for any one word, but all of them have to be somewhere the cards can reach quickly.

Nvidia publishes a version of this model whose experts are stored in a compact four-bit form it calls NVFP4. The expert values in that pack equal the ones in DeepSeek’s own release. Our aim with this build was simple to state: serve every expert from that pack, from card memory, with nothing compressed further and nothing read from the slower memory of the host computer. On eight cards that is not possible. On twelve it is.

The model runs as three sections, one per board, each section split across the four cards of its board. A request passes through the first board, then the second, then the third, like stations on a line. The build keeps four conversations ready at once, each with up to 524,288 tokens of context, and it is tuned for the way we use it: one long piece of agent work at a time, on prompts of sixty thousand to two hundred thousand tokens that grow by small turns. It has been our default DeepSeek server since November 17, 2026.

Speed is counted in tokens. A token is roughly three quarters of an English word. A “128K” prompt is about 128,000 tokens, roughly 96,000 words.

Why twelve cards and not eight

The arithmetic decides it. One expert in NVFP4 takes 4,874,240 bytes on each of the four cards that share it. The forty layers of the model hold 278.9 GiB of expert records. Eight cards of 32 GB hold 254 GiB. So an eight-card machine has to compress some experts further or read some of them from host memory, and either choice costs something. Twelve cards hold every expert with room to spare for the conversations.

Where to cut the model into three sections is also decided by arithmetic. Twenty layers of experts are 34.86 GiB per card, more than a card holds, so the first cut cannot fall at the middle. A section has to begin at a layer of a particular kind, and the last section also has to carry the model’s drafting network and its output stage. That leaves one legal cut that fits: layers 0 to 13, 14 to 27 and 28 to 39. It is the layout a pull request of ours to the shared engine had already run on twelve cards, at about 3 tokens a second on an unoptimized engine. Our starting point was that someone else had shown it could load.

Two four-card V100 boards stacked in a rack, seen from the front, with their heatsink fins and green status lights
Two of the three boards in the rack. Inside a board the four cards share a fast link; between boards the model’s sections hand off over the host’s PCIe lanes.

What had to be built

The engine we started from accepted exactly one arrangement of this model: two sections, cut at layer 20, and nothing else. The work of this build was a set of changes, published as install kits, that teach it a third section, fix its scheduler for a line of three, cut the middle section’s memory to fit, and add a memory limit on every card so that a boot that would not fit fails at once instead of failing under load. After that came three days of test windows, one change per boot, each with its pass-or-fail rule written before the boot ran. Eleven of those changes are switched on in what serves today. The rest are recorded with their numbers, including the ones that failed.

How fast it is

All of the figures below were measured on November 19, 2026, on the configuration that serves today, with every card limited to 175 watts and a clock ceiling of 1,452 MHz, four conversation slots open, the model’s reasoning mode off, 1,024 tokens written per request, each prompt read cold (nothing of it in the cache), and each cell the mean of two runs. The prompts are real files of three kinds: a source-code file, a legal brief, and literary prose.

Three columns matter most. Decode is how fast the model writes once it has read the prompt. Prompt read is how fast it takes in the prompt before the first word. First token is the wait before the first word appears, which for a long document is almost entirely the read.

Code (a source file as the prompt)

Context (prompt tokens)Decode, one request (tok/s)Prompt read (tok/s)First token, coldThree requests together (tok/s, summed)First token, document already cached
1K1841,5670.7 s4430.2 s*
8K1763,2672.5 s4210.2 s
32K1764,4787.3 s4040.3 s
64K1574,69713.9 s3640.3 s
128K1574,67028.1 s3600.4 s
256K1534,51858.0 s3440.6 s

Legal (a brief as the prompt)

Context (prompt tokens)Decode, one request (tok/s)Prompt read (tok/s)First token, coldThree requests together (tok/s, summed)First token, document already cached
1K1021,5260.7 s2490.2 s*
8K933,2532.5 s2190.2 s
32K924,3547.5 s2130.3 s
64K934,56114.4 s2150.3 s
128K934,53828.9 s2130.4 s
256K924,42259.3 s2070.7 s

* At 1K the request’s fixed cost is most of the read, so the cache cannot show. Literary prose, the third kind, writes at 86 tokens a second at 1K, 86 at 128K and 85 at 256K, and reads at 4,586 tokens a second at 128K and 4,456 at 256K. Every cell of all three kinds, with its minimum and maximum, is in the build’s measurements file, linked below.

Chart: decode speed by context length for code and legal prompts, one request alone and three together summed, 1K to 256K tokens
Writing speed by context length. One request alone, and three requests together with each stream’s rate summed. Code above, legal below.

Three things to read from the tables.

Several requests at once

The build keeps four conversations ready and works on three at once. Because the three sections of the model form a line, each step of the line carries one request, so the three requests take turns rather than batching. Two consequences follow, and both are measured.

First, long prompts are read one after another. Three cold 128K code requests released together got their first words after 77, 90 and 90 seconds, where one alone takes 28. Second, once every prompt is in, the three write together: 360 tokens a second summed on code at 128K, 213 on legal, against 157 and 93 for one request. That is about 2.2 to 2.5 times one stream, not three, and it is the throughput of this machine, not its latency. The “three requests together” column in the tables is that sum. Counted from the first request’s first word to the last request’s end, including the waiting, the three give 138 tokens a second on code and 111 on legal. This is not a batched-decode engine; more requests than sections gain little.

Documents it has already read

The largest gain for daily use is not in either table’s writing column. A request on a document the machine has already read gets its first word in under a second at every length: 0.4 seconds at 128K and 0.6 to 0.7 seconds at 256K, against 28 and 58 seconds cold. In a working session the prompt grows by a question and an answer each turn, and nearly all of it has been read before, so the wait before each answer is the new part only.

The same holds when several people work on one document: the first time a second or third stream shares a file the machine reads it once more, and after that every stream hits. Three code streams on one cached 128K document wrote 427 tokens a second over their window, within 2 percent of the sum of the three alone.

Chart, log scale: time to the first token on a cold read against a document already cached, code and legal, 1K to 256K tokens
The wait before the first word, cold against already cached, on a log scale. The cached line stays under a second at every length.

The hardware

Three boards of four Tesla V100 SXM2 cards, 32 GB each. Inside a board the four cards share Nvidia’s fast link and split each section of the model between them; between boards the hand-off from one section to the next goes over the host’s PCIe lanes, through two different PCIe switches, so the second hand-off crosses host bridges and costs more than the first. Each card is limited to 175 watts and a clock ceiling of 1,452 MHz. We ran at 200 watts for a day and the host tripped once under four long chats at once, so 175 is where it stays; long prompts read about a third slower at 150 watts than at 200, and the figures above are 175-watt figures.

These cards have no hardware for four-bit or eight-bit arithmetic. Every compact number is unpacked in software into the half-precision form the cards can multiply. The compact experts pay off because the cards read fewer bytes, not because they compute faster, and the reading is what a model waits on when it writes.

The host computer's PCIe expansion cards with braided cables, the links that carry each board's hand-off to the next
The host side. The cards and cables here carry the hand-off between boards over PCIe.

What did not work

Every window of this build is recorded with its rule, its result and, where the prediction was wrong, the prediction. A few that matter:

What is next

As this is published, two things are being switched on in the serving configuration: the conversation store, re-cut so that a matter’s documents can be pinned in memory and survive a restart, and a count of cached tokens in every reply, so that a hit can be told from a re-read without a stopwatch. The next study is batched decoding on the line, whose first question, written down before any data, is whether rows written together round the way single ones do. After that, the measurements file and this article will be updated again.

Questions

Frequently asked

Does a third board make a single answer faster?

No. One request still passes through all forty layers of the model, so its writing speed comes from the cards’ memory speed, not from the number of boards. The third board buys room: every expert at its published precision, four conversations kept ready, and 524,288 tokens of context.

Why is a writing speed given as a band?

Because the figure belongs to the text that was written. A change that alters rounding changes the text, and the speed of a cell moves with its text by up to about 10 percent either way, while two repeats of one text agree within 2 percent. The band is the honest number.

Is the model exactly DeepSeek’s release?

The experts are served from Nvidia’s NVFP4 pack, whose expert values equal the official release’s. Other parts of the model are stored in compact forms, and the cards’ arithmetic rounds differently from a data-center card’s. How closely the build follows a reference version is measured and published; it is a measurement about the model, not a promise about any particular answer.

Built for your organization

Want this working on your own matters?