Proprietary Legal Intelligence
  1. Home
  2. Research
  3. Owned hardware

Owned hardware

A current AI model on used 2018 graphics cards: what we learned.

PLI Labs runs a large open AI model in a law office on graphics cards that data centers retired years ago. We have now published everything about how: the measurements, the mistakes, and the list of ideas that failed. This article is the short version for lawyers and firm owners. The engineering is one link away.

In this article: why a firm would care · what we built · why an old card can do it · what it cost · what it cannot do · the three builds · what surprised us · the lesson about checking · who this suits · where the engineering is

Why a firm would care

Most firms that use AI send their text to a company that runs the model for them. For most work that is a reasonable arrangement. For some files it is not one the firm, or the client, is comfortable with. The alternative is an open model: one whose makers publish it for anyone to download and run on a machine they own.

The usual objection is cost. The machines built for these models are priced for technology companies. Our question was narrower: can a small firm run a serious open model on hardware it can afford, fast enough to work with, and can the result be measured honestly? The answer is yes, with limits that we state below.

Running a model in your own office removes one outside party from the handling of a file. It does not settle a firm’s confidentiality or privilege duties, and the firm still has to secure its own machine.

What we built

The cards are Nvidia Tesla V100s, sold to data centers from 2017. Each has 32 GB of very fast memory. They are mounted four to a board. We started with two boards (eight cards) and now run three (twelve cards).

The model is DeepSeek V4.1 Flash, released in 2026. It is far too large for one card, so it is cut into sections and each board runs one section. To fit eight cards, most of the model is stored in a compressed form.

Speed is counted in tokens. A token is roughly three quarters of an English word. These are our measurements of November 5, 2026, on eight cards, one request at a time:

What it is doingMeasuredIn plain terms
Writing legal text118 tokens a secondabout 90 words a second
Writing computer code215 tokens a secondcode is more predictable, so it goes faster
Reading a 128,000-token file before it answers59 secondsroughly 96,000 words read in a minute
Longest request it accepts524,000 tokensroughly 390,000 words in one request

On a standard set of legal reasoning questions (LegalBench, 22 tasks) the same build scored 0.716 in balanced accuracy, where 1.0 is a perfect score. The measurements, the method and the test questions’ sources are public, so anyone can check them or run them again.

Why an eight-year-old card can still do this

When a model writes, it produces one word at a time, and for each word it has to read through a large part of itself. The machine is not waiting on arithmetic. It is waiting on how fast it can fetch its own numbers from memory. On that one measure these cards were built generously, and they still hold up: each fetches data more than three times as fast as an entire current desktop AI machine from the same maker.

What the old cards lack is everything added since: newer cards do arithmetic directly on compressed numbers, and these cannot. Software has to unpack the numbers first. That is slower when the machine reads a long document, which is where old cards fall furthest behind new ones.

The cards also come in boards of four, and the connection between boards is much slower than the connection inside one. So a large model is divided so that each board does its part and hands the result to the next, like stations on a line. Adding a board adds room for a bigger or more precise model. It does not make a single answer arrive sooner, because a request still has to pass through every section in turn.

What it cost

What it cannot do

Three builds, each for a purpose

A model’s numbers can be stored the way its makers released them, or compressed further so that more of the model fits. We built the same model three ways. Each is published with its own measurements.

Volta sPLIt runs on eight cards. To fit, most of the model is compressed throughout. It is the build whose speeds are in the table above. Measured against the original model, it chooses the same next word about nine times in ten.

Volta Max uPLIft also runs on eight cards. It keeps the parts of the model that do the most work at the precision of the version Nvidia publishes, and compresses the rest. In the release we tested for accuracy, it chose the same next word as Nvidia’s version 95 times in 100.

Volta 12X AmPLIfy runs on twelve cards. With a third board, every part of the model fits at that precision. It writes at about 128 tokens a second for one request, reads a 93,000-token file at about 3,800 tokens a second, and keeps four conversations ready.

One caution applies to all three: how closely a build follows a reference version is a measurement about the model, not a promise about the quality of any particular answer.

What surprised us

Most of the speed was other people’s work. The software that lets these old cards run current models is maintained by volunteers and small teams who publish it openly. We credit each of them by name in the research. Our own contribution was a set of improvements on top, and we sent those back: twenty-seven proposed changes to the shared project, fifteen of them accepted so far.

The biggest gains were not exotic. The largest single improvement in day-to-day use was making the machine keep what it had already read. In a long working session, the wait before an answer fell from 31 seconds to under one. No new hardware was involved.

More than thirty ideas failed. Some made the machine faster and changed its answers, and we refused them. All of them are published with their numbers, because a failed experiment that is written down saves the next person a week.

The part that applies to any firm using AI

The lesson we would pass to a firm that never buys a graphics card is about checking, not about hardware.

Every change we tested had its pass-or-fail rule written down before the test ran, so that a result could not be explained into a success afterwards. A test that could not decide was recorded as undecided, never as a pass. That discipline caught a great deal. It did not catch everything.

Our own checker was wrong. We asked a second AI system to review the whole record, with no ability to change anything. It found that one of our checks, the one that confirms two answers are identical, had been comparing them only as far as the shorter one. An answer cut off halfway still passed. We fixed the check and re-examined the earlier results on complete answers. They held, but we had not known that they would.

A lawyer will recognize the pattern. The review step is itself work product, and it needs its own review. A firm that adopts AI should ask of every safeguard: has anyone tested whether this check can fail?

Who this suits

A firm with files it has decided should stay in the building, a small number of people using the model at once, someone willing to keep an unusual machine running, and realistic expectations: first-pass work that a lawyer reviews. It does not suit a firm that needs many people served at once, a quiet room, or a vendor to call when something breaks. For most firms the better first step is to find the one task where a model would save the most time, and solve that with whatever tools fit.

Questions

Frequently asked

Does a law firm need its own AI server?

Most do not, and none needs one to start. It is worth considering for files that the firm or its clients have decided should not be sent to an outside provider, and only after the firm knows what work it wants the model to do.

Is a model on old graphics cards as capable as the best cloud models?

No. In our own practice about half of the model work still goes to cloud models. The local model does first-pass reading, extraction, summaries and routine first drafts, and a lawyer reviews all of it.

What does the hardware cost?

When we bought ours, a board of four cards cost about $2,400. Listings had more than doubled by November 2026 according to Tom’s Hardware. The price depends on the used market, so check current listings.

Does running a model in the office satisfy confidentiality duties?

It removes one outside party from the handling of a file. Whether that satisfies a firm’s confidentiality and privilege obligations is a judgment for the firm’s lawyers, and the firm remains responsible for securing its own systems.

Built for your organization

Want this working on your own matters?