Skip to content
fraudforensicspdfarchitecturetestingtamperlens

Document Fraud Detection Without ML: in defense of the deterministic engine

Douglas Haruo 12 min 9/24/2026

Tamperlens analyzes documents for signs of tampering and contains not a gram of machine learning: no model, no GPU, no third-party call. The most exotic dependency in its package.json is a SQLite driver. In a market that sells “AI” in the headline, that looks like backwardness. And it is the most deliberate architecture decision in the product.

This post is its defense, in two parts. The first: why deterministic rules, which always give the same output for the same input, beat ML in this specific domain. The answer does not generalize to all detection. The second, less obvious: what the promise of determinism demands of the engineering to be true. Because “same bytes in, same report out” does not happen by accident and does not survive without tests.


Part 1: why no ML (four arguments, one domain)

1. Evidence in this domain is structural, and structure is verified, not estimated

Document fraud has an asymmetry most detection domains lack: appearance is the layer the forger controls; structure is the one he forgets. An edited PDF almost never looks edited. But it carries incremental revisions, replaced objects, dates that disagree with each other, a signature covering fewer bytes than the file has. These facts are binary and verifiable: either the revision chain exists in the bytes, or it does not.

A trained classifier answers “this document resembles the fraudulent ones I’ve seen”. A structural rule answers “this document was saved again after it was signed, and here is the offset”, the exact position in the file. For someone who has to act on the result, the second answer is usable and the first is a guess with a percentage attached. Acting here means refusing a document, escalating a case, defending the decision in a dispute.

2. In an adversarial game, the model learns the past; the rule verifies the present

Fraud detection is a game against an adapting adversary. A model learns the distribution of yesterday’s frauds. Tomorrow’s forger adjusts the appearance, which is exactly the feature the model sees. And there is a public measurement illustrating the ceiling: a frontier multimodal LLM evaluated on forged documents scored an AUC of 0.509. Statistically, that is a coin flip.

The structural rule is not unbeatable. It is falsifiable in a different way. The adversary can rewrite the whole file to erase a structural trace, and that is measurable. The product’s evasion bench runs the fixtures, the test files, through real rewriting tools. It records, family by family, what survives and what dies, in versioned JSON. Knowing exactly what a file-laundering pass erases, and publishing it, is only possible because the detector is deterministic. There is no honest evasion matrix for a model that changes its answer with the temperature.

3. Your buyer needs to be able to regression-test your detector

The property is written into the architecture documentation as a technical sales argument. Same input bytes → same report is something a fintech customer can pin in a regression test. The customer integrates the API and stores input/output pairs for the cases that matter. Their CI breaks if a detector upgrade changes a verdict their flow depends on. And there is a changelog to read, because a behavior change is a version change.

Try promising that with a periodically retrained model: every retrain silently moves the decision boundary, in directions not even the vendor can enumerate. For a system in a compliance chain, a detector that changes its mind without a version is not a component. It is a source of incidents.

4. Sensitive documents should not travel

With no external model and no third-party call, the document comes in, is analyzed on CPU, and the report goes out. Nothing from the content feeds training, nothing crosses a data boundary. In a product that analyzes contracts, statements, and identity documents, “the file goes nowhere” is not a privacy feature. It is a precondition for adoption.

Part 2: what determinism costs, or engineering the promise

Saying “deterministic” is easy. The five decisions below are what keep the word from rotting.

The clock is read once and injected

The sneakiest source of non-determinism in an analyzer is not randomness. It is Date.now(). Date-anomaly signals compare document dates with “now”. If each module reads the clock at its own instant, the same file produces different reports on different runs. And a test that pins the output cannot exist.

In the engine, the analysis motor, now is resolved once per inspection and propagated to all signals as an argument. The caller can pin it, including the test bench. “Same bytes in, same report out” becomes a property the suite genuinely asserts, with the clock pinned to a known instant.

Even the order of signals is specified

The report orders signals by severity and, within a severity, by id, in alphabetical and stable order. It sounds cosmetic. It is not. Unspecified order means three things. Snapshot diffs that change without cause. Clients that accidentally depend on an object’s iteration order. And a “same report” that is only the same after normalization. Determinism that requires normalization on the verifier’s side is determinism by half.

Weights are versioned: recalibrating IS releasing a version

The aggregate score, the final number, is born from per-severity weights, floors, and a diminishing-returns function. The comment in the scoring module states the house rule: weights are versioned by the engine version, and changing them is a version bump. There is no “little calibration tweak” outside the changelog. For anyone regression-testing against the detector, recalibration is a behavior change, indistinguishable from any other.

And the score’s design carries semantics, not just arithmetic. Two examples worth stealing:

  • Diminishing returns per severity: from the third signal of the same severity onward, the contribution decays geometrically. A pile of weak signals converges to a ceiling (~2.5× the severity’s weight) instead of summing to 100. Ten weak warnings are one suspicious document, not ten times a suspicious document.
  • The high band’s reserve: with no high-severity signal present, the score is capped at 69, one point below the high band. The gravest band is reserved, by construction, for “there is at least one finding that alone establishes alteration”. Never for an accumulation of circumstantial ones. The consumer can trust the band’s meaning, not just the number.

One signal failing neither crashes nor goes silent

A signal module that throws has its family skipped, and the failure recorded as a structure warning in the report itself. The alternatives are both worse. One is crashing the whole inspection over a bug in a peripheral signal. The other, the truly dangerous failure mode, is swallowing the exception and letting the family vanish from the report as if the document were clean along that dimension. A silent absence of verification is indistinguishable from verification that passed, and that is how a detector starts lying without anyone changing a line of rules.

The engine is forbidden to hand down verdicts, by test

The most important separation in the system: the engine measures the signals, the evidence and the score. What decides (accept / review / reject) is the service layer, where policy lives. And the boundary is not a code-review convention: a test guarantees that nothing named “verdict” exists under the engine’s directory.

The reason is the same one that separates the lab from the judge. Decision thresholds change per customer, per risk appetite, per context. And they must be able to change without touching the measurement. The day a business rule sneaks into the engine, the customer’s regression test starts breaking on policy changes. The measurement loses the only authority it had: that of having no opinion.


Where this architecture does not fit, and the summary

The honesty that closes the argument: deterministic rules work here because the domain’s evidence is structural and enumerable. Sometimes the evidence is perceptual: “is this face photo the same person?”, “does this handwritten signature match?”. There, no structural rule will do, and ML is the right tool. The choice is not ideological. It is about where your domain’s evidence lives.

For anyone building a detector in a structural domain, the summary in six lines:

  1. A rule that cites evidence beats an estimate that cites confidence. That matters for anyone who must act and stand behind the action.
  2. Against an adversary, publish your evasion matrix. And note that only a deterministic detector can have one.
  3. “Same bytes, same report” is a sellable feature: your customer wants to regression-test you.
  4. Determinism is engineered: injected clock, specified order, versioned weights.
  5. A failing signal becomes a visible warning, never a silent absence.
  6. Measurement and policy in separate layers, with a test guarding the boundary.

None of those six lines mentions intelligence. Honest detection rarely needs it. It needs evidence, versions, and a system that does not change its mind in the dark.

Want these numbers for your own alert queue?

Douglas Haruo, KYC/AML engineering at Chipper Cash from 2020 to 2025. The two-week diagnostic measures alert volume, false-positive rate, minutes per alert and cost per alert on the vendor tool you already run, inside your environment.

See the two-week diagnostic →