ClaimsDock

Overview

Most insurance claims are simple. A small number are genuinely ambiguous, missing information, or fraudulent, and those are the ones that eat an adjuster's day — because separating the merely-flagged from the actually-wrong takes judgment, every time, at volume. I wanted to build something that does that sorting honestly: surface what's truly uncertain, ground every claim in evidence and policy language, and leave the decision with a person who can be held accountable for it. A system that does the reading so a person can do the deciding faster, with better information than they'd have otherwise.

I chose medical claims because it's a domain with real regulatory stakes. ERISA governs exactly how a denial has to be justified; fraud patterns worth naming correctly, and enough structural complexity, deductibles, network status, benefit caps, linked claims, that guessing my way through it was never going to work.

This started as a question about what a real agentic system looks like when a router, a grounding layer, a confidence signal, and a human decision-maker all have to hold together as one working system. Medical claims became the domain because nothing about it forgives a shortcut.

Designing around Denise Marsh

I built ClaimsDock's interface around a persona, Denise Marsh. She's a senior claims adjuster nine years into the field, the kind who gets routed the disputed cases rather than the routine ones. Her caution toward automated recommendations isn't invented color. It's grounded in peer-reviewed research, and that research shaped decisions in the product.

Algorithm aversion, and the edit path it justifies

A key tension I designed against is well-documented: people trust algorithms readily on analytical tasks, but that trust drops sharply the moment they see the algorithm make a mistake, more sharply than the same mistake made by a person, and they lean back on their own judgment afterward. Researchers call this algorithm aversion, and its boundaries are studied too: letting people edit an algorithm's output, rather than just accept or reject it, measurably softens the effect. That finding is a direct line to the Human Gate's "Approve with edit" path, which exists so an adjuster never has to reject a correct finding outright just because one number in it is wrong.

Confidence tiers have to be accurate, not just present

One finding changed how I thought about Anchor's confidence tiers specifically: reliance tracks an algorithm's stated accuracy right up until it's actually wrong more often than it said it would be, at which point trust doesn't just decline, it collapses. Anchor's tiers have to be accurate, not just present — an honest "Confident" that turns out wrong does more damage to trust than showing no tier at all. I built the tiers around how directly the evidence supports a conclusion instead of a number meant to look precise.

The same research shaped the Human Gate a second way: people lean on an algorithm's correct suggestions more than its incorrect ones, meaning they're curating a recommendation, not just accepting it. Anchor's citation-plus-disclaimer pattern is my own version of the well-timed nudge that research points toward.

A human checkpoint isn't automatically oversight

The most uncomfortable finding, and the one I take most seriously, challenges an assumption I might otherwise have made by default: that putting a person in the approval path guarantees oversight. Research testing that assumption directly — a study titled “Zombies in the Loop?” — finds it doesn't reliably hold: people accepted AI-generated advice even when told the AI's training data was unknown or presumably biased, and even when explicitly warned to be skeptical of it. A human checkpoint can become a rubber stamp if the human isn't made to engage with the decision.

That's the entire reason denying a claim in ClaimsDock isn't a single click. It requires a four-field justification, checked for coherence against the claim's evidence, catching cases where a stated reason cited the wrong fraud category even though the underlying denial was correct.

One user, four jobs

ClaimsDock is four systems sharing a small number of hard rules, not one AI system with a few features bolted on. A claim moves through all four differently:

• Evaluation Pipeline. Reads every incoming claim automatically and produces evidence, a confidence tier, and a recommendation — no human involved yet.
• Router and Anchor. An embedded assistant an adjuster can ask a question of at any time, in plain language, grounded in the same evidence and policy documents the Pipeline used.
• Human Gate. Sits between any AI output and any actual action, backed by its own guardrails.
• Retrieval layer. Shared by the Pipeline and Anchor, grounds every reply in reference documents instead of the model's memory.

There's exactly one user here, an adjuster, and the complexity lives entirely in how many different jobs the system does around that one person.

Deterministic math meets LLM reasoning

The rule that ties all four together, and the one I held to most strictly through the whole build: severity, SLA math, coverage math, and the category-to-recommended-action mapping are all deterministic code, never a model's judgment call. I locked this in during the first planning pass, before any pipeline code existed, because the alternative — asking a model to compute a dollar amount or a deadline — trades a testable function for a probabilistic one with no upside. This boundary is itself a design decision about reliability: the adjuster can trust the numbers because they're testable, and trust the model's judgment because it's scoped to things only judgment can do.

Why two calls, not one

Every claim goes through exactly two model calls, a design based on my research for AI trustworthiness and reliability. The first call reads the claim and produces evidence and a proposed category. The second call sees only that evidence and category — never the first call's reasoning or the original claim — and independently judges confidence before drafting a recommendation.

This is LLM-as-judge, actually implemented. I split the calls so a model never grades its own homework. A single call that finds a pattern and immediately rates its own confidence is checking its own work. Separating them creates a genuine second opinion.

Confidence tiers over confidence scores

Anchor's four tiers — High Confidence, Confident, Suspected, Uncertain — describe how directly evidence supports a conclusion rather than a probability score. AI is less good at picking a range between 1–100, and much better at following a rubric to determine a category. A tier tied to evidence strength reads like a colleague's assessment, not a statistic to either trust blindly or ignore.

Scaling the test set to 132 claims

The claim set grew from 20 to 132 authored scenarios across clean, ambiguous, missing-data, complex-math, and fraud cases. I rebalanced the distribution when too many ambiguous claims dead-ended at "escalate," making the demo unsatisfying to explore. At this scale I batched claims into chunked calls with a round-robin cursor to keep category mix and token cost even — a distribution that took three live-tested attempts to get right.

The Pipeline runs on Kimi in production — a cost decision made after head-to-head testing against Claude, not by default.

Three documents, cited by name

Every recommendation ClaimsDock makes traces back to one of three reference documents:

• Coverage and Adjudication Policy. Deductible logic, network rates, prior-authorization rules, benefit caps — synthetic but structurally realistic.
• Fraud-Indicator Reference. Standard industry terminology for billing fraud, sourced from OIG enforcement history, DOJ False Claims Act settlements, and CMS's own coding-integrity program.
• Regulatory Deadline Reference. ERISA's tiered decision deadlines.

Anchor and the Pipeline both cite these by name so nothing gets invented or hallucinated.

Embedded RAG hosting

I built retrieval on local, self-hosted embeddings to avoid a dependency this corpus size doesn't need. Each document is chunked on natural section headings, embedded once, and held in memory. No vector database — at a few dozen chunks, direct comparison is instant. I used Hugging Face's transformers.js port, Xenova/all-MiniLM-L6-v2 model.

Fixing a ranking problem with hybrid scoring

A smoke test found the ERISA deadline chunk ranking fifth of six for the exact question it was built to answer. The problem wasn't the content; it was structural. Mean-pooled embeddings dilute a long chunk's vector toward its average, letting short, keyword-dense fragments outrank passages that actually hold the answer.

I added hybrid scoring — blending embedding similarity with keyword overlap — which recovered the ERISA chunk to second place. I left ranking imperfect on purpose. Both Anchor and the Pipeline retrieve several top chunks for synthesis, not just the top result, so a small margin doesn't produce wrong answers.

What Anchor actually does

Anchor is the part of ClaimsDock an adjuster talks to directly. It's a plain-language question box that can look up a claim, re-analyze it, draft a recommendation, select and deselect claims for bulk action, or answer a general policy question, independent of whatever the automatic Pipeline already decided. Underneath it is an agentic tool-use loop: a Router classifies each question into one of six tools, extracts the right parameters, executes the tool, and can chain a second tool call if the first result implies a follow-up is needed.

Three failure modes, three different fixes

I designed against three distinct, separately testable failure modes, because they're different problems with different fixes:

• Wrong tool entirely. The Router picks the wrong one of the six tools for the question.
• Right tool, wrong parameters. The right kind of analysis, pointed at the wrong claim.
• Misjudged scope. Mistaking whether a question is about one particular claim or nothing in particular.

A wrong tool choice is a routing-prompt problem. A right tool with wrong parameters is usually a context problem — the Router defaulting to whichever claim happened to be on screen instead of the one actually named in the question. I tested both separately with an adversarial question set, and caught a bug on the first live run: a question needing two tools in sequence had its second tool call silently dropped, because the original round-trip only handled a single dispatch round. I fixed it as a bounded loop instead of assuming every question resolves in one call.

Anchor remembers across a model switch

Anchor runs on either provider, Claude or Kimi, switchable from a settings toggle, independent of whichever one the Pipeline uses. Anchor's memory survives that switch, too: ask a follow-up after flipping from Kimi to Claude mid-conversation, and Anchor still knows what "that claim" refers to, because the one turn of prior-question memory it carries is stored independent of which model answers it.

Designing the comparison, and the metric that actually mattered

I didn't pick a model provider by reputation. I built a comparison harness and designed the metric I cared about before trusting any accuracy number.

Raw accuracy treats every wrong answer as equally bad, but the deterministic layer doesn't. A wrong category paired with cautious confidence still reaches a human for review. The dangerous failure is a wrong "clean" call paired with a confident tier — a silent miss that auto-resolves with nobody reviewing it. That's the one failure mode the safety architecture can't catch after the fact, so I tracked it specifically.

Accuracy and reliability split into two different findings

Claude averaged 92% category accuracy with a 5% silent-miss rate, but completed cleanly on only four of five runs. Kimi averaged 82% with a 13% silent-miss rate initially, yet finished every run cleanly and runs roughly five times cheaper. I fixed a mislabeled ground-truth claim in my own test set that had been skewing Kimi's numbers, then saw its rate improve to near-Claude levels.

I routed the Pipeline to Kimi in production — a cost-driven choice appropriate for a public prototype with no real patients or dollars at stake. I kept Anchor switchable between both, since its answer quality deserved testing on its own terms. If this were actual enterprise software, I'd have been more careful about locking anything to a single provider.

Personally, I found Kimi's answers inside Anchor were, if anything, more thorough than Claude's for the same questions, and it reached for generated tables and bulleted structure far more often. Claude consistently fell back to prose with the occasional list for the same kind of request.

A free check that never blocks anyone

Nothing in ClaimsDock becomes an approval, a denial, or an escalation without a person clicking to make it happen, and I built two guardrails for the moment that click happens. The first is deterministic and costs nothing: it compares whatever action an adjuster chooses against the Pipeline's recommendation and surfaces a soft confirmation, citing the evidence already on file, if the two disagree. It never blocks anyone from proceeding with their own choice. I scoped this check to every action except Approve-with-edit, on purpose: an override on approval doesn't carry a denial's regulatory stakes.

Four fields, eleven rounds

The second guardrail is where most of the build's difficulty lived. Denying a claim requires four structured justification fields — specific reason, policy provision violated, internal standard applied, and what could reverse the decision — graded for coherence against the claim's evidence before acceptance.

• Wrong document cited. The justification pointed to a policy source that didn't actually apply.
• Fabricated policy provision. The model invented language no reference document contained.
• A genuine corpus gap. A real hole in my reference material that no amount of prompt tuning could paper over.
• A retrieval-recall problem. The same correct citation ranked differently depending on how clinically-worded a claim's evidence happened to read.
• Contradicting rules. Twice, two of my own guardrail rules disagreed with each other on different claims.

Removing a field from judgment, rather than another prompt patch

The eventual fix wasn't another prompt patch. One field was never a genuine judgment call: a fraud denial always follows one confidence path, so the correct policy citation is fixed text. I removed it from judgment entirely rather than keep asking a model to rediscover an answer that never varied.

That saga's lesson led to a permanent regression suite — every prior round had been checked only against the single claim that just failed, with nothing verifying fixes didn't break previously-passing cases. The suite caught exactly that regression on its first run.

Every decision, logged

Every status change, override, escalation, and denial writes to an audit log with who, when, and why. A claim's current status is always its latest log entry, so there's no separate status value that can drift out of sync with the record.

Five directions, narrowed to three with a point of view

The ClaimsDock UI started from a plain layout I sketched early: a masthead, a claims table, some stat tiles, and a side panel of mini-cards for the assistant, before any of the architecture above existed as running code. That rough shape became the seed for five style directions, explored side by side. I narrowed those five to three, keeping the three that each carried a distinct point of view on the same underlying data:

• Ledger. Reads dense and tabular, built for a high-volume queue where information density is the point.
• Clinical. Sans-serif, precision-first, closer to how a clinical record reads.
• Field. Warmer and more approachable, built for a less adversarial read of the same claims.

One token system, six combinations

Each of the three ships in both light and dark, six combinations total, driven by a single token architecture. A style-and-brightness pair is one entry in a shared color and type system, so adding a seventh combination later means adding one token block. The interface went through a lot of hands-on iteration — it's the layer an adjuster spends their whole day looking at.