AUGUR • AN AI-NATIVE TOOL FOR HEURISTIC & ACCESSIBILITY EVALUATION
PROJECT BRIEF
Heuristic and accessibility audits are two of the most consistently valuable — and consistently skipped — steps in real design work. They're slow, they require specific expertise, and under deadline pressure they're the first thing cut. AUGUR is a tool that removes the excuse: submit any URL, and it returns a structured heuristic evaluation against the Nielsen Norman Group's ten usability heuristics, and a WCAG 2.2 accessibility evaluation, side by side, in the time it takes to load a page.
That's only possible because of what's running underneath it. A rules-based tool could check contrast ratios and missing alt text — and AUGUR does exactly that, deterministically. But judging whether a page's structure is confusing, whether its feedback is legible, whether its navigation matches user expectations — that's interpretation, not measurement. AUGUR pairs the two: a real accessibility scanning engine for what can be measured, and Claude for what has to be judged. Building that pairing well — deciding what the model should be trusted to interpret, what it should never be allowed to guess at, and how its output earns trust from someone reading it — is the actual design problem this project solves.
Role: Designer & Builder (solo) • Timeline: July 2026 • Stack: Claude (Sonnet 5), Claude Code, Figma, Next.js, Vercel, Upstash Redis
Problem: heuristic and accessibility judgment is valuable, expert-dependent, and the first thing skipped under deadline pressure
Solution: an AI-native tool that performs both evaluations — one measured, one interpreted — for any URL, structured and trust-labeled, with production-grade safeguards around the cost of running it
▶ TRY AUGUR LIVE ▶ VIEW SOURCE ON GITHUB
Not asking the model to remember a standard — supplying it
A model asked to evaluate a page "against Nielsen Norman's heuristics" from memory alone will produce something plausible-sounding regardless of whether its recollection is accurate. That's a real failure mode for any AI evaluation tool, and it's invisible unless you deliberately test for it. AUGUR doesn't rely on recall: the ten heuristics, their definitions, and the questions each one asks are supplied directly in the system prompt as explicit ground truth, sourced from Nielsen Norman Group's own published Heuristic Evaluation Workbook. The model's job is to apply the standard it's been handed, not to reconstruct one from training data.
The same reasoning shaped the accessibility side. WCAG 2.2 was chosen specifically because it's the current, legally referenced standard — not WCAG 3.0, which remains an incomplete draft; a Candidate Recommendation isn't expected until late 2027, with a final Recommendation, and the years of legal adoption that follow, trailing well beyond that. Citing a standard that isn't finished yet would be citing a moving target.
A third framework — Everett McKay's Intuitive Design — was seriously considered and deliberately cut, for two independent reasons rather than one. First, his principles exist only inside a commercial book with no public reference implementation; encoding them, even reworded, would mean building a free tool partly out of someone else's proprietary IP. Second, and more interesting: a direct test of whether Claude's training actually held McKay's specific enumerated principles with any real fidelity came back negative — the model recognized his premise but explicitly declined to reproduce a list, correctly naming the risk that doing so would be confabulation, not recall. Nielsen Norman's ten heuristics passed that same test cleanly. Two unrelated failure modes, one framework, the same conclusion — cut it, keep the two that were actually grounded.
1) Two layers: what's measured, and what's judged
Every evaluation runs two engines in the same pass. axe-core scans the live DOM and returns deterministic, verifiable violations — a missing alt attribute either exists or it doesn't. Claude never re-evaluates or second-guesses that data; the system prompt instructs it to treat the scan results as verified fact and explain them, not judge them. Heuristic evaluation is the opposite case by necessity — there's no scanner for "does this page's structure match user expectations" — so that's where the model is asked to actually interpret, against the grounded framework described above.
Keeping those two roles separate, rather than asking one model call to estimate everything from a screenshot, is what makes the accessibility tab's citations trustworthy: every WCAG reference on that tab traces back to a real, reproducible scan finding, not a visual guess.
2) Constraining output with a schema, not a hope
Every response is returned through Claude's tool-use API against a defined JSON schema, not parsed out of free-text. Severity, confidence, and rating fields are all constrained to fixed enums — Critical / Severe / Moderate / Minor, High Confidence / Confident / Suspected / Uncertain — so the model can't drift into inconsistent wording across evaluations, and the frontend never has to guess what shape a response will take. This is a trust decision as much as an engineering one: a badge system only works if the vocabulary behind it is actually fixed.
Confidence tagging is deliberately scoped to heuristic findings only. Accessibility findings come from a deterministic scan and are already verified fact — a confidence badge there would just be noise. The distinction between "measured" and "interpreted" is made once, structurally, rather than repeated on every card.
3) When the schema wasn't enough on its own
A schema constrains the field type, not necessarily the exact word choice inside it. axe-core labels one of its own severity levels "serious" — and despite both the prompt and the enum instructing the model to report that as AUGUR's own "Severe" label, the raw word occasionally leaked straight through into the badge. The screenshot alongside this is that bug, live, from mid-build testing.
The fix wasn't a bigger prompt — it was a second, independent layer: a small server-side normalization step that remaps any known alias before the response ever reaches the frontend, plus a client-side fallback style so that any future unrecognized value degrades gracefully instead of rendering an invisible badge. Prompting for correctness and verifying it in code aren't the same thing, and this is the specific case that proved it.
axe-core's own wording ("Serious") slipping past both the prompt instruction and the schema enum.
The system prompt itself went through eleven tracked revisions over the course of the build — kept as a versioned log, not edited in place — as specific failure modes surfaced under real testing: truncated output, inconsistent severity ordering, over- and under-eager issue counts. None of that tuning is visible in a finished product; it's the difference between a prompt that works on the examples you tried and one that holds up against a URL you've never seen.
1) The empty state teaches, before it evaluates anything
Most of AUGUR's likely first-time visitors won't know what "heuristic evaluation" means, or why WCAG 2.2 specifically — and a tool built on AI judgment has an extra burden most tools don't: explaining why its output should be trusted, before it's produced any. The pre-submission state isn't a blank placeholder; it's a short, plain-language explanation of both frameworks and why each was chosen, written to be read before a single URL is entered.
2) A loading state that admits the wait, honestly
A real evaluation launches a browser, screenshots the live page, runs a full accessibility scan, and waits on a Claude response — realistically tens of seconds, not milliseconds. Rather than a generic spinner, a slow cross-fading sequence of phrases keeps the wait legible and on-brand, timed deliberately (a slow fade in, a long hold, a slow fade out) so it reads as considered rather than anxious.
3) A real component system, not one-off cards
Every card in the interface is a Figma component built on the same token set the app's CSS uses directly — the header bar's color and icon are driven entirely by the severity or rating value, so a "Critical" issue and an "Excellent" working-well finding are visually unmistakable from across the room, not just on close reading. Two card types share the same shell: an issue card (severity, description, recommendation) and a working-well card (rating, confidence) — different content, same visual grammar.
From a bare prototype to a working pipeline
The project started under a working name, HEURISTIC — a single input field and a title, wired to a raw Claude response with no structure at all. AUGUR came later, tested against options across several registers — the clinical (RUBRIK, Site-Crit), the irreverent (RoastMySite.com) — before landing on a name that works as both noun and verb: something that reads surface patterns to surface a hidden, structural truth. The renaming happened before the product had a card system, a color language, or a second evaluation lens — it was a bet on what the tool was for, made early.
Directing Claude Code through the build meant working the way I'd brief an engineering team: reviewing every response against intent, catching what didn't match, and deciding what "correct" meant in cases neither of us had fully anticipated. Testing deliberately included sites nothing like a typical client homepage — a single-purpose novelty page, a frozen 1996 web relic, a page that's just a solid color block — specifically because edge cases are where a prompt's real failure modes show up, not the well-behaved sites it was tuned against.
Testing against sites designed to break assumptions
A novelty single-purpose site answers one question and nothing else — exactly the kind of page a rigid rubric might penalize for "missing" navigation or content it was never meant to have. AUGUR's evaluation of a one-word answer site correctly identified it as, arguably, one of the most efficient websites in existence — question in, answer out, zero clicks required — rather than dinging it for the empty nav it was never going to need. Recognizing restraint as a legitimate design choice, not a deficiency, is a genuinely harder judgment call than flagging an actual violation. A blank, single-color page is the opposite stress test: confirming the tool reports "nothing here to evaluate" honestly, rather than inventing plausible-sounding findings to fill space.
Debugging production, not just development
The gap between "works locally" and "works deployed" showed up hard on first launch. Vercel's build silently dropped the accessibility scanner's script file — it's read via a runtime-constructed path rather than a static import, deliberately, to route around a separate Next.js dev-bundler bug, but that meant the build's automatic file tracing couldn't detect it as a dependency and never shipped it. Fixed by explicitly declaring the file in next.config.js, confirmed by inspecting the actual build trace manifest rather than guessing.
The next deploy broke differently: Vercel has no browser pre-installed at all, and the serverless-Chromium package's bundled binary — 64.8MB on its own — silently exceeded the platform's function size limit. The fix was a variant of that same package that downloads its binary from a hosted release at cold start instead of bundling it. A third, harder-to-see bug followed once that was live: warm serverless containers getting reused across requests meant Chromium's default profile directory was never cleaned up between invocations, intermittently crashing the browser mid-navigation — traced to a documented issue in the library itself, fixed by giving every invocation its own disposable profile directory.
Protecting the product from its own cost
Every evaluation is a real, billed AI call — a public tool with no usage limit is a public tool with an unbounded bill. AUGUR caps evaluations at three per visitor per rolling seven-day window, checked before the browser launches, the scan runs, or Claude is called — the whole point is avoiding that cost for a request over the limit, not spending it and rejecting the result afterward.
Visitors are identified by two independent signals — a persistent cookie and IP address — each checked separately, blocked if either is exhausted. A single combined key would have meant either signal alone (clearing cookies, one click) resets the count to zero; requiring both to be defeated at once is a meaningfully higher bar for the casual case this exists to stop. The counters live in Upstash Redis with a true sliding-window algorithm — not a fixed-TTL counter that resets all at once, which is a subtly different (and weaker) guarantee than what "3 per week" actually implies.
What I'm proud of
The two-layer architecture held up under real, unpredictable testing — dense sponsor-heavy pages, minimalist novelty sites, and outright blank pages all produced honest, appropriately-scoped output rather than the tool straining to find a fixed number of things to say. The trust signals (confidence tagging, cited WCAG criteria, the severity taxonomy) aren't decoration; they're doing the actual job of telling a reader which parts of the output are measured fact and which are judgment worth double-checking.
What I'd improve next
The most important unbuilt piece is disclosure pacing — a deliberate design decision, not yet implemented, for how many findings to surface at once. A flat cap creates a real dishonesty problem in either direction: too low and a genuinely broken site has real failures silently hidden; too high, or padded to hit a round number on a clean site, and the tool starts manufacturing content to look thorough. The intended model: two strengths and five to six priority issues shown by default, strictly ordered by severity with nothing critical ever hidden, the rest revealed progressively rather than dumped all at once. Worth building once the cost of doing so is weighed against everything else still ahead of it.
Annotation markers directly on the captured screenshot — mapped to axe-core's precise element coordinates for verified findings, approximate positioning for interpreted ones — would close the last gap between "here's what's wrong" and "here's exactly where." A shareable, persistent evaluation link is the other piece: encoding a full report into a compressed URL rather than a database row, so a result can be sent to a stakeholder without either side needing an account.
VISA BENEFITS
Visa Digital Benefits Platform: Designing the tools for banks to assign benefits to cardholders.
VISA BENEFITS CASE STUDY
KAISER MEDICAL
Streamlining the Kaiser Permanente medical claims experience to make billing more comprehensible.
KAISER CASE STUDY
VISA DASHBOARD
The Visa Government Insights Dashboard, a real-time economic view for government and non-government policy makers.
VISA DASHBOARD CASE STUDY
LinkedIn
Dribbble
Medium
Behance
AUGUR: Heuristics Tab — Production Result
Nielsen Norman Group: Jakob's Ten Usability Heuristics
W3C: How to Meet WCAG (Quick Reference)
AUGUR: Accessibility Tab — WCAG 2.2 Citations
AUGUR: Figma Badge Component System
AUGUR: The "Serious" Severity Leak, Mid-Build
AUGUR: Empty State / Onboarding Copy
AUGUR: Loading State
AUGUR: Figma Card Component, Header Variants & Rendered Types
AUGUR: Day One (as "HEURISTICS")
AUGUR: First Real AI Response
AUGUR: Testing Against a Minimal Novelty Site
AUGUR: Testing Against a Blank Page
AUGUR: First Production Deployment Failure
AUGUR: Live in Production
AUGUR: Running, Full Card Interface