DESIGNING TRUST INTO A MACHINE'S BEST GUESS — WITH THE AI TOOLS AVAILABLE IN 2018
PROJECT BRIEF
In 2018, I joined Alation's product design team. Alation's data catalog included a machine-learning feature — the AI Index — that scanned a company's data and suggested plain-English names for the cryptic shorthand database architects use day to day ("sls_rev" → "Sales Revenue"). It was a genuinely useful idea, built on the AI available at the time. It was also nearly invisible: buried in an admin menu, never tested with a real user, and unknown to most of the customers it could have helped.
This case study covers the project that followed: rebuilding the Business Glossary the AI Index was meant to feed into, and redesigning the suggestion feature itself so a machine's best guess could actually earn a user's trust — the same problem, in cruder form and with cruder tools, that AI product design is still solving today.
Role: Product Designer, solo UX • Timeline: 2018–2019 • Company: Alation — product name and visuals altered here to respect confidentiality
Problem: a machine-learning feature that suggested real-world names for cryptic data shorthand existed, but was buried, untested, and effectively unused
Solution: redesigned the suggestion interface so users could see and trust the model's reasoning, closed the feedback loop so corrections visibly improved future suggestions, and integrated the tool into the Business Glossary workflow it was built to feed
Results: both the redesigned Business Glossary and the AI Index shipped and were adopted — customer success reported active use post-launch, particularly among newer customers who hadn't yet built up a manual glossary
Two products, barely connected
At its core, Alation's product is a comprehensive data catalog: every data object a company owns — a database, a table, a single column — gets its own page, in a consistent format, so analysts and business users can find, understand, and share it. Database architects tend to name these objects in shorthand ("tokens," in Alation's terminology) that's fast to write in SQL and nearly opaque to everyone else. The AI Index used machine learning to scan those tokens and suggest an expanded, human-readable title for each one — a real-world name a user could actually recognize.
The two products that should have worked together didn't. The AI Index's suggestions had nowhere obvious to go once approved, and the Business Glossary — the feature meant to house company-wide definitions — required its own separate, manual entry process with no connection back to the AI Index at all. Two tools solving related problems, built and shipped without ever being asked to talk to each other.
The numbers made the stakes concrete
Usage data told the same story from the Glossary side: more than half of our customers had fewer than 20 entries in their Business Glossaries. Almost a third had none at all. For a feature meant to be a company's shared source of truth for its own terminology, an empty glossary isn't a minor gap — it's the feature failing at the one thing it exists to do.
The two problems were really one problem: neither tool gave a user a reason to trust it enough to rely on it. I split the resulting project into two connected workstreams — rebuild the Business Glossary so it was worth filling in, and redesign the AI Index so its suggestions were worth accepting — with the second workstream feeding directly into the first.
Interviews surfaced a consistent pattern
I ran a series of customer interviews to understand why Glossary engagement was so low. The pattern was consistent: customers described the process for creating a Business Glossary entry as confusing and unintuitive. Several said they'd tried, then abandoned it early. Others — including customers with several hundred existing glossary terms, who clearly valued the feature in principle — still called the process overly complicated.
Underneath both complaints was the same structural issue: the workflow required a template to organize documents into individual glossaries, that template could only be set from the Document page rather than the Glossary page itself, and there was no Glossary home page and no way to start a new term from where a user would naturally be looking. Every step added friction a busy analyst wasn't going to push through.
1) A suggestion had to look like a suggestion, not a fact
The AI Index's expanded titles appeared in Alation as subtitles beneath the original shorthand name — a small but deliberate choice. The model's guess was never allowed to replace the token it was explaining; it sat alongside it, clearly attributed, waiting on a human decision. Given the AI available in 2018, the model was going to be wrong sometimes. The interface's job was to make that acceptable — visibly a proposal, never a silent substitution.
2) The feedback loop had to be visible, not just functional
Every suggestion could be approved, rejected, or edited, and the model used that input to improve future suggestions. That loop existed at the engineering level from early on. The design problem was making it legible to the user — someone correcting a bad suggestion needed to feel like they were teaching the system something, not just dismissing a wrong answer into the void. Framing every correction as an input the model would actually use was what made rejecting a suggestion feel productive instead of like a dead end.
3) Letting a suggestion become an action, without overstepping
The most valuable move available was letting an approved token expansion automatically create a Business Glossary entry — the token "rev" approved as "Revenue" could generate a Revenue glossary page and link back to every data object using that token, with almost no manual effort. But Glossary entries are typically owned and governed by a company's Data Stewards, and an AI-generated entry appearing unannounced in their glossary risked feeling like the tool overstepping its role. The design had to make the automation obviously reviewable and attributable — a Data Steward's territory, populated by a machine, never seized by one.
4) None of it mattered if the Glossary itself was still hard to use
An AI Index feeding a Business Glossary nobody could navigate was still a dead end. In parallel with the AI Index work, I redesigned the Glossary's information architecture: a real home page, an "Add New" glossary action available from where a user would actually be looking, and an "Add Term" button inside each individual glossary so starting an entry no longer required leaving the page to find a buried menu. I looked at Google Docs and Dropbox for reference — not for their subject matter, but for how mature document-organization tools handle create, browse, and organize as one continuous flow instead of three disconnected ones.
Two rounds, two related tools
Both redesigns were validated with click-through prototypes built in InVision, tested in-house first and then with paid remote subjects recruited through Respondent.io. For the Business Glossary, subjects were tested on creating a new glossary, adding and deleting entries, adjusting glossary settings, and adding an existing document to a glossary. Results were strong overall; the one recurring stumble was the settings panel, which I revised directly in response.
For the AI Index, testing focused on comprehension as much as usability — whether subjects understood what the tool was doing at all, and whether they could confirm or reject a suggested token expansion with confidence. Because the concepts of tokens and expansions needed some prior context, I recruited subjects with SQL or analytics backgrounds rather than a general population — testing the design against the audience it was actually built for. Every subject completed all four tasks successfully.
Impact
Both redesigns shipped. I handed off to engineering in two phases — Business Glossary improvements first, the Integrated AI Index second — running quality assurance and bi-weekly check-ins through each build. I left Alation shortly after the AI Index hand-off, but stayed in touch: customer success confirmed the rebuilt Business Glossary was in active use, particularly among newer customers who hadn't yet built up a manual glossary of their own, and follow-up feedback on the Integrated AI Index was positive — customers specifically called out how much easier it was to create a Glossary entry directly from a suggested expansion, which was the entire point of connecting the two tools in the first place.
What I'd do differently
The trust signal I designed was binary: a suggestion was either approved or it wasn't. A token expansion the model was highly confident about and one it was barely guessing at were presented identically. That's a thinner signal than the problem actually deserved, and it's the piece I'd push hardest on revisiting today — not because the idea was wrong, but because the AI available in 2018 didn't give me much more than a binary output to design around. The underlying question, though — how do you get a user to correctly calibrate their trust in a machine's output — is the same one I've been working on ever since.
AUGUR AI
A tool for Heuristics and Accessibility evaluation, powered by AI via the Claude API, for any URL.
AUGUR CASE STUDY
VISA DASHBOARD
The Visa Government Insights Dashboard, a real-time economic view for government and non-government policy makers.
VISA DASHBOARD CASE STUDY
KAISER MEDICAL
Streamlining the Kaiser Permanente medical claims experience to make billing more comprehensible.
KAISER CASE STUDY
LinkedIn
Dribbble
Medium
Behance
