Ablution
Years before the current AI boom, I designed the trust layer for a machine-learning and NLP feature that translated cryptic database shorthand into plain English. The suggestion existed — nobody trusted it enough to use it. I redesigned the interface so a model's guess could earn a human's confidence, and rebuilt the Business Glossary it fed into so an accepted suggestion actually went somewhere.
Highlights
- Designed the trust interface for an ML/NLP suggestion feature in 2018, years before "AI product design" was an established discipline, on the same trust-calibration problem the industry is still solving today.
- A suggestion never replaced the token it explained; it sat alongside it, attributed and reversible, so the model's guess stayed visibly a proposal, never a silent substitution.
- Closed the feedback loop end to end: every approve, reject, or edit fed back into the model, and an approved suggestion could auto-generate a Business Glossary entry with almost no manual effort.
- More than half of customers had fewer than 20 Business Glossary entries (a third had none) before the redesign. Both the rebuilt Glossary and the AI Index shipped and were adopted, particularly among newer customers.
01Project Brief
In 2018 I joined Alation's product design team. Their data catalog included a machine-learning and NLP feature, the AI Index, that scanned a company's data and suggested plain-English names for cryptic database shorthand — genuinely useful, and nearly invisible: buried in an admin menu, never tested with a real user, unknown to most of the customers it could have helped.
Project Brief
In 2018 I joined Alation's product design team. Their data catalog included a machine-learning and NLP feature, the AI Index, that scanned a company's data and suggested plain-English names for cryptic database shorthand — genuinely useful, and nearly invisible: buried in an admin menu, never tested with a real user, unknown to most of the customers it could have helped.
In 2018, I joined Alation's product design team. Alation's data catalog included a machine-learning and NLP feature, the AI Index, that scanned a company's data and suggested plain-English names for the cryptic shorthand database architects use day to day ("sls_rev" to "Sales Revenue"). It was a genuinely useful idea, built on the AI available at the time, years before "AI product design" existed as its own discipline. It was also nearly invisible: buried in an admin menu, never tested with a real user, and unknown to most of the customers it could have helped.
This case study covers the project that followed: rebuilding the Business Glossary the AI Index was meant to feed into, and redesigning the suggestion feature itself so a machine's best guess could actually earn a user's trust, the same problem, in cruder form and with cruder tools, that AI product design is still solving today.
02A Trust Problem, Not a Navigation One
The AI Index and the Business Glossary were two disconnected tools — an approved suggestion had nowhere obvious to go, and the Glossary required its own separate, manual entry process. More than half of customers had fewer than 20 Glossary entries, and almost a third had none at all.
A Trust Problem, Not a Navigation One
The AI Index and the Business Glossary were two disconnected tools — an approved suggestion had nowhere obvious to go, and the Glossary required its own separate, manual entry process. More than half of customers had fewer than 20 Glossary entries, and almost a third had none at all.
Two products, barely connected
The AI Index suggested a human-readable title for a shorthand token, but an approved suggestion had nowhere obvious to go once accepted. The Business Glossary, meanwhile, required its own separate, manual entry process with no connection back to the AI Index at all.
The numbers made the stakes concrete
More than half of our customers had fewer than 20 entries in their Business Glossaries. Almost a third had none at all. This was one problem, trust, split into two workstreams: rebuild the Glossary, and redesign the AI Index, with the second workstream feeding directly into the first.
03What Customers Actually Said
Interviews found Glossary entry creation confusing and unintuitive, rooted in a structural problem: the entry template could only be set from the Document page, not the Glossary page, and there was no Glossary home page at all.
What Customers Actually Said
Interviews found Glossary entry creation confusing and unintuitive, rooted in a structural problem: the entry template could only be set from the Document page, not the Glossary page, and there was no Glossary home page at all.
Interviews surfaced a consistent pattern
Customer interviews consistently found Glossary entry creation confusing and unintuitive. The root structural issue: the template could only be set from the Document page, not the Glossary page, there was no Glossary home page, and no way to start a new term from where a user would naturally be looking.
04Key Decisions & Tradeoffs
Four decisions shaped the redesign: a suggestion had to visually stay a suggestion, never overwrite the original; the feedback loop had to feel like teaching the system, not shouting into a void; an approved suggestion could become a Glossary entry without ever overstepping the human steward who owns it; and none of it mattered without a real Glossary IA underneath.
Key Decisions & Tradeoffs
Four decisions shaped the redesign: a suggestion had to visually stay a suggestion, never overwrite the original; the feedback loop had to feel like teaching the system, not shouting into a void; an approved suggestion could become a Glossary entry without ever overstepping the human steward who owns it; and none of it mattered without a real Glossary IA underneath.
1) A suggestion had to look like a suggestion, not a fact
Expanded titles were placed as subtitles beneath the original shorthand token, never replacing it. The model's guess stayed visibly a proposal, attributed and reversible, never a silent substitution.
2) Making the feedback loop visible
Every suggestion approve, reject, or edit fed the model. The design challenge was making that loop legible to the user, so correcting a bad suggestion felt like teaching the system something, not just dismissing a wrong answer into the void.
3) Letting a suggestion become an action, without overstepping
An approved token expansion could auto-create a Business Glossary entry ("rev" to a "Revenue" glossary page, linking every object that used that token). Glossary entries are owned by Data Stewards, so the automation had to be obviously reviewable and attributable, a Data Steward's territory, populated by a machine, never seized by one.
4) None of it mattered if the Glossary itself was still hard to use
A parallel IA rework covered the Glossary itself, studying Google Docs and Dropbox for how they treat create, browse, and organize as one continuous flow:
- A real home page. None existed before.
- An "Add New" glossary action available from where a user would actually be looking.
- An "Add Term" button inside each individual glossary.
05Testing With Real Users
Both redesigns were validated with InVision click-through prototypes, first in-house, then with paid remote subjects. AI Index testing recruited people with SQL and analytics backgrounds specifically, and every subject completed all four comprehension tasks successfully.
Testing With Real Users
Both redesigns were validated with InVision click-through prototypes, first in-house, then with paid remote subjects. AI Index testing recruited people with SQL and analytics backgrounds specifically, and every subject completed all four comprehension tasks successfully.
Two rounds, two related tools
Both redesigns were validated via InVision click-through prototypes, tested in-house then with paid remote subjects via Respondent.io. Glossary testing covered creating a glossary, adding and deleting entries, adjusting settings, and adding an existing document. One recurring stumble was the settings panel, which I revised in response.
AI Index testing focused on comprehension, and I recruited subjects with SQL and analytics backgrounds specifically, not a general population. Every subject completed all four tasks successfully.
06Outcome & Reflection
Both redesigns shipped and were adopted, particularly among newer customers. What I'd do differently: the trust signal was binary, a token expansion the model was highly confident about and one it was barely guessing at looked identical — a limit of 2018-era AI, not a design mistake.
Outcome & Reflection
Both redesigns shipped and were adopted, particularly among newer customers. What I'd do differently: the trust signal was binary, a token expansion the model was highly confident about and one it was barely guessing at looked identical — a limit of 2018-era AI, not a design mistake.
Impact
Both redesigns shipped, handed off to engineering in two phases (Business Glossary first, AI Index second) with QA and bi-weekly check-ins. I left Alation shortly after the AI Index hand-off but stayed in touch. Customer Success confirmed the rebuilt Glossary was in active use, particularly among newer customers, and AI Index feedback was positive, with customers specifically praising creating a Glossary entry directly from a suggested expansion.
What I'd do differently
The trust signal was binary, approved or not. A token expansion the model was highly confident about, and one it was barely guessing at, were presented identically. That's a limit of 2018-era AI, not a design mistake: the AI available in 2018 didn't give me much more than a binary output to design around. The underlying question, though, how do you get a user to correctly calibrate their trust in a machine's output, is the same one I've been working on ever since.
What This Demonstrates
I was solving AI trust-calibration problems in 2018, before the discipline had a name, with tooling far cruder than what's available now. The specific mechanisms were simpler, but the underlying question, attributed versus silent, reversible versus final, was the same one I still design against today.
Rebuilding the Glossary alongside the suggestion feature, rather than treating them as separate problems, is the same instinct that shows up throughout my later work: a good AI-facing interface almost always needs the system it feeds into fixed too, not just the AI-facing surface polished.