Blog · July 15, 2026 · 6 min read
Building an AI Email Assistant: Notes from Shipping Emmy
Architecture notes from building Emmy, a production AI Gmail assistant: OAuth done once, continuous learning from user behavior, safe auto-send, and the trust ladder.
At Infolyze I built Emmy, an AI Gmail assistant that runs 24/7: it sorts the inbox, creates labels, drafts replies, and learns each user's habits over time. It was one of four AI products I shipped there, and email turned out to be the best teacher about trust in AI products. Here's what generalizes.
OAuth once, then be invisible
The entire onboarding is a single Google OAuth consent. Everything after that happens server-side against the Gmail API — no extension, no client to keep open. The lesson: every additional setup step halves activation. If your AI product needs the user present to function, it's a tool; if it works while they sleep, it's staff.
Learn from behavior, not questionnaires
Nobody fills in a 20-field preferences form about how they like email sorted. Emmy watches instead: which drafts get sent unedited, which labels the user moves messages out of, who they always reply to fast. Preferences are inferred, stored as explicit facts, and applied on the next pass — a live version of the memory principle from production agent systems.
The trust ladder
- 01Observe: read-only, show the user what it *would* have done.
- 02Organize: apply labels and priorities — reversible actions only.
- 03Draft: prepare replies that wait for one-tap approval.
- 04Act: auto-send for narrow, proven categories the user has explicitly promoted.
Every user climbs at their own pace, and any wrong action drops that category one rung. Auto-send is earned per-category, never global. This ladder — reversible before irreversible, observed before autonomous — applies to every AI product that touches something users care about.
Guardrails that mattered
- Hard blocklist of actions: never delete, never send to new recipients without approval, never touch threads with legal/financial keywords.
- Every action logged with a one-click undo where the API allows it.
- Daily digest of what was done — silence breeds suspicion, summaries build trust.
- Rate limits per hour, because a runaway loop in someone's inbox is unrecoverable reputationally.
Email is a high-stakes, high-frequency surface — exactly where AI automation pays off and exactly where mistakes are unforgivable. If you're weighing a build like this for your own workflow, start with the framework in when to automate.
Haider Farooq is an AI engineer and data scientist based in Lahore, Pakistan — core engineer on TryCook.ai, developer at Aligno, and creator of MarkSafe.net. He builds agentic AI systems, RAG pipelines, and automation for teams worldwide. Work with him.
RELATED WORK