TRAVIS: NYC DOE Conversational AI Assistant
NDA2022 to present · NYC Department of Education, Office of Pupil Transportation & District Programs
Production AI assistant that turns plain English staff questions into live, permission scoped data queries and workflows, plus a public demo of the same pattern you can run right now.
The production system's architecture and outcomes are shareable at this level of detail; internal prompts, code, and proprietary implementation details are restricted under NDA. The live demo below is a separate, from scratch rebuild of the pattern over a fictional company and fully synthetic data, with no client material in it.
TRAVIS is a production conversational AI assistant for NYC DOE's Office of Pupil Transportation. It turns plain English staff questions into live, permission scoped data queries, reports, and multi step workflows, cutting certification processing time by over 20%. That system runs on real staff and student adjacent data, so it cannot be published. I rebuilt the pattern from scratch as Fleetline: a public assistant over a fictional logistics fleet, with synthetic data, that you can open and interrogate yourself. Same shape end to end, none of the client material.
Live demoAsk a fictional fleet's assistant a plain English question and watch the same pattern run end to end: auth and role scope, a plan cache check, two stage classify then generate query planning, schema and permission validation, guardrails, and an answer as a table, a report, or a guided workflow. Fictional company, generated data, no key needed.Open the live demoInteractive prototype
The problem
Before TRAVIS, staff had to navigate multiple systems and manual reports to answer routine operational questions, slowing down certification and day to day decisions. The hard part was never the chat box. It was making a language model safe enough to sit in front of live operational data: it must never invent a number, never return a row the person asking is not allowed to see, and never write anything without an explicit confirmation. And because the production system is under NDA, I needed a way to show that machinery actually running. That is what the Fleetline demo does.
System design
TRAVIS request flow
Safety layers (as rebuilt in the Fleetline demo)
Process & prototyping
In production (under NDA)
- Two stage LLM agentIntent and domain classification first, then schema validated query generation with Pydantic.
- Async FastAPI backendPython 3.11, 15 endpoints, migrated from an earlier Node.js implementation.
- Retrieval augmented query planningFew shot retrieval plus a semantic query plan cache over Qdrant vector embeddings, cutting redundant LLM calls.
- Layered guardrailsNeMo Guardrails, PII masking, and prompt injection defense.
- Auth and permissionsJWT authentication, role based access control, and a Redis backed permission cache.
- Containerized deliveryMulti stage Docker builds.
In the public demo (Fleetline)
- A fictional fleet, generated end to endThe whole pipeline rebuilt from scratch over an invented logistics company: roughly 260 people, 340 vehicles, and 450 credentials across 8 depots, all from a seeded script.
- The model never writes a queryStage one picks the intent and data domain, stage two emits a small structured query plan against a compact schema description, and a validation pass repairs or drops anything that does not match before a single row is read.
- A plan cache keyed on the questionNot on the answer, and only reused above a strict similarity threshold, so a near identical question skips both model calls while a subtly different one does not silently inherit the wrong plan.
- Role scoping that actually bitesFour disclosed test roles gate which data domains are visible and filter results to the caller's own depots. Switching role in the UI visibly changes what comes back.
- Guardrails on screen, not hiddenThe cache hit or miss, every guardrail pass or fail, and per stage timings are all visible, and a multi step workflow writes nothing until the final confirm step.
- Runs with no API keyMock mode swaps the two model calls for deterministic rules while the cache, validation, permission checks, and real queries still run, so its reported numbers are measured rather than claimed.
Outcome & impact
TRAVIS is live in production, and the public rebuild is live alongside it so the pattern can be inspected without touching client data.
In production
In the public demo (Fleetline)
- Every canonical question answers correctly.
- A row scoped role cannot see another depot's people.
- A count that cannot be answered honestly says so instead of guessing.
- A workflow writes nothing before the confirm step.
Role & collaboration
Team
Built as part of Kshitij's ongoing NYC DOE engagement via Athreya Inc.