Pharmacy Virtual Assistant
2025A multi-agent virtual assistant for retail pharmacy, designed and prototyped as a proof of concept.
Figma · Voiceflow · Airtable REST API · SSML
Role
Wrote the system specification, wireframed the interaction flow in Figma, and authored the intent model with its training utterances. Built the prototype in Voiceflow across 27 nodes, including live API integration with a synthetic Airtable database and a RAG knowledge base covering FAQs (hours, address, etc.).
- specialized agents, each with exactly one job
- 5specialized agents, each with exactly one jobspecification §2, Multi-Agent Topology
- clinical questions the bot is permitted to answer
- 0clinical questions the bot is permitted to answerspecification §1, guardrail 2 (enforced as a routing rule, not a prompt instruction)
- path to protected data, and it runs through identity verification
- 1path to protected data, and it runs through identity verificationspecification §1, guardrail 1; the flow has no other edge into the Prescription Status Agent
Problem
A pharmacy phone line has to serve three main types of users: those seeking general information ("How late are you open today?"), those asking about their own prescriptions ("Do I have any refills left on my inhaler?"), and those seeking professional advice ("Is it safe to take aspirin that expired last month?"). The first is routine and low-stakes. The second touches protected patient information. The third demands high-risk professional judgment that a machine has no business providing.
Collapsed into one flat script, those three collide in predictable ways. A single agent that can both answer FAQs and read prescription records has to remember, at every turn, not to read the record until it has established who it is talking to. A system whose privacy guarantee is "the model was instructed not to" has no guarantee, only a strong preference. A single agent that can hold a fluent conversation about medication will eventually be asked a dosage question, and it will answer, because answering is what it does.
The brief was to design that phone line for a fictional retail pharmacy, Gower's, as a production-grade specification that an engineering team could run with, plus a prototype that brings the product vision to life.
Strip the pharmacy out and the shape underneath is the one every regulated phone line has: a class of information that must stay behind an identity check, a class of question that belongs to a licensed human, and a user who does not know or care which class they are in when they dial.
Architecture
Scroll diagramsideways for more →
The full interaction flow in Figma, organized around one axis: whether an intent requires patient verification. Everything on the right of the canvas is unreachable until the API call in the middle returns a match. The identity-repair loop is the branch that checks a retry against the values already captured instead of re-querying the database. Click to zoom.
Research & Data
This was a self-directed design exercise, so the inputs were domain constraints and common failure modes in voice and text systems rather than original user research. Two constraints were fixed before a single node was drawn, and nearly everything downstream is a consequence of one of them: no protected health information is disclosed until identity is positively verified, and no clinical judgment is presumed or conveyed by the system. Establishing both as immutable facts up front is what made the rest of the design arguable. Once they are settled, most design questions have a right answer instead of a preference.
- Splitting the router from the specialist agents (rather than one large prompt handling everything) turns the identity gate into a structural checkpoint instead of an instruction the model has to remember to obey on every turn. The difference only shows up under pressure, which is exactly when it matters: a user who is confused, insistent, or deliberately probing is applying conversational pressure to a rule, and a rule that lives in a prompt is negotiable in a way that a missing edge in a flow graph is not.
- Speech-to-text failing on a spoken date of birth is an expected case, not an edge case. Dates are close to the worst thing you can ask a recognizer for: "oh five twelve eighty," "May twelfth nineteen eighty," and "five twelve nineteen eighty" are one date in three shapes, and recognition degrades hardest exactly where the stakes are highest: older users, non-native speakers, and accents underrepresented in training data. The synthetic database's birth years deliberately span 1942 to 2005 so the flow was designed against that whole range rather than an imagined median user.
- The database's own status vocabulary is not speakable. The raw stored values are strings like "On Hold," "Expired," "Ready for Pickup," and "Refill Req Sent." "On Hold" is the dangerous one: internally it is a neutral operational state, but to a user it sounds like the pharmacy has decided to withhold their medication. Any status a user hears has to be re-authored, not read out.
- A verified user usually has more than one prescription, so the interaction problem is disambiguation, not lookup. Every patient in the synthetic database carries two or three active prescriptions on purpose: a single-prescription happy path hides the question that actually has to be designed: which one are we talking about? A flow that only works for a patient with one script is not a flow.
- Escalation has to be reachable from a cold start, not only from a failure. Users open with "let me talk to a person" all the time, and a system that can only route to a human after the user has failed at something has made the human a punishment. The Human Escalation Agent is a peer of the router, not a terminal state hanging off the bottom of the identity flow.
- The intent model has to be trained on the user's words, not the pharmacy's. Training utterances were authored from how people actually ask ("is my script ready," "do you have my medicine," "did my doctor call one in") rather than from the internal name of the transaction: 22 utterances across the patient-specific intents, plus a 34-value entity model covering medication names, dosage forms, dosage units, task types, and insurance carriers.
Explorations
Tried: A Figma wireframe of the full interaction flow, walked end to end before any agent logic existed.
Rejected: Building directly in Voiceflow first was rejected. The identity-gate branching and the escalation paths needed to be legible on a canvas (where an unhandled branch or an escalation with no trigger is visible at a glance) before being committed to a tool that makes structural changes expensive.
Tried: Organizing the entire wireframe around a single axis: whether a given intent requires patient verification. The canvas is split into a "Verification: NOT Required" half and a "Verification: REQUIRED" half, and every intent is placed on one side or the other before anything else about it is designed.
Rejected: Verifying every user at the top of the interaction was rejected, even though it is simpler to build and simpler to reason about. It would mean asking a user to prove their identity to find out whether the pharmacy is open on Sunday. That's a privacy control applied where there is no privacy to protect, which teaches users that verification is a toll booth rather than a gate that means something.
Tried: Routing clinical questions to a human as a hard interrupt, from any point in the interaction.
Rejected: A hedged clinical answer ("I'm not a pharmacist, but generally...") was rejected outright. It is the most tempting option in the design, because it feels helpful and it is what a fluent model will do by default. It is also the single worst outcome available: a user who receives a plausible, confident, wrong answer acts on it, and the hedge does not travel with the advice.
Tried: A synthetic Airtable database populated with real prescription shapes: eleven fields per record including sig directions, prescriber, date written, refills remaining, and fill history.
Rejected: A flat mock lookup returning a canned status string was rejected. The disambiguation flow, the status-normalization matrix, and the multi-prescription readout are only designable against records that have the messiness of real ones: multiple scripts per patient, the same medication at different strengths, statuses that do not map cleanly onto anything a user would recognize.
Guardrails & Human Oversight
An agent that can act on a user's behalf needs a different set of design questions than a screen does. Not "can they find the button" but: what can the user stop, what does the system admit when it fails, what happens when the input channel itself breaks, and who is accountable when the answer matters. Here are six key design decisions built directly into the agent specifications.
User control
A user can ask for a human at any point and get one: the Human Escalation Agent is a peer of the router, reachable as an opening line, not only as the end of a failed path. On the clinical branch the bot does not transfer silently either: it explains that it cannot answer clinical questions, then offers the transfer as a choice the user accepts or declines.
Figma wireframe, clinical branch ("Explain cannot answer" → "Offer transfer" → Yes/No); specification §5, Agent 5, trigger 3
Error recovery
After two failed speech-to-text attempts on a date of birth, the agent stops retrying the channel that is not working and changes input modes entirely, to DTMF keypad entry, with a worked example spoken aloud: "enter your 8-digit date of birth, for example 0-5-1-2-1-9-8-0 for May twelfth, 1980." The example is there because "8-digit date of birth" is ambiguous until someone shows you one.
specification §5, Agent 2, step 2 (DOB capture and fallback mechanics)
Repair, not just retry
If a user's second attempt produces the same name and date of birth as the first, the flow does not re-send an identical query to the database and report the same failure. It compares the new input against the values already captured, and when they match, tells the user no record was found rather than looping.
Figma wireframe, identity-repair branch ("Match existing variables?" → Yes → "Inform user no record found")
Transparency
When the retry cap is reached the user is told why, in plain language, and told what happens next: "For your security and privacy, I cannot access patient records without verification. I will transfer you to a pharmacy team member who can help." Not a vague failure, not a silent stonewall, and not a quiet relaxation of the gate to end an awkward moment.
specification §5, Agent 2, step 3 (retries ≥ 3)
Human oversight
Three named escalation triggers (clinical question, failed authentication, explicit user request) each write an audit reason into shared state before the transfer executes: CLINICAL_QUERY, AUTH_FAILED, USER_REQUESTED. The reason an interaction left the automated system is recorded as data at the moment it happens, rather than reconstructed from transcripts afterward.
specification §3, v_transfer_reason; §5, Agent 5 triggers
Feedback
Verification is confirmed out loud ("Thank you, your identity has been verified") before anything protected is spoken, so the user knows the gate opened and can tell the difference between a system that is looking something up and one that is still deciding whether it is allowed to.
specification §5, Agent 2, step 3 (1 record returned)
Interaction Decisions
Some interaction design choices and the concrete failures they prevent.
A dedicated Identity Gate Agent collects and verifies name and date of birth against the live Airtable database before any other agent can reach patient-specific data. It is the only edge in the graph that leads there.
Prevents: A single conversational agent being talked into disclosing another patient's prescription details under conversational pressure: the failure where the model was told the rule, understood the rule, and set it aside because the user was persuasive and the model's job is to be helpful.
Verification is required per intent, not per interaction. A user asking about store hours or the drive-thru is never asked who they are.
Prevents: Making every user authenticate to learn whether the pharmacy is open on Sunday: a privacy control applied where there is no privacy at stake, which trains users to treat the verification prompt as an obstacle to get past rather than a boundary that means something when it appears.
If speech-to-text fails to parse a date of birth twice, the agent switches to DTMF keypad entry rather than asking the user to repeat themselves a third time.
Prevents: A user stuck in an unwinnable retry loop against a recognizer that was never going to understand them, with no exit but hanging up, and the specific unfairness of that loop closing hardest on the users whose speech the recognizer handles worst.
A second verification attempt that submits the same name and date of birth as the first is treated as a repair failure, not as a new query.
Prevents: Spending a user's remaining attempts asking a database a question it has already answered, and the particular indignity of a system that says "let's try again," receives exactly what it received before, and says "let's try again."
Verification retries are capped at three, and hitting the cap triggers an explicit, honest explanation ("For your security and privacy, I cannot access patient records without verification") before transferring to a human.
Prevents: An unverified user either being silently stonewalled with no reason given, or the system quietly relaxing the gate to end an uncomfortable interaction. The cap is a number in the spec rather than "a few attempts," so it cannot be reinterpreted during implementation.
Clinical questions (side effects, dosage, interactions, "can I take this expired pill") are classified by the router and sent straight to human escalation. The bot never attempts an answer, including a hedged one.
Prevents: A model producing a plausible-sounding but unverified clinical answer in a domain where plausible and correct are different properties, and where the user cannot tell them apart from the audio.
Prescription status is normalized into five categories before being spoken, and every category maps to an authored sentence rather than to the raw stored string.
Prevents: A user hearing an internal status code like "Refill Req Sent" and having to guess what it means for them, or hearing "On Hold" and concluding the pharmacy is refusing to release their medication.
Escalation writes an audit reason into shared state (CLINICAL_QUERY, AUTH_FAILED, USER_REQUESTED) before the transfer executes.
Prevents: A pharmacist picking up a transferred conversation with no idea why it arrived, and an operations team having to infer escalation causes from transcripts after the fact instead of counting them directly.
Content Design
The database stores prescription status as an operational string: the value a pharmacy technician needs to see on a screen while working a queue. Several of those values are unsafe to say out loud to the person whose medication they describe. "On Hold" reads to a user as deliberate withholding. "Refill Req Sent" is not a sentence. "No Refills Remaining" is accurate and tells the user nothing about what to do next.
So status is normalized twice before it reaches a user: raw database string → one of five categories → one authored sentence. The middle step is the load-bearing one. It exists so that a status value nobody anticipated maps into an existing spoken response instead of falling through to something unhandled; the fallback is a real answer rather than an error.
Scroll tablesideways for more →
| Raw database value | Normalized category | What the user hears |
|---|---|---|
| Filled · Ready · Ready for Pickup | READY | …is filled and ready for pickup at our main pharmacy counter. |
| Pending Doctor · Refill Req Sent | AWAITING_PRESCRIBER | …is awaiting prescriber authorization. We sent a request to your doctor. |
| Expired · No Refills Remaining | EXPIRED | …has expired with no remaining refills. We'll need a new prescription from your provider. |
| Prior Auth Required · Insurance Hold | INSURANCE_DELAY | …is currently delayed pending insurance prior authorization. |
| null · any unmapped value | OTHER | …is actively being processed by our pharmacy staff. |
Two of these carry a next action inside the sentence, because the status alone leaves the user stranded: EXPIRED says whose job it is to fix it, and AWAITING_PRESCRIBER says the request has already been sent so the user does not call their doctor to ask for something the pharmacy has already asked for.
Delivery is specified alongside the wording, because on a phone line they are the same decision. Lists of prescriptions carry an explicit 200ms SSML break between items, slightly rising or "continuing" intonation on all but the final item, and falling intonation on the last item, so a user can use the prosody to infer whether there is more to come or the list is complete. Dates and numeric strings are wrapped in <say-as interpret-as="digits">, so a birth year is read back as "one nine four two" rather than as "one thousand nine hundred forty-two." A user confirming their own date of birth needs to hear the digits they said.
Iteration
Wireframe → prototype: six transactions specified, one built end to end
Shipped: The Figma flow mapped six patient-specific transactions sitting behind the identity gate (status, history, billing, refill, hold, and billing change), each with its own API call and its own report-back to the user.
Broke: Building all six in the prototype would have produced six shallow branches and no evidence that the hard part worked. The hard part was never the sixth transaction. It was the gate standing in front of all of them, and the readout on the far side.
Fixed: The prototype builds one transaction (prescription status) all the way through: routing, verification, live Airtable lookup, disambiguation across multiple prescriptions, and a normalized spoken answer. The other five stay specified in the wireframe as designed-but-unbuilt, which is what they are, and the case study says so rather than implying a fuller build.
Multi-prescription readout: hardcoded slots → indexed loop
Shipped: The first working version of the readout handled a patient's medications as three named slots (med_1_name, med_2_name, med_3_name, each with its own index variable) because every patient in the synthetic database has two or three prescriptions.
Broke: Three slots covered every record that happened to exist and would have failed on the fourth. Worse, the readout logic had to be written once per slot, which is three separate places for the phrasing to drift apart; the phrasing is the entire product on a voice channel.
Fixed: Replaced with a single indexed loop over the returned array: current_index, current_medication_name, has_more_medications, medication_count. One authored sentence now serves a patient with two prescriptions or twelve. Both generations are still visible in the exported project's variable registry: the abandoned med_1/med_2/med_3 slots sit alongside the loop variables that replaced them.
Debugging the Airtable response shape
Shipped: The identity lookup was written expecting the patient record to carry the prescriptions themselves.
Broke: It does not. Airtable returns the patient record plus an array of linked prescription record IDs (the prescriptions are a second hop away), so the readout was reaching for medication names that were not in the payload it had.
Fixed: A debug node was dropped into the canvas to print the captured fields (patient ID, record ID, and the prescription ID array), so the actual response shape was visible on the flow instead of inferred from documentation. The lookup was then restructured around the two-hop reality. That node is still in the exported project, which is the honest state of a prototype: the scaffolding is there because the thing it held up is there.
Technical Collaboration
Built solo, but the specification is written as a hand-off document rather than as notes to myself: a system overview and developer prompt meant to be legible to someone who did not design it, and to be built from without a discovery call.
The parts an engineer would otherwise have to invent are the parts that got specified: every state variable's name, type, and default; the exact filterByFormula the identity lookup sends and the field IDs it sends it against; the full mapping from raw database status to spoken sentence; and the retry cap as the number 3 rather than as "a few attempts." Design intent that lives only in a designer's head gets re-decided during implementation, usually by whoever hits it first, at four o'clock on a Friday, in whatever way unblocks them.
Where the built prototype and the specification disagree, both are on this page and the disagreement is named. The spec describes a Prescription Status Agent with three disambiguation branches; the build resolves status through six inquiry-type outcomes. The wireframe scopes six patient-specific transactions; the prototype builds one. A specification quietly edited after the fact to match what got built is not a specification, it is a changelog.
That is the same posture behind the Cookbook's AGENTS.md and PORTFOLIO.md (see that case study's Technical Collaboration section): write the spec so that a reviewer (human or agentic) can hold you to it.
The two artifacts an engineer actually needs, reproduced here as they appear in the specification rather than described. The first is the shared state every agent reads and writes; the second is the request the identity gate sends.
Scroll tablesideways for more →
| Variable | Type | Default | Role |
|---|---|---|---|
| v_first_name | String | "" | Spoken or transcribed first name of the caller |
| v_last_name | String | "" | Spoken or transcribed last name of the caller |
| v_dob | String (ISO) | "" | Date of birth, normalized to YYYY-MM-DD |
| v_dob_dtmf | String | "" | Digits captured via the keypad fallback, MMDDYYYY |
| v_is_authenticated | Boolean | false | The gate. Must be true before any PHI node is reachable |
| v_auth_retries | Number | 0 | Verification failure counter; cap is 3 |
| v_airtable_record_id | String | "" | Airtable recordId, populated only on a positive match |
| v_patient_id | String | "" | Internal patient ID retrieved from the matched record |
| v_rx_raw_data | Array | [] | Raw prescription records returned from the Airtable payload |
| v_target_drug_name | String | "" | Medication named by the caller during disambiguation |
| v_rx_status_category | String | "" | Normalized status: READY · AWAITING_PRESCRIBER · EXPIRED · INSURANCE_DELAY · OTHER |
| v_intent_category | String | "" | Router output: FAQ · PATIENT_SPECIFIC · CLINICAL |
| v_transfer_reason | String | "" | Audit reason written before escalation: CLINICAL_QUERY · AUTH_FAILED · USER_REQUESTED |
GET https://api.airtable.com/v0/{baseId}/{tableId}
Authorization: Bearer {AIRTABLE_API_KEY}
Content-Type: application/json
filterByFormula = AND(
{First Name} = '{v_first_name}',
{Last Name} = '{v_last_name}',
{Date of Birth} = '{v_dob}'
)
→ 1 record set v_is_authenticated = true, store recordId + patient ID,
load prescriptions into v_rx_raw_data
→ 0 records increment v_auth_retries; at 3, set v_transfer_reason
= AUTH_FAILED and route to Human EscalationAll three verification parameters go in one query rather than in a sequence of narrowing lookups. A staged check (confirm the surname, then ask for the date of birth) leaks information: a user learns from the second prompt that the first value matched somebody. One query returns one record or zero, and zero is the only thing an unverified user is ever told.
Recommended Evaluation Protocol
Nothing on this page was measured. The prototype ran against a synthetic database on a trial account with me as the only user, which is worth exactly what that sounds like: it demonstrates the flow holds together, not that it works for anyone else. So what follows is the instrumentation plan rather than a results table. It's the part of the design I would most want to argue about before a pilot, because choosing the wrong headline metric here quietly redesigns the system.
The obvious metric for an assistant like this is containment: what fraction of interactions ended without a human. Containment is the wrong headline for this one. The design deliberately gives interactions away (every clinical question is a transfer by intent, not by failure), so optimizing containment would create direct pressure to answer the questions the entire architecture exists to refuse.
False-verification rate
Proposed: not measuredHow often the gate opens for the wrong person. The only criterion here with a target rather than a baseline, and the target is zero.
Why this one: Name plus date of birth is a weak credential. Two patients sharing both is an ordinary event at retail-pharmacy scale, not a hypothetical. Everything else on this list is a quality measure, and this one is the system's reason for existing.
Gate-failure composition
Proposed: not measuredOf users who never get verified, the split between the wrong person being correctly refused and the right person the recognizer could not hear.
Why this one: These are opposite failures with opposite fixes, and the aggregate number hides which one is happening. A rising failure rate means "the gate is working" or "the gate is broken" depending entirely on this split.
Keypad fallback rate, and who reaches it
Proposed: not measuredHow often the DTMF fallback fires, segmented across whatever correlates with recognizer performance: user age band, region, time of day.
Why this one: This is the accessibility measure. If one group of users hits the fallback far more than others, the fallback is doing its job and the primary path is failing them. A fallback that works is invisible in an aggregate success rate, which is precisely why it needs its own number.
Escalation precision, measured in both directions
Proposed: not measuredOf interactions transferred as clinical, how many the receiving pharmacist agrees were clinical. And the harder half: how many clinical questions the router classified as something else and the system answered.
Why this one: Over-escalation costs staff time and shows up immediately in a transfer log. Under-escalation is the failure the whole guardrail exists to prevent, and it appears nowhere in a transfer log. It can only be found by sampling interactions that were never transferred.
Status comprehension
Proposed: not measuredWhether a user can state what happens next after hearing a normalized status. Not whether the status was correct: whether the sentence did its job.
Why this one: The normalization matrix above is a content-design hypothesis, and it is cheaply testable: five users, one question afterward: "so what do you do now?" If EXPIRED does not produce "call my doctor," the sentence is wrong no matter how accurate it is.
Time to first useful answer, split by intent
Proposed: not measuredMeasured separately for FAQ, patient-specific, and clinical interactions. Never blended into a single average.
Why this one: A blended average improves when the FAQ path gets faster, which conceals a verification flow that has gotten slower. Verification is where users actually abandon. The three intents are different products that happen to share a phone number.
Outcome
The Voiceflow trial's credits ran out before a persistent public demo could be recorded, so this entry documents the design through its artifacts rather than a live link. The artifacts are the checkable part. What exists: a 27-node, rough prototype that routes a live interaction, verifies a user against a real API, and speaks back a normalized prescription status; a 13-variable state schema and an API contract an engineer could implement from without asking a question; and an interaction flow whose privacy guarantee is a property of its topology rather than a request made to a language model. The specification predates the build and still disagrees with it in the places named above, which is the honest state of a proof of concept rather than an oversight.
- nodes in the working Voiceflow prototype
- 27nodes in the working Voiceflow prototypeexported project file: 11 agent nodes, 13 logic blocks, 1 live API call, 1 message, 1 entry point
- state variables specified with type, default, and role
- 13state variables specified with type, default, and rolespecification §3 (the full registry, reproduced under Technical Collaboration above)
- authored for the user-facing intent model
- 22 Utterances 34 Entity valuesauthored for the user-facing intent modelexported NLU model: patient-specific intents, plus entities for medication names, dosage forms, dosage units, task types, and insurance carriers
What Transfers
Almost nothing in this design is exclusive to the pharmacy use case I have presented. Three structures do the load-bearing work, and all three exist on any regulated phone line: claims, policy service, banking, benefits, anywhere a user and a record and a licensed human meet.
A record class that has to stay behind an identity check
Prescriptions here; claim files, policy details, and account balances elsewhere. The design decision that matters is not whether to verify (everyone verifies) but where the check physically lives. An instruction inside a prompt that the model is asked to honor on every turn, or a boundary in the flow graph that has no edge across it. Those produce the same behavior on a good day and very different behavior on a bad one, and the bad day is the one the control exists for.
A question class that belongs to a licensed human
Dosage and interactions here; coverage determinations, liability, and anything that constitutes advice elsewhere. The failure mode is identical and specific in both domains: the model produces a fluent, confident, plausible answer, and fluent-and-wrong is worse than silent, because the user acts on it and the hedge does not travel with the advice.
A user who does not know which class they are in
Nobody dials a phone line having pre-classified their own question, and the user with the most urgent one is the least likely to phrase it in the system's categories. Classification is the router's job and it has to happen before the user can be harmed by getting it wrong. That's why the clinical branch is a hard interrupt reachable from any point in the interaction, rather than a topic the conversation is allowed to drift into.
And one process claim that generalizes further than the design does. The specification was written before the prototype, the prototype disagreed with it in several places, and both versions are on this page with the disagreements labeled. That is the part worth carrying into any proof of concept: the value of writing the spec first is that it can be wrong in a way you can see.