Getting 150 builders from the lobby to Room 209 without a queue
0 wrong guests admitted in 28 noisy scenesA voice concierge on a lobby screen for a meetup night. She gives directions, checks guests in by QR pass or by name, and suggests people to meet. Check-in is plain Python rules; the model only talks and picks tools.
AI Tinkerers Doha meets in the Tech 2 building at Qatar Science and Technology Park. For Round 4, on 19 October 2026, the organisers approved about 150 builders for Room 209 on floor 2. Most had never been in the building, and most would arrive in the same half hour, while one or two volunteers ran the desk.
I built Ada for that half hour and gave the talk about her at the event. She's a voice concierge on a lobby screen: she gives directions, checks guests in by QR pass or by name, and names people worth meeting. The model talks and picks tools, and plain Python rules decide who gets in. In 28 scenes spoken over crowd noise, nobody was admitted who shouldn't have been.
- 28
- end-to-end scenes, crowd noise mixed into the audio
- 0
- wrong guests admitted
- 0 to 1
- cloud calls per check-in
Watching one guest's evening
A talk about a kiosk is easy to fake with slides. A slide can't show what the kiosk heard in a noisy lobby, or what it got wrong.
I opened the deck on a recording instead: one guest, start to finish, on the real Godot kiosk and server with lobby chatter in the microphone. The captions show what Whisper heard, slips included. Pick a step to jump to it.
Starting from the research
A meetup lobby has problems a language model can't see: a rush at the door, strangers who won't say their name out loud, and a room too loud to hear in. I had no time for interviews before the event, so the design started from published research and the event's own data.
I read check-in benchmarks, wayfinding studies, work on talking to strangers, and Whisper's own noise results. Each finding turned into one rule for Ada.
- Most guests arrive in the first half hour.
Check-in vendors plan for 60 to 70% of attendees arriving within 30 minutes of doors opening, and aim for under 60 seconds per person. Self-service kiosks see 15 to 30% of first-time users needing help.
So: Ada has to clear the 6:00 to 6:30 rush on her own and send only the exceptions to the desk.
Nunify, Event check-in time benchmarks (vendor data, 2026) ↗
- People get lost inside buildings too.
Wayfinding research puts the cause in three places: the building's layout, the signs, and the visitor's own picture of the space. First-time visitors have no picture yet.
So: Directions name a landmark and a count ("the lifts to the right of reception", "the third door"), and a second kiosk on floor 2 picks the visitor up.
- People hold back from talking to strangers, and misjudge how it goes.
After a first conversation, people consistently underestimate how much the other person liked them. Separately, professional networking for instrumental reasons makes many people feel uncomfortable.
So: Ada can make the first move for a guest: name someone who works on their topic and say where to find them.
- Speaking to a device in public feels exposed.
In studies of voice assistants, people were less willing to say private information out loud in public places than at home.
So: A guest never has to say their name: holding up the QR pass is the first option, and Ada never reads names or the guest list aloud.
- Speech recognition gets worse fast as a room gets louder.
Whisper's own paper shows every model it tested degrading as background noise rises, with white noise and with pub noise. People also raise their voice and change how they speak in noise (the Lombard effect).
So: Push-to-talk marks exactly when the guest is speaking, the reply is captioned on screen, and anything with a right answer (check-in) goes through rules, not recognised words.
- Doha is an English-speaking room with many first languages.
About 88% of Qatar's residents are foreign nationals. Arabic is the official language and English is the language of business.
So: Ada speaks English by default and handles accented speech and names from many languages. A name has to match one guest clearly, so a mishearing goes to the desk instead of admitting the wrong person.
Framing the problem for four people
One kiosk serves four people with different stakes. A first-time guest wants the room and no queue. A presenter wants to set up early and be found afterwards. The volunteer at the desk wants only the hard cases. The organisers want an accurate headcount and a guest list nobody ever reads out.
I turned those stakes into three questions to design for:
- How might we get a guest from the front door to Room 209 without asking a person?
- How might we check a guest in within seconds, in a loud lobby, without them saying their name out loud?
- How might we help a guest start one useful conversation without exposing anyone who didn't opt in?
Nadia Rahman
“I just want to find the room and not stand in a line.”
Goals
- Find Room 209 fast
- Check in without fuss
- Meet someone working on Arabic speech
Pains
- Unfamiliar building, signs she can't see in the crowd
- Doesn't want to say her name to a screen in a busy lobby
- Walking up to strangers
How they use Ada
- Directions
- QR check-in
- Find people (she opted in)
A presenter
“Tell me where to set up, and point people my way.”
Goals
- Get to the front table before 6:40
- Be easy to find afterwards for people who share the topic
Pains
- Stuck in the check-in rush before setup
- Answering where-is-the-room on the way in
How they use Ada
- Presenter line on check-in: "Presenters sit at the front table"
- Named in find-people for their demo's topics
Joseph Mensah
“If the kiosk handles the easy ones, I can deal with the real problems.”
Goals
- Keep the line short during the rush
- Handle only the exceptions
Pains
- Looking up names one by one
- Passes used twice
- Being the directions desk too
How they use Ada
- Ada sends waitlist, reused-pass and not-found cases to him
The organisers
“Nobody reads out our guest list. Ever.”
Goals
- Accurate headcount
- No guest data spoken or sent anywhere it shouldn't go
Pains
- Walk-ins they can't admit
- A model that might improvise a yes
How they use Ada
- Rules decide check-in
- QR decoded on the machine
- Opt-in networking only
Edge personas
- Priya NairBrings a colleague: a party of two on one pass.
plus_ones 1 - Daniel OrtizOn the waitlist. Needs a kind, clear no and where to go next.
waitlist - Sara KimCancelled her RSVP, then came anyway.
cancelled - Nadia KarimShares a first name with Nadia Rahman, and opted out of networking.
opt-out
I drew the personas from the research and the event's data, with no interviews yet, so they are proto-personas and every guest is invented. The four edge cases at the bottom shaped the check-in rules more than the main four did.
Mapping Nadia's evening
Nadia Rahman's evening · feeling from −2 (stressed) to +2 (delighted)
Check in
- Doing
- Presses C, then Q, and holds her phone up.
- Thinking
- Please don't make me spell my name.
- What Ada does
- "You're on the list, Nadia." A pass card appears with the room and gate time.
- Pain
- Glare or a tilted phone; a pass that's already been used.
- Opportunity
- Decide in rules in under a second; send real problems to the desk.
Nadia is the primary persona: a speech researcher new to Doha, with her QR pass on her phone. Her evening runs from two days before the event to leaving through the gates at 9:10, and it dips twice: arriving in a loud lobby, and walking up to strangers.
I drew the map before building anything. Each dip became a feature. The room number goes on screen because the lobby is loud, and Ada names people who opted in because Nadia won't walk up to random ones.
Storyboarding four moments, then breaking them
The storyboard has four panels: the lobby, the pass scan, the second kiosk on floor 2, and networking. Each one looked simple on paper and stopped being simple once I listed what could go wrong in it.
I gave every panel its edge cases, and each edge case became a test scene. The stills come from those test runs, so they show the lobby kiosk: the floor 2 kiosk is still only a panel in the storyboard.




Keeping admission out of the model
A language model that can take actions will sometimes take the wrong one. OWASP lists this as Excessive Agency ↗. At a door, the wrong action means admitting someone who isn't on the list, or telling a stranger who else is here.
A first name alone gets the same reply whether one guest or two share it, so the answer gives nothing away.
I split the job in two. The model understands the guest and picks a tool. Rules in server/meetup.py decide every check-in and write the sentence Ada says, so the model never sees a full name or a pass code.
What the model does
- Understands what the guest asked
- Picks one of 14 tools
- Writes the spoken reply
What plain Python rules do
- Check the event time
- Check RSVP status
- Check earlier check-ins
- Read the QR pass with OpenCV, no model call
a pass scan makes no cloud call at all; a check-in by name makes at most one
The rules can return 16 outcomes, and every one ends somewhere a guest can act on: admitted, asked for more, sent to the desk, refused, or asked to try again. Filter them by what happens next.
checked_inAdmitted"You're on the list, Nadia." Pass card, directions, what's on now.welcome_backAdmittedSeen at the other kiosk: "Hi again, Nadia."already_checked_inAdmittedWithin 2 minutes: "You're all set." Later: the original time, and a pointer to the desk.need_surnameAskFirst name only. Same reply whether one or two guests share it.need_nameAsk"What name is the RSVP under?"name_not_foundDeskNo clear match. Ada never guesses between guests or lists names.unknown_codeDeskA well-formed pass that isn't on the list.wrong_eventRefuseA pass for another night.not_a_passRefuseAny other QR code, like a flyer link.waitlistedDeskKind no, and what to do next.rsvp_cancelledDeskThe guest cancelled; the desk decides.closedRefuseAfter gates close at 10 PM.not_todayRefuseThe pass is real but the event is another day.event_cancelledRefuseThe whole night is off.no_codeRetryNothing readable: hold it closer, screen bright, press Q again.camera_unavailableRetryOffers the spoken route instead.
Names are the tricky part, because Whisper spells them however it likes. The server compares how names sound, so “Kabib Bi Han” can match “Cabibihan”, but only when one guest is the clear winner: at least 0.72 alike, and 0.15 ahead of the next closest guest. A first name alone gets the same reply whether one guest or two share it, so the answer gives nothing away.
Tracing each flow to the function
Two programs share one Mac. Godot draws the lobby and owns Ada's body. A Python server built on Pipecat listens, thinks and speaks. They talk over one local WebSocket, and only the text of each turn leaves the machine.
The stack is chosen so nothing personal leaves the building.
I named every box after the real function, so the talk could follow one question from the microphone to Ada's footsteps. Pick a flow and step through it.
"Where's the AI meetup?"
The guest just talks; Silero VAD and Smart Turn decide when they have finished. Whisper and Kokoro run on the Mac; only the text of the turn goes to the model.
This stepOn this flowNot usedCloudRulesPlain code the model can’t override
01 / 19Keys
main._unhandled_input
Space pressed: voice.set_push_to_talk_active(true), guide.set_state("listening")
The stack is chosen so nothing personal leaves the building. Whisper and Kokoro run on the Mac's GPU, OpenCV reads the pass locally, and the model sees only the words of the turn and the guide entries.
Kiosk
- Godot4.7Local
Draws the lobby, Ada and the HUD; runs the frame loop, navigation and input
A real 3D scene with a navigation mesh, so Ada can walk a guest somewhere instead of pointing at a map
- GDScriptLocal
Kiosk logic: message router, voice client, avatar, HUD
No build step; the scripts reload with the scene
- GL Compatibility rendererLocal
OpenGL 4.1 through Metal on Apple silicon
Light enough to share the laptop GPU with Whisper and Kokoro (51 fps in rehearsal)
- VRM + MToon addonsVRM 1.0Local
Lets Ada wear VRM avatars beside the Ready Player Me body
Swap the body (M key) without touching the face or lip-sync code
Transport
- WebSocketwebsockets 15.0.1Local
One socket on 127.0.0.1:8766: JSON messages plus AUD0 binary audio
Godot speaks WebSocket natively; one ordered channel for both audio and events
Voice pipeline
- Pipecat1.1.0Local
Frame pipeline: speech in, context, model, speech out, interruptions
Barge-in and turn handling come built in; each step is a small FrameProcessor
- Python3.13.5Local
The server, the rules and the tests
Pipecat, MLX and OpenCV are all Python
Speech in
- Whisper large-v3-turbo, 4-bitmlx-whisper 0.4.3, MLX 0.32.0Local
Transcribes the turn when Space is released, English, temperature 0
Runs on the Mac's GPU, so no guest audio leaves the building
Model
- gemma-4-31b-itvia OpenRouter, Cerebras firstCloud
Answers and calls tools; max 140 tokens, temperature 0.2, no reasoning
Fast first token on Cerebras; only the text of the turn and the guide entries are sent
Speech out
- Kokoro v1.0kokoro-onnx 0.6.1Local
Ada's voice, 24 kHz mono, made sentence by sentence and sent as it's ready
Small enough to run beside Whisper; sounds natural at 1.05× speed
- MMS TTSLocal
Filipino and English voices, used when MUSEUM_TTS isn't kokoro or Kokoro's files are missing; Whisper then detects the language
Kokoro speaks English only; this keeps a Filipino path and a fallback
Rules and data
- Check-in rulesLocal
Every check-in outcome, find_people, event phases
Deterministic and unit-tested; the model can't admit anyone
- CheckInStoreLocal
Who has checked in, as a JSON file with an fcntl lock and atomic writes
Safe if two kiosk processes share it; nothing to install
- Visitor guideLocal
Facts retrieved into each turn
Answers come from the guide, with the source shown on screen
- OpenCV QRCodeDetectoropencv-python-headless 5.0.0.93Local
Reads the pass off the webcam frame, trying several views of each frame
Pass check-in needs no model call, and no image leaves the Mac
This site and testing
- React + ViteReact 18.3.1, Vite 5.4.11Build tool
This presentation
One page for the story, the architecture and the live view
- Node22.22.3Build tool
Tails the trace files and streams them by Server-Sent Events; serves function source
Standard library only, no packages
- JSONL traceBuild tool
Both processes append one line per event
A file both sides can write, and any tool can read
- pytest + e2e runnerpytest 9.1.1Build tool
Unit tests, plus 28 scenes spoken through the real stack over crowd noise
Tests the whole loop as a guest would use it, not only the parts
Inside Godot, one frame runs in a fixed order, with audio on its own thread beside it. That order is why a guest can cut Ada off and the walk to the lifts keeps going.
- 01
Input
main._unhandled_inputMuseumPlayer._unhandled_inputSpace, C, Q, mouse look. Each key becomes a message or a state change.
- 02
Physics (60 Hz)
MuseumPlayer._physics_processAvatarGuide._physics_processVisitor movement and the gaze ray; Ada's navigation step, gaze turn, and AvatarFace.update for blinks, mouth and expression.
- 03
Process (every frame)
main._processMuseumVoiceClient._processLocator arrow and the trace heartbeat. The socket: poll(), _read_packets() into message_received and _receive_audio, then _capture_microphone() while Space is held.
- 04
Signals
main._on_voice_messageAvatarGuide.feed_audioMessages route to the HUD and Ada inside the same frame, through Godot signals.
- 05
Render
RenderingServerThe lobby, Ada and the HUD, through the GL compatibility renderer.
- 06
Audio thread
AudioStreamGeneratorAudioEffectCaptureAudioEffectSpectrumAnalyzerRuns beside the frame loop: plays Ada's queued speech, fills the mic capture buffer, measures the 7 speech bands for the lips.
Testing 28 scenes in a loud room
Unit tests can't tell you whether a guest gets through the door. So every scene from the storyboard was spoken through the whole stack, Godot to server and back, over synthetic lobby chatter from ten macOS voices.
A mishearing makes Ada ask again. It can't admit anyone, because admission never trusts the transcript.
When a loud room got in the way, it was the hearing: Whisper turned “AI Tinkerers” into “AIVP” in a busy lobby, a guest cut in as a check-in landed, and the loudest room drowned the question. Ada asked again each time, and none of it let in the wrong guest.
Two of the scenes are on video at the top of this section. Scene 27 is the loudest room; scene 23 is a guest cutting Ada off mid-answer.
Pick a scene to see what the guest said, what Whisper heard, what Ada said back, and how loud the room was.
28 scenes
Wayfinding 4
Pass check-in 11
Spoken check-in 5
Event info and people 5
Turn-taking in noise 3
Directions in a busy lobby
- Guest said
- “Hi! Where's the AI meetup?”
- Whisper heard
- “Hi, where's the AIVP?”
- Ada said
- “I do not have a detail for the AIVP in my guide. Please ask at reception.”
- Time to first audio
- 7.3 s
- Why it failed
- missing ['209']
Holding to six principles
A kiosk at a door makes a small call for every guest, and a guest notices each one that goes wrong: a name read out to the room, a wrong admission, a reply lost in the noise, a refusal with nowhere to go next.
I held every decision above to six principles: one for each of those four, one for knowing when a guest has finished talking, and one for keeping personal data on the Mac. Each principle has a measure the test run reports.
| Principle | What it means in the build |
|---|---|
| Rules decide, the model talks. | Check-in outcomes come from server/meetup.py. The model only chooses when to call it, and the rules wrote the line Ada says. |
| Just talk, or hold to talk. | Ada listens hands-free and works out when the guest has finished. In a room too loud for that, push-to-talk lets Space mark the turn, and taps under 0.4 s are dropped. The 28 test scenes used push-to-talk. |
| Nothing personal leaves the Mac. | Speech recognition, QR reading and the guest list stay on the machine. The model only sees an outcome, never the full name or code. |
| Opt in, or stay invisible. | Only presenters and guests who opted in can be named in networking answers. |
| Always a person to go to. | Every refusal ends with the event desk outside Room 209. |
| Say it and show it. | Every reply is captioned, and the pass card stays on screen, for loud rooms and for guests who are hard of hearing. |
| Measure | Result | Where it comes from |
|---|---|---|
| Check-in, pass shown to decision | under 1 s | pass_read metric, measured on every scan |
| Start of the turn to Ada's first word | median 5.9 s (1.4 to 16 s) | first_audio_s over the scenes where Ada spoke. It includes the guest speaking; pass check-ins, with no speech and no model, are 1.4 to 3 s |
| Scenes run in a loud room | 28 | tools/record_e2e.sh, 19 Oct 2026 |
| Wrong admissions | 0 | every admit path is a rule, tested in server/tests/test_meetup.py |
| Names or list read out | 0 | organiser and "is X here" scenes pass |
Planning the next round
The demo runs one kiosk on one Mac with invented guests. Before it meets a real door, it needs the second floor, a better pass reader, and people who aren't me.
I split what's left into two lists. The build list comes from the rehearsal logs, where OpenCV missed passes on bright phone screens and stray words from the room started replies. The research list checks my proto-personas against real guests, starting with a count at the desk from 6:00 to 6:30 at the next event.
Build next
- The floor 2 kiosk and the lift ride: designed in the storyboard, not built
- A stronger QR decoder, tested with real AI Tinkerers passes on bright phone screens
- Voice barge-in tuned against a real room's echo
- Drop stray words like "uh" or "okay" before they start a reply
Research next
- Watch the desk at the next event from 6:00 to 6:30: count arrivals, time each check-in, note every question that isn't check-in.
- Five short interviews after the demos: what was hard about getting here and meeting people?
- Run the kiosk beside the desk for a week, opt-in, and compare queue length with and without it.
- Test with Arabic-accented and South Asian-accented English speakers, and with a guest who is hard of hearing.
Notes
- Meridian Tech Park, its rooms and every guest in the demo are fictional.
- Conversation state lives in memory, so a restart forgets the chat but not the check-ins.
- The talk site ↗ has the 16 slides this post is built from, and replays one recorded evening with no server.
Sources
- LLM06: Excessive Agency ↗, OWASP Top 10 for LLM Applications, 2025.
- Pipecat ↗, the open-source voice pipeline the server is built on.
- Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision ↗ (Whisper), 2022.
- The research findings above link their own sources.