Designing for a network that drops
The sync layer assumes connectivity is the exception, not the rule. Quests resolve locally and reconcile later rather than blocking the wearer on a round trip that may never return.
3
live signals fused (GPS, weather, time)
0
keyboard inputs required
36h
build window
A context-aware AR platform for MentraOS smart glasses that generates personalized quests from where you are and what the world is doing.
Why it mattered
Smart glasses are the first platform in years where the interface constraints are genuinely hostile — no keyboard, unreliable network, tiny compute budget. Designing for that changes how you think about every API call.
The problem
MentraOS runs on hardware with strict compute and connectivity limits. Building anything real-time there means solving data sync and degraded-network behavior before you write a single feature.
What I built
- Context-aware quest generation combining Claude with GPS position and live weather APIs.
- Fully voice-controlled interface using Vosk and Whisper so the platform never assumes a keyboard.
- A data sync layer engineered specifically for low-connectivity operation.
- Google Gemini integration for dynamic quest management as conditions change.
Engineering notes
Voice as the only input
With no keyboard, intent recognition is not a convenience feature — it is the entire input surface. Vosk handles the fast local path, Whisper the accurate one.
Curious about any of this?
I am glad to go deeper on the architecture, the tradeoffs, or the parts that did not work the first time. That conversation is usually more useful than the README.