Matterly
Role
Timeline
Team
Skills
Matterly reveals the material story behind the everyday objects you use, from extraction and manufacturing, through use, to reuse, recycling, landfill, or compost. It is primarily an educational tool for deeply understanding sustainability in daily life, with a softer lifestyle layer for logging items and tracking small improvements.
Final Design
Takeaways
AI-assisted scripting became a fast way to learn a new platform while still designing intentionally.
Building for Spectacles meant learning a new device, a new OS, and a new interaction model under hackathon time pressure. Vibe coding helped me move faster through unfamiliar APIs and accelerate the creation of custom UI scripts throughout the experience. Instead of using AI to replace design decisions, I used it as a way to prototype interaction ideas, debug platform-specific code, and keep momentum while learning how the Spectacles ecosystem worked.
A polished hackathon project still has to align with what the platform is looking for.
It is really not easy to win a hackathon. We put a lot of care into the research, experience design, and polish of Matterly, but I suspect we did not place because our concept may not have been the type of educational experience the upcoming Spectacles platform needed most. Even so, the project taught me a lot about designing for spatial learning, building on emerging hardware, and balancing ambition with platform fit.
Hackathon Prompt
Can Spatial AI help people learn from the physical world around them?
For this hackathon, we were asked to create an educational experience for Spectacles that used AI, spatial context, and extended reality to transform real-world environments into learning environments. The prompt encouraged teams to think beyond traditional screens and imagine how AI could amplify human learning in places like classrooms, museums, homes, and everyday spaces.
"Spectacles" evolving to "Specs"
How do we make an educational experience specific enough to feel useful, but still relatable to the judges?
During ideation, we kept balancing between two extremes. A concept that was too niche could become a one-time demo, while something too broad could feel like a generic AI assistant in AR. Matterly landed in the middle by focusing on sustainability, which is both relatable but unexplored.
Process
Researching
We began by researching previous hackathon projects and looking for ideas that had not already been done. A lot of spatial learning concepts naturally move toward museums, language learning, object labeling, or guided tutorials, so we wanted to find a direction that felt educational but still distinct. We eventually decided to explore sustainability through a scientific, research-backed lens. The challenge was making that information feel understandable rather than overwhelming. Instead of presenting sustainability as a dashboard of abstract numbers, Matterly breaks down objects into material stories. Users can scan something in front of them, understand what it is made of, and learn how its lifecycle affects carbon, water, energy, recyclability, and waste.
Storyboarding
Once we established sustainability and material literacy as the direction, we moved into storyboarding. Using the Snap OS Figma kit, we mapped out how the experience could unfold on Spectacles, from scanning an object to selecting it, expanding its lifecycle, asking follow-up questions, and comparing objects side by side. Storyboarding helped us think through the experience before building. Since Spectacles interactions are still emerging, we could not rely on familiar mobile or headset patterns. We had to ask where information should appear, how much should be visible at once, and how the UI could feel spatial without overwhelming the user’s view.
Pseudocode and Interaction Planning
Before implementing the UI, we mapped out new interaction patterns that were not already covered by the Spectacles Interaction Kit.
We wanted Matterly to feel native to Spectacles, but the default interaction patterns did not cover everything we imagined. During pseudocode planning, I explored UI paradigms such as dynamic wrist menus, menus that track to a grabbed object, proximity anchors, persistent HUD elements, and wavy point clouds that show which parts of the environment or object have been scanned. These early interaction sketches helped us define what needed to be custom-built. The goal was not just to add visual polish, but to create spatial feedback that made the system easier to understand. When an object is scanned, the user should be able to see what Matterly has recognized, what is still being processed, and where the next interaction should happen.
Custom UI Scripting
When the Interaction Kit felt limiting, I treated custom scripting as part of the design process. Nearly a quarter of the hackathon was spent learning the platform and building custom UI scripts. I worked on scripts that made it easier to change UI asset colors, animate elements, and create interface behaviors that felt more responsive to the user’s actions. This was where vibe coding became especially useful. Since I was working with a new platform and unfamiliar constraints, AI-assisted scripting helped me move from design intent to working prototype more quickly. I could describe the behavior I wanted, test generated code, and then refine the implementation based on what actually worked on device.
Some Custom UI Scripts
Other Custom UI
Current Design
Spatial AI Anchor Experience
Grab Anchor Experience
Thank you!
Thank you to my teammates for building Matterly with me and pushing the idea forward throughout the hackathon. Thank you to all the judges who tried our app and gave us thoughtful feedback. And thank you to the Reality Hack team, the AWE team, and Spectacles team for hosting us and creating the space to experiment with what learning on emerging hardware could become.