Mon Oct 30 2023
TheChallenge
The challenge assigned by the course Advance Programming of Interactive System was to use cutting-edge technologies to create innovative user interactions within an application in a 4 week timeframe.
We needed to create something unexplored and innovative.
TheSolution
We created a gesture-controlled music production platform that revolutionizes how people interact with music creation tools.
Web App
Chromium browsers with webcam and microphone support for universal accessibility
Smartwatch App
Heartbeat communication and tactile feedback for enhanced user experience
Gesture Recognition
MediaPipe integration for accurate hand gesture interpretation in real-time
Voice Commands
Touch-free music control with voice recognition for hands-busy scenarios
Key Innovations
Tune Crafter brings together multiple interaction modalities to create an intuitive music creation experience that's accessible to everyone, regardless of musical background.
💡 Accessibility First
Designed for musicians and non-musicians alike with an intuitive interface
âš¡ Real-time Processing
Ultra-low latency between gesture detection and sound output
MyRole
As the Lead UI Designer & Developer, I spearheaded the design and implementation process.
UX/UI Leadership
Led sketching sessions, user flows, and wireframes for multiple interaction scenarios
Technical Architecture
Benchmarked and selected tech stack optimized for complex gesture tracking
Feature Development
Implemented gesture recognition system and state machine for multi-modal interactions
DesignProcess
User Research & Ideation
Initial user interviews identified key pain points in music creation. We conducted 3 rapid ideation sessions, producing 20+ concept sketches to address these needs.
Interface Exploration
Tested two interface approaches with users: a familiar DJ console layout and an innovative gesture-first design. The hybrid approach yielded 85% higher user engagement.
Iterative Prototyping
Created 3 low-fidelity prototypes with increasing complexity. Each iteration was user-tested, resulting in a 40% improvement in gesture recognition accuracy.
Iterative design process with rapid prototyping cycles
GestureMapping
We designed an intuitive gesture system that transforms natural hand movements into musical controls.
UX Design Approach
- •Used cognitive mapping principles to match gestures to expected sound outputs
- •Conducted usability testing with 12 users to refine gesture recognition thresholds
- •Implemented visual feedback cues to confirm gesture recognition
SeeitinAction
Results&Impact
Growth
60 new monthly users through word-of-mouth from the app's "wow effect"
Tech Implementation
Integrated MediaPipe hand tracking and web-smartwatch communication via sockets
Visibility
Live demos at Université Paris-Saclay and featured interview on Good Soup Media
WanttotryTuneCrafter?
Experience the future of music production with Tune Crafter. Use your hands to create beats and control music flow in an immersive way.
More Projects
Full UX Design Process - Digital Nomads App
Mobile app that connects like-minded professionals and helps them find ideal workspaces.
From Idea to MVP - Rosterbook
LinkedIn for Italian minor basketball teams and players.
From Idea to MVP - Weko.ai
AI-powered productivity platform that reduces digital overwhelm through mindful distraction blocking and personalized focus environments.