Play Instruments in VR by Using Wearable Tech Gloves
Platform:
Unity and Arduino
Duration:
April - May 2023, Boston
Deliverable:
Concept Statment
Precedents Research
User Test
Data Analysis
Unity & Arduino Development
Concept
What: This device explores transitioning from requiring a physical instrument to play music to using a wearable device and virtual reality to allow people to explore music from anywhere. The device can be used to collect rhythmic data to help musicians understand their timing in relation to a metronome. It is built with good piano technique in mind.
Who: Initially, new or seasoned musicians. In the future, it could be a tool for music teachers, composers, or physical therapists.
How: Unity, Arduino, finger movements, & haptic feedback.
Why: It's expensive to buy quality instruments and difficult to carry them around. With this wearable device, people could try out instruments and track how their rhythmic preciseness improves over time.
Research Goals
Play from Anywhere
Exploring activating the surfaces around you
​
​
Interact with Virtual Reality
Exploring using wearable tech to trigger sounds in Unity for use in virtual reality
​
​
Collect Rhythmic Data
Exploring collecting and analyzing data related to a musician's ability to play to a metronome
​
​
Experience Different Environments
Exploring how different environments affect the music created
Video: Single Row, Play Music of Lalaland
Video: Two Rows, Experimental Instrument
Gathering Data, Built Metronome
Data Analysis
How close is close enough? Timestamps show if the musician is before or after the beat.
Success rate by thousandths of a second: 68%
Success rate by hundredths of a second: 95%
Success rate by a tenth of a second: 100%
Use Case: Tracking rhythmic data over time. Musicians can track how "in time" they are with a metronome and see how they improve over time.
Precedents Secondary Research
User Testing
Build Process & Iterations
Next Steps
Music Collaboration of the Future
How could this be used to facilitate music collaboration across time zones?
Decrease Latency
​
When a button is clicked a sound should be immediately heard.
Increase Durability
​
Make the glove more durable to increase longevity and to make it easier to don and doff.