The goal of this project is to use the accelerometer data from an ESP32-S3 smart watch (T-Watch-S3) to control a synthesised web output. It lets you select 8 directions depending on the x and y values of the accelerometer, and each direction you tilt to plays a different note. When a second watch is connected it allows you to change the set of notes that will be played depending on direction.
Instructions:
- Create AWS instance and open port 8080 for all ipv4. This will allow for the watch to send data to the server.
- sudo apt-get update
- sudo apt-get install node.js npm
- npm install cors express
- Clone this repository
- If you are using one watch, flash it with 8wayMidiaccel1.4.ino and run server.js. If you use two watches flash 8wayMidiaccel1.5 and run server2.js. (Note when flashing you must enter your WIFI and AWS server public IP. If you flash 2 watches you must also change the deviceID from 1 to 2 for the second watch)
- Enter your AWS public IP + :8080 into your browser and you are ready to go.
Notes:
Low latency is important for musical expression and I am aiming for 10 to 15ms which I have been able to achieve using UDP. HTTP gave me latencies around 35ms locally and 400ms when sent to AWS. Using UDP there is no difference when sending to AWS as it maintained the low latency because there is no need for a response. Unfortunately I found that sending UDP at a rate of 10 to 15ms overwhelms the AWS server with my current configurations, but by limiting my latency to 30ms I was able to recieve data without being throttled. At 30ms there is also no fluctuations in the latency rate. Using Websockets I have been able to consistantly stay under 15ms even while using 2 watches at the same time on the same port which is very promising. For the future I could like to expand this project to use web BLE, which will eliminate the need to flash the watch every time you have a new AWS server or WIFI.
To activate web audio (no longer needed):
chrome.exe --autoplay-policy=no-user-gesture-required