qSound

qSound is an interactive Python project that translates real-time audio input into visual quantum state representations. Using PyAudio for audio capture, Qiskit for quantum simulations, and OpenGL for rendering, this project provides a unique visualization of sound properties such as amplitude, frequency, and phase as they influence simulated qubit interactions.

Table of Contents

• Overview
• Features
• Prerequisites
• Project Structure
• Running Instructions
• 🏆 Awards

---

Overview

qSound captures real-time audio input and decomposes it into essential components like frequency, amplitude, and phase. These components are then mapped to quantum states, providing a visualization of quantum behaviors influenced by sound. It’s ideal for those interested in the intersection of sound and quantum mechanics, and could also be to some extent an educational tool, though primarily serves as a fun, extremely overcomplicated, audiovisual experience.

Features

• Real-Time Audio Processing: Captures live audio input with customizable device selection.
• Quantum Simulation and Visualization: Converts sound parameters into quantum states and displays them in 3D.
• Particle-Based Visualization: Creates a particle system responsive to the audio’s intensity.
• Neural Network Intensity Determines a time period of a sound wave's intensity to determine fun visual flavors
• OpenGL Rendering: Provides real-time rendering for quantum transformations and particle effects.

Prerequisites

• Python 3.8 or above
• PyAudio
• Qiskit
• Pytorch
• SKlearn
• Librosa
• OpenGL
• Virtual Audio Device (e.g., BlackHole for macOS)

Project Structure

• audio_processing_pyaudio.py: Handles audio input using PyAudio, extracting amplitude, frequency, phase, and other features.
• quantum_process.py: Defines and applies quantum transformations based on sound attributes. Creates quantum registers and applies state changes to qubits based on sound parameters.
• finalscript.py: Initializes and manages threads for audio capture, quantum processing, and OpenGL rendering. Coordinates the components in a main application loop.
• files_for_nn: Trains a neural network on a custom dataset developed by our team. The network employs supervised learning to analyze the input and provides intensity predictions for the given song at 15 ms intervals.

The audio_processing_pyaudio.py script is responsible for capturing live audio input and analyzing various sound features. It uses PyAudio for real-time audio streaming. Key steps in audio processing include:

1. Amplitude Calculation:

   • Measures the signal strength by computing the root mean square (RMS) of the audio sample.
   • This value is used to control the strength of quantum transformations in the simulation.

2. Frequency Detection:

   • Utilizes Fast Fourier Transform (FFT) to determine the dominant frequency within the audio sample.
   • The detected frequency is mapped to qubit oscillation rates, influencing their behavior in the visualization.

3. Phase Extraction:

   • Determines the phase angle of the dominant frequency component.
   • This phase information is applied as a rotation parameter in the quantum simulation, affecting qubit orientation.

Other parameters such as the BPM of a sound wave are approximated to further scale the rate at which we rotate qubits in the bloch sphere.

The quantum_process.py` module leverages Qiskit to create a dynamic quantum visualization that responds to audio input:

1. Quantum State Mapping:

   • Sound features are mapped to quantum transformations as follows:
     • Amplitude -> Controls the rotation angle of qubits, dictating the strength of transformations.
     • Frequency -> Determines oscillation rates, influencing the frequency of qubit rotations.
     • Phase -> Sets phase shifts, directly affecting qubit orientation in space.

2. Quantum Circuit Construction:

   • Register Initialization: Six groups of quantum registers are initialized in different basis states, including:
     • |+⟩ and |−⟩ states: Created using Hadamard and Z gates.
     • |0⟩ and |1⟩ states: Standard basis states.
     • Y-basis states |i+⟩ and |i−⟩: Generated by combining S gates and Hadamard transformations.
   • Sound-Based Transformations: Each group of qubits is transformed based on its mapped sound parameters, creating diverse behaviors in the visualization.
   • Entanglement: Qubits in different groups are entangled, allowing complex interactions and collective responses to sound input.

3. Visualization:

   • The transformed quantum states are rendered as a wave in an OpenGL environment.
   • Particles represent intensity changes, creating an immersive visual experience based on real-time audio features.

This script is responsible for training a neural network to predict audio intensity based on extracted audio features. It leverages machine learning techniques to create a robust model for real-time analysis. Key functionalities include: 1.Data Preprocessing: The script reads audio feature datasets from a CSV file, where each row corresponds to the features of a song segment and the target intensity value. The dataset is split into training and testing sets using an 80-20 split for model evaluation. Features are standardized using StandardScaler to ensure consistent scaling, improving model performance and convergence. 2.Neural Network Design: A fully connected feedforward neural network is implemented using PyTorch. The network consists of: -An input layer matching the number of audio features. -A single hidden layer with customizable dimensions, activated using ReLU for non-linearity. -An output layer that predicts the intensity of the audio segment. 3.Training: The model is trained using supervised learning and optimized with the Adam optimizer to minimize mean squared error (MSE) loss. Training runs for a specified number of epochs, with batch processing facilitated by PyTorch's DataLoader. At the end of training, the script provides detailed loss metrics for each epoch, ensuring transparency and performance tracking. 4.Model Saving and Reusability: The trained model, along with the feature scaler, can be saved for future use. This allows for quick deployment without retraining. 5.Prediction: A separate function, test_model, enables predictions on new audio data. The input features are scaled using the saved scaler, and the model predicts the intensity value for the new data segment.

Running Instructions

To execute qSound, follow these steps:

1. Install Dependencies:
   Ensure all the dependencies listed in the prerequisites section are installed. Use Python 3 for compatibility and package management.

2. Run the Main Script:
   The primary execution point for the application is final_script.py. Execute this script to launch the program.

3. Pre-Trained Models:
   There is no need to retrain the quantum bits or the neural network, as their states have been pre-saved. The system will utilize these pre-trained models for immediate functionality.

Enjoy exploring the quantum-enhanced audio visualization with qSound!

🏆 Awards

✨ This project was honored with the NordVPN Award as one of the Best Projects at HackUMass 2024, competing among over 500 talented participants! 🎉

We are proud of this achievement and grateful for the recognition! 🚀