diff --git a/.vscode/settings.json b/.vscode/settings.json new file mode 100644 index 0000000..7b016a8 --- /dev/null +++ b/.vscode/settings.json @@ -0,0 +1,3 @@ +{ + "java.compile.nullAnalysis.mode": "automatic" +} \ No newline at end of file diff --git a/MODEL_SETUP.md b/MODEL_SETUP.md new file mode 100644 index 0000000..1868cf0 --- /dev/null +++ b/MODEL_SETUP.md @@ -0,0 +1,232 @@ +# STT Model Setup Guide + +## Overview + +This app uses RunAnywhere SDK with Sherpa-ONNX Whisper for speech-to-text transcription. + +## Automatic Model Download + +**The app will automatically download the STT model on first use!** + +- Model: `sherpa-onnx-whisper-base.en` +- Size: ~150MB +- Download happens automatically when you first start a live session +- Model is cached locally after download + +## What Happens on First Launch + +1. App initializes RunAnywhere SDK +2. Registers default models (including STT) +3. When you start a live session: + - Checks if STT model is downloaded + - If not: Downloads it automatically with progress + - Loads model into memory + - Starts transcription + +## Manual Model Management + +You can also pre-download models using the ModelService: + +```typescript +import { useModelService } from './services/ModelService'; + +const { downloadAndLoadSTT, isSTTLoaded } = useModelService(); + +// Download and load STT model +await downloadAndLoadSTT(); +``` + +## Model Information + +### Sherpa-ONNX Whisper Base + +- **Model ID**: `sherpa-onnx-whisper-base.en` +- **Size**: ~150MB +- **Format**: ONNX (tar.gz archive) +- **Language**: English only +- **Speed**: Very fast +- **Accuracy**: Good for most use cases +- **Source**: https://github.com/RunanywhereAI/sherpa-onnx + +## Troubleshooting + +### Error: "Failed to load STT model" + +**Possible causes:** + +1. No internet connection (model needs to download) +2. Insufficient storage space +3. Download was interrupted + +**Solutions:** + +1. Check your internet connection +2. Ensure you have at least 200MB free space +3. Restart the app to retry download +4. Enable Debug Mode to test without the model (see below) + +### Error: "Model download failed" + +**Solution:** + +1. Check internet connection +2. Try again later (GitHub releases might be temporarily down) +3. Use Debug Mode to test the app without audio + +### Download taking too long? + +The model is ~150MB. Download time depends on your connection: + +- WiFi: 30 seconds - 2 minutes +- 4G: 1-3 minutes +- 3G: 3-5 minutes + +You'll see download progress in the logs: + +``` +[SpeechService] đŸ“Ĩ Download progress: 10% +[SpeechService] đŸ“Ĩ Download progress: 20% +... +[SpeechService] ✅ Model downloaded +``` + +## Debug Mode (No Model Required) + +For testing without a model or internet: + +1. Go to **Settings** +2. Scroll to "**Debug & Testing**" +3. Toggle "**Debug Mode**" ON +4. Start a new session + +Debug mode injects hardcoded test transcripts every 7 seconds to test the pattern detection pipeline without needing real audio or the STT model. + +## Architecture + +### Model Loading Flow + +``` +App Launch + ↓ +App.tsx: Initialize SDK & Register backends + ↓ +App.tsx: Call registerDefaultModels() + ├─ Registers LLM model + ├─ Registers STT model (sherpa-onnx-whisper-base.en) + └─ Registers TTS model + ↓ +App.tsx: Call loadSTTModel() + ├─ Checks if model downloaded + ├─ Downloads if needed (with progress) + └─ Loads model using RunAnywhere.loadSTTModel() + ↓ +User starts live session + ↓ +LiveSessionScreen checks model status + ├─ If loaded: Start immediately + ├─ If loading: Wait for it + └─ If failed: Offer debug mode + ↓ +Audio captured + ↓ +RunAnywhere.transcribe(audioPath) + ↓ +Transcript returned +``` + +### Key Files + +- `src/services/SpeechService.ts` - STT model loading & transcription +- `src/services/ModelService.tsx` - Model registry & download management +- `src/App.tsx` - App initialization & model preloading +- `src/screens/LiveSessionScreen.tsx` - Model status checks & error handling + +## Logs to Check + +**Successful model load:** + +``` +[App] 🚀 Starting initialization... +[App] 🔧 Initializing RunAnywhere SDK... +[App] ✅ RunAnywhere SDK initialized +[App] đŸ“Ļ Registering backends... +[App] ✅ Backends registered +[App] 🤖 Registering default models... +[App] ✅ Default models registered +[App] 🎤 Loading STT model... +[SpeechService] đŸŽ¯ loadSTTModel() called +[SpeechService] 🤖 Starting STT model load... +[SpeechService] 🔍 Checking if model is downloaded... +[SpeechService] ✅ Model already downloaded +[SpeechService] 🔄 Loading STT model from: /data/.../sherpa-onnx-whisper-base.en +[SpeechService] ✅ STT model loaded successfully in XXXms +[App] ✅ STT model ready +``` + +**First-time download:** + +``` +[SpeechService] đŸ“Ĩ Model not downloaded, downloading... +[SpeechService] đŸ“Ĩ Download progress: 10% +[SpeechService] đŸ“Ĩ Download progress: 20% +... +[SpeechService] đŸ“Ĩ Download progress: 100% +[SpeechService] ✅ Model downloaded +[SpeechService] 🔄 Loading STT model from: /data/.../sherpa-onnx-whisper-tiny.en +[SpeechService] ✅ STT model loaded successfully +``` + +## Viewing Logs + +### Android + +```bash +adb logcat | grep -E "\[App\]|\[SpeechService\]" +``` + +### React Native Debugger + +Open the app and check the console for emoji-tagged logs. + +## Privacy & Offline Usage + +- ✅ Model downloads once and is cached locally +- ✅ All transcription happens on-device +- ✅ No external API calls during transcription +- ✅ After first download, works in **airplane mode** +- ✅ No audio data ever leaves your device + +## Alternative Models + +You can modify `ModelService.tsx` to use different models: + +```typescript +// In registerDefaultModels() +await ONNX.addModel({ + id: 'sherpa-onnx-whisper-base.en', // Larger, more accurate + name: 'Sherpa Whisper Base (ONNX)', + url: 'https://github.com/RunanywhereAI/sherpa-onnx/releases/download/...', + modality: ModelCategory.SpeechRecognition, + artifactType: ModelArtifactType.TarGzArchive, + memoryRequirement: 150_000_000, +}); +``` + +Then update `SpeechService.ts`: + +```typescript +const STT_MODEL_ID = 'sherpa-onnx-whisper-base.en'; +``` + +## Support + +If issues persist: + +1. Enable Debug Mode to verify the rest of the app works +2. Check RunAnywhere SDK docs: https://docs.runanywhere.ai +3. Verify @runanywhere packages in package.json: + - @runanywhere/core: ^0.18.1 + - @runanywhere/llamacpp: ^0.18.1 + - @runanywhere/onnx: ^0.18.1 +4. Check logs for specific error messages +5. Try uninstalling and reinstalling the app diff --git a/android/app/src/main/assets/fonts/Lora-Regular.ttf b/android/app/src/main/assets/fonts/Lora-Regular.ttf new file mode 100644 index 0000000..ee1914c Binary files /dev/null and b/android/app/src/main/assets/fonts/Lora-Regular.ttf differ diff --git a/android/app/src/main/java/ai/runanywhere/starter/NativeAudioModule.kt b/android/app/src/main/java/ai/runanywhere/starter/NativeAudioModule.kt index 4802a71..276f0d8 100644 --- a/android/app/src/main/java/ai/runanywhere/starter/NativeAudioModule.kt +++ b/android/app/src/main/java/ai/runanywhere/starter/NativeAudioModule.kt @@ -7,6 +7,7 @@ import android.media.AudioRecord import android.media.AudioTrack import android.media.MediaRecorder import android.util.Base64 +import android.util.Log import androidx.core.app.ActivityCompat import com.facebook.react.bridge.* import java.io.ByteArrayOutputStream @@ -26,11 +27,13 @@ class NativeAudioModule(reactContext: ReactApplicationContext) : ReactContextBas private var recordingThread: Thread? = null private var recordedData: ByteArrayOutputStream? = null private var recordingFilePath: String? = null + private var snapshotCounter = 0 private var audioTrack: AudioTrack? = null private var isPlaying = false companion object { + const val TAG = "NativeAudioModule" const val SAMPLE_RATE = 16000 const val CHANNEL_CONFIG = AudioFormat.CHANNEL_IN_MONO const val AUDIO_FORMAT = AudioFormat.ENCODING_PCM_16BIT @@ -70,7 +73,14 @@ class NativeAudioModule(reactContext: ReactApplicationContext) : ReactContextBas } recordedData = ByteArrayOutputStream() + snapshotCounter = 0 isRecording = true + + // Create the recording WAV file path upfront + val tempFile = File(reactApplicationContext.cacheDir, "recording_live.wav") + recordingFilePath = tempFile.absolutePath + Log.d(TAG, "Recording will be saved to: $recordingFilePath") + audioRecord?.startRecording() // Start recording thread @@ -88,7 +98,9 @@ class NativeAudioModule(reactContext: ReactApplicationContext) : ReactContextBas val result = Arguments.createMap().apply { putString("status", "recording") + putString("path", recordingFilePath) } + Log.d(TAG, "Recording started, path: $recordingFilePath") promise.resolve(result) } catch (e: Exception) { @@ -96,6 +108,121 @@ class NativeAudioModule(reactContext: ReactApplicationContext) : ReactContextBas } } + /** + * Create a snapshot WAV file from the current recorded audio data. + * This allows mid-recording transcription without stopping the recording. + * Each call writes the accumulated audio so far to a new temp file. + */ + @ReactMethod + fun getRecordingSnapshot(promise: Promise) { + if (!isRecording || recordedData == null) { + promise.reject("NOT_RECORDING", "No recording in progress") + return + } + + try { + val pcmData = synchronized(recordedData!!) { + recordedData?.toByteArray() ?: ByteArray(0) + } + + if (pcmData.isEmpty()) { + Log.d(TAG, "Snapshot: No audio data yet") + val result = Arguments.createMap().apply { + putString("path", "") + putInt("fileSize", 0) + } + promise.resolve(result) + return + } + + // Create WAV from accumulated PCM data + val wavData = createWavFromPcm(pcmData, SAMPLE_RATE, 1, 16) + + // Write to a snapshot file (use counter to avoid file locking issues) + snapshotCounter++ + val snapshotFile = File(reactApplicationContext.cacheDir, "recording_snapshot_${snapshotCounter % 2}.wav") + FileOutputStream(snapshotFile).use { it.write(wavData) } + + Log.d(TAG, "Snapshot created: ${snapshotFile.absolutePath}, size: ${wavData.size} bytes, PCM: ${pcmData.size} bytes") + + val result = Arguments.createMap().apply { + putString("path", snapshotFile.absolutePath) + putInt("fileSize", wavData.size) + putDouble("durationMs", (pcmData.size.toDouble() / (SAMPLE_RATE * 2)) * 1000) + } + promise.resolve(result) + + } catch (e: Exception) { + Log.e(TAG, "Failed to create snapshot: ${e.message}", e) + promise.reject("SNAPSHOT_ERROR", "Failed to create recording snapshot: ${e.message}", e) + } + } + + /** + * Create a snapshot WAV file containing ONLY the last N milliseconds of audio. + * This enables fast, chunk-based transcription instead of processing the + * endlessly growing full audio buffer. + */ + @ReactMethod + fun getRecentAudioSnapshot(durationMs: Double, promise: Promise) { + if (!isRecording || recordedData == null) { + promise.reject("NOT_RECORDING", "No recording in progress") + return + } + + try { + val pcmData = synchronized(recordedData!!) { + recordedData?.toByteArray() ?: ByteArray(0) + } + + if (pcmData.isEmpty()) { + Log.d(TAG, "Recent Snapshot: No audio data yet") + val result = Arguments.createMap().apply { + putString("path", "") + putInt("fileSize", 0) + } + promise.resolve(result) + return + } + + // Calculate bytes needed for the requested duration. + // 1 sec = SAMPLE_RATE samples = SAMPLE_RATE * 2 bytes (16-bit Mono) + val bytesRequired = ((durationMs / 1000.0) * SAMPLE_RATE * 2).toInt() + + // Ensure we use an even number of bytes to avoid splitting a 16-bit sample + val safeBytesRequired = bytesRequired - (bytesRequired % 2) + + // Extract only the recent chunk from the end of the PCM data + val extractSize = minOf(safeBytesRequired, pcmData.size) + val startIndex = pcmData.size - extractSize + + // Create a new ByteArray containing only the recent data + val recentPcmData = pcmData.copyOfRange(startIndex, pcmData.size) + + Log.d(TAG, "Recent Snapshot: Extracting last ${durationMs}ms -> ${extractSize} bytes out of ${pcmData.size} total") + + // Create WAV from the recent PCM data only + val wavData = createWavFromPcm(recentPcmData, SAMPLE_RATE, 1, 16) + + // Write to a snapshot file + snapshotCounter++ + // Use _recent prefix to keep separate from full snapshots + val snapshotFile = File(reactApplicationContext.cacheDir, "recording_snapshot_recent_${snapshotCounter % 2}.wav") + FileOutputStream(snapshotFile).use { it.write(wavData) } + + val result = Arguments.createMap().apply { + putString("path", snapshotFile.absolutePath) + putInt("fileSize", wavData.size) + putDouble("durationMs", (recentPcmData.size.toDouble() / (SAMPLE_RATE * 2)) * 1000) + } + promise.resolve(result) + + } catch (e: Exception) { + Log.e(TAG, "Failed to create recent snapshot: ${e.message}", e) + promise.reject("SNAPSHOT_ERROR", "Failed to create recent recording snapshot: ${e.message}", e) + } + } + @ReactMethod fun stopRecording(promise: Promise) { if (!isRecording) { @@ -120,11 +247,16 @@ class NativeAudioModule(reactContext: ReactApplicationContext) : ReactContextBas val wavData = createWavFromPcm(pcmData, SAMPLE_RATE, 1, 16) val base64Audio = Base64.encodeToString(wavData, Base64.NO_WRAP) - // Save to temp file + // Save to final file val tempFile = File(reactApplicationContext.cacheDir, "recording_${System.currentTimeMillis()}.wav") FileOutputStream(tempFile).use { it.write(wavData) } recordingFilePath = tempFile.absolutePath + // Clean up snapshot files + cleanupSnapshots() + + Log.d(TAG, "Recording stopped, saved to: $recordingFilePath, size: ${wavData.size} bytes") + val result = Arguments.createMap().apply { putString("status", "stopped") putString("path", recordingFilePath) @@ -150,6 +282,9 @@ class NativeAudioModule(reactContext: ReactApplicationContext) : ReactContextBas recordedData = null recordingFilePath = null + // Clean up snapshot files + cleanupSnapshots() + promise.resolve(true) } catch (e: Exception) { // Ignore errors during cancel @@ -157,10 +292,30 @@ class NativeAudioModule(reactContext: ReactApplicationContext) : ReactContextBas } } + private fun cleanupSnapshots() { + try { + for (i in 0..1) { + val snapshotFile = File(reactApplicationContext.cacheDir, "recording_snapshot_$i.wav") + if (snapshotFile.exists()) { + snapshotFile.delete() + } + } + val liveFile = File(reactApplicationContext.cacheDir, "recording_live.wav") + if (liveFile.exists()) { + liveFile.delete() + } + } catch (e: Exception) { + Log.w(TAG, "Failed to cleanup snapshots: ${e.message}") + } + } + @ReactMethod fun getAudioLevel(promise: Promise) { if (!isRecording || audioRecord == null) { - promise.resolve(0.0) + val result = Arguments.createMap().apply { + putDouble("level", 0.0) + } + promise.resolve(result) return } @@ -181,9 +336,15 @@ class NativeAudioModule(reactContext: ReactApplicationContext) : ReactContextBas } kotlin.math.sqrt(sum / (lastChunk.size / 2)) / 32768.0 } - promise.resolve(level) + val result = Arguments.createMap().apply { + putDouble("level", level) + } + promise.resolve(result) } catch (e: Exception) { - promise.resolve(0.0) + val result = Arguments.createMap().apply { + putDouble("level", 0.0) + } + promise.resolve(result) } } diff --git a/android/app/src/main/res/drawable/ic_launcher_foreground.xml b/android/app/src/main/res/drawable/ic_launcher_foreground.xml deleted file mode 100644 index 5358249..0000000 --- a/android/app/src/main/res/drawable/ic_launcher_foreground.xml +++ /dev/null @@ -1,13 +0,0 @@ - - - - - diff --git a/android/app/src/main/res/mipmap-anydpi-v26/ic_launcher.xml b/android/app/src/main/res/mipmap-anydpi-v26/ic_launcher.xml index 5ed0a2d..0c0a45c 100644 --- a/android/app/src/main/res/mipmap-anydpi-v26/ic_launcher.xml +++ b/android/app/src/main/res/mipmap-anydpi-v26/ic_launcher.xml @@ -1,5 +1,5 @@ - - + + diff --git a/android/app/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml b/android/app/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml index 5ed0a2d..46a0ddb 100644 --- a/android/app/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml +++ b/android/app/src/main/res/mipmap-anydpi-v26/ic_launcher_round.xml @@ -1,5 +1,5 @@ - - + + diff --git a/android/app/src/main/res/mipmap-hdpi/ic_launcher.png b/android/app/src/main/res/mipmap-hdpi/ic_launcher.png new file mode 100644 index 0000000..b6bf71d Binary files /dev/null and b/android/app/src/main/res/mipmap-hdpi/ic_launcher.png differ diff --git a/android/app/src/main/res/mipmap-hdpi/ic_launcher.xml b/android/app/src/main/res/mipmap-hdpi/ic_launcher.xml deleted file mode 100644 index c77a778..0000000 --- a/android/app/src/main/res/mipmap-hdpi/ic_launcher.xml +++ /dev/null @@ -1,5 +0,0 @@ - - - - - diff --git a/android/app/src/main/res/mipmap-hdpi/ic_launcher_round.png b/android/app/src/main/res/mipmap-hdpi/ic_launcher_round.png new file mode 100644 index 0000000..b6bf71d Binary files /dev/null and b/android/app/src/main/res/mipmap-hdpi/ic_launcher_round.png differ diff --git a/android/app/src/main/res/mipmap-hdpi/ic_launcher_round.xml b/android/app/src/main/res/mipmap-hdpi/ic_launcher_round.xml deleted file mode 100644 index c77a778..0000000 --- a/android/app/src/main/res/mipmap-hdpi/ic_launcher_round.xml +++ /dev/null @@ -1,5 +0,0 @@ - - - - - diff --git a/android/app/src/main/res/mipmap-mdpi/ic_launcher.png b/android/app/src/main/res/mipmap-mdpi/ic_launcher.png new file mode 100644 index 0000000..5306e83 Binary files /dev/null and b/android/app/src/main/res/mipmap-mdpi/ic_launcher.png differ diff --git a/android/app/src/main/res/mipmap-mdpi/ic_launcher.xml b/android/app/src/main/res/mipmap-mdpi/ic_launcher.xml deleted file mode 100644 index c77a778..0000000 --- a/android/app/src/main/res/mipmap-mdpi/ic_launcher.xml +++ /dev/null @@ -1,5 +0,0 @@ - - - - - diff --git a/android/app/src/main/res/mipmap-mdpi/ic_launcher_round.png b/android/app/src/main/res/mipmap-mdpi/ic_launcher_round.png new file mode 100644 index 0000000..5306e83 Binary files /dev/null and b/android/app/src/main/res/mipmap-mdpi/ic_launcher_round.png differ diff --git a/android/app/src/main/res/mipmap-mdpi/ic_launcher_round.xml b/android/app/src/main/res/mipmap-mdpi/ic_launcher_round.xml deleted file mode 100644 index c77a778..0000000 --- a/android/app/src/main/res/mipmap-mdpi/ic_launcher_round.xml +++ /dev/null @@ -1,5 +0,0 @@ - - - - - diff --git a/android/app/src/main/res/mipmap-xhdpi/ic_launcher.png b/android/app/src/main/res/mipmap-xhdpi/ic_launcher.png new file mode 100644 index 0000000..705e97b Binary files /dev/null and b/android/app/src/main/res/mipmap-xhdpi/ic_launcher.png differ diff --git a/android/app/src/main/res/mipmap-xhdpi/ic_launcher.xml b/android/app/src/main/res/mipmap-xhdpi/ic_launcher.xml deleted file mode 100644 index c77a778..0000000 --- a/android/app/src/main/res/mipmap-xhdpi/ic_launcher.xml +++ /dev/null @@ -1,5 +0,0 @@ - - - - - diff --git a/android/app/src/main/res/mipmap-xhdpi/ic_launcher_round.png b/android/app/src/main/res/mipmap-xhdpi/ic_launcher_round.png new file mode 100644 index 0000000..705e97b Binary files /dev/null and b/android/app/src/main/res/mipmap-xhdpi/ic_launcher_round.png differ diff --git a/android/app/src/main/res/mipmap-xhdpi/ic_launcher_round.xml b/android/app/src/main/res/mipmap-xhdpi/ic_launcher_round.xml deleted file mode 100644 index c77a778..0000000 --- a/android/app/src/main/res/mipmap-xhdpi/ic_launcher_round.xml +++ /dev/null @@ -1,5 +0,0 @@ - - - - - diff --git a/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.png b/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.png new file mode 100644 index 0000000..52772c2 Binary files /dev/null and b/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.png differ diff --git a/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.xml b/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.xml deleted file mode 100644 index c77a778..0000000 --- a/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.xml +++ /dev/null @@ -1,5 +0,0 @@ - - - - - diff --git a/android/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.png b/android/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.png new file mode 100644 index 0000000..52772c2 Binary files /dev/null and b/android/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.png differ diff --git a/android/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.xml b/android/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.xml deleted file mode 100644 index c77a778..0000000 --- a/android/app/src/main/res/mipmap-xxhdpi/ic_launcher_round.xml +++ /dev/null @@ -1,5 +0,0 @@ - - - - - diff --git a/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png new file mode 100644 index 0000000..8886f4c Binary files /dev/null and b/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png differ diff --git a/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.xml b/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.xml deleted file mode 100644 index c77a778..0000000 --- a/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.xml +++ /dev/null @@ -1,5 +0,0 @@ - - - - - diff --git a/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.png b/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.png new file mode 100644 index 0000000..8886f4c Binary files /dev/null and b/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.png differ diff --git a/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.xml b/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.xml deleted file mode 100644 index c77a778..0000000 --- a/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher_round.xml +++ /dev/null @@ -1,5 +0,0 @@ - - - - - diff --git a/android/app/src/main/res/values/colors.xml b/android/app/src/main/res/values/colors.xml index a5a754f..280bdbc 100644 --- a/android/app/src/main/res/values/colors.xml +++ b/android/app/src/main/res/values/colors.xml @@ -1,5 +1,5 @@ - #1A1F2C + #FFFFFF #FFFFFF diff --git a/android/app/src/main/res/values/strings.xml b/android/app/src/main/res/values/strings.xml index e4b971f..11abb4c 100644 --- a/android/app/src/main/res/values/strings.xml +++ b/android/app/src/main/res/values/strings.xml @@ -1,3 +1,3 @@ - RunAnywhere Starter + Latent diff --git a/android/link-assets-manifest.json b/android/link-assets-manifest.json new file mode 100644 index 0000000..117befc --- /dev/null +++ b/android/link-assets-manifest.json @@ -0,0 +1,9 @@ +{ + "migIndex": 1, + "data": [ + { + "path": "assets/fonts/Lora-Regular.ttf", + "sha1": "2b770fae41c7e1ee781ed458bd3df94e3dc0fce3" + } + ] +} \ No newline at end of file diff --git a/assets/app.icns b/assets/app.icns new file mode 100644 index 0000000..fa94e8f Binary files /dev/null and b/assets/app.icns differ diff --git a/assets/app.ico b/assets/app.ico new file mode 100644 index 0000000..fa1d1e9 Binary files /dev/null and b/assets/app.ico differ diff --git a/assets/favicon-120.png b/assets/favicon-120.png new file mode 100644 index 0000000..e569de5 Binary files /dev/null and b/assets/favicon-120.png differ diff --git a/assets/favicon-128.png b/assets/favicon-128.png new file mode 100644 index 0000000..5d9a163 Binary files /dev/null and b/assets/favicon-128.png differ diff --git a/assets/favicon-144.png b/assets/favicon-144.png new file mode 100644 index 0000000..dbe3bf0 Binary files /dev/null and b/assets/favicon-144.png differ diff --git a/assets/favicon-152.png b/assets/favicon-152.png new file mode 100644 index 0000000..18226e1 Binary files /dev/null and b/assets/favicon-152.png differ diff --git a/assets/favicon-195.png b/assets/favicon-195.png new file mode 100644 index 0000000..1b4003a Binary files /dev/null and b/assets/favicon-195.png differ diff --git a/assets/favicon-228.png b/assets/favicon-228.png new file mode 100644 index 0000000..a10a976 Binary files /dev/null and b/assets/favicon-228.png differ diff --git a/assets/favicon-32.png b/assets/favicon-32.png new file mode 100644 index 0000000..892944e Binary files /dev/null and b/assets/favicon-32.png differ diff --git a/assets/favicon-48.png b/assets/favicon-48.png new file mode 100644 index 0000000..3a81a9e Binary files /dev/null and b/assets/favicon-48.png differ diff --git a/assets/favicon-57.png b/assets/favicon-57.png new file mode 100644 index 0000000..f2a89da Binary files /dev/null and b/assets/favicon-57.png differ diff --git a/assets/favicon-72.png b/assets/favicon-72.png new file mode 100644 index 0000000..692fccb Binary files /dev/null and b/assets/favicon-72.png differ diff --git a/assets/favicon-96.png b/assets/favicon-96.png new file mode 100644 index 0000000..61a62f7 Binary files /dev/null and b/assets/favicon-96.png differ diff --git a/assets/favicon.ico b/assets/favicon.ico new file mode 100644 index 0000000..7b5a5b6 Binary files /dev/null and b/assets/favicon.ico differ diff --git a/assets/fonts/Lora-Regular.ttf b/assets/fonts/Lora-Regular.ttf new file mode 100644 index 0000000..ee1914c Binary files /dev/null and b/assets/fonts/Lora-Regular.ttf differ diff --git a/assets/icon.png b/assets/icon.png new file mode 100644 index 0000000..1676225 Binary files /dev/null and b/assets/icon.png differ diff --git a/docs/MULTI_LANGUAGE_FEATURE.md b/docs/MULTI_LANGUAGE_FEATURE.md new file mode 100644 index 0000000..8a39e99 --- /dev/null +++ b/docs/MULTI_LANGUAGE_FEATURE.md @@ -0,0 +1,354 @@ +# Multi-Language STT/TTS Feature + +This document describes the multi-language Speech-to-Text (STT) and Text-to-Speech (TTS) feature added to the RunAnywhere React Native Starter App. + +## Overview + +The app now supports multiple languages for both speech recognition and voice synthesis, allowing users to: + +- Select their preferred language for speech-to-text transcription +- Choose from available TTS voices in different languages +- Switch languages dynamically without reloading the app + +## Features Implemented + +### 1. Language Configuration System (`src/config/languages.ts`) + +A centralized configuration file that defines: + +- **Supported languages**: English, Spanish, French, German, Chinese, Japanese +- **Language metadata**: Name, native name, flag emoji, language code +- **Model mappings**: STT and TTS model IDs for each language +- **Helper functions**: `getLanguageByCode()`, `hasSTTSupport()`, `hasTTSSupport()` + +```typescript +export interface LanguageConfig { + code: string; // ISO language code (e.g., 'en', 'es') + name: string; // English name + nativeName: string; // Native language name + flag: string; // Flag emoji + sttModelId?: string; // STT model identifier + ttsModelId?: string; // TTS model identifier + sttModelUrl?: string; // Model download URL + ttsModelUrl?: string; // Model download URL +} +``` + +### 2. LanguageSelector Component (`src/components/LanguageSelector.tsx`) + +A beautiful modal component for language selection featuring: + +- **Filterable list**: Can filter by STT/TTS support +- **Visual indicators**: Shows which languages support STT and TTS +- **Clean UI**: Modal with gradient header and smooth animations +- **Accessibility**: Touch-friendly, large tap targets + +**Usage:** + +```tsx + setShowLanguageSelector(false)} + filterSTT={true} // Only show languages with STT support +/> +``` + +### 3. VoiceSelector Component (`src/components/VoiceSelector.tsx`) + +A modal for selecting TTS voices with: + +- **Voice metadata display**: Name, language, quality, gender +- **Type indicators**: Neural vs System voices +- **Download status**: Shows if voice requires download +- **Language filtering**: Can filter voices by language code +- **Fallback handling**: Shows mock voices if SDK API unavailable + +**Usage:** + +```tsx + setShowVoiceSelector(false)} + languageFilter={selectedLanguage.code} +/> +``` + +### 4. Enhanced SpeechToTextScreen + +**New features:** + +- Language selection button at the top +- Displays selected language with flag emoji +- Passes language code to `RunAnywhere.transcribe()` API +- Filters language selector to only show languages with STT support + +**Code changes:** + +```typescript +// Language selector UI + setShowLanguageSelector(true)} +> + {selectedLanguage.flag} + + Language + {selectedLanguage.name} + + + +// Transcribe with selected language +const result = await RunAnywhere.transcribe(audioBase64, { + sampleRate: 16000, + language: selectedLanguage.code, +}); +``` + +### 5. Enhanced TextToSpeechScreen + +**New features:** + +- Dual selection buttons: Language and Voice +- Language selection changes available voices +- Voice selection filtered by selected language +- Selected voice ID passed to synthesis API + +**Code changes:** + +```typescript +// Language & Voice selectors + + setShowLanguageSelector(true)}> + {/* Language button */} + + setShowVoiceSelector(true)}> + {/* Voice button */} + + + +// Synthesize with selected voice +const result = await RunAnywhere.synthesize(text, { + voice: selectedVoice?.id || 'default', + rate: speechRate, + pitch: 1.0, + volume: 1.0, +}); +``` + +## Supported Languages + +| Language | Code | STT Support | TTS Support | +| ------------ | ---- | --------------- | --------------- | +| English (US) | `en` | ✅ Yes | ✅ Yes | +| Spanish | `es` | ⚠ī¸ Model needed | ⚠ī¸ Model needed | +| French | `fr` | ⚠ī¸ Model needed | ⚠ī¸ Model needed | +| German | `de` | ⚠ī¸ Model needed | ⚠ī¸ Model needed | +| Chinese | `zh` | ⚠ī¸ Model needed | ❌ No | +| Japanese | `ja` | ⚠ī¸ Model needed | ❌ No | + +**Note:** Currently only English models are downloaded by default. To add support for other languages, you need to: + +1. Download the appropriate STT/TTS models for that language +2. Register them in `ModelService.tsx` +3. Update the model URLs in `src/config/languages.ts` + +## How It Works + +### Language Selection Flow + +1. User taps language button +2. `LanguageSelector` modal opens +3. User selects a language +4. Selected language is saved to component state +5. Language code is passed to STT/TTS APIs +6. For TTS: Available voices are filtered by language + +### Model Management + +The app uses RunAnywhere SDK's model management system: + +```typescript +// Per RunAnywhere docs: https://docs.runanywhere.ai/react-native/stt/options + +// STT with language option +await RunAnywhere.transcribe(audioData, { + language: 'es', // Spanish + sampleRate: 16000, +}); + +// TTS with voice option +await RunAnywhere.synthesize(text, { + voice: 'en_US-lessac-medium', // English voice + rate: 1.0, +}); +``` + +## Adding New Languages + +To add support for a new language: + +### 1. Add language to configuration + +Edit `src/config/languages.ts`: + +```typescript +{ + code: 'it', + name: 'Italian', + nativeName: 'Italiano', + flag: '🇮🇹', + sttModelId: 'sherpa-onnx-whisper-tiny-it', + ttsModelId: 'vits-piper-it_IT-medium', + sttModelUrl: 'https://github.com/.../whisper-tiny-it.tar.gz', + ttsModelUrl: 'https://github.com/.../piper-it-IT.tar.gz', +} +``` + +### 2. Register models in ModelService + +Edit `src/services/ModelService.tsx`: + +```typescript +// Add model registration +await ONNX.addModel({ + id: 'sherpa-onnx-whisper-tiny-it', + name: 'Sherpa Whisper Tiny (Italian)', + url: 'https://github.com/.../whisper-tiny-it.tar.gz', + modality: ModelCategory.SpeechRecognition, + artifactType: ModelArtifactType.TarGzArchive, + memoryRequirement: 75_000_000, +}); +``` + +### 3. Download and load the models + +Models can be downloaded: + +- **Automatically**: When user first accesses STT/TTS screens +- **Manually**: Via model management in your app settings +- **Programmatically**: Using `RunAnywhere.downloadModel()` + +## Architecture + +``` +src/ +├── config/ +│ ├── languages.ts # Language definitions & helpers +│ └── index.ts # Config exports +├── components/ +│ ├── LanguageSelector.tsx # Language picker modal +│ ├── VoiceSelector.tsx # Voice picker modal +│ └── index.ts # Component exports +└── screens/ + ├── SpeechToTextScreen.tsx # STT with language selection + └── TextToSpeechScreen.tsx # TTS with voice selection +``` + +## Performance Considerations + +- **Model size**: Each language model is 50-100MB +- **Memory usage**: Only one model loaded at a time +- **Switching languages**: Requires model unload/reload +- **Download time**: 30-60 seconds per model on average connection + +## Future Enhancements + +Potential improvements for the multi-language feature: + +1. **Model pre-loading**: Download popular language models during onboarding +2. **Language detection**: Auto-detect spoken language in STT +3. **Voice preview**: Let users hear voice samples before selection +4. **Model caching**: Keep recently used models in memory +5. **Compression**: Use smaller quantized models for faster downloads +6. **Language packs**: Bundle STT+TTS models together +7. **Offline indicator**: Show which languages work offline +8. **Translation**: Add real-time translation between languages + +## API Reference + +### LanguageConfig + +```typescript +interface LanguageConfig { + code: string; + name: string; + nativeName: string; + flag: string; + sttModelId?: string; + ttsModelId?: string; + sttModelUrl?: string; + ttsModelUrl?: string; +} +``` + +### TTSVoiceInfo + +```typescript +interface TTSVoiceInfo { + id: string; + name: string; + language: string; + type: 'neural' | 'system'; + quality: 'low' | 'medium' | 'high'; + gender?: 'male' | 'female' | 'neutral'; + requiresDownload: boolean; + isAvailable: boolean; +} +``` + +## Testing + +To test the multi-language feature: + +1. **Language Selection**: + - Open STT or TTS screen + - Tap language selector button + - Verify languages show with correct flags and names + - Select a language and verify it's displayed + +2. **Voice Selection** (TTS screen): + - Select a language first + - Tap voice selector button + - Verify only voices for selected language are shown + - Select a voice and verify synthesis uses it + +3. **Speech Recognition**: + - Select a non-English language + - Record audio in that language + - Verify transcription accuracy + +4. **Voice Synthesis**: + - Select a non-English voice + - Enter text in that language + - Verify audio is generated in correct language/voice + +## Troubleshooting + +### "No voices available for language" + +- The selected language may not have TTS models downloaded +- Check model availability in `src/config/languages.ts` +- Download required TTS model for that language + +### STT not recognizing speech + +- Ensure correct language is selected +- Verify STT model for that language is downloaded and loaded +- Check microphone permissions + +### Wrong voice used for synthesis + +- Verify voice is selected before synthesis +- Check that selected voice matches language +- Fallback to 'default' voice if none selected + +## Resources + +- [RunAnywhere STT Options Docs](https://docs.runanywhere.ai/react-native/stt/options) +- [RunAnywhere TTS Voices Docs](https://docs.runanywhere.ai/react-native/tts/voices) +- [Whisper Model Languages](https://github.com/openai/whisper#available-models-and-languages) +- [Piper TTS Voices](https://github.com/rhasspy/piper/blob/master/VOICES.md) diff --git a/ios/RunAnywhereStarter.xcodeproj/project.pbxproj b/ios/RunAnywhereStarter.xcodeproj/project.pbxproj index c6529e2..d7f7d5e 100644 --- a/ios/RunAnywhereStarter.xcodeproj/project.pbxproj +++ b/ios/RunAnywhereStarter.xcodeproj/project.pbxproj @@ -17,6 +17,7 @@ 95FAA6247F992A3A1B818127 /* PrivacyInfo.xcprivacy in Resources */ = {isa = PBXBuildFile; fileRef = 13B07FB81A68108700A75B9A /* PrivacyInfo.xcprivacy */; }; NATIVE_AUDIO_OBJC /* NativeAudioModule.m in Sources */ = {isa = PBXBuildFile; fileRef = NATIVE_AUDIO_OBJC_REF /* NativeAudioModule.m */; }; NATIVE_AUDIO_SWIFT /* NativeAudioModule.swift in Sources */ = {isa = PBXBuildFile; fileRef = NATIVE_AUDIO_SWIFT_REF /* NativeAudioModule.swift */; }; + D67CA02AB17A45E4B7A343BC /* Lora-Regular.ttf in Resources */ = {isa = PBXBuildFile; fileRef = E8D10F655D094F99B944453C /* Lora-Regular.ttf */; }; /* End PBXBuildFile section */ /* Begin PBXContainerItemProxy section */ @@ -51,6 +52,7 @@ ED297162215061F000B7C4FE /* JavaScriptCore.framework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = JavaScriptCore.framework; path = System/Library/Frameworks/JavaScriptCore.framework; sourceTree = SDKROOT; }; NATIVE_AUDIO_OBJC_REF /* NativeAudioModule.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; name = NativeAudioModule.m; path = RunAnywhereStarter/NativeAudioModule.m; sourceTree = ""; }; NATIVE_AUDIO_SWIFT_REF /* NativeAudioModule.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; name = NativeAudioModule.swift; path = RunAnywhereStarter/NativeAudioModule.swift; sourceTree = ""; }; + E8D10F655D094F99B944453C /* Lora-Regular.ttf */ = {isa = PBXFileReference; name = "Lora-Regular.ttf"; path = "../assets/fonts/Lora-Regular.ttf"; sourceTree = ""; fileEncoding = undefined; lastKnownFileType = unknown; explicitFileType = undefined; includeInIndex = 0; }; /* End PBXFileReference section */ /* Begin PBXFrameworksBuildPhase section */ @@ -133,6 +135,7 @@ 83CBBA001A601CBA00E9B192 /* Products */, 2D16E6871FA4F8E400B85C8A /* Frameworks */, BBD78D7AC51CEA395F1C20DB /* Pods */, + 612D5D6699364B9B9514468C /* Resources */, ); indentWidth = 2; sourceTree = ""; @@ -159,6 +162,15 @@ path = Pods; sourceTree = ""; }; + 612D5D6699364B9B9514468C /* Resources */ = { + isa = "PBXGroup"; + children = ( + E8D10F655D094F99B944453C /* Lora-Regular.ttf */, + ); + name = Resources; + sourceTree = ""; + path = ""; + }; /* End PBXGroup section */ /* Begin PBXNativeTarget section */ @@ -255,6 +267,7 @@ 81AB9BB82411601600AC10FF /* LaunchScreen.storyboard in Resources */, 13B07FBF1A68108700A75B9A /* Images.xcassets in Resources */, 95FAA6247F992A3A1B818127 /* PrivacyInfo.xcprivacy in Resources */, + D67CA02AB17A45E4B7A343BC /* Lora-Regular.ttf in Resources */, ); runOnlyForDeploymentPostprocessing = 0; }; diff --git a/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/1024x1024.png b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/1024x1024.png new file mode 100644 index 0000000..75ecb18 Binary files /dev/null and b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/1024x1024.png differ diff --git a/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/20x20@2x.png b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/20x20@2x.png new file mode 100644 index 0000000..366f2ee Binary files /dev/null and b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/20x20@2x.png differ diff --git a/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/20x20@3x.png b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/20x20@3x.png new file mode 100644 index 0000000..13057b0 Binary files /dev/null and b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/20x20@3x.png differ diff --git a/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/29x29@2x.png b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/29x29@2x.png new file mode 100644 index 0000000..2c74156 Binary files /dev/null and b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/29x29@2x.png differ diff --git a/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/29x29@3x.png b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/29x29@3x.png new file mode 100644 index 0000000..6c87eb4 Binary files /dev/null and b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/29x29@3x.png differ diff --git a/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/40x40@2x.png b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/40x40@2x.png new file mode 100644 index 0000000..91c9821 Binary files /dev/null and b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/40x40@2x.png differ diff --git a/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/40x40@3x.png b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/40x40@3x.png new file mode 100644 index 0000000..54b7e35 Binary files /dev/null and b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/40x40@3x.png differ diff --git a/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/60x60@2x.png b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/60x60@2x.png new file mode 100644 index 0000000..54b7e35 Binary files /dev/null and b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/60x60@2x.png differ diff --git a/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/60x60@3x.png b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/60x60@3x.png new file mode 100644 index 0000000..1482112 Binary files /dev/null and b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/60x60@3x.png differ diff --git a/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/Contents.json b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/Contents.json index 8121323..9cd008a 100644 --- a/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/Contents.json +++ b/ios/RunAnywhereStarter/Images.xcassets/AppIcon.appiconset/Contents.json @@ -3,47 +3,56 @@ { "idiom" : "iphone", "scale" : "2x", - "size" : "20x20" + "size" : "20x20", + "filename": "20x20@2x.png" }, { "idiom" : "iphone", "scale" : "3x", - "size" : "20x20" + "size" : "20x20", + "filename": "20x20@3x.png" }, { "idiom" : "iphone", "scale" : "2x", - "size" : "29x29" + "size" : "29x29", + "filename": "29x29@2x.png" }, { "idiom" : "iphone", "scale" : "3x", - "size" : "29x29" + "size" : "29x29", + "filename": "29x29@3x.png" }, { "idiom" : "iphone", "scale" : "2x", - "size" : "40x40" + "size" : "40x40", + "filename": "40x40@2x.png" }, { "idiom" : "iphone", "scale" : "3x", - "size" : "40x40" + "size" : "40x40", + "filename": "40x40@3x.png" }, { "idiom" : "iphone", "scale" : "2x", - "size" : "60x60" + "size" : "60x60", + "filename": "60x60@2x.png" }, { "idiom" : "iphone", "scale" : "3x", - "size" : "60x60" + "size" : "60x60", + "filename": "60x60@3x.png" }, { "idiom" : "ios-marketing", "scale" : "1x", - "size" : "1024x1024" + "size" : "1024x1024", + "filename": "1024x1024.png" } ], "info" : { diff --git a/ios/RunAnywhereStarter/Info.plist b/ios/RunAnywhereStarter/Info.plist index 89457f4..ceecc26 100644 --- a/ios/RunAnywhereStarter/Info.plist +++ b/ios/RunAnywhereStarter/Info.plist @@ -46,7 +46,7 @@ NSLocalNetworkUsageDescription This app uses the local network to connect to Metro bundler during development NSLocationWhenInUseUsageDescription - + NSMicrophoneUsageDescription This app needs microphone access for speech recognition and voice agent features NSSpeechRecognitionUsageDescription @@ -67,5 +67,9 @@ UIViewControllerBasedStatusBarAppearance + UIAppFonts + + Lora-Regular.ttf + diff --git a/ios/link-assets-manifest.json b/ios/link-assets-manifest.json new file mode 100644 index 0000000..117befc --- /dev/null +++ b/ios/link-assets-manifest.json @@ -0,0 +1,9 @@ +{ + "migIndex": 1, + "data": [ + { + "path": "assets/fonts/Lora-Regular.ttf", + "sha1": "2b770fae41c7e1ee781ed458bd3df94e3dc0fce3" + } + ] +} \ No newline at end of file diff --git a/package-lock.json b/package-lock.json index 2125746..c461acb 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,13 +1,14 @@ { - "name": "runanywhere-starter-app", + "name": "latent", "version": "1.0.0", "lockfileVersion": 3, "requires": true, "packages": { "": { - "name": "runanywhere-starter-app", + "name": "latent", "version": "1.0.0", "dependencies": { + "@react-native-async-storage/async-storage": "^2.2.0", "@react-navigation/native": "^7.1.24", "@react-navigation/stack": "^7.6.16", "@runanywhere/core": "^0.18.1", @@ -21,7 +22,8 @@ "react-native-live-audio-stream": "^1.1.1", "react-native-nitro-modules": "^0.31.10", "react-native-safe-area-context": "~5.6.2", - "react-native-sound": "^0.13.0" + "react-native-sound": "^0.13.0", + "react-native-svg": "^15.15.3" }, "devDependencies": { "@babel/core": "^7.25.2", @@ -72,6 +74,7 @@ "resolved": "https://registry.npmjs.org/@babel/core/-/core-7.29.0.tgz", "integrity": "sha512-CGOfOJqWjg2qW/Mb6zNsDm+u5vFQ8DxXfbM09z69p5Z6+mE1ikP2jUXw+j42Pf1XTYED2Rni5f95npYeuwMDQA==", "license": "MIT", + "peer": true, "dependencies": { "@babel/code-frame": "^7.29.0", "@babel/generator": "^7.29.0", @@ -103,6 +106,7 @@ "integrity": "sha512-QGmsKi2PBO/MHSQk+AAgA9R6OHQr+VqnniFE0eMWZcVcfBZoA2dKn2hUsl3Csg/Plt9opRUWdY7//VXsrIlEiA==", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@nicolo-ribaudo/eslint-scope-5-internals": "5.1.1-v1", "eslint-visitor-keys": "^2.1.0", @@ -2526,12 +2530,25 @@ "node": ">= 8" } }, + "node_modules/@react-native-async-storage/async-storage": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/@react-native-async-storage/async-storage/-/async-storage-2.2.0.tgz", + "integrity": "sha512-gvRvjR5JAaUZF8tv2Kcq/Gbt3JHwbKFYfmb445rhOj6NUMx3qPLixmDx5pZAyb9at1bYvJ4/eTUipU5aki45xw==", + "license": "MIT", + "dependencies": { + "merge-options": "^3.0.4" + }, + "peerDependencies": { + "react-native": "^0.0.0-0 || >=0.65 <1.0" + } + }, "node_modules/@react-native-community/cli": { "version": "20.1.1", "resolved": "https://registry.npmjs.org/@react-native-community/cli/-/cli-20.1.1.tgz", "integrity": "sha512-aLPUx43+WSeTOaUepR2FBD5a1V0OAZ1QB2DOlRlW4fOEjtBXgv40eM/ho8g3WCvAOKfPvTvx4fZdcuovTyV81Q==", "devOptional": true, "license": "MIT", + "peer": true, "dependencies": { "@react-native-community/cli-clean": "20.1.1", "@react-native-community/cli-config": "20.1.1", @@ -3081,6 +3098,7 @@ "integrity": "sha512-1rjYZf62fCm6QAinHmRAKnJxIypX0VF/zBPd0qWvWABMZugrS0eACuIbk9Wk0StBod4yL8KnwEJyg77ak8xYzQ==", "devOptional": true, "license": "MIT", + "peer": true, "dependencies": { "@react-native/js-polyfills": "0.83.1", "@react-native/metro-babel-transformer": "0.83.1", @@ -3151,6 +3169,7 @@ "resolved": "https://registry.npmjs.org/@react-navigation/native/-/native-7.1.28.tgz", "integrity": "sha512-d1QDn+KNHfHGt3UIwOZvupvdsDdiHYZBEj7+wL2yDVo3tMezamYy60H9s3EnNVE1Ae1ty0trc7F2OKqo/RmsdQ==", "license": "MIT", + "peer": true, "dependencies": { "@react-navigation/core": "^7.14.0", "escape-string-regexp": "^4.0.0", @@ -3196,6 +3215,7 @@ "resolved": "https://registry.npmjs.org/@runanywhere/core/-/core-0.18.1.tgz", "integrity": "sha512-M4epS8GdesQ0g4CUCvqnVA/YXVOBNwOsQUBh7mbEfl3oFrjzJdLzR1UP5X9an8SDqXMy87+Uap3o0dEKmMWwEA==", "license": "MIT", + "peer": true, "peerDependencies": { "react": ">=18.0.0", "react-native": ">=0.74.0", @@ -3394,6 +3414,7 @@ "integrity": "sha512-Qec1E3mhALmaspIrhWt9jkQMNdw6bReVu64mjvhbhq2NFPftLPVr+l1SZgmw/66WwBNpDh7ao5AT6gF5v41PFA==", "devOptional": true, "license": "MIT", + "peer": true, "dependencies": { "csstype": "^3.0.2" } @@ -3446,6 +3467,7 @@ "integrity": "sha512-1y/MVSz0NglV1ijHC8OT49mPJ4qhPYjiK08YUQVbIOyu+5k862LKUHFkpKHWu//zmr7hDR2rhwUm6gnCGNmGBQ==", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@eslint-community/regexpp": "^4.12.2", "@typescript-eslint/scope-manager": "8.55.0", @@ -3475,6 +3497,7 @@ "integrity": "sha512-4z2nCSBfVIMnbuu8uinj+f0o4qOeggYJLbjpPHka3KH1om7e+H9yLKTYgksTaHcGco+NClhhY2vyO3HsMH1RGw==", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@typescript-eslint/scope-manager": "8.55.0", "@typescript-eslint/types": "8.55.0", @@ -3739,6 +3762,7 @@ "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.15.0.tgz", "integrity": "sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==", "license": "MIT", + "peer": true, "bin": { "acorn": "bin/acorn" }, @@ -4310,6 +4334,12 @@ "devOptional": true, "license": "MIT" }, + "node_modules/boolbase": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/boolbase/-/boolbase-1.0.0.tgz", + "integrity": "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww==", + "license": "ISC" + }, "node_modules/brace-expansion": { "version": "2.0.2", "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.2.tgz", @@ -4351,6 +4381,7 @@ } ], "license": "MIT", + "peer": true, "dependencies": { "baseline-browser-mapping": "^2.9.0", "caniuse-lite": "^1.0.30001759", @@ -4856,6 +4887,56 @@ "node": ">= 8" } }, + "node_modules/css-select": { + "version": "5.2.2", + "resolved": "https://registry.npmjs.org/css-select/-/css-select-5.2.2.tgz", + "integrity": "sha512-TizTzUddG/xYLA3NXodFM0fSbNizXjOKhqiQQwvhlspadZokn1KDy0NZFS0wuEubIYAV5/c1/lAr0TaaFXEXzw==", + "license": "BSD-2-Clause", + "dependencies": { + "boolbase": "^1.0.0", + "css-what": "^6.1.0", + "domhandler": "^5.0.2", + "domutils": "^3.0.1", + "nth-check": "^2.0.1" + }, + "funding": { + "url": "https://github.com/sponsors/fb55" + } + }, + "node_modules/css-tree": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/css-tree/-/css-tree-1.1.3.tgz", + "integrity": "sha512-tRpdppF7TRazZrjJ6v3stzv93qxRcSsFmW6cX0Zm2NVKpxE1WV1HblnghVv9TreireHkqI/VDEsfolRF1p6y7Q==", + "license": "MIT", + "dependencies": { + "mdn-data": "2.0.14", + "source-map": "^0.6.1" + }, + "engines": { + "node": ">=8.0.0" + } + }, + "node_modules/css-tree/node_modules/source-map": { + "version": "0.6.1", + "resolved": "https://registry.npmjs.org/source-map/-/source-map-0.6.1.tgz", + "integrity": "sha512-UjgapumWlbMhkBgzT7Ykc5YXUT46F0iKu8SGXq0bcwP5dz/h0Plj6enJqjz1Zbq2l5WaqYnrVbwWOWMyF3F47g==", + "license": "BSD-3-Clause", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/css-what": { + "version": "6.2.2", + "resolved": "https://registry.npmjs.org/css-what/-/css-what-6.2.2.tgz", + "integrity": "sha512-u/O3vwbptzhMs3L1fQE82ZSLHQQfto5gyZzwteVIEyeaY5Fc7R4dapF/BvRoSYFeqfBk4m0V1Vafq5Pjv25wvA==", + "license": "BSD-2-Clause", + "engines": { + "node": ">= 6" + }, + "funding": { + "url": "https://github.com/sponsors/fb55" + } + }, "node_modules/csstype": { "version": "3.2.3", "resolved": "https://registry.npmjs.org/csstype/-/csstype-3.2.3.tgz", @@ -5058,6 +5139,61 @@ "node": ">=6.0.0" } }, + "node_modules/dom-serializer": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-2.0.0.tgz", + "integrity": "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg==", + "license": "MIT", + "dependencies": { + "domelementtype": "^2.3.0", + "domhandler": "^5.0.2", + "entities": "^4.2.0" + }, + "funding": { + "url": "https://github.com/cheeriojs/dom-serializer?sponsor=1" + } + }, + "node_modules/domelementtype": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.3.0.tgz", + "integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/fb55" + } + ], + "license": "BSD-2-Clause" + }, + "node_modules/domhandler": { + "version": "5.0.3", + "resolved": "https://registry.npmjs.org/domhandler/-/domhandler-5.0.3.tgz", + "integrity": "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w==", + "license": "BSD-2-Clause", + "dependencies": { + "domelementtype": "^2.3.0" + }, + "engines": { + "node": ">= 4" + }, + "funding": { + "url": "https://github.com/fb55/domhandler?sponsor=1" + } + }, + "node_modules/domutils": { + "version": "3.2.2", + "resolved": "https://registry.npmjs.org/domutils/-/domutils-3.2.2.tgz", + "integrity": "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw==", + "license": "BSD-2-Clause", + "dependencies": { + "dom-serializer": "^2.0.0", + "domelementtype": "^2.3.0", + "domhandler": "^5.0.3" + }, + "funding": { + "url": "https://github.com/fb55/domutils?sponsor=1" + } + }, "node_modules/dunder-proto": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz", @@ -5100,6 +5236,18 @@ "node": ">= 0.8" } }, + "node_modules/entities": { + "version": "4.5.0", + "resolved": "https://registry.npmjs.org/entities/-/entities-4.5.0.tgz", + "integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==", + "license": "BSD-2-Clause", + "engines": { + "node": ">=0.12" + }, + "funding": { + "url": "https://github.com/fb55/entities?sponsor=1" + } + }, "node_modules/env-paths": { "version": "2.2.1", "resolved": "https://registry.npmjs.org/env-paths/-/env-paths-2.2.1.tgz", @@ -5371,6 +5519,7 @@ "deprecated": "This version is no longer supported. Please see https://eslint.org/version-support for other options.", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@eslint-community/eslint-utils": "^4.2.0", "@eslint-community/regexpp": "^4.6.1", @@ -7054,6 +7203,15 @@ "node": ">=8" } }, + "node_modules/is-plain-obj": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/is-plain-obj/-/is-plain-obj-2.1.0.tgz", + "integrity": "sha512-YWnfyRwxL/+SsrWYfOpUtz5b3YD+nyfkHvjbcanzk8zgyO4ASD67uVMRt8k5bM4lLMDnXfriRhOpemw+NfT1eA==", + "license": "MIT", + "engines": { + "node": ">=8" + } + }, "node_modules/is-regex": { "version": "1.2.1", "resolved": "https://registry.npmjs.org/is-regex/-/is-regex-1.2.1.tgz", @@ -7925,6 +8083,12 @@ "node": ">= 0.4" } }, + "node_modules/mdn-data": { + "version": "2.0.14", + "resolved": "https://registry.npmjs.org/mdn-data/-/mdn-data-2.0.14.tgz", + "integrity": "sha512-dn6wd0uw5GsdswPFfsgMp5NSB0/aDe6fK94YJV/AJDYXL6HVLWBsxeq7js7Ad+mU2K9LAlwpk6kN2D5mwCPVow==", + "license": "CC0-1.0" + }, "node_modules/media-typer": { "version": "0.3.0", "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-0.3.0.tgz", @@ -7941,6 +8105,18 @@ "integrity": "sha512-zYiwtZUcYyXKo/np96AGZAckk+FWWsUdJ3cHGGmld7+AhvcWmQyGCYUh1hc4Q/pkOhb65dQR/pqCyK0cOaHz4Q==", "license": "MIT" }, + "node_modules/merge-options": { + "version": "3.0.4", + "resolved": "https://registry.npmjs.org/merge-options/-/merge-options-3.0.4.tgz", + "integrity": "sha512-2Sug1+knBjkaMsMgf1ctR1Ujx+Ayku4EdJN4Z+C2+JzoeF7A3OZ9KM2GY0CpQS51NR61LTurMJrRKPhSs3ZRTQ==", + "license": "MIT", + "dependencies": { + "is-plain-obj": "^2.1.0" + }, + "engines": { + "node": ">=10" + } + }, "node_modules/merge-stream": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/merge-stream/-/merge-stream-2.0.0.tgz", @@ -8441,6 +8617,18 @@ "node": ">=8" } }, + "node_modules/nth-check": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/nth-check/-/nth-check-2.1.1.tgz", + "integrity": "sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w==", + "license": "BSD-2-Clause", + "dependencies": { + "boolbase": "^1.0.0" + }, + "funding": { + "url": "https://github.com/fb55/nth-check?sponsor=1" + } + }, "node_modules/nullthrows": { "version": "1.1.1", "resolved": "https://registry.npmjs.org/nullthrows/-/nullthrows-1.1.1.tgz", @@ -8856,6 +9044,7 @@ "integrity": "sha512-UOnG6LftzbdaHZcKoPFtOcCKztrQ57WkHDeRD9t/PTQtmT0NHSeWWepj6pS0z/N7+08BHFDQVUrfmfMRcZwbMg==", "dev": true, "license": "MIT", + "peer": true, "bin": { "prettier": "bin/prettier.cjs" }, @@ -9044,6 +9233,7 @@ "resolved": "https://registry.npmjs.org/react/-/react-19.2.0.tgz", "integrity": "sha512-tmbWg6W31tQLeB5cdIBOicJDJRR2KzXsV7uSK9iNfLWQ5bIZfxuPEHp7M8wiHyHnn0DD1i7w3Zmin0FtkrwoCQ==", "license": "MIT", + "peer": true, "engines": { "node": ">=0.10.0" } @@ -9084,7 +9274,6 @@ "resolved": "https://registry.npmjs.org/react-freeze/-/react-freeze-1.0.4.tgz", "integrity": "sha512-r4F0Sec0BLxWicc7HEyo2x3/2icUTrRmDjaaRyzzn+7aDyFZliszMDOgLVwSnQnYENOlL1o569Ze2HZefk8clA==", "license": "MIT", - "peer": true, "engines": { "node": ">=10" }, @@ -9103,6 +9292,7 @@ "resolved": "https://registry.npmjs.org/react-native/-/react-native-0.83.1.tgz", "integrity": "sha512-mL1q5HPq5cWseVhWRLl+Fwvi5z1UO+3vGOpjr+sHFwcUletPRZ5Kv+d0tUfqHmvi73/53NjlQqX1Pyn4GguUfA==", "license": "MIT", + "peer": true, "dependencies": { "@jest/create-cache-key-function": "^29.7.0", "@react-native/assets-registry": "0.83.1", @@ -9161,6 +9351,7 @@ "resolved": "https://registry.npmjs.org/react-native-fs/-/react-native-fs-2.20.0.tgz", "integrity": "sha512-VkTBzs7fIDUiy/XajOSNk0XazFE9l+QlMAce7lGuebZcag5CnjszB+u4BdqzwaQOdcYb5wsJIsqq4kxInIRpJQ==", "license": "MIT", + "peer": true, "dependencies": { "base-64": "^0.1.0", "utf8": "^3.0.0" @@ -9180,6 +9371,7 @@ "resolved": "https://registry.npmjs.org/react-native-gesture-handler/-/react-native-gesture-handler-2.30.0.tgz", "integrity": "sha512-5YsnKHGa0X9C8lb5oCnKm0fLUPM6CRduvUUw2Bav4RIj/C3HcFh4RIUnF8wgG6JQWCL1//gRx4v+LVWgcIQdGA==", "license": "MIT", + "peer": true, "dependencies": { "@egjs/hammerjs": "^2.0.17", "hoist-non-react-statics": "^3.3.0", @@ -9212,6 +9404,7 @@ "integrity": "sha512-hcvjTu9YJE9fMmnAUvhG8CxvYLpOuMQ/2eyi/S6GyrecezF6Rmk/uRQEL6v09BRFWA/xRVZNQVulQPS+2HS3mQ==", "hasInstallScript": true, "license": "MIT", + "peer": true, "peerDependencies": { "react": "*", "react-native": "*" @@ -9222,6 +9415,7 @@ "resolved": "https://registry.npmjs.org/react-native-safe-area-context/-/react-native-safe-area-context-5.6.2.tgz", "integrity": "sha512-4XGqMNj5qjUTYywJqpdWZ9IG8jgkS3h06sfVjfw5yZQZfWnRFXczi0GnYyFyCc2EBps/qFmoCH8fez//WumdVg==", "license": "MIT", + "peer": true, "peerDependencies": { "react": "*", "react-native": "*" @@ -9252,6 +9446,21 @@ "react-native": "*" } }, + "node_modules/react-native-svg": { + "version": "15.15.3", + "resolved": "https://registry.npmjs.org/react-native-svg/-/react-native-svg-15.15.3.tgz", + "integrity": "sha512-/k4KYwPBLGcx2f5d4FjE+vCScK7QOX14cl2lIASJ28u4slHHtIhL0SZKU7u9qmRBHxTCKPoPBtN6haT1NENJNA==", + "license": "MIT", + "dependencies": { + "css-select": "^5.1.0", + "css-tree": "^1.1.3", + "warn-once": "0.1.1" + }, + "peerDependencies": { + "react": "*", + "react-native": "*" + } + }, "node_modules/react-native/node_modules/@react-native/virtualized-lists": { "version": "0.83.1", "resolved": "https://registry.npmjs.org/@react-native/virtualized-lists/-/virtualized-lists-0.83.1.tgz", @@ -10469,6 +10678,7 @@ "integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==", "dev": true, "license": "MIT", + "peer": true, "engines": { "node": ">=12" }, @@ -10649,6 +10859,7 @@ "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", "dev": true, "license": "Apache-2.0", + "peer": true, "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" @@ -10854,8 +11065,7 @@ "version": "0.1.1", "resolved": "https://registry.npmjs.org/warn-once/-/warn-once-0.1.1.tgz", "integrity": "sha512-VkQZJbO8zVImzYFteBXvBOZEl1qL175WH8VmZcxF2fZAoudNhNDvHi+doCaAEdU2l2vtcIwa2zn0QK5+I1HQ3Q==", - "license": "MIT", - "peer": true + "license": "MIT" }, "node_modules/wcwidth": { "version": "1.0.1", @@ -11116,6 +11326,7 @@ "integrity": "sha512-rftlrkhHZOcjDwkGlnUtZZkvaPHCsDATp4pGpuOOMDaTdDDXF91wuVDJoWoPsKX/3YPQ5fHuF3STjcYyKr+Qhg==", "dev": true, "license": "MIT", + "peer": true, "funding": { "url": "https://github.com/sponsors/colinhacks" } diff --git a/package.json b/package.json index 647e846..2b60bb1 100644 --- a/package.json +++ b/package.json @@ -1,16 +1,19 @@ { - "name": "runanywhere-starter-app", + "name": "latent", "version": "1.0.0", - "description": "A comprehensive starter app demonstrating RunAnywhere SDK capabilities - on-device AI for React Native", + "description": "Latent - Offline Meeting Intelligence. Privacy-first negotiation assistant powered by on-device AI.", "main": "index.js", "scripts": { "android": "react-native run-android", "ios": "react-native run-ios", "start": "react-native start", "test": "jest", - "lint": "eslint . --ext .js,.jsx,.ts,.tsx" + "lint": "eslint . --ext .js,.jsx,.ts,.tsx", + "check-models": "bash scripts/check-models.sh", + "rebuild-android": "cd android && ./gradlew clean && cd .. && react-native run-android" }, "dependencies": { + "@react-native-async-storage/async-storage": "^2.2.0", "@react-navigation/native": "^7.1.24", "@react-navigation/stack": "^7.6.16", "@runanywhere/core": "^0.18.1", @@ -24,7 +27,8 @@ "react-native-live-audio-stream": "^1.1.1", "react-native-nitro-modules": "^0.31.10", "react-native-safe-area-context": "~5.6.2", - "react-native-sound": "^0.13.0" + "react-native-sound": "^0.13.0", + "react-native-svg": "^15.15.3" }, "devDependencies": { "@babel/core": "^7.25.2", diff --git a/react-native.config.js b/react-native.config.js index 0ebe959..425fc29 100644 --- a/react-native.config.js +++ b/react-native.config.js @@ -6,6 +6,7 @@ * the RN 0.83 CLI. Pods must be installed manually: cd ios && pod install && cd .. */ module.exports = { + assets: ['./assets/fonts/'], project: { ios: { automaticPodsInstallation: false, diff --git a/scripts/check-models.sh b/scripts/check-models.sh new file mode 100755 index 0000000..b3c725d --- /dev/null +++ b/scripts/check-models.sh @@ -0,0 +1,53 @@ +#!/bin/bash + +echo "🔍 Checking for STT Model Files..." +echo "" + +MODELS_DIR="android/app/src/main/assets/models" + +# Check if directory exists +if [ ! -d "$MODELS_DIR" ]; then + echo "❌ Models directory does not exist: $MODELS_DIR" + echo "" + echo "Creating directory..." + mkdir -p "$MODELS_DIR" + echo "✅ Directory created: $MODELS_DIR" + echo "" + echo "đŸ“Ĩ Next steps:" + echo " 1. Download a Whisper model (whisper-tiny recommended)" + echo " 2. Place it in: $MODELS_DIR" + echo " 3. Update the model name in src/services/SpeechService.ts if needed" + echo " 4. Run: npm run rebuild-android" + exit 1 +fi + +echo "✅ Models directory exists: $MODELS_DIR" +echo "" + +# Check for model files +MODEL_FILES=$(find "$MODELS_DIR" -type f \( -name "*.gguf" -o -name "*.onnx" -o -name "*whisper*" \) 2>/dev/null) + +if [ -z "$MODEL_FILES" ]; then + echo "❌ No model files found in $MODELS_DIR" + echo "" + echo "đŸ“Ĩ Next steps:" + echo " 1. Download a Whisper model" + echo " 2. Place it in: $MODELS_DIR" + echo " 3. See MODEL_SETUP.md for detailed instructions" + exit 1 +fi + +echo "✅ Found model files:" +echo "" +echo "$MODEL_FILES" | while read file; do + size=$(du -h "$file" | cut -f1) + echo " đŸ“Ļ $(basename "$file") - $size" +done + +echo "" +echo "🎉 Model setup looks good!" +echo "" +echo "📝 Notes:" +echo " - Model will load on app startup" +echo " - Check logs for: [SpeechService] STT model loaded" +echo " - If issues persist, try Debug Mode in Settings" diff --git a/scripts/patch-rnfs.sh b/scripts/patch-rnfs.sh new file mode 100755 index 0000000..a508bdd --- /dev/null +++ b/scripts/patch-rnfs.sh @@ -0,0 +1,29 @@ +#!/bin/bash +# Patch react-native-fs to fix NullPointerException crash on React Native New Architecture +# Bug: promise.reject(null, ...) crashes because PromiseImpl.reject requires non-null code +RNFS_MANAGER="node_modules/react-native-fs/android/src/main/java/com/rnfs/RNFSManager.java" + +if [ -f "$RNFS_MANAGER" ]; then + sed -i.bak 's/promise.reject(null, ex.getMessage())/promise.reject("EUNSPECIFIED", ex.getMessage())/' "$RNFS_MANAGER" + rm -f "${RNFS_MANAGER}.bak" + echo "✅ Patched RNFSManager.java (null code fix)" +fi + +# Patch Downloader.java buffer sizes for faster downloads (8KB -> 256KB) +DOWNLOADER="node_modules/react-native-fs/android/src/main/java/com/rnfs/Downloader.java" + +if [ -f "$DOWNLOADER" ]; then + sed -i.bak 's/new BufferedInputStream(connection.getInputStream(), 8 \* 1024)/new BufferedInputStream(connection.getInputStream(), 512 * 1024)/' "$DOWNLOADER" + sed -i.bak 's/new byte\[8 \* 1024\]/new byte[256 * 1024]/' "$DOWNLOADER" + rm -f "${DOWNLOADER}.bak" + echo "✅ Patched Downloader.java (256KB buffer)" +fi + +# Patch RunAnywhere SDK to use foreground downloads (Android OEMs throttle background downloads) +SDK_FILE="node_modules/@runanywhere/core/src/services/FileSystem.ts" + +if [ -f "$SDK_FILE" ]; then + sed -i.bak 's/background: true,/background: false,/' "$SDK_FILE" + rm -f "${SDK_FILE}.bak" + echo "✅ Patched FileSystem.ts (foreground download)" +fi diff --git a/src/App.tsx b/src/App.tsx index 661a51b..fb2c1dc 100644 --- a/src/App.tsx +++ b/src/App.tsx @@ -1,21 +1,13 @@ import 'react-native-gesture-handler'; // Must be at the top! -import React, { useEffect } from 'react'; +import React from 'react'; import { NavigationContainer } from '@react-navigation/native'; import { createStackNavigator, TransitionPresets } from '@react-navigation/stack'; import { StatusBar } from 'react-native'; import { GestureHandlerRootView } from 'react-native-gesture-handler'; // Note: react-native-screens is shimmed in index.js for iOS New Architecture compatibility -import { RunAnywhere, SDKEnvironment } from '@runanywhere/core'; -import { ModelServiceProvider, registerDefaultModels } from './services/ModelService'; +import { ModelServiceProvider } from './services/ModelService'; import { AppColors } from './theme'; -import { - HomeScreen, - ChatScreen, - ToolCallingScreen, - SpeechToTextScreen, - TextToSpeechScreen, - VoicePipelineScreen, -} from './screens'; +import { HomeScreen, LiveSessionScreen, InsightsScreen, SettingsScreen, DisclaimerScreen, OutcomeReplayScreen, PreSessionFormScreen, PreSessionStrategyScreen } from './screens'; import { RootStackParamList } from './navigation/types'; // Using JS-based stack navigator instead of native-stack @@ -23,87 +15,65 @@ import { RootStackParamList } from './navigation/types'; const Stack = createStackNavigator(); const App: React.FC = () => { - useEffect(() => { - // Initialize SDK - const initializeSDK = async () => { - try { - // Initialize RunAnywhere SDK (Development mode doesn't require API key) - await RunAnywhere.initialize({ - environment: SDKEnvironment.Development, - }); - - // Register backends (per docs: https://docs.runanywhere.ai/react-native/quick-start) - const { LlamaCPP } = await import('@runanywhere/llamacpp'); - const { ONNX } = await import('@runanywhere/onnx'); - - LlamaCPP.register(); - ONNX.register(); - - // Register default models - await registerDefaultModels(); - - console.log('RunAnywhere SDK initialized successfully'); - } catch (error) { - console.error('Failed to initialize RunAnywhere SDK:', error); - } - }; - - initializeSDK(); - }, []); + // SDK initialization is now handled inside ModelServiceProvider return ( - + + + diff --git a/src/ai/BehavioralAnalyticsEngine.ts b/src/ai/BehavioralAnalyticsEngine.ts new file mode 100644 index 0000000..f4a1ef4 --- /dev/null +++ b/src/ai/BehavioralAnalyticsEngine.ts @@ -0,0 +1,76 @@ +import { Session, NegotiationMode } from '../types/session'; + +export interface BehavioralProfile { + archetype: string[]; + leverageCaptureScore: number; + objectionHandlingScore: number; + fillerWordCount: number; + hesitationMoments: number; +} + +/** + * 🧠 Strategic Outcome Replayâ„ĸ - Behavioral Pressure Analytics + * Consumes the entire transcript text to calculate psychological profile markers + * like hesitation, filler spam, and over-talking ratios. + */ +export const BehavioralAnalyticsEngine = { + analyzeTranscript(session: Session): BehavioralProfile { + console.log(`[BehavioralAnalyticsEngine] Profiling Transcript for Session: ${session.id}`); + + if (!session.transcript || session.transcript.length === 0) { + return this.getEmptyProfile(); + } + + const fullText = session.transcript.map(t => t.text).join(' ').toLowerCase(); + + // 1. Calculate Filler Words + const fillerList = ['um ', 'uh ', 'like ', 'you know', 'basically', 'actually']; + let fillerCount = 0; + fillerList.forEach(word => { + const regex = new RegExp(word, 'gi'); + const matches = fullText.match(regex); + if (matches) fillerCount += matches.length; + }); + + // 2. Calculate Hesitation Markers + const hesitationList = ['i think maybe', 'if possible', 'sort of', 'kind of']; + let hesitationCount = 0; + hesitationList.forEach(word => { + const regex = new RegExp(word, 'gi'); + const matches = fullText.match(regex); + if (matches) hesitationCount += matches.length; + }); + + // Generate Archetype + const archetypes = []; + if (fillerCount > 5 || hesitationCount > 3) { + archetypes.push("Defensive Under Pressure"); + archetypes.push("Misses Leverage Signals"); + } else { + archetypes.push("Strong Frame Control"); + archetypes.push("Direct Communicator"); + } + + if (fullText.includes("budget") && hesitationCount > 2) { + archetypes.push("Accepts Anchors Quickly"); + } + + return { + archetype: archetypes, + leverageCaptureScore: Math.round(100 - (hesitationCount * 15) - (fillerCount * 5)) > 0 ? Math.round(100 - (hesitationCount * 15) - (fillerCount * 5)) : 30, + objectionHandlingScore: Math.round(100 - (fillerCount * 8)) > 0 ? Math.round(100 - (fillerCount * 8)) : 40, + fillerWordCount: fillerCount, + hesitationMoments: hesitationCount + }; + }, + + getEmptyProfile(): BehavioralProfile { + return { + archetype: ["Insufficient Data"], + leverageCaptureScore: 0, + objectionHandlingScore: 0, + fillerWordCount: 0, + hesitationMoments: 0 + }; + } +}; diff --git a/src/ai/OutcomeReplayEngine.ts b/src/ai/OutcomeReplayEngine.ts new file mode 100644 index 0000000..29aa225 --- /dev/null +++ b/src/ai/OutcomeReplayEngine.ts @@ -0,0 +1,152 @@ +import { Session, NegotiationMode, DetectedPattern, NegotiationPattern } from '../types/session'; + +export interface TacticalImprovement { + originalQuote: string; + originalStrengthScore: number; + improvedReframing: string; + improvedStrengthScore: number; + tacticType: NegotiationPattern; + delta: number; +} + +export interface PostSessionSummary { + whatWorked: string[]; + signalsOfInterest: string[]; + hiddenObjections: string[]; + followUpStrategy: string; +} + +export interface ReplaySimulationResult { + sessionId: string; + opportunities: TacticalImprovement[]; + totalDeltaGained: number; + postSessionSummary: PostSessionSummary; +} + +/** + * 🧠 Strategic Outcome Replayâ„ĸ - Counterfactual Simulation Engine + * Consumes an entire offline session transcript and identifies key + * moments where the user could have deployed stronger structural phrasing. + */ +export const OutcomeReplayEngine = { + generateSimulation(session: Session): ReplaySimulationResult { + console.log(`[OutcomeReplayEngine] Analyzing Session: ${session.id}`); + + const opportunities: TacticalImprovement[] = []; + + // We look through all the raw patterns detected during the session + // Wait, the session saves patterns that crossed the >60% threshold. + // We want to simulate what the user SHOULD have said instead. + // If we want counterfactuals, we analyze the *user's* weak responses to anchors, + // or moments the user deployed weak objections. + + session.detectedPatterns.forEach((pattern: DetectedPattern) => { + const opportunity = this.analyzePatternCounterfactual(pattern, session.mode); + if (opportunity) { + opportunities.push(opportunity); + } + }); + + const totalDelta = opportunities.reduce((acc, curr) => acc + curr.delta, 0); + + const postSessionSummary: PostSessionSummary = this.generatePostSessionSummary(session); + + return { + sessionId: session.id, + opportunities: opportunities.sort((a,b) => b.delta - a.delta), // highest improvement first + totalDeltaGained: totalDelta, + postSessionSummary + }; + }, + + generatePostSessionSummary(session: Session): PostSessionSummary { + const patterns = session.detectedPatterns.map(p => p.pattern); + const hasPositive = patterns.includes(NegotiationPattern.POSITIVE_SIGNAL) || patterns.includes(NegotiationPattern.COMMITMENT_LANGUAGE); + const hasStrength = patterns.includes(NegotiationPattern.STRENGTH_SIGNAL) || patterns.includes(NegotiationPattern.ANCHORING); + const hasBudget = patterns.includes(NegotiationPattern.BUDGET_OBJECTION); + const hasAuthority = patterns.includes(NegotiationPattern.AUTHORITY_PRESSURE); + + const summary: PostSessionSummary = { + whatWorked: [], + signalsOfInterest: [], + hiddenObjections: [], + followUpStrategy: '' + }; + + // What worked + if (hasStrength) summary.whatWorked.push("You actively asserted boundaries and anchored the discussion framework."); + if (session.duration > 180000) summary.whatWorked.push("You maintained a long-form engagement, indicating deep exploration."); + if (summary.whatWorked.length === 0) summary.whatWorked.push("You laid foundational groundwork for future discovery."); + + // Signals of Interest + if (hasPositive) summary.signalsOfInterest.push("Explicit verbal agreement or enthusiasm detected."); + if (patterns.includes(NegotiationPattern.TIME_PRESSURE)) summary.signalsOfInterest.push("They exhibited urgency to close or rush timelines."); + if (summary.signalsOfInterest.length === 0) summary.signalsOfInterest.push("No overwhelming buy-in detected. They are still evaluating risk."); + + // Hidden Objections + if (hasBudget) summary.hiddenObjections.push("Financial constraints or explicit budget ceilings."); + if (hasAuthority) summary.hiddenObjections.push("Decision-making power is delegated elsewhere."); + if (patterns.includes(NegotiationPattern.DEFLECTION)) summary.hiddenObjections.push("Unwillingness to commit to specific next steps."); + if (summary.hiddenObjections.length === 0) summary.hiddenObjections.push("No explicit structural traps observed."); + + // Follow up Strategy + if (hasAuthority) { + summary.followUpStrategy = "Draft an executive summary specifically targeting the hidden decision-maker they mentioned."; + } else if (hasBudget) { + summary.followUpStrategy = "Send a phased implementation plan that breaks your pricing into lower-risk milestones."; + } else if (hasPositive) { + summary.followUpStrategy = "Strike while the iron is hot. Send a concrete timeline summarizing today's verbal agreement."; + } else { + summary.followUpStrategy = "Follow up with a targeted question probing their greatest risk-aversion concern."; + } + + return summary; + }, + + analyzePatternCounterfactual(pattern: DetectedPattern, mode: NegotiationMode): TacticalImprovement | null { + // Base math algorithm: We penalize the original text by generating an arbitrary baseline + // and calculate the mathematical Persuasion Delta Score. + const originalStrength = Math.round(Math.random() * 20 + 30); // 30-50% + const improvedStrength = Math.round(Math.random() * 20 + 75); // 75-95% + const delta = improvedStrength - originalStrength; + + switch (pattern.pattern) { + case NegotiationPattern.ANCHORING: + return { + originalQuote: pattern.context ?? "Transcript unavailable", + originalStrengthScore: originalStrength, + improvedReframing: `Based on your stated scope, the floor requirement sits closer to X to ensure deployment success without sacrificing reliability.`, + improvedStrengthScore: improvedStrength, + tacticType: NegotiationPattern.ANCHORING, + delta + }; + case NegotiationPattern.BUDGET_OBJECTION: + return { + originalQuote: pattern.context ?? "Transcript unavailable", + originalStrengthScore: originalStrength, + improvedReframing: `If the current allocation is locked, we can strip away Phase 2 deliverables to meet that exact figure today.`, + improvedStrengthScore: improvedStrength, + tacticType: NegotiationPattern.BUDGET_OBJECTION, + delta + }; + case NegotiationPattern.STRENGTH_SIGNAL: + return { + originalQuote: pattern.context ?? "Transcript unavailable", + originalStrengthScore: originalStrength, + improvedReframing: `My specific architectural decision here directly accelerated Q3 delivery by 40%, generating $X in early pipeline.`, + improvedStrengthScore: improvedStrength, + tacticType: NegotiationPattern.STRENGTH_SIGNAL, + delta + }; + default: + return { + originalQuote: pattern.context ?? "Transcript unavailable", + originalStrengthScore: originalStrength, + improvedReframing: `Applying a structural pivot here neutralizes the frame and returns leverage to your court.`, + improvedStrengthScore: improvedStrength, + tacticType: pattern.pattern, + delta + }; + } + } +}; diff --git a/src/ai/StrategicPreparationEngine.ts b/src/ai/StrategicPreparationEngine.ts new file mode 100644 index 0000000..81a89ed --- /dev/null +++ b/src/ai/StrategicPreparationEngine.ts @@ -0,0 +1,317 @@ +/** + * 🔒 PRIVACY NOTICE + * All strategic preparation generation runs locally on device using deterministic logic. + * No LLM calls. No external API calls. No data leaves this device. + */ + +import { NegotiationMode, StrategicAnalysis, PreSessionInputs } from '../types/session'; + +export interface FormField { + id: string; + label: string; + placeholder: string; + type: 'text' | 'number' | 'multiline'; + required: boolean; +} + +export class StrategicPreparationEngine { + + /** + * Generates the required input fields based on the selected negotiation mode + */ + public static getFormConfigForMode(mode: NegotiationMode): FormField[] { + switch (mode) { + case NegotiationMode.JOB_INTERVIEW: + return [ + { id: 'role', label: 'Role applying for', placeholder: 'e.g. Senior Product Manager', type: 'text', required: true }, + { id: 'company_type', label: 'Company type', placeholder: 'Startup / MNC / Agency', type: 'text', required: true }, + { id: 'years_exp', label: 'Years of experience', placeholder: 'e.g. 5', type: 'number', required: true }, + { id: 'top_skills', label: 'Top 3 skills', placeholder: 'e.g. Python, Leadership, System Design', type: 'text', required: true }, + { id: 'achievement', label: 'Biggest measurable achievement', placeholder: 'e.g. Scaled revenue by 40%', type: 'multiline', required: true }, + { id: 'expected_salary', label: 'Expected salary', placeholder: 'e.g. $150,000', type: 'text', required: true }, + { id: 'weakness', label: 'Known weaknesses', placeholder: 'e.g. Lack of enterprise experience', type: 'multiline', required: false }, + ]; + case NegotiationMode.SALES: + return [ + { id: 'product', label: 'Product/Service', placeholder: 'e.g. Enterprise SaaS Platform', type: 'text', required: true }, + { id: 'target_profile', label: 'Target customer profile', placeholder: 'e.g. CTOs at mid-market companies', type: 'text', required: true }, + { id: 'price_range', label: 'Price range', placeholder: 'e.g. $50k - $100k ARR', type: 'text', required: true }, + { id: 'objections', label: 'Known objections', placeholder: 'e.g. Too expensive, difficult integration', type: 'multiline', required: true }, + { id: 'competitors', label: 'Competitor names', placeholder: 'e.g. Salesforce, Oracle', type: 'text', required: false }, + { id: 'goal', label: 'Goal of call', placeholder: 'e.g. Schedule technical demo', type: 'text', required: true }, + ]; + case NegotiationMode.STARTUP_PITCH: + return [ + { id: 'problem', label: 'Problem statement', placeholder: 'Describe the core problem in 1 sentence', type: 'multiline', required: true }, + { id: 'solution', label: 'Solution summary', placeholder: 'How do you fix it?', type: 'multiline', required: true }, + { id: 'traction', label: 'Traction metrics', placeholder: 'e.g. 10k MRR, 50% MoM growth', type: 'multiline', required: true }, + { id: 'revenue_model', label: 'Revenue model', placeholder: 'e.g. B2B Subscription', type: 'text', required: true }, + { id: 'funding_ask', label: 'Funding ask', placeholder: 'e.g. $2M Seed', type: 'text', required: true }, + { id: 'market_size', label: 'Target market size', placeholder: 'e.g. $10B TAM', type: 'text', required: true }, + ]; + case NegotiationMode.SALARY_RAISE: + return [ + { id: 'current_role', label: 'Current role', placeholder: 'e.g. Marketing Director', type: 'text', required: true }, + { id: 'current_salary', label: 'Current salary', placeholder: 'e.g. $120,000', type: 'text', required: true }, + { id: 'market_salary', label: 'Market salary range', placeholder: 'e.g. $140,000 - $160,000', type: 'text', required: true }, + { id: 'achievements', label: 'Key achievements', placeholder: 'What did you do this year?', type: 'multiline', required: true }, + { id: 'manager_type', label: 'Manager personality type', placeholder: 'e.g. Analytical, Supportive, Strict', type: 'text', required: true }, + { id: 'desired_raise', label: 'Desired raise %', placeholder: 'e.g. 15%', type: 'text', required: true }, + ]; + case NegotiationMode.INVESTOR_MEETING: + return [ + { id: 'fund_name', label: 'Fund Name', placeholder: 'e.g. Sequoia Capital', type: 'text', required: false }, + { id: 'fund_thesis', label: 'Fund Thesis/Focus', placeholder: 'e.g. Deeptech SaaS', type: 'text', required: false }, + { id: 'burn_rate', label: 'Current monthly burn', placeholder: 'e.g. $50k/mo', type: 'text', required: true }, + { id: 'runway', label: 'Months of runway left', placeholder: 'e.g. 6 months', type: 'number', required: true }, + { id: 'valuation_cap', label: 'Target Valuation Cap', placeholder: 'e.g. $15M Post-Money', type: 'text', required: true }, + { id: 'weakness', label: 'Biggest risk to thesis', placeholder: 'e.g. High churn rate', type: 'multiline', required: true }, + ]; + case NegotiationMode.CLIENT_NEGOTIATION: + return [ + { id: 'client_name', label: 'Client / Company Name', placeholder: 'e.g. Acme Corp', type: 'text', required: true }, + { id: 'project_scope', label: 'Project Scope', placeholder: 'Briefly describe the deliverables', type: 'multiline', required: true }, + { id: 'timeline', label: 'Proposed Timeline', placeholder: 'e.g. 3 Months', type: 'text', required: true }, + { id: 'budget', label: 'Client Budget (if known)', placeholder: 'e.g. $25k', type: 'text', required: false }, + { id: 'leverage', label: 'Your Leverage', placeholder: 'Why do they need YOU specifically?', type: 'multiline', required: true }, + ]; + case NegotiationMode.CUSTOM_SCENARIO: + default: + return [ + { id: 'scenario_desc', label: 'Describe the scenario', placeholder: 'Who are you talking to and what do you want?', type: 'multiline', required: true }, + { id: 'your_goal', label: 'Your ultimate goal', placeholder: 'What is a "win" for you?', type: 'text', required: true }, + { id: 'their_goal', label: 'Their likely goal', placeholder: 'What do they want out of this?', type: 'text', required: true }, + { id: 'leverage', label: 'Your Leverage', placeholder: 'What advantages do you hold?', type: 'multiline', required: true }, + { id: 'risks', label: 'Biggest Risk / Weakness', placeholder: 'What are you afraid they will bring up?', type: 'multiline', required: false }, + ]; + } + } + + /** + * Deterministically generates the 10-point Strategic Analysis plan completely offline + */ + public static generateStrategicAnalysis(mode: NegotiationMode, inputs: PreSessionInputs): StrategicAnalysis { + console.log(`[StrategicPreparationEngine] Generating analysis for ${mode}...`); + + // Base template + const analysis: StrategicAnalysis = { + powerPositioning: '', + likelyObjections: [], + psychologicalTactics: [], + recommendedResponses: [], + highImpactPhrases: [], + phrasesToAvoid: [], + confidenceTriggers: [], + openingScript: '', + closingScript: '', + mistakesToAvoid: [], + }; + + switch (mode) { + case NegotiationMode.JOB_INTERVIEW: + analysis.powerPositioning = `Position yourself not as an applicant, but as a peer evaluating a mutual fit. Your anchor is your achievement: ${inputs.achievement || 'your strong track record'}. Leverage your ${inputs.years_exp || '*'} years of experience as proof of execution risk reduction for the ${inputs.company_type || 'company'}.`; + + analysis.likelyObjections = [ + `"We are looking for someone with more experience in [Specific Niche]."`, + `"Your salary expectation of ${inputs.expected_salary || 'this range'} is above our current band."`, + inputs.weakness ? `"Can you explain your background regarding ${inputs.weakness}?"` : `"Are you comfortable operating outside your core skillset?"` + ]; + + analysis.psychologicalTactics = [ + 'The Pause: They may stay silent after you answer to force you to over-explain. Do not fill the silence.', + 'The Low Anchor: Suggesting a title or compensation lower than market to test your boundary.', + 'Pressure Testing: Deliberately challenging your achievements to see if you get defensive.' + ]; + + analysis.recommendedResponses = [ + `If they challenge your weakness: "That is an area I'm actively bridging, but my core strength in ${inputs.top_skills?.split(',')[0] || 'execution'} allows me to drive results while I adapt quickly."`, + `If they press on salary early: "I'd prefer to ensure I'm the perfect fit for the ${inputs.role || 'role'} before we lock in compensation. Does that work for you?"`, + `If they offer a low anchor: "Based on the market rate for ${inputs.years_exp || 'my'} years of experience and the scope of this role, my floor is higher than that."` + ]; + + analysis.highImpactPhrases = [ + '"I drove...", "I implemented...", "The measurable outcome was..."', + '"My thesis for this role is..."', + '"How does this role directly impact top-line revenue?"' + ]; + + analysis.phrasesToAvoid = [ + '"I think...", "I believe...", "I feel like..."', + '"I was part of a team that..." (Use "I led" or "I owned")', + '"I really need this job."' + ]; + + analysis.confidenceTriggers = [ + 'Maintain 3 seconds of eye contact when stating your achievement.', + 'Keep your hands visible on the table (if in person or deep frame video).', + 'Drop your vocal pitch slightly at the end of sentences to convey authority.' + ]; + + analysis.openingScript = `"It's great to connect. I've been following [Company]'s recent moves, and I'm excited to explore how my background in ${inputs.top_skills?.split(',')[0] || 'my field'} can help accelerate your goals for this quarter."`; + + analysis.closingScript = `"Based on our conversation, I'm confident I can execute on [Their Core Metric]. What timelines should I expect for next steps so I can align my other conversations?"`; + + analysis.mistakesToAvoid = [ + 'Revealing your current salary (it caps your future salary).', + 'Rambling for more than 90 seconds on a single answer.', + 'Asking logistical questions (hours, vacation) before an offer is extended.' + ]; + break; + + case NegotiationMode.SALARY_RAISE: + analysis.powerPositioning = `You are presenting a business case, not a personal plea. You are a highly-performing asset (${inputs.current_role || 'in your role'}) seeking market alignment. Your request of ${inputs.desired_raise || 'a raise'} is justified by the ROI you provided: ${inputs.achievements || 'your recent contributions'}.`; + + analysis.likelyObjections = [ + `"The budget is frozen until next quarter."`, + `"You are already at the top of the band for your title."`, + `"We need to see more leadership before we can authorize a ${inputs.desired_raise || 'raise'} jump."` + ]; + + analysis.psychologicalTactics = [ + 'Sympathy Play: "I really wish I could, but my hands are tied by HR/Finance." (Authority Pressure)', + 'The Delay: "Let\'s circle back to this during annual reviews in 6 months." (Deflection)', + 'The Guilt Trip: "Times are tough for the team right now." (Negative Signal)' + ]; + + analysis.recommendedResponses = [ + `If budget is frozen: "If base compensation is locked, are we able to explore an off-cycle bonus structure or equity grant tied to my recent ${inputs.achievements?.substring(0,20) || 'successes'}?"`, + `If deferred to later: "To ensure we have a productive conversation then, can we put in writing the exact metrics required to unlock the ${inputs.market_salary || 'market'} range?"`, + `If given a hard no: "I appreciate the transparency. For me to continue operating at peak capacity, I need a clear pathway to market-rate compensation. How can we build that?"` + ]; + + analysis.highImpactPhrases = [ + '"Market alignment," "Value capture," "Data shows..."', + '"Based on the scope of my current contributions..."', + '"My priority is continuing to drive value here."' + ]; + + analysis.phrasesToAvoid = [ + '"I need the money for [personal reason]."', + '"I haven\'t had a raise in X years." (Focus on value, not time)', + '"If I don\'t get this, I will quit." (Never issue ultimatums unless you have an offer in hand)' + ]; + + analysis.confidenceTriggers = [ + 'Bring physical data: Have a printed sheet or shared screen showing the market data and your achievements.', + 'Silence after the ask. State your number, then stop talking.', + 'Maintain a collaborative, non-combative posture.' + ]; + + analysis.openingScript = `"Thank you for taking the time to meet. The purpose of this sync is to review my recent contributions—specifically ${inputs.achievements?.substring(0,30) || 'my recent wins'}—and discuss aligning my compensation with both my current output and the market rate."`; + + analysis.closingScript = `"I appreciate your time going over this. I’ll send a summary email outlining the milestones we discussed to reach the ${inputs.market_salary || 'target'} band by [Date]."`; + + analysis.mistakesToAvoid = [ + 'Apologizing for asking for money.', + 'Getting visibly emotional or defensive if rejected.', + 'Negotiating against yourself before they even respond.' + ]; + break; + + case NegotiationMode.SALES: + analysis.powerPositioning = `You are a high-value consultant diagnosing a painful problem, not a vendor pushing a product. Your ${inputs.product || 'solution'} is the antidote to the friction plaguing their ${inputs.target_profile || 'team'}. Control the frame by asking diagnostic questions.`; + + analysis.likelyObjections = [ + inputs.objections || `"Your solution is too expensive for our current budget."`, + inputs.competitors ? `"We are already looking at ${inputs.competitors.split(',')[0]} and they are cheaper."` : `"We are satisfied with our current manual process."`, + `"This isn't a priority for this quarter."` + ]; + + analysis.psychologicalTactics = [ + 'The Silent Treatment: Letting you pitch into the void to drain your confidence and make you offer discounts.', + 'Phantom Authority: Pretending they have decision power to extract information, then claiming they need to "run it by the boss." (Authority Pressure)', + 'Commoditization: Comparing your complex solution to a basic feature to drive down the price.' + ]; + + analysis.recommendedResponses = [ + `If they claim it's too expensive: "Expensive compared to what? What is the current cost of doing nothing for another 6 months?"`, + `If they mention competitors: "We respect ${inputs.competitors?.split(',')[0] || 'them'}. They are great for basic needs. Our clients choose us when they need [Your Unique Value Proposition]."`, + `If urgency is low: "I hear you. If we pause this until next quarter, how will you handle the fallout from [Specific Pain Point] in the meantime?"` + ]; + + analysis.highImpactPhrases = [ + '"What happens if you do nothing?"', + `"Typically, ${inputs.target_profile || 'leaders'} in your position tell me..."`, + '"Is it fair to say..." (Calibrated question)' + ]; + + analysis.phrasesToAvoid = [ + '"Just checking in / following up..." (Provides zero value)', + '"To be honest with you..." (Implies you weren\'t honest before)', + '"Does that make sense?" (Can sound patronizing)' + ]; + + analysis.confidenceTriggers = [ + 'Mirroring: Repeat the last 1-3 words of their sentence to encourage them to elaborate without you asking a question.', + 'Slow your cadence down by 20%.', + 'Use downward inflection at the end of statements to project certainty.' + ]; + + analysis.openingScript = `"I appreciate you carving out time. My goal today isn't to pitch you ${inputs.product || 'our product'}—it's to determine if we are a fit. Are you open to me asking a few targeted questions about how you're currently handling [Problem Area]?"`; + + analysis.closingScript = `"Based on what you've shared about [Their Pain Point], I believe there is a strong fit. To ensure we respect your time, are you the sole decision-maker for the ${inputs.price_range || 'allocated'} budget, or should we include anyone else on the next ${inputs.goal || 'demo'}?"`; + + analysis.mistakesToAvoid = [ + 'Pitching features instead of diagnosing pain.', + 'Answering unasked objections out of nervousness.', + 'Ending the call without a concrete commitment for the next step on the calendar.' + ]; + break; + + // ... Add similar logic branches for Startup Pitch, Investor Meeting, Client, and Custom + // Default to Custom logic if unhandled + default: + analysis.powerPositioning = `Root your frame in absolute certainty. You have leverage because ${inputs.leverage || 'you bring unique value'}. Do not enter the conversion from a defensive posture; you are exploring a mutual exchange of value.`; + + analysis.likelyObjections = [ + inputs.risks || `"I don't think we can accommodate that request."`, + `"This requires further review from other stakeholders."`, + `"We need to see a timeline shift to make this viable."` + ]; + + analysis.psychologicalTactics = [ + 'Anchoring: Opening with an extreme position to drag the midpoint in their favor.', + 'Time Restraints: "I only have 5 minutes." (Rushing you into poor decisions).', + 'Flinching: Visibly acting shocked at your proposal to induce guilt.' + ]; + + analysis.recommendedResponses = [ + `If they flinch: Remain completely silent. Do not justify your position until they articulate a specific argument.`, + `If they claim lack of authority: "Who else needs to be involved, and can we get them on the line now?"`, + `To protect your goal of ${inputs.your_goal || 'success'}: "If we cannot agree on this, what is your proposed alternative that still solves my core requirement?"` + ]; + + analysis.highImpactPhrases = [ + '"It sounds like [Their goal] is a priority for you..." (Labeling)', + '"How am I supposed to do that?" (The ultimate deferral question)', + '"Let\'s put a pin in that and address [Your Priority] first."' + ]; + + analysis.phrasesToAvoid = [ + '"I\'m sorry but...", "I hope...", "I\'ll try..."', + '"Is that okay with you?" (Weak framing)', + 'Any nervous laughter.' + ]; + + analysis.confidenceTriggers = [ + 'Use Late-Night FM DJ Voice: Deep, slow, calming inflection.', + 'Take up physical space. Do not cross arms or shrink posture.', + 'Embrace the awkward silence after you make a demand.' + ]; + + analysis.openingScript = `"I'm glad we could connect. My objective today is to find a pathway to ${inputs.your_goal || 'our mutual goal'}, while addressing your need for ${inputs.their_goal || 'efficiency'}. Are you open to exploring the variables on the table?"`; + + analysis.closingScript = `"We've covered significant ground. To formalize this, I will draft a summary emphasizing how we leverage ${inputs.leverage?.substring(0, 20) || 'our mutual assets'}. Let's target [Date] of next week for finalizing signatures."`; + + analysis.mistakesToAvoid = [ + 'Speaking first after laying a major proposal on the table.', + 'Conceding a variable without demanding something in return.', + 'Letting the opponent dictate the agenda.' + ]; + break; + } + + console.log('[StrategicPreparationEngine] Analysis generated successfully.'); + return analysis; + } +} diff --git a/src/ai/WhisperAutoCorrector.ts b/src/ai/WhisperAutoCorrector.ts new file mode 100644 index 0000000..c858187 --- /dev/null +++ b/src/ai/WhisperAutoCorrector.ts @@ -0,0 +1,124 @@ +import { NegotiationMode } from '../types/session'; +import { PATTERN_LIBRARY, MODE_INTENT_MATRIX } from './patternLibrary'; + +/** + * Calculates the Levenshtein distance between two strings. + */ +export function levenshteinDistance(s1: string, s2: string): number { + if (s1.length === 0) return s2.length; + if (s2.length === 0) return s1.length; + + const m = s1.length; + const n = s2.length; + const dp: number[][] = Array.from({ length: m + 1 }, () => Array(n + 1).fill(0)); + + for (let i = 0; i <= m; i++) dp[i][0] = i; + for (let j = 0; j <= n; j++) dp[0][j] = j; + + for (let i = 1; i <= m; i++) { + for (let j = 1; j <= n; j++) { + const cost = s1[i - 1].toLowerCase() === s2[j - 1].toLowerCase() ? 0 : 1; + dp[i][j] = Math.min( + dp[i - 1][j] + 1, // deletion + dp[i][j - 1] + 1, // insertion + dp[i - 1][j - 1] + cost // substitution + ); + } + } + return dp[m][n]; +} + +/** + * Extracts a flattened, unique list of vocabulary words for a given negotiation mode. + * This includes both topic tags and simple structural pattern words. + */ +function getModeVocabulary(mode: NegotiationMode): string[] { + const modeMatrix = MODE_INTENT_MATRIX[mode] || {}; + const intentKeys = Object.keys(modeMatrix) as Array; + + const vocabSet = new Set(); + + intentKeys.forEach((intent) => { + // Only process intents that are actually weighted for this mode > 0 + if ((modeMatrix as any)[intent] > 0) { + const tactic = PATTERN_LIBRARY[intent]; + + // Add topic tags + if (tactic.topicTags) { + tactic.topicTags.forEach((tag: string) => vocabSet.add(tag.toLowerCase())); + } + + // Attempt to extract raw words from simple structural regexes + if (tactic.structuralPatterns) { + tactic.structuralPatterns.forEach((pattern: string) => { + // Remove regex artifacts like .* or ^ or \b + const words = pattern.replace(/[.*^$\\?|()[\]{}]/g, ' ') + .split(/\s+/) + .filter((w: string) => w.length > 3); // Only preserve meaningful words > 3 chars + words.forEach((w: string) => vocabSet.add(w.toLowerCase())); + }); + } + } + }); + + return Array.from(vocabSet); +} + +// Cache vocabulary per mode to prevent recalculating on every transcript chunk +const vocabCache: Record = {}; + +function getCachedVocabulary(mode: NegotiationMode): string[] { + if (!vocabCache[mode]) { + vocabCache[mode] = getModeVocabulary(mode); + } + return vocabCache[mode]; +} + +/** + * Auto-corrects a Whisper transcription chunk by fuzzy matching misheard words + * against the expected negotiation mode vocabulary. + */ +export function autoCorrectTranscript(transcript: string, mode: NegotiationMode): string { + if (!transcript) return transcript; + + const vocab = getCachedVocabulary(mode); + if (vocab.length === 0) return transcript; + + // Split into words, preserving punctuation if possible by matching words + // e.g. "Wait, what?" -> ["Wait", ",", " ", "what", "?"] + // We'll just tokenize by word boundaries and check the alphabetic chunks + const tokens = transcript.split(/(\b[a-zA-Z]+\b)/); + + const correctedTokens = tokens.map(token => { + // Only fuzz alpha words that are reasonably long + if (/^[a-zA-Z]+$/.test(token) && token.length >= 4) { + let bestMatch = token; + let minDistance = Infinity; + + for (const vWord of vocab) { + // Skip comparing if lengths are vastly different + if (Math.abs(vWord.length - token.length) > 2) continue; + + const dist = levenshteinDistance(token, vWord); + + // Threshold logic: + // Length 4-5: allow 1 error + // Length 6+: allow 2 errors + const maxErrors = token.length >= 6 ? 2 : 1; + + if (dist <= maxErrors && dist < minDistance) { + minDistance = dist; + // Match the original casing if possible. Simple heuristic: match first letter casing + const isCapitalized = token[0] === token[0].toUpperCase(); + bestMatch = isCapitalized + ? vWord.charAt(0).toUpperCase() + vWord.slice(1) + : vWord; + } + } + return bestMatch; + } + return token; + }); + + return correctedTokens.join(''); +} diff --git a/src/ai/intentClassifier.ts b/src/ai/intentClassifier.ts new file mode 100644 index 0000000..66103bc --- /dev/null +++ b/src/ai/intentClassifier.ts @@ -0,0 +1,212 @@ +/** + * 🔒 PRIVACY NOTICE + * All pattern classification runs locally on device using rule-based NLP. + * No external API calls. No data leaves this device. + */ + +import { + NegotiationPattern, + NegotiationMode, + DetectedPattern, + PatternSeverity, +} from '../types/session'; +import { PATTERN_LIBRARY, PatternDefinition, getModeConfig, MODE_INTENT_MATRIX } from './patternLibrary'; + +/** + * Classify text and detect negotiation patterns using Structural Intelligence Rules + */ +export const classifyIntent = ( + text: string, + mode: NegotiationMode, + sensitivityMultiplier: number = 1.0 +): DetectedPattern[] => { + console.log('[IntentClassifier] 🔍 classifyIntent() called'); + console.log('[IntentClassifier] 📝 Text:', text); + console.log('[IntentClassifier] đŸŽ¯ Mode:', mode); + + const lowerText = text.toLowerCase(); + const timestamp = Date.now(); + const candidates: { patternDef: PatternDefinition; score: number }[] = []; + + // Check if text contains numbers + const hasNumbers = /\d+/.test(lowerText) || /\b(one|two|three|four|five|six|seven|eight|nine|ten|hundred|thousand|million|k|m)\b/.test(lowerText); + + // 1. Evaluate every pattern + for (const pattern of Object.values(NegotiationPattern)) { + const patternDef = PATTERN_LIBRARY[pattern]; + if (!patternDef) continue; + + // Structural Match Score + let structuralScore = 0; + for (const structPattern of patternDef.structuralPatterns) { + const regex = new RegExp(structPattern, 'i'); + if (regex.test(lowerText)) { + structuralScore = 1.0; + break; // Only need one structural match + } + } + + // Topic Match Score + let topicScore = 0; + for (const tag of patternDef.topicTags) { + const regex = new RegExp(`\\b${tag}\\b`, 'i'); + if (regex.test(lowerText)) { + topicScore += 0.5; // Need at least 2 distinct topic words to max out, or 1 is 50% + } + } + topicScore = Math.min(topicScore, 1.0); + + if (structuralScore === 0 && topicScore === 0) continue; // Skip if completely irrelevant + + // Numeric Boost Score + let numericScore = 0; + if (patternDef.requiresNumber) { + if (hasNumbers) { + numericScore = 1.0; + } else { + // Punish severely if required number is missing + continue; + } + } + + // Negative Signal Penalty + let negativePenalty = 0; + for (const neg of patternDef.negativeSignals) { + if (lowerText.includes(neg.toLowerCase())) { + negativePenalty = 1.0; + break; + } + } + + // Dynamic Base Weighting: + // If structure matches exactly, they get full baseWeight (e.g. 0.70) + // If only isolated topic words match, they get partial base weight (e.g. 0.49) + let primaryMatchScore = structuralScore > 0 ? patternDef.baseWeight : (patternDef.baseWeight * 0.7); + + // Combine Base Scores + let rawScore = primaryMatchScore + + (0.25 * topicScore) + + (0.10 * numericScore) + - (0.25 * negativePenalty); + + // Apply Mode Intelligence Filter Matrix + let modeAdjustment = 0; + const modeMatrix = MODE_INTENT_MATRIX[mode]; + if (modeMatrix) { + if (modeMatrix.highWeight.includes(patternDef.intent)) { + modeAdjustment += 0.15; + } else if (modeMatrix.lowWeight.includes(patternDef.intent)) { + modeAdjustment -= 0.15; + } + } + + const finalScore = (rawScore + modeAdjustment) * sensitivityMultiplier; + + // Convert to percentage + const finalConfidenceScore = Math.min(Math.max(finalScore * 100, 0), 100); + + candidates.push({ patternDef, score: finalConfidenceScore }); + + // Format logs for Demo display + console.log(`[IntentClassifier] ⚡ Evaluated: ${patternDef.intent}`); + console.log(`[IntentClassifier] ├─ Structural Score: ${structuralScore}`); + console.log(`[IntentClassifier] ├─ Topic Score: ${topicScore}`); + console.log(`[IntentClassifier] ├─ Numeric Score: ${numericScore} (Required: ${patternDef.requiresNumber})`); + console.log(`[IntentClassifier] ├─ Negative Penalty: ${negativePenalty}`); + console.log(`[IntentClassifier] ├─ Mode Adjustment: ${modeAdjustment > 0 ? '+' : ''}${Math.round(modeAdjustment * 100)}% (${mode})`); + console.log(`[IntentClassifier] └─ FINAL CONFIDENCE: ${Math.round(finalConfidenceScore)}%`); + } + + // 2. Highest Confidence Wins. Only push the Absolute Winner > 60% + if (candidates.length === 0) return []; + + candidates.sort((a, b) => b.score - a.score); + const winner = candidates[0]; + + console.log('[IntentClassifier] 🏆 Winning Tactic:', winner.patternDef.intent, 'at', Math.round(winner.score) + '%'); + + if (winner.score >= 60) { + const suggestion = pickRandomSuggestion(winner.patternDef.suggestions); + + return [{ + id: `${winner.patternDef.intent}_${timestamp}`, + pattern: winner.patternDef.intent, + confidenceScore: winner.score, + suggestion, + severity: winner.patternDef.severity, + timestamp, + transcript: text, + context: text.substring(0, 50) + "...", + }]; + } + + return []; +}; + + +/** + * Pick a random suggestion from array + */ +const pickRandomSuggestion = (suggestions: string[]): string => { + if (!suggestions || suggestions.length === 0) return "Acknowledge and pivot."; + return suggestions[Math.floor(Math.random() * suggestions.length)]; +}; + +/** + * Filter patterns by severity + */ +export const filterBySeverity = ( + patterns: DetectedPattern[], + minSeverity: PatternSeverity +): DetectedPattern[] => { + const severityOrder: Record = { + low: 1, + medium: 2, + high: 3, + }; + + const minLevel = severityOrder[minSeverity]; + + return patterns.filter((p) => severityOrder[p.severity] >= minLevel); +}; + +/** + * Get most common pattern from array + */ +export const getMostCommonPattern = (patterns: DetectedPattern[]): NegotiationPattern | null => { + if (patterns.length === 0) return null; + + const counts: Record = {}; + + patterns.forEach((p) => { + counts[p.pattern] = (counts[p.pattern] || 0) + 1; + }); + + let maxCount = 0; + let mostCommon: NegotiationPattern | null = null; + + Object.entries(counts).forEach(([pattern, count]) => { + if (count > maxCount) { + maxCount = count; + mostCommon = pattern as NegotiationPattern; + } + }); + + return mostCommon; +}; + +// Legacy stubs missing from PatternLibrary refactor requirement +export const containsPattern = () => false; +export const getSuggestionForPattern = () => ""; +export const analyzeTranscriptWindow = ( + chunks: Array<{ text: string; timestamp: number }>, + mode: NegotiationMode, + sensitivityMultiplier: number = 1.0, + windowSize: number = 2 // Update to Last 2 chunks as requested +): DetectedPattern[] => { + // Take last 2 chunks + const recentChunks = chunks.slice(-windowSize); + const combinedText = recentChunks.map((c) => c.text).join(' '); + + return classifyIntent(combinedText, mode, sensitivityMultiplier); +}; diff --git a/src/ai/patternLibrary.ts b/src/ai/patternLibrary.ts new file mode 100644 index 0000000..fa71319 --- /dev/null +++ b/src/ai/patternLibrary.ts @@ -0,0 +1,389 @@ +/** + * 🔒 PRIVACY NOTICE + * All pattern detection runs locally on device. + * No data leaves this device. + */ + +import { NegotiationPattern, NegotiationMode, ModeConfig } from '../types/session'; + +/** + * Pattern definition with structural detection rules + */ +export interface PatternDefinition { + intent: NegotiationPattern; + displayName: string; + description: string; + structuralPatterns: string[]; // Regex strings + topicTags: string[]; // Context keywords + requiresNumber: boolean; // Does it require numeric value like $50k or 60? + negativeSignals: string[]; // Penalize score if present (e.g. "maybe") + baseWeight: number; // Base confidence 0.0 - 1.0 + severity: 'low' | 'medium' | 'high'; + suggestions: string[]; +} + +/** + * Complete pattern library for negotiation detection using Structural Logic + */ +export const PATTERN_LIBRARY: Record = { + [NegotiationPattern.ANCHORING]: { + intent: NegotiationPattern.ANCHORING, + displayName: 'Anchoring', + description: 'Setting initial price/expectation reference point', + structuralPatterns: [ + 'we .* offer', + 'the .* range', + 'typically .* is', + 'industry standard', + 'expecting .* around', + 'base .* is', + 'looking .* for', + 'budget .* is', + ], + topicTags: ['salary', 'compensation', 'package', 'budget', 'price', 'cost', 'pay', 'base', 'range'], + requiresNumber: true, + negativeSignals: ['maybe', 'if possible', 'eventually'], + baseWeight: 0.65, + severity: 'high', + suggestions: [ + 'Ask for a detailed cost breakdown', + 'Present your own alternative anchor point', + 'Request comparison data from multiple sources', + ], + }, + + [NegotiationPattern.BUDGET_OBJECTION]: { + intent: NegotiationPattern.BUDGET_OBJECTION, + displayName: 'Budget Objection', + description: 'Claiming budget constraints or financial limitations', + structuralPatterns: [ + 'too expensive', + 'outside .* budget', + 'cost.* high', + 'we cannot afford', + 'beyond .* means', + 'budget is', + 'pricy', + 'not in .* budget', + "don't have .* budget", + 'too much', + 'pricey', + ], + topicTags: ['money', 'budget', 'cost', 'expensive', 'price', 'afford', 'funds', 'capital'], + requiresNumber: false, + negativeSignals: ['flexible', 'might work', 'open to'], + baseWeight: 0.70, + severity: 'high', + suggestions: [ + 'Ask about budget flexibility and approval thresholds', + 'Break pricing into smaller phases or milestones', + 'Highlight ROI and value instead of focusing on cost', + ], + }, + + [NegotiationPattern.AUTHORITY_PRESSURE]: { + intent: NegotiationPattern.AUTHORITY_PRESSURE, + displayName: 'Authority Pressure', + description: 'Deferring to higher authority or claiming lack of decision power', + structuralPatterns: [ + 'as per policy', + 'management decided', + 'company standard', + 'HR guideline', + 'need .* approval', + 'check with .* boss', + 'run it by', + 'up to my manager', + "don't have .* authority", + 'out of my hands', + ], + topicTags: ['manager', 'boss', 'approval', 'policy', 'guideline', 'hr', 'director', 'vp', 'board'], + requiresNumber: false, + negativeSignals: ['i decide', 'my call', 'we can do'], + baseWeight: 0.75, + severity: 'medium', + suggestions: [ + 'Ask for the specific decision criteria being used', + 'Suggest a joint review with all stakeholders', + 'Delay final commitment until decision-maker is present', + ], + }, + + [NegotiationPattern.TIME_PRESSURE]: { + intent: NegotiationPattern.TIME_PRESSURE, + displayName: 'Time Pressure', + description: 'Creating urgency or imposing deadlines', + structuralPatterns: [ + 'we need this today', + 'deadline is', + 'urgent decision', + 'limited time', + 'offer expires', + 'need this by', + 'as soon as possible', + 'right away', + 'move fast', + 'quickly', + 'by the end of', + ], + topicTags: ['today', 'tomorrow', 'urgent', 'deadline', 'asap', 'quickly', 'rush', 'speed', 'now'], + requiresNumber: false, + negativeSignals: ['no rush', 'take your time', 'whenever'], + baseWeight: 0.65, + severity: 'medium', + suggestions: [ + 'Ask if the deadline is truly final or flexible', + 'Introduce a new variable to reset the timeline', + 'Propose a pause — revisit with fresh perspective', + ], + }, + + [NegotiationPattern.DEFLECTION]: { + intent: NegotiationPattern.DEFLECTION, + displayName: 'Deflection', + description: 'Avoiding commitment or postponing decision via topic shift', + structuralPatterns: [ + "let's focus on", + "not the main issue", + "get back to that", + "right now .* discussing", + "circle back", + "talk about .* later", + "park that", + "take that offline", + "moving on", + ], + topicTags: ['later', 'another time', 'focus', 'discussing', 'issue', 'offline', 'park'], + requiresNumber: false, + negativeSignals: ['i agree', 'let us decide now', 'perfect'], + baseWeight: 0.60, + severity: 'medium', + suggestions: [ + 'Pin down specific concerns driving the hesitation', + 'Set a concrete follow-up date and time right now', + 'Ask what information would help them decide today', + ], + }, + + [NegotiationPattern.STRENGTH_SIGNAL]: { + intent: NegotiationPattern.STRENGTH_SIGNAL, + displayName: 'Strength Signal', + description: 'Showcasing positive applicant background or achievements', + structuralPatterns: [ + 'led a team', + 'achieved', + 'improved .* by', + 'increased revenue', + 'managed .* projects', + 'successfully delivered', + 'spearheaded', + 'was responsible for', + 'top performer', + 'exceeded goals', + ], + topicTags: ['leadership', 'revenue', 'successful', 'achieved', 'driven', 'managed', 'delivered', 'exceeded', 'impact'], + requiresNumber: false, + negativeSignals: ['assisted', 'helped', 'was part of'], + baseWeight: 0.70, + severity: 'low', + suggestions: [ + 'Leverage this momentum to position yourself firmly', + 'Directly tie this achievement to their current needs', + 'Use this high-value moment to pivot to compensation framing', + ], + }, + + [NegotiationPattern.POSITIVE_SIGNAL]: { + intent: NegotiationPattern.POSITIVE_SIGNAL, + displayName: 'Positive Signal', + description: 'Expressions of interest, agreement, or enthusiasm', + structuralPatterns: [ + 'sounds good', + 'makes sense', + 'that works', + 'i like that', + 'great idea', + 'we agree', + 'we can do that', + 'looks perfect', + 'exactly right', + ], + topicTags: ['yes', 'agree', 'perfect', 'exactly', 'love', 'great', 'awesome', 'good'], + requiresNumber: false, + negativeSignals: ['however', 'but'], + baseWeight: 0.80, + severity: 'low', + suggestions: [ + 'Capitalize on momentum — move toward commitment', + 'Summarize agreed points and lock them in writing', + 'Ask about next steps while enthusiasm is high', + ], + }, + + [NegotiationPattern.NEGATIVE_SIGNAL]: { + intent: NegotiationPattern.NEGATIVE_SIGNAL, + displayName: 'Negative Signal', + description: 'Expressions of concern, disagreement, or rejection', + structuralPatterns: [ + 'not sure', + 'worried about', + "don't think", + 'is a problem', + 'have concerns', + 'cannot work', + 'does not work', + 'impossible', + 'dealbreaker', + ], + topicTags: ['issue', 'problem', 'concerned', 'skeptical', 'worry', 'no', 'cannot', 'dealbreaker'], + requiresNumber: false, + negativeSignals: ['maybe', 'could work'], + baseWeight: 0.75, + severity: 'high', + suggestions: [ + 'Probe for the specific concern behind the negativity', + 'Acknowledge their worry and provide concrete evidence', + 'Offer an alternative approach that addresses the issue', + ], + }, + + [NegotiationPattern.COMMITMENT_LANGUAGE]: { + intent: NegotiationPattern.COMMITMENT_LANGUAGE, + displayName: 'Commitment Language', + description: 'Strong commitment or decision-making language', + structuralPatterns: [ + "let's do it", + 'we are in', + 'ready to start', + 'sign the', + 'move forward', + 'go ahead', + "let's proceed", + ], + topicTags: ['deal', 'agreed', 'commit', 'proceed', 'sign', 'forward', 'start'], + requiresNumber: false, + negativeSignals: ['maybe later', 'soon'], + baseWeight: 0.85, + severity: 'low', + suggestions: [ + 'Document the agreement immediately in writing', + 'Clarify all remaining terms before finalizing', + 'Confirm timeline and deliverables for next steps', + ], + } +}; + +/** + * Filter Weights Arrays + */ +export interface ModeMatrix { + highWeight: NegotiationPattern[]; + mediumWeight: NegotiationPattern[]; + lowWeight: NegotiationPattern[]; +} + +/** + * Global Mode Intelligence Filter Matrix + * Controls penalty / boost multipliers internally for IntentClassifier Context Check + */ +export const MODE_INTENT_MATRIX: Record = { + [NegotiationMode.SALARY_RAISE]: { + highWeight: [NegotiationPattern.ANCHORING, NegotiationPattern.BUDGET_OBJECTION], + mediumWeight: [NegotiationPattern.AUTHORITY_PRESSURE, NegotiationPattern.COMMITMENT_LANGUAGE], + lowWeight: [NegotiationPattern.STRENGTH_SIGNAL, NegotiationPattern.DEFLECTION], + }, + [NegotiationMode.JOB_INTERVIEW]: { + highWeight: [NegotiationPattern.STRENGTH_SIGNAL, NegotiationPattern.DEFLECTION], + mediumWeight: [NegotiationPattern.POSITIVE_SIGNAL, NegotiationPattern.TIME_PRESSURE], + lowWeight: [NegotiationPattern.ANCHORING, NegotiationPattern.BUDGET_OBJECTION], + }, + [NegotiationMode.STARTUP_PITCH]: { + highWeight: [NegotiationPattern.BUDGET_OBJECTION, NegotiationPattern.DEFLECTION], + mediumWeight: [NegotiationPattern.TIME_PRESSURE, NegotiationPattern.POSITIVE_SIGNAL], + lowWeight: [NegotiationPattern.STRENGTH_SIGNAL], + }, + [NegotiationMode.SALES]: { + highWeight: [NegotiationPattern.BUDGET_OBJECTION, NegotiationPattern.TIME_PRESSURE, NegotiationPattern.COMMITMENT_LANGUAGE], + mediumWeight: [NegotiationPattern.AUTHORITY_PRESSURE, NegotiationPattern.DEFLECTION], + lowWeight: [NegotiationPattern.STRENGTH_SIGNAL], + }, + [NegotiationMode.INVESTOR_MEETING]: { + highWeight: [NegotiationPattern.DEFLECTION, NegotiationPattern.BUDGET_OBJECTION, NegotiationPattern.NEGATIVE_SIGNAL], + mediumWeight: [NegotiationPattern.TIME_PRESSURE, NegotiationPattern.AUTHORITY_PRESSURE], + lowWeight: [NegotiationPattern.STRENGTH_SIGNAL], + }, + [NegotiationMode.CLIENT_NEGOTIATION]: { + highWeight: [NegotiationPattern.ANCHORING, NegotiationPattern.BUDGET_OBJECTION, NegotiationPattern.TIME_PRESSURE], + mediumWeight: [NegotiationPattern.AUTHORITY_PRESSURE, NegotiationPattern.DEFLECTION], + lowWeight: [NegotiationPattern.STRENGTH_SIGNAL], + }, + [NegotiationMode.CUSTOM_SCENARIO]: { + highWeight: [NegotiationPattern.ANCHORING, NegotiationPattern.DEFLECTION, NegotiationPattern.NEGATIVE_SIGNAL], + mediumWeight: [NegotiationPattern.TIME_PRESSURE, NegotiationPattern.BUDGET_OBJECTION], + lowWeight: [NegotiationPattern.STRENGTH_SIGNAL], + }, +}; + +/** + * Legacy interface for Type Compiling (to prevent UI from breaking). + * Used generically to fetch icons and names but not used in the Universal NLP pipeline itself. + */ +export const MODE_CONFIGS: Record = { + [NegotiationMode.JOB_INTERVIEW]: { + mode: NegotiationMode.JOB_INTERVIEW, + displayName: 'Job Interview', + description: 'Optimize for employment negotiations and interview scenarios', + icon: 'đŸ’ŧ', + patternWeights: {} as any, + }, + [NegotiationMode.SALES]: { + mode: NegotiationMode.SALES, + displayName: 'Sales', + description: 'Optimize for sales conversations and client negotiations', + icon: '💰', + patternWeights: {} as any, + }, + [NegotiationMode.STARTUP_PITCH]: { + mode: NegotiationMode.STARTUP_PITCH, + displayName: 'Startup Pitch', + description: 'Optimize for investor pitches and funding negotiations', + icon: '🚀', + patternWeights: {} as any, + }, + [NegotiationMode.SALARY_RAISE]: { + mode: NegotiationMode.SALARY_RAISE, + displayName: 'Salary Raise', + description: 'Optimize for salary negotiation and raise discussions', + icon: '📈', + patternWeights: {} as any, + }, + [NegotiationMode.INVESTOR_MEETING]: { + mode: NegotiationMode.INVESTOR_MEETING, + displayName: 'Investor Meeting', + description: 'Optimize for venture capital and angel funding rounds', + icon: 'đŸĻ', + patternWeights: {} as any, + }, + [NegotiationMode.CLIENT_NEGOTIATION]: { + mode: NegotiationMode.CLIENT_NEGOTIATION, + displayName: 'Client Negotiation', + description: 'Optimize for contract, scoping, and B2B client delivery discussions', + icon: '🤝', + patternWeights: {} as any, + }, + [NegotiationMode.CUSTOM_SCENARIO]: { + mode: NegotiationMode.CUSTOM_SCENARIO, + displayName: 'Custom Scenario', + description: 'Optimize for unmapped, highly specific personal negotiations', + icon: 'đŸŽ¯', + patternWeights: {} as any, + }, +}; + +export const getAllPatterns = (): PatternDefinition[] => Object.values(PATTERN_LIBRARY); +export const getPatternDefinition = (pattern: NegotiationPattern): PatternDefinition => PATTERN_LIBRARY[pattern]; +export const getModeConfig = (mode: NegotiationMode): ModeConfig => MODE_CONFIGS[mode]; +export const getAllModes = (): ModeConfig[] => Object.values(MODE_CONFIGS); + +export const FILLER_WORDS = ['um', 'uh', 'like', 'you know', 'basically', 'actually', 'literally', 'just', 'right', 'okay', 'so', 'well', 'anyway']; +export const FILLER_WORD_PATTERN = new RegExp(`\\b(${FILLER_WORDS.join('|')})\\b`, 'gi'); diff --git a/src/ai/scoringEngine.ts b/src/ai/scoringEngine.ts new file mode 100644 index 0000000..bf9157a --- /dev/null +++ b/src/ai/scoringEngine.ts @@ -0,0 +1,205 @@ +/** + * 🔒 PRIVACY NOTICE + * All scoring calculations run locally on device. + * No data leaves this device. + */ + +import { CognitiveMetrics } from '../types/session'; +import { FILLER_WORD_PATTERN } from './patternLibrary'; + +/** + * Calculate confidence score based on multiple factors + */ +export const calculateConfidenceScore = ( + baseConfidence: number, + factors: { + keywordMatches: number; + contextMatches: number; + patternWeight: number; + sensitivityMultiplier: number; + } +): number => { + // Base confidence from pattern definition + let score = baseConfidence; + + // Boost confidence based on keyword matches (up to +20) + const keywordBoost = Math.min(factors.keywordMatches * 5, 20); + score += keywordBoost; + + // Boost confidence based on context clues (up to +10) + const contextBoost = Math.min(factors.contextMatches * 3, 10); + score += contextBoost; + + // Apply mode-specific pattern weight + score *= factors.patternWeight; + + // Apply user sensitivity setting (0.5 - 1.5 range) + score *= factors.sensitivityMultiplier; + + // Clamp to 0-100 range + return Math.max(0, Math.min(100, Math.round(score))); +}; + +/** + * Calculate focus score from cognitive metrics + * Formula: 100 - (gapPenalty * 0.3 + fillerPenalty * 0.4 + ratePenalty * 0.3) + */ +export const calculateFocusScore = ( + metrics: Partial, + sessionDuration: number // in milliseconds +): number => { + const durationMinutes = sessionDuration / 60000; + + // Avoid division by zero + if (durationMinutes < 0.1) { + return 100; + } + + // Calculate speech gaps penalty (0-40 points) + // More than 5 gaps per minute is concerning + const speechGaps = metrics.speechGaps || 0; + const gapsPerMinute = speechGaps / durationMinutes; + const gapPenalty = Math.min((gapsPerMinute / 5) * 40, 40); + + // Calculate filler words penalty (0-50 points) + // More than 10 filler words per minute is concerning + const fillerWords = metrics.fillerWords || 0; + const fillersPerMinute = fillerWords / durationMinutes; + const fillerPenalty = Math.min((fillersPerMinute / 10) * 50, 50); + + // Calculate speech rate penalty (0-30 points) + // Normal speech: 120-150 WPM, too slow (<80) or too fast (>200) is concerning + const avgSpeechRate = metrics.avgSpeechRate || 130; + let ratePenalty = 0; + if (avgSpeechRate < 80) { + ratePenalty = ((80 - avgSpeechRate) / 80) * 30; + } else if (avgSpeechRate > 200) { + ratePenalty = ((avgSpeechRate - 200) / 100) * 30; + } + ratePenalty = Math.min(ratePenalty, 30); + + // Calculate final focus score + const focusScore = 100 - (gapPenalty * 0.3 + fillerPenalty * 0.4 + ratePenalty * 0.3); + + return Math.max(0, Math.min(100, Math.round(focusScore))); +}; + +/** + * Analyze text for filler words + */ +export const countFillerWords = (text: string): number => { + const matches = text.match(FILLER_WORD_PATTERN); + return matches ? matches.length : 0; +}; + +/** + * Detect speech gaps in transcript chunks + * Returns count of gaps > 2 seconds + */ +export const detectSpeechGaps = ( + chunks: Array<{ timestamp: number; text: string }>, + gapThreshold: number = 2000 // 2 seconds in milliseconds +): number => { + let gapCount = 0; + + for (let i = 1; i < chunks.length; i++) { + const timeDiff = chunks[i].timestamp - chunks[i - 1].timestamp; + // If gap is larger than threshold and previous chunk had text + if (timeDiff > gapThreshold && chunks[i - 1].text.trim().length > 0) { + gapCount++; + } + } + + return gapCount; +}; + +/** + * Calculate words per minute from transcript + */ +export const calculateSpeechRate = ( + totalWords: number, + speechDuration: number // in milliseconds +): number => { + const durationMinutes = speechDuration / 60000; + if (durationMinutes === 0) return 0; + return Math.round(totalWords / durationMinutes); +}; + +/** + * Count total words in text + */ +export const countWords = (text: string): number => { + return text + .trim() + .split(/\s+/) + .filter((word) => word.length > 0).length; +}; + +/** + * Calculate cognitive metrics from transcript chunks + */ +export const calculateCognitiveMetrics = ( + chunks: Array<{ timestamp: number; text: string }>, + sessionDuration: number +): CognitiveMetrics => { + // Combine all text + const fullText = chunks.map((c) => c.text).join(' '); + + // Count metrics + const totalWords = countWords(fullText); + const fillerWords = countFillerWords(fullText); + const speechGaps = detectSpeechGaps(chunks); + + // Calculate speech duration (total session time minus gaps) + let speechDuration = sessionDuration; + for (let i = 1; i < chunks.length; i++) { + const timeDiff = chunks[i].timestamp - chunks[i - 1].timestamp; + if (timeDiff > 2000) { + speechDuration -= timeDiff - 2000; // Subtract gap time beyond 2s + } + } + speechDuration = Math.max(speechDuration, 1000); // Minimum 1 second + + const avgSpeechRate = calculateSpeechRate(totalWords, speechDuration); + const focusScore = calculateFocusScore( + { speechGaps, fillerWords, avgSpeechRate, totalWords, speechDuration }, + sessionDuration + ); + + return { + focusScore, + speechGaps, + fillerWords, + avgSpeechRate, + totalWords, + speechDuration, + }; +}; + +/** + * Determine pattern severity based on confidence score + */ +export const determineSeverity = (confidenceScore: number): 'low' | 'medium' | 'high' => { + if (confidenceScore >= 80) return 'high'; + if (confidenceScore >= 60) return 'medium'; + return 'low'; +}; + +/** + * Calculate leverage indicator (Low/Medium/High) + * Based on ratio of positive to negative signals + */ +export const calculateLeverageLevel = ( + positiveSignals: number, + negativeSignals: number, + totalPatterns: number +): 'low' | 'medium' | 'high' => { + if (totalPatterns === 0) return 'medium'; + + const positiveRatio = positiveSignals / totalPatterns; + const negativeRatio = negativeSignals / totalPatterns; + + if (positiveRatio > negativeRatio * 2) return 'high'; + if (negativeRatio > positiveRatio * 2) return 'low'; + return 'medium'; +}; diff --git a/src/components/AudioVisualizer.tsx b/src/components/AudioVisualizer.tsx deleted file mode 100644 index ea78d8b..0000000 --- a/src/components/AudioVisualizer.tsx +++ /dev/null @@ -1,52 +0,0 @@ -import React from 'react'; -import { View, StyleSheet } from 'react-native'; -import { AppColors } from '../theme'; - -interface AudioVisualizerProps { - level: number; // 0.0 to 1.0 - barCount?: number; -} - -export const AudioVisualizer: React.FC = ({ - level, - barCount = 7, -}) => { - const bars = Array.from({ length: barCount }, (_, i) => { - // Create wave effect - const waveEffect = Math.sin((i / barCount) * Math.PI); - const height = Math.max(0.2, level * waveEffect); - return height; - }); - - return ( - - {bars.map((height, index) => ( - - ))} - - ); -}; - -const styles = StyleSheet.create({ - container: { - flexDirection: 'row', - alignItems: 'center', - justifyContent: 'center', - height: 60, - gap: 6, - }, - bar: { - width: 6, - backgroundColor: AppColors.accentViolet, - borderRadius: 3, - minHeight: 12, - }, -}); diff --git a/src/components/ChatMessageBubble.tsx b/src/components/ChatMessageBubble.tsx deleted file mode 100644 index 63780a6..0000000 --- a/src/components/ChatMessageBubble.tsx +++ /dev/null @@ -1,136 +0,0 @@ -import React from 'react'; -import { View, Text, StyleSheet } from 'react-native'; -import { AppColors } from '../theme'; - -export interface ChatMessage { - text: string; - isUser: boolean; - timestamp: Date; - tokensPerSecond?: number; - totalTokens?: number; - isError?: boolean; - wasCancelled?: boolean; -} - -interface ChatMessageBubbleProps { - message: ChatMessage; - isStreaming?: boolean; -} - -export const ChatMessageBubble: React.FC = ({ - message, - isStreaming = false, -}) => { - const { text, isUser, tokensPerSecond, totalTokens, isError, wasCancelled } = message; - - return ( - - - - {text} - - - {!isUser && !isStreaming && (tokensPerSecond || totalTokens) && ( - - {tokensPerSecond && ( - - ⚡ {tokensPerSecond.toFixed(1)} tok/s - - )} - {totalTokens && ( - 📊 {totalTokens} tokens - )} - - )} - - {wasCancelled && ( - ⚠ī¸ Generation cancelled - )} - - {isStreaming && ▊} - - - ); -}; - -const styles = StyleSheet.create({ - container: { - marginVertical: 4, - paddingHorizontal: 16, - }, - userContainer: { - alignItems: 'flex-end', - }, - assistantContainer: { - alignItems: 'flex-start', - }, - bubble: { - maxWidth: '85%', - padding: 12, - borderRadius: 16, - marginVertical: 2, - }, - userBubble: { - backgroundColor: AppColors.accentCyan, - borderBottomRightRadius: 4, - }, - assistantBubble: { - backgroundColor: AppColors.surfaceCard, - borderBottomLeftRadius: 4, - borderWidth: 1, - borderColor: AppColors.textMuted + '20', - }, - errorBubble: { - backgroundColor: AppColors.error + '20', - borderColor: AppColors.error + '40', - }, - text: { - fontSize: 15, - lineHeight: 21, - }, - userText: { - color: '#FFFFFF', - }, - assistantText: { - color: AppColors.textPrimary, - }, - errorText: { - color: AppColors.error, - }, - metricsContainer: { - flexDirection: 'row', - marginTop: 8, - gap: 12, - }, - metrics: { - fontSize: 11, - color: AppColors.textMuted, - }, - cancelledText: { - fontSize: 11, - color: AppColors.warning, - marginTop: 4, - }, - streamingIndicator: { - fontSize: 16, - color: AppColors.accentCyan, - marginTop: 2, - }, -}); diff --git a/src/components/CircularScore.tsx b/src/components/CircularScore.tsx new file mode 100644 index 0000000..8685e4a --- /dev/null +++ b/src/components/CircularScore.tsx @@ -0,0 +1,106 @@ +import React from 'react'; +import { View, Text, StyleSheet } from 'react-native'; +import Svg, { Circle } from 'react-native-svg'; + +interface CircularScoreProps { + /** Score value 0–100 */ + score: number; + /** Outer diameter in dp (default 120) */ + size?: number; + /** Stroke width of the donut ring (default 10) */ + strokeWidth?: number; + /** Color of the progress arc */ + progressColor?: string; + /** Color of the background track */ + trackColor?: string; +} + +const CircularScore: React.FC = ({ + score, + size = 120, + strokeWidth = 10, + progressColor = '#FFA726', + trackColor = 'rgba(255,255,255,0.25)', +}) => { + const radius = (size - strokeWidth) / 2; + const circumference = 2 * Math.PI * radius; + const clampedScore = Math.min(100, Math.max(0, score)); + const strokeDashoffset = circumference * (1 - clampedScore / 100); + + // Responsive font sizing based on the donut size + const fontSize = Math.round(size * 0.28); + const percentFontSize = Math.round(fontSize * 0.55); + const lineHeight = Math.round(fontSize * 1.1); + + return ( + + {/* SVG donut ring */} + + {/* Background track */} + + {/* Progress arc */} + + + + {/* Overlay – perfectly centred text */} + + + {clampedScore} + + % + + + + + ); +}; + +const styles = StyleSheet.create({ + wrapper: { + position: 'relative', + }, + overlay: { + position: 'absolute', + top: 0, + left: 0, + right: 0, + bottom: 0, + alignItems: 'center', + justifyContent: 'center', + }, + scoreText: { + fontWeight: '700', + color: '#FFFFFF', + textAlign: 'center', + // Ensure no hidden padding pushes the text off-centre + includeFontPadding: false, + textAlignVertical: 'center', + }, +}); + +export { CircularScore }; +export default CircularScore; diff --git a/src/components/CognitiveMeter.tsx b/src/components/CognitiveMeter.tsx new file mode 100644 index 0000000..7b66206 --- /dev/null +++ b/src/components/CognitiveMeter.tsx @@ -0,0 +1,128 @@ +import React, { useEffect, useRef } from 'react'; +import { View, Text, StyleSheet, Animated } from 'react-native'; +import { AppColors } from '../theme'; + +interface CognitiveMeterProps { + focusScore: number; + size?: number; + showLabel?: boolean; +} + +export const CognitiveMeter: React.FC = ({ focusScore, size = 100, showLabel = true }) => { + const animatedValue = useRef(new Animated.Value(0)).current; + + useEffect(() => { + Animated.timing(animatedValue, { toValue: focusScore, duration: 1000, useNativeDriver: false }).start(); + }, [focusScore, animatedValue]); + + const getColor = () => { + if (focusScore >= 80) return '#34C759'; + if (focusScore >= 60) return '#F5A623'; + return '#FF3B30'; + }; + + const getLabelColor = () => { + if (focusScore >= 80) return '#2DA44E'; + if (focusScore >= 60) return '#D4901A'; + return '#D32F2F'; + }; + + const getPillBg = () => { + if (focusScore >= 80) return 'rgba(52,199,89,0.15)'; + if (focusScore >= 60) return 'rgba(245,166,35,0.15)'; + return 'rgba(255,59,48,0.15)'; + }; + + const getLabel = () => { + if (focusScore >= 80) return 'High Focus'; + if (focusScore >= 60) return 'Moderate'; + return 'Low Focus'; + }; + + const scoreFontSize = Math.round(size * 0.28); + const percentFontSize = Math.round(size * 0.16); + + return ( + + {/* Circle container — fixed size for the donut */} + + {/* Background track */} + + {/* Animated progress arc */} + + {/* Center text overlay */} + + + + {Math.round(focusScore)} + + + % + + + + + + {/* Status label */} + {showLabel && ( + + {getLabel()} + + )} + + ); +}; + +const styles = StyleSheet.create({ + wrapper: { + alignItems: 'center', + justifyContent: 'center', + marginBottom: 12, + }, + circleContainer: { + position: 'relative', + }, + circle: { + position: 'absolute', + top: 0, + left: 0, + }, + overlay: { + position: 'absolute', + top: 0, + left: 0, + right: 0, + bottom: 0, + alignItems: 'center', + justifyContent: 'center', + }, + scoreRow: { + flexDirection: 'row', + alignItems: 'center', + }, + scoreText: { + fontWeight: '700', + color: '#FFFFFF', + includeFontPadding: false, + textAlignVertical: 'center', + }, + percentText: { + fontWeight: '600', + color: 'rgba(255,255,255,0.7)', + includeFontPadding: false, + textAlignVertical: 'center', + marginLeft: 1, + }, + statusText: { + marginTop: 8, + fontSize: 15, + fontWeight: '600', + letterSpacing: 0.5, + textAlign: 'center', + includeFontPadding: false, + }, +}); diff --git a/src/components/CounterStrategyCard.tsx b/src/components/CounterStrategyCard.tsx new file mode 100644 index 0000000..b6ced0e --- /dev/null +++ b/src/components/CounterStrategyCard.tsx @@ -0,0 +1,233 @@ +import React, { useEffect, useRef } from 'react'; +import { View, Text, StyleSheet, Animated } from 'react-native'; +import LinearGradient from 'react-native-linear-gradient'; +import { CounterStrategy } from '../types/session'; +import { AppColors } from '../theme'; + +interface CounterStrategyCardProps { + strategy: CounterStrategy; + onDismiss?: () => void; +} + +/** + * CounterStrategyCard - Animated card displaying counter-strategy suggestions + * when a negotiation tactic is detected with high confidence. + */ +export const CounterStrategyCard: React.FC = ({ + strategy, + onDismiss, +}) => { + const slideAnim = useRef(new Animated.Value(120)).current; + const fadeAnim = useRef(new Animated.Value(0)).current; + const scaleAnim = useRef(new Animated.Value(0.95)).current; + + useEffect(() => { + // Reset animations for new strategy + slideAnim.setValue(120); + fadeAnim.setValue(0); + scaleAnim.setValue(0.95); + + // Animate in: slide up + fade in + scale up + Animated.parallel([ + Animated.spring(slideAnim, { + toValue: 0, + tension: 60, + friction: 9, + useNativeDriver: true, + }), + Animated.timing(fadeAnim, { + toValue: 1, + duration: 350, + useNativeDriver: true, + }), + Animated.spring(scaleAnim, { + toValue: 1, + tension: 60, + friction: 8, + useNativeDriver: true, + }), + ]).start(); + }, [strategy.timestamp, slideAnim, fadeAnim, scaleAnim]); + + const getConfidenceColor = () => { + if (strategy.confidence >= 85) return AppColors.error; + if (strategy.confidence >= 70) return AppColors.warning; + return AppColors.accentCyan; + }; + + const getGradientColors = (): string[] => { + if (strategy.confidence >= 85) return ['#DC262620', '#0A0E1A']; + if (strategy.confidence >= 70) return ['#F59E0B20', '#0A0E1A']; + return ['#00D9FF20', '#0A0E1A']; + }; + + return ( + + + {/* Header: Tactic Name + Confidence Badge */} + + + + ⚠ī¸ + Tactic Detected + + + + + {Math.round(strategy.confidence)}% + + + + + {strategy.tacticDisplayName} + + + {/* Divider */} + + + {/* Counter Suggestions */} + + 🛡ī¸ Counter Actions + {strategy.suggestions.map((suggestion, index) => ( + + + {suggestion} + + ))} + + + {/* Explanation */} + + {strategy.explanation} + + + + ); +}; + +const styles = StyleSheet.create({ + container: { + marginHorizontal: 16, + marginBottom: 12, + borderRadius: 16, + overflow: 'hidden', + elevation: 10, + shadowColor: '#000', + shadowOffset: { width: 0, height: 6 }, + shadowOpacity: 0.35, + shadowRadius: 10, + borderWidth: 1, + borderColor: AppColors.accentViolet + '30', + }, + gradient: { + padding: 16, + }, + header: { + marginBottom: 12, + }, + titleRow: { + flexDirection: 'row', + justifyContent: 'space-between', + alignItems: 'center', + marginBottom: 8, + }, + tacticLabelContainer: { + flexDirection: 'row', + alignItems: 'center', + }, + warningIcon: { + fontSize: 14, + marginRight: 6, + }, + tacticLabel: { + fontSize: 12, + fontWeight: '600', + color: AppColors.textMuted, + textTransform: 'uppercase', + letterSpacing: 1, + }, + confidenceBadge: { + flexDirection: 'row', + alignItems: 'center', + paddingHorizontal: 10, + paddingVertical: 4, + borderRadius: 12, + }, + confidenceDot: { + width: 6, + height: 6, + borderRadius: 3, + marginRight: 6, + }, + confidenceText: { + fontSize: 13, + fontWeight: '700', + }, + tacticName: { + fontSize: 20, + fontWeight: '800', + color: AppColors.textPrimary, + letterSpacing: 0.3, + }, + divider: { + height: 1, + backgroundColor: AppColors.textMuted + '20', + marginBottom: 12, + }, + suggestionsContainer: { + marginBottom: 12, + }, + suggestionsLabel: { + fontSize: 13, + fontWeight: '700', + color: AppColors.accentCyan, + marginBottom: 10, + letterSpacing: 0.5, + }, + suggestionRow: { + flexDirection: 'row', + alignItems: 'flex-start', + marginBottom: 8, + paddingLeft: 4, + }, + bulletDot: { + width: 6, + height: 6, + borderRadius: 3, + backgroundColor: AppColors.accentViolet, + marginTop: 7, + marginRight: 10, + }, + suggestionText: { + fontSize: 14, + color: AppColors.textPrimary, + lineHeight: 20, + flex: 1, + }, + explanationContainer: { + backgroundColor: AppColors.surfaceCard + 'AA', + borderRadius: 10, + padding: 12, + borderLeftWidth: 3, + borderLeftColor: AppColors.accentViolet + '80', + }, + explanationText: { + fontSize: 12, + color: AppColors.textSecondary, + lineHeight: 18, + fontStyle: 'italic', + }, +}); diff --git a/src/components/FeatureCard.tsx b/src/components/FeatureCard.tsx deleted file mode 100644 index 01f7c1f..0000000 --- a/src/components/FeatureCard.tsx +++ /dev/null @@ -1,87 +0,0 @@ -import React from 'react'; -import { - TouchableOpacity, - Text, - StyleSheet, - ViewStyle, -} from 'react-native'; -import LinearGradient from 'react-native-linear-gradient'; -import { AppColors } from '../theme'; - -interface FeatureCardProps { - title: string; - subtitle: string; - icon: string; - gradientColors: string[]; - onPress: () => void; -} - -export const FeatureCard: React.FC = ({ - title, - subtitle, - gradientColors, - onPress, -}) => { - return ( - - - {getIconEmoji(title)} - {title} - {subtitle} - - - ); -}; - -// Helper to get emoji icon based on title -const getIconEmoji = (title: string): string => { - const iconMap: Record = { - Chat: 'đŸ’Ŧ', - Tools: '🛠', - Speech: '🎤', - Voice: '🔊', - Pipeline: '✨', - }; - return iconMap[title] || '⚡'; -}; - -const styles = StyleSheet.create({ - container: { - flex: 1, - margin: 8, - borderRadius: 20, - overflow: 'hidden', - elevation: 8, - shadowColor: AppColors.accentCyan, - shadowOffset: { width: 0, height: 4 }, - shadowOpacity: 0.3, - shadowRadius: 12, - } as ViewStyle, - gradient: { - padding: 20, - minHeight: 160, - justifyContent: 'center', - alignItems: 'center', - } as ViewStyle, - icon: { - fontSize: 48, - marginBottom: 12, - }, - title: { - fontSize: 20, - fontWeight: '700', - color: '#FFFFFF', - marginBottom: 4, - textAlign: 'center', - }, - subtitle: { - fontSize: 13, - color: 'rgba(255, 255, 255, 0.85)', - textAlign: 'center', - }, -}); diff --git a/src/components/LiveTranscript.tsx b/src/components/LiveTranscript.tsx new file mode 100644 index 0000000..a46a4e6 --- /dev/null +++ b/src/components/LiveTranscript.tsx @@ -0,0 +1,65 @@ +import React, { useRef, useEffect } from 'react'; +import { View, Text, StyleSheet, FlatList, Animated } from 'react-native'; +import { TranscriptChunk } from '../types/session'; +import { AppColors } from '../theme'; + +interface LiveTranscriptProps { + transcript: TranscriptChunk[]; + highlightPatterns?: boolean; +} + +export const LiveTranscript: React.FC = ({ transcript, highlightPatterns = true }) => { + const flatListRef = useRef(null); + const fadeAnim = useRef(new Animated.Value(0)).current; + + useEffect(() => { + if (transcript.length > 0) { + flatListRef.current?.scrollToEnd({ animated: true }); + Animated.timing(fadeAnim, { toValue: 1, duration: 300, useNativeDriver: true }).start(); + } + }, [transcript.length, fadeAnim]); + + const formatTime = (timestamp: number) => + new Date(timestamp).toLocaleTimeString('en-US', { hour: '2-digit', minute: '2-digit', second: '2-digit' }); + + const renderItem = ({ item, index }: { item: TranscriptChunk; index: number }) => ( + + {formatTime(item.timestamp)} + {item.text} + {item.hasPattern && highlightPatterns && ( + Pattern Detected + )} + + ); + + if (transcript.length === 0) return ( + + 🎤 + Start speaking... + Your transcript will appear here + + ); + + return ( + i.id} + style={styles.list} contentContainerStyle={styles.contentContainer} + showsVerticalScrollIndicator indicatorStyle="default" + onContentSizeChange={() => flatListRef.current?.scrollToEnd({ animated: true })} /> + ); +}; + +const styles = StyleSheet.create({ + list: { flex: 1 }, + contentContainer: { padding: 16 }, + item: { marginBottom: 12, padding: 14, backgroundColor: '#FFFFFF', borderRadius: 16, borderLeftWidth: 3, borderLeftColor: '#DDD6FE', elevation: 1, shadowColor: '#000', shadowOffset: { width: 0, height: 1 }, shadowOpacity: 0.03, shadowRadius: 3 }, + itemHighlighted: { backgroundColor: '#F5F0FF', borderLeftColor: '#7B61FF', borderLeftWidth: 4 }, + timestamp: { fontSize: 11, color: AppColors.textMuted, marginBottom: 6, fontWeight: '500' }, + text: { fontSize: 15, color: AppColors.textPrimary, lineHeight: 22 }, + patternBadge: { marginTop: 8, paddingVertical: 4, paddingHorizontal: 10, backgroundColor: '#EDE9FE', borderRadius: 8, alignSelf: 'flex-start' }, + patternBadgeText: { fontSize: 11, color: '#7B61FF', fontWeight: '600' }, + emptyContainer: { flex: 1, justifyContent: 'center', alignItems: 'center', padding: 32 }, + emptyIconCircle: { width: 80, height: 80, borderRadius: 28, backgroundColor: '#EDE9FE', justifyContent: 'center', alignItems: 'center', marginBottom: 20 }, + emptyIcon: { fontSize: 36 }, + emptyText: { fontSize: 18, fontWeight: '600', color: AppColors.textPrimary, marginBottom: 8 }, + emptySubtext: { fontSize: 14, color: AppColors.textSecondary, textAlign: 'center' }, +}); diff --git a/src/components/ModelLoaderWidget.tsx b/src/components/ModelLoaderWidget.tsx deleted file mode 100644 index dc690b3..0000000 --- a/src/components/ModelLoaderWidget.tsx +++ /dev/null @@ -1,185 +0,0 @@ -import React from 'react'; -import { - View, - Text, - TouchableOpacity, - ActivityIndicator, - StyleSheet, -} from 'react-native'; -import { AppColors } from '../theme'; - -interface ModelLoaderWidgetProps { - title: string; - subtitle: string; - icon: string; - accentColor: string; - isDownloading: boolean; - isLoading: boolean; - progress: number; - onLoad: () => void; -} - -export const ModelLoaderWidget: React.FC = ({ - title, - subtitle, - accentColor, - isDownloading, - isLoading, - progress, - onLoad, -}) => { - const getIconEmoji = () => { - if (title.includes('LLM')) return '🤖'; - if (title.includes('STT')) return '🎤'; - if (title.includes('TTS')) return '🔊'; - if (title.includes('Voice')) return '✨'; - return 'đŸ“Ļ'; - }; - - return ( - - - - {getIconEmoji()} - - - {title} - {subtitle} - - {(isDownloading || isLoading) && ( - - - - {isDownloading - ? `Downloading... ${Math.round(progress)}%` - : 'Loading model...'} - - {isDownloading && ( - - - - )} - - )} - - {!isDownloading && !isLoading && ( - - Download & Load Model - - )} - - - - 🔒 All processing happens on your device. Your data never leaves your phone. - - - - - ); -}; - -const styles = StyleSheet.create({ - container: { - flex: 1, - backgroundColor: AppColors.primaryDark, - justifyContent: 'center', - alignItems: 'center', - padding: 24, - }, - content: { - maxWidth: 400, - alignItems: 'center', - }, - iconContainer: { - width: 100, - height: 100, - borderRadius: 50, - justifyContent: 'center', - alignItems: 'center', - marginBottom: 24, - }, - iconEmoji: { - fontSize: 56, - }, - title: { - fontSize: 24, - fontWeight: '700', - color: AppColors.textPrimary, - marginBottom: 8, - textAlign: 'center', - }, - subtitle: { - fontSize: 14, - color: AppColors.textSecondary, - textAlign: 'center', - marginBottom: 32, - lineHeight: 20, - }, - loadingContainer: { - alignItems: 'center', - marginVertical: 24, - }, - loadingText: { - marginTop: 16, - fontSize: 14, - color: AppColors.textSecondary, - }, - progressBarContainer: { - width: 200, - height: 6, - backgroundColor: AppColors.surfaceCard, - borderRadius: 3, - marginTop: 12, - overflow: 'hidden', - }, - progressBar: { - height: '100%', - borderRadius: 3, - }, - button: { - paddingHorizontal: 32, - paddingVertical: 16, - borderRadius: 12, - elevation: 4, - shadowColor: '#000', - shadowOffset: { width: 0, height: 4 }, - shadowOpacity: 0.3, - shadowRadius: 8, - minWidth: 220, - alignItems: 'center', - justifyContent: 'center', - alignSelf: 'center', - }, - buttonText: { - fontSize: 16, - fontWeight: '700', - color: '#FFFFFF', - textAlign: 'center', - }, - infoBox: { - marginTop: 32, - padding: 16, - backgroundColor: AppColors.surfaceCard, - borderRadius: 12, - borderWidth: 1, - borderColor: AppColors.textMuted + '20', - }, - infoText: { - fontSize: 12, - color: AppColors.textSecondary, - textAlign: 'center', - lineHeight: 18, - }, -}); diff --git a/src/components/SessionSummaryCard.tsx b/src/components/SessionSummaryCard.tsx new file mode 100644 index 0000000..03d9bbe --- /dev/null +++ b/src/components/SessionSummaryCard.tsx @@ -0,0 +1,125 @@ +import React from 'react'; +import { View, Text, StyleSheet, TouchableOpacity } from 'react-native'; +import { Session } from '../types/session'; +import { AppColors } from '../theme'; +import { getModeConfig } from '../ai/patternLibrary'; + +interface SessionSummaryCardProps { + session: Session; + onPress: () => void; +} + +/** + * SessionSummaryCard - Transaction-style session card (like the reference image) + */ +export const SessionSummaryCard: React.FC = ({ session, onPress }) => { + const formatDate = (timestamp: number): string => { + const date = new Date(timestamp); + const now = new Date(); + const diff = now.getTime() - date.getTime(); + const days = Math.floor(diff / (1000 * 60 * 60 * 24)); + if (days === 0) { + return 'Today, ' + date.toLocaleTimeString('en-US', { hour: 'numeric', minute: '2-digit' }); + } else if (days === 1) { + return 'Yesterday, ' + date.toLocaleTimeString('en-US', { hour: 'numeric', minute: '2-digit' }); + } else { + return date.toLocaleDateString('en-US', { month: 'short', day: 'numeric' }) + ', ' + + date.toLocaleTimeString('en-US', { hour: 'numeric', minute: '2-digit' }); + } + }; + + const formatDuration = (ms: number): string => { + const minutes = Math.floor(ms / 60000); + return `${minutes}min`; + }; + + const getFocusColor = (score: number): string => { + if (score >= 80) return '#34C759'; + if (score >= 60) return '#FF9F0A'; + return '#FF3B30'; + }; + + const getIconBg = (index: number): string => { + const colors = ['#EDE9FE', '#DCFCE7', '#FEF3C7', '#FCE7F3', '#DBEAFE']; + return colors[index % colors.length]; + }; + + const modeConfig = getModeConfig(session.mode); + + return ( + + {/* Icon */} + + {modeConfig.icon} + + + {/* Info */} + + {modeConfig.displayName} + {formatDate(session.timestamp)} + + + {/* Score */} + + + {Math.round(session.cognitiveMetrics.focusScore)}% + + {formatDuration(session.duration)} + + + ); +}; + +const styles = StyleSheet.create({ + container: { + flexDirection: 'row', + alignItems: 'center', + backgroundColor: '#FFFFFF', + padding: 16, + borderRadius: 18, + marginBottom: 10, + elevation: 2, + shadowColor: '#000', + shadowOffset: { width: 0, height: 1 }, + shadowOpacity: 0.05, + shadowRadius: 4, + }, + iconCircle: { + width: 48, + height: 48, + borderRadius: 16, + justifyContent: 'center', + alignItems: 'center', + marginRight: 14, + }, + modeIcon: { + fontSize: 22, + }, + info: { + flex: 1, + }, + modeName: { + fontSize: 16, + fontWeight: '600', + color: AppColors.textPrimary, + marginBottom: 3, + }, + date: { + fontSize: 12, + color: AppColors.textMuted, + fontWeight: '400', + }, + scoreContainer: { + alignItems: 'flex-end', + }, + score: { + fontSize: 17, + fontWeight: '700', + marginBottom: 2, + }, + duration: { + fontSize: 11, + color: AppColors.textMuted, + fontWeight: '500', + }, +}); diff --git a/src/components/SuggestionCard.tsx b/src/components/SuggestionCard.tsx new file mode 100644 index 0000000..fcefc16 --- /dev/null +++ b/src/components/SuggestionCard.tsx @@ -0,0 +1,66 @@ +import React, { useEffect, useRef } from 'react'; +import { View, Text, StyleSheet, Animated } from 'react-native'; +import LinearGradient from 'react-native-linear-gradient'; +import { DetectedPattern } from '../types/session'; +import { AppColors } from '../theme'; +import { getPatternDefinition } from '../ai/patternLibrary'; + +interface SuggestionCardProps { + pattern: DetectedPattern; + onDismiss?: () => void; +} + +export const SuggestionCard: React.FC = ({ pattern }) => { + const slideAnim = useRef(new Animated.Value(100)).current; + const fadeAnim = useRef(new Animated.Value(0)).current; + + useEffect(() => { + Animated.parallel([ + Animated.spring(slideAnim, { toValue: 0, tension: 50, friction: 8, useNativeDriver: true }), + Animated.timing(fadeAnim, { toValue: 1, duration: 300, useNativeDriver: true }), + ]).start(); + }, [slideAnim, fadeAnim]); + + const patternDef = getPatternDefinition(pattern.pattern); + + const getSeverityGradient = (): [string, string] => { + switch (pattern.severity) { + case 'high': return ['#FF3B30', '#C53030']; + case 'medium': return ['#FF9F0A', '#D97706']; + case 'low': return ['#7B61FF', '#9B82FF']; + default: return ['#7B61FF', '#9B82FF']; + } + }; + + return ( + + + + + {patternDef.displayName} + {Math.round(pattern.confidenceScore)}% + + {patternDef.description} + + + 💡 Suggestion + {pattern.suggestion} + + + + ); +}; + +const styles = StyleSheet.create({ + container: { marginHorizontal: 16, marginBottom: 10, borderRadius: 20, overflow: 'hidden', elevation: 6, shadowColor: '#7B61FF', shadowOffset: { width: 0, height: 4 }, shadowOpacity: 0.25, shadowRadius: 12 }, + gradient: { padding: 16 }, + header: { marginBottom: 12 }, + titleRow: { flexDirection: 'row', justifyContent: 'space-between', alignItems: 'center', marginBottom: 6 }, + patternName: { fontSize: 18, fontWeight: '700', color: '#FFFFFF', flex: 1 }, + badge: { paddingHorizontal: 10, paddingVertical: 4, borderRadius: 12, marginLeft: 8, backgroundColor: 'rgba(255,255,255,0.25)' }, + badgeText: { fontSize: 12, fontWeight: '700', color: '#FFFFFF' }, + description: { fontSize: 13, color: 'rgba(255,255,255,0.8)', lineHeight: 18 }, + suggestionBox: { backgroundColor: 'rgba(255,255,255,0.15)', padding: 14, borderRadius: 14 }, + suggestionLabel: { fontSize: 12, fontWeight: '600', color: 'rgba(255,255,255,0.7)', marginBottom: 4 }, + suggestionText: { fontSize: 14, color: '#FFFFFF', lineHeight: 20 }, +}); diff --git a/src/components/index.ts b/src/components/index.ts index 354e4c3..aa84643 100644 --- a/src/components/index.ts +++ b/src/components/index.ts @@ -1,4 +1,6 @@ -export * from './FeatureCard'; -export * from './ModelLoaderWidget'; -export * from './ChatMessageBubble'; -export * from './AudioVisualizer'; +export { LiveTranscript } from './LiveTranscript'; +export { SuggestionCard } from './SuggestionCard'; +export { CognitiveMeter } from './CognitiveMeter'; +export { SessionSummaryCard } from './SessionSummaryCard'; +export { CounterStrategyCard } from './CounterStrategyCard'; +export { CircularScore } from './CircularScore'; diff --git a/src/config/index.ts b/src/config/index.ts new file mode 100644 index 0000000..71a94e8 --- /dev/null +++ b/src/config/index.ts @@ -0,0 +1 @@ +export * from './languages'; diff --git a/src/config/languages.ts b/src/config/languages.ts new file mode 100644 index 0000000..5d2038f --- /dev/null +++ b/src/config/languages.ts @@ -0,0 +1,121 @@ +/** + * Language configuration for multi-language STT/TTS support + * Per RunAnywhere docs: https://docs.runanywhere.ai/react-native/stt/options + */ + +export interface LanguageConfig { + code: string; + name: string; + nativeName: string; + flag: string; + sttModelId?: string; + ttsModelId?: string; + sttModelUrl?: string; + ttsModelUrl?: string; +} + +/** + * Supported languages with their model configurations + * Models from: https://github.com/RunanywhereAI/sherpa-onnx/releases + */ +export const SUPPORTED_LANGUAGES: LanguageConfig[] = [ + { + code: 'en', + name: 'English', + nativeName: 'English', + flag: 'đŸ‡ē🇸', + sttModelId: 'sherpa-onnx-whisper-base.en', + ttsModelId: 'vits-piper-en_US-lessac-medium', + sttModelUrl: + 'https://github.com/RunanywhereAI/sherpa-onnx/releases/download/runanywhere-models-v1/sherpa-onnx-whisper-base.en.tar.gz', + ttsModelUrl: + 'https://github.com/RunanywhereAI/sherpa-onnx/releases/download/runanywhere-models-v1/vits-piper-en_US-lessac-medium.tar.gz', + }, + { + code: 'es', + name: 'Spanish', + nativeName: 'EspaÃąol', + flag: 'đŸ‡Ē🇸', + sttModelId: 'sherpa-onnx-whisper-tiny-es', + ttsModelId: 'vits-piper-es_ES-sharvard-medium', + // Note: These are example URLs - actual multilingual models would need to be hosted + sttModelUrl: + 'https://github.com/RunanywhereAI/sherpa-onnx/releases/download/runanywhere-models-v1/sherpa-onnx-whisper-tiny.tar.gz', + ttsModelUrl: + 'https://github.com/RunanywhereAI/sherpa-onnx/releases/download/runanywhere-models-v1/vits-piper-es_ES-sharvard-medium.tar.gz', + }, + { + code: 'fr', + name: 'French', + nativeName: 'Français', + flag: 'đŸ‡Ģ🇷', + sttModelId: 'sherpa-onnx-whisper-tiny-fr', + ttsModelId: 'vits-piper-fr_FR-upmc-medium', + sttModelUrl: + 'https://github.com/RunanywhereAI/sherpa-onnx/releases/download/runanywhere-models-v1/sherpa-onnx-whisper-tiny.tar.gz', + ttsModelUrl: + 'https://github.com/RunanywhereAI/sherpa-onnx/releases/download/runanywhere-models-v1/vits-piper-fr_FR-upmc-medium.tar.gz', + }, + { + code: 'de', + name: 'German', + nativeName: 'Deutsch', + flag: '🇩đŸ‡Ē', + sttModelId: 'sherpa-onnx-whisper-tiny-de', + ttsModelId: 'vits-piper-de_DE-thorsten-medium', + sttModelUrl: + 'https://github.com/RunanywhereAI/sherpa-onnx/releases/download/runanywhere-models-v1/sherpa-onnx-whisper-tiny.tar.gz', + ttsModelUrl: + 'https://github.com/RunanywhereAI/sherpa-onnx/releases/download/runanywhere-models-v1/vits-piper-de_DE-thorsten-medium.tar.gz', + }, + { + code: 'zh', + name: 'Chinese', + nativeName: '中文', + flag: '🇨đŸ‡ŗ', + sttModelId: 'sherpa-onnx-whisper-tiny-zh', + ttsModelId: 'vits-piper-zh_CN-huayan-medium', + sttModelUrl: + 'https://github.com/RunanywhereAI/sherpa-onnx/releases/download/runanywhere-models-v1/sherpa-onnx-whisper-tiny.tar.gz', + }, + { + code: 'ja', + name: 'Japanese', + nativeName: 'æ—ĨæœŦčĒž', + flag: 'đŸ‡¯đŸ‡ĩ', + sttModelId: 'sherpa-onnx-whisper-tiny-ja', + ttsModelId: 'vits-piper-ja_JP-medium', + sttModelUrl: + 'https://github.com/RunanywhereAI/sherpa-onnx/releases/download/runanywhere-models-v1/sherpa-onnx-whisper-tiny.tar.gz', + }, +]; + +/** + * Get language configuration by code + */ +export const getLanguageByCode = (code: string): LanguageConfig | undefined => { + return SUPPORTED_LANGUAGES.find((lang) => lang.code === code); +}; + +/** + * Get default language (English) + */ +export const getDefaultLanguage = (): LanguageConfig => { + return SUPPORTED_LANGUAGES[0]; +}; + +/** + * Check if language has STT support + */ +export const hasSTTSupport = (code: string): boolean => { + const lang = getLanguageByCode(code); + return !!(lang?.sttModelId && lang?.sttModelUrl); +}; + +/** + * Check if language has TTS support + */ +export const hasTTSSupport = (code: string): boolean => { + const lang = getLanguageByCode(code); + return !!(lang?.ttsModelId && lang?.ttsModelUrl); +}; diff --git a/src/hooks/useCounterStrategy.ts b/src/hooks/useCounterStrategy.ts new file mode 100644 index 0000000..c420e18 --- /dev/null +++ b/src/hooks/useCounterStrategy.ts @@ -0,0 +1,85 @@ +/** + * 🔒 PRIVACY NOTICE + * All counter-strategy logic runs locally on device. + * No data leaves this device. + */ + +import { useState, useEffect, useRef, useCallback } from 'react'; +import { DetectedPattern, CounterStrategy } from '../types/session'; +import { generateCounterStrategies, resetAllCooldowns } from '../services/CounterStrategyEngine'; + +interface UseCounterStrategyReturn { + /** The currently active counter-strategy to display, or null */ + activeStrategy: CounterStrategy | null; + /** Dismiss the current strategy card */ + dismiss: () => void; + /** Reset all cooldowns (e.g., on session start) */ + reset: () => void; +} + +/** + * Hook that processes detected patterns and produces counter-strategy suggestions. + * + * - Only triggers when confidence > threshold (default 70%) + * - Applies 10-second cooldown per tactic + * - Only updates UI when tactic actually changes (no flicker) + * + * @param detectedPatterns - Array of detected patterns from the live session + * @param confidenceThreshold - Minimum confidence to trigger (default 70) + */ +export const useCounterStrategy = ( + detectedPatterns: DetectedPattern[], + confidenceThreshold: number = 70, +): UseCounterStrategyReturn => { + const [activeStrategy, setActiveStrategy] = useState(null); + const lastTacticRef = useRef(null); + const lastTimestampRef = useRef(0); + + useEffect(() => { + if (detectedPatterns.length === 0) return; + + // Get the highest-confidence pattern (array is sorted by confidence, highest first) + const topPattern = detectedPatterns[0]; + + if (!topPattern) return; + + console.log('[useCounterStrategy] 🔍 Evaluating top pattern:', topPattern.pattern, 'confidence:', topPattern.confidenceScore, 'id:', topPattern.id); + + // Skip if same pattern ID (exact same detection, prevents flicker) + if (topPattern.id === lastTacticRef.current) { + console.log('[useCounterStrategy] ⏭ī¸ Same pattern ID, skipping'); + return; + } + + // Try to generate counter-strategy (engine handles cooldown internally) + const strategy = generateCounterStrategies( + topPattern.pattern, + topPattern.confidenceScore, + 10_000, // 10s cooldown + confidenceThreshold, + ); + + if (strategy) { + console.log('[useCounterStrategy] ✅ New counter-strategy:', strategy.tacticDisplayName); + lastTacticRef.current = topPattern.id; + lastTimestampRef.current = Date.now(); + setActiveStrategy(strategy); + } else { + console.log('[useCounterStrategy] ❌ No strategy generated (cooldown or below threshold)'); + } + }, [detectedPatterns, detectedPatterns.length, confidenceThreshold]); + + const dismiss = useCallback(() => { + setActiveStrategy(null); + lastTacticRef.current = null; + }, []); + + const reset = useCallback(() => { + setActiveStrategy(null); + lastTacticRef.current = null; + lastTimestampRef.current = 0; + resetAllCooldowns(); + }, []); + + return { activeStrategy, dismiss, reset }; +}; diff --git a/src/hooks/useLiveSession.ts b/src/hooks/useLiveSession.ts new file mode 100644 index 0000000..f181066 --- /dev/null +++ b/src/hooks/useLiveSession.ts @@ -0,0 +1,335 @@ +/** + * 🔒 PRIVACY NOTICE + * All session management runs locally on device. + * No data leaves this device. Uses RunAnywhere SDK for on-device inference only. + */ + +/** + * useLiveSession — Unified hook for the live session lifecycle. + * + * Replaces both useLiveTranscription and useCounterStrategy with a single + * useReducer + SessionEngine integration. + * + * WHY DEBOUNCE REDUCES FLICKER: + * - Without debounce, tactic analysis runs on every transcript chunk (every few seconds). + * - This causes rapid state changes → suggestion cards flash in/out. + * - The 250ms debounce waits for a speech pause before triggering analysis, + * so suggestions only update after the user finishes a phrase. + * + * HOW isLiveRef PREVENTS UNMOUNTED UPDATES: + * - Async inference (analyzeSession) can resolve AFTER stopSession() is called. + * - isLiveRef.current is set to false immediately on stop/cancel. + * - All async callbacks check isLiveRef.current before dispatching. + * - This prevents "Can't perform state update on unmounted component" errors. + */ + +import { useReducer, useRef, useCallback, useEffect } from 'react'; +import { NegotiationMode, Session, AnalysisResult } from '../types/session'; +import { + sessionReducer, + createInitialState, + SessionState, + SessionAction, +} from '../state/sessionReducer'; +import { SessionEngine } from '../services/SessionEngine'; +import { LocalStorageService } from '../services/LocalStorageService'; +import { resetAllCooldowns } from '../services/CounterStrategyEngine'; + +// ─────────────────────────── Return Type ─────────────────────────── + +export interface UseLiveSessionReturn { + /** Current session state (single source of truth) */ + state: SessionState; + /** Whether the session is actively recording */ + isRecording: boolean; + /** Start a new session in the given mode */ + startSession: (mode: NegotiationMode) => Promise; + /** Stop the session and save it */ + stopSession: () => Promise; + /** Cancel the session without saving */ + cancelSession: () => Promise; + /** Most recent error message */ + error: string | null; +} + +// ─────────────────────────── Constants ─────────────────────────── + +/** + * WHY 250ms DEBOUNCE: + * Tactic inference doesn't need to run for every single word. + * We wait 250ms after the last transcript chunk before running analysis. + * This ensures analysis only triggers during speech pauses. + */ +const ANALYSIS_DEBOUNCE_MS = 250; + +// ─────────────────────────── Hook ─────────────────────────── + +export const useLiveSession = (): UseLiveSessionReturn => { + const [state, dispatch] = useReducer(sessionReducer, createInitialState()); + + /** + * isLiveRef: Prevents async callbacks from dispatching after session ends. + * Set to true on startSession, false on stopSession/cancelSession. + * Every async callback checks this before calling dispatch. + */ + const isLiveRef = useRef(false); + + /** SessionEngine handles recording, transcription, and auto-save */ + const sessionEngineRef = useRef(null); + + /** Duration timer interval */ + const durationIntervalRef = useRef(null); + + /** + * Track the last dispatched transcript chunk ID to avoid duplicates. + * The SessionEngine callback fires on every state update (audio level, duration, etc.), + * not just new transcripts. Without this, we'd dispatch the same chunk multiple times. + */ + const lastDispatchedChunkIdRef = useRef(null); + + /** + * Debounce timer for tactic analysis. + * WHY: We don't want to run NegotiationAnalyzer on every transcript chunk. + * Instead we wait 250ms after the last chunk, then run analysis once. + * This dramatically reduces suggestion flicker. + */ + const analysisDebounceRef = useRef(null); + + // ─── Initialize SessionEngine ─── + useEffect(() => { + sessionEngineRef.current = new SessionEngine(); + + return () => { + // Cleanup on unmount + if (sessionEngineRef.current) { + sessionEngineRef.current.cleanup(); + } + if (durationIntervalRef.current) { + clearInterval(durationIntervalRef.current); + } + if (analysisDebounceRef.current) { + clearTimeout(analysisDebounceRef.current); + } + isLiveRef.current = false; + }; + }, []); + + // ─── Safe Dispatch (checks isLiveRef) ─── + const safeDispatch = useCallback((action: SessionAction) => { + if (!isLiveRef.current && action.type !== 'RESET') { + console.log( + `[useLiveSession] ⚠ī¸ Dispatch blocked (session ended): ${action.type}`, + ); + return; + } + dispatch(action); + }, []); + + // ─── Start Session ─── + const startSession = useCallback( + async (mode: NegotiationMode): Promise => { + if (!sessionEngineRef.current) { + dispatch({ type: 'SET_ERROR', message: 'Session engine not initialized' }); + return false; + } + + // Reset cooldowns from previous session + resetAllCooldowns(); + + // Set live flag BEFORE starting anything + isLiveRef.current = true; + + // Dispatch START to reducer + dispatch({ + type: 'START_SESSION', + mode, + startTime: Date.now(), + }); + + try { + // Start SessionEngine — it calls our callbacks for transcript + analysis + const started = await sessionEngineRef.current.startSession( + mode, + (engineState) => { + // Guard: don't dispatch if session was stopped + if (!isLiveRef.current) { + console.log('[useLiveSession] ⚠ī¸ SessionEngine callback blocked — session ended'); + return; + } + + // ─── Sync transcript chunks from engine to reducer ─── + // Sync the entire array to handle mutative string concatenations (paragraph building) + if (engineState.transcript.length > 0) { + const latestChunk = + engineState.transcript[engineState.transcript.length - 1]; + + // Check if the actual text content changed, since IDs remain the same during concatenation + if (latestChunk.text !== lastDispatchedChunkIdRef.current) { + lastDispatchedChunkIdRef.current = latestChunk.text; + + // Dispatch the synced transcript array + safeDispatch({ + type: 'SYNC_TRANSCRIPT', + transcript: engineState.transcript, + }); + } + } + + // ─── Debounced tactic analysis ─── + // ALWAYS evaluate the latest intent detection patterns, regardless of string change + // since intent resolution is purely asynchronous! + if (engineState.detectedPatterns.length > 0) { + if (analysisDebounceRef.current) { + clearTimeout(analysisDebounceRef.current); + } + + analysisDebounceRef.current = setTimeout(() => { + if (!isLiveRef.current) return; + safeDispatch({ + type: 'TACTIC_DETECTED', + patterns: engineState.detectedPatterns, + focusScore: engineState.currentFocusScore, + timestampMs: Date.now(), + }); + }, ANALYSIS_DEBOUNCE_MS); + } + + // ─── Sync audio level ─── + safeDispatch({ + type: 'UPDATE_AUDIO_LEVEL', + level: engineState.audioLevel, + }); + }, + ); + + if (!started) { + isLiveRef.current = false; + dispatch({ + type: 'SET_ERROR', + message: 'Failed to start recording. Please check microphone permissions.', + }); + return false; + } + + // ─── Duration Timer ─── + durationIntervalRef.current = setInterval(() => { + if (!isLiveRef.current) return; + if (sessionEngineRef.current) { + sessionEngineRef.current.updateDuration(); + const engineState = sessionEngineRef.current.getState(); + safeDispatch({ + type: 'UPDATE_DURATION', + duration: engineState.duration, + }); + + // Also check for new patterns from continuous analysis + if (engineState.detectedPatterns.length > 0) { + safeDispatch({ + type: 'TACTIC_DETECTED', + patterns: engineState.detectedPatterns, + focusScore: engineState.currentFocusScore, + timestampMs: Date.now(), + }); + } + } + }, 1000); + + console.log('[useLiveSession] ✅ Session started'); + return true; + } catch (err) { + isLiveRef.current = false; + const errorMessage = + err instanceof Error ? err.message : 'Unknown error'; + dispatch({ + type: 'SET_ERROR', + message: `Failed to start session: ${errorMessage}`, + }); + return false; + } + }, + [safeDispatch], + ); + + // ─── Stop Session ─── + const stopSession = useCallback(async (): Promise => { + if (!sessionEngineRef.current) { + return null; + } + + // Immediately prevent further async dispatches + isLiveRef.current = false; + + // Clear timers + if (durationIntervalRef.current) { + clearInterval(durationIntervalRef.current); + durationIntervalRef.current = null; + } + if (analysisDebounceRef.current) { + clearTimeout(analysisDebounceRef.current); + analysisDebounceRef.current = null; + } + + // Dispatch STOP to reducer + dispatch({ type: 'STOP_SESSION' }); + + try { + const session = await sessionEngineRef.current.stopSession(); + + // Reset state for next session + dispatch({ type: 'RESET' }); + + return session; + } catch (err) { + const errorMessage = + err instanceof Error ? err.message : 'Unknown error'; + dispatch({ + type: 'SET_ERROR', + message: `Failed to stop session: ${errorMessage}`, + }); + return null; + } + }, []); + + // ─── Cancel Session ─── + const cancelSession = useCallback(async (): Promise => { + if (!sessionEngineRef.current) { + return; + } + + // Immediately prevent further async dispatches + isLiveRef.current = false; + + // Clear timers + if (durationIntervalRef.current) { + clearInterval(durationIntervalRef.current); + durationIntervalRef.current = null; + } + if (analysisDebounceRef.current) { + clearTimeout(analysisDebounceRef.current); + analysisDebounceRef.current = null; + } + + try { + await sessionEngineRef.current.cancelSession(); + } catch (err) { + const errorMessage = + err instanceof Error ? err.message : 'Unknown error'; + dispatch({ + type: 'SET_ERROR', + message: `Failed to cancel session: ${errorMessage}`, + }); + } + + // Reset state + dispatch({ type: 'RESET' }); + }, []); + + return { + state, + isRecording: state.status === 'RUNNING', + startSession, + stopSession, + cancelSession, + error: state.error, + }; +}; diff --git a/src/hooks/useLiveTranscription.ts b/src/hooks/useLiveTranscription.ts new file mode 100644 index 0000000..6db6b66 --- /dev/null +++ b/src/hooks/useLiveTranscription.ts @@ -0,0 +1,166 @@ +/** + * Custom hook for managing live session state + */ + +import { useState, useEffect, useRef, useCallback } from 'react'; +import { NegotiationMode, LiveSessionState, Session } from '../types/session'; +import { SessionEngine } from '../services/SessionEngine'; + +export interface UseSessionReturn { + sessionState: LiveSessionState; + isRecording: boolean; + startSession: (mode: NegotiationMode) => Promise; + stopSession: () => Promise; + cancelSession: () => Promise; + error: string | null; +} + +/** + * Hook for managing live transcription and session + */ +export const useLiveTranscription = (): UseSessionReturn => { + const [sessionState, setSessionState] = useState({ + isRecording: false, + startTime: 0, + duration: 0, + transcript: [], + detectedPatterns: [], + currentFocusScore: 100, + audioLevel: 0, + lastAutoSave: 0, + }); + const [error, setError] = useState(null); + const sessionEngineRef = useRef(null); + const durationIntervalRef = useRef(null); + + // Initialize session engine + useEffect(() => { + sessionEngineRef.current = new SessionEngine(); + + return () => { + // Cleanup on unmount + if (sessionEngineRef.current) { + sessionEngineRef.current.cleanup(); + } + if (durationIntervalRef.current) { + clearInterval(durationIntervalRef.current); + } + }; + }, []); + + // Start duration timer + const startDurationTimer = useCallback(() => { + durationIntervalRef.current = setInterval(() => { + if (sessionEngineRef.current) { + sessionEngineRef.current.updateDuration(); + } + }, 1000); // Update every second + }, []); + + // Stop duration timer + const stopDurationTimer = useCallback(() => { + if (durationIntervalRef.current) { + clearInterval(durationIntervalRef.current); + durationIntervalRef.current = null; + } + }, []); + + // Start session + const startSession = useCallback( + async (mode: NegotiationMode): Promise => { + if (!sessionEngineRef.current) { + setError('Session engine not initialized'); + return false; + } + + setError(null); + + try { + const started = await sessionEngineRef.current.startSession(mode, (state) => { + setSessionState(state); + }); + + if (started) { + startDurationTimer(); + return true; + } else { + setError('Failed to start recording. Please check microphone permissions.'); + return false; + } + } catch (err) { + const errorMessage = err instanceof Error ? err.message : 'Unknown error'; + setError(`Failed to start session: ${errorMessage}`); + return false; + } + }, + [startDurationTimer] + ); + + // Stop session + const stopSession = useCallback(async (): Promise => { + if (!sessionEngineRef.current) { + return null; + } + + stopDurationTimer(); + + try { + const session = await sessionEngineRef.current.stopSession(); + + // Reset state + setSessionState({ + isRecording: false, + startTime: 0, + duration: 0, + transcript: [], + detectedPatterns: [], + currentFocusScore: 100, + audioLevel: 0, + lastAutoSave: 0, + }); + + return session; + } catch (err) { + const errorMessage = err instanceof Error ? err.message : 'Unknown error'; + setError(`Failed to stop session: ${errorMessage}`); + return null; + } + }, [stopDurationTimer]); + + // Cancel session + const cancelSession = useCallback(async (): Promise => { + if (!sessionEngineRef.current) { + return; + } + + stopDurationTimer(); + + try { + await sessionEngineRef.current.cancelSession(); + + // Reset state + setSessionState({ + isRecording: false, + startTime: 0, + duration: 0, + transcript: [], + detectedPatterns: [], + currentFocusScore: 100, + audioLevel: 0, + lastAutoSave: 0, + }); + } catch (err) { + const errorMessage = err instanceof Error ? err.message : 'Unknown error'; + setError(`Failed to cancel session: ${errorMessage}`); + } + }, [stopDurationTimer]); + + return { + sessionState, + isRecording: sessionState.isRecording, + startSession, + stopSession, + cancelSession, + error, + }; +}; diff --git a/src/hooks/useSessionAnalyzer.ts b/src/hooks/useSessionAnalyzer.ts new file mode 100644 index 0000000..18c4f94 --- /dev/null +++ b/src/hooks/useSessionAnalyzer.ts @@ -0,0 +1,119 @@ +/** + * Custom hook for session analysis and statistics + */ + +import { useState, useEffect, useCallback } from 'react'; +import { Session, SessionStats } from '../types/session'; +import { LocalStorageService } from '../services/LocalStorageService'; + +export interface UseSessionAnalyzerReturn { + sessions: Session[]; + stats: SessionStats | null; + isLoading: boolean; + error: string | null; + refreshSessions: () => Promise; + getSession: (sessionId: string) => Promise; + deleteSession: (sessionId: string) => Promise; + deleteAllSessions: () => Promise; +} + +/** + * Hook for analyzing and managing sessions + */ +export const useSessionAnalyzer = (): UseSessionAnalyzerReturn => { + const [sessions, setSessions] = useState([]); + const [stats, setStats] = useState(null); + const [isLoading, setIsLoading] = useState(true); + const [error, setError] = useState(null); + + // Load sessions and stats + const loadData = useCallback(async () => { + setIsLoading(true); + setError(null); + + try { + const [loadedSessions, loadedStats] = await Promise.all([ + LocalStorageService.getAllSessions(), + LocalStorageService.getStats(), + ]); + + setSessions(loadedSessions); + setStats(loadedStats); + } catch (err) { + const errorMessage = err instanceof Error ? err.message : 'Unknown error'; + setError(`Failed to load sessions: ${errorMessage}`); + } finally { + setIsLoading(false); + } + }, []); + + // Initial load + useEffect(() => { + loadData(); + }, [loadData]); + + // Refresh sessions + const refreshSessions = useCallback(async () => { + await loadData(); + }, [loadData]); + + // Get single session + const getSession = useCallback(async (sessionId: string): Promise => { + try { + return await LocalStorageService.getSession(sessionId); + } catch (err) { + const errorMessage = err instanceof Error ? err.message : 'Unknown error'; + setError(`Failed to get session: ${errorMessage}`); + return null; + } + }, []); + + // Delete session + const deleteSession = useCallback(async (sessionId: string): Promise => { + try { + await LocalStorageService.deleteSession(sessionId); + + // Update local state + setSessions((prev) => prev.filter((s) => s.id !== sessionId)); + + // Recalculate stats + const newStats = await LocalStorageService.calculateStats(); + setStats(newStats); + } catch (err) { + const errorMessage = err instanceof Error ? err.message : 'Unknown error'; + setError(`Failed to delete session: ${errorMessage}`); + } + }, []); + + // Delete all sessions + const deleteAllSessions = useCallback(async (): Promise => { + try { + await LocalStorageService.deleteAllSessions(); + + // Update local state + setSessions([]); + setStats({ + totalSessions: 0, + avgFocusScore: 0, + avgDuration: 0, + mostCommonPattern: null, + totalPatterns: 0, + lastSessionDate: null, + }); + } catch (err) { + const errorMessage = err instanceof Error ? err.message : 'Unknown error'; + setError(`Failed to delete all sessions: ${errorMessage}`); + } + }, []); + + return { + sessions, + stats, + isLoading, + error, + refreshSessions, + getSession, + deleteSession, + deleteAllSessions, + }; +}; diff --git a/src/navigation/types.ts b/src/navigation/types.ts index 8f849dd..12c7b42 100644 --- a/src/navigation/types.ts +++ b/src/navigation/types.ts @@ -1,8 +1,26 @@ +import { NegotiationMode } from '../types/session'; + export type RootStackParamList = { + Disclaimer: undefined; Home: undefined; - Chat: undefined; - ToolCalling: undefined; - SpeechToText: undefined; - TextToSpeech: undefined; - VoicePipeline: undefined; + PreSessionForm: { + mode: NegotiationMode; + }; + PreSessionStrategy: { + mode: NegotiationMode; + inputs: import('../types/session').PreSessionInputs; + analysis: import('../types/session').StrategicAnalysis; + }; + LiveSession: { + mode: NegotiationMode; + preSessionInputs?: import('../types/session').PreSessionInputs; + strategicAnalysis?: import('../types/session').StrategicAnalysis; + }; + OutcomeReplay: { + sessionId?: string; // Optional: If empty, load latest + }; + Insights: { + sessionId: string; + }; + Settings: undefined; }; diff --git a/src/screens/ChatScreen.tsx b/src/screens/ChatScreen.tsx deleted file mode 100644 index 8bdc707..0000000 --- a/src/screens/ChatScreen.tsx +++ /dev/null @@ -1,330 +0,0 @@ -import React, { useState, useRef, useEffect } from 'react'; -import { - View, - Text, - TextInput, - TouchableOpacity, - FlatList, - StyleSheet, - KeyboardAvoidingView, - Platform, -} from 'react-native'; -import LinearGradient from 'react-native-linear-gradient'; -import { RunAnywhere } from '@runanywhere/core'; -import { AppColors } from '../theme'; -import { useModelService } from '../services/ModelService'; -import { ChatMessageBubble, ChatMessage, ModelLoaderWidget } from '../components'; - -export const ChatScreen: React.FC = () => { - const modelService = useModelService(); - const [messages, setMessages] = useState([]); - const [inputText, setInputText] = useState(''); - const [isGenerating, setIsGenerating] = useState(false); - const [currentResponse, setCurrentResponse] = useState(''); - const flatListRef = useRef(null); - const streamCancelRef = useRef<(() => void) | null>(null); - const responseRef = useRef(''); // Track response for closure - - useEffect(() => { - // Scroll to bottom when messages change - if (messages.length > 0) { - setTimeout(() => { - flatListRef.current?.scrollToEnd({ animated: true }); - }, 100); - } - }, [messages, currentResponse]); - - const handleSend = async () => { - const text = inputText.trim(); - if (!text || isGenerating) return; - - // Add user message - const userMessage: ChatMessage = { - text, - isUser: true, - timestamp: new Date(), - }; - setMessages(prev => [...prev, userMessage]); - setInputText(''); - setIsGenerating(true); - setCurrentResponse(''); - - try { - // Per docs: https://docs.runanywhere.ai/react-native/quick-start#6-stream-responses - const streamResult = await RunAnywhere.generateStream(text, { - maxTokens: 256, - temperature: 0.8, - }); - - streamCancelRef.current = streamResult.cancel; - responseRef.current = ''; - - // Stream tokens as they arrive - for await (const token of streamResult.stream) { - responseRef.current += token; - setCurrentResponse(responseRef.current); - } - - // Get final metrics - const finalResult = await streamResult.result; - - // Add assistant message (use ref to get final text due to closure) - const assistantMessage: ChatMessage = { - text: responseRef.current, - isUser: false, - timestamp: new Date(), - tokensPerSecond: finalResult.performanceMetrics?.tokensPerSecond, - totalTokens: finalResult.performanceMetrics?.totalTokens, - }; - setMessages(prev => [...prev, assistantMessage]); - setCurrentResponse(''); - responseRef.current = ''; - setIsGenerating(false); - } catch (error) { - const errorMessage: ChatMessage = { - text: `Error: ${error}`, - isUser: false, - timestamp: new Date(), - isError: true, - }; - setMessages(prev => [...prev, errorMessage]); - setCurrentResponse(''); - setIsGenerating(false); - } - }; - - const handleStop = () => { - if (streamCancelRef.current) { - streamCancelRef.current(); - if (responseRef.current) { - const message: ChatMessage = { - text: responseRef.current, - isUser: false, - timestamp: new Date(), - wasCancelled: true, - }; - setMessages(prev => [...prev, message]); - } - setCurrentResponse(''); - responseRef.current = ''; - setIsGenerating(false); - } - }; - - const handleClearChat = () => { - setMessages([]); - }; - - const renderSuggestionChip = (text: string) => ( - { - setInputText(text); - handleSend(); - }} - > - {text} - - ); - - if (!modelService.isLLMLoaded) { - return ( - - ); - } - - return ( - - {messages.length === 0 ? ( - - - đŸ’Ŧ - - Start a Conversation - - Ask anything! The AI runs entirely on your device. - - - {renderSuggestionChip('Tell me a joke')} - {renderSuggestionChip('What is AI?')} - {renderSuggestionChip('Write a haiku')} - - - ) : ( - ( - - )} - keyExtractor={(_, index) => index.toString()} - contentContainerStyle={styles.messageList} - showsVerticalScrollIndicator={false} - /> - )} - - {/* Input Area */} - - - - {isGenerating ? ( - - - ⏚ - - - ) : ( - - - 📤 - - - )} - - - - ); -}; - -const styles = StyleSheet.create({ - container: { - flex: 1, - backgroundColor: AppColors.primaryDark, - }, - messageList: { - padding: 16, - }, - emptyState: { - flex: 1, - justifyContent: 'center', - alignItems: 'center', - padding: 32, - }, - emptyIconContainer: { - width: 100, - height: 100, - backgroundColor: AppColors.accentCyan + '20', - borderRadius: 50, - justifyContent: 'center', - alignItems: 'center', - marginBottom: 24, - }, - emptyIcon: { - fontSize: 48, - }, - emptyTitle: { - fontSize: 24, - fontWeight: '700', - color: AppColors.textPrimary, - marginBottom: 12, - }, - emptySubtitle: { - fontSize: 14, - color: AppColors.textSecondary, - textAlign: 'center', - marginBottom: 32, - }, - suggestionsContainer: { - flexDirection: 'row', - flexWrap: 'wrap', - justifyContent: 'center', - gap: 8, - }, - suggestionChip: { - paddingHorizontal: 16, - paddingVertical: 8, - backgroundColor: AppColors.surfaceCard, - borderRadius: 20, - borderWidth: 1, - borderColor: AppColors.accentCyan + '40', - }, - suggestionText: { - fontSize: 12, - color: AppColors.textPrimary, - }, - inputContainer: { - padding: 16, - backgroundColor: AppColors.surfaceCard + 'CC', - borderTopWidth: 1, - borderTopColor: AppColors.textMuted + '1A', - }, - inputWrapper: { - flexDirection: 'row', - alignItems: 'center', - gap: 12, - }, - input: { - flex: 1, - backgroundColor: AppColors.primaryMid, - borderRadius: 24, - paddingHorizontal: 20, - paddingVertical: 12, - fontSize: 15, - color: AppColors.textPrimary, - maxHeight: 100, - }, - sendButton: { - width: 48, - height: 48, - borderRadius: 24, - justifyContent: 'center', - alignItems: 'center', - elevation: 4, - shadowColor: AppColors.accentCyan, - shadowOffset: { width: 0, height: 4 }, - shadowOpacity: 0.3, - shadowRadius: 8, - }, - sendIcon: { - fontSize: 20, - }, - stopButton: { - width: 48, - height: 48, - borderRadius: 24, - backgroundColor: AppColors.error + '33', - justifyContent: 'center', - alignItems: 'center', - }, - stopIcon: { - width: 48, - height: 48, - justifyContent: 'center', - alignItems: 'center', - }, - stopIconText: { - fontSize: 20, - color: AppColors.error, - }, -}); diff --git a/src/screens/DisclaimerScreen.tsx b/src/screens/DisclaimerScreen.tsx new file mode 100644 index 0000000..c673480 --- /dev/null +++ b/src/screens/DisclaimerScreen.tsx @@ -0,0 +1,166 @@ +import React, { useEffect, useState } from 'react'; +import { View, Text, StyleSheet, TouchableOpacity, SafeAreaView } from 'react-native'; +import AsyncStorage from '@react-native-async-storage/async-storage'; +import { AppColors } from '../theme'; +import { StackNavigationProp } from '@react-navigation/stack'; +import { RootStackParamList } from '../navigation/types'; + +type DisclaimerScreenNavigationProp = StackNavigationProp; + +interface DisclaimerScreenProps { + navigation: DisclaimerScreenNavigationProp; +} + +const DISCLAIMER_KEY = '@latent_disclaimer_accepted'; + +export const DisclaimerScreen: React.FC = ({ navigation }) => { + const [checking, setChecking] = useState(true); + + useEffect(() => { + checkDisclaimer(); + }, []); + + const checkDisclaimer = async () => { + try { + const isAccepted = await AsyncStorage.getItem(DISCLAIMER_KEY); + if (isAccepted === 'true') { + navigation.replace('Home'); + } else { + setChecking(false); + } + } catch (e) { + setChecking(false); + } + }; + + const handleAccept = async () => { + try { + await AsyncStorage.setItem(DISCLAIMER_KEY, 'true'); + navigation.replace('Home'); + } catch (e) { + console.error('Failed to save disclaimer acceptance', e); + } + }; + + if (checking) { + return ( + + Loading... + + ); + } + + return ( + + + + ⚠ī¸ + + + Important Notice + + + + â€ĸ This app records and analyzes conversations locally on this device. + + + + â€ĸ You must comply with all local, state, and federal recording laws (e.g., two-party consent laws) in your jurisdiction. + + + + â€ĸ Live mode is intended strictly for permitted environments such as authorized business negotiations, mock interviews, or startup pitches. + + + + â€ĸ This software is not intended for covert or prohibited usage. You are solely responsible for how you apply this technology. + + + + + I Understand & Agree + + + + ); +}; + +const styles = StyleSheet.create({ + container: { + flex: 1, + backgroundColor: '#0A0E1A', // Dark professional theme + }, + loadingContainer: { + flex: 1, + backgroundColor: '#0A0E1A', + justifyContent: 'center', + alignItems: 'center', + }, + loadingText: { + color: '#8B949E', + fontSize: 16, + }, + content: { + flex: 1, + padding: 24, + justifyContent: 'center', + }, + iconContainer: { + alignItems: 'center', + marginBottom: 24, + }, + icon: { + fontSize: 64, + }, + title: { + fontSize: 28, + fontWeight: '800', + color: '#FFFFFF', + marginBottom: 32, + textAlign: 'center', + letterSpacing: 0.5, + }, + card: { + backgroundColor: '#161B22', + borderRadius: 16, + padding: 24, + borderWidth: 1, + borderColor: '#30363D', + marginBottom: 40, + }, + bulletPoint: { + fontSize: 16, + color: '#E6EDF3', + lineHeight: 24, + marginBottom: 20, + }, + bullet: { + color: AppColors.error, + fontWeight: '900', + }, + bold: { + fontWeight: '700', + color: '#FFFFFF', + }, + acceptButton: { + backgroundColor: AppColors.accentViolet, + borderRadius: 12, + paddingVertical: 18, + alignItems: 'center', + shadowColor: AppColors.accentViolet, + shadowOffset: { width: 0, height: 4 }, + shadowOpacity: 0.3, + shadowRadius: 8, + elevation: 5, + }, + acceptButtonText: { + color: '#FFFFFF', + fontSize: 18, + fontWeight: '700', + letterSpacing: 0.5, + }, +}); diff --git a/src/screens/HomeScreen.tsx b/src/screens/HomeScreen.tsx index e734d68..d40544e 100644 --- a/src/screens/HomeScreen.tsx +++ b/src/screens/HomeScreen.tsx @@ -1,134 +1,373 @@ -import React from 'react'; +import React, { useEffect, useState } from 'react'; import { View, Text, ScrollView, StyleSheet, + TouchableOpacity, + RefreshControl, StatusBar, + Alert, + Modal, + Dimensions, } from 'react-native'; import LinearGradient from 'react-native-linear-gradient'; import { StackNavigationProp } from '@react-navigation/stack'; import { AppColors } from '../theme'; -import { FeatureCard } from '../components'; import { RootStackParamList } from '../navigation/types'; +import { useSessionAnalyzer } from '../hooks/useSessionAnalyzer'; +import { SessionSummaryCard } from '../components/SessionSummaryCard'; +import { getAllModes } from '../ai/patternLibrary'; +import { NegotiationMode, ModeConfig } from '../types/session'; + +const { width: SCREEN_WIDTH } = Dimensions.get('window'); type HomeScreenProps = { navigation: StackNavigationProp; }; export const HomeScreen: React.FC = ({ navigation }) => { + const { sessions, stats, isLoading, refreshSessions } = useSessionAnalyzer(); + const [showModeSelector, setShowModeSelector] = useState(false); + const allModes = getAllModes(); + + useEffect(() => { + const unsubscribe = navigation.addListener('focus', () => { + refreshSessions(); + }); + return unsubscribe; + }, [navigation, refreshSessions]); + + const handleStartSession = (mode: NegotiationMode) => { + setShowModeSelector(false); + navigation.navigate('PreSessionForm', { mode }); + }; + + const handleSessionPress = (sessionId: string) => { + navigation.navigate('Insights', { sessionId }); + }; + + const formatDuration = (ms: number): string => { + const minutes = Math.floor(ms / 60000); + return `${minutes}min`; + }; + + const getGreeting = (): string => { + const hour = new Date().getHours(); + if (hour < 12) return 'Good Morning'; + if (hour < 17) return 'Good Afternoon'; + return 'Good Evening'; + }; + return ( - + + + } > {/* Header */} - + - ⚡ + L + + {getGreeting()}, + Latent Strategist + - - RunAnywhere - React Native SDK Starter - + navigation.navigate('Settings')} + > + 🔔 + - {/* Privacy Banner */} - - 🔒 - - Privacy-First On-Device AI - - All AI processing happens locally on your device. No data ever leaves your phone. - - + {/* Balance Card */} + + TOTAL SESSIONS + + {stats ? stats.totalSessions : 0} + + {stats && stats.totalSessions > 0 && ( + + + 📈 {Math.round(stats.avgFocusScore)}% avg focus + + + )} + {!stats || stats.totalSessions === 0 ? ( + + ✨ Ready to start + + ) : null} + + + {/* Action Buttons */} + + {/* Live Tactical Mode */} + setShowModeSelector(true)} + activeOpacity={0.8} + > + + 🎤 + + + Live Tactical Mode + Real-time negotiation intelligence + + + + {/* Strategic Outcome Replay */} + { + if (sessions.length > 0) { + navigation.navigate('OutcomeReplay', { sessionId: sessions[0].id }); + } else { + Alert.alert('No Sessions', 'Complete a session first to view replays.'); + } + }} + activeOpacity={0.8} + > + + 🧠 + + + Strategic Outcome Replayâ„ĸ + Counterfactual behavior modeling + + + + {/* Practice Simulation */} + Alert.alert('Coming Soon', 'Practice Simulation Engine is under construction.')} + activeOpacity={0.8} + > + + ⚔ī¸ + + + Practice Simulation + Mock scenarios against AI + + - {/* Feature Cards Grid */} - - - navigation.navigate('Chat')} - /> - navigation.navigate('ToolCalling')} - /> - - - navigation.navigate('SpeechToText')} - /> - navigation.navigate('TextToSpeech')} - /> - - - navigation.navigate('VoicePipeline')} - /> - + {/* Recent Sessions */} + + + Recent Sessions + {sessions.length > 0 && ( + + See All + + )} + + {sessions.length === 0 ? ( + + + 📊 + + No sessions yet + + Start your first session to see insights here + + setShowModeSelector(true)} + > + + Start Session + + + + ) : ( + sessions.map((session) => ( + handleSessionPress(session.id)} + /> + )) + )} - {/* Model Info Section */} - - - 🤖 - LLM - - SmolLM2 360M + {/* Quick Stats */} + {stats && stats.totalSessions > 0 && ( + + Quick Stats + + + + đŸŽ¯ + + + {Math.round(stats.avgFocusScore)}% + + Avg Focus + + + + 📈 + + + {stats.totalPatterns} + + Patterns + + + + ⏱ī¸ + + + {formatDuration(stats.avgDuration)} + + Avg Duration + + + + 🔒 + + 100% + Private + + - - 🎤 - STT - - Whisper Tiny - - - 🔊 - TTS - - Piper TTS - - + )} + + {/* Floating Action Button */} + setShowModeSelector(true)} + activeOpacity={0.85} + > + + 🎤 + + + + {/* Bottom Navigation Bar */} + + + 🏠 + Home + + { + if (sessions.length > 0) { + handleSessionPress(sessions[0].id); + } + }} + > + 📊 + Insights + + + + 📁 + History + + navigation.navigate('Settings')} + > + 👤 + Profile + + + + {/* Mode Selector Modal */} + setShowModeSelector(false)} + > + + + + Select Live Mode Category + Live mode is recommended for permitted business and negotiation environments. Avoid usage where recording is restricted. + + + {allModes.map((modeConfig: ModeConfig) => ( + handleStartSession(modeConfig.mode)} + activeOpacity={0.7} + > + + {modeConfig.icon} + + + {modeConfig.displayName} + {modeConfig.description} + + â€ē + + ))} + + + setShowModeSelector(false)} + > + Cancel + + + + ); }; @@ -136,7 +375,7 @@ export const HomeScreen: React.FC = ({ navigation }) => { const styles = StyleSheet.create({ container: { flex: 1, - backgroundColor: AppColors.primaryDark, + backgroundColor: AppColors.primaryLight, }, gradient: { flex: 1, @@ -145,105 +384,417 @@ const styles = StyleSheet.create({ flex: 1, }, scrollContent: { - padding: 24, - paddingTop: 60, + paddingHorizontal: 24, + paddingTop: 56, + paddingBottom: 100, }, + + // Header header: { flexDirection: 'row', + justifyContent: 'space-between', alignItems: 'center', - marginBottom: 40, + marginBottom: 28, }, - logoContainer: { - marginRight: 16, + headerLeft: { + flexDirection: 'row', + alignItems: 'center', }, - logoGradient: { - width: 60, - height: 60, - borderRadius: 16, + avatar: { + width: 48, + height: 48, + borderRadius: 24, justifyContent: 'center', alignItems: 'center', - elevation: 8, - shadowColor: AppColors.accentCyan, - shadowOffset: { width: 0, height: 4 }, - shadowOpacity: 0.4, - shadowRadius: 12, + marginRight: 14, }, - logoIcon: { - fontSize: 32, + avatarText: { + fontSize: 20, + fontWeight: '700', + color: '#FFFFFF', }, - headerText: { - flex: 1, + headerTextContainer: { + justifyContent: 'center', }, - title: { - fontSize: 28, + greeting: { + fontSize: 14, + color: '#6B6B80', + fontWeight: '400', + marginBottom: 4, + }, + userName: { + fontSize: 18, fontWeight: '700', - color: AppColors.textPrimary, - letterSpacing: -0.5, + letterSpacing: 0.5, + color: '#1E1E2C', }, - subtitle: { - fontSize: 14, - fontWeight: '500', - color: AppColors.accentCyan, - marginTop: 2, + notificationButton: { + width: 44, + height: 44, + borderRadius: 22, + backgroundColor: 'rgba(255, 255, 255, 0.7)', + justifyContent: 'center', + alignItems: 'center', + borderWidth: 1, + borderColor: 'rgba(123, 97, 255, 0.08)', + }, + notificationIcon: { + fontSize: 20, + }, + + // Balance Card + balanceCard: { + borderRadius: 24, + padding: 28, + marginBottom: 28, + alignItems: 'center', + elevation: 12, + shadowColor: '#7B61FF', + shadowOffset: { width: 0, height: 10 }, + shadowOpacity: 0.35, + shadowRadius: 20, + }, + balanceLabel: { + fontSize: 13, + fontWeight: '600', + color: 'rgba(255, 255, 255, 0.7)', + letterSpacing: 2, + marginBottom: 12, + }, + balanceAmount: { + fontSize: 48, + fontWeight: '700', + color: '#FFFFFF', + letterSpacing: -1, + marginBottom: 14, + }, + balanceBadge: { + backgroundColor: 'rgba(255, 255, 255, 0.2)', + paddingHorizontal: 16, + paddingVertical: 6, + borderRadius: 20, + }, + balanceBadgeText: { + fontSize: 13, + fontWeight: '600', + color: '#FFFFFF', + }, + + actionColumn: { + flexDirection: 'column', + gap: 16, + marginBottom: 36, }, - privacyBanner: { + actionItemLarge: { flexDirection: 'row', + alignItems: 'center', + backgroundColor: '#FFFFFF', padding: 20, - backgroundColor: AppColors.surfaceCard + 'CC', - borderRadius: 16, - borderWidth: 1, - borderColor: AppColors.accentCyan + '33', - marginBottom: 32, + borderRadius: 24, + elevation: 4, + shadowColor: '#000', + shadowOffset: { width: 0, height: 4 }, + shadowOpacity: 0.05, + shadowRadius: 10, + }, + actionCircleLarge: { + width: 64, + height: 64, + borderRadius: 32, + justifyContent: 'center', + alignItems: 'center', + marginRight: 20, }, - privacyIcon: { + actionIconLarge: { fontSize: 28, - marginRight: 16, }, - privacyText: { + actionTextContent: { flex: 1, }, - privacyTitle: { - fontSize: 16, - fontWeight: '600', - color: AppColors.textPrimary, + actionLabelLarge: { + fontSize: 18, + fontWeight: '800', + color: '#1A1A2E', marginBottom: 4, }, - privacySubtitle: { - fontSize: 12, - color: AppColors.textSecondary, - lineHeight: 18, + actionSubLabelLarge: { + fontSize: 14, + color: '#8B949E', }, - gridContainer: { - marginBottom: 24, + + // Section + sessionsSection: { + marginBottom: 28, }, - row: { + sectionHeader: { flexDirection: 'row', + justifyContent: 'space-between', + alignItems: 'center', marginBottom: 16, - gap: 0, }, - infoSection: { - padding: 20, - backgroundColor: AppColors.surfaceCard + '80', + sectionTitle: { + fontSize: 18, + fontWeight: '700', + color: AppColors.textPrimary, + }, + seeAllText: { + fontSize: 14, + color: AppColors.accentPrimary, + fontWeight: '600', + }, + + // Empty State + emptyState: { + alignItems: 'center', + backgroundColor: '#FFFFFF', + borderRadius: 24, + padding: 36, + elevation: 2, + shadowColor: '#000', + shadowOffset: { width: 0, height: 2 }, + shadowOpacity: 0.06, + shadowRadius: 8, + }, + emptyIconCircle: { + width: 72, + height: 72, + borderRadius: 36, + backgroundColor: '#EDE9FE', + justifyContent: 'center', + alignItems: 'center', + marginBottom: 16, + }, + emptyIcon: { + fontSize: 32, + }, + emptyText: { + fontSize: 18, + fontWeight: '700', + color: AppColors.textPrimary, + marginBottom: 6, + }, + emptySubtext: { + fontSize: 14, + color: AppColors.textSecondary, + textAlign: 'center', + lineHeight: 20, + marginBottom: 20, + }, + emptyButton: { borderRadius: 16, - borderWidth: 1, - borderColor: AppColors.textMuted + '1A', + overflow: 'hidden', + }, + emptyButtonGradient: { + paddingHorizontal: 32, + paddingVertical: 14, + borderRadius: 16, + }, + emptyButtonText: { + fontSize: 15, + fontWeight: '700', + color: '#FFFFFF', + }, + + // Quick Stats + quickStatsSection: { + marginBottom: 20, }, - infoRow: { + quickStatsGrid: { flexDirection: 'row', + flexWrap: 'wrap', + gap: 12, + marginTop: 14, + }, + quickStatCard: { + width: (SCREEN_WIDTH - 60) / 2, + backgroundColor: '#FFFFFF', + borderRadius: 20, + padding: 18, + alignItems: 'center', + elevation: 2, + shadowColor: '#000', + shadowOffset: { width: 0, height: 2 }, + shadowOpacity: 0.05, + shadowRadius: 8, + }, + quickStatIcon: { + width: 48, + height: 48, + borderRadius: 16, + justifyContent: 'center', alignItems: 'center', + marginBottom: 12, + }, + quickStatEmoji: { + fontSize: 22, + }, + quickStatValue: { + fontSize: 22, + fontWeight: '700', + color: AppColors.textPrimary, + marginBottom: 4, + }, + quickStatLabel: { + fontSize: 12, + color: AppColors.textSecondary, + fontWeight: '500', + }, + + // FAB + fab: { + position: 'absolute', + bottom: 72, + alignSelf: 'center', + zIndex: 10, + elevation: 12, + shadowColor: '#7B61FF', + shadowOffset: { width: 0, height: 6 }, + shadowOpacity: 0.4, + shadowRadius: 12, + }, + fabGradient: { + width: 60, + height: 60, + borderRadius: 30, + justifyContent: 'center', + alignItems: 'center', + }, + fabIcon: { + fontSize: 26, + }, + + // Bottom Nav + bottomNav: { + flexDirection: 'row', + backgroundColor: '#FFFFFF', paddingVertical: 8, + paddingHorizontal: 16, + paddingBottom: 20, + borderTopLeftRadius: 24, + borderTopRightRadius: 24, + alignItems: 'center', + justifyContent: 'space-around', + elevation: 10, + shadowColor: '#000', + shadowOffset: { width: 0, height: -4 }, + shadowOpacity: 0.08, + shadowRadius: 12, }, - infoIcon: { - fontSize: 20, - marginRight: 12, + navItem: { + alignItems: 'center', + justifyContent: 'center', + flex: 1, + paddingVertical: 6, + }, + navItemSpacer: { + width: 60, + }, + navIcon: { + fontSize: 22, + marginBottom: 4, + opacity: 0.4, + }, + navIconActive: { + fontSize: 22, + marginBottom: 4, + }, + navLabel: { + fontSize: 10, + color: AppColors.textMuted, + fontWeight: '500', + }, + navLabelActive: { + fontSize: 10, + color: AppColors.accentPrimary, + fontWeight: '700', }, - infoLabel: { + + // Modal + modalOverlay: { + flex: 1, + backgroundColor: 'rgba(0, 0, 0, 0.4)', + justifyContent: 'flex-end', + }, + modalContent: { + backgroundColor: '#FFFFFF', + borderTopLeftRadius: 28, + borderTopRightRadius: 28, + padding: 24, + maxHeight: '80%', + }, + modalHandle: { + width: 40, + height: 4, + borderRadius: 2, + backgroundColor: '#E5E7EB', + alignSelf: 'center', + marginBottom: 20, + }, + modalTitle: { + fontSize: 24, + fontWeight: '700', + color: AppColors.textPrimary, + marginBottom: 6, + }, + modalSubtitle: { fontSize: 14, color: AppColors.textSecondary, + marginBottom: 24, }, - infoValue: { - fontSize: 12, - color: AppColors.accentCyan, - fontWeight: '500', + modesScroll: { + maxHeight: 400, + }, + modeCard: { + flexDirection: 'row', + alignItems: 'center', + backgroundColor: '#F8F5FF', + padding: 16, + borderRadius: 18, + marginBottom: 10, + borderWidth: 1, + borderColor: 'rgba(123, 97, 255, 0.08)', + }, + modeIconCircle: { + width: 48, + height: 48, + borderRadius: 16, + backgroundColor: '#EDE9FE', + justifyContent: 'center', + alignItems: 'center', + marginRight: 14, + }, + modeIcon: { + fontSize: 22, + }, + modeInfo: { + flex: 1, + }, + modeTitle: { + fontSize: 16, + fontWeight: '600', + color: AppColors.textPrimary, + marginBottom: 3, + }, + modeDescription: { + fontSize: 13, + color: AppColors.textSecondary, + lineHeight: 17, + }, + modeArrow: { + fontSize: 22, + color: AppColors.textMuted, + marginLeft: 8, + }, + cancelButton: { + marginTop: 14, + padding: 16, + backgroundColor: '#F3F4F6', + borderRadius: 16, + alignItems: 'center', + }, + cancelButtonText: { + fontSize: 16, + fontWeight: '600', + color: AppColors.textSecondary, }, }); diff --git a/src/screens/InsightsScreen.tsx b/src/screens/InsightsScreen.tsx new file mode 100644 index 0000000..e2c239f --- /dev/null +++ b/src/screens/InsightsScreen.tsx @@ -0,0 +1,252 @@ +import React, { useEffect, useState } from 'react'; +import { + View, Text, ScrollView, StyleSheet, TouchableOpacity, StatusBar, Alert, +} from 'react-native'; +import LinearGradient from 'react-native-linear-gradient'; +import { StackNavigationProp } from '@react-navigation/stack'; +import { RouteProp } from '@react-navigation/native'; +import { AppColors } from '../theme'; +import { RootStackParamList } from '../navigation/types'; +import { Session } from '../types/session'; +import { useSessionAnalyzer } from '../hooks/useSessionAnalyzer'; +import { getModeConfig, getPatternDefinition } from '../ai/patternLibrary'; +import { CognitiveMeter } from '../components/CognitiveMeter'; + +type InsightsScreenProps = { + navigation: StackNavigationProp; + route: RouteProp; +}; + +export const InsightsScreen: React.FC = ({ navigation, route }) => { + const { sessionId } = route.params; + const { getSession, deleteSession } = useSessionAnalyzer(); + const [session, setSession] = useState(null); + const [selectedTab, setSelectedTab] = useState<'transcript' | 'patterns' | 'analysis'>('analysis'); + + useEffect(() => { loadSession(); }, [sessionId]); + + const loadSession = async () => { + const s = await getSession(sessionId); + if (s) setSession(s); + else Alert.alert('Error', 'Session not found', [{ text: 'OK', onPress: () => navigation.goBack() }]); + }; + + const handleDelete = () => { + Alert.alert('Delete Session', 'This cannot be undone.', [ + { text: 'Cancel', style: 'cancel' }, + { text: 'Delete', style: 'destructive', onPress: async () => { await deleteSession(sessionId); navigation.goBack(); } }, + ]); + }; + + if (!session) return ( + + Loading session... + + ); + + const modeConfig = getModeConfig(session.mode); + const formatDate = (t: number) => new Date(t).toLocaleString('en-US', { month: 'long', day: 'numeric', year: 'numeric', hour: '2-digit', minute: '2-digit' }); + const formatDuration = (ms: number) => { const m = Math.floor(ms / 60000); const s = Math.floor((ms % 60000) / 1000); return `${m}m ${s}s`; }; + + return ( + + + + {/* Header */} + + + + {modeConfig.icon} + {modeConfig.displayName} + + + 🗑ī¸ + + + {formatDate(session.timestamp)} + + + Duration + {formatDuration(session.duration)} + + + Patterns + {session.detectedPatterns.length} + + + Words + {session.cognitiveMetrics.totalWords} + + + + + + + + {/* Tabs */} + + {(['analysis', 'patterns', 'transcript'] as const).map((tab) => ( + setSelectedTab(tab)}> + + {tab.charAt(0).toUpperCase() + tab.slice(1)} + + + ))} + + + {/* Content */} + + {selectedTab === 'analysis' && ( + + {session.summary.keyInsights.length > 0 && ( + + 💡 Key Insights + {session.summary.keyInsights.map((insight, i) => ( + + + {insight} + + ))} + + )} + {session.summary.tacticalSuggestions.length > 0 && ( + + đŸŽ¯ Tactical Suggestions + {session.summary.tacticalSuggestions.map((s, i) => ( + + {i + 1} + {s} + + ))} + + )} + {session.summary.leverageMoments.length > 0 && ( + + ✅ Leverage Moments + {session.summary.leverageMoments.map((m, i) => ( + {m} + ))} + + )} + {session.summary.missedOpportunities.length > 0 && ( + + ⚠ī¸ Missed Opportunities + {session.summary.missedOpportunities.map((o, i) => ( + {o} + ))} + + )} + + 🧠 Cognitive Metrics + + {session.cognitiveMetrics.speechGaps}Speech Gaps + {session.cognitiveMetrics.fillerWords}Filler Words + {Math.round(session.cognitiveMetrics.avgSpeechRate)}WPM + + + + )} + {selectedTab === 'patterns' && ( + + {session.detectedPatterns.map((p) => { + const d = getPatternDefinition(p.pattern); + return ( + + + {d.displayName} + {Math.round(p.confidenceScore)}% + + {d.description} + Suggestion:{p.suggestion} + {p.context && "{p.context}"} + + ); + })} + {session.detectedPatterns.length === 0 && No patterns detected} + + )} + {selectedTab === 'transcript' && ( + + {session.transcript.map((c) => ( + + {new Date(c.timestamp).toLocaleTimeString()} + {c.text} + + ))} + {session.transcript.length === 0 && No transcript available} + + )} + + + ); +}; + +const styles = StyleSheet.create({ + container: { flex: 1, backgroundColor: AppColors.primaryLight }, + centered: { justifyContent: 'center', alignItems: 'center' }, + loadingText: { fontSize: 16, color: AppColors.textSecondary }, + + header: { paddingTop: 50, paddingBottom: 24, paddingHorizontal: 20, borderBottomLeftRadius: 28, borderBottomRightRadius: 28 }, + headerTop: { flexDirection: 'row', justifyContent: 'space-between', alignItems: 'center', marginBottom: 8 }, + modeTag: { flexDirection: 'row', alignItems: 'center', backgroundColor: 'rgba(255,255,255,0.2)', paddingVertical: 6, paddingHorizontal: 12, borderRadius: 10 }, + modeIcon: { fontSize: 16, marginRight: 6 }, + modeText: { fontSize: 13, fontWeight: '600', color: '#FFFFFF' }, + deleteButton: { padding: 8, backgroundColor: 'rgba(255,255,255,0.15)', borderRadius: 10 }, + deleteIcon: { fontSize: 20 }, + dateText: { fontSize: 14, color: 'rgba(255,255,255,0.7)', marginBottom: 16 }, + + statsRow: { flexDirection: 'row', gap: 10, marginBottom: 20 }, + headerStatCard: { flex: 1, alignItems: 'center', backgroundColor: 'rgba(255,255,255,0.15)', borderRadius: 16, padding: 14 }, + headerStatLabel: { fontSize: 11, color: 'rgba(255,255,255,0.6)', marginBottom: 6, fontWeight: '500' }, + headerStatValue: { fontSize: 18, fontWeight: '700', color: '#FFFFFF' }, + focusSection: { alignItems: 'center' }, + + tabs: { flexDirection: 'row', backgroundColor: '#FFFFFF', paddingHorizontal: 4, paddingVertical: 4, marginHorizontal: 16, borderRadius: 16, marginTop: -6, marginBottom: 16, elevation: 3, shadowColor: '#000', shadowOffset: { width: 0, height: 2 }, shadowOpacity: 0.06, shadowRadius: 6 }, + tab: { flex: 1, paddingVertical: 10, alignItems: 'center', borderRadius: 12 }, + tabActive: { backgroundColor: '#7B61FF' }, + tabText: { fontSize: 14, fontWeight: '600', color: AppColors.textSecondary }, + tabTextActive: { color: '#FFFFFF' }, + + content: { flex: 1 }, + contentContainer: { padding: 16 }, + section: { marginBottom: 24 }, + sectionTitle: { fontSize: 18, fontWeight: '700', color: AppColors.textPrimary, marginBottom: 12 }, + + insightCard: { flexDirection: 'row', backgroundColor: '#FFFFFF', borderRadius: 16, marginBottom: 8, overflow: 'hidden', elevation: 2, shadowColor: '#000', shadowOffset: { width: 0, height: 1 }, shadowOpacity: 0.04, shadowRadius: 4 }, + insightAccent: { width: 4, backgroundColor: '#7B61FF' }, + insightText: { flex: 1, fontSize: 14, color: AppColors.textPrimary, lineHeight: 20, padding: 14 }, + + suggestionCard: { flexDirection: 'row', backgroundColor: '#FFFFFF', padding: 14, borderRadius: 16, marginBottom: 8, alignItems: 'flex-start', elevation: 2, shadowColor: '#000', shadowOffset: { width: 0, height: 1 }, shadowOpacity: 0.04, shadowRadius: 4 }, + suggestionNum: { width: 28, height: 28, borderRadius: 10, backgroundColor: '#EDE9FE', justifyContent: 'center', alignItems: 'center', marginRight: 12 }, + suggestionNumText: { fontSize: 14, fontWeight: '700', color: '#7B61FF' }, + suggestionText: { flex: 1, fontSize: 14, color: AppColors.textPrimary, lineHeight: 20 }, + + momentCard: { backgroundColor: '#DCFCE7', padding: 14, borderRadius: 14, marginBottom: 8 }, + momentText: { fontSize: 13, color: '#166534', lineHeight: 18 }, + oppCard: { backgroundColor: '#FEF3C7', padding: 14, borderRadius: 14, marginBottom: 8 }, + oppText: { fontSize: 13, color: '#92400E', lineHeight: 18 }, + + metricsGrid: { flexDirection: 'row', gap: 10 }, + metricCard: { flex: 1, backgroundColor: '#FFFFFF', padding: 18, borderRadius: 18, alignItems: 'center', elevation: 2, shadowColor: '#000', shadowOffset: { width: 0, height: 1 }, shadowOpacity: 0.04, shadowRadius: 4 }, + metricValue: { fontSize: 24, fontWeight: '700', color: '#7B61FF', marginBottom: 6 }, + metricLabel: { fontSize: 11, color: AppColors.textSecondary, textAlign: 'center', fontWeight: '500' }, + + patternCard: { backgroundColor: '#FFFFFF', padding: 16, borderRadius: 18, marginBottom: 12, elevation: 2, shadowColor: '#000', shadowOffset: { width: 0, height: 1 }, shadowOpacity: 0.04, shadowRadius: 4 }, + patternHeader: { flexDirection: 'row', justifyContent: 'space-between', alignItems: 'center', marginBottom: 8 }, + patternName: { fontSize: 16, fontWeight: '700', color: AppColors.textPrimary, flex: 1 }, + patternBadge: { backgroundColor: '#EDE9FE', paddingHorizontal: 10, paddingVertical: 4, borderRadius: 10 }, + patternBadgeText: { fontSize: 12, fontWeight: '700', color: '#7B61FF' }, + patternDesc: { fontSize: 13, color: AppColors.textSecondary, marginBottom: 10, lineHeight: 18 }, + patternSugg: { backgroundColor: '#F8F5FF', padding: 12, borderRadius: 12, marginBottom: 8 }, + patternSuggLabel: { fontSize: 11, fontWeight: '600', color: AppColors.textMuted, marginBottom: 4 }, + patternSuggText: { fontSize: 13, color: AppColors.textPrimary, lineHeight: 18 }, + patternCtx: { padding: 12, backgroundColor: '#F9FAFB', borderRadius: 12, borderLeftWidth: 3, borderLeftColor: '#7B61FF' }, + patternCtxText: { fontSize: 12, fontStyle: 'italic', color: AppColors.textSecondary, lineHeight: 16 }, + + tChunk: { marginBottom: 16, paddingBottom: 16, borderBottomWidth: 1, borderBottomColor: '#F3F4F6' }, + tTime: { fontSize: 12, color: AppColors.textMuted, marginBottom: 6, fontWeight: '500' }, + tText: { fontSize: 15, color: AppColors.textPrimary, lineHeight: 22 }, + + emptyState: { padding: 40, alignItems: 'center' }, + emptyText: { fontSize: 16, color: AppColors.textSecondary }, +}); diff --git a/src/screens/LiveSessionScreen.tsx b/src/screens/LiveSessionScreen.tsx new file mode 100644 index 0000000..abd9a6b --- /dev/null +++ b/src/screens/LiveSessionScreen.tsx @@ -0,0 +1,283 @@ +import React, { useEffect, useState } from 'react'; +import { View, Text, StyleSheet, TouchableOpacity, Alert, StatusBar } from 'react-native'; +import LinearGradient from 'react-native-linear-gradient'; +import { StackNavigationProp } from '@react-navigation/stack'; +import { RouteProp } from '@react-navigation/native'; +import { AppColors } from '../theme'; +import { RootStackParamList } from '../navigation/types'; +import { LiveTranscript } from '../components/LiveTranscript'; +import { SuggestionCard } from '../components/SuggestionCard'; +import { CounterStrategyCard } from '../components/CounterStrategyCard'; +import { CognitiveMeter } from '../components/CognitiveMeter'; +import { getModeConfig } from '../ai/patternLibrary'; +import { LocalStorageService } from '../services/LocalStorageService'; +import { useModelService } from '../services/ModelService'; +import { useLiveSession } from '../hooks/useLiveSession'; + +type LiveSessionScreenProps = { + navigation: StackNavigationProp; + route: RouteProp; +}; + +export const LiveSessionScreen: React.FC = ({ navigation, route }) => { + const { mode } = route.params; + + // ─── Single source of truth: useReducer-based session hook ─── + // Replaces useLiveTranscription + useCounterStrategy + const { state, isRecording, startSession, stopSession, cancelSession, error } = + useLiveSession(); + + const [hasStarted, setHasStarted] = useState(false); + const [modelReady, setModelReady] = useState(false); + const [modelError, setModelError] = useState(false); + const { downloadAndLoadSTT, isSTTLoaded, isSTTLoading, isSTTDownloading, isSDKReady, sttDownloadProgress } = useModelService(); + + const modeConfig = getModeConfig(mode); + + // ─── Suggestions and strategy from reducer state (no separate hooks) ─── + // The reducer handles all tactic detection, cooldown, and counter-strategy + // generation inside the TACTIC_DETECTED action. We just read the result. + const { activeStrategy, suggestions, detectedPatterns } = state; + + // Kick off model download+load when SDK is ready + useEffect(() => { + if (!isSDKReady) { + console.log('[LiveSessionScreen] âŗ Waiting for SDK to initialize...'); + return; + } + + const initModel = async () => { + console.log('[LiveSessionScreen] 🔍 Checking STT model status...'); + + // Check if debug mode is enabled + const settings = await LocalStorageService.getSettings(); + const debugMode = settings.debugMode || false; + + if (debugMode) { + console.log('[LiveSessionScreen] 🐛 Debug mode enabled, skipping STT model check'); + setModelReady(true); + return; + } + + // If already loaded, we're done + if (isSTTLoaded) { + console.log('[LiveSessionScreen] ✅ STT model already loaded'); + setModelReady(true); + return; + } + + // Kick off download+load (fire and forget — we'll watch isSTTLoaded state) + console.log('[LiveSessionScreen] 🤖 Starting STT model download and load...'); + downloadAndLoadSTT().catch((err) => { + console.error('[LiveSessionScreen] ❌ downloadAndLoadSTT error:', err); + setModelError(true); + }); + }; + + initModel(); + }, [isSDKReady]); // Only run once when SDK becomes ready + + // Watch for model loading completion from ModelService state + useEffect(() => { + if (isSTTLoaded && !modelReady) { + console.log('[LiveSessionScreen] ✅ STT model loaded (via ModelService state)'); + setModelReady(true); + setModelError(false); + } + }, [isSTTLoaded, modelReady]); + + // Watch for errors — show failure only when download/load finishes with error + // (not while still downloading) + useEffect(() => { + if (modelError && !isSTTDownloading && !isSTTLoading && !isSTTLoaded) { + Alert.alert( + 'Model Loading Failed', + 'The speech recognition model could not be loaded.\n\nOptions:\nâ€ĸ Enable Debug Mode in Settings to test without audio\nâ€ĸ Check your internet connection\nâ€ĸ Try again', + [ + { text: 'Go to Settings', onPress: () => navigation.navigate('Settings') }, + { + text: 'Try Again', + onPress: () => { + setModelError(false); + downloadAndLoadSTT().catch(() => setModelError(true)); + }, + }, + { text: 'Cancel', onPress: () => navigation.goBack() }, + ] + ); + } + }, [modelError, isSTTDownloading, isSTTLoading, isSTTLoaded, navigation, downloadAndLoadSTT]); + + useEffect(() => { + if (error) Alert.alert('Error', error, [{ text: 'OK' }]); + }, [error]); + + useEffect(() => { + if (!hasStarted && modelReady) { + setHasStarted(true); + startSession(mode).catch(() => navigation.goBack()); + } + }, [hasStarted, modelReady, mode, startSession, navigation]); + + const handleStop = () => { + Alert.alert('Stop Session', 'Your insights will be saved.', [ + { text: 'Cancel', style: 'cancel' }, + { + text: 'Stop', style: 'destructive', onPress: async () => { + const session = await stopSession(); + session ? navigation.navigate('Insights', { sessionId: session.id }) : navigation.goBack(); + }, + }, + ]); + }; + + const handleCancel = () => { + Alert.alert('Cancel Session', 'This session will not be saved.', [ + { text: 'No', style: 'cancel' }, + { + text: 'Yes, Cancel', style: 'destructive', onPress: async () => { + await cancelSession(); navigation.goBack(); + }, + }, + ]); + }; + + const formatDuration = (ms: number): string => { + const totalSeconds = Math.floor(ms / 1000); + const hours = Math.floor(totalSeconds / 3600); + const minutes = Math.floor((totalSeconds % 3600) / 60); + const seconds = totalSeconds % 60; + if (hours > 0) return `${hours}:${minutes.toString().padStart(2, '0')}:${seconds.toString().padStart(2, '0')}`; + return `${minutes}:${seconds.toString().padStart(2, '0')}`; + }; + + if (!modelReady) { + const loadingTitle = !isSDKReady + ? '🔧 Initializing AI Engine...' + : isSTTDownloading + ? `đŸ“Ĩ Downloading Model... ${Math.round(sttDownloadProgress)}%` + : isSTTLoading + ? '🤖 Loading AI Model...' + : '🤖 Preparing AI Model...'; + const loadingText = !isSDKReady + ? 'Setting up on-device AI' + : isSTTDownloading + ? 'First-time setup (~75MB)' + : isSTTLoading + ? 'Almost ready...' + : 'Preparing speech recognition'; + const loadingSubtext = !isSDKReady + ? 'This only takes a moment' + : isSTTDownloading + ? 'This only happens once' + : 'Please wait...'; + + return ( + + + + {loadingTitle} + {loadingText} + {loadingSubtext} + + + ); + } + + return ( + + + + {/* Top Bar */} + + + + + {modeConfig.icon} + {modeConfig.displayName} + + + + {formatDuration(state.duration)} + + + + + + + + + {/* Transcript — reads from reducer state */} + + + + + {/* Counter Strategy Card — only shown when tactic is detected */} + {activeStrategy && state.tactic != null && ( + + + + )} + + {/* Bottom Actions */} + + + Cancel + + + + ⏚ī¸ + Stop Session + + + + + {state.audioLevel > 0 && ( + + + + )} + + ); +}; + +const styles = StyleSheet.create({ + container: { flex: 1, backgroundColor: AppColors.primaryLight }, + loadingContainer: { flex: 1, justifyContent: 'center', alignItems: 'center', padding: 32 }, + loadingIconCircle: { width: 80, height: 80, borderRadius: 28, backgroundColor: '#EDE9FE', justifyContent: 'center', alignItems: 'center', marginBottom: 24 }, + loadingIcon: { fontSize: 36 }, + loadingTitle: { fontSize: 22, fontWeight: '700', color: AppColors.textPrimary, marginBottom: 8, textAlign: 'center' }, + loadingText: { fontSize: 15, color: AppColors.textSecondary, marginBottom: 32, textAlign: 'center' }, + loadingBarBg: { width: '60%', height: 6, borderRadius: 3, backgroundColor: '#EDE9FE', overflow: 'hidden' }, + loadingBar: { width: '45%', height: '100%', borderRadius: 3 }, + + topBar: { paddingTop: 50, paddingBottom: 16, paddingHorizontal: 20, backgroundColor: '#FFFFFF', borderBottomLeftRadius: 24, borderBottomRightRadius: 24, elevation: 4, shadowColor: '#000', shadowOffset: { width: 0, height: 2 }, shadowOpacity: 0.06, shadowRadius: 8 }, + topBarContent: { flexDirection: 'row', justifyContent: 'space-between', alignItems: 'center' }, + topBarLeft: { flex: 1 }, + modeTag: { flexDirection: 'row', alignItems: 'center', backgroundColor: '#EDE9FE', paddingVertical: 6, paddingHorizontal: 12, borderRadius: 10, alignSelf: 'flex-start', marginBottom: 12 }, + modeIconText: { fontSize: 16, marginRight: 6 }, + modeText: { fontSize: 13, fontWeight: '600', color: AppColors.accentPrimary }, + recordingIndicator: { flexDirection: 'row', alignItems: 'center' }, + recordingDot: { width: 12, height: 12, borderRadius: 6, backgroundColor: AppColors.error, marginRight: 8 }, + recordingText: { fontSize: 24, fontWeight: '700', color: AppColors.textPrimary, fontVariant: ['tabular-nums'] }, + topBarRight: { marginLeft: 16 }, + + transcriptContainer: { flex: 1, backgroundColor: AppColors.primaryLight, minHeight: 150 }, + + counterStrategyPanel: { backgroundColor: '#FFFFFF', paddingTop: 12, paddingBottom: 4, borderTopWidth: 1, borderTopColor: AppColors.accentViolet + '20', maxHeight: '45%' }, + + suggestionsPanel: { backgroundColor: '#FFFFFF', paddingVertical: 16, borderTopWidth: 1, borderTopColor: '#F3F4F6', maxHeight: 300 }, + suggestionsTitle: { fontSize: 16, fontWeight: '700', color: AppColors.textPrimary, marginHorizontal: 16, marginBottom: 12 }, + + bottomActions: { flexDirection: 'row', padding: 16, paddingBottom: 28, backgroundColor: '#FFFFFF', borderTopWidth: 1, borderTopColor: '#F3F4F6', gap: 12 }, + cancelButton: { flex: 1, padding: 16, backgroundColor: '#F3F4F6', borderRadius: 16, alignItems: 'center' }, + cancelButtonText: { fontSize: 16, fontWeight: '600', color: AppColors.textSecondary }, + stopButton: { flex: 2, borderRadius: 16, overflow: 'hidden', elevation: 4, shadowColor: AppColors.error, shadowOffset: { width: 0, height: 2 }, shadowOpacity: 0.3, shadowRadius: 6 }, + stopButtonGradient: { flexDirection: 'row', alignItems: 'center', justifyContent: 'center', padding: 16 }, + stopButtonIcon: { fontSize: 20, marginRight: 8 }, + stopButtonText: { fontSize: 16, fontWeight: '700', color: '#FFFFFF' }, + + audioVisualizer: { position: 'absolute', bottom: 0, left: 0, right: 0, height: 3, backgroundColor: '#EDE9FE' }, + audioBar: { height: '100%', borderRadius: 2 }, +}); diff --git a/src/screens/OutcomeReplayScreen.tsx b/src/screens/OutcomeReplayScreen.tsx new file mode 100644 index 0000000..4095f8c --- /dev/null +++ b/src/screens/OutcomeReplayScreen.tsx @@ -0,0 +1,263 @@ +import React, { useEffect, useState } from 'react'; +import { View, Text, StyleSheet, ScrollView, ActivityIndicator, TouchableOpacity } from 'react-native'; +import { RouteProp } from '@react-navigation/native'; +import { StackNavigationProp } from '@react-navigation/stack'; +import LinearGradient from 'react-native-linear-gradient'; +import { RootStackParamList } from '../navigation/types'; +import { AppColors } from '../theme'; +import { useSessionAnalyzer } from '../hooks/useSessionAnalyzer'; +import { OutcomeReplayEngine, ReplaySimulationResult } from '../ai/OutcomeReplayEngine'; +import { BehavioralAnalyticsEngine, BehavioralProfile } from '../ai/BehavioralAnalyticsEngine'; + +type OutcomeReplayScreenRouteProp = RouteProp; +type OutcomeReplayScreenNavigationProp = StackNavigationProp; + +interface OutcomeReplayScreenProps { + route: OutcomeReplayScreenRouteProp; + navigation: OutcomeReplayScreenNavigationProp; +} + +export const OutcomeReplayScreen: React.FC = ({ route, navigation }) => { + const { sessionId } = route.params; + const { getSession } = useSessionAnalyzer(); + const [loading, setLoading] = useState(true); + const [simulation, setSimulation] = useState(null); + const [profile, setProfile] = useState(null); + + useEffect(() => { + const loadEngines = async () => { + if (!sessionId) { + setLoading(false); + return; + } + const session = await getSession(sessionId); + if (session) { + setSimulation(OutcomeReplayEngine.generateSimulation(session)); + setProfile(BehavioralAnalyticsEngine.analyzeTranscript(session)); + } + setLoading(false); + }; + loadEngines(); + }, [sessionId, getSession]); + + if (loading || !simulation || !profile) { + return ( + + + Running Counterfactual Simulations... + + ); + } + + return ( + + + + {/* Header */} + + SESSION ANALYSIS + Strategic Outcome Replayâ„ĸ + + + {/* SECTION 3: Behavioral Performance Profile */} + + Behavioral Performance Profile + + + {profile.archetype.map((trait, index) => ( + + {trait} + + ))} + + + + + Leverage Score + 70 ? AppColors.success : AppColors.error }]}> + {profile.leverageCaptureScore}% + + + + Hesitations + {profile.hesitationMoments} + + + Filler Words + {profile.fillerWordCount} + + + + + + {/* SECTION 4: Post-Session Summary */} + + Tactical Post-Session Summary + + + What Worked + {simulation.postSessionSummary.whatWorked.map((item, idx) => ( + + ✓ + {item} + + ))} + + + + Signals of Interest + {simulation.postSessionSummary.signalsOfInterest.map((item, idx) => ( + + â€ĸ + {item} + + ))} + + + + Hidden Objections Detected + {simulation.postSessionSummary.hiddenObjections.map((item, idx) => ( + + ! + {item} + + ))} + + + + Follow-up Strategy + + {simulation.postSessionSummary.followUpStrategy} + + + + + {/* SECTION 1 & 2: Tactical Missed Opportunities & Improvement Simulation */} + + Tactical Counterfactuals + + {simulation.opportunities.length === 0 ? ( + + No major tactical errors detected. + + ) : ( + simulation.opportunities.map((opp, idx) => ( + + {/* Type Label */} + + {opp.tacticType.replace('_', ' ')} + + + {/* Original String */} + Original Response + "{opp.originalQuote}" + + {/* Improved Sim */} + + Improved Strategic Framing + "{opp.improvedReframing}" + + + Persuasion Strength: + + {opp.originalStrengthScore}% + {' → '} + {opp.improvedStrengthScore}% + + + + + )) + )} + + + {/* Next Session Reccomendations */} + navigation.navigate('Home')} + > + Return to Dashboard + + + + + + ); +}; + +const styles = StyleSheet.create({ + container: { flex: 1, backgroundColor: '#0A0E1A' }, + loadingContainer: { flex: 1, backgroundColor: '#0A0E1A', justifyContent: 'center', alignItems: 'center' }, + loadingText: { color: '#8B949E', marginTop: 16, fontSize: 16 }, + gradient: { flex: 1 }, + scrollView: { flex: 1 }, + scrollContent: { padding: 24, paddingBottom: 60, paddingTop: 60 }, + header: { marginBottom: 32 }, + subtitle: { color: '#8B949E', fontSize: 13, fontWeight: '700', letterSpacing: 1.5, marginBottom: 4 }, + title: { fontSize: 28, fontWeight: '800', color: '#FFFFFF', letterSpacing: 0.5 }, + section: { marginBottom: 36 }, + sectionHeader: { fontSize: 20, fontWeight: '700', color: '#E6EDF3', marginBottom: 16 }, + + // Profile + profileCard: { backgroundColor: '#161B22', borderRadius: 16, padding: 20, borderWidth: 1, borderColor: '#30363D' }, + archetypeContainer: { flexDirection: 'row', flexWrap: 'wrap', gap: 10, marginBottom: 20 }, + badge: { backgroundColor: 'rgba(123, 97, 255, 0.15)', paddingHorizontal: 12, paddingVertical: 6, borderRadius: 12, borderWidth: 1, borderColor: 'rgba(123, 97, 255, 0.3)' }, + badgeText: { color: '#B19CFF', fontSize: 13, fontWeight: '600' }, + statsRow: { flexDirection: 'row', justifyContent: 'space-between', borderTopWidth: 1, borderTopColor: '#30363D', paddingTop: 16 }, + statBox: { alignItems: 'center' }, + statLabel: { color: '#8B949E', fontSize: 12, marginBottom: 4 }, + statValue: { fontSize: 24, fontWeight: '800' }, + statValueAlt: { color: '#E6EDF3', fontSize: 24, fontWeight: '700' }, + + // Counterfactuals + simulationCard: { backgroundColor: '#161B22', borderRadius: 16, padding: 20, borderWidth: 1, borderColor: '#30363D', marginBottom: 16 }, + tacticLabelContainer: { alignSelf: 'flex-start', backgroundColor: '#30363D', paddingHorizontal: 10, paddingVertical: 4, borderRadius: 8, marginBottom: 16 }, + tacticLabelText: { color: '#C9D1D9', fontSize: 11, fontWeight: '700', textTransform: 'uppercase', letterSpacing: 1 }, + oppTitle: { color: '#8B949E', fontSize: 12, fontWeight: '700', textTransform: 'uppercase', marginBottom: 8 }, + originalQuote: { color: '#E6EDF3', fontSize: 16, fontStyle: 'italic', lineHeight: 24, marginBottom: 20 }, + improvedBox: { borderRadius: 12, padding: 16, borderWidth: 1, borderColor: 'rgba(16, 185, 129, 0.2)' }, + oppTitleGood: { color: '#10B981', fontSize: 12, fontWeight: '700', textTransform: 'uppercase', marginBottom: 8 }, + improvedQuote: { color: '#FFFFFF', fontSize: 16, fontWeight: '600', lineHeight: 24, marginBottom: 16 }, + deltaBox: { flexDirection: 'row', alignItems: 'center', justifyContent: 'space-between', backgroundColor: 'rgba(0,0,0,0.2)', padding: 10, borderRadius: 8 }, + deltaLabel: { color: '#8B949E', fontSize: 13, fontWeight: '600' }, + deltaScores: { fontSize: 16, fontWeight: '700' }, + deltaBad: { color: AppColors.error }, + deltaGood: { color: '#10B981' }, + + emptyCard: { padding: 20, backgroundColor: '#161B22', borderRadius: 12, alignItems: 'center' }, + emptyText: { color: '#8B949E' }, + + doneButton: { backgroundColor: AppColors.accentPrimary, borderRadius: 12, paddingVertical: 16, alignItems: 'center', marginTop: 10 }, + doneButtonText: { + color: '#FFFFFF', + fontSize: 16, + fontWeight: '700', + }, + bulletRow: { + flexDirection: 'row', + marginBottom: 8, + }, + bulletText: { + color: '#e0e0e0', + fontSize: 15, + flex: 1, + lineHeight: 22, + }, + divider: { + height: 1, + backgroundColor: '#333333', + marginVertical: 12, + }, + + // Summary Card + summaryCard: { backgroundColor: '#161B22', borderRadius: 16, padding: 20, borderWidth: 1, borderColor: '#30363D' }, + summaryLabel: { color: '#8B949E', fontSize: 13, fontWeight: '700', textTransform: 'uppercase', marginBottom: 12 }, + bulletPointSuccess: { color: '#10B981', fontSize: 16, marginRight: 10, marginTop: 1, fontWeight: '700' }, + bulletPointPrimary: { color: AppColors.accentPrimary, fontSize: 16, marginRight: 10, marginTop: 1, fontWeight: '700' }, + bulletPointWarning: { color: '#F59E0B', fontSize: 16, marginRight: 10, marginTop: 1, fontWeight: '700' }, + strategyBox: { borderRadius: 12, padding: 16, borderWidth: 1, borderColor: 'rgba(123, 97, 255, 0.3)' }, + strategyText: { color: '#FFFFFF', fontSize: 15, fontWeight: '600', lineHeight: 22 }, +}); diff --git a/src/screens/PreSessionFormScreen.tsx b/src/screens/PreSessionFormScreen.tsx new file mode 100644 index 0000000..cb8ba41 --- /dev/null +++ b/src/screens/PreSessionFormScreen.tsx @@ -0,0 +1,221 @@ +import React, { useState, useEffect } from 'react'; +import { + View, + Text, + StyleSheet, + ScrollView, + TextInput, + TouchableOpacity, + KeyboardAvoidingView, + Platform, + Alert, +} from 'react-native'; +import { RouteProp } from '@react-navigation/native'; +import { StackNavigationProp } from '@react-navigation/stack'; +import LinearGradient from 'react-native-linear-gradient'; +import { RootStackParamList } from '../navigation/types'; +import { AppColors } from '../theme'; +import { StrategicPreparationEngine, FormField } from '../ai/StrategicPreparationEngine'; +import { getModeConfig } from '../ai/patternLibrary'; +import { PreSessionInputs } from '../types/session'; + +type PreSessionFormRouteProp = RouteProp; +type PreSessionFormNavigationProp = StackNavigationProp; + +interface Props { + route: PreSessionFormRouteProp; + navigation: PreSessionFormNavigationProp; +} + +export const PreSessionFormScreen: React.FC = ({ route, navigation }) => { + const { mode } = route.params; + const modeConfig = getModeConfig(mode); + + const [fields, setFields] = useState([]); + const [formValues, setFormValues] = useState({}); + + useEffect(() => { + const generatedFields = StrategicPreparationEngine.getFormConfigForMode(mode); + setFields(generatedFields); + + // Initialize empty form state + const initialValues: PreSessionInputs = {}; + generatedFields.forEach(f => { + initialValues[f.id] = ''; + }); + setFormValues(initialValues); + }, [mode]); + + const handleInputChange = (id: string, text: string) => { + setFormValues(prev => ({ + ...prev, + [id]: text + })); + }; + + const handleGenerateStrategy = () => { + // Basic validation + const missingRequired = fields.filter(f => f.required && !formValues[f.id].trim()); + + if (missingRequired.length > 0) { + Alert.alert( + 'Missing Information', + `Please fill out: ${missingRequired.map(f => f.label).join(', ')}` + ); + return; + } + + // Generate strict 10-point analysis based on input + const analysis = StrategicPreparationEngine.generateStrategicAnalysis(mode, formValues); + + navigation.navigate('PreSessionStrategy', { + mode, + inputs: formValues, + analysis, + }); + }; + + return ( + + + + + + + {modeConfig.icon} + + STRATEGIC PREPARATION + {modeConfig.displayName} + + Fill out this briefing document to instantly generate your 10-point power positioning and tactical response strategy. + + + + + {fields.map((field) => ( + + + {field.label} + {!field.required && (Optional)} + + + handleInputChange(field.id, text)} + placeholder={field.placeholder} + placeholderTextColor="#4B5563" + multiline={field.type === 'multiline'} + numberOfLines={field.type === 'multiline' ? 3 : 1} + keyboardType={field.type === 'number' ? 'numeric' : 'default'} + /> + + ))} + + + + + Generate Tactical Strategy ⚡ + + + + navigation.replace('LiveSession', { mode })} + activeOpacity={0.7} + > + Skip Setup & Start Session 👉 + + + + + + ); +}; + +const styles = StyleSheet.create({ + container: { flex: 1, backgroundColor: '#0A0E1A' }, + gradient: { flex: 1 }, + scrollView: { flex: 1 }, + scrollContent: { padding: 24, paddingBottom: 60, paddingTop: 60 }, + + header: { marginBottom: 32, alignItems: 'center' }, + iconContainer: { + width: 64, height: 64, borderRadius: 32, + backgroundColor: 'rgba(123, 97, 255, 0.15)', + justifyContent: 'center', alignItems: 'center', + marginBottom: 16, borderWidth: 1, borderColor: 'rgba(123, 97, 255, 0.3)' + }, + headerIcon: { fontSize: 32 }, + subtitle: { color: AppColors.accentPrimary, fontSize: 13, fontWeight: '800', letterSpacing: 1.5, marginBottom: 8 }, + title: { fontSize: 32, fontWeight: '800', color: '#FFFFFF', marginBottom: 12, textAlign: 'center' }, + description: { fontSize: 15, color: '#8B949E', textAlign: 'center', lineHeight: 22 }, + + formContainer: { marginBottom: 20 }, + inputGroup: { marginBottom: 20 }, + labelRow: { flexDirection: 'row', alignItems: 'center', marginBottom: 8 }, + label: { color: '#E6EDF3', fontSize: 15, fontWeight: '600' }, + optionalText: { color: '#4B5563', fontSize: 13, marginLeft: 8 }, + + input: { + backgroundColor: '#161B22', + borderWidth: 1, + borderColor: '#30363D', + borderRadius: 12, + color: '#FFFFFF', + fontSize: 16, + paddingHorizontal: 16, + paddingVertical: 14, + }, + inputMultiline: { + minHeight: 100, + textAlignVertical: 'top', + }, + + generateButton: { + borderRadius: 16, + overflow: 'hidden', + marginTop: 10, + elevation: 8, + shadowColor: '#10B981', + shadowOffset: { width: 0, height: 4 }, + shadowOpacity: 0.3, + shadowRadius: 10, + }, + generateGradient: { + paddingVertical: 18, + alignItems: 'center', + justifyContent: 'center', + }, + generateButtonText: { + color: '#FFFFFF', + fontSize: 18, + fontWeight: '800', + }, + skipButton: { + paddingVertical: 16, + alignItems: 'center', + justifyContent: 'center', + marginTop: 8, + }, + skipButtonText: { + color: '#8B949E', + fontSize: 16, + fontWeight: '600', + }, +}); diff --git a/src/screens/PreSessionStrategyScreen.tsx b/src/screens/PreSessionStrategyScreen.tsx new file mode 100644 index 0000000..fca772c --- /dev/null +++ b/src/screens/PreSessionStrategyScreen.tsx @@ -0,0 +1,218 @@ +import React from 'react'; +import { View, Text, StyleSheet, ScrollView, TouchableOpacity } from 'react-native'; +import { RouteProp } from '@react-navigation/native'; +import { StackNavigationProp } from '@react-navigation/stack'; +import LinearGradient from 'react-native-linear-gradient'; +import { RootStackParamList } from '../navigation/types'; +import { AppColors } from '../theme'; +import { getModeConfig } from '../ai/patternLibrary'; + +type PreSessionStrategyRouteProp = RouteProp; +type PreSessionStrategyNavigationProp = StackNavigationProp; + +interface Props { + route: PreSessionStrategyRouteProp; + navigation: PreSessionStrategyNavigationProp; +} + +export const PreSessionStrategyScreen: React.FC = ({ route, navigation }) => { + const { mode, inputs, analysis } = route.params; + const modeConfig = getModeConfig(mode); + + const handleStartSession = () => { + // Navigate to actual Live Recording passing the pre-session analysis forward + navigation.replace('LiveSession', { + mode, + preSessionInputs: inputs, + strategicAnalysis: analysis + }); + }; + + return ( + + + + + + OFFLINE INTELLIGENCE MODE + Strategic Plan: {modeConfig.displayName} + + + {/* 1. Power Positioning summary */} + + 1. Power Positioning + + {analysis.powerPositioning} + + + + {/* 2. Opening Script */} + + 2. Opening Statement Script + + {analysis.openingScript} + + + + {/* 3 & 4. Objections & Responses */} + + 3. Likely Objections & Hard Counters + {analysis.likelyObjections.map((obj, i) => ( + + If they say: + {obj} + + + + Strategic Counter: + {analysis.recommendedResponses[i]} + + ))} + + + {/* 5. Target Phrases */} + + 4. High-Impact Phrases to Inject + + {analysis.highImpactPhrases.map((phrase, i) => ( + + {phrase} + + ))} + + + + {/* 6. Words to Avoid */} + + 5. Phrases to Avoid + + {analysis.phrasesToAvoid.map((phrase, i) => ( + + {phrase} + + ))} + + + + {/* 7. Psychological Tactics */} + + 6. Opponent's Psychological Tactics + + {analysis.psychologicalTactics.map((tactic, i) => ( + + â€ĸ + {tactic} + + ))} + + + + {/* 8. Confidence Triggers */} + + 7. Confidence Triggers + + {analysis.confidenceTriggers.map((tactic, i) => ( + + â€ĸ + {tactic} + + ))} + + + + {/* 9. Mistakes to Avoid */} + + 8. Fatal Mistakes To Avoid + + {analysis.mistakesToAvoid.map((mistake, i) => ( + + ! + {mistake} + + ))} + + + + {/* 10. Closing Script */} + + 9. Closing Script (To force commitment) + + {analysis.closingScript} + + + + {/* ACTION BUTTON */} + + + + Begin Target Session â€ē + Start local microphone tracking + + + + + + + + + ); +}; + +const styles = StyleSheet.create({ + container: { flex: 1, backgroundColor: '#0A0E1A' }, + gradient: { flex: 1 }, + scrollView: { flex: 1 }, + scrollContent: { padding: 24, paddingBottom: 60, paddingTop: 60 }, + + header: { marginBottom: 32 }, + subtitle: { color: AppColors.accentPrimary, fontSize: 13, fontWeight: '800', letterSpacing: 1.5, marginBottom: 8 }, + title: { fontSize: 32, fontWeight: '800', color: '#FFFFFF', marginBottom: 12 }, + + section: { marginBottom: 32 }, + sectionHeader: { fontSize: 18, fontWeight: '700', color: '#E6EDF3', marginBottom: 16 }, + + cardPrimary: { backgroundColor: '#161B22', borderRadius: 16, padding: 20, borderWidth: 1, borderColor: '#30363D' }, + cardTextPrimary: { color: '#FFFFFF', fontSize: 16, lineHeight: 24, fontWeight: '500' }, + + scriptBox: { borderRadius: 12, padding: 20, borderWidth: 1, borderColor: 'rgba(123, 97, 255, 0.3)' }, + scriptText: { color: '#FFFFFF', fontSize: 18, fontStyle: 'italic', fontWeight: 'bold', lineHeight: 26 }, + + objectionCard: { backgroundColor: '#161B22', padding: 20, borderRadius: 16, borderWidth: 1, borderColor: '#30363D', marginBottom: 16 }, + objectionLabel: { color: '#F87171', fontSize: 12, fontWeight: '700', textTransform: 'uppercase', marginBottom: 6 }, + objectionText: { color: '#FFFFFF', fontSize: 16, fontStyle: 'italic', marginBottom: 16 }, + divider: { height: 1, backgroundColor: '#30363D', marginBottom: 16 }, + responseLabel: { color: '#10B981', fontSize: 12, fontWeight: '700', textTransform: 'uppercase', marginBottom: 6 }, + responseText: { color: '#FFFFFF', fontSize: 16, fontWeight: '600', lineHeight: 24 }, + + wrapContainer: { flexDirection: 'row', flexWrap: 'wrap', gap: 10 }, + goodBadge: { backgroundColor: 'rgba(16, 185, 129, 0.1)', paddingHorizontal: 14, paddingVertical: 10, borderRadius: 8, borderWidth: 1, borderColor: 'rgba(16, 185, 129, 0.3)' }, + goodBadgeText: { color: '#10B981', fontSize: 14, fontWeight: '600' }, + + badBadge: { backgroundColor: 'rgba(239, 68, 68, 0.1)', paddingHorizontal: 14, paddingVertical: 10, borderRadius: 8, borderWidth: 1, borderColor: 'rgba(239, 68, 68, 0.3)' }, + badBadgeText: { color: '#F87171', fontSize: 14, fontWeight: '600', textDecorationLine: 'line-through' }, + + cardSecondary: { backgroundColor: '#161B22', borderRadius: 16, padding: 20, borderWidth: 1, borderColor: '#30363D' }, + cardError: { backgroundColor: 'rgba(239, 68, 68, 0.05)', borderRadius: 16, padding: 20, borderWidth: 1, borderColor: 'rgba(239, 68, 68, 0.2)' }, + + bulletRow: { flexDirection: 'row', alignItems: 'flex-start', marginBottom: 12 }, + bulletPoint: { color: '#8B949E', fontSize: 16, marginRight: 10, marginTop: 2 }, + bulletText: { color: '#C9D1D9', fontSize: 15, lineHeight: 22, flex: 1 }, + + errorBulletPoint: { color: '#F87171', fontSize: 16, marginRight: 10, fontWeight: 'bold' }, + errorText: { color: '#F87171', fontSize: 15, lineHeight: 22, flex: 1, fontWeight: '500' }, + + startButton: { borderRadius: 16, overflow: 'hidden', elevation: 8, shadowColor: '#E11D48', shadowOffset: { width: 0, height: 4 }, shadowOpacity: 0.3, shadowRadius: 10 }, + startGradient: { paddingVertical: 20, alignItems: 'center', justifyContent: 'center' }, + buttonStack: { alignItems: 'center' }, + startButtonText: { color: '#FFFFFF', fontSize: 20, fontWeight: '800', marginBottom: 4 }, + startButtonSubtext: { color: 'rgba(255, 255, 255, 0.7)', fontSize: 13, fontWeight: '600' } +}); diff --git a/src/screens/SettingsScreen.tsx b/src/screens/SettingsScreen.tsx new file mode 100644 index 0000000..7b1148f --- /dev/null +++ b/src/screens/SettingsScreen.tsx @@ -0,0 +1,220 @@ +import React, { useState, useEffect } from 'react'; +import { + View, Text, ScrollView, StyleSheet, TouchableOpacity, Alert, Switch, StatusBar, +} from 'react-native'; +import LinearGradient from 'react-native-linear-gradient'; +import { StackNavigationProp } from '@react-navigation/stack'; +import { AppColors } from '../theme'; +import { RootStackParamList } from '../navigation/types'; +import { LocalStorageService } from '../services/LocalStorageService'; +import { AppSettings, NegotiationMode } from '../types/session'; +import { getModeConfig } from '../ai/patternLibrary'; + +type SettingsScreenProps = { navigation: StackNavigationProp }; + +export const SettingsScreen: React.FC = ({ navigation }) => { + const [settings, setSettings] = useState(null); + const [storageInfo, setStorageInfo] = useState<{ keys: number; estimatedSize: string } | null>(null); + + useEffect(() => { loadSettings(); loadStorageInfo(); }, []); + const loadSettings = async () => { setSettings(await LocalStorageService.getSettings()); }; + const loadStorageInfo = async () => { setStorageInfo(await LocalStorageService.getStorageInfo()); }; + const saveSettings = async (s: AppSettings) => { await LocalStorageService.saveSettings(s); setSettings(s); }; + + const handleClearData = () => { + Alert.alert('Clear All Data', 'Delete all sessions? This cannot be undone.', [ + { text: 'Cancel', style: 'cancel' }, + { text: 'Delete All', style: 'destructive', onPress: async () => { await LocalStorageService.clearAllData(); await loadStorageInfo(); Alert.alert('Success', 'All data cleared'); } }, + ]); + }; + + if (!settings) return Loading...; + const modeConfig = getModeConfig(settings.defaultMode); + + return ( + + + + + + Settings + Customize your Latent experience + + + {/* Mode Section */} + + Default Mode + { + Alert.alert('Select Default Mode', 'Choose your default mode', + [NegotiationMode.JOB_INTERVIEW, NegotiationMode.SALES, NegotiationMode.STARTUP_PITCH, NegotiationMode.SALARY_RAISE] + .map((m) => ({ text: getModeConfig(m).displayName, onPress: () => saveSettings({ ...settings, defaultMode: m }) })) + ); + }}> + {modeConfig.icon} + + {modeConfig.displayName} + {modeConfig.description} + + â€ē + + + + {/* Pattern Detection */} + + Pattern Detection + + + Sensitivity + + {settings.patternSensitivity < 0.8 ? 'Low - Fewer false positives' : settings.patternSensitivity <= 1.2 ? 'Normal - Balanced' : 'High - More patterns'} + + + + {[{ label: 'Low', val: 0.7, check: settings.patternSensitivity <= 0.7 }, + { label: 'Normal', val: 1.0, check: settings.patternSensitivity === 1.0 }, + { label: 'High', val: 1.3, check: settings.patternSensitivity >= 1.3 }].map((b) => ( + saveSettings({ ...settings, patternSensitivity: b.val })}> + {b.label} + + ))} + + + + + {/* Session Settings */} + + Session Settings + {[ + { label: 'Auto-Save', desc: 'Saves session every 45 seconds', key: 'enableAutoSave' as const }, + { label: 'Haptic Feedback', desc: 'Vibrate on pattern detection', key: 'enableHapticFeedback' as const }, + { label: 'Suggestion Notifications', desc: 'Show notifications', key: 'enableSuggestionNotifications' as const }, + ].map((s) => ( + + + {s.label} + {s.desc} + + saveSettings({ ...settings, [s.key]: v })} + trackColor={{ false: '#E5E7EB', true: '#C4B5FD' }} thumbColor={settings[s.key] ? '#7B61FF' : '#D1D5DB'} /> + + ))} + + + {/* Debug */} + + 🐛 Debug & Testing + + + Debug Mode + Use test transcripts + + { + saveSettings({ ...settings, debugMode: v }); + if (v) Alert.alert('Debug Mode Enabled', 'Hardcoded transcripts will be used. Restart your session.', [{ text: 'Got it' }]); + }} trackColor={{ false: '#E5E7EB', true: '#FDE68A' }} thumbColor={settings.debugMode ? '#F59E0B' : '#D1D5DB'} /> + + {settings.debugMode && ( + ⚠ī¸ Debug mode active. Test transcripts injected every 7s. + )} + + + {/* Storage */} + + Storage + {storageInfo && ( + + Sessions{storageInfo.keys} + + Used{storageInfo.estimatedSize} + + )} + + Clear All Data + + + + {/* Privacy */} + + 🔒 + + 100% Private & Offline + All data stays on your device. No cloud. + + + + {/* About */} + + + L + + Latent + Offline Meeting Intelligence + Version 1.0.0 + + + + + ); +}; + +const styles = StyleSheet.create({ + container: { flex: 1, backgroundColor: AppColors.primaryLight }, + gradient: { flex: 1 }, + centered: { justifyContent: 'center', alignItems: 'center' }, + loadingText: { fontSize: 16, color: AppColors.textSecondary }, + scrollView: { flex: 1 }, + scrollContent: { padding: 24, paddingTop: 56, paddingBottom: 40 }, + + header: { marginBottom: 28 }, + title: { fontSize: 32, fontWeight: '700', color: AppColors.textPrimary, marginBottom: 4 }, + subtitle: { fontSize: 14, color: AppColors.textSecondary }, + + section: { marginBottom: 28 }, + sectionTitle: { fontSize: 16, fontWeight: '700', color: AppColors.textPrimary, marginBottom: 14 }, + + modeButton: { flexDirection: 'row', alignItems: 'center', backgroundColor: '#FFFFFF', padding: 16, borderRadius: 18, elevation: 2, shadowColor: '#000', shadowOffset: { width: 0, height: 1 }, shadowOpacity: 0.05, shadowRadius: 4 }, + modeIconCircle: { width: 48, height: 48, borderRadius: 16, backgroundColor: '#EDE9FE', justifyContent: 'center', alignItems: 'center', marginRight: 14 }, + modeIcon: { fontSize: 24 }, + modeInfo: { flex: 1 }, + modeName: { fontSize: 16, fontWeight: '600', color: AppColors.textPrimary, marginBottom: 3 }, + modeDesc: { fontSize: 13, color: AppColors.textSecondary, lineHeight: 17 }, + chevron: { fontSize: 24, color: AppColors.textMuted }, + + settingRow: { flexDirection: 'row', justifyContent: 'space-between', alignItems: 'center', backgroundColor: '#FFFFFF', padding: 16, borderRadius: 18, marginBottom: 10, elevation: 2, shadowColor: '#000', shadowOffset: { width: 0, height: 1 }, shadowOpacity: 0.04, shadowRadius: 4 }, + settingInfo: { flex: 1, marginRight: 16 }, + settingLabel: { fontSize: 15, fontWeight: '600', color: AppColors.textPrimary, marginBottom: 3 }, + settingDesc: { fontSize: 12, color: AppColors.textSecondary, lineHeight: 16 }, + + sensitivityBtns: { flexDirection: 'row', gap: 6 }, + sensBtn: { paddingHorizontal: 14, paddingVertical: 8, borderRadius: 12, backgroundColor: '#F3F4F6' }, + sensBtnActive: { backgroundColor: '#7B61FF' }, + sensBtnText: { fontSize: 12, fontWeight: '600', color: AppColors.textSecondary }, + sensBtnTextActive: { color: '#FFFFFF' }, + + storageCard: { backgroundColor: '#FFFFFF', padding: 16, borderRadius: 18, marginBottom: 14, elevation: 2, shadowColor: '#000', shadowOffset: { width: 0, height: 1 }, shadowOpacity: 0.04, shadowRadius: 4 }, + storageRow: { flexDirection: 'row', justifyContent: 'space-between', paddingVertical: 10 }, + storageDivider: { height: 1, backgroundColor: '#F3F4F6' }, + storageLabel: { fontSize: 14, color: AppColors.textSecondary }, + storageValue: { fontSize: 14, fontWeight: '600', color: AppColors.textPrimary }, + + dangerButton: { backgroundColor: '#FEE2E2', padding: 16, borderRadius: 16, alignItems: 'center' }, + dangerButtonText: { fontSize: 15, fontWeight: '600', color: '#DC2626' }, + + debugNotice: { backgroundColor: '#FEF3C7', padding: 14, borderRadius: 14, marginTop: 10, borderLeftWidth: 3, borderLeftColor: '#F59E0B' }, + debugNoticeText: { fontSize: 12, color: '#92400E', lineHeight: 16 }, + + privacyCard: { flexDirection: 'row', backgroundColor: '#FFFFFF', padding: 18, borderRadius: 20, marginBottom: 28, alignItems: 'center', elevation: 2, shadowColor: '#000', shadowOffset: { width: 0, height: 1 }, shadowOpacity: 0.05, shadowRadius: 6 }, + privacyIconCircle: { width: 44, height: 44, borderRadius: 14, backgroundColor: '#EDE9FE', justifyContent: 'center', alignItems: 'center', marginRight: 14 }, + privacyIcon: { fontSize: 22 }, + privacyText: { flex: 1 }, + privacyTitle: { fontSize: 15, fontWeight: '600', color: AppColors.textPrimary, marginBottom: 3 }, + privacyDesc: { fontSize: 12, color: AppColors.textSecondary, lineHeight: 17 }, + + aboutSection: { alignItems: 'center', paddingTop: 8, paddingBottom: 20 }, + aboutLogo: { width: 56, height: 56, borderRadius: 18, justifyContent: 'center', alignItems: 'center', marginBottom: 12, elevation: 4, shadowColor: '#7B61FF', shadowOffset: { width: 0, height: 3 }, shadowOpacity: 0.3, shadowRadius: 8 }, + aboutLogoText: { fontSize: 24, fontWeight: '700', color: '#FFFFFF' }, + aboutTitle: { fontSize: 22, fontWeight: '700', color: AppColors.textPrimary, marginBottom: 4 }, + aboutSubtitle: { fontSize: 14, color: '#7B61FF', marginBottom: 8 }, + aboutVersion: { fontSize: 12, color: AppColors.textMuted }, +}); diff --git a/src/screens/SpeechToTextScreen.tsx b/src/screens/SpeechToTextScreen.tsx deleted file mode 100644 index ab1aad7..0000000 --- a/src/screens/SpeechToTextScreen.tsx +++ /dev/null @@ -1,425 +0,0 @@ -import React, { useState, useRef, useEffect } from 'react'; -import { - View, - Text, - TouchableOpacity, - ScrollView, - StyleSheet, - NativeModules, - Alert, - Platform, - PermissionsAndroid, -} from 'react-native'; -import LinearGradient from 'react-native-linear-gradient'; -import { RunAnywhere } from '@runanywhere/core'; -import { AppColors } from '../theme'; -import { useModelService } from '../services/ModelService'; -import { ModelLoaderWidget, AudioVisualizer } from '../components'; - -// Native Audio Module - records in WAV format (16kHz mono) optimal for Whisper STT -const { NativeAudioModule } = NativeModules; - -export const SpeechToTextScreen: React.FC = () => { - const modelService = useModelService(); - const [isRecording, setIsRecording] = useState(false); - const [isTranscribing, setIsTranscribing] = useState(false); - const [transcription, setTranscription] = useState(''); - const [transcriptionHistory, setTranscriptionHistory] = useState([]); - const [audioLevel, setAudioLevel] = useState(0); - const [recordingDuration, setRecordingDuration] = useState(0); - const recordingPathRef = useRef(null); - const audioLevelIntervalRef = useRef | null>(null); - const recordingStartRef = useRef(0); - - // Cleanup on unmount - useEffect(() => { - return () => { - if (audioLevelIntervalRef.current) { - clearInterval(audioLevelIntervalRef.current); - } - if (isRecording && NativeAudioModule) { - NativeAudioModule.cancelRecording().catch(() => {}); - } - }; - }, [isRecording]); - - const startRecording = async () => { - try { - // Check if native module is available - if (!NativeAudioModule) { - console.error('[STT] NativeAudioModule not available'); - Alert.alert('Error', 'Native audio module not available. Please rebuild the app.'); - return; - } - - // Request microphone permission on Android - if (Platform.OS === 'android') { - const granted = await PermissionsAndroid.request( - PermissionsAndroid.PERMISSIONS.RECORD_AUDIO, - { - title: 'Microphone Permission', - message: 'This app needs access to your microphone for speech recognition.', - buttonNeutral: 'Ask Me Later', - buttonNegative: 'Cancel', - buttonPositive: 'OK', - } - ); - if (granted !== PermissionsAndroid.RESULTS.GRANTED) { - Alert.alert('Permission Denied', 'Microphone permission is required for speech recognition.'); - return; - } - } - - console.warn('[STT] Starting native recording...'); - const result = await NativeAudioModule.startRecording(); - - recordingPathRef.current = result.path; - recordingStartRef.current = Date.now(); - setIsRecording(true); - setTranscription(''); - setRecordingDuration(0); - - // Poll for audio levels - audioLevelIntervalRef.current = setInterval(async () => { - try { - const levelResult = await NativeAudioModule.getAudioLevel(); - setAudioLevel(levelResult.level || 0); - setRecordingDuration(Date.now() - recordingStartRef.current); - } catch (e) { - // Ignore errors during polling - } - }, 100); - - console.warn('[STT] Recording started at:', result.path); - } catch (error) { - console.error('[STT] Recording error:', error); - Alert.alert('Recording Error', `Failed to start recording: ${error}`); - } - }; - - const stopRecordingAndTranscribe = async () => { - try { - // Clear audio level polling - if (audioLevelIntervalRef.current) { - clearInterval(audioLevelIntervalRef.current); - audioLevelIntervalRef.current = null; - } - - if (!NativeAudioModule) { - throw new Error('NativeAudioModule not available'); - } - - console.warn('[STT] Stopping recording...'); - const result = await NativeAudioModule.stopRecording(); - setIsRecording(false); - setAudioLevel(0); - setIsTranscribing(true); - - // Get the base64 audio data directly from native module (bypasses RNFS sandbox issues) - const audioBase64 = result.audioBase64; - if (!audioBase64) { - throw new Error('No audio data received from recording'); - } - - console.warn('[STT] Recording stopped, audio base64 length:', audioBase64.length, 'file size:', result.fileSize); - - if (result.fileSize < 1000) { - throw new Error('Recording too short - please speak longer'); - } - - // Check if STT model is loaded - const isModelLoaded = await RunAnywhere.isSTTModelLoaded(); - if (!isModelLoaded) { - throw new Error('STT model not loaded. Please download and load the model first.'); - } - - // Transcribe using base64 audio data directly from native module - console.warn('[STT] Starting transcription...'); - const transcribeResult = await RunAnywhere.transcribe(audioBase64, { - sampleRate: 16000, - language: 'en', - }); - - console.warn('[STT] Transcription result:', transcribeResult); - - if (transcribeResult.text) { - setTranscription(transcribeResult.text); - setTranscriptionHistory(prev => [transcribeResult.text, ...prev]); - } else { - setTranscription('(No speech detected)'); - } - - recordingPathRef.current = null; - setIsTranscribing(false); - } catch (error) { - console.error('[STT] Transcription error:', error); - const errorMessage = error instanceof Error ? error.message : String(error); - setTranscription(`Error: ${errorMessage}`); - Alert.alert('Transcription Error', errorMessage); - setIsTranscribing(false); - } - }; - - const handleClearHistory = () => { - setTranscriptionHistory([]); - setTranscription(''); - }; - - const formatDuration = (ms: number): string => { - const totalSeconds = Math.floor(ms / 1000); - const minutes = Math.floor(totalSeconds / 60); - const seconds = totalSeconds % 60; - return `${minutes}:${seconds.toString().padStart(2, '0')}`; - }; - - if (!modelService.isSTTLoaded) { - return ( - - ); - } - - return ( - - - {/* Recording Area */} - - {isRecording ? ( - <> - - - Listening... - - - {formatDuration(recordingDuration)} - - - ) : isTranscribing ? ( - <> - - âŗ - - Transcribing... - - ) : ( - <> - - 🎤 - - Tap to Record - On-device speech recognition (WAV 16kHz) - - )} - - - {/* Current Transcription */} - {(transcription || isTranscribing) && ( - - - LATEST - - - {isTranscribing ? 'Processing...' : transcription} - - - )} - - {/* History */} - {transcriptionHistory.length > 0 && ( - - - History - - Clear - - - {transcriptionHistory.map((item, index) => ( - - {item} - - ))} - - )} - - - {/* Record Button */} - - - - {isRecording ? '⏚' : '🎤'} - - {isRecording ? 'Stop Recording' : 'Start Recording'} - - - - - - ); -}; - -const styles = StyleSheet.create({ - container: { - flex: 1, - backgroundColor: AppColors.primaryDark, - }, - scrollView: { - flex: 1, - }, - scrollContent: { - padding: 24, - }, - recordingArea: { - padding: 32, - backgroundColor: AppColors.surfaceCard, - borderRadius: 24, - borderWidth: 1, - borderColor: AppColors.textMuted + '1A', - alignItems: 'center', - marginBottom: 24, - }, - recordingActive: { - borderColor: AppColors.accentViolet + '80', - borderWidth: 2, - shadowColor: AppColors.accentViolet, - shadowOffset: { width: 0, height: 0 }, - shadowOpacity: 0.3, - shadowRadius: 20, - elevation: 8, - }, - micContainer: { - width: 100, - height: 100, - backgroundColor: AppColors.accentViolet + '20', - borderRadius: 50, - justifyContent: 'center', - alignItems: 'center', - marginBottom: 24, - }, - micIcon: { - fontSize: 48, - }, - loadingContainer: { - width: 80, - height: 80, - justifyContent: 'center', - alignItems: 'center', - marginBottom: 24, - }, - loadingIcon: { - fontSize: 48, - }, - statusTitle: { - fontSize: 20, - fontWeight: '700', - color: AppColors.textPrimary, - marginBottom: 8, - }, - statusSubtitle: { - fontSize: 14, - color: AppColors.textSecondary, - }, - transcriptionCard: { - padding: 20, - backgroundColor: AppColors.surfaceCard, - borderRadius: 16, - borderWidth: 1, - borderColor: AppColors.accentViolet + '40', - marginBottom: 24, - }, - badge: { - alignSelf: 'flex-start', - paddingHorizontal: 8, - paddingVertical: 4, - backgroundColor: AppColors.accentViolet + '33', - borderRadius: 8, - marginBottom: 12, - }, - badgeText: { - fontSize: 10, - fontWeight: '700', - color: AppColors.accentViolet, - }, - transcriptionText: { - fontSize: 15, - color: AppColors.textPrimary, - lineHeight: 22, - }, - historySection: { - marginBottom: 24, - }, - historyHeader: { - flexDirection: 'row', - justifyContent: 'space-between', - alignItems: 'center', - marginBottom: 12, - }, - historyTitle: { - fontSize: 16, - fontWeight: '600', - color: AppColors.textMuted, - }, - clearButton: { - fontSize: 14, - color: AppColors.accentViolet, - }, - historyItem: { - padding: 16, - backgroundColor: AppColors.surfaceCard + '80', - borderRadius: 12, - borderWidth: 1, - borderColor: AppColors.textMuted + '1A', - marginBottom: 12, - }, - historyText: { - fontSize: 14, - color: AppColors.textSecondary, - lineHeight: 20, - }, - buttonContainer: { - padding: 24, - backgroundColor: AppColors.surfaceCard + 'CC', - borderTopWidth: 1, - borderTopColor: AppColors.textMuted + '1A', - }, - recordButton: { - flexDirection: 'row', - height: 72, - borderRadius: 36, - justifyContent: 'center', - alignItems: 'center', - gap: 12, - elevation: 8, - shadowColor: AppColors.accentViolet, - shadowOffset: { width: 0, height: 8 }, - shadowOpacity: 0.4, - shadowRadius: 20, - }, - recordIcon: { - fontSize: 28, - }, - recordButtonText: { - fontSize: 16, - fontWeight: '700', - color: '#FFFFFF', - }, -}); diff --git a/src/screens/TextToSpeechScreen.tsx b/src/screens/TextToSpeechScreen.tsx deleted file mode 100644 index 21c0588..0000000 --- a/src/screens/TextToSpeechScreen.tsx +++ /dev/null @@ -1,451 +0,0 @@ -import React, { useState, useEffect } from 'react'; -import { - View, - Text, - TextInput, - TouchableOpacity, - ScrollView, - StyleSheet, - NativeModules, -} from 'react-native'; -import LinearGradient from 'react-native-linear-gradient'; -import RNFS from 'react-native-fs'; -import { RunAnywhere } from '@runanywhere/core'; -import { AppColors } from '../theme'; -import { useModelService } from '../services/ModelService'; -import { ModelLoaderWidget } from '../components'; - -// Native Audio Module for better audio session management -const { NativeAudioModule } = NativeModules; - -const SAMPLE_TEXTS = [ - 'Hello! Welcome to RunAnywhere. Experience the power of on-device AI.', - 'The quick brown fox jumps over the lazy dog.', - 'Technology is best when it brings people together.', - 'Privacy is not something that I am merely entitled to, it is an absolute prerequisite.', -]; - -export const TextToSpeechScreen: React.FC = () => { - const modelService = useModelService(); - const [text, setText] = useState(''); - const [isSynthesizing, setIsSynthesizing] = useState(false); - const [isPlaying, setIsPlaying] = useState(false); - const [speechRate, setSpeechRate] = useState(1.0); - const [currentAudioPath, setCurrentAudioPath] = useState(null); - - // Cleanup on unmount - useEffect(() => { - return () => { - if (NativeAudioModule && isPlaying) { - NativeAudioModule.stopPlayback().catch(() => {}); - } - }; - }, [isPlaying]); - - const synthesizeAndPlay = async () => { - if (!text.trim()) { - return; - } - - setIsSynthesizing(true); - - try { - // Per docs: https://docs.runanywhere.ai/react-native/tts/synthesize - // result.audio contains base64-encoded float32 PCM - // Using same config as sample app for consistent voice output - const result = await RunAnywhere.synthesize(text, { - voice: 'default', - rate: speechRate, - pitch: 1.0, - volume: 1.0, - }); - - console.log(`[TTS] Synthesized: duration=${result.duration}s, sampleRate=${result.sampleRate}Hz, numSamples=${result.numSamples}`); - - // Use SDK's built-in WAV converter (same as sample app) - const tempPath = await RunAnywhere.Audio.createWavFromPCMFloat32( - result.audio, - result.sampleRate || 22050 - ); - - console.log(`[TTS] WAV file created: ${tempPath}`); - - setCurrentAudioPath(tempPath); - setIsSynthesizing(false); - setIsPlaying(true); - - // Play using native audio module - if (NativeAudioModule) { - try { - const playResult = await NativeAudioModule.playAudio(tempPath); - console.log(`[TTS] Playback started, duration: ${playResult.duration}s`); - - // Wait for playback to complete (approximate based on duration) - setTimeout(() => { - setIsPlaying(false); - setCurrentAudioPath(null); - // Clean up file - RNFS.unlink(tempPath).catch(() => {}); - }, (result.duration + 0.5) * 1000); - } catch (playError) { - console.error('[TTS] Native playback error:', playError); - setIsPlaying(false); - } - } else { - console.error('[TTS] NativeAudioModule not available'); - setIsPlaying(false); - } - } catch (error) { - console.error('[TTS] Error:', error); - setIsSynthesizing(false); - setIsPlaying(false); - } - }; - - const stopPlayback = async () => { - if (NativeAudioModule) { - try { - await NativeAudioModule.stopPlayback(); - } catch (e) { - // Ignore - } - } - setIsPlaying(false); - - // Clean up file - if (currentAudioPath) { - RNFS.unlink(currentAudioPath).catch(() => {}); - setCurrentAudioPath(null); - } - }; - - if (!modelService.isTTSLoaded) { - return ( - - ); - } - - return ( - - - {/* Input Section */} - - - - - 📝 {text.length} characters - - {text.length > 0 && ( - setText('')}> - Clear - - )} - - - - {/* Controls */} - - Speech Rate - - 🐌 - {speechRate.toFixed(1)}x - 🚀 - - - {[0.5, 0.75, 1.0, 1.5, 2.0].map((rate) => ( - setSpeechRate(rate)} - style={[ - styles.rateButton, - speechRate === rate && styles.rateButtonActive, - ]} - > - - {rate}x - - - ))} - - - - {/* Playback Area */} - - {isPlaying ? ( - <> - - {[...Array(7)].map((_, i) => ( - - ))} - - Playing... - - ) : isSynthesizing ? ( - <> - âŗ - Synthesizing... - - ) : ( - <> - 🔊 - Tap to synthesize - - )} - - {/* Play Button */} - - - - {isSynthesizing ? 'âŗ' : isPlaying ? '⏚' : 'â–ļī¸'} - - - - - - {/* Sample Texts */} - - Sample Texts - {SAMPLE_TEXTS.map((sample, index) => ( - setText(sample)} - style={styles.sampleItem} - > - - {sample} - - ➕ - - ))} - - - - ); -}; - -const styles = StyleSheet.create({ - container: { - flex: 1, - backgroundColor: AppColors.primaryDark, - }, - scrollView: { - flex: 1, - }, - scrollContent: { - padding: 24, - }, - inputCard: { - backgroundColor: AppColors.surfaceCard, - borderRadius: 20, - borderWidth: 1, - borderColor: AppColors.accentPink + '33', - marginBottom: 24, - overflow: 'hidden', - }, - input: { - padding: 20, - fontSize: 15, - color: AppColors.textPrimary, - minHeight: 120, - textAlignVertical: 'top', - }, - inputFooter: { - flexDirection: 'row', - justifyContent: 'space-between', - alignItems: 'center', - padding: 12, - paddingHorizontal: 16, - backgroundColor: AppColors.primaryMid, - }, - characterCount: { - fontSize: 12, - color: AppColors.textMuted, - }, - clearText: { - fontSize: 14, - color: AppColors.accentPink, - fontWeight: '600', - }, - controlsCard: { - padding: 20, - backgroundColor: AppColors.surfaceCard, - borderRadius: 16, - borderWidth: 1, - borderColor: AppColors.textMuted + '1A', - marginBottom: 24, - }, - controlLabel: { - fontSize: 16, - fontWeight: '600', - color: AppColors.textPrimary, - marginBottom: 16, - }, - sliderContainer: { - flexDirection: 'row', - alignItems: 'center', - justifyContent: 'space-between', - marginBottom: 16, - }, - sliderIcon: { - fontSize: 20, - }, - sliderValue: { - fontSize: 18, - fontWeight: '700', - color: AppColors.accentPink, - paddingHorizontal: 16, - paddingVertical: 8, - backgroundColor: AppColors.accentPink + '20', - borderRadius: 12, - }, - rateButtons: { - flexDirection: 'row', - gap: 8, - }, - rateButton: { - flex: 1, - paddingVertical: 8, - backgroundColor: AppColors.surfaceElevated, - borderRadius: 8, - alignItems: 'center', - }, - rateButtonActive: { - backgroundColor: AppColors.accentPink + '40', - }, - rateButtonText: { - fontSize: 14, - color: AppColors.textSecondary, - fontWeight: '600', - }, - rateButtonTextActive: { - color: AppColors.accentPink, - }, - playbackArea: { - padding: 24, - backgroundColor: AppColors.surfaceCard, - borderRadius: 20, - borderWidth: 1, - borderColor: AppColors.textMuted + '1A', - alignItems: 'center', - marginBottom: 32, - }, - playbackActive: { - borderColor: AppColors.accentPink + '80', - borderWidth: 2, - shadowColor: AppColors.accentPink, - shadowOffset: { width: 0, height: 0 }, - shadowOpacity: 0.3, - shadowRadius: 20, - elevation: 8, - }, - waveform: { - flexDirection: 'row', - height: 60, - alignItems: 'center', - justifyContent: 'center', - gap: 6, - marginBottom: 24, - }, - waveBar: { - width: 6, - height: 40, - backgroundColor: AppColors.accentPink, - borderRadius: 3, - }, - playbackIcon: { - fontSize: 48, - marginBottom: 16, - }, - loadingIcon: { - fontSize: 48, - marginBottom: 16, - }, - playbackStatus: { - fontSize: 14, - color: AppColors.textSecondary, - marginBottom: 24, - }, - playButtonWrapper: { - marginTop: 8, - }, - playButton: { - width: 80, - height: 80, - borderRadius: 40, - justifyContent: 'center', - alignItems: 'center', - elevation: 8, - shadowColor: AppColors.accentPink, - shadowOffset: { width: 0, height: 8 }, - shadowOpacity: 0.4, - shadowRadius: 20, - }, - playButtonIcon: { - fontSize: 32, - }, - samplesSection: { - marginBottom: 24, - }, - samplesTitle: { - fontSize: 16, - fontWeight: '600', - color: AppColors.textMuted, - marginBottom: 12, - }, - sampleItem: { - flexDirection: 'row', - alignItems: 'center', - padding: 16, - backgroundColor: AppColors.surfaceCard + '80', - borderRadius: 12, - borderWidth: 1, - borderColor: AppColors.textMuted + '1A', - marginBottom: 12, - }, - sampleText: { - flex: 1, - fontSize: 12, - color: AppColors.textSecondary, - lineHeight: 18, - }, - sampleIcon: { - fontSize: 20, - color: AppColors.accentPink + '99', - marginLeft: 8, - }, -}); diff --git a/src/screens/ToolCallingScreen.tsx b/src/screens/ToolCallingScreen.tsx deleted file mode 100644 index 562decc..0000000 --- a/src/screens/ToolCallingScreen.tsx +++ /dev/null @@ -1,616 +0,0 @@ -import React, { useState, useRef, useEffect } from 'react'; -import { - View, - Text, - TextInput, - TouchableOpacity, - ScrollView, - StyleSheet, - KeyboardAvoidingView, - Platform, - ActivityIndicator, -} from 'react-native'; -import LinearGradient from 'react-native-linear-gradient'; -import { - RunAnywhere, - ToolDefinition, - ToolCall, - ToolResult, - ToolCallingResult, -} from '@runanywhere/core'; -import { AppColors } from '../theme'; -import { useModelService } from '../services/ModelService'; -import { ModelLoaderWidget } from '../components'; - -// ─── Tool Definitions ──────────────────────────────────────────── - -const DEMO_TOOLS: ToolDefinition[] = [ - { - name: 'get_weather', - description: 'Get the current weather for a given city', - parameters: [ - { - name: 'city', - type: 'string', - description: 'The city name, e.g. "San Francisco"', - required: true, - }, - { - name: 'unit', - type: 'string', - description: 'Temperature unit: "celsius" or "fahrenheit"', - required: false, - defaultValue: 'celsius', - enum: ['celsius', 'fahrenheit'], - }, - ], - }, - { - name: 'calculate', - description: 'Perform a mathematical calculation', - parameters: [ - { - name: 'expression', - type: 'string', - description: 'A math expression to evaluate, e.g. "2 + 2"', - required: true, - }, - ], - }, - { - name: 'get_time', - description: 'Get the current date and time for a timezone', - parameters: [ - { - name: 'timezone', - type: 'string', - description: 'IANA timezone, e.g. "America/New_York"', - required: false, - defaultValue: 'UTC', - }, - ], - }, -]; - -// ─── Mock Tool Executors ───────────────────────────────────────── - -const mockWeather = async (args: Record) => { - const city = (args.city as string) || 'Unknown'; - const unit = (args.unit as string) || 'celsius'; - const temp = Math.floor(Math.random() * 30) + 5; - return { - city, - temperature: unit === 'fahrenheit' ? Math.round(temp * 1.8 + 32) : temp, - unit, - condition: ['Sunny', 'Cloudy', 'Rainy', 'Partly Cloudy'][Math.floor(Math.random() * 4)], - humidity: Math.floor(Math.random() * 60) + 30, - }; -}; - -const mockCalculate = async (args: Record) => { - const expr = (args.expression as string) || '0'; - try { - // Simple safe eval for basic math - const sanitized = expr.replace(/[^0-9+\-*/().% ]/g, ''); - const result = Function(`"use strict"; return (${sanitized})`)(); - return { expression: expr, result: Number(result) }; - } catch { - return { expression: expr, error: 'Could not evaluate expression' }; - } -}; - -const mockGetTime = async (args: Record) => { - const tz = (args.timezone as string) || 'UTC'; - try { - const now = new Date().toLocaleString('en-US', { timeZone: tz }); - return { timezone: tz, datetime: now }; - } catch { - return { timezone: tz, datetime: new Date().toISOString() }; - } -}; - -// ─── Log Entry Types ───────────────────────────────────────────── - -type LogType = 'info' | 'prompt' | 'tool_call' | 'tool_result' | 'response' | 'error'; - -interface LogEntry { - id: number; - type: LogType; - title: string; - detail?: string; - timestamp: Date; -} - -// ─── Screen Component ──────────────────────────────────────────── - -export const ToolCallingScreen: React.FC = () => { - const modelService = useModelService(); - const [inputText, setInputText] = useState(''); - const [isRunning, setIsRunning] = useState(false); - const [logs, setLogs] = useState([]); - const [toolsRegistered, setToolsRegistered] = useState(false); - const scrollRef = useRef(null); - const logIdRef = useRef(0); - - // Auto-scroll on new logs - useEffect(() => { - if (logs.length > 0) { - setTimeout(() => scrollRef.current?.scrollToEnd({ animated: true }), 100); - } - }, [logs]); - - const addLog = (type: LogType, title: string, detail?: string) => { - const id = Date.now() * 1000 + Math.floor(Math.random() * 1000); - setLogs(prev => [ - ...prev, - { id, type, title, detail, timestamp: new Date() }, - ]); - }; - - // ─── Register tools ────────────────────────────────────────── - - const handleRegisterTools = () => { - try { - RunAnywhere.clearTools(); - - RunAnywhere.registerTool(DEMO_TOOLS[0], mockWeather); - RunAnywhere.registerTool(DEMO_TOOLS[1], mockCalculate); - RunAnywhere.registerTool(DEMO_TOOLS[2], mockGetTime); - - setToolsRegistered(true); - addLog('info', 'Tools Registered', `Registered ${DEMO_TOOLS.length} tools: ${DEMO_TOOLS.map(t => t.name).join(', ')}`); - } catch (error) { - addLog('error', 'Registration Failed', String(error)); - } - }; - - // ─── Run tool calling generation ───────────────────────────── - - const handleGenerate = async () => { - const prompt = inputText.trim(); - if (!prompt || isRunning) return; - - setInputText(''); - setIsRunning(true); - addLog('prompt', 'User Prompt', prompt); - - try { - const result: ToolCallingResult = await RunAnywhere.generateWithTools(prompt, { - tools: DEMO_TOOLS, - maxToolCalls: 3, - autoExecute: true, - temperature: 0.7, - maxTokens: 512, - }); - - // Log tool calls - if (result.toolCalls.length > 0) { - for (let i = 0; i < result.toolCalls.length; i++) { - const tc = result.toolCalls[i]; - addLog( - 'tool_call', - `Tool Call: ${tc.toolName}`, - JSON.stringify(tc.arguments, null, 2), - ); - if (result.toolResults[i]) { - const tr = result.toolResults[i]; - addLog( - 'tool_result', - `Result: ${tr.toolName} (${tr.success ? 'success' : 'failed'})`, - tr.success ? JSON.stringify(tr.result, null, 2) : tr.error, - ); - } - } - } else { - addLog('info', 'No Tool Calls', 'The model responded without calling any tools'); - } - - // Log final response - addLog('response', 'Model Response', result.text || '(empty)'); - } catch (error) { - addLog('error', 'Generation Failed', String(error)); - } finally { - setIsRunning(false); - } - }; - - // ─── Manual parse test ─────────────────────────────────────── - - const handleParseSample = async () => { - addLog('info', 'Parse Test', 'Testing parseToolCall with sample output...'); - - const sampleOutput = `I'll check the weather for you.\n{"name": "get_weather", "arguments": {"city": "San Francisco"}}`; - - try { - const parsed = await RunAnywhere.parseToolCall(sampleOutput); - addLog( - 'tool_call', - 'Parsed Tool Call', - parsed.toolCall - ? `Tool: ${parsed.toolCall.toolName}\nArgs: ${JSON.stringify(parsed.toolCall.arguments, null, 2)}\nClean text: "${parsed.text}"` - : `No tool call detected. Text: "${parsed.text}"`, - ); - } catch (error) { - addLog('error', 'Parse Failed', String(error)); - } - }; - - // ─── Render ────────────────────────────────────────────────── - - if (!modelService.isLLMLoaded) { - return ( - - ); - } - - return ( - - {/* Action Buttons */} - - - - {toolsRegistered ? 'Tools Ready' : 'Register Tools'} - - - - - Parse Test - - - setLogs([])} - > - Clear - - - - {/* Tool chips */} - - {DEMO_TOOLS.map(tool => ( - - {tool.name} - - ))} - - - {/* Log output */} - - {logs.length === 0 ? ( - - 🛠 - Tool Calling Test - - Register tools, then ask the model to use them.{'\n'} - Try: "What's the weather in Tokyo?" or "Calculate 42 * 17" - - - ) : ( - logs.map(log => ( - - - {LOG_ICONS[log.type]} - {log.title} - - {log.timestamp.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit', second: '2-digit' })} - - - {log.detail ? ( - {log.detail} - ) : null} - - )) - )} - {isRunning && ( - - - Generating... - - )} - - - {/* Suggestion chips */} - - {['What\'s the weather in Tokyo?', 'Calculate 123 * 456', 'What time is it in New York?'].map(s => ( - setInputText(s)} - > - {s} - - ))} - - - {/* Input */} - - - - - - â–ļ - - - - - - ); -}; - -// ─── Constants ───────────────────────────────────────────────── - -const LOG_ICONS: Record = { - info: 'ℹī¸', - prompt: 'đŸ’Ŧ', - tool_call: '🔧', - tool_result: 'đŸ“Ļ', - response: '🤖', - error: '❌', -}; - -// ─── Styles ──────────────────────────────────────────────────── - -const styles = StyleSheet.create({ - container: { - flex: 1, - backgroundColor: AppColors.primaryDark, - }, - - // Action bar - actionBar: { - flexDirection: 'row', - padding: 12, - gap: 8, - }, - actionBtn: { - flex: 1, - paddingVertical: 10, - borderRadius: 12, - backgroundColor: AppColors.surfaceCard, - borderWidth: 1, - borderColor: AppColors.accentOrange + '40', - alignItems: 'center', - }, - actionBtnActive: { - backgroundColor: AppColors.accentOrange + '20', - borderColor: AppColors.accentOrange, - }, - actionBtnText: { - fontSize: 13, - fontWeight: '600', - color: AppColors.accentOrange, - }, - actionBtnClear: { - paddingVertical: 10, - paddingHorizontal: 16, - borderRadius: 12, - backgroundColor: AppColors.surfaceCard, - borderWidth: 1, - borderColor: AppColors.textMuted + '40', - alignItems: 'center', - }, - actionBtnClearText: { - fontSize: 13, - fontWeight: '600', - color: AppColors.textMuted, - }, - - // Tool chips - toolChips: { - flexDirection: 'row', - paddingHorizontal: 12, - paddingBottom: 8, - gap: 6, - }, - toolChip: { - paddingHorizontal: 10, - paddingVertical: 4, - backgroundColor: AppColors.surfaceElevated, - borderRadius: 8, - }, - toolChipText: { - fontSize: 11, - fontFamily: Platform.OS === 'ios' ? 'Menlo' : 'monospace', - color: AppColors.textSecondary, - }, - - // Log area - logArea: { - flex: 1, - }, - logContent: { - padding: 12, - paddingBottom: 8, - }, - - // Empty state - emptyState: { - flex: 1, - justifyContent: 'center', - alignItems: 'center', - paddingVertical: 80, - }, - emptyIcon: { - fontSize: 56, - marginBottom: 16, - }, - emptyTitle: { - fontSize: 22, - fontWeight: '700', - color: AppColors.textPrimary, - marginBottom: 8, - }, - emptySubtitle: { - fontSize: 13, - color: AppColors.textSecondary, - textAlign: 'center', - lineHeight: 20, - paddingHorizontal: 32, - }, - - // Log entries - logEntry: { - marginBottom: 8, - padding: 12, - borderRadius: 12, - borderWidth: 1, - }, - logHeader: { - flexDirection: 'row', - alignItems: 'center', - gap: 6, - }, - logIcon: { - fontSize: 14, - }, - logTitle: { - flex: 1, - fontSize: 13, - fontWeight: '600', - color: AppColors.textPrimary, - }, - logTime: { - fontSize: 10, - color: AppColors.textMuted, - fontFamily: Platform.OS === 'ios' ? 'Menlo' : 'monospace', - }, - logDetail: { - marginTop: 6, - fontSize: 12, - color: AppColors.textSecondary, - fontFamily: Platform.OS === 'ios' ? 'Menlo' : 'monospace', - lineHeight: 18, - }, - - // Log type-specific colors - log_info: { - backgroundColor: AppColors.info + '10', - borderColor: AppColors.info + '30', - }, - log_prompt: { - backgroundColor: AppColors.accentCyan + '10', - borderColor: AppColors.accentCyan + '30', - }, - log_tool_call: { - backgroundColor: AppColors.accentOrange + '10', - borderColor: AppColors.accentOrange + '30', - }, - log_tool_result: { - backgroundColor: AppColors.accentGreen + '10', - borderColor: AppColors.accentGreen + '30', - }, - log_response: { - backgroundColor: AppColors.accentViolet + '10', - borderColor: AppColors.accentViolet + '30', - }, - log_error: { - backgroundColor: AppColors.error + '10', - borderColor: AppColors.error + '30', - }, - - // Loading - loadingRow: { - flexDirection: 'row', - alignItems: 'center', - gap: 8, - paddingVertical: 12, - justifyContent: 'center', - }, - loadingText: { - fontSize: 13, - color: AppColors.accentOrange, - }, - - // Suggestions - suggestions: { - flexDirection: 'row', - paddingHorizontal: 12, - paddingBottom: 8, - gap: 6, - }, - suggestionChip: { - flex: 1, - paddingHorizontal: 8, - paddingVertical: 6, - backgroundColor: AppColors.surfaceCard, - borderRadius: 10, - borderWidth: 1, - borderColor: AppColors.accentOrange + '30', - }, - suggestionText: { - fontSize: 10, - color: AppColors.textSecondary, - textAlign: 'center', - }, - - // Input - inputContainer: { - padding: 12, - backgroundColor: AppColors.surfaceCard + 'CC', - borderTopWidth: 1, - borderTopColor: AppColors.textMuted + '1A', - }, - inputWrapper: { - flexDirection: 'row', - alignItems: 'center', - gap: 10, - }, - input: { - flex: 1, - backgroundColor: AppColors.primaryMid, - borderRadius: 20, - paddingHorizontal: 16, - paddingVertical: 10, - fontSize: 14, - color: AppColors.textPrimary, - maxHeight: 80, - }, - sendButton: { - width: 44, - height: 44, - borderRadius: 22, - justifyContent: 'center', - alignItems: 'center', - }, - sendButtonDisabled: { - opacity: 0.4, - }, - sendIcon: { - fontSize: 18, - color: '#FFFFFF', - }, -}); diff --git a/src/screens/VoicePipelineScreen.tsx b/src/screens/VoicePipelineScreen.tsx deleted file mode 100644 index 5a5f1b5..0000000 --- a/src/screens/VoicePipelineScreen.tsx +++ /dev/null @@ -1,606 +0,0 @@ -import React, { useState, useRef, useCallback } from 'react'; -import { - View, - Text, - TouchableOpacity, - ScrollView, - StyleSheet, - Platform, - NativeModules, -} from 'react-native'; -import LinearGradient from 'react-native-linear-gradient'; -import RNFS from 'react-native-fs'; -import { RunAnywhere, VoiceSessionEvent, VoiceSessionHandle } from '@runanywhere/core'; -import { AppColors } from '../theme'; -import { useModelService } from '../services/ModelService'; -import { ModelLoaderWidget, AudioVisualizer } from '../components'; - -// Conditionally import Sound - disabled on iOS via react-native.config.js -let Sound: any = null; -if (Platform.OS === 'android') { - try { - Sound = require('react-native-sound').default; - } catch (e) { - console.log('react-native-sound not available'); - } -} - -// iOS uses NativeAudioModule -const { NativeAudioModule } = NativeModules; - -interface ConversationMessage { - role: 'user' | 'assistant'; - text: string; - timestamp: Date; -} - -// Model IDs - must match those registered in ModelService -const MODEL_IDS = { - llm: 'lfm2-350m-q8_0', - stt: 'sherpa-onnx-whisper-tiny.en', - tts: 'vits-piper-en_US-lessac-medium', -}; - -export const VoicePipelineScreen: React.FC = () => { - const modelService = useModelService(); - const [isActive, setIsActive] = useState(false); - const [status, setStatus] = useState('Ready'); - const [conversation, setConversation] = useState([]); - const [audioLevel, setAudioLevel] = useState(0); - - // Refs for session and audio - const sessionRef = useRef(null); - const currentSoundRef = useRef(null); - const isPlayingRef = useRef(false); - - // Handle voice session events per docs: - // https://docs.runanywhere.ai/react-native/voice-agent#voicesessionevent - const handleVoiceEvent = useCallback((event: VoiceSessionEvent) => { - switch (event.type) { - case 'sessionStarted': - setStatus('Listening...'); - setAudioLevel(0.2); - break; - - case 'listeningStarted': - setStatus('Listening...'); - setAudioLevel(0.3); - break; - - case 'speechDetected': - setStatus('Hearing you...'); - setAudioLevel(0.7); - break; - - case 'speechEnded': - setAudioLevel(0.1); - break; - - case 'transcribing': - setStatus('Processing speech...'); - setAudioLevel(0.4); - break; - - case 'transcriptionComplete': - if (event.data?.transcript) { - const userMessage: ConversationMessage = { - role: 'user', - text: event.data.transcript, - timestamp: new Date(), - }; - setConversation(prev => [...prev, userMessage]); - } - setStatus('Thinking...'); - setAudioLevel(0.5); - break; - - case 'generating': - setStatus('Generating response...'); - setAudioLevel(0.5); - break; - - case 'generationComplete': - if (event.data?.response) { - const assistantMessage: ConversationMessage = { - role: 'assistant', - text: event.data.response, - timestamp: new Date(), - }; - setConversation(prev => [...prev, assistantMessage]); - } - setStatus('Synthesizing...'); - setAudioLevel(0.6); - break; - - case 'synthesizing': - setStatus('Preparing voice...'); - break; - - case 'synthesisComplete': - setStatus('Speaking...'); - // Play audio if provided - if (event.data?.audio) { - playResponseAudio(event.data.audio); - } - break; - - case 'speaking': - setStatus('Speaking...'); - setAudioLevel(0.8); - break; - - case 'turnComplete': - setStatus('Listening...'); - setAudioLevel(0.3); - break; - - case 'error': - setStatus(`Error: ${event.data?.error || 'Unknown error'}`); - setAudioLevel(0); - console.error('Voice session error:', event.data?.error); - break; - } - }, []); - - // Play synthesized audio response - platform-specific - const playResponseAudio = async (base64Audio: string) => { - try { - if (Platform.OS === 'ios' && NativeAudioModule) { - // iOS: Use NativeAudioModule - isPlayingRef.current = true; - setAudioLevel(0.8); - await NativeAudioModule.playAudioBase64(base64Audio, 22050); - isPlayingRef.current = false; - setAudioLevel(0.3); - } else if (Platform.OS === 'android' && Sound) { - // Android: Use react-native-sound - const wavData = createWavFromBase64Float32(base64Audio, 22050); - const tempPath = `${RNFS.TemporaryDirectoryPath}/voice_response_${Date.now()}.wav`; - await RNFS.writeFile(tempPath, wavData, 'base64'); - - const sound = new Sound(tempPath, '', (error: any) => { - if (error) { - console.error('Failed to load sound:', error); - return; - } - - currentSoundRef.current = sound; - setAudioLevel(0.8); - - sound.play((success: boolean) => { - sound.release(); - currentSoundRef.current = null; - setAudioLevel(0.3); - }); - }); - } else { - console.warn('No audio playback module available'); - } - } catch (error) { - console.error('Error playing audio:', error); - isPlayingRef.current = false; - setAudioLevel(0.3); - } - }; - - // Convert base64 float32 PCM to WAV format - const createWavFromBase64Float32 = (base64Audio: string, sampleRate: number): string => { - const binaryStr = atob(base64Audio); - const bytes = new Uint8Array(binaryStr.length); - for (let i = 0; i < binaryStr.length; i++) { - bytes[i] = binaryStr.charCodeAt(i); - } - const float32Samples = new Float32Array(bytes.buffer); - const numSamples = float32Samples.length; - - const wavBuffer = new ArrayBuffer(44 + numSamples * 2); - const view = new DataView(wavBuffer); - - // WAV header - const writeString = (offset: number, str: string) => { - for (let i = 0; i < str.length; i++) { - view.setUint8(offset + i, str.charCodeAt(i)); - } - }; - - writeString(0, 'RIFF'); - view.setUint32(4, 36 + numSamples * 2, true); - writeString(8, 'WAVE'); - writeString(12, 'fmt '); - view.setUint32(16, 16, true); - view.setUint16(20, 1, true); - view.setUint16(22, 1, true); - view.setUint32(24, sampleRate, true); - view.setUint32(28, sampleRate * 2, true); - view.setUint16(32, 2, true); - view.setUint16(34, 16, true); - writeString(36, 'data'); - view.setUint32(40, numSamples * 2, true); - - let offset = 44; - for (let i = 0; i < float32Samples.length; i++) { - const s = Math.max(-1, Math.min(1, float32Samples[i])); - view.setInt16(offset, s < 0 ? s * 0x8000 : s * 0x7fff, true); - offset += 2; - } - - const uint8Array = new Uint8Array(wavBuffer); - let result = ''; - for (let i = 0; i < uint8Array.length; i++) { - result += String.fromCharCode(uint8Array[i]); - } - return btoa(result); - }; - - // Start voice session per docs: - // https://docs.runanywhere.ai/react-native/voice-agent#startvoicesession - const startVoiceAgent = async () => { - setIsActive(true); - setStatus('Starting...'); - - try { - // Per docs: Use startVoiceSession with VoiceSessionConfig and callback - sessionRef.current = await RunAnywhere.startVoiceSession( - { - agentConfig: { - llmModelId: MODEL_IDS.llm, - sttModelId: MODEL_IDS.stt, - ttsModelId: MODEL_IDS.tts, - systemPrompt: 'You are a helpful, friendly voice assistant. Keep your responses brief and conversational.', - generationOptions: { - maxTokens: 150, - temperature: 0.7, - }, - }, - enableVAD: true, - vadSensitivity: 0.5, - speechTimeout: 3000, // 3 seconds timeout for speech - }, - handleVoiceEvent - ); - } catch (error) { - console.error('Voice agent error:', error); - setStatus(`Error: ${error}`); - setIsActive(false); - } - }; - - const stopVoiceAgent = async () => { - try { - // Stop any playing audio - platform-specific - if (Platform.OS === 'ios' && isPlayingRef.current && NativeAudioModule) { - await NativeAudioModule.stopPlayback(); - isPlayingRef.current = false; - } else if (currentSoundRef.current) { - currentSoundRef.current.stop(() => { - currentSoundRef.current?.release(); - currentSoundRef.current = null; - }); - } - - // Stop the voice session - if (sessionRef.current) { - await sessionRef.current.stop(); - sessionRef.current = null; - } - - setIsActive(false); - setStatus('Ready'); - setAudioLevel(0); - } catch (error) { - console.error('Stop voice agent error:', error); - } - }; - - const clearConversation = () => { - setConversation([]); - }; - - if (!modelService.isVoiceAgentReady) { - return ( - - ); - } - - return ( - - - {/* Status Area */} - - {isActive ? ( - <> - - - {status} - - - Voice agent is running - - - ) : ( - <> - - ✨ - - Voice Agent - - Full speech-to-speech AI conversation - - - )} - - - {/* Conversation */} - {conversation.length > 0 && ( - - - Conversation - - Clear - - - {conversation.map((message, index) => ( - - - - {message.role === 'user' ? '👤' : '🤖'} - - - {message.role === 'user' ? 'You' : 'Assistant'} - - - {message.text} - - ))} - - )} - - {/* Pipeline Info */} - {!isActive && conversation.length === 0 && ( - - How it works: - - 1ī¸âƒŖ - Voice Activity Detection (VAD) listens for speech - - - 2ī¸âƒŖ - Speech is transcribed (STT with Whisper) - - - 3ī¸âƒŖ - AI generates response (LLM with SmolLM2) - - - 4ī¸âƒŖ - Response is spoken (TTS with Piper) - - - )} - - - {/* Control Button */} - - - - - {isActive ? '⏚' : '✨'} - - - {isActive ? 'Stop Agent' : 'Start Voice Agent'} - - - - - - ); -}; - -const styles = StyleSheet.create({ - container: { - flex: 1, - backgroundColor: AppColors.primaryDark, - }, - scrollView: { - flex: 1, - }, - scrollContent: { - padding: 24, - }, - statusArea: { - padding: 32, - backgroundColor: AppColors.surfaceCard, - borderRadius: 24, - borderWidth: 1, - borderColor: AppColors.textMuted + '1A', - alignItems: 'center', - marginBottom: 24, - }, - statusActive: { - borderColor: AppColors.accentGreen + '80', - borderWidth: 2, - shadowColor: AppColors.accentGreen, - shadowOffset: { width: 0, height: 0 }, - shadowOpacity: 0.3, - shadowRadius: 20, - elevation: 8, - }, - agentIconContainer: { - width: 100, - height: 100, - backgroundColor: AppColors.accentGreen + '20', - borderRadius: 50, - justifyContent: 'center', - alignItems: 'center', - marginBottom: 24, - }, - agentIcon: { - fontSize: 48, - }, - statusText: { - fontSize: 20, - fontWeight: '700', - color: AppColors.textPrimary, - marginBottom: 8, - }, - statusSubtitle: { - fontSize: 14, - color: AppColors.textSecondary, - }, - conversationSection: { - marginBottom: 24, - }, - conversationHeader: { - flexDirection: 'row', - justifyContent: 'space-between', - alignItems: 'center', - marginBottom: 16, - }, - conversationTitle: { - fontSize: 18, - fontWeight: '700', - color: AppColors.textPrimary, - }, - clearButton: { - fontSize: 14, - color: AppColors.accentGreen, - fontWeight: '600', - }, - messageCard: { - padding: 16, - borderRadius: 16, - marginBottom: 12, - borderWidth: 1, - }, - userMessage: { - backgroundColor: AppColors.accentCyan + '20', - borderColor: AppColors.accentCyan + '40', - alignSelf: 'flex-end', - maxWidth: '85%', - }, - assistantMessage: { - backgroundColor: AppColors.surfaceCard, - borderColor: AppColors.textMuted + '20', - alignSelf: 'flex-start', - maxWidth: '85%', - }, - messageHeader: { - flexDirection: 'row', - alignItems: 'center', - marginBottom: 8, - }, - roleIcon: { - fontSize: 18, - marginRight: 8, - }, - roleText: { - fontSize: 12, - fontWeight: '600', - color: AppColors.textSecondary, - textTransform: 'uppercase', - }, - messageText: { - fontSize: 14, - color: AppColors.textPrimary, - lineHeight: 20, - }, - infoCard: { - padding: 20, - backgroundColor: AppColors.surfaceCard + '80', - borderRadius: 16, - borderWidth: 1, - borderColor: AppColors.textMuted + '1A', - }, - infoTitle: { - fontSize: 16, - fontWeight: '700', - color: AppColors.textPrimary, - marginBottom: 16, - }, - infoStep: { - flexDirection: 'row', - alignItems: 'center', - marginBottom: 12, - }, - stepNumber: { - fontSize: 20, - marginRight: 12, - }, - stepText: { - fontSize: 14, - color: AppColors.textSecondary, - flex: 1, - }, - buttonContainer: { - padding: 24, - backgroundColor: AppColors.surfaceCard + 'CC', - borderTopWidth: 1, - borderTopColor: AppColors.textMuted + '1A', - }, - controlButton: { - flexDirection: 'row', - height: 72, - borderRadius: 36, - justifyContent: 'center', - alignItems: 'center', - gap: 12, - elevation: 8, - shadowColor: AppColors.accentGreen, - shadowOffset: { width: 0, height: 8 }, - shadowOpacity: 0.4, - shadowRadius: 20, - }, - controlIcon: { - fontSize: 28, - }, - controlButtonText: { - fontSize: 16, - fontWeight: '700', - color: '#FFFFFF', - }, -}); diff --git a/src/screens/index.ts b/src/screens/index.ts index 27b3c8c..e70edd7 100644 --- a/src/screens/index.ts +++ b/src/screens/index.ts @@ -1,6 +1,9 @@ -export * from './HomeScreen'; -export * from './ChatScreen'; -export * from './ToolCallingScreen'; -export * from './SpeechToTextScreen'; -export * from './TextToSpeechScreen'; -export * from './VoicePipelineScreen'; +export { HomeScreen } from './HomeScreen'; +export { LiveSessionScreen } from './LiveSessionScreen'; +export { InsightsScreen } from './InsightsScreen'; +export { SettingsScreen } from './SettingsScreen'; +export { DisclaimerScreen } from './DisclaimerScreen'; +export { OutcomeReplayScreen } from './OutcomeReplayScreen'; + +export { PreSessionFormScreen } from './PreSessionFormScreen'; +export { PreSessionStrategyScreen } from './PreSessionStrategyScreen'; diff --git a/src/services/CounterStrategyEngine.ts b/src/services/CounterStrategyEngine.ts new file mode 100644 index 0000000..1c0dbf8 --- /dev/null +++ b/src/services/CounterStrategyEngine.ts @@ -0,0 +1,220 @@ +/** + * 🔒 PRIVACY NOTICE + * All counter-strategy generation runs locally on device. + * No data leaves this device. No cloud APIs. No LLM calls. + * Pure rule-based mapping from detected tactics to counter-strategies. + */ + +import { NegotiationPattern, CounterStrategy } from '../types/session'; +import { getPatternDefinition } from '../ai/patternLibrary'; + +/** + * Counter strategy definition for each tactic + */ +interface CounterStrategyDefinition { + suggestions: string[]; + explanation: string; +} + +/** + * Complete rule-based mapping: tactic → counter-strategies + */ +const COUNTER_STRATEGY_MAP: Record = { + [NegotiationPattern.ANCHORING]: { + suggestions: [ + 'Ask for a detailed cost breakdown', + 'Present your own alternative anchor point', + 'Request comparison data from multiple sources', + ], + explanation: + 'Anchoring attempts to set a psychological reference point. Counter by introducing your own data-backed reference or questioning their basis.', + }, + + [NegotiationPattern.BUDGET_OBJECTION]: { + suggestions: [ + 'Ask about budget flexibility and approval thresholds', + 'Break pricing into smaller phases or milestones', + 'Highlight ROI and value instead of focusing on cost', + ], + explanation: + 'Budget objection often masks value hesitation rather than a real constraint. Shift the conversation from cost to return on investment.', + }, + + [NegotiationPattern.AUTHORITY_PRESSURE]: { + suggestions: [ + 'Ask for the specific decision criteria being used', + 'Suggest a joint review with all stakeholders', + 'Delay final commitment until decision-maker is present', + ], + explanation: + 'Authority pressure reduces your negotiation space by introducing an unseen decision-maker. Get access to the real authority.', + }, + + [NegotiationPattern.TIME_PRESSURE]: { + suggestions: [ + 'Ask if the deadline is truly final or flexible', + 'Introduce a new variable to reset the timeline', + 'Propose a pause — revisit with fresh perspective', + ], + explanation: + 'Urgency framing creates artificial pressure to force quick decisions. Verify the deadline and resist rushing critical commitments.', + }, + + [NegotiationPattern.DEFLECTION]: { + suggestions: [ + 'Pin down specific concerns driving the hesitation', + 'Set a concrete follow-up date and time right now', + 'Ask what information would help them decide today', + ], + explanation: + 'Deflection delays decisions without surfacing real objections. Gently identify what\'s actually holding them back.', + }, + + [NegotiationPattern.POSITIVE_SIGNAL]: { + suggestions: [ + 'Capitalize on momentum — move toward commitment', + 'Summarize agreed points and lock them in writing', + 'Ask about next steps while enthusiasm is high', + ], + explanation: + 'Positive signals indicate openness. Strike while the iron is hot — clarify terms and advance toward agreement.', + }, + + [NegotiationPattern.NEGATIVE_SIGNAL]: { + suggestions: [ + 'Probe for the specific concern behind the negativity', + 'Acknowledge their worry and provide concrete evidence', + 'Offer an alternative approach that addresses the issue', + ], + explanation: + 'Negative signals reveal underlying objections. Address them directly with empathy and data rather than ignoring them.', + }, + + [NegotiationPattern.COMMITMENT_LANGUAGE]: { + suggestions: [ + 'Document the agreement immediately in writing', + 'Clarify all remaining terms before finalizing', + 'Confirm timeline and deliverables for next steps', + ], + explanation: + 'Commitment language signals readiness to close. Ensure all details are clear and get confirmation in writing.', + }, + + [NegotiationPattern.STRENGTH_SIGNAL]: { + suggestions: [ + 'Leverage this momentum to position yourself firmly', + 'Directly tie this achievement to their current needs', + 'Use this high-value moment to pivot to compensation framing', + ], + explanation: + 'Strength signals display your leverage points. Use them to establish value before discussing terms.', + }, +}; + +/** + * Cooldown tracker — stores last trigger timestamp per tactic + */ +const cooldownTracker: Map = new Map(); + +/** + * Default cooldown period in milliseconds (10 seconds) + */ +const COOLDOWN_MS = 10_000; + +/** + * Default confidence threshold (70%) + */ +const CONFIDENCE_THRESHOLD = 70; + +/** + * Generate counter-strategies for a detected tactic. + * + * Returns null if: + * - Confidence is below threshold + * - Same tactic was triggered within cooldown period + * + * @param tactic - The detected NegotiationPattern + * @param confidence - Confidence score (0-100) + * @param cooldownMs - Optional custom cooldown in ms (default: 10000) + * @param threshold - Optional custom confidence threshold (default: 70) + */ +export const generateCounterStrategies = ( + tactic: NegotiationPattern, + confidence: number, + cooldownMs: number = COOLDOWN_MS, + threshold: number = CONFIDENCE_THRESHOLD, +): CounterStrategy | null => { + console.log('[CounterStrategyEngine] đŸŽ¯ generateCounterStrategies() called'); + console.log('[CounterStrategyEngine] 📋 Tactic:', tactic); + console.log('[CounterStrategyEngine] 📊 Confidence:', confidence); + + // 1. Check confidence threshold + if (confidence < threshold) { + console.log( + `[CounterStrategyEngine] ❌ Confidence ${confidence}% below threshold ${threshold}%`, + ); + return null; + } + + // 2. Check cooldown + const now = Date.now(); + const lastTriggered = cooldownTracker.get(tactic); + + if (lastTriggered && now - lastTriggered < cooldownMs) { + const remaining = Math.round((cooldownMs - (now - lastTriggered)) / 1000); + console.log( + `[CounterStrategyEngine] âŗ Cooldown active for "${tactic}" — ${remaining}s remaining`, + ); + return null; + } + + // 3. Look up counter-strategy + const definition = COUNTER_STRATEGY_MAP[tactic]; + + if (!definition) { + console.log(`[CounterStrategyEngine] ❌ No counter-strategy for tactic: ${tactic}`); + return null; + } + + // 4. Get display name from pattern library + const patternDef = getPatternDefinition(tactic); + + // 5. Update cooldown tracker + cooldownTracker.set(tactic, now); + console.log(`[CounterStrategyEngine] ✅ Counter-strategy generated for "${tactic}"`); + + return { + tactic, + tacticDisplayName: patternDef.displayName, + confidence, + suggestions: definition.suggestions, + explanation: definition.explanation, + timestamp: now, + }; +}; + +/** + * Reset cooldown for a specific tactic (for testing) + */ +export const resetCooldown = (tactic: NegotiationPattern): void => { + cooldownTracker.delete(tactic); +}; + +/** + * Reset all cooldowns (for testing or session reset) + */ +export const resetAllCooldowns = (): void => { + cooldownTracker.clear(); +}; + +/** + * Check if a tactic is currently on cooldown + */ +export const isOnCooldown = ( + tactic: NegotiationPattern, + cooldownMs: number = COOLDOWN_MS, +): boolean => { + const lastTriggered = cooldownTracker.get(tactic); + if (!lastTriggered) return false; + return Date.now() - lastTriggered < cooldownMs; +}; diff --git a/src/services/LocalStorageService.ts b/src/services/LocalStorageService.ts new file mode 100644 index 0000000..fda7fbf --- /dev/null +++ b/src/services/LocalStorageService.ts @@ -0,0 +1,320 @@ +/** + * 🔒 PRIVACY NOTICE + * All data is stored locally on device using AsyncStorage. + * No cloud sync. No external storage. No data leaves this device. + */ + +import AsyncStorage from '@react-native-async-storage/async-storage'; +import { + Session, + AppSettings, + SessionStats, + NegotiationMode, + NegotiationPattern, +} from '../types/session'; + +// Storage keys +const KEYS = { + SESSIONS: '@latent:sessions', + SESSION_PREFIX: '@latent:session:', + SETTINGS: '@latent:settings', + STATS: '@latent:stats', +}; + +/** + * Default app settings + */ +const DEFAULT_SETTINGS: AppSettings = { + defaultMode: NegotiationMode.SALES, + patternSensitivity: 1.0, + enableAutoSave: true, + autoSaveInterval: 45000, // 45 seconds as requested + enableHapticFeedback: true, + enableSuggestionNotifications: true, +}; + +/** + * LocalStorageService - Handles all AsyncStorage operations + */ +export class LocalStorageService { + /** + * Save a session to storage + */ + static async saveSession(session: Session): Promise { + try { + const sessionKey = `${KEYS.SESSION_PREFIX}${session.id}`; + await AsyncStorage.setItem(sessionKey, JSON.stringify(session)); + + // Update session list + const sessionIds = await this.getAllSessionIds(); + if (!sessionIds.includes(session.id)) { + sessionIds.push(session.id); + await AsyncStorage.setItem(KEYS.SESSIONS, JSON.stringify(sessionIds)); + } + + console.log('[LocalStorage] Session saved:', session.id); + } catch (error) { + console.error('[LocalStorage] Error saving session:', error); + throw error; + } + } + + /** + * Get a session by ID + */ + static async getSession(sessionId: string): Promise { + try { + const sessionKey = `${KEYS.SESSION_PREFIX}${sessionId}`; + const data = await AsyncStorage.getItem(sessionKey); + return data ? JSON.parse(data) : null; + } catch (error) { + console.error('[LocalStorage] Error getting session:', error); + return null; + } + } + + /** + * Get all session IDs + */ + static async getAllSessionIds(): Promise { + try { + const data = await AsyncStorage.getItem(KEYS.SESSIONS); + return data ? JSON.parse(data) : []; + } catch (error) { + console.error('[LocalStorage] Error getting session IDs:', error); + return []; + } + } + + /** + * Get all sessions (sorted by timestamp, newest first) + */ + static async getAllSessions(): Promise { + try { + const sessionIds = await this.getAllSessionIds(); + const sessions: Session[] = []; + + for (const id of sessionIds) { + const session = await this.getSession(id); + if (session) { + sessions.push(session); + } + } + + // Sort by timestamp (newest first) + return sessions.sort((a, b) => b.timestamp - a.timestamp); + } catch (error) { + console.error('[LocalStorage] Error getting all sessions:', error); + return []; + } + } + + /** + * Delete a session + */ + static async deleteSession(sessionId: string): Promise { + try { + const sessionKey = `${KEYS.SESSION_PREFIX}${sessionId}`; + await AsyncStorage.removeItem(sessionKey); + + // Update session list + const sessionIds = await this.getAllSessionIds(); + const updatedIds = sessionIds.filter((id) => id !== sessionId); + await AsyncStorage.setItem(KEYS.SESSIONS, JSON.stringify(updatedIds)); + + console.log('[LocalStorage] Session deleted:', sessionId); + } catch (error) { + console.error('[LocalStorage] Error deleting session:', error); + throw error; + } + } + + /** + * Delete all sessions + */ + static async deleteAllSessions(): Promise { + try { + const sessionIds = await this.getAllSessionIds(); + + // Delete each session + for (const id of sessionIds) { + const sessionKey = `${KEYS.SESSION_PREFIX}${id}`; + await AsyncStorage.removeItem(sessionKey); + } + + // Clear session list + await AsyncStorage.setItem(KEYS.SESSIONS, JSON.stringify([])); + + console.log('[LocalStorage] All sessions deleted'); + } catch (error) { + console.error('[LocalStorage] Error deleting all sessions:', error); + throw error; + } + } + + /** + * Get app settings + */ + static async getSettings(): Promise { + try { + const data = await AsyncStorage.getItem(KEYS.SETTINGS); + return data ? { ...DEFAULT_SETTINGS, ...JSON.parse(data) } : DEFAULT_SETTINGS; + } catch (error) { + console.error('[LocalStorage] Error getting settings:', error); + return DEFAULT_SETTINGS; + } + } + + /** + * Save app settings + */ + static async saveSettings(settings: AppSettings): Promise { + try { + await AsyncStorage.setItem(KEYS.SETTINGS, JSON.stringify(settings)); + console.log('[LocalStorage] Settings saved'); + } catch (error) { + console.error('[LocalStorage] Error saving settings:', error); + throw error; + } + } + + /** + * Calculate and cache session statistics + */ + static async calculateStats(): Promise { + try { + const sessions = await this.getAllSessions(); + + if (sessions.length === 0) { + return { + totalSessions: 0, + avgFocusScore: 0, + avgDuration: 0, + mostCommonPattern: null, + totalPatterns: 0, + lastSessionDate: null, + }; + } + + // Calculate averages + const totalFocusScore = sessions.reduce((sum, s) => sum + s.cognitiveMetrics.focusScore, 0); + const avgFocusScore = Math.round(totalFocusScore / sessions.length); + + const totalDuration = sessions.reduce((sum, s) => sum + s.duration, 0); + const avgDuration = Math.round(totalDuration / sessions.length); + + // Count patterns + const patternCounts: Record = {}; + let totalPatterns = 0; + + sessions.forEach((session) => { + session.detectedPatterns.forEach((pattern) => { + patternCounts[pattern.pattern] = (patternCounts[pattern.pattern] || 0) + 1; + totalPatterns++; + }); + }); + + // Find most common pattern + let mostCommonPattern: NegotiationPattern | null = null; + let maxCount = 0; + + Object.entries(patternCounts).forEach(([pattern, count]) => { + if (count > maxCount) { + maxCount = count; + mostCommonPattern = pattern as NegotiationPattern; + } + }); + + const stats: SessionStats = { + totalSessions: sessions.length, + avgFocusScore, + avgDuration, + mostCommonPattern, + totalPatterns, + lastSessionDate: sessions[0].timestamp, + }; + + // Cache stats + await AsyncStorage.setItem(KEYS.STATS, JSON.stringify(stats)); + + return stats; + } catch (error) { + console.error('[LocalStorage] Error calculating stats:', error); + return { + totalSessions: 0, + avgFocusScore: 0, + avgDuration: 0, + mostCommonPattern: null, + totalPatterns: 0, + lastSessionDate: null, + }; + } + } + + /** + * Get cached stats (faster, but may be slightly outdated) + */ + static async getCachedStats(): Promise { + try { + const data = await AsyncStorage.getItem(KEYS.STATS); + return data ? JSON.parse(data) : null; + } catch (error) { + console.error('[LocalStorage] Error getting cached stats:', error); + return null; + } + } + + /** + * Get stats (tries cache first, calculates if needed) + */ + static async getStats(): Promise { + const cached = await this.getCachedStats(); + if (cached) { + return cached; + } + return await this.calculateStats(); + } + + /** + * Clear all data (for settings screen) + */ + static async clearAllData(): Promise { + try { + await this.deleteAllSessions(); + await AsyncStorage.removeItem(KEYS.STATS); + console.log('[LocalStorage] All data cleared'); + } catch (error) { + console.error('[LocalStorage] Error clearing all data:', error); + throw error; + } + } + + /** + * Get storage size estimate (for debugging) + */ + static async getStorageInfo(): Promise<{ keys: number; estimatedSize: string }> { + try { + const allKeys = await AsyncStorage.getAllKeys(); + const latentKeys = allKeys.filter((key) => key.startsWith('@latent:')); + + // Get approximate size + let totalSize = 0; + for (const key of latentKeys) { + const value = await AsyncStorage.getItem(key); + if (value) { + totalSize += value.length; + } + } + + const sizeInKB = (totalSize / 1024).toFixed(2); + + return { + keys: latentKeys.length, + estimatedSize: `${sizeInKB} KB`, + }; + } catch (error) { + console.error('[LocalStorage] Error getting storage info:', error); + return { keys: 0, estimatedSize: '0 KB' }; + } + } +} diff --git a/src/services/ModelService.tsx b/src/services/ModelService.tsx index 56da189..cbbe787 100644 --- a/src/services/ModelService.tsx +++ b/src/services/ModelService.tsx @@ -1,5 +1,5 @@ -import React, { createContext, useContext, useState, useCallback } from 'react'; -import { RunAnywhere, ModelCategory } from '@runanywhere/core'; +import React, { createContext, useContext, useState, useCallback, useEffect } from 'react'; +import { RunAnywhere, ModelCategory, SDKEnvironment } from '@runanywhere/core'; import { LlamaCPP } from '@runanywhere/llamacpp'; import { ONNX, ModelArtifactType } from '@runanywhere/onnx'; @@ -12,6 +12,10 @@ const MODEL_IDS = { } as const; interface ModelServiceState { + // SDK readiness + isSDKReady: boolean; + sdkError: string | null; + // Download state isLLMDownloading: boolean; isSTTDownloading: boolean; @@ -56,6 +60,10 @@ interface ModelServiceProviderProps { } export const ModelServiceProvider: React.FC = ({ children }) => { + // SDK readiness + const [isSDKReady, setIsSDKReady] = useState(false); + const [sdkError, setSDKError] = useState(null); + // Download state const [isLLMDownloading, setIsLLMDownloading] = useState(false); const [isSTTDownloading, setIsSTTDownloading] = useState(false); @@ -76,6 +84,40 @@ export const ModelServiceProvider: React.FC = ({ chil const [isTTSLoaded, setIsTTSLoaded] = useState(false); const isVoiceAgentReady = isLLMLoaded && isSTTLoaded && isTTSLoaded; + + // Initialize SDK on mount + useEffect(() => { + const initializeSDK = async () => { + try { + console.log('[ModelService] 🚀 Initializing RunAnywhere SDK...'); + + // Initialize SDK + await RunAnywhere.initialize({ + environment: SDKEnvironment.Development, + }); + console.log('[ModelService] ✅ SDK initialized'); + + // Register backends + console.log('[ModelService] đŸ“Ļ Registering backends...'); + await LlamaCPP.register(); + await ONNX.register(); + console.log('[ModelService] ✅ Backends registered'); + + // Register default models + console.log('[ModelService] 🤖 Registering default models...'); + await registerDefaultModels(); + console.log('[ModelService] ✅ Default models registered'); + + setIsSDKReady(true); + console.log('[ModelService] ✅ SDK fully ready'); + } catch (error) { + console.error('[ModelService] ❌ SDK initialization failed:', error); + setSDKError(error instanceof Error ? error.message : 'SDK initialization failed'); + } + }; + + initializeSDK(); + }, []); // Check if model is downloaded (per docs: use getModelInfo and check localPath) const checkModelDownloaded = useCallback(async (modelId: string): Promise => { @@ -123,36 +165,133 @@ export const ModelServiceProvider: React.FC = ({ chil // Download and load STT const downloadAndLoadSTT = useCallback(async () => { - if (isSTTDownloading || isSTTLoading) return; + if (isSTTDownloading || isSTTLoading) { + console.log('[ModelService] ⏭ī¸ STT download/load already in progress, skipping'); + return; + } + + if (!isSDKReady) { + console.log('[ModelService] âŗ SDK not ready yet, waiting...'); + return; + } try { - const isDownloaded = await checkModelDownloaded(MODEL_IDS.stt); + console.log('[ModelService] 🎤 Starting STT download and load...'); - if (!isDownloaded) { + // Step 1: Check if model files already exist on disk (e.g., pre-pushed via adb) + let modelLocalPath: string | null = null; + + try { + // Check if the SDK already knows about the model + const isDownloaded = await checkModelDownloaded(MODEL_IDS.stt); + console.log('[ModelService] đŸ“Ļ SDK reports model downloaded:', isDownloaded); + + if (isDownloaded) { + const info = await RunAnywhere.getModelInfo(MODEL_IDS.stt); + if (info?.localPath) { + modelLocalPath = info.localPath; + console.log('[ModelService] ✅ Model found via SDK at:', modelLocalPath); + } + } + } catch (checkErr) { + console.log('[ModelService] ⚠ī¸ SDK check failed, trying filesystem:', checkErr); + } + + // If SDK doesn't know about it, check the filesystem directly + if (!modelLocalPath) { + try { + const RNFS = require('react-native-fs'); + const documentsDir = RNFS.DocumentDirectoryPath; + const possiblePaths = [ + `${documentsDir}/RunAnywhere/Models/ONNX/${MODEL_IDS.stt}/${MODEL_IDS.stt}`, + `${documentsDir}/RunAnywhere/Models/ONNX/${MODEL_IDS.stt}`, + ]; + + for (const path of possiblePaths) { + const exists = await RNFS.exists(path); + if (exists) { + // Verify it has model files + try { + const contents = await RNFS.readDir(path); + const hasOnnx = contents.some((f: any) => f.name.endsWith('.onnx')); + if (hasOnnx) { + modelLocalPath = path; + console.log('[ModelService] ✅ Model found on disk at:', path); + console.log('[ModelService] 📁 Contents:', contents.map((f: any) => f.name).join(', ')); + break; + } + } catch { + // Not a directory, might be a file + } + } + } + } catch (fsErr) { + console.log('[ModelService] ⚠ī¸ Filesystem check failed:', fsErr); + } + } + + // Step 2: Download if not found on disk + if (!modelLocalPath) { setIsSTTDownloading(true); setSTTDownloadProgress(0); - await RunAnywhere.downloadModel(MODEL_IDS.stt, (progress) => { - setSTTDownloadProgress(progress.progress * 100); - }); + console.log('[ModelService] đŸ“Ĩ Model not found on disk, downloading...'); + try { + await RunAnywhere.downloadModel(MODEL_IDS.stt, (progress) => { + const pct = progress.progress * 100; + setSTTDownloadProgress(pct); + }); + console.log('[ModelService] ✅ STT model download completed'); + } catch (downloadError: any) { + setIsSTTDownloading(false); + const errMsg = downloadError?.message || String(downloadError); + console.error('[ModelService] ❌ STT download failed:', errMsg); + const { Alert } = require('react-native'); + Alert.alert('Download Error', `STT model download failed:\n\n${errMsg}`); + throw downloadError; + } setIsSTTDownloading(false); + + // Get the path from SDK after download + try { + const modelInfo = await RunAnywhere.getModelInfo(MODEL_IDS.stt); + modelLocalPath = modelInfo?.localPath || null; + } catch (e) { + console.error('[ModelService] ❌ Failed to get model path after download:', e); + } + } + + if (!modelLocalPath) { + const errMsg = 'Could not find model files on disk after download'; + console.error('[ModelService] ❌', errMsg); + const { Alert } = require('react-native'); + Alert.alert('Model Error', errMsg); + throw new Error(errMsg); } - // Load the STT model (per docs: loadSTTModel(localPath, 'whisper')) + // Step 3: Load the STT model setIsSTTLoading(true); - const modelInfo = await RunAnywhere.getModelInfo(MODEL_IDS.stt); - if (modelInfo?.localPath) { - await RunAnywhere.loadSTTModel(modelInfo.localPath, 'whisper'); + console.log('[ModelService] 🔄 Loading STT model from:', modelLocalPath); + try { + await RunAnywhere.loadSTTModel(modelLocalPath, 'whisper'); + console.log('[ModelService] ✅ STT model loaded successfully'); setIsSTTLoaded(true); + } catch (loadError: any) { + const errMsg = loadError?.message || String(loadError); + console.error('[ModelService] ❌ STT model load failed:', errMsg); + const { Alert } = require('react-native'); + Alert.alert('Model Load Error', `Path: ${modelLocalPath}\nError: ${errMsg}`); + throw loadError; } + setIsSTTLoading(false); - } catch (error) { - console.error('STT download/load error:', error); + } catch (error: any) { + console.error('[ModelService] ❌ STT pipeline error:', error?.message || error); setIsSTTDownloading(false); setIsSTTLoading(false); } - }, [isSTTDownloading, isSTTLoading, checkModelDownloaded]); + }, [isSTTDownloading, isSTTLoading, checkModelDownloaded, isSDKReady]); // Download and load TTS const downloadAndLoadTTS = useCallback(async () => { @@ -211,6 +350,8 @@ export const ModelServiceProvider: React.FC = ({ chil }, []); const value: ModelServiceState = { + isSDKReady, + sdkError, isLLMDownloading, isSTTDownloading, isTTSDownloading, @@ -260,21 +401,23 @@ export const registerDefaultModels = async () => { }); // STT Model - Sherpa Whisper Tiny English - // Using tar.gz from RunanywhereAI/sherpa-onnx for fast native extraction + // Using tar.gz wrapper from RunAnywhere models + // Fallback: wrapping github download in ghproxy CDN to prevent DNS resolution errors + // Update: Switching to Hugging Face directly since ISP blocks ghproxy and github.com await ONNX.addModel({ id: MODEL_IDS.stt, name: 'Sherpa Whisper Tiny (ONNX)', - url: 'https://github.com/RunanywhereAI/sherpa-onnx/releases/download/runanywhere-models-v1/sherpa-onnx-whisper-tiny.en.tar.gz', + url: 'https://huggingface.co/runanywhere/sherpa-onnx-whisper-tiny.en/resolve/main/sherpa-onnx-whisper-tiny.en.tar.gz', modality: ModelCategory.SpeechRecognition, artifactType: ModelArtifactType.TarGzArchive, - memoryRequirement: 75_000_000, + memoryRequirement: 100_000_000, }); // TTS Model - Piper TTS (US English - Medium quality) await ONNX.addModel({ id: MODEL_IDS.tts, name: 'Piper TTS (US English - Medium)', - url: 'https://github.com/RunanywhereAI/sherpa-onnx/releases/download/runanywhere-models-v1/vits-piper-en_US-lessac-medium.tar.gz', + url: 'https://huggingface.co/runanywhere/vits-piper-en_US-lessac-medium/resolve/main/vits-piper-en_US-lessac-medium.tar.gz', modality: ModelCategory.SpeechSynthesis, artifactType: ModelArtifactType.TarGzArchive, memoryRequirement: 65_000_000, diff --git a/src/services/NegotiationAnalyzer.ts b/src/services/NegotiationAnalyzer.ts new file mode 100644 index 0000000..15b157f --- /dev/null +++ b/src/services/NegotiationAnalyzer.ts @@ -0,0 +1,261 @@ +/** + * 🔒 PRIVACY NOTICE + * All pattern analysis runs locally on device. + * No external API calls. No data leaves this device. + */ + +import { InteractionManager } from 'react-native'; +import { + NegotiationMode, + NegotiationPattern, + DetectedPattern, + TranscriptChunk, + AnalysisResult, +} from '../types/session'; +import { classifyIntent, analyzeTranscriptWindow } from '../ai/intentClassifier'; +import { calculateCognitiveMetrics, calculateFocusScore } from '../ai/scoringEngine'; +import { getPatternDefinition } from '../ai/patternLibrary'; + +/** + * NegotiationAnalyzer - Analyzes transcript for negotiation patterns + * Runs analysis in background to avoid blocking UI + */ +export class NegotiationAnalyzer { + private mode: NegotiationMode; + private sensitivityMultiplier: number; + private analysisInterval: NodeJS.Timeout | null = null; + private isAnalyzing: boolean = false; + + constructor(mode: NegotiationMode = NegotiationMode.SALES, sensitivityMultiplier: number = 1.0) { + this.mode = mode; + this.sensitivityMultiplier = sensitivityMultiplier; + } + + /** + * Analyze a single transcript chunk + */ + analyzeChunk(text: string): DetectedPattern[] { + return classifyIntent(text, this.mode, this.sensitivityMultiplier); + } + + /** + * Analyze multiple transcript chunks (window-based analysis) + */ + analyzeWindow(chunks: TranscriptChunk[], windowSize: number = 1): DetectedPattern[] { + const recentChunks = chunks.slice(-windowSize).map((c) => ({ + text: c.text, + timestamp: c.timestamp, + })); + + return analyzeTranscriptWindow(recentChunks, this.mode, this.sensitivityMultiplier, windowSize); + } + + /** + * Perform comprehensive analysis + * Returns patterns, cognitive metrics, and suggestions + */ + async analyzeSession( + chunks: TranscriptChunk[], + sessionDuration: number + ): Promise { + // Run analysis in background to avoid blocking UI + return new Promise((resolve) => { + InteractionManager.runAfterInteractions(() => { + try { + // Analyze patterns from the most recent chunk to optimize speed and maintain high keyword density + const detectedPatterns = this.analyzeWindow(chunks, 1); + + // Calculate cognitive metrics + const cognitiveMetrics = calculateCognitiveMetrics( + chunks.map((c) => ({ text: c.text, timestamp: c.timestamp })), + sessionDuration + ); + + // Calculate focus score + const focusScore = calculateFocusScore(cognitiveMetrics, sessionDuration); + + // Generate suggestions based on patterns + const suggestions = this.generateSuggestions(detectedPatterns); + + resolve({ + detectedPatterns, + cognitiveMetrics, + suggestions, + focusScore, + }); + } catch (error) { + console.error('[NegotiationAnalyzer] Analysis error:', error); + resolve({ + detectedPatterns: [], + cognitiveMetrics: {}, + suggestions: [], + focusScore: 100, + }); + } + }); + }); + } + + /** + * (Removed `startContinuousAnalysis` to migrate to debounced execution inline) + */ + + /** + * (Removed `stopContinuousAnalysis` to migrate to debounced execution inline) + */ + + /** + * Generate tactical suggestions based on detected patterns + */ + private generateSuggestions(patterns: DetectedPattern[]): string[] { + const suggestions: string[] = []; + const seenSuggestions = new Set(); + + // Get top 3 highest confidence patterns + const topPatterns = patterns.sort((a, b) => b.confidenceScore - a.confidenceScore).slice(0, 3); + + for (const pattern of topPatterns) { + if (!seenSuggestions.has(pattern.suggestion)) { + suggestions.push(pattern.suggestion); + seenSuggestions.add(pattern.suggestion); + } + } + + return suggestions; + } + + /** + * Generate session summary + */ + generateSummary( + allPatterns: DetectedPattern[], + chunks: TranscriptChunk[] + ): { + leverageMoments: string[]; + missedOpportunities: string[]; + objectionCount: number; + positiveSignalCount: number; + tacticalSuggestions: string[]; + keyInsights: string[]; + } { + // Leverage moments (positive signals and commitments) + const leverageMoments: string[] = []; + allPatterns + .filter( + (p) => + p.pattern === NegotiationPattern.POSITIVE_SIGNAL || + p.pattern === NegotiationPattern.COMMITMENT_LANGUAGE + ) + .sort((a, b) => b.confidenceScore - a.confidenceScore) + .slice(0, 5) + .forEach((p) => { + leverageMoments.push( + `${this.formatTimestamp(p.timestamp)}: ${p.context || p.transcript.substring(0, 50)}` + ); + }); + + // Missed opportunities (deflections and negative signals without follow-up) + const missedOpportunities: string[] = []; + const deflections = allPatterns.filter( + (p) => + p.pattern === NegotiationPattern.DEFLECTION || + p.pattern === NegotiationPattern.NEGATIVE_SIGNAL + ); + deflections.slice(0, 3).forEach((p) => { + const patternDef = getPatternDefinition(p.pattern); + missedOpportunities.push( + `${this.formatTimestamp(p.timestamp)}: ${patternDef.displayName} - Consider: ${p.suggestion}` + ); + }); + + // Count objections (budget + authority) + const objectionCount = allPatterns.filter( + (p) => + p.pattern === NegotiationPattern.BUDGET_OBJECTION || + p.pattern === NegotiationPattern.AUTHORITY_PRESSURE + ).length; + + // Count positive signals + const positiveSignalCount = allPatterns.filter( + (p) => p.pattern === NegotiationPattern.POSITIVE_SIGNAL + ).length; + + // Tactical suggestions (most common patterns) + const tacticalSuggestions: string[] = []; + const patternCounts: Record = {}; + + allPatterns.forEach((p) => { + patternCounts[p.pattern] = (patternCounts[p.pattern] || 0) + 1; + }); + + const sortedPatterns = Object.entries(patternCounts) + .sort(([, a], [, b]) => b - a) + .slice(0, 3); + + sortedPatterns.forEach(([pattern]) => { + const patternDef = getPatternDefinition(pattern as NegotiationPattern); + tacticalSuggestions.push( + `Prepare for ${patternDef.displayName}: ${patternDef.suggestions[0]}` + ); + }); + + // Key insights + const keyInsights: string[] = []; + const totalPatterns = allPatterns.length; + + if (positiveSignalCount > objectionCount) { + keyInsights.push('Strong positive momentum - capitalize on enthusiasm'); + } + if (objectionCount > 5) { + keyInsights.push('High objection frequency - focus on value proposition'); + } + if (deflections.length > 3) { + keyInsights.push('Multiple deferrals detected - establish concrete next steps'); + } + if (totalPatterns < 5) { + keyInsights.push('Low pattern detection - conversation may need more depth'); + } + + return { + leverageMoments, + missedOpportunities, + objectionCount, + positiveSignalCount, + tacticalSuggestions, + keyInsights, + }; + } + + /** + * Format timestamp for display + */ + private formatTimestamp(timestamp: number): string { + const date = new Date(timestamp); + return date.toLocaleTimeString('en-US', { + hour: '2-digit', + minute: '2-digit', + second: '2-digit', + }); + } + + /** + * Update mode + */ + setMode(mode: NegotiationMode): void { + this.mode = mode; + } + + /** + * Update sensitivity + */ + setSensitivity(multiplier: number): void { + this.sensitivityMultiplier = multiplier; + } + + /** + * Cleanup + */ + cleanup(): void { + // no continuous analysis to stop + } +} diff --git a/src/services/SessionEngine.ts b/src/services/SessionEngine.ts new file mode 100644 index 0000000..60a7f13 --- /dev/null +++ b/src/services/SessionEngine.ts @@ -0,0 +1,549 @@ +/** + * 🔒 PRIVACY NOTICE + * SessionEngine orchestrates all local processing. + * All data stays on device. Auto-saves every 45 seconds. + * No external API calls. No cloud sync. + */ + +import { + Session, + LiveSessionState, + TranscriptChunk, + DetectedPattern, + NegotiationMode, + CognitiveMetrics, + SessionSummary, +} from '../types/session'; +import { SpeechService } from './SpeechService'; +import { NegotiationAnalyzer } from './NegotiationAnalyzer'; +import { LocalStorageService } from './LocalStorageService'; +import { calculateCognitiveMetrics } from '../ai/scoringEngine'; +import { autoCorrectTranscript } from '../ai/WhisperAutoCorrector'; + +export interface SessionUpdateCallback { + (state: LiveSessionState): void; +} + +const DEBUG_TRANSCRIPTS: Record = { + [NegotiationMode.JOB_INTERVIEW]: [ + 'We usually offer around 6 LPA for this position.', + 'I need to check with my manager before making any decision.', + 'Your profile looks interesting, but we have strict budget constraints.', + 'Let me think about it and get back to you next week.', + "This looks great! I'm really excited about moving forward.", + 'Are you willing to relocate for this role?', + 'I have authority to approve up to 8 LPA maximum.', + ], + [NegotiationMode.SALES]: [ + 'Your product looks interesting, but the price is too high.', + 'We already use a competitor for this service.', + 'I need to get approval from the procurement department.', + 'What kind of discount can you offer us?', + 'We have budget constraints this quarter.', + 'Can we start with a pilot program first?', + ], + [NegotiationMode.STARTUP_PITCH]: [ + 'What is your customer acquisition cost?', + 'We usually invest in later stage companies.', + 'Your valuation expectations seem a bit high.', + 'How do you plan to scale this next year?', + 'I need to check with my partners before committing.', + 'This looks great! I am really excited about moving forward.', + ], + [NegotiationMode.SALARY_RAISE]: [ + 'Company performance has been slow this quarter.', + 'We usually only do appraisals at the end of the year.', + 'I agree you have done good work, but 30% is too much.', + 'Let me review the budget with HR and get back to you.', + 'Can we look at performance bonuses instead of a base hike?', + 'You are a valuable asset to the team.', + ], + [NegotiationMode.INVESTOR_MEETING]: [ + 'Your burn rate is concerning for this stage.', + 'What is the exact runway you have left?', + 'We need to see more traction before releasing the next tranche.', + 'Who else is participating in this funding round?', + 'Your Go-to-market strategy seems expensive.', + ], + [NegotiationMode.CLIENT_NEGOTIATION]: [ + 'The timeline for these deliverables is too tight.', + 'Can we reduce the retainer fee for the first three months?', + 'We need unlimited revisions included in this contract.', + 'We are waiting on our legal team to review the MSA.', + 'This scope is larger than what we initially discussed.', + ], + [NegotiationMode.CUSTOM_SCENARIO]: [ + 'I need to check with my manager before making any decision.', + 'Let me think about it and get back to you next week.', + "This looks great! I'm really excited about moving forward.", + 'We have budget constraints this quarter.', + 'Can you send me more information via email?', + ], +}; + +/** + * SessionEngine - Orchestrates the entire session lifecycle + * Manages recording, transcription, analysis, and auto-save + */ +export class SessionEngine { + private sessionId: string | null = null; + private mode: NegotiationMode = NegotiationMode.SALES; + private speechService: SpeechService; + private analyzer: NegotiationAnalyzer; + private state: LiveSessionState; + private updateCallback: SessionUpdateCallback | null = null; + private autoSaveInterval: NodeJS.Timeout | null = null; + private analysisDebounceTrigger: NodeJS.Timeout | null = null; + private autoSaveIntervalMs: number = 45000; // 45 seconds as requested + private debugTranscriptIndex: number = 0; + private debugInterval: NodeJS.Timeout | null = null; + private debugMode: boolean = false; + + constructor() { + this.speechService = new SpeechService(); + this.analyzer = new NegotiationAnalyzer(); + this.state = this.createInitialState(); + } + + /** + * Create initial session state + */ + private createInitialState(): LiveSessionState { + return { + isRecording: false, + startTime: 0, + duration: 0, + transcript: [], + detectedPatterns: [], + currentFocusScore: 100, + audioLevel: 0, + lastAutoSave: 0, + }; + } + + /** + * Start a new session + */ + async startSession(mode: NegotiationMode, onUpdate: SessionUpdateCallback): Promise { + try { + // Generate session ID + this.sessionId = `session_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`; + this.mode = mode; + this.updateCallback = onUpdate; + + // Reset state + this.state = this.createInitialState(); + this.state.startTime = Date.now(); + this.state.isRecording = true; + + // Configure analyzer + const settings = await LocalStorageService.getSettings(); + this.analyzer.setMode(mode); + this.analyzer.setSensitivity(settings.patternSensitivity); + + this.debugMode = settings.debugMode || false; + + if (this.debugMode) { + console.log('[SessionEngine] 🐛 DEBUG MODE ENABLED - Using hardcoded transcripts'); + console.log('[SessionEngine] 🐛 This bypasses STT and injects test data every 7 seconds'); + + // Start debug transcript injection + this.startDebugTranscripts(); + } else { + // Start speech service + const started = await this.speechService.startRecording( + this.onTranscription.bind(this), + this.onAudioLevel.bind(this) + ); + + if (!started) { + console.error('[SessionEngine] Failed to start speech service'); + return false; + } + } + + // Removed: analyzer.startContinuousAnalysis + // Now running debounced in onTranscription + + // Start auto-save (every 45 seconds) + this.startAutoSave(); + + console.log('[SessionEngine] Session started:', this.sessionId); + this.notifyUpdate(); + + return true; + } catch (error) { + console.error('[SessionEngine] Error starting session:', error); + return false; + } + } + + /** + * Stop the current session and save + */ + async stopSession(): Promise { + try { + if (!this.sessionId) { + return null; + } + + console.log('[SessionEngine] Stopping session:', this.sessionId); + + // Stop services + if (this.analysisDebounceTrigger) clearTimeout(this.analysisDebounceTrigger); + this.stopAutoSave(); + this.stopDebugTranscripts(); + + if (!this.debugMode) { + await this.speechService.stopRecording(); + } + + this.state.isRecording = false; + this.state.duration = Date.now() - this.state.startTime; + + // Calculate final cognitive metrics + const cognitiveMetrics = calculateCognitiveMetrics( + this.state.transcript.map((c) => ({ text: c.text, timestamp: c.timestamp })), + this.state.duration + ); + + // Generate summary + const summaryData = this.analyzer.generateSummary( + this.state.detectedPatterns, + this.state.transcript + ); + + const summary: SessionSummary = { + leverageMoments: summaryData.leverageMoments, + missedOpportunities: summaryData.missedOpportunities, + objectionCount: summaryData.objectionCount, + positiveSignalCount: summaryData.positiveSignalCount, + tacticalSuggestions: summaryData.tacticalSuggestions, + keyInsights: summaryData.keyInsights, + }; + + // Create final session object + const session: Session = { + id: this.sessionId, + timestamp: this.state.startTime, + duration: this.state.duration, + mode: this.mode, + transcript: this.state.transcript, + detectedPatterns: this.state.detectedPatterns, + cognitiveMetrics, + summary, + }; + + // Save to storage + await LocalStorageService.saveSession(session); + + // Recalculate stats + await LocalStorageService.calculateStats(); + + console.log('[SessionEngine] Session saved:', session.id); + + // Clean up + this.sessionId = null; + this.updateCallback = null; + + return session; + } catch (error) { + console.error('[SessionEngine] Error stopping session:', error); + return null; + } + } + + /** + * Cancel the current session without saving + */ + async cancelSession(): Promise { + console.log('[SessionEngine] Canceling session'); + + if (this.analysisDebounceTrigger) clearTimeout(this.analysisDebounceTrigger); + this.stopAutoSave(); + this.stopDebugTranscripts(); + + if (!this.debugMode) { + await this.speechService.cancelRecording(); + } + + this.state = this.createInitialState(); + this.sessionId = null; + this.updateCallback = null; + } + + private onTranscription(text: string, timestamp: number): void { + const rawText = text.trim(); + if (!rawText) return; + + // Remove blank tokens just in case + let cleanedText = rawText.replace(/\[BLANK_AUDIO\]/g, '').replace(/\[ Pause \]/gi, '').trim(); + if (!cleanedText) return; + + // 💡 Auto-correct Whisper 'tiny' hallucinations based on context vocabulary + cleanedText = autoCorrectTranscript(cleanedText, this.mode); + + const transcriptLen = this.state.transcript.length; + let isNewSentence = true; + + if (transcriptLen > 0) { + const lastChunk = this.state.transcript[transcriptLen - 1]; + // Only break to a new bubble if the previous chunk ended with punctuation or it's a long pause + const isFinished = /[.!?]\s*$/.test(lastChunk.text); + const timeSinceLastUpdate = timestamp - lastChunk.timestamp; + const isLongPause = timeSinceLastUpdate > 10000; + + if (!isFinished && !isLongPause) { + // BREAK REACT'S REFERENCE EQUALITY: Create a completely new object + // so FlatList detects that this item changed and actually re-renders! + this.state.transcript[transcriptLen - 1] = { + ...lastChunk, + text: `${lastChunk.text} ${cleanedText}`.trim(), + timestamp: timestamp, + }; + isNewSentence = false; + } + } + + if (isNewSentence) { + const chunk: TranscriptChunk = { + id: `chunk_${timestamp}_${Math.random().toString(36).substr(2, 9)}`, + text: cleanedText, + timestamp, + }; + this.state.transcript.push(chunk); + } + this.state.duration = Date.now() - this.state.startTime; + this.triggerDebouncedAnalysis(); + } + + private triggerDebouncedAnalysis() { + this.notifyUpdate(); + + // Trigger debounced analysis inline + if (this.analysisDebounceTrigger) { + clearTimeout(this.analysisDebounceTrigger); + } + + this.analysisDebounceTrigger = setTimeout(() => { + if (this.state.isRecording || this.debugMode) { + this.analyzer.analyzeSession(this.state.transcript, this.state.duration) + .then((result) => this.onAnalysisResult(result)) + .catch((err) => console.error('[SessionEngine] Analysis error:', err)); + } + }, 300); + } + + /** + * Handle audio level updates + */ + private onAudioLevel(level: number): void { + this.state.audioLevel = level; + this.notifyUpdate(); + } + + /** + * Handle analysis results + */ + private onAnalysisResult(result: any): void { + console.log('[SessionEngine] 🧠 onAnalysisResult() called'); + console.log('[SessionEngine] đŸŽ¯ Detected patterns count:', result.detectedPatterns.length); + console.log('[SessionEngine] 📊 Focus score:', result.focusScore); + + // Add new patterns (avoid duplicates) + const existingPatternIds = new Set(this.state.detectedPatterns.map((p) => p.id)); + let newPatternsCount = 0; + + result.detectedPatterns.forEach((pattern: DetectedPattern) => { + if (!existingPatternIds.has(pattern.id)) { + this.state.detectedPatterns.push(pattern); + newPatternsCount++; + + console.log('[SessionEngine] 🆕 New pattern detected:', { + pattern: pattern.pattern, + confidence: pattern.confidenceScore, + suggestion: pattern.suggestion, + }); + + // Mark related transcript chunks + this.state.transcript.forEach((chunk) => { + if (chunk.text === pattern.transcript) { + chunk.hasPattern = true; + } + }); + } + }); + + // Sort patterns by confidence (highest first) so counter-strategy picks the best one + this.state.detectedPatterns.sort((a, b) => b.confidenceScore - a.confidenceScore); + + // Update focus score + this.state.currentFocusScore = result.focusScore; + + console.log('[SessionEngine] ✅ Analysis complete'); + console.log('[SessionEngine] 🆕 New patterns added:', newPatternsCount); + console.log('[SessionEngine] 📊 Total patterns:', this.state.detectedPatterns.length); + + if (newPatternsCount > 0) { + // Force new array references so React detects the change + this.state.detectedPatterns = [...this.state.detectedPatterns]; + } + + this.notifyUpdate(); + } + + /** + * Start auto-save timer + */ + private startAutoSave(): void { + this.autoSaveInterval = setInterval(() => { + this.performAutoSave(); + }, this.autoSaveIntervalMs); + + console.log('[SessionEngine] Auto-save started (every 45s)'); + } + + /** + * Stop auto-save timer + */ + private stopAutoSave(): void { + if (this.autoSaveInterval) { + clearInterval(this.autoSaveInterval); + this.autoSaveInterval = null; + console.log('[SessionEngine] Auto-save stopped'); + } + } + + /** + * Perform auto-save (saves current state) + */ + private async performAutoSave(): Promise { + if (!this.sessionId) { + return; + } + + try { + // Calculate current cognitive metrics + const cognitiveMetrics = calculateCognitiveMetrics( + this.state.transcript.map((c) => ({ text: c.text, timestamp: c.timestamp })), + this.state.duration + ); + + // Create partial session for auto-save + const partialSession: Session = { + id: this.sessionId, + timestamp: this.state.startTime, + duration: Date.now() - this.state.startTime, + mode: this.mode, + transcript: this.state.transcript, + detectedPatterns: this.state.detectedPatterns, + cognitiveMetrics, + summary: { + leverageMoments: [], + missedOpportunities: [], + objectionCount: 0, + positiveSignalCount: 0, + tacticalSuggestions: [], + keyInsights: [], + }, + }; + + await LocalStorageService.saveSession(partialSession); + this.state.lastAutoSave = Date.now(); + + console.log('[SessionEngine] Auto-save completed'); + } catch (error) { + console.error('[SessionEngine] Auto-save error:', error); + } + } + + /** + * Get current session state + */ + getState(): LiveSessionState { + return { ...this.state }; + } + + /** + * Notify update callback + */ + private notifyUpdate(): void { + if (this.updateCallback) { + console.log('[SessionEngine] 🔔 Notifying state update to UI'); + // Create new array references so React's useState detects the change + this.updateCallback({ + ...this.state, + transcript: [...this.state.transcript], + detectedPatterns: [...this.state.detectedPatterns], + }); + } else { + console.log('[SessionEngine] ⚠ī¸ No update callback registered'); + } + } + + /** + * Update duration (call from UI timer) + */ + updateDuration(): void { + if (this.state.isRecording) { + this.state.duration = Date.now() - this.state.startTime; + this.notifyUpdate(); + } + } + + /** + * Cleanup + */ + cleanup(): void { + if (this.analysisDebounceTrigger) clearTimeout(this.analysisDebounceTrigger); + this.analyzer.cleanup(); + this.speechService.cleanup(); + this.stopAutoSave(); + this.stopDebugTranscripts(); + } + + /** + * Start debug transcript injection (bypasses STT) + */ + private startDebugTranscripts(): void { + console.log('[SessionEngine] 🐛 Starting debug transcript injection'); + this.debugTranscriptIndex = 0; + + // Inject first transcript immediately + this.injectDebugTranscript(); + + // Inject new transcript every 7 seconds + this.debugInterval = setInterval(() => { + this.injectDebugTranscript(); + }, 7000); + } + + /** + * Stop debug transcript injection + */ + private stopDebugTranscripts(): void { + if (this.debugInterval) { + clearInterval(this.debugInterval); + this.debugInterval = null; + console.log('[SessionEngine] 🐛 Debug transcript injection stopped'); + } + } + + /** + * Inject a single debug transcript + */ + private injectDebugTranscript(): void { + const activeTranscripts = DEBUG_TRANSCRIPTS[this.mode] || DEBUG_TRANSCRIPTS[NegotiationMode.CUSTOM_SCENARIO]; + + if (this.debugTranscriptIndex >= activeTranscripts.length) { + console.log('[SessionEngine] 🐛 All debug transcripts injected, cycling back to start'); + this.debugTranscriptIndex = 0; + } + + const text = activeTranscripts[this.debugTranscriptIndex]; + console.log(`[SessionEngine] 🐛 Injecting debug transcript (${this.mode}):`, text); + + this.onTranscription(text, Date.now()); + this.debugTranscriptIndex++; + } +} diff --git a/src/services/SpeechService.ts b/src/services/SpeechService.ts new file mode 100644 index 0000000..3e82dc3 --- /dev/null +++ b/src/services/SpeechService.ts @@ -0,0 +1,451 @@ +/** + * 🔒 PRIVACY NOTICE + * All speech processing runs locally on device using RunAnywhere SDK. + * Audio is processed in memory and transcribed locally. + * No audio data is sent to external servers. + */ + +import { NativeModules, Platform, PermissionsAndroid } from 'react-native'; +import { RunAnywhere } from '@runanywhere/core'; + +const { NativeAudioModule } = NativeModules; + +// ============================================================================ +// STT MODEL STATUS HELPERS +// NOTE: Model loading is handled by ModelService.downloadAndLoadSTT() in App.tsx +// ============================================================================ + +/** + * Check if STT model is ready (async check with RunAnywhere SDK) + */ +export const checkSTTModelReady = async (): Promise => { + try { + // First check if a model is actually loaded in the ONNX backend + const isLoaded = await RunAnywhere.isSTTModelLoaded(); + if (isLoaded) { + console.log('[SpeechService] ✅ STT model is loaded (isSTTModelLoaded=true)'); + return true; + } + + // Fallback: check if model has a local path (downloaded but maybe not loaded) + const modelInfo = await RunAnywhere.getModelInfo('sherpa-onnx-whisper-base.en'); + const hasLocalPath = !!modelInfo?.localPath; + console.log('[SpeechService] STT model check: isLoaded=false, hasLocalPath=', hasLocalPath); + + if (hasLocalPath) { + // Model is downloaded but not loaded - try to load it + console.log('[SpeechService] 🔄 Model downloaded but not loaded, attempting to load...'); + try { + await RunAnywhere.loadSTTModel(modelInfo!.localPath!, 'whisper'); + console.log('[SpeechService] ✅ STT model loaded successfully'); + return true; + } catch (loadError) { + console.error('[SpeechService] ❌ Failed to load STT model:', loadError); + return false; + } + } + + return false; + } catch (error) { + console.log('[SpeechService] STT model check failed:', error); + return false; + } +}; + +export interface TranscriptionCallback { + (text: string, timestamp: number): void; +} + +export interface AudioLevelCallback { + (level: number): void; +} + +/** + * SpeechService - Handles audio recording and speech-to-text + */ +export class SpeechService { + private isRecording: boolean = false; + private recordingPath: string | null = null; + private recordingStartTime: number = 0; + private audioLevelInterval: NodeJS.Timeout | null = null; + private transcriptionInterval: NodeJS.Timeout | null = null; + private transcriptionCallback: TranscriptionCallback | null = null; + private audioLevelCallback: AudioLevelCallback | null = null; + private lastTranscriptionTime: number = 0; + + /** + * Request microphone permission (Android only) + */ + async requestPermission(): Promise { + if (Platform.OS !== 'android') { + console.log('[SpeechService] ✅ Platform is iOS, permission auto-granted'); + return true; + } + + try { + console.log('[SpeechService] 🎤 Requesting RECORD_AUDIO permission...'); + + const granted = await PermissionsAndroid.request( + PermissionsAndroid.PERMISSIONS.RECORD_AUDIO, + { + title: 'Microphone Permission', + message: 'Latent needs access to your microphone to transcribe speech.', + buttonNeutral: 'Ask Me Later', + buttonNegative: 'Cancel', + buttonPositive: 'OK', + } + ); + + const isGranted = granted === PermissionsAndroid.RESULTS.GRANTED; + console.log(`[SpeechService] ${isGranted ? '✅' : '❌'} Permission ${granted}`); + + return isGranted; + } catch (error) { + console.error('[SpeechService] ❌ Permission error:', error); + return false; + } + } + + /** + * Start recording audio + */ + async startRecording( + onTranscription: TranscriptionCallback, + onAudioLevel?: AudioLevelCallback + ): Promise { + try { + console.log('[SpeechService] 🎙ī¸ startRecording() called'); + + // Check native module + if (!NativeAudioModule) { + console.error('[SpeechService] ❌ NativeAudioModule not available'); + return false; + } + console.log('[SpeechService] ✅ NativeAudioModule available'); + + // Request permission + const hasPermission = await this.requestPermission(); + if (!hasPermission) { + console.error('[SpeechService] ❌ Microphone permission denied'); + return false; + } + console.log('[SpeechService] ✅ Microphone permission granted'); + + // Start native recording + console.log('[SpeechService] đŸ“ŧ Starting native recording...'); + const result = await NativeAudioModule.startRecording(); + + this.isRecording = true; + this.recordingPath = result.path; + this.recordingStartTime = Date.now(); + this.transcriptionCallback = onTranscription; + this.audioLevelCallback = onAudioLevel || null; + + console.log('[SpeechService] ✅ Recording started'); + console.log('[SpeechService] 📁 Recording path:', result.path); + console.log('[SpeechService] ⏰ Recording start time:', this.recordingStartTime); + + // Start audio level polling + if (onAudioLevel) { + this.startAudioLevelPolling(); + console.log('[SpeechService] 🎚ī¸ Audio level polling started'); + } + + // Start continuous transcription (every 5 seconds) + this.startContinuousTranscription(); + console.log('[SpeechService] 🔄 Continuous transcription started'); + + return true; + } catch (error) { + console.error('[SpeechService] ❌ Error starting recording:', error); + this.isRecording = false; + return false; + } + } + + /** + * Stop recording and transcribe + */ + async stopRecording(): Promise { + try { + if (!this.isRecording || !NativeAudioModule) { + return null; + } + + // Stop audio level polling + this.stopAudioLevelPolling(); + + // Stop continuous transcription + this.stopContinuousTranscription(); + + // Stop native recording + console.log('[SpeechService] Stopping recording...'); + const result = await NativeAudioModule.stopRecording(); + + this.isRecording = false; + const audioPath = result.path || this.recordingPath; + + if (!audioPath) { + console.error('[SpeechService] No audio path available'); + return null; + } + + // Transcribe the audio + console.log('[SpeechService] Transcribing audio...'); + const transcription = await this.transcribeAudio(audioPath); + + // Clean up + this.recordingPath = null; + this.transcriptionCallback = null; + this.audioLevelCallback = null; + + return transcription; + } catch (error) { + console.error('[SpeechService] Error stopping recording:', error); + this.isRecording = false; + return null; + } + } + + /** + * Cancel recording without transcribing + */ + async cancelRecording(): Promise { + try { + if (!this.isRecording || !NativeAudioModule) { + return; + } + + // Stop audio level polling + this.stopAudioLevelPolling(); + + // Stop continuous transcription + this.stopContinuousTranscription(); + + // Cancel native recording + await NativeAudioModule.cancelRecording(); + + this.isRecording = false; + this.recordingPath = null; + this.transcriptionCallback = null; + this.audioLevelCallback = null; + + console.log('[SpeechService] Recording cancelled'); + } catch (error) { + console.error('[SpeechService] Error cancelling recording:', error); + this.isRecording = false; + } + } + + /** + * Transcribe audio file using RunAnywhere STT + */ + private async transcribeAudio(audioPath: string): Promise { + try { + console.log('[SpeechService] đŸŽ¯ transcribeAudio() called'); + console.log('[SpeechService] 📁 Audio path:', audioPath); + + // Check if STT model is ready + const modelReady = await checkSTTModelReady(); + if (!modelReady) { + console.error('[SpeechService] ❌ STT model not ready'); + throw new Error('STT model not loaded. Please ensure the model is downloaded.'); + } + + console.log('[SpeechService] ✅ STT model ready'); + + // Use the provided exact domain prompt bias string + const DOMAIN_PROMPT = "This conversation includes words like salary, fresher, offer, budget, negotiation, interview, compensation, package, client, proposal, anchor, objection, startup, candidate."; + console.log("[STT] Using domain prompt bias."); + + // Use RunAnywhere.transcribeFile() API for file-based transcription + console.log('[SpeechService] 🤖 Running STT inference on file...'); + const result = await RunAnywhere.transcribeFile(audioPath, { + initialPrompt: DOMAIN_PROMPT + } as any); + + const transcription = result.text || ''; + console.log('[SpeechService] ✅ Transcription complete'); + console.log('[SpeechService] 📝 Text length:', transcription.length, 'chars'); + console.log('[SpeechService] 📝 Text:', transcription); + + // Call callback with transcription + if (this.transcriptionCallback) { + console.log('[SpeechService] 📤 Calling transcription callback'); + this.transcriptionCallback(transcription, Date.now()); + } else { + console.log('[SpeechService] ⚠ī¸ No transcription callback registered'); + } + + return transcription; + } catch (error) { + console.error('[SpeechService] ❌ Transcription error:', error); + return null; + } + } + + private startContinuousTranscription(): void { + this.lastTranscriptionTime = Date.now(); + + // Check interval every 500ms, but only transcribe when 1.5 seconds have passed + this.transcriptionInterval = setInterval(async () => { + try { + if (!this.isRecording) return; + + const currentTime = Date.now(); + const timeSinceLast = currentTime - this.lastTranscriptionTime; + + // Extract and transcribe every 5.0 seconds for maximum stability and sentence context + if (timeSinceLast >= 5000) { + console.log(`[SpeechService] 🔄 Processing ${timeSinceLast}ms chunk...`); + + if (!NativeAudioModule) { + console.warn('[SpeechService] ⚠ī¸ NativeAudioModule not available'); + return; + } + + try { + // Get EXACTLY the audio recorded since the last transcription. No padding, no overlap. + // This guarantees that Whisper evaluates the new chunk as a completely unique, sequential audio segment. + const snapshot = await NativeAudioModule.getRecentAudioSnapshot(timeSinceLast); + const snapshotPath = snapshot.path; + + if (!snapshotPath || snapshot.fileSize === 0) { + console.log('[SpeechService] ⚠ī¸ No audio data in snapshot yet'); + return; + } + + console.log( + `[SpeechService] 📸 Got ${timeSinceLast}ms snapshot:`, + snapshotPath, + 'size:', + snapshot.fileSize + ); + + // Use the provided exact domain prompt bias string + const DOMAIN_PROMPT = "This conversation includes words like salary, fresher, offer, budget, negotiation, interview, compensation, package, client, proposal, anchor, objection, startup, candidate."; + console.log("[STT] Using domain prompt bias."); + + // Transcribe the small snapshot file (O(1) time) + const result = await RunAnywhere.transcribeFile(snapshotPath, { + initialPrompt: DOMAIN_PROMPT + } as any); + const newText = (result.text || '').trim(); + + if (newText && newText.length > 0) { + // Sanitize whisper STT tokens that appear during silence + const sanitized = newText + .replace(/\[BLANK_AUDIO\]/g, '') + .replace(/\[ Pause \]/gi, '') + .replace(/\(Pause\)/gi, '') + .trim(); + + if (sanitized.length > 0) { + console.log('[SpeechService] ✅ New chunk transcription:', sanitized); + if (this.transcriptionCallback) { + this.transcriptionCallback(sanitized, currentTime); + } + } else { + console.log('[SpeechService] ⚠ī¸ Chunk was only silence/pause tokens'); + } + } else { + console.log('[SpeechService] ⚠ī¸ Empty transcription result for chunk'); + } + + // Important: Only update time if successful, so we don't drop audio if it failed + this.lastTranscriptionTime = currentTime; + + } catch (transcribeError) { + console.log('[SpeechService] ⚠ī¸ Continuous transcription skipped:', transcribeError); + } + } + } catch (error) { + console.error('[SpeechService] ❌ Continuous transcription error:', error); + } + }, 1000); // Check interval + } + + /** + * Stop continuous transcription + */ + private stopContinuousTranscription(): void { + if (this.transcriptionInterval) { + clearInterval(this.transcriptionInterval); + this.transcriptionInterval = null; + } + } + + /** + * Start polling audio levels + */ + private startAudioLevelPolling(): void { + let sampleCount = 0; + + this.audioLevelInterval = setInterval(async () => { + try { + if (!NativeAudioModule || !this.isRecording) { + return; + } + + const levelResult = await NativeAudioModule.getAudioLevel(); + const level = levelResult.level || 0; + + // Log first 3 samples for debugging + if (sampleCount < 3) { + console.log(`[SpeechService] 🎚ī¸ Audio level sample ${sampleCount + 1}:`, level); + sampleCount++; + } else if (sampleCount === 3) { + console.log('[SpeechService] 🎚ī¸ Audio level polling working (further logs suppressed)'); + sampleCount++; + } + + if (this.audioLevelCallback) { + this.audioLevelCallback(level); + } + } catch (error) { + // Ignore polling errors + } + }, 100); // Poll every 100ms + } + + /** + * Stop polling audio levels + */ + private stopAudioLevelPolling(): void { + if (this.audioLevelInterval) { + clearInterval(this.audioLevelInterval); + this.audioLevelInterval = null; + } + } + + /** + * Get recording status + */ + getIsRecording(): boolean { + return this.isRecording; + } + + /** + * Get recording duration in milliseconds + */ + getRecordingDuration(): number { + if (!this.isRecording) { + return 0; + } + return Date.now() - this.recordingStartTime; + } + + /** + * Cleanup on service destruction + */ + cleanup(): void { + this.stopAudioLevelPolling(); + this.stopContinuousTranscription(); + if (this.isRecording) { + this.cancelRecording(); + } + } +} + +// Export singleton instance +export const speechService = new SpeechService(); diff --git a/src/state/sessionReducer.ts b/src/state/sessionReducer.ts new file mode 100644 index 0000000..e385356 --- /dev/null +++ b/src/state/sessionReducer.ts @@ -0,0 +1,305 @@ +/** + * 🔒 PRIVACY NOTICE + * All session state management runs locally on device. + * No data leaves this device. + */ + +/** + * SESSION REDUCER + * + * Single source of truth for the entire live session lifecycle. + * + * WHY REDUCER PREVENTS STALE STATE: + * - Each action produces a deterministic new state from the previous state. + * - No stale closures: the reducer always gets the latest state as its first arg. + * - Unlike multiple useState() hooks, there's one atomic update per action — + * no half-updated states where transcript is new but suggestions are old. + * + * WHY COOLDOWN AVOIDS UI SPAM: + * - TACTIC_DETECTED checks lastTacticAtMs before updating suggestions. + * - If the same tactic fires within 8000ms, the action is ignored. + * - This prevents the suggestion card from flickering on/off rapidly. + */ + +import { + NegotiationMode, + NegotiationPattern, + TranscriptChunk, + DetectedPattern, + CounterStrategy, +} from '../types/session'; +import { generateCounterStrategies } from '../services/CounterStrategyEngine'; + +// ─────────────────────────── Session Status ─────────────────────────── + +export type SessionStatus = 'IDLE' | 'RUNNING' | 'ENDED'; + +// ─────────────────────────── State ─────────────────────────── + +export interface SessionState { + /** Current session lifecycle status */ + status: SessionStatus; + /** Negotiation mode for pattern detection weights */ + mode: NegotiationMode; + /** Live transcript chunks */ + transcript: TranscriptChunk[]; + /** Full transcript text (appended each chunk) */ + transcriptText: string; + /** All detected patterns so far */ + detectedPatterns: DetectedPattern[]; + /** Currently detected tactic (highest confidence), or null */ + tactic: NegotiationPattern | null; + /** Confidence of the current tactic (0-100) */ + confidence: number; + /** Counter-strategy suggestions for the current tactic */ + suggestions: string[]; + /** Active counter-strategy card data, or null */ + activeStrategy: CounterStrategy | null; + /** Timestamp (ms) when the last tactic was accepted (for cooldown) */ + lastTacticAtMs: number; + /** Session start time */ + startTime: number; + /** Session duration in ms */ + duration: number; + /** Current cognitive focus score (0-100) */ + focusScore: number; + /** Current audio level (0-1) */ + audioLevel: number; + /** Last error message */ + error: string | null; +} + +// ─────────────────────────── Initial State ─────────────────────────── + +export const createInitialState = ( + mode: NegotiationMode = NegotiationMode.SALES, +): SessionState => ({ + status: 'IDLE', + mode, + transcript: [], + transcriptText: '', + detectedPatterns: [], + tactic: null, + confidence: 0, + suggestions: [], + activeStrategy: null, + lastTacticAtMs: 0, + startTime: 0, + duration: 0, + focusScore: 100, + audioLevel: 0, + error: null, +}); + +// ─────────────────────────── Action Types ─────────────────────────── + +export type SessionAction = + | { type: 'START_SESSION'; mode: NegotiationMode; startTime: number } + | { type: 'STOP_SESSION' } + | { type: 'TRANSCRIPT_CHUNK'; chunk: TranscriptChunk } + | { type: 'SYNC_TRANSCRIPT'; transcript: TranscriptChunk[] } + | { + type: 'TACTIC_DETECTED'; + patterns: DetectedPattern[]; + focusScore: number; + timestampMs: number; + } + | { type: 'UPDATE_DURATION'; duration: number } + | { type: 'UPDATE_AUDIO_LEVEL'; level: number } + | { type: 'SET_ERROR'; message: string } + | { type: 'RESET' }; + +// ─────────────────────────── Constants ─────────────────────────── + +/** Minimum confidence to accept a tactic (45%) */ +const CONFIDENCE_THRESHOLD = 45; + +/** Cooldown period to prevent same-tactic spam (8 seconds) */ +const TACTIC_COOLDOWN_MS = 8000; + +// ─────────────────────────── Reducer ─────────────────────────── + +export const sessionReducer = ( + state: SessionState, + action: SessionAction, +): SessionState => { + switch (action.type) { + // ─── Start Session ─── + case 'START_SESSION': { + console.log('[sessionReducer] â–ļī¸ START_SESSION mode:', action.mode); + return { + ...createInitialState(action.mode), + status: 'RUNNING', + startTime: action.startTime, + }; + } + + // ─── Stop Session ─── + case 'STOP_SESSION': { + console.log('[sessionReducer] ⏚ī¸ STOP_SESSION'); + return { + ...state, + status: 'ENDED', + }; + } + + // ─── Append Transcript Chunk ─── + case 'TRANSCRIPT_CHUNK': { + // Guard: ignore if session not running + if (state.status !== 'RUNNING') { + console.log('[sessionReducer] ⚠ī¸ TRANSCRIPT_CHUNK ignored — session not running'); + return state; + } + + console.log('[sessionReducer] 📝 TRANSCRIPT_CHUNK:', action.chunk.text.substring(0, 40)); + + return { + ...state, + transcript: [...state.transcript, action.chunk], + transcriptText: state.transcriptText + ' ' + action.chunk.text, + }; + } + + // ─── Sync Transcript Array ─── + case 'SYNC_TRANSCRIPT': { + if (state.status !== 'RUNNING') return state; + return { + ...state, + transcript: action.transcript, + transcriptText: action.transcript.map(c => c.text).join(' '), + }; + } + + // ─── Tactic Detected (from debounced analysis) ─── + case 'TACTIC_DETECTED': { + // Guard 1: ignore if session not running (prevents unmounted updates) + if (state.status !== 'RUNNING') { + console.log('[sessionReducer] ⚠ī¸ TACTIC_DETECTED ignored — session not running'); + return state; + } + + const { patterns, focusScore, timestampMs } = action; + + // Merge new patterns (avoid duplicates) + const existingIds = new Set(state.detectedPatterns.map((p) => p.id)); + const newPatterns = patterns.filter((p) => !existingIds.has(p.id)); + const allPatterns = [...state.detectedPatterns, ...newPatterns]; + + // Mark transcript chunks that have patterns + const updatedTranscript = state.transcript.map((chunk) => { + const hasPattern = newPatterns.some((p) => p.transcript === chunk.text); + return hasPattern ? { ...chunk, hasPattern: true } : chunk; + }); + + // If no NEW patterns were detected on this tick, do not reset or calculate strategies. + // Just update the base focusScore. + if (newPatterns.length === 0) { + return { + ...state, + detectedPatterns: allPatterns, + transcript: updatedTranscript, + focusScore, + }; + } + + // Get the strongest pattern from the CURRENT newly analyzed audio block! + // We ignore allPatterns entirely here to prevent old history from overriding fresh intel. + newPatterns.sort((a, b) => b.confidenceScore - a.confidenceScore); + const topPattern = newPatterns[0] || null; + + if (!topPattern) { + return { + ...state, + detectedPatterns: allPatterns, + transcript: updatedTranscript, + focusScore, + }; + } + + console.log( + '[sessionReducer] đŸŽ¯ NEW TACTIC_DETECTED:', + topPattern.pattern, + 'confidence:', topPattern.confidenceScore, + ); + + // Guard 2: confidence threshold + if (topPattern.confidenceScore < CONFIDENCE_THRESHOLD) { + console.log( + `[sessionReducer] ❌ Confidence ${topPattern.confidenceScore}% below threshold ${CONFIDENCE_THRESHOLD}%`, + ); + return { + ...state, + detectedPatterns: allPatterns, + transcript: updatedTranscript, + focusScore, + }; + } + + // Guard 3: cooldown — don't repeat same tactic within 8s + if ( + topPattern.pattern === state.tactic && + timestampMs - state.lastTacticAtMs < TACTIC_COOLDOWN_MS + ) { + const remaining = Math.round( + (TACTIC_COOLDOWN_MS - (timestampMs - state.lastTacticAtMs)) / 1000, + ); + console.log( + `[sessionReducer] âŗ Cooldown: "${topPattern.pattern}" — ${remaining}s remaining`, + ); + return { + ...state, + detectedPatterns: allPatterns, + transcript: updatedTranscript, + focusScore, + }; + } + + // All guards passed — generate counter-strategy suggestions + console.log('[sessionReducer] ✅ Generating counter-strategy for:', topPattern.pattern); + + const strategy = generateCounterStrategies( + topPattern.pattern, + topPattern.confidenceScore, + TACTIC_COOLDOWN_MS, + CONFIDENCE_THRESHOLD, + ); + + return { + ...state, + detectedPatterns: allPatterns, + transcript: updatedTranscript, + tactic: topPattern.pattern, + confidence: topPattern.confidenceScore, + suggestions: strategy ? strategy.suggestions : state.suggestions, + activeStrategy: strategy || state.activeStrategy, + lastTacticAtMs: timestampMs, + focusScore, + }; + } + + // ─── Update Duration ─── + case 'UPDATE_DURATION': { + return { ...state, duration: action.duration }; + } + + // ─── Update Audio Level ─── + case 'UPDATE_AUDIO_LEVEL': { + return { ...state, audioLevel: action.level }; + } + + // ─── Set Error ─── + case 'SET_ERROR': { + console.log('[sessionReducer] ❌ SET_ERROR:', action.message); + return { ...state, error: action.message }; + } + + // ─── Reset ─── + case 'RESET': { + console.log('[sessionReducer] 🔄 RESET'); + return createInitialState(state.mode); + } + + default: + return state; + } +}; diff --git a/src/theme/colors.ts b/src/theme/colors.ts index 39be4bc..ef39f29 100644 --- a/src/theme/colors.ts +++ b/src/theme/colors.ts @@ -1,31 +1,66 @@ /** - * App color palette - Inspired by modern AI/tech aesthetics - * Matching the Flutter app's beautiful theme + * App color palette - Premium Light Purple Fintech Theme + * Inspired by modern banking/fintech app design + * Soft lavender backgrounds with purple accent cards */ export const AppColors = { - // Primary gradient colors - Deep space with electric accents - primaryDark: '#0A0E1A', - primaryMid: '#141B2D', - surfaceCard: '#1C2438', - surfaceElevated: '#242F4A', - - // Accent colors - Electric cyan, violet, and more - accentCyan: '#00D9FF', - accentViolet: '#8B5CF6', - accentPink: '#EC4899', - accentGreen: '#10B981', - accentOrange: '#F59E0B', + // Backgrounds - Light lavender/purple + primaryLight: '#F5F0FF', + primaryBg: '#EDE5FF', + primaryMid: '#E8DFFF', + surfaceCard: '#FFFFFF', + surfaceElevated: '#F8F5FF', + + // NOT USED for backward compat — mapped to new palette + primaryDark: '#F5F0FF', + + // Purple gradient accent + purpleGradientStart: '#7B61FF', + purpleGradientEnd: '#9B82FF', + purpleLight: '#C4B5FD', + purpleMuted: '#DDD6FE', + purpleSoft: '#EDE9FE', + + // Primary accent - Purple + accentPrimary: '#7B61FF', + accentSecondary: '#9B82FF', + accentTeal: '#7B61FF', // backward compat alias + accentTealLight: '#9B82FF', + accentTealDark: '#6C4DE6', + accentCyan: '#7B61FF', // backward compat + accentViolet: '#6C4DE6', + accentPink: '#F472B6', + accentGreen: '#34C759', + accentOrange: '#FF9F0A', // Text colors - textPrimary: '#F1F5F9', - textSecondary: '#94A3B8', - textMuted: '#64748B', + textPrimary: '#1A1A2E', + textSecondary: '#6B7280', + textMuted: '#9CA3AF', // Status colors - success: '#22C55E', - warning: '#F59E0B', - error: '#EF4444', - info: '#3B82F6', + success: '#34C759', + warning: '#FF9F0A', + error: '#FF3B30', + info: '#007AFF', + + // Transaction icon colors + iconOrange: '#FF9F43', + iconBlue: '#54A0FF', + iconGreen: '#00D2D3', + iconPurple: '#A29BFE', + iconPink: '#FF6B6B', + iconTeal: '#1DD1A1', + + // Glassmorphism / card styling + glassBg: 'rgba(255, 255, 255, 0.7)', + glassBgLight: 'rgba(255, 255, 255, 0.85)', + glassBorder: 'rgba(123, 97, 255, 0.12)', + glassBorderLight: 'rgba(123, 97, 255, 0.08)', + + // Shadow + shadowPurple: '#7B61FF', + shadowDark: 'rgba(0, 0, 0, 0.08)', } as const; export type AppColorsType = typeof AppColors; diff --git a/src/types/session.ts b/src/types/session.ts new file mode 100644 index 0000000..3db6c67 --- /dev/null +++ b/src/types/session.ts @@ -0,0 +1,211 @@ +/** + * 🔒 PRIVACY NOTICE + * All data structures defined here are stored locally on device. + * No data leaves this device. + */ + +/** + * Negotiation modes with different pattern detection weights + */ +export enum NegotiationMode { + JOB_INTERVIEW = 'job_interview', + SALES = 'sales', + STARTUP_PITCH = 'startup_pitch', + SALARY_RAISE = 'salary_raise', + INVESTOR_MEETING = 'investor_meeting', + CLIENT_NEGOTIATION = 'client_negotiation', + CUSTOM_SCENARIO = 'custom_scenario', +} + +/** + * Negotiation patterns that can be detected + */ +export enum NegotiationPattern { + ANCHORING = 'anchoring', + BUDGET_OBJECTION = 'budget_objection', + AUTHORITY_PRESSURE = 'authority_pressure', + TIME_PRESSURE = 'time_pressure', + DEFLECTION = 'deflection', + POSITIVE_SIGNAL = 'positive_signal', + NEGATIVE_SIGNAL = 'negative_signal', + COMMITMENT_LANGUAGE = 'commitment_language', + STRENGTH_SIGNAL = 'strength_signal', +} + +/** + * Severity levels for detected patterns + */ +export type PatternSeverity = 'low' | 'medium' | 'high'; + +/** + * Detected pattern with metadata + */ +export interface DetectedPattern { + id: string; + pattern: NegotiationPattern; + confidenceScore: number; // 0-100 + suggestion: string; + severity: PatternSeverity; + timestamp: number; + transcript: string; + context?: string; +} + +/** + * Cognitive metrics tracked during session + */ +export interface CognitiveMetrics { + focusScore: number; // 0-100 + speechGaps: number; // Count of pauses > 2 seconds + fillerWords: number; // Count of um, uh, like, etc. + avgSpeechRate: number; // Words per minute + totalWords: number; + speechDuration: number; // Milliseconds of actual speech (excluding gaps) +} + +/** + * Session summary generated after session ends + */ +export interface SessionSummary { + leverageMoments: string[]; // Key moments where user had advantage + missedOpportunities: string[]; // Moments where better response was possible + objectionCount: number; + positiveSignalCount: number; + tacticalSuggestions: string[]; // Suggestions for next meeting + keyInsights: string[]; +} + +/** + * Transcript chunk with timestamp + */ +export interface TranscriptChunk { + id: string; + text: string; + timestamp: number; + speaker?: 'user' | 'other'; // Future enhancement for multi-speaker + hasPattern?: boolean; // True if this chunk has detected patterns +} + +/** + * Live session state (in-memory during recording) + */ +export interface LiveSessionState { + isRecording: boolean; + startTime: number; + duration: number; + transcript: TranscriptChunk[]; + detectedPatterns: DetectedPattern[]; + currentFocusScore: number; + audioLevel: number; + lastAutoSave: number; +} + +/** + * Adaptive Inputs collected before session start + */ +export interface PreSessionInputs { + [key: string]: string; // Key-value maps for dynamic form questions +} + +/** + * 10-Point Strategic Analysis Plan generated offline + */ +export interface StrategicAnalysis { + powerPositioning: string; + likelyObjections: string[]; + psychologicalTactics: string[]; + recommendedResponses: string[]; + highImpactPhrases: string[]; + phrasesToAvoid: string[]; + confidenceTriggers: string[]; + openingScript: string; + closingScript: string; + mistakesToAvoid: string[]; +} + +/** + * Persisted session data (saved to AsyncStorage) + */ +export interface Session { + id: string; + timestamp: number; // Session start time + duration: number; // Total duration in milliseconds + mode: NegotiationMode; + preSessionInputs?: PreSessionInputs; + strategicAnalysis?: StrategicAnalysis; + transcript: TranscriptChunk[]; + detectedPatterns: DetectedPattern[]; + cognitiveMetrics: CognitiveMetrics; + summary: SessionSummary; + title?: string; // Optional user-provided title + notes?: string; // Optional user notes +} + +/** + * Quick stats for home screen dashboard + */ +export interface SessionStats { + totalSessions: number; + avgFocusScore: number; + avgDuration: number; + mostCommonPattern: NegotiationPattern | null; + totalPatterns: number; + lastSessionDate: number | null; +} + +/** + * App settings stored in AsyncStorage + */ +export interface AppSettings { + defaultMode: NegotiationMode; + patternSensitivity: number; // 0.5 - 1.5 multiplier for confidence thresholds + enableAutoSave: boolean; + autoSaveInterval: number; // Milliseconds (default 45000) + enableHapticFeedback: boolean; + enableSuggestionNotifications: boolean; + debugMode?: boolean; // Enable debug mode with hardcoded transcripts +} + +/** + * Pattern detection configuration per mode + */ +export interface ModeConfig { + mode: NegotiationMode; + displayName: string; + description: string; + icon: string; + patternWeights: { + [key in NegotiationPattern]: number; // Multiplier for confidence score + }; +} + +/** + * Audio buffer chunk for processing + */ +export interface AudioChunk { + data: number[]; + timestamp: number; + duration: number; +} + +/** + * Analysis result from NegotiationAnalyzer + */ +export interface AnalysisResult { + detectedPatterns: DetectedPattern[]; + cognitiveMetrics: Partial; + suggestions: string[]; + focusScore: number; +} + +/** + * Counter strategy result from the CounterStrategyEngine + */ +export interface CounterStrategy { + tactic: NegotiationPattern; + tacticDisplayName: string; + confidence: number; // 0-100 + suggestions: string[]; + explanation: string; + timestamp: number; +} diff --git a/tech_stack_info.md b/tech_stack_info.md new file mode 100644 index 0000000..db5a0e6 --- /dev/null +++ b/tech_stack_info.md @@ -0,0 +1,65 @@ +# Latent: Technical Architecture & Stack Overview + +**Latent** is a privacy-first, on-device Conversation Intelligence Engine built primarily to analyze live negotiations, interviews, and pitches in real-time without ever sending audio or transcriptions to the cloud. + +--- + +## Core Technology Stack + +- **Application Framework:** React Native (TypeScript) +- **Target Platforms:** Android & iOS +- **State Management & UI:** React Hooks, Functional Components, Custom CSS-in-JS (React Native StyleSheets). +- **Routing:** React Navigation (Stack Navigator) +- **Local Storage:** AsyncStorage (for saving post-session analysis & settings offline) + +--- + +## On-Device AI: The RunAnywhere SDKs + +The entire artificial intelligence pipeline is powered exclusively by the **RunAnywhere SDK**. This allows massive neural network models (LLMs, STT, TTS) to run locally on the mobile phone's CPU/GPU. + +The following SDK modules are integrated into Latent: + +1. **`@runanywhere/core`** + - The foundational bridging layer that interfaces between React Native JavaScript and the underlying C++ native environment. +2. **`@runanywhere/onnx`** + - The ONNX Runtime backend. This powers the **Speech-to-Text (STT)** engine using the _Sherpa Whisper Tiny_ model. + - Also acts as the execution engine for the **Text-to-Speech (TTS)** pipeline using the _Piper TTS_ model. +3. **`@runanywhere/llamacpp`** + - The Llama.cpp backend used for running full **Large Language Models (LLMs)** on-device, such as _Liquid LFM2 (350M)_ or _SmolLM2 (360M)_, without cloud latency. + +--- + +## Data Transmission & Input Flow + +The most critical architectural pillar of Latent is the **Zero-Cloud Data Policy**. Here is how data is inputted, transmitted, and processed entirely within the phone's memory. + +### 1. Audio Acquisition (Input) + +- The application requests local microphone permissions (`android.permission.RECORD_AUDIO`). +- Audio is captured via native audio modules (`react-native-live-audio-stream` / underlying C++ audio capture buffers). +- **Transmission:** Audio data is passed directly from the microphone hardware into local RAM (Float32 buffers or temporary WAV snapshots). **It never leaves the device.** + +### 2. Live Transcription Pipeline (STT) + +- The `SpeechService.ts` module routinely chunks the incoming audio (e.g., every 1.5 to 5 seconds). +- These localized audio snapshots are handed to the `RunAnywhere.transcribeFile()` C++ bridge. +- The **Sherpa Whisper Tiny ONNX** model analyzes the audio purely on local hardware. We inject a _Domain Bias Prompt_ natively to optimize the STT context for words involving finance, startup, and negotiation (e.g., prioritizing "salary" over "celery"). +- The C++ bridge returns the recognized text string back up to the JavaScript layer. + +### 3. Contextual Auto-Correction & Parsing + +- Upon receiving the raw text string, Latent passes it through a zero-latency, synchronous TypeScript `WhisperAutoCorrector`. +- This uses **Levenshtein Distance** algorithms to fuzzy-match phonetic hallucinations against the active _Negotiation Mode_ (e.g., Job Interview vs. Investor Meeting) vocabulary. + +### 4. Intent & Counter-Strategy Generation + +- The sanitized transcript string is injected into the `SessionEngine`, which feeds it to the `IntentClassifier`. +- The engine evaluates the sentence against an offline `Universal Scoring Engine` (powered by structural pattern matching, negation rules, and numeric markers). +- If a specific tactic is detected (e.g., _Authority Pressure_ or _Budget Objection_), the `CounterStrategyEngine` instantly surfaces actionable suggestions on the React Native UI. + +### 5. Final Output & Offline Storage + +- When a session ends, the full dialog map and cognitive metrics are compiled. +- Data is saved directly to local flash storage via `AsyncStorage`. +- The data is then displayed on the _Outcome Replay Screen_ for post-meeting analysis, remaining 100% private to the physical device owner. diff --git a/tech_stack_info.pdf b/tech_stack_info.pdf new file mode 100644 index 0000000..b02243b Binary files /dev/null and b/tech_stack_info.pdf differ