AI-Powered Voice Translation Application with Google Gemini
A modern, cost-effective translation application featuring voice recognition, text-to-speech, real-time streaming responses, and comprehensive Bible translation assistant tools.
- AI Translation - Context-aware translations using Google Gemini 2.0 Flash (~$0.13/M tokens)
- Speech-to-Text - Gemini Audio integration for voice transcription
- Text-to-Speech - Google Cloud TTS with Neural2 voices
- Real-time Streaming - Server-Sent Events for live AI responses
- Chat Management - Persistent conversation history
- Storyteller - Creates engaging, spoken-language stories about biblical themes
- Oral Performer - Transforms biblical passages into oral versions for live audiences
- Conversation Partner - General translation assistance
- OBT Health Assessor - Evaluates Oral Bible Translation projects
- Back Translation Checker - Verifies translation accuracy against original texts
- Multiple TTS voices (Alloy, Echo, Fable, Onyx, Nova, Shimmer)
- Voice-specific caching for instant replay
- Transcription editing before sending
| Layer | Technologies |
|---|---|
| Frontend | React 18, TypeScript, Vite, Tailwind CSS, shadcn/ui |
| Backend | Node.js, Express.js, TypeScript, Drizzle ORM |
| Database | PostgreSQL (Neon serverless) |
| AI | Google Gemini 2.0 Flash, Google Cloud TTS |
| Infrastructure | Docker, Google Cloud Run, Terraform, GitHub Actions |
- Node.js 18+
- PostgreSQL database (Neon recommended)
- Google Gemini API key (Get one here)
- Google Cloud account (for TTS)
# Clone repository
git clone https://github.com/shemaobt/translation-helper.git
cd translation-helper
# Install dependencies
npm install
# Configure environment
cp .env.example .env
# Edit .env with your credentials
# Set up database
npm run db:push
# Start development server
npm run devDATABASE_URL=postgresql://user:pass@host/db
GOOGLE_API_KEY=your_gemini_api_key
SESSION_SECRET=your_session_secretTranslation-Helper-WebApp/
├── client/ # React frontend
│ ├── src/
│ │ ├── components/ # UI components
│ │ ├── pages/ # Page components
│ │ ├── hooks/ # Custom React hooks
│ │ └── lib/ # Utilities
│ └── nginx.conf # Production web server config
├── server/ # Node.js backend
│ ├── gemini.ts # AI integration
│ ├── prompts.ts # Agent prompt definitions
│ ├── routes.ts # API endpoints
│ ├── storage.ts # Database layer
│ └── db.ts # Database connection
├── shared/
│ └── schema.ts # Shared types & DB schema
├── tests/ # Integration tests
├── Dockerfile.backend # Backend container
├── Dockerfile.frontend # Frontend container
└── docker-compose.yml # Local Docker setup
| Endpoint | Method | Description |
|---|---|---|
/api/public/info |
GET | Available endpoints and rate limits |
/api/public/translate |
POST | Translate text between languages |
/api/public/transcribe |
POST | Convert speech to text |
/api/public/speak |
POST | Convert text to speech |
Rate Limit: 50 requests per 15 minutes per IP
curl -X POST https://your-app.com/api/public/translate \
-H "Content-Type: application/json" \
-d '{
"text": "Hello world",
"fromLanguage": "en-US",
"toLanguage": "es-ES"
}'| Endpoint | Method | Description |
|---|---|---|
/api/chats |
GET/POST | List or create chats |
/api/chats/:id/messages |
GET/POST | Get or send messages |
/api/chats/:id/stream |
GET | Stream AI responses (SSE) |
/api/auth/user |
GET | Current user info |
/api/user/profile-image |
POST | Upload profile picture |
/api/user/change-password |
POST | Update password |
| Endpoint | Method | Description |
|---|---|---|
/api/admin/prompts |
GET | List all agent prompts |
/api/admin/prompts/:id |
GET/PUT | Get or update a prompt |
/api/admin/prompts/:id/reset |
POST | Reset prompt to default |
/api/admin/users |
GET | List all users |
/api/admin/feedback |
GET | View user feedback |
Agent prompts are stored in the database and can be managed via the Admin UI at /admin/prompts.
Edit default prompts in server/prompts.ts:
export const AGENT_PROMPTS = {
storyteller: `You are the Storyteller Assistant...`,
conversation: `You are the Conversation Partner...`,
performer: `You are the Oral Performer...`,
// ...
};- Add to schema in
shared/schema.ts:
export const ASSISTANTS = {
// ...existing agents...
newAgent: {
id: 'newAgent',
name: 'New Agent Name',
description: 'Agent description',
model: 'gemini-2.0-flash-exp',
}
};- Add prompt in
server/prompts.ts:
export const AGENT_PROMPTS = {
// ...existing prompts...
newAgent: `Your prompt instructions here...`,
};- Run
npm run db:pushto seed the prompt to the database.
docker compose up -d
docker compose logs -f- Set up infrastructure with Terraform:
cd ../tf/environments/translation-prod
terraform init
terraform apply -var-file="translation-prod.tfvars"-
Configure GitHub Secrets:
GCP_PROJECT_IDGCP_WORKLOAD_IDENTITY_PROVIDERGCP_WORKLOAD_IDENTITY_SERVICE_ACCOUNTNEON_DATABASE_URLGOOGLE_API_KEYSESSION_SECRET
-
Deploy via push to main:
git push origin mainGitHub Actions will automatically build, push, and deploy to Cloud Run.
# Install test dependencies
pip install -r requirements-test.txt
# Run all tests
./run_tests.sh all
# Quick tests only
./run_tests.sh quick
# With coverage
./run_tests.sh coverage| Service | Cost |
|---|---|
| Cloud Run (100k requests/month) | ~$5/month |
| Gemini 2.0 Flash (1M tokens) | ~$0.13 |
| Google Cloud TTS (1M chars) | ~$16 |
| Total typical usage | ~$15-20/month |
| Command | Description |
|---|---|
npm run dev |
Start development server |
npm run build |
Build for production |
npm start |
Start production server |
npm run db:push |
Push schema changes to database |
npm run check |
TypeScript type checking |
# Verify DATABASE_URL format
postgresql://user:password@host:port/database
# Test connection
psql "$DATABASE_URL"# Verify API key
curl -H "Content-Type: application/json" \
-d '{"contents":[{"parts":[{"text":"test"}]}]}' \
"https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash-exp:generateContent?key=YOUR_KEY"# Check logs
gcloud run services logs read translation-helper-backend \
--region=us-central1 --limit=50
# Check service status
gcloud run services describe translation-helper-backend \
--region=us-central1- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
MIT License - see LICENSE for details.