Subtitle Generator

A website for AI-based transcript and subtitle generation including a small user editor using

• Stable Whisper (which is a slight modification of OpenAI Whisper) for audio transcription,
• NVIDIA NeMo for speaker diarization,
• PANNs inferece for sound event detection,
• DeepL Translator for translation,
• FFmpeg for video/audio editing,
• Flask for webframework
• and Bootstrap for website CSS.

This application was developed as part of a bachelor thesis.

Example

example_3.mp4

Video-URL: https://www.youtube.com/watch?v=L6yE7fUE220

More examples can be found here.

Requirements

• Python 3.10.11 (Download here)
• Microsoft Visual C++ 14.0 or greater (Download here)
• FFmpeg 6.0 or greater (Download here)
• DeepL API Key (Create a free account here)

Setup

1. Clone repository:

git clone https://github.com/philipp821/subtitle-generator.git

2. Move into repository directory and create virtual environment:

python -m venv venv

3. Activate virtual environment:

venv\Scripts\activate

4. Install packages:

pip install Cython
pip install -r requirements.txt
python -m textblob.download_corpora lite

5. Download pretrained model for sound event detection and store it in:

\data\configs\panns_inference\Cnn14_DecisionLevelMax_mAP=0.385.pth

6. Put your DeepL API Key in a file named deepl.key and store it in root directory:

\deepl.key

Usage

• Activate virtual environment if not already done:

venv\Scripts\activate

• Run the webserver.py file:

python src\webserver.py

• Enter http://localhost:5000/ in your browser if it does not open by itself.