Video Similarity Comparison Tool (ViT-model)

This project uses Vision Transformer (ViT) models to compare videos and find similarities between them. It extracts embeddings from video frames and computes cosine similarity to determine how similar two videos are.

Features

• Extract frame embeddings from videos using Vision Transformer models
• Compare videos based on visual similarity
• Process multiple videos in batch
• Generate similarity reports in JSON format

Requirements

• Python 3.6+
• PyTorch
• OpenCV
• NumPy
• timm (PyTorch Image Models)

Installation

1. Clone this repository:

git clone https://github.com/yourusername/ViT-model.git
cd ViT-model

2. Install the required dependencies:

pip install torch torchvision opencv-python numpy timm

Usage

1. Place videos to be monitored in the monitored folder
2. Place videos to be compared against in the watched folder
3. Run the main script:

python test/main.py

4. Results will be saved to similarity_results.json

How It Works

1. The tool samples frames from each video at a specified rate (default: 1 frame per second)
2. Each frame is preprocessed and normalized
3. A Vision Transformer model extracts embeddings from the frames
4. Frame embeddings are aggregated to create a video-level embedding
5. Cosine similarity is computed between video embeddings
6. Results are sorted by similarity and saved to a JSON file

Project Structure

• test/main.py: Main script for video comparison
• monitored/: Directory for videos to be monitored
• watched/: Directory for videos to be compared against
• similarity_results.json: Output file with similarity results

License

MIT License

Acknowledgements

This project uses the timm library for Vision Transformer models.