The vitrivr frontend, vitrivr-web, was developed as a React application, as this framework enables straightforward modularization of components and facilitates deployment.
The frontend architecture follows a modular, component-based approach, making it easily adaptable and extendable to accommodate potential changes in the backend. Queries are formulated through different components that act as building blocks. These components can either be used individually or combined to form temporal queries.
Each block provides a selection for the modality type (CLIP, emotions, OCR, ASR).
- CLIP enables users to submit either textual queries or perform image-to-image searches to retrieve the k nearest neighbors.
- In the context of emotions, a textual query (CLIP) can be integrated with a filter to restrict results to a particular emotion.
- Emotion detection is a multimodal process that integrates facial expression analysis, automatic speech emotion recognition, and OCR-based text emotion analysis into a unified score.
- Furthermore, the system is equipped with the capability to process both OCR and ASR through textual queries.
The frontend further provides functionality to display and filter search results for faster access. The available filters encompass options such as media type, including videos, images, and 3D models.
The ensuing results are presented in descending order of relevance. With regard to the video results, playback is initiated at the frame that most closely corresponds to the query.
The filtering functionality is of particular importance in the context of video retrieval competitions, as it enables the refinement of the result set.
Moreover, the frontend incorporates the DRES framework, which was developed to assess the performance of multimedia retrieval systems.
frontend-vitrivr/
├── vitrivr-web/
│ ├── public/
│ ├── src/
│ │ ├── assets/
│ │ ├── components/
│ │ ├── dres/
│ │ ├── lib/
│ │ ├── state/
│ │ ├── styles/
│ │ ├── utils/
│ │ ├── vitrivr/
│ │ ├── App.tsx
│ │ └── main.tsx
- Navigate to the frontend directory
cd frontend-vitrivr- Create the .env file In the .env file the address of the vitrivr-engine, DRES-server, etc. are saved.
VITE_VITRIVR_BASE_URL=http://<ip>:<port>
VITE_MEDIA_ORIGIN=http://<ip>:<port>
VITE_THUMBNAIL_ORIGIN=http://<ip>:<port>
VITE_DRES_BASE_URL=https://vbs.videobrowsing.org/
VITE_DEBUG=0- Install all dependencies
npm install- Build the application
npm run build- Run the application in development mode
npm run devThis frontend was specifically developed for the VBS Prague 2026. This affects some parts of the code, mainly:
- The image-to-image search, as this search operation is not permitted at the VBS, and is therefore not fully implemented. The components exist, but they need to be slightly adapted.
- DRES is included with a login on the upper left. The user is not forced to log in, but it is highly recommended that if you want to run the frontend without DRES that you remove it entirely.