Emotion Prediction Finetuning

This repo prepares emotion-regression datasets and trains two baselines:

• ResNet18 regressor (PyTorch) — numeric regression baseline.
• Qwen2.5-VL-7B-Instruct + LoRA (SFT) — vision-language model fine-tuned to output a single numeric rating.

---

Setup

Option A: venv

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Option B: conda (if available)

conda create -n emo-qwen python=3.10 -y
conda activate emo-qwen
pip install -r requirements.txt

Hugging Face login (required for gated datasets)

Some datasets/models may require accepting terms or having access on Hugging Face.

huggingface-cli login

---

Data access (important)

Parquet files and images are NOT stored in this Git repository.
They are downloaded from Hugging Face and may be gated (requires login + access approval).

If you already have the Parquet files locally (e.g. from a previous run), you can skip the download step and go straight to processing.

---

Option 1: If you already have Parquet files locally

Make sure these files exist (or update PARQUET_PATHS in scripts/make_processed.py):

• data/IAPS/train.parquet
• data/LAIGAI/train.parquet
• data/NAPS/train.parquet

Then build processed splits + extracted images:

python scripts/make_processed.py

---

Option 2: Full pipeline from scratch

1) Download datasets (HF → Parquet)

python download_data/download_all.py

2) Build processed splits + extracted images

Creates:

• data/processed/train.jsonl
• data/processed/val.jsonl
• data/processed/test.jsonl
• data/processed/images/

python scripts/make_processed.py

3) Build Qwen SFT dataset

Creates:

• data/qwen_sft/train.jsonl
• data/qwen_sft/val.jsonl
• data/qwen_sft/test.jsonl
• data/qwen_sft/stats.json

python scripts/make_qwen_sft_dataset.py

---

Baseline (ResNet18)

Train:

python scripts/baseline/train.py

Evaluate:

python scripts/baseline/evaluate.py

Infer (single image):

python scripts/baseline/infer.py

Outputs:

• runs/baseline_resnet18/

---

Qwen LoRA

Note: Qwen LoRA training/inference expects a CUDA GPU by default (DEVICE="cuda" in scripts).

0) (Optional) Pre-download base Qwen model to cache

python scripts/download_qwen.py

1) Train LoRA adapter

python scripts/train_qwen.py

2) Evaluate on test split

Creates:

• runs/qwen_lora/test_metrics.csv
• runs/qwen_lora/test_metrics.json
• runs/qwen_lora/test_predictions.csv (per-record: filename/source/emotion/true/pred)

python scripts/evaluate_qwen.py

3) (Optional) Rate images from data/processed/test.jsonl

Creates:

• runs/qwen_ratings.csv (one row per image: filename + 5 emotion columns)

python scripts/rate_images.py

4) Infer (single image, 5 emotions)

Creates:

• runs/qwen_lora/prediction.json

python scripts/infer_qwen.py

Outputs:

• runs/qwen_lora/adapter/ (LoRA adapter)
• runs/qwen_lora/train_log.json
• runs/qwen_lora/val_metrics.json Trained LoRA adapter is included in runs/qwen_lora/adapter.

---

Sample input/output (for reproducibility/demo)

This repo includes a small sample (one image + expected format) to demonstrate the I/O format without shipping the full dataset:

• data/sample_input/
• data/sample_output/

Use scripts/infer_qwen.py or scripts/baseline/infer.py to run a single-image demo.