🧠 STL10 Self-Supervised Pretraining & Intel Image Transfer Learning

This project implements a two-stage deep learning workflow:

1. Self-Supervised Learning (SSL) using a rotation-prediction task on the STL10 dataset.
2. Transfer Learning using the Intel Image Classification dataset to compare performance across multiple pretrained and scratch models.

---

🏃‍♂️ Running Procedure

1. Extract the dataset

   • Extract the Intel Image Classification dataset into the data folder.

2. Run SSL pretraining

   • Open and run the ssl_pretrain.ipynb notebook located in the notebooks folder.

3. Run Intel transfer

   • After SSL pretraining completes, open and run the intel_transfer.ipynb notebook in the same folder.

4. Dataset location

   • Ensure the Intel Image Classification dataset is located at:

     data/Intel Image Classification/
     

   • (I simply extracted the dataset folder in place.)
   • https://www.kaggle.com/datasets/puneet6060/intel-image-classification

---

💾 Pretrained Backbone

A pretrained backbone from the SSL portion is included, so you do not need to rerun the pretraining notebook (only run it once).

• File: stl10_backbone_pretrained.pth

---

⚙️ Environment Setup

• Device: GPU (CPU fallback if unavailable)
• Random Seed: 42 (applied to both NumPy and PyTorch)
• Normalization: All images are resized to 224 × 224 to match the ImageNet format for consistency.

---

🧠 Training Procedure

STL10 SSL Model

• Trained until no improvement over 3 consecutive epochs or until achieving 99% training accuracy to prevent overfitting.
• Batch size: 128 (you may need to reduce this if GPU memory is limited).

Intel Image Classification

• Batch size: 64 (may be reduced if needed).

---

🔧 Learning Rates

Model Name	Description	Learning Rate
s0_model	Random initialization (trained from scratch)	0.001
i_frozen	ImageNet pretrained — frozen backbone	0.001
i_ft	ImageNet pretrained — fine-tuned backbone	0.00005
ssl_frozen	SSL pretrained — frozen backbone	0.001
ssl_ft	SSL pretrained — fine-tuned backbone	0.00005

---

🧩 Linear Head Architecture

All models use the same MLPHead class (defined in models/heads.py):

• Two hidden layers → output layer
• Batch Normalization
• ReLU activations
• Dropout layers

---

📊 Output

Running intel_transfer.ipynb generates and optionally saves the confusion matrices The plots comparing the different models is in the intel_transfer notebook with extended discussion in the "experiment.pdf" file that contains my final report.