Tactile Material Classifier

What This Repo Does

Classifies tactile images (from YCB-Sight) into 4 material categories using UniTouch encoder:

• plastic
• paper-box
• wood
• metal

Quick Start with Docker

Pull the Docker Image

docker pull timttu/unitouch:v3

The checkpoint is already wrapped in the workspace.

Explore the Container (Interactive Mode)

To get into the container and explore files:

docker run --gpus all -it \
  -v <your-path-where-repo-is>:/pers_vol/YCB-material \
  timttu/unitouch:v3 /bin/bash

Once inside, you can check files, verify paths, or run commands manually.

Download and Prepare Data

First, navigate to the repository directory and run the preprocessing script:

cd /pers_vol/YCB-material/unitouch_ycbsight
bash preprocessing.sh

This will download and extract the YCB-Sight datasets into the data/ folder.

Run Inference

python3 inference_classifier.py \
    --config configs/inference.yaml \
    --checkpoint workspace/best_model.pth \
    --input /path/to/your/tactile/images/ \
    --output results.json \
    --batch-size 64

Installation (Without Docker)

pip install torch torchvision tqdm pyyaml tensorboard pillow easydict

Data Structure

Your tactile images should be organized like this:

data/
├── plastic/
│   ├── image_001.jpg
│   └── ...
├── paper-box/
├── wood/
└── metal/

Training (Optional)

python train_classifier.py --config configs/simple_classifier.yaml --gpu 0

Citation

@inproceedings{yang2024binding,
  title={Binding touch to everything: Learning unified multimodal tactile representations},
  author={Yang, Fengyu and Feng, Chao and Chen, Ziyang and others},
  booktitle={CVPR},
  year={2024}
}