16-825 Assignment 2: Single View to 3D

This project explores 3D reconstruction in voxel, point cloud, and meshe respresentations from single-view images using a simple MLP decoder model.

Table of Contents

1. Exploring Loss Functions
2. Single View 3D Reconstruction
3. Training on Extended Dataset

---

1. Exploring Loss Functions

To run the first question:

python fit_data.py --type vox|point|mesh

1.1 Fitting a Voxel Grid

Ground Truth Reconstruction	Fitted Voxels

1.2 Fitting a Point Cloud

Ground Truth Reconstruction	Fitted Point Clouds

1.3 Fitting a Mesh

Ground Truth Reconstruction	Fitted Mesh

---

2. Single View 3D Reconstruction

To run the second question:

python train_model.py --type vox|point|mesh

To evaluate it:

python eval_model.py --type vox|point|mesh

2.1 Image to Voxel Grid

Single View RGB Image	Ground Truth Reconstruction	Fitted Voxels

2.2 Image to Point Cloud

Single View RGB Image	Ground Truth Reconstruction	Fitted Point Cloud

2.3 Image to Mesh

Single View RGB Image	Ground Truth Reconstruction	Fitted Mesh

2.4 Quantitative Comparisons

Voxel F1-score	Point Cloud F1-score	Mesh F1-score

2.5 Effects of Hyperparameter Variations

Single View RGB Image	Ground Truth Reconstruction	n_points = 1000

n_points = 700	n_points = 1500

2.6 Model Interpretation

Input RGB Image	Ground Truth Reconstruction	Final Fitted Points

At 500 Iterations	At 1000 Iterations	At 3000 Iterations

---

3. Training on Extended Dataset

To run the third question:

1. Change use_full_dataset = True in dataset_location.py.
2. Run the following commands:

python train_model.py --type vox|point|mesh
python eval_model.py --type vox|point|mesh

3.1 Extended Dataset Results

Input RGB Image	Ground Truth Reconstruction	Fitted Points

Single Class F1-Score	Multi-Class F1-Score