This is an official PyTorch implementation of the EBAD-NeRF. Click here to see the video and supplementary materials in our project website.
The code is based on the offical Pytorch implementation of the BAD-NeRF and use the same environment. Please refer to its github website for the environment installation.
The configs of the synthetic data are in the config_blender.txt file. Please download the synthetic data below and put it into the corresponding file (./data/blender_llff/). Then you can use the command below to train the model.
python train_blender.py --config config_blender.txt
The configs of the real-world data are in the config_davis.txt file. Please download the real-world data below and put it into the corresponding file (./data/davis_llff/). Then you can use the command below to train the model.
python train_davis.py --config config_davis.txt
The synthetic data can be downloaded at here. We use five Blender scenes from BAD-NeRF to construct this dataset. To increase the difficulty of the data, we add non-uniform camera shake. As shown in the folder, each scene folder contains five parts:
"images": images for training.
"images_gt_blur": ground truth images of blur view for testing.
"images_gt_novel": ground truth images of novel view for testing.
"events.pt": the event data for training.
"pose_bounds.npy": the initial poses for training.
The real-world data can be downloaded at here. We use Davis 346 event camera to capture the real-world data. The data consist of two scenes as shown in the folder. Each folder contains four parts:
"images": images for training.
"images_gt_novel": ground truth images of novel view for testing.
"events.pt": the event data for training.
"pose_bounds.npy": the initial poses for training.
Notice that we set b=6 for real-world data because the blur degree is larger than the synthetic data. Additionally, for real-world data experiement in the paper, we select four novel view images (No.[3, 10, 17, 24]) in the "images_gt_novel" folder for testing.
If you find this useful, please consider citing our paper:
@inproceedings{qi2024deblurring,
title={Deblurring neural radiance fields with event-driven bundle adjustment},
author={Qi, Yunshan and Zhu, Lin and Zhao, Yifan and Bao, Nan and Li, Jia},
booktitle={Proceedings of the 32nd ACM International Conference on Multimedia},
pages={9262--9270},
year={2024}
}The overall framework and camera trajectory metrics computing are derived from BAD-NeRF. We appreciate the effort of the contributors to these repositories. Additionally the event loss is derived from our previous work E2NeRF.