SelfCriticalSequenceTrainingforImageCaptioning

TensorFlow implementation of Self-critical Sequence Training for Image Captioning.
General framework for reinforcement learning image caption tasks.

References

This work is highly based on @yunjey 's implementation of show-attend-and-tell

Pytorch implementation self-critical.pytorch

Getting Started

Prerequisites

• python2.7
• tensorflow 0.12
• numpy
• matplotlib
• scipy
• scikit-image
• hickle
• Pillow
• pycocoevalcap

data

• In this work, we used MSCOCO data set
• Data download & process is the same as @yunjey 's work. Please check show-attend-and-tell "Getting Started" secton for details.

Train the model

To train the image captioning model, run command below.

$ python train.py

For reference, the RL model is in ./core/model.py

Results

After 5 epochs, our result has 5% improvement. Note that we are using different data set to test and we used different feature net.