Xinhuo_competition

This repo generates multi-language caption for images. We provide training code and 2 versions of demo:

• DTU version demo: Demo in this version run on DTU.
• GUI version demo: Demo in this version has GUI interaction interface. You can run it on GPU or CPU with pt or onnx model. Note that, it can't run on DTU, because existing DTU environment doesn't support GUI.
• Training codes: Codes and dataset for training image caption task.

DTU version

Environment

Please make sure that you have install following package in your environment:

pip install --upgrade pytorch torchvision
pip install onnxruntime

pip install transformers
pip install datasets
pip install sacrebleu
pip install sentencepiece

pip install scikit-image

Inference

You can run this demo with the following script, and get the result in the last row of log:

python -m inference_DTU_total --language zh --type DTU

Note, you can change parameters as followings:

• language: We support languages in [en, zh, de, fr, ro].
• type: You can change type to torch or onnx, which infer the type of model without DTU.

GUI version

You can run this demo with the following script, and interact with GUI interface:

python gui_demo.py

Training codes

How to download dataset and train model are described in PyTorch-Tutorial-to-Image-Captioning/README.md.