I hope the author will update the model with more, thank you!

[yahiko-l]

[vvvm23]

Greetings,

I'm currently busy with final year exams and coursework. Also, parts of the SUNDAE model I am using in my dissertation, so I won't update further until after my deadline to avoid any issues with plagiarism.

Once I graduate, I plan on updating with text8 checkpoints, EN-DE and EN-ZH translation, support for inpainting, and argmax unrolled sampling.

Please be patient in the meantime :) I currently don't have time to update much. Thank you (´ω｀)

[yahiko-l]

Hi,
I am a first year PhD student in China. Thanks to the author for replying to my message. But now, I'm curious to understand if the translation model (EN-ZH) based on DENOISING AUTOENCODERS is really useful, and the related implementation with pytorch.
I will continue to follow the progress of your project, thank you(^_^).

[vvvm23]

I think SUNDAE has many properties that make it a good choice for translation:

• fast to sample from
• can use a bidirectional context
• can correct errors during sampling

Why EN-ZH specifically? Just think Chinese language is cool so want to try applying it there. I don't think it would beat state-of-the-art, but fun to try.

Thanks~

[yahiko-l]

Because Chinese language is different from English language. English consists of a definite number of letters (a-z), but Chinese has many characters and it is indefinite. I just want to verify whether the diffusion model is valid in EN-ZH or ZH-EN.

SO, I would like to do some research based on this paper, but I do not currently have the code for that paper.

Thank you~(^_^)

[vvvm23]

SO, I would like to do some research based on this paper, but I do not currently have the code for that paper.

I haven't reimplemented all experiments in the original paper, but I have verified this implementation works. It should be okay to use this code for further research~

Because Chinese language is different from English language. English consists of a definite number of letters (a-z), but Chinese has many characters and it is indefinite. I just want to verify whether the diffusion model is valid in EN-ZH or ZH-EN.

This is true, however in English NLP we rarely tokenize individual letters (except in some datasets like text8) and instead tokenize parts of words, or even entire words.

In Chinese you can kinda do the same thing by converting individual characters into byte sequences and tokenize common sequences of bytes into a single token, or even common sequences of characters. Or just pick the top N most common characters and replace all others with an unknown token . I think any approach would be valid for discrete diffusion.

I am not too sure what is the best way to tokenize Chinese, it tends to be more complex than English as there are (usually) no clear delimiters between words.

Thanks~

[vvvm23]

Hi,
I am working on code for MT tasks in the mt branch. I'll merge it into main when I am done.
Hope this will helpful for your research once my implementation is complete.