Design a new file format like po/pot for game development

[shitake2333]

Gettext is an excellent software internationalization and localization solution, and godot also supports po/pot to complete game localization.

However, in addition to text resources, there are many other non-text resources in game development that require internationalization and localization support. This is something that po/pot cannot handle.

Do we need to design a new extension file (poe/poet) dedicated to the internationalization and localization of non-text resources to achieve better support?

In this way, godot only needs to generate pot and poet files at the same time to complete internationalization. Localization personnel generate po and poe files in the corresponding language based on these template files, and the engine loads these files at runtime to achieve game localization.

Furthermore, the existing TMS (Translation management system) only supports text processing. We can develop a new TMS to better support game localization (by providing SDK or engine direct integration to work together).

[Delsin-Yu]

Can you provide an example of a specific scenario? I'm having trouble imaging how your poe/poet addresses the i18n related issue. Is it some kind of InternationalizedResource?

[shitake2333]

poet is similar to pot, but it records multilingual resources in other formats (such as audio and images) instead of strings that need to be localized.

I am not sure what to use as the unique key (file hash and file path are both OK, but each has its own problems)

We assume that the file hash is used as the key and uses a format similar to pot. Here is a hypothetical poet file item:

white-space
#  translator-comments
#. extracted-comments
#: reference…
#, flag…
#| msgid previous-untranslated-string
resid 4d9e57cdded3a0ec5caa05b73741b71d
respath res/icon/flag.png
lreshash localization resource id
lrespath localization resource path

[shitake2333]

The poe file is an index table of non-text localized resources

[Calinou]

Godot already supports defining translation remaps in the Project Settings.

[shitake2333]

The poet file can work with existing translation remaps.

translation remaps requires manual addition of resources that need to be mapped, while poet is similar to pot and generates a table of resources that need to be mapped according to the designed rules. In this way, the translation remaps system can load this table, and the staff only needs to add the corresponding mapping to each resource to be translated according to the table.

[shitake2333]

This is an upgrade of the existing system rather than a redevelopment

[timothyqiu]

the existing TMS (Translation management system) only supports text processing

This is why I said you might be reinventing the wheel on RocketChat. Godot does support remapping resources based on the current language, see Project Settings > Localization > Remaps.

Do we need to design a new extension file (poe/poet) dedicated to the internationalization and localization of non-text resources to achieve better support?

A proprietary format means that translators can't use their existing tools to create & manage translations, e.g. Poedit and gettext CLI tools.

If you just want to make wiring resource remaps for different languages easier, I think adding an API for setting these remaps dynamically at runtime might be enough (currently users can only set that in the project settings). It does not need to touch the generation and management of the translation files.

[shitake2333]

A proprietary format means that translators can't use their existing tools to create & manage translations, e.g. Poedit and gettext CLI tools.

Oh, sorry, I didn't make it clear, TMS refers to translation content management websites such as weblate. So the TMS improvement I mentioned is to solve this problem.

This is why I said you might be reinventing the wheel on RocketChat. Godot does support remapping resources based on the current language, see Project Settings > Localization > Remaps.

I already know about Remaps...

If you just want to make wiring resource remaps for different languages easier, I think adding an API for setting these remaps dynamically at runtime might be enough

I want to improve the internationalization process and translator experience. Here are some examples I've used in Unity

If non-text multilingual resources use the same processing flow as existing text multilingual resources, there are many benefits. For example, the most common non-text multilingual resource is the dubbing file of the text. Using the same processing flow, we can match the two in TMS, which is very helpful for localizers.