Possibilty to add translations via BGforge's web based translation tool

[brussell1]

I'm currently translating various Fallout 2 mods (UPU,RPU etc) from BGforgeNet to german. The translation is done via Weblate, a web based translation tool, at https://tra.bgforge.net/. After you connect your git repo to it, everyone, who is registered, can help with translating string by string, without caring for the formatting of the source files.
Since you ask for help and this project surely gets more popular in the future, I think it could be a good idea to consider this opportunity.
Info on how to set this up can be found here: https://forums.bgforge.net/viewtopic.php?f=9&t=26

[fedprod]

While I can't speak for Lexx, I'm personally not a fan of string for string based systems, I need my ressources, a few Notepad++ Windows/Tabs and maybe good music and a drink. That way I can see what I've done and worked on (and bitching about it when one file is waaay too big), but just me beeing old fashioned^^"

Edit: And that's how I finished the translation for 1.1.1946, meaning that readme is not quite up to date ^^ Okay, it can still be better, especially some vanilla stuff and TMAs.

[Lexx2k]

Personally I have no idea how this stuff works and how it is kept up to date.

[burner1024]

Well, the platform is open, if you want to use it, feel free to put an integration request. The system comes with various translation-related perks, but it does have some requirements/limitations as well. The details are on the forum, if something's unclear, feel free to ask too.
And if you prefer not to, it's up to you, of course. Whatever works best for translators.

[egornovivan]

So what did you decide? weblate is a very convenient tool for translating text, I see no reason to refuse it.

[Lexx2k]

Not going to happen for this branch. Primary points against it is that 1. you won't be able to directly edit text anymore. Everything needs to be done via that website, and 2. as far as I can tell it works string by string .. so it's possible translators will not understand the context of what exactly they are translating.

Most of the text files in ettu aren't very different from vanilla Fo1. IMO it'll be more convenient to just use a tool like WinMerge and compare vanilla Fo1 with ettu text files and then modify the original text files of language version X to work with ettu.

To give an example, it took me roughly half a day to get most of the german translation up to speed. fedprod did the rest of the files in a couple more days. So all in all it was pretty fast and not a big deal.
Hell I didn't even use WinMerge in this case. Just opened every single dialog file and quickly scanned it for things that were obviously different.

[wipe2238]

So. To summary recent IRC talk, translations done on bgforge-tra went from "not going to happen" to "nice to have, if...".
Big if being - can we find a working solution to "you won't be able to directly edit text anymore" problem.

For testing, i made a translations branch with script which follows integration instructions and generates .pot/.po files from scratch (based on current state of .msg/.sve/.txt files in master). This is intended to be a base for allowing handmade edits in repo monitored by bgforge-tra. I skipped steps 7+ as i need @burner1024 input before i can continue :p

After reading some of Weblate docs i believe separating .po/.pot is a step in good direction; Weblate will work with its native format only, and EtTu can check translation before it lands into game files (for example by running ScanMsg, as bgforge-tra won't protect mod from broken translations, see BGforgeNet/Fallout2_Restoration_Project#99). If i got every piece right (i'm a total Weblate noob) that's how it's gonna work:

Translation changed on bgforge-tra

• .po file(s) in translations branch are updated as usual
• @antalaskaya runs unpoify (to converts .po files to game format) and ScanMsg
• if everything seems right, master branch is updated (either manually or automagically)

.msg files changed on master branch

• @antalaskaya converts handmade changes to .pot/.po files and commits to translations branch
• webhook informs bgforge-tra about .pot/.po changes
• translators rush to bgforge-tra to translate new string with multiple smiles on their faces

Now the question left is if there's a chance to get following things (bottom of this NMA post makes me little pessimistic about that, but let's bite...):

• using non-default branch for sending/receiving changes
  all examples i could find uses default branch
• completely disable unpoify commits for everyone
  that includes translation admins or however they're triggered
• some info about how unpoify commit is actually generated server-side
  so i can replicate it on my own at the right moment; best if server-side scripts would be included in msg2po repo as well (without any sensitive data, ofc)

[burner1024]

Ahhhh... yes... come, come... join the Dark Side.

First off, cool that you decided to give it a try. It just so happens, however, that right now I'm planning an upgrade to the latest version. It's a big upgrade by itself: 3.1.1 -> 4.7, with intermediate steps, so it's a litlle bumpy. What's making it problematic is that I have to transfer/recreate the customizations added back then to handle female translations. It should be less hacky now with new functionality, but still takes time. msg2po utils will have to be adjusted accordingly too.
And while working on that, I discovered a blocker in the said functionality, which prevents me from proceeding with upgrade.

So for now the best thing to do is just sit tight. First I need a fix from upstream, then I'll do the upgrade and add customizations, then probably a week or so to stabilize things.

---

To answer your questions and raise some other random points (gonna be a long list, so I guess I'll numerate for easier reference):

1. Manual edits. Yes, you can edit the texts manually and then update po files manually. I never meant it otherwise. It's simply that if there's one source of truth, everything goes from it. If there's more than one, then someone has to keep them synced manually anyway, resolve conflicts, etc. Which really goes for anything, not just this system. So, to me it's extra work for no gain.
2. And when you do manual syncing, your continuous localization is... less continuous. Obviously it matters less in projects where source changes rarely, but still latency adds up.
3. Integration instruction you linked is for WeiDU mods, for Fallout it's not so strict, less steps. (Basically, can just look at UPU repo). You do want to lowercase dialog files, or at least make sure to keep the case consistent, otherwise you'll be looking at a lot of diff noise.
4. Checks. True, the system doesn't have built-in tests for msg file format. It was never the goal. Still, using msg2po utils gives pretty safe output. If I remember correctly, in that case the translations weren't broken as in "game is broken", gameplay wise it's just strings with typos.
   1. It can probably be made even more safe by escaping/removing/replacing curly brackets in unpoify.
   2. Weblate has its own list of checks that can be applied, and it includes auto fixups, and custom ones can be added too, see the same doc. It is a long list, though, and can become overbearing if used without caution.
   3. In the end, no tool can find all typos, but it's good to have automatic tools to run checks for common issues. Even better when they overlap and cross-check each other.
   4. I tried ScanMsg for UPU, I think it was throwing warnings on some english source strings, not allowing {000} strings or something, and I didn't want to delete them from the source, because they were actually useful comments. Didn't find how to get around that, so disabled the action for the time being.
5. Custom branches are possible, weblate itself uses a non-master branch. You can also do pull requests and stuff. I don't do either in my repos, because
   • don't really see the need
   • see point 2)
   • uniform component configuration is much easier to maintain
6. Unpoify
   1. Not sure who do you mean by "everyone/translation admins". If you mean weblate admins, so far there's one - me.
   2. But no one runs unpoify manually. It is launched by a pre-commit hook.
   3. Hook can be disabled.
   4. What it does literally is unpoify && git add ., that's all.
   5. I actually thought about making poify/unpoify into github actions. On one hand, it's going to take some load off the server. On the other, that's extra configuration steps for every repo (and since most repos now are mine, that means many more steps for me). Plus msg2po releases will have to be handled more carefully. Plus it's extra vendor lock on github/microsoft. So... undecided.
7. Extra random points you may or may not know yet, but pointing them out just in case so that I don't get accussed of ruining everything later :):
   1. PO format stuffs all identical strings into a single translation unit, which reduces workload, speeds up translation and ensures far better consistency. However, there are edge cases where that is undesirable, and you have to resort to workarounds, see 1, 2, 3. (Practically, I had 2-3 cases of this in several years)
   2. bgforge-config.py can be used to get values from .bgforge.yml
   3. dir2msgstr which I see you using is a crude tool. It was created as a quick and dirty way to load existing old translations into empty POs once and use that as a base for continuous translation. Not meant for continuous loading, you're on your own if things break. Didn't look at it for a long time, but in particular, I'm pretty sure it doesn't have the smarts to handle female translations like unpoify does.
   4. Speaking of female translations, you know about those, right? So it's not 2 sources of truth to keep in sync, but 3 at least.
   5. While uniformity goes a long way, you don't have to use the system for all languages. If you want to keep translating, say, German manually, you can just not generate a po for it and translate it manually like before. You can add languages gradually. Start with Polish or something.
   6. Holodisk entries aren't great for unit translation, because you need to align them to display well, so recently I skipped pipboy.msg.