What is the merit of having heading IDs translated per localization?

[debiru]

For example, the following URL is available:

• https://docs.astro.build/en/install-and-setup/#browser-compatibility

The URL to access the same section in the Japanese localized version is as follows:

• https://docs.astro.build/ja/install-and-setup/#%E3%83%96%E3%83%A9%E3%82%A6%E3%82%B6%E3%81%AE%E4%BA%92%E6%8F%9B%E6%80%A7

Since they point to the same section, doesn't it make sense that even the localized version should be accessible at the following URL?

• https://docs.astro.build/ja/install-and-setup/#browser-compatibility

Why do you use different heading IDs for the English and localized versions?

I found the following pull request and I agree with the policy of such a modification. #9346

For reference, a similar discussion is also taking place at MDN. Translation of IDs of headings

I would like to seek your views on this.

[delucis]

In our case we chose this mainly for ease of maintenance — using English IDs in non-English pages introduces quite a lot of extra work (for example, every translated heading would need to make sure it also includes an explicit heading ID, and then those would need to be kept in sync somehow with the English — or the English would also need explicit heading IDs). That’s a lot to juggle for contributors and would also require more CI tooling to ensure IDs are always valid. We didn’t see a clear benefit to justify all that effort.

[debiru]

Thank you for your response.

You had a clear intention in deciding on the policy.

Indeed, it may take more effort to keep the ID the same compared to writing a localized version of Markdown without thinking.

However, if you want to do a static analysis to see if the English and localized translations are in sync, having the same heading IDs should be one important factor to consider. Personally, I also do not like URLs to be percent-encoded and long, as it is unwieldy and not elegant.

If the localized translation is maintained as an independent document from the English version, it is difficult to mechanically examine which pages are synchronized and which pages are out of sync.

Although this is separate from heading ID, I believe that localization work should not be done by creating independent documents, but by using dictionary files to prepare translated sentences that correspond to English for each sentence. That way, it should be possible to find out which paragraphs are synchronized on a paragraph-by-paragraph basis, which should make the task of updating the localized version easier.

I made a similar suggestion at MDN, but MDN has more than 10,000 pages and it was already too late to change the policy. However, it is not too late for Astro. We believe that we can create a more efficient localization system now, when there are only about 500 pages at most.

Does anyone agree with my idea? Of course, opposing views are welcome. I would be happy to consider creating a better localization mechanism.

[delucis]

We do already track what is up-to-date in case you haven’t seen it!

The dashboard is here: https://i18n.docs.astro.build/

Granted, this is at the page-level, rather than able to have subheading granularity, but like I mentioned above the extra maintenance burden on our volunteer translators seems too much just to get slightly better progress tracking. Especially considering we’ve managed to grow our docs site and translations very successfully using this existing system.

[debiru]

Certainly that dashboard has a certain value.

However, in localizing the documentation, there are a variety of issues that need to be resolved.

• (1) That dashboard only looks at the update date and time, not the content.
• (2) There is no way to check if the translated document is outdated from the English version in terms of content.
• (3) There is no way to check if the heading structure of the translated document matches that of the English version.
• (4) There is no way to detect when word-for-word translations differ from page to page.
• (5) If page A is translated and there is an internal link to page B, there is no way to know what hash fragment to set if page B does not exist in the localized version.

I believe that one way to solve all these localization problems at once is to import the English file into the localized version and then translate it in dictionary format. In addition to that, I believe that once a mechanism is in place, it will not only be a hassle for the translator, but will even reduce the hassle.

By the way, how do you guys localize new pages?

My current procedure for translation

(1) Copy the mdx files from the English directory to the Japanese directory, then commit

---
title: About Astro
description: This page is fictitious
---

# About Astro

The web framework for content-driven websites.

(2) Translate the front matter and commit (at this time, delete the original English)

---
title: Astroについて
description: このページは架空のものです
---

# About Astro

The web framework for content-driven websites.

(3) Translate the content and commit (at this time, the original English is deleted)

---
title: Astroについて
description: このページは架空のものです
---

# Astroについて

コンテンツ駆動型ウェブサイトのためのウェブフレームワークです。

One idea for a new mechanism

(1) Copy the mdx files from the English directory to the Japanese directory, then commit

---
title: About Astro
description: This page is fictitious
---

# About Astro

The web framework for content-driven websites.

(2) Translate the front matter and commit (at this time, leave original props)

---
originalProps:
  title: About Astro
  description: This page is fictitious
title: Astroについて
description: このページは架空のものです
---

# About Astro

The web framework for content-driven websites.

(3) Translate the content and commit (at this time, leave the original English)

---
originalProps:
  title: About Astro
  description: This page is fictitious
title: Astroについて
description: このページは架空のものです
---

<Localization>
<Original>
# About Astro
</Original>
<Translation>
# Astroについて
</Translation>

<Original>
The web framework for content-driven websites.
</Original>
<Translation>
コンテンツ駆動型ウェブサイトのためのウェブフレームワークです。
</Translation>
</Localization>

• The content of the original language is basically hidden in the localized version, but heading IDs and other information from the English version can be automatically transferred to the localized version.

This is just one idea and may need to be brushed up for practical use. However, this kind of system may make it possible to do various things. Are these ideas of mine worthless? Do you localization contributors have any opinions?

[vrabe]

What you are pointing out is not a problem.

If page A is translated and there is an internal link to page B, there is no way to know what hash fragment to set if page B does not exist in the localized version.

Just leave the hash fragment untranslated. Page B falls back to English version.

The problem is it causes dependencies between translated pages. If page B gets translated in the future, we'll need to modify page A together to update the link. If page A is outdated at that time, we'll also need to update it to date.

[debiru]

The problem is it causes dependencies between translated pages. If page B gets translated in the future, we'll need to modify page A together to update the link. If page A is outdated at that time, we'll also need to update it to date.

I think this situation is problematic and a matter that should be remedied, in my opinion.

I believe that it is a heavy burden on the translator to have to revise other pages that link to page B just to translate page B. This burden may cause the translator to stop translating page B.

[ArmandPhilippot]

I haven't intervened before because I'm happy with the current process, but if it helps, here's my reaction to your thoughts.

Do you localization contributors have any opinions?

I started translating the docs in French less than a year ago and the process seems quite simple to me so I don't necessarily see any improvement to be made. Certainly, managing links can be problematic but I don't have a better solution.

If the localized translation is maintained as an independent document from the English version, it is difficult to mechanically examine which pages are synchronized and which pages are out of sync.

This is only true when the file is very outdated and many commits have occurred between the English version and the translation. Otherwise, with Git Diff, it's quite easy to check what has changed. But I understand that this can be problematic.

Personally in such a case, I just use the English file and compare line by line in my IDE to see if that's match. Perhaps it takes a bit more time but I'm not bothered by that.
I don't know which tool you use but for example, with VS Code, casungo had suggested two plugins on Discord to help translators: Sync Scroll and Make Hidden.

using dictionary files to prepare translated sentences that correspond to English for each sentence

A dictionary is an interesting idea when certain strings repeat in exactly the same way, but it seems to me that this is more or less what is already done with i18n/en.yml.

There are also the i18n-guides to help being consistent. But to be honest I only noticed their existence long after I started to translate... So, yeah, there is no automatic way to do that.

I think a dictionary for sentences could introduce other problems. Sometimes the same set of words could be translated in another way according to the context.

(1) That dashboard only looks at the update date and time, not the content.
(2) There is no way to check if the translated document is outdated from the English version in terms of content.

Well, if the English version is updated, the translation should be updated as well. From the dashboard, you can view the commit list to verify the changes. So I'm not sure I understand the issue here...

(3) There is no way to check if the heading structure of the translated document matches that of the English version.

If you use VS Code, I believe the Outline panel can help you here.
In the browser, you could also use a plugin to compare the English version and the translation. I don't have a name to suggest but I know some plugins helps you extract all the headings in the page to check the hierarchy.

(5) If page A is translated and there is an internal link to page B, there is no way to know what hash fragment to set if page B does not exist in the localized version.

I have to admit that this is the most complicated point when the translation is not up to date or there are big changes related to a version change (e.g. the v5 release). And I don't have any solutions to solve this issue either. For the French translation, we managed to get by. But, yeah, it's not "easy".

By the way, how do you guys localize new pages?

1. I create a new file with the same name.
2. I copy the English content.
3. I start the translation paragraph by paragraph.
4. If it takes a while, I commit and eventually push to my fork to have a backup.
5. I continue the translation.
6. I commit or amend the commit and push (or push force) to my fork. And repeat steps 5 and 6 if needed.
7. I create the PR.

In between, I use the dev server to check the rendering and update the links.

When there are link issues... well, I handle them on a case-by-case basis. Often, if I know I'm going to be working on the linked page that's causing the issue, I remove the anchor and put it back when I translate the other page. I'm confident with this process because I keep track of these issues and I follow the PRs made in my language and I can alert the author PR if the translation has been made by someone else. So, I'm not saying this is a solution for others, but it works for me.

One idea for a new mechanism

To be honest, I'm not convinced here. Looking at (3) I feel like it will be harder to follow the changes made in the English version... IMO, this solution adds noise and we can no longer rely on line numbers when looking at git diffs.

Maybe if I saw this process in action I would have a different idea. But right now, I'm having trouble seeing the point. I think it's easier to have everything in the same language to make sure that each paragraph is consistent with each other. But that's just my opinion.

having the same heading IDs should be one important factor to consider. Personally, I also do not like URLs to be percent-encoded and long, as it is unwieldy and not elegant.

Yeah, it's not ideal with some alphabets. But I'd say this is more a browser issue...
I know Firefox has a hidden config to avoid the encoding when copy pasting a link. I don't know if there is something similar with other browsers. Perhaps a plugin can help here too.

Using the English ID could maybe be a better translator experience but I don't think this a better user experience. Ideally, I think links should be fully translated, so anchors (ids) as well. But my opinion may be biased in relation to my alphabet.
So like Chris said, not sure there is clear benefit here.

[debiru]

Thank you for your detailed response. I am a little late in replying.

I think my proposal is still in the process of being brushed up and contains parts that are not practical. The feedback I received on my proposal helped me better understand what was good and what was bad.

I would like to think more about whether we can make some suggestions in the direction of further improvement without denying the way things have been done. I have not yet come up with a good way to do this, but I hope we can come up with some answers through further study.

[vrabe]

If we can't have English IDs because of maintenance burden, can we come out with alternative solutions to this problem?

Like limit link checks in CI? Or create a field in frontmatter of translation to declare a corresponding English doc version like what MDN does (i10n.sourceCommit)?(Seems not that useful)

[delucis]

can we come out with alternative solutions to this problem?

Could you explain a bit more what “this problem” refers to? What are the issues you are looking to find a solution for?

[vrabe]

As mentioned in my previous comment.

[sarah11918]

The problem is it causes dependencies between translated pages. If page B gets translated in the future, we'll need to modify page A together to update the link. If page A is outdated at that time, we'll also need to update it to date.

It does, but we have tests to check for this! When this happens, checks don't pass and we fix them. So, it's not really a problem in that we know when it happens, and how to fix it. This is simpler to maintain than every model we've thought about for not localizing the headings.