Two spaces are not treated as line break in gettext builder

[ivandkhn]

Describe the bug

Suppose we have the following Markdown file:

Some info with two images:  
![](a.png)  
![](b.png)

Note that it uses two spaces at the end of the lines to produce line breaks:

Some info with two images:<SPACE><SPACE>
![](a.png)<SPACE><SPACE>
![](b.png)

When gettext generates .po files for this document, it seems that it ignores the line breaks and threats everything as a single paragraph:

#: ../source/test.md:1
msgid "Some info with two images: ![](a.png) ![](b.png)"
msgstr ""

However, I would expect the lines to be separated:

#: ../source/test.md:1
msgid "Some info with two images:"
msgstr ""

#: ../source/test.md:3
msgid "![](a.png)"
msgstr ""

#: ../source/test.md:5
msgid "![](b.png)"
msgstr ""

How to Reproduce

Run python3 -m sphinx -b gettext source/ build/ for source/test.md:

Some info with two images:
![](a.png)
![](b.png)

build/test.pot will have the following:

# SOME DESCRIPTIVE TITLE.
# Copyright (C) 2023, test
# This file is distributed under the same license as the test package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: test \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2023-10-26 10:44+0900\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"

#: ../source/test.md:1
msgid "Some info with two images: ![](a.png) ![](b.png)"
msgstr ""

Environment Information

Platform:              linux; (Linux-6.1.0-0.deb11.7-amd64-x86_64-with-glibc2.31)
Python version:        3.9.2 (default, Feb 28 2021, 17:03:44) 
[GCC 10.2.1 20210110])
Python implementation: CPython
Sphinx version:        5.3.0
Docutils version:      0.17.1
Jinja2 version:        2.10.3

Sphinx extensions

extensions = []

Additional context

No response

[picnixz]

We don't support older versions of Sphinx so could you tell me if the bug persists with 7.2.6?

Btw, is it common to do that in markdown? I mean, is it supported by the MD specs? why not using real new lines?

[ivandkhn]

We don't support older versions of Sphinx so could you tell me if the bug persists with 7.2.6?

I've tried on 7.2.6 and got the same result – all text is concatenated in a single string.

Platform:              linux; (Linux-6.1.0-0.deb11.7-amd64-x86_64-with-glibc2.31)
Python version:        3.9.2 (default, Feb 28 2021, 17:03:44) 
[GCC 10.2.1 20210110])
Python implementation: CPython
Sphinx version:        7.2.6
Docutils version:      0.20.1
Jinja2 version:        3.1.2
Pygments version:      2.17.2

Btw, is it common to do that in markdown? I mean, is it supported by the MD specs? why not using real new lines?

To be honest, I was surprised to learn myself that such a feature exists. But some people just use it because it's a part of the specs – both the original:

When you do want to insert a <br /> break tag using Markdown, you end a line with two or more spaces, then type return.

and CommonMarkdown, example 633:

A line ending (not in a code span or HTML tag) that is preceded by two or more spaces and does not occur at the end of a block is parsed as a hard line break (rendered in HTML as a <br /> tag):

[picnixz]

Thank you. Now I am not sure whether we expected the source to be markdown for the gettext rule.

If so, I would say it's a bug and we need to fix it. If not, we should probably tell users what format we actually support.

I won't have time for checking or making a PR so anyone willing to help is welcomed.

[j123b567]

I think that the biggest problem here is that gettext does not preserve important newlines in this situation. So if the text has line breaks in the source text and it is still one paragraph, it should be one msgid bit with line breaks \n. Unfortunately all new lines are converted to space so \n generates forced line break but will not.

Same problem is with MyST line break \ which works only if it is followed by real new line but it is not preserved during translation.