Handling of invalid and dev versions when building from source

[PhrozenByte]

When attempting to build Borg 1.4 from source with setuptools_scm failing to determine a valid version string automatically, any Borg invocation fails with the following ValueError:

$ borg --version
Traceback (most recent call last):
  File "/borg-build/src/borg/archiver.py", line 47, in <module>
    from .archive import Archive, ArchiveChecker, ArchiveRecreater, Statistics, is_special
  File "/borg-build/src/borg/archive.py", line 24, in <module>
    from .cache import ChunkListEntry
  File "/borg-build/src/borg/cache.py", line 31, in <module>
    from .crypto.key import PlaintextKey
  File "/borg-build/src/borg/crypto/key.py", line 30, in <module>
    from .nonces import NonceManager
  File "/borg-build/src/borg/crypto/nonces.py", line 8, in <module>
    from ..remote import InvalidRPCMethod
  File "/borg-build/src/borg/remote.py", line 41, in <module>
    BORG_VERSION = parse_version(__version__)
  File "/borg-build/src/borg/version.py", line 24, in parse_version
    raise ValueError('Invalid version string %s' % version)
ValueError: Invalid version string 0.1.dev1+gfb10207fe

It looks like you tried to handle this gracefully in __init__.py:

borg/src/borg/__init__.py

Lines 11 to 21 in b91fb89

	# assert that all semver components are integers
	# this is mainly to show errors when people repackage poorly
	# and setuptools_scm determines a 0.1.dev... version
	assert all(isinstance(v, int) for v in __version_tuple__), \
	"""\
	broken borgbackup version metadata: %r
	version metadata is obtained dynamically on installation via setuptools_scm,
	please ensure your git repo has the correct tags or you provide the version
	using SETUPTOOLS_SCM_PRETEND_VERSION in your build script.
	""" % __version__

This actually pretty helpful error message is never shown unfortunately.

Furthermore, the SETUPTOOLS_SCM_PRETEND_VERSION env variable is nowhere mentioned in the docs. It might be a good idea to mention it in the "Installation - From Source - Using git" section of the docs.

Lastly, the auto-generated versions of setuptools_scm don't match Borg's version string format and are currently treated like stable versions (parse_version() e.g. yields (1,4,2,-1) for the auto-generated version 1.4.2.dev77+gfb10207fe, it should be something like (1,4,2,-9)).

[RonnyPfannschmidt]

This looks like a shallow clone use real git

@ThomasWaldmann more recent versions support a export subst based annotation if you like we can add that to borg to deal with part of the issue

[ThomasWaldmann]

Yes, I also suspect it is a shallow clone or misses tags.

Not sure what "export subst based annotation" means, but I am fine with it if it improves the situation.

[PhrozenByte]

This looks like a shallow clone use real git

It indeed is. However, just to make this clear, I'm absolutely fine with this failing and requiring manual intervention. I just feel like that the env variable should be documented and that this already existing and pretty helpful error message should be shown instead.

---

Related topic:

The version parser doesn't know what a dev version is, just alphas/betas/rcs. The parser is fine with prefixes and suffixes, but they are ignored when comparing versions (i.e. Borg happily accepts e.g. 1.4.2.dev77+gfb10207fe as version (which is the auto-generated version for the matching full clone btw), but it is treated like a stable version and the parser yields (1,4,2,-1); it should be something like (1,4,2,-9) instead).

Updating parse_version() accordingly is trivial and I could add that to #9015 if you think that's a reasonable addition and doesn't break anything (I don't know Borg's code, so I can't know)? However, there's one issue: The auto-generated dev versions don't match the known format: Note the . before dev. Is there a way to tell setuptools-scm that the auto-generated version should be 1.4.2dev77+gfb10207fe, not 1.4.2.dev77+gfb10207fe? Or we tell parse_version() to optionally accept a dot there… WDYT?

[ThomasWaldmann]

Looks like documenting SETUPTOOLS_SCM_PRETEND_VERSION usage is now fixed in master and 1.4-maint.

[PhrozenByte]

Thanks for merging the PRs Thomas 👍

I just updated the issue title and original text to also reflect the findings in parse_version().