Restore master branch and get point releases back online

[xuhdev]

The Security Problem with the develop Branch

The develop branch of Spacemacs pulls a lot of packages from Melpa, directly
from their latest releases. Neither Spacemacs nor Melpa examines these new
versions. When a user update packages, they install the latest versions
directly, and usually do not examine the changes in updated packages -- the
sheer number of packages is often quite large!

The whole usage pattern poses a severe security hazard: If any of the packages
(Spacemacs pulls 300+ packages on my system) go malicious, even just for one
day, a lot of Spacemacs users are attacked. This is commonly known as one kind of supply chain attack. [1]

Solution

The security problem can be mostly mitigated by the normal software engineering
practice of making point releases with freezing periods. The master branch
(which is not virtually unupdated today) uses a dedicated registry with packages
of frozen versions. Given a short freezing period of time before releasing,
due to the open source nature of the packages, it's unlikely that a malicious
package can remain hidden.

I recommend the following practice:

1. Cut point releases regularly
2. Mirror relevant packages of Melpa on that day to spacelpa or a similare registry
3. Freeze the release for one or two weeks

I believe it's better to be safe than sorry later.

[1] Packages can go malicious for many reasons, such as inactive account being stolen, owner organization acquired by a malicious actor, etc.

[xuhdev]

A bit clarification on maintenance burden

The core of this recommendation is point release practice. The point release doesn't need to be maintained with backporting bug fixes like Debian stable, perhaps with the exception of severe security vulnerability. Therefore, the maintenance burden should be low, and likely can be automated.

People on point releases may complain about some bugs. But this can be mitigated by releasing frequently. In theory, you can even release daily, it's just the packages from melpa need a freezing period.

[xuhdev]

I'm working on a project to create melpa snapshots: https://github.com/delpa-org/snapshot-2025-01-02

Using such repos should also be able to address the concern. However, spacemacs doesn't seem to allow me to adjust package repositories...

[fnussbaum]

I would support not pulling directly from MELPA by default. If we decide to implement some kind of mirror, I think it would be great to have sort of a semi-automatic update process for the mirror were, ideally, package updates are reviewed, and were one can easily hold back updates for certain packages or pin them to specific commits.

Some related projects I found: github-elpa, mirror-elpa, elpa-mirror.

In theory, you can even release daily, it's just the packages from melpa need a freezing period.

I think it would indeed be possible to keep the current process without explicit releases, and sync the Spacemacs development and state of the mirror.

[xuhdev]

I've created a convenient snapshot project at https://delpa.org/ .

It's not at its best shape yet, as users can only point to a fixed Melpa snapshot for now. In the future I plan to create a moving redirection (e.g., https://packages.delpa.org/at-least-days-old/7 points to the most recent snapshot that is at least 7 days old).

Regardless of using a fixed snapshot or a moving one in the future, Spacemacs need to provide a way for user to override the default respository. This has been requested many times in the past but the hacks kept losing their usefulness...

• request to consider local package archives #4230
• [question] how do I tell spacemacs to look for a local elpa mirror? #7103

[fnussbaum]

I've created a convenient snapshot project at https://delpa.org/ .

Thanks for the initiative, this looks good to me! Would it make sense to also snapshot GNU ELPA/Non-GNU ELPA, or at least the packages that are only published there and not on MELPA?

It might also be a good idea to discuss the problem and the approach in general with the MELPA maintainers.

Spacemacs need to provide a way for user to override the default respository.

Yes this should be made easy to configure in any case.

[fnussbaum]

@smile13241324 @bcc32 What do you think about this?

[bcc32]

Definitely +1 to making package-archives and package-archive-priorities more configurable by the user. Having snapshots of GNU/Non-GNU would be good too, so that you could have a full historical picture of packages and (maybe) eventually eliminate the need for Spacemacs to provide its own custom package rollback mechanisms (i.e., moving the whole configuration in the direction of being more declarative, version-controllable, and reproducible).

In that vein, I'd like to suggest not providing a rolling "7 days ago" snapshot by default, but instead providing commands to automatically bump the timestamped snapshot referenced in the user's config. For example, you could then configure SPC f e U to (1) bump the configured snapshot 7 days ago and then (2) upgrade packages via that snapshot.

[bcc32]

As for point-releases of Spacemacs itself, we should perhaps track that discussion in a separate issue. It's not entirely related to package security (obviously some overlap in concerns).

My main dissatisfaction with the current state of releases in Spacemacs is that it's very easy to push breaking (i.e., non-backwards-compatible) or broken (e.g., errors at startup, typos, etc.) changes to the develop branch, and roughly all users will be impacted on their next pull. Granted, people upgrade at different times and hopefully the people who upgrade early will report any issues quickly, but it would still be nice to at least have a separate branch/tags/whatever, where we'd have some defense against automatically pushing any possibly-regression-containing commits.

[smile13241324]

I like the idea but we need to discuss how to do this properly. I will have my great appointment this weekend then I should be more available again.

EDIT
After having read through the entire thread I would definitely give my thumbs up for an updated master branch and a configurable package archive.

The stale master gives a wrong impression for new users and should definitely be fixed. On this branch we should point to fixed package versions by default, similar to what doom emacs does.

On develop the default should be a different package archive so that all tests with the same package versions. Once we reached a freeze event we should do a new release and move the packages from dev to stable then upgrade dev again.

Of course users must be able to configure this conveniently via the dotfile if they want to use melpa directly.

The only concern I have is what to do with pure github based packages. Here we would need to refer to fixed releases as well but I am unsure if quelpa allows that.

[xuhdev]

Definitely +1 to making package-archives and package-archive-priorities more configurable by the user. Having snapshots of GNU/Non-GNU would be good too, so that you could have a full historical picture of packages and (maybe) eventually eliminate the need for Spacemacs to provide its own custom package rollback mechanisms (i.e., moving the whole configuration in the direction of being more declarative, version-controllable, and reproducible).

I don't have plans for gnu and non-gnu, as I don't see an urgent need for them. The packages there are generally carefully versions and aren't really breaking. Each version is also verified, similar to a package in most Linux distributions.

In that vein, I'd like to suggest not providing a rolling "7 days ago" snapshot by default, but instead providing commands to automatically bump the timestamped snapshot referenced in the user's config. For example, you could then configure SPC f e U to (1) bump the configured snapshot 7 days ago and then (2) upgrade packages via that snapshot.

I think it's a good idea. Available snapshots can be accessed from https://delpa.org/melpa_snapshot_versions.json . Spacemacs can read from that file.

[xuhdev]

I also opened melpa/melpa#9318 and see if we can get any better ideas on Delpa.

[fnussbaum]

The only concern I have is what to do with pure github based packages. Here we would need to refer to fixed releases as well but I am unsure if quelpa allows that.

This is possible with both quelpa and package-vc, by specifying a commit hash and/or branch. (Actually, I haven't tried with package-vc, but with quelpa it works for sure.)

I don't have plans for gnu and non-gnu, as I don't see an urgent need for them. The packages there are generally carefully versions and aren't really breaking. Each version is also verified, similar to a package in most Linux distributions.

I think the package updates are signed, but not necessarily reviewed, see also https://lists.gnu.org/archive/html/emacs-devel/2013-09/msg00450.html (not sure how much has changed since then though).