gem sources --prepend and --append allow finer grained control of sources

[martinemde]

What was the end-user or developer problem that led to this PR?

I need a way to programmatically and idempotently add a source to the top of the sources list.

Previously, sources could only be appended using the gem sources --add command since rubygems.org was always re-added as the first source when there were no other sources.

What is your fix for the problem, implemented in this PR?

This improves the ability to add sources programmatically by enabling sources to be added in front of rubygems.org.
It also renames the primary command line options to --append and --prepend and allows you to use the flag multiple times (in which case it behaves as if the command was called multiple time with each flag).

One edgecase I didn't fix: if you --append and --prepend the same source, the order is indeterminate. I hope that's just not a thing people do, because fixing it seemed like unnecessary complexity.

Make sure the following tasks are checked

• Describe the problem / feature
• Write tests for features and bug fixes
• Write code to solve the problem
• Make sure you follow the current code style and write meaningful commit messages without tags

[deivid-rodriguez]

This sounds good!

I doubt usefulness of current --add option, so maybe we could consider changing its behavior, or removing it, too?

Currently

$ gem source --add https:/my.gem.source # includes test-gem-0.0.1
$ gem install test-gem -v 0.0.1 # installs test-gem-0.0.1 from rubygems.org

I've never seen private/custom sources used as a rubygems.org fallback, but the other way around. People want gems installed from their private sources, and their dependencies (if missing) fallback to being picked from rubygems.org.

Should we deprecate --add in favor of --prepend, or directly change --add behavior for RubyGems 4.0?

[martinemde]

I think the best solution would be to keep the --add behavior but deprecate the flag and switch to --append.

That makes the interface clear and doesn't break any uses that we aren't imagining right now. If we want to add a deprecation now, we could fully remove in 4.0 or just leave it there as an alias for --append, which I don't think would cause any harm.

I vaguely remember dealing with this problem in 2010ish. This has probably been the API as long as rubygems has existed so whatever the shortcomings of --add, people have been dealing with it for a very long time.

Edit: One argument for removing --add is that it's usually not what you want and we want the default to be the right choice. You usually want prepend. Forcing that would help a lot of people that don't work with rubygems very often (since mostly people work with bundler for these things).

[martinemde]

I'm inclined to merge this. If I'm not mistaken, we want --prepend right now regardless of our choice about --add since even if we replace --add we need a deprecation period.

[deivid-rodriguez]

Yeah, the only alternative I see would be to change add behavior directly and introduce the breaking change in RubyGems 4.0. But I think it's better to add prepend for now, and consider deprecations/removals later.

A couple of things I noticed:

• prepend deviates from add in that it also moves the source to the front if already there. So if we were to add append, it wouldn't be exactly add, since for symmetry we'd probably want to move the source to the bottom if already there?
• There are existing tests for typo squatting, for checking that the default source uses https, etc, when using --add. Since we're most duplicating the add implementation, we should probably also duplicate those tests to make sure that same stuff applied to add also applies to prepend. If we ever remove add, we could end up losing the tests for these checks otherwise.

[deivid-rodriguez]

Oh, sorry, I missed your other PR. That makes sense to me since now there's a nice symmetry between append and prepend. They're both slightly different to add, but I'm not sure if anyone will need the source appended, but left in the first position if it's already there (although I'm not sure either if anyone needs the source appended at all in the first place).

[martinemde]

Yeah, I do wonder if append is ever the right thing to do. I sort of doubt anyone uses the sources command in a serious way.

[martinemde]

Now I'm thinking about --add-first and --add-last. Is that more clear than prepend and append?

[deivid-rodriguez]

Now I'm thinking about --add-first and --add-last. Is that more clear than prepend and append?

I think they are!

[martinemde]

I've been thinking about this and I think the most obvious declarative interface is:

$ gem sources --first https://mysource.dev
Source added (or moved) to first position
$ gem sources --last https://rubygems.org
Source moved to last position

With just the declarative, and short, first and last, we can imply both movement and addition.

I also think that sources --clear should fully erase all sources, leaving only rubygems.org. This allows fully idempotent setting of source without having to edit the file directly.

We can move the cache clearing to a more explicit --cache-clear.

[martinemde]

Nope... I don't like gem sources --list and gem sources --last being so close. I'm going back to --append and --prepend because it's immediately obvious what they do.

I think the semantic confusion of "if it's already there, we're going to move it" is really not bad.

We have three choices that I see for when the source is already added:

1. fail
2. move the source
3. do nothing

Between the choices, option 2 seems more desirable since it does what you asked: the source is now in the position requested without requiring prior knowledge of the order or contents.

We can either be helpful and do it, or leave you in a weird state where you have to respond to the output and adjust.

My specific use case is being able to blindly run this on dev machines where I don't know sources and state. I could be heavy handed and delete all sources and start completely fresh, but if sources move properly I can ensure that the private source is first and then not care about the rest of the contents.

[deivid-rodriguez]

I agree to add --append and --prepend with semantics as implemented in this PR. But I'd keep --add as it is (and all of its tests) for now. Then deprecate it separately. Then eventually remove it on a major version.

[martinemde]

Updated to make this only additive (besides a little refactoring)

[deivid-rodriguez]

Great!