How to restrict search to subdirectories via glob, but still respect .gitignore for files?

[RalfJung]

Is there a way to use -g to restrict rg to certain subdirectories, but still respect .gitignore for files?

A representative example of the situation I have:

mkdir test
echo test > test/foo.v
echo test > test/foo.glob
echo '*.glob' > .ignore

Now when I do rg -l test I get what I hope for, only test/foo.v.

However, when I do rg -l test --glob 'test/**', it also finds the foo.glob file, which I want to have ignored. It looks like 'glob for filename, .ignore directory' works as expected (as was explained in #1589), but 'glob for directory, .ignore filename' does not.

I know that on the CLI I can just use rg -l test test, however, this comes up in particular when searching in vscode (Cc microsoft/vscode#26574) where one can only specify glob search restrictions, not paths (it seems).

[BurntSushi]

There isn't a way, no. The issue is that, conceptually, the -g/--glob flags are meant to act like a "gitignore file that overrides everything else, but at CLI invocation time." So if you have a more specific gitignore file, its rules will override rules from a less specific gitignore file. ripgrep treats -g/--glob as the most specific as possible gitignore file. So when you say something like --glob 'test/**' as something you want to include, ripgrep matches that against test/foo.glob and whitelists it. While a more specific ignore file can override a whitelist match, there isn't anything more specific than -g/--glob. So ripgrep includes it in the search.

The simplest way I can think of to achieve what you want would be something like this:

$ rg --files | rg '^test/' | xargs rg test

or something like that.

The problem with things like VS Code that only give a limited interface to CLI tools is that you lose the compositional power of the shell. This in turn puts pressure on tools to break with the Unix philosophy and build everything into the tool. I'm not a Unix zealot and there are obviously many things in ripgrep that already break the Unix philosophy, but the answer can't be "build everything into it."

With that said, ripgrep's filtering logic breaks down at various rough edges like this. The issue tracker has lots of issues like this. Unfortunately, the whole thing needs a rethink (and I've been saying that for years), but haven't been able to manage to do it yet.

[RalfJung]

I see, thanks. I can almost make sense of this, except for the part which has been previously discussed at #1589 -- when doing --glob index.js, this does not override .gitignore ignoring entire directories. So superficially it doesn't seem as simple as "gitignore file that overrides everything else, but at CLI invocation time"?

My particular needs here would be covered by a flag that decides whether -g or .ignore has precedence, or with a --glob-early that has weaker precedence than .ignore, but I also understand that starting to add flags like this is a never-ending rabbit hole.

[BurntSushi]

--glob index.js only whitelists that one file and ignores everything else, regardless of gitignore.

But yes, definitely won't be adding more weird flags that make things even more inscrutable. This is really blocked on a rethink.