brew test and sandbox

[timsutton]

I am looking to add a new formula for a CLI tool: https://github.com/ninxsoft/mist

My formula's now passing various audits and I'm looking to add a test to at least assert that the command runs and outputs the matching version (and potentially something else functional if I can add it too).

However, by nature of how it calls macOS Foundation APIs to fetch a URL (it does a "get me the latest known version" check as part of printing the tool's version), the runtime ends up wanting to write some cache data into ~/Library/Caches/<foo>/*.db and the sandbox that brew test runs in prevents this from working, so the test block fails as it can't execute this command.

Is there any sane way to side-step this inside of how the brew test block is executed? I'm thinking that even if the sandbox could be disabled when the test block happens, the cache cleanup logic starts to make this messy. Alternatively I can look to add a feature to the tool to disable the network lookup via some flag. But just wanted to ask since I figured that this may have been an issue that's come up before?

[SMillerDev]

The HOME variable should be set to the test path, so ~/Library/Caches should work just fine.

---

it does a "get me the latest known version" check as part of printing the tool's version

Just checking the version of a tool is considered a bad test, see https://docs.brew.sh/Formula-Cookbook#add-a-test-to-the-formula

[timsutton]

Thanks! I'll take a closer look.

I'll see what else I can do for adding something more functional rather than just comparing the version. I can, for example, have it assert that when it does its list operation to show all discoverable installers, and has output configured in json, that the output is valid parseable JSON. It sounds from the guidance in the docs that if I have something like this it is preferable than just executing the command and then asserting that (for example) there's some specific known usage output like -h, --help etc.