[Dev Tools] extremely slow forge tests

[vmaark]

Team or Project

Treasure

Environment

Mainnet

Select the Dev Tool you are using

Foundry

Provide the version of the tool (if applicable)

forge 0.0.7 (0a4af94 2025-01-31T21:19:40.678985000Z)

Provide a brief description of the functionality you're trying to implement and the issue you are running into.

I have a test suite which is extremely slow to run, 20-30 minutes instead of seconds without the --zksync flag. I have the same suite in multiple projects, as it's a UniswapV2 fork. Is this a known issues, or are others experiencing the same?

Repo Link (Optional)

No response

Additional Details

This is the only clue I get at the beginning:

[⠊] Compiling...
No files changed, compilation skipped
[⠃] Using zksolc-1.5.11
[⠊] Compiling (zksync)
No files changed, compilation skipped
proptest: Aborting shrinking after the PROPTEST_MAX_SHRINK_ITERS environment variable or ProptestConfig.max_shrink_iters iterations (set 0 to a large(r) value to shrink more; current configuration: 0 iterations)
proptest: Aborting shrinking after the PROPTEST_MAX_SHRINK_ITERS environment variable or ProptestConfig.max_shrink_iters iterations (set 0 to a large(r) value to shrink more; current configuration: 0 iterations)

[bxpana]

Thanks for the heads up, do you have a repo or something to share for us to try and reproduce?

[vmaark]

sorry still don't have the minimal reproduction, just wanted to share another detail, during the test run the forge process is continuously using 90-99% CPU (M1 Macbook Pro).

[MexicanAce]

@vmaark if it's possible to share the repo as-is and steps to reproduce, we can take a look to see what optimization settings can be enabled for foundry-zksync (for example forge build --zk-force-evmla=true --zksync or in your foundry TOML config)

[MexicanAce]

@vmaark I'm going to mark this discussion as answered with my previous comment, but if you are still experiencing long build times, I would please post a repo that we can replicate the issue and unmark the previous post as the answer.

Once we have that information, we can offer some foundry TOML config settings that might help reduce the CPU/build time.

[vmaark]

hey @MexicanAce, finally got a repo set up for showing https://github.com/vmaark/slow-zksync-tests-reproduction

[vmaark]

btw tried --zk-force-evmla=true, didn't change the test run result

[dutterbutter]

@vmaark really, really, really appreciate you setting up a minimal reproduction repo here 🙏 This will help us in benchmarking as we making improvements.

[thepluck]

Hi @MexicanAce, I experienced the same issue, the build time is negligible but the time required to run tests is extremely long, my project only has 26 fuzz tests with 16 runs for each but it took up to 2 hours to finish.

I use foundry-zksync-v0.0.8. Here is my foundry.toml:

## Profile configs
[profile.default]
src = 'contracts'
libs = ['node_modules', 'lib']
verbosity = 3
cache_path  = 'forge-cache'
fs_permissions = [{ access = 'read', path = './'}]
optimizer = true
optimizer_runs = 200
gas_reports = ['*']
ignored_error_codes = ['func-mutability', 'unused-var', 'unused-param']
evm_version = 'cancun'
ffi = true
no_match_path = "test/prod/*"
seed = 42

[profile.default.zksync]
zksolc = "1.5.11"
compile = true
force_evmla = true

## Testing configs
[fuzz]
runs = 16
max_test_rejects = 100_000

[vmaark]

@dutterbutter @MexicanAce any chance there are some tricks to already speed things up, based on the example repo?

[dutterbutter]

We just triaged out the issue today: https://github.com/matter-labs/foundry-zksync/issues so hopefully we can provide more insights soon. We are aware of this issue for fuzzing / invariant testing but the example provided shouldnt cause a slow down so we need to look deeper.

Apart from including the following zksolc optimization referenced in config here:

[profile.default.zksync]
# Enable optimizer on zksolc (defaults to true)
optimizer = true 
# zksolc optimizer mode (0 | 1 | 2 | 3 | s | z)
optimizer_mode = '3'

Apologies for the inconvenience but we are actively investigating!

[vmaark]

thanks!

[dutterbutter]

Sir @vmaark we've just made a significant performance improvement to foundry-zksync based on this provided info. You can see the details here: matter-labs/foundry-zksync#962

Will be included in the 0.0.12 release next week. 🙏

[vmaark]

sounds great, can't wait to test it 👍

[vmaark]

hey @dutterbutter, just wanted to verify that indeed this release was a big improvement, as it went from 75s to 17s. Still far from the 100ms with regular foundry, so in real life scenarios with larger test suites this is still problematic.

[bxpana]

Great to hear that it made a big improvement. Is getting it up to par a blocker for you guys?

We are currently working on boojumos support so no further improvements are planned while we do that.

[vmaark]

it's not a blocker, just wanted to follow up. gl with boojums!