[ZK Stack] - Errors with zkstack ecosystem init on Sepolia

[pashakhromov]

Team or Project

No response

Module Affected

• System contracts (era-system-contracts)
• Smart contracts (era-contracts)
• zkEVM (era-zk_evm)
• Server (zksync-era)
• Prover (era-boojum)
• Fee withdrawer (era-fee-withdrawer)
• Block Explorer (clock-explorer)
• Portal - Wallet + Bridge (dapp-portal)
• Other (please specify below)

Rust Version

1.82.0

Operating System

• Ubuntu
• MacOS
• Windows
• Other (Please specify below)

Issue Description

Hello! I'm trying to deploy on Sepolia with zkstack ecosystem init --l1-rpc-url=[RPC_URL] but getting numerous errors depending on RPC chosen. Tried all on RPCs from https://sepolia.dev/ and failed. Then created account on https://www.infura.io/ and tried with my key https://sepolia.infura.io/v3/[API_KEY] and also failed.

Expected Behavior

Successful deployment

Current Behavior

Just going to copy a couple of examples here

Example 1

zkstack ecosystem init --l1-rpc-url=https://rpc.sepolia.org/ 

┌   ZK Stack CLI 
│
◇  Do you want to deploy some test ERC20s?
│  Yes 
│
◇  Do you want to deploy ecosystem contracts? (Not needed if you already have an existing one)
│  Yes 
│
◇  Do you want to setup observability? (Grafana)
│  Yes 
│
●  Initializing ecosystem
│  
●  Era observability already setup
│  
◇  Installing and building dependencies... done in 8.008185 secs
│
◒  Deploying ecosystem contracts...                                                                                                                                             [crates/zkstack/src/commands/ecosystem/common.rs:31:5] config.get_default_configs_path() = "/Users/pasha/test_sepolia_ecosystem/zksync-era/etc/env/file_based"
■  Deserialization Error: expected value at line 1 column 1. Response: error code: 502
│  
└  Failed to run command

Example 2: error despite moving > 5 Sepolia ETH to the address

zkstack ecosystem init --l1-rpc-url=https://rpc2.sepolia.org  

┌   ZK Stack CLI 
│
◇  Do you want to deploy some test ERC20s?
│  Yes 
│
◇  Do you want to deploy ecosystem contracts? (Not needed if you already have an existing one)
│  Yes 
│
◇  Do you want to setup observability? (Grafana)
│  Yes 
│
●  Initializing ecosystem
│  
●  Era observability already setup
│  
◇  Installing and building dependencies... done in 8.114509083 secs
│
◒  Deploying ecosystem contracts...                                                                                                                                             [crates/zkstack/src/commands/ecosystem/common.rs:31:5] config.get_default_configs_path() = "/Users/pasha/test_sepolia_ecosystem/zksync-era/etc/env/file_based"
◒  Deploying ecosystem contracts...                                                                                                                                             ◇  Address 0x9a935149d049497353564a88d707f6df91be3c32 doesn't have enough money to deploy contracts only 0.000000000000000000 ETH but expected: 5.000000000000000000 ETH do you want to try again?
│  No 
│
Command: forge script deploy-scripts/DeployL1.s.sol --legacy --ffi --rpc-url=https://rpc2.sepolia.org --private-key=[KEY]--broadcast
◓  Deploying ecosystem contracts...                                                                                                                                             │
■  Command failed to run
│  
■    Status:
│      exit status: 1
│    Stdout:
│      No files changed, compilation skipped
│      Script ran successfully.
│      
│      == Logs ==
│      Deploying L1 contracts
│      Using deterministic Create2Factory address: 0x4e59b44847b379578588920cA78FbF26c0B4956C
│      BytecodesSupplier deployed at: 0xEa22Ae6CFE12e86Fa52Fa5852b181311D1B09ac2
│      Verifier deployed at: 0xF893582517C87E5480Ba85e9845f625cC0C258c6
│      DefaultUpgrade deployed at: 0x308e1e7c743315DC0b5E81196a49Fa6C3CA43009
│      GenesisUpgrade deployed at: 0xb40719CB0772038EC5B7053fA72102C98879413F
│      L1RollupDAValidator deployed at: 0xe8BF466f3A6F0Cc8b82b10e523400Ca2D9583cDB
│      L1NoDAValidiumDAValidator deployed at: 0x476CA4b3f6F2D3f12545f122eE2f49f00d05CdcF
│      AvailL1DAValidator deployed at: 0x899ba9075765b14B694e6CF1C2A3CeFe4C97d0AB
│      ValidatorTimelock deployed at: 0xf842bf3CF8a13e8a7A81011B1733E239c64b7E5b
│      Governance deployed at: 0x876dDa15d632b7384F9DE447c62F48f3F7F69E54
│      ChainAdminOwnable deployed at: 0x0000000000000000000000000000000000000000
│      Transparent Proxy Admin deployed at: 0xdBB023e64Ab569751Fc74194DF28d4dC40fBdfd7
│      Bridgehub Implementation deployed at: 0xDa3C76974e57F700644ca7AB52ED9D65E4A058EE
│      Bridgehub Proxy deployed at: 0xc86a952056C9e890e944BD3C0f0fbEf74AB76Be8
│      MessageRoot Implementation deployed at: 0xdbdB59BCF0075Af86a5fb0f9838269e166A44Bf6
│      Message Root Proxy deployed at: 0x044BC07a7baDd78986b49E23b4F321AA87234e4c
│      L1NullifierImplementation deployed at: 0x7DB9f45Aa68F392804E3E73A3BDE0F9F8952E4f1
│      L1NullifierProxy deployed at: 0xce2403c8B93b43CB2Fed8F7023cE388c26B0156e
│      SharedBridgeImplementation deployed at: 0x6ce785BB05A0a63b85531f68a76CbA50213FF3FB
│      SharedBridgeProxy deployed at: 0xa766891861C1028b3De82A9B9B410DA87A2bf0e2
│      BridgedStandardERC20Implementation deployed at: 0x32872f0010E1097c1a5917CA1D33F507712048Ba
│      BridgedTokenBeacon deployed at: 0x101f6Aaf8530f1870f0B45f8A4dF28Eb365aD127
│      L1NativeTokenVaultImplementation deployed at: 0xF7034A98D1C7Af0508faEa57B4bbc8d1E3b2F0cF
│      L1NativeTokenVaultProxy deployed at: 0x96dAf00E68E0Cd5462C2de10860825365D24Aa2E
│      Erc20BridgeImplementation deployed at: 0xaa1BB301F913C5398D416D98aF8f125D13019096
│      Erc20BridgeProxy deployed at: 0x5C953A512D0399bc8eA7eC0Ddb6F3cE280D9737C
│      SharedBridge updated with ERC20Bridge address
│      CTM Deployment Tracker Implementation deployed at: 0x4a275CA91d0f1fec298aBCf3a95ed756603649B0
│      CTM Deployment Tracker Proxy deployed at: 0x9A0dA8126eD7091FCB8468ab169eFe0bB0B1e658
│      SharedBridge registered
│      BlobVersionedHashRetriever deployed at: 0x7758883e025810C108FE28AFFdB1FD5F7c0764a5
│      ExecutorFacet deployed at: 0x94E1f4AFeCA4D9777bA79D38A8A87f5cf2699B81
│      AdminFacet deployed at: 0x7B9e2fb1b005a23977fA1fB68CAdD4AbEe14C1c6
│      MailboxFacet deployed at: 0xDBeCd5D14dc6926DE68034F61DF7aC2Af77E6391
│      GettersFacet deployed at: 0x3F74c6D00bcf18993AB1AB59CD09813a24C11B04
│      DiamondInit deployed at: 0xd52a21729387e7d20f2aB665D522805F38950E77
│      ChainTypeManagerImplementation deployed at: 0xC222528c4CA7c0931d201339cAA46fe0d35Dde0b
│      ChainTypeManagerProxy deployed at: 0x29774003dB8ED52842D7Cd232C0268C175aDf043
│      ChainTypeManager registered
│      CTM DT whitelisted
│      CTM registered in CTMDeploymentTracker
│      CTM in router 1 0xc86a952056C9e890e944BD3C0f0fbEf74AB76Be8 0x29774003dB8ED52842D7Cd232C0268C175aDf043
│      ChainTypeManager set in ValidatorTimelock
│      Owners updated
│      
│      ## Setting up 1 EVM.
│      
│      ==========================
│      
│      Chain 11155111
│      
│      Estimated gas price: 12.707925766 gwei
│      
│      Estimated total gas used for script: 74852230
│      
│      Estimated amount required: 0.95121658225955818 ETH
│      
│      ==========================
│      
│      Transactions saved to:
│      /Users/pasha/test_sepolia_ecosystem/zksync-era/contracts/l1-contracts/broadcast/DeployL1.s.sol/11155111/run-latest.json
│      
│      Sensitive values saved to:
│      /Users/pasha/test_sepolia_ecosystem/zksync-era/contracts/l1-contracts/cache-forge/DeployL1.s.sol/11155111/run-latest.json
│      
│      
│  
│    Stderr:
│      Error: Failed to send transaction
│      
│      Context:
│      - server returned an error response: error code -32000: insufficient funds for gas * price + value: balance 0, tx cost
│      5393256403016166, overshot 5393256403016166
│      
│  
■  Command failed to run: forge script deploy-scripts/DeployL1.s.sol --legacy --ffi --rpc-url=https://rpc2.sepolia.org --private-key=[KEY]--broadcast
│  Error: Failed to send transaction
│  
│  Context:
│  - server returned an error response: error code -32000: insufficient funds for gas * price + value: balance 0, tx cost 5393256403016166, overshot 5393256403016166
│  
│  
▲    0: Command failed to run: forge script deploy-scripts/DeployL1.s.sol --legacy --ffi --rpc-url=https://rpc2.sepolia.org --private-key=[KEY]--broadcast
│  
└  Failed to run command

Example 3: too many requests error

zkstack ecosystem init --l1-rpc-url=https://sepolia.infura.io/v3/[API_KEY]

For which I'm getting too many requests error

No files changed, compilation skipped
│      2025-01-25T03:05:48.162075Z ERROR alloy_provider::blocks: failed to fetch block
│      number=7565144 err=Max retries exceeded HTTP error 429 with body:
│      {"jsonrpc":"2.0","id":38,"error":{"code":429,"event":-33200,"message":"Too Many Requests","details":"You have surpassed

│ your allowed throughput limit. Reduce the amount of requests per second or upgrade for more capacity."}}

Repository Link (if applicable)

No response

Additional Details

No response

Prior Research

• Yes, I searched for similar issues and did not find any matching ones.

[bogatyy]

Try Alchemy free tier as well?
https://www.alchemy.com/ethereum

[dutterbutter]

It is not recommended to use public rpcs given the amount of requests so your intuition to use infura and the suggestion above for alchemy as well is correct.

To resolve the error of insufficient funds, I would check the balance of the addresses in the wallet.yaml files. Recall there is more than one address that will need to be funded (e.g. deployer, governor, etc). To simplify things, and just to test you can replace all required addresses with the existing funded one in the wallet.yaml files.

curl --request POST \
  --url <YOUR-RPC-URL> \
  --header 'Content-Type: application/json' \
  --data '{
      "jsonrpc": "2.0",
      "id": 1,
      "method": "eth_getBalance",
      "params": [
        "<REPLACE_WITH_ADDRESS>",
        "latest"
      ]
    }'

[pashakhromov]

Thank you for taking a look @bogatyy and @dutterbutter!

Alchemy so far doesn't complain about too many requests and I think the single address trick for testing is neat.

Now when running init I'm getting "future transaction tries to replace pending" error. This is with and without --resume flag and also waiting for 6+ hours and creating new wallets as per #713

zkstack ecosystem init --l1-rpc-url=https://eth-sepolia.g.alchemy.com/v2/[API-KEY]          

┌   ZK Stack CLI 
│
◇  Do you want to deploy some test ERC20s?
│  Yes 
│
◇  Do you want to deploy ecosystem contracts? (Not needed if you already have an existing one)
│  Yes 
│
◇  Do you want to setup observability? (Grafana)
│  Yes 
│
●  Initializing ecosystem
│  
●  Era observability already setup
│  
◇  Installing and building dependencies... done in 8.558410834 secs
│
◒  Deploying ecosystem contracts...                                                                                                                                             [crates/zkstack/src/commands/ecosystem/common.rs:31:5] config.get_default_configs_path() = "/Users/pasha/test_sepolia_ecosystem/zksync-era/etc/env/file_based"
◒  Deploying ecosystem contracts...                                                                                                                                             Command: forge script deploy-scripts/DeployL1.s.sol --legacy --ffi --rpc-url=https://eth-sepolia.g.alchemy.com/v2/y3zza9KZyFEX4kRgt4_Qcdp6B-KGdsBY --private-key=[KEY]--broadcast
◐  Deploying ecosystem contracts...                                                                                                                                             │
■  Command failed to run
│  
■    Status:
│      exit status: 1
│    Stdout:
│      No files changed, compilation skipped
│      Script ran successfully.
│      
│      == Logs ==
│      Deploying L1 contracts
│      Using deterministic Create2Factory address: 0x4e59b44847b379578588920cA78FbF26c0B4956C
│      BytecodesSupplier deployed at: 0xc773CF06e282DfD825534ab1be0790B0Bd3b61Da
│      Verifier deployed at: 0xFeAad59138a6Cd8Dd4A21667C02b320F49842C9B
│      DefaultUpgrade deployed at: 0xb8419CD3ae87C3087F7A2D64853837C04F4A047b
│      GenesisUpgrade deployed at: 0xf25AB1ff671C479B671b85D61Be0F88E170D97C5
│      L1RollupDAValidator deployed at: 0xec5F4b3f43CF580f60dacA509289F0aAd1aD8f3F
│      L1NoDAValidiumDAValidator deployed at: 0x46d85410e7b67C508956346DABb4E56a885Eba2D
│      AvailL1DAValidator deployed at: 0xe02bdb887de3ea46715427134bb8271EE5336888
│      ValidatorTimelock deployed at: 0x3e4e86d119FC1cd9E92f357E9cdBeDF00750a816
│      Governance deployed at: 0xe12FCbAEc56760C0621C67D464CF209D55f08d34
│      ChainAdminOwnable deployed at: 0x0000000000000000000000000000000000000000
│      Transparent Proxy Admin deployed at: 0xBBf796ce779Bf908afA0980eaB97b1Ae6a657E40
│      Bridgehub Implementation deployed at: 0x342a551f6F625F8D5e5B675a40B83e3996F6fcaB
│      Bridgehub Proxy deployed at: 0x105A1504B2614cB905195beE37c2D34C3AfDfD83
│      MessageRoot Implementation deployed at: 0xEcE786C7b6c22B54c55C76f4AB4b770b74730276
│      Message Root Proxy deployed at: 0x72CC9C48188e62Ff9b5142e546c0880076673371
│      L1NullifierImplementation deployed at: 0x47A484269344cf0034e2b27290d3B6Bf8787D910
│      L1NullifierProxy deployed at: 0x956F1D600F43358069AF464432F2986E13C78440
│      SharedBridgeImplementation deployed at: 0x1b1093C1C1D909B08862a96660375012F6CC27b0
│      SharedBridgeProxy deployed at: 0xd2ae77be5C2a7F569AF8d5E12c7ED2f17Ef6e520
│      BridgedStandardERC20Implementation deployed at: 0x6c1Cd0aF2EEd67e34E65D644d44FCDc8014382C6
│      BridgedTokenBeacon deployed at: 0x5937C7606Ba65b82e0FF96e1255dCE20a55a605b
│      L1NativeTokenVaultImplementation deployed at: 0xD5C2e855D8e2ff0dD78775E13e91eC940167E4D4
│      L1NativeTokenVaultProxy deployed at: 0x55B76857aB9f750C055Dc44B30D66876e6e01b3C
│      Erc20BridgeImplementation deployed at: 0x59bcFE6efC45907841aA9cA14f0bDfc4149EEF4C
│      Erc20BridgeProxy deployed at: 0x886AA30D1a22952B9aF69E0137f6D2627C42fD14
│      SharedBridge updated with ERC20Bridge address
│      CTM Deployment Tracker Implementation deployed at: 0xFfEdb0d1de94Ac6aEd0fD21eFBe620A356215c17
│      CTM Deployment Tracker Proxy deployed at: 0x9d658d4284Cb0256C8DB0A93A46cBD4707f4b909
│      SharedBridge registered
│      BlobVersionedHashRetriever deployed at: 0x6731d91f07C02DB57269940757D891CA161336dA
│      ExecutorFacet deployed at: 0x27D0566BF5a0188412e4445f2DFA4B0D21439084
│      AdminFacet deployed at: 0xf4A808d24F6100e4C5bdff1a2aA79Db437957A2F
│      MailboxFacet deployed at: 0xebeDb3C88184BDC20Bc7C76Bbc424276C43062f3
│      GettersFacet deployed at: 0x2E9B40f30285c47BA1DA7fF5f73c171D193bfCfe
│      DiamondInit deployed at: 0x89b01B38532fF9D8BeA0CC79abe45Ed90dBC9f00
│      ChainTypeManagerImplementation deployed at: 0xbc5Fc1FCb88607BEE38c645c8E4726878a3898EF
│      ChainTypeManagerProxy deployed at: 0x732bf24961c81cBf6A380db0819656EFf8d26419
│      ChainTypeManager registered
│      CTM DT whitelisted
│      CTM registered in CTMDeploymentTracker
│      CTM in router 1 0x105A1504B2614cB905195beE37c2D34C3AfDfD83 0x732bf24961c81cBf6A380db0819656EFf8d26419
│      ChainTypeManager set in ValidatorTimelock
│      Owners updated
│      
│      ## Setting up 1 EVM.
│      
│      ==========================
│      
│      Chain 11155111
│      
│      Estimated gas price: 19.144026363 gwei
│      
│      Estimated total gas used for script: 50537721
│      
│      Estimated amount required: 0.967495463149938723 ETH
│      
│      ==========================
│      
│      Transactions saved to:
│      /Users/pasha/test_sepolia_ecosystem/zksync-era/contracts/l1-contracts/broadcast/DeployL1.s.sol/11155111/run-latest.json
│      
│      Sensitive values saved to:
│      /Users/pasha/test_sepolia_ecosystem/zksync-era/contracts/l1-contracts/cache-forge/DeployL1.s.sol/11155111/run-latest.json
│      
│      
│  
│    Stderr:
│      Error: Failed to send transaction
│      
│      Context:
│      - server returned an error response: error code -32000: future transaction tries to replace pending
│      
│  
■  Command failed to run: forge script deploy-scripts/DeployL1.s.sol --legacy --ffi --rpc-url=https://eth-sepolia.g.alchemy.com/v2/y3zza9KZyFEX4kRgt4_Qcdp6B-KGdsBY --private-key=[KEY] --broadcast
│  Error: Failed to send transaction
│  
│  Context:
│  - server returned an error response: error code -32000: future transaction tries to replace pending
│  
│  
▲    0: Command failed to run: forge script deploy-scripts/DeployL1.s.sol --legacy --ffi --rpc-url=https://eth-sepolia.g.alchemy.com/v2/y3zza9KZyFEX4kRgt4_Qcdp6B-KGdsBY --private-key=[KEY] --broadcast
│  
└  Failed to run command

[pashakhromov]

Hey @dutterbutter no worries at all and thanks so much for debugging with me!

I tried

1. Creating new app and new API key in infura under current account and recreating an ecosystem from scratch. That lead to "too many requests" error
2. Creating new account on infura and recreating an ecosystem from scratch. That lead to "too many requests" error again
3. Creating new app and new API key in alchemy under current account and recreating an ecosystem from scratch. That lead to "future transaction tries to replace pending" error

Since I can't easily create a new alchemy account (need to go through phone number verification) and can't easily delete my current alchemy account (looks like only support can do that), are there alternative ways of dealing with the error other than paying for alchemy/infura accounts?

[dutterbutter]

You don't need to change the rpc but rather the funded ethereum address in the wallets.yaml files.

[sachanayush47]

Regarding "future transaction tries to replace pending", check this thread. You cannot deploy with infura free RPC. You need to upgrade your plan. If you plan to use other rpc providers like Alchemy or chainlink, most probably you will encounter this error "initcode too large".
Also, fund chain's and ecosystem's deployer and governor address. Governor address should also be funded with base token.

[pashakhromov]

Hi @dutterbutter and @sachanayush47, thanks for taking a look.

I did change RPCs along with the wallet addresses and did fund the necessary accounts.

After some experimentation what I found to be working reliably was to get a paid account (infura in my case) and if running first init was not successful to continue running the init with --resume flag after encountering "Failure on receiving a receipt for ... error sending request for url ..." errors