[ZK Stack] - Debugging Local Hyperchain Quickstart

[ffarall]

Module Affected

• System contracts (era-system-contracts)
• Smart contracts (era-contracts)
• zkEVM (era-zk_evm)
• Server (zksync-era)
• Prover (era-boojum)
• Fee withdrawer (era-fee-withdrawer)
• Block Explorer (clock-explorer)
• Portal - Wallet + Bridge (dapp-portal)
• Other (please specify below)

Rust Version

1.74.0-nightly

Operating System

• Ubuntu
• MacOS
• Windows
• Other (Please specify below)

Issue Description

This is not really a specific issue, but a quick list of the issues encountered when following the steps in the Getting Started Guide for running a hyperchain locally with the ZK Stack.

The tutorial was followed on three different machines:

1. MacBook Pro M3 Max.
2. MacBook Pro M1 Max.
3. Ubuntu 22.04.3 LTS with AMD Ryzen 9 7950X 16-Core Processor and 64GB RAM.

The following issues were encountered:

1. Permissions for new folder created are set for root not the current user. When running on the Remote Server, connected through SSH, the user is not root, but one with sudo access. In the initial setup, the volumes folder is created and root is the owner, therefore the script fails when trying to create files in it. Can be solved changing ownership of the volumes folder, but requires re-running the script.
2. Hardcoded ports for PostgreSQL and Geth. Not really an issue, but it would be nice to be able to pass the ports as optional arguments in the command line. Coincidentally I was using the 5432 port in the Remote Server, as it is the default for PostgreSQL, so I run into an error when trying to run the containers. Furthermore, this prevents running multiple hyperchains in the same machine.
3. Missing instructions on how to install solc in Ubuntu. Although easy to find online, this prerequisites setup includes only the instructions for MacOS.
4. TypeScript not compiling in Remote Server. This was due to the yarn.lock file not being locked when installing the dependencies. The zk and zk stack init scripts both run yarn at some point, but without the --frozen-lockfile, meaning each machine would modify the yarn.lock file and the dependencies would be slightly different. In the Mac machine, this didn't cause any issues, but in the Ubuntu it made the TypeScript compiler fail. The solution was to run yarn install --frozen-lockfile in the zk and zk stack init scripts, which is added in this commit. Below, the compile error:

   scripts/utils.ts:201:93 - error TS2339: Property ‘zksolc’ does not exist on type ‘HardhatConfig’.
   
   201   const url = getZksolcUrl(“https://github.com/matter-labs/zksolc-prerelease”, hre. config.zksolc.version);
   

5. After setting up a new hyperchain for the first time with zk stack init (let’s call it my-hyperchain) if you execute zk clean —-all it deletes my-hyperchain.env (among others). However, as soon as you run another zk command, the file is regenerated with the old content. For example, running zk down, regenerates the my-hyperchain.env that was just deleted. This is not really an issue, but it is a bit confusing/annoying. It can be solved by also having zk clean delete the .current file, as done in this commit.
6. The command for running the server has more components than the one executed at the end of zk stack init, which only has http_api, eth, state_keeper, housekeeper, tree. Including these other components make it fail:

   zk server --components “http_api,tree_lightweight,eth,data_fetcher,state_keeper,housekeeper”
   

7. When executing the server, which is Rust code running in the local machine, the script tries to use sqlx to access the PostgreSQL database (which was running in a docker container named postgres), using the URL DATABASE_URL=postgres://postgres:notsecurepassword@postgres:5432/zksync_local. That is the environment variable that is autogenerated in hyperchain_wizards.ts, in this line:

   wrapEnvModify('DATABASE_URL', 'postgres://postgres:notsecurepassword@postgres:5432/zksync_local');

   The problem is that the local machine cannot resolve the path postgres:5432 by name, as it does not have the IP of the postgres container. That is why it could not find the database. Luckily, the container is forwarding the ports, so changing hyperchain_wizards.ts to generate the env variable as DATABASE_URL=postgres://postgres:notsecurepassword@postgres:5432/zksync_local solves the problem.
   There is a similar issue when the server tries to find geth:8545, the docker container running the Ethereum node. Both issues are solved in this commit.
   Interestingly enough, manually modifying the DATABASE_URL env variable after executing zk stack init would not work, as running zk server again would still use the autogenerated value.
8. Running zk stack init after doing zk down for stopping the docker containers doesn't boot the same containers again, but other ones. Not really an issue, but it would be nice to not have to go through zk stack init again.

Expected Behavior

Going through the steps of the Quickstart, i.e. running zk, then zk stack init and configuring all the prompted choices should end with a running hyperchain server.

Current Behavior

Aside from some minor issues in Ubuntu, the main issue is that the zk server running in the local machine cannot resolve the addresses of the postgres and geth containers launched, just by name, so after going through the configuration process of zk stack init, running the server generates a Rust panic.

Repository Link (if applicable)

https://github.com/ffarall/zksync-era

Additional Details

No response

Prior Research

• Yes, I searched for similar issues and did not find any matching ones.

[zk-Lumi]

Amazing thanks @ffarall 🔥

We will be looking into all of these.

[githubdoramon]

@ffarall we are working on rewriteing the zk stack tools to make it way more robust and modular. Please wait a little bit longer and all your problems will be solved :)

[bxpana]

Keep an eye on https://twitter.com/zkSyncDevs and https://github.com/zkSync-Community-Hub/zksync-developers/discussions/categories/announcements for updates on the ZK Stack tool updates