Introduce community mirrors

mlugg · mlugg · commit d3c8c7477cf5 · 2025-06-30T15:35:59.000+01:00
The 'Download' page has a link to the new 'Community Mirrors' page which
describes how to use them. The mirror list is accessible at
'/download/community-mirrors.txt', which is generated from the Ziggy
source in 'assets/community-mirrors.ziggy'.

I have confirmed with the owners of these Zig mirrors that they are
happy for their mirrors to be used in this context.
diff --git a/MIRRORS.md b/MIRRORS.md
@@ -0,0 +1,95 @@
+> [!NOTE]
+> Are you trying to download Zig from a mirror? If so, you're in the wrong place! This documentation is for anyone who wants to *host* a community mirror.
+> For instructions on *using* the mirrors, check out the [Community Mirrors](https://ziglang.org/download/community-mirrors/) page on the Zig website.
+
+# Community Mirrors
+
+The Zig mirrors in this repository are all community-maintained and unaffiliated with the ZSF. Anyone is welcome to host their own mirror and add it to the
+list. Because Zig's tarballs are cryptographically signed, no mirror need be trusted. Of course, if a mirror is found to be malicious, it will be removed from
+the list.
+
+The following rules define how a mirror works and the requirements it is expected to fulfil.
+
+* A mirror provides a single base URL, which we will call `X`.
+* `X` **may** include a path component, but is not required to. For instance, `https://foo.bar/zig/` is okay,
+  as is `https://zig.baz.qux/`.
+* The mirror **must** support HTTPS with a valid signed certificate. `X` **must** start with `https://`.
+* The mirror **must** cache tarballs locally. For instance, it may not simply forward all requests to another mirror.
+* The mirror **may** routinely evict its local tarball caches based on any reasonable factor, such as age, access frequency, or the existence of newer versions. This does not affect whether the mirror may respond with 404 Not Found for requests to these files (see below).
+* The mirror **must** download tarballs from `https://ziglang.org/` or a valid mirror of it.
+  * Typically, it is best to download only from `https://ziglang.org/`. One possible exception is when a pre-release is requested which is more likely to be available from an alternative mirror.
+* When a mirror receives a GET request for `X/<filename>`, the behavior depends on `<filename>`.
+  * Parse the file name to extract the Zig version string. More details on this are [below](#parsing-versions).
+  * Determine whether this is a "normal" or "pre-release" [Semantic Version](https://semver.org/).
+  * If this is a "normal" version, respond with 200 OK and the file found at `https://ziglang.org/download/<version>/<filename>`.
+    * If the release version is "0.5.0" or older (according to Semantic Versioning), the mirror **may** respond with 404 Not Found, but is not required to.
+    * Otherwise, the mirror **must** return the tarball.
+    * These requirements apply equally to source tarballs (e.g. `zig-0.14.1.tar.xz`), bootstrap source tarballs (e.g. `zig-bootstrap-0.14.1.tar.xz`), and binary tarballs (e.g. `zig-x86_64-linux-0.14.1.tar.xz`).
+  * Otherwise (i.e. if this is a "pre-release" version), respond with 200 OK and the file found at `https://ziglang.org/builds/<filename>`.
+    * If the version is older than the latest "release" version (according to Semantic Versioning), the mirror **may** respond with 404 Not Found, but is not required to.
+    * Otherwise, the mirror **must** return the tarball.
+    * These requirements apply equally to source tarballs (e.g. `zig-0.15.0-dev.671+c907866d5.tar.xz`), bootstrap source tarballs (e.g. `zig-bootstrap-0.15.0-dev.671+c907866d5.tar.xz`), and binary tarballs (e.g. `zig-x86_64-linux-0.15.0-dev.671+c907866d5.tar.xz`).
+  * Invalid accesses, for instance to malformed filenames, **may** cause a 404 Not Found response.
+  * Accesses to file names which do not end with ".zip", ".tar.xz", ".zip.minisig", or ".tar.xz.minisig", **may** cause a 404 Not Found response.
+* Files provided by the mirror **must** be bit-for-bit identical to their `https://ziglang.org/` counterparts.
+* If a mirror is required to serve a tarball which is has not yet cached locally, it **must** immediately download it from its source at `https://ziglang.org`, and respond with that downloaded file.
+* The mirror **may** rate-limit accesses. If an access failed due to rate-limiting, the mirror **should** respond with 429 Too Many Requests.
+* The mirror **may** undergo maintenance, upgrades, and other scheduled downtime. If an access fails for this reason, where possible, the mirror **should** respond with 503 Unavailable. The mirror **should** try to minimize such downtime.
+* The mirror **may** undergo occasional unintended and unscheduled downtime. The mirror **must** go to all efforts to minimize such outages, and **must** resolve such outages within a reasonable time upon being notified of them.
+* The mirror **may** observe the query parameter named `source` to learn about the origin of its traffic.
+  * Clients are encouraged, but not required, to provide this query parameter to indicate what service triggered the request. For instance, the `mlugg/setup-zig` GitHub Action passes this query parameter as `?source=github-mlugg-setup-zig`.
+
+## Parsing Versions
+
+Mirrors are required to parse tarball file names to extract the Zig version from them. Unfortunately,
+this is complicated a little by two factors: the existence of "source" tarballs, and the fact that
+Zig's tarball naming schema changed with the 0.14.1 release.
+
+If you have access to Regular Expressions, a simple strategy is to search the tarball name for
+`/\d+\.\d+\.\d+(-dev\.\d+\+[0-9a-f]+)?/`. This is probably inefficient, but that is unlikely to
+matter in practice.
+
+An alternative strategy which is simpler but requires slightly more code is as follows:
+* Strip a supported extension (`.zip`, `.tar.xz`, `.zip.minisig`, `.tar.xz.minisig`) from the end of the file name.
+* Find the last occurrence of "-" in the file name. If that byte is followed by the string "dev", find the previous occurence of "-" instead.
+* The string after that "-" byte is the Zig version.
+
+It is strongly discouraged for checks to rely on specific architecture or OS names, since the set of targets for which tarballs are provided could be extended at any time.
+
+## Adding A Mirror
+
+Once a mirror complies with the requirements listed above, you can add it to the list of mirrors on ziglang.org by modifying the list in
+`assets/community-mirrors.ziggy` and opening a pull request with your changes. The diff should just add a single entry to the end of the list:
+
+```diff
+ [
+     {
+         .url = "https://a.com",
+         .username = "a",
+         .email = "a@a.com",
+     },
+     {
+         .url = "https://b.com/zig",
+         .username = "b",
+         .email = "b@b.com",
+     },
++    {
++        .url = "https://mymirror.net",
++        .username = "my-github-username",
++        .email = "my@email.com",
++    },
+ ]
+```
+
+Note that the Zig core team always reserves the right to exclude mirrors from this list for any (or no) reason.
+
+## Community-Written Solutions
+
+Some members of the Zig community have written software for hosting Zig mirrors, making it easier to comply with the requirements listed above. If you have an
+open-source repository which would fit here, feel free to open a pull request adding it below.
+
+> [!WARNING]
+> The Zig Software Foundation has not written or audited this code. As such, no guarantees can be made regarding its correctness, and it should **not** be
+> implicitly trusted. You are strongly encouraged to audit this software before using it, particularly if it will not be run in an isolated environment.
+
+* [hexops/wrench](https://github.com/hexops/wrench?tab=readme-ov-file#run-your-own-ziglangorgdownload-mirror)
diff --git a/assets/community-mirrors.ziggy b/assets/community-mirrors.ziggy
@@ -0,0 +1,17 @@
+[
+    {
+        .url = "https://pkg.machengine.org/zig",
+        .username = "emidoots",
+        .email = "emi@hexops.com",
+    },
+    {
+        .url = "https://zigmirror.hryx.net/zig",
+        .username = "hryx",
+        .email = "codroid@gmail.com",
+    },
+    {
+        .url = "https://zig.linus.dev/zig",
+        .username = "linusg",
+        .email = "mail@linusgroh.de",
+    },
+]
diff --git a/assets/css/style.css b/assets/css/style.css
@@ -467,7 +467,7 @@ blockquote {
 }
 
 
-code.zig {
+code {
   --light-yellow: #e5c07b;
   --dark-yellow: #d19a66;
   --blue: #61afef;
@@ -517,3 +517,49 @@ video {
   margin-top: 1em;
   display: inline-block;
 }
+
+code.python {
+  color: var(--cyan);
+}
+
+code.python .punctuation {
+  color: var(--cyan);
+}
+
+code.python .string {
+  color: var(--dark-yellow);
+}
+
+code.python .variable,
+code.python .parameter,
+code.python .exception,
+code.python .field {
+  color: var(--light-yellow);
+}
+
+code.python .keyword.function {
+  color: var(--light-red);
+}
+
+code.python .bracket {
+  color: var(--cyan);
+}
+
+code.python .function {
+  color: var(--blue);
+}
+
+code.python .builtin {
+  color: var(--magenta);
+}
+
+code.python .number {
+  color: var(--magenta);
+}
+
+code.python .operator,
+code.python .qualifier,
+code.python .keyword,
+code.python .attribute {
+  color: var(--light-red);
+}
diff --git a/build.zig.zon b/build.zig.zon
@@ -13,8 +13,8 @@
             .hash = "doctest-0.1.0-lY-6k0W2AQB2cw2vD3306oCziQrVnG013naZd8hMkuAQ",
         },
         .zine = .{
-            .url = "git+https://github.com/kristoff-it/zine#5408d3dff6107fc449a6d24d90f379b6fffdb4e6",
-            .hash = "zine-0.9.0-ou6nIBcqFgCkkL8OQnHRzEKboVcAvIp76bnf6n9qeBvU",
+            .url = "git+https://github.com/kristoff-it/zine#ff0b7b1222fe6529fc36e876a71a99379ad761c4",
+            .hash = "zine-0.9.0-ou6nIEJTFgDwdlQnB6NtTnZneWKgg-qoVPaXyU2PjWF2",
         },
     },
 }
diff --git a/content/en-US/download.smd b/content/en-US/download.smd
@@ -18,3 +18,6 @@ Files are signed with [minisign](https://jedisct1.github.io/minisign/) using thi
 ```
 RWSGOq2NVecA2UPNdBUZykf1CCb147pkmdtYxgb3Ti+JO/wCYvhbAb/U
 ```
+
+> # [Setting up an automation?]($block)
+> If you're automating the process of downloading Zig, you might want to learn about [Community Mirrors](/download/community-mirrors) to avoid downtime and help us save on bandwidth costs.
diff --git a/content/en-US/download/community-mirrors.smd b/content/en-US/download/community-mirrors.smd
@@ -0,0 +1,91 @@
+---
+.title = "Community Mirrors",
+.author = "",
+.date = @date("2024-08-07:00:00:00"),
+.layout = "download/community-mirrors.shtml",
+.alternatives = [{
+    .name = "list",
+    .layout = "download/community-mirrors-list.shtml",
+    .output = "/download/community-mirrors.txt",
+}],
+.custom = { "mobile_menu_title": "Mirrors" },
+---
+
+If you're setting something up which will automatically download Zig, like CI, you might be interested in using community mirrors instead of downloading from
+ziglang.org.
+
+The ziglang.org website does not offer any uptime or speed guarantees, meaning that your CI will sporadically fail or have slower runs if it hardcodes it as a
+download URL. In fact, configuring your CI to fetch from ziglang.org directly contributes to uptime and speed issues, because this site is [intentionally hosted
+on a simple one-computer configuration](/news/migrate-to-self-hosting). Instead, it is often a good idea to fetch Zig from one of many community-maintained
+mirrors. These mirrors are not officially endorsed by the Zig Software Foundation, but they can be used without security risks thanks to our signing of
+archives. While no individual mirror has an uptime or speed guarantee, configuring your automation to cycle through the list of available mirrors can
+effectively guarantee high uptime in practice.
+
+> # [Security Notice]($block)
+> Community mirrors are not officially trusted or endorsed by the Zig Software Foundation, and could in theory serve malicious binaries. If you are using them,
+> you **must** make sure to validate the minisign signature for every tarball you download against the ZSF's public key, available on [the download
+> page](/download).
+
+## GitHub Actions
+
+If you are setting up an automation using GitHub Actions, you may be interested in the
+[mlugg/setup-zig](https://github.com/marketplace/actions/setup-zig-compiler) Action (note that this is not an official ZSF project). Not only does it install a
+Zig version of your choice from a community mirror, but it also saves your Zig cache directory between workflow runs, allowing for faster rebuilds.
+
+## Using Mirrors
+
+The list of community mirrors is available in a newline-separated ASCII text file at https://ziglang.org/download/community-mirrors.txt. Tooling is recommended
+to fetch this list and try mirrors in a randomized order (to avoid putting excessive load on any one mirror, as this slows it down for everyone).
+
+Every Zig tarball is associated with a [minisign](https://jedisct1.github.io/minisign/) signature file, which can also be downloaded from mirrors. **When you
+download a tarball from a mirror, you must also download its associated signature and verify the tarball against it.** Failing to check the signature could
+theoretically leave you vulnerable to malicious mirrors hosting modified tarballs.
+
+Put simply, the recommended strategy is approximately this pseudocode:
+
+```python
+pubkey = "(copy this from https://ziglang.org/download)"
+tarball_name = "zig-x86_64-linux-0.14.1.tar.xz"
+# To improve uptime, optionally cache this GET:
+mirrors = http_get("https://ziglang.org/download/community-mirrors.txt")
+# ASCII-encoded, one mirror per line, newlines are LF, there is a trailing newline.
+shuffled = shuffle_lines(mirrors)
+for mirror_url in shuffled:
+    tarball = http_get(f"{mirror_url}/{tarball_name}?source=my_automation_name")
+    if success:
+        # NEVER SKIP THIS STEP. The signature must be verified before the tarball is deemed safe.
+        signature = http_get(f"{mirror_url}/{tarball_name}.minisig?source=my_automation_name")
+        if success and minisign_verify(tarball, signature, pubkey):
+            print("Successfully fetched Zig 0.14.1!")
+```
+
+Because ziglang.org does not have guaranteed uptime, the `community-mirrors.txt` file may at times become inaccessible. For this reason, you may wish to
+consider caching its contents to prevent disruption in the event that ziglang.org encounters downtime. The recommended refetch interval is approximately
+once per day. At this point in time, mirrors may be added or removed on a monthly basis as the ecosystem evolves, so periodic re-fetching is essential.
+
+Written more precisely, here is the key information and recommend workflow for downloading Zig tarballs:
+
+* The mirror list file is available at https://ziglang.org/download/community-mirrors.txt.
+  * Because ziglang.org does not guarantee uptime, it may be desirable to cache this file.
+* The mirror list file contains ASCII-encoded mirror URLs, separated with newline characters (ASCII LF 0x20). There is a trailing newline. There is no other whitespace. There are no blank lines.
+* Mirrors are required to support HTTPS. Every line in the mirror list file begins with "https://".
+* Mirrors cannot guarantee uptime, so if one fails to serve you a tarball, you should try another. Ideally, shuffle the list, and try each mirror in turn.
+  * Usually, the first one will work. If no mirror works, you may choose to try `ziglang.org` as a final fallback.
+* To download a tarball from a mirror, perform a GET request to "mirror/filename", where "mirror" is the mirror URL, and "filename" is the basename of the corresponding tarball on ziglang.org (e.g. `zig-x86_64-linux-0.14.1.tar.xz`).
+  * You are highly encouraged to include in your request a query parameter named `source` containing a string indicating what is making this request. For instance, the `mlugg/setup-zig` GitHub Action passes it as `?source=github-mlugg-setup-zig`.
+  * Source tarballs, bootstrap tarballs, and binary tarballs are available from all listed mirrors, as well as minisign signatures for all such files.
+  * Binary tarballs for recent Zig versions are of the form `zig-x86_64-linux-0.14.1.tar.xz`.
+  * If a mirror responds with a HTTP status code other than 200 OK:
+    * `503 Unavilable` may indicate scheduled downtime.
+    * `429 Too Many Requests` may indicate intentional rate-limiting.
+    * `404 Not Found` is a permitted response when requesting Zig releses 0.5.0 or earlier, or Zig development builds earlier than the current latest release.
+    * Otherwise, feel free to [open an issue](https://github.com/ziglang/www.ziglang.org/issues/new) to inform us of the problem.
+* The Zig Software Foundation can never guarantee the security of any mirror, so every time a tarball is downloaded, it is **essential** to also download the minisign signature (suffix the filename with ".minisig") and verify it against the ZSF's public key (which you should copy from the ziglang.org/download page). **Never skip this step.**
+  * If a mirror responds with `200 OK` but signature validation fails on the returned tarball, feel free to [open an
+    issue](https://github.com/ziglang/www.ziglang.org/issues/new) to inform us of the problem.
+
+## Hosting a Mirror
+
+If you are interested in hosting a mirror, please consult the [documentation in the www.ziglang.org
+repository](https://github.com/ziglang/www.ziglang.org/blob/main/MIRRORS.md). Thank you for helping
+to improve and decentralize the Zig ecosystem!
diff --git a/layouts/download.shtml b/layouts/download.shtml
@@ -6,6 +6,22 @@
     rel="stylesheet"
     href="$site.asset('css/home.css').link()"
   >
+  <style>
+.block {
+  border: 2px solid grey;
+}
+.block h1 {
+  margin: 0;
+  font-size: 1.1em;
+  background: #f7a41d;
+  padding: 0.4em 0.7em;
+  color: black;
+}
+.block p {
+  margin: 0;
+  padding: 0.5em 0 0.5em 1.0em;
+}
+  </style>
 </head>
 <div id="content">
   <div class="container">
@@ -138,4 +154,4 @@
       </div>
     </ctx>
   </div>
-</div>
+</div>
diff --git a/layouts/download/community-mirrors-list.shtml b/layouts/download/community-mirrors-list.shtml
@@ -0,0 +1,6 @@
+<ctx :loop="$site.asset('community-mirrors.ziggy').ziggy()"><ctx :text="$loop.it.get('url')"></ctx>
+</ctx><ctx :if="$page.draft">
+This file intentionally does not have a trailing newline in order to prevent a double-newline after
+the final mirror. If you modify this file, carefully inspect `community-mirrors.txt` to check you
+haven't made an unintentional change to the output.
+</ctx>
diff --git a/layouts/download/community-mirrors.shtml b/layouts/download/community-mirrors.shtml
@@ -0,0 +1,39 @@
+<extend template="locale.shtml">
+<title id="title" :text="$page.title"></title>
+<head id="head">
+  <link
+  type="text/css"
+  rel="stylesheet"
+  href="$site.asset('css/home.css').link()"
+>
+  <style>
+.block {
+  border: 2px solid grey;
+}
+.block h1 {
+  margin: 0;
+  font-size: 1.1em;
+  background: #f7a41d;
+  padding: 0.4em 0.7em;
+  color: black;
+}
+.block p {
+  margin: 0;
+  padding: 0.5em 0 0.5em 1.0em;
+}
+  </style>
+</head>
+<div id="content">
+  <div class="container">
+    <ctx back="$page.parentSection()">
+      <a id="back" href="$ctx.back.link()">
+        <ctx :text="$i18n.get('back')"></ctx>
+        <b :text="$ctx.back.title"></b>
+      </a>
+    </ctx>
+    <h1 class="title" :text="$page.title"></h1>
+    <ctx :if="$page.custom.get?('toc')" :html="$page.toc()">
+    </ctx>
+    <ctx :html="$page.content()"></ctx>
+  </div>
+</div>
