[docs][ctf] Update introduction for CTF

Bug: 339893257
Change-Id: I8129153984d0524c9d24cd6b65ba2092392bdb02
Reviewed-on: https://fuchsia-review.googlesource.com/c/fuchsia/+/1054553
Reviewed-by: Kyo Lee <kyol@google.com>
Commit-Queue: Christopher Johnson <crjohns@google.com>
Fuchsia-Auto-Submit: Christopher Johnson <crjohns@google.com>

Author: crjohns

docs/development/testing/ctf/_toc.yaml

@@ -8,8 +8,10 @@
 toc:
 - title: "Overview"
   path: /docs/development/testing/ctf/overview.md
-- title: "Introduction"
-  path:  /docs/development/testing/ctf/compatibility_testing.md
+- title: "Getting started"
+  section:
+  - title: "Introduction"
+    path:  /docs/development/testing/ctf/compatibility_testing.md
 - title: "Contributing"
   section:
   - title: "Contributing Tests"

docs/development/testing/ctf/compatibility_testing.md

@@ -23,34 +23,195 @@ CTF fundamentally consists of the following parts:
   FIDL methods and syscalls are tested for compatibility across
   versions.
 
-CTF was originally proposed as [RFC
-0015](/docs/contribute/governance/rfcs/0015_cts.md), and the project
-code is located at
-[//sdk/ctf](https://fuchsia.googlesource.com/fuchsia/+/refs/heads/main/sdk/ctf/).
+CTF was originally proposed as [RFC 0015][rfc-0015], and the project
+code is located at [`//sdk/ctf`][sdk-ctf-directory].
 
-## Background, motivation, and goals
+## Motivation
 
-The CTF exists to prevent breaking changes to the implemented Fuchsia
+The Fuchsia platform defines a surface area consisting of a number of
+FIDL protocols which are exposed in the Fuchsia SDK. Developers may
+download and use the Fuchsia SDK to write software targeting Fuchsia systems
+at a specific *API level*.
+
+As the Fuchsia platform surface area evolves over time, changes are
+represented through availability annotations on FIDL protocols:
+
+```fidl
+protocol Entry {
+  @available(added=12)
+  SetValue(struct { value string; });
+
+  @available(added=20)
+  GetTimestamp() -> (struct { timestamp int64; });
+
+  @available(added=13, removed=20)
+  Encrypt();
+};
+```
+
+In the above example, the protocol `Entry` has three methods:
+
+- `SetValue`, which was added at API level 12
+- `GetTimestamp`, which was added at API level 20
+- `Encrypt`, which was added at API level 13 but removed at API level
+20 when it was found to not be needed any longer.
+
+Components built using the Fuchsia SDK must declare the API level
+they target, affecting which definitions are visible. Consider the following
+scenarios:
+
+- A component is built targeting API level 19:
+  - `SetValue` and `Encrypt` are visible.
+  - `GetTimestamp` is absent, because it did not yet exist at API level 19.
+- A component is built targeting API level 21:
+  - `SetValue` and `GetTimestamp` are visible.
+  - `Encrypt` is absent, because it was removed at API level 20.
+
+Not all API levels are currently supported by the Fuchsia Platform.
+The canonical list of [supported API levels][version-history] is checked
+in to the `fuchsia.git` repository and is used to determine which API
+levels will be supported in releases of the Fuchsia SDK.
+
+Targeting a supported API level ensures that a client can connect
+only to protocols that are supported and guaranteed to be implemented,
+but it does not provide a guarantee that the behavior of the platform
+will remain consistent for pre-built components targeting that API
+level.
+
+Ensuring consistent behavior is important to support components
+built outside of the `fuchsia.git` repository, especially when those
+components are downloaded as pre-built binaries and assembled into
+product images.
+
+### Example: A compatibility problem
+
+Consider a simplified version of the `Entry` protocol above:
+
+```fidl
+protocol Entry {
+  @available(added=20)
+  GetTimestamp() -> (struct { timestamp int64; });
+};
+```
+
+The `GetTimestamp` method returns the timestamp of an `Entry` *in seconds*.
+
+We can test it as follows:
+
+```c++
+TEST(Timestamp) {
+  EntrySyncPtr entry = CreateEntryAtMidnightJan1stUTC();
+  ASSERT_EQ(entry.GetTimestamp(), 1704067200);
+}
+```
+
+Suppose that we decide we actually want nanosecond granularity. We
+will change the *implementation* of `GetTimestamp` to return
+nanoseconds, but note that the *FIDL definition* does not change.
+
+We would update our test as follows:
+
+```diff
+TEST(Timestamp) {
+  EntrySyncPtr entry = CreateEntryAtMidnightJan1stUTC();
+- ASSERT_EQ(entry.GetTimestamp(), 17040672000);
++ ASSERT_EQ(entry.GetTimestamp(), 17040672000000000);
+}
+```
+
+This will pass all checks in `fuchsia.git` and submit, but we have now
+introduced a **compatibility breakage to the Fuchsia Platform**.
+
+#### Failure timeline
+
+Let's look at a complete timeline that leads to a breakage.
+
+1. **Before the SDK change**: Suppose the FooWidget team wants to
+   implement their FooWidget for Fuchsia. They download the latest
+   Fuchsia SDK and build their component targeting API level 20. This
+   component uses the `Entry` protocol and calls the `GetTimestamp`
+   method. At this point in time, `GetTimestamp` returns seconds.
+
+   This works for them, so the FooWidget team will publish a Fuchsia
+   package called `foo-widget`. Upon inspection, users of this package
+   see it targets API level 20.
+
+1. **FooWidget is included in a Fuchsia product**: A Fuchsia product
+   is assembled that includes the `foo-widget` package and the Fuchsia
+   platform from an SDK. The `foo-widget` package was ingested as
+   a pre-built binary without source code access, but product assembly
+   sees that the package was built targeting API level 20 and that the
+   Fuchsia platform image supports API level 20.
+
+1. **The SDK change is rolled**: At this point, the change we defined
+   above lands in `fuchsia.git`, a new SDK is produced, and it is
+   released.
+
+1. **The product is assembled with the new SDK**: The product
+   repository rolls to the new SDK and assembles the product using
+   a new Fuchsia platform image combined with the existing pre-built
+   `foo-widget`.
+
+1. **FooWidget is broken on that product**: When the `foo-widget`
+   component calls `GetTimestamp()`, the results will be in nanoseconds
+   rather than seconds, which can have very strange results.
+   For instance, the FooWidget UI may start displaying times millions
+   of years in the future because it is treating nanoseconds as
+   seconds!
+
+   It is dangerous to change the behavior of platform functionality
+   when pre-existing software depends on the old behavior.
+
+### Catching compatibility problems
+
+CTF tests simulate the above scenario by taking advantage of Fuchsia's
+release branching process. If a CTF test fails, it means that
+pre-built components targeting an old Fuchsia platform implementation
+will fail when targeting the latest platform implementation. It works
+as follows:
+
+1. Each Fuchsia [milestone release][milestone-release] has an
+   associated release branch in `fuchsia.git`. This represents the
+   state of Fuchsia's platform implementation at the time of release.
+1. That platform implementation passes the set of tests on that
+   release branch.
+1. A set of artifacts are included in a *CTF Artifacts* bundle for
+   that release branch.
+1. When the release branch is created and/or modified, the CTF
+   Artifacts for that branch are compiled and uploaded to CIPD.
+1. The CTF Artifacts for each supported API level are downloaded as
+   pre-builts on the `main` branch and are *thawed* by turning them
+   into test packages.
+1. Those test packages are run against the latest Fuchsia platform
+   on the `main` branch, which will block CL submission if they fail.
+
+This exactly maps to the scenario in which a component is built
+against an old Fuchsia SDK and platform, and then is run as a
+pre-built against the current Fuchsia platform.
+
+## Background and history
+
+CTF exists to prevent breaking changes to the implemented Fuchsia
 API and ABI from landing in the platform.
 
 CTF originally was designed to provide confidence that an arbitrary
 Fuchsia system correctly implemented the platform surface area for
-a particular ABI revision. This would mean that components build
+a particular ABI revision. This would mean that components built
 for that revision would work on the system, and that the system is
 backwards compatible with that revision.
 
 The current purpose of CTF is to provide a mechanism for freezing
-and "thawing" test artifacts across release branches. For example,
-a test on the `f18` release branch is downloaded as a prebuilt at
-the top of the `main` branch. This test can then be executed against
-the Fuchsia platform as it exists today; a failure indicates that
-the Fuchsia platform broke compatibility with F18 in some way, which
-means downstream platform integrators may run in to unexpected
-failures if they try to run a pre-compiled binary on this new
-platform image.
+and "thawing" test artifacts across release branches.
 
 This mechanism can be applied to ensure the current Fuchsia platform
 supports the old API and ABI behavior for each version that is
 supposed to be supported. Additionally, the frozen tests may be
 applied to arbitrary Fuchsia images to achieve the original goal
-of CTF if desired.
+of CTF if desired.
+
+<!-- Reference links -->
+
+[rfc-0015]: /docs/contribute/governance/rfcs/0015_cts.md
+[sdk-ctf-directory]: https://fuchsia.googlesource.com/fuchsia/+/refs/heads/main/sdk/ctf/
+[version-history]: https://fuchsia.googlesource.com/fuchsia/+/HEAD/sdk/version_history.json
+[milestone-release]: /docs/concepts/versioning/release.md#milestone