Suzuka DA Migrations Format

[l-monninger]

Summary

In order to allow for safe upgrades, data model changes, and DA migrations, it has been proposed that we provide a data format which describes the canonical state of the DA over time. This would be hardfork migration approach to upgrades. In a simple flexible form, this would be a list of tuples of (a) block height range tuples indexing (b) the following fields:

• DA namespace/identifier
• DA transaction format version
• DA block format version
• Transaction ordering rule version
• Full node version (generally)
• Movement Aptos version (framework and node)
• Aptos state model version (more specifically)

[mzabaluev]

block height range

I think the scalar height key by which these states become normative is more maintainable. We will not have gaps in coverage, so the next entry's height implicitly defines the last height for which the preceding entry applies.

the following fields:

Should all of these fields be enforced? I'd like to avoid overengineering here.

• DA transaction format version
• DA block format version
• Transaction ordering rule version

Makes sense.

• Full node version (generally)
• Movement Aptos version (framework and node)

For these, we would need something like semver minimal version spec. Also introspection into the version information.
What is "Movement Aptos node version", BTW? Should it reflect upstream feature parity?

Aptos state model version (more specifically)

Where is this defined?

[l-monninger]

BCS encoding

What is the emphasis on the encoding here?

[l-monninger]

I would prefer not to get carried away with versioning, besides making the current version a top-level parameter to enable breaking changes in the future.

So does this mean using versioned enums, but not worrying too much about the internal structure? I think that's kind of the point of the versioned approach in general.

[mzabaluev]

I would prefer not to get carried away with versioning, besides making the current version a top-level parameter to enable breaking changes in the future.

So does this mean using versioned enums, but not worrying too much about the internal structure? I think that's kind of the point of the versioned approach in general.

Essentially a hybrid: put in a "version": 1 field to be future-proof, but evolve the structure backward-compatibly wrt. adding new fields without bumping the version as long as we can get away with it. This way, we don't need versioned enums until we actually have to introduce version 2.

[l-monninger]

Essentially a hybrid: put in a "version": 1 field to be future-proof, but evolve the structure backward-compatibly wrt. adding new fields without bumping the version as long as we can get away with it. This way, we don't need versioned enums until we actually have to introduce version 2.

Interesting. So, you are trying to avoid enums in code not their implication in the format? Because for most formats essentially how Serde will serialize an enum is some variant of this. For example, with JSON it names the field containing the data according to the version.

[mzabaluev]

While we are on version 1, it's possible to serialize and deserialize the schema with plain structs.
Even later, you can parse into a serde_json::Value, check the top-level version field, hen convert the items into struct instances of a version-appropriate type with serde_json::from_value.

[mzabaluev]

Something else that we should think about is the location of the migration table and requirements arising out of this.

• Well-known configuration file for the initial governance phase: preferably human-readable, likely JSON.
• On-chain parameter: should it be a binary format that allows extensibility (which excludes BCS)? Protobuf? Do we have a general approach for chain in Movement?

[l-monninger]

In the beginning, this will mainly be just for node operators. I think the initial governance suggestion that this is just a human-readable JSON stored somewhere is perfect. Perhaps it can be MIP that houses this in some manner.

[l-monninger]

@mzabaluev do you maybe want to open an MD for this?: https://github.com/movementlabsxyz/MIP

This will be part of a governance lineage, so it would be good to have there.

[mzabaluev]

Drafting the data types and rules of their application on the basis of the discussion.

DA Settings

The format of a well-known file defining migrations should be JSON serialization of a map where keys indicate the block height at which the configuration in the value starts applying. The values are objects. The top-level configuration envelope specifies the current version of the format to permit breaking changes in the future.

{
  "version": 1,
  "migrations": {
    "123456": {
      "namespace": "deadbeef01234567890",
      "transaction_format": 1,
      "block_format": 2,
      "ordering_rule": 3,
      "node_version": "1.2"
    }
  }
}

the Rust definition of the value struct can look like this:

#[derive(Serialize, Deserialize)]
pub struct DASettings {
    namespace: Namespace,
    transaction_format: TransactionFormatVersion,
    block_format: BlockFormatVersion,
    ordering_rule: OrderingRuleVersion,
    node_version: NodeVersionReq,
}

Principles of introducing migrations

Each new entry in the migration table SHALL use a unique Celestia namespace identifier to prevent confusion and detect accidental misconfiguration early.

The configuration in all nodes in the network should be updated well in advance of the block height at which a new change is introduced. The expected process in governance stage 0 is a coordinated upgrade.

The node_version field gives the semver minimum version of the Suzuka full node software that is required to use these settings, in the "major.minor" format. A node that does not satisfy the version requirement for any of the encountered entries MUST reject the configuration and report an error.

Applying the migrations

The producers and consumers of DA blobs must implement all versions encountered in the configuration file and apply the settings accordingly to the block height. There is not to be mixing of different DA formats between blobs submitted by correct nodes for the same block height.

Evolving the migration format

JSON objects are self-describing and allow adding new fields in a backward-compatible way, should we need them later.

For the on-chain parameter to be introduced with later governance stages, we can consider retaining the JSON format, or if a binary encoding is required, use an extensible encoding such as Protobuf.

[l-monninger]

Yeah, so apart from the questions about the versioning I still have from above, I think this is good.

I would basically copy over to an MIP, polish a bit, then look to assign issues.

https://github.com/movementlabsxyz/MIP

[mzabaluev]

MIP-13: movementlabsxyz/MIP#13