Back out Canonical ABI mangling scheme in preparation for future alternative

Author: lukewagner

design/mvp/CanonicalABI.md

@@ -1313,281 +1313,6 @@ the AOT compiler as requiring an intermediate copy to implement the above
 `lift`-then-`lower` semantics.
 
 
-## Canonical ABI
-
-The above `canon` definitions are parameterized, giving each component a small
-space of ABI options for interfacing with its contained core modules. Moreover,
-each component can choose its ABI options independently of each other component,
-with compiled adapter trampolines handling any conversions at cross-component
-call boundaries. However, in some contexts, it is useful to fix a **single**,
-"**canonical**" ABI that is fully determined by a given component type (which
-itself is fully determined by a set of [`wit`](WIT.md) files). For example,
-this allows existing Core WebAssembly toolchains to continue targeting [WASI]
-by importing and exporting fixed Core Module functions signatures, without
-having to add any new component-model concepts.
-
-To support these use cases, the following section defines two new mappings:
-1. `canonical-module-type : componenttype -> core:moduletype`
-2. `lift-canonical-module : core:module -> component`
-
-The `canonical-module-type` mapping defines the collection of core function
-signatures that a core module must import and export to implement the given
-component type via the Canonical ABI.
-
-The `lift-canonical-module` mapping defines the runtime behavior of a core
-module that has successfully implemented `canonical-module-type` by fixing
-a canonical set of ABI options that are passed to the above-defined `canon`
-definitions.
-
-Together, these definitions are intended to satisfy the invariant:
-```
-for all m : core:module and ct : componenttype:
-  module-type(m) = canonical-module-type(ct) implies ct = type-of(lift-canonical-module(m))
-```
-One consequence of this is that the canonical `core:moduletype` must encode
-enough high-level type information for `lift-canonical-module` to be able to
-reconstruct a working component. This is achieved using [name mangling]. Unlike
-traditional C-family name mangling, which have a limited character set imposed
-by linkers and aim to be space-efficient enough to support millions of
-*internal* names, the Canonical ABI can use any valid UTF-8 string and only
-needs to mangle *external* names, of which there will only be a handful.
-Therefore, squeezing out every byte is a lower concern and so, for simplicity
-and readability, type information is mangled using a subset of the
-[`wit`](WIT.md) syntax.
-
-One final point of note is that `lift-canonical-module` is only able to produce
-a *subset* of all possible components (e.g., not covering nesting and
-virtualization scenarios); to express the full variety of components, a
-toolchain needs to emit proper components directly. Thus, the Canonical ABI
-serves as an incremental adoption path to the full component model, allowing
-existing Core WebAssembly toolchains to produce simple components simply by
-emitting module imports and exports with the appropriate mangled names (e.g.,
-in LLVM using the [`import_name`] and [`export_name`] attributes).
-
-
-### Canonical Module Type
-
-For the same reason that core module and component [binaries](Binary.md)
-include a version number (that is intended to never change after it reaches
-1.0), the Canonical ABI defines its own version that is explicitly declared by
-a core module. Before reaching stable 1.0, the Canonical ABI is explicitly
-allowed to make breaking changes, so this version also serves the purpose of
-coordinating breaking changes in pre-1.0 tools and runtimes.
-```python
-CABI_VERSION = '0.1'
-```
-Working top-down, a canonical module type is defined by the following mapping:
-```python
-def canonical_module_type(ct: ComponentType) -> ModuleType:
-  start_params, import_funcs = mangle_instances(ct.imports)
-  start_results, export_funcs = mangle_instances(ct.exports)
-
-  imports = []
-  for name,ft in import_funcs:
-    flat_ft = flatten_functype(ft, 'lower')
-    imports.append(CoreImportDecl('', mangle_funcname(name, ft), flat_ft))
-
-  exports = []
-  exports.append(CoreExportDecl('cabi_memory', CoreMemoryType(initial=0, maximum=None)))
-  exports.append(CoreExportDecl('cabi_realloc', CoreFuncType(['i32','i32','i32','i32'], ['i32'])))
-
-  start_ft = FuncType(start_params, start_results)
-  start_name = mangle_funcname('cabi_start{cabi=' + CABI_VERSION + '}', start_ft)
-  exports.append(CoreExportDecl(start_name, flatten_functype(start_ft, 'lift')))
-
-  for name,ft in export_funcs:
-    flat_ft = flatten_functype(ft, 'lift')
-    exports.append(CoreExportDecl(mangle_funcname(name, ft), flat_ft))
-    if any(contains_dynamic_allocation(t) for t in ft.results):
-      exports.append(CoreExportDecl('cabi_post_' + name, CoreFuncType(flat_ft.results, [])))
-
-  return ModuleType(imports, exports)
-
-def contains_dynamic_allocation(t):
-  match despecialize(t):
-    case String()       : return True
-    case List(t)        : return True
-    case Record(fields) : return any(contains_dynamic_allocation(f.t) for f in fields)
-    case Variant(cases) : return any(contains_dynamic_allocation(c.t) for c in cases)
-    case _              : return False
-```
-This definition starts by mangling all nested instances into the names of the
-leaf fields, so that instances can be subsequently ignored. Next, each
-component-level function import/export is mapped to corresponding core function
-import/export with the function type mangled into the name. Additionally, each
-export whose return type implies possible dynamic allocation is given a
-`post-return` function so that it can deallocate after the caller reads the
-return value. Lastly, all value imports and exports are concatenated into a
-synthetic `cabi_start` function that is called immediately after instantiation.
-
-For imports (which in Core WebAssembly are [two-level]), the first-level name
-is set to be a zero-length string so that the entire rest of the first-level
-string space is available for [shared-everything dynamic linking].
-
-For imports and exports, the Canonical ABI assumes that `_` is not a valid
-character in a component-level import/export (as is currently the case in `wit`
-[identifiers](WIT.md#identifiers)) and thus can safely be used to prefix
-auxiliary Canonical ABI-induced imports/exports.
-
-#### Instance type mangling
-
-Instance-type mangling recursively builds a dotted path string (of instance names)
-that is included in the mangled core import/export name:
-```python
-def mangle_instances(xs, path = ''):
-  values = []
-  funcs = []
-  for x in xs:
-    name = path + x.name
-    match x.t:
-      case ValueType(t):
-        values.append( (name, t) )
-      case FuncType(params,results):
-        funcs.append( (name, x.t) )
-      case InstanceType(exports):
-        vs,fs = mangle_instances(exports, name + '.')
-        values += vs
-        funcs += fs
-      case TypeType(bounds):
-        assert(False) # TODO: resource types
-      case ComponentType(imports, exports):
-        assert(False) # TODO: `canon instantiate`
-      case ModuleType(imports, exports):
-        assert(False) # TODO: canonical shared-everything linking
-  return (values, funcs)
-```
-The three `TODO` cases are intended to be filled in by future PRs extending
-the Canonical ABI.
-
-#### Function type mangling
-
-Function types are mangled into [`wit`](WIT.md)-compatible syntax:
-```python
-def mangle_funcname(name, ft):
-  params = mangle_named_types(ft.params)
-  if len(ft.results) == 1 and isinstance(ft.results[0], ValType):
-    results = mangle_valtype(ft.results[0])
-  else:
-    results = mangle_named_types(ft.results)
-  return f'{name}: func{params} -> {results}'
-
-def mangle_named_types(nts):
-  assert(all(type(nt) == tuple and len(nt) == 2 for nt in nts))
-  mangled_elems = (nt[0] + ': ' + mangle_valtype(nt[1]) for nt in nts)
-  return '(' + ', '.join(mangled_elems) + ')'
-```
-
-#### Value type mangling
-
-Value types are similarly mangled into [`wit`](WIT.md)-compatible syntax,
-recursively:
-
-```
-def mangle_valtype(t):
-  match t:
-    case Bool()           : return 'bool'
-    case S8()             : return 's8'
-    case U8()             : return 'u8'
-    case S16()            : return 's16'
-    case U16()            : return 'u16'
-    case S32()            : return 's32'
-    case U32()            : return 'u32'
-    case S64()            : return 's64'
-    case U64()            : return 'u64'
-    case Float32()        : return 'float32'
-    case Float64()        : return 'float64'
-    case Char()           : return 'char'
-    case String()         : return 'string'
-    case List(t)          : return 'list<' + mangle_valtype(t) + '>'
-    case Record(fields)   : return mangle_recordtype(fields)
-    case Tuple(ts)        : return mangle_tupletype(ts)
-    case Flags(labels)    : return mangle_flags(labels)
-    case Variant(cases)   : return mangle_varianttype(cases)
-    case Enum(labels)     : return mangle_enumtype(labels)
-    case Union(ts)        : return mangle_uniontype(ts)
-    case Option(t)        : return mangle_optiontype(t)
-    case Result(ok,error) : return mangle_resulttype(ok,error)
-
-def mangle_recordtype(fields):
-  mangled_fields = (f.label + ': ' + mangle_valtype(f.t) for f in fields)
-  return 'record { ' + ', '.join(mangled_fields) + ' }'
-
-def mangle_tupletype(ts):
-  return 'tuple<' + ', '.join(mangle_valtype(t) for t in ts) + '>'
-
-def mangle_flags(labels):
-  return 'flags { ' + ', '.join(labels) + ' }'
-
-def mangle_varianttype(cases):
-  mangled_cases = ('{label}{payload}'.format(
-                     label = c.label,
-                     payload = '' if c.t is None else '(' + mangle_valtype(c.t) + ')')
-                   for c in cases)
-  return 'variant { ' + ', '.join(mangled_cases) + ' }'
-
-def mangle_enumtype(labels):
-  return 'enum { ' + ', '.join(labels) + ' }'
-
-def mangle_uniontype(ts):
-  return 'union { ' + ', '.join(mangle_valtype(t) for t in ts) + ' }'
-
-def mangle_optiontype(t):
-  return 'option<' + mangle_valtype(t) + '>'
-
-def mangle_resulttype(ok, error):
-  match (ok, error):
-    case (None, None) : return 'result'
-    case (None, _)    : return 'result<_, ' + mangle_valtype(error) + '>'
-    case (_, None)    : return 'result<' + mangle_valtype(ok) + '>'
-    case (_, _)       : return 'result<' + mangle_valtype(ok) + ', ' + mangle_valtype(error) + '>'
-```
-As an example, given a component type:
-```wasm
-(component
-  (import "foo" (func))
-  (import "a" (instance
-    (export "bar" (func (param "x" u32) (param "y" u32) (result u32)))
-  ))
-  (import "v1" (value string))
-  (export "baz" (func (param "s" string) (result string)))
-  (export "v2" (value list<list<string>>))
-)
-```
-the `canonical_module_type` would be:
-```wasm
-(module
-  (import "" "foo: func() -> ()" (func))
-  (import "" "a.bar: func(x: u32, y: u32) -> u32" (func param i32 i32) (result i32))
-  (export "cabi_memory" (memory 0))
-  (export "cabi_realloc" (func (param i32 i32 i32 i32) (result i32)))
-  (export "cabi_start{cabi=0.1}: func(v1: string) -> (v2: list<list<string>>)" (func (param i32 i32) (result i32)))
-  (export "baz: func(s: string) -> string" (func (param i32 i32) (result i32)))
-  (export "cabi_post_baz" (func (param i32)))
-)
-```
-
-### Lifting Canonical Modules
-
-TODO
-
-```python
-class Module:
-  t: ModuleType
-  instantiate: Callable[typing.List[typing.Tuple[str,str,Value]], typing.List[typing.Tuple[str,Value]]]
-
-class Component:
-  t: ComponentType
-  instantiate: Callable[typing.List[typing.Tuple[str,any]], typing.List[typing.Tuple[str,any]]]
-
-def lift_canonical_module(module: Module) -> Component:
-  # TODO: define component.instantiate by:
-  #  1. creating canonical import adapters
-  #  2. creating a core module instance that imports (1)
-  #  3. creating canonical export adapters from the exports of (2)
-  pass
-```
-
-
 
 [Canonical Definitions]: Explainer.md#canonical-definitions
 [`canonopt`]: Explainer.md#canonical-definitions