Merge pull request #10088 from github/parameterisedModules

parameterised modules in the QL language reference

Author: ginsbach

docs/codeql/ql-language-reference/index.rst

@@ -15,6 +15,8 @@ Learn all about QL, the powerful query language that underlies the code scanning
 
 - :doc:`Modules <modules>`: Modules provide a way of organizing QL code by grouping together related types, predicates, and other modules. 
 
+- :doc:`Signatures <signatures>`: Signatures provide a typing mechanism to parameters of parameterized modules.
+
 - :doc:`Aliases <aliases>`: An alias is an alternative name for an existing QL entity. 
 
 - :doc:`Variables <variables>`: Variables in QL are used in a similar way to variables in algebra or logic. They represent sets of values, and those values are usually restricted by a formula.
@@ -44,6 +46,7 @@ Learn all about QL, the powerful query language that underlies the code scanning
    queries
    types
    modules
+   signatures
    aliases
    variables
    expressions

docs/codeql/ql-language-reference/modules.rst

@@ -133,6 +133,92 @@ defined :ref:`above <library-modules>`:
 This defines an explicit module named ``M``. The body of this module defines
 the class ``OneTwo``.
 
+.. _parameterized-modules:
+
+Parameterized modules
+=====================
+
+Parameterized modules are QL's approach to generic programming.
+Similar to explicit modules, parameterized modules are defined within other modules using the keywork ``module``.
+In addition to the module name, parameterized modules declare one or more parameters between the name and the module body.
+
+For example, consider the module ``M``, which takes two predicate parameters and defines a new predicate
+that applies them one after the other:
+
+.. code-block:: ql
+
+    module M<transformer/1 first, transformer/1 second> {
+      bindingset[x]
+      int applyBoth(int x) {
+        result = second(first(x))
+      }
+    }
+
+Parameterized modules cannot be directly referenced.
+Instead, you instantiate a parameterized module by passing arguments enclosed in angle brackets (``<`` and ``>``) to the module.
+Instantiated parameterized modules can be used as a :ref:`module expression <name-resolution>`, identical to explicit module references.
+
+For example, we can instantiate ``M`` with two identical arguments ``increment``, creating a module
+containing a predicate that adds 2:
+
+.. code-block:: ql
+
+    bindingset[result] bindingset[x]
+    int increment(int x) { result = x + 1 }
+
+    module IncrementTwice = M<increment/1, increment/1>;
+
+    select IncrementTwice::applyBoth(40) // 42
+
+The parameters of a parameterized module are (meta-)typed with :ref:`signatures <signatures>`.
+
+For example, in the previous two snippets, we relied on the predicate signature ``transformer``:
+
+.. code-block:: ql
+
+    bindingset[x]
+    signature int transformer(int x);
+
+The instantiation of parameterized modules is applicative.
+That is, if you instantiate a parameterized module twice with identical arguments, the resulting object is the same.
+This is particularly relevant for type definitions inside parameterized modules as :ref:`classes <classes>`
+or via :ref:`newtype <algebraic-datatypes>`, because the duplication of such type definitions would result in
+incompatible types.
+
+The following example instantiates module ``M`` inside calls to predicate ``foo`` twice.
+The first call is valid but the second call generates an error.
+
+.. code-block:: ql
+
+    bindingset[this]
+    signature class TSig;
+
+    module M<TSig T> {
+      newtype A = B() or C()
+    }
+
+    string foo(M<int>::A a) { ... }
+
+    select foo(M<int>::B()),  // valid: repeated identical instantiation of M does not duplicate A, B, C
+           foo(M<float>::B()) // ERROR: M<float>::B is not compatible with M<int>::A
+
+Module parameters are dependently typed, meaning that signature expressions in parameter definitions can reference
+preceding parameters.
+
+For example, we can declare the signature for ``T2`` dependent on ``T1``, enforcing a subtyping relationship
+between the two parameters:
+
+.. code-block:: ql
+
+    signature class TSig;
+
+    module Extends<TSig T> { signature class Type extends T; }
+
+    module ParameterizedModule<TSig T1, Extends<T1>::Type T2> { ... }
+
+Dependently typed parameters are particularly useful in combination with
+:ref:`parameterized module signatures <parameterized-module-signatures>`.
+
 .. _module-bodies:
 
 Module bodies

docs/codeql/ql-language-reference/name-resolution.rst

@@ -21,7 +21,7 @@ In summary, the kinds of expressions are:
   - **Module expressions**
       - These refer to modules.
       - They can be simple :ref:`names <names>`, :ref:`qualified references <qualified-references>` 
-        (in import statements), or :ref:`selections <selections>`.
+        (in import statements), :ref:`selections <selections>`, or :ref:`instantiations <parameterized-modules>`.
   - **Type expressions**
       - These refer to types.
       - They can be simple :ref:`names <names>` or :ref:`selections <selections>`.

docs/codeql/ql-language-reference/signatures.rst

@@ -0,0 +1,120 @@
+:tocdepth: 1
+
+.. index:: signature
+
+.. _signatures:
+
+Signatures
+##########
+
+Parameterized modules use signatures as a type system for their parameters.
+There are three categories of signatures: **predicate signatures**, **type signatures**, and **module signatures**.
+
+Predicate signatures
+====================
+
+Predicate signatures declare module parameters that will be substituted with predicates when the module is instantiated.
+
+The substitution of predicate signatures relies on structural typing. That is, predicates do not have to be explicitly
+defined as implementing a predicate signature - they just have to match the return and argument types.
+
+Predicate signatures are defined much like predicates themselves, but they do not have a body.
+In detail, a predicate signature definition consists of:
+
+#. The keyword ``signature``.
+#. The keyword ``predicate`` (allows subsitution with a :ref:`predicate without result <predicates-without-result>`),
+   or the type of the result (allows subsitution with a :ref:`predicate with result <predicates-with-result>`).
+#. The name of the predicate signature. This is an `identifier <https://codeql.github.com/docs/ql-language-reference/ql-language-specification/#identifiers>`_
+   starting with a lowercase letter.
+#. The arguments to the predicate signature, if any, separated by commas.
+   For each argument, specify the argument type and an identifier for the argument variable.
+#. A semicolon ``;``.
+
+For example:
+
+.. code-block:: ql
+
+    signature int operator(int lhs, int rhs);
+
+Type signatures
+===============
+
+Type signatures declare module parameters that will be substituted with types when the module is instantiated.
+Type signatures are used to specify supertypes and are the simplest category of signatures.
+
+The substitution of type signatures relies on structural typing. That is, types do not have to be explicitly defined as
+implementing a type signature - they just need to have the specified (transitive) supertypes.
+
+In detail, a type signature definition consists of:
+
+#. The keyword ``signature``.
+#. The keyword ``class``.
+#. The name of the type signature. This is an `identifier <https://codeql.github.com/docs/ql-language-reference/ql-language-specification/#identifiers>`_
+   starting with a uppercase letter.
+#. Optionally, the keyword ``extends`` followed by a list of types, separated by commas.
+#. A semicolon ``;``.
+
+For example:
+
+.. code-block:: ql
+
+    signature class ExtendsInt extends int;
+
+Module signatures
+=================
+
+Module signatures declare module parameters that will be substituted with modules when the module is instantiated.
+Module signatures specify a collection of types and predicates that a module needs to contain under given names and
+matching given signatures.
+
+Unlike type signatures and predicate signatures, the substitution of type signatures relies on nominal typing.
+That is, the definition of a module must declare the module signatures it implements.
+
+In detail, a type signature definition consists of:
+
+#. The keyword ``signature``.
+#. The keyword ``module``.
+#. The name of the module signature. This is an `identifier <https://codeql.github.com/docs/ql-language-reference/ql-language-specification/#identifiers>`_
+   starting with a uppercase letter.
+#. Optionally, a list of parameters for :ref:`parameterized module signatures <parameterized-module-signatures>`.
+#. The module signature body, consisting of type signatures and predicate signatures enclosed in braces.
+   The ``signature`` keyword is omitted for these contained signatures.
+
+For example:
+
+.. code-block:: ql
+
+    signature module MSig {
+      class T;
+      predicate restriction(T t);
+    }
+
+    module Module implements MSig {
+      newtype T = A() or B();
+
+      predicate restriction(T t) { t = A() }
+    }
+
+.. _parameterized-module-signatures:
+
+Parameterized module signatures
+-------------------------------
+
+Module signatures can themselves be parameterized in exactly the same way as parameterized modules.
+This is particularly useful in combination with the dependent typing of module parameters.
+
+For example:
+
+.. code-block:: ql
+
+    signature class NodeSig;
+
+    signature module EdgeSig<NodeSig Node> {
+      predicate apply(Node src, Node dst);
+    }
+
+    module Reachability<NodeSig Node, EdgeSig<Node> Edge> {
+      Node reachableFrom(Node src) {
+        Edge::apply+(src, result)
+      }
+    }