Address comments from PR

Author: aeisenberg

docs/codeql/codeql-cli/about-codeql-packs.rst

@@ -15,7 +15,7 @@ There are two types of CodeQL packs: query packs and library packs.
 You can use the package management commands in the CodeQL CLI to create CodeQL packs, add dependencies to packs, and install or update dependencies. For more information, see ":ref:`Creating and working with CodeQL packs <creating-and-working-with-codeql-packs>`." You can also publish and download CodeQL packs using the CodeQL CLI. For more information, see ":doc:`Publishing and using CodeQL packs <publishing-and-using-codeql-packs>`."
 
 
-The standard CodeQL packages for all supported languages are published in the `GitHub package registry <https://github.com/orgs/codeql/packages>`__.`
+The standard CodeQL packages for all supported languages are published in the `GitHub Container registry <https://github.com/orgs/codeql/packages>`__.`
 The `CodeQL repository <https://github.com/github/codeql>`__ contains sources for the standard CodeQL packs for all supported languages.
 
 .. _codeql-pack-structure:
@@ -59,7 +59,7 @@ The following properties are supported in ``qlpack.yml`` files.
 
    * - Property
      - Example
-     - Required
+     - Required by
      - Purpose
    * - ``name``
      - .. code-block:: yaml
@@ -82,14 +82,14 @@ The following properties are supported in ``qlpack.yml`` files.
           codeql/cpp-all: ^0.0.2
 
      - Packs that define CodeQL package dependencies on other packs
-     - A map from pack references to the semantic version range that is compatible with this pack. Supported from CLI version v2.6.0 and onwards.
+     - A map from pack references to the semantic version range that is compatible with this pack. Supported for CLI versions v2.6.0 and later.
    * - ``defaultSuiteFile``
      - .. code-block:: yaml
 
         defaultSuiteFile: cpp-code-scanning.qls
 
      - Packs that export a set of default queries to run
-     - The path to a query suite file containing all of the queries that are run by default when this pack is passed to the ``codeql database analyze`` command. Supported from CLI version v2.6.0 and onwards.
+     - The path to a query suite file relative to the package root, containing all of the queries that are run by default when this pack is passed to the ``codeql database analyze`` command. Supported from CLI version v2.6.0 and onwards. Only one of ``defaultSuiteFile`` or ``defaultSuite`` can be defined.
    * - ``defaultSuite``
      - .. code-block:: yaml
 
@@ -99,7 +99,7 @@ The following properties are supported in ``qlpack.yml`` files.
             precision: medium
 
      - Packs that export a set of default queries to run
-     - An inlined query suite containing all of the queries that are run by default when this pack is passed to the ``codeql database analyze`` command. Supported from CLI version v2.6.0 and onwards.
+     - An inlined query suite containing all of the queries that are run by default when this pack is passed to the ``codeql database analyze`` command. Supported from CLI version v2.6.0 and onwards. Only one of ``defaultSuiteFile`` or ``defaultSuite`` can be defined.
    * - ``library``
      - .. code-block:: yaml
 
@@ -113,7 +113,7 @@ The following properties are supported in ``qlpack.yml`` files.
         suites: octo-org-query-suites
 
      - Optional
-     - The path to a directory in the pack that contains the query suites you want to make known to the CLI, defined relative to the pack directory. CodeQL pack users can run "well-known" suites stored in this directory by specifying the pack name, without providing their full path. This is not supported for CodeQL packs downloaded from a package registry. For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
+     - The path to a directory in the pack that contains the query suites you want to make known to the CLI, defined relative to the pack directory. CodeQL pack users can run "well-known" suites stored in this directory by specifying the pack name, without providing their full path. This is not supported for CodeQL packs downloaded from the Container registry. For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
    * - ``extractor``
      - .. code-block:: yaml
 
@@ -141,7 +141,7 @@ The following properties are supported in ``qlpack.yml`` files.
         upgrades: .
 
      - Core language packs only
-     - The path to a directory within the pack that contains upgrade scripts, defined relative to the pack directory. The ``database upgrade`` action uses these scripts to upgrade databases that were created by an older version of an extractor or downgrade databases that were created by a newer version of the extractor so they're compatible with the current extractor (see `Upgrade scripts for a language <#upgrade-scripts-for-a-language>`__ below.)
+     - The path to a directory within the pack that contains database upgrade scripts, defined relative to the pack directory. Database upgrades are used internally to ensure a databasae created with a different version of the CodeQL CLI is compatible with the current version.
    * - ``authors``
      - .. code-block:: yaml
 
@@ -169,7 +169,7 @@ The following properties are supported in ``qlpack.yml`` files.
         libraryPathDependencies: codeql/javascript-all
 
      - Optional, deprecated
-     - Use the ``dependencies`` property instead. The names of any CodeQL packs that this CodeQL pack depends on, as a sequence. This gives the pack access to any libraries, database schema, and query suites defined in the dependency.
+     - Use the ``dependencies`` property instead. The names of any CodeQL packs that this CodeQL pack depends on, as an array. This gives the pack access to any libraries, database schema, and query suites defined in the dependency.
 
 .. _about-codeql-pack-lock:
 
@@ -199,6 +199,9 @@ The ``codeql-pack.lock.yml`` file will contain something like the following:
      my-user/transitive-dependency:
        version: 1.2.4
 
+..
+  TODO: Add a link to the CodeQL CLI documentation for query resolution, specifically in regards to resolving from source
+
 The ``codeql/cpp-all`` dependency is locked to version 0.1.4. The ``my-user/my-lib`` dependency is locked to version 0.2.4. The ``my-user/transitive-dependency``, which is a transitive dependency and is not specified in the ``qlpack.yml`` file, is locked to version 1.2.4. The ``other-dependency/from-source`` is absent from the lock file since it is resolved from source. This dependency must be available in the same CodeQL workspace as the pack.
 
 .. _custom-codeql-packs:
@@ -208,26 +211,43 @@ Examples of custom CodeQL packs
 
 When you write custom queries or tests, you should save them in custom CodeQL packs. For simplicity, try to organize each pack logically. For more information, see `CodeQL pack structure <#codeql-pack-structure>`__. Save files for queries and tests in separate packs and, where possible, organize custom packs into specific folders for each target language. This is particuarly useful if you intend to publish your CodeQL packs so they can be shared with others or used in GitHub `Code scanning <https://docs.github.com/en/code-security/secure-coding/automatically-scanning-your-code-for-vulnerabilities-and-errors/about-code-scanning>`__.
 
+CodeQL packs for custom libraries
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A custom CodeQL pack containing custom C++ libraries, with no queries or tests, may have a ``qlpack.yml`` file containing:
+
+.. code-block:: yaml
+
+   name: my-github-user/my-custom-libraries
+   version: 1.2.3
+   library: true
+   dependencies:
+     codeql/cpp-all: ^0.1.2
+
+where ``codeql/cpp-all`` is the name of the CodeQL pack for C/C++ analysis included in the CodeQL repository. The version range ``^0.1.2`` indicates that this pack is compatible with all versions of ``codeql/cpp-all`` that are greater than or equal to ``0.1.2`` and less than ``0.2.0``. Any CodeQL library file (a file with a ``.qll`` extension) defined in this pack will be available to queries defined in any query pack that includes this pack in its dependencies block.
+
+The ``library`` property indicates that this pack is a library pack and does not contain any queries.
+
 CodeQL packs for custom queries
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-A custom CodeQL pack for queries must include a ``qlpack.yml`` file at the pack root, containing ``name``, ``version``, and ``dependencies`` properties. If the pack contains query suites, you can use the ``suites`` property to define their location. Query suites defined here are called "well-known" suites, and can be used on the command line by referring to their name only, rather than their full path. For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
-
-For example, a ``qlpack.yml`` file for a CodeQL pack featuring custom C++ queries
-and libraries may contain:
+A custom CodeQL pack containing custom C++ queries and libraries may have a ``qlpack.yml`` file containing:
 
 .. code-block:: yaml
 
    name: my-github-user/my-custom-queries
    version: 1.2.3
    dependencies:
      codeql/cpp-all: ^0.1.2
+     my-github-user/my-custom-libraries: ^1.2.3
    suites: my-custom-suites
 
-where ``codeql/cpp-all`` is the name of the CodeQL pack for C/C++ analysis included in the CodeQL repository. The version range ``^0.1.2`` indicates that this pack is compatible with all versions of ``codeql/cpp-all`` that are greater than or equal to ``0.1.2`` and less than ``0.2.0``.
+where ``codeql/cpp-all`` is the name of the CodeQL pack for C/C++ analysis included in the CodeQL repository. The version range ``^0.1.2`` indicates that this pack is compatible with all versions of ``codeql/cpp-all`` that are greater than or equal to ``0.1.2`` and less than ``0.2.0``. ``my-github-user/my-custom-libraries`` is the name of a CodeQL pack containing custom CodeQL libraries for C++. Any CodeQL library file (a file with a ``.qll`` extension) defined in this pack will be available to queries in the ``my-github-user/my-custom-queries`` pack.
+
+The ``suites`` property indicates a directory where "well-known" query suites can be found. These suites can be used on the command line by referring to their name only, rather than their full path. For more information about query suites, see ":doc:`Creating CodeQL query suites <creating-codeql-query-suites>`."
 
-CodeQL packs for custom test files
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+CodeQL packs for custom tests
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 For custom CodeQL packs containing test files, you also need to include an
 ``extractor`` property so that the ``test run`` command knows how to create test
@@ -246,23 +266,18 @@ Examples of CodeQL packs in the CodeQL repository
 Each of the languages in the CodeQL repository has four main CodeQL packs:
 
 - Core library pack for the language, with the :ref:`database schema <codeql-database-schema>`
-  used by the language, and CodeQL libraries, and queries at ``ql/<language>/ql/lib``
+  used by the language, and CodeQL libraries, and queries at ``<language>/ql/lib``
 - Core query pack for the language that includes the default queries for the language, along
-  with their query suites at ``ql/<language>/ql/src``
-- Tests for the core language libraries and queries at ``ql/<language>/ql/test``
-- Example queries for the language at ``ql/<language>/ql/examples``
+  with their query suites at ``<language>/ql/src``
+- Tests for the core language libraries and queries at ``<language>/ql/test``
+- Example queries for the language at ``<language>/ql/examples``
 
 Core library pack
 ~~~~~~~~~~~~~~~~~
 
-The ``qlpack.yml`` file for a core library pack uses the following properties:
-``name``, ``version``, ``dbscheme``.
-The ``dbscheme`` property should only be defined in the core QL
-pack for a language.
-
-For example, the ``qlpack.yml`` file for `C/C++ analysis libraries
+Here is an example ``qlpack.yml`` file for `C/C++ analysis libraries
 <https://github.com/github/codeql/blob/main/cpp/ql/lib/qlpack.yml>`__
-contains:
+core language pack:
 
 .. code-block:: yaml
 
@@ -272,15 +287,18 @@ contains:
    library: true
    upgrades: upgrades
 
+Some extra notes on the following properties:
+
+- ``library``: Indicates that this is a library pack with no executable queries. It is only meant to be used as a dependency for other packs.
+- ``dbscheme`` and ``upgrades``: Properties internal to the CodeQL CLI and should only be defined in the core QL pack for a language.
+
+
 Core query pack
 ~~~~~~~~~~~~~~~
 
-The ``qlpack.yml`` file for a core query pack uses the following properties:
-``name``, ``version``, ``suites``, ``defaultSuiteFile``, ``dependencies`` .
-
-For example, the ``qlpack.yml`` file for `C/C++ analysis queries
-<https://github.com/github/codeql/blob/main/cpp/ql/lib/qlpack.yml>`__
-contains:
+Here is an example ``qlpack.yml`` file for `C/C++ analysis queries
+<https://github.com/github/codeql/blob/main/cpp/ql/src/qlpack.yml>`__
+core query pack:
 
 .. code-block:: yaml
 
@@ -292,16 +310,18 @@ contains:
    suites: codeql-suites
    defaultSuiteFile: codeql-suites/cpp-code-scanning.qls
 
-This query pack depends on ``codeql/cpp-all`` and ``codeql/suite-helpers``. Since these dependencies are resolved from source, it does not matter what version of the CodeQL pack they are compatible with.
+Some extra notes on the following properties:
+
+- ``dependencies``: This query pack depends on ``codeql/cpp-all`` and ``codeql/suite-helpers``. Since these dependencies are resolved from source, it does not matter what version of the CodeQL pack they are compatible with.
+- ``suites``: Indicates the directory containing "well-known" query suites.
+- ``defaultSuiteFile``: The name of the default query suite file that is used when no query suite is specified.
 
 Tests for the core CodeQL pack
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The ``qlpack.yml`` file for the tests for the core CodeQL packs use the following properties: ``name``, ``dependencies``, ``extractor``, ``tests``.
-
-For example, the ``qlpack.yml`` file for `C/C++ analysis tests
-<https://github.com/github/codeql/blob/main/cpp/ql/test/qlpack.yml>`__
-contains:
+Here is an example ``qlpack.yml`` file for `C/C++ analysis tests
+<https://github.com/github/codeql/blob/main/cpp/ql/src/qlpack.yml>`__
+core test pack:
 
 .. code-block:: yaml
 
@@ -312,4 +332,9 @@ contains:
    extractor: cpp
    tests: .
 
-The ``dependencies`` property always specifies the core CodeQL pack. The ``extractor`` property specifies that all the tests will use the same C++ extractor to create the database for the tests. The ``tests`` property specifies the location of the tests. In this case, the tests are in the root folder (and all sub-folders) of the pack. Note that there is no ``version`` property for the tests pack since this pack is not meant to be published.
+Some extra notes on the following properties:
+
+- ``dependencies``: This pack depends on the core CodeQL query and library packs for C++.
+- ``extractor``: This specifies that all the tests will use the same C++ extractor to create the database for the tests.
+- ``tests``: This specifies the location of the tests. In this case, the tests are in the root folder (and all sub-folders) of the pack.
+- ``version``: There is no ``version`` property for the tests pack. This prevents test packs from accidentally being published.

docs/codeql/codeql-cli/analyzing-databases-with-the-codeql-cli.rst

@@ -58,7 +58,7 @@ You can also specify:
   - a path to a directory containing query files
   - a path to a query suite file
   - the name of a CodeQL query pack
-    - with an optional a version range
+    - with an optional version range
     - with an optional path to a query, directory, or query suite inside the pack
 
   If omitted, the default query suite for the language of the database being analyzed will be used. For more information, see the :ref:`examples <database-analyze-examples>` below.
@@ -250,7 +250,7 @@ For example, to execute all Python queries contained in the ``Functions`` direct
 
    codeql database analyze <python-database> ../ql/python/ql/src/Functions/ --format=sarif-latest --output=python-analysis/python-results.sarif
 
-If you do not have the codeql repository checked out, you can execute the same queries by specifying the query pack name and the path to the queries:
+If you do not have the CodeQL repository checked out, you can execute the same queries by specifying the query pack name and the path to the queries:
 
    codeql database analyze --download <python-database> codeql/python-queries:Functions/ --format=sarif-latest --output=python-analysis/python-results.sarif
 

docs/codeql/codeql-cli/creating-and-working-with-codeql-packs.rst

@@ -41,7 +41,7 @@ If you already have a ``qlpack.yml`` file, you can edit it manually to convert i
 
 #. Edit the ``name`` property so that it matches the format ``<scope>/<name>``, where ``<scope>`` is the name of the GitHub organization or user account that you will publish to.
 #. In the ``qlpack.yml`` file, include a ``version`` property with a semver identifier, as well as an optional ``dependencies`` block.
-#. Migrate the list of dependencies in ``libraryPathDependencies`` to the ``dependencies`` block. Specify the version range for each dependency. If the range is unimportant, or you are unsure of compatibility, you can specify ``"*"``, which indicates that any version is acceptable, with will default to the latest version when ``codeql pack install`` is run.
+#. Migrate the list of dependencies in ``libraryPathDependencies`` to the ``dependencies`` block. Specify the version range for each dependency. If the range is unimportant, or you are unsure of compatibility, you can specify ``"*"``, which indicates that any version is acceptable and will default to the latest version when ``codeql pack install`` is run.
 
 For more information about the properties, see ":ref:`About CodeQL packs <about-codeql-packs>`."
 

docs/codeql/codeql-cli/creating-codeql-query-suites.rst

@@ -240,7 +240,7 @@ the reusable instructions to the current suite. For example::
 This will filter the queries in ``queries/cpp/custom`` to only include those that match the reusable conditions.
 
 You can also create a suite definition using ``reusable-instructions.yml`` on
-queries in a different codeQL pack. If the ``.qls`` file is in the same CodeQL pack as
+queries in a different CodeQL pack. If the ``.qls`` file is in the same CodeQL pack as
 the queries, you can add a ``from`` field immediately after the ``apply``
 instruction::
 

docs/codeql/codeql-cli/getting-started-with-the-codeql-cli.rst

@@ -91,7 +91,7 @@ Alternatively, you can download ``codeql.zip``, which contains the CLI for all s
       iv. Ensure the check-box labeled **Open "safe" files after downloading**.
           is unchecked.
 
-1. Extract the zip archive
+2. Extract the zip archive
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 For Linux, Windows, and macOS users (version 10.14 "Mojave", and earlier)

docs/codeql/reusables/test-qlpack.rst

@@ -1,10 +1,9 @@
 .. code-block:: yaml
 
    name: my-github-user/my-query-tests
-   version: 0.0.0
    dependencies:
      my-github-user/my-custom-queries: ^1.2.3
    extractor: java
    tests: .
 
-This ``qlpack.yml`` file states that ``my-github-user/my-query-tests`` depends on ``my-github-user/my-custom-queries`` at a version greater than or equal to 1.2.3 and less than 2.0.0. It also declares that the CLI should use the Java ``extractor`` when creating test databases. The ``tests: .`` line declares that all ``.ql`` files in the pack should be run as tests when ``codeql test run`` is run with the ``--strict-test-discovery`` option.
+This ``qlpack.yml`` file states that ``my-github-user/my-query-tests`` depends on ``my-github-user/my-custom-queries`` at a version greater than or equal to 1.2.3 and less than 2.0.0. It also declares that the CLI should use the Java ``extractor`` when creating test databases. The ``tests: .`` line declares that all ``.ql`` files in the pack should be run as tests when ``codeql test run`` is run with the ``--strict-test-discovery`` option. Typically, test packs do not contain a ``version`` property. This prevents them from being inadvertently not published.