Skip to content

Commit fb8a0fc

Browse files
authored
Merge pull request #10182 from github/aeisenberg/codeql-workspace-docs
Add docs for codeql workspaces
2 parents 059d320 + a065974 commit fb8a0fc

File tree

5 files changed

+100
-7
lines changed

5 files changed

+100
-7
lines changed

docs/codeql/codeql-cli/about-codeql-packs.rst

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -199,11 +199,7 @@ The ``codeql-pack.lock.yml`` file will contain something like the following:
199199
my-user/transitive-dependency:
200200
version: 1.2.4
201201
202-
203-
..
204-
TODO: Add a link to the CodeQL CLI documentation for query resolution, specifically in regards to resolving from source
205-
206-
The ``codeql/cpp-all`` dependency is locked to version 0.1.4. The ``my-user/my-lib`` dependency is locked to version 0.2.4. The ``my-user/transitive-dependency``, which is a transitive dependency and is not specified in the ``qlpack.yml`` file, is locked to version 1.2.4. The ``other-dependency/from-source`` is absent from the lock file since it is resolved from source. This dependency must be available in the same CodeQL workspace as the pack.
202+
The ``codeql/cpp-all`` dependency is locked to version 0.1.4. The ``my-user/my-lib`` dependency is locked to version 0.2.4. The ``my-user/transitive-dependency``, which is a transitive dependency and is not specified in the ``qlpack.yml`` file, is locked to version 1.2.4. The ``other-dependency/from-source`` is absent from the lock file since it is resolved from source. This dependency must be available in the same CodeQL workspace as the pack. For more information about CodeQL workspaces and resolving dependencies from source, see ":doc:`About CodeQL Workspaces <about-codeql-workspaces>`."
207203

208204
In most cases, the ``codeql-pack.lock.yml`` file is only relevant for query packs since library packs are non-executable and usually do not need their transitive dependencies to be fixed. The exception to this is for library packs that contain tests. In this case, the ``codeql-pack.lock.yml`` file is used to ensure that the tests are always run with the same versions of dependencies to avoid spurious failures when there are mismatched dependencies.
209205

@@ -316,7 +312,7 @@ core query pack:
316312
317313
Some extra notes on the following properties:
318314

319-
- ``dependencies``: This query pack depends on ``codeql/cpp-all`` and ``codeql/suite-helpers``. Since these dependencies are resolved from source, it does not matter what version of the CodeQL pack they are compatible with.
315+
- ``dependencies``: This query pack depends on ``codeql/cpp-all`` and ``codeql/suite-helpers``. Since these dependencies are resolved from source, it does not matter what version of the CodeQL pack they are compatible with. For more information about resolving dependencies from source, see ":ref:`Source Dependencies <source-dependencies>`."
320316
- ``suites``: Indicates the directory containing "well-known" query suites.
321317
- ``defaultSuiteFile``: The name of the default query suite file that is used when no query suite is specified.
322318

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
.. _about-codeql-workspaces:
2+
3+
About CodeQL Workspaces
4+
=======================
5+
6+
.. include:: ../reusables/beta-note-package-management.rst
7+
8+
CodeQL workspaces are used to group multiple CodeQL packs together. A typical use case for a CodeQL workspace is to develop a set of CodeQL library and query packs that are mutually dependent. For more information on CodeQL packs, see ":doc:`About CodeQL packs <about-codeql-packs>`."
9+
10+
The main benefit of a CodeQL workspace is that it makes it easier for you to develop and maintain multiple CodeQL packs. When you use a CodeQL workspace, all the CodeQL packs in the workspace are available as *source dependencies* for each other when you run a CodeQL command that resolves queries. This makes it easier to develop, maintain, and publish multiple, related CodeQL packs.
11+
12+
In most cases, you should store the CodeQL workspace and the CodeQL packs contained in it in one git repository. This makes it easier to share your CodeQL development environment.
13+
14+
The ``codeql-workspace.yml`` file
15+
----------------------------------
16+
17+
A CodeQL workspace is defined by a ``codeql-workspace.yml`` yaml file. This file contains a ``provide`` block, and optionally an ``ignore`` block.
18+
19+
* The ``provide`` block contains a list of glob patterns that define the CodeQL packs that are available in the workspace.
20+
* The ``ignore`` block contains a list of glob patterns that define CodeQL packs that are not available in the workspace.
21+
22+
Each entry in the ``provide`` or ``ignore`` section must map to the location of a ``qlpack.yml`` file. All glob patterns are defined relative to the directory that contains the workspace file. For a list of patterns accepted in this file, see "`@actions/glob <https://github.com/actions/toolkit/tree/main/packages/glob#patterns>`__ ."
23+
24+
For example, the following ``codeql-workspace.yml`` file defines a workspace that contains all the CodeQL packs recursively found in the ``codeql-packs`` directory, except for the packs in the ``experimental`` directory:
25+
26+
.. code-block:: yaml
27+
28+
provide:
29+
- "*/codeql-packs/**/qlpack.yml"
30+
ignore:
31+
- "*/codeql-packs/**/experimental/**/qlpack.yml"
32+
33+
To verify that your ``codeql-workspace.yml`` file includes the CodeQL packs that you expect, run the ``codeql pack ls`` command in the same directory as your workspace. The result of the command is a list of all CodeQL packs in the workspace.
34+
35+
.. _source-dependencies:
36+
37+
Source Dependencies
38+
-------------------
39+
40+
Source dependencies are CodeQL packs that are resolved from the local file system outside of the CodeQL package cache. These dependencies can be in the same CodeQL workspace, or specified as a path option using the ``--additional-packs`` argument. When you compile and run queries locally, source dependencies override any dependencies found in the CodeQL package cache as well as version constraints defined in the ``qlpack.yml``. All references to CodeQL packs in the same workspace are resolved as source dependencies.
41+
42+
This is particularly useful in the following situations:
43+
44+
- One of the dependencies of the query pack you are running is not yet published. Resolving from source is the only way to reference that pack.
45+
- You are making changes to multiple packs at the same time and want to test them together. Resolving from source ensures that you are using the version of the pack with your changes in it.
46+
47+
CodeQL workspaces and query resolution
48+
--------------------------------------
49+
50+
All CodeQL packs in a workspace are available as source dependencies for each other when you run any CodeQL command that resolves queries or packs. For example, when you run ``codeql pack install`` in a pack directory in a workspace, any dependency that can be found in the workspace will be used instead of downloading that dependency to the package cache and adding it to the ``codeql-pack.lock.yml`` file. For more information, see ":ref:`Adding and Installing Dependencies <adding-and-installing-dependencies>`."
51+
52+
Similarly, when you publish a CodeQL query pack to the GitHub container registry using ``codeql pack publish`` the command will always use the dependencies from the workspace instead of using dependencies found in the local package cache.
53+
54+
This ensures that any local changes you make to a query library in a dependency are automatically reflected in any query packs you publish from that workspace.
55+
56+
Example
57+
~~~~~~~
58+
59+
Consider the following ``codeql-workspace.yml`` file:
60+
61+
.. code-block:: yaml
62+
63+
provide:
64+
- "**/qlpack.yml"
65+
66+
And the following CodeQL library pack ``qlpack.yml`` file in the workspace:
67+
68+
.. code-block:: yaml
69+
70+
name: my-company/my-library
71+
library: true
72+
version: 1.0.0
73+
74+
And the following CodeQL query pack ``qlpack.yml`` file in the workspace:
75+
76+
.. code-block:: yaml
77+
78+
name: my-company/my-queries
79+
version: 1.0.0
80+
dependencies:
81+
my-company/my-library: "*"
82+
codeql/cpp-all: ~0.2.0
83+
84+
Notice that the ``dependencies`` block for the CodeQL query pack, ``my-company/my-queries``, specifies ``"*"`` as the version of the library pack. Since the library pack is already defined as a source dependency in ``codeql-workspace.yml``, the library pack's content is always resolved from inside the workspace. Any version constraint you define will be ignored in this case. We recommend that you use ``"*"`` for source dependencies to make it clear that the version is inherited from the workspace.
85+
86+
When you execute ``codeql pack install`` from the query pack directory, an appropriate version of ``codeql/cpp-all`` is downloaded to the local package cache. Also, a ``codeql-pack.lock.yml`` file is created that contains the resolved version of ``codeql/cpp-all``. The lock file won't contain an entry for ``my-company/my-library`` since it is resolved from source dependencies. The ``codeql-pack.lock.yml`` file will look something like this:
87+
88+
.. code-block:: yaml
89+
90+
dependencies:
91+
codeql/cpp-all:
92+
version: 0.2.2
93+
94+
When you execute ``codeql pack publish`` from the query pack directory, the ``codeql/cpp-all`` dependency from the package cache and the ``my-company/my-library`` from the workspace are bundled with ``my-company/my-queries`` and published to the GitHub container registry.

docs/codeql/codeql-cli/analyzing-databases-with-the-codeql-cli.rst

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,6 @@ You can also specify:
9292
For databases that were created by CodeQL CLI v2.3.4 or later, the CLI will implicitly run any
9393
required upgrades. Explicitly running the upgrade command is not necessary.
9494

95-
9695
For full details of all the options you can use when analyzing databases, see
9796
the `database analyze reference documentation <../manual/database-analyze>`__.
9897

docs/codeql/codeql-cli/codeql-cli-reference.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,12 +10,14 @@ Learn more about the files you can use when running CodeQL processes and the res
1010
:hidden:
1111

1212
about-codeql-packs
13+
about-codeql-workspaces
1314
query-reference-files
1415
sarif-output
1516
exit-codes
1617
extractor-options
1718

1819
- :doc:`About CodeQL packs <about-codeql-packs>`: CodeQL packs are created with the CodeQL CLI and are used to create, depend on, publish, and run CodeQL queries, libraries, and query suites.
20+
- :doc:`About CodeQL workspaces <about-codeql-workspaces>`: CodeQL workspaces are used to group multiple CodeQL packs together.
1921
- :doc:`Query reference files <query-reference-files>`: A query reference file is text file that defines the location of one query to test.
2022
- :doc:`SARIF output <sarif-output>`: CodeQL supports SARIF as an output format for sharing static analysis results.
2123
- :doc:`Exit codes <exit-codes>`: The CodeQL CLI reports the status of each command it runs as an exit code.

docs/codeql/codeql-cli/creating-and-working-with-codeql-packs.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,8 @@ If you already have a ``qlpack.yml`` file, you can edit it manually to convert i
4545

4646
For more information about the properties, see ":ref:`About CodeQL packs <about-codeql-packs>`."
4747

48+
.. _adding-and-installing-dependencies:
49+
4850
Adding and installing dependencies to a CodeQL pack
4951
---------------------------------------------------
5052
You can add dependencies on CodeQL packs using the command ``codeql pack add``. You must specify the scope, name, and (optionally) a compatible version range.

0 commit comments

Comments
 (0)