Skip to content

Commit 8e0a006

Browse files
authored
Merge pull request #10270 from jf205/download-db-vs-code
Update documentation about downloading CodeQL databases using the VS Code extension
2 parents 6a0e480 + b427e8a commit 8e0a006

File tree

9 files changed

+37
-73
lines changed

9 files changed

+37
-73
lines changed

docs/codeql/codeql-cli/creating-codeql-databases.rst

Lines changed: 14 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,17 @@ Creating CodeQL databases
55

66
Before you analyze your code using CodeQL, you need to create a CodeQL
77
database containing all the data required to run queries on your code.
8+
You can create CodeQL databases yourself using the CodeQL CLI,
9+
or download them from GitHub.com.
810

911
CodeQL analysis relies on extracting relational data from your code, and
1012
using it to build a :ref:`CodeQL database <codeql-database>`. CodeQL
1113
databases contain all of the important information about a codebase, which can
12-
be analyzed by executing CodeQL queries against it.
14+
be analyzed by executing CodeQL queries against it. GitHub creates and
15+
stores CodeQL databases for a large number of open-source projects. For more information,
16+
see ":ref:`Downloading CodeQL databases from GitHub.com <downloading-codeql-databases-from-github-com>`."
17+
18+
You can also create CodeQL databases yourself using the CodeQL CLI.
1319
Before you generate a CodeQL database, you need to:
1420

1521
- Install and set up the CodeQL CLI. For more information, see
@@ -379,24 +385,15 @@ The following example shows how you could use indirect build tracing in an Azure
379385
# `codeql database analyze`
380386
# then `codeql github upload-results` ...
381387

382-
Obtaining databases from LGTM.com
383-
---------------------------------
384-
385-
`LGTM.com <https://lgtm.com>`__ analyzes thousands of open-source projects using
386-
CodeQL. For each project on LGTM.com, you can download an archived CodeQL
387-
database corresponding to the most recently analyzed revision of the code. These
388-
databases can also be analyzed using the CodeQL CLI or used with the CodeQL
389-
extension for Visual Studio Code.
390-
391-
.. include:: ../reusables/download-lgtm-database.rst
392-
393-
.. pull-quote::
394-
395-
Note
396-
397-
.. include:: ../reusables/index-files-note.rst
388+
.. _downloading-codeql-databases-from-github-com:
398389

390+
Downloading databases from GitHub.com
391+
-------------------------------------
399392

393+
.. include:: ../reusables/download-github-database.rst
394+
395+
Before running an analysis with the CodeQL CLI, you must unzip the databases.
396+
400397
Further reading
401398
---------------
402399

docs/codeql/codeql-cli/getting-started-with-the-codeql-cli.rst

Lines changed: 4 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -47,22 +47,11 @@ Conditions <https://securitylab.github.com/tools/codeql/license>`__.
4747

4848
.. pull-quote:: Important
4949

50-
There are different versions of the CLI available to download, depending
50+
There are several versions of the CLI available to download, depending
5151
on your use case:
5252

53-
- If you want to use the most up to date CodeQL tools and features, download the
54-
version tagged ``latest``.
55-
56-
- If you want to create CodeQL databases to upload to LGTM Enterprise, download
57-
the version that is compatible with the relevant LGTM Enterprise version
58-
number. Compatibility information is included in the description for each
59-
release on the `CodeQL CLI releases page
60-
<https://github.com/github/codeql-cli-binaries/releases>`__ on GitHub. Using the
61-
correct version of the CLI ensures that your CodeQL databases are
62-
compatible with your version of LGTM Enterprise. For more information,
63-
see `Preparing CodeQL databases to upload to LGTM
64-
<https://help.semmle.com/lgtm-enterprise/admin/help/prepare-database-upload.html>`__
65-
in the LGTM admin help.
53+
* If you want to use the most up to date CodeQL tools and features, download the version tagged ``latest``.
54+
* If you want to generate code scanning data to upload to GitHub Enterprise server, then download the version that is compatible with the CodeQL CLI used in your CI system. For more information, see "`Installing CodeQL CLI in your CI system <https://docs.github.com/en/enterprise-server/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system#downloading-the-codeql-cli>`__" in the GitHub documentation.
6655

6756
If you use Linux, Windows, or macOS version 10.14 ("Mojave") or earlier, simply
6857
`download the zip archive
@@ -232,15 +221,7 @@ see ":doc:`About CodeQL packs <about-codeql-packs>`."
232221
- For the most up to date CodeQL queries, check out the ``main`` branch.
233222
This branch represents the very latest version of CodeQL's analysis.
234223

235-
- For the queries used in a particular LGTM Enterprise release, check out the
236-
branch tagged with the relevant release number. For example, the branch
237-
tagged ``v1.27.0`` corresponds to LGTM Enterprise 1.27. You must use this
238-
version if you want to upload data to LGTM Enterprise. For further
239-
information, see `Preparing CodeQL databases to upload to LGTM
240-
<https://help.semmle.com/lgtm-enterprise/admin/help/prepare-database-upload.html>`__
241-
in the LGTM admin help.
242-
243-
1. Extract the zip archive
224+
4. Extract the zip archive
244225
~~~~~~~~~~~~~~~~~~~~~~~~~~
245226

246227
For Linux, Windows, and macOS users (version 10.14 "Mojave", and earlier)

docs/codeql/codeql-for-visual-studio-code/analyzing-your-projects.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ To analyze a project, you need to add a :ref:`CodeQL database <codeql-database>`
1414

1515
#. Open the CodeQL Databases view in the sidebar.
1616

17-
#. Hover over the **Databases** title bar and click the appropriate icon to add your database. You can add a database from a local ZIP archive or folder, from a public URL, or from a project slug or URL on LGTM.com.
17+
#. Hover over the **Databases** title bar and click the appropriate icon to add your database. You can add a database from a local ZIP archive or folder, from a public URL, or from a project's URL on GitHub.com.
1818

1919
.. image:: ../images/codeql-for-visual-studio-code/choose-database.png
2020
:width: 350
@@ -31,14 +31,14 @@ If you have a CodeQL database saved locally, as an unarchived folder or as a ZIP
3131

3232
- To create a database with the CodeQL CLI, see ":ref:`Creating CodeQL databases <creating-codeql-databases>`."
3333

34-
- .. include:: ../reusables/download-lgtm-database.rst
35-
3634
- To analyze a test database, add a ``.testproj`` folder to the Databases view.
3735
Test databases (that is, folders with a ``.testproj`` extension) are generated when you run regression tests on custom queries using the :ref:`CodeQL CLI <codeql-cli>`.
3836
If a query fails a regression test, you may want to analyze the test database in Visual Studio Code to debug the failure.
3937

4038
For more information about running query tests, see ":ref:`Testing custom queries <testing-custom-queries>`" in the CodeQL CLI help.
4139

40+
.. include:: ../reusables/download-github-database.rst
41+
4242
Running a query
4343
------------------------
4444

docs/codeql/codeql-for-visual-studio-code/customizing-settings.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ Editing settings
2828
Choosing a version of the CodeQL CLI
2929
--------------------------------------
3030

31-
The CodeQL extension uses the CodeQL CLI to run commands. If you already have the CLI installed and added to your ``PATH``, the extension uses that version. This might be the case if you create your own CodeQL databases instead of downloading them from LGTM.com. Otherwise, the extension automatically manages access to the executable of the CLI for you. For more information about creating databases, see ":ref:`Creating CodeQL databases <creating-codeql-databases>`" in the CLI help.
31+
The CodeQL extension uses the CodeQL CLI to run commands. If you already have the CLI installed and added to your ``PATH``, the extension uses that version. This might be the case if you create your own CodeQL databases instead of downloading them from GitHub.com. Otherwise, the extension automatically manages access to the executable of the CLI for you. For more information about creating databases, see ":ref:`Creating CodeQL databases <creating-codeql-databases>`" in the CLI help.
3232

3333
To override the default behavior and use a different CLI, you can specify the CodeQL CLI **Executable Path**.
3434

docs/codeql/codeql-for-visual-studio-code/setting-up-codeql-in-visual-studio-code.rst

Lines changed: 1 addition & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ Configuring access to the CodeQL CLI
2828

2929
The extension uses the CodeQL CLI to compile and run queries.
3030

31-
If you already have the CLI installed and added to your ``PATH``, the extension uses that version. This might be the case if you create your own CodeQL databases instead of downloading them from LGTM.com. For more information, see ":ref:`CodeQL CLI <codeql-cli>`."
31+
If you already have the CLI installed and added to your ``PATH``, the extension uses that version. This might be the case if you create your own CodeQL databases instead of downloading them from GitHub.com. For more information, see ":ref:`CodeQL CLI <codeql-cli>`."
3232

3333
Otherwise, the extension automatically manages access to the executable of the CLI for you. This ensures that the CLI is compatible with the CodeQL extension. You can also check for updates with the **CodeQL: Check for CLI Updates** command.
3434

@@ -57,20 +57,6 @@ There are two ways to do this:
5757

5858
For CLI users there is a third option: If you have followed the instructions in ":ref:`Getting started with the CodeQL CLI <getting-started-with-the-codeql-cli>`" to create a CodeQL directory (for example ``codeql-home``) containing the CodeQL libraries, you can open this directory in VS Code. This also gives the extension access to the CodeQL libraries.
5959

60-
.. container:: toggle
61-
62-
.. container:: name
63-
64-
**Click to show information for LGTM Enterprise users**
65-
66-
Your local version of the CodeQL queries and libraries should match your version of LGTM Enterprise. For example, if you
67-
use LGTM Enterprise 1.27, then you should clone the ``1.27.0`` branch of the `starter workspace <https://github.com/github/vscode-codeql-starter/>`__ (or the appropriate ``1.27.x`` branch, corresponding to each maintenance release).
68-
69-
This ensures that the queries and libraries you write in VS Code also work in the query console on LGTM Enterprise.
70-
71-
If you prefer to add the CodeQL queries and libraries to an :ref:`existing workspace <existing-workspace>` instead of the starter workspace, then you should
72-
clone the appropriate branch of the `CodeQL repository <https://github.com/github/codeql>`__ and add it to your workspace.
73-
7460
.. _starter-workspace:
7561

7662
Using the starter workspace
Loading
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
GitHub stores CodeQL databases for over 200,000 repos on GitHub.com, which you can download using the REST API. The list of repos is constantly growing and evolving to make sure that it includes the most interesting codebases for security research.
2+
3+
You can check if a repository has any CodeQL databases available for download using the ``/repos/<owner>/<repo>/code-scanning/codeql/databases`` endpoint.
4+
For example, to check for CodeQL databases using the `GitHub CLI <https://cli.github.com/manual/gh_api>`__ you would run::
5+
6+
gh api /repos/<owner>/<repo>/code-scanning/codeql/databases/
7+
8+
This command returns information about any CodeQL databases that are available for a repository, including the language the database represents, and when the database was last updated. If no CodeQL databases are available, the response is empty.
9+
10+
When you have confirmed that a CodeQL database exists for the language you are interested in, you can download it using the following command::
11+
12+
gh api /repos/<owner>/<repo>/code-scanning/codeql/databases/<language> -H 'Accept: application/zip' > path/to/local/database.zip
13+
14+
For more information, see the documentation for the `Get CodeQL database <https://docs.github.com/en/rest/reference/code-scanning#get-codeql-database>`__ endpoint in the GitHub REST API documentation.

docs/codeql/reusables/download-lgtm-database.rst

Lines changed: 0 additions & 6 deletions
This file was deleted.

docs/codeql/reusables/index-files-note.rst

Lines changed: 0 additions & 8 deletions
This file was deleted.

0 commit comments

Comments
 (0)