Skip to content

Documents for scancode-license-data #4024 #4461

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 36 additions & 35 deletions docs/source/cli-reference/other-commands.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,16 @@ Other available CLIs

----

.. include:: /rst_snippets/scancode-reindex-licenses.rst
.. include:: /rst_snippets/extract.rst

----

.. include:: /rst_snippets/extract.rst
.. include:: /rst_snippets/scancode-license-data.rst


----

``scancode-reindex-licenses`` command
-------------------------------------
.. include:: /rst_snippets/scancode-reindex-licenses.rst

ScanCode maintains a license index to search for and detect licenses. When ScanCode is
configured for the first time, a license index is built and used in every scan thereafter.
Expand All @@ -26,6 +26,38 @@ displays the following message to the terminal::

This has several CLI options as follows:


``--all-languages`` Option:
^^^^^^^^^^^^^^^^^^^^^^^^^^^

Rebuild the license index including texts all languages (and not only
English) and exit. This is an EXPERIMENTAL option.


``--only-builtin`` Option:
^^^^^^^^^^^^^^^^^^^^^^^^^^

Rebuild the license index excluding any additional license directory or additional
license plugins which were added previously, i.e. with only builtin scancode license and rules.

This is applicable when there are additional license plugins installed already and you want to
reindex the licenses without these licenses from the additional plugins.

.. note::

Running the ``--only-builtin`` command won't get rid of the installed license plugins, it
would just reindex without the licenses from these plugins for once. Another reindex afterwards
without this option would bring back the licenses from the plugins again in the index.


``--load-dump`` Option
^^^^^^^^^^^^^^^^^^^^^^

Load all licenses and rules from their respective files and then dump them
to their respective files. This is done to make small formatting changes across
all licenses and rules, to be consistent across them.


``--additional-directory`` Option:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Expand Down Expand Up @@ -77,34 +109,3 @@ need to reindex the license index with those directories as parameters::
better reproducibility and distribution of those license/rules
for use in conjunction with scancode-toolkit licenses.
See :ref:`install_new_license_plugin`


``--only-builtin`` Option:
^^^^^^^^^^^^^^^^^^^^^^^^^^

Rebuild the license index excluding any additional license directory or additional
license plugins which were added previously, i.e. with only builtin scancode license and rules.

This is applicable when there are additional license plugins installed already and you want to
reindex the licenses without these licenses from the additional plugins.

.. note::

Running the ``--only-builtin`` command won't get rid of the installed license plugins, it
would just reindex without the licenses from these plugins for once. Another reindex afterwards
without this option would bring back the licenses from the plugins again in the index.


``--all-languages`` Option:
^^^^^^^^^^^^^^^^^^^^^^^^^^^

Rebuild the license index including texts all languages (and not only
English) and exit. This is an EXPERIMENTAL option.


``--load-dump`` Option
^^^^^^^^^^^^^^^^^^^^^^

Load all licenses and rules from their respective files and then dump them
to their respective files. This is done to make small formatting changes across
all licenses and rules, to be consistent across them.
67 changes: 50 additions & 17 deletions docs/source/rst_snippets/extract.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,53 @@
All Extractcode Options
-----------------------
``extractcode`` Usage
---------------------

Usage: extractcode [OPTIONS] <input>

extract archives and compressed files in the <input> file or directory tree.

Archives found inside an extracted archive are extracted recursively. Use
--shallow for a shallow extraction. Extraction for each archive is done in-
place in a new directory named '<archive file name>-extract' created side-
by-side with an archive.

Options:

.. _cli_extract:

--verbose Print verbose file-by-file progress messages.
--quiet Do not print any summary or progress message.
--shallow Do not extract recursively nested archives in archives.
--replace-originals Replace extracted archives by the extracted content.
--ignore TEXT Ignore files/directories matching this glob pattern.
--all-formats Extract archives from all known formats. The default is
to extract only the common format of these kinds:
"regular", "regular_nested" and "package". To show all
supported formats use the option --list-formats .
--list-formats Show the list of supported archive and compressed file
formats and exit.
-h, --help Show this message and exit.
--about Show information about ExtractCode and its licensing
and exit.
--version Show the version and exit.

Examples:

(Note for Windows: use '\' backslash instead of '/' slash for paths.)

Extract all archives found in the 'samples' directory tree:

extractcode samples

Note: If an archive contains other archives, all contained archives will be
extracted recursively. Extraction is done directly in the 'samples'
directory, side-by-side with each archive. Files are extracted in a
directory named after the archive with an '-extract' suffix added to its
name, created side-by-side with the corresponding archive file.

Extract a single archive. Files are extracted in the directory
'samples/arch/zlib.tar.gz-extract/':

extractcode samples/arch/zlib.tar.gz

This is intended to be used as an input preparation step, before running the scan. Archives found
in an extracted archive are extracted **recursively** by default. Extraction is done in-place
Expand All @@ -14,18 +62,3 @@ To extract the packages in the ``samples`` directory
This extracts the zlib.tar.gz package:

.. image:: /rst_snippets/data/extractcode.png

.. _cli_extract:

--shallow Do not extract recursively nested archives (e.g. Not
archives in archives).

--verbose Print verbose file-by-file progress messages.

--quiet Do not print any summary or progress message.

-h, --help Show the extractcode help message and exit.

--about Show information about ScanCode and licensing and exit.

--version Show the version and exit.
43 changes: 43 additions & 0 deletions docs/source/rst_snippets/scancode-license-data.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
``scancode-license-data`` Usage
-------------------------------

Usage: ``scancode-license-data [OPTIONS]``

Dump scancode license data in various formats, and the licenseDB static
website at `path`.

Options:
--path DIR Dump the license data in this directory in the LicenseDB format
and exit. Creates the directory if it does not exist.
-h, --help Show this message and exit.


``--path`` Option:
^^^^^^^^^^^^^^^^^^

The ``--path`` option is mandatory and defines the directory where the
license data will be stored.

Here is an example of dumping license data with the ``--path DIR`` option::

scancode-license-data --path ~/licenses

Dumping license data to: /home/user/licenses
Done dumping #2465 licenses.

Each of the licenses contains four files: '.LICENSE', '.html', '.json', and '.yml'.

The dumped licenses directory look like this::

licenses/
├── 389-exception.LICENSE
├── 389-exception.html
├── 389-exception.json
├── 389-exception.yml
├── 3com-microcodeLICENSE
├── 3com-microcode.html
├── 3com-microcode.json
├── 3com-microcode.yml
.
.
.
20 changes: 10 additions & 10 deletions docs/source/rst_snippets/scancode-reindex-licenses.rst
Original file line number Diff line number Diff line change
@@ -1,23 +1,23 @@
``scancode-reindex-licenses`` Usage
-----------------------------------

Usage: ``scancode-reindex-licenses [OPTIONS]``
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't CLI scripts/othe CLI commands best be wrapped by `` so they are highlighted differently?

Usage: scancode-reindex-licenses [OPTIONS]

Reindex scancode licenses and exit

Options
-------
Reindex scancode licenses and exit

Options:
--all-languages [EXPERIMENTAL] Rebuild the license index
including texts all languages (and not only
English) and exit.
--only-builtin Rebuild the license index excluding any
additional license directory or additional
license plugins which were added previously, i.e.
with only builtin scancode license and rules.
license plugins which were added previously,
i.e. with only builtin scancode license and
rules.
--load-dump Load all the license objects from file and then
dump them back to their respective license
files.
--additional-directory DIR Include this directory with additional custom
licenses and license rules in the license
detection index.
--load-dump Load all license and rules from their respective
files and then dump them back to those same files.
-h, --help Shows the options and explanations.
-h, --help Show this message and exit.
Loading