From 3fb4520083b34198ea2bd80f857d10f47228cd8e Mon Sep 17 00:00:00 2001 From: Chin Yeung Li Date: Mon, 7 Jul 2025 09:05:03 +0800 Subject: [PATCH 1/4] Updated extractcode text and title for consistency Signed-off-by: Chin Yeung Li --- docs/source/rst_snippets/extract.rst | 67 +++++++++++++++++++++------- 1 file changed, 50 insertions(+), 17 deletions(-) diff --git a/docs/source/rst_snippets/extract.rst b/docs/source/rst_snippets/extract.rst index 59d3b337ab8..97e9a990581 100644 --- a/docs/source/rst_snippets/extract.rst +++ b/docs/source/rst_snippets/extract.rst @@ -1,5 +1,53 @@ -All Extractcode Options ------------------------ +``extractcode`` Usage +--------------------- + +Usage: extractcode [OPTIONS] + + extract archives and compressed files in the file or directory tree. + + Archives found inside an extracted archive are extracted recursively. Use + --shallow for a shallow extraction. Extraction for each archive is done in- + place in a new directory named '-extract' created side- + by-side with an archive. + +Options: + +.. _cli_extract: + + --verbose Print verbose file-by-file progress messages. + --quiet Do not print any summary or progress message. + --shallow Do not extract recursively nested archives in archives. + --replace-originals Replace extracted archives by the extracted content. + --ignore TEXT Ignore files/directories matching this glob pattern. + --all-formats Extract archives from all known formats. The default is + to extract only the common format of these kinds: + "regular", "regular_nested" and "package". To show all + supported formats use the option --list-formats . + --list-formats Show the list of supported archive and compressed file + formats and exit. + -h, --help Show this message and exit. + --about Show information about ExtractCode and its licensing + and exit. + --version Show the version and exit. + +Examples: + + (Note for Windows: use '\' backslash instead of '/' slash for paths.) + + Extract all archives found in the 'samples' directory tree: + + extractcode samples + + Note: If an archive contains other archives, all contained archives will be + extracted recursively. Extraction is done directly in the 'samples' + directory, side-by-side with each archive. Files are extracted in a + directory named after the archive with an '-extract' suffix added to its + name, created side-by-side with the corresponding archive file. + + Extract a single archive. Files are extracted in the directory + 'samples/arch/zlib.tar.gz-extract/': + + extractcode samples/arch/zlib.tar.gz This is intended to be used as an input preparation step, before running the scan. Archives found in an extracted archive are extracted **recursively** by default. Extraction is done in-place @@ -14,18 +62,3 @@ To extract the packages in the ``samples`` directory This extracts the zlib.tar.gz package: .. image:: /rst_snippets/data/extractcode.png - -.. _cli_extract: - ---shallow Do not extract recursively nested archives (e.g. Not - archives in archives). - ---verbose Print verbose file-by-file progress messages. - ---quiet Do not print any summary or progress message. - --h, --help Show the extractcode help message and exit. - ---about Show information about ScanCode and licensing and exit. - ---version Show the version and exit. From 39a9eed7089ea21108f9f19c7111c3c8352bf5d1 Mon Sep 17 00:00:00 2001 From: Chin Yeung Li Date: Mon, 7 Jul 2025 09:09:51 +0800 Subject: [PATCH 2/4] Updated `scancode-reindex-licenses` text and reordered options Signed-off-by: Chin Yeung Li --- .../scancode-reindex-licenses.rst | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/source/rst_snippets/scancode-reindex-licenses.rst b/docs/source/rst_snippets/scancode-reindex-licenses.rst index 83304762c21..8472a0b357b 100644 --- a/docs/source/rst_snippets/scancode-reindex-licenses.rst +++ b/docs/source/rst_snippets/scancode-reindex-licenses.rst @@ -1,23 +1,23 @@ ``scancode-reindex-licenses`` Usage ----------------------------------- -Usage: ``scancode-reindex-licenses [OPTIONS]`` +Usage: scancode-reindex-licenses [OPTIONS] -Reindex scancode licenses and exit - -Options -------- + Reindex scancode licenses and exit +Options: --all-languages [EXPERIMENTAL] Rebuild the license index including texts all languages (and not only English) and exit. --only-builtin Rebuild the license index excluding any additional license directory or additional - license plugins which were added previously, i.e. - with only builtin scancode license and rules. + license plugins which were added previously, + i.e. with only builtin scancode license and + rules. + --load-dump Load all the license objects from file and then + dump them back to their respective license + files. --additional-directory DIR Include this directory with additional custom licenses and license rules in the license detection index. - --load-dump Load all license and rules from their respective - files and then dump them back to those same files. - -h, --help Shows the options and explanations. + -h, --help Show this message and exit. From 67f7634c790749e303517c729929565da691f8bc Mon Sep 17 00:00:00 2001 From: Chin Yeung Li Date: Mon, 7 Jul 2025 09:13:37 +0800 Subject: [PATCH 3/4] Added documents for scancode-license-data #4024 This commit also alphabetizes the options list and reorders `scancode-reindex-licenses` options to match the help output. Signed-off-by: Chin Yeung Li --- docs/source/cli-reference/other-commands.rst | 100 ++++++++++++------ .../rst_snippets/scancode-license-data.rst | 12 +++ 2 files changed, 77 insertions(+), 35 deletions(-) create mode 100644 docs/source/rst_snippets/scancode-license-data.rst diff --git a/docs/source/cli-reference/other-commands.rst b/docs/source/cli-reference/other-commands.rst index 53c56f01301..a88799a6aa7 100644 --- a/docs/source/cli-reference/other-commands.rst +++ b/docs/source/cli-reference/other-commands.rst @@ -5,16 +5,45 @@ Other available CLIs ---- -.. include:: /rst_snippets/scancode-reindex-licenses.rst +.. include:: /rst_snippets/extract.rst ---- -.. include:: /rst_snippets/extract.rst +.. include:: /rst_snippets/scancode-license-data.rst + + +``--path`` Option: +^^^^^^^^^^^^^^^^^^ + +The ``--path`` option defines the directory where the license data will be stored. + +Here is an example of dumping license data with the ``--path DIR`` option:: + + scancode-license-data --path ~/licenses + + Dumping license data to: /home/user/licenses + Done dumping #2465 licenses. + +Each of the licenses contains four files: '.LICENSE', '.html', '.json', and '.yml'. + +The dumped licenses directory look like this:: + + licenses/ + ├── 389-exception.LICENSE + ├── 389-exception.html + ├── 389-exception.json + ├── 389-exception.yml + ├── 3com-microcodeLICENSE + ├── 3com-microcode.html + ├── 3com-microcode.json + ├── 3com-microcode.yml + . + . + . ---- -``scancode-reindex-licenses`` command -------------------------------------- +.. include:: /rst_snippets/scancode-reindex-licenses.rst ScanCode maintains a license index to search for and detect licenses. When ScanCode is configured for the first time, a license index is built and used in every scan thereafter. @@ -26,6 +55,38 @@ displays the following message to the terminal:: This has several CLI options as follows: + +``--all-languages`` Option: +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Rebuild the license index including texts all languages (and not only +English) and exit. This is an EXPERIMENTAL option. + + +``--only-builtin`` Option: +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Rebuild the license index excluding any additional license directory or additional +license plugins which were added previously, i.e. with only builtin scancode license and rules. + +This is applicable when there are additional license plugins installed already and you want to +reindex the licenses without these licenses from the additional plugins. + +.. note:: + + Running the ``--only-builtin`` command won't get rid of the installed license plugins, it + would just reindex without the licenses from these plugins for once. Another reindex afterwards + without this option would bring back the licenses from the plugins again in the index. + + +``--load-dump`` Option +^^^^^^^^^^^^^^^^^^^^^^ + +Load all licenses and rules from their respective files and then dump them +to their respective files. This is done to make small formatting changes across +all licenses and rules, to be consistent across them. + + ``--additional-directory`` Option: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -77,34 +138,3 @@ need to reindex the license index with those directories as parameters:: better reproducibility and distribution of those license/rules for use in conjunction with scancode-toolkit licenses. See :ref:`install_new_license_plugin` - - -``--only-builtin`` Option: -^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Rebuild the license index excluding any additional license directory or additional -license plugins which were added previously, i.e. with only builtin scancode license and rules. - -This is applicable when there are additional license plugins installed already and you want to -reindex the licenses without these licenses from the additional plugins. - -.. note:: - - Running the ``--only-builtin`` command won't get rid of the installed license plugins, it - would just reindex without the licenses from these plugins for once. Another reindex afterwards - without this option would bring back the licenses from the plugins again in the index. - - -``--all-languages`` Option: -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Rebuild the license index including texts all languages (and not only -English) and exit. This is an EXPERIMENTAL option. - - -``--load-dump`` Option -^^^^^^^^^^^^^^^^^^^^^^ - -Load all licenses and rules from their respective files and then dump them -to their respective files. This is done to make small formatting changes across -all licenses and rules, to be consistent across them. diff --git a/docs/source/rst_snippets/scancode-license-data.rst b/docs/source/rst_snippets/scancode-license-data.rst new file mode 100644 index 00000000000..0a5815d2140 --- /dev/null +++ b/docs/source/rst_snippets/scancode-license-data.rst @@ -0,0 +1,12 @@ +``scancode-license-data`` Usage +------------------------------- + +Usage: scancode-license-data [OPTIONS] + + Dump scancode license data in various formats, and the licenseDB static + website at `path`. + +Options: + --path DIR Dump the license data in this directory in the LicenseDB format + and exit. Creates the directory if it does not exist. + -h, --help Show this message and exit. From 38a89f417ba6b2bb874a53ca23ce336fd757a0f0 Mon Sep 17 00:00:00 2001 From: Chin Yeung Li Date: Thu, 10 Jul 2025 11:11:26 +0800 Subject: [PATCH 4/4] Update doc fomat and reorganize content location Signed-off-by: Chin Yeung Li --- docs/source/cli-reference/other-commands.rst | 29 ---------------- .../rst_snippets/scancode-license-data.rst | 33 ++++++++++++++++++- 2 files changed, 32 insertions(+), 30 deletions(-) diff --git a/docs/source/cli-reference/other-commands.rst b/docs/source/cli-reference/other-commands.rst index a88799a6aa7..666f755d661 100644 --- a/docs/source/cli-reference/other-commands.rst +++ b/docs/source/cli-reference/other-commands.rst @@ -12,35 +12,6 @@ Other available CLIs .. include:: /rst_snippets/scancode-license-data.rst -``--path`` Option: -^^^^^^^^^^^^^^^^^^ - -The ``--path`` option defines the directory where the license data will be stored. - -Here is an example of dumping license data with the ``--path DIR`` option:: - - scancode-license-data --path ~/licenses - - Dumping license data to: /home/user/licenses - Done dumping #2465 licenses. - -Each of the licenses contains four files: '.LICENSE', '.html', '.json', and '.yml'. - -The dumped licenses directory look like this:: - - licenses/ - ├── 389-exception.LICENSE - ├── 389-exception.html - ├── 389-exception.json - ├── 389-exception.yml - ├── 3com-microcodeLICENSE - ├── 3com-microcode.html - ├── 3com-microcode.json - ├── 3com-microcode.yml - . - . - . - ---- .. include:: /rst_snippets/scancode-reindex-licenses.rst diff --git a/docs/source/rst_snippets/scancode-license-data.rst b/docs/source/rst_snippets/scancode-license-data.rst index 0a5815d2140..ef68e918172 100644 --- a/docs/source/rst_snippets/scancode-license-data.rst +++ b/docs/source/rst_snippets/scancode-license-data.rst @@ -1,7 +1,7 @@ ``scancode-license-data`` Usage ------------------------------- -Usage: scancode-license-data [OPTIONS] +Usage: ``scancode-license-data [OPTIONS]`` Dump scancode license data in various formats, and the licenseDB static website at `path`. @@ -10,3 +10,34 @@ Options: --path DIR Dump the license data in this directory in the LicenseDB format and exit. Creates the directory if it does not exist. -h, --help Show this message and exit. + + +``--path`` Option: +^^^^^^^^^^^^^^^^^^ + +The ``--path`` option is mandatory and defines the directory where the +license data will be stored. + +Here is an example of dumping license data with the ``--path DIR`` option:: + + scancode-license-data --path ~/licenses + + Dumping license data to: /home/user/licenses + Done dumping #2465 licenses. + +Each of the licenses contains four files: '.LICENSE', '.html', '.json', and '.yml'. + +The dumped licenses directory look like this:: + + licenses/ + ├── 389-exception.LICENSE + ├── 389-exception.html + ├── 389-exception.json + ├── 389-exception.yml + ├── 3com-microcodeLICENSE + ├── 3com-microcode.html + ├── 3com-microcode.json + ├── 3com-microcode.yml + . + . + .