Skip to content

Improve documentation #4467

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: develop
Choose a base branch
from
Open

Improve documentation #4467

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b9954a6
Added note on recommended process number for optimal performance #2980
chinyeungli Jul 7, 2025
a4d9219
Removed duplicate content from "All Available Options"
chinyeungli Jul 7, 2025
bc1c655
Enhance description of the `--from-json` option #2562
chinyeungli Jul 7, 2025
6c46ab4
Update documentation format
chinyeungli Jul 10, 2025
3c251f0
Update note for ``--processess`` option
chinyeungli Jul 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/source/cli-reference/core-options.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,20 @@ Comparing Progress Message Options
This inputs the scan results from ``sample.json``, runs the post-scan plugin ``--classify`` and
outputs the results for this scan to ``sample_2.json``.

You can also convert the input .json file to other output format
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
You can also convert the input .json file to other output format
You can also convert the input ``.json`` file to other output format

without using any scan option.

For instance,::

scancode --from-json sample.json --yaml sample.yml

The above command takes ``sample.json`` as input and generates file in .yml formats.

.. note::

This only works for options that require only the input JSON file;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally the post-scan options (like --summary, --todo) are configured to just operate on the codebase (with top-level licenses/packages etc and all resource level results) and this should work fine with --from-json
The core scan options (like --license --info) and pre-scan options cannot be run here. There are some post-scan options which have required options as dependencies, so those also need to be there in the loaded scan if those options are being specified.

Copy link
Contributor Author

@chinyeungli chinyeungli Jul 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AyanSinhaMahapatra I am wondering is it true that all the "Pre-Scan" options cannot be run with --from-json?
https://scancode-toolkit.readthedocs.io/en/latest/cli-reference/list-options.html#all-pre-scan-options
It seems to be the "--ignore" and "--include" don't really need the code files to have it work (I didn't look at the source code to verify it). Also, https://scancode-toolkit.readthedocs.io/en/latest/cli-reference/core-options.html#from-json-option , the current --from-json` is usung --classify`` as a sample.

For the post-scans, I want to verify that if an option's dependencies are missing, the command still executes without errors, it just won’t return results related to the missing dependencies, correct?

options that rely on other files or source code will not work.

----

.. _max_in_memory_option:
Expand Down
16 changes: 2 additions & 14 deletions docs/source/cli-reference/list-options.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
All Available Options
=====================

This section contains an exhaustive list of all ScanCode options, arranged in various sections.
This section contains an exhaustive list of all ScanCode-Toolkit options,
arranged in various sections.
The sections are as follows:

- Basic Scan Options
Expand All @@ -13,25 +14,12 @@ The sections are as follows:
- Pre-Scan Options
- Post-Scan Options

There's also another section for ``extractcode`` options.

The order of the sections and all their options is the same as in the :ref:`cli_help_text`,
available in the command line.

.. _cli_basic:

.. include:: /rst_snippets/basic_options.rst

----

.. include:: /rst_snippets/extract.rst
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we want to remove these sections, they are pretty useful IMHO

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AyanSinhaMahapatra I'm removing this from the list-options since extract.rst is already included in other-commands.rst. Having it in both places seem redundant.


----

.. include:: /rst_snippets/scancode-reindex-licenses.rst

----

.. include:: /rst_snippets/core_options.rst

----
Expand Down
9 changes: 9 additions & 0 deletions docs/source/rst_snippets/core_options.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,12 @@ All "Core" Scan Options
including and below the starting point. INTEGER
must be positive or zero for no limit.
[Default: 0]


.. note::

For optimal performance, it's recommended to set the `-n, --processes`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chinyeungli this was merged as default with #4104
See also docs above which was changed:

[Default: (number of CPUs)-1]

This explanation is nice, but we need to modify this a bit.
We could also say it's safer to use (number of CPUs)-2 for local machines with other applications (browser/IDEs/chatrooms etc) open, or even less after looking at current CPU usage.

We should probably have a seperate section on performance and explanations, but this can be done a bit more explicitly as a sub-task of #4069 eith detailed explanations and recommendations on CPU/memory

option to "number of CPU cores minus 1".
For example, on a quad-core/eight-thread machine, use ``-n 7``.


Loading