Merge remote-tracking branch 'upstream/develop' into update-license-rules

Author: pombredanne

CHANGELOG.rst

@@ -12,7 +12,27 @@ v33.0.0 (next next, roadmap)
 - Fallback packages for non-native dependencies of SCTK.
 - Dependencies for
 - Support for copyright detection objects.
-- Bump commoncode to v31.0.3
+
+- A new field in packages with the license category for the
+  detected license expression and also an API function to
+  compute license categories from license expressions.
+  See https://github.com/nexB/scancode-toolkit/issues/2897
+
+- More support for tabular output formats: New command-line
+  options for XSLX output, and the old `--csv` command line
+  option is removed.
+  See https://github.com/nexB/scancode-toolkit/issues/830
+
+- `--unknown-licenses` is removed and this is always enabled
+  and only used in case of improper detections automatically.
+  Also tag all license rules with required phrases to improve
+  license detection and reduce false positives.
+  See https://github.com/nexB/scancode-toolkit/issues/3300
+
+- File categorization support added, a post scan plugin tagging
+  files with priority levels for review, and also take advantage
+  of these in other summary plugins.
+  See https://github.com/nexB/scancode-toolkit/issues/1745
 
 v32.1.0 (next, roadmap)
 ----------------------------
@@ -68,41 +88,33 @@ Changes in Output Data Structure:
     referenced_filenames, and the boolean attributes (like
     is_license_notice, is_license_intro etc, as applicable).
 
-- A new field in packages with the license category for the
-  detected license expression and also an API function to
-  compute license categories from license expressions.
-  See https://github.com/nexB/scancode-toolkit/issues/2897
-
-- More support for tabular output formats: New command-line
-  options for XSLX output, and the old `--csv` command line
-  option is removed.
-  See https://github.com/nexB/scancode-toolkit/issues/830
-
-- `--unknown-licenses` is removed and this is always enabled
-  and only used in case of improper detections automatically.
-  Also tag all license rules with required phrases to improve
-  license detection and reduce false positives.
-  See https://github.com/nexB/scancode-toolkit/issues/3300
-
-- A new `--todo` option is added to show the todo items that
-  should be reviewed, which are ambiguous license/package
-  detections.
-
-- File categorization support added, a post scan plugin tagging
-  files with priority levels for review, and also take advantage
-  of these in other summary plugins.
-  See https://github.com/nexB/scancode-toolkit/issues/1745
-
 - New and updated licenses, including support for newly released
   SPDX license list version 3.22. This release of the SPDX license
   list had 48 new licenses, and several of them we already had as
   licenses/rules, and these has been modified to be consistent with
   the SPDX list. And the rest have been added as new licenses.
   For more details see https://github.com/nexB/scancode-toolkit/pull/3554
 
+- Improve debian namespace detection based on clues and fix
+  namespace and qualifier bugs for debian purls.
+  For more details see https://github.com/nexB/scancode.io/issues/899
+  and https://github.com/nexB/scancode-toolkit/issues/3443
+  Also improve debian manifests parsing and purl parsing from
+  filenames. Support for https://github.com/nexB/purldb/issues/245
+  Bumps debian-inspector to v31.1.0 
+
+- Bump commoncode to v31.0.3
+
 - Upgraded spdx-tools dependency to v0.8.
   See https://github.com/nexB/scancode-toolkit/issues/3455
 
+Support for Conan package parser:
+
+- We now support the parsing of Conan manifest files, such as
+  `conanfile.py`, as described here https://docs.conan.io/2.0/reference/conanfile.html.
+  We also support source extraction from `conandata.yml`, as described here
+  https://docs.conan.io/2/tutorial/creating_packages/handle_sources_in_packages.html#using-the-conandata-yml-file.
+
 
 v32.0.8 - 2023-10-11
 ------------------------

docs/source/reference/available_package_parsers.rst

@@ -177,6 +177,18 @@ parsers in scancode-toolkit during documentation builds.
      - ``php_composer_lock``
      - PHP
      - https://getcomposer.org/doc/01-basic-usage.md#commit-your-composer-lock-file-to-version-control
+   * - conan external source
+     - ``*/conandata.yml``
+     - ``conan``
+     - ``conan_conandata_yml``
+     - C++
+     - https://docs.conan.io/2/tutorial/creating_packages/handle_sources_in_packages.html#using-the-conandata-yml-file
+   * - conan recipe
+     - ``*/conanfile.py``
+     - ``conan``
+     - ``conan_conanfile_py``
+     - C++
+     - https://docs.conan.io/2.0/reference/conanfile.html
    * - Conda meta.yml manifest
      - ``*/meta.yaml``
      - ``conda``
@@ -243,6 +255,13 @@ parsers in scancode-toolkit during documentation builds.
      - ``debian_copyright_in_source``
      - None
      - https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+   * - Debian machine readable file standalone
+     - ``*/copyright``
+       ``*_copyright``
+     - ``deb``
+     - ``debian_copyright_standalone``
+     - None
+     - https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
    * - Debian binary package archive
      - ``*.deb``
      - ``deb``

requirements.txt

@@ -13,7 +13,7 @@ commoncode==31.0.3
 construct==2.10.68
 container-inspector==31.1.0
 cryptography==37.0.4
-debian-inspector==31.0.0
+debian-inspector==31.1.0
 dockerfile-parse==1.2.0
 dparse2==0.7.0
 extractcode==31.0.0

setup-mini.cfg

@@ -72,7 +72,7 @@ install_requires =
     colorama >= 0.3.9
     commoncode >= 31.0.2
     container-inspector >= 31.0.0
-    debian-inspector >= 31.0.0
+    debian-inspector >= 31.1.0
     dparse2 >= 0.7.0
     fasteners
     fingerprints >= 0.6.0

setup.cfg

@@ -72,7 +72,7 @@ install_requires =
     colorama >= 0.3.9
     commoncode >= 31.0.3
     container-inspector >= 31.0.0
-    debian-inspector >= 31.0.0
+    debian-inspector >= 31.1.0
     dparse2 >= 0.7.0
     fasteners
     fingerprints >= 0.6.0

src/licensedcode/data/licenses/bcrypt-solar-designer.LICENSE

@@ -0,0 +1,30 @@
+---
+key: bcrypt-solar-designer
+short_name: bcrypt Solar Designer License
+name: bcrypt Solar Designer License
+category: Permissive
+owner: Openwall
+notes: |
+    This was previously the openwall-md5-permissive_8.RULE
+spdx_license_key: bcrypt-Solar-Designer
+other_urls:
+    - https://github.com/bcrypt-ruby/bcrypt-ruby/blob/master/ext/mri/crypt_blowfish.c
+ignorable_copyrights:
+    - Copyright (c) 1998-2014 Solar Designer
+ignorable_holders:
+    - Solar Designer
+ignorable_authors:
+    - Solar Designer
+---
+
+Written by Solar Designer <solar at openwall.com> in 1998-2014.
+No copyright is claimed, and the software is hereby placed in the public
+domain.  In case this attempt to disclaim copyright and place the software
+in the public domain is deemed null and void, then the software is
+Copyright (c) 1998-2014 Solar Designer and it is hereby released to the
+general public under the following terms:
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted.
+
+There's ABSOLUTELY NO WARRANTY, express or implied.

src/licensedcode/data/licenses/bison-exception-2.0.LICENSE

@@ -5,8 +5,12 @@ name: Bison exception to GPL 2.0 or later
 category: Copyleft Limited
 owner: Free Software Foundation (FSF)
 is_exception: yes
-spdx_license_key: LicenseRef-scancode-bison-exception-2.0
+spdx_license_key: Bison-exception-1.24
+other_spdx_license_keys:
+    - LicenseRef-scancode-bison-exception-2.0
 faq_url: http://www.gnu.org/software/bison/manual/bison.html#Conditions
+other_urls:
+    - https://github.com/arineng/rwhoisd/blob/master/rwhoisd/mkdb/y.tab.c#L180
 standard_notice: |
     This library is free software; you can redistribute it and/or modify it
     under the terms of the GNU General Public License as published by the Free

src/licensedcode/data/licenses/brian-gladman.LICENSE

@@ -5,9 +5,14 @@ name: Brian Gladman License
 category: Permissive
 owner: Brian Gladman
 homepage_url: http://www.gladman.me.uk/AES
-spdx_license_key: LicenseRef-scancode-brian-gladman
+spdx_license_key: Brian-Gladman-2-Clause
+other_spdx_license_keys:
+    - LicenseRef-scancode-brian-gladman
 text_urls:
     - http://gladman.plushost.co.uk/oldsite/AES/aes-src-12-09-11.zip
+other_urls:
+    - https://web.mit.edu/kerberos/krb5-1.21/doc/mitK5license.html
+    - https://github.com/krb5/krb5/blob/krb5-1.21.2-final/NOTICE#L140-L156
 ---
 
 The redistribution and use of this software (with or without changes)

src/licensedcode/data/licenses/bsd-simplified-darwin.LICENSE

@@ -4,7 +4,11 @@ short_name: BSD Simplified Darwin
 name: BSD Simplified Darwin
 category: Permissive
 owner: Ian Darwin
-spdx_license_key: LicenseRef-scancode-bsd-simplified-darwin
+spdx_license_key: BSD-2-Clause-Darwin
+other_spdx_license_keys:
+    - LicenseRef-scancode-bsd-simplified-darwin
+other_urls:
+    - https://github.com/file/file/blob/master/COPYING
 ---
 
 This software is not subject to any export provision of the United States

src/licensedcode/data/licenses/bsd-systemics-w3works.LICENSE

@@ -0,0 +1,73 @@
+---
+key: bsd-systemics-w3works
+short_name: Systemics W3Works License
+name: Systemics W3Works BSD variant license
+category: Permissive
+owner: Systemics Ltd
+spdx_license_key: BSD-Systemics-W3Works
+other_urls:
+    - https://metacpan.org/release/DPARIS/Crypt-Blowfish-2.14/source/COPYRIGHT#L7
+ignorable_authors:
+    - Systemics Ltd (http://www.systemics.com/)
+    - W3Works, LLC (http://www.w3works.com)
+ignorable_urls:
+    - http://www.systemics.com/
+    - http://www.w3works.com/
+---
+
+Current implimentation contains modifications made by W3Works, LLC.  The 
+modifications remain copyright of W3Works, LLC and attribution for these 
+modification should be made to W3Works, LLC.  These modifications and 
+this copyright must remain with this package.
+ 
+Additions to the Restrictions set out below are:
+1. All advertising materials mentioning features or use of this software
+   must display the following acknowledgement:
+   This product inculdes software developed by W3Works, LLC (http://www.w3works.com)
+ 
+   NO ADDITIONAL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+   THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+   ARE EXTENDED BY THIS DISTRIBUTION.
+ 
+   Any subsequent derrivations of this package must retainl this copyright.
+ 
+ 
+Original Copyright Below
+ 
+This library and applications are FREE FOR COMMERCIAL AND NON-COMMERCIAL USE
+as long as the following conditions are adhered to.
+ 
+Copyright remains with Systemics Ltd, and as such any Copyright notices
+in the code are not to be removed.  If this code is used in a product,
+Systemics should be given attribution as the author of the parts used.
+This can be in the form of a textual message at program startup or
+in documentation (online or textual) provided with the package.
+ 
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+1. Redistributions of source code must retain the copyright
+   notice, this list of conditions and the following disclaimer.
+2. Redistributions in binary form must reproduce the above copyright
+   notice, this list of conditions and the following disclaimer in the
+   documentation and/or other materials provided with the distribution.
+3. All advertising materials mentioning features or use of this software
+   must display the following acknowledgement:
+   This product includes software developed by Systemics Ltd (http://www.systemics.com/)   
+ 
+   THIS SOFTWARE IS PROVIDED BY SYSTEMICS LTD ``AS IS'' AND
+   ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+   IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+   ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
+   FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+   DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+   OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+   HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+   LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+   OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+   SUCH DAMAGE.
+ 
+   The licence and distribution terms for any publically available version or
+   derivative of this code cannot be changed.  i.e. this code cannot simply be
+   copied and put under another distribution licence
+   [including the GNU Public Licence.]