Merge branch 'dev' into static-group

Author: LeoOMaia

.github/workflows/close-stale.yml

@@ -0,0 +1,27 @@
+name: Close Stale Issues and PRs
+
+on:
+  schedule:
+    # Run daily at 02:00 UTC
+    - cron: '0 2 * * *'
+  workflow_dispatch:
+    # Allow manual triggering
+
+permissions:
+  issues: write
+  pull-requests: write
+
+jobs:
+  close-stale:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Close stale issues and PRs
+        uses: actions/stale@v9
+        with:
+          # Disable automatic stale marking - only close manually labeled items
+          days-before-stale: -1
+          days-before-close: 7
+          stale-issue-label: 'stale'
+          stale-pr-label: 'stale'
+          close-issue-message: 'This issue has been automatically closed because it was manually labeled as stale. If you believe this was closed in error, please reopen it and remove the stale label.'
+          close-pr-message: 'This PR has been automatically closed because it was manually labeled as stale. If you believe this was closed in error, please reopen it and remove the stale label.'

.gitignore

@@ -147,3 +147,4 @@ docs/.devcontainer/devcontainer.json
 docs/.devcontainer/Dockerfile
 docs/LICENSE
 docs/.hugo_build.lock
+.cursor-rules

docker-compose.yml

@@ -120,7 +120,7 @@ services:
           source: ./docker/extra_settings
           target: /app/docker/extra_settings
   postgres:
-    image: postgres:17.5-alpine@sha256:ec01646a894c7c5aec8a29e0fe723ab093e24e2e055940090d282d8ab1c79d26
+    image: postgres:17.5-alpine@sha256:6567bca8d7bc8c82c5922425a0baee57be8402df92bae5eacad5f01ae9544daa
     environment:
       POSTGRES_DB: ${DD_DATABASE_NAME:-defectdojo}
       POSTGRES_USER: ${DD_DATABASE_USER:-defectdojo}
@@ -129,7 +129,7 @@ services:
       - defectdojo_postgres:/var/lib/postgresql/data
   redis:
     # Pinning to this version due to licensing constraints
-    image: redis:7.2.10-alpine@sha256:b49011d66bd28754587d44ac9079d6088086fd6063cb7c554b77a5c2b2cbec1c
+    image: redis:7.2.10-alpine@sha256:395ccd7ee4db0867de0d0410f4712a9e0331cff9fdbd864f71ec0f7982d3ffe6
     volumes:
       - defectdojo_redis:/data
 volumes:

docs/content/en/about_defectdojo/faq.md

@@ -48,7 +48,7 @@ There are two different methods to import a report from a security tool into Def
 - **Import** handles the report as a single point-in-time record.  Importing a report creates a Test within DefectDojo that holds the Findings rendered from that report.
 - **Reimport** is used to extend an existing Test.  If you have a more open-ended approach to your testing process, you continuously Reimport the latest version of your report to an existing Test.  DefectDojo will compare the results of the incoming report to your existing data, record any changes, and then adjust the Findings in the Test so that they match the latest report.
 
-Both methods also use **Deduplication** differently: while two discrete Imported Tests in the same Product will identify and label duplicate Findings, Reimport will discard duplicate Findings altogether.
+Both methods also use **Deduplication** differently: while two discrete Imported Tests in the same Product will identify and label duplicate Findings, Reimport will skip duplicates in uploaded reports as theses Findings already exist in Defect Dojo.
 
 Generally speaking - if a point-in-time report is what you need, Import is the best method to use.  If you are continuously running and ingesting reports from a tool, Reimport is the better method for keeping things organized.
 

docs/content/en/open_source/archived_docs/usage/features.md

@@ -386,15 +386,12 @@ details about the deduplication process : switch
 
 ### Deduplication - APIv2 parameters
 
-- `close_old_findings` : if true, findings that are not
-    duplicates and that were in the previous scan of the same type
-    (example ZAP) for the same engagement (or product in case of
-    \"close_old_findings_product_scope\") and that are not present in the new
-    scan are closed (Inactive, Verified, Mitigated). 
-- `close_old_findings_product_scope` : if true, close_old_findings applies
-    to all findings of the same type in the product. Note that
-    \"Deduplication on engagement\" is no longer used to determine the
-    scope of close_old_findings.
+| Parameter | Import behaviour | Reimport Behaviour |
+|-----------|------------------|-------------------|
+| `close_old_findings` | if `true`, findings that are not duplicates and that were in the previous scan of the same type (example ZAP) for the same **engagement** (or product in case of `close_old_findings_product_scope`) and that are not present in the new scan are closed (`Inactive`, `Verified`, `Mitigated`). | if `true`, findings that that are in the same **test** and that are not present in the new scan are closed (`Inactive`, `Verified`, `Mitigated`) |
+| `close_old_findings_product_scope` | if true, `close_old_findings` applies to all findings of the same type in the whole **product**. Note that "Deduplication on engagement" is no longer used to determine the scope of `close_old_findings` | has no effect | 
+
+The `close_old_findings` feature will respect the value of the `service` field to only close findings with an identical `service` value.
 
 ### Deduplication / Similar findings
 

docs/content/en/open_source/upgrading/2.48.2.md

@@ -0,0 +1,9 @@
+---
+title: 'Upgrading to DefectDojo Version 2.48.2'
+toc_hide: true
+weight: -20250602
+description: Tag invalid character cleanup
+---
+
+## Tag Formatting Update
+In [2.46.0](../2.46.md) tag validation was added to disallow commas, spaces and quotes in tags. Some parsers were still creating tags with invalid characters. This is fixed in this release and this release will run another data migration to replace any invalid character in tag with an underscore '`_`'.

dojo/api_v2/serializers.py

@@ -2302,14 +2302,17 @@ class ImportScanSerializer(CommonImportScanSerializer):
     close_old_findings = serializers.BooleanField(
         required=False,
         default=False,
-        help_text="Select if old findings no longer present in the report get closed as mitigated when importing. "
-        "If service has been set, only the findings for this service will be closed.",
+        help_text="Old findings no longer present in the new report get closed as mitigated when importing. "
+                    "If service has been set, only the findings for this service will be closed. "
+                    "This only affects findings within the same engagement.",
     )
     close_old_findings_product_scope = serializers.BooleanField(
         required=False,
         default=False,
-        help_text="Select if close_old_findings applies to all findings of the same type in the product. "
-        "By default, it is false meaning that only old findings of the same type in the engagement are in scope.",
+        help_text="Old findings no longer present in the new report get closed as mitigated when importing. "
+                    "If service has been set, only the findings for this service will be closed. "
+                    "This only affects findings within the same product."
+                    "By default, it is false meaning that only old findings of the same type in the engagement are in scope.",
     )
     version = serializers.CharField(
         required=False, help_text="Version that was scanned.",
@@ -2380,15 +2383,15 @@ class ReImportScanSerializer(CommonImportScanSerializer):
     # also for ReImport.
     close_old_findings = serializers.BooleanField(
         required=False,
-        default=True,
-        help_text="Select if old findings no longer present in the report get closed as mitigated when importing.",
+        default=False,
+        help_text="Old findings no longer present in the new report get closed as mitigated when importing. "
+                    "If service has been set, only the findings for this service will be closed. "
+                    "This only affects findings within the same test.",
     )
     close_old_findings_product_scope = serializers.BooleanField(
         required=False,
         default=False,
-        help_text="Select if close_old_findings applies to all findings of the same type in the product. "
-        "By default, it is false meaning that only old findings of the same type in the engagement are in scope. "
-        "Note that this only applies on the first call to reimport-scan.",
+        help_text="This has no effect on reimport",
     )
     version = serializers.CharField(
         required=False,

dojo/db_migrations/0234_alter_system_settings_maximum_password_length_and_more.py

@@ -0,0 +1,24 @@
+# Generated by Django 5.1.8 on 2025-07-17 20:45
+
+import django.core.validators
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('dojo', '0233_remove_test_actual_time_remove_test_estimated_time'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='system_settings',
+            name='maximum_password_length',
+            field=models.IntegerField(default=48, help_text='Requires user to set passwords less than maximum length.', validators=[django.core.validators.MinValueValidator(9), django.core.validators.MaxValueValidator(48)], verbose_name='Maximum password length'),
+        ),
+        migrations.AlterField(
+            model_name='system_settings',
+            name='minimum_password_length',
+            field=models.IntegerField(default=9, help_text='Requires user to set passwords greater than minimum length.', validators=[django.core.validators.MinValueValidator(9), django.core.validators.MaxValueValidator(48)], verbose_name='Minimum password length'),
+        ),
+    ]

dojo/db_migrations/0235_alter_system_settings_time_zone_zoneinfo.py

@@ -7,7 +7,7 @@
 class Migration(migrations.Migration):
 
     dependencies = [
-        ('dojo', '0234_finding_cvssv4_finding_cvssv4_score'),
+        ('dojo', '0234_alter_system_settings_maximum_password_length_and_more'),
     ]
 
     operations = [

dojo/db_migrations/0236_clean_tags.py

@@ -0,0 +1,123 @@
+# Generated by Django 5.0.8 on 2024-09-12 18:22
+
+import logging
+from django.db import migrations
+from django.db.models import Q
+
+logger = logging.getLogger(__name__)
+
+# Only apply the process to models that _could_ have tags
+model_names = [
+    "Product",
+    "Endpoint",
+    "Engagement",
+    "Test",
+    "Finding",
+    "Finding_Template",
+    "App_Analysis",
+    "Objects_Product",
+]
+
+
+def clean_tag_value(tag: str) -> str:
+    """
+    Clean each tag value by:
+    - Converting all commas to hyphens
+    - Converting all spaces to underscores
+    - Removing all single/double quotes
+    """
+    return tag.replace(",", "-").replace(" ", "_").replace('"', "").replace("'", "")
+
+
+def clean_all_tag_fields(apps, schema_editor):
+    """
+    Cleans tag values for all models in the `model_names` list, removing unwanted characters.
+    Updates both 'tags' and 'inherited_tags' fields where applicable.
+    """
+    updated_count = {}
+    for model_name in model_names:
+        TaggedModel = apps.get_model("dojo", model_name)
+        unique_tags_per_model = {}
+        count_per_model = 0
+        # Only fetch the objects with tags that contain a character in violation
+        queryset = (
+            TaggedModel.objects.filter(
+                Q(**{"tags__name__icontains": ","})
+                | Q(**{"tags__name__icontains": " "})
+                | Q(**{"tags__name__icontains": '"'})
+                | Q(**{"tags__name__icontains": "'"})
+            )
+            .distinct()
+            .prefetch_related("tags")
+        )
+        # Iterate over each instance to clean the tags. The iterator is used here
+        # to prevent loading the entire queryset into memory at once. Instead, we
+        # will only process 500 objects at a time
+        for instance in queryset.iterator(chunk_size=500):
+            # Get the current list of tags to work with
+            raw_tags = instance.tags.all()
+            # Clean each tag here while preserving the original value
+            cleaned_tags = {tag.name: clean_tag_value(tag.name) for tag in raw_tags}
+            # Quick check to avoid writing things without impact
+            if cleaned_tags:
+                instance.tags.set(list(cleaned_tags.values()), clear=True)
+                count_per_model += 1
+                # Update the running list of cleaned tags with the changes on this model
+                unique_tags_per_model.update(cleaned_tags)
+            # Add a quick logging statement every 100 objects cleaned
+            if count_per_model > 0 and count_per_model % 100 == 0:
+                logger.info(
+                    f"{TaggedModel.__name__}.tags: cleaned {count_per_model} tags..."
+                )
+        # Update the final count of the tags cleaned for the given model
+        if count_per_model:
+            updated_count[f"{TaggedModel.__name__}"] = (
+                count_per_model,
+                unique_tags_per_model,
+            )
+    """
+    Write a helpful statement about what tags were changed for each model in the list.
+    It looks something like this:
+
+    Product: 1 instances cleaned
+      "quoted string with spaces" -> quoted_string_with_spaces
+      "quoted with spaces, and also commas!" -> quoted_with_spaces-_and_also_commas!
+      "quoted,comma,tag" -> quoted-comma-tag
+    Engagement: 1 instances cleaned
+      "quoted string with spaces" -> quoted_string_with_spaces
+      "quoted with spaces, and also commas!" -> quoted_with_spaces-_and_also_commas!
+      "quoted,comma,tag" -> quoted-comma-tag
+    Test: 1 instances cleaned
+      "quoted string with spaces" -> quoted_string_with_spaces
+      "quoted with spaces, and also commas!" -> quoted_with_spaces-_and_also_commas!
+      "quoted,comma,tag" -> quoted-comma-tag
+    Finding: 1 instances cleaned
+      "quoted string with spaces" -> quoted_string_with_spaces
+      "quoted with spaces, and also commas!" -> quoted_with_spaces-_and_also_commas!
+      "quoted,comma,tag" -> quoted-comma-tag
+    """
+    for key, (count, tags) in updated_count.items():
+        logger.info(f"{key}: {count} instances cleaned")
+        for old, new in tags.items():
+            if old != new:
+                logger.info(f"  {old} -> {new}")
+
+
+def cannot_turn_back_time(apps, schema_editor):
+    """
+    We cannot possibly return to the original state without knowing
+    the original value at the time the migration is revoked. Instead
+    we will do nothing.
+    """
+    pass
+
+
+class Migration(migrations.Migration):
+    dependencies = [
+        ('dojo', '0235_alter_system_settings_time_zone_zoneinfo'),
+    ]
+
+    operations = [
+        migrations.RunPython(clean_all_tag_fields, cannot_turn_back_time),
+    ]
+