Merge branch 'bugfix' into wizcli-improvements

Author: OsamaMahmood

.dryrunsecurity.yaml

@@ -52,21 +52,21 @@ sensitiveCodepaths:
   - 'docker/entrypoint-celery-beat.sh'
   - 'docker/entrypoint-celery-worker.sh'
   - 'docker/entrypoint-initializer.sh'
+  - 'docker/entrypoint-first-boot.sh'
   - 'docker/entrypoint-nginx.sh'
   - 'docker/entrypoint-uwsgi.sh'
   - 'docker/wait-for-it.sh'
 allowedAuthors:
   usernames:
     - mtesauro
     - devGregA
-    - grendel513
     - cneill
     - Maffooch
     - blakeaowens
     - kiblik
     - dsever
     - dogboat
     - hblankenship
+    - valentijnscholten
 notificationList:
   - '@mtesauro'
-  - '@grendel513'

.github/CODEOWNERS

@@ -0,0 +1,19 @@
+# Any kind of package updates only need 2 approvals,
+# So let's add three folks here
+requirements.txt @cneill @mtesauro @Maffooch
+# Any dockerfile or compose changes will need to be viewed by
+# these people
+Dockerfile.* @mtesauro @Maffooch 
+docker-compose.* @mtesauro @Maffooch 
+/docker/ @mtesauro @Maffooch 
+# Documentation changes 
+/docs/content/ @paulOsinski @valentijnscholten @Maffooch
+# Kubernetes should be reviewed by reviewed first by those that know it
+/helm/ @cneill @kiblik @Maffooch
+# Anything UI related needs to be checked out by those with the eye for it
+/dojo/static/ @blakeaowens @Maffooch
+/dojo/templates/ @blakeaowens @Maffooch
+# Any model changes should be closely looked at
+/dojo/models.py @Maffooch
+# All other code changes should be reviewed by someone
+* @Maffooch @mtesauro

.github/ISSUE_TEMPLATE/bug_report.md

@@ -33,10 +33,11 @@ A clear and concise description of what you expected to happen.
 
 **Environment information**
  - Operating System: [e.g. Ubuntu 18.04]
+ - Docker Compose or Helm version (Output of `docker compose version` or `helm version`)
  - DefectDojo version (see footer) or commit message: [use `git show -s --format="[%ci] %h: %s [%d]"`]
 
 **Logs** 
-Use `docker-compose logs` (or similar, depending on your deployment method) to get the logs and add the relevant sections here showing the error occurring (if applicable).
+Use `docker compose logs` (or similar, depending on your deployment method) to get the logs and add the relevant sections here showing the error occurring (if applicable).
 
 **Sample scan files**
 If applicable, add sample scan files to help reproduce your problem.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -8,7 +8,7 @@ assignees: ''
 ---
 ## :warning: Note on feature completeness :warning:
 
-We are narrowing the scope of acceptable enhancements to DefectDojo in preparation for v3. Learn more here:
+We are narrowing the scope of acceptable enhancements to DefectDojo. Learn more here:
 https://github.com/DefectDojo/django-DefectDojo/blob/master/readme-docs/CONTRIBUTING.md
 
 **Is your feature request related to a problem? Please describe**

.github/ISSUE_TEMPLATE/support_request.md

@@ -7,7 +7,7 @@ assignees: ''
 
 ---
 **Slack us first!**
-The easiest and fastest way to help you is via Slack. There's a free and easy signup to join our #defectdojo channel in the OWASP Slack workspace: [Get Access.](https://owasp-slack.herokuapp.com/)
+The easiest and fastest way to help you is via Slack. There's a free and easy signup to join our #defectdojo channel in the OWASP Slack workspace: [Get Access.](https://owasp.org/slack/invite)
 If you're confident you've found a bug, or are allergic to Slack, you can submit an issue anyway.
 
 **Be informative**
@@ -33,10 +33,11 @@ A clear and concise description of what you expected to happen.
 
 **Environment information**
  - Operating System: [e.g. Ubuntu 18.04]
+ - Docker Compose or Helm version (Output of `docker compose version` or `helm version`)
  - DefectDojo version (see footer) or commit message: [use `git show -s --format="[%ci] %h: %s [%d]"`]
 
 **Logs** 
-Use `docker-compose logs` (or similar, depending on your deployment method) to get the logs and add the relevant sections here showing the error occurring (if applicable).
+Use `docker compose logs` (or similar, depending on your deployment method) to get the logs and add the relevant sections here showing the error occurring (if applicable).
 
 **Sample scan files**
 If applicable, add sample scan files to help reproduce your problem.

.github/labeler.yml

@@ -60,3 +60,8 @@ localization:
     - any-glob-to-any-file:
       - dojo/locale/*
       - dojo/locale/**/*
+
+lint:
+  - changed-files:
+    - any-glob-to-any-file:
+      - ruff.toml

.github/pr-reminder.py

@@ -0,0 +1,220 @@
+import logging
+import os
+from datetime import datetime, timedelta
+
+import requests
+
+logger = logging.getLogger(__name__)
+
+# Set up the GitHub and Slack tokens from environment variables
+GH_TOKEN = os.getenv("GH_TOKEN")
+SLACK_TOKEN = os.getenv("SLACK_TOKEN")
+REPO_OWNER = "DefectDojo"
+REPO_NAME = "django-DefectDojo"
+GITHUB_USER_NAME_TO_SLACK_EMAIL = {
+    "Maffooch": "cody@defectdojo.com",
+    "mtesauro": "matt@defectdojo.com",
+    "devGregA": "greg@defectdojo.com",
+    "blakeaowens": "blake@defectdojo.com",
+    "dogboat": "sean@defectdojo.com",
+    "cneill": "charles@defectdojo.com",
+    "hblankenship": "harold@defectdojo.com",
+    "valentijnscholten": "valentijn@defectdojo.com",
+}
+
+
+# Helper function to calculate the prior Thursday from a given date
+def get_prior_thursday(date: datetime) -> str:
+    # Calculate the day of the week (0=Monday, 1=Tuesday, ..., 6=Sunday)
+    weekday = date.weekday()
+    # If today is Thursday (weekday 3), return the same day.
+    if weekday == 3:
+        return date
+    # Calculate how many days to subtract to reach the most recent Thursday
+    days_to_subtract = (
+        weekday - 3
+    ) % 7  # (weekday - 3) gives the number of days past Thursday
+    prior_thursday = date - timedelta(days=days_to_subtract)
+
+    return prior_thursday.strftime("%Y-%m-%d")
+
+
+# Helper function to get Slack User ID from Slack Email
+def get_slack_user_id(slack_email: str) -> int:
+    headers = {"Authorization": f"Bearer {SLACK_TOKEN}"}
+    params = {"email": slack_email}
+    response = requests.get(
+        "https://slack.com/api/users.lookupByEmail",
+        headers=headers,
+        params=params,
+        timeout=30,
+    )
+
+    if response.status_code != 200 or not response.json().get("ok"):
+        logger.info(f"Error fetching Slack user ID for email {slack_email}: {response.text}")
+        return None
+
+    return response.json().get("user", {}).get("id")
+
+
+# Helper function to fetch pull requests from GitHub
+def get_pull_requests() -> dict:
+    headers = {"Authorization": f"token {GH_TOKEN}"}
+    response = requests.get(
+        f"https://api.github.com/repos/{REPO_OWNER}/{REPO_NAME}/pulls",
+        headers=headers,
+        timeout=30,
+    )
+
+    if response.status_code != 200:
+        logger.info(f"Error fetching PRs: {response.text}")
+        response.raise_for_status()
+
+    return response.json()
+
+
+# Helper function to get PR reviews (approved, changes requested, or pending)
+def get_pr_reviews(pull_request: dict) -> list[dict]:
+    pr_number = pull_request["number"]
+    reviews_url = f"https://api.github.com/repos/{REPO_OWNER}/{REPO_NAME}/pulls/{pr_number}/reviews"
+    headers = {"Authorization": f"token {GH_TOKEN}"}
+    response = requests.get(
+        reviews_url,
+        headers=headers,
+        timeout=30,
+    )
+
+    if response.status_code != 200:
+        logger.info(f"Error fetching reviews for PR {pr_number}: {response.text}")
+        return []
+
+    reviews = response.json()
+    # Dictionary to store the latest review for each user
+    latest_reviews = {}
+    # Iterate over each review to find the latest one for each user
+    for review in reviews:
+        user = review["user"]["login"]
+        submitted_at = review["submitted_at"]
+        state = review["state"]
+        # Convert the submitted_at timestamp to a datetime object for comparison
+        review_time = datetime.strptime(submitted_at, "%Y-%m-%dT%H:%M:%SZ")
+        # If the user doesn't have a review or the current one is later, update
+        if user not in latest_reviews or (
+            review_time > latest_reviews[user]["submitted_at"] and state != "COMMENTED"
+        ):
+            latest_reviews[user] = {
+                "user": user,
+                "state": state,
+                "submitted_at": review_time,
+                "url": review["html_url"],
+            }
+    # Determine if there are any pending reviewers
+    latest_reviews.update(
+        {
+            user_dict.get("login"): {
+                "user": user_dict.get("login"),
+                "state": "PENDING",
+            }
+            for user_dict in pull_request.get("requested_reviewers", [])
+        },
+    )
+    # Return the latest review state and URL for each user
+    return latest_reviews.values()
+
+
+# Helper function to send a message via Slack
+def send_slack_message(slack_user_id: int, message: str) -> None:
+    headers = {
+        "Content-Type": "application/json",
+        "Authorization": f"Bearer {SLACK_TOKEN}",
+    }
+    payload = {"channel": slack_user_id, "text": message}
+    response = requests.post(
+        "https://slack.com/api/chat.postMessage",
+        json=payload,
+        headers=headers,
+        timeout=30,
+    )
+
+    if response.status_code != 200 or not response.json().get("ok"):
+        logger.info(f"Error sending Slack message: {response.text}")
+        response.raise_for_status()
+
+
+# Helper function to format the PR message with review statuses
+def format_pr_message(pull_request: dict, reviews: list[dict]) -> str:
+    repo_name = pull_request["head"]["repo"]["name"]
+    pull_request_title = pull_request["title"]
+    pull_request_url = pull_request["html_url"]
+    pull_request_number = pull_request["number"]
+    constructed_title = f"{repo_name} (#{pull_request_number}): {pull_request_title}"
+    message = f"• <{pull_request_url}|{constructed_title}>"
+    # Fetch the milestone due date and URL
+    if (milestone := pull_request.get("milestone")) is not None and (
+        (milestone_due_date := milestone.get("due_on"))
+        and (milestone_url := milestone.get("html_url"))
+        and (milestone_title := milestone.get("title"))
+    ):
+        message += f"\n    Merge by: {get_prior_thursday(datetime.strptime(milestone_due_date, '%Y-%m-%dT%H:%M:%SZ'))} for release <{milestone_url}|{milestone_title}>"
+    # Format reviews and append to the message (only latest review status per user)
+    message += "\n    Review Status:\n"
+    for review in reviews:
+        user = review["user"]
+        state = review["state"]
+        if url := review.get("url"):
+            message += f"    • {user}: <{url}|{state.lower().capitalize()}>\n"
+        else:
+            message += f"    • {user}: {state.lower().capitalize()}\n"
+
+    return message
+
+
+# Main function to process PRs and notify Slack users
+def notify_reviewers():
+    try:
+        user_pr_map = {}
+        slack_email_to_slack_id = {}
+        pull_requests = get_pull_requests()
+        # Logging all fetched PR details
+        logger.info(f"Fetched {len(pull_requests)} PRs from GitHub.")
+        for pull_request in pull_requests:
+            title = pull_request["title"]
+            pr_number = pull_request["number"]
+            logger.info(f"Processing PR: {pr_number} - {title}")
+            reviews = get_pr_reviews(pull_request)
+            logger.info(f"Found {len(reviews)} reviews for PR {pr_number}.")
+            message = format_pr_message(pull_request, reviews)
+            # Map Slack users to PR messages
+            for review in reviews:
+                github_username = review["user"]
+                if github_username not in user_pr_map:
+                    user_pr_map[github_username] = ""
+                # Determine if we should prune any non pending reviews
+                if f"{github_username}: Pending" in message:
+                    user_pr_map[github_username] += message + "\n"
+        # Add the Header at the beginning of the list
+        header_message = "Here are the PRs that are still requiring review:"
+        # Add Tips and Tricks at the end of the list
+        tips_message = "*Tips and Tricks*\n"
+        tips_message += (
+            "• This is how to remove a PR from the list: Approve, Request changes, or leave a general comment.\n"
+            "• If someone else has requested changes, then leave a general comment to remove the pending review from yourself."
+        )
+        # Send Slack messages to reviewers
+        for github_username, pr_list in user_pr_map.items():
+            if pr_list:
+                if slack_email := GITHUB_USER_NAME_TO_SLACK_EMAIL.get(github_username):
+                    if slack_user_id := slack_email_to_slack_id.get(
+                        slack_email,
+                        get_slack_user_id(slack_email),
+                    ):
+                        message_content = f"Hello {github_username}! {header_message}\n{pr_list}\n{tips_message}"
+                        # logger.info("\n\n", message_content, "\n\n")
+                        send_slack_message(slack_user_id, message_content)
+    except Exception as e:
+        logger.info(f"Error occurred: {e}")
+        raise
+
+
+if __name__ == "__main__":
+    notify_reviewers()

.github/release-drafter.yml

@@ -41,6 +41,8 @@ categories:
     label: 'ui'
   - title: '🗣 Updates in localization'
     label: 'localization'
+  - title: '🔧 Improved code quality with linters'
+    label: 'lint'
   - title: '🧰 Maintenance'
     collapse-after: 3
     labels:
@@ -51,7 +53,7 @@ exclude-labels:
 
 change-template: '- $TITLE @$AUTHOR (#$NUMBER)'
 template: |
-  Please consult the [Upgrade notes in the documentation ](https://documentation.defectdojo.com/getting_started/upgrading/) for specific instructions for this release, and general upgrade instructions. Below is an automatically generated list of all PRs merged since the previous release.
+  Please consult the [Upgrade notes in the documentation ](https://docs.defectdojo.com/en/open_source/upgrading/upgrading_guide/) for specific instructions for this release, and general upgrade instructions. Below is an automatically generated list of all PRs merged since the previous release.
 
   ## Changes since $PREVIOUS_TAG
   $CHANGES

.github/renovate.json

@@ -6,6 +6,7 @@
   "dependencyDashboardApproval": false,
   "baseBranches": ["dev"],
   "rebaseWhen": "conflicted",
+  "separateMinorPatch": true,
   "ignorePaths": ["requirements.txt", "requirements-lint.txt", "components/package.json", "components/package-lock.json", "dojo/components/yarn.lock", "dojo/components/package.json", "Dockerfile**"],
   "ignoreDeps": [],
   "packageRules": [{