added basic documentation

Author: superstes

.gitignore

@@ -9,3 +9,5 @@ dist/
 testdata/custom_*
 scripts/cli_custom.sh
 tmp/
+
+docs/build/

CONTRIBUTE.md

@@ -0,0 +1,16 @@
+# Contribute
+
+Contributions are welcome (:
+
+What would be helpful for now:
+* Feel free to discuss the ideas and roadmap for this project with us: [GitHub discussions](https://github.com/O-X-L/firewall-testing-framework/discussions) or [contact us directly](mailto://contact@oxl.at)
+* Open [issues](https://github.com/O-X-L/firewall-testing-framework/issues) if you think you have found a problem with the existing code (be aware that it might not yet be in a usable state)
+
+* Please do not post any generic AI-slop.. thanks.
+* Be friendly and respectful
+
+----
+
+## Documentation
+
+Please read the developer-documentation: [ftf.oxl.app](https://ftf.oxl.app/dev/1_intro.html)

README.md

@@ -3,139 +3,60 @@
 [![Lint](https://github.com/O-X-L/firewall-testing-framework/actions/workflows/lint.yml/badge.svg?branch=latest)](https://github.com/O-X-L/firewall-testing-framework/actions/workflows/lint.yml)
 [![Test](https://github.com/O-X-L/firewall-testing-framework/actions/workflows/test.yml/badge.svg?branch=latest)](https://github.com/O-X-L/firewall-testing-framework/actions/workflows/test.yml)
 
-A framework for testing and troubleshooting firewall Layer 3-4 rulesets.
-
-**WARNING**: This project is still in the conception-phase.
+A framework for **testing and troubleshooting firewall rulesets**.
 
 ----
 
-## Goal / Why?
-
-When having to administer IT infrastructure and networks - we will often have multiple firewalls in place.
-
-Maintaining these might be time-consuming. You might also face some challenges:
-
-* **Troubleshooting & Analysis**:
-
-  Even for senior network engineers it can be a challenge to find the source of an unexpected block/accept in large rulesets that are distributed across multiple systems and firewall vendors. 
-
-  Infrastructure-as-Code does help to keep the rulesets in a consistent state - but it does not solve the issue of having to manually analyze/troubleshoot existing rulesets.
+## Documentation
 
-  This project wants to provide one interface for simulating traffic over multiple firewall systems.
-
-* **Automated Regression-Tests**:
-
-  Why would you want to do ruleset-regression-tests?
-  * You may want/need to periodically verify that the currently active rulesets actually allow/deny the traffic you expect
-    This can be a tedious task - you might overlook some edge-case.
-  * Especially when a ruleset is administered by teams of engineers over a long time period - it can be a challenge to:
-    * detect configuration errors/mistakes before they can be exploited
-    * make sure the design-choices for the ruleset are adhered to
-
-  * If you already utilize Infrastructure-as-Code and change-reviews for updating your rulesets you might want to also validate the functionality of that ruleset via automated CI-jobs.
-
-  How do regression-tests work?
-  * You define test-cases that simulate traffic over one or multiple firewalls
-  * You assert that the traffic was allowed/denied/rejected
-  * You might even want to assert that the traffic took a specific outbound route or was NATed to a specific IP
-
-  This way you can continuously extend these test-cases and easily verify that the currently active rulesets comply with them.
-
-----
-
-## Idea
-
-Take a look at this topology:
+You can find the documentation at: [ftf.oxl.app](https://ftf.oxl.app)
 
 <img src="https://raw.githubusercontent.com/O-X-L/firewall-testing-framework/refs/heads/latest/docs/source/_static/img/topology.svg" max-width="700"></img>
 
-The flow is planned to be:
-
-1. Either:
-  * manually pull the current config from the existing firewalls
-  * or utilize existing `pull-plugins` to do so (p.e. via API)
-
-2. The vendor-specific configuration gets parsed by `translation-plugins` which output a standardized firewall config-schema.
-
-3. The user provides a high-level `topology-config`
-
-4. If automated tests should be run: The user needs to provide a `test-traffic config`
-
-  Else the user has the option to enter an interactive shell where traffic can be sent manually
-
-5. The `firewall/network simulator`
-
-  * parses the provided config
-  * generates the network-topology
-  * finds where the packet originates from (or notifies the user if more information is required)
-  * finds the route the packet should take
-  * tests the traffic against the rulesets of firewalls that are hops of that route
-
-Thanks already go to @MikPisula, the creator of the [MikPisula/packet-simulator](https://github.com/MikPisula/packet-simulator) for creating a simulator for netfilter (IPTables/NFTables) firewalls.
-
-Also thanks to the [go-ftw (Web Application Firewall Testing Framework) project](https://github.com/coreruleset/go-ftw) that inspired us to support regression-tests.
-
-----
-
-## Principles
-
-* **Strict separation of vendor-specific plugins** from the core traffic-simulator. 
-
-  Plugins CAN be used to pull the current configuration (rulesets, interfaces, routes) from a firewall system, but admins should always be able to manually provide this information.
-
-  Some might not want to trust some 'nice-to-have' tool with access to their firewalls.
-
-* The user should be able to choose the **output verbosity**.
-
-  We want to provide full transparency (*show every rule the traffic interacts with*) but if not required (*p.e in automated/CI-mode*) it should be brief.
-
-----
-
-## Contribute
-
-Contributions are welcome (:
-
-What would be helpful for now:
-* Feel free to discuss the ideas and roadmap for this project with us: [GitHub discussions](https://github.com/O-X-L/firewall-testing-framework/discussions) or [contact us directly](mailto://contact@oxl.at)
-* Open [issues](https://github.com/O-X-L/firewall-testing-framework/issues) if you think you have found a problem with the existing code (be aware that it might not yet be in a usable state)
-
-* Please do not post any generic AI-slop.. thanks.
-* Be friendly and respectful
-
 ----
 
 ## Roadmap
 
 ### 2025
 
 **Core Simulator**:
-* Generating Layer 3 Topology
-* Generating multiple Firewalls
-* Detect Firewall-chaining (one firewall routes to another one - p.e. over VPN)
-* Run modes:
-  * Basic interactive shell
-  * Automated/CI mode
-* Defining basic config-schema (Topology, Rulesets, Tests)
-* Run multiple Test-cases from config (CLI pytest-like?)
-* Option to Output results to JSON
-* Security Features to protect users:
-  * Warn before executing non-verified (code review) plugin
+- [ ] Fundamental Features
+  - [x] Routing
+  - [x] Network Interfaces
+  - [x] Firewall Tables
+  - [x] Firewall Chains
+  - [x] Firewall Rules
+  - [x] System-Specific Translate-Plugins
+  - [x] System-Specific Rule-Matching
+- [ ] Run modes:
+  - [x] One-Shot CLI
+  - [ ] Basic interactive shell
+  - [ ] Automated/CI mode
+    - [ ] Run multiple Test-cases from config (CLI pytest-like?)
+- [ ] Defining basic config-schema (Topology, Rulesets, Tests)
+- [ ] Option to Output results to JSON
+- [ ] Supporting multiple Firewalls
+  - [ ] Generating Layer 3 Topology
+  - [ ] Detect Firewall-chaining (one firewall routes to another one - p.e. over VPN)
 
 **Development**:
-* Create Plugin Templates
-* Create Guide on how to develop Plugins
+- [ ] Create Plugin Templates
+- [ ] Create Guide on how to develop Plugins
 
 **Firewall Support**:
-* Netfilter (NFTables/IPTables)
-* OPNsense (Plugin that parses Config-Backup-File)
+- [ ] Netfilter (NFTables/IPTables)
+- [ ] OPNsense (Information from Config-Backup-File and runtime-infos like routes from API)
+
+----
+
+## Contribute
+
+See: [CONTRIBUTING](https://github.com/O-X-L/firewall-testing-framework/blob/latest/CONTRIBUTING.md)
 
 ----
 
-### What will be out-of-scope for now
+## Credits
 
-Why? Because we initially need to focus on building the core simulator!
+* Thanks to the [go-ftw (Web Application Firewall Testing Framework) project](https://github.com/coreruleset/go-ftw) that inspired us to create this project
 
-* Transparent firewalls (layer 2 interception)
-* Application-Level Protocols
-* Connection-Tracking helpers (rules that use these CT-states)
-* Non-static routing (dynamic routing, rule-based routing via fwmark and routing-table lookup)
+* Thanks go to [@MikPisula](https://github.com/MikPisula) for some inspiration on how to simulate network-traffic over a firewall ([MikPisula/packet-simulator](https://github.com/MikPisula/packet-simulator))

docs/html.sh

@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+
+set -e
+
+cd "$(dirname "$0")"
+
+rm -rf build
+mkdir build
+
+sphinx-build -b html source/ build/

docs/html_infra.sh

@@ -0,0 +1,72 @@
+#!/bin/bash
+
+if [ -z "$1" ]
+then
+  DEST_DIR='build'
+else
+  DEST_DIR="$1"
+fi
+
+set -euo pipefail
+
+function log() {
+  msg="$1"
+  echo ''
+  echo "### ${msg} ###"
+  echo ''
+}
+
+cd "$(dirname "$0")"
+
+SRC_DIR="$(pwd)"
+
+TS="$(date +%s)"
+TMP_DIR="/tmp/${TS}"
+mkdir -p "${TMP_DIR}"
+
+VENV_BIN='/tmp/.oxl-docs-venv/bin/activate'
+if [ -f "$VENV_BIN" ]
+then
+  source "$VENV_BIN"
+fi
+
+log 'BUILDING DOCS'
+export PYTHONWARNINGS='ignore'
+sphinx-build -b html source/ "${TMP_DIR}/" >/dev/null
+
+log 'PATCHING METADATA'
+cp "${SRC_DIR}/meta/"* "${TMP_DIR}/"
+
+HTML_META_SRC="<meta charset=\"utf-8\" />"
+HTML_META="${HTML_META_SRC}<meta http-equiv=\"Content-Security-Policy\" content=\"default-src 'self'; img-src 'self' https://files.oxl.at data:; style-src 'self' https://files.oxl.at 'unsafe-inline'; script-src 'self' https://files.oxl.at 'unsafe-inline' 'unsafe-eval'; connect-src 'self' https://api.github.com;\">"
+HTML_META="${HTML_META}<link rel=\"icon\" type=\"image/webp\" href=\"https://files.oxl.at/img/oxl3_sm.webp\">"
+HTML_META_EN="${HTML_META}"  # <link rel=\"alternate\" href=\"https://docs.o-x-l.at\" hreflang=\"de\">
+# HTML_LOGO_LINK_SRC='href=".*Go to homepage"'
+# HTML_LOGO_LINK_EN='href="https://www.o-x-l.com" class="oxl-nav-logo" title="OXL IT Services Website"'
+HTML_LANG_NONE='<html'
+HTML_LANG_EN='html lang="en"'
+
+cd "${TMP_DIR}/"
+
+sed -i "s|$HTML_META_SRC|$HTML_META_EN|g" *.html
+sed -i "s|$HTML_META_SRC|$HTML_META_EN|g" */*.html
+# sed -i "s|$HTML_LOGO_LINK_SRC|$HTML_LOGO_LINK_EN|g" *.html
+# sed -i "s|$HTML_LOGO_LINK_SRC|$HTML_LOGO_LINK_EN|g" */*.html
+sed -i "s|$HTML_LANG_NONE|<$HTML_LANG_EN|g" *.html
+sed -i "s|$HTML_LANG_NONE|<$HTML_LANG_EN|g" */*.html
+
+log 'ACTIVATING'
+cd "$SRC_DIR"
+if [ -d "$DEST_DIR" ]
+then
+  rm -r "$DEST_DIR"
+fi
+mkdir -p "${DEST_DIR}/"
+
+mv "${TMP_DIR}/"* "${DEST_DIR}/"
+
+touch "${DEST_DIR}/${TS}"
+
+rm -rf "$TMP_DIR"
+
+log 'FINISHED'

docs/meta/robots.txt

@@ -0,0 +1,27 @@
+User-agent: anthropic-ai
+Disallow: /
+
+User-agent: Claude-Web
+Disallow: /
+
+User-agent: ClaudeBot
+Disallow: /
+
+User-agent: PerplexityBot
+Disallow: /
+
+User-agent: CCBot
+Disallow: /
+
+User-agent: Google-Extended
+Disallow: /
+
+User-agent: GPTBot
+Disallow: /
+
+User-agent: ChatGPT-User
+Disallow: /
+
+User-agent: *
+Disallow:
+Sitemap: sitemap.xml

docs/meta/sitemap.xml

@@ -0,0 +1,12 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
+   <url><loc>https://ftf.oxl.app/</loc></url>
+
+   <url><loc>https://ftf.oxl.app/usage/1_intro.html</loc></url>
+   <url><loc>https://ftf.oxl.app/usage/2_system_support.html</loc></url>
+   <url><loc>https://ftf.oxl.app/usage/3_run.html</loc></url>
+   <url><loc>https://ftf.oxl.app/usage/4_config.html</loc></url>
+
+   <url><loc>https://ftf.oxl.app/dev/1_intro.html</loc></url>
+   <url><loc>https://ftf.oxl.app/dev/2_plugins.html</loc></url>
+</urlset>

docs/requirements.txt

@@ -0,0 +1,2 @@
+sphinx
+sphinx_immaterial

docs/source/_include/head.rst

@@ -0,0 +1,2 @@
+.. warning::
+    This project still in early development! **DO NOT USE IN PRODUCTION!**