-
Notifications
You must be signed in to change notification settings - Fork 2
AIK-3167 : Switch the python firewall agent to an IPC socket #14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
0414da0
Update aikido env variables
c2627c1
Update agent __init__.py : Create IPC
be15ca9
Create a reporting thread
e15bf5b
Merge branch 'main' into AIK-3167
b028c78
Actually read from queue and send to queue, also temp crit. logs
d4fe2e8
Actually read from queue and send to queue, also temp crit. logs
175ca4e
Merge branch 'AIK-3167' of github.com:AikidoSec/firewall-python into …
51a7550
Add tests for agent init function
91e8f9c
Add test cases for the start_ipc and get_ipc functions
e307854
Add tests to make sure AIKIDO_SECRET_KEY needs to be set
4220ec6
Add some tests for protect() funtion
0abfd66
Test connection refused
53849e5
Linting
5b4c426
Use monkeypatch for environment vars instead of mocker
29890c5
Comment out bc not working in CI?CD
fe05991
Testing/Linting
364b142
get environment variables using the more standard way
01571d6
Update testing so that init test does not fail bc of env vars
6a91bca
Update how con is imported for testing + basic test case (no asserts)
81fae9c
Change interval to 10 minutes
4db88c6
Remove any mention of Agent and use Background process instead
7853f5f
Renames
772b34b
Remove AIKIDO_SECRET_KEY and generate on the spot
f7dfb36
Remove AIKIDO_SECRET_KEY and generate on the spot
5fb1694
Merge branch 'AIK-3167' of github.com:AikidoSec/firewall-python into …
8553645
Merge branch 'main' into AIK-3167
bitterpanda63 d2c98e7
Rename action to "ATTACK" for comms
8015a98
Merge branch 'AIK-3167' of github.com:AikidoSec/firewall-python into …
682512f
Merge branch 'main' into AIK-3167
bitterpanda63 bb4ce9f
Update aikido_firewall/background_process/__init__.py
willem-delbare 8b92619
Update aikido_firewall/background_process/__init__.py
willem-delbare 24061c7
Linting
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,126 @@ | ||
""" | ||
Aikido background process, this will create a new process | ||
and listen for data sent by our sources and sinks | ||
""" | ||
|
||
import time | ||
import os | ||
import secrets | ||
import multiprocessing.connection as con | ||
from multiprocessing import Process | ||
from threading import Thread | ||
from queue import Queue | ||
from aikido_firewall.helpers.logging import logger | ||
|
||
REPORT_SEC_INTERVAL = 600 # 10 minutes | ||
IPC_ADDRESS = ("localhost", 9898) # Specify the IP address and port | ||
|
||
|
||
class AikidoBackgroundProcess: | ||
""" | ||
Aikido's background process consists of 2 threads : | ||
- (main) Listening thread which listens on an IPC socket for incoming data | ||
- (spawned) reporting thread which will collect the IPC data and send it to a Reporter | ||
""" | ||
|
||
def __init__(self, address, key): | ||
logger.debug("Background process started") | ||
listener = con.Listener(address, authkey=key) | ||
self.queue = Queue() | ||
# Start reporting thread : | ||
Thread(target=self.reporting_thread).start() | ||
|
||
while True: | ||
conn = listener.accept() | ||
logger.debug("connection accepted from %s", listener.last_accepted) | ||
while True: | ||
data = conn.recv() | ||
logger.error(data) # Temporary debugging | ||
if data[0] == "ATTACK": | ||
self.queue.put(data[1]) | ||
elif data[0] == "CLOSE": # this is a kind of EOL for python IPC | ||
conn.close() | ||
break | ||
|
||
def reporting_thread(self): | ||
"""Reporting thread""" | ||
logger.debug("Started reporting thread") | ||
while True: | ||
self.send_to_reporter() | ||
time.sleep(REPORT_SEC_INTERVAL) | ||
|
||
def send_to_reporter(self): | ||
""" | ||
Reports the found data to an Aikido server | ||
""" | ||
items_to_report = [] | ||
while not self.queue.empty(): | ||
items_to_report.append(self.queue.get()) | ||
logger.debug("Reporting to aikido server") | ||
logger.critical("Items to report : %s", items_to_report) | ||
# Currently not making API calls | ||
|
||
|
||
# pylint: disable=invalid-name # This variable does change | ||
ipc = None | ||
|
||
|
||
def get_comms(): | ||
""" | ||
Returns the globally stored IPC object, which you need | ||
to communicate to our background process. | ||
""" | ||
return ipc | ||
|
||
|
||
def start_background_process(): | ||
""" | ||
Starts a process to handle incoming/outgoing data | ||
""" | ||
# pylint: disable=global-statement # We need this to be global | ||
global ipc | ||
# Generate a secret key : | ||
generated_key_bytes = secrets.token_bytes(32) | ||
|
||
ipc = IPC(IPC_ADDRESS, generated_key_bytes) | ||
ipc.start_aikido_listener() | ||
|
||
|
||
class IPC: | ||
""" | ||
Facilitates Inter-Process communication | ||
""" | ||
|
||
def __init__(self, address, key): | ||
self.address = address | ||
self.key = key | ||
self.background_process = None | ||
|
||
def start_aikido_listener(self): | ||
""" | ||
This will start the aikido background process which listens | ||
and makes calls to the API | ||
""" | ||
self.background_process = Process( | ||
target=AikidoBackgroundProcess, | ||
args=( | ||
self.address, | ||
self.key, | ||
), | ||
) | ||
logger.debug("Starting the background process") | ||
self.background_process.start() | ||
|
||
def send_data(self, action, obj): | ||
""" | ||
This creates a new client for comms to the background process | ||
""" | ||
try: | ||
conn = con.Client(self.address, authkey=self.key) | ||
logger.debug("Created connection %s", conn) | ||
conn.send((action, obj)) | ||
conn.send(("CLOSE", {})) | ||
conn.close() | ||
logger.debug("Connection closed") | ||
except Exception as e: | ||
logger.info("Failed to send data to bg process : %s", e) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
import pytest | ||
from aikido_firewall.background_process import ( | ||
IPC, | ||
start_background_process, | ||
get_comms, | ||
IPC_ADDRESS, | ||
) | ||
|
||
|
||
def test_ipc_init(): | ||
address = ("localhost", 9898) | ||
key = "secret_key" | ||
ipc = IPC(address, key) | ||
|
||
assert ipc.address == address | ||
assert ipc.background_process is None | ||
|
||
|
||
# Following function does not work | ||
def test_start_background_process(monkeypatch): | ||
assert get_comms() == None | ||
start_background_process() | ||
|
||
assert get_comms() != None | ||
assert get_comms().address == IPC_ADDRESS | ||
|
||
get_comms().background_process.kill() | ||
|
||
|
||
def test_send_data_exception(monkeypatch, caplog): | ||
def mock_client(address, authkey): | ||
raise Exception("Connection Error") | ||
|
||
monkeypatch.setitem(globals(), "Client", mock_client) | ||
monkeypatch.setitem(globals(), "logger", caplog) | ||
|
||
ipc = IPC(("localhost", 9898), "mock_key") | ||
ipc.send_data("ACTION", "Test Object") | ||
|
||
assert "Failed to send data to bg" in caplog.text | ||
# Add assertions for handling exceptions | ||
|
||
|
||
def test_send_data_successful(monkeypatch, caplog, mocker): | ||
ipc = IPC(("localhost"), "mock_key") | ||
mock_client = mocker.MagicMock() | ||
monkeypatch.setattr("multiprocessing.connection.Client", mock_client) | ||
|
||
# Call the send_data function | ||
ipc.send_data("ACTION", {"key": "value"}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
import pytest | ||
from aikido_firewall import protect | ||
from aikido_firewall.background_process import get_comms | ||
|
||
|
||
def test_protect_with_django(monkeypatch, caplog): | ||
monkeypatch.setitem( | ||
globals(), "aikido_firewall.sources.django", "dummy_django_module" | ||
) | ||
monkeypatch.setitem( | ||
globals(), "aikido_firewall.sinks.pymysql", "dummy_pymysql_module" | ||
) | ||
|
||
protect(module="django") | ||
|
||
assert "Aikido python firewall started" in caplog.text | ||
assert get_comms() != None | ||
get_comms().background_process.kill() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.