Add journal to system logs #3456
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Reviewer's GuideThis PR implements an end-to-end system journal export workflow by adding a Python CLI utility to fetch and timestamp systemd logs, exposing it via a new FastAPI endpoint that runs arbitrary blueOS commands, integrating the feature into the Vue settings UI to trigger dumps and downloads, and packaging the script into the Docker image. Sequence diagram for system journal export workflowsequenceDiagram
actor User
participant Frontend as SettingsMenu.vue
participant Backend as FastAPI /command/blueos
participant Script as dump_journal.py
participant FileSystem as /var/logs/blueos/services/journal/
User->>Frontend: Click 'Download System Journal Logs'
Frontend->>Backend: POST /command/blueos?command=dump_journal.py
Backend->>Script: Execute dump_journal.py
Script->>FileSystem: Write journal log file
Backend-->>Frontend: Command execution result
Frontend->>FileSystem: Download journal log file (via filebrowser)
FileSystem-->>Frontend: Send log file
Frontend-->>User: Downloaded log file
Class diagram for dump_journal.py scriptclassDiagram
class dump_journal {
+get_boot_info(boot_index: int) Optional[dict]
+dump_latest_journal_logs(output_dir: str, boot_index: int) bool
}
dump_journal <|-- __main__ : uses
class __main__ {
+boot: int
}
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Williangalvani
force-pushed
the
dump_journal
branch
4 times, most recently
from
2c7dd1b to
56a9b5f
Compare
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey @Williangalvani - I've reviewed your changes - here's some feedback:
Blocking issues:
- Detected subprocess function 'run' without a static string. If this data can be controlled by a malicious actor, it may be an instance of command injection. Audit the use of this call to ensure it is not controllable by an external resource. You may consider using 'shlex.escape()'. (link)
General comments:
- The new /command/blueos endpoint allows arbitrary command execution—consider implementing a whitelist or command mapping to avoid security risks.
- Returning subprocess.CompletedProcess directly may not serialize cleanly; wrap stdout, stderr, and return code in a JSON-friendly response schema instead.
- Rather than triggering a CLI dump and then using the filebrowser, add a dedicated backend download endpoint for journal logs to simplify the frontend flow.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The new /command/blueos endpoint allows arbitrary command execution—consider implementing a whitelist or command mapping to avoid security risks.
- Returning subprocess.CompletedProcess directly may not serialize cleanly; wrap stdout, stderr, and return code in a JSON-friendly response schema instead.
- Rather than triggering a CLI dump and then using the filebrowser, add a dedicated backend download endpoint for journal logs to simplify the frontend flow.
## Individual Comments
### Comment 1
<location> `core/services/commander/main.py:74` </location>
<code_context>
return message
+@app.post("/command/blueos", status_code=status.HTTP_200_OK)
+@version(1, 0)
+async def command_blueos(command: str, i_know_what_i_am_doing: bool = False) -> Any:
+ check_what_i_am_doing(i_know_what_i_am_doing)
+ logger.debug(f"Running command: {command}")
+ return subprocess.run(
+ shlex.split(command),
+ check=False,
</code_context>
<issue_to_address>
Returning subprocess.run() directly may expose sensitive process details.
Sanitize or structure the response to avoid exposing internal command output or errors to clients.
</issue_to_address>
### Comment 2
<location> `core/services/commander/main.py:79` </location>
<code_context>
+async def command_blueos(command: str, i_know_what_i_am_doing: bool = False) -> Any:
+ check_what_i_am_doing(i_know_what_i_am_doing)
+ logger.debug(f"Running command: {command}")
+ return subprocess.run(
+ shlex.split(command),
+ check=False,
+ text=True,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ )
+
</code_context>
<issue_to_address>
No timeout is set for subprocess.run, risking hanging processes.
Specify a timeout in subprocess.run to avoid potential indefinite hangs and resource exhaustion.
</issue_to_address>
<suggested_fix>
<<<<<<< SEARCH
async def command_blueos(command: str, i_know_what_i_am_doing: bool = False) -> Any:
check_what_i_am_doing(i_know_what_i_am_doing)
logger.debug(f"Running command: {command}")
return subprocess.run(
shlex.split(command),
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
=======
async def command_blueos(
command: str,
i_know_what_i_am_doing: bool = False,
timeout: float = 30.0,
) -> Any:
check_what_i_am_doing(i_know_what_i_am_doing)
logger.debug(f"Running command: {command} with timeout {timeout}s")
return subprocess.run(
shlex.split(command),
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=timeout,
)
>>>>>>> REPLACE
</suggested_fix>
### Comment 3
<location> `core/tools/dump_journal/dump_journal.py:69` </location>
<code_context>
+ logger.error(f"Failed to get journal logs: {result.stderr}")
+ return False
+
+ with open(filepath, "w", encoding="utf-8") as f:
+ f.write(f"# Journal logs for boot {boot_id}\n")
+ f.write(f"# Boot started: {dt.strftime('%Y-%m-%d %H:%M:%S')}\n")
</code_context>
<issue_to_address>
No error handling for file write operations.
Wrap the file write in a try/except block and log any exceptions to handle potential write failures.
</issue_to_address>
<suggested_fix>
<<<<<<< SEARCH
with open(filepath, "w", encoding="utf-8") as f:
f.write(f"# Journal logs for boot {boot_id}\n")
f.write(f"# Boot started: {dt.strftime('%Y-%m-%d %H:%M:%S')}\n")
f.write("# Format: ISO timestamps with log levels (loguru-like)\n")
f.write(f"# Generated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n")
f.write("#" + "=" * 60 + "\n\n")
f.write(result.stdout)
logger.info(f"Logs saved to: {filepath}")
return True
=======
try:
with open(filepath, "w", encoding="utf-8") as f:
f.write(f"# Journal logs for boot {boot_id}\n")
f.write(f"# Boot started: {dt.strftime('%Y-%m-%d %H:%M:%S')}\n")
f.write("# Format: ISO timestamps with log levels (loguru-like)\n")
f.write(f"# Generated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n")
f.write("#" + "=" * 60 + "\n\n")
f.write(result.stdout)
except Exception as e:
logger.error(f"Failed to write journal logs to {filepath}: {e}")
return False
logger.info(f"Logs saved to: {filepath}")
return True
>>>>>>> REPLACE
</suggested_fix>
## Security Issues
### Issue 1
<location> `core/services/commander/main.py:79` </location>
<issue_to_address>
**security (python.lang.security.audit.dangerous-subprocess-use-audit):** Detected subprocess function 'run' without a static string. If this data can be controlled by a malicious actor, it may be an instance of command injection. Audit the use of this call to ensure it is not controllable by an external resource. You may consider using 'shlex.escape()'.
*Source: opengrep*
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Williangalvani
force-pushed
the
dump_journal
branch
from
56a9b5f to
6ed2bec
Compare
joaomariolago
requested changes
Oct 9, 2025
There was a problem hiding this comment.
dump_journal.py fails in BlueOS running on PI4 with Bullseye, seems to be related to journalctl itself ignoring --output arg.
Outpout of dump_journal.py
root@blueos:~# dump_journal.py
2025-10-09 16:20:09.976 | INFO | commonwealth.utils.commands:run_command:73 - Host: 'journalctl --list-boots --output=json' : returned 0
2025-10-09 16:20:09.976 | INFO | commonwealth.utils.commands:run_command:77 - stdout: -21 d52810bca79847438062d810eade0fdf Wed 2025-09-24 15:57:02 BST—Wed 2025-09-24 15:57:36 BST
-20 1c61118d030c4a1cb19f7cc217f7efd1 Wed 2025-09-24 15:57:47 BST—Wed 2025-09-24 17:42:06 BST
-19 f8840a690ef44cc097b3359554220fde Wed 2025-09-24 17:42:17 BST—Wed 2025-09-24 21:40:00 BST
-18 c7c8a39f413544e7a827adf3981712e2 Thu 2025-09-25 14:27:24 BST—Thu 2025-09-25 15:17:01 BST
-17 b9a2548adea04bcea747872091ff8da6 Thu 2025-09-25 15:17:01 BST—Thu 2025-09-25 17:32:06 BST
-16 d6ba15cf0b404169974f98490951b6b9 Thu 2025-09-25 17:32:06 BST—Thu 2025-09-25 18:00:30 BST
-15 243fc658d280477283eae63005baf6ab Thu 2025-09-25 18:06:39 BST—Thu 2025-09-25 20:55:02 BST
-14 8fb084a408924ee8aa9aca139cf7a368 Thu 2025-09-25 20:55:02 BST—Fri 2025-09-26 02:17:01 BST
-13 b780abd7941b46979bc02644db5b6316 Fri 2025-09-26 02:17:01 BST—Fri 2025-09-26 17:17:02 BST
-12 5eed26a1cad34f24ad65603c30be5184 Fri 2025-09-26 17:17:02 BST—Fri 2025-09-26 22:20:45 BST
-11 d6c289673ee448b492002fe7e554101d Fri 2025-09-26 22:26:38 BST—Mon 2025-09-29 21:38:07 BST
-10 91ada533f0d84e3092b9fb0878a3e95f Tue 2025-09-30 17:46:22 BST—Tue 2025-09-30 17:47:35 BST
-9 010dd7c4768a49b6821d865dc1c1722b Tue 2025-09-30 17:48:26 BST—Wed 2025-10-01 05:17:01 BST
-8 6f339f9f2a0d4d83bbc50cb9515cd797 Wed 2025-10-01 05:17:01 BST—Wed 2025-10-01 14:09:13 BST
-7 cfcaae96469b4e2ebb7b729e8b956931 Wed 2025-10-01 14:43:30 BST—Wed 2025-10-01 19:00:27 BST
-6 acad36499de84d51b23757f576d2d835 Wed 2025-10-01 19:00:27 BST—Wed 2025-10-01 21:17:01 BST
-5 cf1fc7ae6f484185bbedf40bbc3b0e36 Wed 2025-10-01 21:17:01 BST—Fri 2025-10-03 03:34:03 BST
-4 061bf2df99e64cd6b390bb9dd59f50b0 Fri 2025-10-03 03:45:05 BST—Sat 2025-10-04 01:50:50 BST
-3 8b073d40809a4f33bb8875f98a4c5ac9 Sat 2025-10-04 01:58:19 BST—Mon 2025-10-06 21:50:22 BST
-2 2ff2a73fc004407d866429f6c9681f89 Mon 2025-10-06 21:53:54 BST—Wed 2025-10-08 17:43:05 BST
-1 46fa6b8088a14ca1b65b153982e444f9 Wed 2025-10-08 17:43:05 BST—Wed 2025-10-08 21:24:20 BST
0 7b7ba847de0f4d6fa426cd45361bda66 Thu 2025-10-09 15:08:56 BST—Thu 2025-10-09 17:20:09 BST
ERROR:__main__:Error getting boot info: Extra data: line 1 column 5 (char 4)
Manually running journalctl --list-boots --output=json
pi@blueos:~ $ journalctl --list-boots --output=json
-21 d52810bca79847438062d810eade0fdf Wed 2025-09-24 15:57:02 BST—Wed 2025-09-24 15:57:36 BST
-20 1c61118d030c4a1cb19f7cc217f7efd1 Wed 2025-09-24 15:57:47 BST—Wed 2025-09-24 17:42:06 BST
-19 f8840a690ef44cc097b3359554220fde Wed 2025-09-24 17:42:17 BST—Wed 2025-09-24 21:40:00 BST
-18 c7c8a39f413544e7a827adf3981712e2 Thu 2025-09-25 14:27:24 BST—Thu 2025-09-25 15:17:01 BST
-17 b9a2548adea04bcea747872091ff8da6 Thu 2025-09-25 15:17:01 BST—Thu 2025-09-25 17:32:06 BST
-16 d6ba15cf0b404169974f98490951b6b9 Thu 2025-09-25 17:32:06 BST—Thu 2025-09-25 18:00:30 BST
-15 243fc658d280477283eae63005baf6ab Thu 2025-09-25 18:06:39 BST—Thu 2025-09-25 20:55:02 BST
-14 8fb084a408924ee8aa9aca139cf7a368 Thu 2025-09-25 20:55:02 BST—Fri 2025-09-26 02:17:01 BST
-13 b780abd7941b46979bc02644db5b6316 Fri 2025-09-26 02:17:01 BST—Fri 2025-09-26 17:17:02 BST
-12 5eed26a1cad34f24ad65603c30be5184 Fri 2025-09-26 17:17:02 BST—Fri 2025-09-26 22:20:45 BST
-11 d6c289673ee448b492002fe7e554101d Fri 2025-09-26 22:26:38 BST—Mon 2025-09-29 21:38:07 BST
-10 91ada533f0d84e3092b9fb0878a3e95f Tue 2025-09-30 17:46:22 BST—Tue 2025-09-30 17:47:35 BST
-9 010dd7c4768a49b6821d865dc1c1722b Tue 2025-09-30 17:48:26 BST—Wed 2025-10-01 05:17:01 BST
-8 6f339f9f2a0d4d83bbc50cb9515cd797 Wed 2025-10-01 05:17:01 BST—Wed 2025-10-01 14:09:13 BST
-7 cfcaae96469b4e2ebb7b729e8b956931 Wed 2025-10-01 14:43:30 BST—Wed 2025-10-01 19:00:27 BST
-6 acad36499de84d51b23757f576d2d835 Wed 2025-10-01 19:00:27 BST—Wed 2025-10-01 21:17:01 BST
-5 cf1fc7ae6f484185bbedf40bbc3b0e36 Wed 2025-10-01 21:17:01 BST—Fri 2025-10-03 03:34:03 BST
-4 061bf2df99e64cd6b390bb9dd59f50b0 Fri 2025-10-03 03:45:05 BST—Sat 2025-10-04 01:50:50 BST
-3 8b073d40809a4f33bb8875f98a4c5ac9 Sat 2025-10-04 01:58:19 BST—Mon 2025-10-06 21:50:22 BST
-2 2ff2a73fc004407d866429f6c9681f89 Mon 2025-10-06 21:53:54 BST—Wed 2025-10-08 17:43:05 BST
-1 46fa6b8088a14ca1b65b153982e444f9 Wed 2025-10-08 17:43:05 BST—Wed 2025-10-08 21:24:20 BST
0 7b7ba847de0f4d6fa426cd45361bda66 Thu 2025-10-09 15:08:56 BST—Thu 2025-10-09 18:12:16 BST
Journalctl version
pi@blueos:~ $ journalctl --version
systemd 247 (247.3-6+rpi1)
+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +ZSTD +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=unified
Journalctl help
pi@blueos:~ $ journalctl --help
...
-o --output=STRING Change journal output mode (short, short-precise,
short-iso, short-iso-precise, short-full,
short-monotonic, short-unix, verbose, export,
json, json-pretty, json-sse, json-seq, cat,
with-unit)
...
Journalctl Args validation
pi@blueos:~ $ journalctl --list-boots --output=POTATO
Unknown output format 'POTATO'.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
needs some more testing
fix #78
Summary by Sourcery
Provide a system journal export workflow by adding a CLI utility, exposing a new backend endpoint for blueOS commands, integrating the feature into the frontend settings, and packaging the tool in the Docker image.
New Features:
Summary by Sourcery
Add system journal export functionality end-to-end by introducing a CLI script, exposing a backend command endpoint, integrating it into the frontend settings, and packaging the tool in the Docker image.
New Features:
Build: