LLM Integration - POC

[timothyriceherc]

Thank you for all the contributions to the cyber security community.

Has anyone thought of LLM integration? I have built a LLM integration that will add additional logic and has a re-scoring capability. For flexibility, I decided to integrate with OpenWebUI.

I did not find an official way to contribute, but would be happy to provide my POC code should there be a need or desire to integrate the power of LLM.

This works by parsing the reports.json for each report injecting a response written in python.
I do NOT consider myself a power coder by any means, but I am happy to contribute whatever I can.

I can be reached at tim[.]j[.]rice@hackrange[.]com

[doomedraven]

hey, i was considering that in past, but my usage of capev2 is completely different from the rest of the user. you can PR your code i can check it, i won't promise that i will merge it, but it can be a community reporting module(i can adapt that). Its all dends how good is the system prompt is

[kevoreilly]

I think that the interesting part that is within the remit of the core cape project is the interface to LLMs, but I don't think it's the right place for any direct integration of a given LLM.

It's perhaps not obvious but it can be taken as intentional that malscore and other scoring mechanisms are absent from cape in its default configuration, and in the demonstration instance at capesandbox.com. This is both because experience has shown how poor they are and because ultimately I think maliciousness scores themselves are just completely flawed.

I remain unconvinced that an LLM will actually produce superior output than previous procedural scoring mechanisms but feel certain that to enable a half-decent result firstly requires a good interface to the information presented in cape's output. That is where my interest lies; ensuring that all of the relevant data is not only captured by the LLM but with appropriate weight to produce a real-world benefit. Signatures are an obvious place to start but the scope is wide when considering all the sources a highly skilled human might harness.

Ultimately I would be happy to see an LLM harness integrated that allows different LLMs to be plugged in, but am a long way from being convinced that any particular LLM deserves to benefit from the publicity of direct integration and having its output badged with cape's good name.

[timothyriceherc]

Thanks @kevoreilly & @doomedraven .

If there is any interest in a feature, feel free to see the code that I am using. I by default don't actually re-score, but I have an option to. That is in the var called REWRITE_SCORE. By default it is 0, but it still writes a LLM summary.
I love CAPE. We use this platform to run about 60,000 analysis annually. We have been using Cuckoo and then CAPE for years.

The LLM provides great context from my testing.

cape_rescore_v1.0.0.py

[doomedraven]

Thank you for code. I will review it tomorrow. Yes it can do a magic with proper instructions. So let see how it going.

[doomedraven]

i did a quick look, i like the idea, some parts need to be worked on. right now i don't have much time to do that. but as quick note for myself.

1. reporting module with on demand feature
2. if enabled/requested on demand, create a new tab with summary by LLM
3. ask Gemini and Claude to refine the system prompt, define the system role, make it configurable which LLM to use, etc

so please don't close the issue, leave it here till I or someone will have time to properly integrate it, im very busy those weeks so not sure when i will be able to work on it

[kevoreilly]

You probably have an issue that can be tackled in normal fashion, by creating a new issue with all the details rather than replying to a completely unrelated subject.

Helpful information such as example hashes, linksto public jobs to which you are comparing and specific details on the observed disparities.

While I am typing this I see you copy/pasted the same reply into other issues?!?! You are about to get banned with that behaviour.

[doomedraven]

Is more about how do you setup windows VM itself than configuration of the server side El lun, 13 oct 2025, 15:58, Kronos ***@***.***> escribió:

…

*Kronos-ux* left a comment (kevoreilly/CAPEv2#2719) <#2719 (comment)> Hey @doomedraven <https://github.com/doomedraven> @kevoreilly <https://github.com/kevoreilly>, I hope you both are doing well. I’m currently conducting research using CAPE Sandbox and noticed a significant difference in the results when analyzing the same sample locally versus on your hosted instance ( www.capesandbox.com). Would it be possible for you to please share the configuration file (or at least the relevant sections) that your setup uses? I’ve already prepared my workstation and downloaded the necessary samples, but this difference in behavior is impacting my research consistency. I completely understand if you can’t share the entire configuration, but even partial guidance or anonymized snippets would be incredibly helpful. I’d really appreciate your help on this — it would mean a lot for my research. 🙏 Thank you so much in advance! — Reply to this email directly, view it on GitHub <#2719 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOFH354Q3U4CUS2CFTL4WL3XOVZXAVCNFSM6AAAAACIP67YOGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGOJXGY2TKMRXGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Kronos-ux]

sorry @doomedraven @kevoreilly

i am really very frusted and upset

it's just i can't understand how to get help and solve my issue

really sorry for my behaviour