accessing PCAP for processing remotely?

[mmguero]

Asked in a PM on Reddit:

Hi there. Malcolm question. I'm considering deploying malcolm alongside an existing deployment that already features opensearch & arkime, but we're interested in your zeek parsers, and other features such as ClamAV, yara, dashboards, etc. that we don't support. We're unable to get rid of our current deployment without presenting a better solution, hence the side-by-side deployment. We already have existing full PCAP, but it is spread on various sensors. I understand we don't need hedgehog in this case because i'm not looking for live analysis. Are you aware of a way we can reference (e.g., symlinks) the full PCAP we have stored without having to duplicate full PCAP into the ./pcap/upload/ directory? Or do we have to script out uploading individual PCAP spread throughout various sensors? Thank you for your contribution!

[mmguero]

@mmguero 's reply

Hmmm, no unfortunately I'm not aware of any way that malcolm could see that PCAP spread out on different remote sensors. Even with something like sshfs, Malcolm's going to want to be taking its PCAP files and moving them around within its ./pcap folder for processing, which I don't think would work in that case.

So for scripting PCAP upload, you've got a few options:

• just use rsync/scp to the box running Malcolm and drop files in its ./pcap/upload directory like you mentioned
• Malcolm can be configured to surface its own internal SCP/SFTP server (search for sftp here and here)
• you could use curl to do it, see this example for the syntax