-
Notifications
You must be signed in to change notification settings - Fork 523
FAQ
Install / Update / Upgrade
Users / Passwords
Support / Help
Error messages
IPS/IDS engines
Security Onion internals
Tuning
sostat output
Miscellaneous
We only support the English language at this time:
https://github.com/Security-Onion-Solutions/security-onion/wiki/Installation#language
The Keyboard Layout screen may be larger than your screen resolution and so the Continue button may be off the screen to the right like this:
https://launchpadlibrarian.net/207213663/Screenshot_wilyi386deskmanual_2015-05-22_13%3A05%3A41.png
You can simply slide the window over until you see the Continue button. For more information, please see:
https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/1458039
Some users running the Snort engine with the Snort Subscriber (Talos) ruleset are experiencing barnyard2 failing with errors like Returned signature_id is not equal to updated signature_id. This is due to some wrong entries in the database left by the previous version of barnyard2. One of the barnyard2 developers wrote a MySQL script to fix these entries and I've packaged it into a shell script called so-snorby-fix-sigs and included it in the rule-update package. If you're running the Snort engine with the Snort Subscriber (Talos) ruleset, please run so-snorby-fix-sigs and follow the directions (including shutting down all barnyard2 instances before proceeding with the database changes).
http://blog.securityonion.net/2014/06/new-securityonion-rule-update-package.html
Ubuntu is saying that my kernel has reached EOL (End Of Life). Should I update to the newer HWE stack?
Please see our HWE page.
Usually this happens when you clone a VM. VMware asks if you moved it or copied it. If you select "copied", it will change the MAC address to avoid duplication. At the next boot, Ubuntu's udev will see a new MAC address and create a new network interface (eth1). To fix this:
sudo rm /etc/udev/rules.d/70-persistent-net.rules
sudo reboot
If you have a 64-bit machine, use our 64-bit ISO image or use a 64-bit version of Ubuntu.
If you have a 32-bit machine, you'll need to use a 32-bit version of Ubuntu and then install the PAE kernel as described here:
https://help.ubuntu.com/community/EnablingPAE
No, we only support x86 and x86-64 architectures. Please see the hardware page.
box
Definition: A physical or virtual machine running the Security Onion operating system.
server
Definition: A set of processes that receive data from sensors and allow analysts to see and investigate that data. The set of processes includes sguild, mysql, and optionally the Elastic stack (Elasticsearch, Logstash, Kibana) and Curator. The server is also responsible for ruleset management.
Naming convention: The collection of server processes has a server name separate from the hostname of the box. Security Onion always sets the server name to securityonion.
Configuration files: /etc/nsm/securityonion/
Controlled by: /usr/sbin/nsm_server
server box
Definition: A machine running the server processes. May optionally be running sensor processes.
Example 1: User runs Quick Setup on machine with hostname securityonion and two ethernet interfaces. Setup creates a server and two sensors (securityonion-eth0 and securityonion-eth1).
Example 2: User runs Advanced Setup and chooses Server. Setup creates a server only (no sensor processes).
sensor
Definition: A set of processes listening on a network interface. The set of processes currently includes Snort/Suricata, netsniff-ng, argus, prads, and bro (although this is in constant flux as we add new capabilities and find better tools for existing capabilities).
Naming convention: $HOSTNAME-$INTERFACE
Configuration files: /etc/nsm/$HOSTNAME-$INTERFACE/
Example: sensor1-eth0
Controlled by: /usr/sbin/nsm_sensor
sensor box
Definition: A machine having one or more sensors that transmit to a central server. Does not run server processes. Pulls ruleset from server box. (In some contexts, I refer to this a slave pulling rules from the master.)
Example: A machine named sensor1 having sensors sensor1-eth0 and sensor1-eth1.
back to top
Adding Sguil accounts
back to top
Yes, please see:
https://securityonionsolutions.com
back to top
Please see https://github.com/Security-Onion-Solutions/security-onion/wiki/Proxy#pulledpork.
Why does rule-update fail with an error like "Error 404 when fetching s3.amazonaws.com/snort-org/www/rules/community/community-rules.tar.gz.md5"?
The Snort Community ruleset has moved to a different URL. You can run the following command to update the Snort Community URL in pulledpork.conf:
sudo sed -i 's\rule_url=https://s3.amazonaws.com/snort-org/www/rules/community/|community-rules.tar.gz|Community\rule_url=https://snort.org/downloads/community/|community-rules.tar.gz|Community\g' /etc/nsm/pulledpork/pulledpork.conf
For more information, please see: http://blog.snort.org/2015/10/are-you-getting-404-errors-attempting.html
Why does soup fail with an error message like "find: `/usr/lib/python2.7/dist-packages/salt/': No such file or directory"?
This is a bug in the salt packages that can manifest when skipping salt versions. Resolve with the following:
sudo mkdir -p /usr/lib/python2.7/dist-packages/salt/
sudo apt-get -f install
sudo soup
Why does barnyard2 keep failing with errors like "Returned signature_id is not equal to updated signature_id"?
Please see: http://blog.securityonion.net/2014/06/new-securityonion-rule-update-package.html
I just updated Snort and it's now saying 'ERROR: The dynamic detection library "/usr/local/lib/snort_dynamicrules/chat.so" version 1.0 compiled with dynamic engine library version 2.1 isn't compatible with the current dynamic engine library "/usr/lib/snort_dynamicengine/libsf_engine.so" version 2.4.'
Run the following:
sudo rule-update
For more information, please see:
http://blog.securityonion.net/2014/12/new-version-of-securityonion-rule.html
I get periodic MySQL crashes and/or error code 24 "out of resources" when searching in Sguil. How do I fix that?
Recent versions of Setup should set MySQL's open-files-limit to 90000 to avoid this problem:
http://blog.securityonion.net/2014/02/new-securityonion-setup-package.html
If you ran Setup before February 2014, you can set this manually as follows.
First, stop sguil and mysql:
sudo nsm_server_ps-stop
sudo service mysql stop
Next, edit /etc/mysql/my.cnf and add the following in the mysqld section (please use hyphens not underscores):
open-files-limit = 90000
Finally, start mysql and sguil:
sudo service mysql start
sudo nsm_server_ps-start
For more information, please see:
http://nsmwiki.org/Sguil_FAQ#I.27m_seeing_error_code_24_from_MySQL._How_do_I_fix_that.3F
Barnyard2 is failing with an error like "ERROR: sguil: Expected Confirm 13324 and got: Failed to insert 13324: mysqlexec/db server: Duplicate entry '9-13324' for key 'PRIMARY'". How do I fix this?
Sometimes, just restarting Barnyard will clear this up:
sudo nsm_sensor_ps-restart --only-barnyard2
Other times, restarting Sguild and then restarting Barnyard will clear it up:
sudo nsm_server_ps-restart
sudo nsm_sensor_ps-restart --only-barnyard2
If that doesn't work, then try also restarting mysql:
sudo service mysql restart
sudo nsm_server_ps-restart
sudo nsm_sensor_ps-restart --only-barnyard2
If that still doesn't fix it, you may have to perform MySQL surgery on the database securityonion_db as described in the Sguil FAQ:
http://nsmwiki.org/Sguil_FAQ#Barnyard_dies_at_startup.2C_with_.22Duplicate_Entry.22_error
Application initialization failed: no display name and no $DISPLAY environment variable
ERROR: Cannot fine the Iwidgets extension.
The iwidgets package is part of the incr tcl extension and is
available as a port/package most systems.
See http://www.tcltk.com/iwidgets/ for more info.
This is related to this question. See tcl.
7:01 AM is the time of the daily PulledPork rules update. If you're running Snort with the Snort Subscriber (Talos) ruleset, this includes updating the SO rules. There is a known issue when running Snort with the Snort Subscriber (Talos) ruleset and updating the SO rules:
https://groups.google.com/d/topic/pulledpork-users/1bQDkh3AhNs/discussion
After updating the rules, Snort is restarted, and the segfault occurs in the OLD instance of Snort (not the NEW instance). Therefore, the segfault is merely a nuisance log entry and can safely be ignored.
This usually means that there is an unexpected file in the dailylogs directory. Run the following:
ls /nsm/sensor_data/*/dailylogs/
You should see a bunch of date stamped directories and you may see some extraneous files. Remove any extraneous files and restart pcap_agent:
sudo nsm_sensor_ps-restart --only-pcap-agent
I'm running the Security Onion 12.04.5 ISO image and Chromium crashes and/or displays a black screen.
This is a known issue with certain versions of VMware. You can either:
- go into the VM configuration and disable 3D in the video adapter
OR
- upgrade the VM hardware level (may require upgrading to a new version of VMware)
The GeoIP CITY database is not free and thus we cannot include it in the distro. Bro fails to find it and falls back to the GeoIP COUNTRY database (which is free). As long as you are seeing some country codes in your conn.log, then everything should be fine. If you really need the CITY database, see this thread for some options:
https://groups.google.com/d/topic/security-onion-testing/gtc-8ZTuCi4/discussion
Since Ubuntu kernel 4.4.0-20, the EFI_SECURE_BOOT_SIG_ENFORCE kernel configuration has been enabled. This prevents the loading of unsigned third party modules if UEFI Secure Boot is enabled.
An example of this can be found here:
https://groups.google.com/d/msg/security-onion/r64yl58KGJ4/uRedkKTBCAAJ
If Secure Boot is enabled for your machine, you can disable it, following the steps found here:
https://wiki.ubuntu.com/UEFI/SecureBoot/DKMS
http://askubuntu.com/questions/762254/why-do-i-get-required-key-not-available-when-install-3rd-party-kernel-modules
back to top
Please note that, if you're running the Snort Talos ruleset, Snort Shared Object rules will not load in Suricata. Most folks who choose the Suricata engine choose to run the Emerging Threats ruleset.
sudo nsm_sensor_ps-stop
sudo sed -i 's|ENGINE=snort|ENGINE=suricata|g' /etc/nsm/securityonion.conf
sudo rule-update
sudo nsm_sensor_ps-start
sudo nsm_sensor_ps-stop
sudo sed -i 's|ENGINE=suricata|ENGINE=snort|g' /etc/nsm/securityonion.conf
sudo rule-update
sudo nsm_sensor_ps-start
Running Security Onion as an IPS requires manual configuration and is not supported.
I talked about this on the Packet Pushers podcast:
http://packetpushers.net/show-95-security-onion-with-doug-burks-or-why-ids-rules-and-ips-drools/
back to top
Sguil uses netsniff-ng to record full packet captures to disk. These pcaps are stored in nsm/sensor_data/$HOSTNAME-$INTERFACE/dailylogs/. /etc/cron.d/sensor-clean is a cronjob that runs every minute that should delete old pcaps when the disk reaches your defined disk usage threshold (90% by default). It's important to properly size your disk storage so that you avoid filling the disk to 100% between purges.
/etc/init/securityonion.conf waits 60 seconds after boot to ensure network interfaces are fully initialized before starting services.
You can lower the DAYSTOKEEP setting in /etc/nsm/securityonion.conf.
Also see UNCAT_MAX:
http://blog.securityonion.net/2015/01/new-version-of-sguil-db-purge-helps.html
https://groups.google.com/d/topic/security-onion/xLxTGQs30ho/discussion
In the Sguil client, click the File menu and then go to Change Font. You can change both the Standard and Fixed fonts.
Interface stops receiving traffic
In /etc/default/grub, change this line:
GRUB_CMDLINE_LINUX_DEFAULT="splash quiet"
to:
GRUB_CMDLINE_LINUX_DEFAULT="text"
Then run:
sudo update-grub
For more information, please see:
http://ubuntuforums.org/showthread.php?t=1690118
If you're doing a new installation, you can avoid this altogether by installing our packages on top of Ubuntu Server (minimal installation, no GUI) instead of using the Security Onion ISO image.
I'm running Security Onion in a VM and the screensaver is using lots of CPU. How do I change/disable the screensaver?
- Click Applications.
- Click Settings.
- Click Screensaver.
- Screensaver Preferences window appears. Click the Mode dropdown and select "Disable Screen Saver" or "Blank Screen Only".
- Close the Screensaver Preferences window.
From Gianluca Costa:
Xplico has embedded (in its PHP code) a Http-proxy, this proxy is used to show the web pages, emulating, for example, the original cache of the user.
By default the XI url must be an IP address (wiki: http://wiki.xplico.org/doku.php?id=interface#browser ), the only exception to this rule is the url http://demo.xplico.org (for obvious reasons). If you use as url a name (not an ip) then XI give you a blank page, because XI searches your url in the decoded data. To change this behavior you must modify the PHP code. Edit /opt/xplico/xi/cake/dispatcher.php and change demo.xplico.org to your host name (used in the url).
There are usually 2 main reasons for this:
- low on RAM
OR
- ungraceful shutdown (perhaps power outage) resulted in database corruption
If you think you have sufficient RAM, then search /nsm/elsa/data/elsa/log/node.log for errors:
sudo grep syslogs_archive_1 /nsm/elsa/data/elsa/log/node.log
If you see errors like "Can't find file: 'syslogs_archive_1'", then you'll need to reset the archive. If you have all updates installed, you can simply run the following command:
sudo securityonion-elsa-reset-archive
Otherwise, you can manually reset the archive by running the following commands:
# Stop services
sudo service nsm stop
sudo service syslog-ng stop
# Cleanup database tables and entries
sudo mysql --defaults-file=/etc/mysql/debian.cnf syslog_data -e "DROP TABLE syslog_data.syslogs_archive_1"
sudo mysql --defaults-file=/etc/mysql/debian.cnf syslog_data -e "DELETE FROM syslog.tables WHERE table_name='syslog_data.syslogs_archive_1'"
# Cleanup database files
sudo rm /nsm/elsa/data/elsa/mysql/syslogs_archive_1*
sudo rm /var/lib/mysql/syslog_data/syslogs_archive_1*
# Restart services
sudo service mysql restart
sudo service syslog-ng restart
sudo service nsm start
Also see https://groups.google.com/d/topic/security-onion/O3uBjCR5jYk/discussion.
Sguild has to load uncategorized events into memory when it starts and it won't accept connections until that's complete.
You can either:
- wait for sguild to start up (may take a LONG time), then log into Sguil, and
F8LOTS of events
OR
- stop sguild
sudo nsm_server_ps-stop
and manually categorize events using mysql
(see http://taosecurity.blogspot.com/2013/02/recovering-from-suricata-gone-wild.html)
OR
lower your DAYSTOKEEP setting in /etc/nsm/securityonion.conf and run
sudo sguil-db-purge
To keep Uncategorized Events from getting too high, you should log into Sguil/Squert on a daily/weekly basis and categorize events.
Some users have reported issues with Perl processes continually spinning up and consuming a large amount of CPU. This is likely related to /opt/elsa/web/cron.pl trying to load duplicate entries in the saved_results table of the elsa_web database. Try the following steps if you notice a continuous spawning of Perl processes, paired with high load/CPU usage:
-
To temporarily deal with the load, comment out the last line of
/etc/cron.d/elsa.
Make sure to uncomment this line when you are done troubleshooting, as this is not a a permanent solution. Disabling this script prevents ELSA (cron.pl) from loading new data into it's mysql database, therefore it should only be disabled for troubleshooting purposes. -
Try running cron.pl manually, observing the output:
sudo sh /opt/elsa/contrib/securityonion/contrib/securityonion-elsa-cron.sh
or if you're still on the old 12.04:
sudo perl /opt/elsa/web/cron.pl -c /etc/elsa_web.conf -
Look for references to a failure or an error in ELSA's web.log:
sudo grep failed /nsm/elsa/data/elsa/log/web.log -
Check to see if any error messages are returned similar to the following:
Duplicate entry '800' for key 'PRIMARY' QUERY: INSERT INTO saved_results
If so, you can attempt to remedy this issue by performing the following actions:
-
Back up elsa_web database:
sudo cp -R /var/lib/mysql/elsa_web /var/lib/mysql/elsa_web_backup -
Remove offending entries:
sudo mysql --defaults-file=/etc/mysql/debian.cnf -Delsa_web -e 'delete from saved_results where qid=[offendingentry#]';
Be sure to uncomment the final line of /etc/cron.d/elsa if you previously commented it out, and monitor processes to verify the issue is no longer present. If the issue persists, pose the question to the mailing list for further assistance.
Also see: https://groups.google.com/forum/#!searchin/security-onion/elsa$20cron.pl/security-onion/t3o5rcTf_-U/VJKMdkDsBAAJ
back to top
Network Security Monitoring as a whole is considered "best effort". It is not a "mission critical" resource like a file server or web server. Since we're dealing with "big data" (potentially terabytes of full packet capture), backups would be prohibitively expensive. Most organizations don't do any backups and instead just rebuild boxes when necessary.
Adding local rules and testing them with scapy
You can download the full source code for any of our packages like this:
apt-get source PACKAGE-NAME
where PACKAGE-NAME is usually something like securityonion-snort. Here's a list of all of our packages:
https://launchpad.net/~securityonion/+archive/stable
In the new Security Onion 14.04, ELSA should display bar charts automatically. In the old Security Onion 12.04,
ELSA's bar charts required flash, so one option would be to replace Chromium with Google Chrome (which includes flash). If you install Chrome on Security Onion and want to make it your default browser so that you can pivot from Sguil to Chrome, do the following:
sudo update-alternatives --set x-www-browser /usr/bin/google-chrome-stable
A few options:
- "ssh -X" - any program started in the SSH session will be displayed on your local desktop (requires a local X server)
-
xrdp - sudo apt-get install xrdp - requires an rdp client
- You can use FreeNX but we don't recommend or support it
If the Squert map is not showing the country for IPs, try running the following:
sudo /usr/bin/php -e /var/www/so/squert/.inc/ip2c.php 0'/
Why does the ELSA web interface not recognize one of my ELSA log nodes even though the APIKEY is correct?
Could be due to clocks not matching between ELSA log node and ELSA web interface. Please see: https://groups.google.com/d/topic/security-onion/K_5vWQpd8VM/discussion
This is a known issue with Ubuntu 10.04 and ESXi 4.1 and is unrelated to Security Onion. Please see:
http://ubuntuforums.org/showthread.php?t=1674759
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/659422
We're not allowed to redistribute the unrar plugin, so you'll need to install it manually:
sudo apt-get update
sudo apt-get install unrar
Security Onion is based on Ubuntu, but we don't provide community support for the Ubuntu OS itself. If you have questions about Ubuntu, you should check the Ubuntu website, forums, and Google.
back to top
- Introduction
- Use Cases
- Hardware Requirements
- Release Notes
- Download/Install
- Booting Issues
- After Installation
- UTC and Time Zones
- Services
- VirtualBox Walkthrough
- VMWare Walkthrough
- Videos
- Architecture
- Cheat Sheet
- Conference
- Elastic Stack
- Elastic Architecture
- Elasticsearch
- Logstash
- Kibana
- ElastAlert
- Curator
- FreqServer
- DomainStats
- Docker
- Redis
- Data Fields
- Beats
- Pre-Releases
- ELSA to Elastic
- Network Configuration
- Proxy Configuration
- Firewall/Hardening
- Email Configuration
- Integrating with other systems
- Changing IP Addresses
- NTP
- Managing Alerts
- Managing Rules
- Adding Local Rules
- Disabling Processes
- Filtering with BPF
- Adjusting PF_RING for traffic
- MySQL Tuning
- Adding a new disk
- High Performance Tuning
- Trimming PCAPs