Do I Have Enough Disk Space?

If the underlying Linux system runs out of space, that can lead to processes dying unexpectedly or tasks not completing. To check if you’re running low, run:

df -h

This shows the amount of free space on each of your partitions. Example – your output will be different:

df -h

Filesystem Size Used Avail Use% Mounted on


/dev/vda1 60G 9.9G 51G 17% /

/dev/vda2 1.0T 101G 899G 10% /var

devtmpfs 1.9G 0 1.9G 0% /dev

tmpfs 1.9G 180K 1.9G 1% /dev/shm

tmpfs 1.9G 193M 1.7G 11% /run

tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup

tmpfs 380M 0 380M 0% /run/user/1001

tmpfs 380M 0 380M 0% /run/user/0

The important lines are the ones associated with the root filesystem(“/”), the home partition (“/home/”, if there is one), and “/var/”, if there is one. If any of these are very low on space, you’ll need to free some up.

A common tool for this is delete-databases.sh , available on the Zeek and AC-Hunter/Rita systems. When run with no command line parameters, it lists the available databases.

Once you’ve found some you no longer need to keep, run the tool again with the name(s) of one or more of them to remove. We recommend running the same command on both systems to remove them from both.


Read More

Is There Any Way to Analyze PCAP Files?

Yes there is! Run the following commands on your AC-Hunter system, depending on your version.

 

If you are using AC-Hunter 5.3.0 or higher, please run the following:

/usr/local/bin/import_pcaps.sh -p pcap_filename.pcap -d database_name

 

If you have AC-Hunter 5.2.0 or lower:

Set the following variables to your own values:

export PCAP_FILE=/absolute/path/to/file.pcap
export BRO_DIR=/absolute/path/you/want/bro/logs/
export DATABASE=yourdatabasename

Convert your pcap to Bro logs:

sudo docker run --rm --volume "$PCAP_FILE:/capture.pcap" --volume "$BRO_DIR:/pcap" --env BRO_DNS_FAKE=true blacktop/bro:2.5 -r /capture.pcap local

Import the Bro logs into AI-Hunter using RITA:

~/AIH-source/AI-Hunter-latest/rita import $BRO_DIR $DATABASE

 

At this point your database should be visible in AC-Hunter.

 


Read More

Installing tcpdump

This package is provided in all supported Linux distributions.

On Debian and Ubuntu linux, run:

sudo apt-get -y install tcpdump

On Centos, RHEL, or Fedora Linux, run:

sudo yum -y install tcpdump


Read More

How Can I View Logs from Specific Docker Containers?

sudo hunt logs web
sudo hunt logs api
sudo hunt logs db
sudo hunt logs auth

 


Read More

Can I Change Where the Docker Files Are Stored?

Here’s how you can switch the docker storage location.

First, stop the docker daemon.

sudo systemctl stop docker

Next, move your docker directory. In this example we are moving the directory to /hunt/docker but you can choose your own location as long as you change the directory in the subsequent steps as well.

sudo mv /var/lib/docker /hunt/docker

Create /etc/docker/daemon.json and make it look like this:

{
"data-root": "/hunt/docker"
}

Or if the file already exists then add the “data-root” line immediately after the opening brace like this:

{
"data-root": "/hunt/docker",
...existing contents
}

Then start the docker daemon again:

sudo systemctl start docker

At this point you should be able to access AC-Hunter through the web interface, but if not you can try starting it manually using:

sudo hunt up -d --force-recreate


Read More

Can I Send Zeek Logs from an Existing Zeek Sensor to AC-Hunter to Be Analyzed?

Yes.

 

  1. Log in to your Zeek sensor as a user that can read the Zeek logs and can run commands under sudo
  2. Run the following command.  You’ll need to replace “my.achunter.system” with the hostname or ip address of your AC-Hunter system.  “[/zeek/log/top/dir/]” is an optional parameter pointing at the top level directory under which your Zeek logs can be found.  You only need to specify this if it’s not automatically detected.
    curl -fsSL https://raw.githubusercontent.com/activecm/zeek-log-transport/master/connect_sensor.sh -O
    
    curl -fsSL https://raw.githubusercontent.com/activecm/shell-lib/master/acmlib.sh -O
    
    curl -fsSL https://raw.githubusercontent.com/activecm/zeek-log-transport/master/zeek_log_transport.sh -O
    
    bash connect_sensor.sh my.achunter.system [/zeek/log/top/dir/]
  3. If you wish to send your logs to a second AC-Hunter system, repeat step 2 using the second system name or IP address.

 

 

Note; common directories that hold Zeek logs include:

/opt/zeek/logs/ #Zeek as installed by Rita
/usr/local/zeek/logs/ #Zeek default
/var/lib/docker/volumes/var_log_zeek/_data/ #Blue Vector
/nsm/zeek/logs #Security Onion
/storage/zeek/logs/

 


Read More

Mongo Login Failures in delete-databases.sh

If you have trouble running delete-databases.sh , such as:

./delete-databases.sh
exception: login failed
No pattern specified on the command line (such as ./delete-databases.sh '2018-05-31', so we will just list databases available to delete. Press enter when ready to see list.


2019-04-25T11:46:18.589-0400 Error: 18 { ok: 0.0, errmsg: "auth failed", code: 18, codeName: "AuthenticationFailed" } at src/mongo/shell/db.js:1292
Exiting.

This may be a result of a mismatch between the mongo command line client and the mongo server software. To check, run:

mongo --version

and see if you have version 3.6. If not, follow the instructions to upgrade the client at https://docs.mongodb.com/v3.6/administration/install-on-linux/ . It amounts to adding a third party package repo and then installing the “mongodb-org-shell” package.

You can tell if you were successful by running:

mongo --version

and checking that the MongoDB shell version is v3.6.x.


Read More

Troubleshooting Log Transfer

The logs for each machine get placed in a different directory on the AC-Hunter system for each Zeek sensor. They should be under /opt/zeek/remotelogs/{zeek_sensor_name}/ , with additional directories under that for each calendar day (such as /opt/zeek/remotelogs/zeek1__1921681213/2021-04-01/).

Please log in to your Zeek system as the user under which you installed Zeek and make sure you can ssh to the AC-Hunter system with:

ssh [email protected] -i "$HOME/.ssh/id_rsa_dataimport" 'echo Successfully connected.'

The “Successfully connected.” response should come back without having to enter a password; if you are asked for a password there’s something wrong with the ssh key setup.

As that same user on the Zeek sensor, please run:

/usr/local/bin/zeek_log_transport.sh --dest ACH.IP.ADDRESS --localdir /opt/zeek/logs/

Replace ACH.IP.ADDRESS with the address of the AC-Hunter system. If your logs are stored somewhere other than /opt/zeek/logs/ on this sensor, adjust that too. This should start sending logs over to the AC-Hunter system. It’s OK to leave this running; any files you successfully transfer now will not be resent later.

Please check the file that initiates sending logs:

cat /etc/cron.d/zeek_log_transport

It should look like the following:

5 * * * * senduser /usr/local/bin/zeek_log_transport.sh --dest ACH.IP.ADDRESS --localdir /opt/zeek/logs/

“senduser” will need to be the account name on this system under which you did the installation, “ACH.IP.ADDRESS” should be the AC-Hunter system’s IP, and “/opt/zeek/logs/” will need to be the directory where you have Zeek logs on this system.


Read More

Deleting Old Zeek Logs from the AC-Hunter System

The following steps will delete any Zeek logs older than 4 days on the AC-Hunter system. Note that this will not delete any AC-Hunter databases, just the raw Zeek log files that were imported. It will also not delete them from your actual sensors, just the copies that were sent to the AC-Hunter system, so if you ever needed them again you could manually copy them from the originals on your Zeek sensors.

To see what files would be deleted by this command, you can run the following under the “dataimport” account on your AC-Hunter server (run “sudo su – dataimport” if you’re not already logged in as that user, and then run):

find "/opt/zeek/remotelogs/" -type f -mtime +4 -print0 | xargs -0 -r -n 20 echo

 

To set up a daily automatic delete, add the following line to /etc/cron.d/delete_old_zeek_logs . Example command, though you can feel free to use any editor you like:

sudo vi /etc/cron.d/delete_old_zeek_logs

 

Everything from “0 3” to “-f'” is one line. Please be especially careful when typing the path “/opt/zeek/remotelogs/” and make sure there are no spaces between the first and last slash. The quotes on this line (before find and after dash-f) are single quotes (below the double quotes on a US keyboard) and we use double quotes around the log directory.

0 3 * * * dataimport /bin/bash -c 'find "/opt/zeek/remotelogs/" -type f -mtime +4 -print0 | xargs -0 -r -n 20 rm -f'

 

Run both of the following:

sudo service cron reload 2>/dev/null
sudo service crond reload 2>/dev/null

 

Side note: You should not reduce the +4 in the above command; if you deleted logs that were 1, 2, or 3 days old, you’d run the risk of deleting files that zeek_log_transport would then have to copy over again (it sends any new files from the previous 3 days worth of logs every time it runs).


Read More

Safelist (Whitelist) File – JSON Format

The safelist is stored in json format, an industry standard for sharing data. Here’s a small part of the top of the default safelist:

[
{
"Name": "8075",
"Type": "asn",
"Modules": [
{
"Name": "Beacons",
"Src": false,
"Dst": true
}
],
"Comment": "Microsoft patching and time servers"
},
{
"Name": "41231",
"Type": "asn",
"Modules": [
{
"Name": "Beacons",
"Src": false,
"Dst": true
}
],
"Comment": "Ubuntu patching servers"
},
...

 

You have the ability to edit this file to add new entries, take out existing entries, or modify entries. If you do, here are a few notes about the formatting in this file:

You must use double quotes, not single quotes, backquotes or “smart quotes” as used in word processors for all strings. For example, “Name” and “8075” from above are valid; ‘Name’ and `8075` are not.

Whenever using true, false, or null as values, these must be all lowercase.

Inside each matched pair of left and right square brackets (“[” and “]”, json lists), and inside each matched pair of curly braces (“{” and “}”, json dictionaries), the entries are separated by commas, but you don’t use a comma after the final entry. For example:

 {
"Name": "Beacons",
"Src": false,
"Dst": true
}

 

Most Linux distributions and the Mac OS offer a tool called jq (“json query”) that allows you to extract data from and modify json files. It’s not commonly installed by default, but should be available in your package manager. Once installed you can do the following:

To see the full contents in pretty-printed format (like the example above where entries are indented according to how deep they are), use:

cat edited-safelist.json | jq . | less

 

To check whether a json file is in a valid format, run:

$ cat edited-safelist.json | jq . >/dev/null
$

 

When you’re returned to a prompt directly, that means the format appears correct. If the file is not valid json, such as this one where I used single quotes instead of double quotes:

$ cat malformed-safelist.json | jq . >/dev/null
parse error: Invalid numeric literal at line 1, column 16
$

 

you’ll get back some kind of error.

The default json output format (pretty-printing, as seen above) takes a lot of lines to display, especially when you have a large safelist. To give each safelist entry a single line run the following (all on one line, even if wrapped in this document):

( echo '[' ; cat edited-safelist.json | jq -c '.[]' | sed -e '$!s/$/,/' ; echo ']' ) >safelist-perline.json

 

The safelist-perline.json contains the same content and is still a valid json file, but shows the safelist entries one per line, such as:

[
{"Name":"8075","Type":"asn","Modules":[{"Name":"Beacons","Src":false,"Dst":true}],"Comment":"Microsoft patching and time servers"},
{"Name":"41231","Type":"asn","Modules":[{"Name":"Beacons","Src":false,"Dst":true}],"Comment":"Ubuntu patching servers"},
{"Name":"16625","Type":"asn","Modules":[{"Name":"Beacons","Src":false,"Dst":true}],"Comment":"Akamai CDN"},
]

 


Read More