Beacons Proxy and Whitelisting

If you’re intending to use the Beacons Proxy tab to investigate connections sent through an HTTP proxy and you’re running AC-Hunter 5.3.1 or lower, you should be aware of an interaction with the whitelisting module. Let’s say your HTTP proxy is at IP address 8.7.6.5 and has a hostname of “proxy1.example.com”. If you whitelist any of the following you will not see any entries in the Beacons Proxy tab:

• The IP address 8.7.6.5
• The hostname proxy1.example.com
• The domain example.com with a wildcard

 

To fix this, please do all of the following:

1) Upgrade to AC-Hunter 5.4.0 or above. You can download this from your portal account.

2) Edit /etc/AC-Hunter/rita.yaml with your preferred editor under sudo.

sudo nano /etc/AC-Hunter/rita.yaml

Edit the “AlwaysInclude” line so that the IP address of the proxy is included in the list. Here’s an example:

AlwaysInclude: ["8.8.8.8/32","place_proxy_internal_ip_here/32"]

Make sure this line continues to have the same number of leading spaces as the InternalSubnets line, has no leading tabs, uses standard double quotes, has a “/32” after the proxy IP address, and has no spaces between the square brackets.

3) Restart AC-Hunter with:

hunt down
hunt up -d --force-recreate

4) The entries will start to show up in the Beacons Proxy tab over the next 24 hours. You do not need to remove the whitelist entry to handle this issue in the Beacons Proxy tab.

 

Long Connections and Whitelisting

Long Connections that are sent through the proxy will also see this problem if the proxy is whitelisted with any of the 3 methods listed above in AC-Hunter versions 5.4.0 and below (5.4.0 is the most recent version as of 9/14/2021).

To see Long Connections routed through a proxy, please remove any whitelist entries that match this proxy system. As new data is imported it will show up again in your rolling databases over the next 24 hours.

We anticipate that a future release of AC-Hunter will address this and allow inspecting proxy-routed Long Connections while still whitelisting the proxy system.

 

Direct Link to this FAQ Item: https://portal.activecountermeasures.com/support/faq/?Display_FAQ=3884

If you have a safelist (whitelist) file that was created under AC-Hunter 4.0.0 or earlier (by selecting “Download Whitelist” from the Settings box) and wish to convert it to the new format needed for version 5.0.1 or above, please do the following:

# Please note that you must put your safelist (whitelist) file in /opt/zeek/remotelogs/

cp -p path_to_your_old_whitelist_file.json /opt/zeek/remotelogs/old-whitelist.json

cd /opt/zeek/remotelogs/

hunt run --rm -u 0 api /home/api/middleware upgrade-whitelist /opt/zeek/remotelogs/old-whitelist.json /opt/zeek/remotelogs/new-whitelist.json

 

You can now copy /opt/zeek/remotelogs/new-whitelist.json to where you keep your whitelist backups. You can also import this new safelist (whitelist) into AC-Hunter version 5.0.1 and above. (Note, if you’re running 5.0.0, please upgrade to 5.0.1).

 

Direct Link to this FAQ Item: https://portal.activecountermeasures.com/support/faq/?Display_FAQ=3229

(NOTE: This only applies to AI-Hunter version 3.8.0; 3.8.0 was the first version with BeaKer included.  Versions 4.0.0 and higher have this bug fixed.  To find out your AI-Hunter version, go to the Dashboard, Settings (gear in the upper right), and “About”.)

On your AI-Hunter system, look at /etc/AI-Hunter/config.yaml . Near the bottom of that file you should have a line starting with “BeakerHost:”, like one of the following forms:

BeakerHost: “https://14.96.107.22:5601”
or
BeakerHost: “https://beakerhostname.example.com:5601”
or
BeakerHost: “https://[2604:a340:206:d94::13:8001]:5601”
or
BeakerHost: “https://2604:a340:206:d94::13:8001:5601”

Each of these forms has the IP address or hostname of the BeaKer server, as well as the port on which that server is run (5601 by default).

The first three forms are fine – they tell AI-Hunter how to reach the BeaKer server when it’s on an IPv4 address, hostname, or IPv6 address, respectively. If you have one of these first three, this faq entry doesn’t apply to you; contact [email protected] for help.

The fourth form is almost identical to the third, but is missing the square brackets around the IPv6 address. These are required, so if your “BeakerHost” line is missing them as well, do the following:

• While still logged in to your AI-Hunter server, edit the file /etc/AI-Hunter/config.yaml (substitute your favorite editor for vim):

sudo vim /etc/AI-Hunter/config.yaml

• Scroll down to the BeakerHost line.
• Edit that line and add a left square bracket immediately after “://”
• Add a right square bracket immediately before “:5601”
• While the IPv6 address inside the brackets will be different, your line should look like:

BeakerHost: "https://[2604:a340:206:d94::13:8001]:5601"

• Save the file and exit
• Run the following commands:

cd ~/AIH-source/AI-Hunter-latest/
sudo ./hunt down
sudo ./hunt up -d --force-recreate

Go back to your AI-Hunter console, force a reload of the page with shift-ctrl-R, and try clicking on the BeaKer icon again – a new tab should be opened with the BeaKer console.

 

Direct Link to this FAQ Item: https://portal.activecountermeasures.com/support/faq/?Display_FAQ=2757

To use this feature, you must be using version 3.4.0 or higher (we strongly recommend 3.4.1 or higher).

Overview: To create your own blacklist, you’ll create a file (“/etc/AC-Hunter/blacklist/ips.txt”) on the Rita/AC-Hunter system with the ipv4 and ipv6 addresses listed one per line, instruct Rita to use this file by editing “rita.yaml”, and load these new addresses into Mongo. Once this is done, these addresses will be tagged as blacklisted on new data imported from Zeek (though old Zeek logs will not be modified).

Detailed steps:

1. Add the following block to “/etc/AC-Hunter/rita.yaml” , verbatim. We don’t recommend changing the filename in this release.  Note that the second and third lines need to be indented with spaces.

BlackListed:
  # Lists containing both IPv4 and IPv6 addresses are acceptable
  CustomIPBlacklists: ["/etc/AC-Hunter/blacklist/ips.txt"]

2. Create “/etc/AC-Hunter/blacklist/ips.txt” and add your IPs, one per line.

3. After creating this file – and every time you make a change to it – run the following commands:

If you’re running AC-Hunter 4.0.0 or higher:

rita test-config
hunt run --rm db_client mongo_cmd.sh 'db.getSiblingDB("rita-bl").dropDatabase()'
hunt up -d --force-recreate

 

If you’re running AI-Hunter 3.8.0 or lower:

cd ~/AIH-source/AI-Hunter-latest/
rita test-config
./hunt run --rm db_client mongo_cmd.sh 'db.getSiblingDB("rita-bl").dropDatabase()'
./hunt up -d --force-recreate

The “rita test-config” will tell you if there are any errors in the rita configuration file.

The ip addresses you’ve placed in ips.txt will be tagged as blacklisted in log files imported from this point on. Logs that were imported previously will not show these IP addresses as blacklisted.

Direct Link to this FAQ Item: https://portal.activecountermeasures.com/support/faq/?Display_FAQ=1890

Yes there is! Run the following commands on your AC-Hunter system, depending on your version.

 

If you are using AC-Hunter 5.3.0 or higher, please run the following:

/usr/local/bin/import_pcaps.sh -p pcap_filename.pcap -d database_name

 

If you have AC-Hunter 5.2.0 or lower:

Set the following variables to your own values:

export PCAP_FILE=/absolute/path/to/file.pcap
export BRO_DIR=/absolute/path/you/want/bro/logs/
export DATABASE=yourdatabasename

Convert your pcap to Bro logs:

sudo docker run --rm --volume "$PCAP_FILE:/capture.pcap" --volume "$BRO_DIR:/pcap" --env BRO_DNS_FAKE=true blacktop/bro:2.5 -r /capture.pcap local

Import the Bro logs into AI-Hunter using RITA:

~/AIH-source/AI-Hunter-latest/rita import $BRO_DIR $DATABASE

 

At this point your database should be visible in AC-Hunter.

 

Direct Link to this FAQ Item: https://portal.activecountermeasures.com/support/faq/?Display_FAQ=869

---

The safelist is stored in json format, an industry standard for sharing data. Here’s a small part of the top of the default safelist:

[
{
"Name": "8075",
"Type": "asn",
"Modules": [
{
"Name": "Beacons",
"Src": false,
"Dst": true
}
],
"Comment": "Microsoft patching and time servers"
},
{
"Name": "41231",
"Type": "asn",
"Modules": [
{
"Name": "Beacons",
"Src": false,
"Dst": true
}
],
"Comment": "Ubuntu patching servers"
},
...

 

You have the ability to edit this file to add new entries, take out existing entries, or modify entries. If you do, here are a few notes about the formatting in this file:

You must use double quotes, not single quotes, backquotes or “smart quotes” as used in word processors for all strings. For example, “Name” and “8075” from above are valid; ‘Name’ and `8075` are not.

Whenever using true, false, or null as values, these must be all lowercase.

Inside each matched pair of left and right square brackets (“[” and “]”, json lists), and inside each matched pair of curly braces (“{” and “}”, json dictionaries), the entries are separated by commas, but you don’t use a comma after the final entry. For example:

 {
"Name": "Beacons",
"Src": false,
"Dst": true
}

 

Most Linux distributions and the Mac OS offer a tool called jq (“json query”) that allows you to extract data from and modify json files. It’s not commonly installed by default, but should be available in your package manager. Once installed you can do the following:

To see the full contents in pretty-printed format (like the example above where entries are indented according to how deep they are), use:

cat edited-safelist.json | jq . | less

 

To check whether a json file is in a valid format, run:

$ cat edited-safelist.json | jq . >/dev/null
$

 

When you’re returned to a prompt directly, that means the format appears correct. If the file is not valid json, such as this one where I used single quotes instead of double quotes:

$ cat malformed-safelist.json | jq . >/dev/null
parse error: Invalid numeric literal at line 1, column 16
$

 

you’ll get back some kind of error.

The default json output format (pretty-printing, as seen above) takes a lot of lines to display, especially when you have a large safelist. To give each safelist entry a single line run the following (all on one line, even if wrapped in this document):

( echo '[' ; cat edited-safelist.json | jq -c '.[]' | sed -e '$!s/$/,/' ; echo ']' ) >safelist-perline.json

 

The safelist-perline.json contains the same content and is still a valid json file, but shows the safelist entries one per line, such as:

[
{"Name":"8075","Type":"asn","Modules":[{"Name":"Beacons","Src":false,"Dst":true}],"Comment":"Microsoft patching and time servers"},
{"Name":"41231","Type":"asn","Modules":[{"Name":"Beacons","Src":false,"Dst":true}],"Comment":"Ubuntu patching servers"},
{"Name":"16625","Type":"asn","Modules":[{"Name":"Beacons","Src":false,"Dst":true}],"Comment":"Akamai CDN"},
]

 

Direct Link to this FAQ Item: https://portal.activecountermeasures.com/support/faq/?Display_FAQ=859

---