If you have read anything I have already published here, you will likely see I am somewhat biased to using TheHive as a Cyber security Case Management tool. There are many other tools, but my focus is on enabling a SOC operator no matter the budget for a business (something is well and truly better than nothing).

So with that, I will be walking through the deployment of a collection of VMs designed to operate as a SOC enclave for Incident Investigation and Analysis. All using Open Source tools and technologies which can be acquired for no cost outlay.

First of all let's talk about what we need to occur in this environment, and how we are going to achieve it:

The Environment

This environment will be scaled to the manual job creation process for the moment, ingesting new events as cases or alerts will come in a later article. So for this use case, our input space for cases will be TheHive, which will store the case data in ElasticSearch. TheHive and Cortex are usually installed together, however as analysis scripts become more complex and cumbersome it may be worth separating them out and scaling on more virtual hardware.

In my use case, the following environment will be built:

Elasticsearch Node

Elasticsearch 5.x will be used for this particular deployment of TheHive and Cortex, unfortunately this is part of the compatibility issue with the current product. But once it has been migrated to more recent version I will update the below.

First we need to prep the Elasticsearch Node - In my case Elasticsearch is occupying the IP address: 172.16.0.100/24

sudo apt update -y
sudo apt upgrade -y
sudo apt install wget -y
sudo apt install openjdk-11-jre-headless -y
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list
sudo apt install apt-transport-https -y
sudo apt update -y
sudo apt install elasticsearch -y
sudo apt hold elasticsearch
echo "LimitMEMLOCK=infinity" >> /usr/lib/sysctl.d/elasticsearch.conf
echo "network.host: 0.0.0.0" >> /etc/elasticsearch/elasticsearch.yml
echo "bootstrap.memory_lock: true" >> /etc/elasticsearch/elasticsearch.yml
echo "script.inline: true" >> /etc/elasticsearch/elasticsearch.yml
echo "cluster.name: hive" >> /etc/elasticsearch/elasticsearch.yml
echo "thread_pool.index.queue_size: 100000" >> /etc/elasticsearch/elasticsearch.yml
echo "thread_pool.search.queue_size: 100000" >> /etc/elasticsearch/elasticsearch.yml
echo "thread_pool.bulk.queue_size: 100000" >> /etc/elasticsearch/elasticsearch.yml
systemctl start elasticsearch
systemctl enable elasticsearch

Assuming all goes well in the first 15 seconds or so, Elasticsearch will now be advertising on all interfaces on port 9200. You should however update this in your configuration or put other security measures in place to protect the Elasticsearch indexes.

vi /etc/elasticsearch/elasticsearch.yml

And update the network.host line to reflect a more appropriate value (or interface).

TheHive and Cortex

You may be wondering why would I suggest separating Elasticsearch and TheHive/Cortex in this manner. I feel this is better from a performance perspective once you start ingesting huge volumes of indicators into the environment. Elasticsearch starts to bog down at about the same rate as Cortex performs analysis operations. So separating them onto different hosts allows for expansion of resources (potentially beyond a single host limitation - depending on scale).

TheHive and Cortex will be installed on 172.16.0.50/24, and will need to be able to contact Elasticsearch once configured.

sudo apt update -y
sudo apt upgrade -y
sudo apt install wget curl -y
sudo apt install openjdk-11-jre-headless -y
echo "deb https://dl.bintray.com/thehive-project/debian-stable any main" | sudo tee -a /etc/apt/sources.list.d/thehive-project.list
curl https://raw.githubusercontent.com/TheHive-Project/TheHive/master/PGP-PUBLIC-KEY | sudo apt-key add -
curl https://raw.githubusercontent.com/TheHive-Project/Cortex/master/PGP-PUBLIC-KEY | sudo apt-key add -
sudo apt update -y

sudo apt install -y --no-install-recommends python-pip python2.7-dev python3-pip python3-dev ssdeep libfuzzy-dev libfuzzy2 libimage-exiftool-perl libmagic1 build-essential git libssl-dev
sudo pip install -U pip setuptools
sudo pip3 install -U pip setuptools

sudo apt install cortex -y
sudo apt-mark hold cortex
(cat << _EOF_ 
# Secret key
# ~~~~~
# The secret key is used to secure cryptographics functions.
# If you deploy your application to several instances be sure to use the same key!
play.http.secret.key="$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 64 | head -n1)"
search.host = ['172.16.0.100:9300']
analyzer.urls = ["/opt/Cortex-Analyzers/analyzers"]
_EOF_
) | sudo tee -a /etc/cortex/application.conf 
sudo systemctl enable cortex 
sudo systemctl start cortex
sudo systemctl status cortex

cd /opt/
sudo git clone https://github.com/TheHive-Project/Cortex-Analyzers
chown -R root:cortex Cortex-Analyzers
sudo pip2 install wheel
sudo pip3 install wheel
for I in $(find Cortex-Analyzers -name 'requirements.txt'); do sudo -H pip2 install -r $I; done
for I in $(find Cortex-Analyzers -name 'requirements.txt'); do sudo -H pip3 install -r $I; done

sudo apt install thehive -y
sudo apt hold thehive
(cat << _EOF_
play.http.secret.key="$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 64 | head -n1)"
_EOF_
) | sudo tee -a /etc/thehive/application.conf

systemctl enable thehive
systemctl start thehive

Now that Cortex is operational, we need to configure it to contact our Elasticsearch node (172.16.0.100), and then add our first organisation, and non-administrative user. This user will need to also have an API Key generated, which we will integrate with TheHive to allow usage of the Cortex analyzers. TheHive can be used in isolation of Cortex, but without that API key there will be very little orchestrated analysis conducted.

Configuring Cortex

Open the Cortex URL first (http://172.16.0.50:9001) and you will be prompted to Update the Database, and then create an Administrative User. This admin user will control the whole Cortex instance, but cannot initiate analyses. Once logged in, create your Organisation, and then a Read, Analyse, and Orgadmin user under that Organisation.

Now that TheHiveAPI user has been created, we can now create the API key we will need for TheHive integration. 

 

 

 

Configuring TheHive

Now that TheHive has been installed, we need to update configuration to look at Cortex, and add the API key we generated above to the configuration.

sudo vi /etc/thehive/application.conf

We now need to update some values in application.conf, those being:

# Elasticsearch
search {
  ## Basic configuration
  # Index name.
  index = the_hive
  # ElasticSearch cluster name.
  cluster = hive
  # ElasticSearch instance address.
  host = ["172.16.0.100:9300"]

# Cortex
# TheHive can connect to one or multiple Cortex instances. Give each
# Cortex instance a name and specify the associated URL.
#
# In order to use Cortex, first you need to enable the Cortex module by uncommenting the next line

play.modules.enabled += connectors.cortex.CortexConnector ## UNCOMMENT THIS ##

cortex {
  "CORTEX-ADAMMCHUGH" {
    url = "http://127.0.0.1:9001"
    key = "<API key as generated in Cortex>"
    # HTTP client configuration (SSL and proxy)
    ws {}
  }
}

Once we have those configuration elements updated, we can now restart TheHive and see what happens with its interface.

sudo systemctl restart thehive

Open your favourite browser and browse to http://172.16.0.50:9000 to access TheHive's web interface.

Assuming everything works and the Elasticsearch cluster is reachable on 172.16.0.100, you should see a prompt regarding Database Migration for TheHive. Click through, and create your primary administrator account in here. Once you are logged in, to confirm we are connected to Cortex you will need to look in the bottom right of TheHive's web panel to see the following graphic.

If you can see this image, you are now connected to Cortex through TheHive. Now we just need to walk through configuring the Analyzers, Reporters, Case Templates, and Report Templates... but we're almost at a point now where you could start working on cases.