IBM Resilient SOAR, STIX, QRadar, QNI
By Rukhsar Khan on Friday, December 21, 2018
Populate data table and create STIX bundle – steps 7-10
As part of this solution draft we are expressing the CTI provided IOCs in STIX Indicator SDOs and their local relevance in STIX Observed Data SDOs. We are further generating Sighting SROs in order to visualize that we sighted IOCs in the local context. We are also leveraging the Identity SDO in order to specify Recorded Future as an organization that we are receiving indicators from and myself – Rukhsar Khan, Security Analyst – as an individual. In order to put the Identity SDO into a relationship with other SDOs we are making use of the Relationship SRO. For generating a visualized graph we are finally creating a STIX bundle. A STIX bundle packages multiple STIX SDOs and SROs into a single entity which can be loaded into a STIX visualizer1.
Figure 1.24 is illustrating with steps 7-10 how we are achieving the above goal. Step 7 is first of all re-querying QRadar in order to receive flow details on the previously RF matched IP addresses. These flow details are required to provide context around the local relevance. We are expressing the local relevance in a STIX Observed Data SDO and the RF matched IP addresses in an Indicator SDO. As QRadar currently doesn’t provide result sets in STIX format, we need to convert the received QRadar JSON of step 8 into STIX. For this we are using a free IBM Python library called STIX Shifter2 that we have customized to our needs. We are then loading the flow details into a data table available in the Resilient incident (step 9). In step 10 we are further loading the STIX bundle into our Postgres test database.
Figure 1.24: Steps 7-10 – Functions diagram
As you can see in figure 1.25, steps 7-10 are triggered by an automatic rule that has a match condition. The rule name is STIX: Create Indicator and observed data SDO + Sighting SRO and the following match condition is true for these steps: If an Artifact is created AND its Description is equal to “Provided by RECORDED FUTURE TI match”. This calls a corresponding workflow that has a single function. When the function is processed by the defined pre-process script, function processor and postprocess script, this workflow eventually populates the Resilient incident data table and creates the STIX bundle.
The pre-process script is shown in example 1.3. It defines the input variable name inputs.ariel_sql_query with a select statement as the value. This select statement is searching for multiple field values and has also an aggregate function that sums up some fields (sourcebytes, sourcepackets, destinationbytes, destinationpackets) from the QRadar flows table where the sourceip is derived from the Resilient field incident. properties.source_ip_address AND the destinationip is equal to the value of the Resilient incident artifact (artifact.value) that actually triggered this rule/workflow. As this is an aggregate function query, we have a GROUP BY clause that is aggregating on unique sourceip-destinationip-protocol field values. The query time range is further limited to the discovered date of the incident until “now”. As you can also see in lines 3-6, additional input variable names with corresponding values have been defined and are taken as input to the function processor script.
The function processor script does multiple things:
Figure 1.25: Steps 7-10 – Automatic rule / Workflow
1. It sends a query to QRadar by using it’s REST APIs (step 7).
2. It returns a string in QRadar JSON format that becomes the input string of the post-process script (step 8).
3. It converts the QRadar JSON into a STIX Observed Data SDO by using IBM’s STIX Shifter.
4. It creates an Indicator SDO from the artifact value.
5. It creates a Sighting SRO and Identity SROs.
6. It creates a STIX bundle and loads it into our Postgres test database (step 10).
Example 1.3: Steps 7-10 – Pre-process script
The post-process script is represented in example 1.4. The variable in line 1 defines the mapping between the fields returned in the result set and the column names specified in the Resilient data table named “Observed data network traffic”. The API access name of this data table is observed_data_network_traffic. Lines 16- 28 are showing a “for loop” that is iterating over the lines in the result set in order to populate the data table.
Example 1.4: Steps 7-10 – Post-process script
Next, example 1.5 is showing the STIX bundle that has been created and was loaded into our Postgres test database. It starts with a unique id (line 2) and specifies the type which is a bundle (line 3). Line 4 defines an objects list which consists of the Identity SDO for the individual Rukhsar Khan, Security Analyst (lines 5-10), Observed Data SDO (lines 11-42), the Identity SDO for the organization Recorded Future (lines 43-50), and Indicator SDO including the STIX pattern [ipv4addr : value = '18.104.22.168'] that is specifying the object that we are searching for (lines 51-62) and a Sighting SDO for linking the Indicator SDO with the Observed Data SDO (lines 63-72).
Example 1.5: Steps 7-10 – STIX bundle JSON
The graphical representation of the above STIX bundle can be seen in figure 1.26. We are using the open source STIX visualizer introduced above but have done some minor changes in the code.
Figure 1.26: Steps 7-10 – STIX bundle visualized