As a SOC Analyst or Manager who are working on Azure Sentinel you would like to have a view of how productive your team is (response time, resolution..). As being familiar with Log Analytics query, you might wish to do kind of query to get Azure Sentinel incident without writing any script to call Azure Sentinel Incident API.
In this article, let’s see how to ingest Azure Sentinel incident data using Logic App to make Azure Sentinel incident data available in Log Analytics workspace.
Use Case
With Azure Sentinel incident data available in a Log Analytics workspace, you can query them to get specific incident info quickly. You can also give your boss a view of incident with different metrics like:
- Total number of incidents per week/month
- Incident rate (last month vs. this month)
- Resolution time average
- Top resources with incident
- Top productive analyst (by whom triaging and closing incident..)
There are many more metrics you would like to put in your dashboard (workbook) .
Pre-requisites
There are a couple of pre-requisites that you need before building the Logic App workflow:
- Register Microsoft.Logic resource provider in your subscription. Refer to this article if you don’t know how to do it/want to verify.
- Enable System-assigned Managed Identity (SAMI) on the Logic App you are going to build a workflow. Refer to this article fore more information.
- Grant SAMI object a Read role or Azure Sentinel Reader in order to let it read data from the workspace
- Basic understanding of building Logic App. I’d highly recommend you to read “Notify Azure Sentinel alert to your email automatically” step-by-step article
Step-by-step Guidance
Step 1: Create a recurrence trigger
Our goal is to make sure to trigger Logic App to run at regular for example every 24 hours. In Logic App, you can use Recurrence trigger.
By specifying 24 hours interval my Logic App is triggered every 24 hours to get incidents from Azure Sentinel.
To name the step for more meaningful, you can click […] > Rename
Step 2: Create a subtract from Now() time span
The idea here is to get incident for a specific period of time. For example you would want to get Azure Sentinel incidents for the last 24h from the date & time when the Logic App is triggered. In Azure Logic App, you can use subtractFromTime function from Date and Time action to subtract a number of time units from a timestamp.
In Base time field, you can set an expression utcNow() in order to specify the start time you want. Moreover, UTC is the accepted format in a Request URI which is needed later.
In Interval and Time unit field, set interval and unit time. In this article, I would like to get incidents for the last 24 hours
Step 3: Initialize Log Analytics workspace Variable and Incident
A request to Azure Sentinel Incident API requires a workspace ID. In other words Log Analytics workspace name, resource group and subscription ID are required.
In Azure Logic App, you can use Initialize variable function from Variable action when you want to create an object that is used in another Logic App step.
Next step is to initialize incident object which stores all incident data which we will send to Log Analytics workspace.
Step 4: Create Azure Sentinel Incident Request URI
This is a very important step in the entire solution. You will need to make a valid request URL that is accepted by Azure Sentinel API back-end. The action we need is Initialize variable function that was described in step 3. It data type is String.
In Value field, we need to pass initialized variable’s value from step 3. In fact you don’t need step 3 for the workflow. You can create a string for your request with a constant value as follows:
https://management.azure.com/subscriptions/67d6179d-a11d-4ccd-1c16-4d3ff2e13349/resourcegroups/azsec-corporate-rg/providers/Microsoft.OperationalInsights/workspaces/azsec-shared-workspace/providers/Microsoft.SecurityInsights/incidents/?api-version=2020-01-01
Since we use variable as well as we want to get query from the last 7 days you need to append filter in the request.
For each of variable you need an expression:
- subscriptionId: variables(‘workspace’)[‘subscriptionId’]
- resourceGroup: variables(‘workspace’)[‘resourceGroup’]
- workspaceName: variables(‘workspace’)[‘workspaceName’]
These are referenced from previous step.
For date & time filter, we can add $filter=properties/createdTimeUtc ge to the request URL as it is accepted by OData standard. “ge” stands for greater – equal. The last one is to add subtracted time you created in step 2.
Step 5: Retrieve Incident with Azure Sentinel API
Azure Sentinel Incident API wouldn’t give you all incidents when being called. It has paging though. So we wouldn’t just call the API once. We would need to use nextLink to get all incidents. Technically speaking, there should be a reapeting request call until nextLink becomes null.
In Azure Logic App, you can use Until function (under Control action) to run and repeat action until a condition is met. This said condition is when nextLink becomes null.
Inside Until branch we need a HTTP action. Method used in our approach is GET. URI is referenced from the requestUrl value we created in previous step.
For Authentication, we use Managed Identity because it’d be the most seamless way. You can use Active Directory OAuth but you need to create a new service principal then.
Now you need to set a condition for your Until loop. Click Edit in Advanced mode and set the following condition:
@equals(body('Send_GET_request_to_Incident_API')?['nextLink'], null)
Click Edit in basic mode, you can see that the condition is automatically converted to human-readable format.
Step 6: Join Incident Array
For each of loop’s response there is a response which we need to join to the initialized incident array (created in Step 3). To do this we have a few options. In this article I’d like to use the basic Compose action with union() function. Below is the sample expression to be set in Inputs field.
union(variables('incidents'),body('Send_GET_request_to_Incident_API')?['value'])
Step 7: Compose final Incident data set
Similar to step 6, this step is to compose a final incident data set prior to sending it to the Log Analytics workspace.
You can find the output from Dynamic content under Append Incident Response Value which is in step 6.
Step 8: Create Log Analytics Send Data action
Send Data is an action under Azure Log Analytics Data Collector allowing you to send data (only JSON is accepted) to a pre-connected Log Analytics workspace.
The action requires a connection name (just a general name), workspace and workspace key to connect to your workspace. Once it is connected you can specify input data which is from the preview step.
Step 9: Save and Run
Once everything is done you can save the Logic App workflow and run it.
Known-issue
The workflow is not fully optimal. The article shows you some capabilities to build a ingestion pipeline. There would be issues as follows:
- Data duplicate: the workflow doesn’t provide any de-duplicate solution. That said, you would see same incident with same GUID in the Log Analytics workspace. In this case you could use distinct() or summarize count() function in your query.
- nextLink: I cannot provide 100% guarantee on nextLink call. There might be lack of data from the call. Sample reference from nextLink paging is here and here
Data you see in the Log Analytics workspace is the same data from the API response. If you would like to process it in the workflow you would need more steps with more actions (e.g. Parse JSON…)
Query and Workbook
When data is available, it is the time for doing some queries, or building a workbook. For example you can compare incident severity
azsec_incident_CL | summarize count() by properties_severity_s
and make it a chart
or count by incident title
azsec_incident_CL | summarize count() by properties_title_s
and make a chart
Conclusion
This article introduces some helpful actions to help you build a Logic App workflow to transform Azure Sentinel incident to a Log Analytics workspace so you can do query or build a dashboard for your SOC monitoring.
If you have any feedback or issue please feel free to email to azsecblog@gmail.com I’d be very happy to help.
Pingback: Extract all Azure Sentinel incidents