Azure Firewall Monitoring 101

My last article was to give you an overview of Azure Firewall – a managed firewall service Microsoft recently announced in public preview, and also guidance on how to set it up. There have been some positive feedbacks along with questions about monitoring Azure Firewall traffic. In fact, without monitoring, you wouldn’t know what would have happened in your network, specific to traffic gone through your firewall to the Internet

This article is going to give you guidance on how to monitor Azure Firewall traffic using Azure Log Analytics. This also gives you some sample queries which are hopefully helpful to your security monitoring plan.

Disclaimer: this article only focuses on using built-in Azure service to monitor your Azure Firewall. For external party or building a monitoring system programmatically, hopefully something will come out from my blog.

Azure Firewall Log Overview

When it comes to Azure Logging, there are commonly two types of log:

  1. Activity Log: this is cloud-based resource log. When you modify or change something on an Azure resource, the activity is logged. For example, you create a new firewall rule in Azure Firewall.
  2. Resource Log: it targets specific to resource configuration.

Azure Firewall activity log initially provides the following event:

  • Microsoft.network/azureFirewalls/write
  • Microsoft.network/azureFirewalls/read
  • Microsoft.network/azureFirewalls/delete

Activity Log is imperative to monitoring resource modification. It’s to ensure the integrity of cloud resource and prevent resource from being modified by unauthorized access. Activity logs can be viewed from every service or Azure Monitor. You can query Azure Firewall activity log from Log Analytics.

Note: This article doesn’t focus on Activity Log monitoring, for more information about it, read here https://docs.microsoft.com/en-us/azure/monitoring-and-diagnostics/monitoring-overview-activity-logs

Azure Firewall resource log is categorized into two types: AzureFirewallApplicationRule and AzureFirewallNetworkRule. Each type represents each firewall rule type. Azure Firewall log can be stored in a storage account, or stream to event hub or is sent to Log Analytics.  Depending on the need and scenario, the log location may vary.

Storing Azure Firewall log in storage account can be considered the cheapest solution and would be helpful in log archival. Log in storage account can be streamed and pre-processed with Stream Analytics before being pushed to external service (e.g. Event Hub, PowerBI for visualization, SIEM for advanced monitoring).

Using Event Hub to store Azure Firewall log is a good choice in case you’d like to handle near real-time event of your Azure Firewall. Logs are continuously sent to Event Hub in which a consumer (e.g. a SIEM, real-time dashboard, telemetry…) receives. This approach is normally used when there is a large numeric network volume.

The last option is Log Analytics. Log Analytics has been significantly improved and now is considered a single source of log for almost Azure service. Not only log repository, Azure Log Analytics provides a powerful framework of query language (aka Kusto as code name)which you can query very specific Azure resource’s event. If there is not a SIEM or 3rd network monitoring system, I’d recommend you to review Log Analytics as a log and monitoring tool in your Azure environment.

Lastly, no matter where you chose to store your Azure Firewall log, remember you are always given the ability to query and manipulate log. All of the three options allow you to query through REST API.

Azure Firewall Log construct

Before writing query to retrieve log, you should understand Azure Firewall log construct. Below is the body of application rule in Azure Firewall

TimeGenerated    :7/22/2018 1:30:43.738 PM
Category         :AzureFirewallApplicationRule
OperationName    :AzureFirewallApplicationRuleLog
msg_s            :HTTPS request from 192.168.2.4:49439 to md-qcwlf4kmk11q.blob.core.windows.net:443. Action: Allow. Azure internal traffic.
Resource         :INFRA-RGAZFW
SourceSystem     :Azure
ResourceId       :/SUBSCRIPTIONS/A52B7C5C-C629-48E1-A01A-54D27BB40BE4/RESOURCEGROUPS/INFRA-RG/PROVIDERS/MICROSOFT.NETWORK/AZUREFIREWALLS/INFRA-RGAZFW
SubscriptionId   :a52b7c5c-c629-48e1-a01a-54d27bb40be4
ResourceGroup    :INFRA-RG
ResourceProvider :MICROSOFT.NETWORK
ResourceType     :AZUREFIREWALLS

And network rule as follows:

TimeGenerated    :7/22/2018 12:56:11.909 PM
Category         :AzureFirewallNetworkRule
OperationName    :AzureFirewallNetworkRuleLog
msg_s            :TCP request from 192.168.2.4:49352 to 13.107.21.200:443. Action: Deny
Resource         :INFRA-RGAZFW
SourceSystem     :Azure
ResourceId       :/SUBSCRIPTIONS/A52B7C5C-C629-48E1-A01A-54D27BB40BE4/RESOURCEGROUPS/INFRA-RG/PROVIDERS/MICROSOFT.NETWORK/AZUREFIREWALLS/INFRA-RGAZFW
SubscriptionId   :a52b7c5c-c629-48e1-a01a-54d27bb40be4
ResourceGroup    :INFRA-RG
ResourceProvider :MICROSOFT.NETWORK
ResourceType     :AZUREFIREWALLS

Because Azure Firewall is still under public preview, it may not meet your expectation. For example, you would need a specific Action property so you would not have to query keyword inside the msg_s . Or the TCP packet should be recorded. Here is what I’d expect

{
    "msg_s": {
        "source": "192.168.1.2:49032",
        "destination": "13.107.21.200:443",
        "action": "Deny",
        "protocol": [
            "https",
            "http"
        ],
        "message": "Network packet is dropped due to firewall rule",
    }
}

Azure Firewall Log Query

Before you can start writing a query, make sure you have already enabled diagnostic log to be sent to Log Analytics.

The simplest query to get started with Azure Firewall log is to retrieve category as follows:

AzureDiagnostics 
| where Category == "AzureFirewallNetworkRule"

If you want to count number of log items, use the following query:

AzureDiagnostics 
| where Category == "AzureFirewallNetworkRule" 
| summarize count()

If you have more than one Azure Firewall in your subscription and you want to retrieve log from one of them:

AzureDiagnostics 
| where Category == "AzureFirewallNetworkRule" 
    and Resource  == "INFRA-RGAZFW"

…if you want to retrieve log within 24 hours:

AzureDiagnostics 
| where TimeGenerated > ago(24h)
    and Category == "AzureFirewallNetworkRule"

If you want to query only Deny log:

AzureDiagnostics 
| where TimeGenerated > ago(24h)
    and Category == "AzureFirewallNetworkRule" 
    and msg_s contains "Deny"

Using contains operator should not be a problem, but would not be a good approach since the msg_s is just a string. Perhaps when Azure Firewall gets mature, we would see the JSON format containing details of TCP packet. If you’d like to count the number of Deny log items, just add | summarize count()  into your query.

If you don’t want to retrieve all log items coming from Azure Firewall:

AzureDiagnostics 
| where ResourceType == "AZUREFIREWALLS"

If you would like to know where a denied traffic comes from a specific IP address:

AzureDiagnostics 
| where ResourceType == "AZUREFIREWALLS" 
    and msg_s contains "request from 192.168.2.4" 
    and msg_s contains "Deny"

or kind of fancy like the below (try yourself and you will understand what it does)

let deny_log = 
    AzureDiagnostics
    | where Category == "AzureFirewallApplicationRule"
        and msg_s contains "Deny";
deny_log
| summarize count() by msg_s

or you would like to shorten using let .

let source = "request from 192.168.2.4";
let action = "Deny";
AzureDiagnostics 
|where ResourceType == "AZUREFIREWALLS" 
    and msg_s contains (source)
    and msg_s contains (action)

and if you’d like to count and render both log categories into pie chart:

let source = "request from 192.168.2.4";
let action = "Deny";
AzureDiagnostics 
|where ResourceType == "AZUREFIREWALLS" 
    and msg_s contains (source)
    and msg_s contains (action)
|summarize count() by Category
|render piechart

There are probably more queries you’d like to write for getting more information about rule. You can get started with Log Analytics query from here.

Log visualization 

Writing and executing each query individually is not enough. You’d expect to see all of query’s results in a single place. In Azure Log Analytics you can create and design your dashboard to show result. Here is where you can get started with.

Although Workspace designer doesn’t provide you many charts like PowerBI and the way you visualize is not really flexible, it is still acceptable as a single source of result. Below is my sample workspace you can download to have a look.

Conclusion

The article shows you the way to retrieve Azure Firewall log and how to visualize it using Workspace designer feature in Log Analytics. As said, the gallery provides limited number of charts. If you need more than that, you can import Log Analytics to Power BI and build your own dashboard.

There may be more complex advanced queries to manipulate value inside Azure Firewall message content (msg_s). This content should have been constructed in a way people can easily retrieve. I will come back with more queries soon.

This entry was posted in Monitoring & Detection and tagged , . Bookmark the permalink.

2 Responses to Azure Firewall Monitoring 101

  1. Pingback: All about security on Microsoft Azure

  2. Pingback: Azure Firewall (Public Preview) Automation – Part 1 | All about security on Microsoft Azure

Leave a Reply

Your email address will not be published. Required fields are marked *