Security Monitoring and Detection Tips for your Storage Account – Part 1

Capital One breach was one of the biggest data breaches in 2019 which affected over 100 million people. There was a compromised access key that was used to access to an S3 storage bucket (equivalent to Azure Storage Account) to exfiltrate highly sensitive data that Capital One stored. That catastrophic left an open discussion: what would be a security measurement to protect against data breach?

This series is not going to answer the question above. Security is a very broad topic and it would be an endless journey to everyone who has been on. Instead, this series would hopefully provide enough information to build a security monitoring and detection capability in your Azure that could help monitor and detect suspicious and anomaly activity on your storage account.

This series would consist of the following parts:

Understanding Storage Account Logging

Without log, monitoring and detection makes no sense. In a simplest word, it becomes impossible to monitor and detect security threat without having security event logging. In Azure Storage Account, there are three type of logs (without building your own custom log) you would need for security monitoring and detection:

  • Operation Log
  • Storage Metric
  • Storage Analytics

Each of these types are important for monitoring, detection and incident response.

Operation Log

With Operation log, you can get visibility on who created a storage account, who tried to list a Storage Account access key for instance. The operation activity is logged and is retrievable in Azure Activity Log.

The following information would be helpful:

  • Timestamp: when an activity occurred.
  • Operation Name: what was done on a given storage account.
  • Authorization Action: it is a operation namespace of the operation.
  • Scope: the scope that the operation was made against.
  • Caller: who did on a given storage account.
  • Caller IP Address: the client IP address of the caller.
  • Status: status of the request. This tells you whether request was made successfully or not.

Take a look at the following extracted log from Azure Activity that I removed unnecessary info:

{
    "authorization": {
        "action": "Microsoft.Storage/storageAccounts/listKeys/action",
        "scope": "/subscriptions/67d6179d-a99d-4ccd-8c56-4d3ff2e13349/resourceGroups/azsec-corporate-rg/providers/Microsoft.Storage/storageAccounts/az46bex6otpb4ttk"
    },
    "caller": "brian.macdonald@azsec.net",
    "eventTimestamp": "2019-12-09T16:06:14.923547Z",
    "operationName": {
        "value": "Microsoft.Storage/storageAccounts/listKeys/action",
        "localizedValue": "List Storage Account Keys"
    },
    "resourceGroupName": "azsec-corporate-rg",
    "resourceType": {
        "value": "Microsoft.Storage/storageAccounts",
        "localizedValue": "Microsoft.Storage/storageAccounts"
    },
    "resourceId": "/subscriptions/67d6179d-a99d-4ccd-8c56-4d3ff2e13349/resourceGroups/azsec-corporate-rg/providers/Microsoft.Storage/storageAccounts/az46bex6otpb4ttk",
    "status": {
        "value": "Succeeded",
        "localizedValue": "Succeeded"
    },
    "submissionTimestamp": "2019-12-09T16:07:24.2100045Z",
    "subscriptionId": "67d6179d-a99d-4ccd-8c56-4d3ff2e13349"
}

In this example, we have information that a username brian.macdonald (caller ) who accessed to a storage account named az46bex6otpb4ttk in azsec-coporate-rg resource group(resourceId ) to attempted to retrieve storage account access key (authorization action/operation value ) successfully (status ) in 12/09/2019 16:06:14 (UTC time) (eventTimestamp )

To list out all supported operations on storage account you would see in log, run the following PowerShell command:

Get-AzureRmProviderOperation microsoft.storage/* `
        | Where-Object {$_.IsDataAction -eq $false} `
        | Select-Object Operation

Storage Metric Log

Storage metric is another type of log that provides you usage, trace request. They don’t really give you to the bottom but they are very helpful for monitoring from cloud-level API request.

Successful anonymous request on storage account named az46bex6otpb4ttk

There are 6 default metrics in Azure Storage:

  • Transactions
  • Ingress
  • Egress
  • SuccessServerLatency
  • SuccessE2ELatency
  • Availability

There are mix of operational metric versus security metric here. No matter what they are supposed to be they are all useful for incident investigation at some point. For example, look at Transactions metric that provides you the information of request made to a storage service or specific API operation. The dimensions Transactions provide would be good enough. Take a look at the sample construct of this metric:

{
    "ResponseType": "Success",
    "GeoType": "Primary",
    "ApiName": [
        "GetBlob"
    ],
    "Authentication": [
        "Anonymous"
    ]
}

From the sample construct, we know there is a succeeded anonymous request that was made against a blob in a storage account. We don’t know which blob file specifically as well as where the request was originated from but at least we know if this storage is not purposely used for storing public file then there must be something wrong. We will cover more monitoring and detection use cases in the next part of this series.

Email alert when successful anonymous request on storage account named az46bex6otpb4ttk

For details about Azure Storage supported metric, refer to this article.

Storage Analytics Log

Similar to any kind of diagnostics log in Azure, Storage Analytics log provides you much better visibility on what has happened with your storage account. Storage Analytics log can answer the following things:

  • What was the request type (operation) made?
  • What was the blob file?
  • What was the authentication type?
  • What was the IP address the request originated from?
  • What was the user agent used?
  • Was that request made successfully? 

See the example below:

{
    "request_start_time": "2019-12-09T07:01:33.863Z",
    "operation_type": "GetBlob",
    "request_status": "AnonymousSuccess",
    "authentication_type": "anonymous",
    "owner_account_name": "az46bex6otpb4ttk",
    "request_url": "https://az46bex6otpb4ttk.blob.core.windows.net/customerdata/customerB.json",
    "requested_object_key": "/az46bex6otpb4ttk/customerdata/customerB.json",
    "requester_ip_address": "138.59.18.110:43919"
}

The blob file named customerB.json stored in a container customerdata in storage account named az46bex6otpb4ttk that was successfully accessed anonymously from IP 138.59.18.110 (with a NAT port 43919). With such an info you should know what you gotta do then!

To get fully understanding of what is collected in Storage Analytics log, refer to this article.

Conclusion

Understanding what you can collect from Azure Storage service is the very first and imperative step to building security monitoring and detection capabilities for your storage account. You must understand each log type as well as specific log definition so you can analyze it.

In the upcoming path we will explore ways to collect log, as well as where logs should be so we don’t miss any log we need for monitoring, detection and investigation.

You may need to read an article Deploy a compliant Storage account service to get to the right artifact.

This entry was posted in Monitoring & Detection and tagged , . Bookmark the permalink.

5 Responses to Security Monitoring and Detection Tips for your Storage Account – Part 1

  1. Pingback: Security Monitoring and Detection Tips for your Storage Account – Part 2 | All about security on Microsoft Azure

  2. Pingback: Deploy a compliant Storage Account service | All about security on Microsoft Azure

  3. This article is very helpful. Thanks for your time writing in detail.

  4. Pingback: Security Monitoring and Detection Tips for your Storage Account – Part 4

  5. Pingback: Enable storage account analytics logging on all storage accounts

Leave a Reply