Notes on Azure Backup Soft-delete feature in a cybersecurity context

Backup would be the last hope for you in an attempt of recovering your infrastructure after a cyber attack. Malware doesn’t only steal and exfiltrate data but also scans and deletes your backup. The soft delete feature is designed to address such a concern of data destruction.

In this article, let’s look into some aspects of the soft delete feature in¬† Azure Backup.

The soft delete feature protects backup data even if it is deleted accidentally by you or malware. The backup data is retained for 14 additional days. You can undelete backup data within 14 days as well. The backup data will be permanently deleted on the 15th day.

In the cyberattack context, the attacker’s target is not only critical data but also to find a way to destroy your environment to cover track. One of the targets to destroy is the backup and recovery system.

Disabling Soft delete feature

As a cloud administrator, there are a couple of reasons you may want to disable the soft delete feature:

  • You are planning to move your virtual machines to a new Recovery Services vault in a hard deadline.
  • You are working on testing a Recovery Services vault and would like to delete it. Soft delete needs disabling and backup data needs deleting before a recovery services vault can be deleted.

To an attacker, he wants to make sure you can’t recover the system with your backup data. In this particular case, he would need to disable the soft delete feature then delete all backup data.¬†

That can be also done by PowerShell, Azure CLI or Azure REST API. With PowerShell, you can run the following code snippet:

$vaultName = "azsec-vault"
$vaultRgName = "azsec-test-backup-rg"

$vault = Get-AzRecoveryServicesVault -ResourceGroupName "$vaultRgName" `
                                     -Name "$vaultName"
Set-AzRecoveryServicesVaultProperty -VaultId $vault.Id `
                                    -SoftDeleteFeatureState Disable

You can do with Azure CLI as follows:

vaultName="azsec-vault"
vaultRgName="azsec-test-backup-rg"

az backup vault backup-properties set \
                        --name $vaultName \
                        --resource-group $vaultRgName \
                        --soft-delete-feature-state Disable

Once the script is successfully executed the attacker can delete a backup data and backup item (virtual machine) to ensure no backup in the soft delete state is retained.

$vaultName = "azsec-vault"
$vaultRgName = "azsec-test-backup-rg"
$backupManagementType = "AzureVM"
$containerType = "AzureVM"

$vault = Get-AzRecoveryServicesVault -ResourceGroupName "$vaultRgName" `
                                     -Name "$vaultName"

$containers = Get-AzRecoveryServicesBackupContainer -ContainerType $containerType `
                                                    -BackupManagementType $backupManagementType `
                                                    -VaultId $vault.ID

foreach ($container in $containers) {
    $backupItem = Get-AzRecoveryServicesBackupItem -Container $container `
                                                   -WorkloadType $containerType `
                                                   -VaultId $vault.ID
    Disable-AzRecoveryServicesBackupProtection -Item $backupItem `
                                               -RemoveRecoveryPoints `
                                               -VaultId $vault.ID `
                                               -Force
}

Here is what may happen. If you try to delete a soft deleted backup item Azure will deny the request. You must undelete it first using Undo-AzRecoveryServicesBackupItemDeletion

Once undelete operation is executed successfully you can safely remove the recovery services vault

Monitoring and Detection

Alright here is the important part. Disabling Soft delete requires Backup Contributor only which may be granted unattended. An insider threat actor with this permission could disable Soft delete and wait for his malware to do the rest.

Given the Role Assignment watchlist (refer to this article), you can do write a query to audit people/groups that are being granted Backup Contributor.

_GetWatchlist('azsec_role_assignment')
| where RoleDefinitionName == "Backup Contributor"
| project LastUpdatedTimeUTC, DisplayName, ObjectId, SignInName, Scope

Every soft delete removal causes an alert with Critical severity. Administrator will receive a warning email.

You can go to the Recovery Services vault and check alert under Monitoring

For AzureActivity log, pay attention to the following operations:

  • Microsoft.RecoveryServices/vaults/backupconfig/write
  • Microsoft.RecoveryServices/vaults/monitoringAlerts/write
  • Microsoft.RecoveryServices/vaults/backupFabrics/protectionContainers/protectedItems/delete

Prevention

Well, there isn’t really a good preventive way in this case because you still need someone that is in charge of backup and recovery operation in your environment. The key thing is to monitor and keep an eye on the Soft delete feature. If someone needs to disable it he must submit a ticket or seeks approval. Otherwise, any action to disable the feature is considered suspicious activity.

Recovery Services Vault with Soft Delete disabled

For testing purposes use the following ARM template to create a Recovery Services vault that has the Soft delete feature disabled:

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "baseName": {
      "type": "string",
      "defaultValue": "azsec",
      "metadata": {
        "description": "The name of Recovery Service Vault"
      }
    }
  },
  "variables": {
    "vaultName": "[concat(parameters('baseName'), 'vault')]"
  },
  "resources": [
    {
      "type": "Microsoft.RecoveryServices/vaults",
      "apiVersion": "2015-11-10",
      "name": "[variables('vaultName')]",
      "location": "[resourceGroup().location]",
      "sku":{
          "name": "RS0",
          "tier": "Standard"
      },
      "properties": {
      },
      "resources": [
        {
          "type": "backupconfig",
          "apiVersion": "2021-04-01",
          "name": "vaultconfig",
          "location": "[resourceGroup().location]",
          "dependsOn": [
            "[resourceId('Microsoft.RecoveryServices/vaults', variables('vaultName'))]"
          ],
          "properties": {
            "enhancedSecurityState": "Enabled",
            "softDeleteFeatureState": "Disabled"
          }
        }
      ]
    }
  ]
}

enhancedSecurityState¬†must be explicitly set “Enabled”. Otherwise, you will receive Status Message: Enhanced Security is enabled for this vault. You cannot disable Enhanced Security after enabling it. Enhanced Security is enabled for this vault. You cannot disable Enhanced Security after enabling it. (Code:UserErrorChangeEnhancedSecurityStateNotAllowed)

Make sure to only deploy in your test environment. You should never disable soft delete from the beginning of your deployment.

Conclusion

This article gives you a bit about the Soft delete feature in a cybersecurity context. Backup and Recovery system is always a target of a cyber attack. And the Soft delete feature would be a blocker to destroying backup data. Hence, it would be a list of targets.

This entry was posted in Monitoring & Detection, Security Operation and tagged . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *