What Blue Team needs to know about Run Script feature in Azure

Run Script is great feature that help cloud system admin perform command or script execution on target virtual machine without RDP or setting up a PsRemote that may not be allowed in your organization. Nontheless Run Script also allows bad actor to perform a malicious command if he has enough permission. That would become worst if the malcicous execution is succeeded.

As a Blue teamer working on Azure, there should be a deep understanding of how Run Script works as well as how to detect and trace what were run on a compromised virtual machine.

Feature Overview

The very first thing we normally do when trying to understand something is to exercise. Specific to this case, we need to perform a Run Script on a virtual machine and observe it. In Azure virtual machine, you can use Azure Portal to execute a script or command. Microsoft provides 10 built-in scripts (except RunPowerShellScrit is just an editor to author a script).

When you click on any of built-in scripts you can see the script that is made. For example take a look at EnableRemotePS which is kind of interesting:

You can execute your own custom script by using RunPowerShellScript. Let’s try this one below to get OS information of the virtual machine.

Not only Azure Portal, you can use Azure CLI or PowerShell to run a script. Let’s say you want to run a script below on a virtual machine:

To run this script using Azure PowerShell module, use Invoke-AzVMRunCommand  as follows:

Once the script execution is succeeded the Message field gives you the script output.

Deeper Understanding

So far you know how to run a script remotely. You may check VM Extension to from Azure Portal to see if there is one. Unfortunately Run Script doesn’t actually create an extension. All stuffs that come with Run Script on Windows virtual machine are stored in C:\Packages\Plugins\Microsoft.CPlat.Core.RunCommandWindows\{version}.

For Linux you can find Run Script extension, configuration in /var/lib/waagent/run-command

There are two things that are involved entirely from triggering a Run Script API to executing that script:

  • Command Invocation
  • Script Execution

Command Invocation

Because we don’t know what API does behind the scene so we could only dig into PowerShell module DLL. We can find module path by simple command below:

There are two DLLs that we would need to dig into is Microsoft.Azure.Management.Compute.dll and Microsoft.Azure.PowerShell.Cmdlets.Compute.dll . There are free decompilers you can use. I use JetBrains dotPeek to dig into DLL in Windows. There are a lot of classes in this DLL. However I’d only pay attention to Microsoft.Azure.Commands.Compute.Automation  . There is a class named InvokeAzureRmVMRunCommand  . In this command, there is a code snippet that catches my attention:

This code simply checks null in ScriptPath parameter and then read the file and split the script with ReadFileAsText()  then use Split()  function with \r  and \n to split the script string. So the friendly format of the above HelloWord example becomes the following

Now we have an idea about your script content. Next step is to find out where script content is stored before it is sent along with request body to Azure REST API. Normally there is a client object that is initialized. Specific to Azure Computer, it is ComputerManagementClient . During the search in this object, I found an initiation:

The next step is to locate VirtualMachineRunCommandsOperations() as well as request body in a HTTP Request made to Azure API. From the search, I found an object named RunCommandDocumentBase  that is a JSON object.

You may find more useful information of Run Command/Run Script feature. At this point, we know that the request body contains a long string , and parameter to be used.

Script Execution

Most of the features that interact with Azure virtual machine Microsoft provides an extension – they call it extension. And they are dropped to the path C:\Packages\Plugins and extension log is at C:\WindowsAzure\Logs\Plugins .You may have known common extensions like:

  • Microsoft.Azure.Diagnostics.IaaSDiagnostics
  • Microsoft.Azure.Security.IaaSAntimalware
  • Microsoft.Compute.VMAccessAgent
  • Microsoft.EnterpriseCloud.Monitoring.MicrosoftMonitoringAgent

Specific to Run Script feature, the extension is called Microsoft.CPlat.Core.RunCommandWindows. The latest version is 1.1.3 (as of this article). Checking directories and files in C:\Packages\Plugins\Microsoft.CPlat.Core.RunCommandWindows\1.1.3 there are three places that ask for attention:

  • bin
  • Downloads
  • RuntimeSettings

In bin there are some DLLs as well as an application named RunCommandExtension.exe that we should look into. Again, to do the work I use dotPeek. Luckily this application can be decompiled

There are 4 namespaces here. I checked around and noticed a class named Constants under Microsoft.Azure.ComputeResourceProvider.Extensions.RunCommandExtension

This one is interesting. This gives you an idea on what RunCommandExtension application does and what are being set. We know that script file name follows the pattern script{0}.ps1 . Configuration file is stored in a file that has extension *.settings . You may see a constant named NumberOfScriptFiles  which tells the maximum scripts you may run once at a time is 100.

The main program()  is in the Program  class. There are bunch of helpful information that uncover a lot of things about RunCommandExtension application. We see registry key at path HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Azure\RunCommandExtension  is set. We can also see the argument/command that are supported in RunCommandExtension. Each argument is checked to be handled by a handler. You can simply jump to a handler for checking what it does e.g. InstallHandler()

The next one is protectedSetting path, I’d like to keep it for another article coming. In a nutshell, this part stores a signing key that the application uses to decrypt a secret to run the your script.

The next one is command that RunCommandExtension invokes to run. It is straightforward as follows

It looks like that the command to be executed is

Alright you gotta have an idea now. The application finds a script in Download directory and execute it by invoking PowerShell script. But you may be wondering how that script file is created? Did something pull script from somewhere in the cloud and wrote down to the local file? While I still cannot still figure out the part that the agent downloads a setting file from Azure back-end, I found a part that the RunCommandExtension application reads settings and write the script to a local script file.

Until now we know something that is much brighter and rather than just simply Run Script allows you to execute a script on Azure virtual machine.  To ensure this is a true, you can check files in Downloads and RuntimeSettings directory. Sample setting file is as follows:

Here is the flow diagram that illustrates how Run Command/Run Script works:

Run Script Logging

As being part of Blue Team, if something badly happens you would normally ask what left in the system that supports investigation and incident report. First, if you can access to the virtual machine you can find Run Command/Run Script extension log in this path C:\WindowsAzure\Logs\Plugins\Microsoft.CPlat.Core.RunCommandWindows\1.1.3. The file named RunCommandExtension.txt stores history including setting and script that was used.

In the case that you cannot access to check, technically you can still run another script using Run Command/Run Script feature to check:

You may hit the limitation of 4096 bytes output or outbound traffic on port 443 to Azure Public IP (when an incident occurs traffic normally routes to nowhere). Refer to this article for limitations.

With cloud-level logging, Azure logs the operation in Azure Activity Log.

If you write Azure Activity Log to a Log Analytics workspace, you can query:

If your virtual machine is covered  by Azure Security Center then you have data in SecurityEvent table. Do the query below to see if there is a Run Script activity:

As of this article, script content is not present in Azure Activity Log or anywhere else so you must capture script binaries on the virtual machine. Depending on how critical and policy you may have, should you have to block Run Command feature or write things to a Log Analyics workspace or your SIEM.

If there is a breach and you cannot get back to desired state of the virtual machine you would have to dump its memory and perform forensic.

Alert and Detection

In a large environment Contributor is normally granted to developer and system admin which includes RunCommand role

  • Microsoft.Compute/locations/runCommands/read
  • Microsoft.Compute/virtualMachines/runCommand/action
  • Microsoft.Compute/virtualMachineScaleSets/virtualMachines/runCommand/action

From Azure Monitor Alert or Azure Sentinel, you can create a query like

or

or combine both to increase fidelity

Conclusion

Run Command and Run Script are cool features for SysOps guys even SecOps guys (CVE verification). It could be abused by bad actor and if you don’t monitor it you may lose visibility in security monitoring.

This article shows you the Windows part only but what Run Command and Script run in Linux is not much different from Windows. You will see things like setting file, decryption path as well as Run Command utility.

To me personally I wouldn’t recommend Run Command and Run Script. There are good management tools out there providing much faster remote communication and execution. It is good for quick and simple audit though.

Finally, here are a few articles to check:

If you find something not correct or would like to add more feedback, please feel free to leave a comment.

This entry was posted in Monitoring & Detection and tagged , . Bookmark the permalink.

1 Response to What Blue Team needs to know about Run Script feature in Azure

  1. olivera56 says:

    Good work! Thanks for your time doing detailed analysis

Leave a Reply