Maintaining Microsoft Sentinel Analytic Rules in JSON and YAML with GitHub Actions

Microsoft Sentinel Analytic Rules can be shared in both the YAML and ARM format, whereas the ARM format leverages JSON as file type. Within this short post I want to demonstrate an approach that leverages a GitHub Action to automatically build and update the rules in YAML format — so you can just export and update existing rules without any manual conversion effort.

Fabian Bader built a cool solution called SentinelARConverterthat allows conversion of exported Sentinel Analctic rules from ARM/JSON to YAML (and vice-versa). To emphasize sharing of analytic rules I wanted to adopt also the YAML format without the need to always manually convert the rules. Therefore I incorporated his solution into a GitHub Action.

Analytic Rules can be exported from Sentinel as ARM template in JSON format.

Building a GitHub Action

The automation of this task is fairly simple if you are already familiar with GitHub actions. In case you want to directly see the full pipeline, you can find it here. Otherwise keep on reading.

The GitHub action should be triggered as soon as I upload a new Export of an Analytics Rule to the repository. For that, we need to define a folder structure. I maintain the rules within a folder called AnalyticRules. Based on that we can define the triggers for the workflow and filter only for the analytic rules path. This will only run the Action, when a file within that folder get’s changed. Additionally, I added a workflow_dispatch trigger, this allows manual execution of the pipeline.

on:
  push:
    branches: [main]
    paths:
      - "AnalyticRules/*.json"
  workflow_dispatch:

Besides the initial conversion, the Action should reflect changes to existing ARM/JSON Analytic Rules based on the last file modification timestamp.

So we do the following things:

• Enumerate all JSON files within the AnalyticRules folder
• Change the destination file type to YAML
• Check whether the destination file already exists or whether the JSON file was modified
• Convert the actual rule from JSON to YAML

Install-Module SentinelARConverter -AcceptLicense -Force
Get-ChildItem -Path 'AnalyticRules' -Filter '*.json' | ForEach-Object {
    Write-Output "Processing file: $($_.Name)"
    $yamlFilePath = $_.FullName.Replace('.json', '.yaml')
    if (-not (Test-Path $yamlFilePath) -or (Get-Item $yamlFilePath).LastWriteTime -lt $_.LastWriteTime) {
        Write-Host "Converting $($_.FullName) to $yamlFilePath"
        Convert-SentinelARArmToYaml -Filename $_.FullName -UseOriginalFilename
    }
}

After running the conversion, the Action should automatically commit and push the changes into the repository as I don’t want to do this manually.

For this, the workflow requires contents:write permissions, as GitHub will automatically grant the workflow permissions to commit and push changes to the repository.

git config --global user.name 'SentinelARConverter'
git config --global user.email 'nicolonsky@users.noreply.github.com'
git add AnalyticRules
git commit -am "SentinelARConverter"
git push

Now we just need to combine the individual steps within a GitHub Action workflow file and store it within the: .github/workflows folder in the repository:

on:
  push:
    branches: [main]
    paths:
      - "AnalyticRules/*.json"
  workflow_dispatch:

permissions:
  contents: write

name: Build YAML Analytic Rules

jobs:
  build:
    runs-on: ubuntu-latest
    name: Build YAML Analytic Rules
    steps:
      - uses: actions/checkout@v3
      - name: Convert rules to YAML
        run: |
          Install-Module SentinelARConverter -AcceptLicense -Force
          Get-ChildItem -Path 'AnalyticRules' -Filter '*.json' | ForEach-Object {
              Write-Output "Processing file: $($_.Name)"
              $yamlFilePath = $_.FullName.Replace('.json', '.yaml')
              if (-not (Test-Path $yamlFilePath) -or (Get-Item $yamlFilePath).LastWriteTime -lt $_.LastWriteTime) {
                  Write-Host "Converting $($_.FullName) to $yamlFilePath"
                  Convert-SentinelARArmToYaml -Filename $_.FullName -UseOriginalFilename
              }
          }
        shell: pwsh
      - name: Commit and push changes
        run: |
          git config --global user.name 'SentinelARConverter'
          git config --global user.email 'nicolonsky@users.noreply.github.com'
          git add AnalyticRules
          git commit -am "SentinelARConverter"
          git push

After committing or changing an Analytics Rule in JSON format, the GitHub action will take over and automatically build, commit and push the changes:

Modifications and new rules will be automatically committed by the GitHub action

Detailed info can then also be found within the run details of the workflow:

Detailed GitHub action output

Ciao 👋

I hope this helps you to simplify the sharing and maintenance of both analytic rules in YAML and JSON. Kudos again to Fabian Bader for building the converter!

Image proudly generated by DALL·E :)