Configure Azure Stream Analytics Alert Rule using Insights REST API

Azure Stream Analytics (ASA) helps us to quickly develop the real time streaming solution for Internet of Things (IoT) scenarios such as processing messages from the real devices. In an actual IoT implementation, there could be multiple ASA jobs to process different types of messages from the devices and perhaps this number could go beyond hundreds. The question is how the monitoring would happen for this large number of ASA jobs to make sure that there is no issue in message processing by the ASA job.

The answer is using automated alert for Azure Platform-as-a-Service (PaaS) out of the box feature. The alert rule can be created based on the available metrics with the threshold value and the mail address. The mail would be triggered by the PaaS service when the condition satisfies and action can be taken to resolve the issue by the engineers. Lets take a look how to setup an alert rule for an ASA job from the Ibiza portalI have already created an ASA job to process the message from IoT Hub to Blob Storage. The screenshot is given below

Once the ASA job is created and in running state, it starts capturing the metrics data like input events, output events, data conversion errors, runtime errors, SU% utilization etc as depicted below
The metric events can also be used to setup the alert which is triggered when the condition satisfies. The following alert rule triggers when SU% utilization is more than 90% for the duration of 15 minutes.
The above steps outline to set up the alert rule for an ASA job. If the number of ASA job is less than 5 or 10, the steps are achievable but if the number is significantly high (scale of hundreds) then it is a cumbersome process to create the alert rule for the ASA jobs. In order to create the alert rule programmatically, Insights REST API is used with the Azure PowerShell module and the process is really straightforward.The Insights REST API is authenticated using Azure Active Directory and the details can be found in my earlier post.  I will use a json file to store the name of the ASA jobs and details of the alert rules.

Following script is written using Azure Powershell module

The above powershell script does following jobs

  1. Authenticate over Azure Active Directory using username and password
  2. Get the JWT access token from Azure Active Directory
  3. Read the input from ReferenceData.json file
  4. Call the Insights REST API with the access token and input json data

In Step 4, the input json data is populated to post to the Insights REST API. The condition is on the total error metrics with threshold value of 1500 over a period of 15 minutes. The details of the json data is given below

More details can be found here for the input json data. The alert rule would be in action once the script execution is completed and it does not require any restart of the ASA jobs.