In today's fast-paced business environment, it is crucial to have efficient and reliable systems in place to manage and monitor critical processes. One such essential component in Microsoft Dynamics 365 Business Central is the Job Queue. In this blog post, we will discuss what the Business Central Job Queue and Telemetry are and how to implement a low-code solution for automatic alerts when a Job Queue fails using BC Telemetry, Azure Application Insights, Power Automate, and Office 365 distribution lists.
What is BC Telemetry?
Business Central emits telemetry data for various activities and operations on environments and apps/extensions. Monitoring telemetry gives you a look at the activities and general health of your environments/apps, so you can diagnose problems and analyse operations that affect performance.
What is BC Job Queue?
The Job Queue in Microsoft Dynamics 365 Business Central is a powerful feature that allows you to schedule and manage tasks and processes. It enables you to run tasks in the background, ensuring that your system remains responsive and efficient. The Job Queue is particularly useful for automating repetitive tasks, such as generating reports, processing data, or updating records.
However, when a Job Queue fails, it is essential to be notified promptly to address the issue and minimize any potential impact on your business operations.
This proposed low-code solution leverages BC Telemetry, Azure Application Insights, Power Automate and Office 365 distribution lists to provide automatic email notifications when a Job Queue run fails.
Here's a step-by-step guide on how to implement this low-code solution.
Pre-Requirements:
Azure subscription (Azure Application Insights)
Dynamics 365 Business Central SaaS subscription
Office 365 license
Power Automate Premium license.
Proposed Approach (Not mandatory)
Have a dedicate user account for Office 365, Dynamics 365 BC and Power Automate. Let's call it bcsysadmin@[customertenant].com.
This way, this user can be owning the Job Queue runs in BC, Power Automate Cloud Flow and Office account to send emails.
Set up Azure Application Insights:
Create an Azure Application Insights resource in the Azure portal.
Configure Business Central to send telemetry data to Application Insights by adding the Instrumentation Key to the Business Central administration console.
Create a KQL query to query in Application Insights:
Navigate to the Data Explorer dataexplorer.azure.com.
Connect to Application Insights (Min 8:00 of https://youtu.be/pPSe19mOG30). Thanks Søren.
Identify an existing query to tweak from Telemetry library https://learn.microsoft.com/en-us/dynamics365/business-central/dev-itpro/administration/telemetry-event-ids. Thanks Kennie.
Write a custom query to filter Job Queue failures
// Job queue entry run failed
traces
| where timestamp > ago(60d) // adjust as needed
| where customDimensions.eventId == 'AL0000HE7'
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, environmentType = customDimensions.environmentType
, companyName = customDimensions.companyName
// jobQueueApp* dimensions contains the information about the app where the jobQueueObject* is located
, jobQueueAppId = customDimensions.alCallerAppId
, jobQueueAppName = customDimensions.alCallerAppName
, jobQueueAppPublisher = customDimensions.alCallerPublisher
, jobQueueAppVersion = customDimensions.alCallerAppVersion
// jobQueueObject* dimensions contains the information about the object containing the code being run
, jobQueueObjectId = customDimensions.alJobQueueObjectId
, jobQueueObjectName = customDimensions.alJobQueueObjectName // added in 22.0
, jobQueueObjectType = customDimensions.alJobQueueObjectType
// jobQueueExecution dimensions contain information about the job queue entry definition
, jobQueueEntryId = customDimensions.alJobQueueId
, jobQueueEntryIsRecurring = customDimensions.alJobQueueIsRecurring
, jobQueueEntryDescription = customDimensions.alJobQueueObjectDescription // added in 22.2
, jobQueueEntryMaxNumberOfAttemptsToRun = customDimensions.alJobQueueMaxNumberOfAttemptsToRun // added in 21.5
// jobQueueExecution dimensions contain information about the attempted run of the job queue entry
, jobQueueExecutionNumberOfAttemptsToRun = customDimensions.alJobQueueNumberOfAttemptsToRun // added in 21.5
, jobQueueExecutionStacktrace = customDimensions.alJobQueueStacktrace
, jobQueueExecutionStatus = customDimensions.alJobQueueStatus
, jobQueueExecutionTaskId = customDimensions.alJobQueueScheduledTaskId // you can join to task scheduler telemetry on the taskId
Save the query for future use.
Configure the Office 365 distribution list:
Create a new distribution list in Office 365, or use an existing one.
Add the relevant recipients to the list.
Set up Power Automate:
Create a new flow in Power Automate with four steps
Step 1 - Add a "Recurrence" trigger to define how often the flow should run (e.g., every 10 minutes).
Step 2 - Add an "Run Analytics query" using Azure Application Insights action to run the custom log query
Copy and Paste the KQL and adjust to your recurrence and required additional filters (Another KQL query proposed at the end of this post)
Step 3 - Parse JSON. (Min 20:45 of https://youtu.be/pPSe19mOG30). Thanks again Søren.
Step 4 - "Office 365 Outlook - Send an Email" action to send an email to the distribution list.
Test the solution:
Trigger a Job Queue failure in Business Central.
Verify that the recipients in the distribution list receive an email notification.
This proposal is a short version than the one proposed by Søren (https://youtu.be/pPSe19mOG30).
After deploying in LIVE for a customer, few lessons learned to be considered.
Job Queue in BC
Most of the Job Queue entries are recurring. Therefore, if a Job Queue run fails, it's possible to setup an automatic re-run.
Fields to be considered:
Maximum No. of Attempts to Run
Specifies how many times a job queue task should be rerun after a job queue fails to run. This is useful for situations in which a task might be unresponsive. For example, a task might be unresponsive because it depends on an external resource that is not always available.
By default the Maximum No. of Attempts to Run value is set to 0. That means that if the Job Queue run fails, the Job Queue entry will be stuck in Error and a manual trigger will be required by a BC User.
Consider to set to a number greater than 0 to allow an automatic re-run.
Rerun Delay (sec.)
Specifies how many seconds to wait before re-running this job queue task in the event of a failure.
Consider to set to a number (value is in seconds) depending on typical duration of the run with success. In this example, 15 seconds was configured and this Job Queue entry finishes in less than 10 sec.
Using Data Explorer to analyse Telemetry
When using Data Explorer to query data generated from Telemetry, it is required access to Application Insights. A read on this document https://learn.microsoft.com/en-us/azure/azure-monitor/app/resources-roles-access-control is mandatory. User to be used to query Application Insights should be have Reader permissions.
KQL Query - without false positives
After adjusting previous proposed configuration in Job Queue entries, it's critical to adjust the proposed KQL query to avoid "false positives".
Example of a potential false positive:
Job Queue runs and ended in error.
Maximum No. of Attempts to Run is set to 3
Rerun Delay (sec.) is set to 15
Job Queue will be re-run after 15 sec. If the re-run is successful, the email will be send and it's a false positive as the Job Queue won't be stuck in error.
Proposed new KQL Query (Thanks ChatGPT for helping me with the final version)
traces
| where timestamp > ago(10m)
| where customDimensions.eventId == 'AL0000HE7'
| where toint(customDimensions.alJobQueueNumberOfAttemptsToRun) == toint(customDimensions.alJobQueueExecutionNumberOfAttemptsToRun)
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, environmentType = customDimensions.environmentType
, companyName = customDimensions.companyName
, jobQueueAppId = customDimensions.alCallerAppId
, jobQueueAppName = customDimensions.alCallerAppName
, jobQueueAppPublisher = customDimensions.alCallerPublisher
, jobQueueAppVersion = customDimensions.alCallerAppVersion
, jobQueueObjectId = customDimensions.alJobQueueObjectId
, jobQueueObjectName = customDimensions.alJobQueueObjectName
, jobQueueObjectType = customDimensions.alJobQueueObjectType
, jobQueueEntryId = customDimensions.alJobQueueId
, jobQueueEntryIsRecurring = customDimensions.alJobQueueIsRecurring
, jobQueueEntryDescription = customDimensions.alJobQueueObjectDescription
, jobQueueEntryMaxNumberOfAttemptsToRun = customDimensions.alJobQueueMaxNumberOfAttemptsToRun
, jobQueueExecutionNumberOfAttemptsToRun = customDimensions.alJobQueueNumberOfAttemptsToRun
, jobQueueExecutionStacktrace = customDimensions.alJobQueueStacktrace
, jobQueueExecutionStatus = customDimensions.alJobQueueStatus
, jobQueueExecutionTaskId = customDimensions.alJobQueueScheduledTaskId
With this new propose KQL query, the false positives will be avoided. Lines will be only returned when the Max. No. of attempts is reached.
Other considerations:
Instead of using Power Automate to trigger the emails, it's possible to use directly Application Insights. Please read this post -> https://demiliani.com/2022/06/23/dynamics-365-business-central-set-up-alerts-on-job-queue-failures/. Thanks Stefano.
Using Power Automate, it is possible to custom the email body but it adds one more component to the overall solution.
Be careful about data volume generated by Telemetry. Please read this post -> Use Transformation Rules to lower the costs of BC Telemetry | MSDyn365 Business Central - Tom Kapitan (kepty.cz) and this one -> https://www.waldo.be/2023/11/27/a-bc-telemetry-story-monitor-your-telemetry-usages/. Thanks Tomas and Waldo.
If this is the first Azure subscription from your customer, don't forget to assign the subscription to your record as Partner. Please read this -> https://blogs.partner.microsoft.com/mpn-sweden/wp-content/uploads/sites/41/2017/01/Attaching-DPOR-for-Azure.pdf.
Thanks for reading!