Azure Logic App Unexpected File Size

Problem

Everything had been running fine in the integration platform, but then one day, alerts started coming out that the budget threshold had been hit.

The below image shows where the budget had been close to being hit for a few days then eventually got hit.

Analysis of What’s going on

Taking a look into the cost data we could see there was an ongoing increase in costs with the blue line showing that one of the Logic Apps is increasing significantly each day.

Looking at the run history of the Logic App you can see a number of runs where they are taking multiple days to complete.

If you look at the metrics for the Logic App billable actions you can see its regularly doing quite a bit of work but suddenly it starts getting very busy.

Root Cause

What was happening here was that the 3rd party who is the source for a file that is submitted to the customer was supposed to be a delta file.  The partner had a bug and they were not removing records from the file when they were complete.

In this case, the Logic App would loop over the batch and process all records in the file.  Since the partner stopped removing complete records the Logic App was incurring more cost because of the consumption pricing model.

Solution

The solution here is for the partner to correct the bug, which is causing complete and unchanged records to not appear in the file as they used to.

Risk Mitigation

One of the challenges here is that the solution was unchanged and it was a bug at the partner which caused this issue.  We need to think about how we can mitigate this risk.

Read more: How to Optimize Azure Logic Apps Cost to Maximize Savings?

Step 1

The alerting for budget and anomaly detection already mitigates the risk to a good degree.  This may be enough for many customers

Step 2 - Circuit Breaker

Using Turbo360 Monitoring, the customer can implement a circuit breaker pattern whereby if the Logic App has too many runs in a short period, Turbo360 can disable the Logic App, which will trigger the monitoring alerts