Azure Logic Apps as a Proxy That Throttles the Number of Parallel HTTP(S) Requests to a Service

Once upon a time, a client asked me to provide a solution that would prohibit his apps from making requests to a specific API endpoint in parallel.

The thing is, that specific API endpoint was a legacy third-party closed source solution, deployed years ago, and the company that developed it went out of business, but they still needed that legacy service because – reasons.

So the problem with it is that when it was developed no one was thinking about making concurrent calls to it because that endpoint was being used from a desktop app by a single user. Fast forward, the desktop app is now a web app, and that single user is now a single department. The biggest problem was that during the web app development, no one discovered those race conditions. They were discovered after the web app was in production for some time, the users were getting blocked, the data was messed up, etc.

They needed a "quick hack" ASAP, so they've pinged me, and we've solved it by using Azure Logic Apps in a couple of hours. Later on, we've implemented a proper solution with Durable Functions and some other services, but more about that in another blog post.

Get an Azure subscription and create a Logic App

The first step was to get them an Azure subscription and create a Logic App.

To skip having redundant steps in my blog posts, I've extracted that process in a separate blog post. Please follow the steps on this post, but stop once you reach the "Define the JSON Schema" section. We won't need that part for this tutorial.

Make an HTTP(S) request to the legacy API endpoint

The next step is to call the legacy API endpoint. What you need to do is click the "+ Next step" button, start writing "http", and select the "HTTP" step.

Once the HTTP step is added you need to select the HTTP verb, which in our case was POST, set the URL, and add a body. Since we're just "forwarding" the body from the original request, once you click in the Body field just select the Body content from the windows on the right.

You can also set the headers, queries, and authentication, but in our case, there was no need to do it.

(The URL I'm using in this example is of an "echo" Logic App that I've created just to have something which I can showcase to take screenshots. It simply returns the body from the request in the body of the response.)

Return a response

We need to add yet another step which will return us the response from the legacy service. Click the "+ Next step" button, and start writing "response". Then, select the Response step from the list.

Once it's added, instead of the pre-populated value in the Response Code field select the Response code value from the previous step inside of the windows that will appear on the right.

Do the same thing for the Body field - click in it, and then select the Body item from the list that will appear on the right. At the end click the "Save" button at the top left corner of the Logic Apps Designer.

Congratulations... if you copy the URL from the first step which was just generated after you've clicked "Save", you'll have a fully working proxy.

But, that's not why we started this adventure... we've wanted to make sure that the legacy API endpoint which we're calling from the second step will get hit sequentially, instead of in parallel.

To limit the number of parallel executions click on the ellipsis in the top right corner of the first step, and then select Settings.

A popup will appear in which you'll have to switch on the Limit switch, and then drag the slider to leftmost, to enable only one execution at the time. When you're done with that click "Done", and you might click "Save".

The problem is that you won't be able to save the Logic App because of an error. The error says that you can't limit the number of parallel executions to one and have the Logic App return a response unless you make that response asynchronous.

To fix that error, switch from the Designer view to the Code View in the top bar.

Once you're in the Code view, look for the section named "Response", and then add this value to it:

    "operationOptions": "Asynchronous",

At this point, you'll be able to save the logic app.

Testing your Logic App

The Logic App that we've just built will call the legacy API endpoint sequentially, forward it the body of the original request, and then return the response from the legacy API endpoint.

Since I've replaced the legacy API endpoint with a Logic App that just echoes the request for the purpose of this blog post, the response that we get back from our Logic App is the body which we've sent originally.

But there's a catch. Logic Apps is a serverless service, and not only that it's supposed to run in parallel, but it wouldn't be really useful if Logic Apps would just dismiss your request if there's one already "in the pipeline".

So the solution actually works by accepting all the requests that come in parallel, but instead of trying to execute them in parallel, it adds them to a "queue" invisible to us and executes them one at a time.

Since we had to change the Response step from being synchronous to asynchronous, what we're going to get as a response once we call our Logic App won't be the response that we expect, but a "ticket" to check the status of our request, and retrieve the result once it's done.

If you take a look at the screenshot below, you'll see the response we got. The response says that the current status is "Waiting", and it gives us a way to check the status. What's most important to us, is the "outputsLink" URL.

By making a GET request at that URL we'll get a response consisting of two "top-level" fields - headers and body. In this case, we're only interested in the body, but headers might be useful for your case.

Now that we've received this object, all we have to do is deserialize it, and use the body.

How to use this solution?

This probably isn't what you had in mind, but it definitely beats a solution where multiple web apps are hammer-pooling an endpoint which either turns them down with some meaningless response code (for example, my favorite, 418 - I'm a teapot), or keeps the connection on until it gets the response back, or the connection timeouts.

The second solution is actually pretty bad because there's an edge case where your connection gets its turn after waiting for a certain amount of time, but connection timeouts before the legacy API endpoint returned the response, and the response was returned to us. Since we don't know how the legacy API endpoint works there's a chance that changes made during a connection that broke won't be rolled back.

To use this, the clients' developers had to call our Logic App, extract the "results URL" from the response, and ping it every few seconds in a loop, until the app got the response back. Since the loop wasn't a blocking one, but actually a timer was used to trigger the call, this implementation wasn't blocking threads.

One of these days I'm going to write a post about how we've ended up solving it using Durable Functions and Azure Service Bus, but that took somewhat longer, and the solution from this post got from inception to production in a few hours, so it can be forgiven for being a bit rough.