How Dynamics 365 Business Central handles incoming web service calls (for dummies).

In the last two days I was in Assago (Milano) at the WPC Conference, probably the most important IT conference in Italy, where I presented two sessions related to Azure and AI. The conference is absolutely not a Business Central-related conference, but some of the attendees of my Semantic Kernel session were companies and partners working with Dynamics 365 Business Central.

The IT manager of one of this customer talked with me about some problems they have on incoming web service calls (high number of incoming calls arriving from external systems). They started to have the need to handle the HTTP 429 – Too many requests error message and they asked me questions about the operational limits, why they are so “strict” and what to do in such cases.

I think that probably all of you know that Dynamics 365 Business Central has some operational limits (and every SaaS application need to have operational limits). You can see details here.

Related to OData service calls, Dynamics 365 Business Central has now a per-user limit that helps on improving concurrency and throughput:

• Max concurrent requests: maximum number of OData V4 requests that can be processed at the same time before waiting in a queue: 5 per user.
• Max connections: maximum number of simultaneous OData V4 requests (processed and queued): 100 per user.
• Speed (rate): The maximum number of OData V4 requests that can be submitted within a 5-minute sliding window: 6000 per user.

The per-user limit permits you to avoid users blocking each other during incoming calls.

Incoming we service calls are handled by the NST and the NST has resource limits for handling them (it cannot handle an unlimited number of requests). The limit is about 500 parallel calls for NST. If you have a lots of incoming web service calls, these are spreaded across different NSTs:

This is a standard load balancing process that every scalable cloud application must have in place.

Spreading incoming calls across different NSTs is a must but it can also have a main problem: transaction consistency.

Imagine to have an external application sending two API requests:

• Request 1: update the availability for item IT01 to 5
• Request 2: read item IT01 availability

and now imagine that these two requests are handled by different NSTs:

Here is where you can have a transaction consistency problem: if the second request is forwarded to a different NST than the first request (NST 2 in the example), there’s a risk that NST 2 is not seeing the data modification coming from the first call, so you can read a wrong value for the item’s availability.

What Microsoft is currently doing to prevent this problem and to achieve high concurrency at the same time is introducing the concept of user session: incoming web service requests are spreaded across different NSTs accordingly to the user initiating the request. And this is the reason of the operational limits for users too (max concurrent requests per users = 5).

The same applies to the speed rate (6000 requests per user in a 5 minute sliding window): if USER1 sends 4000 requests in the first minute, it’s ok. Then it sends 1000 requests in the second minute, it’s ok. If it then sends 2000 requests in the third minute, you have a HTTP 429 error:

Are those limits so strict? I honestly don’t think so, at least now that they are per-user.

What can you do more?

In the session I’ve done with Duilio at the last Directions EMEA in Vienna, I’ve explained how we’re currently handle very big volumes of incoming web service requests in Business Central tenants. I hope we’ll have the chance to repeat this session with more time and technical details, but here my suggestions in small words:

• use API batch calls (this is the first basic option you can adopt).
• use an intermediate layer where you handle caching (for GET requetss) and queuing of requests before sending them to Business Central (decoupling): this is the advanced solution I’ve explained at Directions EMEA, when you have an extremely high number of incoming requests it’s absolutely a good architectural pattern to adopt.

Original Post https://demiliani.com/2024/11/29/how-dynamics-365-business-central-handles-incoming-web-service-calls-for-dummies/