Web Services Connector hitting 429 Rate Limit

IdentityIQ (IIQ) IIQ Discussion and Questions
,
1

Share all details about your problem, including any error messages you may have received.

Hi,

We are using IdentityIQ Web Services connector to integrate with an application using REST API.

we are frequently getting the following error:

HTTP 429 - Request rate exceeded
Retry-After: 60

What is the recommended best practice in IdentityIQ Web Services Connector to handle API rate limits like this , as the api has a limit of 40 calls/60 seconds

Best Regards

2

@alshahim04 - did you configure Retry-After response header attribute. if you have not configured it.
The aggregation is implicitly retried on the basis of the Retry-After response header attribute. The allowed Retry-After value range is from 1 to 180 seconds. If the response header has no Retry-After value, then the connector retries a failed process after one second. If the response header has a Retry-After value higher than the maximum 180 seconds (three minutes), then the connector retries a failed process after 180 seconds. If the response header has an accepted Retry-After value (three minutes or less), then the retry will take place after that time duration.

3

Hi @narayanag

Thanks for the clarification.

In our case, the issue is happening mainly during (provisioning). We often have scenarios where multiple user accounts are modified at the same time, which triggers several concurrent provisioning requests to the target REST API.

Because of this parallel execution, we frequently hit HTTP 429 / rate limit

How can we configure the connector to limit the number of api calls ?

Thanks

4

Can you please try below best practices. Might be very helpfull

Implement retry logic that respects the Retry-After header

Add delays between API calls in your connector customization

Reduce aggregation frequency

Use pagination appropriately

Consider delta/incremental aggregation instead of full aggregations

5

Hello @jchinnapareddy

Thank you for your reply ,

We have all of pagination , aggregation frequency reduced and delta aggregation , the error does not happen in aggregation,

it mainly happens in provisionning only as we have a use case where many account accouts should be provisionned simultanously ,

Thanks again

6

@alshahim04 - Could you provide more information about the Web Services endpoint and the API rate-limiting policy? It may be worth discussing with the target application team whether a higher rate limit can be granted for the SailPoint integration user.

7 
You can try these steps.

Step1: Add retryableErrors to the application debug XML so 429 is treated as retryable:

<entry key="retryableErrors"> 
<value> <List> 
<String>429</String>
 <String>Request rate exceeded</String>
 </List>
</value> 
</entry>
<entry key="retryCount" value="5"/>
<entry key="retryWait" value="65000"/>


Step2: Ask the app team to increase the rate limit.
8

Add Thread.sleep() in the After Rule with 429 retry

In your Web Services connector, add this logic to the After Rule for every operation (Aggregate, Create, Update, etc.):

java

import connector.common.JsonUtil;

int statusCode = responseMap.get("responseCode");

if (statusCode == 429) {
    Thread.sleep(60000); // honour Retry-After: 60
    // returning null tells IIQ to retry the same request
    return null;
}


Thread.sleep(1600);

return responseMap;
9

@alshahim04 Try adding provisioning retries in the app xml. You can list down all possible errors you are aware of to this list and set the timer accordingly.

I would prefer this approach over sleeping the thread.

Because with retries, IIQ creates a workitem to trigger it after wait period and it releases the thread to be used by other process. But in case of thread.sleep, it doesn’t release the thread. If you sleep it for let’s say 30 mins, that thread will be unavailable to use. If you have parallel execution enabled, then lot of threads will not be available and you can observe performance degradation with other modules.