How to Configure Retries for Connector Aggregations and Provisioning Actions

Overview

Connectors in both IdentityIQ and Identity Security Cloud support retryable errors during both aggregation and provisioning. This document showcases how to configure these.

Keep in mind retries can happen at the platform /Identity Security Cloud level(distinct) to what can happen at the ‘connector’ level. There are many points on retries, most provisioning actions are retriable, but some connectors do have fidelity around which errors can be set as retriable

Aggregation Retries

Identity Security Cloud, like IdentityIQ, has support built-in for aggregation retries. This is configured per source by configuring aggregationRetryErrors under the connectorAttributes. Additional options retryWaitTime and maxRetryCount can also be configured.

curl --request PATCH '{api-url}/v3/sources/{id}' \
--header 'Content-Type: application/json-patch+json' \
--header 'Authorization: Bearer {token}' \
--data '[
    {
        "op": "add",
        "path": "/connectorAttributes/aggregationRetryErrors",
        "value": [
          "Connection reset".
          "Read time out"
        ]
    },
    {
        "op": "add",
        "path": "/connectorAttributes/retryWaitTime",
        "value": "5000"
    },
    {
        "op": "add",
        "path": "/connectorAttributes/maxRetryCount",
        "value": "3"
    }
]
  • aggregationRetryErrors
    • A string list containing the errors for which you want to retry again.
    • This would be the input steam e.g. ‘Connection Reset’ or Read Timeout. That would be set and be used to hook into this e.g. think dynamic error retry process.
  • retryWaitTime
    • A string containing the amount of retry time to wait before attempting a retry. This is in milliseconds.
    • Default value of 5000 milliseconds (or 5 seconds).
  • maxRetryCount
    • A string defining the the maximum retry attempts that will be performed.
    • Default value is 3.

Example Scenario

If aggregationRetryErrors is configured for “Connection reset” and “Read time out”, then anytime this error is encountered, the aggregation iteration would be retried.

Provisioning Retries

Identity Security Cloud, like IdentityIQ, has support built-in for provisioning retries. This is configured per source by configuring retryableErrors under the connectorAttributes. In addition, provisioningMaxRetries and provisioningRetryThreshold can also be optionally specified to control the amount of retrying that is done.

Please see below definition of what can be set for provisioning retries. So we can set for instance on the connector, retries X times, every Y minutes, if it the error matches on any of these strings “xyz”.

curl --request PATCH '{api-url}/v3/sources/{id}' \
--header 'Content-Type: application/json-patch+json' \
--header 'Authorization: Bearer {token}' \
--data '[
    {
        "op": "add",
        "path": "/connectorAttributes/retryableErrors",
        "value": [
          "Connection reset",
          "Read time out"
        ]
    },
    {
        "op": "add",
        "path": "/connectorAttributes/provisioningRetryThreshold",
        "value": "1"
    },
    {
        "op": "add",
        "path": "/connectorAttributes/provisioningMaxRetries",
        "value": "3"
    }
]
  • retryableErrors
    • A list of string specifying the retryable errors to match against.
    • e.g., retry on any error with “Connection timed out”
  • provisioningRetryThreshold
    • A string specifying the retry looping threshold in minutes. Default is 1.
    • i.e., retry every X minutes.
  • provisioningMaxRetries
    • A string specifying the maximum number of retries to attempt. Default is 3.
    • i.e., retry up to Y times

Example Scenario

If retryableErrors is configured for “Unknown Host”, with a provisioningRetryThreshold of 3 minutes, and a provisioningMaxRetries of 2 attempts, the system would look to see if an error matches “Unknown Host”, and retry up to 2 times, every 3 minutes.

Example Retryable Errors

"Cannot open connection"
"Connect failed"
"Connect Failed"
"Connect to SAP gateway failed"
"Connection refused: connect"
"Connection refused"
"Connection Refused"
"Connection reset"
"Connection Reset"
"Connection timed out"
"ConnectionFailedException"
"Errors returned from IQService. The network path was not found."
"Errors returned from IQService"
"Exact or partial message phrase returned from IQService"
"Exception while executing the request"
"Failed to connect to IQService"
"Failed to connect to server"
"Failed to connect to the server"
"failed to respond"
"Failed"
"Internal Server Error"
"Interrupted during LDAP operation"
"Invalid keyId or incorrect keystore"
"Invalid token"
"InvalidConfigurationException"
"Item failed due to retry timeout"
"java.io.IOException"
"java.lang.ClassCastException"
"java.net.ConnectException"
"java.net.SocketException"
"java.net.SocketTimeoutException"
"javax.naming.CommunicationException"
"javax.naming.InsufficientResourcesException"
"javax.naming.NamingException"
"javax.naming.ServiceUnavailableException"
"javax.net.ssl.SSLException"
"LDAP connection has been closed"
"LDAP response read timed out"
"locked by user"
"Login failed"
"No such host is known"
"Not a JSON Object"
"ObjectNotFound"
"Read timed out"
"Remote host terminated the handshake"
"sailpoint.connector.ConnectorException"
"Server is not operational"
"ServerErrorException"
"Service Unavailable"
"The server is unwilling to process the request"
"TimeoutException"
"Unable to create iterator"
"unable to obtain exclusive access to this record"
"Unknown host"
"Unknown Host"
"UnknownHostException"
"Unreachable Host"
"Unrecognized SSL message, plaintext connection"

Note: not all error message configured under retryableErrors are retried (there is no definitive list), it’s not possible to separate the count and try matching, the configurations work in tandem.

15 Likes

Does this also apply to entitlement aggregations?

Hi Omar, thanks for this, saved us a lot. Let me ask a question, is there some way to configure the provisioningRetryThreshold and provisioningMaxRetries for “all” errors? Perhaps with empty array of /connectorAttributes/retryableErrors? Or do you recommend adding all string values?

@omar_khote_iam ,
How to know if the retry is really happening for the provisioning failure , Does event data show that system is trying to retry the failures ?

I tried to add the required entry for ISC Governance connector and don’t have any clue if the retry is really happening for the failure .

@ Colin McKibben

Hi Omar,

Quick question, what would be error message I should specify for below aggregation error?

@vishal_kejriwal1 did you get an answer for this? I am looking for something similar.

Hello Mark, thanks for your comment, not that I’m aware of unfortunately.

Hello @jsosa , thanks for your comment, I don’t think that’s supported, I’d simply copy all values and reuse the ones that are relevant to your connector , use search for instance to find common errors witnessed and compile a list of effective errors to add to those problematic sources.

Hi @vishal_kejriwal1 thanks for your comment, if there is a provisioning error we should see this event , on the identity, access history and search. Are we able to check the CLI and tail logs too?
Debugging | SailPoint Developer Community.

Hi @Shubhams_009 thanks for your comment, I am unsure, what have we tried ?please try Communications link failure

This should be part of standard documentation from sailpoint. There is section under each connector doc : Timeout Errors and Settings
where they have mentioned timeout settings, these instructions should be shared there or there sbhould be specific doc written and it should be referred in each connector guide.

Hi @omar_khote_iam,

We attempted to implement retry at connector level for provisioning in January 2024 but this was not working as expected.

According to support (in 2024), the retry configuration in the source was applicable only to IIQ and retry behavior in ISC is configured at tenant level.

Has this evolved since the beginning of 2024 ?

Thanks

1 Like

Hi Omar,

I implemented the configuration as mentioned in your post but I was unable to make it works.

Was this working on your environment ?

Hi @omar_khote_iam and All …

Thanks for all your updates! This seems to be a useful solution / workaround for our use case (retry failed provisioning actions / access requests) but we haven’t implemented this yet.

@omar_khote_iam I am wondering, have you tested the retries of failed provisioning actions in ISC and has it worked as expected?

Also, can you think or give us an example of the scenario(s) where the retries of failed provisioning actions wouldn’t happen/work?

Thanks

@omar_khote_iam Could you please advise regarding my previous reply? I would appreciate this. Thanks

Hi @omar_khote_iam

Could you please give us an update on this?