Okta Account aggregation taking more time to complete

IdentityIQ (IIQ) IIQ Discussion and Questions
1

The Okta Full Account Aggregation task has been running for over 5 hours, even with partitioning enabled. Is there any way to reduce the average runtime? Thanks in advance.

2

Hi @bhoyars_1 ,

Use the Delta aggregation: by default, delta aggregation fetches changes to accounts’ user profile attributes and entitlements. In order to aggregate the complete list of attributes present in a schema, add the following entry key to the application XML via the Identity IQ Debug page:

<entry key="enableFullROinDelta"; value="true">

For more info please go through the Okta connector guide.

3

Okta’s API is notoriously slow, not even including the rate limits. Adding partitions won’t help if rate limiting is your problem, either.

The mentioned delta aggregation may be useful, if that’s something you can support.

That said, a five hour aggregation that just runs in the background and doesn’t get in the way isn’t a real “problem”, right? You may as well use the compute time you’re already paying for. It’s just a long-running task that you’ll want to schedule in a way that doesn’t get in the way of deployments.

Another, weirder option if Okta is your authoritative source is to use its “SCIM” provisioning capabilities to push identities directly in to IIQ.

4

We enabled caching on the Okta application object that made our aggregations run a little bit faster.

      <entry key="enableMultiThreadedCacheCreationGroups" value="true"/>
      <entry key="groupCacheThreadSize" value="%%OKTA_GROUP_CACHE_SIZE%%"/>
      <entry key="enableMultiThreadedCacheCreationApps" value="true"/>
      <entry key="appCacheThreadSize" value="%%OKTA_APP_CACHE_SIZE%%"/>

Note: Like mentioned above you still can’t do much within IIQ to increase the performance of the Okta API’s & the rate limit situation. But the above flags did help us decrease the time a bit. The documentation has a good bit of information on this as well. I would also leverage full & incremental aggregations and run the full overnight so it doesn’t impact your system and bring it to a screeching halt.

If you look at the connector documentation it’ll provide you guidelines on wether to perform caching vs sequential on the aggregation:

  • Caching– This works best when the number of connected entitlements is large.

  • Sequential– This works best when there is large number of accounts with relatively fewer connected entitlements with an ability to execute multiple partitions in parallel.

2 Likes
5

What size did you set for the app and group cache?

6

You’ll need to base the thread settings on your system’s available resources, particularly CPU and memory, to effectively handle increased threading. Adjust the thread sizes based on your system’s capacity and performance observations.

We initially set the thread size to 4, gradually increasing it and monitoring CPU and memory usage until we reached an optimal setting of 10 for our organization. It’s a bit of a balancing act to ensure the system isn’t overburdened, so keep an eye on performance metrics as you make adjustments.

1 Like