Share all details about your problem, including any error messages you may have received.
IIQ is aggregating Workday every hour, then running an Identity Refresh for all Workday identities with only three options selected:
Refresh identity attributes
Process events
Enable partitioning
We have nine Lifecycle Events that we want to check each hour after we aggregate Workday.
We are aggregating 8300 users from Workday.
We have two Task servers. Both servers have 8 CPU and 16GB of memory each. We have the “Identity Refresh Partition” RequestDefinition set to have a maxThreads value of 8.
The refresh (where we are simply wanting to process events) takes about 10 minutes.
If I disable all Lifecycle events, the refresh, with only Process Events selected, still takes over 5 minutes to run.
This setup is currently what I am testing in our QA environment. I am concerned about taking this setup to Prod as our Task servers will be taxed with CPU/RAM for 10 minutes out of every hour of the day. We want our data from Workday to be more-or-less real-time, so that is why we are processing events every hour.
Does anyone have any suggestions for a better approach to this problem?
Or, does anyone have any ideas on how to tune this better?
The identity refresh for Process Events actually seems to run fairly quickly for the part where it reads through each identity. However, it then kicks off this sub-task as a single thread on a single server:
That sub-task takes a long time to run and is the largest part of the overall time that it takes for the identity refresh to run.
another option to decrease the execution time of refresh is mark the Refresh only identities marked as needing refresh during aggregation.
During the aggregation, IIQ set to true the needsRefresh if the account changes the identity; this flag reduces the number of identity examinated during the refresh.
If you have 2 task server with 8 CPU and 8 avaible threads you can configure up to 16 threds on request definition and use 16 partitions.
I guess I’m not familiar with that Identity Refresh option. How does an identity get marked with “needs refresh”? Simply if they had an identity attribute change based on an aggregation?
During the aggregation if an account changes, the identity will mark that needs to be refreshed (needsRefresh=true on the idenity key). Later, you can mark the flag into the Refresh task to relaborate only the identies that have this value.
Be sure on your aggregation task that the below flag is UNMARKED:
