We are looking for input on the performance characteristics of the Workday connector in IdentityIQ for a large-scale aggregation scenario.
Our use case involves aggregating 80K+ users, and we are trying to assess the impact of running a full aggregation at that volume. We would like to understand what kind of performance considerations we should plan for and whether there are any common issues or tuning recommendations for this type of deployment.
Specifically, we would appreciate feedback on the following:
Expected performance impact of aggregating 80,000+ users
Best practices for full aggregation with the Workday connector
Key variables that influence performance most
If anyone has implemented the Workday connector at a similar scale, we would value any insights, benchmarks, or lessons learned you can share.
We are planning to use workday service not RAAS.
Version : 8.5p2 (IdentityIQ)
You should be able to do a full aggregation in an hour or less. Most of the customers I have worked with have about 2-3x that number and we run it several times a day.
Delta aggregation may give you better aggregation performance. You can run a full account aggregation once in a week and rest of the days, you can execute the delta aggregation for workday.
The aggregation time depends on the latency of workday APIs and the number of attributes (OOB & custom) are being aggregated. In general with minimum details a full aggregation may take around 1 to 2 hours.
@Krank Workday connector doesn’t support partitioning but you can configure multiple threads at the app xml. you need to set “aggregationThreadSize”, you can try value=“8” or “4”. you may need to run it few times in your non prod to figure out how many threads effectively work in your environment. Rough estimates: for 85K in our environment with 8 thread size, it completes in ~45 minutes.
@Krank - For an 80K+ Workday population, I would not suggest a daily full aggregation. Instead, Use delta aggregation for normal daily changes and keep a weekly full aggregation for reconciliation and for any connector/schema configuration change.
The supported Workday optimization is parallel aggregation using aggregationThreadSize; As per the SailPoint documents 4 as the default. Start at 4 in non-production, then test 6 and 8 with the same dataset, keeping the lowest value that gives a repeatable improvement without Workday throttling, timeouts, IIQ heap pressure, or database contention.
Thanks Soumya for sharing your insight, we are planning to do full to aggregate the custom attributes, I’m under the impression with delta we can’t get the custom attribute changes.
@Krank Yes. that’s the limitation of workday delta aggregation.. it doesn’t aggregate custom attributes. That’s the reason we had to rely on full aggregation twice a day. we were also considering getting a custom rest endpoints and convert the standard connector to webservices based, we had to halt this approach as we decided to move to different HR app.
First option is full aggregation which is obvious take more time so lets say 2 time we can run this
We can run delta aggregation , yes there is limitation to get all the attribute there we can do out customization which we can do single account aggregation from resource object customization rule
This option we can go with webservice approach completely and then we can use the feature and complete end to end process.
Adding a customization rule will add more delays. so, we should be careful with it. Also, delta payload doesn’t capture all the attributes, so there is a risk of wrong data evaluation. What we had observed, delta was not sending the employee type attribute.. it was coming as null.
We intend to proceed with Options 1 and 2 at this time, as Option 3 would require substantial customization efforts. I would appreciate further clarification on the specific scenarios or use cases that would necessitate the implementation of Option 3.
Additionally, as noted by other team members, the aggregation process for 80,000+ identities is estimated to take approximately 1–2 hours when accounting for a number of custom attributes.