For one of our clients, we are migrating their HR application. As part of this process, we are creating a new authoritative application that uses a delimited format. We receive user records (delta) on an hourly basis. My question is: Is there a recommended frequency for HR account aggregation? We are considering triggering the aggregation twice by iterating through all files to improve performance and avoid loading the system every hour. However, I would like to know if there are any drawbacks to scheduling authoritative aggregation hourly. What are the best practices?
you may want to explore incremental aggregation or batch processing methods to minimize system load, as well as monitor system performance regularly to ensure it meets your needs.
Hi @Arpitha1 - I actually think it depends on your use cases.
I’ve operated successfully on daily HR delta updates.
Think of the use cases: Joiner, Mover, Leaver. The majority of these events in HR systems are scheduled to occur on a date (ie midnight); there are very few events, in my experience, which happen “in real time”. So I tend to schedule early in the morning (say 03:00, to avoid TZ issues between systems).
Even for “Immediate leaver” vs “Scheduled leaver” use cases. It is unlikely that your Immediate leaver process (someone marched off site) would be triggered from HR.
There is a way to check this. If you are currently doing an hourly aggregation, check when the majority of your updates occur and schedule accordingly.