HI @vinnysail
You’re correct that IdentityIQ doesn’t have a fully “built-in” automated detection and recovery mechanism for this specific scenario. The TaskResult object’s status changing to Failed or Terminated is the primary built-in indicator, but in your case, it remains Running
Your approach of monitoring long-running tasks via a scheduled rule is a good starting point and a common practice. Let’s delve deeper into potential causes and more robust detection/recovery strategies.
Understanding Why Partitioned Tasks Get “Stuck” (Common Causes)
When partitioned tasks hang without errors, it’s often due to:
Database Issues:
- Deadlocks/Lock Contention: One or more partitions might be waiting for a database lock held by another process (IIQ or external), leading to a deadlock.
- Connection Exhaustion/Stalling: The database connection pool might be exhausted, or a database connection might have silently dropped/stalled for a specific partition thread.
- Slow Queries: A particular data set within a partition might hit a very inefficient query, causing it to run excessively long without erroring out.
Application Server (JVM) Issues:
- Thread Starvation/Deadlock: A thread executing a partition might enter a deadlock with another thread within the JVM, or simply get stuck in a busy-wait state.
- Memory Exhaustion: While less common for “no error” scenarios, a subtle memory leak within a partition’s processing could lead to a slow crawl before a full crash.
- JVM Pauses: Long garbage collection (GC) pauses can make tasks appear stuck.
- Network Issues: Transient network problems between IIQ server(s) and the database, or between IIQ and the target application (for aggregation), can cause indefinite waits.
Network Issues:
- Transient network problems between IIQ server(s) and the database, or between IIQ and the target application (for aggregation), can cause indefinite waits.
Partitioning Configuration:
- Uneven Distribution: If partitions are heavily uneven in data size, one very large partition might naturally take a long time, making it seem stuck.
- Too Many Partitions/Threads: Over-partitioning or setting too many threads can sometimes exacerbate contention issues.