Hi all,
Is there a way to check for tasks that run for a very long time in IIQ?
How can I achieve this?
Thanks in advance
Hi @rishavghoshacc ,
You can check Task results or Administrator Console –> Task
There all task are listed with execution time. You can also use filters if needed.
@rishavghoshacc Do you just want to see or you want to programmatically get it?
If you just want to see, then @mandarsane shared the path.
in case you need in code, in the task result you will have start datetime, completed datetime, running time and average run time. There are methods available on Task Result class to get these details: getCreated: for Start Date Time, getCompleted for Completed date and time, getRunLengthAverage for Average run time, and getRunLength for current run time.
Yes what Mandar said that is correct, also look at the documentation below for more details
yes correct, jut see the task result from there, and if you want to kill a long running task, go to debug and search for taskresult object and then select the task which is in stale status, and delete it, and then trigger it again.
@neel193 We want to write a code and schedule it to check all the running tasks in the system and notify if anything runs more than 6 hours.
@naveenkumar3 We want to write a code and schedule it to check all the running tasks in the system and notify if anything runs more than 6 hours.
@santhirajumunganda We want to write a code and schedule it to check all the running tasks in the system and notify if anything runs more than 6 hours.
@rishavghoshacc Cool. You can get all task result whose status is Pending and use getRunLength to get the current execution duration and compare it with 6 hours and keep adding the task result to a list. Then after the loop send the email notification.
Please give it a try and if you need any help with the script, please let us know.
This will get you 95% of the way there. I’ll leave it up to you to implement the notification.
This rule logs any task running for 6 hours and deletes anything over 12 hours. It’s also designed to ignore tasks that aren’t really tasks, e.g. Dispatch Access History.
<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE Rule PUBLIC "sailpoint.dtd" "sailpoint.dtd">
<Rule language="beanshell" name="Monitor Tasks">
<Source><![CDATA[
//////////////////////////////////////
int timeThreshold = 360; // Default 6 hours in minutes
int deleteThreshold = 720; // Default 12 hours in minutes
//////////////////////////////////////
List exemptTasks = new ArrayList(Arrays.asList("Dispatch Access History",
"Dispatch Access History daily"));
List deletedTasks = new ArrayList();
TaskResult taskResult = (taskResult != void) ? taskResult : new TaskResult(); // Initialize in case rule ran from debug
QueryOptions qo = new QueryOptions();
qo.addFilter(Filter.and(Filter.ne("type", "LCM"),
Filter.ne("type", "LiveReport"),
Filter.ne("type", "Workflow"),
Filter.notnull("type")));
qo.addFilter(Filter.isnull("completed"));
qo.addOrdering("name", true);
qo.setCloneResults(true);
List tasks = context.getObjects(TaskResult.class, qo);
List rv = new ArrayList();
String status = "Success";
for (TaskResult tr : tasks) {
String taskName = tr.getName();
if (exemptTasks.contains(taskName)) {
continue; // Do not process this task
}
Date launched = (tr.getLaunched() != null) ? tr.getLaunched() : tr.getCreated();
Date targetTime = Util.incrementDateByMinutes(launched, timeThreshold);
Date deleteTime = Util.incrementDateByMinutes(launched, deleteThreshold);
Calendar cal = Calendar.getInstance();
Date timeNow = cal.getTime();
if (timeNow.after(targetTime)) {
log.info("The following task has exceeded threshold limit: " + taskName);
rv.add(taskName);
status = "Alerted";
if (timeNow.after(deleteTime)) {
deletedTasks.add(taskName);
}
}
}
for (String deleteTask : deletedTasks) {
TaskResult tr = context.getObjectByName(TaskResult.class, deleteTask);
Terminator term = new Terminator(context);
term.deleteObject(tr);
context.commitTransaction();
}
if ("Alerted".equals(status)) {
log.info("Detected " + rv.size() + " task(s) running beyond threshold: " + Util.listToCsv(rv));
// Implement your email notification logic here to alert administrators of the long-running tasks
taskResult.setCompletionStatus(TaskResult.CompletionStatus.Warning);
}
if (!deletedTasks.isEmpty()) {
log.info("Deleted " + deletedTasks.size() + " task(s) running beyond " + (deleteThreshold / 60) + " hour(s): " + Util.listToCsv(deletedTasks));
// Implement your email notification logic here to alert administrators of the long-running tasks that were deleted
}
taskResult.addMessage(status);
return status;
]]></Source>
</Rule>
It’s worth noting that while deleting the TaskResult makes the task disappear, the task may continue running.
Hi @rishavghoshacc You can utilize the code shared by @sunny_tsang . We had a similar usecase where one of our task gets stuck after running for few minutes in some cases. We developed a rule to find if that task is running for more than 1 hour and delete the task result.
It will be useful for you if you want to achieve the same usecase.
Let me know if you still need any help in building this code.
Thanks
Manish Singh
@msingh900 Does the task run in the background even if we delete the task result as @paul_hilchey mentioned in the previously.
If so, what is the best way to stop a task?
If the task is running and processing then it is not good idea to delete but in somecases tasks gets stuck after sometime. So, IN that case, only option you have is to delete the taskresult.
if it is running, then you can try cancelling it from UI. It usually does not end as soon as you click Cancel/Terminate in UI.
@rishavghoshacc You should check for average run time and add some threshold, if any task is running beyond the limit and task result attributes is not showing any increment on the objects, you can assume it is stuck and delete it.
Could you please re-specify what is your requirement? Do you want to cancel or send email?
If you want to keep it manual, then monitor the task results and cancel the task. Some time it takes few attempts + seconds to cancel the task from UI. In case it is not cancelling, you can delete the task and restart the server hosting the task.