Terminate and Delete stuck Perform Maintenance task

Hi All!

I am looking at options to terminate and delete stuck multi-threaded Perform Maintenance task.

We have seen that our perform maintenance task is stuck at times. I am trying to create a rule runner task, that will look at the Perform Maintenance task, look at how long it has been running and then terminate the task and then delete task result. This way the Perform Maintenance task will run again as scheduled(i.e. within next 5 mins)

I have I have an idea of how I will do it. The only thing that I am confused is, will I be able to stop a multi-threaded Perform Maintenance task via beanshell?

Also, is there anyone who have implemented something similar. Any help and suggestion is appreciated.

Hi @kpudasaini - You can use something like this to detect a hung PM task. This allows you to add a config item to a rule runner task to determine how long to let the PM task run using “noOfHours”. If it is hung or running longer than you specify, you can get the task result and delete it (allows it to run again at the 5 min schedule) or you can try to set taskResult.setTerminateRequested() to true and see if that stops it in a timely manner. The rule run task can also send a failure message based on the result so you can get an email when it is detected as hung.


<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE Rule PUBLIC "sailpoint.dtd" "sailpoint.dtd">
<Rule language="beanshell" name="Check Perform Maintenance Status" >
  <Description>
    Import template-defined Roles from a CSV file.
  </Description>
  <Signature returnType="Map">
    <Inputs>
      <Argument name="noOfHours">
        <Description>
          Input the number of hours to check perfrom maintenance task status
        </Description>
      </Argument>
      <Argument name="log">
        <Description>
          The log object associated with the SailPointContext.
        </Description>
      </Argument>
      <Argument name="context">
        <Description>
          A sailpoint.api.SailPointContext object that can be used to query the database if necessary.
        </Description>
      </Argument>
    </Inputs>
  </Signature>
  <Source>


  import java.util.*;
  import java.io.File;
  import java.util.List;
  import java.util.HashMap;

  import sailpoint.object.Identity;

  import sailpoint.object.Filter;
  import sailpoint.object.QueryOptions;

  import org.apache.commons.logging.Log;
  import org.apache.commons.logging.LogFactory;
  import sailpoint.object.TaskItem;
  import sailpoint.object.TaskDefinition;
  import sailpoint.object.TaskResult;
  import sailpoint.tools.Message;
  import sailpoint.tools.Message.Type;


  QueryOptions qo = new QueryOptions(); 
  List ordering = new ArrayList();
  ordering.add(new QueryOptions.Ordering("modified", false));
  qo.setOrderings(ordering);

  Filter f = Filter.and(Filter.eq("name","Perform Maintenance"),Filter.eq("completionStatus","Success"));
  qo.addFilter(f);
  List taskList = new ArrayList(context.getObjects(TaskResult.class, qo));
  Date mod = null;
  Date launchedTime = null;
  
  Filter checkMaintenanceFilter = Filter.eq("name","Check Perform Maintenance Task");
  TaskResult checkMaintenanceResult = context.getUniqueObject(TaskResult.class, checkMaintenanceFilter);
		 

  boolean flag = false;
  for(TaskResult taskres : taskList){

    mod = taskres.getModified();
    launchedTime = taskres.getLaunched();
    long modifiedDate = mod.getTime();
    long millis = launchedTime.getTime();
    String noOfhrs = (String)config.get("noOfHours");

    long nh;
    long finalVal;
    if(!noOfhrs.equals("0.5"))
    {
      nh  = Long.parseLong(noOfhrs);
      finalVal = (nh*3600000)+millis;
      System.out.println("1 hr:: "+finalVal);
    }else 
    {
      finalVal = ((1/2)*3600000)+millis;
      System.out.println("30 mins ::: "+finalVal);

    }

    if(modifiedDate &lt; finalVal){

   
      List msgList= new ArrayList();
      Message msg = new Message();
      msg.setType(Message.Type.Error);
      msg.setKey("Perform Maintenance Task is hung for more than "+noOfhrs+" hours");
      msgList.add(msg);

      checkMaintenanceResult.setCompletionStatus(TaskResult.CompletionStatus.Error);
      checkMaintenanceResult.setMessages(msgList);

      flag = true;
      context.saveObject(checkMaintenanceResult);
      context.commitTransaction();
      context.decache();

    }

    break;
  }	

  return flag?"Error":"Success";


  </Source>
</Rule>

TaskManager tm = new TaskManager(context);
tm.terminateTask(result.getId()); might be helpful

Thank you! This is a partitioned perform maintenance task. Do you know how can I terminate it?

Thank you! This is a partitioned perform maintenance task. I see the method terminateTask() cannot be used for a partitioned task.

Looks like after the partitions are launched the requestExecutor for that task does not include a terminate option. Have you tried getting the task result and deleting it? This will end all the threads once it is gone. May be worth a try.