Identity Processing Threshold in IIQ 8.4p2: facing some issues

[daniel_neubert]

Which IIQ version are you inquiring about?

8.4p2

Share all details about your problem, including any error messages you may have received.

Dear IIQ experts,

currently we are implementing and testing the Identity Processing Thresholds on some of our Lifecycle Events and we are facing a couple of challenges we would like to share as well as discuss potential options to overcome those issues.

First of all we had to add the following parameter to our Identity Refresh tasks (which are processing events) in order to make the feature working as expected:

<entry key="enableTriggerIdentityQueue" value="true"/> 

But we are still facing some behavior which we are not yet sure how to deal with.

1. In case attribute-change based Identity Triggers are not executed because of the threshold is exceeded those changes are “lost” according to our test results.
   How are you handling these cases?
2. When using rule based event triggering we are facing the issue that our “multiple launch prevention” approach (details provided here: BSDG 15 - Workflow Trigger Rule Best Practices - Compass) is preventing the Identity Trigger from launching at all.
   This happens due to the fact that the trigger rule is evaluated twice and already in the first place it seems that the change is persisted on the identity cube.

In case anybody already found a suitable workaround especially for the issue mentioned under 2) we would highly appreciated your input on this.
That’s because of the fact that without “double launch prevention” the worklfows will be triggered multiple times.

Regarding the events not being re-triggered we think that this is acceptable compared to the risk we are mitigating by the usage of that feature.

Your thoughts and input is highly appreciated.

Thanks,
Daniel

[paul_hilchey]

Hi @daniel_neubert

For 1) I looked into the identity processing threshold and also concluded it was not useful as it would lose events.

For 2) for some triggers we check if there is a workflow already running (TaskResult with completed=null). If there is, we schedule a Request to run the workflow 2 hours later. If there is an existing request to run the workflow, we delete that so there is only the one. It is an awkward solution, but does work.

Let me know if you want more details.

[daniel_neubert]

Hi @paul_hilchey ,

thank you very much for your reply.

Re. 2) may I ask if you are doing all those checks within the Trigger Rule?
In the past we were also checking for inflight workflows already present but this was not always successful and we still had duplicate workflow launches under some conditions.

I’m pretty sure you’re not doing the checks in the workflow because from our experience this is definitely too late

As your approach sounds very interesting any details you would be willing to share are highly appreciatad.
Thank you very much in advance.

Best regards,
Daniel

P. S.: We are following the approach described here (https://community.sailpoint.com/t5/Technical-White-Papers/BSDG-15-Workflow-Trigger-Rule-Best-Practices/ta-p/73110) and as the document is still active and not withdrawn I would expected that this is still working… Also with new features

[daniel_neubert]

Hi again @paul_hilchey ,

I guess technically you are scheduling the workflow launch as described e. g. in the following post on Compass:
Scheduling a workflow to run in the future from BeanShell/rule code - Compass

I definitely like the concept and will evaluate options to amend at least one of our trigger rules (leaver) to solely kick off a single Leaver WF.

Thanks again,
Daniel

[paul_hilchey]

Hi @daniel_neubert ,

We are doing like this:

    import org.apache.logging.log4j.Logger;
    import org.apache.logging.log4j.LogManager;
    import sailpoint.api.IdentityService;
    import sailpoint.api.SailPointContext;
    import sailpoint.api.SailPointFactory;
    import sailpoint.object.Attributes;
    import sailpoint.object.Filter;
    import sailpoint.object.Identity;
    import sailpoint.object.QueryOptions;
    import sailpoint.object.Request;
    import sailpoint.object.RequestDefinition;
    import sailpoint.object.TaskResult;

    ...

    if (trigger) {
      // Schedule for later if there is already a task running
      String displayName = newIdentity.getDisplayName();
      String taskName = "Azure Provisioning: " + displayName;
      QueryOptions qo = new QueryOptions();
      qo.addFilter(Filter.isnull("completed"));
      qo.addFilter(Filter.like("name", taskName, Filter.MatchMode.START));
      
      if (context.countObjects(TaskResult.class, qo) > 0) {
        log.info("Task already running");
        Date date = new Date();
        date.setHours(date.getHours() + 2);
        
        Map args = new HashMap();
        args.put("workflow", "Azure Provisioning");
        args.put("identityName", newIdentity.getName());
        args.put("isRetry", "true");
        args.put("oldIdentity", previousIdentity);
        
        SailPointContext privateContext = SailPointFactory.createPrivateContext();
        
        qo = new QueryOptions();
        qo.addFilter(Filter.eq("definition.name", "Deferred Task"));
        qo.addFilter(Filter.isnull("launched"));
        qo.addFilter(Filter.isnull("completed"));
        qo.addFilter(Filter.eq("name", taskName));
        List requestList = privateContext.getObjects(Request.class, qo);
        Request request = null;
        if (requestList.isEmpty()) {
          request = new Request();
          request.setName(taskName);
          request.setString1(newIdentity.getName());
          request.setDefinition(privateContext.getObjectByName(RequestDefinition.class, "Deferred Task"));
        }
        else {
          request = requestLink.get(0);
        }
        request.setNextLaunch(date);
        request.setLauncher(context.getUserName());
        request.setAttributes(new Attributes(args));
        
        privateContext.saveObject(request);
        privateContext.commitTransaction();
        SailPointFactory.releasePrivateContext(privateContext);
        
        log.info("Not firing trigger");
        return false;
      }

      return true;
    }

    log.info("Not firing trigger");
    return false;

Note the use of a private context, to avoid affecting any transaction that the Identizer may have open.

[daniel_neubert]

Thank you very much, @paul_hilchey .
I’ll check our trigger rules and will try to amend them accordingly.

Kind regards,
Daniel

[daniel_neubert]

Thanks again, @paul_hilchey .

In the meantime we changed our “double-launch prevention” and it works as expected.
However the approach is quite simple as it worked out for us to “just” check for existing WorkflowCases for the identity in scope.

Further testing reveals some other behavior we are not very happy with:
Let’s assume the following situation:
IdentityTrigger1: Fixed Threshold defined: 10
IdentityTrigger2: Fixed Threshold defined: 30

We are refreshing 25 identity cubes and IdentityTrigger1 would fire 25 times (for each identity in that test case).
However IdentityTrigger2 would solely fire for a single cube.
Test result: no triggers are fired at all because threshold for IdentityTrigger1 is exceeded.

At least for us this is not expected behavior.
Not sure what the general expectations are.

Just keep it in mind in case you are facing a similar situation.

Best regards,
Daniel

[daniel_neubert]

Dear colleagues,

please kindly note that if you are using Identity Processing Thresholds together with partitioned Identity Refresh tasks those thresholds might not work as expected due to erroneous calculation/processing.

We are able to reproduce an issue which leads to the unexpected firing of a high number of Identity Triggers even though a processing threshold is defined.

A support case (CS0389766) is open and I’ll share an ETN as soon as we received it.

Just be aware that under certain conditions (account aggregation for trigger-relevant attributes and cube refresh running concurrently) the Identity Trigger Thresholds might not work as expected.

Best regards,
Daniel