Recursive Workflows in IdentityNow

IdentityNow Workflows provides a loop operator that allows a workflow to process many items in a list of data in a quick and efficient manner. While the loop operator is a built-in component of Workflows, there is another way to achieve looping functionality using features that are currently supported in the product. In this blog, I will demonstrate how to implement recursion in Workflows to achieve looping behavior without the need for the loop operator.

What is recursion?

Recursion is a form of iteration that is achieved by creating a function that will call itself. To demonstrate this concept, let’s compute the factorial of a number using a loop.

num = 7
factorial = 1

for i in range(1, num + 1):
       factorial = factorial*i
   
print("The factorial of",num,"is",factorial)

Now let’s compute the factorial of a number using recursion.

def factorial(x):
    if x == 1:
        return 1
    else:
        return (x * factorial(x-1))


num = 3
print("The factorial of", num, "is", factorial(num))

As you can see, recursion employs the use of a function that will call itself in order to perform the iterations necessary to calculate the result. In most cases, loops are easier to implement and understand, but recursion is an option in many programming languages that can make some problems easier to implement.

Recursive Workflows

Recursion in Workflows employs the use of the following three features to process a list of items.

  1. External Trigger
  2. HTTP Request Action
  3. JSONpath

A recursive workflow will continuously invoke itself until there are no more items left in the list. In this example, the workflow is started with a list of three numbers. It will process each number in the list in turn, and then invoke itself to process the next number. It will repeat this process until there are no more numbers in the list.

    flowchart TD
    Z[Invoke workflow with list of items] --> |"[1,2,3]"| A
    A[External Trigger] --> B{If length of list > 1}
    B -->|no| C[Perform actions on last item]
    B -->|yes| D[Perform actions on first item]
    D --> E[HTTP Request to invoke this workflow with first item removed]
    E -->|"[2,3]"| A

Pros and Cons of Recursive Workflows

Recursive workflows have some advantages compared to the loop operator, but there are also drawbacks to using recursion. When possible, prefer to use the loop operator. However, recursion is a viable option if the pros outweigh the cons for your particular use case.

Pros

  1. There is an option to implement synchronous recursion, which allows you to execute a series of commands on each item in the input list in the order in which they are received. The loop operator is strictly asynchronous. Asynchronous is usually better, as it is much faster than synchronous, but in situations where you need synchronous execution, recursion allows for that.
  2. There is no strict input limit for recursive workflows. If you need to process more items than the loop operator can support, then recursion allows you to do this. However, it is a good idea to check the length of your input before invoking a recursive workflow to make sure you don’t accidentally iterate over a very large data set, which can have performance implications.

Cons

  1. Recursive workflows are harder to implement and understand, especially for less advanced users.
  2. Recursive workflows can only accept arrays as input, not objects. This means that they cannot work with a context like the loop operator can, which limits the use cases they can support. For example, if you need to revoke a list of entitlements for an identity, then you would normally pass the list of entitlements to the loop input and the identity ID to the loop context. However, since recursion can only accept an array as input, you can only pass it the list of entitlements and not the identity ID as well, which means this use case won’t work with recursion. Support for objects as an input type in recursion may be coming in the future.

Synchronous Recursion

Recursive workflows can be configured to run synchronously, which means the recursion will process each item in the list in the order in which they are received. For example, if the list [1,2,3] is provided to the recursive workflow, it will process 1 first, then 2, and finally 3.

Implementation

A fully implemented synchronous recursive workflow will look like this, at a minimum. This workflow may be larger depending on how many actions you need to perform on each item in the list. You can download the workflow script and follow along in your tenant. Continue reading to learn what each step in this workflow does, and how to configure it.

Synchronousrecursion20230818.json (2.6 KB)

External Trigger

Click on External Trigger and generate a new access token. Make sure to save the client ID, client secret, and client URL, as you will need these to configure the HTTP Request actions later.

Verify Data Type

The Verify Data Type step doesn’t require any further configuration. It is configured to check if the type of data coming from the External Trigger is an array or not. This check is necessary since Workflows will remove the array brackets from single item arrays. If the trigger input is an array, then it takes the right path, which will process the first item in the array and then pass the remaining items to the next invocation of the workflow. If it is not an array, then it knows it is at the last item in the list and it will take the left path to process that last item and finish the recursion.

Verify Data Type 1

If the External Trigger input is not an array, that means the recursion has arrived at the last item in the original list. We need to do one more check to see if the trigger input is null or not, which depends on the initial payload sent to the External Trigger.

If the initial trigger input is an array of one item (ex. [{"id":"123"}]), then the workflow will take the right path at Verify Data Type. It will perform the actions on the only item in the array, but it will invoke the workflow one more time with a null object, since $.trigger[1:] results in a null value for an array with only one item. Therefore, we need to check for a null value which indicates that there is no more input data left and the workflow should terminate. If there is a null value, then the workflow will take the right path of Verify Data Type 1, which will terminate the workflow.

However, if the initial trigger input is an array of more than one item, then the last item in the recursion will not be null. It will be a single object. Verify Data Type 1 will take the left path to process the last item of the recursion one last time, before terminating for good.

HTTP Request 3

If the External Trigger input is not an array and it is not null, that means the recursion has arrived at the last item in the original list. The only thing left to do is perform the series of actions necessary to process the item and then terminate the recursion for good. To keep this example simple, the only action we perform on the list item is an HTTP Request action to send the data to webhook.site (read more about webhook.site in workflows). In a real scenario, you would have one or more actions that process the item, or you might have a separate workflow that you can invoke via an External Trigger to better encapsulate the processing actions.

It’s important that both the left and right branch of the Verify Data Type step perform the same series of actions to process the data. There will be a lot of copy and paste. Just make sure that you use the correct JSONpath expression for actions in each branch based on whether the trigger input is an array or not. The JSONpath used in the HTTP Request 3 action is simply $.trigger since the input is not an array.

HTTP Request

If the External Trigger input is an array, that means there is more than one item left to process. The right branch of the Verify Data Type step is followed by a series of actions that will process the next item in the list. This could be creating a cert campaign, submitting an access request, to name a few examples. To keep this example simple, the action that will process the next item in the list is an HTTP Request that will send the data to webhook.site.

It’s important that both the left and right branch of the Verify Data Type step perform the same series of actions to process the data. The right branch will be an exact duplicate of the left branch, with the only difference being the JSONpath used to reference the input data. The JSONpath used in the HTTP Request action of the right branch is $.trigger[0] since the input is an array and we want to process the first item of the array.

HTTP Request 1

The last step in the right branch is to use an HTTP Request to recursively invoke this workflow with the remaining items in the array. Since the previous action(s) processed the first item in the trigger payload, we need to use the JSONpath $.trigger[1:] to send everything but the first element in a new invocation of this workflow.

image

The target of this HTTP Request is the External Trigger of this workflow, so you will need to configure the authentication and request URL to point to the trigger of this workflow.

Caveats

Synchronous recursion is slower than asynchronous recursion. The workflow must wait for all actions that process the current item to complete before invoking the next iteration. If the order in which items are processed is important, then synchronous recursion is the right choice. Otherwise, use asynchronous recursion for much faster execution times.

Asynchronous Recursion

Recursive workflows can be configured to run asynchronously, which means the order in which items are processed is not guaranteed. In many use cases, processing items in order is not necessary. For example, if you need to disable a list of identities the order in which they are disabled probably doesn’t matter. Asynchronous recursion is much faster than synchronous recursion since the workflow doesn’t have to wait for processing steps to complete before invoking the next iteration of the workflow. In most cases, you should prefer to use asynchronous recursion.

Implementation

A fully implemented asynchronous (async) recursive workflow will look like this, at a minimum. Notice the use of the loop operator in async recursion. Using the loop operator greatly speeds up the execution of the workflow. This workflow may be larger depending on how many actions you need to perform on each item in the list. You can download the workflow script and follow along in your tenant. Continue reading to learn what each step in this workflow does, and how to configure it.

AsynchronousRecursionLoopsDemo20230818.json (4.3 KB)

External Trigger

Click on External Trigger and generate a new access token. Make sure to save the client ID, client secret, and client URL, as you will need these to configure the HTTP Request actions later.

Loop

Since the loop operator processes items asynchronously, we use it in our async recursive workflow to process the input items asynchronously as well, greatly speeding up execution times. Since loops currently have a limit of 100 input items, the input to Loop will be the first 100 items of the External Trigger payload.

HTTP Request 3

Inside Loop is a single action; an HTTP Request to invoke webhook.site with the data in the current item, referenced with the JSONpath $.loop.loopInput. This is a simple demonstration, but in practice you will likely have more actions inside Loop to process each item accordingly. This could be disabling an identity, or starting a certification campaign.

image

Compare Numbers

The loop will process the first 100 items asynchronously. After the loop, we need to check how many items were in the External Trigger payload to determine how to proceed. Compare Numbers will check the length of the trigger input. If the length is greater than 100 (the limit of the loop step), then there may be more items to process, which will require a recursive call to this workflow with the remaining items in the payload. If the length is less than 100, then we know the loop was able to process the last of the items in the trigger payload, and we can terminate the workflow.

Compare Numbers 1

Since Workflows will remove the array brackets of single item arrays, we have to handle the edge case where the trigger payload contains 101 items. If we attempt to invoke this workflow with a JSONpath expression of $.trigger[100], then the array brackets will be stripped and the trigger payload will contain an object. This will cause an error, as the loop is expecting an array, not an object. Therefore, we need another compare numbers operator (Compare Numbers 1) to check if the trigger input is more than 101 items. If it is more than 101 items, then we can recursively invoke this workflow with the remaining items since the array will be preserved. However, if we hit the edge case of 101 items, then we just have to process that last item with the exact same steps performed in the loop and then terminate the workflow.

HTTP Request 1

If we have 101 items left in the trigger input, then we need to process that last item (item #101) with the exact same steps performed in the loop. In this demonstration, the only step that we performed in the loop was to send the item data to webhook.site. HTTP Request 1 will use the JSONpath $.trigger[100] to send item 101 to webhook.site.

JSONpath array indexing is 0 based. $.trigger[0] points to the first item in the array, while $.trigger[100] will point to the 101st item in the array.

image

HTTP Request

If there are more than 101 items in the trigger input, then we need to recursively invoke this workflow, passing all of the items after the first 100, which were already processed in the loop. HTTP Request is configured to invoke the External Trigger of this workflow with the remaining input items.

image

The target of this HTTP Request is the External Trigger of this workflow, so you will need to configure the authentication and request URL to point to the trigger of this workflow.

Testing Recursive Workflows

Recursive workflows require the workflow to be enabled in order to call the External Trigger with the remaining input. Because of this fact, the “Test Workflow” feature is of minimal value. You can use “Test Workflow” to test a single run of the workflow, but since the workflow isn’t live it won’t be able to invoke itself again. Therefore, the only way to truly test this is to enable the workflow and invoke the External Trigger with small test payloads. You can view the execution results to see if it worked or not. You can use a tool like Postman to test recursive workflows. This will require you to make two API requests: one request to get an access token, and one request to trigger the workflow with test input.

Get An Access Token

Using the Client ID and Client Secret provided by the External Trigger of your workflow, make the following API call to get an access_token.

POST https://{your tenant}.api.identitynow.com/oauth/token?grant_type=client_credentials&client_id={your client id}&client_secret={your secret}

Copy the access_token for use in the next step.

Invoke the Workflow

The External Trigger will give you the Client URL needed to invoke the workflow. Copy it from the external trigger and use it in your next API call to send test data.

Then, make an API request to the Client URL with the data you want to test. In this demonstration, we will send a very basic array of objects that will be sent to webhook.site.

curl --location 'https://{tenant}.api.identitynow.com/beta/workflows/execute/external/{workflow_id}' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {your access_token}' \
--data '[
    {
        "key": "1"
    },
    {
        "key": "2"
    },
    {
        "key": "3"
    }
]'

In this example, the workflow sends the data to webhook.site. After invoking the test payload, I see the following:

Using a Recursive Workflow

Recursive workflows can be used anywhere you would normally use a loop operator, as long as the loop doesn’t need a context defined. At this time, recursive workflows can only work with an array input since there is no way to pass additional information as is supported by the loop context variable.

The following workflow is a simple example of where a recursive workflow can be used. This workflow runs on a scheduled trigger that periodically queries the list identity certifications endpoint for any certifications still in the “Active” phase. Normally, a loop might be used here to check the created date of each certification to see if the certification has been active for more than a certain period of time, say 14 days. If a certification has been active for at least 14 days, then send a reminder email to the reviewer to complete the certification.

This workflow will fail if there are more than 100 certifications in the “Active” phase. However, if you use a recursive workflow to process the list of certifications instead of a loop, you will be able to process more than 100 certifications. To replace the loop operator in this workflow, you first must create a recursive workflow that performs the same actions that are performed inside the loop operator. Then, replace the loop operator in your main workflow with an HTTP request that invokes the recursive workflow with the list of certifications. Building the recursive workflow is an exercise left to the reader, but the main workflow should look like this. The first HTTP Request invokes the list identity certifications endpoint to get the list of active certifications, and the second HTTP Request invokes the recursive workflow with the list of certifications in the request body.

RevokeEntitlement20230821 (3)

Debugging Recursive Workflows

Synchronous

Synchronous recursive workflows will create a new workflow execution for each iteration. As a result, you will see many executions that are close together in time. If any individual execution fails, you can click on that execution to see why it failed. For example, if I invoke the workflow with five items, you will see five executions very close together in time.

Asynchronous

Since asynchronous recursive workflows use the loop operator to speed up execution, the execution history will look different than for synchronous recursion. If you have an input loop of less than 100 items, then only one execution will be triggered, and the details for each item in the loop will be contained within that single execution. You will need to download that execution report to check each iteration of the loop for any failures, just like you would for a normal workflow that uses loops.

If you have an input of more than 100 items, then you will see one execution for each time the workflow was recursively invoked. For example, if you invoked the workflow with 230 items, then you will see three executions.

  • The first execution report will contain the results of the loop for items 1-100
  • The second execution report will contain the results of the loop for items 101-200
  • The third execution report will contain the results of the loop for items 201-230

Stopping infinite loops

If you don’t properly handle the termination conditions of a recursive workflow or correctly pass the remaining list items to the next iteration of the recursion, you may end up creating an infinite loop, which means the recursion will never end and just keep calling itself. If you encounter a scenario like this, the correct course of action is to disable the recursive workflow and wait for all running executions to finish. Once there are no pending or running executions left, you can then fix the workflow so that it does not exhibit infinite looping anymore and then enable the workflow again.

17 Likes

Hi @colin_mckibben We are building the recursive workflow with external trigger and I notice sometime recursive workflow doesn’t trigger second execution even though first execution logs show it has successfully trigger the second execution and have workflowexecutionId in logs. Not sure if it sandbox environment issue. Just wanted to check if you have encounter any such issues before.

Possibly a delay in the workflow execution queue. I have noticed at times during my testing that the execution service can slow down and it can take a a while for the queued actions to be processed. If I recall correctly, the queue will eventually pick up again and process the requests.

Hi @colin_mckibben Initially I thought the same but interesting part is, List workflow execution API also doesn’t show anything on the queue. Is there some other API / method to see if execution is in queue or not ?

If it gave you a workflow execution ID but you aren’t seeing it being executed, then I would open a support ticket to investigate further. I would treat this as any other workflow not being triggered correctly.

1 Like

Hello @colin_mckibben we have tried this solution in dev. It works fine for first element of input array but while invoking second iteration at(as per your example-HTTP Request1) it failed with error “Error Parsing Input, retryable: false): Unable to parse input as JSON” on jsonRequestBody containing $.trigger[1:]. Even we tried to change the value of jsonRequestBody to {{$.trigger[1:]}} but no luck.

Nice stuff!

We need granular certifications for some Access Profiles due to different ownerships, and the amount of APs is 100+.

I’m wondering, is this possible to use recursion as a solution to this use-case or will it be difficult due to the fact that I can’t check for identity IDs?

It might be possible, but I’m not sure of the details necessary to make this work. What would this workflow look like if you tried doing a loop? That would tell me all of the inputs you need to kick off certs and I could tell you if recursion would be possible or not.

I think it’ll work!

Just using the search API instead of v3/access-profiles so I don’t have to paginate and also I do get all the info I need in each object (owner id and id of AP). So It’s looking bright atm.

Thanks.

Glad to hear it :slight_smile: If you manage to get it working, would you mind creating a colab and/or blog post sharing your implementation?

Good idea!

Will do once I have a good working implementation:

1 Like

I’m wondering if it’s possible to make it even better:

Using recursion workflow:

Fetch all owners of roles and access profiles in the tenant. Send this list to the recursive workflow.

Inside the recursion workflow, have the owner-list as input to the loop and fetch the objects that are owned. Create Certification Campaign for each owner along with the owned access-objects.

Is it possible to make a certification campaign via HTTP or via the “Create Certification Campaign Action” that includes both access profiles and roles?

I recall there being difficulty passing a list to a HTTP request as the access-list was formatted as a map. This may have been changed since I tried about 6 months ago or so?

Perhaps there is a workflow already configured in the community that deals with owner certifications like this?

I realised in the original plan, we’d end up with 1 certification campaign per access profile which… Well it’s not the most pleasant to look at when suddenly you have 200+ campaigns containing 1 AP each. Would be nice to bundle as much as possible together and possibly pivot to some form of “Owner certification campaign” for all Roles and Access Profiles in a tenant.

1 Like

Inline variables that are not strings don’t resolve to JSON. They resolve to Golang maps and arrays. This will present a problem when you try to mix two types of object arrays.

Perhaps you can look into utilizing a search campaign to create your campaign.

Thanks Colin,

I’ve managed to create a recursive workflow now that loops through all owners of access (entitlements, access profiles, roles) in ISC and then creates a Search Campaign using the query “owner.id:{{ownerid}}” to fetch all the access items that each owner owns and certify all users with access to these items. So basically a full-scale “Access Owner Certification Workflow”

Might be a great addition to the co-lab once I have customers confirmation on letting me post it. Not sure about the legal stuff, as I believe technically it’s not my property to share, gonna check this anyway.

Followup question: Are there any ideas or items in the pipeline for extending the OOB certification functionality? For example to include stuff like this, application campaigns, owner campaigns, etc etc? Wondering if this workflow might not even be needed in the future.

Thanks.

1 Like

Hi Colin,

I think it is worth having a note somewhere above that if implementers intend to use recursive workflows to call a SailPoint endpoint, then they need to be sure that the SailPoint IP addresses from https://files.accessiq.sailpoint.com/network/region/source_ips.txt are included in their SailPoint tenant’s IP Allow list. I ran into this issue and could not find it called out anywhere and SailPoint Support did not know to check for this.

A post was split to a new topic: Remove roles for multiple users using asynchronous recursion

Hi Sebastian - Can the workflow you created be customized to trigger a role owner/access profile owner for a particular source? We are interested in this. I know there are ways to do a role owner cert using powershell scripts, but if we can do it using just SailPoint, that would be great.

Hey Vidhi,

It should be possible yea! It really just comes down to what Search Query you use, currently we are using owns.accessProfiles:* AND owns.roles:* just to fetch all owners of APs and Roles, but you could probably filter on source as well!

Seb

Can we use other triggers such as Account Aggregation Completed instead of External Trigger for Asynchronous Workflows?

When using recursive workflows, your main workflow uses any one of the available triggers, like Account Aggregation Completed, as the entry point. However, you need to create a second workflow that must use the External trigger to process the list of data. Since recursive workflows call themselves, the only trigger that can call itself is the External trigger.