Workflow HTTP Search API only returns 250 active identities — how do I paginate to retrieve all ~6000?

Identity Security Cloud (ISC) ISC Discussion and Questions
,
1

Hi everyone,

I’m using a scheduled workflow in ISC and calling the Search API through the HTTP workflow operation to retrieve all active identities.

My environment has about 6000 active users, but my workflow is only returning 250 identities.

This is the API I’m calling:

POST https://companytenant-sb.api.identitynow.com/v3/search

Request body:

{
  "indices": ["identities"],
  "query": {
    "query": "attributes.cloudLifecycleState:active"
  },
  "queryResultFilter": {
    "includes": ["id", "displayName", "email"]
  }
}

My goal is to make sure the workflow processes all active identities, not just the first 250.

From what I’ve found, Search can return only the default first page unless pagination is handled explicitly, and recent SailPoint guidance indicates that if you are only getting 250 results you may need to pass a limit value and then page through the remaining results. SailPoint’s standard collection docs say the Search endpoint supports up to 10,000 records per page, while community guidance also notes that 250 is what many users are currently seeing by default if limit is not specified.

What I’m trying to understand is:

  1. What is the correct way to paginate /v3/search inside a workflow HTTP action?

  2. Should I be using limit and offset, or is there a better pagination method for Search in workflows?

  3. If the workflow has to loop through pages, what is the recommended pattern to continue calling Search until all records are returned?

  4. Has anyone successfully retrieved thousands of identities in a single scheduled workflow run using the HTTP action?

If helpful, I’d also appreciate an example of:

  • the full HTTP request including pagination parameters

Thanks in advance.

2

image
image1364×692 34 KB

3

Update the Search API HTTP Request body to include "limit": 500 and "offset": 0 so the workflow returns more than the default 250 identities.

4

Have you tried setting the limit to 6000 to bring back all the indentities?
You are now bringing back that much information so you shouldn’t hit a sizing issue

5

@aomololu You cannot process more than 250 entries in a loop in the workflow. It is a limitation of the loop in the workflow - Workflow Operators - SailPoint Identity Services

Even though you could retrieve from a HTTP call, it cannot be processed in the workflow.

6

Since you cannot turn pages in workflow and since loop is a max of 250, May be if you explain the purpose of this workflow we might help find an alternative approach

7

Hi Ade,

You can do this using a Recursive Workflow. Recursive Workflows in IdentityNow

Like others have mentioned though, there is likely a better way to achieve your desired result. If you share your use case, I’m sure the community can help to provide ideas.

Thanks

Margo

8

Depending on what you are trying to achieve, another option would be create a script using the Sailpoint SDK. It is very easy to paginate with the SDK, and then take the data and make your other API calls.

The downside is you would have to run the script from a machine outside of ISC. We setup a server to house our scripts, and then use a very basic scheduling app to run them.