Pulling More Than 10K records with Sailpoint Search API Using Webservices Connector

[imagavis (Sivagami Annamalai)]

I understand that to pull more than 10K records via Search API, we got to use SearchAfter something like below.

POST https://orgname-api.identitynow.com/v3/search?limit=100&count=true { "indices":["identities"], “queryType":"SAILPOINT", "query":{ “query":"*" }, "sort":["id"], "searchAfter”:["2c9180835d38ca0c015d606b50851b1e"] }

How to retrieve more than 10K records using search API

Hi Team, Greetings of the Day! We are trying to generate custom reports using identity now’s search API. We kept the limit as 500 and we are incrementing 500 to offset value per API call. Example: API call 1: https://{{tenant}}.api.identitynow.com/v3/search?limit=500&offset=0 API call 2: https://{{tenant}}.api.identitynow.com/v3/search?limit=500&offset=500 By doing this, we have observed that once offset value is reached to 10000, we are receiving 400 as the response code. And we have al…

Does Sailpoint Webservices connector allow to update the request body OOB? If so, how to read the last user’s id from response and pass it over in request body of the next call?

[gmilunich (Geoff Milunich)]

Take a look at this article by Fernando, where he uses the Before Operation and After Operation rules to store and retrieve the SearchAfter Value: SailPoint IdentityNow - Introduction to loopback connectors

I think this is what you are looking for, but if not, let us know what I missed in your question

[imagavis (Sivagami Annamalai)]

Thanks @gmilunich! This helps but I already have an webservices after operations rule that will concatenate 2 account attributes & sets it to another account attribute. Code below.

import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Iterator;

Map updatedMapInfo = new HashMap();
if (processedResponseObject != null){
    for (Map iterateMap : processedResponseObject) {
        if (iterateMap != null ) {
            String roleName = (String) iterateMap.get("role_name");
            String roleScope = (String) iterateMap.get("role_scope");
            if (roleName != null && roleScope != null) {
                    iterateMap.put("role", roleName + " | "+ roleScope);
            }
       }
   }
}
updatedMapInfo.put("data", processedResponseObject);
return updatedMapInfo;

How do I add an other after operations rule for the same account aggregation operation to use searchafter attribute in request body of the next call?

[iamnithesh (Nithesh Rao)]

Have you checked this page:

Paging with searchAfter Parameter

[gmilunich (Geoff Milunich)]

@imagavis

How do I add an other after operations rule for the same account aggregation operation to use searchafter attribute in request body of the next call?

You can only have one After Operation rule per operation, so you would need to do the After Portion from the suggested values in the same rule that you update the values.

I would look at the suggestion by @iamnithesh as it seems to add the “hasMore” key into the mix and uses the existing Pagination process, which was not present in the link I provided.

[imagavis (Sivagami Annamalai)]

Thanks much @gmilunich & @iamnithesh for the guidance.

I combined the logic into one rule like below & it worked

import java.util.HashMap;
import java.util.Map;
import connector.common.JsonUtil;

Map updatedMapInfo = new HashMap();

Map values = new HashMap();
Map jsonBody = JsonUtil.toMap(requestEndPoint.getBody().get("jsonBody"));
String attribute = jsonBody.get("sort").remove(0).replace("-", "");

String searchAfter = processedResponseObject.get(processedResponseObject.size() - 1).get(attribute);
values.put("searchAfter", searchAfter);

application.setAttribute("transientValues", values);

if (processedResponseObject != null) {
    for (Map iterateMap : processedResponseObject) {
        if (iterateMap != null) {
            String roleName = (String) iterateMap.get("role_name");
            String roleScope = (String) iterateMap.get("role_scope");
            if (roleName != null && roleScope != null) {
                iterateMap.put("role", roleName + " - " + roleScope);
            }
        }
    }
}

updatedMapInfo.put("data", processedResponseObject);

return updatedMapInfo;