How to do paging in web service connector

[amanKsingh (Aman Singh)]

Which IIQ version are you inquiring about?

Version 8.3

Share all details related to your problem, including any error messages you may have received.

At the time of aggregation operation I am using below URL:
/v1/Users?count=100&startIndex=101

An in pagination I am using below script:

TERMINATE_IF $response.itemsPerPage$ == 0
$offset$ = $offset$ + $limit$
$endpoint.fullUrl$ = $application.baseUrl$ + $endpoint.relativeUrl$ +“?startIndex=”+ $offset$

This script is not ending it is continuously in loop. Forcefully I am terminating it, and below is my response coming from postman:

{
“totalResults”: 170,
“startIndex”: 201,
“itemsPerPage”: 100,
“Resources”:
}

I want to terminate task when resources does not contain any value. Please help me out of this.

[vishal_kejriwal1 (Vishal Kejriwal)]

check below sample if this helps

$limit$ = 100
TERMINATE_IF $RECORDS_COUNT$ < $limit$
$offset$ = $offset$ + $limit$
$endpoint.fullUrl$ = $application.baseUrl$ + "/api/organizations/abcd/vishal?limit=100&offset=" + $offset$"

[Remold (Remold Krol)]

The documentation is very good for the pagination of the Web Services Connector:
https://documentation.sailpoint.com/connectors/identityiq8_3/webservices/help/integrating_webservices/pagination.html

In the information you provided:
Please remove the &startIndex=101 from the URL. So the first call will be starting from 0 and for the next page startindex will be added.
Also replace the ? with a & in $endpoint.fullUrl$ = $application.baseUrl$ + $endpoint.relativeUrl$ +“?startIndex=”+ $offset$ ( There is already a ? in the relative URL

So:
Relative URL = v1/Users?count=100
Paging step:

$offset$ = $offset$ + $limit$
TERMINATE_IF $response.totalResults$ < $offset$
$endpoint.fullUrl$ = $application.baseUrl$ + $endpoint.relativeUrl$ +“&startIndex=”+ $offset$

– Remold

[amanKsingh (Aman Singh)]

Hi @vishal_kejriwal1,
Thanks for your script, I have used your script but it is not working so I have updated some changes in screenshot but It is also not working, can you please provide me some other solution.

[MuhammadMustafa]

Hi Aman. change the limit in the 2nd line replace it with offset and see.

[amanKsingh (Aman Singh)]

Hi @MuhammadMustafa, Thanks for quick response.
I have done changes as per your suggestion, now it is giving 122 record but it having 172 record below is my screenshot for sailpoint and postman.

[Nadim (Nadim El Khoury)]

Hello Aman.

I think that in your paging steps the “TERMINATE_IF” line should be above your $offset$ addition line, or else the condition will be satisfied before you make all your paging calls.

[amanKsingh (Aman Singh)]

Hi @Nadim,

TERMINATE_IF $response.totalResults$ < $offset$
$offset$ = $offset$ + $limit$
$endpoint.fullUrl$ = $application.baseUrl$ + “/v1/Users?startIndex=” + $offset$ +“&count=100”

I have changed the code as per your suggestion but still It is fetching 122 records

[MuhammadMustafa]

Hi @amanKsingh I can join a call with you after the Developer Day activity/sessions and troubleshoot it with you (if you want).

[amanKsingh (Aman Singh)]

Hi all, Still I am facing the same issue.

TERMINATE_IF $response.totalResults$ < $offset$
$offset$ = $offset$ + $limit$
$endpoint.fullUrl$ = $application.baseUrl$ + “/v1/Users?startIndex=” + $offset$ +“&count=100”

I have changed the code as per your suggestion but still It is fetching 122 records

[rajeshjakkula]

Hi Aman,

I am facing same issue,
Were you able to find solution to this

If so, can you share
What did you set for context url and the pagination config

Thanks,
Rajesh

[Remold (Remold Krol)]

Hi @rajeshjakkula,

The documentation is very good for the pagination of the Web Services Connector:
https://documentation.sailpoint.com/connectors/identityiq8_3/webservices/help/integrating_webservices/pagination.html

All webservices are slightly different when looking for the pagination.

For the above issue, I doubt that this issue is related to the pagination, as it gets more accounts than the page size (122 over 100). It looks like there is an issue with the identity attribute (multiple accounts with the same identity attribute).

To see what is response is from the web servers call, you can use the following code for the Web Services After Operation Rule:

log.error("rawResponseObject=" + rawResponseObject);
log.error("processedResponseObject=" + processedResponseObject);

Map updatedMapInfo = new HashMap();
updatedMapInfo.put("data", processedResponseObject);
return updatedMapInfo;

If you have further queries, please open a separate thread in this forum. So we can isolate your issue from Aman’s.

– Remold

[MuhammadMustafa]

Hi @rajeshjakkula and Welcome to the community!!!

If you can elaborate more about what you are doing and the nature of the APIs that would be great so we can able to help you because maybe your issue is different than @amanKsingh’s one.

However, if it’s the same, I can go with you and @amanKsingh on a quick call as previously offered to check with you guys, because again as mentioned by @Remold & I second that “All webservices are slightly different when looking for the pagination.
For the above issue, I doubt that this issue is related to the pagination”

[HarithaGollavilli]

Hi all, am also facing the same issue for the same requirement. Could anyone provide the solution.

[dylanfoggan]

Hi @amanKsingh , @HarithaGollavilli ,

I think you should assign the limit first, please take a look at this example and let me know if this helps.

$sysparm_limit$ = 100
TERMINATE_IF $RECORDS_COUNT$ < $sysparm_limit$

$offset_index$ = $start_index$ + $sysparm_limit$

$endpoint.fullUrl$ = $application.baseUrl$ + "/devapi/user_management/v6/users?offset=" + $offset_index$ + "&limit=100&include_custom_fields=true"

Thanks,

[HarithaGollavilli]

Hi @dylanfoggan

It is pulling only 300 users (totalResults are 1420) and response is

“totalResults”: 1420,
“startIndex”: 1,
“itemsPerPage”: 100,

Below are the paging steps.

$sysparm_limit$ = 100
TERMINATE_IF $RECORDS_COUNT$ < $sysparm_limit$
$offset_index$ = $start_index$ + $sysparm_limit$
$endpoint.fullUrl$ = $application.baseUrl$ + “/scim/v3/Users?startIndex=” + $offset_index$ + “&count=100”

Could you please help me to fix this

[dylanfoggan]

Hi @HarithaGollavilli ,

One thing to confirm, do you perhaps have any AfterOperationRules attached to this aggregation?

Another thing, could you please provide the context URL you are using in the Operation.

Thanks,

[HarithaGollavilli]

HI @dylanfoggan

yes am using AfterOpertionRule.
if (processedResponseObject != null) {
for (Map account : processedResponseObject) {
boolean accountStatus = account.get(“active”) != null
? Boolean.parseBoolean((String) account.get(“active”))
: false;
log.error("accountStatus: " + accountStatus);
if (accountStatus) {
account.put(“IDN_AccountUAR”, “Review Account”);
}
}
}
ContextURL is /scim/v3/Users

[dylanfoggan]

Hi @HarithaGollavilli

Could you please adjust your context URL as such:

/scim/v3/Users?startIndex=1&count=100

another thing you could perhaps try along this is remove the AfterOperationRule for testing to ensure it is not causing any issues.

Thanks,

[bhanuprakashkuruva]

Besides, above all replies, the native application should support pagination; then only that will work. Otherwise, it will not work. I have seen a couple of applications that do not support pagination, like Axero, etc. Please consult with the application team and get confirmation that this is going to allow us to do all these things.