IDN Web services connector pagination ends too early for SCIM V3 apis

Identity Security Cloud (ISC) ISC Discussion and Questions
,
1

Hi I am creating a web services connector for the application OneTrust trying to connect to its scim V3 apis.

We tried pagnation like so:
/Users?startIndex=$offset$&count=1&filter=email co “@company.com”

The pagination steps is:

TERMINATE_IF $offset$ > $response.totalResults$
$endpoint.fullUrl$ = $application.baseUrl$ + "/Users?filter=email co “@company.com”&count=1&startIndex=" + $offset$
$offset$ = $response.startIndex$ + 1

I checked the logs while doing the aggregation and have found that the pagination never stops with the TERMINATE_IF statement. It never stops with the TERMINATE_IF statement. Here is what the logs show:

“==> Calculating data hash value for endpoint: Aggregate Accounts”
“==> Data hash is same. Paging will be terminated for endpoint: Aggregate Accounts”

It only runs the aggregate accounts endpoint twice and then terminates.
https://uat.onetrust.com/api/scim/v3/Users?filter=emails co "@company.com"&startIndex=1&count=1

//uat.onetrust.com/api/scim/v3/Users?filter=emails co "@company.com"&startIndex=1&count=1

As you can see the startIndex never changes eventhough it should on the second run.
I am not really sure why startIndex never changes or why it never ends on the TERMINATE_IF I removed it and it still terminated eventhough it shouldnt.

2

@Yaseenl you can try the below option and see

type $sysparm_size$ = 1

TERMINATE_IF $response.$content[*].enabled$ == NULL

$nextpage$ = $nextpage$ + $sysparm_size$

$endpoint.fullUrl$ = $application.baseUrl$ + $endpoint.relativeUrl$ + "?page=" + $nextpage$ + "&size" + $sysparm_size$or paste code here
3

@schattopadhy I have tried going through the SCIM documentation and we are not able to use pages just the standindex to paginate

4

@Yaseenl try this we used this for and scim source:

TERMINATE_IF $response.$Resources[*].id$ == NULL
$startIndex$ = $response.startIndex$ + $response.itemsPerPage$
$endpoint.fullUrl$ = "https://&startIndex=" + $startIndex$

5

That didnt work. It is still showing the same number of users.

6

----------------EDIT 1----------------------
I increased the count to 3.
I found that it does page twice, but it still brings back 3 accounts.
There are 6 accounts on the source.
So I can assume the endpoint doesnt change on the second page through.

7

I checked the logs again.
just before the paging steps end it does create the new updated endpoint as seen here:
URL after replacing place holders :https://uat.app.com/api/scim/v3/Users?filter=emails co "@company.com"&startIndex=4&count=3

but then when it updates the endpoint with V1 endpoint related paging. It reverts back to the original endpoint with regards to the offset.