Pagination not working on web service connector

Identity Security Cloud (ISC) ISC Discussion and Questions
, ,
1

I have an API where pagination need to work as follows:

  1. {{baseUrl}}/v1/users?index=1&size=100
  2. {{baseUrl}}/v1/users?index=2&size=100
  3. {{baseUrl}}/v1/users?index=3&size=100
  4. {{baseUrl}}/v1/users?index=4&size=100
  5. {{baseUrl}}/v1/users?index=5&size=100

The first call always returns the first 100 records:

image
image1254×599 32 KB

The response looks like this:

{
    "users": [...],    
        "pagination": {
        "lastPage": false,
        "total": 521,
        "pageCount": 6
    }
}

After return all records:

{
    "users": [],
    "pagination": {
        "lastPage": true,
        "total": 0,
        "pageCount": 0
    }
}

My pagination on sailpoint was configured like this:

TERMINATE_IF $offset$ > $response.pagination.pageCount$
$offset$ = $offset$ + 1
$endpoint.fullUrl$ = $application.baseUrl$ + "/v1/users?index="+$offset$+"&size="+$limit$

image
image1274×608 33.1 KB

After aggregation, it only returns the first 100 accounts. I’ve seen several examples and I can’t understand why it doesn’t work.
Can anyone help me?

2

Hi @raibom ,

I have not tried this myself, but looks like this might help your case - Web Service Connector not Aggregating all Accounts - Identity Security Cloud (ISC) / ISC Discussion and Questions - SailPoint Developer Community

Thanks,
Shailee

3

Hi @raibom , can you please try the following paging config ?

> $size$ = 100
> TERMINATE_IF $RECORDS_COUNT$ < $size$
> $index$ = $index$ + 1
> $endpoint.fullUrl$ = $application.baseUrl$ + "/v1/users?index=" + $index$ + "&size=100"
4

Hi @MeKhalbi ,

It didn’t work either. What I see here is that the iteration increment doesn’t work. Shouldn’t this work since the documentation examples use it this way?

5

@shaileeM In the example provided, it worked when pagination was done through the response header. This does not apply to my case because I do not have pagination information in the header.

6

@raibom you can try something like this



TERMINATE_IF $response.$content[*].enabled$ == NULL

 paste code here
7

Hi @raibom,

Have you tried adding a value for $limit$ or hardcoding the value (100) directly in the $endpoint.fullUrl$.?

$limit$ = 100
TERMINATE_IF $offset$ > $response.pagination.pageCount$
$offset$ = $offset$ + 1
$endpoint.fullUrl$ = $application.baseUrl$ + "/v1/users?index="+$offset$+"&size="+$limit$

You may also want to try out with the initial offset value as 0.

Also, try out a different terminate condition as below :

TERMINATE_IF $response.pagination.lastPage$ == true

8

Hi @jesvin90,

As I showed in the last image of the post, it is configured in the page size and page offset variables.

I tried it the way you showed and it returns a maximum of 200 records.

image
image1235×509 32.3 KB

image

9

I was able to implement pagination after asking the team responsible for the system to make a change to the API.
Talking to Sailpoint support, it seems that the response always needs to bring pagination information, either in the response body or header, so that this dynamic data can be iterate, thus creating the next URL. Iterating heardcoded does not work (although it is stated in the documentation, this seems to only work in the documentation system).
A change was made to the API, bringing another attribute called “page”, which indicates the current page, as shown below:

{
    "users": [...],
    "pagination": {
        "lastPage": false,
        "total": 521,
        "pageCount": 6,
        "page": 1,
        "index": "bi004870"
    }
}

The code was like this:

TERMINATE_IF $response.pagination.lastPage$ == true
$page$ = $response.pagination.page$ + 1
$endpoint.fullUrl$ = $application.baseUrl$ + "/v1/users?index=" + $page$ + "&size=100"