Web Services paging based on response

Identity Security Cloud (ISC) ISC Discussion and Questions
,
1

Hey, I’ve read through the docs and it seems like what I’m trying to do doesn’t exactly follow the standard methods (offset/limit, cursor and/or header links). But looking at the docs, I feel like it should be possible without a rule.

The endpoint returns data sequentially and only accepts record ID based paging.

For example, let’s say 10 records are returned per page. The ID of the last record, present in the json element, is 9. To get the next page, I’d need to pass idgt=9 as a query parameter and so on.

I’m trying to get that object by declaring a variable within the paging configuration where $variable$ = $response[-1:].id$ but this always returns nothing (a null value). My fullUrl then ends up as service.com/api?idgt=null

Do I need a rule for this or am I just doing something wrong? The paging docs seem to suggest that you can access the response body.

2

@M_rtenH - Can you share a screenshot of your configuration here for better understanding.

3

Try to follow this one
https://documentation.sailpoint.com/connectors/webservices/help/integrating_webservices/paging_based_on_limit-offset.html

Basicaly the idea is that you have 2 variables that are standard ones $RECORDS_COUNT$ which contains number of records returned in last page and $sysparm_limit$ which is expected page size. As you see in this example:

$sysparm_limit$ = 100
TERMINATE_IF $RECORDS_COUNT$ < $sysparm_limit$
$sysparm_offset$ = $sysparm_offset$ + $sysparm_limit$
$endpoint.fullUrl$ = $application.baseUrl$ + "/api/now/v1/table/sys_user?sysparm_fields=sys_id&sysparm_limit=100&sysparm_offset=" + $sysparm_offset$

if RECORDS_COUNT is less than expected page size than it’s the last page and we can terminate. Of course you need to adjust this example to your needs.

4 
$response[-1:]._id$ = $page$
TERMINATE_IF ($responseHeaders.Content-Length$ > 300000)
$endpoint.fullUrl$ = $application.baseUrl$ + "/tables/WebexpensesUsers?idgt=" + $page$

I’m using content-length for termination for now but I can’t verify how accurately that works until the basic pagination works. Issue is that $page$ always resolves to null.

Only thing I can think of is that while you can access the response object, it won’t work with an array type. Or the jsonPath implementation doesn’t support [-1:]

5

Yeah that would work as well, but currently my issue is getting the ID out of the response body - the IDs aren’t simple linearly incremented integers but rather a unique ID generated by the service

6

Hello,
Have you found a solution?

Thank you

7

Not necessarily, I think I’d need to use a rule for this. But I’m in talks with the vendor to change the service instead.

8

Hi Marten,
Can you give a try for this pagination?

TERMINATE_IF $RECORDS_COUNT$ < 1
$lastElement$ =  $RECORDS_COUNT$ - 1
$endpoint.fullUrl$ = $application.baseUrl$ + "/tables/WebexpensesUsers?idgt=" + $response.[$lastElement$]._id$
9

Hello everyone on this thread,

We were facing a similar requirement to retrieve all Entitlements from a specific Source, plus, the Pagination not working/stopping while using filters.

The issue was fixed after escaping special characters. So I paste the solution here, just for the record. Thanks and greets !

In-Loop: @moreno_carbone @guillermo_fernandez

$pageSize$ = 250
TERMINATE_IF $RECORDS_COUNT$ < $pageSize$
$pageOffset$ =  $pageSize$ + $pageOffset$
$endpoint.fullUrl$ = $application.baseUrl$ + "/beta/entitlements?limit=250&offset=" + $pageOffset$ + "&filters=source.id%20eq%20%22YOUR_SOURCE_ID%22"