Web Service Connector not Aggregating all Accounts

[anthnyhm (Anthony Hamilton)]

I have created and connected a Web Services connector. I then created an HTTP Operation to aggregate all system accounts. However, the aggregation is only pulling over 100 accounts out of 674. I attempted to change the paging settings according to SailPoint’s Web Services documentation and the connector then scanned 200 accounts out of 674, but again only aggregated 100. Not sure where to go from here?

Root path is: $.entities

I’m pretty sure I’m not setting up the paging correctly:

relative url: /1.0/entities

Paging:

$paging_limit$ = 50
TERMINATE_IF $RECORDS_COUNT$ < $paging_limit$
$offset$ = $offset$ + 1
$endpoint.fullUrl$ = $application.baseUrl$ + “/1.0/entities”
$request.num_pages$ = $offset$

Example of API Output (Redacted):

HTTP 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

{
    "count": 614,
    "next": null,
    "previous": "https://api.zerofox.com/1.0/entities/?page=6",
    "num_pages": 7,
    "entities": [
        {
            "id": 5******,
            "name": "Wi*****th",
            "email_address": "W*********.com",
            "image": "**********",
            "organization": "",
            "labels": [
                "Advisors"
            ],
            "strict_name_matching": false,
            "policy_id": 3020,
            "profile": null,
            "entity_group": {
                "id": 201,
                "name": "Default"
            },
            "type": {
                "id": 3,
                "name": "Executive/VIP"
            },
            "type_id": 3,
            "next_account_expiration": null,
            "next_account_expiration_iso": null,
            "status": {
                "id": 1,
                "name": "enabled",
                "display_name": "Enabled"
            },
            "protection_status": {
                "id": 1,
                "name": "protected",
                "display_name": "Protected"
            },
            "source": {
                "id": 1,
                "name": "manual",
                "display_name": "Manual"
            },
            "address": "",
            "phone": "",
            "description": ""
        },

[anthnyhm (Anthony Hamilton)]

This issue has been resolved by switching to pagination by response headers. Paging steps were updated with specific response attributes. This allowed the HTTP Operation to pull all accounts. See below:

TERMINATE_IF $response.next$ == null 
$endpoint.fullUrl$ = $response.next$
$request.cursor$ = $response.cursor$
REMOVE $request.limit$

[MVKR7T (Krishna Reddy Mummadi)]

Hi @anthnyhm

Welcome to SailPoint developer community.

Glad to see that you found the solution, please mark your response as solution. So that others can rely on it.

Thanks
Krish