shijingg
February 17, 2025, 7:07am
1
8.4
I am trying to retrieve all groups during account group aggregation, however, the pagination code does not seem to work and I am only able to retrieve 50 groups. Based on the API specifications, I have specified my page to be of size 50 and if $response.nextlink$ contains a string then it should continue paging? What am I doing wrongly? The paging is not working. I have a total of 70 groups.
Arpitha1
February 17, 2025, 8:00am
3
In your context url append this at last
?size=$limit$
Arpitha1
February 17, 2025, 10:20am
5
This should work if this attribute itself not coming in last response payload.
shijingg
February 18, 2025, 1:43am
6
Hi @Arpitha1
Am I right to say if I have 70 records and I defined my page size to be 50 if nextlink IS NOT NULL, it should return me
Page 1 => 50 records
Is that correct?
Arpitha1
February 18, 2025, 3:48am
7
Yes Correct. Can you send the Page 2 API response payload ?
Also, try keeping Page Size as 10, update context url ?size=$limit$ (Not with hardcoded value like size=50) and let me know the result.
shijingg
February 18, 2025, 4:26am
8
Hi @Arpitha1
Page 2 API response payload
Am I using $limit$ in the correct way? Getting runtime exception. But there is indeed a nextLink value.
Arpitha1
February 18, 2025, 5:29am
9
Can you share your application xml over chat, by masking sensitive data. I need to view context url and other details.
shijingg
February 18, 2025, 6:15am
10
Hi @Arpitha1
<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE Application PUBLIC "sailpoint.dtd" "sailpoint.dtd">
<Application connector="sailpoint.connector.webservices.WebServicesConnector" created="1724734668973" featuresString="PROVISIONING, SYNC_PROVISIONING, SEARCH" id="0af403fa917a11b48191923280ad0603" modified="1739852392618" name="*******" profileClass="" significantModified="1739852392618" type="Web Services">
<AccountCorrelationConfig>
<Reference class="sailpoint.object.CorrelationConfig" id="0af403dc925c1b1b81926bb8ecbf74c1" name="*******"/>
</AccountCorrelationConfig>
<Attributes>
<Map>
<entry key="addRemoveEntInSingleReq">
<value>
<Boolean>true</Boolean>
</value>
</entry>
<entry key="afterProvisioningRule"/>
<entry key="aggregationPartitioned">
<value>
<Boolean></Boolean>
</value>
</entry>
<entry key="aggregationType" value="account"/>
<entry key="authenticationMethod" value="No Auth"/>
<entry key="beforeProvisioningRule"/>
<entry key="clientCertAuthEnabled">
<value>
<Boolean></Boolean>
</value>
</entry>
<entry key="compositeDefinition"/>
<entry key="connectionParameters">
<value>
<List>
<Map>
<entry key="afterRule"/>
<entry key="beforeRule"/>
<entry key="body">
<value>
<Map>
<entry key="bodyFormData">
<value>
<Map>
<entry key="client_id" value="*******"/>
<entry key="client_secret" value="*******"/>
<entry key="grant_type" value="client_credentials"/>
<entry key="scope" value="*******"/>
</Map>
</value>
</entry>
<entry key="bodyFormat" value="formData"/>
<entry key="jsonBody"/>
</Map>
</value>
</entry>
<entry key="contextUrl"/>
<entry key="curlCommand"/>
<entry key="curlEnabled">
<value>
<Boolean></Boolean>
</value>
</entry>
<entry key="customAuthUrl" value="*******"/>
<entry key="header">
<value>
<Map>
<entry key="Content-Type" value="application/x-www-form-urlencoded"/>
</Map>
</value>
</entry>
<entry key="httpMethodType" value="POST"/>
<entry key="operationType" value="Custom Authentication"/>
<entry key="paginationSteps"/>
<entry key="pagingInitialOffset">
<value>
<Integer>0</Integer>
</value>
</entry>
<entry key="pagingSize">
<value>
<Integer>50</Integer>
</value>
</entry>
<entry key="parentEndpointName"/>
<entry key="resMappingObj">
<value>
<Map>
<entry key="access_token" value="access_token"/>
</Map>
</value>
</entry>
<entry key="responseCode">
<value>
<List>
<String>2**</String>
</List>
</value>
</entry>
<entry key="rootPath" value="$"/>
<entry key="sequenceNumberForEndpoint" value="1"/>
<entry key="uniqueNameForEndPoint" value="Get Access Token"/>
<entry key="xpathNamespaces"/>
</Map>
<Map>
<entry key="afterRule"/>
<entry key="beforeRule" value="******* Bind Members to Group"/>
<entry key="body">
<value>
<Map>
<entry key="bodyFormData"/>
<entry key="bodyFormat" value="raw"/>
<entry key="jsonBody" value="{
"emails": [
"$plan.nativeIdentity$"
]
}"/>
</Map>
</value>
</entry>
<entry key="contextUrl" value="/*******/namespace"/>
<entry key="curlCommand"/>
<entry key="curlEnabled">
<value>
<Boolean></Boolean>
</value>
</entry>
<entry key="customAuthUrl"/>
<entry key="header">
<value>
<Map>
<entry key="Accept" value="application/json"/>
<entry key="Authorization" value="Bearer $application.access_token$"/>
<entry key="x-api-key" value="*******"/>
</Map>
</value>
</entry>
<entry key="httpMethodType" value="PUT"/>
<entry key="operationType" value="Add Entitlement"/>
<entry key="paginationSteps"/>
<entry key="pagingInitialOffset">
<value>
<Integer>0</Integer>
</value>
</entry>
<entry key="pagingSize">
<value>
<Integer>50</Integer>
</value>
</entry>
<entry key="parentEndpointName"/>
<entry key="resMappingObj">
<value>
<Map>
<entry key="email" value="emails[*]"/>
<entry key="groups" value="groups[*]"/>
</Map>
</value>
</entry>
<entry key="responseCode">
<value>
<List>
<String>200</String>
</List>
</value>
</entry>
<entry key="rootPath" value="$"/>
<entry key="sequenceNumberForEndpoint" value="2"/>
<entry key="uniqueNameForEndPoint"/>
<entry key="xpathNamespaces"/>
</Map>
<Map>
<entry key="afterRule"/>
<entry key="beforeRule"/>
<entry key="body">
<value>
<Map>
<entry key="bodyFormData"/>
<entry key="bodyFormat" value="raw"/>
<entry key="jsonBody"/>
</Map>
</value>
</entry>
<entry key="contextUrl" value="/Organisations"/>
<entry key="curlCommand"/>
<entry key="curlEnabled">
<value>
<Boolean></Boolean>
</value>
</entry>
<entry key="customAuthUrl"/>
<entry key="header">
<value>
<Map>
<entry key="Accept" value="application/json"/>
<entry key="Authorization" value="Bearer $application.access_token$"/>
<entry key="x-api-key" value="*******"/>
</Map>
</value>
</entry>
<entry key="httpMethodType" value="GET"/>
<entry key="operationType" value="Test Connection"/>
<entry key="paginationSteps"/>
<entry key="pagingInitialOffset">
<value>
<Integer>0</Integer>
</value>
</entry>
<entry key="pagingSize">
<value>
<Integer>50</Integer>
</value>
</entry>
<entry key="parentEndpointName"/>
<entry key="resMappingObj"/>
<entry key="responseCode">
<value>
<List>
<String>2**</String>
</List>
</value>
</entry>
<entry key="rootPath"/>
<entry key="sequenceNumberForEndpoint" value="3"/>
<entry key="uniqueNameForEndPoint"/>
<entry key="xpathNamespaces"/>
</Map>
<Map>
<entry key="afterRule"/>
<entry key="beforeRule"/>
<entry key="body">
<value>
<Map>
<entry key="bodyFormData"/>
<entry key="bodyFormat" value="raw"/>
<entry key="jsonBody"/>
</Map>
</value>
</entry>
<entry key="contextUrl" value="/*******/namespace/*******/groups?size=$limit$"/>
<entry key="curlCommand"/>
<entry key="curlEnabled">
<value>
<Boolean></Boolean>
</value>
</entry>
<entry key="customAuthUrl"/>
<entry key="header">
<value>
<Map>
<entry key="Accept" value="application/json"/>
<entry key="Authorization" value="Bearer $application.access_token$"/>
<entry key="x-api-key" value="*******"/>
</Map>
</value>
</entry>
<entry key="httpMethodType" value="GET"/>
<entry key="operationType" value="Group Aggregation"/>
<entry key="paginationSteps" value="TERMINATE_IF $response.nextLink$ == NULL
$endpoint.fullUrl$ = $application.baseUrl$ + "/*******/namespace/*******/groups?nextLink=" + $response.nextLink$"/>
<entry key="pagingInitialOffset">
<value>
<Integer>0</Integer>
</value>
</entry>
<entry key="pagingSize">
<value>
<Integer>10</Integer>
</value>
</entry>
<entry key="parentEndpointName"/>
<entry key="resMappingObj">
<value>
<Map>
<entry key="group" value="grn"/>
<entry key="groupName" value="name"/>
<entry key="id" value="id"/>
</Map>
</value>
</entry>
<entry key="responseCode">
<value>
<List>
<String>2**</String>
</List>
</value>
</entry>
<entry key="rootPath" value="$.groups[*]"/>
<entry key="sequenceNumberForEndpoint" value="4"/>
<entry key="uniqueNameForEndPoint"/>
<entry key="xpathNamespaces"/>
</Map>
<Map>
<entry key="afterRule"/>
<entry key="beforeRule" value="******* Unbind Members from Group"/>
<entry key="body">
<value>
<Map>
<entry key="bodyFormData"/>
<entry key="bodyFormat" value="raw"/>
<entry key="jsonBody"/>
</Map>
</value>
</entry>
<entry key="contextUrl" value="/*******/namespace"/>
<entry key="curlCommand"/>
<entry key="curlEnabled">
<value>
<Boolean></Boolean>
</value>
</entry>
<entry key="customAuthUrl"/>
<entry key="header">
<value>
<Map>
<entry key="Accept" value="application/json"/>
<entry key="Authorization" value="Bearer $application.access_token$"/>
<entry key="x-api-key" value="*******"/>
</Map>
</value>
</entry>
<entry key="httpMethodType" value="DELETE"/>
<entry key="operationType" value="Remove Entitlement"/>
<entry key="paginationSteps"/>
<entry key="pagingInitialOffset">
<value>
<Integer>0</Integer>
</value>
</entry>
<entry key="pagingSize">
<value>
<Integer>50</Integer>
</value>
</entry>
<entry key="parentEndpointName"/>
<entry key="resMappingObj"/>
<entry key="responseCode">
<value>
<List>
<String>2**</String>
</List>
</value>
</entry>
<entry key="rootPath"/>
<entry key="sequenceNumberForEndpoint" value="5"/>
<entry key="uniqueNameForEndPoint"/>
<entry key="xpathNamespaces"/>
</Map>
<Map>
<entry key="afterRule"/>
<entry key="beforeRule"/>
<entry key="body">
<value>
<Map>
<entry key="bodyFormData"/>
<entry key="bodyFormat" value="raw"/>
<entry key="jsonBody"/>
</Map>
</value>
</entry>
<entry key="contextUrl" value="/*******/namespace/*******/groups/*******"/>
<entry key="curlCommand"/>
<entry key="curlEnabled">
<value>
<Boolean></Boolean>
</value>
</entry>
<entry key="customAuthUrl"/>
<entry key="header">
<value>
<Map>
<entry key="Accept" value="application/json"/>
<entry key="Authorization" value="Bearer $application.access_token$"/>
<entry key="x-api-key" value="*******"/>
</Map>
</value>
</entry>
<entry key="httpMethodType" value="GET"/>
<entry key="operationType" value="Account Aggregation"/>
<entry key="paginationSteps" value="TERMINATE_IF $response.nextLink$ == NULL
$endpoint.fullUrl$ = $application.baseUrl$ + "/*******/namespace/*******/groups/*******?size=100&nextLink=" + $response.nextLink$"/>
<entry key="pagingInitialOffset">
<value>
<Integer>0</Integer>
</value>
</entry>
<entry key="pagingSize">
<value>
<Integer>100</Integer>
</value>
</entry>
<entry key="parentEndpointName"/>
<entry key="resMappingObj">
<value>
<Map>
<entry key="accountType" value="accountType"/>
<entry key="description" value="description"/>
<entry key="displayName" value="displayName"/>
<entry key="email" value="email"/>
<entry key="firstName" value="firstName"/>
<entry key="id" value="id"/>
<entry key="lastName" value="lastName"/>
<entry key="status" value="status"/>
<entry key="upn" value="upn"/>
<entry key="userPrincipalName" value="userPrincipalName"/>
</Map>
</value>
</entry>
<entry key="responseCode">
<value>
<List>
<String>2**</String>
</List>
</value>
</entry>
<entry key="rootPath" value="$.users[*]"/>
<entry key="sequenceNumberForEndpoint" value="6"/>
<entry key="uniqueNameForEndPoint" value="Get List of Users in Namespace"/>
<entry key="xpathNamespaces"/>
</Map>
<Map>
<entry key="afterRule"/>
<entry key="beforeRule"/>
<entry key="body">
<value>
<Map>
<entry key="bodyFormData"/>
<entry key="bodyFormat" value="raw"/>
<entry key="jsonBody"/>
</Map>
</value>
</entry>
<entry key="contextUrl" value="/*******/users/$response.email$/memberOf"/>
<entry key="curlCommand"/>
<entry key="curlEnabled">
<value>
<Boolean></Boolean>
</value>
</entry>
<entry key="customAuthUrl"/>
<entry key="header">
<value>
<Map>
<entry key="Accept" value="application/json"/>
<entry key="Authorization" value="Bearer $application.access_token$"/>
<entry key="x-api-key" value="*******"/>
</Map>
</value>
</entry>
<entry key="httpMethodType" value="GET"/>
<entry key="operationType" value="Account Aggregation"/>
<entry key="paginationSteps"/>
<entry key="pagingInitialOffset">
<value>
<Integer>0</Integer>
</value>
</entry>
<entry key="pagingSize">
<value>
<Integer>10</Integer>
</value>
</entry>
<entry key="parentEndpointName" value="Get List of Users in Namespace"/>
<entry key="resMappingObj">
<value>
<Map>
<entry key="groupCreatedAt" value="createdAt"/>
<entry key="groupDescription" value="description"/>
<entry key="groupId" value="id"/>
<entry key="groupName" value="name"/>
</Map>
</value>
</entry>
<entry key="responseCode">
<value>
<List>
<String>2**</String>
</List>
</value>
</entry>
<entry key="rootPath" value="$.groups[*]"/>
<entry key="sequenceNumberForEndpoint" value="7"/>
<entry key="uniqueNameForEndPoint" value="Get Groups"/>
<entry key="xpathNamespaces"/>
</Map>
<Map>
<entry key="afterRule"/>
<entry key="beforeRule"/>
<entry key="body">
<value>
<Map>
<entry key="bodyFormData"/>
<entry key="bodyFormat" value="raw"/>
<entry key="jsonBody"/>
</Map>
</value>
</entry>
<entry key="contextUrl" value="/*******/namespace/*******/users"/>
<entry key="curlCommand"/>
<entry key="curlEnabled">
<value>
<Boolean></Boolean>
</value>
</entry>
<entry key="customAuthUrl"/>
<entry key="header"/>
<entry key="httpMethodType" value="POST"/>
<entry key="operationType" value="Create Account"/>
<entry key="paginationSteps"/>
<entry key="pagingInitialOffset">
<value>
<Integer>0</Integer>
</value>
</entry>
<entry key="pagingSize">
<value>
<Integer>50</Integer>
</value>
</entry>
<entry key="parentEndpointName"/>
<entry key="resMappingObj"/>
<entry key="rootPath"/>
<entry key="sequenceNumberForEndpoint" value="8"/>
<entry key="uniqueNameForEndPoint"/>
<entry key="xpathNamespaces"/>
</Map>
</List>
</value>
</entry>
<entry key="createAccountWithEntReq">
<value>
<Boolean></Boolean>
</value>
</entry>
<entry key="custom_auth_token_info" value=""/>
<entry key="enableHasMore" value="true"/>
<entry key="enableStatus" value="status=enabled"/>
<entry key="encrypted" value="accesstoken,refresh_token,oauth_token_info,client_secret,private_key,private_key_password,clientCertificate,clientKeySpec,resourceOwnerPassword,custom_auth_token_info"/>
<entry key="fixedPlanMultivaluedAttribute" value="true"/>
<entry key="genericWebServiceBaseUrl" value="*******"/>
<entry key="httpCookieSpecsStandard" value="true"/>
<entry key="isGetObjectRequiredForPTA">
<value>
<Boolean>true</Boolean>
</value>
</entry>
<entry key="lastAggregationDate_account" value="2024-12-04T04:52:17Z"/>
<entry key="lastAggregationDate_group" value="2025-02-18T04:07:35Z"/>
<entry key="nativeChangeDetectionAttributeScope" value="entitlements"/>
<entry key="nativeChangeDetectionAttributes"/>
<entry key="nativeChangeDetectionEnabled">
<value>
<Boolean></Boolean>
</value>
</entry>
<entry key="nativeChangeDetectionOperations"/>
<entry key="oAuthJwtHeader">
<value>
<Map>
<entry key="alg" value="RS256"/>
</Map>
</value>
</entry>
<entry key="oAuthJwtPayload">
<value>
<Map>
<entry key="aud"/>
<entry key="exp" value="15f"/>
<entry key="iss"/>
<entry key="sub"/>
</Map>
</value>
</entry>
<entry key="private_key_to_update"/>
<entry key="saml_assertion_url"/>
<entry key="saml_request_body"/>
<entry key="sysDescriptions">
<value>
<Map>
<entry key="en_GB"/>
<entry key="en_US"/>
</Map>
</value>
</entry>
<entry key="templateApplication" value="Web Services"/>
<entry key="throwProvAfterRuleException">
<value>
<Boolean>true</Boolean>
</value>
</entry>
<entry key="throwProvBeforeRuleException">
<value>
<Boolean>true</Boolean>
</value>
</entry>
<entry key="timeoutInSeconds"/>
<entry key="version" value="v2"/>
</Map>
</Attributes>
<CorrelationRule>
<Reference class="sailpoint.object.Rule" id="0af4038394811cd58194826b43060ae2" name="*******"/>
</CorrelationRule>
<Owner>
<Reference class="sailpoint.object.Identity" id="0af402c190dc19c38190de5d45112287" name="*******"/>
</Owner>
<Schemas>
<Schema created="1739798447082" displayAttribute="displayName" groupAttribute="" id="0af4057b951411418195141183ea0046" identityAttribute="email" instanceAttribute="" modified="1739852392618" nativeObjectType="email" objectType="account" significantModified="1739852392618">
<AttributeDefinition name="accountType" type="string">
<Description></Description>
</AttributeDefinition>
<AttributeDefinition name="displayName" type="string">
<Description></Description>
</AttributeDefinition>
<AttributeDefinition name="email" type="string">
<Description></Description>
</AttributeDefinition>
<AttributeDefinition name="firstName" type="string">
<Description></Description>
</AttributeDefinition>
<AttributeDefinition name="id" type="string">
<Description></Description>
</AttributeDefinition>
<AttributeDefinition name="lastName" type="string">
<Description></Description>
</AttributeDefinition>
<AttributeDefinition name="status" type="string">
<Description></Description>
</AttributeDefinition>
<AttributeDefinition name="upn" type="string">
<Description></Description>
</AttributeDefinition>
<AttributeDefinition name="userPrincipalName" type="string">
<Description></Description>
</AttributeDefinition>
<AttributeDefinition entitlement="true" multi="true" name="groupName" schemaObjectType="group" type="string">
<Description></Description>
</AttributeDefinition>
</Schema>
<Schema created="1739798447082" descriptionAttribute="" displayAttribute="group" id="0af4057b951411418195141183ea0047" identityAttribute="group" instanceAttribute="" modified="1739852392618" nativeObjectType="group" objectType="group" significantModified="1739852392618">
<AttributeDefinition entitlement="true" multi="true" name="group" type="string">
<Description></Description>
</AttributeDefinition>
</Schema>
</Schemas>
<ApplicationScorecard created="1739798447082" id="0af4057b951411418195141183ea0048" modified="1739852392618" significantModified="1739852392618"/>
</Application>
Arpitha1
February 18, 2025, 7:29am
11
As per xml, in the first iteration, the URL shouldn’t have nextLink, and in the second iteration, it shouldn’t have size, yet the error message includes both. So I’m just confused how’s that.
On another note, I see you are not appending size in the pagination tab for full url. Correct it and trigger task again
shijingg
February 18, 2025, 9:22am
12
Hi @Arpitha1
Added for both context url and paging steps, still same error.
May I check do I need to define what is $limit$ anywhere?
Arpitha1
February 18, 2025, 9:32am
13
Update URL step pagination tab, something as below.
"size=" + $limit$ + "nextlink=" + $response.nextLink$
shijingg
February 18, 2025, 9:37am
14
Hi @Arpitha1
Now it is not terminating.
This is the result when trying with page size = 10. I don’t have the latest screenshot, environment is down.
Arpitha1
February 18, 2025, 9:51am
16
As per document, NULL would work for both null and empty values.
However, you can also try with empty value check as below.TERMINATE_IF $response.nextLink$ == ""
If that doesn’t work, then try with below line
TERMINATE_IF $RECORDS_COUNT$ < $limit$
shijingg
February 18, 2025, 10:04am
17
Tested both, same endless looping behaviour
Terminate IF using == “”
Record count method
Arpitha1
February 18, 2025, 10:34am
18
Can you try printing requestEndPoint in the before operation rule and check the logs? By examining the URL value, we can identify where the loop is occurring.
shijingg
February 19, 2025, 5:33am
19
Hi @Arpitha1 ,
I captured all these for now, seems to be looping endlessly and the nextlink are all different values.
Arpitha1
February 19, 2025, 5:52am
20
Since nextLink value is not getting repeated, we can’t say it’s looping.
Can you try to replicate this in postman, try coping exact context URLs and observe the behaviour
shijingg
February 19, 2025, 8:39am
21
I tested using curl, the number of nextlink is correct. meaning it is the way I paginate in sailpoint that is having an issue
