Webservices Aggregation Pagination not working properly

IdentityIQ (IIQ) IIQ Discussion and Questions
1

Which IIQ version are you inquiring about?

[8.4p2]

I am working on a webservices connector . For aggregation I have configured the pagination steps in paging step .

Paging steps :-

TERMINATE_IF $response.itemsPerPage$ == 0
$offset$ = $offset$ + $limit$
TERMINATE_IF $offset$ > $response.totalResults$
$endpoint.fullUrl$ = $application.baseUrl$+“/v2/users?startIndex=”+$offset$+“&count=”+$limit$

→ In logs I am seeing that the url gets updated correctly but again it get reset to startIndex=1 , creating an loop and aggregating same accounts again and again.

Additional information

Initial offset = 1
pageSize/count = 1000 ($limit$)

[2026-11-06T07:21:55,299 DEBUG QuartzScheduler_Worker-1 connector.webservices.paging.WebServicesPaginator:80 - PAGING: Attempting to evaluate paging step: $endpoint.fullUrl$ = $application.baseUrl$+“/v2/users?startIndex=”+$offset$+“&count=”+$limit$
2026-11-06T07:21:55,299 DEBUG QuartzScheduler_Worker-1 connector.webservices.paging.WebServicesJsonVariableResolver:51 - PAGING: Attempting to resolve: $offset$
2026-11-06T07:21:55,299 DEBUG QuartzScheduler_Worker-1 connector.webservices.paging.WebServicesJsonVariableResolver:51 - PAGING: Attempting to resolve: $limit$
2026-11-06T07:21:55,299 DEBUG QuartzScheduler_Worker-1 connector.webservices.v2.RequestOrchestratorV2:303 - ==> Calculating data hash value for endpoint: Account_Aggregation
2026-11-06T07:21:55,304 DEBUG QuartzScheduler_Worker-1 connector.webservices.v2.RequestOrchestratorV2:324 - ==> Paging will continue for endpoint: Account_Aggregation
2026-11-06T07:21:55,304 DEBUG QuartzScheduler_Worker-1 connector.webservices.v2.RequestOrchestratorV2:343 - ==> Paging logic processing ended for endpoint ‘‘Account_Aggregation’’. Paging terminated: false
2026-11-06T07:21:57,462 DEBUG QuartzScheduler_Worker-1 connector.webservices.v2.AbstractHttpRequestBuilder:133 - URL after replacing place holders :https://euiopq.scim.know.com/v2/users?startIndex=1001&count=1000
2026-11-06T07:21:57,462 DEBUG QuartzScheduler_Worker-1 connector.webservices.v2.RequestOrchestratorV2:237 - Request after updating V1 endpoint related paging information: HttpRequestWrapper [url=https://euiopq.scim.know.com/v2/users?startIndex=1&count=1000, headers={Authorization=***, Accept=application/scim+json, Content-Type=application/scim+json}, payload=null, type=GET, allowedSuccessCodes=[2**], possibleHttpErrorCodes=, possibleHttpErrorMessages=]

app.xml (16.2 KB)

2

Can you share application XML? Seems like something else in your app config might overwrite pagination settings.

3

Hey I have attached the app xml above.

4

Try these pagination set up:

TERMINATE_IF $RECORDS_COUNT$ < $limit$
$offset$ = $offset$ + $limit$
$endpoint.fullUrl$ = $application.baseUrl$ + "/v2/users?startIndex=" + $offset$ + "&count=" + $limit$
5

Got this error

Exception during aggregation. Reason: java.lang.RuntimeException: sailpoint.connector.ConnectorException: Url: null, Message: 0 : sailpoint.connector.ConnectorException: Final URL can not be null, HTTP Error Code: 0

6

Log for reference:-
2026-11-06T11:38:59,198 DEBUG Thread-2412 connector.webservices.v2.AbstractHttpRequestBuilder:133 - URL after replacing place holders :https://euiopq.scim.know.com/v2/users?startIndex=1001&count=1000
2026-11-06T11:38:59,198 DEBUG Thread-2412 connector.webservices.v2.RequestOrchestratorV2:237 - Request after updating V1 endpoint related paging information: HttpRequestWrapper [url=null, headers={Authorization=***, Accept=application/scim+json, Content-Type=application/scim+json}, payload=null, type=GET, allowedSuccessCodes=[2**], possibleHttpErrorCodes=[], possibleHttpErrorMessages=[]]
2026-11-06T11:38:59,198 ERROR Thread-2412 connector.webservices.v2.WebServiceFacadeV2:2083 - Exception occurred for account aggregation: Url: null, Message: 0 : sailpoint.connector.ConnectorException: Final URL can not be null, HTTP Error Code: 0

7

Hi,
It seems to be executing now , I had turned on the partitioning.
Quick thing , I have now configured the partitioning also in webservices but again this error popped up I can provide the updated logs and also the getPartitions and Partitioned account aggregation config too.

8

Do you have set

   <entry key="genericWebServiceBaseUrl" value=""/>
9

Hi ,
Yes this is populated in the app xml in the environment

10

Updated app xml with partitioning.
FYI the list used to create partitions from getPartitions op is created in the after op rule so you can ignore that operation.

updated_app.xml (22.7 KB)

Updated logs:-
2026-11-06T12:21:46,356 DEBUG Thread-2972 connector.webservices.v2.AbstractHttpRequestBuilder:133 - URL after replacing place holders :https://euiopq.scim.know.com/v2/users?startIndex=1001&count=1000&filter=(userName+sw+“A”+or+userName+sw+“a”+or+userName+sw+“J”+or+userName+sw+“j”+or+userName+sw+“S”+or+userName+sw+“s”)+and+(urn%3Aietf%3Aparams%3Ascim%3Aschemas%3Aextension%3Aknowbe4%3Akmsat%3A2.0%3AUser%3AcustomField1+eq+“Employee”+or+urn%3Aietf%3Aparams%3Ascim%3Aschemas%3Aextension%3Aknowbe4%3Akmsat%3A2.0%3AUser%3AcustomField1+eq+“employee”) 2026-11-06T12:21:46,356 DEBUG Thread-2972 connector.webservices.v2.RequestOrchestratorV2:237 - Request after updating V1 endpoint related paging information: HttpRequestWrapper [url=null, headers={Authorization=***, Accept=application/scim+json, Content-Type=application/scim+json}, payload=null, type=GET, allowedSuccessCodes=[2], possibleHttpErrorCodes=, possibleHttpErrorMessages=] 2026-11-06T12:21:46,357 ERROR Thread-2972 connector.webservices.v2.WebServiceFacadeV2:2083 - Exception occurred for account aggregation: Url: null, Message: 0 : sailpoint.connector.ConnectorException: Final URL can not be null, HTTP Error Code: 0

11

If I understand correctly it works in non partitioned mode, but fails with partitioning?

12

Yes that is a correct statement

13

Hi @PrathamChoreria,

When you tried @lmoren’s steps earlier with partitioning on, you got the same url=null, and that step has no partition token at all. So I don’t think the filter is what’s breaking it. It looks more like the fullUrl paging step itself doesn’t resolve inside the partition threads, most likely $application.baseUrl$.

Even so, I’d keep $partition.filter$ out of the paging steps. It’s only documented for the Context URL, request body and before/after rules (Partitioning), not as a paging keyword.

I’d try moving the filter into the Context URL and dropping the fullUrl line completely:

/v2/users?startIndex=$offset$&count=$limit$&filter=$partition.filter$


TERMINATE_IF $RECORDS_COUNT$ < $limit$
$offset$ = $offset$ + $limit$

Your log already shows the Context URL being rebuilt with the new offset, so this may be all you need. Check the account count after the run, though. If the connector doesn’t rebuild the URL each page, it won’t throw an error. It’ll just fetch page 1 again and finish cleanly.

If it still errors or the count comes up short, post the PAGING: debug lines from one partition thread (page 1 → page 2) and we can dig in from there.