I am trying to configure paging using the web services connector.
The url https://api.brivo.com/v1/api/users?offset=200&pageSize=100 gives results via postman.
In the aggregation operation of the web services connector, i am configuring paging as below
The api returns 20 records which is the default page size.
I have not configured the before/after rules. Are they required for paging configuration?
Pagination not required Before or After rule
Yes, Looks pretty standard. I have observed that the configured paging steps are not considered at all. Even if I mistyped the url in the paging steps, the aggregation still runs returning just 20 records
Hey @nileshnikalje,
Can you please let me know how many records are present in the system and does the API response provide any total records count in the system?
Thank you.
yes there are 5000+ records in the system. The response do send the info
"offset": 400,
"pageSize": 100,
"count": 5844
Can you try the below steps and let me know if it is working:
$pageSize$=100
TERMINATE_IF $response.count$ < $offset$
$offset$ = $offset$ + $pageSize$
$endpoint.fullUrl$ = $application.baseUrl$ + "/v1/api/users?pageSize=100&offset=" + $offset$
@nileshnikalje as given above :
Pagination not required Before or After rule.
This work for us :
In your case you your define pagination logic should work:`
$pageSize$=100
TERMINATE_IF $RECORDS_COUNT$ < $pageSize$
$offset$ = $offset$ + $pageSize$
$endpoint.fullUrl$ = $application.baseUrl$ + "/v1/api/users?pageSize=100&offset=" + $offset$
Yes. Thanks.It works. But what is the logic behind comparing if $response.count$ < $offset$. Shouldnt it be $response.count$ < $pageSize$?
What does $response.count$ represent? the number of records fetched in one request or the total records fetched till now?
Hey @nileshnikalje,
Page size is the response limit or the maximum number of records you can fetch per page. This will be fixed value so you can’t compare this with the total records that are present in your system. Therefore, you need to compare this with offset because for every API call you are incrementing the offset until it fetches the last record in the system.
That $response.count$ is the total number of records present in the application, and you are getting that value in your API response, so we have used that for our comparison to fetch all the records until the offset is more than your response count.
Hope this clarifies your query.
Thank you.
Shanmukh
Yes. it should have worked. However the solution posted by @Shanmukh worked.
Thanks for the clarification. I was asking for $RECORDS_COUNT$.
This standard variable is used in the documentation and it would represent the number of records fetched in the current request.
The issue with my config was that in the context url of the operation I didnt mentioned the offset and pagesize. Therefore it was returning 20 records as default for the first request. As 20 < pagesize or offset, it would terminate.
Fixing the context url to fetch the first page passing the pagesize and offset would guarantee returning 100 records as per pageSize. Then the comparision of $RECORDS_COUNT$ < pagesize would make sense.
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.