Enhancements: Updates to API Paging Limitations

Announcements Product News
, ,
1

In order to better serve our customers, SailPoint will begin enforcing a page limit of 250 for all API calls in 30 days time. Any calls to our pre-V3 APIs may be out of compliance.

Please review your integration code and make the necessary changes by Monday, June 13th for Sandbox and Monday, June 27th for Production tenants. Our V3 APIs already include this functionality, so we encourage you to update to the latest version. You can find a list of our V3 APIs here V3 APIs | SailPoint Developer Community and documentation on API pagination here Standard Collection Parameters | SailPoint Developer Community.

If you have any additional questions or concerns about making this change please reach out to your SailPoint contact or our Support team.

Post edited to include the following update:

On Monday, May 16th, the IdentityNow Team posted an announcement to customers via multiple channels that, beginning in 30 days, IdentityNow will being rolling out an update to Pre-V3 APIs to limit results to 250. We have fielded a few questions coming from customers on this announcement. Below is an FAQ to help provide context and explanation for this change.

FAQ

  • What is the change and what is impacted?
    • For Pre-V3 APIs (aka V1 and V2 APIs) only, this change will enforce a maximum value of 250 for the “limit” parameter. After this change, API calls with a limit parameter greater than 250 will ERROR. API calls with a limit set equal to or less than this maximum value will continue to function.
    • This change impacts Pre-V3 (aka V1 and V2 APIs) only. Please note that /cc APIs are considered V1 and ARE impacted by this change.
    • This change DOES NOT impact any Beta or V3 APIs. Beta and V3 APIs have been designed with the appropriate limitations in mind, where necessary. For specific limits on each V3 and Beta API, please refer to the posted documentation for our V3 API and Beta API specifications. If a limit is NOT explicitly stated in the specification, developers should assume the limit is 250 .
    • This change DOES NOT impact the IdentityNow UI and any functionality therein.
  • Are utilities provided by SailPoint Professional Services impacted by this change?
    • Yes, for any utilities provided by SailPoint Professional Services calling V1 or V2 APIs with a limit set >250, those API calls will begin to receive an ERROR response. Customers will need to update the utility to adhere to the max 250 item limit. As a reminder, tools provided by SailPoint Professional Services are always considered custom work and provided “as is.” Additional assistance can be requested from SailPoint Professional Services either through an ongoing IdentityNow project or by using Expert Services hours.
  • But what if I need more than 250 items returned by the API?
    • Pagination controls are standard and can be used to request multiple calls of max 250 until the entire desired data set has been returned. Read more on the use of pagination here.
  • Why is this change being made?
    • These APIs are handled by shared services within IdentityNow and the resources required to generate large responses has a negative performance impact. This change is being made to enforce best practices and ensure expected system performance for all SailPoint customers.
  • Who is impacted?
    • As part of the research on this issue, the team has been monitoring API activity across IdentityNow.
    • Over the prior 30 days, >50 IdentityNow tenants (either production or sandbox) called a V1/V2 API in excess of this 250 item limit. Of the >50 tenants, most had a very low count of over-limit API calls during this window.
    • The vast majority of these over-limit V1/V2 API calls were made by only a few customers. These customers were directly contacted by our Support team ahead of the announcement to assist them in making the requested necessary changes quickly.
    • Based on this research, it was determined that the vast majority of IdentityNow customers would not be impacted by this change.
  • When will these changes be rolled out?
    • Monday, June 13, 2022 - Sandbox tenants will be enabled with the new behavior for V1 and V2 APIs. Customers can use this time to test any concerns.
    • Monday, June 27, 2022 - Production tenants will BEGIN to be enabled with the new behavior for V1 and V2 APIs. Production tenants will be enabled in batches.

Thank you all for your questions! Please continue asking us here, or if you have any additional questions or concerns about making this change please reach out to your SailPoint contact or our Support Team.

2

hi Rachael,
Is this for all APIs? I have a API call to external application endpoint which collects 500 items, will that be restricted too?

3

Hello @ranjanik,

I would like to better understand your question.

What SailPoint API endpoint are you calling?

If you are calling a SailPoint API, the limit of results per page is now 250. In order to get 500 results you will need to call the API twice using the limit and offset parameters.

For example, looking at our public identities API:

To get the first 250 results you’ll call the endpoint like so. This will return the results 0-249

https://{{tenant}}.api.identitynow.com/v3/public-identities

To get the next 250 results you’ll call the same API with the offset of 250. This will return results 250-499

https://{{tenant}}.api.identitynow.com/v3/public-identities?offset=250

If you are referring calling an external url in your code not related to SailPoint APIs. We do not limit the result set coming back for non-SailPoint APIs.

4

Hi, @ranjanik -

Yes, the max 250 paging limit will now be enforced across all SailPoint APIs.

The V3 version of our APIs enforces this limit out of the box, but not all of the older versions of our APIs have enforced this limit before now.

We will begin enforcing these limits in 30 days. We’re asking that any necessary changes be made by Monday, June 13th for Sandbox and Monday, June 27th for Production tenants.

5

ok I think my question is answered. If any of the sailpoint APIs (v3,beta and others) are being called, they will be limited to 250, you have to paginate if you have more than 250 items.

Thank you
Ranjani

1 Like
7

The Identity IO utility is only retrieving the max of 250 Roles and Access Profiles even though the tenant has hundreds more.

1 Like
8

Hello @shaun_wallace,

Does this positively impact APIs whose defaults are less than 50?

No, if there was a previous limit set for the API it will remain the same.

Does this impact SailPoint IdentityNow UI pages? Will the page still show all sources?

The UI will not change, it is currently setup for pagination. If you are calling the API /cc/api/source that will now enforce the limit of 250.

Does this impact SailPoint Professional Services utilities?

Yes, it will. When this change is implemented any calls exceeding the 250 limit will return an error. This includes any utilities using the APIs. As a reminder, tools provided by SailPoint Services are always considered custom work and provided “as is.”

9

Hi @colin_mckibben,

You indicate that V3 already enforces this limits, however I use the V3 search API call with a 5000 record limit that does return 5000 records:

[org].api.identitynow.com/v3/search?limit=5000

 So I hope this is the expected behaviour and the it will be left as it is; could you confirm?

Thanks,
Gustavo

10

@gsanr - Great question! And yes, this is expected behavior.

The V3 Search API does work slightly differently than the other V3 APIs because it simply acts as a passthrough to Elasticsearch. So in this specific V3 API, SailPoint will support the settings supported by Elasticsearch.

This behavior will not change as result of this announcement, which is pertinent only for pre-V3 APIs.

For additional information on what is supported by Elasticsearch, see here: Paginate search results | Elasticsearch Guide [7.17] | Elastic

1 Like
11

Thanks for confirming Rachel; would you mind requesting the v3/search doc to be updated to reflect this?

Gustavo

1 Like
12

Yes, of course! We’ll get these updated. Again. Great question - thank you!

13

Hello @colin_mckibben,

just so I understand this correctly - extending the limit past 250 by specifying the limit parameter will no longer be an option across the board (all V1/V2/V3? endpoints), correct? And is the V3 search the only exception to this rule or are there other endpoints where the limit can still be extended?

Thanks in advance for the clarification.

Marek

14

Hello @Prooxie,

This change to limit results to 250 is specific to pre-V3 APIs (V1 and V2). The Beta and V3 APIs have been designed with this already in mind. To get specific limits for each API refer to our API specs at V3 APIs | SailPoint Developer Community and Beta APIs | SailPoint Developer Community. If a limit is NOT stated you can assume that it is 250.

15

Hi @tyler_mairose,

Understood, so the only endpoint that will still accept a value higher than 250 in the limit parameter is the V3 search then? And if not, what other exceptions are there with a limit still extendable past the 250? Also, I assume it will still be possible to specify a limit lower than the 250 for the old endpoints?

Thanks again.

Marek

16

On Monday, May 16th, the IdentityNow Team posted an announcement to customers via multiple channels that, beginning in 30 days, IdentityNow will being rolling out an update to Pre-V3 APIs to limit results to 250. We have fielded a few questions coming from customers on this announcement. Below is an FAQ to help provide context and explanation for this change.

FAQ

  • What is the change and what is impacted?
    • For Pre-V3 APIs (aka V1 and V2 APIs) only, this change will enforce a maximum value of 250 for the “limit” parameter. After this change, API calls with a limit parameter greater than 250 will ERROR. API calls with a limit set equal to or less than this maximum value will continue to function.
    • This change impacts Pre-V3 (aka V1 and V2 APIs) only. Please note that /cc APIs are considered V1 and ARE impacted by this change.
    • This change DOES NOT impact any Beta or V3 APIs. Beta and V3 APIs have been designed with the appropriate limitations in mind, where necessary. For specific limits on each V3 and Beta API, please refer to the posted documentation for our V3 API and Beta API specifications. If a limit is NOT explicitly stated in the specification, developers should assume the limit is 250.
    • This change DOES NOT impact the IdentityNow UI and any functionality therein.
  • Are utilities provided by SailPoint Professional Services impacted by this change?
    • Yes, for any utilities provided by SailPoint Professional Services calling V1 or V2 APIs with a limit set >250, those API calls will begin to receive an ERROR response. Customers will need to update the utility to adhere to the max 250 item limit. As a reminder, tools provided by SailPoint Professional Services are always considered custom work and provided “as is.” Additional assistance can be requested from SailPoint Professional Services either through an ongoing IdentityNow project or by using Expert Services hours.
  • But what if I need more than 250 items returned by the API?
    • Pagination controls are standard and can be used to request multiple calls of max 250 until the entire desired data set has been returned. Read more on the use of pagination here.
  • Why is this change being made?
    • These APIs are handled by shared services within IdentityNow and the resources required to generate large responses has a negative performance impact. This change is being made to enforce best practices and ensure expected system performance for all SailPoint customers.
  • Who is impacted?
    • As part of the research on this issue, the team has been monitoring API activity across IdentityNow.
    • Over the prior 30 days, >50 IdentityNow tenants (either production or sandbox) called a V1/V2 API in excess of this 250 item limit. Of the >50 tenants, most had a very low count of over-limit API calls during this window.
    • The vast majority of these over-limit V1/V2 API calls were made by only a few customers. These customers were directly contacted by our Support team ahead of the announcement to assist them in making the requested necessary changes quickly.
    • Based on this research, it was determined that the vast majority of IdentityNow customers would not be impacted by this change.
  • When will these changes be rolled out?
    • Monday, June 13, 2022 - Sandbox tenants will be enabled with the new behavior for V1 and V2 APIs. Customers can use this time to test any concerns.
    • Monday, June 27, 2022 - Production tenants will BEGIN to be enabled with the new behavior for V1 and V2 APIs. Production tenants will be enabled in batches.

Thank you all for your questions! Please continue asking us here, or if you have any additional questions or concerns about making this change please reach out to your SailPoint contact or our Support Team.

17

So if we have 5000 access profiles, we have to redo the query 20 times? Every single time we do this for audit?

SailPoint didn’t provide adequate reports for public audits and now we can’t run our own.

How do we get this fixed to be able to run one single query? Because repeatedly running 20+ queries is not an acceptable solution to the lack of adequate reporting this system has.

18

Welcome to the developer community @julieldsp . Search is the preferred way to do reporting in IdentityNow since it doesn’t have the paging limits. Check out the screenshot below to see how to get all access profiles in your tenant, which you can then download as a CSV file. You can even save a search and schedule it to automatically email you the report when you want.

image
image2555Ă—752 86.4 KB

19

Thanks for that info and I pulled a report like that to see what data exported. However, that doesn’t list the governance group or manager for the access profile. We need to export all that detail in one query.

I also couldn’t get one report that exports Roles with Access Profiles with Entitlements in a single report via the search. (Every time we have to export multiple reports and put them together is one more thing to prove completeness and accuracy on to the auditors.)

Can these reports be performed inside the SailPoint system?
We were told to use Postman to get these audit reports before but this change breaks all those.

1 Like
20

Hi Julie,

Looks like the limit can be circumvented by using the v3/search api endpoint instead. So a basic access profiles extract can still be obtained via a POST request to https://{tenant}.api.identitynow.com/v3/search with the request below:

{
  "indices": [
    "accessprofiles"
  ],
  "query": {
    "query": "*"
  }
}
21

The search endpoint is a little limited in terms of what you can see about access profiles - if you need a report that shows you who approves the requests etc, search won’t be very helpful.

@julieldsp are you familiar at all with python? I’ve written a script to get all applications, access profiles, corresponding approval configurations and governance group members. Auditors were also curious but mostly the use case is so that users can check who they need to chase to expedite governance group approvals instead of coming to our team.
The current output is in json since the data goes to a confluence page but with some fiddling around you could get it to CSV. Happy to share if you think this could be helpful at all for your use case.

4 Likes