# /v3/search results not accurate with limit and offset if dont specify sort criteria

**URL:** <https://developer.sailpoint.com/discuss/t/v3-search-results-not-accurate-with-limit-and-offset-if-dont-specify-sort-criteria/99527>\
**Category:** Bugs\
**Tags:** not-a-bug, identity-security-cloud, apis\
**Created:** [January 31, 2025, 12:00am UTC](https://developer.sailpoint.com/discuss/t/v3-search-results-not-accurate-with-limit-and-offset-if-dont-specify-sort-criteria/99527 "2025-01-31T00:00:50Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jrossicare](https://avatars.discourse-cdn.com/v4/letter/j/a698b9/32.png) [@jrossicare](https://developer.sailpoint.com/discuss/u/jrossicare)\
**Post date:** [January 31, 2025, 12:00am UTC](https://developer.sailpoint.com/discuss/t/v3-search-results-not-accurate-with-limit-and-offset-if-dont-specify-sort-criteria/99527/1 "2025-01-31T00:00:50Z")

</div>

# What problem are you observing?

Is this expected behavior?  
Using /v3/search with offset and limit, you _need_ to specify sort, otherwise results are not accurate. There are duplicates and missed results. See example below

# What is the correct behavior?

I would like search results to be accurate without having to specify sort criteria. It makes me lose trust in search results.

# What product feature is this related to?

ISC Search API. I only tried v3, not v2024 or beta etc

# What are the steps to reproduce the issue?

Using a script (say ruby) compare the search results of the following.  
In my example, getting all identities that have an account on source X

Without sort, I had duplicates in the results. and after removing duplicates I had _less_ rows than I had with sort option (which had no duplicates, as expected)

```ruby
  objList = Array.new
  offset = 0
  count = 250
  # This is payload without sort
  payload = "{\"indices\":[\"identities\"],,\"query\":{\"query\":\"@accounts(source.id:#{mySource})\"}}"
  while count == 250
    response = IDNAPI.post_json("#{$config['baseUrl']}/v3/search?limit=250&offset=#{offset}", $config['access_token'],payload)
    responseBody = JSON.parse( response.body )
    count = responseBody.length
    objList = objList.concat(responseBody)
    offset = offset + count
    puts "Count: #{count}, offset #{offset}, length #{objList.length()}"
  end
  puts "identitiesList size: #{objList.length}"
  objList.each do | identity |
    writeToFile(identity)
  end

```

With sort:

```ruby
  objList = Array.new
  offset = 0
  count = 250
  # Note: Without the sort, it was not giving accurate returns, there were duplicates and missing entries
  payload = "{\"indices\":[\"identities\"],\"sort\": [\"+name\"],\"query\":{\"query\":\"@accounts(source.id:#{mySource})\"}}"
  while count == 250
    response = IDNAPI.post_json("#{$config['baseUrl']}/v3/search?limit=250&offset=#{offset}", $config['access_token'],payload)
    responseBody = JSON.parse( response.body )
    count = responseBody.length
    objList = objList.concat(responseBody)
    offset = offset + count
    puts "Count: #{count}, offset #{offset}, length #{objList.length()}"
  end
  puts "identitiesList size: #{objList.length}"
  objList.each do | identity |
    writeToFile(identity)
  end

```

# Do you have any other information about your environment that may help?

N/A

---

<div class="post-metadata">

**Author:** ![colin\_mckibben](https://sea1.discourse-cdn.com/sailpoint/discuss/user_avatar/developer.sailpoint.com/colin_mckibben/32/37182_2.png) [@colin\_mckibben](https://developer.sailpoint.com/discuss/u/colin_mckibben)\
**Post date:** [January 31, 2025, 1:32pm UTC](https://developer.sailpoint.com/discuss/t/v3-search-results-not-accurate-with-limit-and-offset-if-dont-specify-sort-criteria/99527/2 "2025-01-31T13:32:03Z")

</div>

Hi Jason,

The Search API has an upper limit of 10,000 records instead of 250, although the default is set to 250. Using offset pagination is not really necessary and will lead to more API calls than necessary. I recommend setting the limit higher, up to 10,000, to avoid having to paginate. If you need to paginate past 10,000 records, you will have to use `searchAfter`, which is documented here:

> **[Standard Collection Parameters | SailPoint Developer Community](https://developer.sailpoint.com/docs/api/v2024/api/v2024/standard-collection-parameters#paginating-search-queries)**
>
> ISC API pagination, filtering, and sorting.

All of this to say, this is not a bug and is working as intended. Search just behaves differently due to the technical differences of how it is implemented.

---

<div class="post-metadata">

**Author:** ![jrossicare](https://avatars.discourse-cdn.com/v4/letter/j/a698b9/32.png) [@jrossicare](https://developer.sailpoint.com/discuss/u/jrossicare)\
**Post date:** [February 2, 2025, 10:07pm UTC](https://developer.sailpoint.com/discuss/t/v3-search-results-not-accurate-with-limit-and-offset-if-dont-specify-sort-criteria/99527/3 "2025-02-02T22:07:30Z")

</div>

Hi Colin,  
I tried 10,000 and also 1,000, but the response size / HTTP content size is too large

If you have access, please see case CS0354934

Response if I try 10,000:

```auto
{
    "detailCode": "500.1.503 Downstream service unavailable",
    "trackingId": "f5a8f1039cb64e65a81bff07ac155329",
    "messages": [
        {
            "locale": "und",
            "localeOrigin": "REQUEST",
            "text": "A downstream resource was unavailable."
        },
        {
            "locale": "en-US",
            "localeOrigin": "DEFAULT",
            "text": "A downstream resource was unavailable."
        }
    ],
    "causes": []
}

```

---

<div class="post-metadata">

**Author:** ![colin\_mckibben](https://sea1.discourse-cdn.com/sailpoint/discuss/user_avatar/developer.sailpoint.com/colin_mckibben/32/37182_2.png) [@colin\_mckibben](https://developer.sailpoint.com/discuss/u/colin_mckibben)\
**Post date:** [February 3, 2025, 1:46pm UTC](https://developer.sailpoint.com/discuss/t/v3-search-results-not-accurate-with-limit-and-offset-if-dont-specify-sort-criteria/99527/4 "2025-02-03T13:46:51Z")

</div>

Try limiting the fields returned by your search query to reduce the response size. Please see this post for an example of how to do it.

> [@Workflow execution failed](https://developer.sailpoint.com/discuss/t/workflow-execution-failed/89157/2):
>
> Hi @JackSparrow Can you try adding a query result filter to limit the output size and see if it works for you. I know there is a limit for the allowed size in a WF and in a loop eg: { "indices": ["identities"], "query": { "fields": ["name"], "query": "attributes.uid:123" }, "queryResultFilter": { "includes": ["id", "name"] } }

---

<div class="post-metadata">

**Author:** ![jrossicare](https://avatars.discourse-cdn.com/v4/letter/j/a698b9/32.png) [@jrossicare](https://developer.sailpoint.com/discuss/u/jrossicare)\
**Post date:** [February 3, 2025, 9:29pm UTC](https://developer.sailpoint.com/discuss/t/v3-search-results-not-accurate-with-limit-and-offset-if-dont-specify-sort-criteria/99527/5 "2025-02-03T21:29:46Z")

</div>

Thanks Colin. I will try that.