Powershell, offset and sort=asc

Identity Security Cloud (ISC) ISC Discussion and Questions
1

Hi,
Hoping that someone can point out what I am doing wrong with the following snippet. The code works, but observationally the API is doing the offset THEN sorting the results. This means that it brings back the same results occasionally. I’ve added code to fix that problem, but whilst I like fudge, I’d rather not have it in my code. How do I structure the API call so that it sorts the results before setting the offset - is that even possible?

    # --- Step 2: Get all Identities and their Assigned Access (Roles and Access Profiles) ---
    Write-Host "Fetching all Identities and their assigned Access..."
    $identityQueryBody    = @{
        indices           = @("identities")
        query             = @{
            query         = "attributes.cloudLifecycleState:active"
            sort          = @(    
                @{field   = "id"; order = "asc" }
            )
        }
        offset                = 0
        limit                 = 50
    }

    $identityQueryBody.offset = 0

    while ($continueIdentities) {

        try {
            $identityResponse = Invoke-RestMethod -Uri "https://$tenant.api.identitynow.com/v3/search?offset=$($identityQueryBody.offset)&limit=$($identityQueryBody.limit)" -Method 'POST' -Headers $headers -Body ($identityQueryBody | ConvertTo-Json -Depth 10) -ErrorAction Stop
        }
        .
        .
        .
2

To prevent this, you should do use ‘searchAfter’:

3

Hi,
Its a nice idea, but that assumes that the order that the ‘search’ api brings the items back is consistent, and if it’s consistent, then my code won’t fail.

I run my code everyday, and each time the results come back in a different order.

4

That is the idea that you sort on ID and then use the searchAfter to get the results after the last ID that was retrieved instead of relying on the offset attribute.

5

One of us is missing the point here.
My statement is that the API is bringing back the data in a random order BEFORE applying the offset. Why would applying a ‘searchAfter’ filter work when the data is in a random order.
All ‘searchAfter’ is doing is changing the parameters of the offset from numerical number to a variable point, AFTER the API call.
Both instances are assuming that the data held in Atlas is in a fixed order, and (observationally on many months of data requests) its not stored in any order.

6

@phil_awlings, I understand your concern, but the searchAfter is supposed to be the cure for exactly what you are saying, i.e. that you might end up with missing data and/or getting duplicate date.

Have you tried to implement the searchAfter pagination in your script? And if so, did it fail? If that is the case, it might be good to raise that as a bug to SailPoint.

7

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.