ISC JSON path evaluator from SailPoint does not behave the same as ISC workflows

Hi all, :slight_smile:

SailPoint has written a Json path evaluator that should be as similar as possible to the implementation used by SailPoint in each area of the UI (only workflow and event trigger implementations are supported for now).

This is great and can be useful, but I noticed during testing that it is actually not matching in behavior. Using this evaluator to convince yourself that your workflow implementation will work correctly is therefore not a working strategy.
Since the default JSON of the given evaluator does not have much data to test on (empty arrays, multivalue arrays, booleans, integers), I have tested it on this JSON object:

{
    "id": "1234",
    "name": "fruits",
    "items": [
        {
            "name": "apple",
            "color": "red",
            "tasty": true,
            "cost": 10,
            "vendors": [
                {
                    "name": "Good Fruit Hood",
                    "id": "A"
                },
                {
                    "name": "Steve",
                    "id": "B"
                }
            ]
        },
        {
            "name": "apple",
            "color": "green",
            "tasty": true,
            "cost": 9,
            "vendors": [
                {
                    "name": "Good Fruit Hood",
                    "id": "A"
                },
                {
                    "name": "Steve",
                    "id": "B"
                }
            ]
        },
        {
            "name": "pear",
            "color": "green",
            "tasty": true,
            "cost": 11,
            "vendors": []
        },
        {
            "name": "lemon",
            "color": "yellow",
            "tasty": false,
            "cost": 10,
            "vendors": [
                {
                    "name": "CJ",
                    "id": "C"
                }
            ]
        }
    ]
}

Some examples I found while testing, more might exist, but it is demonstrating that they are actually not using the same implementation:

JSON path behavior in json path evalator from SailPoint actual behavior in workflows
$.name.length() 6 8 [edit: This has been fixed to 6 since 12-12-2024]
$.items[?(@.name == ‘pear’)] [{…}] {…}
$.items[?(@.name == ‘banana’)] [ ] null

Also, I saw that the json validator is breaking on the paths below, where we do expect a value (have not compared this to workflows, but just saw it breaking):

  1. $.items[0].tasty
  2. $.items[?(@.cost == 10)] results in an array with two values, but $.items[?(@.cost == 10)][0] gives an empty array instead of the first object.
  3. $.items[?(@.tasty)] also returned the lemon.

Tagging @tyler_mairose in response to:

Kind regards,
Angelo

2 Likes

Thanks for reporting this Angelo. To add a little more context to what @tyler_mairose said, the Javascript library we use in the web validator more closely resembles thew event trigger implementation, but the Workflows implementation still needs some work. The good news is that we have now taken ownership of our JSONpath validation tools instead of offloading that to third party websites that have the same issues, which means we can fix them. Please keep the bug reports coming as you notice any other inaccuracies and we will work to fix them.

1 Like

Bug ticket opened. DEVREL-1894

1 Like

I fixed the code for the workflows string length function. It will now add 2 to the length for strings since GoSlice includes the double quotes when checking length.

As for the other two:

$.items[?(@.name == ‘pear’)] [{…}] {…}
$.items[?(@.name == ‘banana’)] null

These are actually a divergence from the original GoSlice library code. The workflows team early on implemented an auto-flattening feature for single item arrays that causes the behavior you are experiencing. This was done to help support variable insertion from arrays, as they otherwise would have been persisted as single item arrays. The workflows team is actively working on reverting this (PLTWRKFLW-5787) so that the auto-flattening no longer occurs. They are making good progress, so I’m not going to implement auto-flattening and wait for them to complete their ticket.

@Darrell is working on fixing the breaking validation paths you mentioned. Once he’s done, a new version will be published with the fixes.

Thank you @colin_mckibben,

I agree that is makes sense that the auto-flattening feature will be reverted as it was causing more harm than good (you don’t always know if the filter will result in just one value, so you still want to iterate over it, even if it ends up containing only one value. Similar thing holds for no results. With an empty array you can still iterate over it, and it will just make no iterations. With a null value, it breaks on an error, requiring workarounds.

Although adding 2 to the length of the strings will make it consistent to the behavior of ISC workflows and consistency is important here, I would argue that the best change here would be to instead make it consistent by subtract 2 from the result of ISC workflows length function. After all, let’s say we do native change detection on the firstname attribute, we notice a firstname changed to Tom, and then we want to send an HTTP request with the length of the firstname, we want to give the number 3, but ISC workflows now gives the number 5. Why would we care amount the number of characters only used for syntax? This was an odd thing to have to tell our client.

Also notice how you would obtain other strange issues as well. With the way you have things implemented now, equations like this one are suddenly no longer true:

($firstname + "." + $lastname).length() == $firstname.length() + 1 + $lastname.length()

Since a defect has been created for this, could you please revert your change when the defect was fixed as well?

See details here:

Kind regards,
Angelo

1 Like

Yes, I am watching PLTWRKFLW-6024. Once that is fixed, I will revert my change.

1 Like

We implemented the actual length behavior of adding 2 to strings. We’ll remove it once that workflows ticket is done. We also fixed the behavior $.items[?(@.tasty)] so it will return just the true items. As for 1. $.items[0].tasty and [?(@.cost == 10)][0], this behavior is present in the goslice library as well, so this is expected.

Sounds reasonable!

Great! :smiley:

Earlier it broke (when I created this topic). Now it actually gives the boolean variable correctly. So you say the behavior was as expected according to the goslice library, but you (or someone else) then fixed it anyway?

So if I understand you correctly, the “goslice library” is a JSONpath implementation, where

  1. If you first try to filter an array of multiple json objects (which gives you a array of fewer items) and then try to get the first element of the filtered array, it will give you an empty array.
  2. If you ask for the length of a firstname, it will be two more than the actual length, because it also adds the quotes.

If this is truly the case, then why would SailPoint ever choose to implement it according to this implementation? I hope you agree that these two behaviors are quite counter intuitive, undesired behaviors.

I may have confused myself. This is working the same in Goslice and the online validator now.

I agree, that intuitively it makes sense that you should be able to refer to an index of a filtered array. However, this doesn’t work in Goslice, the library we use for the jsonpath web UI, https://www.javainuse.com/jsonpath, or https://jsonpath.com/. So the problem seems to be a general JSONpath issue and not any one library. Maybe we are the ones who are misunderstanding how JSONpath works :person_shrugging:

Oh yeah, you are completely right here @colin_mckibben !

JSONpath indeed doesn’t seem to support first filtering a JSON array, and then taking the first element from the subarray. According to the RFC, there is no way to get the first element of the filtered result of a JSON array using only one JSONpath expression. Instead the workaround suggestion seems to perform this task outside of JSONpath, for example by using two JSONpaths. One JSONpath expression to filter your JSON array, then ensuring you get the results and ensure it is still a JSON object (a JSON array), and then use another JSONpath to obtain the results. In a programming language like Java, Python or Ruby this is easily done. Follow-up question then is. How can we achieve this in ISC workflows?

Suppose my workflow performs an action (for example to get all accounts of a specific identity) and then gets a JSON response which is an array. Is it then possible in ISC workflows to apply a filter on the results with a JSONpath expression, store the result in a variable, and then apply a second JSONpath expression on that variable, to get the first record? I tried with the define variable operator, but did not succeed. Maybe because it expects to return a string instead of a structured JSON object?