ISC JSON path evaluator from SailPoint does not behave the same as ISC workflows

Feedback Bugs
, ,
1

Hi all, :slight_smile:

SailPoint has written a Json path evaluator that should be as similar as possible to the implementation used by SailPoint in each area of the UI (only workflow and event trigger implementations are supported for now).

This is great and can be useful, but I noticed during testing that it is actually not matching in behavior. Using this evaluator to convince yourself that your workflow implementation will work correctly is therefore not a working strategy.
Since the default JSON of the given evaluator does not have much data to test on (empty arrays, multivalue arrays, booleans, integers), I have tested it on this JSON object:

{
    "id": "1234",
    "name": "fruits",
    "items": [
        {
            "name": "apple",
            "color": "red",
            "tasty": true,
            "cost": 10,
            "vendors": [
                {
                    "name": "Good Fruit Hood",
                    "id": "A"
                },
                {
                    "name": "Steve",
                    "id": "B"
                }
            ]
        },
        {
            "name": "apple",
            "color": "green",
            "tasty": true,
            "cost": 9,
            "vendors": [
                {
                    "name": "Good Fruit Hood",
                    "id": "A"
                },
                {
                    "name": "Steve",
                    "id": "B"
                }
            ]
        },
        {
            "name": "pear",
            "color": "green",
            "tasty": true,
            "cost": 11,
            "vendors": []
        },
        {
            "name": "lemon",
            "color": "yellow",
            "tasty": false,
            "cost": 10,
            "vendors": [
                {
                    "name": "CJ",
                    "id": "C"
                }
            ]
        }
    ]
}

Some examples I found while testing, more might exist, but it is demonstrating that they are actually not using the same implementation:

JSON path behavior in json path evalator from SailPoint actual behavior in workflows
$.name.length() 6 8
$.items[?(@.name == ‘pear’)] [{…}] {…}
$.items[?(@.name == ‘banana’)] [ ] null

Also, I saw that the json validator is breaking on the paths below, where we do expect a value (have not compared this to workflows, but just saw it breaking):

  1. $.items[0].tasty
  2. $.items[?(@.cost == 10)] results in an array with two values, but $.items[?(@.cost == 10)][0] gives an empty array instead of the first object.
  3. $.items[?(@.tasty)] also returned the lemon.

Tagging @tyler_mairose in response to:

Kind regards,
Angelo

2 Likes
2

Thanks for reporting this Angelo. To add a little more context to what @tyler_mairose said, the Javascript library we use in the web validator more closely resembles thew event trigger implementation, but the Workflows implementation still needs some work. The good news is that we have now taken ownership of our JSONpath validation tools instead of offloading that to third party websites that have the same issues, which means we can fix them. Please keep the bug reports coming as you notice any other inaccuracies and we will work to fix them.

1 Like
3

Bug ticket opened. DEVREL-1894

1 Like
4

I fixed the code for the workflows string length function. It will now add 2 to the length for strings since GoSlice includes the double quotes when checking length.

As for the other two:

$.items[?(@.name == ‘pear’)] [{…}] {…}
$.items[?(@.name == ‘banana’)] null

These are actually a divergence from the original GoSlice library code. The workflows team early on implemented an auto-flattening feature for single item arrays that causes the behavior you are experiencing. This was done to help support variable insertion from arrays, as they otherwise would have been persisted as single item arrays. The workflows team is actively working on reverting this (PLTWRKFLW-5787) so that the auto-flattening no longer occurs. They are making good progress, so I’m not going to implement auto-flattening and wait for them to complete their ticket.

@Darrell is working on fixing the breaking validation paths you mentioned. Once he’s done, a new version will be published with the fixes.

5

Thank you @colin_mckibben,

I agree that is makes sense that the auto-flattening feature will be reverted as it was causing more harm than good (you don’t always know if the filter will result in just one value, so you still want to iterate over it, even if it ends up containing only one value. Similar thing holds for no results. With an empty array you can still iterate over it, and it will just make no iterations. With a null value, it breaks on an error, requiring workarounds.

Although adding 2 to the length of the strings will make it consistent to the behavior of ISC workflows and consistency is important here, I would argue that the best change here would be to instead make it consistent by subtract 2 from the result of ISC workflows length function. After all, let’s say we do native change detection on the firstname attribute, we notice a firstname changed to Tom, and then we want to send an HTTP request with the length of the firstname, we want to give the number 3, but ISC workflows now gives the number 5. Why would we care amount the number of characters only used for syntax? This was an odd thing to have to tell our client.

Also notice how you would obtain other strange issues as well. With the way you have things implemented now, equations like this one are suddenly no longer true:

($firstname + "." + $lastname).length() == $firstname.length() + 1 + $lastname.length()

Since a defect has been created for this, could you please revert your change when the defect was fixed as well?

See details here:

Kind regards,
Angelo

1 Like
6

Yes, I am watching PLTWRKFLW-6024. Once that is fixed, I will revert my change.

1 Like