requiresPeriodicRefresh not triggering after daily ISC Identity Refresh

Identity Security Cloud (ISC) ISC Discussion and Questions
,
1

Hi All,

Following previous discussions (just for context):

https://developer.sailpoint.com/discuss/t/requiresperiodicrefresh-changes-and-wrong-documentation/211788?u=leolopes_verisure

I got the point where requiresPeriodicRefresh is not behaving as expected, I kindly ask for help to see if I’m missing something. Here is the deal:

I have a logic where I use transform evaluation based on date/time and 2 LCS to move from one LCS to another in order to make a “retry” mechanism on LCS operations.

All this logic is working perfectly. How do I know it works? Because it works when triggered from these events:

  1. When I use “Process Identity”
  2. When I aggregate a “Single Account” of the Identity account source.
  3. When I perfom “Refresh all identities” in the IdentityProfile using API (VS Code).
  4. When identity naturally refreshes due to changes in its data.

So, I presume the logic itself is clearly working. As a final part, I need it to work by itself where requiresPeriodicRefresh would fit perfectly, performs the logic twice a day, but it is not triggering anything.

In IdentityProfile, cloudLifeCycle is calculated by a Transform where I set the “requiresPeriodicRefresh”:

image
image1617×137 5.83 KB

image
image419×204 4.95 KB

So, what else could be missing to properly makes all identities to be processed during daily ISC identity refresh?

Best Regards,
Leonilson Lopes

2

Hi @leolopes_verisure

I believe requiresPeriodRefresh should only be set on dateMath or dateCompare transform types. It looks like you are setting it on the top level lookup based transform when it should be set in your dateMath/dateCompare transforms. Are you also setting it in those transforms as well? If not, I would definitely try that and see if it resolves your issue.

Here is an example using it in a static transform:

{
    "id": "IDHERE",
    "name": "Employee cloudLifecycleState",
    "type": "static",
    "attributes": {
        "currentLifecycleState": {
            "attributes": {
                "values": [
                    {
                        "attributes": {
                            "name": "cloudLifecycleState"
                        },
                        "type": "identityAttribute"
                    },
                    "NA"
                ]
            },
            "type": "firstValid"
        },
        "isInTerminationGracePeriod": {
            "attributes": {
                "firstDate": {
                    "attributes": {
                        "expression": "+3d/d",
                        "input": {
                            "attributes": {
                                "input": {
                                    "attributes": {
                                        "values": [
                                            {
                                                "attributes": {
                                                    "attributeName": "endDate",
                                                    "sourceName": "Replication Source"
                                                },
                                                "type": "accountAttribute"
                                            },
                                            {
                                                "attributes": {
                                                    "attributeName": "termdate",
                                                    "sourceName": "Workday"
                                                },
                                                "type": "accountAttribute"
                                            },
                                            "2099-01-01"
                                        ]
                                    },
                                    "type": "firstValid"
                                },
                                "inputFormat": "yyyy-MM-dd",
                                "outputFormat": "ISO8601"
                            },
                            "type": "dateFormat"
                        }
                    },
                    "requiresPeriodicRefresh": true,
                    "type": "dateMath"
                },
                "negativeCondition": "false",
                "operator": "gte",
                "positiveCondition": "true",
                "secondDate": {
                    "attributes": {
                        "expression": "/d",
                        "input": {
                            "attributes": {
                                "id": "Global - UTC to Herb Time",
                                "input": {
                                    "attributes": {
                                        "expression": "now"
                                    },
                                    "requiresPeriodicRefresh": true,
                                    "type": "dateMath"
                                }
                            },
                            "type": "reference"
                        }
                    },
                    "requiresPeriodicRefresh": true,
                    "type": "dateMath"
                }
            },
            "requiresPeriodicRefresh": true,
            "type": "dateCompare"
        },
        "isTerminatedRecently": {
            "attributes": {
                "firstDate": {
                    "attributes": {
                        "expression": "+45d/d",
                        "input": {
                            "attributes": {
                                "input": {
                                    "attributes": {
                                        "values": [
                                            {
                                                "attributes": {
                                                    "attributeName": "endDate",
                                                    "sourceName": "Replication Source"
                                                },
                                                "type": "accountAttribute"
                                            },
                                            {
                                                "attributes": {
                                                    "attributeName": "termdate",
                                                    "sourceName": "Workday"
                                                },
                                                "type": "accountAttribute"
                                            },
                                            "2099-01-01"
                                        ]
                                    },
                                    "type": "firstValid"
                                },
                                "inputFormat": "yyyy-MM-dd",
                                "outputFormat": "ISO8601"
                            },
                            "type": "dateFormat"
                        }
                    },
                    "requiresPeriodicRefresh": true,
                    "type": "dateMath"
                },
                "negativeCondition": "inactive",
                "operator": "lte",
                "positiveCondition": "delete",
                "secondDate": {
                    "attributes": {
                        "expression": "/d",
                        "input": {
                            "attributes": {
                                "id": "Global - UTC to Herb Time",
                                "input": {
                                    "attributes": {
                                        "expression": "now"
                                    },
                                    "requiresPeriodicRefresh": true,
                                    "type": "dateMath"
                                }
                            },
                            "type": "reference"
                        }
                    },
                    "requiresPeriodicRefresh": true,
                    "type": "dateMath"
                }
            },
            "requiresPeriodicRefresh": true,
            "type": "dateCompare"
        }
        "value": "#LOGICHERE"
    },
    "internal": false
}
3

Hi @trettkowski It is my understanding that it should be on the top level transform, not the nested transforms, as those would not be triggered unless the top level was triggered.

Hi @leolopes_verisure I believe that requiresPeriodicRefresh only occurs on the evening (20:00) re-calculation, not the 08:00. Could that be what you are seeing?

4

“requiresPeriodicRefresh=true” is evaluated only during periodic identity refresh processing and may not be triggered by all daily ISC refresh operations.

Verify that the identity profile’s refresh interval has elapsed and that the identity meets the condition for a periodic refresh.

If the flag remains set but no refresh occurs after expected interval, opening SailPoint Support case would be a good idea.

5

Hi @trettkowski ,

Thanks for the suggestions.

While requiresPeriodicRefresh is intended to be used where I have “now” date comparisons, I didn’t see anything saying that it only works within it, specially because this function is not really exclusive for dates.

I could try it, but it would create another issue for me: Indeed a use date/time check with “now”, but this is nested within the transform. So far I have learned that the requiresPeriodicRefresh must be in the top transform, the one which is really verified. If i put it within a nested transform, it may not be checked during refresh.

6

That’s not my understanding, but I would love to be proven wrong just so everyone is clear on how this works.

I’ll run a test tonight with the two different types, top-level vs lower-level, and see if I see any differences.

7

Good point. I guess this all depends on how ISC evaluates whether to evaluate it only at the top level or if it checks the lower level transforms as well. I’ll run a test tonight and see if I can find a difference.

8

Hi @j_place ,

Thanks for your answers.

Following the topics I have mentioned, despite documentation says requiresPeriodicRefresh should be in “top level” the truth is that it is only accepted within “attributes”. If you put in top level, it saves, but once you check again the parameter disappears.

I’m already testing it for days, if the identity refresh happens at least once a day I should see my process working, but nothing really happens.

9

Hi @BBR1 ,

Thanks for your suggestion.

I’ve been testing for days, and refresh interval never triggers my process.

You mentioned “identity meets the condition for a periodic refresh”, what are these conditions? Is it documented? What I assume so far is that it would refresh ALL identities in that profile. Currently, these identities are INACTIVE_LONG_TERM and source account disabled.

10

Hi @trettkowski ,

If you have already set for top-level, go back there and see if the property is there, I think it won’t be anymore! :sweat_smile:

11

Agreed that it should be in attributes, i meant it should be on the top level transform, not the top level of the top level transform, if you see what i mean :wink:

requiresPeriodicRefresh works for my LCS transform (at 20:00), maybe you could provide more detail on your “2 LCS retry” mechanism?

12

Yeah, like @j_place already mentioned, I meant the top level transform, so if you’re looking at my previous example, it would be like this:

{
    "id": "IDHERE",
    "name": "Employee cloudLifecycleState",
    "type": "static",
    "attributes": {
        "requiresPeriodicRefresh": true,
        "currentLifecycleState": {
            "attributes": {
                "values": [
                    {
                        "attributes": {
                            "name": "cloudLifecycleState"
                        },
                        "type": "identityAttribute"
                    },
                    "NA"
                ]
            },
            "type": "firstValid"
        },
        "isInTerminationGracePeriod": {
            "attributes": {
                "firstDate": {
                    "attributes": {
                        "expression": "+3d/d",
                        "input": {
                            "attributes": {
                                "input": {
                                    "attributes": {
                                        "values": [
                                            {
                                                "attributes": {
                                                    "attributeName": "endDate",
                                                    "sourceName": "Replication Source"
                                                },
                                                "type": "accountAttribute"
                                            },
                                            {
                                                "attributes": {
                                                    "attributeName": "termdate",
                                                    "sourceName": "Workday"
                                                },
                                                "type": "accountAttribute"
                                            },
                                            "2099-01-01"
                                        ]
                                    },
                                    "type": "firstValid"
                                },
                                "inputFormat": "yyyy-MM-dd",
                                "outputFormat": "ISO8601"
                            },
                            "type": "dateFormat"
                        }
                    },
                    "type": "dateMath"
                },
                "negativeCondition": "false",
                "operator": "gte",
                "positiveCondition": "true",
                "secondDate": {
                    "attributes": {
                        "expression": "/d",
                        "input": {
                            "attributes": {
                                "id": "Global - UTC to Herb Time",
                                "input": {
                                    "attributes": {
                                        "expression": "now"
                                    },
                                    "type": "dateMath"
                                }
                            },
                            "type": "reference"
                        }
                    },
                    "type": "dateMath"
                }
            },
            "type": "dateCompare"
        },
        "isTerminatedRecently": {
            "attributes": {
                "firstDate": {
                    "attributes": {
                        "expression": "+45d/d",
                        "input": {
                            "attributes": {
                                "input": {
                                    "attributes": {
                                        "values": [
                                            {
                                                "attributes": {
                                                    "attributeName": "endDate",
                                                    "sourceName": "Replication Source"
                                                },
                                                "type": "accountAttribute"
                                            },
                                            {
                                                "attributes": {
                                                    "attributeName": "termdate",
                                                    "sourceName": "Workday"
                                                },
                                                "type": "accountAttribute"
                                            },
                                            "2099-01-01"
                                        ]
                                    },
                                    "type": "firstValid"
                                },
                                "inputFormat": "yyyy-MM-dd",
                                "outputFormat": "ISO8601"
                            },
                            "type": "dateFormat"
                        }
                    },
                    "type": "dateMath"
                },
                "negativeCondition": "inactive",
                "operator": "lte",
                "positiveCondition": "delete",
                "secondDate": {
                    "attributes": {
                        "expression": "/d",
                        "input": {
                            "attributes": {
                                "id": "Global - UTC to Herb Time",
                                "input": {
                                    "attributes": {
                                        "expression": "now"
                                    },
                                    "type": "dateMath"
                                }
                            },
                            "type": "reference"
                        }
                    },
                    "type": "dateMath"
                }
            },
            "type": "dateCompare"
        }
        "value": "#LOGICHERE"
    },
    "internal": false
}
13

Hi @j_place ,

As a high-level, this is how it works:

Today I have 2 LCS “normal” LCS, as result of HR identity data:

  1. Terminated - Disable accounts, set INACTIVE-LONG-TERM
  2. postterminated30days - Disable account, delete accounts, set INACTIVE-LONG-TERM

As name suggests, postterminated30days happens 30 days after terminated, in order to delete some accounts.

So, due to the lack of retry mechanism in the LCS (I’m aware of error message retries, but not good solution for us), I had to come up with something. Despite of the different variations possible, I went this path:

  1. I have a FLAG in the identity, which tells me if there are accounts that missed termination/deletion.
  2. I have created a new LCS called “transitionForRetry”, to serve as “bounce”, so identity gets this LCS and then later returns to terminated or postterminated30days, this movement triggers again the disable/delete accounts, recreating the “retry” effect.
  3. I make this “bounce” by adding this LCS + FLAG + TimeTable in the final calculation. So, assuming the daily identity refresh I expected: at 20hs it would move to transitionForRetry those identities FLAGGED with pending accounts to be terminated. At 8:00 it would return to terminated or postterminated30days, performing the “retry”.

So, as I mentioned, the mechanism work perfectly in all scenarios where I can control the identity refresh, but it is not being triggered by the background daily refreshes.

14

I do not have any doc confirming that,

But if identities in the profile are not being refreshed after several intervals have elapsed and if they are INACTIVE_LONG_TERM with disabled source accounts, opening a Support case with SailPoint can confirm the expected behavior for inactive identities.

15

Hi @leolopes_verisure There’s your prob. Identities in INACTIVE_LONG_TERM Identity State are exempted from scheduled processing. See Identity States Overview - SailPoint Identity Services

16

Hi @j_place ,

Thank you! Finally found something :sweat_smile: !

I need to review the process… I have couple options.

Thank you!

17

I would edit your Identity State for Terminated to Inactive (Short Term). Then you should be able to use requiresPeriodicRefresh, which I think should remove your need for the retry mechanism.

18

Note that the requiresPeriodicRefresh doesn’t apply to the morning scheduled refresh, only the evening scheduled refresh.

From the linked thread:

The requiresPeriodicRefresh option is there to force us to include the Identities with this configured on their IdentityProfiles into the 8pm refresh

19

Hi @j_place ,

Could you please explain better what you mean “which I think should remove your need for the retry mechanism”?

Change to inactive_short_term could be a solution to solve the problem of the identity being skipped in the daily refresh. However, it does not make a retry to happen by itself, and LCS disable/delete triggers is “one shot only”, if any account fails, there won’t be a retry (here is where my retry mechanism enters).

20

Apologies, I didn’t read the full details of the post, I thought it was failing to put them in the correct LCS. My understanding, though, is retryable errors are retried, and if you are getting a non-retryable error, shouldn’t that be investigated, rather than re-attempted. I appreciate I don’t have full knowledge of your environment, so I could be off the mark.