Can Transform convert special character (UTF8) to normal character

[Rpalos (Rainiel Anne Palos)]

Hi Experts,

I am trying to use the decomposeDiacriticalMarks tranform to convert frstname with special character like “ñ” to n. However it is not working.

I created tranform referencing in this link

Will Identitynow supports UTF-8 encoding

{attributes=com.sailpoint.seaspray.transform.ConcatTransform@1e55313f} That is my output when I try to use the in built transform: See below. Any idea why this is going wrong? I’ve tried wrapping the concat in a static field but no change "name": "TEST", "type": "decomposeDiacriticalMarks", "attributes": { "input": { "attributes": { "type": "concat", "attributes": { "values": [ { …

and here is my transform.

{
    "name": "Decompose Diacritical Marks Transform",
    "type": "decomposeDiacriticalMarks",
    "attributes": {
        "input": {
            "attributes": {
                "name": "firstname"
            },
            "type": "identityAttribute"
        }
    },
    "internal": false
}

Has anyone successfully used this transform?

Thanks a lot in advance.

[Mahesh_Mukku]

Hi @Rpalos ,
Greetings of the Day!

please use ReplaceAll Transform please check the below screenshot hope it is helpful for you.

Thank You

[vijayasaikoppineni]

Hello @Rpalos

Good day!

Please check below transform, it is worked for me, can you try once.

{
    "attributes": {
        "input": {
            "attributes": {
                "name": "firstname"
            },
            "type": "identityAttribute"
        }
    },
    "type": "decomposeDiacriticalMarks",
    "name": "Decompose Diacritical Marks Transform1"
}

Thank you

[Rpalos (Rainiel Anne Palos)]

Hi @vijayasaikoppineni

This transform works with you? may I know what value of your firstname?

I Use this transform but still it’s returning

This is what I set in the csv for the firstname: Test642Lñ

And I find it weird, why the replace and decomposeDiacriticalMarks not working on my end

[Abhishek_1995 (Abhishek Kumar)]

Hi @Rpalos,
Good Day!
Can you use the ReplaceAll transform and let us know the output. As suggested by the @Mahesh_Mukku.

Thank you!

[Rpalos (Rainiel Anne Palos)]

Hi @Mahesh_Mukku

I also tried this one, if normal character “Abc”, this transform works. but If I tried using it with ñ it’s returning

Transform:

{
        "name": "Replace All Special Character Transform",
        "type": "replaceAll",
        "attributes": {
            "input": {
                "attributes": {
                    "sourceName": "TEST_SAP HCM",
                    "attributeName": "familyName"
                },
                "type": "accountAttribute"
            },
            "table": {
                "ñ": "n",
                "Ñ": "N",
                "e": "a"
            }
        },
        "internal": false
    }

[vijayasaikoppineni]

Hi @Rpalos ,
Good Day!

I have tired Replace All Transform and decomposeDiacriticalMarks its an intermediate issue , please raise this to SailPoint team .

Thank You

[sup3rmark (Mark Corsillo)]

I don’t think the transforms are the problem here.

Can you make an API call to SailPoint to see what it returns as the value for that account’s firstname attribute? I think you’ll find that the stored value doesn’t actually contain the ñ you’re expecting, likely because of an encoding issue.

[shekhardas1825 (Shekhar Das)]

Hi @Rpalos,

Try below transform, it should work for you:

{
    "name": "TRN FirstName",
    "type": "replace",
    "attributes": {
        "input": {
            "attributes": {
                "input": {
                    "attributes": {
                        "input": {
                            "attributes": {
                                "name": "firstname"
                            },
                            "type": "identityAttribute"
                        },
                        "regex": "[´]",
                        "replacement": "'"
                    },
                    "type": "replace"
                }
            },
            "type": "decomposeDiacriticalMarks"
        },
        "regex": "[^a-zA-Z0-9-'´`’‘ ]",
        "replacement": ""
    },
    "internal": false
}

[Rpalos (Rainiel Anne Palos)]

Hi @vijayasaikoppineni,

Thanks for the advice.

My decomposeDiacriticalMarks transform is already working. The issue is when I edited the .CSV file in Excel, the characters are being converted to different format. so When I tried to open it via notepad or notepad++, it works now. Thanks for the help.

[Rpalos (Rainiel Anne Palos)]

Hi Everyone,

Thank you all for the help The issue occurs where I am opening and editing the CSV file. so When I tried to open it via notepad or notepad++, decomposeDiacriticalMarks transform can covert special characters now.

Can refer on this link. Special characters will not open correctly from .CSV - Microsoft Community

[Janga_Tulasi (Tulasi Janga)]

Hi All,

We are seeing the same issue where the source is Oracle HCM and when we get the data with special characters, it is represented as question mark (�) in Sailpoint. For e.g. Alarc�n.
I tried the above transform but still no luck. Any insights please?

Thank you,
Tulasi

[phil_awlings (Phil Rawlings)]

The in built transform does not cover all diacritic marks for some reason. I had to add this table to cover some of them. (This table was written about a year ago, so I don’t know if it needs updating):

        "table": {
            "Å": "A",
            "å": "a",
            "Æ": "AE",
            "æ": "ae",
            "Ã": "A",
            "ã": "a",
            "Đ": "D",
            "đ": "d",
            "Ë": "E",
            "ë": "e",
            "Ħ": "H",
            "ħ": "h",
            "ı": "i",
            "Í": "I",
            "í": "i",
            "ĸ": "k",
            "Ł": "L",
            "ł": "l",
            "Ø": "O",
            "ø": "o",
            "Œ": "OE",
            "œ": "oe",
            "Þ": "P",
            "þ": "p",
            "ß": "s",
            "Ŧ": "T",
            "ŧ": "t"
        }

[Janga_Tulasi (Tulasi Janga)]

When we download the data from Sailpoint, all the special characters are represented as � (e.g. Alarc�n), so we are not sure what the special character would be to replace.

[phil_awlings (Phil Rawlings)]

Are you downloading it as a .csv file, or any excel format?
If so, then all special characters will look like that. You will need to clean the data in SP before downloading it.

[Janga_Tulasi (Tulasi Janga)]

Ok, we downloaded as csv file but the actual problem here is that the data, which is coming from HCM, all the special characters are represented as � in SP. So, we are not sure what the special character would be and how to clean up in SP.

[phil_awlings (Phil Rawlings)]

When you look at the source account, are they showing there as special characters or are they being converted to ‘�’.
They ‘should’ be fine there.
We had the situation where they were replaced in AD in the DN, but not in the display name, so I was able to down load a search and then manually fixed them in AD

[Janga_Tulasi (Tulasi Janga)]

In the source account they are showing as: �

[phil_awlings (Phil Rawlings)]

Ok.
Your options seem to be:

• Speak to the HR team to replace all the special characters with latin equivalents
• Create a beforeProvisioning rule and run the transform there.
  Other than that, I’m not sure