NELM Bulk Update API

Identity Security Cloud (ISC) ISC Discussion and Questions
1

I noticed that the NELM Bulk Update API uses a content-type of multipart/form-data in the body. Has anyone developed a script to automate the upload of NELM data? Our client will be using a SharePoint front-end to input NELM data, export that data to a CSV, and would like to then import the data using a scheduled job but I am not seeing an easy way to accomplish that with a multipart/form-data content type. Any guidance here would be helpful!

2

Hey @justinrhaines,

Thanks for posting! Let us do some poking around and get back to you shortly.

3

Hello @justinrhaines,

Is this the endpoint you are referring to?

https://developer.sailpoint.com/idn/api/beta/non-employee-records-bulk-upload

Do you know what programming language you will use for the script?

Here are some examples using cURL and Python:

cURL

curl --location --request POST 'https://{tenant}.api.identitynow.com/beta/non-employee-sources/2c9180887933afa701793d96d9cb5e6d/non-employee-bulk-upload' \
--header 'Authorization: Bearer {access_token}' \
--form 'data=@"/Users/my_user/Documents/non_employees_for_source_Contractors.csv"'

Python using requests library

import requests

url = "https://{tenant}.api.identitynow.com/beta/non-employee-sources/2c9180887933afa701793d96d9cb5e6d/non-employee-bulk-upload"

payload={}
files=[
  ('data',('non_employees_for_source_Contractors.csv',open('/Users/my_user/Documents/non_employees_for_source_Contractors.csv','rb'),'text/csv'))
]
headers = {
  'Authorization': 'Bearer {access_token}'
}

response = requests.request("POST", url, headers=headers, data=payload, files=files)

print(response.text)

To get the source ID, you can use the following endpoint:

curl --location --request GET 'https://{tenant}.api.identitynow.com/beta/non-employee-sources' \
--header 'Authorization: Bearer {access_token}'

Look for the sourceId of the source you want to update in the response:

[
    {
        "id": "ac110004-793d-14e4-8179-3d96e5300000",
        "sourceId": "2c9180887933afa701793d96d9cb5e6d",
        "name": "Contractors",
        "description": "Non-employee contractors",
        "approvers": [],
        "accountManagers": [],
        "created": "2021-05-05T17:32:49.989Z",
        "modified": "2021-05-05T17:32:49.989Z",
        "nonEmployeeCount": null
    }
]
4

Yes, that is the API I am referring to. I would prefer to use Powershell as that is the client’s standard and can be run using the Windows task scheduler, but I am open to other options as well.

5

Here is a PowerShell example using RestMethod

$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Authorization", "Bearer {access_token}")

$multipartContent = [System.Net.Http.MultipartFormDataContent]::new()
$multipartFile = '/Users/my_user/Documents/non_employees_for_source_Contractors.csv'
$FileStream = [System.IO.FileStream]::new($multipartFile, [System.IO.FileMode]::Open)
$fileHeader = [System.Net.Http.Headers.ContentDispositionHeaderValue]::new("form-data")
$fileHeader.Name = "data"
$fileHeader.FileName = "non_employees_for_source_Contractors.csv"
$fileContent = [System.Net.Http.StreamContent]::new($FileStream)
$fileContent.Headers.ContentDisposition = $fileHeader
$multipartContent.Add($fileContent)

$body = $multipartContent

$response = Invoke-RestMethod 'https://{tenant}.api.identitynow.com/beta/non-employee-sources/2c9180887933afa701793d96d9cb5e6d/non-employee-bulk-upload' -ContentType 'multipart/form-data' -Method 'POST' -Headers $headers -Body $body
$response | ConvertTo-Json
1 Like
6

Thanks, this is a great start but I am receiving the following response:

Invoke-RestMethod : {“messages”:[{“localeOrigin”:“DEFAULT”,“locale”:“en-US”,“text”:“Content type ‘application/x-www-form-urlencoded;charset=UTF-8’ not supported”}],“detailCode”:"400.0 Bad request

syntax",“trackingId”:“f7de0c8cc9da420e97a8feec9b341cd7”}

At line:27 char:13

  • $response = Invoke-RestMethod 'https://amfidelity-sb.api.identitynow. …

  •

  • CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException

  • FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

Shouldn’t this be a different content-type than x-www-form-urlencoded?

7

This should have a content-type of multipart/form-data. I haven’t tested this myself as I’m using OS X, but maybe you can try explicitly setting the content-type header by adding the following line in your headers section:

$headers.Add("Content-Type", "multipart/form-data")

If that doesn’t work, then I found this Stack Overflow article that suggests adding an additional flag to the Invoke-RestMethod command:

$response = Invoke-RestMethod 'https://{tenant}.api.identitynow.com/beta/non-employee-sources/2c9180887933afa701793d96d9cb5e6d/non-employee-bulk-upload' -ContentType 'multipart/form-data' -Method 'POST' -Headers $headers -Body $body
8

Hi Collin–

Yes I tried both of those methods and received a 500 Internal Server Error on both. Any other ideas?

9

Let me install a Windows VM and play around with this. I’ll see if I can figure out what’s wrong and get back to you.

10

Hi Colin–

Did you have a chance to test this with your Windows VM?

11

With the IdentityNow PowerShell Module we use multipart/form-data for the Join-IdentityNowAccount cmdlet.
We needed to convert the payload to UTF8 and add formatting to the CSV to get it to play nicely.
Here is an example where the payload is in CSV stringdata in the $csv variable.

$result = Invoke-restmethod -Uri "https://$($IdentityNowConfiguration.orgName).api.identitynow.com/cc/api/source/loadUncorrelatedAccounts/$source" `
                -Method "POST" `
                -Headers @{Authorization = "$($v3Token.token_type) $($v3Token.access_token)"; "Accept-Encoding" = "gzip, deflate, br"} `
                -ContentType "multipart/form-data; boundary=----WebKitFormBoundaryU1hSZTy7cff3WW27" `
                -Body ([System.Text.Encoding]::UTF8.GetBytes("------WebKitFormBoundaryU1hSZTy7cff3WW27$([char]13)$([char]10)Content-Disposition: form-data; name=`"file`"; filename=`"temp.csv`"$([char]13)$([char]10)Content-Type: application/vnd.ms-excel$([char]13)$([char]10)$([char]13)$([char]10)$($csv | out-string)$([char]13)$([char]10)------WebKitFormBoundaryU1hSZTy7cff3WW27--$([char]13)$([char]10)")) `
                -UseBasicParsing

Cheers,
DR

1 Like
12

Thank you @darrenjrobinson. Based on your code I was able to put together the following PowerShell code that will work.

$uri = 'https://{tenant}.api.identitynow.com/beta/non-employee-sources/{sourceId}/non-employee-bulk-upload'
$accessToken = '{accessToken}'

$inFile = 'C:\users\my_user\Documents\contractors.csv'
$fileName = 'contractors.csv'
$csvBytes = [System.IO.File]::ReadAllBytes($inFile)
$csvText = [System.Text.Encoding]::ASCII.GetString($csvBytes)

$result = Invoke-restmethod -Uri $uri `
                -Method "POST" `
                -Headers @{Authorization = "Bearer $accessToken"} `
                -ContentType "multipart/form-data; boundary=----WebKitFormBoundaryU1hSZTy7cff3WW27" `
                -Body ([System.Text.Encoding]::UTF8.GetBytes("------WebKitFormBoundaryU1hSZTy7cff3WW27$([char]13)$([char]10)Content-Disposition: form-data; name=`"data`"; filename=`"$fileName`"$([char]13)$([char]10)Content-Type: text/csv$([char]13)$([char]10)$([char]13)$([char]10)$($csvText)$([char]13)$([char]10)------WebKitFormBoundaryU1hSZTy7cff3WW27--$([char]13)$([char]10)")) `
                -UseBasicParsing

$result
2 Likes
13

Brilliant. Great to see you got it working :+1:t2:

14

Excellent, this is working as expected. Thanks everyone for your input!

1 Like