cc/api/source/loadAccounts/{id} - Error

Im trying to use this endpoint to bulk load accounts into a source (file)

i’m generating the CSV from JSON and putting it into an s3 bucket.
im making the POST with a bearer token that I use for other API requests etc…

I have this as the body

‘disableOptimization=true&file=@“{s3Url.csv}”’

And this is these various errors in the response object.

  {
    code: 'typeMismatch',
    field: 'file',
    internalMsg: 'Could not find matching constructor for: org.springframework.web.multipart.MultipartFile(java.lang.String)',
    rejectedValue: '{s3Url.csv}'
  }
and

{
  file: 'ApiSourceController.groovy',
  method: 'loadAccounts',
  line_number: 1167,
  class: 'com.cloudmasons.ApiSourceController'
}

Any help would be great.

I’m fairly certain you can’t provide a URL for the file. Usually you load the file as binary data into the request before sending it. This works fine when sending it locally, since Postman or other tools can load the file into the request first. Referencing a web URL probably isn’t supported though.

You will need to first download the file from S3 onto the machine that is sending the API request.

See this link for more information than you probably wanted on multipart/form-data :slight_smile: . Forms in HTML documents

When do you guys plan to talk about the Config for the SDK?

Also, I was looking over the SDK for TS. This is great!

As far as the file prop. I assume you are wanting a Buffer for the file vs a csv string etc…?

Im not sure whats going on. I get the same error via Node as I do with the Sailpoint Postman setup.

The Error

{
    "messages": [
        {
            "localeOrigin": "REQUEST",
            "text": "An internal fault occurred.",
            "locale": "en-US"
        },
        {
            "localeOrigin": "DEFAULT",
            "text": "An internal fault occurred.",
            "locale": "en-US"
        }
    ],
    "detailCode": "500.0 Internal fault",
    "trackingId": "f21dbc2701684cee839c5a478c2d43eb"
}

The CSV is straightforward

sourceId,id,name,email,e-mail
"19de130cda854583845d2f307031ed41","admin","Administrator","[email protected]","[email protected]"

Im using a valid bearer token, valid host, and I am also using the older sourceId that is not the long guid.
{{api-url}}/cc/api/source/loadAccounts/211116

It really should be straight forward, I dont know what im doing wrong.

Any help would be great.

I should also ask, is there a better way to bulk add accounts/trigger correlation, programmatically, to a source in IDN. Maybe there is a better approach?

Try removing the sourceId column. The CSV file you upload will be sent to the source that you specify in the url. In your case, source 211116. The CSV file must match exactly to the account schema you specified for that source. I find it helpful to first download the accounts file from the source as a starting point.

2 posts were split to a new topic: Unable to create flat file source via API

@aceiss did this solve your problem?

I did not. I took that out and set the schema to the default values. In trying to get the account schema, is where is was encountering issues with the delimited file error.

I’ll try the provisionAsCsv param and see if I can get the source created correctly, then see about getting the account schema and finally try to bulk upload a csv to provision accounts.

I’ll reply here if I encounter more problems.

Thanks for the response.

@colin_mckibben

Ok, So from the other thread, I am able to create a source and the account schema looks correct.

These are the properties of the schema

id,name,givenName,familyName,e-mail,location,manager,groups

this is 1 of the many objects in the JSON, that are used to create the csv.

  id: '[email protected]',
  name: 'Asive.Marele',
  givenName: '',
  familyName: '',
  'e-mail': '[email protected]',
  location: '',
  manager: '',
  groups: ''

The headers match the above schema and the csv opens in numbers correctly. etc…

Then I use that csv to create a Binary/Buffer and send it over in FormData.

const loadAccountCSVIntoSailpoint = async (request, bearerToken, legacySourceId, filePath) => {
  const formData = new FormData()
  const file = await readFileSync(filePath)

  formData.append('disableOptimization', 'true')
  formData.append('file', file, 'aceiss.csv')
  const apiRequest = buildMultiPartRequest(request, `source/loadAccounts/${legacySourceId}`, bearerToken, formData)

  try {
    // POST  cc/api/source/loadAccounts/legacySourceId
    return executeRequest(request, apiRequest)
  } catch (err) {
    return errorMessage(request, err)
  }
}

Here is the request payload

{
    url: 'https://eu-partner27.api.identitynow.com/cc/api/source/loadAccounts/212343',
    headers: {
      'Retry-After': 1,
      Authorization: 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0ZW5hbnRfaWQiOiJhMTE5Yjg3OS1kZWI2LTQ1NDgtOGNhMS1jYjQ3OTJlOGFmY2IiLCJwb2QiOiJzdGcwMS1ldWNlbnRyYWwxIiwic3Ryb25nX2F1dGhfc3VwcG9ydGVkIjpmYWxzZSwib3JnIjoiZXUtcGFydG5lcjI3IiwiaWRlbnRpdHlfaWQiOiIyYzkxODA4MzgxM2E0MjU5MDE4MTNlNTE0NDc3NTRlOCIsInVzZXJfbmFtZSI6IkdhcnkuQmljY2FyZCIsInNjb3BlIjpbIkJnPT0iXSwic3Ryb25nX2F1dGgiOnRydWUsImV4cCI6MTY3OTkyOTA0OSwiYXV0aG9yaXRpZXMiOlsiT1JHX0FETUlOIiwic3A6dXNlciJdLCJqdGkiOiJEMUNhODFPS2RxNGlRZHhjN2VBT0RabV81d0UiLCJjbGllbnRfaWQiOiI1MGFkYTkzNjE0NWU0OTc1YWM4ZTlmNzVhM2FjMDc0YyJ9.AHOR5OW2Tob5cDsHcuGEqL8fWaPKR53FH6BURpPgh0c',
      'Content-Type': 'multipart/form-data',
      'Cache-Control': 'no-cache'
    },
    method: 'post',
    data: FormData {
      _overheadLength: 327,
      _valueLength: 2478,
      _valuesToMeasure: [],
      writable: false,
      readable: true,
      dataSize: 0,
      maxDataSize: 2097152,
      pauseStreams: true,
      _released: false,
      _streams: [Array],
      _currentStream: null,
      _insideLoop: false,
      _pendingNext: false,
      _boundary: '--------------------------470468635821468462563621'
    }
  }
{
    _overheadLength: 327,
    _valueLength: 2478,
    _valuesToMeasure: [],
    writable: false,
    readable: true,
    dataSize: 0,
    maxDataSize: 2097152,
    pauseStreams: true,
    _released: false,
    _streams: [
      '----------------------------470468635821468462563621\r\n' +
        'Content-Disposition: form-data; name="disableOptimization"\r\n' +
        '\r\n',
      'true',
      [Function: bound ],
      '----------------------------470468635821468462563621\r\n' +
        'Content-Disposition: form-data; name="file"; filename="sailpoint-09b49ac0e4cf4b5593c3f3e71b2738ee-accounts-1679885850288.csv"\r\n' +
        'Content-Type: text/csv\r\n' +
        '\r\n',
      <Buffer 69 64 2c 6e 61 6d 65 2c 67 69 76 65 6e 4e 61 6d 65 2c 66 61 6d 69 6c 79 4e 61 6d 65 2c 65 2d 6d 61 69 6c 2c 6c 6f 63 61 74 69 6f 6e 2c 6d 61 6e 61 67 ... 2424 more bytes>,
      [Function: bound ]
    ],
    _currentStream: null,
    _insideLoop: false,
    _pendingNext: false,
    _boundary: '--------------------------470468635821468462563621'
  }

and then the error back from Sailpoint ( the message part ) I can put the whole thing in here if you want

  [
    {
      localeOrigin: 'DEFAULT',
      locale: 'en-US',
      text: 'An internal fault occurred.'
    },
    {
      localeOrigin: 'REQUEST',
      locale: 'en-US',
      text: 'An internal fault occurred.'
    }
  ]

So, I think to this point I have implemented what you have suggested ( unless you see something im missing)

But the error is the same.

I can’t tell if it’s the code or the CSV being formatted wrong. What if you try manually downloading the accounts from IDN and then uploading that file? If you still get that error then it’s probably how you are constructing the request in your code.

this is the content of said csv

id,name,givenName,familyName,e-mail,location,manager,groups
[email protected],“A.User”,“”,“”,"[email protected]",“”,“”,“”
[email protected],“B.User”,“”,“”,"[email protected]",“”,“”,“”
[email protected],“C.User”,“”,“”,"[email protected]",“”,“”,“”

@colin_mckibben

So, to your point, I think its in my request structure.

Question, when you upload a csv, does it correlate immediately? is there a delay? Is there any other API that I need to call?

Try uploading the file using Postman. If you can get postman to do it successfully, then you can have postman generate the code for your language of choice. This is a quick way to see where you might be going wrong.

And when using the API, if it is successful you will get a 200 response. Depending on how big it is, correlation may take seconds to minutes.

@colin_mckibben

Ok, great! Im now able to use the API in both my code and postman, they return the same response

{
    "success": true,
    "task": {
        "attributes": {
            "appId": "09b49ac0e4cf4b5593c3f3e71b2738ee",
            "eventId": 3413954,
            "optimizedAggregation": "disabled"
        },
        "completed": null,
        "completionStatus": null,
        "created": 1679934325152,
        "description": "Aggregates from the specified application.",
        "id": "d8f9fc295dde43ecadb074d648529dc1",
        "launched": 1679934325166,
        "launcher": "Gary.Biccard",
        "messages": [],
        "name": "Cloud Account Aggregation",
        "parentName": null,
        "progress": null,
        "returns": [
            {
                "attributeName": "applications",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_APPLICATIONS"
            },
            {
                "attributeName": "total",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_TOTAL"
            },
            {
                "attributeName": "optimizedAggregation",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_OPTIMIZED"
            },
            {
                "attributeName": "ignored",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_IGNORED"
            },
            {
                "attributeName": "optimized",
                "displayLabel": "TASK_OUT_UNCHANGED_ACCOUNTS"
            },
            {
                "attributeName": "created",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_CREATED"
            },
            {
                "attributeName": "updated",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_UPDATED"
            },
            {
                "attributeName": "deleted",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_DELETED"
            },
            {
                "attributeName": "managerChanges",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_MANAGER_CHANGES"
            },
            {
                "attributeName": "detectedRoleChanges",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_BUSINESS_ROLE_CHANGES"
            },
            {
                "attributeName": "exceptionChanges",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_EXCEPTION_CHANGES"
            },
            {
                "attributeName": "policies",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_POLICIES"
            },
            {
                "attributeName": "policyViolations",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_POLICY_VIOLATIONS"
            },
            {
                "attributeName": "policyNotifications",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_POLICY_NOTIFICATIONS"
            },
            {
                "attributeName": "scoresChanged",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_SCORES_CHANGED"
            },
            {
                "attributeName": "snapshotsCreated",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_SNAPSHOTS_CREATED"
            },
            {
                "attributeName": "scopesCreated",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_SCOPES_CREATED"
            },
            {
                "attributeName": "scopesCorrelated",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_SCOPES_CORRELATED"
            },
            {
                "attributeName": "scopesSelected",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_SCOPES_SELECTED"
            },
            {
                "attributeName": "scopesDormant",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_SCOPES_DORMANT"
            },
            {
                "attributeName": "unscopedIdentities",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_UNSCOPED_IDENTITIES"
            },
            {
                "attributeName": "certificationsCreated",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_CERTIFICATIONS_CREATED"
            },
            {
                "attributeName": "certificationsDeleted",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_CERTIFICATIONS_DELETED"
            },
            {
                "attributeName": "applicationsGenerated",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_APPLICATIONS_GENERATED"
            },
            {
                "attributeName": "managedAttributesCreated",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_MANAGED_ATTRIBUTES_PROMOTED"
            },
            {
                "attributeName": "managedAttributesCreatedByApplication",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_MANAGED_ATTRIBUTES_PROMOTED_BY_APP"
            },
            {
                "attributeName": "identityEntitlementsCreated",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_IDENTITYENTITLEMENTS_CREATED"
            },
            {
                "attributeName": "groupsCreated",
                "displayLabel": "TASK_OUT_ACCOUNT_AGGREGATION_GROUPS_CREATED"
            }
        ],
        "type": "QUARTZ"
    }
}

Ive waited about 15-20 mins

When I make the request to retrieve the account, I get an empty array in my code, Postman and using the api tool in the docs.

We have been waiting for a few months to get our partner IDN instance, so currently I dont have access to login to the GUI. We only have tokens provided by a client.

based on the response. Could I be missing something still?

Thanks again for all the help. This flying blind has been difficult.

So I took the task ID and ran the status API

API

get-account-aggregation-status

response

{
  "start": "2023-03-27T16:58:35.290328Z",
  "status": "COMPLETED",
  "totalAccounts": 0,
  "processedAccounts": 0,
  "totalAccountsMarkedForDeletion": 0,
  "deletedAccounts": 0,
  "totalIdentities": 0,
  "processedIdentities": 0
}

I guessing I may be missing something in the source setup?