Snowflake & JDBC Connector

Hi,
I’m setting up a JDBC source to integrate to a Snowflake data warehouse for access controls.

Is there a way for IDN to support multiple statements in one query?

For background:

There’s some technical limitations around Snowflake’s stored procedures (only ever returns 1 value) and user controls I’m trying to work around.

What I’m currently stuck on is the single account query that’s performed after each provisioning operation - this fails since the stored procedure only returns a single value. As a work-around, I’m trying to use the Snowflake built-in “show grants…” (this is the quickest and most up to date method) to expose the roles assigned to each user.
However, I also need to modify this data so the response would be what Sailpoint expects. “show grants…” doesn’t support being used in subquery/nested query so I have to send 2 queries: first the “show grants…”, followed by a

select “columnA” as account, “columnB” as entitlement from table(RESULT_SCAN(LAST_QUERY_ID()))

Hello @M_rtenH,

I do not have a Snowflake instance to test but I tested with an Azure SQL Database and the corresponding JDBC driver.

In short: yes it works

However, there are a few things to note:

  • Using the result of a stored procedure in a select is not possible. Looking into this, I found a solution using a temp table
  • I had an issue with multiple statements in the same query. Therefore, I used another trick to force the result set (see below)

The query I used to:

  • create the temp table
  • insert the result of the stored procedure GetUsers
  • join the temp table with other tables
IF OBJECT_ID(N'tempdb..#tempusers`') IS NOT NULL
BEGIN
DROP TABLE #tempusers;
END;
create table #tempusers([id] [int] NOT NULL,
	[login] [varchar](30) NOT NULL,
	[description] [varchar](1024) NULL,
	[first] [varchar](128) NULL,
	[last] [varchar](128) NULL,
	[password] [varchar](26) NULL,
	[status] [char](1) NULL,
	[locked] [char](1) NULL,
	[lastLogin] [date] NULL);
insert into #tempusers exec [dbo].[GetUsers] ;

select t.id, t.[login],t.[first], t.[last]
, groups.name as groups
from #tempusers t left join dbo.users_groups g on g.user_id=t.id left join dbo.groups on groups.id=g.group_id order by t.id;

Wrapping all this in an EXEC (beware to quotes) and forcing the result set:

EXEC('
IF OBJECT_ID(N''tempdb..#tempusers`'') IS NOT NULL
BEGIN
DROP TABLE #tempusers;
END;
create table #tempusers([id] [int] NOT NULL,
	[login] [varchar](30) NOT NULL,
	[description] [varchar](1024) NULL,
	[first] [varchar](128) NULL,
	[last] [varchar](128) NULL,
	[password] [varchar](26) NULL,
	[status] [char](1) NULL,
	[locked] [char](1) NULL,
	[lastLogin] [date] NULL);
insert into #tempusers exec [dbo].[GetUsers] ;


select t.id, t.[login],t.[first], t.[last]
, groups.name as groups
from #tempusers t left join dbo.users_groups g on g.user_id=t.id left join dbo.groups on groups.id=g.group_id order by t.id;')
WITH RESULT SETS (
    (
    [id] [int], 
    [login] [varchar](30),
    [first] [varchar](128) ,
    [last] [varchar](128),
    [groups] [varchar](30)
    )
)

I was able to use this last query in IDN as the “Account SQL Query” in the config of my source and it worked!

Hope this helps!

1 Like

Hi Yannick,

This looks promising, thanks a lot!

Snowflake commands and syntax is somewhat different so I’ll play around with this and see if I can replicate what you’re doing in your example.

Will let you know how I get on.

So a small update - I’ve got a working query (ended up finding a workaround on Snowflake’s side while digging through their documentation) which, according to Snowflake, returns the expected data.

Now the problem is that on Sailpoint’s side I get a

ConnectorException: Identity attribute [ACCOUNT] was not found.

Looking at the DB response - the expected data should be there, 2 columns with names ‘ACCOUNT’ and 'ENTITLEMENT 'are returned which matches the account schema. I’m not sure if it can’t find the identity attribute due to data types/formatting or something else?

Hello @M_rtenH
Did you update the schema of the connector with those attributes?

Hi Yannick,

I’m not entirely sure - I’ve followed the JDBC Source documentation to set this up, I don’t see any mention of “connector schema”.

Is this configured separately from the account/group schemas? (which include the necessary attributes)

Sorry, I did not read carefully your previous answer.
I understand your “big” query returns only 2 columns. That does not seems much.
I would expect something like:

  • Id, as the internal, permanent and unique Id of a user in Snowflake (maybe optional. I don’t know if one is created for a user)
  • login, as the name
  • other basic identity attributes (firstname, lastname, email)
  • ENTITLEMENT, as a multivalue reference to entitlements

Is it what you have?

Yes, in the current iteration we’re only exposing 2 values - ACCOUNT (which is the users email address which is the same as their Sailpoint/Okta username) and ENTITLEMENT (users permissions in Snowflake). ACCOUNT attribute is treated as displayName and accountID.

We’ll include a status field as well but are mostly concerned with getting the basics to work at the moment.

Could you copy paste your query (the content of the “Aggregate Account Query” field)?
I will try to create a Snowflake test instance

Sure, this is the Account SQL Query:

SELECT ACCOUNT, ENTITLEMENT FROM SAILPOINT_INTEGRATION.REPORT_ACCOUNTS_ENTITLEMENTS;

and this is the single account query:

show grants to role “$(identity)”; select “grantee_name” as ACCOUNT, “name” as ENTITLEMENT from table(RESULT_SCAN(LAST_QUERY_ID())) WHERE “granted_on” = ‘ROLE’ ;

I will likely update the account aggregation query at some point - the current “table” it queries is a VIEW built from other tables to filter to only relevant results but the query is slow(~30s) & results have up to 2h lag. The single account query is real time and takes ~400ms to execute. But the output of the query will be the same.

And just to be sure we’re on the same page - the current issue is with the single account query that returns ‘ConnectorException: Identity attribute [ACCOUNT] was not found.’, other operations work fine.

As described in Why does Snowflake LAST_QUERY_ID returns NULL? - Stack Overflow, using LAST_QUERY_ID() seems to be problematic in JDBC.
My test with JDBC (without IdentityNow) confirms that this kind of request does not work.

Hmm, that’s interesting.
The issue I had at first is that Snowflake by default limits amount of statements in a query to 1 to protect against SQL injection but this can be changed by passing multi-statement-count=0 in the connection url.

If I look at query history, I can see that both statements are successfully executed (both the “show grants…” and “…(RESULT_SCAN(LAST_QUERY_ID()))” and the latter query returns 1 row as expected. I can also run the same query directly in the DB and it returns the expected result. I would assume that if LAST_QUERY_ID() were to return a NULL, the second query that tries to scan the results of that query would fail.

I have tried a different single account query as well -

SELECT ACCOUNT, ENTITLEMENT FROM SAILPOINT_INTEGRATION.REPORT_ACCOUNTS_ENTITLEMENTS WHERE ACCOUNT = $(identity)

but this has the same issue, ‘ConnectorException: Identity attribute [ACCOUNT] was not found.’ despite Snowflake query history showing 1 returned row.

REPORT_ACCOUNTS_ENTITLEMENTS is not part of the default views.
As I do not have the definition of your view REPORT_ACCOUNTS_ENTITLEMENTS, I cannot test both queries.
Can you provide me the view definition?

CREATE OR REPLACE VIEW SAILPOINT_INTEGRATION.REPORT_ACCOUNTS_ENTITLEMENTS AS
SELECT
USERS.name AS account,
GRANTS_TO_ROLES.name AS entitlement
FROM “SNOWFLAKE”.“ACCOUNT_USAGE”.“USERS” USERS
LEFT JOIN “SNOWFLAKE”.“ACCOUNT_USAGE”.“GRANTS_TO_ROLES” GRANTS_TO_ROLES ON (GRANTS_TO_ROLES.GRANTEE_NAME = USERS.name)
WHERE USERS.deleted_on IS null
AND GRANTS_TO_ROLES.deleted_on IS null
ORDER BY account, entitlement
;

I’ve removed some WHERE x is NULL clauses since they don’t contribute to explaining the view structure.

If the issue isn’t with the query (I can execute the queries in Snowflake and I can see them successfully being executed via the JDBC driver as well), where else should I look?

I have something working now.
So after creating a database SAILPOINT_INTEGRATION and the view REPORT_ACCOUNTS_ENTITLEMENTS in the public schema, on IdentityNow, I have created a JDBC source:

  1. I used Snowflake JDBC Driver from and upload to the source
  2. I have configured :
    • VA Cluster
    • Username
    • Password
    • JDBC URL (jdbc:snowflake://<account_identifier>.snowflakecomputing.com/
    • JDBC Driver: net.snowflake.client.jdbc.SnowflakeDriver
    • Test Connection SQL Query: show databases
    • Account Query Settings: SELECT ACCOUNT, ENTITLEMENT FROM SAILPOINT_INTEGRATION.public.REPORT_ACCOUNTS_ENTITLEMENTS
    • Single Account SQL Query : SELECT ACCOUNT, ENTITLEMENT FROM SAILPOINT_INTEGRATION.public.REPORT_ACCOUNTS_ENTITLEMENTS WHERE ACCOUNT = '$(identity)'
  3. Save and test connection → Should work
  4. Go to “Import Data > Account Schema” and add attributes to suit the query:
    • ACCOUNT → set as account ID and name
    • ENTITLEMENT → set as multivalued and entitlement

I had some issue like:

ConnectorException ] \n [ Error details ] The server encountered an unexpected error while contacting target system. Please check the logs. ResultSet type 1004 is not supported.

To past this issue, I needed to update the source configuration through the API with a PATCH to {{api-url}}/v3/sources/:sourceid and the payload:

[
    {
        "op": "add",
        "path": "/connectorAttributes/resultSetType",
        "value": "TYPE_FORWARD_ONLY"
    }
]

After that, the aggregation worked.

Thanks for going to great lengths in helping me with this, much appreciated.

Re ResultSet type 1004 - I had the same issue originally and applied the same fix.

So it looks like our set-up is nearly identical but for some reason, single account query fails for me.

Is there anything else you can think of I might’ve skipped during configuration or should I check cgg.log to see what exactly is the response Sailpoint gets and why it can’t find the [ACCOUNT] attribute ONLY during single account aggregation?

From my point of view, single account aggregation is not that important.
The most important is that your “full” account aggregation works.
In any case, can you review your query?
Compared to your query:

  • Mine specified the schema of the view
  • $(identity) is placed between single-quotes

Full account aggregation has been working all along. When you say single account aggregation is not that important, can this be skipped? I understand the use-case for this is to confirm that the provisioning operation was actually successful?

SELECT ACCOUNT, ENTITLEMENT FROM SAILPOINT_INTEGRATION.REPORT_ACCOUNTS_ENTITLEMENTS WHERE ACCOUNT = ‘$(identity)’;

That’s my current single account query and I still get the …attribute [ACCOUNT] was not found.

Is SAILPOINT_INTEGRATION your schema?
Did you specify the database and/or the schema in the attributes of the connection string?

In Snowflake, select * from "SAILPOINT_INTEGRATION"."REPORT_ACCOUNTS_ENTITLEMENTS" does not work, but select * from "SAILPOINT_INTEGRATION"."PUBLIC"."REPORT_ACCOUNTS_ENTITLEMENTS" does.

Yes, that’s the schema. The database and warehouse are specified in the connection URL.

The query works - I can see that it’s sent to Snowflake from Sailpoint via the JDBC driver (which is expected), it’s executed and it returns 1 row, which is also expected. I can also run either of the queries (SELECT ACCOUNT, ENTITLEMENT… or the one using show grants… and last_query_id()) directly in Snowflake and get the expected results. If the query didn’t work either via JDBC or directly in Snowflake, it would either fail with an appropriate status code or not return any rows.

The only part that’s failing is IDN reading the results of that query and mapping it back to the single identity that was aggregated. If you’re saying the single account query is not important perhaps I should just skip it at this point and trust that the provisioning operations will always be successful?