Request timeout error while saving an access review

[milinaphalke]

Which IIQ version are you inquiring about?

IIQ8.4p2

Please share any images or screenshots, if relevant.

Please share any other relevant files that may be required (for example, logs).

[Please insert files here, otherwise delete this section]

Share all details about your problem, including any error messages you may have received.

While trying to save an Access Review decision, getting request timeout error.
This is faced by many endusers. Any pointers what could be the issue ?

[pattabhi (Pattabhi Nandarapu)]

Hi @milinaphalke

I would like to ask for more details.

Could you please provide the application server name and database name?

Additionally, please get the stack trace details from the application logs during the UI error timeframe. (You can find the main IdentityIQ log files on your application server, usually identityiq.log or catalina.out if you’re running on Tomcat.)

How many application servers do you have, especially if this is a Production environment? (Do you have designated UI servers and Task servers?)

Lastly, could you provide information on the volume of data, such as the number of line items, certifiers, or how many concurrent users were logged in?

[milinaphalke]

Thanks @pattabhi for looking into this issue.
Application server - tomcat
Database - SQL server
We could not see any specific error during the timeout error that shows up in SailPoint logs. However we will still monitor.
There are servers - 2 UI and 2 Task servers.
Approximately 20000 users have access to SailPoint.
From login audit approx 100 to 160 users login through out the business hours.
There were overall 56000+ certification items.
Few users have also reported the issue during approving access requests.
So might not be specific to access review.

[pravin_ranjan (Kumar Ranjan)]

@milinaphalke if possible pls share the catalina.sh, just want to see the memory configuration.

[pattabhi (Pattabhi Nandarapu)]

Please check below parameters & pointers, hope these might help you with respect to timeout error.

Just giving some trouble shooting points like where and what needs to be looked into.

1. Tomcat: connectionTimeout

From UI servers, please check the connectionTimeout="XXXXX" from server.xml from location: <Tomcat_Installation_Directory>/conf/server.xml

2. Designated Servers:

You have 2 UI and 2 Task servers. Ensure your Server.xml and ServiceDefinition objects (especially for Request and Task services) are correctly configured, from debug pages(object browser) – > ServiceDefinition – check Request and Task details

3. Insufficient JVM Memory:

search for OutOfMemoryError in the application logs which will be an indication.

4. Database Connection Pool:

from iiq.properties check the details from below section:

##### Data Source Properties #####

Database Health and Performance: – check with DBA team.

I’ll try to replicate the scenario and see if I can replicate the issue.

[milinaphalke]

Thanks @pravin_ranjan for looking into this.
I can see the following configuration in catalina.bat file

[milinaphalke]

Thanks @pattabhi
I will check these recommendations.

[pattabhi (Pattabhi Nandarapu)]

Hi @milinaphalke

After changing these values, try to restart the application servers as instructed below.

Recommended values to give a try.

##### Data Source Properties #####
dataSource.maxWaitMillis=30000
dataSource.maxTotal=250
dataSource.minIdle=5
dataSource.minEvictableIdleTimeMillis=3000000
#dataSource.maxOpenPreparedStatements=-1

dataSource.maxTotal – this can be up to 500- for UI servers as per document reference:
IIQ Properties Settings - Compass

Controls the number of connections that can be attached to the database at one time; the default value is 50, though higher values are usually specified for production installations.

This setting is typically higher for UI servers than for task servers, and often ranges from 200-500. The max HTTP sessions in the application server also may need to be adjusted, and the database server must be configured to support all connections from all clients as well.

Installations taking advantage of the partitioning and parallelized aggregation features available in releases 6.2 and newer should consider increasing this number to 100 or 200 as a minimum for batch and task servers. Servers performing aggregations using partitioning use more database connections than servers performing single threaded aggregations.

Instructions:

1. Stop Tomcat.
2. Update iiq.properties file as recommended above.
3. Delete the temp and work folders to clear the cache.

Note: You MUST stop the Tomcat server before deleting these folders content. While the temp and work folders in Tomcat can generally be deleted safely, stopping the server beforehand is crucial to prevent corruption or issues.

Please refer SailPoint best practices while updating these values.

4. Start Tomcat.

Expected values in UI: from Object Browser – > Wrench Icon – Database

[milinaphalke]

Thanks @pattabhi will check.