We have setup the RACF connector on ISC to connect to RACF on 2980 port. Without enabling TLS we were able to do Test connection, account aggregation and entitlements aggregation. It suddenly stopped working since yesterday and we are seeing the following error message from UI.
**CG logs:[Redacted]**
2024-08-13T13:33:01,714 TRACE Req Handler sailpoint.router.server.ConnectorGateway sailpoint.router.server.ConnectorGateway:146 - Setting selectorTimeout to 7 seconds
2024-08-13T13:33:11,895 ERROR main sailpoint.router.sm.SMConnection sailpoint.router.sm.SMConnection:485 - No data received from Agent Remote Socket address Bytes read : 0
2024-08-13T13:33:11,895 ERROR main sailpoint.router.common.Main sailpoint.router.common.Main:111 - 2024/08/13 13:33:11 Connection with Agent broken.
2024-08-13T13:33:15,728 DEBUG Req Handler sailpoint.router.server.ConnectorGateway sailpoint.router.server.ConnectorGateway:224 - Stopping all active threads.
2024-08-13T13:33:18,896 TRACE main sailpoint.router.sm.SMConnection sailpoint.router.sm.SMConnection:251 - Connecting to host:X.X.X.X
2024-08-13T13:33:19,045 DEBUG main sailpoint.router.sm.SMConnection sailpoint.router.sm.SMConnection:329 - Socket initialized successfully.
2024-08-13T13:33:19,045 INFO main sailpoint.router.common.Main sailpoint.router.common.Main:89 - 2024/08/13 13:33:19 Connected to Agent.
2024-08-13T13:33:19,045 INFO main sailpoint.router.server.ConnectorGateway sailpoint.router.server.ConnectorGateway:61 - Thread pool is intialized with maxActiveTransactions: 10
2024-08-13T13:33:19,046 INFO main sailpoint.router.common.Main sailpoint.router.common.Main:94 - 2024/08/13 13:33:19 Connector Gateway initialized successfully.
2024-08-13T13:33:19,046 INFO main sailpoint.router.common.Main sailpoint.router.common.Main:95 - 2024/08/13 13:33:19 TLSParameters enableCGtoPE2TLS false | enablePE2TLSAuth false | enableCGtoAgentTLS false | enableCGTLSAuth false
2024-08-13T13:33:19,046 TRACE Req Handler sailpoint.router.server.ConnectorGateway sailpoint.router.server.ConnectorGateway:146 - Setting selectorTimeout to 7 seconds
2024-08-13T13:33:29,221 ERROR main sailpoint.router.sm.SMConnection sailpoint.router.sm.SMConnection:485 - No data received from Agent Remote Socket address Bytes read : 0
2024-08-13T13:33:29,222 ERROR main sailpoint.router.common.Main sailpoint.router.common.Main:111 - 2024/08/13 13:33:29 Connection with Agent broken.
2024-08-13T13:33:33,061 DEBUG Req Handler sailpoint.router.server.ConnectorGateway sailpoint.router.server.ConnectorGateway:224 - Stopping all active threads.
2024-08-13T13:33:36,223 TRACE main sailpoint.router.sm.SMConnection sailpoint.router.sm.SMConnection:251 - Connecting to host:X.X.X.X
2024-08-13T13:33:36,352 DEBUG main sailpoint.router.sm.SMConnection sailpoint.router.sm.SMConnection:329 - Socket initialized successfully.
2024-08-13T13:33:36,353 INFO main sailpoint.router.common.Main sailpoint.router.common.Main:89 - 2024/08/13 13:33:36 Connected to Agent.
2024-08-13T13:33:36,353 INFO main sailpoint.router.server.ConnectorGateway sailpoint.router.server.ConnectorGateway:61 - Thread pool is intialized with maxActiveTransactions: 10
2024-08-13T13:33:36,354 INFO main sailpoint.router.common.Main sailpoint.router.common.Main:94 - 2024/08/13 13:33:36 Connector Gateway initialized successfully.
2024-08-13T13:33:36,354 INFO main sailpoint.router.common.Main sailpoint.router.common.Main:95 - 2024/08/13 13:33:36 TLSParameters enableCGtoPE2TLS false | enablePE2TLSAuth false | enableCGtoAgentTLS false | enableCGTLSAuth false
2024-08-13T13:33:36,354 TRACE Req Handler sailpoint.router.server.ConnectorGateway sailpoint.router.server.ConnectorGateway:146 - Setting selectorTimeout to 7 seconds
**VA Logs:[Redacted]**
{"exception":{"stacktrace":"java.io.IOException: Connection with Connector Gateway broken.\n\tat sailpoint.connector.sm.ConnectSM.GetPartialMSG(ConnectSM.java:1126)\n\tat sailpoint.connector.sm.ConnectSM.Receive(ConnectSM.java:925)\n\tat sailpoint.connector.sm.SMUser.updateAdminPwd(SMUser.java:2889)\n\tat sailpoint.connector.sm.SMWrapper.updateAdminPwd(SMWrapper.java:456)\n\tat sailpoint.connector.sm.SMConnector.testConfiguration(SMConnector.java:3217)\n\tat sailpoint.connector.ConnectorProxy.testConfiguration(ConnectorProxy.java:470)\n\tat com.sailpoint.ccg.cloud.container.Container.testConnection(Container.java:431)\n\tat com.sailpoint.ccg.cloud.container.ContainerIntegration.ping(ContainerIntegration.java:73)\n\tat com.sailpoint.ccg.handler.TestConnectionHandler.invoke(TestConnectionHandler.java:31)\n\tat sailpoint.gateway.accessiq.CcgPipelineMessageHandler.handleMessage_aroundBody0(CcgPipelineMessageHandler.java:43)\n\tat sailpoint.gateway.accessiq.CcgPipelineMessageHandler$AjcClosure1.run(CcgPipelineMessageHandler.java:1)\n\tat org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167)\n\tat com.sailpoint.tracing.otel.TracedAspect.lambda$traceExecution$0(TracedAspect.java:38)\n\tat com.sailpoint.tracing.otel.GlobalTracer.trace(GlobalTracer.java:170)\n\tat com.sailpoint.tracing.otel.GlobalTracer.trace(GlobalTracer.java:143)\n\tat com.sailpoint.tracing.otel.TracedAspect.traceExecution(TracedAspect.java:40)\n\tat sailpoint.gateway.accessiq.CcgPipelineMessageHandler.handleMessage(CcgPipelineMessageHandler.java:36)\n\tat com.sailpoint.pipeline.server.PipelineServer$InboundQueueListener$MessageHandler.run(PipelineServer.java:379)\n\tat java.base\/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base\/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base\/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base\/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base\/java.lang.Thread.run(Thread.java:829)\n","exception_class":"java.io.IOException","exception_message":"Connection with Connector Gateway broken."}
{"exception":{"stacktrace":"sailpoint.connector.ConnectorException: [ ConnectionFailedException ] \n [ Possible suggestions ] Verify Connector Gateway and Mainframe Connector communication. \n [ Error details ] Connection issue with Connector Gateway.\n\tat com.sailpoint.ccg.cloud.container.Container.testConnection(Container.java:438)\n\tat com.sailpoint.ccg.cloud.container.ContainerIntegration.ping(ContainerIntegration.java:73)\n\tat com.sailpoint.ccg.handler.TestConnectionHandler.invoke(TestConnectionHandler.java:31)\n\tat sailpoint.gateway.accessiq.CcgPipelineMessageHandler.handleMessage_aroundBody0(CcgPipelineMessageHandler.java:43)\n\tat sailpoint.gateway.accessiq.CcgPipelineMessageHandler$AjcClosure1.run(CcgPipelineMessageHandler.java:1)\n\tat org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:167)\n\tat com.sailpoint.tracing.otel.TracedAspect.lambda$traceExecution$0(TracedAspect.java:38)\n\tat com.sailpoint.tracing.otel.GlobalTracer.trace(GlobalTracer.java:170)\n\tat com.sailpoint.tracing.otel.GlobalTracer.trace(GlobalTracer.java:143)\n\tat com.sailpoint.tracing.otel.TracedAspect.traceExecution(TracedAspect.java:40)\n\tat sailpoint.gateway.accessiq.CcgPipelineMessageHandler.handleMessage(CcgPipelineMessageHandler.java:36)\n\tat com.sailpoint.pipeline.server.PipelineServer$InboundQueueListener$MessageHandler.run(PipelineServer.java:379)\n\tat java.base\/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base\/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base\/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base\/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base\/java.lang.Thread.run(Thread.java:829)\nCaused by: sailpoint.connector.ConnectionFailedException: [ ConnectionFailedException ] \n [ Possible suggestions ] Verify Connector Gateway and Mainframe Connector communication. \n [ Error details ] Connection issue with Connector Gateway.\n\tat sailpoint.connector.sm.SMExceptionHandler.checkException(SMExceptionHandler.java:31)\n\tat sailpoint.connector.sm.SMConnector.testConfiguration(SMConnector.java:3232)\n\tat sailpoint.connector.ConnectorProxy.testConfiguration(ConnectorProxy.java:470)\n\tat com.sailpoint.ccg.cloud.container.Container.testConnection(Container.java:431)\n\t... 16 more\nCaused by: java.io.IOException: Connection with Connector Gateway broken.\n\tat sailpoint.connector.sm.ConnectSM.GetPartialMSG(ConnectSM.java:1126)\n\tat sailpoint.connector.sm.ConnectSM.Receive(ConnectSM.java:925)\n\tat sailpoint.connector.sm.SMUser.updateAdminPwd(SMUser.java:2889)\n\tat sailpoint.connector.sm.SMWrapper.updateAdminPwd(SMWrapper.java:456)\n\tat sailpoint.connector.sm.SMConnector.testConfiguration(SMConnector.java:3217)\n\t... 18 more\n","exception_class":"sailpoint.connector.ConnectorException","exception_message":"[ ConnectionFailedException ] \n [ Possible suggestions ] Verify Connector Gateway and Mainframe Connector communication. \n [ Error details ] Connection issue with Connector Gateway."}
**RACF Logs**
12.27.24 STC10184 ECAT00I CHANNEL(DCSA.TCP) TASK(003.R) LISTENING ON PORT(2471), IP(*), APPLICATION(S)
12.27.24 STC10184 ECAT00I CHANNEL(DCSA.TCP) TASK(002.S) LISTENING ON PORT(2470), IP(*), APPLICATION(S)
12.27.27 STC10184 ECAT02I CHANNEL(DCSA.TCP) TASK(003.R) ESTABLISHED CONNECTION WITH HOST(x.x.x.x), APPLICATION(S)
12.27.27 STC10184 ECAT02I CHANNEL(DCSA.TCP) TASK(002.S) ESTABLISHED CONNECTION WITH HOST(x.x.x.x), APPLICATION(S)
12.27.37 STC10184 ECAT05W CHANNEL(DCSA.TCP) TASK(003.R) DETECTED EVENT(ECONNRESET.43) STAGE(READ) API(READ) ACTION(CLOSE_REINIT)
12.27.37 STC10184 ECAT03I CHANNEL(DCSA.TCP) TASK(002.S) TERMINATED CONNECTION WITH HOST(x.x.x.x)
12.27.37 STC10184 ECAT03I CHANNEL(DCSA.TCP) TASK(003.R) TERMINATED CONNECTION WITH HOST(x.x.x.x)
12.27.41 STC10184 ECAT00I CHANNEL(DCSA.TCP) TASK(003.R) LISTENING ON PORT(2471), IP(*), APPLICATION(S)
12.27.42 STC10184 ECAT00I CHANNEL(DCSA.TCP) TASK(002.S) LISTENING ON PORT(2470), IP(*), APPLICATION(S)
12.27.44 STC10184 ECAT02I CHANNEL(DCSA.TCP) TASK(003.R) ESTABLISHED CONNECTION WITH HOST(x.x.x.x), APPLICATION(S)
12.27.44 STC10184 ECAT02I CHANNEL(DCSA.TCP) TASK(002.S) ESTABLISHED CONNECTION WITH HOST(x.x.x.x), APPLICATION(S)
12.27.54 STC10184 ECAT05W CHANNEL(DCSA.TCP) TASK(003.R) DETECTED EVENT(ECONNRESET.43) STAGE(READ) API(READ) ACTION(CLOSE_REINIT)
12.27.54 STC10184 ECAT03I CHANNEL(DCSA.TCP) TASK(002.S) TERMINATED CONNECTION WITH HOST(x.x.x.x)
12.27.54 STC10184 ECAT03I CHANNEL(DCSA.TCP) TASK(003.R) TERMINATED CONNECTION WITH HOST(x.x.x.x)
12.27.58 STC10184 ECAT00I CHANNEL(DCSA.TCP) TASK(003.R) LISTENING ON PORT(2471), IP(*), APPLICATION(S)
12.27.59 STC10184 ECAT00I CHANNEL(DCSA.TCP) TASK(002.S) LISTENING ON PORT(2470), IP(*), APPLICATION(S)
12.28.01 STC10184 ECAT02I CHANNEL(DCSA.TCP) TASK(003.R) ESTABLISHED CONNECTION WITH HOST(x.x.x.x), APPLICATION(S)
12.28.01 STC10184 ECAT02I CHANNEL(DCSA.TCP) TASK(002.S) ESTABLISHED CONNECTION WITH HOST(x.x.x.x), APPLICATION(S)
12.28.11 STC10184 ECAT05W CHANNEL(DCSA.TCP) TASK(003.R) DETECTED EVENT(ECONNRESET.43) STAGE(READ) API(READ) ACTION(CLOSE_REINIT)
12.28.11 STC10184 ECAT03I CHANNEL(DCSA.TCP) TASK(002.S) TERMINATED CONNECTION WITH HOST(x.x.x.x)
12.28.12 STC10184 ECAT03I CHANNEL(DCSA.TCP) TASK(003.R) TERMINATED CONNECTION WITH HOST(x.x.x.x)
12.28.16 STC10184 ECAT00I CHANNEL(DCSA.TCP) TASK(003.R) LISTENING ON PORT(2471), IP(*), APPLICATION(S)
12.28.16 STC10184 ECAT00I CHANNEL(DCSA.TCP) TASK(002.S) LISTENING ON PORT(2470), IP(*), APPLICATION(S)
12.28.18 STC10184 ECAT02I CHANNEL(DCSA.TCP) TASK(003.R) ESTABLISHED CONNECTION WITH HOST(x.x.x.x), APPLICATION(S)
12.28.19 STC10184 ECAT02I CHANNEL(DCSA.TCP) TASK(002.S) ESTABLISHED CONNECTION WITH HOST(x.x.x.x)), APPLICATION(S)
Please help us in resolving this issue.