How to onboard AD Application in ISC - local lab setup running entirely on VMware Workstation on a single Windows host machine:

Identity Security Cloud (ISC) ISC Discussion and Questions
1

Title: IQService Test Connection fails with “No matching certificate found” even after self-signed cert is created with correct subject (Local VMware lab setup)

Environment / Setup:

This is a local lab setup running entirely on VMware Workstation on a single Windows host machine:

  • VA (Virtual Appliance): deployed locally on VMware Workstation, paired with my ISC tenant and showing Connected in the cluster

  • AD + IQService server: Windows Server 2016 VM running on the same VMware Workstation host (not in cloud, not in a corporate network)

  • Both VMs are on the same VMware NAT network (192.168.75.x)

  • VA IP: 192.168.75.143

  • AD VM IP: 192.168.75.147 (static)

  • AD domain: boa.com, hostname: WIN-TH0V3NI5FQ4

  • Domain controller, IQService, and Windows Firewall all configured on the same VM

  • IQService: installed in TLS mode using IQService.exe -i -o 5050

Issue:

When testing the AD source connection from the ISC UI, I get:

Timeout waiting for response to message 7 from client f13a3f04-... after 15 seconds

IQTrace.log on the IQService server shows:

RpcHandler [ Thread-6 ] ERROR : "An Exception occurred while accepting new client request
System.Exception: No matching certificate found for WIN-TH0V3NI5FQ4.boa.com
   at sailpoint.rpcserver.RpcHandler..ctor(Hashtable services, Hashtable registry, 
      TcpClient client, String port, Boolean useTLS, String subject, String tlsVersion, 
      String registeredClients, String serialNumber)"

What I have verified:

  1. Network connectivity from VA → IQService server is fine (both VMs on same VMware NAT subnet):

    • nc -zv 192.168.75.147 5050 → succeeded
    • nc -zv 192.168.75.147 389 → succeeded
    • nc -zv 192.168.75.147 636 → succeeded

  2. Windows Firewall rule for inbound TCP 5050 is in place on the AD VM.

  3. IQService is running and listening:

    netstat -an | findstr 5050
  TCP    0.0.0.0:5050    0.0.0.0:0    LISTENING
  TCP    [::]:5050       [::]:0       LISTENING

  4. Self-signed cert created with the exact subject IQService is asking for:

    New-SelfSignedCertificate `
  -Subject "CN=WIN-TH0V3NI5FQ4.boa.com" `
  -DnsName "WIN-TH0V3NI5FQ4.boa.com", "WIN-TH0V3NI5FQ4", "192.168.75.147" `
  -CertStoreLocation "Cert:\LocalMachine\My" `
  -KeyAlgorithm RSA -KeyLength 2048 `
  -NotAfter (Get-Date).AddYears(5) `
  -KeyUsage DigitalSignature, KeyEncipherment `
  -KeyExportPolicy Exportable `
  -FriendlyName "IQService TLS Cert"

    Result:

    Subject       : CN=WIN-TH0V3NI5FQ4.boa.com
Thumbprint    : <40-char hex>
HasPrivateKey : True
NotAfter      : 4/26/2031

  5. IQService was restarted after cert creation (confirmed via Application event log timestamp).

  6. NETWORK SERVICE permission granted on the private key folder via icacls.

  7. In ISC source config (IQService Settings):

    • IQService Host: 192.168.75.147
    • IQService Port: 5050
    • Use TLS: :check_box_with_check: Checked
    • IQService User / Password: blank

Despite all of the above, the same “No matching certificate found” error continues to appear in IQTrace.log on every Test Connection attempt.

Questions:

  1. Where exactly does IQService look for the cert? Does it search only LocalMachine\My, or does it also require the cert in Trusted Root Certification Authorities?

  2. Are there additional cert requirements beyond Subject CN match? For example:

    • Specific Enhanced Key Usage (Server Authentication OID 1.3.6.1.5.5.7.3.1)?
    • Specific Subject Alternative Names format?
    • Cert must be signed by a real CA (not self-signed)?

  3. Is there a SailPoint-provided script bundled with IQService for generating TLS certs in the exact expected format? I checked the install folder and didn’t find one.

  4. Does the cert subject need to match the machine FQDN exactly, the NetBIOS name, or some other format specific to the IQService instance?

  5. For a lab/local-VMware test setup, is there a supported way to disable TLS enforcement on IQService and run on plain TCP/5050? (For real client work I’d of course use proper TLS — just want to validate end-to-end flow first.)

Any pointers on what specifically IQService matches against during cert lookup would be very helpful. I’ve reviewed the official TLS configuration documentation but want to confirm the exact cert spec for this IQService version.

Thanks in advance!

2

As it is lab. Can you disable TLS? and see the flow.

3

Hello Sathish,

Based on my research & looking at your logs, this isn’t a connectivity issue. VA is reaching 5050. The failure is happening during IQService certificate lookup.

No matching certificate found for WIN-TH0V3NI5FQ4.boa.com

A few things to fix:

  1. Don’t use IP for IQService Host when TLS is enabled. Use the FQDN:
WIN-TH0V3NI5FQ4.boa.com

The cert CN/SAN must match this, and the VA must be able to resolve it.

  1. Check the certificate — most likely gap here is EKU:

    • Installed under Local Computer > Personal ✓

    • Has a private key ✓

    • Has Server Authentication EKU ← check this

    • Add it to Trusted Root (since it’s self-signed)

  2. Make sure VA trusts the cert:

    • Place the cert/root cert under:
/home/sailpoint/certificates
  • Restart CCG

I would start with FQDN + EKU. This error usually comes down to one of those. Try these & plz let me know if that resolves your issue.

4

In SailPoint Identity Security Cloud:

• Where does IQService look for the cert?
It looks in LocalMachine\My (Personal) store. Trusted Root is not required for selection, but adding it there can help with trust in some setups.

• Additional cert requirements?
Yes — important ones:

  • Must include Server Authentication EKU (1.3.6.1.5.5.7.3.1)

  • Must have a private key

  • SAN is recommended (FQDN/hostname), but not strictly enforced in all cases

• CA-signed vs self-signed?
Self-signed works fine for lab/testing. CA-signed is recommended for production.

• Any SailPoint script for cert creation?
No, there’s no built-in script — standard Windows cert tools are typically used.

• Subject name requirement?
Should match the machine FQDN (preferred). NetBIOS may work in some cases, but FQDN is the safest.

• Can TLS be disabled for lab?
Yes — you can install/run IQService without TLS (plain TCP/5050) for testing purposes.