Programmatically Adding a Single Role to the Full Text Index in IIQ

Introduction

This guide is for implementing a single role addition to the Role Full Text Index programmatically in IdentityIQ. The Role Full Text Index is a feature that allows you to search for roles in the system based on the role name. This can be helpful when you want dynamically created roles to be instantly searchable in the Manage User Access. This guide also contains an example of how to use a workflow to allow users to create bundles using an interactive form called from a QuickLink.

Use Case

The requirement was to develop a QuickLink workflow enabling users to manage Azure groups, including their creation and deletion. This involved designing a workflow where users could input data, leading to the creation of a group in Azure via a provisioning plan. Subsequently, a bundle object was created with the profile set to that entitlement’s native identity. However, a challenge arose because the newly created role was not immediately searchable or manageable due to it not being indexed. Initiating a full indexing process for every user-created role was not viable due to time constraints. The solution involved decompiling IIQ classes to understand their usage and how roles were indexed. I found that an Apache library facilitated the manipulation of index files, which are stored in the IIQs Tomcat directory and can be read and written to. By opening these files programmatically, it was possible to add the newly created bundle to the index. After saving, the role became searchable, allowing users to instantly create roles and assign users to them.

Implementation

  1. Prepare the RequestDefinition
  2. Prepare the RequestExecutor
  3. Prepare the Textifier
  4. Add it to a workflow

In order to prepare the single refresh, we need to have the following components ready. The first one is the request definition and its request executor, which will execute the request to index the bundle on each host that is available because the file is separate for each server. The next one is the textifier itself to index the roles. Then it can be added to a workflow.

RequestDefinition

This is the request definition that will be used to create the execution requests:

<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE RequestDefinition PUBLIC "sailpoint.dtd" "sailpoint.dtd">
<RequestDefinition
    executor="com.ventum.iiq.indexer.FullTextSingleIndexer"
    name="Full Text Single Refresh"
    retryMax="3"
    retryInterval="10"
/>
Field Description
executor A class that extends the sailpoint.request.AbstractRequestExecutor class
name An arbitrary name
retryMax The maximal number of time the request will be retried
retryInterval The time between the retries

Request Executor

This is the request executor class that will be used to execute the request. We will break it down into methods to make it easier to understand.

runOnAllServers
This method will be used to execute the request on all available servers. It will iterate over all the servers and execute the request on each one.

public static void runOnAllServers(SailPointContext context, List<String> bundlesToAdd) throws GeneralException {
    RequestDefinition requestDefinition = context.getObjectByName(RequestDefinition.class, "Full Text Single Refresh");
    
    if (requestDefinition == null) {
        throw new GeneralException("Missing request definition!");
    }
    
    List<Server> servers = context.getObjects(Server.class);
    
    for (Server server : servers) {
        if (server.isInactive()) {
            continue;
        }
        
        Request request = new Request();
        request.setName("FullTextSingle: " + server.getName());
        request.setDefinition(requestDefinition);
        request.setHost(server.getName());
        request.put("bundlesToAdd", bundlesToAdd);
        
        RequestManager.addRequest(context, request);
    }
}

The runOnAllServers method iterates over all servers obtained from the SailPoint context, creating and submitting a request for each active server with the specified bundles to add. It skips inactive servers and throws an exception if the required request definition is not found.

execute
This method will be used to execute the request on a single server. It will read the bundles to add from the request and execute the indexing process.

Note: The log variable is provided by the class.

@Override
public void execute(SailPointContext context, Request request, Attributes<String, Object> attributes) throws RequestPermanentException, RequestTemporaryException {
    // Get the current request definition        
    RequestDefinition requestDefinition = request.getDefinition();
    
    // Get the maximum number of retries and the retry interval from the request definition
    int maxRetries    = requestDefinition.getRetryMax();
    int retryInterval = requestDefinition.getRetryInterval();
    
    try {
        boolean retry      = false;
        int     retryCount = 0;
        
        List<String> bundlesToAdd = (List<String>) attributes.get("bundlesToAdd");
        
        if (!_terminate) {
            do {
                // Create the textifier and index the bundles
                try (SingleTextifier textifier = new SingleTextifier(sailPointContext)) {
                    // Iterate over the bundles to add
                    for (String bundleName : bundlesToAdd) {
                        // Check if the request was terminated
                        if (_terminate) {
                            break;
                        }
                        
                        // Index the bundle
                        textifier.index(bundleName);
                    }
                    
                    log.info(String.format("Indexing on '%s' completed", Util.getHostName()));
                } catch (IOException e) {
                    log.warn("Could not creat the SingleTextifier!", e);
                    
                    // Check if the maximum number of retries has been reached
                    if (retryCount++ >= maxRetries) {
                        throw new RequestPermanentException(new Message(Message.Type.Error, "Could not create the SingleTextifier", e.getMessage()));
                    }
                    
                    log.warn("Retrying in " + retryInterval + " seconds");
                    
                    // Sleep for the retry interval.
                    // Request execute in their own thread, so this will not block the other requests.
                    Thread.sleep(retryInterval * 1000L);
                    retry = true;
                }
            } while (retry && !_terminate);
            
        } else {
            log.info("Request terminated");
        }
    } catch (Exception e) {
        log.error("Error while indexing bundles", e);
    }
}

The execute method is responsible for executing the queued request. It retrieves configuration from the request definition, such as maximum retries and retry interval, and attempts to index each bundle provided in the request’s attributes. It also retrieves the bundle names passed to the request to index. If indexing fails due to an IOException, which could occur if another user requested the role creation at the same time, it retries the operation based on the retry parameters. The process is terminated if the _terminate flag is set, ensuring the method can gracefully stop its operation upon request.

Note: The execute method uses the SingleTextifier class, which we will discuss in the next section

Full Class Example

package com.ventum.iiq.indexer;

import org.apache.log4j.LogManager;
import org.apache.log4j.Logger;
import sailpoint.api.RequestManager;
import sailpoint.api.SailPointContext;
import sailpoint.object.*;
import sailpoint.request.AbstractRequestExecutor;
import sailpoint.request.RequestPermanentException;
import sailpoint.request.RequestTemporaryException;
import sailpoint.tools.GeneralException;
import sailpoint.tools.Message;
import sailpoint.tools.Util;

import java.io.IOException;
import java.util.List;


public class FullTextSingleIndexer extends AbstractRequestExecutor {
    public static final String REQUEST_DEFINITION_NAME = "Full Text Single Refresh";
    
    public static final String KEY_BUNDLES_TO_ADD = "bundlesToAdd";
    
    private static final Logger log = LogManager.getLogger(FullTextSingleIndexer.class);
    
    private boolean _terminate = false;
    
    public FullTextSingleIndexer() {
    }
    
    
    @Override
    @SuppressWarnings({"BusyWait", "unchecked"})
    public void execute(SailPointContext context, Request request, Attributes<String, Object> attributes) throws RequestPermanentException, RequestTemporaryException {
        
        RequestDefinition requestDefinition = request.getDefinition();
        
        int maxRetries    = requestDefinition.getRetryMax();
        int retryInterval = requestDefinition.getRetryInterval();
        
        try {
            boolean retry      = false;
            int     retryCount = 0;
            
            List<String> bundlesToAdd = (List<String>) attributes.get(KEY_BUNDLES_TO_ADD);
            
            if (!_terminate) {
                do {
                    try (SingleTextifier textifier = new SingleTextifier(context)) {
                        for (String bundleName : bundlesToAdd) {
                            if (_terminate) {
                                break;
                            }
                            
                            textifier.index(bundleName);
                        }
                        
                        log.info(String.format("Indexing on '%s' completed", Util.getHostName()));
                    } catch (IOException e) {
                        log.warn("Could not create the SingleTextifier!", e);
                        
                        if (retryCount++ >= maxRetries) {
                            throw new RequestPermanentException(new Message(Message.Type.Error, "Could not create the SingleTextifier", e.getMessage()));
                        }
                        
                        log.warn("Retrying in " + retryInterval + " seconds");
                        Thread.sleep(retryInterval * 1000L);
                        retry = true;
                    }
                } while (retry && !_terminate);
                
            } else {
                log.info("Request terminated");
            }
        } catch (Exception e) {
            log.error("Error while indexing bundles", e);
        }
    }
    
    @Override
    public boolean terminate() {
        _terminate = true;
        
        return true;
    }
    
    public static void runOnAllServers(SailPointContext context, List<String> bundlesToAdd) throws GeneralException {
        RequestDefinition requestDefinition = context.getObjectByName(RequestDefinition.class, REQUEST_DEFINITION_NAME);
        
        if (requestDefinition == null) {
            throw new GeneralException("Missing request definition: " + REQUEST_DEFINITION_NAME);
        }
        
        List<Server> servers = context.getObjects(Server.class);
        
        for (Server server : servers) {
            if (server.isInactive()) {
                continue;
            }
            
            Request request = new Request();
            request.setName("FullTextSingle: " + server.getName());
            request.setDefinition(requestDefinition);
            request.setHost(server.getName());
            request.put(KEY_BUNDLES_TO_ADD, bundlesToAdd);
            
            RequestManager.addRequest(context, request);
        }
        
        
    }
}

Single Textifier

This class is responsible for indexing the roles. It will read the index file, add the new role, and save the file. Again, we will break it down into methods to make it easier to understand.

Constructor
Constructor of the class. It will initialize the class with the SailPoint context, the indexer and the index writer.

public SingleTextifier(SailPointContext context) throws GeneralException, IOException {
    this.context     = context;
    this.indexer     = getIndexer(getIndex());
    this.indexWriter = getIndexWriter(getIndex());
}

getIndex
This method simply returns the FullTextIndex object using getObjectByName.

private FullTextIndex getIndex() throws GeneralException {
    FullTextIndex index = context.getObjectByName(FullTextIndex.class, "BundleManagedAttribute");
    
    if (index == null) {
        throw new RuntimeException("Index not found");
    }
    
    return index;
}

getIndexer
This method reuses the out-of-the-box IIQ methods to create an instance of the BundleIndexer. Since I only need this class to create the Doc object, and I want to handle writing myself, I use reflection to only set the fields I need for this use case. The reason for wanting to handle writing myself is that OOTB SailPoint classes open the index file in CREATE mode, not CREATE_OR_APPEND, which is needed here.

private AbstractIndexer getIndexer(FullTextIndex index) {
    AbstractIndexer indexer = new BundleIndexer();
    
    Builder builder = new Builder(context, null, null);
    
    try {
        RefUtil.setFieldValue(builder, "_index", index);
        indexer.prepare(builder);
    } catch (NoSuchFieldException | IllegalAccessException | GeneralException e) {
        throw new RuntimeException(e);
    }
    
    indexer.setAddObjectClass(true);
    
    return indexer;
}

Inner Class - RefUtil
The RefUtil class provides utility methods for reflection operations, specifically for setting field values on objects. It contains two methods: setFieldValue and getField. The setFieldValue method allows setting the value of a specified field on an object, even if the field is private or protected, by making it accessible. The getField method retrieves a Field object for a given field name, searching through the class hierarchy if necessary.

private static class RefUtil {
    @SuppressWarnings("UnusedReturnValue")
    public static Field setFieldValue(Object object, String fieldName, Object valueTobeSet) throws NoSuchFieldException, IllegalAccessException {
        Field field = getField(object.getClass(), fieldName);
        field.setAccessible(true);
        field.set(object, valueTobeSet);
        return field;
    }
    
    private static Field getField(Class<?> mClass, String fieldName) throws NoSuchFieldException {
        try {
            return mClass.getDeclaredField(fieldName);
        } catch (NoSuchFieldException e) {
            Class<?> superClass = mClass.getSuperclass();
            if (superClass == null) {
                throw e;
            } else {
                return getField(superClass, fieldName);
            }
        }
    }
}

getIndexWriter
This method creates an IndexWriter since the OOTB IIQ classes do not provide a way to append to the index file. It uses the StandardAnalyzer to analyze the text and the FSDirectory to open the index directory.

private IndexWriter getIndexWriter(FullTextIndex index) throws IOException, GeneralException {
    Analyzer          analyzer = new StandardAnalyzer();
    IndexWriterConfig iwc      = new IndexWriterConfig(analyzer);
    iwc.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
    
    log.trace("Got analyzer");
    
    String indexDir = LuceneUtil.getIndexPath(index);
    
    FSDirectory dir = FSDirectory.open(Paths.get(indexDir));
    
    return new IndexWriter(dir, iwc);
}

index
The most important method. This method is called by the FullTextSingleIndexer class, and it performs the indexing and the saving of the index file.

public void index(String bundleName) throws GeneralException {
    log.trace("Indexing bundle: " + bundleName);
    
    Bundle bundle = context.getObjectByName(Bundle.class, bundleName);
    
    if (bundle == null) {
        throw new ObjectNotFoundException(new Message(String.format("Bundle '%s' not found", bundleName)));
    }
    
    
    
    try {
        Document doc = indexer.index(bundle);
        
        if (doc == null) {
            throw new RuntimeException("Document is null");
        }
        
        indexWriter.addDocument(doc);
    } catch (IOException e) {
        throw new RuntimeException(e);
    }
    
    
    log.trace("Indexed bundle: " + bundleName);
}

close
This method is comes from the Closeable interface and is used to close the IndexWriter object.

Note: If you forget to close the IndexWriter, the OOTB index refresh task will fail.

@Override
public void close() throws IOException {
    if (indexWriter != null) {
        indexWriter.close();
        log.trace("Closed indexWriter");
    }
}

Full Class Example

package com.ventum.iiq.indexer;

import org.apache.log4j.LogManager;
import org.apache.log4j.Logger;
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.store.FSDirectory;
import sailpoint.api.SailPointContext;
import sailpoint.api.SailPointFactory;
import sailpoint.fulltext.AbstractIndexer;
import sailpoint.fulltext.Builder;
import sailpoint.fulltext.BundleIndexer;
import sailpoint.fulltext.LuceneUtil;
import sailpoint.object.*;
import sailpoint.task.AbstractTaskExecutor;
import sailpoint.tools.GeneralException;
import sailpoint.tools.Message;
import sailpoint.tools.ObjectNotFoundException;

import java.io.Closeable;
import java.io.IOException;
import java.lang.reflect.Field;
import java.nio.file.Paths;


public class SingleTextifier implements Closeable {
    public static final  String INDEX_NAME = "BundleManagedAttribute";
    private static final Logger log        = LogManager.getLogger(SingleTextifier.class);
    
    private final SailPointContext context;
    private final IndexWriter      indexWriter;
    private final AbstractIndexer  indexer;
    
    
    private static class RefUtil {
        @SuppressWarnings("UnusedReturnValue")
        public static Field setFieldValue(Object object, String fieldName, Object valueTobeSet) throws NoSuchFieldException, IllegalAccessException {
            Field field = getField(object.getClass(), fieldName);
            field.setAccessible(true);
            field.set(object, valueTobeSet);
            return field;
        }
        
        private static Field getField(Class<?> mClass, String fieldName) throws NoSuchFieldException {
            try {
                return mClass.getDeclaredField(fieldName);
            } catch (NoSuchFieldException e) {
                Class<?> superClass = mClass.getSuperclass();
                if (superClass == null) {
                    throw e;
                } else {
                    return getField(superClass, fieldName);
                }
            }
        }
    }
    
    
    public SingleTextifier(SailPointContext context) throws GeneralException, IOException {
        this.context     = context;
        this.indexer     = getIndexer(getIndex());
        this.indexWriter = getIndexWriter(getIndex());
    }
    
    private FullTextIndex getIndex() throws GeneralException {
        FullTextIndex index = context.getObjectByName(FullTextIndex.class, INDEX_NAME);
        
        if (index == null) {
            throw new RuntimeException("Index not found");
        }
        
        return index;
    }
    
    private AbstractIndexer getIndexer(FullTextIndex index) {
        AbstractIndexer indexer = new BundleIndexer();
        
        Builder builder = new Builder(context, null, null);
        
        try {
            RefUtil.setFieldValue(builder, "_index", index);
            indexer.prepare(builder);
        } catch (NoSuchFieldException | IllegalAccessException | GeneralException e) {
            throw new RuntimeException(e);
        }
        
        indexer.setAddObjectClass(true);
        
        return indexer;
    }
    
    private IndexWriter getIndexWriter(FullTextIndex index) throws IOException, GeneralException {
        Analyzer          analyzer = new StandardAnalyzer();
        IndexWriterConfig iwc      = new IndexWriterConfig(analyzer);
        iwc.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
        
        log.trace("Got analyzer");
        
        String indexDir = LuceneUtil.getIndexPath(index);
        
        FSDirectory dir = FSDirectory.open(Paths.get(indexDir));
        
        return new IndexWriter(dir, iwc);
    }
    
    public void index(String bundleName) throws GeneralException {
        log.trace("Indexing bundle: " + bundleName);
        
        Bundle bundle = context.getObjectByName(Bundle.class, bundleName);
        
        if (bundle == null) {
            throw new ObjectNotFoundException(new Message(String.format("Bundle '%s' not found", bundleName)));
        }
        
        log.trace("Got bundle");
        log.trace(bundle.toXml());
        
        
        try {
            Document doc = indexer.index(bundle);
            
            if (doc == null) {
                throw new RuntimeException("Document is null");
            }
            
            indexWriter.addDocument(doc);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
        
        
        log.trace("Indexed bundle: " + bundleName);
    }
    
    @Override
    public void close() throws IOException {
        if (indexWriter != null) {
            indexWriter.close();
            log.trace("Closed indexWriter");
        }
    }
}

Example Usage in an Interactive Workflow

Here’s an example of how to use the FullTextSingleIndexer in a workflow. This example is a workflow that allows users to create a bundle using an interactive form. After the bundle is created, the workflow indexes the bundle to make it searchable.

Please bear in mind that this is a simplified example and may require additional error handling and validation in a production environment.

To learn more about:

Prerequisites:

  • The FullTextSingleIndexer and SingleTextifier classes are implemented and deployed.
  • The Full Text Single Refresh request definition is created and deployed.

Create a Form

  1. First, we need to define the Form in IIQ. Go to the IIQ Debug page (/debug), click on the Select an Action and select New:
  2. Copy and paste the following xml:
<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE Form PUBLIC "sailpoint.dtd" "sailpoint.dtd">
<Form name="ventum_create_bundle_form" type="Workflow">
  <Attributes>
    <Map>
      <entry key="pageTitle" value="Create a Business Role"/>
      <entry key="title" value="Create a Business Role"/>
    </Map>
  </Attributes>

  <Section name="Section 10">
    <Field displayName="Bundle Name" name="bundleName" required="true"
           type="string"/>

    <Field displayName="Bundle Display Name" name="bundleDisplayName" required="true"
           type="string"/>
  </Section>

  <Button action="next" label="Submit"/>
  <Button action="back" label="Cancel"/>
</Form>
  1. Click on Save

Create a Workflow

  1. On the IIQ Debug page, click again on the Select an Action and select New.
  2. Copy and paste the following xml:
<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE Workflow PUBLIC "sailpoint.dtd" "sailpoint.dtd">
<Workflow explicitTransitions="true" name="ventum_create_bundle" libraries="Identity">

  <Variable name="transient" initializer="true"/>
  <Variable name="LOGGER_NAME" initializer="com.ventum.workflow.CreateBundleWorkflow"/>

  <Step icon="Start" name="Start" posX="15" posY="375">
    <Transition to="Show Form"/>
  </Step>

  <Step icon="Provision" name="Show Form" posX="115" posY="375">
    <Approval name="Get Bundle Info" owner="ref:launcher"
              return="bundleName, bundleDisplayName">
      <Arg name="workItemForm" value="ventum_create_bundle_form"/>
      <Arg name="workItemType" value="Form"/>
    </Approval>

    <Transition to="end" when="!ref:approved"/>
    <Transition to="Create Role"/>
  </Step>


  <Step icon="task" name="Create Role">
    <Script>
      <Source>
        <![CDATA[
        import org.apache.log4j.LogManager;
        import org.apache.log4j.Logger;
        import sailpoint.object.Bundle;

          Logger log = LogManager.getLogger(LOGGER_NAME);

          {
            Bundle newBundle = new Bundle();

            newBundle.setName(bundleName);
            newBundle.setDisplayName(bundleDisplayName);

            newBundle.setType("business");

            context.saveObject(newBundle);
            context.commitTransaction();
          }
        ]]>
      </Source>
    </Script>

    <Transition to="Index Roles"/>
  </Step>

  <Step name="Index Roles">
    <Script>
      <Source>
        <![CDATA[
        import java.util.Collections;

        import com.ventum.iiq.indexer.FullTextSingleIndexer;
        import org.apache.log4j.Logger;
        import org.apache.log4j.LogManager;

          Logger log = LogManager.getLogger(LOGGER_NAME);

          try {
            log.trace("Indexing roles");
            FullTextSingleIndexer.runOnAllServers(context, Collections.singletonList(bundleName));
          } catch (Exception e) {
            log.error(e, e);
            throw new RuntimeException(e);
          }
        ]]>
      </Source>
    </Script>

    <Transition to="end"/>
  </Step>
</Workflow>
  1. Click on “Save

What does the workflow do?
This workflow allows users to create a bundle using an interactive form. It starts the execution by showing the form to the user (the Show Form step). After the user fills in the form and submits it, the workflow proceeds to the Create Role step. In this step, the workflow creates a new bundle with the provided name and display name. If the bundle is successfully created, the workflow continues with the Index Roles step. Here, the workflow indexes the newly created bundle using the FullTextSingleIndexer class. If the indexing is successful, the workflow ends. If not, it also ends, but with an error in the logs.

Create a QuickLink

  1. Go to the IIQ Debug page and click on Select an Action and select New.
  2. Copy and paste the following xml:
<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE QuickLink PUBLIC "sailpoint.dtd" "sailpoint.dtd">
<QuickLink action="workflow" category="Access" messageKey="Create Business Role"
           name="ventum_create_bundle" ordering="3">
  <Attributes>
    <Map>
      <entry key="workflowName" value="ventum_create_bundle"/>
    </Map>
  </Attributes>

  <Description>Allows the user to create a business role</Description>

  <QuickLinkOptions allowSelf="true">
    <DynamicScopeRef>
      <Reference class="sailpoint.object.DynamicScope" name="Manager"/>
    </DynamicScopeRef>
  </QuickLinkOptions>

</QuickLink>
  1. Click on Save

What does the QuickLink do?
Not much. It checks if the user is a manager (i.e. is a member of the Manager dynamic scope) and calls the ventum_create_bundle workflow if clicked on.

Test the Workflow

  1. Log in to IIQ as a user with the necessary permissions. If you are already logged in, log out and log back in to refresh the visible QuickLinks.

  2. Click on the hamburger menu icon in the top left corner, select Manage Access, and then click on the Create Business Role item.

  3. Fill in the form with the desired bundle name and display name and click Submit.

  4. Check the logs for any errors or warnings. If everything is successful, the bundle should be created and indexed.

Conclusion

This guide provides a step-by-step explanation of how to implement a single role addition to the Role Full Text Index programmatically in IIQ.
The instructions provided demonstrate how you can create a request definition and executor to index the bundle on each available host, as well as a textifier to perform the indexing. The guide also includes an example of how to use a workflow to allow users to create bundles using an interactive form called from a QuickLink.

By following these steps, you can ensure that dynamically created roles are instantly searchable and manageable in the system, enhancing the user experience and efficiency of role management in IIQ.

4 Likes