Edge Documentation

{{DISPLAYTITLE:Data Processing Scripts}}
[[Category:AppBoard 2.6]]

Data Processing Scripts are a method of processing data returned from Data Sources before it's made available to the rest of the system. This is a very flexible method to transform data in order to meet the specific needs of a solution.

Scripts are written in the Groovy programming language, which is a Java like dynamic language, and attached to Data Source entities.


== Applying a Data Processing Script ==
#Open the AppBoard Builder and click on "Data Sources"
#Select the data source to be affected and click the "Edit" button
#Advance through the data source wizard to the "Query" stage for database, IBM Omnibus/Netcool, or ServiceNow adapters, or the "Explore" stage otherwise. 
#Next, access the "Data Processing Script" area:
#*For database, IBM Omnibus/Netcool, or ServiceNow adapters click the table you want to transform and click the "Edit" button.
#*Otherwise, select the table you want to transform and then click the "Advanced" button.
# Click "Add" in the Data Processing Script section.
#Configure the Data Processing Script
##Input the path name of the script
##*Typically Data Processing Script are kept in the following directory:  <tt>${appboard_server}/webapps/enportal/WEB-INF/groovy-script/custom</tt>
##Set the priority (default 10)
##*Priority is only important if you have multiple scripts active
##Set script to run on "Response" or "Refresh"
###Response (called every time the client requests)
###*Example when to use:  performing a join or status determination by requesting data from another dataset
###Refresh (called only when data is fetched from source)
###*Example when to use:  substringing a value
#Click through the wizard to finish
#View the appropriate data collection (re-query if necessary) to see if data is being updated

{{Note|If you have a Sub-Query based on a data source that is using a Data Processing Script, in order for the Sub-Query to have access to any new attributes created by the script, you must first query the data collection of the data source with the script prior to configuring or editing the Sub-Query.}}

== Examples ==

=== Enrich with Session Information ===

In AppBoard there are many places where it may be useful to have access to session information such as action filters, icon/color/shape filters, labels, and a variety of actions. A way to do this is to enrich the data response with session information such as the username, domain, role, and session variables.

AppBoard ships with an example <tt>SessionEnrichment.groovy</tt> which automatically adds additional fields with the session information listed above.

'''SessionEnrichment.groovy'''
<code>[java,N]
import com.edgetech.services.ServiceRequestFacade
import com.edgetech.services.ServiceResponseFacade
import com.edgetech.services.GenericRecord
import com.edgetech.services.Association
import com.edgetech.services.DataAdapter;
import com.edgetech.services.GenericRecord;
import com.edgetech.services.NamespaceService;
import com.edgetech.services.model.EntityDef;
import com.edgetech.services.model.Query;

import java.util.Collection;
import java.util.Iterator;
import java.util.HashMap;

ServiceResponseFacade serviceResponse = responseFacade;
ServiceRequestFacade serviceRequest = requestFacade;

Collection<GenericRecord> records = serviceResponse.getOrderedGenericRecords();
if ( ( records != null ) && !records.isEmpty() ) {

  Iterator<GenericRecord> recIter = records.iterator();
  while ( recIter.hasNext() ) {

    GenericRecord rec = recIter.next();

    rec.setAttributeValue( "_session_user", serviceRequest.getSessionContext().getActor().getUser().getID() );
    rec.setAttributeValue( "_session_domain", serviceRequest.getSessionContext().getActor().getUser().getDomain().getName() );
    rec.setAttributeValue( "_session_role", serviceRequest.getSessionContext().getRole().getName() );
    try {
      rec.setAttributeValue( "_session_var_bob", serviceRequest.getDataAdapter().performSubstitution("\${shim:session.var.get('bob')}" ) );
    } catch (Exception e) {
      rec.setAttributeValue( "_session_var_bob", "undef" );
    }
  }
}
</code>

=== Trim Whitespace ===

In this example extra leading and trailing whitespace is removed from the <tt>SITE_CODE</tt> field. It also creates a new field <TT>SITE_CODE2</tt> which is a concatenation of the <tt>SITE_CODE</tt> and <tt>STATE</tt> fields.

'''TrimSiteCode.groovy'''
<code>[java,N]
import com.edgetech.services.ServiceRequestFacade
import com.edgetech.services.ServiceResponseFacade
import com.edgetech.services.GenericRecord
import com.edgetech.services.Association
import com.edgetech.services.impl.ServiceRequestHelper
 
import java.util.Collection;
import java.util.List;
import java.util.ArrayList;
import java.util.Iterator;
 
// trim extra whitespace and concatenate two fields into a new one
 
ServiceResponseFacade serviceResponse = responseFacade;
ServiceRequestFacade serviceRequest = requestFacade;
 
Collection recordsToClean = serviceResponse.getOrderedGenericRecords();
if ( ( recordsToClean != null ) && !recordsToClean.isEmpty() ) {
    for ( GenericRecord recordToClean:recordsToClean ) {
        String siteCode = recordToClean.getAttributeValueAsString("SITE_CODE");
        if ( siteCode != null ) { 
            recordToClean.setAttributeValue("SITE_CODE", siteCode.trim());
            String stateCode = recordToClean.getAttributeValueAsString("STATE");
            if ( stateCode != null ) { 
                recordToClean.setAttributeValue("SITE_CODE2", siteCode.trim() + stateCode.trim());
            }   
        }
        String region = recordToClean.getAttributeValueAsString("REGION");
        if ( region != null ) { 
            recordToClean.setAttributeValue("REGION", region.trim());
        }   
    }
}
</code>