Appboard/2.4/builder/data processing scripts: Difference between revisions

imported>Mike.berman
(create page for AppBoard 2.4)
 
imported>David.moore
 
(12 intermediate revisions by 3 users not shown)
Line 1: Line 1:
{{DISPLAYTITLE:AppBoard Data Processing Script}}
{{DISPLAYTITLE:Data Processing Scripts}}
Data Processing Script is a tool for customizing Data Sources in AppBoard to meet specific needs of a customer solution. By inserting Groovy script into the data processing, AppBoard can process the data according to a set of coded instructions. This allows data to be manipulated or pre-processed while passing between the raw back-end data source and AppBoard.
[[Category:AppBoard 2.4]]
 
Data Processing Scripts are a method of processing data returned from Data Sources before it's made available to the rest of the system. This is a very flexible method to transform data in order to meet the specific needs of a solution.
 
Scripts are written in the Groovy programming language, which is a Java like dynamic language, and attached to Data Source entities.




Line 6: Line 10:
#Open the AppBoard Builder and click on "Data Sources"
#Open the AppBoard Builder and click on "Data Sources"
#Select the data source to be affected and click the "Edit" button
#Select the data source to be affected and click the "Edit" button
#Advance through the data source wizard to the "explore" stage
#Advance through the data source wizard to the "Query" stage for database, IBM Omnibus/Netcool, or ServiceNow adapters, or the "Explore" stage otherwise.
#Select the table you want to transform and then click the "Advanced" button
#Next, access the "Data Processing Script" area:
#*For database, IBM Omnibus/Netcool, or ServiceNow adapters click the table you want to transform and click the "Edit" button.
#*Otherwise, select the table you want to transform and then click the "Advanced" button.
# Click "Add" in the Data Processing Script section.
#Configure the Data Processing Script
#Configure the Data Processing Script
##Input the path name of the script
##Input the path name of the script
###typically Data Processing Script are kept in the following directory:  ${appboard_server}/webapps/enportal/WEB-INF/groovy-script
##*Typically Data Processing Script are kept in the following directory:  <tt>${appboard_server}/webapps/enportal/WEB-INF/groovy-script/custom</tt>
##Set the priority (default 10)
##Set the priority (default 10)
###priority is only important if you have multiple scripts active
##*Priority is only important if you have multiple scripts active
##Set script to run on "Response" or "Refresh"
##Set script to run on "Response" or "Refresh"
###Response (called every time the client requests)
###Response (called every time the client requests)
####Example when to use:  performing a join or status determination by requesting data from another dataset
###*Example when to use:  performing a join or status determination by requesting data from another dataset
###Refresh (called only when data is fetched from source)
###Refresh (called only when data is fetched from source)
####Example when to use:  substringing a value
###*Example when to use:  substringing a value
#Click through the wizard to finish
#Click through the wizard to finish
#View the appropriate data collection (re-query if necessary) to see if data is being updated
#View the appropriate data collection (re-query if necessary) to see if data is being updated


{{Note|If you have a Sub-Query based on a data source that is using a Data Processing Script, in order for the Sub-Query to have access to any new attributes created by the script, you must first query the data collection of the data source with the script prior to configuring or editing the Sub-Query.}}
== Examples ==
=== Enrich with Session Information ===
In AppBoard there are many places where it may be useful to have access to session information such as action filters, icon/color/shape filters, labels, and a variety of actions. A way to do this is to enrich the data response with session information such as the username, domain, role, and session variables.
AppBoard ships with an example <tt>SessionEnrichment.groovy</tt> which automatically adds additional fields with the session information listed above.
'''SessionEnrichment.groovy'''
<code>[java,N]
import com.edgetech.services.ServiceRequestFacade
import com.edgetech.services.ServiceResponseFacade
import com.edgetech.services.GenericRecord
import com.edgetech.services.Association
import com.edgetech.services.DataAdapter;
import com.edgetech.services.GenericRecord;
import com.edgetech.services.NamespaceService;
import com.edgetech.services.model.EntityDef;
import com.edgetech.services.model.Query;
import java.util.Collection;
import java.util.Iterator;
import java.util.HashMap;
ServiceResponseFacade serviceResponse = responseFacade;
ServiceRequestFacade serviceRequest = requestFacade;


== Data Processing Script Example ==
Collection<GenericRecord> records = serviceResponse.getOrderedGenericRecords();
if ( ( records != null ) && !records.isEmpty() ) {
 
  Iterator<GenericRecord> recIter = records.iterator();
  while ( recIter.hasNext() ) {
 
    GenericRecord rec = recIter.next();
 
    rec.setAttributeValue( "_session_user", serviceRequest.getSessionContext().getActor().getUser().getID() );
    rec.setAttributeValue( "_session_domain", serviceRequest.getSessionContext().getActor().getUser().getDomain().getName() );
    rec.setAttributeValue( "_session_role", serviceRequest.getSessionContext().getRole().getName() );
    try {
      rec.setAttributeValue( "_session_var_bob", serviceRequest.getDataAdapter().performSubstitution("\${shim:session.var.get('bob')}" ) );
    } catch (Exception e) {
      rec.setAttributeValue( "_session_var_bob", "undef" );
    }
  }
}
</code>
 
=== Trim Whitespace ===
 
In this example extra leading and trailing whitespace is removed from the <tt>SITE_CODE</tt> field. It also creates a new field <TT>SITE_CODE2</tt> which is a concatenation of the <tt>SITE_CODE</tt> and <tt>STATE</tt> fields.


'''TrimSiteCode.groovy'''
'''TrimSiteCode.groovy'''
<pre>import com.edgetech.services.ServiceRequestFacade
<code>[java,N]
import com.edgetech.services.ServiceRequestFacade
import com.edgetech.services.ServiceResponseFacade
import com.edgetech.services.ServiceResponseFacade
import com.edgetech.services.GenericRecord
import com.edgetech.services.GenericRecord
Line 41: Line 99:
ServiceRequestFacade serviceRequest = requestFacade;
ServiceRequestFacade serviceRequest = requestFacade;
   
   
                Collection recordsToClean = serviceResponse.getOrderedGenericRecords();
Collection recordsToClean = serviceResponse.getOrderedGenericRecords();
                if ( ( recordsToClean != null ) && !recordsToClean.isEmpty() ) {
if ( ( recordsToClean != null ) && !recordsToClean.isEmpty() ) {
                  for ( GenericRecord recordToClean:recordsToClean ) {
    for ( GenericRecord recordToClean:recordsToClean ) {
                                String siteCode = recordToClean.getAttributeValueAsString("SITE_CODE");
        String siteCode = recordToClean.getAttributeValueAsString("SITE_CODE");
                                if ( siteCode != null ) {
        if ( siteCode != null ) {  
                                                recordToClean.setAttributeValue("SITE_CODE", siteCode.trim());
            recordToClean.setAttributeValue("SITE_CODE", siteCode.trim());
                                                String stateCode = recordToClean.getAttributeValueAsString("STATE");
            String stateCode = recordToClean.getAttributeValueAsString("STATE");
                                                if ( stateCode != null ) {
            if ( stateCode != null ) {  
                                                                recordToClean.setAttributeValue("SITE_CODE2", siteCode.trim() + stateCode.trim());
                recordToClean.setAttributeValue("SITE_CODE2", siteCode.trim() + stateCode.trim());
                                                }
            }  
                                }
        }
                                String region = recordToClean.getAttributeValueAsString("REGION");
        String region = recordToClean.getAttributeValueAsString("REGION");
                                if ( region != null ) {
        if ( region != null ) {  
                                                recordToClean.setAttributeValue("REGION", region.trim());
            recordToClean.setAttributeValue("REGION", region.trim());
                                }
        }  
                  }
    }
                }
}
 
</code>
</pre>

Latest revision as of 18:48, 30 March 2014


Data Processing Scripts are a method of processing data returned from Data Sources before it's made available to the rest of the system. This is a very flexible method to transform data in order to meet the specific needs of a solution.

Scripts are written in the Groovy programming language, which is a Java like dynamic language, and attached to Data Source entities.


Applying a Data Processing Script

  1. Open the AppBoard Builder and click on "Data Sources"
  2. Select the data source to be affected and click the "Edit" button
  3. Advance through the data source wizard to the "Query" stage for database, IBM Omnibus/Netcool, or ServiceNow adapters, or the "Explore" stage otherwise.
  4. Next, access the "Data Processing Script" area:
    • For database, IBM Omnibus/Netcool, or ServiceNow adapters click the table you want to transform and click the "Edit" button.
    • Otherwise, select the table you want to transform and then click the "Advanced" button.
  5. Click "Add" in the Data Processing Script section.
  6. Configure the Data Processing Script
    1. Input the path name of the script
      • Typically Data Processing Script are kept in the following directory: ${appboard_server}/webapps/enportal/WEB-INF/groovy-script/custom
    2. Set the priority (default 10)
      • Priority is only important if you have multiple scripts active
    3. Set script to run on "Response" or "Refresh"
      1. Response (called every time the client requests)
        • Example when to use: performing a join or status determination by requesting data from another dataset
      2. Refresh (called only when data is fetched from source)
        • Example when to use: substringing a value
  7. Click through the wizard to finish
  8. View the appropriate data collection (re-query if necessary) to see if data is being updated
Template-note.png
If you have a Sub-Query based on a data source that is using a Data Processing Script, in order for the Sub-Query to have access to any new attributes created by the script, you must first query the data collection of the data source with the script prior to configuring or editing the Sub-Query.

Examples

Enrich with Session Information

In AppBoard there are many places where it may be useful to have access to session information such as action filters, icon/color/shape filters, labels, and a variety of actions. A way to do this is to enrich the data response with session information such as the username, domain, role, and session variables.

AppBoard ships with an example SessionEnrichment.groovy which automatically adds additional fields with the session information listed above.

SessionEnrichment.groovy [java,N] import com.edgetech.services.ServiceRequestFacade import com.edgetech.services.ServiceResponseFacade import com.edgetech.services.GenericRecord import com.edgetech.services.Association import com.edgetech.services.DataAdapter; import com.edgetech.services.GenericRecord; import com.edgetech.services.NamespaceService; import com.edgetech.services.model.EntityDef; import com.edgetech.services.model.Query;

import java.util.Collection; import java.util.Iterator; import java.util.HashMap;

ServiceResponseFacade serviceResponse = responseFacade; ServiceRequestFacade serviceRequest = requestFacade;

Collection<GenericRecord> records = serviceResponse.getOrderedGenericRecords(); if ( ( records != null ) && !records.isEmpty() ) {

 Iterator<GenericRecord> recIter = records.iterator();
 while ( recIter.hasNext() ) {
   GenericRecord rec = recIter.next();
   rec.setAttributeValue( "_session_user", serviceRequest.getSessionContext().getActor().getUser().getID() );
   rec.setAttributeValue( "_session_domain", serviceRequest.getSessionContext().getActor().getUser().getDomain().getName() );
   rec.setAttributeValue( "_session_role", serviceRequest.getSessionContext().getRole().getName() );
   try {
     rec.setAttributeValue( "_session_var_bob", serviceRequest.getDataAdapter().performSubstitution("\${shim:session.var.get('bob')}" ) );
   } catch (Exception e) {
     rec.setAttributeValue( "_session_var_bob", "undef" );
   }
 }

}

Trim Whitespace

In this example extra leading and trailing whitespace is removed from the SITE_CODE field. It also creates a new field SITE_CODE2 which is a concatenation of the SITE_CODE and STATE fields.

TrimSiteCode.groovy [java,N] import com.edgetech.services.ServiceRequestFacade import com.edgetech.services.ServiceResponseFacade import com.edgetech.services.GenericRecord import com.edgetech.services.Association import com.edgetech.services.impl.ServiceRequestHelper

import java.util.Collection; import java.util.List; import java.util.ArrayList; import java.util.Iterator;

// trim extra whitespace and concatenate two fields into a new one

ServiceResponseFacade serviceResponse = responseFacade; ServiceRequestFacade serviceRequest = requestFacade;

Collection recordsToClean = serviceResponse.getOrderedGenericRecords(); if ( ( recordsToClean != null ) && !recordsToClean.isEmpty() ) {

   for ( GenericRecord recordToClean:recordsToClean ) {
       String siteCode = recordToClean.getAttributeValueAsString("SITE_CODE");
       if ( siteCode != null ) { 
           recordToClean.setAttributeValue("SITE_CODE", siteCode.trim());
           String stateCode = recordToClean.getAttributeValueAsString("STATE");
           if ( stateCode != null ) { 
               recordToClean.setAttributeValue("SITE_CODE2", siteCode.trim() + stateCode.trim());
           }   
       }
       String region = recordToClean.getAttributeValueAsString("REGION");
       if ( region != null ) { 
           recordToClean.setAttributeValue("REGION", region.trim());
       }   
   }

}