Appboard/old/XMLWebServiceAdapter

Introduction

The AppBoard XML Web Service Adapter supports data retrieval from HTTP REST services. XML data returned from the REST service is transformed to the AppBoard internal data structure using XSLT. Hierarchical relationships between XML data elements can be represented within AppBoard through associations within the AppBoard data model.

Using the XML Web Service Adapter

The following is an overview of how to use the XML Web Service Adapter to retrieve XML data from REST services and make it available in AppBoard.

REST URL and XML data validation

Prior to configuring the AppBoard Data Adapter, you should configure validate the request URL being used to retrieve the XML data in your browser. Once the URL request is returning the data you require, you will need to write an XSLT stylesheet to transform the returned XML data into AppBoard XML data format.

Writing an XSLT Stylesheet

XSLT (Extensible Stylesheet Language Transformations) is a declarative, XML-based language used for the transformation of XML. The original XML is not changed; rather, a new set of XML is created based on the content of an existing one using the XSLT. The XML Bible contains a tutorial on XSLT authoring that can be used as a reference.

The incoming XSLT is transformed into the AppBoard internal XML structure as defined by the xml-adapter-data.dtd located in the ${application.home}/stylesheets/ directory. Hierarchical relationships within the XML can be captured by using associations in the AppBoard data model as described below.

There are with several sample XSLT files that can also be used as a reference. The files are located in ${application.home}/stylesheets/sample/ directory:

Example transformation for a RSS XML source

${application.home}/stylesheets/sample/rss-1.0-to-adapter.xsl

Example transformation for a VPM Connex XML source

${application.home}/stylesheets/sample/fluke-to-adapter.xsl

Example transformation for an Atlas XML source

${application.home}/stylesheets/sample/atlas-to-adapter.xsl

Also available are several XML sample data files for each XSLT:

Sample RSS feed:

${application.home}/data/pkg/sample/xml/nist-cve/cve.xml

Sample Atlas Global Attacks Summary:

${application.home}/data/pkg/sample/xml/atlas/atlas.xml

Sample Fluke VPM Connex XML data:

${application.home}/data/pkg/sample/xml/fluke/top-n.xml

${application.home}/data/pkg/sample/xml/fluke/topSitesByEurt.xml

${application.home}/data/pkg/sample/xml/fluke/trend.xml

${application.home}/data/pkg/sample/xml/fluke/trendForApp.xml

${application.home}/data/pkg/sample/xml/fluke/trendForAppSite.xml

Representing Hierarchical XML Relationships Using Associations

Hierarchical parent/child relationships is accomplished by defining an association on the parent entity and setting a parent id attribute on the child records. The association then uses a key value in the parent as the "fromKey" and the corresponding value set on each child record as the "toKey". It is usually convenient to define the association via the XSLT, but the association can also be defined in the builder.

Example data with a parent/child relationship between nodes and NICs:

${application.home}/data/pkg/sample/hierarchy/hierarchy.xml

Transform for that data:

${application.home}/stylesheets/sample/hierarchy-to-adapter.xsl

Configuring an AppBoard XML Data Source in Builder

The Data Source Wizard within the AppBoard Builder is used to retrieve data from REST Web Services. The following steps outline the process of connecting to a Data Source using the AppBoard Builder:

1. Click "Data Sources" in the left tool palette.
2. Click on the “Add” button.
3. Enter the name of the Data Source in the “Name” field.
4. Select the “Web Services” category
5. Select the XML REST Adapter
6. In the url field enter the URL of the REST service with parameters.
7. In the styleSheetPath field enter the file path of the sytle sheet on the AppBoard server.
8. Select "next"
9. Enter the "primary keys" of the data source if it has not been defined within the XSLT.
10. Select "next"
11. Create associations between the XML REST data source and other data sources if the associations have not been defined within the XSLT.

Inspect the data in an AppBoard Data Collection

The data that has been retrieved by the new Data Source can be viewed as a Data Collection in the Data Collection Wizard.

1. Click "Data Collections" in the left tool palette.
2. Click on the "i" or “preview” button at the far right side of the row for your Data Collection.

Incorporating Dynamic Variables into a URL Query

AppBoard Data Source queries can incorporate system and custom variables into the URL string using portal SHIM variables. The values of these variables are dynamically inserted into the URL prior to making the request. For example, if you would like a query to with a variable to incorporate the current date / time you can do so as follows:

http://192.10.0.23/Portal/Portal/Services/ConnexDataAccess.ashx?domain=VpmConnex&username=user&password=password&report=EurtTrend&startTime=January%2012,2011%2017:30&endTime=${shim:utility.cryptor.url.encode('${shim:now.format('MMMMMMMdd,yyyy HH:mm')}')}

Where ${shim:now.format('MMMMMMMdd,yyyy HH:mm')} will be substituted for the current data / time in the MMMMMMMdd,yyyy HH:mm format. The “now” shim expression is wrapped within the ${shim:utility.cryptor.url.encode('')} call to apply URL encoding to the result.

Inserting the users current “Role” into the query can be accomplished by inserting the ${shim:session.actor.role.name} expression into the URL string. For example, Customer='${shim:session.actor.role.name}'"

XSLT Tutorial

Using the hierarchical data from the associations example, this tutorial builds the XSL Transformation in a step-by-step manner. It will be helpful in following the tutorial to have the source XML and the target XML in separate windows or printed out.

Source: ${application.home}/data/pkg/sample/hierarchy/hierarchy.xml

note: we don't have the target anywhere, since we never print it out. That might be a good debug function to have on the wizard. I use xsltproc to run it on linux, which is a big time-saver on developing the xsl.

1. Review the basic structure of your source XML.

This example consists of a single root node ("rspec") with some descriptive child elements one of which ("computeResource") contains nested information on compute nodes which contain further information including network interfaces. In outline form:

• rspec
  • aggregate
  • description
  • lifetime
  • computeResource
    • node
      • networkInterface
    • node
      • networkInterface
    • node
      • networkInterface

2. Identify the data elements you want to transform into AppBoard Entities.

In this case, we are interested in nodes and networkInterfaces which we will call Nodes and NICs.

3. Begin the XSLT with the xsl:stylesheet element and an xsl:output element:

[xml,N] <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" version="1.0">

   <xsl:output method="xml" indent="yes" encoding="UTF-8"/>

</xsl:stylesheet> 4. Identify an element that has only a single instance and is the parent of all of the data elements you wish to capture as Entities. In many cases, that would be the root element ("rspec" in this example), but because there is only one "computeResource" instance we can reference it directly as "/rspec/computeResource".

5. Add a match template for that data element that emits the "result" element and definitions for the entities we will capture:

[xml,N] <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" version="1.0">

   <xsl:output method="xml" indent="yes" encoding="UTF-8"/>
   <xsl:template match="/rspec/computeResource">
       <result xmlns="http://www.edgeti.com/xml-data">
           <entity name="Nodes">
               <attributes>
                   <attribute primaryKey="true">NodeId</attribute>
                   <attribute>Address</attribute>
               </attributes>
           </entity>
           <entity name="NICs">
               <attributes>
                   <attribute primaryKey="true">NICId</attribute>
                   <attribute>Address</attribute>
               </attributes>
           </entity>
       </result>            
   </xsl:template>

</xsl:stylesheet>

This only works for single instance elements. If there were multiple "computeResource" elements we would start with "/rspec" even if we weren't capturing an Entity corresponding to "computeResource" instances.

6. Add do-nothing templates for the peers of "computeResource" that we plan to ignore:

[xml,N] <xsl:template match="/rspec/aggregate"/> <xsl:template match="/rspec/description"/> <xsl:template match="/rspec/lifetime"/> 7. Add a match template for the "node" elements to construct "record" elements for each instance:

[xml,N] <template match="node">

   <record>
       <value name="NodeId"><value-of select="@id"/></value>
       <value name="Address"><value-of select="address"/></value>
   </record>

</template>

Note the use of "@id" to extract an attribute of the "node" element and the use of "address" to extract the value in a child element.

8. Add a match template for the "networkInterface" elements. Note that we use "node/networkInterface" since that is the relative "path" from our "computeResource" element.

[xml,N] <xsl:template match="node/networkInterface">

   <record>
       <value name="NICId"><xsl:value-of select="@id"/></value>
       <value name="Address"><xsl:value-of select="ipAddress"/></value>
       <value name="NodeId"><xsl:value-of select="../@id"/></value>
   </record>

</xsl:template> 9. Place these after the template for "computeResource" and then add xsl:apply-templates elements to reference them to construct children of the "entity" elements in the output:

[xml,N] <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" version="1.0">

   <xsl:output method="xml" indent="yes" encoding="UTF-8"/>
   <xsl:template match="/rspec/computeResource">
       <result xmlns="http://www.edgeti.com/xml-data">
           <entity name="Nodes">
               <attributes>
                   <attribute primaryKey="true">NodeId</attribute>
                   <attribute>Address</attribute>
               </attributes>
               <xsl:apply-templates select="node"/>
           </entity>
           <entity name="NICs">
               <attributes>
                   <attribute primaryKey="true">NICId</attribute>
                   <attribute>Address</attribute>
               </attributes>
               <xsl:apply-templates select="node/networkInterface"/>
           </entity>
       </result>            
   </xsl:template>
   <xsl:template match="/rspec/aggregate"/>
   <xsl:template match="/rspec/description"/>
   <xsl:template match="/rspec/lifetime"/>
   <template match="node">
       <record>
           <value name="NodeId"><value-of select="@id"/></value>
           <value name="Address"><value-of select="address"/></value>
       </record>
   </template>
   <xsl:template match="node/networkInterface">
       <record>
           <value name="NICId"><xsl:value-of select="@id"/></value>
           <value name="Address"><xsl:value-of select="ipAddress"/></value>
       </record>
   </xsl:template>

</xsl:stylesheet>

Now we have a working XSL Transform that defines our two entities and constructs all of the records. But, the hierarchy of the data gives us an association that we aren't capturing.

10. Define the association in the "Nodes" entity:

[xml,N] <associations>

   <association fromKey="NodeId" toEntity="NICs" toKey="NodeId"/>

</associations> 11. Add an attribute to "NICs" to store the key:

[xml,N] <entity name="NICs">

   <attributes>
       <attribute primaryKey="true">NICId</attribute>
       <attribute>Address</attribute>
       <attribute>NodeId</attribute>
   </attributes>

</entity> 12. Use "../@id" in an xsl:value-of to add a "NodeId" value to the instances of "NICs".

[xml,N] <xsl:template match="node/networkInterface">

   <record>
       <value name="NICId"><xsl:value-of select="@id"/></value>
       <value name="Address"><xsl:value-of select="ipAddress"/></value>
       <value name="NodeId"><xsl:value-of select="../@id"/></value>
   </record>

</xsl:template> Resulting XSL:

[xml,N] <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.w3.org/1999/xhtml" version="1.0">

   <xsl:output method="xml" indent="yes" encoding="UTF-8"/>
   <xsl:template match="/rspec/computeResource">
       <result xmlns="http://www.edgeti.com/xml-data">
           <entity name="Nodes">
               <attributes>
                   <attribute primaryKey="true">NodeId</attribute>
                   <attribute>Address</attribute>
               </attributes>
               <associations>
                   <association fromKey="NodeId" toEntity="NICs" toKey="NodeId"/>
               </associations>
               <xsl:apply-templates select="node"/>
           </entity>
           <entity name="NICs">
               <attributes>
                   <attribute primaryKey="true">NICId</attribute>
                   <attribute>Address</attribute>
                   <attribute>NodeId</attribute>
               </attributes>
               <xsl:apply-templates select="node/networkInterface"/>
           </entity>
       </result>            
   </xsl:template>
   <xsl:template match="/rspec/aggregate"/>
   <xsl:template match="/rspec/description"/>
   <xsl:template match="/rspec/lifetime"/>
   <template match="node">
       <record>
           <value name="NodeId"><value-of select="@id"/></value>
           <value name="Address"><value-of select="address"/></value>
       </record>
   </template>
   <xsl:template match="node/networkInterface">
       <record>
           <value name="NICId"><xsl:value-of select="@id"/></value>
           <value name="Address"><xsl:value-of select="ipAddress"/></value>
           <value name="NodeId"><xsl:value-of select="../@id"/></value>
       </record>
   </xsl:template>

</xsl:stylesheet>