Developer Guide 2024-1¶
- Developer Guide 2024-1
- Development Environment Setup
- Existing Training Application with Assessment Engine/Tutor
- Creating a new Gateway Module Interop Plugin
- Creating a performance assessment Condition
- Analysis of Training Applications and preferred condition classes
- Creating DKF
- Creating a Course
- Simple Example Training Application
- Integrating an External Assessment Engine
- Integrate a Sensor
- GIFT Server mode
- Configuring the XML-RPC Python Server With GIFT
- Configuring GIFT with RapidMiner
- Improving the GIFT Authoring Tool (GAT)
- Character Servers
- Changing UMS/LMS database schema
- Adding External Applications to GIFT Wrap
- GWT
Introduction¶
This guide is useful for those individuals that would like to customize GIFT for their own purposes or better understand the GIFT architecture. It will help the reader to have an introductory knowledge of computer programming before continuing. Furthermore, your GIFT instance should be setup in developer mode in order to follow along with the examples/references mentioned throughout the document (see Development Environment Setup).
- In most examples the desktop-based XML authoring tools will be mentioned. These tools are meant to be used by low level developers as they provide fine grained interaction with the XML tree. There also exist more user friendly authoring tools like the GIFT Authoring Tool (GAT) which can be found by launching GIFT and going to the Tools section.
- Also, if you develop shortcuts or a better approach to any of the following procedures please share with the rest of the GIFT community on gifttutoring.org.
- The GIFT team would like to personally thank Intelligent Automation Inc. for significant help in writing this document. The full thanks can be seen in the section titled Special Thanks to IAI.
Delivering your changes to the GIFT team¶
Visit the GIFT Co-Developer Guide for information on how your code might be integrated back into GIFT.
Where to Begin?¶
GIFT is a tutoring framework that can be customized for your use case. Below is a list of common situations GIFT users may find themselves in. As GIFT continues to mature and more use cases are realized this list will evolve. If you believe that you have a use case which can help others, we would encourage you to post it to the http://www.gifttutoring.org forums.
Please read through the entire list and then decide which situations you need solutions for.- If you plan on developing software for GIFT, please follow the Development Environment Setup.
I. Do you have a training application (e.g. VBS, TC3/Vmedic, UrbanSim, a flight simulator, a physics tutor, etc. – in other words, an environment where a user will be assessed on performance when conducting one or more tasks) that you would like to integrate with GIFT?
There are two types of training applications that can communicate with GIFT. They differ primarily in there distribution method.
Do you want to deliver your application through the Web?
GIFT currently supports Unity WebGL training applications. For a complete guide on developing Unity WebGL applications for GIFT view the Gift Unity Software Development Guide
Do you want to deliver your application as a desktop application?
Training applications communicate with GIFT via an “interop plugin” in the Gateway module. The reader is free to choose an existing protocol already implemented in an interop plugin (e.g. DIS protocol – uses Socket communication, XML-RPC server/client) or implement a new protocol. There is no standard communication protocol between the Gateway module and an external training application. This is due to the fact that some training applications are already interoperable and therefore already have a protocol in place which should be leveraged by GIFT to reduce additional development.
In some instances your training application and assessment logic are already tightly coupled together in a single application. If so please see Existing Training Application with Assessment Engine/Tutor.
For information on how to integrate a new training application, please see Creating a new Gateway Module Interop Plugin.
II. Do you need to introduce new assessment logic for a concept into GIFT?
Concepts are assessed by one or more conditions in the Domain module. Currently there are 2 assessment engines in GIFT, the Default Assessment engine and SIMILE. In the default assessment engine, logic to assess game state messages for concepts is performed within the Java condition class itself. The conditions are configured by a Domain Knowledge File (DKF) for scenario specific constraints (e.g. coordinates). SIMILE is an external assessment engine that is configured using the output of the SIMILE workbench application.
As more assessment logic is integrated into the Domain module, GIFT will be able to choose the best analysis tool for a particular domain. In most instances the solution will be a hybrid approach selecting among many engines executing simultaneously.
Do you have an existing performance assessment engine (e.g. SIMILE – in other words, logic that can consume ‘game state’ and provide assessment results) that you would like to integrate with GIFT?
Before you attempt to introduce a new assessment engine, determine if the existing engines provide the functionality you need (unless you are heavily invested in this new engine).
In some instances your training application and assessment logic are already tightly coupled together in a single application. If so please see Existing Training Application with Assessment Engine/Tutor.
To see more on integrating an external assessment engine, please see Integrating an External Assessment Engine.
Do you want to extend the Default assessment engine by creating a new condition that can perform assessments?
Please see Creating a performance assessment Condition.
III. Do you have a sensor (e.g. heartbeat, ECG, EEG, Electro dermal, eye tracking, etc.) that you would like to integrate with GIFT?
Sensors communicate with GIFT via the Sensor module. Integrate a Sensor.
IV. Do you want to author a course to execute in GIFT?
A GIFT course contains course elements such as surveys, training application instances and After Action Review (to name a few) and is what a user can select after logging into the GIFT tutor user interface (TUI). Please see Creating a Course.
V. Do you need help with Java Web Start Communication Application (i.e. Gateway module) including configuration and Java Security settings when running GIFT in server mode?
When GIFT is executed in server mode (i.e. “GIFT in the cloud”), all of the modules except for the Gateway module are running on the server. The gateway module is executed on the client’s machine as an on-demand application via Java Web Start technology when a GIFT course requires interaction with a desktop application (e.g. PowerPoint, VBS). When using the GIFT Gateway Java Web Start instance there are configuration and Java Security settings that need to be addressed before using it. Please see GIFT Server mode.
VI. Do you want to how python scripts can be called externally via GIFT?
GIFT has the ability to call python scripts via an XML-RPC interface. This can be used to help filter sensor data before broadcasting the sensor event to GIFT. It also can be used to pre-process training application or sensor data prior to sending the data to RapidMiner. For more information on how this can be configured, please see Configuring the XML-RPC Python Server With GIFT.
VII. Do you want to learn more about how to configure GIFT to use an external RapidMiner model to assess learner state?
GIFT supports the use of external RapidMiner models which can be configured to process training application data and sensor data. The result of this external processing is used to assess various learner states (anxious, frustrated, etc) and yield a confidence (high/low) of the learner state. Please see Configuring GIFT with RapidMiner.
VIII. Did you make a change to GIFT but the GIFT Authoring Tool (GAT) doesn’t natively support the ability for an author to select your changes?
In the past, most changes made to GIFT source and schema files (xsd) was either automatically (java reflection, jaxb, etc.) integrated into the desktop-based XML authoring tools or the level of effort was minimal to do so. However since the advent of the web based GAT, it now requires various levels of client/server side GWT development to expose new authoring elements. Please see Improving the GIFT Authoring Tool (GAT).
IX. Do you need to change the UMS or LMS database tables?
There are some circumstances where developers will need to change existing database tables (i.e. add a column to a table) or add new tables to the UMS or LMS database. More often than not this is due to needing new functionality in GIFT. Please see Changing UMS/LMS database schema.
X. Do you want to add an external application to the GIFT Wrap authoring tool?
One of the most difficult part of authoring in an ITS such as GIFT is the domain assessment logic. The goal of GIFT Wrap authoring tool is to decrease the learning curve normally required to author ITS courses. It creates External Application course objects by leveraging that training application's native scenario authoring user interface.
If you need to add logic to GIFT Wrap to leverage authoring in a newly integrated training application then refer to Adding External Application's to GIFT Wrap.
XI. Do you want to integrate a different character server other than Virtual Human or Media Semantics?
Currently GIFT is integrated with Virtual Human and Media Semantics servers in order to provide various character actions such as text to speech. If you would like to know more about this integration please refer to Character Servers.
Development Environment Setup¶
Eclipse is the preferred IDE to use for GIFT development. The GIFT source includes an Eclipse project file (.project) and an Eclipse preferences file to help with setting up a GIFT development environment.
Configure Eclipse IDE¶
When setting up your IDE to develop GIFT, you need to configure the IDE’s project classpath so that all of GIFT’s dependencies can be used and are loaded in the correct order. Before configuring your IDE, be sure that GIFT’s third-party libraries are located in the ‘external’ in GIFT (this is already done for you if you are using an official GIFT release from gifttutoring.org). Provided with the GIFT source release are the classpath and project files needed to configure the Eclipse IDE. The steps detailed below will set the reader’s workstation up for GIFT software development.
- Install GIFT according to the installation procedures (or just use the installGIFT.bat to run the installer). This includes successfully building GIFT using ant or build.bat script file (which will be performed by the GIFT installer if your GIFT has not been built yet).
- Download/Install Eclipse IDE, win64. (http://eclipse.org/).
- Launch Eclipse
- Select “File,” then “Import.”
- Expand “General,” select “Existing Projects into Workspace”
- Select the “Next>” button.
- Select the “Browse” button next to “Select root directory:”.
- Browse for directory where you placed GIFT and select the GIFT named directory and select the “OK” button.
- Select the Finish button.
- Build newly created GIFT project in Eclipse – select the “Project” in the Package Explorer pane, then “Project” menu option followed by “Build Project.”
- Upon successfully build (view the Problems tab for Errors)…
- Now import the Eclipse preferences file that GIFT provides by selecting “File,” then “Import.”
- Expand “General,” select “Preferences”
- Select the “Browse” button next to “From preference file:”.
- Browse for directory where you placed GIFT and select the file GIFT\Eclipse Preferences.epf. Select the “Open” button.
- Select the Finish button.
Java Compiler¶
Eclipse tends to default the Java compiler version to what it can find on your machine and depending on the version of Eclipse you are using, it may choose the wrong Java version. Follow the steps below to ensure your GIFT Eclipse project is configured correctly. Failure to use the appropriate Java version can result in build warnings and errors and/or Java runtime exceptions.
- Identify the GIFT project Java compiler version by right clicking on the GIFT project in Eclipse and selecting “Properties.” Select “Java Build Path,” then the “Libraries” tab. Scroll to find the “JRE System Library.” It should match the version of Java that was installed using the GIFT provided Java exe. If you don’t remember what that version was, select “JRE System Library” and expand it. Do the paths to the various jars match where you installed the GIFT Java? If not, then click the Edit button and select the GIFT Java folder you provided during the GIFT Java install.
- Make sure the JDK Compliance version matches your Java version. From the project properties window, select “Java Compiler.” From that window, you can set project specific settings if needed.
Note: as of writing this, GIFT uses Open JDK 11 and the latest Eclipse version is called 2023-09. In order to select Java 11 as the JDK Compliance version, you will need the Eclipse 2023-09 or newer version.
Your Eclipse IDE is now configured with the GIFT baseline and launch configurations for the various modules/applications. Please refer to the IDE Debugging section for more information on what to do with launch configurations for each GIFT module/application.
Debugging¶
There are two main options for debugging using an IDE. Both involve using the IDE to place breakpoints anywhere in the code, that when hit will pause the appropriate thread.
Recommended Remote Debugging
Here you will remotely connect to a running module. Each module/application has a specific port assigned to it for remote debugging. It most instances the port is displayed on the console window for that application but, for reference, all ports numbers are in the GIFT/scripts/launchProcess.bat file. In order to remote connect, the GIFT application needs to be running and you will need to provide the IDE the appropriate port number to connect on.
e.g. when running GIFT using launchGIFT.bat you will remote debug on port 50015 (the Single Process Launcher, SPL, debug port)
The other, less tested, option is to run the module within the IDE. In this option, you can either have the IDE manage the build process or build using ant with the build.bat script or console. A GIFT module/application (e.g., tools or modules) can be launched within an IDE using launch configurations (Eclipse reference, may be called different things in different IDEs). After importing/creating a launch configuration you can access or run it by selecting Run, then Debug Configurations from the menu (this is just one way to find them). If you have imported the launch configurations provided with GIFT (after following the procedures in Detailed Steps to Configure IDE), then they will be listed under Java Applications. Simply select the application you wish to run (e.g., DomainModule) and select the Debug button. Once you successfully launch a module you will be able to set breakpoints in the code that can greatly improve debugging capabilities over simple console out techniques.
Existing Training Application with Assessment Engine/Tutor¶
This section is for the reader who already has a training application with an embedded assessment engine or tutor and you would like to integrate with GIFT. The best approach is to determine whether your assessment engine/tutor can be decoupled from the training application and integrated with the GIFT Domain module. The benefits of this are that other GIFT users could gain access to your engine either through an official GIFT release or by some proprietary means. Furthermore, other GIFT users might want to leverage your engine for other Domains and/or improve upon the engine itself. For more information on connecting an external assessment engine to the domain module refer to Integrating an External Assessment Engine.
However this may not be feasible in some circumstances. In that case we advise you to create a light weight version of a Domain module Condition class that will consume training application state messages (in this case they would contain the output of your assessment engine) and translate the contents into the GIFT performance assessment format. The GIFT performance assessment format is currently based on an enumerated result (i.e. Above, At, Below Expectation) for each of the nodes in the concept hierarchy represented by a DKF. To date, the closest complete example (it lacks details of having an assessment engine in the training application) of this type of integration was with SIMILE and is discussed in Integrating an External Assessment Engine. We are aware of various organizations (Vanderbilt, Intelligent Automation, etc.) which are developing/using others.
For further assistance please search for forums or post a new topic at https://gifttutoring.org/projects/gift/boards.
Creating a new Gateway Module Interop Plugin¶
In order to communicate with a new external training application (e.g. VBS), a Gateway module interop plugin may need to be created if an existing plugin can’t be reused (e.g. the DIS plugin is reusable by all DIS complaint training applications). The interop plugin is responsible for communicating between GIFT and your external training Application. If the training application already has networking capabilities (i.e. external interoperability) sufficient enough for use with GIFT, then your GIFT interop plugin logic should utilize that protocol to communicate to and from your training application. Internally, GIFT uses its own message format and message bus. This means that your interop plugin would be responsible for translating the content between the various message formats, depending on which way example data is flowing.
There are several examples of interop plugin classes (i.e. Java files that ‘extend AbstractInteropInterface’) found in GIFT\src\mil\arl\gift\gateway\interop\.
The following procedures provide an example on how to create a new interop plugin using the Eclipse IDE. The interop interface class will communicate with an external training application referenced in Simple Example Training Application using XML-RPC. Even if you decide to use a different communication protocol between an interop plugin and your TA, the basic principles described below are still valid.
- Prerequisite: installed Microsoft Visual C# 2010 (freely available online) which will be need to compile the changes we are going to make to the Simple Example Training Application written in C#. If you can’t install this program then please make sure to not create a new Interop plugin class but instead use the existing SimpleExampleTAPluginInterface class when following this guide.
Creating a new java Package¶
Create a new Java package (i.e. folder) so that we easily identify the location of your new plugin.
- In Eclipse’s Package Explorer right click “GIFT/src/” folder
- In the menu click on “New -> Package”
- In the “New Java Package” window leave the Source folder as GIFT/src
- In the “Name” textbox type in: mil.arl.gift.gateway.interop.myplugin
- Click on “Finish”
Creating a new Plugin Interface java file¶
In this step the reader is going to create a new plugin interface Java file by copying some key functionality from the simple.SimpleExampleTAPluginInterface class.
All plugin interface Java classes must extend the mil.arl.gift.gateway.interop.AbstractInteropInterface class. That abstract class has 5 important abstract methods that must be implemented by all plugin interface classes. Other abstract methods must be defined, but can stay empty or return null unless their functionality is needed for your interface.
Required methods:- configure – configures the interop interface with connection settings (e.g. IP address, network port) specified in the interopConfig.xml.
- getSupportedMessageTypes – Returns a list of supported GIFT training application messages that the interop interface can translate into a message the training application can consume. This information is important because the Gateway module will deliver GIFT messages to the appropriate interop plugin(s) and expect them to act on them accordingly. The most common messages to handle are Simulation Management (SIMAN) messages like Load, Initialize, Start, Pause and Stop which help control and/or synchronize course execution between GIFT and the training application.
- handleGIFTMessage – allows the interop plugin to handle a GIFT message (that it previously stated it supports). More often than not this results in a message being sent to the training application.
- getReqTrainingAppConfigurations – defines the training application types needing to be configured to use this interop interface implementation.
- cleanup – used to notify the plugin that it is no longer needed and should cleanup.
- Right click on the “mil.arl.gift.gateway.interop.myplugin” package in the Package Explorer in Eclipse.
- In the menu go to “New -> Class”
- In the “New Java Class” window, find the “Name” textbox and type in: MyPluginInterface
- In the “Superclass:” field, type “mil.arl.gift.gateway.interop.AbstractInteropInterface”.
- In the checkboxes below "Superclass" and "Interface", ensure that 'Constructors from superclass' and 'Inherited abstract methods' are checked, and no other boxes are.
- Click finish.
- In the Package Explorer go to “mil.arl.gift.gateway.interop.simple” and open “SimpleExampleTAPluginInterface.java”
Because we are about to copy contents, line by line, from this file it would be best to view both the Example plugin class and your MyPlugin class at the same time in Eclipse using a split pane view. Try dragging one of the file’s tabs around the Eclipse window until you are satisfied with the split pane view presented.
- The following steps will identify the important features of the Simple Example TA plugin interface to copy:
import generated.course.MyPluginInputs;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Vector;
import org.apache.xmlrpc.XmlRpcException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import mil.arl.gift.common.ConfigurationException;
import mil.arl.gift.common.Siman;
import mil.arl.gift.common.enums.MessageTypeEnum;
import mil.arl.gift.common.enums.SimanTypeEnum;
import mil.arl.gift.common.ta.state.SimpleExampleState;
import mil.arl.gift.common.ta.state.StopFreeze;
import mil.arl.gift.gateway.GatewayModule;
import mil.arl.gift.gateway.interop.AbstractInteropInterface;
import mil.arl.gift.net.api.message.Message;
import mil.arl.gift.net.xmlrpc.XMLRPCClient;
import mil.arl.gift.net.xmlrpc.XMLRPCServer;
2. Using the GIFT provided logger by copying the following above the configure method in your MyPluginInterface class. Using the log4j library is preferred over “System.out” calls to show text on the console/command-prompt window.
/** Instance of the logger */
private static Logger logger = LoggerFactory.getLogger(MyPluginInterface.class);
3. Simulation Interests – the plugin needs to register with the Gateway module the various types of GIFT messages the plugin is interested in. A plugin is interested in messages it can handle within the plugin and/or translate and send the contents to the training application for processing.
- Copy the following and paste it above the configure method in your MyPluginInterface class. In this example the interop plugin will be received Simulation Management (SIMAN) and Display Feedback request messages. The handling logic for these messages is covered in a later step.
/** * contains the types of GIFT messages this interop interface can translate and handle or send * on to the training application */ private static List<MessageTypeEnum> supportedMsgTypes; static{ supportedMsgTypes = new ArrayList<MessageTypeEnum>(); supportedMsgTypes.add(MessageTypeEnum.SIMAN); supportedMsgTypes.add(MessageTypeEnum.DISPLAY_FEEDBACK_GATEWAY_REQUEST); } - Reference the static list you just copied in your “getSupportedMessageTypes()” method by making it look like:
@Override public List<MessageTypeEnum> getSupportedMessageTypes() { return supportedMsgTypes; }
- Copy the following XML-RPC server and client related code and paste it above the configure method in your MyPluginInterface class
/** * The XML RPC server created by this class to allow the training application to remotely call methods * in this class, mainly the 'handleTrainingApplicationMessage' method. */ private XMLRPCServer server; /** * The XML RPC client created by this class to call remote methods in the training application C# program which * is hosting an XML RPC server. */ private XMLRPCClient client; /** needed for XML-RPC called methods that will need to send messages via the Gateway module */ private static MyPluginInterface instance = null; private static List<TrainingApplicationEnum> REQ_TRAINING_APPS; static{ REQ_TRAINING_APPS = new ArrayList<TrainingApplicationEnum>(); REQ_TRAINING_APPS.add(TrainingApplicationEnum.SIMPLE_EXAMPLE_TA); } /** * This inner class contains the RPC methods the training application RPC client can call. * The only reason I choose not to use the outer class of SimpleExamplePluginTAPluginInterface was to prevent having * to create another constructor (as mentioned in the Notes below) and to organize the RPC methods * together instead of possible exposing other methods in ExamplePLuginInterface. * * Notes: * 1) This class must contain either no class constructor OR an empty constructor. * 2) Every RPC method must return a non-null, non-void, non java.lang.Object value (e.g. Integer). * 3) if using an inner class, the class must be static * * @author mhoffman * */ public static class MyPluginXMLRPC{ /** * This is a RPC method hosted by the RPC server in this class and called by the * training application's RPC client to handle 'game state' messages. * * XML-RPC method name = * "mil.arl.gift.gateway.interop.simple.SimpleExampleTAPluginInterface$SimpleExampleTAPluginXMLRPC.handleTrainingApplicationMessage" * * @param message the training application message content * @return Integer an arbitrary return object to satisfy the XML-RPC library requirements */ public Integer handleTrainingApplicationMessage(String message){ SimpleExampleState exampleState = new SimpleExampleState(message); sendMessageToGIFT(exampleState, MessageTypeEnum.SIMPLE_EXAMPLE_STATE); return XMLRPCServer.EMPTY_RETURN_OBJECT; } /** * This is a RPC method hosted by the RPC server in this class and called by the * training application's RPC client to notify GIFT that the TA is finished. * * XML-RPC method name = * "mil.arl.gift.gateway.interop.simple.SimpleExampleTAPluginInterface$SimpleExampleTAPluginXMLRPC.handleTrainingApplicationFinished" * * @return Integer an arbitrary return object to satisfy the XML-RPC library requirements */ public Integer handleTrainingApplicationFinished(){ StopFreeze stopFreeze = new StopFreeze(0, 0, 0, 0); sendMessageToGIFT(stopFreeze, MessageTypeEnum.STOP_FREEZE); return XMLRPCServer.EMPTY_RETURN_OBJECT; } } /** * Call upon the Gateway module to construct and then send a new GIFT message using the payload * provided. * * @param payload the contents of the message to send * @param messageTypeEnum the enumerated type of the message */ private static void sendMessageToGIFT(TrainingAppState payload, MessageTypeEnum messageTypeEnum){ GatewayModule.getInstance().sendMessageToGIFT(payload, messageTypeEnum, instance); } /** * Call a remote method with the given parameter. * * @param rpcMethodName the name of the method to call on the RPC server * @param param the single parameter to pass to the rpcMethod. Can be null. * @param errorMsg - used to append error content too, if an error occurs * @return Object the return value of the remote method. Can be null. */ private Object callRPCMethod(String rpcMethodName, String param, StringBuilder errorMsg) { Vector<String> params = new Vector<>(1); if(params != null){ params.addElement(param); } return callRPCMethod(rpcMethodName, params, errorMsg); } /** * Call a remote method with the given parameters. * * @param rpcMethodName the name of the method to call on the RPC server * @param params the parameters to pass to the rpcMethod. Can be null. * @param errorMsg - used to append error content too, if an error occurs * @return Object the return value of the remote method. Can be null. */ private Object callRPCMethod(String rpcMethodName, Vector<?> params, StringBuilder errorMsg) { try { Object returnObj = client.callMethod(rpcMethodName, params); return returnObj; } catch (XmlRpcException e) { logger.error("There was an exception thrown when calling the RPC method of "+rpcMethodName+".", e); errorMsg.append("Failed to call RPC method named "+rpcMethodName+"."); } return null; }
5. Create class constructor by adding the following above the new sendMessageToGIFT method in your MyPluginInterface class. The name parameter’s value will come from the interopConfig.xml entry to be created in a later step.
/**
* Class constructor
*
* @param name - the display name of the plugin
*/
public MyPluginInterface(String name){
super(name, true);
instance = this;
}
- Make your configure method look like the following:
@Override public boolean configure(Serializable config) { if (config instanceof generated.gateway.XMLRPC) { try { generated.gateway.XMLRPC pluginConfig = (generated.gateway.XMLRPC) config; server = new XMLRPCServer(pluginConfig.getServerNetworkPort(), MyPluginXMLRPC.class); client = new XMLRPCClient(pluginConfig.getExternalServerNetworkAddress(), pluginConfig.getExternalServerNetworkPort()); } catch (Exception e) { throw new ConfigurationException("Failed to create the XML RPC client.", e.getMessage(), e); } } else { throw new ConfigurationException(getName()+" Plugin interface can't configure.", getName() + " plugin interface uses "+generated.gateway.XMLRPC.class+" interop input and doesn't support using the interop config instance of " + config, null); } return false; }
- Paste the following after your getSupportedMessageTypes method. This logic will start the XML-RPC server when the plugin is enabled and then stop the server when the plugin is disabled during the execution of a course.
@Override public void setEnabled(boolean value){ boolean isEnabledAlready = isEnabled(); super.setEnabled(value); if(!isEnabledAlready){ //listen for RPC requests try { server.start(); } catch (IOException e) { logger.error("Caught exception when trying to start the RPC server.", e); throw new RuntimeException("There was a problem starting the RPC server."); } logger.info("Started listening for incoming training application messages."); }else{ //stop listening for RPC requests server.stop(); logger.info("Stopped listening for incoming training application messages."); } } - To be extra diligent let’s make sure the XML-RPC server is stopped when the Gateway module is being closed. Make your cleanup method look like the following:
@Override public void cleanup() { server.stop(); }
- Copy the following and paste it above the class constructor.
private static List<TrainingApplicationEnum> REQ_TRAINING_APPS; static{ REQ_TRAINING_APPS = new ArrayList<TrainingApplicationEnum>(); REQ_TRAINING_APPS.add(TrainingApplicationEnum.SIMPLE_EXAMPLE_TA); } - Then make your getReqTrainingAppConfigurations method look like:
@Override public List<TrainingApplicationEnum> getReqTrainingAppConfigurations(){ return REQ_TRAINING_APPS; }
- Make your handleGIFTMessage method look like the following. Notice how there is handling logic for the various SIMAN related messages as well as the display feedback message. Other examples of handling SIMAN logic can be found in the other GIFT interop plugins as it is a common practice and encouraged in order to synchronize GIFT and your TA.
@Override public boolean handleGIFTMessage(Message message, StringBuilder errorMsg) { // Below is some arbitrary handling of GIFT messages. // In most instances your interop plugin will need to handle SIMAN messages in order to // allow GIFT to synchronize with the TA. // Look at the other interop plugins for more examples of handling GIFT messages. if (message.getMessageType() == MessageTypeEnum.SIMAN) { Siman siman = (Siman) message.getPayload(); if (siman.getSimanTypeEnum() == SimanTypeEnum.LOAD) { generated.course.InteropInputs interopInputs = getLoadArgsByInteropImpl(this.getClass().getName(), siman.getLoadArgs()); generated.course.MyPluginInputs inputs = (generated.course.MyPluginInputs) interopInputs.getInteropInput(); generated.course.MyPluginInputs.LoadArgs args = inputs.getLoadArgs(); //actually calling a different RPC method here, called "load" to provide another example of an RPC method. Object returnObj = callRPCMethod("load", args.getScenarioName(), errorMsg); logger.info("Receive return value of "+returnObj+" from XML-RPC call for method named 'load'."); }else if(siman.getSimanTypeEnum() == SimanTypeEnum.PAUSE){ callRPCMethod("blob", "Pause message received", errorMsg); }else if(siman.getSimanTypeEnum() == SimanTypeEnum.RESTART){ callRPCMethod("blob", "Restart message received", errorMsg); }else if(siman.getSimanTypeEnum() == SimanTypeEnum.RESUME){ callRPCMethod("blob", "Resume message received", errorMsg); }else if(siman.getSimanTypeEnum() == SimanTypeEnum.START){ callRPCMethod("blob", "Start message received", errorMsg); }else if(siman.getSimanTypeEnum() == SimanTypeEnum.STOP){ callRPCMethod("blob", "Stop message received", errorMsg); }else{ errorMsg.append(getName()+" plugin can't handle siman type of "+siman.getSimanTypeEnum()); logger.error("Found unhandled Siman type of "+siman.getSimanTypeEnum()); } }else if(message.getMessageType() == MessageTypeEnum.DISPLAY_FEEDBACK_GATEWAY_REQUEST){ //send some feedback text to the example training application String text = (String)message.getPayload(); callRPCMethod("blob", "Display Feedback received with feedback of \""+text+ "\"", errorMsg); }else{ logger.error("Received unhandled GIFT message to send over to the "+getName()+" plugin, " + message); errorMsg.append(getName()+" plugin can't handle message of type "+message.getMessageType()); } return false; }
- Now your interop plugin class is complete, however you will not be able to successfully compile/build it until you complete the interop config schema changes in the next section.
Configure new Plugin connection¶
In order for the Gateway module to use your interop plugin, the plugin class must be registered with the module. This is accomplished by the GIFT\config\gateway\interopConfig.xml file contents which are loaded when the Gateway module starts. Every interop plugin instance that could be used by your GIFT needs an entry in the interopConfig.xml. If you wish to have more than one instance of a single interop plugin class, than you will need to create an “InteropInterfaceConfig” element for each (continue reading for more information).
Specifying Interop Plugin Configuration Parameters¶
If your interop plugin requires connection parameters such as an IP address and/or network port, then you will need to modify the XML schema file for interopConfig.xml called GIFT\config\gateway\interopConfig.xsd (An XML schema file is a description of a type of XML document that expresses constraints on the structure of the XML elements). The change to the schema involves creating a new XML element that will contain specific properties for your interop plugin.
- You are highly encouraged to provide configurable network connection parameters at a minimum as an alternative to fixed values in code. This way the user can change the values to fit their situation without having to edit code and recompile GIFT. It also provides the user one location to visit to change connection parameters, rather than having each plugin reference their own parameter file somewhere on disk.
Continuing from the previous section’s example…
- In Windows Explores navigate to GIFT/config/gateway/
- Open up interopConfig.xsd (Note: I use Notepad++, but any text editor would do)
- Because we are using XML-RPC in the newly created MyInteropPlugin class we could just use the existing XML-RPC schema definition, however we will create a new element anyway.
- First find the XML tag that looks like “<xs:element name="XML-RPC">”. Then copy all of the content from that until you reach the corresponding closing element tag that looks like “</xs:element>” (currently that is about 60 lines of XML to be selected).
- Next, find the last “</xs:element>” near the bottom of the file and paste the copied text on a new line after it.
- Change the name of the copied “XML-RPC” element name value to “My-Plugin-XML-RPC” to make it a unique element in the schema.
- Notice the various sub-element names (i.e. ExternalServerNetworkAddress, ExternalServerNetworkPort and ServerNetworkPort) as well as the annotation text for each that describes their purpose.
<xs:element ref="My-Plugin-XML-RPC"/>
Add interface configuration to interopConfig.xml¶
For this guide we will be using the randomly selected port numbers of 10566 and 10567. We need two ports for our XML-RPC configuration, one port for the server and one for the client connection.
Before using these ports, let’s check if that port is available to use on your computer by following the steps below (if the port is in use by another application, GIFT will throw a runtime exception when trying to open a connection to that numbered port).
- Click on the Start button
- In the search box, type cmd and then hit enter
- Type netstat –aon | find “:10566” and then hit enter
- If a blank line is shown then the port is available. If something is printed like
TCP 0.0.0.0:50009 0.0.0.0:0 LISTENING 9420
then the port is in use and you need to select a different value other than 10566.
- Do the same procedure for port 10567
Once you have valid port numbers to use, it is time to open the interopConfig.xml to add a section for your newly created interop interface implementation.
Continuing from the previous section’s example…- In Windows Explores navigate to GIFT/config/gateway/configurations/
- Open up default.interopConfig.xml (Note: I use Notepad++, but any text editor would do)
- Find the last “</InteropInterfaceConfig>” and add the following on a new line after that:
<InteropInterfaceConfig refID="17"> <available>false</available> <impl>gateway.interop.myplugin.MyPluginInterface</impl> <name>My Plugin</name> <XML-RPC> <ExternalServerNetworkAddress>127.0.0.1</ExternalServerNetworkAddress> <ExternalServerNetworkPort>10567</ExternalServerNetworkPort> <ServerNetworkPort>10566</ServerNetworkPort> </XML-RPC> </InteropInterfaceConfig> - Save the file
- Notice that your MyPluginInterface class is used as the implementation class (“impl”) for this instance.
Notes:
- Make sure to update the ‘refID” value to a unique value like the next highest value for all ‘refID’ values in interopConfig.xml.
- Use the appropriate ‘networkPort’ value if you decided to use a different number.
Configure new Interop course inputs¶
In most instances, your interop plugin class will need specific input such as the scenario name (e.g. VBS scenario name) or file to load (e.g. PowerPoint show file) in your training application. This sort of information is course specific and therefore needs to go in the course XML file.
There are two ways to author the inputs needed for your interop plugin. The first is to use the “CustomInteropInputs” course XML element which allows the course author to provide a list of name-value pairs. This solution requires no course XML schema changes; however it places the burden on the author to know the key name values needed by your interop plugin (e.g. “scenarioName”, “abcFile”). The other solution is to create a new course XML element with specific attribute names, value restrictions and the option to show helpful information about the attribute (e.g. purpose, hints, use cases). An example of this solution is explained below.
Create new Course Interop Inputs¶
In this step you will be editing the course XML schema file to create a new element which will be used when authoring a course later on.
- In Windows Explores navigate to GIFT/config/domain/course
- Open up course.xsd (Note: I use Notepad++, but any text editor would do)
- Before “<xs:element name=”VBSInteropInputs”>” insert this
<xs:element name="MyPluginInputs"> <xs:annotation> <xs:appinfo> <fg:node-info exposed="true" message="My Interoperability Interface Inputs"> <fg:message>Used to provide interop inputs for the interop plugin represented by the MyPluginInterface class in the Gateway module.</fg:message> </fg:node-info> </xs:appinfo> </xs:annotation> <xs:complexType> <xs:sequence> <xs:element name="loadArgs"> <xs:annotation> <xs:appinfo> <fg:node-info exposed="true" message="Load Arguments"> <fg:message>These are the arguments sent to the Gateway module with the Siman.Load message. The arguments are used to configure or execute logic in the interop interface.</fg:message> </fg:node-info> </xs:appinfo> </xs:annotation> <xs:complexType> <xs:sequence> <xs:element name="ScenarioName"> <xs:annotation> <xs:appinfo> <fg:node-info exposed="true" message="Scenario Name"> <fg:message>Name of an Example Scenario. The value is used by Example scripting to load the scenario. The value matches the name of a FITS scenario folder and is one of the entries shown in FITS scenario selection screens.</fg:message> </fg:node-info> </xs:appinfo> </xs:annotation> <xs:simpleType> <xs:restriction base="xs:string"> <xs:minLength value="1"/> </xs:restriction> </xs:simpleType> </xs:element> </xs:sequence> </xs:complexType> </xs:element> </xs:sequence> </xs:complexType> </xs:element> - Then search for “<xs:element ref="VBSInteropInputs"/>” which should reside under the “InteropInputs” element.
- Add this line above the “<xs:element ref="VBSInteropInputs"/>” line (and below the “<xs:choice>” line):
<xs:element ref="MyPluginInputs"/> - Save the file. [We will auto-generate classes based on these changes to the schema when building GIFT in a later step]
Game state message(s)¶
At this point your training application can be loaded and configured by the Gateway module. The next step is to determine what GIFT messages are needed to contain your training application game state information into (i.e. one or more training application messages are being sent by the training application to the interop plugin which then needs to translate those messages into GIFT messages). The first step is to take a look at the existing GIFT message classes located in GIFT\src\mil\arl\gift\common\ta\state and see if you can re-use any (this includes extending existing classes to introduce additional attributes). Then determine what GIFT messages need to be created to cover your remaining game state messages not already represented in GIFT. Keep in mind that others may want to use your new messages for other new training applications in the future, therefore try to make your new state classes as generic as possible. Furthermore, it most cases it is not a good idea to place all of your training application state attributes in a single message. Not only does this limit the amount of reusable state classes because it would require the other application to contain the same set of reporting attributes but it is entirely possible that not all attributes will have values every time you want to send a state message thereby leaving you to deal with optional vs required attribute logic (e.g. JSON encoding doesn’t allow for null values).
Game State common class¶
Every GIFT training application or game state message has a class located in mil.arl.gift.common.ta.state (classes in the common package are often referred to “common classes”) and extends mil.arl.gift.common.ta.state.TrainingAppState.
The following steps will create an example game state common class. However, if you would like to skip this step and use an existing game state common class, the class named mil.arl.gift.common.ta.state.SimpleExampleState.java is an exact match to what you are about to author.
- In Eclipse, traverse the GIFT ‘src’ path until you find mil.arl.gift.common.ta.state
- Right click on ‘state’ and select ‘New’ then ‘Class’. A create new class dialog should appear.
- Name the class “ExampleState”.
- Select the Finish button.
- Then edit the class content by replacing it with:
public class ExampleState implements TrainingAppState { /** just an example class attribute for this game state class */ private String var; /** * Class constructor - set attribute(s). * * @param var just an example class attribute for this game state class */ public ExampleState(String var){ if(var == null){ throw new IllegalArgumentException("The var can't be null."); } this.var = var; } /** * Return the var value. * * @return String */ public String getVar(){ return var; } @Override public String toString(){ StringBuffer sb = new StringBuffer(); sb.append("[ExampleState: "); sb.append("var = ").append(getVar()); sb.append("]"); return sb.toString(); } }
Notice the following coding standards (also described in the GIFT Coding Standards document): - Javadoc comments on class attributes and important methods
- toString method mainly useful for logging and debugging purposes
- use of illegalArgumentException to check invalid values, in this case the value of ‘var’ can’t be null.
- Save the file.
Message Type enumeration¶
Every message sent between GIFT modules must have a message type. Message types are enumerated in mil.arl.gift.common.enums.MessageTypeEnum.java. Every time a new message is created you will need to add a new entry by first copying an existing enumeration and then changing the values for your message type.
For our on-going example in this section:
- Open MessageTypeEnum.java in Eclipse.
- Paste this on a new line after the “ENTITY_STATE” class attribute:
public static final MessageTypeEnum EXAMPLE_GAME_STATE = new MessageTypeEnum("ExampleGameState", "Example Game State"); - Save the file.
Message Codec¶
GIFT currently provides a JSON format for all its messages. Whenever you create a new message (or edit an existing message) a corresponding codec class will need to be created/edited, accordingly. The codec classes are responsible for encoding and decoding Java objects/primitives into a JSON string representation. That string is what is sent as a GIFT message on the message bus. JSON codec classes are located in mil.arl.gift.net.api.message.codec.json. Classes must ‘extend JSONCodec’ to be used by GIFT to translate to and from a JSON string. The JSONCodec class has to important methods that must be implemented:
- decode – takes the provided JSONObject and its JSON formatted values and returns a new Java object instance that represents the JSONObject contents. For example the SimanJSON.java class decodes into a mil.arl.gift.common.Siman.class instance.
- encode – encodes the provide Java object’s attributes into the JSONObject using JSON formatting.
- An important note about JSON encoding is that it doesn’t support null values. Therefore make sure that objects like Strings have values if they are required attributes or don’t encode/decode the attribute at all.
If you wanted to send the previously made ‘ExampleState.java’ contents as a GIFT message, you would have to create a codec class. Below is an example of how to do that. However, if you choose to use the existing class named “SimpleExampleState.java” in a previous section you can skip this section as that class already has codec class for it.
- In Eclipse, traverse the GIFT ‘src’ path until you find mil.arl.gift.net.api.message.codec.json
- Right click on ‘json’ and select ‘New’ then ‘Class’. A create new class dialog should appear.
- Name the class “ExampleStateJSON”.
- Click on the “Add…” button next to the “Interfaces” section. A “Implemented Interfaces Selection” dialog appears.
- Start typing in “JSONCodec” and then select the only JSONCodec entry shown. Click the OK button.
- Select the Finish button.
- Then edit the class content by replacing:
public class ExampleStateJSON implements JSONCodec {
@Override
public Object decode(JSONObject jsonObj) throws MessageDecodeException {
// TODO Auto-generated method stub
return null;
}
@Override
public void encode(JSONObject jsonObj, Object payload) {
// TODO Auto-generated method stub
}
with...
import mil.arl.gift.common.ta.state.ExampleState;
import mil.arl.gift.net.api.message.MessageDecodeException;
import org.json.simple.JSONObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ExampleStateJSON implements JSONCodec {
/** instance of the logger */
private static Logger logger = LoggerFactory.getLogger(ExampleStateJSON.class);
/** JSON string keys */
private static final String VAR = "VAR";
@Override
public Object decode(JSONObject jsonObj) throws MessageDecodeException {
try{
String var = (String) jsonObj.get(VAR);
return new ExampleState(var);
}catch(Exception e){
logger.error("caught exception while creating "+this.getClass().getName()+" from "+jsonObj, e);
throw new MessageDecodeException(this.getClass().getName(), "Exception logged while decoding");
}
}
@SuppressWarnings("unchecked")
@Override
public void encode(JSONObject jsonObj, Object payload) {
ExampleState state = (ExampleState)payload;
jsonObj.put(VAR, state.getVar());
}
- Javadoc comments on class attributes and important methods.
- Use of MessageDecodeException to indicate an exception when decoding.
- Using a static final string for keys to help with memory usage and make it easier to maintain fixed strings like these.
- Use of logger to help debugging
- Save the file.
Create the Protobuf file for the Message type¶
After creating a new message type enumeration and a class for the message content, you will now need to implement the class as a Protobuf message. This allows GIFT to convert the message to and from a Protobuf format, to allow for more compact logs and message traffic.
In GIFT\config\protobufMessages\, there are a set of .proto files used to define these messages. Create a new .proto file to match the message you created, such as the following example.
syntax = "proto3";
option java_package = "generated.proto.common";
option java_outer_classname = "ExampleStateProto";
import "google/protobuf/wrappers.proto";
/* This message represents the information needed for an example state. */
message ExampleState {
/* The string data being contained */
string textData = 1;
}
The details of this file's contents will change depending on the contents of the message. When creating other messages, you can reference the contents of GIFT\config\protobufMessages\ for examples, and read documentation at https://protobuf.dev/.
After you save that new protobuf file, building GIFT will create a new generated class in GIFT\generated\external_proto\generated\proto\gateway\. The class name will match that which was defined in the .proto file's java_outer_classname, which in the example case above is "ExampleStateProto". It will, similarly, be generated in the namespace defined in the java_package variable of the .proto file.
Now that the generated class exists, you can create a class to convert from the generated Protobuf message to the GIFT message class you created earlier. This class will implement the mil.arl.gift.net.proto.ProtoCodec class. For the example given in this guide, the following implementation works.
package mil.arl.gift.net.api.message.codec.proto;
import generated.proto.common.ExampleStateProto;
import mil.arl.gift.common.ta.state.ExampleState;
import mil.arl.gift.net.proto.ProtoCodec;
public class ExampleStateProtoCodec implements ProtoCodec<ExampleStateProto.ExampleState, ExampleState> {
@Override
public ExampleState convert(generated.proto.common.ExampleStateProto.ExampleState protoObject) {
if (protoObject == null) {
return null;
}
String textData = protoObject.getTextData();
ExampleState request = new ExampleState(textData);
return request;
}
@Override
public generated.proto.common.ExampleStateProto.ExampleState map(ExampleState commonObject) {
if (commonObject == null) {
return null;
}
ExampleStateProto.ExampleState.Builder builder = ExampleStateProto.ExampleState
.newBuilder();
builder.setTextData(commonObject.getVar());
return builder.build();
}
}
Register Codec class for Message Type¶
After creating a new message type enumeration and a codec class for the message content, you will now need to register the codec for the message type. This allows GIFT to look up the appropriate codec class when a particular message type is being sent or received.
To register the codec follow the procedures below. However, if you choose to use the existing class named “SimpleExampleState.java” in a previous section you can skip this section as that class already has a registered codec. Furthermore, if you decided to use an existing codec but have a new message type enumeration you will need to follow the procedures below.
- Open mil.arl.gift.net.api.message.codec.json.JSONMapper.java in Eclipse.
- Locate the appropriate alphabetically location for your entry in the large static block located near the top of the file based on your MessageTypeEnum name that is going to be added. In our example the name is “EXAMPLE_GAME_STATE”.
- Feel free to copy an existing line and then paste it in the appropriate location.
- Then edit the line of code so it looks like:
messageWrapper.put(MessageTypeEnum.EXAMPLE_GAME_STATE, ExampleStateJSON.class); - Save the file.
Compile changes made¶
In order to use the Java code changes you made, you will need to build GIFT. To build GIFT double click on GIFT\build.bat and wait for it to successfully complete (i.e. the dialog will close automatically when finished). If there is an error the dialog will remain open allowing you to review the build error.
Refer to the Gateway configuration section of the GAT improvements section for information on how to expose the new training application type and/or interop plugin logic to the GIFT course author.
Add Plugin to Course Editor¶
While your plugin now exists in GIFT, it will take some additional work to make it useable in GIFT's course editor, so that you can add it to a course.
The following files will need to be edited to reference your plugin:- DomainKnowledgeUtil.java
- CoursePresenter.java
- CourseHelp.java
- CourseView.java
- CourseViewImpl.java
- CourseViewImpl.ui.xml
- CourseElementUtil.java
- ScenarioAdaptationEditor.java
An explanation of the necessary changes is below.
CourseView and CourseViewImpl¶
The file GIFT\src\mil\arl\gift\tools\authoring\server\gat\client\view\course\CouseViewImpl.ui.xml contains a group of FlowPanels that represent course object options. To create an option for your new plugin, add a new FlowPanel with code like the following.
<g:FlowPanel addStyleNames='courseTransitionObject'
title='drag to add to course' ui:field='mySampleAppButton'>
<g:HTMLPanel addStyleNames='courseTransitionObjectInner'>
<img draggable='false' width='24px' height='24px'
src='images/transitions/ta.png' />
<g:HTML>
My Sample Application
</g:HTML>
</g:HTMLPanel>
</g:FlowPanel>
Now, the corresponding Java class must be modified to use this new FlowPanel, in GIFT\src\mil\arl\gift\tools\authoring\server\gat\client\view\course\CouseViewImpl.java.
Add a UiField Widget with a name that matches the name of your FlowPanel's ui:field, as shown in the following:
@UiField
protected Widget mySampleAppButton;
In that file's CourseViewImpl() method, there is a section that makes the course objects in the left panel draggable. Add a line to include your new app button, like the following:
mySampleAppButton.getElement().setDraggable("true");
In that file, several methods follow the pattern of "getAdd<PluginName>Button". Add a new method to return your new Widget like the following.
@Override
public Widget getAddMySampleAppButton() {
return mySampleAppButton;
}
Now, because that is an override, you will need to create a matching method in GIFT\src\mil\arl\gift\tools\authoring\server\gat\client\view\course\CourseView.java
/**
* Gets the button used to create the My Sample App course objects
*
* @return the Sample App button
*/
Widget getAddMySampleAppButton();
Updating CoursePresenter¶
In GIFT\src\mil\arl\gift\tools\authoring\server\gat\client\presenter\course\CoursePresenter.java a method needs to be added to add functionality for the course object button you created above. Add a method like the following:
private void addMySampleAppHandlers() {
editCourseView.getAddMySampleAppButton().addDomHandler(new DragStartHandler() {
@Override
public void onDragStart(DragStartEvent event) {
//Prevent drag events when the course is read only
if(readOnly) {
event.preventDefault();
return;
}
event.setData("text", "anything"); //need to add some drag data for dragging to work in Firefox
transitionBeingDragged = generateTrainingApp(TrainingApplicationEnum.MY_APPLICATION);
//scroll the course tree as the user drags this object
editCourseView.getCourseTree().startDragScrolling();
}
}, DragStartEvent.getType());
editCourseView.getAddMySampleAppButton().addDomHandler(new ClickHandler() {
@Override
public void onClick(ClickEvent arg0) {
editCourseView.getHelpHTML().setHTML(CourseHelp.EXAMPLE_APP_COURSE_OBJECT_HELP);
//prevent the CourseViewImpl RootPanel from clearing the help text with its click handler
arg0.stopPropagation();
}
}, ClickEvent.getType());
editCourseView.getAddMySampleAppButton().addDomHandler(new DragEndHandler() {
@Override
public void onDragEnd(DragEndEvent event) {
//stop scrolling the course tree as the user drags this object
editCourseView.getCourseTree().endDragScrolling();
}
}, DragEndEvent.getType());
}
Then, call your new method in the block of calls under the comment that labels "Course object action handlers".
Updating DomainKnowledgeUtil¶
The file GIFT\src\mil\arl\gift\tools\authoring\common\util\DomainKnowledgeUtil.java assigns condition classes to constant values for each of the interop plugins which use GIFT conditions. To allow for this in your course object, add code like the following:
// My Sample App (if adding/removing conditions, please update DomainKnowledgeUtilTest.getConditionInfosForTrainingAppTest())
Class<?>[] MY_SAMPLE_APP_CONDITIONS =
{mil.arl.gift.domain.knowledge.condition.StringMatchingExampleCondition.class,
mil.arl.gift.domain.knowledge.condition.ApplicationCompletedCondition.class,
mil.arl.gift.domain.knowledge.condition.TimerCondition.class,
mil.arl.gift.domain.knowledge.condition.SimpleSurveyAssessmentCondition.class,
mil.arl.gift.domain.knowledge.condition.ObservedAssessmentCondition.class};
trainingAppConditionClassesMap.put(TrainingApplicationEnum.MY_APPLICATION, MY_SAMPLE_APP_CONDITIONS);
In the example above, the conditions are those used with our existing Demo Application. If you want to use a different set of conditions, or create new conditions of your own, they will need to be included here to show up in the course object's list of supported conditions.
Updating CourseElementUtil¶
The file GIFT\src\mil\arl\gift\tools\authoring\server\gat\client\view\course\tree\CourseElementUtil.java must be updated to account for any new course objects. In getTypeDisplayName(), one section returns a content type depending on the application being used. You'll need to add a clause for your new application like this:
} else if (TrainingApplicationEnum.MY_APPLICATION.equals(type)) {
return getContentTypeTitle(ContentTypeEnum.MY_APPLICATION);
}
Similarly, in getContentTypeTitle(), you'll need to add a case for your new application like this, before the default case:
case MY_APPLICATION:
return "My sample application";
default:
return "UNKNOWN TYPE - "+contentType;
In getContentTypeGraphic(), you'll also need to set the course object's image by adding a case like the following:
case MY_APPLICATION:
image = new Image(TrainingApplicationEnum.getTrainingAppTypeIcon(TrainingApplicationEnum.MY_APPLICATION));
image.setWidth(imageWidth);
return image.getElement();
Test¶
If you have followed the previous sections than you are now ready to test your new GIFT interop plugin. The following section will have you build a course that references your interop plugin, launch a simple example training application provided by GIFT and finally execute your course to observer the various interactions.
Configure Course to use Interop Plugin¶
In order for GIFT to know when to use your Gateway module interop plugin (and more importantly your training application), you need to author a GIFT course which references the plugin class.
Below is an example on manually creating a simple course, however if you prefer to take a shortcut then use the XML provided after these steps. For more detailed, but generic, instructions on using the CAT, please refer to the Authoring test procedures located in GIFT\docs\tests\.
1. Launch the Course Authoring Tool (CAT) by double clicking on the GIFT\scripts\tools\launchCAT.bat script.
Of course you can use the GIFT Authoring Tool as well by launching GIFT and traversing to the course authoring part of the GAT.
2. Provide a name for the course. For our example use “My Plugin Test”.
3. Provide a description for the course. For our example use “This course will test my new training application interop plugin.”.
4. Expand the transitions section and choose “Guidance” as your first transition to provide some text before the course starts using your interop plugin. This is optional but is normally a good approach for GIFT courses as it eases the learner into what the course they are about to complete.
- For more information on course transitions please refer to the GIFT Domain Course File Documentation.
5. Expand the Guidance element followed by the “Choice:message” element.
6. Provide a message content value of “You are about to use your new interop plugin.”
7. Add a new transition after the Guidance transition you just authored (right click on the “Choice” text next to “Guidance” and select “Insert Node After”).
8. Select “TrainingApplication” for this new choice element.
9. Expand the training application element.
10. Expand the “dkfRef” element.
11. Use the ellipses button for the file element and select “simplest.dkf.xml”. This DKF merely looks for the completion of a lesson which is accomplished with a SIMAN ‘Stop Freeze’ message being sent from the Gateway module to the Domain module.
If you authored your own dkf.xml please replace references to “simplest.dkf.xml” with yours in the procedures in this section.
12. Expand the “interops” element and descendant elements until you see “InteropImpl” element.
13. Select “gateway.interops.example.MyPluginInterface” followed by the OK button.
14. Expand the “InteropInputs” element.
15. Select “MyPluginInputs” which was created earlier when editing the course XML schema file.
If you choose the simpler approach of not editing the course XML schema file, choose “CustomInteropInputs”.
16. Expand the choice until you reach an element where you can type in a Scenario Name. For our example type in “Scenario X”.
17. Choose the Validate File menu option in the CAT to validate the XML inputs you have just authored. Validation will happened against the XML schema (.xsd) as well as GIFT code that will execute validation on Files, URLs, etc.
18. When finished, choose the “Save As” File menu option in the CAT. Name the file “My Plugin Test”.
19. Close the CAT.
For reference here is the content of the course XML file named “My Plugin Test.course.xml” that you authored in the previous steps:
<?xml version="1.0" encoding="UTF-8"?>
<!--
This file was created with the GIFT Course Authoring Tool (CAT).
It contains information about the flow of a course in GIFT.
-->
<Course name="My Plugin Test" version="4.0.1" xmlns:c0="http://GIFT.com/common" xmlns:p0="http://GIFT.com/pedagogicalStrategy" xmlns:x0="http://www.w3.org/2001/XMLSchema">
<description>This course will test my new training application interop plugin.</description>
<transitions>
<Guidance>
<message>
<content>You are about to use your new interop plugin.</content>
</message>
</Guidance>
<TrainingApplication>
<dkfRef>
<file>simplest.dkf.xml</file>
</dkfRef>
<interops>
<interop>
<InteropImpl>gateway.interop.myplugin.MyPluginInterface</InteropImpl>
<InteropInputs>
<MyPluginInputs>
<loadArgs>
<ScenarioName>Scenario X<ScenarioName/>
</loadArgs>
</MyPluginInputs>
</InteropInputs>
</interop>
</interops>
<finishedWhen>Stopped</finishedWhen>
</TrainingApplication>
</transitions>
</Course>
Setup Training Application¶
Instead of having you create a Training Application, we have created a simple one for you to test with located in “Training.Apps\SimpleExampleTrainingApplication” (refer to this path for the following procedures).
If you choose to use the existing Gateway interop plugin of “mil.arl.gift.gateway.interop.simple.SimpleExampleTAPluginInterface.java”, than you can just start the “Simple Example Training Application” as specified in Run The application.
For those readers who followed this guide and created the Gateway interop plugin named “MyPluginInterface.java” there are a few changes we need to make in the “Simple Example Training Application” related to XML-RPC methods hosted by the GIFT XML-RPC server.
Prerequisite: installed Microsoft Visual C# 2010 [freely available online] (let’s call this MVCS for short) which will be need to compile the changes we are going to make. If you can’t install this program then please read through the guide again and make sure to not create a new Interop plugin class but instead use the existing SimpleExampleTAPluginInterface class.
1. Open the SimpleExampleTrainingApplication.sln in MVCS.
2. We need to open Form1.cs. Locate the Solution Explorer window and right-click on “Form1.cs”.
3. Select “view code”.
4. Search for “mil.arl.gift.gateway.interop.simple.SimpleExampleTAPluginInterface$SimpleExampleTAPluginXMLRPC”. These are the 2 methods hosted by the GIFT XML-RPC server. Currently they are referencing the SimpleExampleTAPluginInterface Gateway interop interface class.
5. Copy the 2 methods that have this string in their tags in the “ISumAndDiff” interface and paste them above/below.
6. Replace “simple.SimpleExampleTAPluginInterface$SimpleExampleTAPluginXMLRPC” with “myplugin.MyPluginInterface$MyPluginXMLRPC”
7. Comment the existing methods you copied by highlighting them and then selecting “Edit->Comment Selection”. The code won’t compile if there are multiple methods with the same name.
8. Save All by using the “File” menu option or “Ctrl+Shift+S”.
9. Build solution by using the “Debug” menu option or “F6”.
Run Test¶
This test will involve running a GIFT course, the training application and the GIFT monitor application in order to test the newly created interop plugin.
- Prerequisite: Make sure you compiled any changes made to GIFT as mentioned in the previous section of Compile changes made.
1. Launch GIFT in Developer Mode (aka Power User Mode) by following the “Power User Mode” instructions in GIFT Configuration Settings.
2. Login to GIFT, select “View Available Courses” and then select your course named “My Plugin Test”.
3. Once the course has started, open the Monitor module window.
4. Select the “Main” tab on the top row of tabs followed by its sub-tab named “Active Sessions”.
5. Select what should be the only entry in the “Active Domain Sessions” text area followed by selecting the “Monitor Domain Session” button. A new tab named “Domain Session # Msgs” should appear on the top row of tabs.
6. Click on the Domain session message tab to view domain session messages.
7. Let us filter out all messages that have been received so far by clicking on one of the message type checkboxes on the right hand side and then hold down Left Ctrl button while clicking that same checkbox again.
8. Go back to the web browser tutor and continue past the initial Guidance course transition you previously authored. Now you are in the training application course transition. The “Simple Example Training Application” program should automatically launch and its window will be presented to you in the foreground (in addition an AutoHotKey script was ran to keep the window as the top most window for now). If for some reason the program didn’t start you can manually start it by following the Run The application instructions.
9. Return back to the monitor and see the various SIMAN messages sent between the Domain module and the Gateway module.
10. Then open the Training Application window and see the entries added to the “Received” list box.
11. Now click on “Button 1” and “Button 2” on the training application and look at the messages in the monitor.
12. When finished click on “Button 3” on the training application to end the lesson (i.e. the training application course transition).
13. A final guidance page will be shown. Click the continue button.
14. The training application program should gracefully close at this point and the tutor should return you to the LMS history screen.
Creating a performance assessment Condition¶
The GIFT domain module is responsible for performance assessments of the user during the execution of a lesson in a training application (i.e. during a training application course transition). The algorithms for assessing can be linked to condition classes in “mil.arl.gift.domain.knowledge.condition” package. There are some caveats to this including using an external assessment engine (Integrating an External Assessment Engine) and being unable to separate an existing assessment engine from a training application (Existing Training Application with Assessment Engine/Tutor). Exactly how condition classes are configured and what the concept hierarchy looks like for a particular domain/lesson can be seen in the associated DKF for a training application course transition (Creating DKF and Creating a Course).
In this guide (the following steps), the user will create a new condition class that will be responsible for consuming a game state message and then providing an assessment based on the information contained within that message.
The following procedures provide an example on how to create a new Domain module assessment condition using the Eclipse IDE. In addition it will utilize the “Simple Example Training Application” program (Simple Example Training Application) and associated mil.arl.gift.gateway.interop.simple.SimpleExampleTAPluginInterface Gateway module interop plugin provided by GIFT to create game state messages that this new condition will assess.
Update dkf.xsd¶
1. The new condition input will need to be added into the dkf.xsd as a new element. The element will look similar to this:
<xs:element name="TestExampleInput">
<xs:annotation>
<xs:appinfo>
<fg:node-info exposed="true" message="Test Condition Input">
<fg:message>This input is used to configure the Timer condition for assessment.</fg:message>
</fg:node-info>
</xs:appinfo>
</xs:annotation>
<xs:complexType>
<xs:sequence>
<xs:element name="repeatable" type="common:booleanEnum" default="false">
<xs:annotation>
<xs:appinfo>
<fg:node-info exposed="true" message="Repeatable">
<fg:message>Whether the time based assessment should be repeated every 'interval' milliseconds.</fg:message>
</fg:node-info>
</xs:appinfo>
</xs:annotation>
</xs:element>
<xs:element name="interval">
<xs:annotation>
<xs:appinfo>
<fg:node-info exposed="true" message="Time Interval (sec)">
<fg:message>The amount of time in seconds before the condition assessment happens.</fg:message>
</fg:node-info>
</xs:appinfo>
</xs:annotation>
<xs:simpleType>
<xs:restriction base="xs:decimal">
<xs:minExclusive value="0.0"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
2. The condition input that was just created needs to be added into the "input" element's list of choices. (to find this element you can search for name="input")
<xs:element ref="TestExampleInput"/>
Compile changes made¶
In order to use the new condition input type you made, you will need to build GIFT in order for it to generate the dkf classes. To build GIFT double click on GIFT\build.bat.
Create the Java Class¶
1. Right click on the “mil.arl.gift.domain.knowledge.condition” package in the Package Explorer in Eclipse.2. In the menu go to “New -> Class”
3. In the “New Java Class”, in the “Name” textbox type in: StringCompareExampleCondition
Ø Conditions are meant to be lesson, scenario, file, domain and training application independent (as much as possible) in order for it to be reused by other courses that could provide a different initial configuration as input to the class. In addition, the file name of the condition should describe the condition purposes, i.e. what does the condition assess. In this case your condition class will be comparing a string to see if it matches a specific, pre-configured, value.
4. In the “Superclass:” text field type in: mil.arl.gift.domain.knowledge.condition.AbstractCondition
5. Click finish.
6. The first thing you will do is create a description of what your condition assesses. The description needs to point to an html file (naming standard: <insert name here>.GIFT Domain condition description.html) which should be placed in "<branch>\GIFT\docs\conditions". This description can be used by authoring tools to provide additional information to the author beyond the name of the condition. Add the following below the class definition line that looks like “public class StringCompareExampleCondition extends AbstractCondition”:
private static final ConditionDescription DESCRIPTION = new FileDescription(new File("docs" + File.separator + "conditions" + File.separator + "<MyCondition>.GIFT Domain condition description.html"), "<My Condition Display Name>");
- Then make your “getDescription” method look like this:
@Override public ConditionDescription getDescription() { return DESCRIPTION; }
7. Next we will identify and register for the game state messages this condition needs in order to perform its assessments. Add the following below the class definition line:
/**
* contains the types of GIFT messages this condition needs in order to provide assessments
*/
private static List<MessageTypeEnum> simulationInterests;
static{
simulationInterests = new ArrayList<MessageTypeEnum>();
simulationInterests.add(MessageTypeEnum.SIMPLE_EXAMPLE_STATE);
}
The “Simple Example State” message type enumeration is associated with the Java class mil.arl.gift.common.ta.state.SimpleExampleState. To determine the mapping between a MessageTypeEnum value and a game state Java class refer to mil.arl.gift.net.api.message.codec.json.JSONMapper (admittedly a more straightforward link between a condition and the game state objects is needed in the future).
Then make your “getSimulationInterest” method look like this:
@Override
public List<MessageTypeEnum> getSimulationInterests() {
return simulationInterests;
}
8. Now create a class level attribute that will store the configurable string value this condition is comparing against with each incoming game state message. Add this line above your “handleTrainingAppGameState” method:
/** a key to look for in the game state message */
private String conditionKey;
9. Time to create the class constructors for your class. The first is the default constructor which will be used by authoring tools. Add this above “handleTrainingAppGameState” method:
/**
* Default constructor - required for authoring logic
*/
public StringCompareExampleCondition (){
}
The second class constructor will have an input parameter can contain the necessary configuration information for this condition. In this example the input will contain the string value we are trying to match. Add this above “handleTrainingAppGameState” method:
/**
* Class constructor
*
* @param input configuration parameters for this condition
*/
public StringCompareExampleCondition (generated.dkf.GenericConditionInput input){
//assuming that there are only 1 name:value pair in the list
List<generated.dkf.Nvpair> pairs = input.getNvpair();
generated.dkf.Nvpair pair = pairs.get(0);
//set this condition's key value which will be used for string matching
//with incoming game state messages
conditionKey = pair.getValue();
}
You may have noticed that there in an assumption in that constructor that indicates the first name:value pair in the input will be used to set the conditionKey value. This was used for simplicity so the author didn’t have to edit the DKF schema to create a specific condition input definition OR create logic which searched the list of Nvpair for a particular name value (e.g. “key string”). When you go to run the course to test this condition all you have to do is provide the condition a single name:value pair and the name is irrelavent.
10. Next create the assessment logic in the condition class for when a game state message is received. Change your “handleTrainingAppGameState” message to:
< @Override
public AssessmentLevelEnum handleTrainingAppGameState(Message message) {
AssessmentLevelEnum level = null;
//since this class is only registered to receive 1 type of message, casting w/o checking type
SimpleExampleState state = (SimpleExampleState)message.getPayload();
if(state.getVar().equals(conditionKey)){
//found the condition key in the game state message content
logger.info("Found "+conditionKey+" in the ExampleState game state message.");
// This is used to indicate that an important event for this condition needs to be scored.
// In this case we could use the scoring logic to indicate how many times the string was matched
// and to start a timer to determine how much time elapses until this condition is not satisfied.
// Scoring is used as an overall assessment of a lesson and is normally presented in
// After Action Review (AAR) and stored in an LMS/LRS.
// Note: the DKF responsible for configuring this condition must have scoring attributes in order
// for this call to do anything.
scoringEventStarted();
level = AssessmentLevelEnum.AT_EXPECTATION;
updateAssessment(level);
}else{
// This is used to indicate that an important event for this condition needs to be scored.
// In this case we are using it to indicate the amount of time since this condition was satisfied.
// Note: the DKF responsible for configuring this condition must have scoring attributes in order
// for this call to do anything.
scoringEventEnded();
level = AssessmentLevelEnum.BELOW_EXPECTATION;
updateAssessment(level);
}
return level;
}
In essence the logic you just created will check if the value for the game state message attribute named “var” matches the string that was set from input to this condition during the class constructor call. If the string matches, an ‘At Expectation’ performance assessment result is returned for this condition. Otherwise a ‘Below Expectation’ performance assessment result is returned. The assessment result is then wrapped up in the performance node hierarchy defined by the DKF used to instantiate this condition, where each node (i.e. Task/Concept) will have a single enumerated assessment value based on its descendants.
11. Finally add the following imports lines above the class definition line:
import java.util.ArrayList;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import mil.arl.gift.common.course.dkf.ConditionDescription;
import mil.arl.gift.common.course.dkf.ConditionDescription.FileDescription;
import mil.arl.gift.common.enums.AssessmentLevelEnum;
import mil.arl.gift.common.enums.MessageTypeEnum;
import mil.arl.gift.common.ta.state.SimpleExampleState;
import mil.arl.gift.net.api.message.Message;
And add a logger instance under the class definition line:
/** instance of the logger */
private static Logger logger = LoggerFactory.getLogger(StringCompareExampleCondition.class);
Update DomainKnowledgeUtil.java¶
In DomainKnowledgeUtil there is a static block where the conditions are mapped to specific training applications. Add the new condition to all relevant training application groups. Note that it may relate to multiple training applications.
Update ConditionInputCache.java¶
1. Add the new condition to the list of ConditionType enums. It's name should be "domain.knowledge.condition.<condition class name>".
2. Add the new condition to the createInstance() switch block.
Create the editor implementation .java and .ui.xml¶
1. Right click on the “mil.arl.gift.tools.authoring.server.gat.client.view.dkf.task.condition” package in the Package Explorer in Eclipse.
2. In the menu go to “New -> Class”
3. In the “New Java Class”, in the “Name” textbox type in: "<condition name>InputEditorImpl" (e.g. TestExampleInputEditorImpl).
4. In the “Superclass:” text field type in: mil.arl.gift.tools.authoring.server.gat.client.view.dkf.task.condition.ConditionInputPanel<T>
5. Click finish.
6. Replace the "T" with your DKF generated condition input class
7. Right click on the “mil.arl.gift.tools.authoring.server.gat.client.view.dkf.task.condition” package in the Package Explorer in Eclipse.
8. In the menu go to “New -> File”
9. In the “New File”, in the “File Name” textbox type in: "<condition name>InputEditorImpl.ui.xml" (e.g. TestExampleInputEditorImpl.ui.xml).
10. Click finish.
Update ConditionPanel.java¶
1. Go to the onInputChanged method and add the new condition into the switch block. Refer to the other cases to populate the contents of the case.
Add Validation¶
1. Open ScenarioValidatorUtility.java
2. Create a method to validate the condition data.
3. Go to the validateObject() method add your condition class to the if/else block under the comment
// check if validationObject is a condition type
Compile changes made¶
In order to use the Java code changes you made, you will need to build GIFT. To build GIFT double click on GIFT\build.bat and wait for it to successfully complete (i.e. the dialog will close automatically when finished). If there is an error the dialog will remain open allowing you to review the build error.
Run Test¶
This test will involve running a GIFT course and the “Simple Example training application” in order to test your new condition class. The course will need a training application transition which references a DKF that uses your condition implementation. Instead of creating one from scratch (which you are free to do – see Creating a Course and Creating DKF for general information), we will be editing an existing course called “Simple Example TA Test”. That course is already setup with a training application course transition that utilizes the SimpleExampleTAPluginInterface to communicate between GIFT and the training application. However the DKF it references uses a different Domain module performance assessment condition and you need it to use yours for this test. The DKF does contain 3 concepts which use a similar condition implementation class as the one you created and the file already contains condition input for the string comparison logic.
Setup Course/DKF¶
1. Copy the existing Domain folder named “Simple Example Training App Course” in order to keep the original. Name the new folder “My Condition Test Course”.
2. Open exampleCourse.course.xml in the Course Authoring Tool (CAT). To launch the CAT double click on GIFT\scripts\tools\launchCAT.bat.
3. In the CAT:
i. Change the name of the course to “My Condition Test”.
ii. Optionally, change the description of your course.
iii. Press the expansion bar next to “transitions” and keep expanding until you see a “TrainingApplication” course transition node.
iv. Press the expansion bar for that “TrainingApplication” course transition node.
v. Press the expansion bar for the “dkfRef” followed by the “…” ellipses button.
vi. Select “My Condition Test Course\example.dkf.xml” from the drop down (or use the Browse button if it is not there) and then select the OK button.
vii. Save the course.4. Open example.dkf.xml in the DKF Authoring Tool (DAT). To launch the DAT double click on GIFT\scripts\tools\launchDAT.bat.
5. In the DAT:
i. Keep expanding the various nodes along this path: {assessment -> tasks -> array -> task -> concepts -> array} by pressing the expansion bars. You should now see 4 concepts.
ii. You will now be replacing the condition implementation class with your class in each of the first 3 concepts. Expand the first concept along this path: {concept -> choice:conditions -> array -> condition}. Then select the “…” ellipses button next to “conditionImpl”. Select “domain.knowledge.condition.StringCompareExampleCondition” from the drop down (or use the Browse button if it is not there) and then select the OK button. Do the same for the next 2 concepts.
iii. Save the DKF.
Run Course¶
1. Launch GIFT in Developer Mode (aka Power User Mode) by following the “Power User Mode” instructions in GIFT Run instructions.
2. Login to GIFT, select “View Available Courses” and then select your course named “My Condition Test”.
3. Once the course has started, open the Monitor module window.
4. Select the “Main” tab on the top row of tabs followed by its sub-tab named “Active Sessions”.
5. Select what should be the only entry in the “Active Domain Sessions” text area followed by selecting the “Monitor Domain Session” button. A new tab named “Domain Session # Msgs” should appear on the top row of tabs.
6. Click on the Domain session message tab to view domain session messages.
7. Let us filter out all messages that have been received so far by clicking on one of the message type checkboxes on the right hand side and then hold down Left Ctrl button while clicking that same checkbox again.
8. Go back to the web browser tutor and continue past the initial Guidance course transition you previously authored. Now you are in the training application course transition. The “Simple Example Training Application” program should automatically launch and its window will be presented to you in the foreground (in addition an AutoHotKey script was ran to keep the window as the top most window for now). If for some reason the program didn’t start you can manually start it by following the Run The application instructions.
9. Return back to the monitor and see the various SIMAN messages sent between the Domain module and the Gateway module.
10. Then open the Training Application window and see the entries added to the “Received” list box.
11. Now click on “Button 1” on the training application and look at the messages in the monitor. Find the Performance Assessment message(s) on the left hand side. The first one should be the initial performance assessment sent when the DKF was first used to configure the hierarchy and contains the default assessment values. The second one contains the assessment value created based on your condition class (in the first of your four concepts in the hierarchy) receiving a game state message with the value of “button 1” which matched that conditions input parameter. Try pressing “button 2” and then look at the resulting assessment message. Feel free to keep switching back and forth between these 2 buttons.
12. When you are finished click on “Button 3” on the training application to end the lesson (i.e. the training application course transition).
13. A final guidance page will be shown. Click the continue button.
14. The training application program should gracefully close at this point and the tutor should return you to the LMS history screen.
Analysis of Training Applications and preferred condition classes¶
This analysis was conducted 08/18
The last two columns ('Extraneous Conditions' and 'Consumed Messages') contain the condition classes that consume one or more messages produced by the Gateway module interop plugin classes used to communicate with the training application. These condition classes could be used based on message types alone but are not intended to be used to assess the learner in those training applications as of now. The italicized message types in 'Consumed Messages' column are the message types that match.
| Training Application | Produced Messages | Preferred Condition | Consumed Messages | Extraneous Conditions | Consumed Messages |
| All | SIMAN | TimerCondition | SIMAN | ||
| ObservedAssessmentCondition | |||||
| SimpleSurveyAssessmentCondition | SIMAN | ||||
| PowerPoint (PPTInterface) | StopFreeze | ApplicationCompletedCondition | StartResume | SIMILEInterfaceCondition | GenericJSONState |
| StartResume | StopFreeze | EntityState | |||
| PowerPointState | PowerPointUnderDwellCondition | PowerPointState | WeaponFire | ||
| StopFreeze | Detonation | ||||
| PowerPointOverDwellCondition | PowerPointState | StopFreeze | |||
| StopFreeze | |||||
| VBS, VR-Engage [or any app that produces DIS traffic] (DISInterface) | StopFreeze | CorridorBoundaryCondition | EntityState | HasCollidedCondition | Collision |
| EntityState | CheckpointProgressCondition | EntityState | PowerPointUnderDwellCondition | PowerPointState | |
| WeaponFire | CorridorPostureCondition | EntityState | StopFreeze | ||
| Detonation | CheckpointPaceCondition | EntityState | PowerPointOverDwellCondition | PowerPointState | |
| StartResume | RulesOfEngagementCondition | WeaponFire | StopFreeze | ||
| Collision | ApplicationCompletedCondition | StartResume | SIMILEInterfaceCondition | GenericJSONState | |
| StopFreeze | EntityState | ||||
| AvoidLocationCondition | EntityState | WeaponFire | |||
| IdentifyPOIsCondition | LoSResult | Detonation | |||
| EntityState | StopFreeze | ||||
| LifeformTargetAccuracyCondition | EntityState | HasMovedExcavatorComponentCondition | EntityState | ||
| Detonation | |||||
| EnterAreaCondition | EntityState | ||||
| EliminateHostilesCondition | EntityState | ||||
| HaltCondition | EntityState | ||||
| SpacingCondition | EntityState | ||||
| SpeedLimitCondition | EntityState | ||||
| VBS (VBSPlugin) | StopFreeze | ApplicationCompletedCondition | StartResume | PowerPointUnderDwellCondition | PowerPointState |
| StartResume | StopFreeze | StopFreeze | |||
| LoSResult | IdentifyPOIsCondition | LoSResult | PowerPointOverDwellCondition | PowerPointState | |
| EntityState | StopFreeze | ||||
| SIMILEInterfaceCondition | GenericJSONState | ||||
| EntityState | |||||
| WeaponFire | |||||
| Detonation | |||||
| StopFreeze | |||||
| VR-Engage (VREngageInterface) | StopFreeze | ApplicationCompletedCondition | StartResume | PowerPointUnderDwellCondition | PowerPointState |
| StartResume | StopFreeze | StopFreeze | |||
| LoSResult | IdentifyPOIsCondition | LoSResult | PowerPointOverDwellCondition | PowerPointState | |
| EntityState | StopFreeze | ||||
| SIMILEInterfaceCondition | GenericJSONState | ||||
| EntityState | |||||
| WeaponFire | |||||
| Detonation | |||||
| StopFreeze | |||||
| Unity Embedded | SimpleExampleState | StringMatchingExampleCondition | SimpleExampleState | PowerPointUnderDwellCondition | PowerPointState |
| StopFreeze | ApplicationCompletedCondition | StartResume | StopFreeze | ||
| StopFreeze | PowerPointOverDwellCondition | PowerPointState | |||
| StopFreeze | |||||
| SIMILEInterfaceCondition | GenericJSONState | ||||
| EntityState | |||||
| WeaponFire | |||||
| Detonation | |||||
| StopFreeze | |||||
| TC3 | StopFreeze | SIMILEInterfaceCondition | GenericJSONState | PowerPointUnderDwellCondition | PowerPointState |
| StartResume | ApplicationCompletedCondition | StartResume | StopFreeze | ||
| LOSResult | StopFreeze | PowerPointOverDwellCondition | PowerPointState | ||
| GenericJSONState | StopFreeze | ||||
| IdentifyPOIsCondition | LoSResult | ||||
| EntityState | |||||
| DE Testbed (DIS Interface) | StopFreeze | HasMovedExcavatorComponentCondition | EntityState | CorridorBoundaryCondition | EntityState |
| EntityState | HasCollidedCondition | Collision | CheckpointProgressCondition | EntityState | |
| WeaponFire | ApplicationCompletedCondition | StartResume | CorridorPostureCondition | EntityState | |
| Detonation | StopFreeze | CheckpointPaceCondition | EntityState | ||
| StartResume | RulesOfEngagementCondition | WeaponFire | |||
| Collision | AvoidLocationCondition | EntityState | |||
| IdentifyPOIsCondition | LoSResult | ||||
| EntityState | |||||
| LifeformTargetAccuracyCondition | EntityState | ||||
| Detonation | |||||
| EnterAreaCondition | EntityState | ||||
| EliminateHostilesCondition | EntityState | ||||
| HaltCondition | EntityState | ||||
| SpacingCondition | EntityState | ||||
| SpeedLimitCondition | EntityState | ||||
| DE Testbed (XMLRPCInterface) | |||||
| SimpleExampleTA | StopFreeze | StringMatchingExampleCondition | SimpleExampleState | PowerPointUnderDwellCondition | PowerPointState |
| SimpleExampleState | ApplicationCompletedCondition | StartResume | StopFreeze | ||
| StopFreeze | PowerPointOverDwellCondition | PowerPointState | |||
| StopFreeze | |||||
| SIMILEInterfaceCondition | GenericJSONState | ||||
| EntityState | |||||
| WeaponFire | |||||
| Detonation | |||||
| StopFreeze | |||||
| Sudoku | StopFreeze | ApplicationCompletedCondition | StartResume | PowerPointUnderDwellCondition | PowerPointState |
| StartResume | StopFreeze | StopFreeze | |||
| SimpleExampleState | PowerPointOverDwellCondition | PowerPointState | |||
| StopFreeze | |||||
| SIMILEInterfaceCondition | GenericJSONState | ||||
| EntityState | |||||
| WeaponFire | |||||
| Detonation | |||||
| StopFreeze | |||||
| StringMatchingExampleCondition | SimpleExampleState | ||||
| SCATT | RifleShot | MarksmanshipPrecisionCondition | RifleShot | ||
| MarksmanshipSessionCompleteCondition | RifleShot | ||||
| NumberOfShotsFiredCondition | RifleShot | ||||
| ARES | TrainingAppSurveyResponse | ||||
| TrainingAppSurveySubmit | |||||
Domain Conditions Analysis¶
overall assessment types:- count - the number of times a violation of the condition happens
- vioaltion time - how long the violation happened over all violations
- completion time - how long it took to finish the condition
| Name | Class | Requires Learner | overall assessment type(s) |
| Application Completed | ApplicationCompletedCondition | count, completion time | |
| AutoTutor Conversation | AutoTutorWebServiceInterfaceCondition | ||
| Avoid Location | AvoidLocationCondition | YES | count, violation time |
| Checkpoint Pace | CheckpointPaceCondition | YES | count, violation time, completion time |
| Checkpoint Progress | CheckpointProgressCondition | YES | count, violation time, completion time |
| Corridor Boundary | CorridorBoundaryCondition | YES | count, violation time |
| Corridor Posture | CorridorPostureCondition | YES | count, violation time |
| Eliminate Hostiles | EliminateHostilesCondition | count, violation time, completion time | |
| Enter Area | EnterAreaCondition | YES | count |
| Explosive Hazard Spot Report | ExplosiveHazardSpotReportCondition | count, completion time | |
| Halt | HaltCondition | Yes | count, violation time, completion time |
| Has Collided | HasCollidedCondition | count | |
| Has Moved Excavator Component | HasMovedExcavatorComponentCondition | count, violation time, completion time | |
| Identify Locations | IdentifyPOIsCondition | YES | count, violation time, completion time |
| Lifeform Target Accuracy | LifeformTargetAccuracyCondition | YES | count |
| Marksmanship Precision | MarksmanshipPrecisionCondition | count, completion time | |
| Marksmanship Session Complete | MarksmanshipSessionCompleteCondition | count, completion time | |
| Nine line report | NineLineReportCondition | count, completion time | |
| Number of shots fired | NumberOfShotsFiredCondition | count, completion time | |
| Observed assessment | ObservedAssessmentCondition | ||
| Pace count | PaceCountCondition | Yes | completion time |
| PowerPoint Over Dwell | PowerPointOverDwellCondition | count, completion time | |
| PowerPoint Under Dwell | PowerPointUnderDwellCondition | count, completion time | |
| Rules of Engagement | RulesOfEngagementCondition | YES | count |
| SIMILE | SIMILEInterfaceCondition | ||
| Simple Survey Assessment | SimpleSurveyAssessmentCondition | count, completion time | |
| Spacing | SpacingCondition | Yes | count, violation time |
| Speed limit | SpeedLimitCondition | Yes | count, violation time |
| Spot Report | SpotReportCondition | count, completion time | |
| String Matching | StringMatchingExampleCondition | count, violation time | |
| Timer | TimerCondition | count, completion time | |
| Use Radio | UseRadioCondition | count, completion time |
Creating DKF¶
A DKF contains scenario dependent information to configure the domain module for performance assessment during a Training Application course transition. The configuration information includes items such as the concept hierarchy and how each concept is assessment based on conditions, over-all scoring rules, instructional strategies and tactics. DKFs files can be authored using the DKF Authoring Tool (DAT) which provides a structured view of the XML tree along with the ability to validate your authored content against the XML schema (xsd) and various GIFT validation logic (e.g. URL and File checks).
For more information on DKFs refer to Gift Authoring Guide and GIFT Domain Knowledge File.
To start the DAT, double click on GIFT\scripts\tools\launchDAT.bat.
One way to get familiar with authoring a DKF file is to follow the GIFT Authoring Test procedures located at GIFT\docs\testing\GIFT Test Procedure Authoring.xlsx. Another suggestion is to view the course’s delivered with GIFT by first starting with the various “Course Technical Details.README.docx” documents associated with almost every course. These documents describe the important features used in each course and should be used for referencing examples of what is supported and has been tested thus far. From there you can open the course’s DKFs in the DAT or execute them via the respective course in GIFT.
Also note that there are several, simple example DKFs provided for at the root of the GIFT Domain folder:
- simplest.dkf.xml – one of the simplest DKFs that provides the most value in that it has a single condition which ends the lesson upon receiving the SIMAN Stop_Freeze message. This is useful for determining when a Training Application scenario/lesson has finished and the GIFT course should continue onto the next course transition.
- AutoTutorSession.examle.dkf.xml – contains the logic used to assess an AutoTutor chat that is initialized using an AutoTutor SKO XML file.
For further assistance please search for forums or post a new topic at https://gifttutoring.org/projects/gift/boards.
Creating a Course¶
A course is the top level element in GIFT that orchestrates what will be presented to the user at various points in time. Each course consist of a single course.xml named file that resides somewhere in the GIFT Domain folder. Course files can be authored using either the web-based GIFT Authoring Tool (GAT) or the desktop-based Course Authoring Tool (CAT). The GAT is meant to be an easier to use interface while the CAT is for low level developers that like to get up close and personal with XML (i.e. provides a structured view of the XML tree). Both tools provide the ability to validate your authored content against the XML schema (xsd) and various GIFT validation logic (e.g. URL and File checks) with different levels of detail exposed to the user.
To start the CAT, launch the GIFT control panel by double clicking on GIFT\scripts\launchControlPanel.bat.
To start the GAT, launch GIFT by double clicking on launchGIFT.bat and traverse to the Tools area where the course authoring is exposed.
One way to get familiar with authoring a GIFT course file is to follow the GIFT Authoring Test procedures located at GIFT\docs\testing\GIFT Test Procedure Authoring.xlsx. Another suggestion is to view the course’s delivered with GIFT by first starting with the various “Course Technical Details.README.docx” documents associated with almost every course. These documents describe the important features used in each course and should be used for referencing examples of what is supported and has been tested thus far. From there you can open the courses for authoring or execute them in GIFT.
For further assistance please search for forums or post a new topic at https://gifttutoring.org/projects/gift/boards.
Simple Example Training Application¶
As part of this guide, a simple training application was created that could be used to test the various examples discussed. The training application, including source and a property file, is located in GIFT at “Training.Apps\SimpleExampleTrainingApplication\”.
Run The application¶
First you may need to configure the XML-RPC network parameters to match those in your GIFT\config\gateway\interopConfig.xml file. Refer to the Configuration section below.
To run the application, double-click on “Training.Apps\SimpleExampleTrainingApplication\RunApplication.bat”. .
In order to use this program with GIFT, a GIFT course must be running that references the appropriate Gateway interop plugin of which utilizes XML-RPC connection(s) configured to the same network parameters.
About the application¶
The Simple Example Training Application is a C# program written using Microsoft Visual C# Express 2010. The C# programming language was chosen because we had already provided Java and C++ examples of connecting to a Gateway module interop plugin.
The application creates an XML-RPC client to connect to a GIFT XML-RPC server (e.g. mil.arl.gift.gateway.interop.simple.SimpleExampleTAPluginInterface) as well as creating an XML-RPC server which a GIFT XML-RPC client can connect too. This communication protocol establishes two way communication between GIFT and this program during GIFT course execution.
The program will present a dialog with the following components:
- “button 1” and “button 2” – this will cause the C# XML-RPC client to call the following method in GIFT with the string parameter of “button 1” or “button 2” respectively. mil.arl.gift.gateway.interop.simple.SimpleExampleTAPluginInterface$SimpleExampleTAPluginXMLRPC.handleTrainingApplicationMessage
The SimpleExampleTAPluginXMLRPC inner class in mil.arl.gift.gateway.interop.simple.SimpleExampleTAPluginInterface will then create a GIFT Simple Example game state message with that provided string parameter which the Gateway module will send to the Domain module.
- “button 3” – this will cause the C# XML-RPC client to call the following method in GIFT. mil.arl.gift.gateway.interop.simple.SimpleExampleTAPluginInterface$SimpleExampleTAPluginXMLRPC.handleTrainingApplicationFinished
The SimpleExampleTAPluginXMLRPC inner class in mil.arl.gift.gateway.interop.simple.SimpleExampleTAPluginInterface will then create a GIFT SIMAN ‘Stop Freeze’ game state message which the Gateway module will send to the Domain module.
- The 2 list boxes will show information about sent and received XML-RPC calls.
Configuration¶
Open “Training.Apps\SimpleExampleTrainingApplication\application.properties” to view the various configuration parameters for this application, including the important XML-RPC parameters for communicating with GIFT.
Integrating an External Assessment Engine¶
The current best practice for integrating an external assessment engine into GIFT is via a condition class in the Domain module (specifically in GIFT/src/mil/arl/gift/domain/knowledge/condition). In this situation, the assessment engine has been separated from any Training Application it may have been a part of in the past. It is also advantageous, but not required, if this assessment engine can handle other GIFT training app game state messages that are provided in a GIFT release (see classes in GIFT/src/mil/arl/gift/common/ta/state/ ). This way the engine can be used by other Training Applications for different domains by other organizations.
The condition class you will need to create will be responsible for:
- Registering to receive the types of game state messages it can assess performance on
- Configuring the assessment engine for the current use case (e.g. course/domain/scenario/lesson specific rules)
- Returning an assessment result calculated by the engine for a DKF condition
- Support multiple concurrent users – a single GIFT instance (in this case a single Domain module) can be used by multiple users simultaneously. Make sure you handle this situation in the best way you see fit.
- Cleaning up the assessment engine once the Training App course transition is over
For more information on creating a new simple condition class, Creating a performance assessment Condition.
After your condition class is created you will need a way for the DKF author to reference the implementation in order to execute the assessment engine. This is done by extending the dkf.xsd (GIFT/config/domain/dkf/dkf.xsd) to include a new condition input element. This new element might contain the information needed to configure your engine for a condition (i.e. load ‘xyz.config’) and associate your configuration rules, if any, to the specific condition. The association is useful in cases where your configuration file could contain the rules for many conditions specified in the dkf.xml you are authoring. Therefore it allows a single configuration file to assess multiple conditions and provide different assessment values for each of those conditions.
Currently, the only example of an external assessment engine being integrated into GIFT is with the SIMILE assessment engine. The SIMILE assessment engine uses the “SIMILEInterfaceCondition” condition class (GIFT/src/mil/arl/gift/domain/knowledge/condition/SIMILEInterfaceCondition.java) to receive game state messages of interest from GIFT. This condition class then delivers the content of those messages to SIMILE. At some point SIMILE returns an assessment result based on the rules it was configured with (authored using the SIMILE workbench found in GIFT/external/simile) and the current state of the training application to GIFT via that condition class. Then the condition class provides that assessment to its parent concept just as all the other conditions do in the domain module. To see this in action please explore the “Explicit Feedback within Game-Based Training” courses provided in the GIFT release. One important note is to look at how the SIMILE rules were mapped to DKF conditions using the “conditionKey” element which allows the engine’s performance assessment values to be mapped to the appropriate DKF condition and therefore the concept included in a performance assessment message going from the Domain module to the Learner module.
Integrate a Sensor¶
Sensors provide additional data for determining learner state in GIFT. A sensor can either be hardware based (e.g. Q Sensor, Kinect sensor) or software based (e.g. Self-Assessment sensor). Thus far, software based sensors have been used as surrogates for testing or demo purposes due to the inconvenience of configuring a sensor's hardware (see screenshots below). It is also difficult to control the data values being collected when you need a specific outcome in GIFT. Whether you want to implement a software based sensor or a hardware based sensor, both types of sensors communicate with GIFT via the sensor module. The first step in integrating your own sensor in GIFT is to create a sensor implementation class.
examples of available software based sensors that already exists in GIFT source and can be used or copied for your own needs
Create a Sensor Implementation¶
To begin implementing a sensor into GIFT, you must first create a sensor implementation class for your sensor. A sensor implementation class is responsible for translating raw sensor data feeds into a common GIFT data structures that can be used by sensor filters and writer (writers sensor data to disk, e.g. csv file) in the sensor module. The implementation classes, of which there are at least a dozen of in GIFT, are located in the folder, GIFT/src/mil/arl/gift/sensor/impl.
The implementation class you will need to create will be responsible for:
- Identifying the information the sensor will produce for GIFT (e.g. temperature, humidity, location of left eyebrow, etc.)
- Read the data steam(s) from the sensor at the appropriate rate
- Post unfiltered sensor data events
- Cleanup when appropriate - may involve closing network sockets, terminating threads, sending a shutdown message to the sensor
It is advised that you build the implementation class with static and fixed configuration values (e.g. ports, intervals, rates) first. In the end, the sensor data's path will depend on the specified implementation class, the specified filter, and lastly, the specified writer. Ultimately, the goal is for the sensor data to reach the learner module via a sensor data message.
If you wish to specify an enumeration in your sensor implementation, classes are already in place to do so. You can find these files in the folder GIFT\src\mil\arl\gift\common\enums.
These files are:- SensorAttributeNameEnum.java
- In this enumeration file, you can add features you wish to observe using your sensor. For example, if you are doing an experiment using a Kinect sensor, you might want to get arm data. You can specify an enumeration for this type of data.
- SensorStateEnum.java
- If you wish to specify another state for your sensor, you can specify the enumeration here.
- SensorTypeEnum.java
- You will need to create an enumeration for the name of your sensor in this file. Not necessary, but useful for your sensor implementation.
If you have created an enumeration for your sensor using any of three files. You can specify the enumeration in your sensor implementation class.
Once you have finished building the sensor implementation class, the next step is extending the sensor configuration schema with the required XML configuration elements mentioned in the next section.
Refer to Sensors Configuration section in GAT improvements section for information on exposing this sensor implementation to a course author.
Extend Sensor Configuration schema¶
The sensor configuration schema found in the folder GIFT/config/sensor/sensorConfig.xsd defines the schema for the sensor configuration XML file. For more information on the elements in that XML file, please refer to the Sensors Configuration section of GIFT Configuration Settings.
The schema needs to be extended to support the addition of your sensor. In your sensor implementation file, if you have any discrete parameters specified, you will need to add these elements into the schema alongside your sensor. You have two options in appending your sensor changes into the schema. The first option assumes that your sensor might have a closely related parameter already implemented by another sensor. You have the option of choosing this element for your sensor. The second option is to manually open the sensorConfig.xsd file and add your sensor and it's elements into the schema. The second option is mandatory.
- First option:
- You can append to the schema by taking a look at other "sensorInput" choices available in the sensor configuration tool (SCAT) found in the folder GIFT/scripts/tools. It is possible another sensor has the same or similar elements that you desire. You have the option of using this element as a parameter for your sensor. You will have to manually add this element to the schema using the second option.
- Second mandatory option:
- You can append to the schema by manually opening the sensorConfig.xsd file for editing and adding the elements you need. There are several examples of sensor inputs like the "SelfAssessmentSensor" and "MouseTempHumiditySensor".
Once you are finished changing the sensorConfig.xsd schema, the next step is to build GIFT by running the GIFT/build.bat. Running this .bat file will generate Java code that will be used by your sensor implementation class to configure the attributes you have added.
The Test Sensor section below will advise you on testing your sensor implementation.
Sensor Filters¶
A sensor filter can be used for many reasons such as:
- preventing too much data from passing through at a high rate
- dealing with noisy data streams
- categorizing data streams into useful attributes
Sensor filter implementations are located in GIFT/src/mil/arl/gift/sensor/filter. Each filter implementation receives raw sensor data from a sensor implementation class and is responsible for some type of filtering on that data before ultimately producing a filtered data event.
Refer to Sensors Configuration for information on exposing this sensor implementation to a course author.
Sensor Writers¶
A sensor writer is responsible for writing unfiltered/raw or filtered sensor data to disk (e.g. csv file). Sensor writer implementations are located in GIFT/src/mil/arl/gift/sensor/writer. Each writer implementation should be configured with a file name token (i.e. prefix) that is used by all instantiations of the writer but allow the writer to create unique output file names by adding a unique identifier (e.g. time stamp).
- Be careful when using sensor writer implementations as disk I/O (i.e. writing to a file) can consume a computers processing/memory resources quickly depending on the logic implemented and the rate at which it is called upon.
Refer to Sensors Configuration for information on exposing this sensor implementation to a course author.
Test Sensor + Filter + writer¶
After completing the above sensor related steps, you will need to re-open the SCAT, which can be found in the folder GIFT/scripts/tools. In the SCAT file, you will need to author a sensor configuration XML file that will use the sensor implementation class for your sensor. The filter and writer you created will also be specified in this file. The sensor configuration XML file for the sensors can be found in the folder GIFT\config\sensor\configurations.
You also have the option of manually changing the parameters to fit the needs of your sensor implementation, but make sure the changes you make to the sensor configuration XML file match the changes you made in the sensorConfig.xsd schema file. It is recommended that you take a look at the sensor configurations of other sensors to get a better idea as to how they are configured based on the sensor schema file and their sensor implementation file.
Before launching GIFT, you will need to configure the sensor module to use your sensor configuration file. This is currently done by editing the "SensorConfigurationFile" property in the sensor.properties file located at GIFT/config/sensor/sensor.properties.
To test the sensor, launch GIFT and start any course. Once the course has started, your sensor implementation will become active. It is possible to view a real-time sensor feed by viewing the sensor graph on the sensor tab of the GIFT monitor application (see GIFT monitor (IOS-EOS) instructions for more information on using the monitor. The viewing of the real-time sensor feed requires the implementation of the Sensors Configuration section in the GAT improvements section. In addition, you can go to the output directory of your sensor writer(s) to view the sensor data being written to disk. You can locate these files in the folder GIFT\output\sensor.
You can also refer to the “Author Sensor Configuration” worksheet of the “GIFT Test Procedure Authoring.xls” spreadsheet that covers authoring a sensor configuration file. That sensor configuration is used in the execution of a GIFT course elsewhere in that spreadsheet.
GIFT Server mode¶
One of the major differences between desktop configured GIFT and server configured GIFT is where the Gateway module resides and how it is started. In desktop mode the Gateway module is normally executed on the same computer as the other GIFT modules. This computer has the web-browser as well as any desktop applications needed for that GIFT instance. While in server mode the Gateway module is delivered, when needed for a course, to the user’s computer via the web-browser. In order to do this GIFT utilizes Java Web Start (JWS) technology to distribute and start the Gateway module.
In order to deploy your own instance of the GIFT Server you will need to consider the following:
- Updating JWS server URL values
- GIFT Compilation
- Code Signing versus Java Security Exceptions
- Debugging JWS Gateway module instance
Server URLs¶
The URL of your GIFT server needs to be updated in two files in order for the JWS application to work correctly.
1. GIFT/config/build.properties
The property used in this file is used in the manifest of the GIFT jars used in the JWS application. If you don’t set this property correctly the JWS instance will never fully start correctly.
a. “gateway.jws.server.address” – change the URL of this property to match the address of the GIFT machine.
snippet:
gateway.jws.server.address=http://10.1.21.68:8885After changing the URLs you will need to compile GIFT (only the GIFT jars used in the JWS application should be updated) using the provided build.xml infrastructure (i.e. build scripts). If you haven’t done so already, please read the following Code Signing section.
Code Signing¶
Most browsers these days are automatically configured to ensure some level of security on your computer. When it comes to a Java Web Start application, the code must be signed in order to be granted certain permissions on the user’s computer. Signing code is performed using a keystore and a certificate provided by a reputable code signing authority.
For more information on code signing please refer to the following:
http://docs.oracle.com/javase/tutorial/security/toolsign/When you compile GIFT (using the GIFT build.xml files) there is logic that tries to code sign the JWS Gateway jars. These ant targets can be found in the following files by searching for “signjar”:
- GIFT/src/mil/arl/gift/common/build.xml
- GIFT/src/mil/arl/gift/gateway/build.xml
and those targets use properties set in:
- GIFT/build.xml
<property name="keystore" value="${basedir}/config/ARL Certificate/ArmyResearchLaboratory.jks"/>
<property name="keystorePW" value="giftkeystore"/>
<property name="keystoreAlias" value="server"/>
As part of the GIFT release, the internally used Java keystore and purchased certificate has been removed. Therefore, in order to sign any changes you make to the GIFT JWS jars you can either:
1. Purchase a code signing certificate (we used Digicert) and follow the instructions to create and use your own keystore.
2. Don’t sign your code (which is the same as creating your own keystore and certificate). In this case any client computers that will run your JWS application will need to add Java security exceptions as described in Java Security Exceptions (no code signing certificate work-around).
Java Security Exceptions (no code signing certificate work-around)¶
In order to bypass Java security settings (probably because you don’t have a non-revoked code signing certificate), there are a few things that can be configured as a work around:
- Add your GIFT Server to the “Exception Site List” – this is done through the Java Control Panel on the “Security” tab. The URL should match the “codebase” value mentioned earlier.
- Using a revoked certificate – if you are trying to use a revoked certificate but Java won’t let you continue then you can disable checking through the Java Control Panel on the “Advanced” tab. Under “Perform signed code certificate revocation checks on” select “Do not check (not recommended). Under “Perform TLS certificate revocation checks on” select “Do not check (not recommended)”.
JAVA Console for Debugging¶
One way to debug the JWS Gateway module application is to enable the Java console. The Java console can show standard out (system.out) data. Currently GIFT is directing log4j output to standard out for the JWS instance.
To enable the Java console: https://www.java.com/en/download/help/javaconsole.xml
Configuring the XML-RPC Python Server With GIFT¶
Python scripts can be called from GIFT Java Code using XML remote procedure calls (aka XML-RPC). Python scripts can be called externally from within a SensorFilter class (such as mil.arl.gift.sensor.filter.QrsFromEcgFilter) to further help filter raw sensor data. Python scripts can also be called to ‘pre-process’ data coming from sensors or training applications before it is sent to Rapid Miner. See Configuring GIFT with RapidMiner for more information on how to use python scripts with RapidMiner.
It is expected that the developer is familiar with Python scripts in order to extend these areas.
Guidelines For Developing Python Code with GIFT¶
The following guidelines should be followed when developing/integrating python code with GIFT.
- Python source should be put beneath the GIFT/src.py folder
- Exported python methods (i.e. methods to be called by GIFT via XML RPC) should be part of a single class. Note that if existing python modules are not written as classes this requirement can be met by creating a class wrapper around the procedural module.
- Exported python methods can accept multiple arguments but each argument should be of a primitive type such as int, float, string. Arguments of type "array of string" also work. (NOTE: Arrays of other primitive types are expected to work but have not yet been tested by the GIFT team). (Here again, wrapper classes may be a good approach if existing python code does not meet these requirements.)
- Python methods should return a list of primitive types. The current working example returns a list containing two values. Caution: the XML RPC code as written may not support return of a single value. It won't be hard to extend the baseline to support return of single values, but it hasn't yet been done.
- Until further notice, methods that have no return value, a single return value, or a null or nil return value should be avoided. If necessary use wrappers to achieve these requirements.
Calling Python Scripts For Sensor Filters¶
GIFT supports the ability to use python scripts to help filter raw sensor data before sending the sensor data event within the GIFT application. The following diagram illustrates a sequence call of when the python server is called to filter the raw sensor data.
In the above example, the sensor data is fed into a filter class (mil.arl.gift.sensor.filter.QrsFromEcgFilter) via a call to filterSensorData(). The filterSensorData() function internally calls handleEcgWaveformData(). This function takes the sensor data and calls externally to the Python server by calling the python function “QrsDetect”. Filtered data is returned. The QrsFromEcgFilter then calls handleSensorDataEvent() which calls into the SensorManager to broadcast the Sensor Event to GIFT.
Further description of this process follows:
Input to this QrsFromEcgFilter is an ECG waveform signal (currently originating from the Bioharness sensor). The waveform signal is passed incrementally to a python module (accessed via XML RPC) which computes the instantaneous heart rate. Filter then outputs the heartrate, sending the value to downstream consumers.
To see details of how this is done please refer to the QrsFromEcgFilter.java file.
Calling Python Scripts for RapidMiner¶
GIFT supports the ability to filter data from sensors or training applications to transform or pre-process the data before being sent to RapidMiner. For more information on how this is done, see Understanding the RapidMiner Data Pipeline Process and Using RapidMiner with Python Script Processing sections.
Adding a Python Script to GIFT¶
1. Place the python script in the GIFT/src.py/xml.rpc.server/ folder location.
2. Once you have suitable python code in the src.py folder, the next step is to make the code accessible via the Xml RPC Service.
Users should edit the GIFT/src.py/xml.rpc.server/XmlRpcServer.py file and add the appropriate import statement. For example the QrsFromEcgFilter requires the IncECGDetection.py module so XmlRpcServer.py contains the following import:
from IncECGdetection import *
3. Once the import statement has been added and saved the XML RPC server can be launched manually using the following on the command line:
python XmlRpcServer.py -p <service_port> -c <classname>
At this point the python work is complete.
To access the python service via XML RPC from java code within GIFT you will need an instance of XmlRpcService.
Calling Python Code From Within GIFT via XML-RPC¶
In order to call python code via XML-RPC, there is a manager class to allow easier access to the XML-RPC interface. GIFT has an XMLRPCPythonServerManager class that is responsible for launching the XML-RPC python server. This class is located in mil.arl.gift.common.python. XMLRPCPythonServerManager.java.
The general outline is to:
1. Start the XML-RPC python server
2. Create an XMLRpcClient to call the python function.
3. Stop the XML-RPC python server when done.
To start the server, the XMLRPCPythonServerManager can be called with a single function:
XMLRPCPythonServerManager.launchServer(int port, String serviceClassName);
Where the first argument is the port that the server will be started on. The serviceClassName is the name of the python class that is registered in the import added in the previous section.
To call a running python server from within GIFT, an XMLRPCClient object needs to be created. An example of what that looks like is here:
XMLRPCClient client = new XMLRPCClient(XMLRPCPythonServerManager.DEFAULT_SERVER_BASE_URL, QRS_DETECT_SERVICE_PORT);
result = (Object[]) client.callMethod(QRS_DETECT_SERVICE_METHOD_NAME, params, errorMsg);
Where QRS_DETECT_SERVICE_METHOD_NAME (in this example) is the name of the python function to call.
To Stop the server, the XMLRPCPythonServerManager can be called via a function:
XMLRPCPythonServerManager.destroyServiceProcess()
For a complete example of this flow, see the mil.arl.gift.sensor.filter.QrsFromEcgFilter.java class.
Configuring GIFT with RapidMiner¶
GIFT allows for configuration of RapidMiner to process game state data and/or sensor data. The result of the RapidMiner process is to evaluate the data and give an assessment on a particular learner state. For example, GIFT can be configured with a Kinect sensor. GIFT can receive the raw data of the Kinect sensor and send it into an externally developed RapidMiner model which may look at the vertex data to assess how engaged the learner is. The output of RapidMiner can produce a result which indicates a confidence of how engaged the learner is in real-time during the course execution.
Why Use RapidMiner?¶
RapidMiner is a highly customizable open-source modern analytics platform that provides an excellent GUI/authoring tool to create and author data pipelines (which can be used for various purposes such as predictive analysis or data transformation). The use of RapidMiner and developing a full rapid miner model is outside the scope of this document, and the reader is referred to query the internet regarding the RapidMiner application documentation.
The primary benefit of RapidMiner within GIFT allows for the author to use the powerful GUI/tools available within RapidMiner specific to creating a data model and insert the models into GIFT which can be used to assess learner states based on 1) training application game states and/or 2) sensor data.
For the purposes of this document it is assumed the reader has already developed a RapidMiner model with which to integrate into the GIFT framework.
General Requirements¶
Configuring RapidMiner in GIFT will require modification to various *.xml files, java classes and (optional) python classes. As such it is expected that any developer wishing to configure GIFT to use RapidMiner has an understanding of how to modify *.xml files, write some java code, write python functions, and have an external RapidMiner model (which has been developed with the RapidMiner application).
The RapidMiner Sample Course¶
To better help understand the use of RapidMiner within GIFT, a sample course has been created which demonstrates the ability of GIFT to:
1. Receive game state data from a training application.
2. Send the game state data to a python function to preprocess the data (in this case compute the average of one of the data points).
3. Send the averaged result set into a RapidMiner model which analyzes the average values to get the learner’s ANXIOUS state.
See the RapidMinerTestCourse in the Domain folder to see this course work and get a better understanding of the data pipeline process. Once the TC3 course is loaded, the BloodVolume parameter is received via periodic updates to GIFT. The learner starts the ANXIOUS state with HIGH as the BloodVolume starts at ~5000. As the BloodVolume decreases, the average of BloodVolume over time is computed and sent to the RapidMiner model. The RapidMiner model is configured to look for the threshold of when the Average of Blood Volume reaches 4990. Once the BloodVolume of the patient reaches 4990, the learner ANXIOUS state is turned to LOW. This is a contrived example used to show the key pieces of the RapidMiner data pipeline and how the learner state can be affected.
Understanding the RapidMiner Data Pipeline Process¶
GIFT supports two types of data pipelines with RapidMiner:
1. Raw Data Feed Pipeline (Figure 1)
2. Processed Data Feed Pipeline (using Python Script Processing) (Figure 2)
Example of the Raw Data Feed Pipeline Into RapidMiner
Example of the Processed Data Feed Pipleline (using Python Script Processing)
GIFT currently supports taking input data from two sources 1) Sensors and 2) Training Applications.
A description of the Raw Data Feed Pipeline (from Figure 1) follows. GIFT can be configured to take input from either a sensor or a training application. In the case of a sensor, a single frame of data is fed into GIFT. This single frame of data is received by a classifier which is configured to monitor a specific learner state (such as Anxious). The single frame of sensor data is stored into a DataModel, which collects the raw sensor feed frames as they come in. The DataModel houses multiple frames of data until a 20 second interval is reached. Once the interval is reached, the multiple frames of raw data are sent into a RapidMiner model. This model operates on the data and computes a ‘confidence value’ (HIGH/LOW) of the learner state (eg. Anxious). The Classifier receives this updated learner state where it is broadcast to the rest of the GIFT application.
For the Processed Data Feed Pipeline (using Python Script Processing) from Figure 2, there is an additional step that is added to ‘preprocess’ the data before it is eventually sent into RapidMiner. Data can be received from a sensor or a training application. In the case of a sensor, a single frame of data is fed into GIFT. This single frame of data is received by a classifier which is configured to monitor a specific learner state (such as Anxious). The single frame of sensor data is sent (via the DataModel) to a configurable python function which can ‘pre-process’ the frame of data. One example of such ‘pre-processing’ may be to compute averages over the frames of data as they are received. The python function will return a transformed frame of data. Similar to the previous pipeline, every 20 second interval, the transformed data is sent to RapidMiner where the RapidMiner model operates on the data to compute a confidence value (HIGH/LOW) of the learner state (eg. Anxious). The Classifier receives this updated learner state where it is broadcast to the rest of the GIFT application.
Which pipeline you need to use depends on the use case. In Figure 1, the advantage is that the RapidMiner model can receive largely raw data as it is sent by the sensor or Training Application and operate directly on the raw data.
In Figure 2, the advantage of the ‘pre-processing’ using Python can be used in scenarios where some ‘adjustment’ of the data is required by the RapidMiner model where it is not possible to use the raw data feed directly. The python script processing can be used to do a ‘pre-processing’ of the data before it is fed into the RapidMiner model.
The following sections now detail the steps on how to configure GIFT to use RapidMiner with each of these pipelines.
Using RapidMiner with Raw Data Feeds¶
The following steps will show how to setup GIFT to use the RapidMiner raw data feed pipeline (from Figure 1 above).
1. Start by choosing a learner state that you will be using to be updated by the sensor or training application. In this example, we will choose the Anxious learner state, but it could be any learner state (bored, confused, surprised, etc). See the LearnerStateAttributeNameEnum.java file in GIFT for a list of supported learner states.
2. Author the RapidMiner model. Before authoring the model, make sure to read the full outline below. The RapidMiner model must be written to expect the data that will be labeled by GIFT. The labels named in the RapidMiner model must match the labels that that GIFT will be outputting to the RapidMiner model. See Step 5 for where this mapping is defined. The authoring of a RapidMiner model is outside the scope of this documentation. Place the RapidMiner model file(s) in the “GIFT/config/learner/model/RapidMiner” directory.
3. Update GIFT/src/mil/arl/gift/learner/common/rapidminer/RapidMinerInterface.java to point to the new rapidminer model file. An example of this (for Anxious is here):
public static final File ANXIOUS_PROCESS_FILE = new File(PackageUtil.getConfiguration() + "/learner/model/RapidMiner/Sample_TC3.process.InApply.xml");
4. Configure the Classifier class that will be used. In this example, we will use the AnxiousClassifier (which is setup to monitor the Anxious state). We need to link the AnxiousClassifier to use the rapidminer model process file that we configured in Step 3, as well as setup which pipeline (raw pipeline). A full example of the AnxiousClassifier is found at: GIFT/src/mil/arl/gift/learner/clusterer/AnxiousClassifier.java. The important changes are listed below:
public AnxiousClassifier() throws IOException, XMLException{
tc3RapidMiner = new RapidMinerProcessor(RapidMinerInterface.ANXIOUS_PROCESS_FILE, LearnerStateAttributeNameEnum.ANXIOUS, new TC3DataModel(), CONFIDENCE_KEY);
}
Where:
ANXIOUS_PROCESS_FILE should match the process file that was added in Step 3.
TC3DataModel – specifies to use the ‘raw data model for TC3’. The options available here are TC3DataModel() for TC3 training application data OR KinectSensorDataModel() for Kinect sensor data processing of raw kinect data.
CONFIDENCE_KEY – is a string value that must match the output of the RapidMiner model. The default output key that is examined is “confidence(1)”.
5. Configure the attribute labels being collected in TC3DataModel.java (GIFT/src/mil/arl/gift/learner/common/rapidminer/TC3DataModel.java) or KinectSensorDataModel.java (GIFT/src/mil/arl/gift/learner/common/rapidminer/KinectSensorDataModel.java). These attributes will be sent to RapidMiner and can be adjusted if needed, but the model MUST be written to expect data with the following labels. An example of these attribute labels for TC3DataModel are listed below:
public static final String BLOOD_VOLUME_LABEL = "BloodVolume";
public static final String HEART_RATE_LABEL = "HeartRate";
public static final String ISSAFE_LABEL = "isSafe";
public static final String BLEED_RATE_LABEL = "BleedRate";
public static final String LEFT_LUNG_EFF_LABEL = "LeftLungEfficiency";
6. IMPORTANT - Configure the inputs and outputs of the RapidMiner model to match the GIFT labels/attributes. The DataModel (TC3DataModel or KindectSensorDataModel) treats all data values as ‘double’ values. The RapidMiner model should expect a list of ‘double’ values that contain the labels (from Step 5) and in the same order that the attributes are defined in the GIFT DataModel java classes. Booleans will be given values of 1.0 or 0.0.
For output, the RapidMiner model is expected to produce a ‘confidence value’ (double) that is a range from 0.0 to 1.0. The CONFIDENCE_KEY that is configured in Step 4 within GIFT must match the the label of this confidence value from RapidMiner.
Mismatches in input to the RapidMiner model OR mismatches in the output of the RapidMiner model will result in invalid processing of the data. The RapidMiner model MUST match in terms of 1) label/attribute names, 2) number of columns of data, 3) data type (doubles) and 4) output/confidence key label name.
7. Update the GIFT LearnerConfiguration.xml file (GIFT/config/learner/LearnerConfiguration.xml) to enable the classifier that is used. An example of setting up the configuration file to use the AnxiousClassifier is listed below:
<input name="Generic Anxious">
<translator>
<translatorImpl>learner.clusterer.data.DefaultTranslator</translatorImpl>
</translator>
<classifier>
<classifierImpl>learner.clusterer.AnxiousClassifier</classifierImpl>
</classifier>
<predictor>
<predictorImpl>learner.predictor.GenericPredictor</predictorImpl>
</predictor>
<producers>
<trainingAppState type="GenericJSONState"/>
</producers>
</input>
Where <producers> can be either <trainingAppState type=”GenericJSONState”/> for TC3 (training application) OR <sensor type=”KINECT”/> for Kinect sensor.
8. Run the course! At this point the pipeline should be configured to take raw data from either sensor or a training application to send the data into a RapidMiner model which will analyze the data and return a confidence value for a learner state (like Anxious). As the confidence value changes, the learner state will be updated to a HIGH or LOW state.
9. The most important part is to make sure the input & outputs are in sync between the RapidMiner model (See Step 6). Any mismatches will cause the data to not get processed properly. To troubleshoot issues with the pipeline, debug or trace logging can be added to the learner module when running the GIFT application to see the input and outputs that are received by the RapidMiner model.
Using RapidMiner with Python Script Processing¶
The following steps will show how to setup GIFT to use the RapidMiner Processed Data Feed Pipeline Using Python Script Processing (from Figure 2 above). The steps will be similar to setting up the Raw Data Feed pipeline. The primary differences are the configuration of the python server as well as switching out the DataModel from a TC3DataModel() or a KinectSensorDataModel() to use a ProcessedTC3DataModel() and ProcessedKinectSensorDataModel(). These java classes are found in the directory: GIFT/src/mil/arl/gift/learner/common/rapidminer/. The “Processed” versions of the datamodel classes indicate they will use python to process (or transform) the data before sending it into RapidMiner. For more information on how XML-RPC python server is integrated with GIFT, refer to Configuring the XML-RPC Python Server With GIFT.
1. Start by choosing a learner state that you will be using to be updated by the sensor or training application. In this example, we will choose the Anxious learner state, but it could be any learner state (bored, confused, surprised, etc). See the LearnerStateAttributeNameEnum.java file in GIFT for a list of supported learner states.
2. Author the python script. To preprocess the data before it is sent into RapidMiner, a python function call will be made. The python script can be added here: GIFT/src.py/xml.rpc.server. In this case, we will say we’ve created a TC3GameStateDataProcessor.py. It should live in that directory. The script should have a function call that is expected to take in an arbitrary table of ‘double’ values of any size and return a table of double values. The script could perform any number of operations on the data and ‘transform’ it before it will be sent to RapidMiner. One example is computing the ‘averages’ of BloodVolume over time rather than the exact state of BloodVolume as it is received each frame. The name of this function will be used later to map it to where it can be called by GIFT.
Note the full authoring of a python script is outside the scope of this document, but the reader is directed to the internet for more details on authoring python script.
3. Register the python script with GIFT. The script class must be registered in the GIFT/src.py/xml.rpc.server/XmlRpcServer.py script. Add the class there as an “import” at the top of the file. An example of this is listed below:
from TC3GameStateDataProcessor import *
4. Author the RapidMiner model. Before authoring the model, make sure to read the full outline below. The RapidMiner model must be written to expect the data that will be labeled by GIFT. The labels named in the RapidMiner model must match the labels that that GIFT will be outputting to the RapidMiner model. See Step 7 for where this mapping is defined. The authoring of a RapidMiner model is outside the scope of this documentation. Place the RapidMiner model file(s) in the “GIFT/config/learner/model/RapidMiner” directory.
5. Update GIFT/src/mil/arl/gift/learner/common/rapidminer/RapidMinerInterface.java to point to the new rapidminer model file. An example of this (for Anxious is here):
public static final File ANXIOUS_PROCESS_FILE = new File(PackageUtil.getConfiguration() + "/learner/model/RapidMiner/Sample_TC3.process.InApply.xml");
6. Configure the Classifier class that will be used. In this example, we will use the AnxiousClassifier (which is setup to monitor the Anxious state). We need to link the AnxiousClassifier to use the rapidminer model process file that we configured in Step 3, as well as setup which pipeline (raw pipeline). A full example of the AnxiousClassifier is found at: GIFT/src/mil/arl/gift/learner/clusterer/AnxiousClassifier.java. The important changes are listed below:
public AnxiousClassifier() throws IOException, XMLException{
tc3RapidMiner = new RapidMinerProcessor(RapidMinerInterface.ANXIOUS_PROCESS_FILE, LearnerStateAttributeNameEnum.ANXIOUS, new ProcessedTC3DataModel(), CONFIDENCE_KEY);
}
Where:
ANXIOUS_PROCESS_FILE should match the process file that was added in Step 3.
ProcessedTC3DataModel – specifies to use the python enabled data model. The options available here are ProcessedTC3DataModel() for TC3 training application data OR ProcessedKinectSensorDataModel() for Kinect sensor data processing of raw kinect data.
CONFIDENCE_KEY – is a string value that must match the output of the RapidMiner model. The default output key that is examined is “confidence(1)”.
7. Configure the attribute labels being collected in ProcessedTC3DataModel.java (GIFT/src/mil/arl/gift/learner/common/rapidminer/ProcessedTC3DataModel.java) or ProcessedKinectSensorDataModel.java (GIFT/src/mil/arl/gift/learner/common/rapidminer/ProcessedKinectSensorDataModel.java). These attributes will be sent to RapidMiner and can be adjusted if needed, but the model MUST be written to expect data with the following labels. Note that the ProcessedTC3DataModel class extends off ofTC3DataModel class where the base attributes are defined. An example of these attribute labels for ProcessedTC3DataModel are listed below:
public static final String BLOOD_VOLUME_LABEL = "BloodVolume";
public static final String HEART_RATE_LABEL = "HeartRate";
public static final String ISSAFE_LABEL = "isSafe";
public static final String BLEED_RATE_LABEL = "BleedRate";
public static final String LEFT_LUNG_EFF_LABEL = "LeftLungEfficiency";
…
8. Configure the python function call that will be made in the ProcessedTC3DataModel.java class or the ProcessedKinectSensorDataModel.java class. These classes are located in: GIFT/src/mil/arl/gift/learner/common/rapidminer. An example of setting this value is given below:
private String SERVICE_METHOD_NAME = "ProcessTC3Data";
Where “ProcessTC3Data” is the name of the python function that was created in Step 2.
IMPORTANT – The python script should take in a table of ‘double’ values and return a table of ‘double’ values back to GIFT. The mappings of what GIFT will send to the python script are defined in the ProcessedT3CDataModel.java and the ProcessedKinectSensorDataModel.java classes. These mappings must match the 1) data type (doubles) and 2) the number of columns and 3) the order of the columns.
It is best practice to keep the ‘raw’ column values defined in the base data model (TC3DataModel and KinectDataModel), and append ‘processed’ columns if python script will process any data (like a column for ‘averages’).
Again, the mapping of the column/attribute names must match GIFT output to Python, Python output to GIFT, GIFT (transformed data) output to RapidMiner and RapidMiner output to GIFT (the confidence key). Making sure the inputs & outputs are aligned at each stage is critical or incorrect processing of data can occur.
For the final output, the RapidMiner model is expected to produce a ‘confidence value’ (double) that is a range from 0.0 to 1.0. The CONFIDENCE_KEY that is configured in Step 6 within GIFT must match the the label of this confidence value from RapidMiner.
Some applications like TC3 may not provide values on every frame for each column. In this case, the python script and GIFT must agree on a value that represents “NO DATA” for the specified column for the given frame. This “NO DATA” value can be changed and an example of where this is used is in GIFT/src/mil/arl/gift/learner/common/rapidminer/ProcessedTC3DataModel.java:
private static final double PYTHON_NO_VALUE = -9999.99;
This PYTHON_NO_VALUE field defaults to -9999.99 and can be adjusted depending on expected values from the training application or sensor. The python script should treat all values as PYTHON_NO_VALUE to mean that GIFT did not receive any value for that column for the specified frame.
Mismatches in input to the RapidMiner model/Python function OR mismatches in the output of the RapidMiner model/Python function will result in invalid processing of the data. The RapidMiner model/Python function MUST match in terms of 1) label/attribute names, 2) number of columns of data, 3) data type (doubles) and 4) output/confidence key label name.
9. Update the GIFT LearnerConfiguration.xml file (GIFT/config/learner/LearnerConfiguration.xml) to enable the classifier that is used. An example of setting up the configuration file to use the AnxiousClassifier is listed below:
<input name="Generic Anxious">
<translator>
<translatorImpl>learner.clusterer.data.DefaultTranslator</translatorImpl>
</translator>
<classifier>
<classifierImpl>learner.clusterer.AnxiousClassifier</classifierImpl>
</classifier>
<predictor>
<predictorImpl>learner.predictor.GenericPredictor</predictorImpl>
</predictor>
<producers>
<trainingAppState type="GenericJSONState"/>
</producers>
</input>
Where <producers> can be either <trainingAppState type=”GenericJSONState”/> for TC3 (training application) OR <sensor type=”KINECT”/> for Kinect sensor.
10. Configure GIFT to use the python server by updating the module.common.properties file (GIFT/config/module.common.properites). An example of enabling the python server is listed below:
StartXMLRpcPythonServer=true
XMLRpcPythonServerPort=9000
XMLRpcPythonServerClassName=TC3GameStateDataProcessor
Where XMLRpcPythonServerClassName must match the name of the python class that was configured in Step 2. Make sure the StartXMLRpcPythonServer is set to true with a valid port that is not in use.
11. Run the course! At this point the pipeline should be configured to take raw data from either sensor or a training application to send the data into a python function which will pre-process and transform the data. The transformed data will be sent to the RapidMiner model. The RapidMiner model will analyze the data and return a confidence value for a learner state (like Anxious). As the confidence value changes, the learner state will be updated to a HIGH or LOW state.
12. The most important part is to make sure the input & outputs are in sync between GIFT, the python function, and RapidMiner model (See Step 8). Any mismatches will cause the data to not get processed properly. To troubleshoot issues with the pipeline, debug or trace logging can be added to the learner module when running the GIFT application to see the input and outputs that are received by the RapidMiner model.
Improving the GIFT Authoring Tool (GAT)¶
The GAT is an easier to use authoring tool when compared to the desktop-based XML authoring tools like the CAT, DAT, SCAT, etc. The following sections provide developer details on how to expose authoring user interface elements necessary to select/use the improvements made to GIFT.
- For reference in this section, the root of the GAT source is located at: GIFT/src/mil/arl/gift/tools/authoring/server/gat.
Gateway configuration¶
After adding a new gateway interop plugin class (see above), you will need to make changes to a few GIFT classes to allow the GAT to author interop configurations for your new training application.
First, you will need to add an enumeration in common.enums.TrainingApplicationEnum to represent your new training application. The enumerations defined in this class are used to determine the list of training applications that course authors will be able to chose from when authoring interop configurations in the GAT. Suppose you wanted to add a new application called "My Application". To add this application to the enumeration, you could add a line like the following:
public static final TrainingApplicationEnum MY_APPLICATION = new TrainingApplicationEnum("MyApplication", "My Application");
Next, you will need to update the maps in common.gwt.shared.TrainingAppPluginUtil to tell the GAT where to find the interop plugins to use with the enumeration you just added. As an example, suppose that "My Application"'s interop plugin is located at gateway.interop.myapplication.MyApplicationPluginInterface. To tell the GAT where to find this plugin, you would have to add the following lines to TrainingAppPluginUtil's static block:
trainingAppToInteropClassNames.put(TrainingApplicationEnum.MY_APPLICATION, new ArrayList<String>());
trainingAppToInteropClassNames.get(TrainingApplicationEnum.MY_APPLICATION).add("gateway.interop.myapplication.MyApplicationPluginInterface");
If your interop plugin does't require any load arguments or if it is set up to allow CustomInteropInputs to be used when retrieving its load arguments, then congratulations, you're done.
Once the previous steps have been completed, you will finally be able to author interop configurations for your application throughout the GAT; however, you'll only be able to author its load arguments using CustomInteropInputs if you stop here. If your interop plugin uses special interop inputs to handle its load arguments, then you'll still have to make a few more changes to let the GAT know how to build the interop inputs your application needs.
The class actually used by the GAT to author load arguments and build interop inputs is gat.client.view.course.ta.TrainingAppInteropEditor, so you'll need to update this class to support your new training application. First, you'll need to decide what UI elements you want the GAT to display whenever a course author selects your training application from the editor. The logic the GAT uses to make this decision is in setTrainingAppEditingMode(TrainingApplicationEnum), so in order to define the UI elements to use with your new training application, you'll need to add a new if-statement that displays the UI elements you want shown. As an example, if you want the GAT to show the same UI elements used to author interop inputs for VBS when a user selects your application, you can use the following code:
} else if(mode == TrainingApplicationEnum.MY_APPLICATION){
applicationArgsDeckPanel.showWidget(applicationArgsDeckPanel.getWidgetIndex(vbsPanel));
Alternatively, if you want to create new UI elements from scratch, you can add them to the "applicationArgsDeckPanel" container in TrainingAppInteropEditor.ui.xml. As an example, if you just wanted to use a plain text box to author your interop inputs, you could add one to "applicationArgsDeckPanel" as follows:
<g:DeckPanel width='100%' ui:field='applicationArgsDeckPanel'>
...
<g:TextBox ui:field='myApplicationBox'/>
...
</g:DeckPanel>
Then, in order to show this new UI element using setTrainingAppEditingMode(TrainingApplicationEnum), you would add a @UiField field identifying the element to TrainingAppInteropEditor.java as follows:
@UiField protected TextBox myApplicationBox;
... and then use the field in setTrainingAppEditingMode(TrainingApplicationEnum) like so:
} else if(mode == TrainingApplicationEnum.MY_APPLICATION){
applicationArgsDeckPanel.showWidget(applicationArgsDeckPanel.getWidgetIndex(myApplicationBox));
After you've decided what UI elements you want to show for your training application, you'll need to determine whether or not TrainingAppInteropEditor is already set up to author the interop input class your training application needs. To do this, check the if-statements in selectTrainingApp(TrainingApplicationEnum) and setTrainingApplication(TrainingApp) to see if they contain cases for your interop input class or not. selectTrainingApp(TrainingApplicationEnum) is used to author new interop input objects while setTrainingApplication(TrainingApp) is used to author existing objects, so if your interop input class is being handled by both of these methods, then you know the GAT is already fully set up to author your interop input class. If your interop input class isn't handled by one of these methods, then you'll have to add a new if-statement telling the GAT how to handle it. This will usually involve making a few calls to get values from or set values on the UI elements associated with your training application. There aren't really any clear-cut steps on how to do this since it varies depending on what interop input class and what UI elements your application is using, but you can find plenty of examples from the nearby if statements.
Last but not least, if you added any new interactive UI elements to handle authoring your application's interop inputs, you'll need to attach event handlers to these elements to handle user interactions with them. Once again, this varies greatly depending on the interop input class and UI elements your application is using, so look at the nearby event handlers for examples on how to do this.
Sensors Configuration¶
After adding new sensor related classes, you will need to expand the GAT to allow the user to select from those parameters (e.g. choose a new sensor type). These files are located in the GAT section of GIFT, in the folder GIFT/src/mil/arl/gift/tools/authoring/gat/client/view/sensor.
In the sensor folder, we have the following files that might need to be edited depending on your sensor implementation:
- FilterTypeEnum.java
- The FilterTypeEnum.java enumeration file specifies an filter enumeration for each sensor that is supported in GIFT.
- SensorsConfigurationFactory.java
- The SensorsConfigurationFactory.java file is important for three reasons. It serves to create unique IDs for the sensors, filters, and writers classes. All classes must have a different ID to function properly.
- It serves to define default values for the attributes specified in the sensor implementation file.
- Lastly, it serves to define default object composition for sensor, filter, and writer classes. For example, a sensor is paired up with a filter, which is then paired with a writer. This class provides methods to construct these compositions.
- SensorsConfigurationMaps.java
- This class creates a mapping between the enumerations to the implementations.
- This class also serves to define the compositional structure between sensors, filters, and writer. For example, a sensor should be paired up with a filter and writer. This is also true for a filter with a writer.
- WriterTypeEnum.java
- This class serves to represent a writer by using an enumeration that can be used in a sensors configuration. Useful because it allows end users to identify a writer by the enumeration's display name rather than using the writer implementation name that it maps to.
Once these files have been edited depending on your implementation file, we can move on to the next set of files.
The next set of files are located in the folder GIFT\src\mil\arl\gift\tools\authoring\gat\client\view\sensor\advanced\steps.
In the steps folder, we have the following files that might need to be edited depending on your sensor implementation:
- ArchiveFilteredDataEditor.java, ArchiveFilteredDataEditor.ui.xml, ArchiveSensorDataEditor.java, and ArchiveSensorDataEditor.ui.xml
- These files need to be edited only when you have created your own writer class for your sensor implementation. Currently, these four files implement the KinectSensorWriter.java writer class and the GenericSensorDelimitedWriter.java class.
- SensorNameAndInputEditor.java, SensorNameAndInputEditor.ui.xml
- These files are required for any sensor implementation.
Once these files have been edited, we can move on to the next set of files.
The next set of files are located in the folder GIFT\src\mil\arl\gift\tools\authoring\gat\client\view\sensor\input
In the input folder, we have the following files that need to be created depending on your sensor implementation:
- SensorNameEditor.java, SensorNameEditor.ui.xml
- It is crucial to create these two files to match the specifications of your sensor implementation class. To get a better idea as to how to create these two files, compare the editors and sensor implementation classes of other sensors.
Once these files have been edited, we can move on to the next set of files.
The next set of files are located in the folder
GIFT\src\mil\arl\gift\tools\authoring\gat\client\view\sensor\wizard
In the wizard folder, we have the following files that need to be edited depending on your sensor implementation:
- BuildSensorDialog.java
- This file creates and returns the SensorInput for your sensor. Make sure that the initialization of your SensorInput in the BuildSensorDialog.java matches the structure of how you setup your sensor in the sensorConfig.xsd schema, it's sensorconfig XML, and in the sensor implementation file.
- ConfigureArchievePane.java, ConfigureArchivePane.ui.xml
- Feel free to take a look at these files if you have specified a writer for your sensor. If not, you may ignore these files.
- ConfigureSensorPane.java, ConfigureSensorPane.ui.xml
- These two files are crucial in the implementation of your sensor. Take a look at how things are defined and implemented in the java file before starting to add changes regarding your sensor. You might need to take a look at your Sensor Editor class in the input folder to make some changes in the java file.
- Follow the same procedure of understanding how other sensors are implemented for the ui.xml file as well.
The files listed in this section help to improve the GAT based on the configuration your sensor requires.
This configuration is based on a sensor's implementation file. For example, if you wish to not implement a writer/filter alongside your sensor, it is completely normal to leave some of the files mentioned in this section alone. As a guideline, mimic the implementation of other sensors while you implement your own sensor to decrease the likelihood of errors.
To test the configuration of your sensor. Please refer to the Test Sensor section of the wiki.
Learner Configuration¶
If you add a new translator, classifier or predictor to the learner module, you will need to expose those options in the GAT. Begin by studying the following code:
- TranslatorTypeEnum defines an enum for each translator.
- ClassifierTypeEnum defines an enum for each classifier.
- PredictorTypeEnum defines an enum for each predictor.
- LearnerConfigurationMaps defines several hardcoded maps very similar to the SensorsConfigurationMaps for the sensors tool.
- BuilderLearnerStateInterpreterDialog is the “wizard” view used to add a new learner state interpreter.
- LearnerStateInterpreterEditor is the UI used to modify an existing learner state interpreter.
Pedagogy Configuration¶
If you add a new learner state attribute or metadata attribute to GIFT the GAT will automatically expose those choices where appropriate. Therefore there is no development needed.
Currently the pedagogy configuration authoring focuses on the Engine for Management of Adaptive Pedagogy (EMAP) and Merrill’s Component Display Theory (CDT). In the future this tool will be expanded to support authoring other types of domain-independent pedagogy configurations.
Domain configuration¶
After adding a new condition input type in dkf.xsd, you will need to make a few changes to the GAT to author performance assessments using your new condition input.
First, you will need to create an editor to allow course authors to edit your new condition input. Generally, the editor should reflect the structure of the underlying condition input class, so there isn't a clear set of instructions to follow when creating the editor. As an example, suppose we want to create an editor for a new condition called MyConditionInput defined by the following code in dkf.xsd:
<xs:element name="MyConditionInput">
<xs:annotation>
<xs:appinfo>
<fg:node-info exposed="true" message="My Condition Input">
<fg:message>This is my condition.</fg:message>
</fg:node-info>
</xs:appinfo>
</xs:annotation>
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="IdealValue">
<xs:annotation>
<xs:appinfo>
<fg:node-info exposed="true" message="Ideal Completion Duration">
<fg:message>The ideal value needed for the learner to satisfy this condition.</fg:message>
</fg:node-info>
</xs:appinfo>
</xs:annotation>
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:minLength value="1"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
Building GIFT with this condition will generate a Java class for it in GIFT/bin/jaxb_generated.jar called generated.dkf.MyConditionInput, which will look something like the following:
@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "", propOrder = {
"idealValue"
})
@XmlRootElement(name = "MyConditionInput")
public class MyConditionInput implements Serializable{
private final static long serialVersionUID = 12343L;
@XmlElement(name = "IdealValue")
protected String idealValue;
public String getIdealValue() {
return idealValue;
}
public void setIdealValue(String value) {
this.idealValue = value;
}
}
In order to allow users to author this new condition input, we'll need to create an editor that allows them to modify its IdealValue field. Since this field takes in a String value, we can use a simple text box to allow users to modify its value. To do this, we can create a new MyConditionInputEditor class in the tools.authoring.gat.client.view.task.condition package that looks like the following:
MyConditionInputEditor.java
package mil.arl.gift.tools.authoring.gat.client.view.dkf.task.condition;
import com.google.gwt.core.client.GWT;
import com.google.gwt.uibinder.client.UiBinder;
import com.google.gwt.uibinder.client.UiField;
import com.google.gwt.user.client.ui.Composite;
import com.google.gwt.user.client.ui.HasValue;
import com.google.gwt.user.client.ui.TextBox;
import com.google.gwt.user.client.ui.Widget;
public class MyConditionInputEditor extends Composite {
private static MyConditionInputUiBinder uiBinder = GWT
.create(MyConditionInputUiBinder.class);
interface MyConditionInputUiBinder extends
UiBinder<Widget, MyConditionInputEditor> {
}
@UiField protected TextBox idealValueTextBox;
public MyConditionInputEditor() {
initWidget(uiBinder.createAndBindUi(this));
}
public HasValue<String> getIdealValue() {
return idealValueTextBox;
}
}
MyConditionInputEditor.ui.xml
<!DOCTYPE ui:UiBinder SYSTEM "http://dl.google.com/gwt/DTD/xhtml.ent">
<ui:UiBinder xmlns:ui="urn:ui:com.google.gwt.uibinder"
xmlns:g="urn:import:com.google.gwt.user.client.ui">
<ui:style>
.padTop{
padding-top: 5px;
}
.padBottom{
margin-bottom: 10px;
}
.largePadBottom{
margin-bottom: 20px;
}
.tabbed{
padding: 0px 30px;
}
</ui:style>
<g:HTMLPanel addStyleNames='{style.padTop}'>
<g:HTML>
Ideal Value:
</g:HTML>
<g:FlowPanel addStyleNames='{style.tabbed} {style.padBottom}'>
<g:TextBox width="100%" ui:field="idealValueTextBox" />
</g:FlowPanel>
</g:HTMLPanel>
</ui:UiBinder>
After creating the editor, you'll have to tell the GAT to show this editor to the user whenever they choose to use your MyConditionInput class in the GAT's condition editor. To do this, you'll first need to add your editor to the "deckPanel" container in tools/gat/client/view/dkf/task/condition/ConditionPanel.ui.xml like so:
<g:DeckPanel animationEnabled='true' addStyleNames='{style.tabbed}' ui:field="deckPanel">
...
<condition:MyConditionEditor ui:field="myConditionEditor"/>
...
</g:DeckPanel>
Next, you'll have to modify tools/gat/client/view/dkf/task/condition/ConditionPanel.java as follows to make your editor available to another class later on:
public class ConditionPanel extends Composite {
...
@UiField protected MyConditionInputEditor myConditionInputEditor;
...
public MyConditionInputEditor getMyConditionInputEditor() {
return myConditionInputEditor;
}
...
}
After that, you'll also have to add the following method to tools/gat/client/view/dkf/task/TaskViewImpl.java:
...
@Override
public MyConditionInputEditor getMyConditionInputEditor() {
return dkfConditionPanel.getMyConditionInputEditor();
}
...
You'll also have to add the following method to tools/gat/client/view/dkf/task/TaskView.java, since TaskViewImpl extends it:
... public MyConditionInputEditor getMyConditionInputEditor(); ...
Then, you'll have to add a new string identifier to the CONDITION_INPUT_TYPES_ARRAY in tools.gat.client.view.dkf.task.TaskView to identify your editor with (Note that the position where you put the array in CONDITION_INPUT_TYPES_ARRAY must match the editor's position in the "deckPanel" container in ConditionPanel.ui.xml, otherwise the wrong editor will be shown):
...
final String MY_CONDITION_INPUT = "MyConditionInput";
...
final String[] CONDITION_INPUT_TYPES_ARRAY = {
...
MY_CONDITION_INPUT,
...
};
...
Finally, you'll need to make a few changes to tools.gat.client.presenter.dkf.task.TaskPresenter to allow the GAT to properly use your editor to author instances of MyConditionInput. To do this, you'll first need to add a new if-statement in the displayCondition(ConditionWithId) method to allow the TaskPresenter to properly populate your MyConditionInputEditor with data from an instance of MyConditionInput like so:
if(input != null) {
...
else if(inputType instanceof MyConditionInput) {
handleConditionInput(inputType, TaskView.MY_CONDITION_INPUT);
}
...
}
Next, you'll need to modify getConditionInput(String) to tell TaskPresenter how to initialize a new instance of MyConditionInput when the user selects that input type for the first time, like so:
...
} else if(inputType.equals(TaskView.MY_CONDITION_INPUT)) {
MyConditionInput myCondition = (MyConditionInput)conditionInputCache.get(inputType);
if(myCondition == null) {
myCondition = new MyConditionInput();
myCondition.setIdealValue(null);
conditionInputCache.put(inputType, myCondition);
}
return myCondition;
}
...
After that, you'll need to modify init() to add event handlers for each of the modifiable elements in MyConditionInputEditor and use those event handlers to update the instance of MyConditionInput being edited, like so:
...
handlerRegistrations.add(taskView.getApplicationCompletedConditionEditor().getIdealCompletionDuration().addValueChangeHandler(new ValueChangeHandler<String>(){
@Override
public void onValueChange(ValueChangeEvent<String> event) {
String idealValue = event.getValue();
MyConditionInput myCondition = (MyConditionInput)currentCondition.getInput().getType();
myCondition.setIdealValue(idealValue);
}
}));
...
Then, you'll need to create a new populateCoditionInput(MyConditionInput) method to populate the modifiable elements in MyConditionInputEditor with values from an instance of MyConditionInput, like so:
...
private void populateConditionInput(MyConditionInput myConditionInput) {
MyConditionInputEditor editor = taskView.getMyConditionInputEditor();
String idealCompletionDuration = myConditionInput.getIdealValue();
editor.getIdealValue().setValue(idealCompletionDuration);
}
...
Last but not least, you'll have to add a new if-statement to populateConditionInput(Serializable) to invoke the new method you just added, like so:
...
else if(conditionInput instanceof MyConditionInput){
populateConditionInput((MyConditionInput) conditionInput);
}
...
With that, the GAT can finally allow course authors to author performance assessments using MyConditionInput through the MyConditionInputEditor. A similar approach can be used when handling other new input types in the GAT as well, although the code used to populate and modify the condition editor will obviously change depending on the elements the editor is using.
We aren't necessarily done yet, though. While the GAT is now perfectly capable of authoring instances of MyConditionInput, it won't ever present the interface to allow authors to do so unless at least one of the condition classes in mil.arl.gift.domain.knowledge.condition has a constructor that takes in an instance of MyConditionInput as an argument. The GAT does this to prevent course authors from accidentally authoring condition inputs that will never be used, so GIFT needs to use MyConditionInput in at least one of these condition classes to allow it to be authored in the GAT. Once one of these condition classes is using MyConditionInput, course authors will finally be able author instances of MyConditionInput in the condition editor by selecting the condition class in question in the Assessment Logic field and then selecting MyConditionInput in the Logic Configuration field.
Character Servers¶
A diagram explaining how Media Semantics is integrated with GIFT is listed below. Any changes made to the characters or their surrounding logic should still support the general flow of messages portrayed by this diagram, particularly the feedback and idle messages between the TUI and the character files embedded in its iframe.
Changing UMS/LMS database schema¶
The default UMS and LMS databases are SQL based databases. Currently GIFT uses Hibernate to interact with the database tables. The Java representation of these tables can be found in the UMS and LMS packages in the source folder.
To edit or add a table to the database you will need to edit or create the corresponding Java class. There are many examples on how to create tables using annotations (e.g. primary keys, non-null columns, value generators) in GIFT as well as found online.
Once you have completed your table changes in the Java class you need to apply those changes to the existing GIFT database so that the changes can be used during GIFT execution. To do this:
1. Open the Hibernate configuration file for the appropriate database. For UMS that is “GIFT/config/ums/ums.hibernate.cfg.xml”.
2. You need to add the following property alongside the other properties in that XML file:
<property name="hbm2ddl.auto">update</property>
There should be a comment in the file with that exact line as well.
3. Save the file.
4. The next time you start the UMS or LMS database manager either by starting the UMS or LMS modules, a Junit test or other tool, the appropriate database will be updated.
If there is a logic issue with your table changes the module will most likely not start and there will be an error somewhere – usually in the modules log file.
Make sure to undo the changes to the Hibernate configuration file.
If you need to share your new default GIFT database with other developers the current procedures is as follows:
1. Clear any user data you don’t want to share such as survey responses, user accounts, LMS history, domain sessions, etc. There are several scripts to help with this in “GIFT/scripts/database/derby/”.
2. Shutdown the database server. This can be done several ways but the graceful way is to run “GIFT\data\derbyDb\stopDerbyDb.bat”.
3. Zip the changed database as the new default database for other GIFT users. For UMS the database is at “GIFT\data\derbyDb\GiftUms\”. Name the zip “DerbyDB.UMS.Backup.Original.zip” so GIFT logic can find it and unzip it if not already unzipped when performing a GIFT install.
Adding External Applications to GIFT Wrap¶
You may want to extend the existing functionality of GIFT Wrap in order to create other types of GIFT course objects or work with other external applications. The overall effort to do so involves:
- Creating a GIFT Gateway interop plugin to communicate with the external application (Creating a new Gateway Module Interop Plugin
- Extending GIFT Wrap to author the course object
GIFT Wrap doesn't produce GIFT courses but rather course objects that can be used to populate parts of a GIFT course. GIFT courses are authored using the GIFT course authoring tool (aka GIFT authoring tool).
How to start GIFT Wrap¶
Currently GIFT Wrap is only available when running GIFT in Desktop deployment mode (not Server mode). GIFT Wrap can be launched in these ways:
1) Outside of the GIFT Course Authoring Tool (new/edit)- Using the GIFT Window's system tray icon
- Using the GIFT Control Panel desktop application (GIFT\scripts\launchControlPanel.bat)
- After creating an External Application course object in the course authoring tool than selecting the "Edit with GIFT Wrap" button on the External Application course object editor. Note that the button will only appear for supported application types (e.g. Augmented REality Sandtable (ARES) ).
Adding External Applications to GIFT¶
Before you update GIFT Wrap to easily author external application course objects for your new training application, you need to make sure GIFT can communicate with that application.
For information on how to integrate a new training application, please see Creating a new Gateway Module Interop Plugin.
GIFT Wrap User Interface¶
The GIFT Wrap user interface is a series of web pages written using Google Web toolkit (GWT). If you plan on extending existing GIFT Wrap functionality than start by looking at the code in:
GIFT\src\mil\arl\gift\tools\authoring\wrap
Notice the separation of client and server side logic in that package.
The main communication between client and server is in server.GIFTWrapServiceImpl.java. There are two underlying data models:
- Server side - found in GIFT\src\mil\arl\gift\tools\services\libraries, helps to build course objects for specific external applications
- Client side - found in GIFT\src\mil\arl\gift\tools\authoring\wrap\shared, used to pass attributes to the GIFT Wrap authoring UI for editing
There are utility methods to help create DKFs, Surveys and communicate with external application Gateway interop plugins in GIFT\src\mil\arl\gift\tools\services\libraries.
Anything that is newly created using GIFT Wrap is saved in a folder under Domain\workspace\Public\TrainingAppsLib. Take a look at GIFT\src\mil\arl\gift\tools\services\libraries\util\TACourseObjectUtil.java for logic that manages saving/loading from that location for you.
Types of course objects¶
Currently GIFT Wrap provides the ability to author a 'check on learning' external application course object. Feel free to expand this selection to other types of useful assessments for various external applications.
Check on Learning Authoring¶
This type of course objects currently allows an author to present an external application scenario to the learner along with an assessment question that is presented in GIFT.
Augmented REality Sandtable (ARES)
The ARES check on learning course object produces the following output:
- trainingApp.xml - this is essentially a wrapper around a course object that normally resides in a course.xml.
- dkf.xml - contains the assessment logic to use when the ARES scenario is presented (i.e. what survey question to present and how to assess the response)
- survey.export - contains the survey question including ARES scenario object ids and survey question choice feedback
- zip - contains the ARES scenario files
When imported into a course being built in the course authoring tool, the surveys are stored in the GIFT survey database and the dkf and zip files are placed into the course folder.
GWT¶
Troubleshooting¶
GWT Compiler Errors¶
“Failed to resolve X via deferred binding“¶
This is a fairly common error to run into when attempting to reference new classes in a GWT application’s “client” or “shared” code. Without going into too much detail, deferred binding is the process that the GWT compiler uses to convert Java classes into JavaScript so that they can be used in clients’ browsers. When an error occurs during the deferred binding process, it typically means that the GWT compiler ran into some sort of problem that prevented it from converting a class to JavaScript, thus preventing the class from being used in client-side code.
The most common cause of this error is attempting to reference Java classes that do not have any sort of protocol in place to be converted to JavaScript. If this happens, the usual deferred binding error will often be accompanied with another error saying “Hint: Check the inheritance chain from your module; it may not be inheriting a required module or a module may not be adding its source path entries properly”. In order to be converted to JavaScript for a GWT application, a Java class has to meet two conditions:
First, the Java class’s source code must be explicitly included for deferred binding by the GWT application’s .gwt.xml file. There are several ways to do this. The first, and by far the easiest, is to simply place the Java class’ source file into the GWT application’s “client” or “shared” folder. Classes in the “client” file will automatically be compiled for the GWT application’s client end, while classes in the “shared” folder will be compiled for both the client end AND the server end. The second approach is to add an <inherits> statement to the GWT application’s .gwt.xml file linking to another GWT application containing the Java class’s source code. This approach is useful for using classes defined in other third-party GWT modules (such as GWT-Bootstrap3, which is commonly used in GIFT) or for reusing the same classes across multiple GWT applications (which is commonly done in GIFT by inheriting the common.gwt.GwtShared module). The third and final approach is to manually specify source bindings directly within the GWT application’s GWT.xml file. This is done by adding one or more <source> statements containing multiple <includes> statements linking to the source files for Java classes that need to be converted to Javascript. A good example of this approach is in common.GwtCommon.gwt.xml, which uses source bindings heavily.
Second, the Java class must not make any references to any other Java classes that cannot also be converted to JavaScript. This condition is usually what trips new GWT developers up, because only a specific subset of the Java runtime library is supported by GWT by default. This is typically because web browsers either lack the functionality to support certain parts of the JRE (such as the java.net package or java.lang.Thread), because JavaScript doesn’t implement some of the JRE’s expected language constructs (which would affect the java.reflect package), or because browser security protocols forbid or greatly limit the operating system support needed by certain classes in the JRE (such as many classes in the java.io package, especially java.io.File). If you only need client-side code to access a subset of the functionality from a Java class that can’t be converted to JavaScript, you can override that class with a GWT-safe version by adding a <super-source> statement to a GWT application’s .gwt.xml file. A good example of this is in common.gwt.GwtShared.gwt.xml, which overrides common.enums.AbstractEnum with a version that doesn’t use the java.reflect package, making it safe for GWT to convert into JavaScript. Super source files should be used sparingly since they can potentially confuse developers that don’t know which classes are overridden or not, but they can come in handy on rare occasions.
If a Java class satisfies both of these conditions but still fails to be resolved via deferred binding, there are a few other potential causes. The first, which is common for classes extending GWT’s Widget or Composite classes, is that the class is improperly using GWT’s UiBinder mechanism in some way. Common reasons for this include having syntax errors in a class’ .ui.xml files or @UiField annotations attached to fields that aren’t declared in said .ui.xml files. Another cause is that the class might be using native JSNI methods containing JavaScript code with syntax errors. Both of these causes can often be detected by the Google Plugin for Eclipse which allows the Eclipse IDE to perform some of the same validation work done by the GWT complier to catch deferred binding errors.
For more information about GWT’s deferred binding process, please refer to the GWT developer guide’s topic on the matter.
GWT Runtime Exceptions¶
RPC exceptions caused by NoClassDefFoundErrors¶
These exceptions can sometimes pop up while performing client-server communications using either GWT’s built-in RPC mechanism or GWT-Dispatch (which is used in the GAT). A good example of this is the following exception that was found in the GAT:
com.google.gwt.user.server.rpc.UnexpectedException: Service method 'public abstract net.customware.gwt.dispatch.shared.Result net.customware.gwt.dispatch.client.standard.StandardDispatchService.execute(net.customware.gwt.dispatch.shared.Action) throws net.customware.gwt.dispatch.shared.DispatchException' threw an unexpected exception: java.lang.NoClassDefFoundError: Could not initialize class mil.arl.gift.common.io.AbstractSchemaHandler
at com.google.gwt.user.server.rpc.RPC.encodeResponseForFailure(RPC.java:389)
at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:579)
…
Caused by: java.lang.NoClassDefFoundError: Could not initialize class mil.arl.gift.common.io.AbstractSchemaHandler
…
At first glance, this might look like a classpath issue, but in this case, what was actually happening was that the class loader used to access this class wasn’t able to fully initialize it because an exception was being thrown in one of the class’ static blocks.
In short, if you see these types of exceptions during client-server communications, check the class in question to make sure it is getting properly initialized by the class loader before being used. Logging exceptions thrown during static blocks can also help to track down these types of errors.
Running GWT Development Mode¶
Since GWT converts all client-side Java code into JavaScript, GIFT's GWT modules need to be run in Development Mode order to properly map the Java files to the JavaScript code compiled by the GWT compiler so they can be stepped through by a debugger.
GWT 2.7 and onward accomplish this by using Super Dev Mode to host JavaScript Source Maps, which are commonly used in conjunction with libraries like TypeScript that use source code written in a language different from JavaScript. Source maps were introduced in HTML5 and are commonly supported across all major browsers, and they allow said browsers's debugging tools to step through a webpage's original source code just like they would with regular JavaScript.
Note that older versions of GWT use the legacy version of Development mode, which debug's client-side code with different approach. Instead of using source maps, the server in this mode actually hosts client-side code on the server and steps through it concurrently with the client and stays in sync via a round trip of server calls. While this approach is nice in that it can be debugged in Eclipse just like the server, it generally causes the client to run slower, consumes more memory, and requires more configuration than Super Dev Mode, so it generally shouldn't be used anymore. Legacy Development Mode can still be enabled on newer versions of GWT if it is needed, but this guide will not document how to use it.
This guide will primarily focus on how to use Super Dev Mode to debug GIFT's client-side source code. Additional documentation on Super Dev Mode can be found here.
Each GIFT instance contains a "scripts" folder containing a few scripts used to launch GIFT's various GWT modules in Dev mode. We'll use scripts/tools/GWT Development Mode/launchDashboard.bat as an example to launch the Dashboard in Development Mode.
Launching the Development Mode server¶
- Start applications that will be needed by the GWT application
- Dashboard (which contains the ERT and GAT/Course Creator) - ActiveMQ, Derby database
- First install GIFT, run GIFT and make sure Hello World course works. Shutdown GIFT.
- Then start ActiveMQ (GIFT\scripts\util\launchActiveMQ.bat)
- Derby should already be running but just in case, start Derby (GIFT\external\db-derby-10.10.1.1-bin\bin\startNetworkServer.bat)
- Run the script to launch Dashboard in Dev Mode GIFT/scripts/tools/GWT Development Mode/launchDashboard.bat.
- Tutor - All of GIFT (i.e. ActiveMQ, Derby database, all GIFT modules, Dashboard)
- First install GIFT, run GIFT and make sure Hello World course works. Shutdown GIFT.
- Then start Control Panel (GIFT\scripts\launchControlPanel.bat)
- Start the Dashboard by clicking the "GIFT Dashboard" button on the Control Panel. A webpage should appear with the GIFT Dashboard.
- Start the Monitor by clicking the "Monitor" button on the Control Panel.
- Click on 'Launch All' button above the UMS and LMS buttons. Also click on all of the GIFT module launch buttons (e.g. Launch Sensor) EXCEPT for the 'Launch Tutor' button.
- Run the script to launch Tutor module in Dev Mode GIFT/scripts/modules/GWT Development Mode/launchTutor.bat.
- Dashboard (which contains the ERT and GAT/Course Creator) - ActiveMQ, Derby database
- Wait for the GWT Development Mode window to appear like so.
- Note that it make take a few minutes to launch the window if you have uncompiled changes to the GWT modules being launched, since they will need to be recompiled.
- Note that it make take a few minutes to launch the window if you have uncompiled changes to the GWT modules being launched, since they will need to be recompiled.
- Select the appropriate startup URL for the site containing the GWT module you want to connect to (e.g. dashboard/Dashboard.html).
- If only one is available, it will be selected by default.
- Click "Launch Default Browser".
- Your system's default browser will automatically be used to connect to the site.
- You can also use "Copy to Clipboard" to get a URL you can paste in a browser other than your default.
- Wait for your GWT module to load into the page.
- The first time you'll connect, you'll likely see a loading screen like the following. This loading screen appears whenever the Dev Mode server is creating source maps for your module and may take a few minutes to go away depending on how complex the GWT module's client-side code is.
- For the tutor the simple sign-on web page will appear but you can still use the Dashboard login path to take a course and debug the tutor client embedded in the Dashboard iframe.
- You may need to change the TutorURL property in common.properties to use localhost since the URL of the Tutor client launched by GWT uses a localhost IP address.
- The first time you'll connect, you'll likely see a loading screen like the following. This loading screen appears whenever the Dev Mode server is creating source maps for your module and may take a few minutes to go away depending on how complex the GWT module's client-side code is.
Debugging Client-side Java code¶
- After launching a GWT module in Dev Mode and loading its client-side into a browser, open your browser's developer tools window.
- In most browsers this can be done by pressing F12. The window that appears will likely look something like the following:
- In most browsers this can be done by pressing F12. The window that appears will likely look something like the following:
- Open the debugger in your browser's developer window.
- In Chrome, this is on the Sources tab
- In Chrome, this is on the Sources tab
- Search for the .java file that you want to add a breakpoint to.
- In Chrome, you can use Ctrl + P to search for the file.
- If the file doesn't appear (e.g. a file in the GAT):
- you might need to enter the GAT or ERT so that the source maps are generated. THIS WILL MAKE THE GAT AND ERT TAKE LONGER TO LOAD!
- you may need to use a 64x Java. See "Things to keep in mind":Developer_Guide_2024-1#Things-to-Keep-in-Mind-when-using-Dev-Mode.
- Open the .java file. The .java file will be displayed in your browser using the source maps provided by the Dev Mode server.
- Click the line number next to a line in order to place a breakpoint on it. The browser will halt its JavaScript execution when you reach this line.
- Note, the W3C web standard requires that JavaScript in browsers be single-threaded, so all activities handled via JavaScript execution will be stopped when your breakpoint is hit, including event handlers for user input and automatic scripts in the page. CSS animations and networking will not be blocked, though the JavaScript used to handle network calls once they complete will also be halted until execution continues normally again.
- Note, the W3C web standard requires that JavaScript in browsers be single-threaded, so all activities handled via JavaScript execution will be stopped when your breakpoint is hit, including event handlers for user input and automatic scripts in the page. CSS animations and networking will not be blocked, though the JavaScript used to handle network calls once they complete will also be halted until execution continues normally again.
- Interact with the client in a way that will hit your breakpoint.
- When it is hit, your main browser window will pause and the developer window will jump to the place in your .java file where execution is halted, like so:
- When it is hit, your main browser window will pause and the developer window will jump to the place in your .java file where execution is halted, like so:
- The call stack leading up to your breakpoint as well as any variable values within the scope of the breakpoint are listed to the right. You can click through the panel on the right to explore these values.
- While the functions in the call stack and variables will use same names as the original Java code, their underlying values will be shown as native JavaScript. This usually isn't a problem for primitive values, but for more-complex Java objects that don't translate easily to JavaScript, you may have to dig through these values a little to understand exactly what they are in the Java code. All JavaScript objects created from Java code using the GWT compiler will have type names similar to the names of the Java classes used to create them (in the example above, the state_0_g$ variable with the ScreenEnum_1_g$ type is the ScreenEum state field in the displayScreen method from the Java file.
- To step through the JavaScript execution, use the appropriate buttons provided in your browser's debugger. In Chrome, these look like the following:
- To return to normal JavaScript execution when finished debugging, use the appropriate button provided in your browser's debugger. In Chrome, this looks like the following:
Recompiling Client-side Java code while Dev Mode is running¶
If a GWT module's client-side Java code has changed since the Dev Mode server was deployed, the server will automatically recompile that module the next time a client attempts to load it (either via refresh or new window). This can be used to quickly build and redeploy client-side changes without bringing down the server
Recompiling Server-side Java code¶
If a GWT module's client-side Java code has changed since the Dev Mode server was deployed, the server will automatically recompile that module the next time you launch the module in Dev Mode (i.e. when you close the Dev Mode window and relaunch it).
Things to Keep in Mind when using Dev Mode¶
- The Dev Mode server is launched using its own configuration that does not acknowledge common.properties, so if common.properties is used to get the URL of one of GIFT's GWT modules, it might not necessarily match the URL that Dev Mode launches it with. This is most noticeable with the GAT, since if the GAS URL in common.properties does not match the Dev Mode server running the GAT, then the GAT IFrame in the Dashboard will not load. This can be fixed in one of two ways:
- Easy way: Modify the appropriate URL in common.properties to match the URL used by the Dev Mode server
- Harder way: Modify the "DevelopmentMode" task in the appropriate build.xml that's referenced by the script you are using the launch your GWT module in Dev Mode. Using the build.xml, you can change the arguments of the Java process that runs Dev Mode, which you can change the address and port that the Dev Mode server binds to. The arguments accepted by the Dev Mode launching process are listed in the Super Dev Mode guide linked above.
Troubleshooting¶
If you see an error (e.g. 503 response) in the browser window after pressing 'Launch Default Browser' button on the 'GWT Development Mode' window, you will need to open the 'Jetty' tab of the 'GWT Development Mode' window to explore the issue. No GIFT logs will be created at this time so you can't rely on that logging. Once on the Jetty tab explore the listed events starting at the top. IF you find a stack trace in the details panel at the bottom of the Jetty tab than see what the issue is and fix it.