GIFT Unity Embedded Application Developer Guide 2017-1¶

• GIFT Unity Embedded Application Developer Guide 2017-1
  • Setting Up the Unity Development Environment
  • Building a Unity Application for GIFT
  • Creating Your First Embedded Unity Application
    • Creating the UI
    • Importing the GIFT Unity SDK
    • Adding the GiftConnection
    • Handling Messages From GIFT
    • Sending GIFT Messages
    • Connecting the UI to the Script
    • Finishing Up
  • GIFT Unity API Overview
    • Mil.Arl.Gift.Unity.Connectivity
    • Mil.Arl.Gift.Unity.Messaging
    • Mil.Arl.Gift.Unity.Messaging.Incoming
    • Mil.Arl.Gift.Unity.Messaging.Outgoing
  • GIFT Unity Communication Architecture
    • Message Examples
  • Supporting New Messages
    • Create a New Type Identifier
    • Create a New C# Payload Class
    • Associating Embedded Messages With GIFT Messages
  • FAQ
    • What happens to uncaught exceptions?

Setting Up the Unity Development Environment¶

The Unity Editor is required in order to build Unity applications for GIFT. The earliest version of the Unity Editor that GIFT supports for creating embedded applications is 5.6. If your version of Unity is older, your application may not be able to be built for GIFT or may not behave as expected.

Specific steps need to be taken during the installation in order to build a Unity application that will work with GIFT.

1. Go to https://store.unity.com/, select a version of Unity, and download the installer. All versions of Unity, including the free version, are capable of creating applications that can communicate with GIFT.
2. Run the Unity installer once the download completes. Proceed through the installer until the 'Choose Components' screen is reached.
3. Make sure that 'WebGL Build Support' is checked within the list of features in addition to the defaults. NOTE: Visual Studio Community can add substantial time to the install time. If you already have an IDE or text editor that you feel comfortable editing C# Unity scripts in, you can leave the Visual Studio Community option unchecked.
   
4. Proceed through the remainder of the installer with the defaults and wait for the installation to complete

Building a Unity Application for GIFT¶

In addition to Unity's build process the output has to be packaged in a way that will allow it to be uploaded to GIFT.

1. Click File > Build Settings... in the menu at the top of the Unity Editor.
2. On the dialog that appears, select WebGL from the Platform list
3. Click Switch Platform to make this the default platform to build for
4. Click Build and select a folder to output the Unity WebGL application to.
5. Once the build process completes a file explorer will open to the location of the output folder. Open the folder to view the build output.
6. Place the entire output of the Unity application inside a zip archive so that its structure matches the folder selected for build output.

Creating Your First Embedded Unity Application¶

Before following this procedure be sure that you've set up the development environment. For this procedure, start with a new, blank Unity 3D project. The following procedures are for creating a new example application but similar steps will need to be taken with existing Unity applications in order to enable communication with GIFT. Once your development environment is set up and a blank project has been created, you are ready to start developing your first embedded Unity application.

Creating the UI¶

For the first application we will create a simple UI with two buttons and a text element. One button for displaying feedback to the learner, and another for ending the training application scenario. The text field is used to display feedback from GIFT in the embedded training application. First create the feedback button.

1. Click 'Create' in the Hierarchy panel and select UI > Button
2. Expand the new Canvas GameObject that appears in the Hierarchy to reveal a Button GameObject.
   
3. Right click Button and choose Rename
4. Name the button FeedbackButton
5. Expand the FeedbackButton GameObject to reveal a Text GameObject
6. Select the Text GameObject
7. Change the Text property in the Inspector panel from Button to Feedback
8. Select FeedbackButton in the Hierarchy panel and use the Rect Transform in the Inspector panel to position the Button on the screen
   

After the feedback button has been created, the end button should be created in a similar fashion.

1. Right click the Canvas GameObject and select UI > Button
2. Right click Button and choose Rename
3. Name the button EndButton
4. Expand the EndButton GameObject to reveal a Text GameObject
5. Select the Text GameObject
6. Change the Text property in the Inspector panel from Button to End
7. Select EndButton in the Hierarchy panel and use the Rect Transform in the Inspector panel to position the Button on the screen

Finally after both buttons have been added, the text element used to display the feedback needs to be added.

1. Right click the Canvas GameObject and select UI > Text
2. Right click the Text GameObject and choose Rename
3. Name the Text element FeedbackText
4. Set the Text in the Inspector to No Feedback
5. Position the feedback text using the Rect Transform in the Inspector

Importing the GIFT Unity SDK¶

In order to make adding GIFT functionality to a Unity application easier, the GIFT Unity SDK was created. It includes prebuilt C# classes, Unity scripts, and Unity prefabs for communicating with GIFT from the context of an embedded application.

1. Download and extract the Simple Example Unity Application Project.zip from the Downloads Page at https://www.gifttutoring.org/projects/gift/files
2. Within the Unity Editor, click Assets > Import Package > Custom Package... from the menu at the top of the Unity Editor
3. In the file explorer that appears, navigate to the extracted folder and select the Gift Unity Sdk.unitypackage within the Gift Unity Resources folder of the extracted zip
4. In the dialog that appears in Unity select all the assets listed and click Import
   

The GIFT Unity SDK is now imported into your Unity project! You are now ready to begin GIFT Unity development.

Adding the GiftConnection¶

The first step for any Unity project is add the GiftConnection GameObject to the scene. The GiftConnection object is an invisible GameObject that allows the Unity application to receive messages from GIFT.

1. Navigate to Assets\Prefabs within the Project panel
2. Drag the GiftConnection prefab from the Project panel to the root of the Hierarchy panel on the left side of the editor
   

Your Unity Application is now able to receive messages from GIFT! You are now ready to write handlers for these messages.

Handling Messages From GIFT¶

Every application needs to be able to handle Siman messages. These messages are used by GIFT to manage the lifecycle of a training application. It is important to respond to these messages correctly so that the course is able to proceed and not end prematurely or hang.

First a GameObject must be created to hold the Siman message handler.

1. Click Create in the Hierarchy panel and select Create Empty
2. Rename the new GameObject to GiftEventHandler
3. Click Add Component in the Inspector and click New Script
4. Name the script GiftEventHandler
5. Go to the root of the Assets folder in the Project panel and double click GiftEventHandler.cs to edit the file

Now add the following using statements and fields to the GiftEventHandler class within the script. The Text object will be a reference to Text used to display feedback messages from GIFT. The AbstractGiftConnector is what allows the Unity application to receive and send GIFT messages from anywhere within the application.

using Mil.Arl.Gift.Unity.Connectivity;
using Mil.Arl.Gift.Unity.Messaging.Incoming;
using UnityEngine;
using UnityEngine.UI;

public Text feedbackText;
private AbstractGiftConnector connector = null;

Now add the following methods to the GiftEventHandler class within the script. This is the method that will be called when a Siman message is received from GIFT. Depending on the type of Siman message that is received from GIFT the handler can perform the appropriate logic. After the message has been handled the event handler needs to send an acknowledgement message back to GIFT to inform it the operation has successfully completed. If this message is not sent the course will become stuck and not proceed further. It is also important to not send a Loaded, Started, or Stopped message if there is no corresponding received Siman message. This can easily happen if there are multiple handlers subscribed to the OnSimanReceivedEvent within the same Unity scene or different Unity scenes.

private void handleSimanMessage(Siman siman) {
   switch(siman.Siman_Type) {
       case Siman.Load:
          connector.SendLoadedMessage();
          break;
       case Siman.Start:
          connector.SendStartedMessage();
          break;
       case Siman.Stop:
          connector.SendStoppedMessage();
          break;
    }
}

private void handleFeedbackMessage(string feedback) {
   feedbackText.text = feedback;
}

Now insert the following code into the Start method. This gets the singleton instance of Unity's connection to GIFT and then subscribes the event handlers to the events that are raised when the connector receives messages from GIFT.

if(connector == null) {
   connector = GiftConnectorFactory.CreateGiftConnector(typeof(UnityWebGLGiftConnector));
   connector.OnSimanReceived += handleSimanMessage;
   connector.OnFeedbackReceived += handleFeedbackMessage;
}

Sending GIFT Messages¶

In order to convey the learner's state or trigger actions such as presenting feedback, messages have to be sent to GIFT. In this example we will send a SimpleExampleState to GIFT containing text that indicates our feedback button was pressed. To accomplish this, a button click handler will be created and subscribe to the button's click event.

First the event handler for the feedback button and stop button will be created by inserting the following methods into the GiftEventHandler class.

public void FeedbackButtonClick() {
   connector.SendSimpleExampleState("feedback");
}

public void EndButtonClick() {
   connector.SendFinishedMessage();
}

The FeedbackButtonClick method will be called when the FeedbackButton is clicked. It sends a SimpleExampleState to GIFT which will process it based on the DKF supplied by the course author. The EndButtonClick method will be called when the EndButton is clicked. It sends a StopFreeze message to GIFT which signals that the training application has completed. GIFT will begin to move to the next course transition if one exists.

Connecting the UI to the Script¶

In order for the behavior in the script to interact with the UI, the two have to be connected. Once the click handlers have been created for both buttons, they can subscribe to the click events of their respective buttons. First connect the FeedbackButton to its event handler in the GiftEventHandler class.

1. Select FeedbackButton in the Hierarchy panel
2. In the Inspector panel click the add button within the On Click() setting
3. Use the GameObject selector (circle icon) beneath Runtime Only to select the GiftEventHandler class
   
4. In the Function drop down list choose GiftEventHandler > FeedbackButtonClick()

The EndButton is connected to its event handler in a similar fashion.

1. Select EndButton in the Hierarchy panel
2. In the Inspector panel click the add button within the On Click() setting
3. Use the GameObject selector beneath Runtime Only to select the GiftEventHandler class
4. In the Function drop down list choose GiftEventHandler > EndButtonClick()

In addition to the click handlers, the GiftEventHandler class needs to be given a reference to the Text object used to display the feedback.

1. Select GiftEventHandler in the Hierarchy panel
2. In the Inspector panel click the GameObject selector next to the Feedback Text field.
3. In the Select Text dialog that appears, select FeedbackText and close the dialog

Now the development of your first embedded application is complete

Finishing Up¶

Now that the application is written it needs to be packaged and used in a GIFT course. To build the Unity application for GIFT follow the instructions in the Building a Unity Application for GIFT section

Once your application is built, use the GAT to build a course with a Unity WebGL course object. A Unity WebGL course object requires the zipped output created by the build instructions and a DKF that instructs GIFT how to react to the GIFT messages being sent from the embedded training application. When specifying the DKF for the Unity training application, create a DKF that responds to a SimpleExampleState with the string value "feedback". More information about GIFT's DKFs can be found here.

Perform the following steps to add a Unity application to an existing GIFT course:

1. Add a Unity WebGL course object to the course
2. Click Select Unity WebGL Application and select the zipped output of the Unity build
3. Save and run the course

GIFT Unity API Overview¶

A brief description of each of the namespaces is included below. For detailed information on the API, read the API Documentation here: https://www.gifttutoring.org/attachments/download/2377/GIFT%20Unity%20SDK%20Documentation%202017-1.pdf

Mil.Arl.Gift.Unity.Connectivity¶

Contains classes for communicating to and from GIFT. Contains AbstractGiftConnector which can be extended to be used to communicate with GIFT on other platforms. The namespace also contains UnityWebGLGiftConnector which is an implementation of AbstractGiftConnector used by Unity WebGL applications embedded in a webpage presented by GIFT to communicate information back to GIFT.

Mil.Arl.Gift.Unity.Messaging¶

Contains the Message class which defines the message format used by the embedded Unity application to communicate with GIFT.

Mil.Arl.Gift.Unity.Messaging.Incoming¶

Contains classes which act as message payloads for messages that are sent from GIFT to the embedded application. These types should only be created by the API and should only be read by the API caller.

Mil.Arl.Gift.Unity.Messaging.Outgoing¶

Contains classes which act as message payloads for messages sent from the embedded application to GIFT. These types are created by the user or the API and are sent to GIFT.

GIFT Unity Communication Architecture¶

GIFT communicates with embedded Unity applications through the Tutor Module where the embedded application resides. Messages sent from the Domain module to embedded training applications are sent to the Tutor client using the same pipeline as the messages that tell the Tutor client what widgets to display. Once a message being sent to the embedded training application reaches the Tutor client, it is sent to the embedded application using the window.postMessage JavaScript API on the iframe that is hosting the embedded application.

Once the index.html page inside the iframe receives the message from the Tutor client, it passes the message into the game engine using an API provided by Unity. Once the message is received by the engine the GIFT-Unity SDK deserializes the message and invokes the appropriate handlers supplied by the Unity developer.

Legend

Key	Description
1	Message received from GIFT as a JSON string. These messages include: SIMAN Messages and Feedback Messages
2	C# object representing the message that was received as a JSON string
3	Message sent to GIFT as a JSON string. These messages include: SimanResponse, StopFreeze, SimpleExampleState

Message Examples¶

Message Type	JSON
SIMAN	{"payload":"{\"Siman_Type\":\"Start\",\"RouteType\":\"Embedded\",\"FileSize\":0}","type":"Siman"}
Feedback	{"payload":"{\"StringPayload\":\"This is feedback because you pressed button 1.\"}","type":"Feedback"}
SimanResponse	{"type":"SimanResponse","payload":"Load"}
StopFreeze	{"type":"StopFreeze","payload":"{\"realWorldTime\":0,\"reason\":0,\"frozenBehavior\":0,\"requestID\":0}"}
SimpleExampleState	{"type":"SimpleExampleState","payload":"{\"VAR\":\"button 1\"}"}

Supporting New Messages¶

The Message class is the mechanism for transmitting a message from an embedded application to GIFT or sending a message from GIFT to an embedded application. The Message class is serialized to JSON and sent to GIFT to be It consists of two fields: payload and type. The payload field is a string which contains the data being sent to or from GIFT in the form of JSON. The type field is a string that identifies the type of the payload and specifies which JSON codec to use on the GIFT server or which class to deserialize and cast the resulting object to in the embedded application. Currently the SDK only supports a subset of the messages that are available in GIFT but adding support for a new message type is a straight forward process. To support a new message:

1. Create a New Type Identifier
2. Create a New C# Payload Class
3. Associating Embedded Messages With GIFT Messages

Throughout the description of the procedure for supporting new message types, example code will be included. This example code reflects the implementation of the SimpleExampleState message type.

Create a New Type Identifier¶

The first step is to create the type identifier for the new payload type. This type identifier needs to be added in two locations.

The new type identifier needs to first be added to the Gift Unity C# API. Within the Mil.Arl.Gift.Unity.Messaging namespace, the Message class contains several public const string fields. The type identifier similarly should be added to the Message class as a public const string field. When creating a new Message the type field of the message can be set to the new type identifier to send a message of that type.

The new type identifier also needs to be added to the GIFT server side code. Within the mil.arl.gift.tutor.web.server package, the EmbeddedAppMessageEncoder contains a private enum EncodedMessageType. The new type identifier needs to be added to this enumeration.

Message.cs

...
public const string SimpleExampleState = "SimpleExampleState";
...

EmbeddedAppMessageEncoder.java

...
private enum EncodedMessageType {
   ...
   SimpleExampleState,
   ...
}
...

IMPORTANT: Make sure that the public const string of the Message class and the new value within the EncodedMessageType enum match each other exactly.

Create a New C# Payload Class¶

A new C# payload class must be created to hold the data of the new message type. The payload class is serialized as JSON before it is transmitted to GIFT. Some important things to note for the payload class:

• The class must be marked with the [Serializable] attribute
• Only the public fields will be serialized
• The name of the public field is the name of the key used in the JSON's key-value pair.

SimpleExampleState.cs

[Serializable]
public class SimpleExampleState {
   public SimpleExampleState(string var) {
      VAR = var;
   }

   public string VAR;
}

IMPORTANT: Make sure that the key names (public field names) match the names that the codec expects in the next step.

Associating Embedded Messages With GIFT Messages¶

Once the type identifier has been added and the payload class has been created, the type identifier needs to be associated with a codec and the type of object that the codec produces needs to be associated with a GIFT message type. First, within the EmbeddedAppMessageEncoder, put a new entry into the messageTypeToCodec map using the new enum value as the key and an instance of the desired JSON codec as the value. JSON codecs used should be created from classes within the mil.arl.gift.net.api.message.codec.json package of GIFT. Next, the type of object that the codec produces needs to be mapped to a MessageTypeEnum. Add an entry into the decodedPayloadClassToMessageType map with the object type of the deserialized payload as the key and the corresponding MessageTypeEnum as the value.

EmbeddedAppMessageEncoder.java

...
private static SimpleExampleStateJSON SIMPLE_EXAMPLE_STATE_JSON_CODEC = new GenericJSONStateJSON();
...
static {
   ...
   messageTypeToCodec.put(EncodedMessageType.SimpleExampleState, SIMPLE_EXAMPLE_STATE_JSON_CODEC);
   ...
}
...
static {
   ...
   decodedPayloadClassToMessageType.put(SimpleExampleState.class, MessageTypeEnum.SIMPLE_EXAMPLE_STATE);
   ...
}
...

FAQ¶

What happens to uncaught exceptions?¶

Exceptions that are thrown by event handlers are caught by the GIFT Unity C# API and displayed in the details section of an error dialog shown to the learner. If such an exception is thrown, the course will end prematurely after the learner acknowledges the dialog.