Below is an article I wrote for the Adobe Developer Connection (Education Category). The full article, published July 2010, can be found at: http://www.adobe.com/devnet/edu/articles/frank_garofalo.html

- - - - - - - - - - - - - - - - - - - - - - - - - - - - -

"Multi-touch" technology has generated a large amount of buzz in the past several months. Some of the buzz is about mobile/handheld devices, and some has come from broadcast media's adoption of touch devices for their programs (such as CNN's "Magic Wall," NBC's Saturday Night Live, and ESPN). This has resulted in the term "multi-touch" being used to describe a broad range of interaction types, interfaces, and devices. Most of the uses for touch interfaces that I've seen have involved a single user interface allowing only one or two touch points at a time through tapping the screen or through gestures. However, the true capabilities of a "multi-touch" interface can accommodate multiple users interacting through any number of taps or gestures on the screen at the same time.

Multi-user Multi-touch

For the purpose of this article I do not refer to "multi-touch" to describe a smart-phone or hand-held device that can recognize only one or two points of touch simultaneously, such as a Motorola DROID; I refer to larger devices like the Microsoft Surface product that can accommodate multiple users and recognize points of touch from several users simultaneously (at least three simultaneous points of touch).

The research conducted as part of my graduate studies in the Department of Computer Graphics Technology at Purdue University (www.tech.purdue.edu/cgt/) has been a qualitative examination of multi-user collaboration with multi-touch devices. The study explored the experience of users performing a common task in a shared environment (in this case the shared environment is the single display screen of the multi-touch device). For more details about this research study visit www.multiusermultitouch.com.

Digitally Replicating a Six Sigma Brainstorming Exercise

For my research, I designed and built my own multi-touch table (which is currently pending a U.S. Patent) that could showcase the interface I developed with the Adobe Flash Platform.

For the hardware, I used an open-source software package called Community Core Vision (CCV), created by the NUI Group, to serve as a liaison between the hardware components and the Adobe Flash Player. I developed an Adobe AIR application to recreate a brainstorming exercise used within Six Sigma, known as both "Affinity Diagrams" and "KJ Analysis." Typically in Affinity Diagram exercises, participants anonymously record and submit individual ideas or concepts on separate sticky notes. Then the participants sort through the sticky notes, grouping and categorizing them by moving them physically. Through this process, unforeseen relationships between the ideas can emerge. Since this activity requires participants to interact in a shared work environment in the physical world, I chose to replicate it in a multi-touch environment to explore the implications for a multi-user interface.

Exploring the Multi-user Interface

A moderator with Six Sigma Green-Belt certification and experience working with both corporations and universities served as a facilitator for the exercises performed during user testing. I selected Adobe AIR as the client since the application could run natively on a laptop computer connected to the multi-touch table, and chose the Adobe Flash Platform to capture the experience of multiple users simultaneously interacting with multi-touch interface and hardware. In the AIR application, users were able to move virtual sticky notes on the multi-touch table, and perform the same grouping and categorizing exercises that people do with physical sticky notes in an Affinity Diagram exercise.

While both the hardware and software of the multi-touch device created a few challenges for the participants on various levels, including occasional software failures, they were able to complete the exercises. That benefited not only this study, aimed at collecting qualitative data from the participants, but also the participants themselves, who were able to take the results of the Affinity Diagram exercises back to their respective organizations on the Purdue campus.

Affirming Research Assumptions

My goal was to determine whether a multi-touch interface and device can be a practical shared environment for a team of individuals to complete a common task. The participants indicated that for completing an Affinity Diagram exercise, the multi-touch device proved to be a useful tool and believe that it improved collaboration. The moderator for the Affinity Diagram exercise was surprised at the speed with which the participants in both rounds completed the tasks, despite a learning curve for them to become familiar with the multi-touch device. She remarked that she has "not done one on paper that quickly." Furthermore, participants in the study expressed appreciation for the usefulness the multi-touch device provided. It allowed them to see all the data in front of them at once, and to see most of the actions of their fellow participants.

About the author

The submission of the Adobe AIR application I developed as the user-interface for my graduate research project has been selected as a semi-finalist in the Adobe Design Achievement Awards. Below are my submission details:

• Submission Category: Non-Browser-Based Design
• Submission #: 484
• Entry title: Multi-user Collaboration
• Number of Pieces: 1

For more details about the Adobe Design Achievement Awards, visit: https://adaa.adobe.com/us/content/adaaLive

My Defense Presentation is tomorrow and I'm ready! The Defense will take place at 1pm EST in room Knoy 373. The presentation slides are done and I've reviewed them several times already.

Grad school has been an interesting and challenging journey. I'll openly admit that there were some points along the way I wasn't sure if I was going to see the end. I'm glad it's finally coming to a conclusion. I have learned a significant amount not only about my research topic and methods for conducting qualitative research studies, but also I've learned more about myself... the cliche of "when the going gets tough, the tough get going" comes to mind.

I have several "thank yous" to give out for my sincere appreciation:

• My parents and brother
• Robert Brophy
  
• Dr. James Mohler
• Prof. Rodney Vandeveer
• Prof. Terry Burton
• Prof. Gail Farnsley
• Dr. Mary Sadowski
• Julie Talz
• My Resident Assistant Staff Members at Cary Quad from the past few years
• ... and all my friends who have stood by me (and let me vent/rant at times!), you know who you are

Thanks!

Today marked the beginning of Round 2 of user testing with Part 1 - the Affinity Diagram exercise. From Round 1 (User Testing - Part 1) the voluntary participant number wasn't high enough so a second round of user testing was required. The testing today went very well overall. I'm glad to say that the system didn't crash, as it did during Round 1 - Part 1 (so the modification I made to the open source code seemed to resolve the issue that was occurring).

The largest challenge with the user testing today seemed to be when the digital sticky notes would either move to the default top left corner (the 0,0 coordinate) or it would move to the location of the last touch point (which was usually another testing participants touch point). This seems to be an issue with Community Core Vision (CCV), the open source code, and the ActionScript 3.0 class which communication with CCV.

The digital post-it notes being sorted included various popular social networking sites, such as Facebook, YouTube, Flickr, Twitter, etc. Special thanks for volunteering to be video taped - Sean S., Brian E., Ben T., and Mike M.

Yesterday (April 19th 2010), I started part 1 of the user testing for my research project regarding the multi-touch device and software application I've been developing. The software application is based on the Six Sigma Affinity Diagram (aka KJ Analysis) exercise. This is typically used to sort through qualitative data (such as ideas from a brainstorming session) to find unforeseen relationships between the the data items / ideas by grouping them into categories.

I was granted permission by Dr. Mary Sadowski, Associate Dean for the College of Technology at Purdue University, to ask faculty members from a committee within the College of Technology to use my multi-touch device and software application to aide them in sorting through some ideas/outcomes regarding a project they are working on.


Image 1 - Doing a run-through prior to user testing during set-up.


Image 2 - User Testing.

During the user testing the software application unfortunately crashed twice. Each time the users had to restart the Affinity Diagram from the beginning. After the second crash, I went into the code and quickly made some alterations. The problem was occurring with an open-source class written in ActionScript 3.0 which processes data received from Community Core Vision (the open-source software which interprets the images received from the camera component of the device's hardware). I added some "try / catch" statements. This seemed to resolve the issue since the software application didn't crash after the modifications.

Overall, the user testing proved to be successful. Excellent data was collected both regarding the interaction of user-to-user and user-to-interface. Also, notes for improvements to the software application.

After looking everywhere to try to figure out how to save data to an existing XML file, this is what I came up with:

//~~~~~~ IMPORT FLASH CLASSES
import flash.filesystem.*;

//~~~~~~ VARIABLE DECLARATION
var dataPathLocal:String = "com/frankgarofalo/data/"; //file path
var destinationFile:File = File.userDirectory; //Where to save the file on local computer
var sourceFile:File = File.createTempFile(); //Temporary File
var writeStream:FileStream = new FileStream(); //File Access
var xmlData:XML = new XML();

//~~~~~~ CODE
xmlData = 1; //XML Data
destinationFile = destinationFile.resolvePath(dataPathLocal);
//trace (destinationFile .exists);
//trace (destinationFile .nativePath);
//trace (destinationFile .url);

//Open File & Write to the File
writeStream.open(sourceFile, FileMode.WRITE);
writeStream.writeUTFBytes(xmlData);
writeStream.close();
sourceFile.copyToAsync(destinationFile, true);
sourceFile.addEventListener(Event.COMPLETE, dataSaved);	

function dataSaved(e:Event):void {
    trace ("~~~~~~ XML SAVED SUCCESSFULLY ~~~~~~");
}

Hopefully I didn't forget anything from copying over the code from my AS file. If I did or if you have any questions, please let me know. Here are some links for reference:

• Adobe LiveDocs: flash.filesystem.File
• Adobe LiveDocs: flash.filesystem.FileStream

I've made progress on my multi-touch software. I can now have multiple assets displayed on the device being touched (selected) at the same time and have events respond respectively to each individual asset selected. Getting this to work was a major accomplishment. Thank goodness for error catching.



Now I've run into another problem trying to save over an existing XML file with updated data. Spent about 3 hours this evening trying to figure it out without much success. I've searched Google to see what other people have tried but I've yet to find a working solution... slightly frustrating. If I find one, I'll post about it.

Adobe Wave: Displaying a desktop notification is as simple as sending an e-mail. Still in Adobe Labs.
http://labs.adobe.com/wiki/index.php/Adobe_Wave

xOpen AIR Chat is the creation of a senior-design project in the Department of Computer Graphics Technology at Purdue University. Members of the team are Kristeen Hudson, Will Grauvogel, and David Auble.

Built with Adobe Flex and XMPP, the application also uses custom Flex compontents created in Adobe Flash. The application runs on the Adobe AIR runtime.

For more details visit http://www.xopenair.com/

Adobe has a free ;poster promotion going on right now. These large posters (2 feet by 3 feet) will cover most walls and show packages and cl es available to Flex developers. (Source: Adobe.com)

https://www.adobe.com/cfusion/entitlement/index.cfm?e=posters

On Monday, October 20th 2008, ;Professor Kellen Maicher and I are presenting a research poster entitled cgCentral: ;An AIR Application for Managing Course Information and Simulation Data at the 2008 IT Summit at Purdue University. Information presented includes details and testing results ;about the lab testing phase of the cgCentral AIR App from the Spring 2008. Also, details about the field testing of cgCentral currently underway this fall.

cgCentral: An AIR Application for ;Managing Course Information and Simulation Data

PROBLEM STATEMENT
Dissemination from CGT 411/450 to the Computer Graphics Technology student population of information regarding COGENT stock prices, individual COGENT accounts and press releases.

RESEARCH QUESTION
Are rich internet applications such as CGCentral more usable than traditional browser-based tools for disseminating information from a senior-level course to an entire department student body?

ABSTRACT
The senior design capstone course in the Department of Computer Graphics Technology at Purdue University provides an opportunity to engage students using COGENT, a fictitious currency used to evaluate performance. A simulation of a stock market for the senior design groups allows for undercl men to receive, exchange and invest COGENT dollars during their time as a Computer Graphics Technology student. Through this interaction undergraduates can accumulate COGENT in an effort to enhance their final grade for the course. Building on previous research in this area, this project addresses which medium is perceived by students to be the most effective to disseminate information: 1) desktop-based rich Internet applications or 2) dynamic web sites and RSS feeds.

We have a date set ;for an Adobe ;Flex and Adobe ;AIR infomation / training session scheduled for Thursday, November 30th at 6pm in the DLC. The event will be open to all Purdue students, however space will be limited.

The presentation will discuss emerging Adobe technologies, such as AIR and Flex. During the presentation we will provide a demo/tutorial (at an introductory level) ;regarding how to build an AIR application with Flex and Flash.

For more details check out the group on Facebook: Adobe on Campus - Purdue RIA User Group

We hope to see you there.

PROBLEM STATEMENT
The current information dissemination mechanisms used in CGT 411/450 are perceived inefficient because of the lack of a centralized source of knowledge regarding the tracking of tasks related to the CGT 411/450 course simulation, in turn detracting from the true goal of the course, the scholarly research endeavor.

RESEARCH QUESTION
Are dedicated standalone rich internet applications such as CGCentral more usable than traditional browser-based tools for disseminating CGT 411/450 information?

ABSTRACT
CGT 411 is a senior capstone course in which students simulate the complexities of real-world corporations and business environments related to applied computer graphics . Part of this simulation includes the student and faculty functions of generating, tracking, and maintaining large amounts of mock data including stock information, press releases, corporate and individual finances. Typically, these functions have been achieved through a variety of mechanisms including web sites, RSS feeds, forums, and conferencing applications. Due to size and complexity of information required for the function of this simulation, the use of the previously mentioned applications have proven to be inefficient and inadequate. The authors propose the creation of a Rich Internet Application using the Adobe AIR technology that will consolidate the functions of the multiple simulation mechanisms into a single, unified application. The RIA will be subjected to usability and essment testing to ensure that an effective application is built that achieves the goals of the project.

On March 4th 2008, our research team was awarded the 2008 Teaching and Learning Technologies 2008 Digital Content Development Grant, presented by Information Technology at Purdue (ITaP). The grant will support the further research of the COGENT AIR Application in which a prototype is currently being built by a CGT 411 team as a senior design project.

More Details

The Adobe Integrated Runtime was launched by Adobe Systems in February 2008. Prior to the launch it was code-named Apollo. Adobe describes the AIR technology as a cross-operating system runtime that allows web application developers to use their existing web development skills (HTML, Javascript, Adobe Flash, Adobe Flex, Ajax) to build and deploy rich Internet applications to the desktop. Retreived on February 10 2008 from http://labs.adobe.com/downloads/air.html?promoid=ZMIZ