I recently read an excerpt from J. Licklider in a book, The Design of Future Things by Donald A. Norman. The discussion in the excerpt discusses "Man-Computer Symbiosis" (a similar article can be found at here). Licklider describes the concept of symbiosis, in terms of user-interface design and human-computer interaction, as "a merger of two components, one human, one machine, where the mix is smooth and fruitful, the resulting collaboration exceeding what either is capable of alone." I found this to be a very interesting description of what the optimum goal for a user-interface should entail. The Merriam-Webster Online Dictionary definition for symbiosis is "the living together in more or less intimate association or close union of two dissimilar organisms (as in parasitism or commensalism); a cooperative relationship." The main concept here is that a user-interface should provide a "cooperative relationship" between the hardware/software interface and the human using such interface.

Below is an article I wrote for the Adobe Developer Connection (Education Category). The full article, published July 2010, can be found at: http://www.adobe.com/devnet/edu/articles/frank_garofalo.html

- - - - - - - - - - - - - - - - - - - - - - - - - - - - -

"Multi-touch" technology has generated a large amount of buzz in the past several months. Some of the buzz is about mobile/handheld devices, and some has come from broadcast media's adoption of touch devices for their programs (such as CNN's "Magic Wall," NBC's Saturday Night Live, and ESPN). This has resulted in the term "multi-touch" being used to describe a broad range of interaction types, interfaces, and devices. Most of the uses for touch interfaces that I've seen have involved a single user interface allowing only one or two touch points at a time through tapping the screen or through gestures. However, the true capabilities of a "multi-touch" interface can accommodate multiple users interacting through any number of taps or gestures on the screen at the same time.

Multi-user Multi-touch

For the purpose of this article I do not refer to "multi-touch" to describe a smart-phone or hand-held device that can recognize only one or two points of touch simultaneously, such as a Motorola DROID; I refer to larger devices like the Microsoft Surface product that can accommodate multiple users and recognize points of touch from several users simultaneously (at least three simultaneous points of touch).

The research conducted as part of my graduate studies in the Department of Computer Graphics Technology at Purdue University (www.tech.purdue.edu/cgt/) has been a qualitative examination of multi-user collaboration with multi-touch devices. The study explored the experience of users performing a common task in a shared environment (in this case the shared environment is the single display screen of the multi-touch device). For more details about this research study visit www.multiusermultitouch.com.

Digitally Replicating a Six Sigma Brainstorming Exercise

For my research, I designed and built my own multi-touch table (which is currently pending a U.S. Patent) that could showcase the interface I developed with the Adobe Flash Platform.

For the hardware, I used an open-source software package called Community Core Vision (CCV), created by the NUI Group, to serve as a liaison between the hardware components and the Adobe Flash Player. I developed an Adobe AIR application to recreate a brainstorming exercise used within Six Sigma, known as both "Affinity Diagrams" and "KJ Analysis." Typically in Affinity Diagram exercises, participants anonymously record and submit individual ideas or concepts on separate sticky notes. Then the participants sort through the sticky notes, grouping and categorizing them by moving them physically. Through this process, unforeseen relationships between the ideas can emerge. Since this activity requires participants to interact in a shared work environment in the physical world, I chose to replicate it in a multi-touch environment to explore the implications for a multi-user interface.

Exploring the Multi-user Interface

A moderator with Six Sigma Green-Belt certification and experience working with both corporations and universities served as a facilitator for the exercises performed during user testing. I selected Adobe AIR as the client since the application could run natively on a laptop computer connected to the multi-touch table, and chose the Adobe Flash Platform to capture the experience of multiple users simultaneously interacting with multi-touch interface and hardware. In the AIR application, users were able to move virtual sticky notes on the multi-touch table, and perform the same grouping and categorizing exercises that people do with physical sticky notes in an Affinity Diagram exercise.

While both the hardware and software of the multi-touch device created a few challenges for the participants on various levels, including occasional software failures, they were able to complete the exercises. That benefited not only this study, aimed at collecting qualitative data from the participants, but also the participants themselves, who were able to take the results of the Affinity Diagram exercises back to their respective organizations on the Purdue campus.

Affirming Research Assumptions

My goal was to determine whether a multi-touch interface and device can be a practical shared environment for a team of individuals to complete a common task. The participants indicated that for completing an Affinity Diagram exercise, the multi-touch device proved to be a useful tool and believe that it improved collaboration. The moderator for the Affinity Diagram exercise was surprised at the speed with which the participants in both rounds completed the tasks, despite a learning curve for them to become familiar with the multi-touch device. She remarked that she has "not done one on paper that quickly." Furthermore, participants in the study expressed appreciation for the usefulness the multi-touch device provided. It allowed them to see all the data in front of them at once, and to see most of the actions of their fellow participants.

About the author

Today I gave my defense presentation to my graduate research committee and the professors signed-off on my research report and passed me! I get to graduate in August with a Master of Science in Computer Graphics Technology.

My Defense Presentation is tomorrow and I'm ready! The Defense will take place at 1pm EST in room Knoy 373. The presentation slides are done and I've reviewed them several times already.

Grad school has been an interesting and challenging journey. I'll openly admit that there were some points along the way I wasn't sure if I was going to see the end. I'm glad it's finally coming to a conclusion. I have learned a significant amount not only about my research topic and methods for conducting qualitative research studies, but also I've learned more about myself... the cliche of "when the going gets tough, the tough get going" comes to mind.

I have several "thank yous" to give out for my sincere appreciation:

• My parents and brother
• Robert Brophy
  
• Dr. James Mohler
• Prof. Rodney Vandeveer
• Prof. Terry Burton
• Prof. Gail Farnsley
• Dr. Mary Sadowski
• Julie Talz
• My Resident Assistant Staff Members at Cary Quad from the past few years
• ... and all my friends who have stood by me (and let me vent/rant at times!), you know who you are

Thanks!

Today I received edits back from the chair of my graduate studies, Dr. James Mohler. His feedback, along with the edits, was:

I have gone through your document and I think it is in pretty good shape. I have edited and you can pick up the changes I am suggesting. Most edits are grammatical errors/small things to tighten up the document. You have done a good job analyzing and summarizing the qualitative data.

So... good news! I've completed making all of his recommended changes and submitted it back to all my committee members: Dr. James Mohler, Prof. Terry Burton and Prof. Rodney Vandeveer.

The main draft of my research paper is now complete. Here is an overview:

• Content Pages: 127 (including the Bibliography)
• Content Page Word Count: 41,894 words (including section titles, not including the table of content and list of figures)
• Total Pages: 154
  

I've completed transcribing the audio recordings of the 8 1-on-1 interviews and 1 of the 2 focus group sessions. Now it's time to start the data analysis...

From all 8 interviews there were 5,846 words transcribed, but to get a visual of the most frequent spoken words from my testing participants (plus filtering out 'common English words') below is the result - thanks to Wordle:

Today marked the beginning of Round 2 of user testing with Part 1 - the Affinity Diagram exercise. From Round 1 (User Testing - Part 1) the voluntary participant number wasn't high enough so a second round of user testing was required. The testing today went very well overall. I'm glad to say that the system didn't crash, as it did during Round 1 - Part 1 (so the modification I made to the open source code seemed to resolve the issue that was occurring).

The largest challenge with the user testing today seemed to be when the digital sticky notes would either move to the default top left corner (the 0,0 coordinate) or it would move to the location of the last touch point (which was usually another testing participants touch point). This seems to be an issue with Community Core Vision (CCV), the open source code, and the ActionScript 3.0 class which communication with CCV.

On April 23rd 2010 a provisional patent application was filed with the United States Patent and Trademark Office (USPTO). The provisional number is 61/327,354 with Frank J. Garofalo and James L. Mohler listed as the inventors, and the Purdue Research Foundation (http://www.prf.org/otc/) as the assignee.

Yesterday (April 19th 2010), I started part 1 of the user testing for my research project regarding the multi-touch device and software application I've been developing. The software application is based on the Six Sigma Affinity Diagram (aka KJ Analysis) exercise. This is typically used to sort through qualitative data (such as ideas from a brainstorming session) to find unforeseen relationships between the the data items / ideas by grouping them into categories.

I was granted permission by Dr. Mary Sadowski, Associate Dean for the College of Technology at Purdue University, to ask faculty members from a committee within the College of Technology to use my multi-touch device and software application to aide them in sorting through some ideas/outcomes regarding a project they are working on.


Image 1 - Doing a run-through prior to user testing during set-up.


Image 2 - User Testing.

During the user testing the software application unfortunately crashed twice. Each time the users had to restart the Affinity Diagram from the beginning. After the second crash, I went into the code and quickly made some alterations. The problem was occurring with an open-source class written in ActionScript 3.0 which processes data received from Community Core Vision (the open-source software which interprets the images received from the camera component of the device's hardware). I added some "try / catch" statements. This seemed to resolve the issue since the software application didn't crash after the modifications.

Overall, the user testing proved to be successful. Excellent data was collected both regarding the interaction of user-to-user and user-to-interface. Also, notes for improvements to the software application.

I originally hoped to use the ColorPicker component in Flash but I was having too many problems trying to modify the code behind the component to get it to recognize the Touch Events from CCV and TUIO. For the sake of time and meeting my approaching deadline, I decided to ditch that idea and just make a set number of colors that will automatically be selected as groups/categories are created.

Below is an example of what I've come up with so far.



Still more to come, next is the ability to vote on the priority of categories/groups.

Received a recommendation from @BobRullo at AdobeMAX for applying the brainstorming multi-touch device app I want to create in a similar process to Six Sigma called Scrum. This is a development process usually used for software development. Wikipedia article on Scrum: http://en.wikipedia.org/wiki/Scrum_(development) Thanks Bob!

Some progress was made with the multi-touch device I've been building. Using the CCV software and some demo Flash apps provided on the CCV web site, here are some pictures and a video:

I've found, in addition to the NUI Group Forum (www.nuigroup.com), the Touch Factors Blog to be a very helpful resource: http://www.touchfactors.com/?p=103

I think there is a possible issue with the silicone layer of my multi-touch device. I had tried to do the method described by Tickerman on the NUI Group Forum (http://nuigroup.com/forums/viewthread/2197/P0/; YouTube Video: http://www.youtube.com/watch?v=IEgoVBfF68Y&feature=player_embedded). Tickerman calls it the "siliconed vellum" method.

I was unable to get the silicone completely smooth and it seems that the infrared camera I have is having difficulty distinguishing the blobs created by fingers in contact with the mt surface and the impurities/imperfections of the silicone.


Screen capture of CCV without contact to the MT surface


Screen capture of CCV with five fingers in contact on the MT surface

Grant me the serenity to accept the things I cannot change,
the courage to change the things I cannot accept,
and the wisdom to hide the bodies of those people I had
to kill today because they pissed me off.

And also, help me to be careful of the toes I step on
today as they may be connected to the that I may have
to kiss tomorrow.

Help me to always give 100 at work...
12% on Monday
23% on Tuesday
40% on Wednesday
20% on Thursday
5% on Fridays

And help me to remember...
When I'm having a really bad day,
and it seems that people are trying to piss me off,
that it takes 42 muscles to frown and
only 4 to extend my middle finger and tell them to bite me!

Amen

Background and Interests
For as long as I can remember, I have had an interest in art. Throughout elementary and middle school I attended several extracurricular art courses. During middle school I developed a curiosity for computers. This appeal led me to explore web design, which for me allowed me to combine my passion for art with my emerging passion technology. I enjoyed the creativity side of generating an interface design in addition to the problem solving side of debugging code. In 1999, I realized that people were paid to make web sites (and the pay was better than cutting grass for neighbors). That year I started my company, Cyber View (www.cyberviewsites.com), and quickly generated a small clientele list. I was able to turn my new found hobby into an aspiring entrepreneurial business. The following year in August of 2000, it was incorporated as an S-Corporation. Since that time I have transitioned from static web sites into dynamic, database-driven web development. Around 2001, I started experimenting with interactive multimedia. At the time I was utilizing Adobe Flash 5 to make small interactive applications for web pages. Today I describe my business as a web and interactive multimedia company. We are proud to state that we serve customers in three countries. In addition to providing custom web design, graphic design, web development, and multimedia development, the company also boasts two software-as-a-service systems which are offered to customers on a licensing basis. These two systems are: a content management system geared specifically for small businesses owners to easily maintain the content of their web sites (www.cyberstudiocms.com); and a residential life management system to centralize record keeping and tracking for the residential life staff of a college or university residence hall (www.myraportal.com).

During my undergraduate studies at Purdue University I studied Computer Graphics Technology, which provided me with the concepts and skills to know how to utilize graphic design tools more creatively as well as how to program more efficiently. Also within this time period, I developed a stronger fascination with interactive multimedia. As technologies evolved, I found an interest in rich internet applications, in other words, a web application that can be delivered in a variety of formats to send and receive data. I completed my Bachelor of Science from Purdue University in 2008 from Computer Graphics Technology.

Focus
For my graduate studies, I have decided to focus on research regarding the interaction between a human and an interactive multimedia application on a multi-touch device. More specifically, I'm interested in developing multi-user applications for multi-touch devices using the Adobe Flash Platform as the delivery method. Several individuals in both industry and in academia have developed multi-touch devices and others have begun designing applications for these devices. However, the pursuit will be to determine through the literature review what testing has been conducted about the interaction of multiple users with these devices.

There is a definite potential for value-added benefit to companies in the future through the use of multi-touch devices, especially for group collaboration. This potential has led my research focus into this general topic area. One initial challenge I have already encountered during my first year of graduate school is the acquisition of a multi-touch device. Professionally manufactured devices are very costly. In the current economic standing our country is in, these high costs are not within reach of most businesses. Other the other hand, many have sought the task of building their own devices through reading blog posts of other individuals who have attempted the same feat. Another challenge to those choosing to build a multi-touch device is the process of working through the quirks and system-bugs of constructing a multi-touch device from raw parts. While I would have preferred to not dive into the hardware side of multi-touch devices and focus solely of the interaction of the interface and the user, it has become evident that I will need to address both topics.

I am currently seeking a Master Degree. I have decided to select the route of a Directed Project. The decision came, in part, based upon the descriptions comparing a Master Thesis and a Directed Project in the Graduate Student Handbook. After running a company for the past ten years, I enjoy the combination of the business/industry focus with emerging technologies. Furthermore, I embrace challenges of finding methods to apply technological solutions towards solving business/industry issues. In the spectrum of research and development for a business setting, my preference lies towards the development side due to my hands-on personality. I thoroughly acknowledge and understand the value in the research component; although it is not one of my stronger skill sets I still strive to improve.

At the Purdue University ;IT Summit ;2008, our research poster received third place in the Graduate Student category. The poster was ;entitled cgCentral: ;An AIR Application for Managing Course Information and Simulation Data (however, it was listed in the program as Using Adobe AIR as a Tool in Universities)

In November 2008, Professor Kellen Maicher and I will be presenting at the 2008 Adobe MAX Conference in San Fransisco, California. The presentation entitled cgCentral: ;An AIR Application for Managing Course Information and Simulation Data will share the story of how cgCentral evolved, including discussion about development and usability testing of the AIR application.

On Monday, October 20th 2008, ;Professor Kellen Maicher and I are presenting a research poster entitled cgCentral: ;An AIR Application for Managing Course Information and Simulation Data at the 2008 IT Summit at Purdue University. Information presented includes details and testing results ;about the lab testing phase of the cgCentral AIR App from the Spring 2008. Also, details about the field testing of cgCentral currently underway this fall.

cgCentral: An AIR Application for ;Managing Course Information and Simulation Data

PROBLEM STATEMENT
Dissemination from CGT 411/450 to the Computer Graphics Technology student population of information regarding COGENT stock prices, individual COGENT accounts and press releases.

RESEARCH QUESTION
Are rich internet applications such as CGCentral more usable than traditional browser-based tools for disseminating information from a senior-level course to an entire department student body?

ABSTRACT
The senior design capstone course in the Department of Computer Graphics Technology at Purdue University provides an opportunity to engage students using COGENT, a fictitious currency used to evaluate performance. A simulation of a stock market for the senior design groups allows for undercl men to receive, exchange and invest COGENT dollars during their time as a Computer Graphics Technology student. Through this interaction undergraduates can accumulate COGENT in an effort to enhance their final grade for the course. Building on previous research in this area, this project addresses which medium is perceived by students to be the most effective to disseminate information: 1) desktop-based rich Internet applications or 2) dynamic web sites and RSS feeds.

Facebook, MySpace, and Blackboard – Analyzing the interfaces as educational tools?

Two out of the three interfaces we have been asked to review are branded as social networking sites. The third is branded as an educational course management system with limited social networking capabilities. ; To examine each of these systems as a potential educational tool to aide communication in a college course, an analysis of the interface in which students and professors will interact is in order.

When you first arrive at the myspace.com web site, animated banner ads immediately bombard you. ; Throughout the MySpace web site the design layout is scattered with ads. On its web pages, not including profile pages of its members, there is a lack of a consistent layout from page to page. For example, from the homepage to the MySpace Music page. Some users may experience challenges of determining how to create an account for membership to MySpace.

Drastically contrasted from the MySpace homepage, the facebook.com homepage is brief and to the point. No flashing banner ads are popping up in your face. The site informs visitors of its purpose, potential uses, login details, and information to join. For a user new to Facebook, they would be able to create an account with great ease from the homepage. Once logged into the site, users will notice that Facebook is currently transitioning their design layout. The old version incorporated a consistent layout throughout the site, however new version Facebook has debuted lacks a consistent page layout. Facebook’s advertising model has two different means of advertising: banner ads and fliers. The animated banner ads you will find throughout Facebook are not overbearing to the user and there are typically less than two per page.

Out of the three, creating an account for Blackboard is the easiest – Purdue University creates it for you. The Career Account username and p word for all Purdue students is their access to Blackboard. Initially logging into Blackboard, users might experience challenges if their computer doesn’t have the latest Java plug-in for their internet browser. Blackboard will prompt them to download any updates they need to utilize the system. Upon accessing the system, users will not encounter any advertisements since the system was purchased by Purdue. The interface is very structured with a similar page layout throughout the system. Unlike Facebook and MySpace, using the internet browser’s back button doesn’t always take the user back to the previous page.

In Designing Interfaces author Jennifer Tidwell (2006) discusses the openness of an interface can have a great effect on how your users are able to accomplish the intended goal(s) for using the system. For this analysis the goal for users will be to exchange information and communications regarding a college course. With MySpace the system is very open. This gives users the complete ability to change the layout, color scheme, and graphics of the interface. For a novice user, they can feel paralyzed, not knowing what to do next, unprepared (Tidwell, 2006). The ability to customize the aesthesis of a MySpace profile page will not improve the communication features of the site. ;I feel Tidwell would agree that Blackboard would be considered an extremely closed system. Tidwell describes closed systems as users feel trapped and unsatisfied, which seems to describe Blackboard well. However, the main purpose of Blackboard is for education. It provides features that the other two systems lack, such as file sharing, grade reporting and progress reports. Facebook is a middle ground between the openness of MySpace and the closedness of Blackboard. While it doesn’t offer the grading features Blackboard has, it does have an easier means to navigate the site. Each of the three options provides similar capabilities, including discussion boards, instant chatting, who's online status indicators, and internal messaging.

I’ve completely ruled out MySpace due to the issues already discussed, in addition to the connotation the site has from the general public wouldn't lend itself well as an educational tool. Facebook obviously has a social connotation to it due to it starting off initially introduced to college students; therefore, many students are very familiar with how it works. Students repetitively use the site throughout the day to check the latest news about their friends. Blackboard’s communication tools don’t seem to be utilized by students, due to the educational/institutional connotation of the system. As educational tools, I believe the best choices based upon the experience users will receive from the system interfaces will be a toss-up between Facebook and Blackboard.

Sources:
Tidwell, ; J. (2006). Users’ Motivation to Learn. Designing Interfaces, ; 8-19.

PROBLEM STATEMENT
The current information dissemination mechanisms used in CGT 411/450 are perceived inefficient because of the lack of a centralized source of knowledge regarding the tracking of tasks related to the CGT 411/450 course simulation, in turn detracting from the true goal of the course, the scholarly research endeavor.

RESEARCH QUESTION
Are dedicated standalone rich internet applications such as CGCentral more usable than traditional browser-based tools for disseminating CGT 411/450 information?

ABSTRACT
CGT 411 is a senior capstone course in which students simulate the complexities of real-world corporations and business environments related to applied computer graphics . Part of this simulation includes the student and faculty functions of generating, tracking, and maintaining large amounts of mock data including stock information, press releases, corporate and individual finances. Typically, these functions have been achieved through a variety of mechanisms including web sites, RSS feeds, forums, and conferencing applications. Due to size and complexity of information required for the function of this simulation, the use of the previously mentioned applications have proven to be inefficient and inadequate. The authors propose the creation of a Rich Internet Application using the Adobe AIR technology that will consolidate the functions of the multiple simulation mechanisms into a single, unified application. The RIA will be subjected to usability and essment testing to ensure that an effective application is built that achieves the goals of the project.

Read the article on the Adobe Developer Connection Web site

When my friends and I considered how to create a capstone project for our senior design course, we decided to use the content of the course itself and proposed a project that would unify the many distribution means for that content in a single rich Internet application.

Computer Graphics Technology (CGT) 411/450 is a senior capstone course in which students simulate the complexities of real-world corporations and business environments related to applied computer graphics. Part of this simulation includes the student and faculty functions of generating, tracking, and maintaining large amounts of mock data, including stock information, press releases, and corporate and individual finances. Typically, these functions have been achieved through multiple mechanisms including websites, RSS feeds, forums, and conferencing applications.

We found it inefficient to disseminate information for the course without a centralized source of knowledge for tracking of tasks and felt that not having such a source detracted from the true goal of the course, the scholarly research endeavor. Our team, composed of Alex Porter, Adam Christ, Brian McCreight, and myself, proposed the creation of a rich Internet application using Adobe AIR technology, with the goal to consolidate the functions of the multiple simulation mechanisms into a single, unified application.

Currently the CGT 411/450 course uses multiple channels to provide information to students regarding the three primary pillars of the course: the scholarly research endeavor, the product development, and the corporation entities. Our team investigated multiple rich Internet application technologies that could be used to create an application to solve the perceived problem. After developing an interest in Adobe AIR, we researched an ortment of competing technology platforms to determine if Adobe AIR was the most effective choice. The technologies platforms we analyzed included Microsoft Silverlight, Google Gears, Java FX, and Mozilla Prism.

The rich Internet application we developed was a culmination of web technologies. These included ActionScript 3.0 using Adobe Flash CS3, PHP, MySQL, and Adobe AIR. We named the application CGCentral to emphasize its goal to serve as a central point of knowledge for students enrolled in the course. We employed the vector graphics capabilities of Adobe Flash CS3 for the end-user interface and programmed using ActionScript 3.0. The AIR platform allowed us to target and package the application for an end-user's desktop. We divided CGCentral into four sections to provide the end user with specific areas from which to receive information: Account Overview, Lecture Schedule, Stocks, and News.

CGCentral is a work in progress. We will continue essment and usability testing to ensure that an effective application is completed that achieves the goals of the project.

About the Author
Frank Garofalo is an interactive multimedia developer with professional experience including operating a web design firm, Cyber View (http://www.cyberviewsites.com/), since 1999; acting as a project manager in marketing departments; and working for the Walt Disney Company and Bank of America. As a graduate of Purdue University's Computer Graphics Technology program, he has focused on web design, multimedia applications, and digital marketing. His research interests are in the areas of usability, accessibility, and user-centered design. He is currently pursuing his master's degree in Applied Computer Graphics at Purdue University.