Organization

University of Central Florida

Project Description

I am a doctoral candidate at the University of Central Florida working on my dissertation – Performance Support and Usability: An Experimental Study of Performance Support Interfaces. I would like to administer a survey via the QuestionPro Software in four classroom labs to undergraduate students. So far, I have enjoyed using the software – it has been very usable. I will definitely site the software in the dissertation.

Project Abstract

Introduction

In the last decade, companies have begun to shift from training employees in the classroom and with stand alone computer based training (CBT), to offering employees training or support on the job (Brown, 1996). The concept of providing people pertinent information at the time of need is performance support. When this support is provided on the job in an electronic environment, it is referred to as electronic performance support (EPS).
Gery (1991) first coined the term Electronic Performance Support System (EPSS) in 1989 and has since been defined by Raybould (1995) as an electronic infrastructure that captures, stores, and distributes knowledge throughout an organization via tools to enable it to learn faster. While there are many terms that have the same meaning or are closely related to EPSS, for the remainder of this document, EPSS and other software applications designed to capture, store, and distribute information and performance support on a just-in-time basis will be referred to as EPS. In addition, the terms “employee(s),” “participant(s),” “user(s),” and “learner(s),” will be interchangeable throughout the document.
Corporations have experienced problems with conventional training, which include inefficacy, high costs, and resources. Training programs, while potentially effective, have proven to be somewhat inefficient because only a certain portion of what is taught in the classroom is actually remembered by the participant, with an even greater loss of information when a delay occurs between instruction and actual application on the job (Puterbaugh, 1990). It is estimated that more than 80% of critical job related learning happens on the job (Lawton, 1999). Corporations view training programs in terms of their impact on results, which only result in short-term learning (1999). The absence of meaningful long-term learning is serious because it transforms itself from an instructional problem into a business problem, which could be detrimental to the corporation. Training costs are also becoming prohibitive due to increasing costs for instructors, travel to and from training events, and lost employee work time for formal classroom sessions and complicated course structures (Horn, 1989). Furthermore, it is difficult to design training programs that emulate job situations because many tasks overlap and usually are performed in a more complex environment.
Due to the high demand for information and ubiquitous delivery methods, the same problems that have begun to plague corporations are now entering the K12 school systems (Branson & Hirumi, 1994). Traditional methods for teacher training and professional development have become inefficient and ineffective (Hirumi, 2003). The negative effects are prevalent in teacher workshops, generally consisting of information dumps resulting in learning devoid of richness. Even Web and computer-based methods are not useful because teachers are not able to utilitize the collective knowledge from the organization (2003).
The aforementioned educational problems have been documented at a virtual K12 school in Ohio (Hirumi, 2003). To assuage these problems, an online training and professional development system is being developed for educators as part of the Educational Classroom of Tomorrow (ECOT) (2003). Included in this system will be an EPS providing educators with the tools, training, and information necessary to complete critical job functions in a just-in-time manner. ECOT will serve as a test bed for the EPS proposed in this study.
With new technologies today, organizations are moving towards EPS as a means to improve the performance of users. There are many reasons why organizations are shifting their views from traditional training methods to performance support, which include decreasing time to competency, increasing performer competency, decentralizing decision making, and increasing customer satisfaction (Brown, 1996). While it may take new employees one to two years to become competent at their job, an EPS can provide a neophyte with expert advice allowing him or her to perform above his or her current level of knowledge (1996). An EPS also automates redesigned work processes to streamline the task allowing a user to be more efficient (1996). With organizations empowering their users to make their own decisions, a well-designed EPS supports this goal by providing the resources for better decision-making (1996). Providing the employee with benefits ultimately yields a better product thereby increasing customer satisfaction. Does this mean that training in the classroom should be discarded? Hardly, but because of these compelling reasons, the EPS approach is being used more often to provide on the job tools during critical times of need.
An EPS consists of performance support tools that may or may not include wizards, coaches, advisors, and intelligent tutors. While organizing these tools may appear easy, making them accessible is not. Designing the system and tools to be accessible is critical because if they are not designed appropriately, the users may not be able to readily access or navigate them, thus causing cognitive load. Usable software products must be designed from a user-centered design approach.
Designing products from a user-centered design (UCD) approach increases their overall usability or simply, makes them easier to use. Products, in this case, refer to the user interface, which contain the content, human factors, design guidelines, and interaction styles (Hix & Hartson, 1993). Equally important is the process by which the product is developed. The process involves the life cycle, methods, techniques, and tools used in designing an interface (1993). Using an integrated process, three important activities ensure a disciplined, integrated, customer based product (Vredenburg, Isensee, & Righi, 2002). The first activity is to understand users in their environment, which also includes understanding tasks users perform frequently and the tasks they may perform in the future. The second activity consists of designing and evaluating iterative prototypes with users. Finally, the last activity is assessing competitor designs. Implementing an integrated process ensures a higher degree of usability because it focuses on users, solutions, teamwork, external designs, user experience, competition, user measurement, and the future customer; as opposed to traditional approaches that focus on technology, components, limited cooperation, internal architecture, limited competition, limited user measurement, and current customers (2002). Designing an EPS from a UCD approach is critical in providing the foundation for a highly usable interface.
The user interface is important in an electronic environment when attempting to design accessible software programs. If designed appropriately, the user interface engages users by inviting them to browse or work through software interface programs. If designed inappropriately, the user interface can lead to user confusion, frustration, and cognitive load.

Statement of the Problem

The problem is there is relatively little research to guide the design of performance support user interfaces. User interface design remains the least researched component of knowledge-based systems (McGraw, 1992). Lawton (1999) states there are no recognized or universal design templates for developing EPS. This study will evaluate the usability of two types of performance support interfaces, which have been designed using informational and experiential approaches. The experiment will determine if there is a relationship between usability and the informational and experiential approaches.

Hypothesis


The EPS interface developed using the informational approach will possess a higher degree of usability than the interface developed from the experiential approach based on measures of efficiency, intuitiveness, errors, and satisfaction.


Significance of the Study

This study is significant for teachers as well as designers tasked with creating EPS. The ECOT employs approximately 150 teachers who require information, training, and performance support almost on a daily basis to complete critical job functions. There are also plans to expand ECOT across the United States over the next five years.
The results of this study may benefit teachers at two levels. First, there is a general benefit of providing teachers access to the EPS and its resources. The second benefit is the optimization of the interface with an increased degree of usability.
The results of this study will also provide a solid, research based, and usable foundation for designers as they develop an EPS interface. The data from this study may determine the guidelines and standards for EPS interface design and usability. This in turn should benefit all teachers and professionals using this system, allowing them to be more efficient and effective users.

Instrument

An online usability questionnaire developed using QuestionPro.com software was constructed using the Purdue Usability Testing Questionnaire (PUTQ) questions. A Task List (see Appendix B) was instrumental in determining what navigation processes the participants engage in while interacting with the interface. The PUTQ will be used as the testing instrument. It is an accepted usability testing method.

Surveys released for this project:
Questions
EPS PUTQ 32
QuestionPro is FREE for Academic Research

This Project Sponsored by: QuestionPro - Web Survey Software
See Research Sponsorship for more information.

×
×
Complex insights made simple. Get Started for FREE