Skip to main content

Full text of "DTIC ADA333501: Advances in Multimedia and Simulation. Human-Machine-Interface Implications. Proceedings of the Europe Chapter of the Human Factors and Ergonomics Society Annual Conference"

See other formats

Proceedings of the Europe Chapter of the 

Human Factors and Ergonomics Society 
Annual Conference in Bochum, 
November 1997 

Klaus-Peter Holzhausen (Ed.) 


Form Approved 0MB No. 0704-0188 

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, 
gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this 
collection of information, including suggestions for reducing this burden to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson 
Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Papenvork Reduction Project (0704-0188), Washington, DC 20503._ 


19 November 1997 

Conference Proceedings 



Advances in Multimedia and Simulation. Human-Machine-Interface Implications. Proceedings 
of the Europe Chapter of the Human Factors and Ergonomics Society Annual Conference 



Conference Committee 


Lennershof Strasse 140 
44801 Bochum 



PSC 802 BOX 14 
FPO 09499-0200 


CSP 97-1070 



Approved for public release; distribution is unlimited. 

13. ABSTRACT (Maximum 200 words) 

The Final Proceedings for Advances in Multimedia and Simulation, 6 November 1997 - 7 November 1997 

Multimedia in Information Systems, Multimedia applications in Higher Education, Multimedia applications in Architecture, Videoconferencing 
Including Multimedia, Teleoperation in Virtual Reality, Multimedia and Virtual Environment, Modeling and Simulation of Ground Vehicles, 
Simulation in Traffic Environment, Flight and Air Traffic Control Simulation, Undenwater Simulation, Factory Simulation, Simulation of Medical 
Systems, Power Plant Simulation and Training, Military and Government Simulation 



Flight Control, Human Factors, Instrumentation 






NSN 7540-01-280-5500 



Standard Form 298 (Rev. 2-89) 
Prescribed by ANSI Std. 239-18 

Advances in Multimedia and Simulation. 
Human-Machine-Interface Implications 

Proceedings of the Human Factors and Ergonomics Society 

Europe Chapter 

Annual Conference in Bochum 

November 1997 

Klaus-Peter Holzhausen (Ed.) 

19980102 015 

Copyright 1997, The authors and the Fachhochschule Bochum, University of Applied 
Sciences, Bochum, Germany 

All rights reserved. No part of this publication may be reproduced, stored in a retrieval 
system or transmitted in any form or by any means, electronic, mechanical, 
photocopying, recording or otherwise, without prior permission in writing of the 
copyright holder. 

Advances in Multimedia and Simulation. Human-Machine-Interface Implications - 
Klaus-Peter Holzhausen (Ed.) - Fachhochschule Bochum, University of Applied 

Proceedings of the Human Factors and Ergonomics Society Europe Chapter 1997 Annual 
Conference in Bochum, November 1997. 

Keywords: ergonomics, human factors, multimedia, simulation, management of 



Plenary Session 


Prof, Dr.-Ing. Klaus-Peter Holzhausen, Fachhochschule Bochum 
Message to the Europe Chapter 

Dr. Harold van Cott, President, Human Factors and Ergonomics Society 

Multimedia and Management of Enterprises 

Dr. h. c. Klaus Steilmann, Member Club of Rome, Steilmann Gruppe, 




AAl-Air Traffic Control and Aviation 1 

Air Traffic Control Simulation and the Human Operator 36 

Dipl.-lng. Ralf Beyer, DLR e.V., Braunschweig 

Horses for Courses - Simulation in Air Traffic Control 46 

Dr. Hugh David, Eurocontrol Experimental Centre, Bretigny-sur-Orge 

PC - Based Training of Air Traffic Controllers 55 

Ir. Bas Kuijpers, NLR, Amsterdam 

AA2- Air Traffic Control and Aviation 2 

Deep Design - Beyond the Interface 65 

Dr. Hugh David, Eurocontrol Experimental Centre, Bretigny-sur-Orge 

Advanced Manoeuvre Flight Training on PC‘s and Transfer to the Real Aircraft 78 
Jan J. Roessingh, NLR, Amsterdam 
Staszek F. Chlopowski 

A Systematic Approach for Applying Multimedia Techniques to Aviation 93 

Margaret T. Shaffer, Paradigm International, Potomac, Maryland 
Dr. Rodney Baldwin, BIS, Heffingen 

Controller Working Position for Future Automatic Dependent Surveillance 103 

Dipl. Psych. Beate Ulbrich, TU Berlin 
Dipl. -Ing. Andre Wattler, TU Berlin 
cand.Inform. Andre Nordwig, TU Berlin 
cand.Inform. Frank Posadny, TU Berlin 



The Diadem Software Development Methodology extended to Multimedia Interfaces 117 
Dr.~ Ing, Bemd-Burkhard Borys, Universitdt-Gesamthochschule Kassel 
Dipl- Ing. Markus Tiemann, TUBerlin 

What is Multimedia? Proposal of a Taxonomy for Human-Machine Communication 126 
Prof. Dr.-Ing. Georg Geiser, Katholische Universitdt Eichstatt 

Comparing Multimedia Concepts by Using Socio-Oriented Modelling Methods 137 
Prof. Dr.-Ing. Thomas Herrmann, Universitdt Dortmund 
Dipl-Ing. Kai-Uwe Loser Universitdt Dortmund 
Dipl-Inform. Klaus Moysich, Universitdt Dortmund 
Dipl-Inform. Thomas Walter, Universitdt Dortmund 

DS- Defense System 

Virtual Environment for the Simulation of a Tactical Situation Display 147 

Dipl-Ing. Morten Grandt, FGAN/FFM-EFS, Wachtberg-Werthhoven 
Dipl-Ing. Thomas Alexander, FGAN/FFM-EFS, Wachtberg-Werthhoven 
Prof. Dr.-Ing. Klaus- Peter Gartner, FGAN/FFM-EFS, 


Multimedia Approaches in Management Information Systems 159 

Dipl - Ing. Jurgen Kaster, FGAN/FFM, Wachtberg-Werthhoven 

Simulation Supported Analysis and Development of Tactical 164 

and Lower Level Ground Battle Units 

Capt. M. Sc. Ame Worm, Swedish Armed Forces, Linkoping 

ATI- Automotive and Traffic Applications 1 

Human Factors, an Emergency Situation, and the Automated Highway System 175 
Dr. Karel J. Brookhuis, Centre for Environmental and 
Traffic Psychology, Groningen 
Dr. Dick De Waard, Centre for Environmental and 
Traffic Psychology, Groningen 

A Model of Individually Acting Drivers with Cognitive Capabilities 184 

for Microscopic Traffic Simulation 

Marita Irmscher, Technische Universitdt Berlin 

Thomas JUrgensohn, Technische Universitdt Berlin 

Prof. Dr.-Ing. Hans-Peter Willumeit, Technische Universitdt Berlin 

Online Traffic Simulation Based on Cellular Automata 196 

J. Esser, Universitdt Duisburg 
Michael Schreckenberg, Universitdt Duisburg 


AT2- Automotive and Traffic Applications 2 

Intelligent Speed Adaptor (ISA) 

Dr. Karel J. Brookhuis, Centre for Environmental and 
Traffic Psychology, Groningen 
Dr. DickDe Waard, Centre for Environmental and 
Traffic Psychology, Groningen 

Simulation and Control of Traffic Flow on Large Motorway Networks 
Prof Dr.-Ing. Soenke Schoof, Fachhochschule Hannover 

False Yield and False Go Decisions at Signalized Left-Turn Intersections: 

A Driving Simulator Study 

Andrea Szymkowiak, University of Massachusetts, Amherst, MA 
Donald L Fisher, University of Massachusetts, Amherst, MA 
Karen A. Connemey, University of Massachusetts, Amherst, MA 

Validation of an Economic and Fast Method to Evaluate Situation Specific 236 

Parameters of Traffic Safety 

Prof Dr. Alf Zimmer, Universitat Regensburg 
Katharina Dahmen-Zimmer, Universitat Regensburg 
Kilian Ehrl, Universitat Regensburg 



AT3- Automotive and Traffic Applications 3 

New Multimedia Components Inside a Car Cockpit 246 

Dipl.~Ing. Jens Ohler, Leopold Kostal GmbH & Co. KG, Ludenscheid 
Dipl.~Ing. Roland Drees, Leopold Kostal GmbH & Co. KG, Ludenscheid 
Prof Dr.- Ing. Klaus-Peter Holzhausen, Fachhochschule Bochum 

Changing Attitudes of Speed-Limit Offenders Using a Multimedia Programme 254 
Frank Steyvers, University of Groningen, Haren 
Anneke J. Menting, Universitat Groningen, Haren 
Dr. Karel J. Brookhuis, Centre for Environmental and 
Traffic Psychology, Groningen 

Driving Simulation Systems as Fast Tools to Evaluate Different Types 262 

of Head-Up Displays in a Vehicle 

Dipl.- Ing. Andreas Penka, TUMUnchen, Garching 
Rolf Gengenbach, TU MUnchen, Garching 
Heiner Bubb, TU MUnchen, Garching 

Analysis of Hand Position in Unstmctured Environments 269 

Lutz Geisen, Deutsche Sporthochschule, Koln 
Prof 0. Bock, Deutsche Sporthochschule, Koln 


IRl- Industrial Automation and Robotics 1 

An Experimental Multimedia Process Control Room 276 

Dr.- Ing. Bemd-Burkhard Borys, Universitdt-Gesamthochschule Kassel 
Prof. Dr.-Ing. Gunnar Johannsen, Universitdt-Gesamthochschule Kassel 

A Multimedia Decision Support System for Workgroups in Flexible Manufacturing 290 
Dietmar Gude, Universitdt Dortmund 
Andreas Bauerle, Universitdt Dortmund 
Andreas Stiegler, Universitdt Dortmund 
Prof. Dr.-Ing. W. Laurig, Universitdt Dortmund 

Incorporating Ergonomic Conciderations into Models of Manufacturing Systems 300 
Henry Herper, Otto-von-Guericke-Universitdt, Magdeburg 
Hansjurgen Gebhardt, Universitdt Wuppertal 

IR2- Industrial Automation and Robotics 2 

Concept of an Object-Oriented Multimedia System for Teleservice Applications 308 
Werner Amshojf, Bergische Uni Gesamthochschule Wuppertal 
Andre Brinkmann, Bergische Uni Gesamthochschule Wuppertal 
Dr.- Ing. ReinhardMdller, Bergische Uni Gesamthochschule Wuppertal 
OlafSchaar, Bergische Uni Gesamthochschule Wuppertal 
Achim Sixtus, Bergische Uni Gesamthochschule Wuppertal 

Architecture of the "Robotic Tele Lab" 317 

Prof. Dr. Armin E. Cremers, Universitdt Bonn 
Wolfram Burgard, Universitdt Bonn 
Dirk Schulz, Universitdt Bonn 

A 3D Interface for a Mobile Robot Using Standard Internet Tools 327 

Eric Colon, Royal Military Academy, Brussels 
Prof. Dr. Yvan Baudoin, Royal Military Academy, Brussels 
Bernard Heggermont, Royal Military Academy, Brussels 

Performance Shaping Factors in Teleoperation using a 6-Axis Active 338 

Hand Controller 

Dipl.-Ing. Gordon Gillet, TU Munchen, Garching 
Herbert Rausch, TU MUnchen, Garching 

VR- Virtual Reality 

Virtual Reality Usability Evaluation Techniques: RECMUVI and TILE VIZ 347 

Pedro Z Caldeira, Instituto Superior de Psicologia Aplicada, Lisbon 
Nuno Otero 


A Multimedia Archaeological Museum ‘ 

Dr.-Ing. Claus Diefienbacher, Universitat Dortmund 
DipL-Ing. Thorsten Henkely Universitat Dortmund 
Prof.Dr.-Ing. Ernst Rank, Universitat Dortmund 

Subjective Realism in a Simulated Squash Game 365 

Dr. Heiko Hecht, Universitat Bielefeld 

Virtual Reality as an Aid to Document and Inventorize Historical Buildings 371 

Prof Dr. -Ing. Martin Trautwein, Fachkochschule Lippe, Detmold 

ICl- Information Technology and Communication 1 

Designing for Telecommunication on the Internet 380 

Marita Franzke, US West Advanced Technologies, Boulder, Colorado 
Anne McClard, US West Advanced Technologies, Boulder,Colorado 
Carrie Rudman, US West Advanced Technologies, Boulder,Colorado 
Pat Somers, US West Advanced Technologies, Boulder, Colorado 

Digital Multimedia Broadcasting 398 

Transmission of Video and High Data Rate Signals into Moving Vehicles 
Prof Dr.-Ing. Gert Siegle, Robert Bosch GmbH, Hildesheim 

IC2- Information Technology and Communication 2 

The Interface - An Invitation To Conununication 403 

Monika Fleischmann, GMD, Sankt Augustin 

The Development of Future Communication 406 

Dr.-Ing. Michael Laskowski, communications GmbH, 


Implementational and Perceptual Aspects of an Immersive Auditory/ Tactile 413 

Virtual Envirmonment 

Dipl-Ing. Jorg Sahrhage, Ruhr-Universitdt-Bochum, Bochum 
Prof Dr.-Ing. Dr. h. c. techn. Jens Blauert, Ruhr-Universitdt-Bochum 

TL- Teaching and Learning 

An Integrated Multimedia Concept for Distance Teaching of Engineering 426 

Prof Dr.Dr. Wolfgang Halang, Fernuniversitdt Hagen 
Martin Witte, Fernuniversitdt Hagen 

Teaching Syntheses of Digital Systems. A Challenge for Multimedia Applications 436 
Peter Conradi, Uni-GH Giefien, Siegen 
Ulrich Heinkel, Universitat Erlangen 
Michael Wahl, Uni-GH Giefien, Siegen 



Interface Design of a Multimedia Training Course for Electrical Engineers 
Including Evaluation Design 

Prof. Dr. Birgit Scherff, Femuniversitdt Hagen 

Dipl-Ing. Dipl-Wirtschaftsing. Dirk Thifien, Femuniversitdt Hagen 

MS- Medical Systems 

Fixation Disparity under Vergence Load: Clinical Optometric Procedures 456 

and Ergonomic Applications 

Dr.-Ing. Wolfgang Jaschinskiy Universitdt Dortmund 

A New Strategy for Incorporating Health and Safety Relevant Factors into Planning 463 
Ina Ehrhardt, Otto-von-Guericke-Universitdt, Magdeburg 
Hansjiirgen Gebhardt, Universitdt Wuppertal 
P. Lorenz, Otto-vonGuericke-Universitdt, Magdeburg 
B.H. Muller, Universitdt Wuppertal 

Simulation of Different Peripheral Blood Flow Patterns for Educational Purposes 474 
Dr.- Ing. Ulrich Schaarschmidt, Universitdt der Bundeswehr, Hamburg 

Perception-Action Compatibility and Eye-Hand Dominance in Using 478 

Visually-Displayed Information 

Dr.rer.nat. Walter H. Ehrenstein, Universitdt Dortmund 
Dr.rer.nat. Birgit E. Arnold-Schulz-Gahmen, Universitdt Dortmund 



Prof-Dr. -Ing. Klaus-Peter Holzhausen 
Fachhochschule Bochum, University of Applied Sciences 

The focus of the Human Factors and Ergonomics Society Europe Chapter 1997 Annual 
Conference in Bochum is on Advances in Multimedia and Simulation from the viewpoint 
of Human-Machine-Interface Implications. This very important and modem topic 
attracted more than 50 scientific papers. The conference enjoys contributions from 
important universities, research establishment as well as associations and firms. 

I am grateful and proud that a major contribution was given by the European Office of 
Advanced Research and Development (EOARD) of the United States Air Force. This 
donation confirms the growing importance of our Chapter not only in Europe but also in 
the United States. 

Another major contribution is given by the ARAL AG. We are especially grateful for this 
one, because the ARAL AG is one of the most important companies in our city of 
Bochum, a major employer in the Ruhr Region, and a leading German enterprise in the 
field of energy. 

The host of this conference is the Gesellschaft der Forderer der Fachhochschule Bochum 
e.V. (Fachhochschule Bochum Alumni Association). The Alumni Association and my 
institute are responsible as local hosts for the organization and realization. The 
symposium would not have been possible without the support of our higher education 
institution, the Fachhochschule Bochum, University of Applied Sciences. The 
Fachhochschule Bochum is co-organizer of this scientific event. The President of our 
University, Rektor Prof. Dr. Martin Grote, is taking the opportunity to open the 
conference with a greeting address. Mrs. Gabriele Schafer, Mayor of the City of Bochum, 
is going to convey the greetings of the City. 

Dr. Harold van Cott, President of the Human Factors and Ergonomics Society in the 
United States, sent us a greeting address, which is in our proceedings. 

I would like to thank Dr, h. c. Klaus Steilmann, Member of the Club of Rome, President 
and C.E.O. of the Bochum based Steilmann Group, for his keynote speech on Multimedia 
and Management of Enterprises that he is going to give on the conference. 

My personal thanks are to all the contributors to this conference, the speakers, members 
of the organizing and the program committees, for their work and advice. I want to 
mention my colleagues of the Fachhochschule Bochum, who made this event happen, and 
I am very grateful to the many students and staff in my Laboratory for Artificial 
Intelligence, Computer Graphics, and System Software (kiss-lab). 


Message to the Europe Chapter 

Harold van Cott, Ph.D., President 
Human Factors and Ergonomics Society 

Professor Cavonius, Ladies and Gentlemen: Greetings to all of you here in Bochum at this 
meeting of the Europe Chapter of the Human Factors and Ergonomics Society. I had hoped to 
be here with you today, but an unavoidable, personal scheduling conflict prevented it. None- 
the-less, on behalf of the Society’s Executive Council, I would like to wish you well for a 
successful conference. 

In September, I attended the 1997 41"' Annual Meeting of the HFES in Albuquerque, a small, 
historic city in central New Mexico. It was a busy week of renewed friendships, symposia, 
papers, posters, receptions, banquets, and tours. Late September is a good time to be in 
Albuquerque. Seeing the awesome Sandia mountain range emerge from morning twilight into 
a pure blue sky invigorates the soul for the day ahead. The days were long but gratifying. 
Evening meals with green chili stew, ostrich and buffalo steak were a novelty to tastes formed 
in the Eastern U.S, 

In 1956, the HFES had its first meeting in Tulsa, Oklahoma. About 150 charter members, who 
had organized what was then called the Human Factors Society of America, were there. I was 
among them: green but enthusiastic. This year over 1.200 members from aU over the world 
attended the Albuquerque meeting. I was there and became the 41"* President of the Society. 
Today, I’m gray but still roaring to go. This is an exiting time to be a member of HFES. It will 
become more exiting as the society’s strategic plan continues to unfold. As members assume 
society leadership roles and take on program activities, the accomplishments of the long-range 
plan will be increasingly felt, the society will flourish, and our profession will grow. This 
strong European Chapter is but one proof of the growing global human factors and 
ergonomics community. We are delighted that distinguished colleagues such as you are 

Today I would like to tell you about the accomplishments in 1997 and the plans for 1998 of 
the Human Factors and Ergonomics Society, 

A lot was accomplished in 1996/7. Our Web Site was running smoothly and was used 
extensively. The directory of graduate programs in human factors education and the author’s 
guide to preparing manuscripts for publication were put on the Web Site. 

Informing others outside the field of the benefits of applying human factors data and 
principles has always been a problem. Last year, an illustrated report. Good Ergonomics is 
good Economics, was published as a removable insert in our magazine. Engineering in 
Design. These case histories drawn from around the world, on the costs and benefits of human 
factors and ergonomics interventions, are available in either paper or electronic format from 
the Society’s email address thfes@compiis e . Slides and text of these stories may also 
be obtained from the Central Office for use in giving presentations. The success stories will 
be expanded and updated as members augment the current file of stories and pictures with 
material form their own experiences. I invite you to do so. 

Feedback from focus groups and from a survey of all members showed that HFES should put 
more effort into reaching out to government and industry with education and information. In 


the past year, and with the combined strength derived from teaming with the Federation of 
Behavioral and Cognitive Sciences and the American Psychological Association, the HFES 
participated in programs to educate staff of the U.S. Congress on the value of the discipline 
and the importance of supporting funding for research and applications. Presentations and an 
exhibit were made at the Medical Device and Manufacturing Show. A policy and guidelines 
for conducting advocacy and outreach efforts was being developed. 

Given that more of our members are moving into small business and independent consulting, 
we decided to give workshops on forming and managing efforts in these areas for the first 
time in Albuquerque. Enrollment and participant enthusiasm was high, and snnilar workshops 
are contemplated for 1998. 

With funding in hand from two founding partner companies- Sun Microsystems, Inc. And the 
Eastman Kodak Company- the HFES Institute became a dream turned in reality. The Institute 
will provide a badly needed mechanism for procuring research data needed to fill critical data 
gaps in new industry standards, guidelines and best practices. 

Volunteers are the life-blood of HFES. More are needed if we are to enhance, strengthen and 
extend our efforts. But good training is required for volunteers to become truly effective. This 
year, again for the first time, a Leadership Day for local and student chapter presidents and 
technical group chairs was held. Over 80 enthusiastic participants shared experiences, 
identified common problems, and started to build plans for solutions to them. Many of those 
present had not met before and formed ties. It was a good start on the way to our goal of 
amplifying the capability of the Executive Council to plan and manage our many new 

Here are some examples of new efforts scheduled for 1997-98: 

Two new collections of papers from the Proceedings will be published on topics of high 
current professional and public interest. One will be on design for the aging population. The 
other will cover papers on musculosceletal disorders and manual handling. 

Two important surveys are planned for 1998. One will be a survey of member needs and 
preferences for the electronic delivery of mail. The other wiU be a detailed survey of salaries. 

If your arms ached or your wallet shrunk from canying or mailing home the two volumes of 
annual meeting proceedings, be of good cheer. Relief form transporting the growing bulk of 
the Proceedings is in sight Next year, they will be available on CD-ROM in a searchable 
format, as well as in traditional hard copy. In addition, a portion of cumulative index of our 
journal may be included on a trial basis on CD-ROM. 

Students are always broke. Next year, student members will be able to receive a 50% discount 
off full member dues. For the first time two years following graduation while they are getting 
professionally established. 

Next year the Executive Council will continue strategic planning, monitor current programs, 
plan new initiatives, and invite feedback and ideas from all members. To do all of these things 
well, the Council will meet quarterly. Additional outreach efforts, leadership forums, 
practitioner workshops and issues of membership recruitment and retention will be given high 
priority at these meetings. 

As for my own agenda as President, I plan to revive the dormant Technical Advisory Group to 
advise the Council on emerging opportunities for the application of our science and 


technology, and for the actions that HFES can take to be proactive in the pursuit of these 
opportunities. After learning about the struggles of our chapters simply to survive, to hold 
meetings and to communicate with members, I plan to explore ways to help chapters and the 
technical groups to establish listservers to enhance communication among tiieir members and 
with other chapters and HFES. I also plan to visit local and student chapters. I will continue to 
support HFES’ standards activities and the new HFES Institute, and I will work with the 
Council of Technical Groups on a plan to create a “NewsletterDigest”, a publication drawing 
from the best, most interesting and useful articles in the newsletters published by the 
Technical Groups. 

In concluding, I invite you, the members of the Europe Chapter, to send me suggestions and 
ideas about how to strengthen the Society and advance the science and practice of human 
factors and ergonomics. Your perspectives as colleagues who live and work outside of North 
America, wiU provide us with insights on new research methods and applications that will 
enrich our conunon endeavors. My email address is HPVANCQTT@ AOL.COM - 

Thank you for inviting me to attend this meeting. I am sorry that I can not be with you. I trust 
that you will find his to be a worth while conference. Have fun as well! 


Multimedia and Management of Enterprises 

Dr. h. c. Klaus Steilmann 
Member Club of Rome 

President and C.E.O. of the Steilmann Group, Bochum-Wattenscheid 


The speed of technical development in the area of new information and communication 
technologies can hardly be correctly estimated. For trend researchers, politicians, and 
“economic gurus“ Multimedia when associated with terms such as information society, 
data highway and digital era promises solutions for the greatest problems of the 2F* 
century: worldwide peace, global growth and unemployment. Technical opportunities 
that are concealed behind the shimmering term, Multimedia, are definitely impressive. 
The combination of communication systems, telephone, data networks, and communica¬ 
tion satellites with other means of communication such as television, computer, fax ma¬ 
chine, and radio offers users the perspective of worldwide interactive connection. In ad¬ 
dition to real corporations, more and more virtual firms will be developed, creating a so 
called “cyber industry“. These virtual corporations are agile, global, “cybemetic“, and 
fast learning companies, which will expedite the evolution of the business worlds in a 
decisive manner. Cyber corporations are part of modem economic systems, which show 
similar behavioral patterns to biological ecosystems. They connect multiple industries 
and compete for synergetic effects and predominance in the jungle of competition. James 
Martin distinguishes three kinds of intelligent Darwinist-like evolution (Martin, 1997a). 
The primary type of evolution relates to the modification of a product or service while 
maintaining a predefined manufacturing process and a fixed structure of the corporation. 
The secondary type of evolution refers to the modification of the production process, the 
methodology or the workflow. In this context the structure of the corporation can be 
adapted appropriately. The tertiaiy type of evolution investigates factors outside of the 
company, termed the framing condition, i.e. relations to other companies. 

My long standing experiences as an entrepreneur proved these evolutions not only to be 
typical for cyber enterprises but also necessary for the development of corporations to¬ 
day. Yet, the concentration on the tertiary type of evolution will be considerably in¬ 
creased as more cyber corporations emerge. Multimedia is literally the combined use of 
different media, more precisely, use of such media that utilizes different human senses 
such as the visual and the auditory systems. So far. Multimedia has had several different 
effects. Multimedia is not only comprised of multiple modes of information, but also 
computer integration and/or computer support. The overall effect is a computer sup¬ 
ported media integration process. The user has time-independent access on information, 
digital processing, storage and proliferation of this data and also the option to interact 
and communicate at any time (Hermanns; Suckrow, 1993. Grabner; Lang 1992. Stippel, 

For my subsequent explanations and arguments, I would like to use the term Multimedia 
synonymously with Multimedia techniques for new information and communication 
technologies. I choose to do this because there are a great number of different interpreta¬ 
tions and definitions for Multimedia, even if these compose a more complex scope than 

original sense. For reflections on changes in business and management this definition is 
very useful. I do not want to concern myself with all technical parameters at this time. 

Many authors have dealt with Multimedia definitions under the viewpoint of assessing 
social aspects of new technologies. Many more authors have described social aspects of 
new technology without even mentioning Multimedia. The reason for is that Multimedia 
is a comparatively new term, though selected elements of it have long existed, subsumed 
under terms like new techniques, computer networking, hypermedia etc. Many books 
included in the bibliography therefore do not explicitly address Multimedia but never¬ 
theless contribute significantly to social aspects of computer applications, the influence 
of computer networking to management and the consequences of connecting enterprises 

Unfortunately I cannot cite and evaluate the literature covering these topics in detail. Yet 
a number of books or studies will be mentioned and cited in brief throughout this paper 
if necessary to further clarify my statements. 

Multimedia in the Sociological Evolution 

The protagonists of new technology address Multimedia as the most important revolution 
since the invention of fire (Spiegel Spezial, 1996a). The term Information Society is fre¬ 
quently used here without analyzing the true content of the term. The following basic 
understanding which is of crucial importance is frequently forgotten: “Society cannot be 
reduced to information. Economical, ecological, social, and psychological conditions 
form the foundation of life and not only information (Hdfling, 1996). 

There are currently four extreme views of Multimedia. They are: 

• A warning that Multimedia destabilizes established sound stmctures in all areas, 

• The tendency to expect a great “hausse” (boom or bull-market) for society, econom¬ 
ics, and consumer, a true thrill, 

• The expectation of a “baisse” (fall or bear-market), a doomsday-like situation, 

• The belief that the world will not change despite technical innovations. 

Probably none of the four statements can truly characterize the consequences of this new 
technology alone. During a study initiated by the Office of Research on Consequences of 
Technology from the German Parliament in Bonn, different application areas of Multi- 
media technologies were covered, including the following: 

• Multimedia in Business Applications 

• New Media worlds in private households 

• Multimedia in the public domain, i.e. for citizens and community 

• Learning by Multimedia 

• The new Multimedia vocabulary 

• Computer support for radio 

The study comprises nearly ten pages of consequences and suggestions (Riem; Wingert, 
1996), that make clear that Multimedia greatly influences all areas of life and will result 
in profound changes in conmiercial activity. I do not want to restrict my analysis to the 
business and management processes as companies are embedded in sociological framing 
conditions, especially since the tertiary evolution constitutes the greatest challenges. 


Changes in the social attitudes of people with respect to behavior while buying, relaxing 
or vacationing have a direct economic effect on values in the ecological situation. This 
is a very important issue for me. I am interested to see how people will take the chal¬ 
lenge of Multimedia, how they will use their chances, see its risks and understand the 
innovative development of the information- and communication technologies that wiU 
become part of new organizational structures in business. I am eager to see new concepts 
of marketing and sales in certain aspects of principal policies. Learning will develop to 
be a complex challenge. From my understanding this challenge will not only relate to 
“learning with Multimedia“, but to learning for and because of Multimedia. It cannot be 
overseen that in the public and in management on many different levels there is already a 
certain level of knowledge regarding the existence of Multimedia applications. Society 
on the other hand possesses only partial competence to use these new opportunities. 
Moreover, the risks of multimedia technologies are perceived very differently in practice 
work, everyday use, and by small and medium sized companies even though the theoreti¬ 
cal aspects are well defined. There is stiU very little routine skill, know-how, and user 
experience available in these technologies on a broad scale. It is not only a requirement 
of school education to further develop these skills, but it is also an essential task for the 
management in businesses. This point I will further elaborate on later in this paper. 
Moreover, we will need a realistic calculation of national economy and business admini¬ 
stration related facts to assess the ever accelerating physical and moral deterioration of 
new information- and communication technologies and software. 

Only in the evolutionary context of society, a clearer understanding of Multimedia can be 
developed which is neither euphoric nor apocalyptic, but realistic. Businesses will be 
forced to approach these new technologies with a much stronger emphasis. They should 
do this on the basis of their critical engagement. 

New Challenges to Democracy and Participatory Leadership in Industry 
General access to information has a basic and profound sociological importance and is 
an important prerequisite for a democratic society and participatory leadership on the 
business level. 

In conjunction with the possibilities to spread out new multimedia techniques, numerous 
new formats to democratize, monopolize, globalize, and respectively regionalize society 
and management will result. The jurisdictional framework for the new information and 
communication services has not yet been defined. Germany is one the first countries in 
the world to have developed such a framework. These new established standards focus 
on the principle of extensive deregulation, which promises fi*ee access, responsibility, 
and data protection specific to different areas. Yet, present laws are not fully accepted by 
the German economy. The main criticism exist against the following split governmental 
responsibility for different electronic services: 

• German federal states are responsible for TV-shopping and Videotext, 

• Internet and telebanking are under the sovereignty of the Federal Government, 

There are no regulations within corporations regarding management. 

The Federal Association of German Industry expects an increase in uncertainty with re¬ 
spect to legal problems and investment policies. They are also afi*aid that many busi- 


nesses will leave the country, because increased requirements of data protection will 
highly influence the flow of data within the corporation. The further development of the 
legal framework is of crucial importance to the full use of labor potentials as documented 
by a study of the Council of Economic Advisors to the United States of America. Ac¬ 
cording to the study, a significant increase in return from investment could be achieved 
by reduction of legal uncertainties, the elimination of restrictive regulations and the pro¬ 
motion of competition in telecommunication and in the telecommunication industry. 
Besides legal regulations, restrictions of access to use new technologies with respect to 
costs have to be taken into account. The new technologies presumably guarantee their 
users an equal access to global knowledge resources, information, and interactive par¬ 
ticipation in the opinion forming process in the corporation. It should be stressed that 
this is true only for potential users. Only those who have access and competence to use 
and exploit this information can derive benefits from these opportunities. Cost effective¬ 
ness plays a crucial role in the further development of management in the direction of 
participatory management. 

There are no clear perspectives on the future price to use multimedia technologies. Many 
experts have the opinion that prices for information transfer will drastically decrease due 
to the complete liberalization of the telecommunication market. They predict that not 
only the promotion of mass access to technologies, such as the Internet and other new 
communication services, will be enforced. Mass access to information can be an oppor¬ 
tunity for democratic processes in society and business. Right now the information 
highways are far too slow, unsafe, and too expensive for many potential users. In many 
companies the rules for the Intranet, the information and data exchange within the corpo¬ 
ration, are just being defined. Whether or not more participation can really be achieved 
or whether fewer software moguls increase their power remains to be seen. 

Even today Bill Gates is known as one of the most influential entrepreneurs in the world. 
He is in the position of acquiring the complete cultural inheritance of the world in nego¬ 
tiation with the great museums of all countries in the sense of buying the right to repro¬ 
duce them. It is expected that access to Internet information, therefore use of Multimedia, 
will become more expensive the very minute when the integration of all business trans¬ 
actions, life experiences and educational processes of the majority of people is achieved. 
Experiences in the past show that we will have to expect additional fiscal and financial 
taxes in new dimensions. 

Meanwhile powerful global multinational companies of information and communication 
technologies and media trusts have developed unknown amounts of power and political 
influence. National governments cannot practically oppose this power with appropriate 
countermeasures besides quarreling, as seen with TV-transmission times and communi¬ 
cation channels. In this respect, there may arise new dangers to democracy. The influence 
of the national governments, as opposed by supranational organizations, regional interest 
groups and by competition of private firms, will be drastically reduced. Even data relat¬ 
ing to individuals, their private consumption, shopping habits, preferences and liking 
will be ever more controlled by economic giants. The companies will be obliged to store 
and use personal data according to the technical state of the art with greatest possible 
care. This applies to the use of personal profiles and potential abuse of personal infor¬ 


The prominent German newspaper FAZ-Magazin predicts that democracy will be the 
great loser in the information society (FAZ-Magazin, 1996). Opinion poUs via Internet 
will surely influence the work of governments and parliaments and thus undermine their 
ability to govern responsibly. Each individual, who is wealthy, educated and has Internet 
access becomes the winners. On the other side are millions and millions of illiterates with 
respect to computer technologies. Five biUion social losers currently have no chance to 
access the net. Members of the industrial society look down contemptuously on “digital 
illiterates“ or “homeless in cyberspace“. 

Even though globalization is a major trend now, there is also a trend towards regionali¬ 
zation. The “technocratic digitalos“ and an increasing number of people turn to tradi¬ 
tional structures to find their identity in small regional units. At the same time there can 
be seen another trend towards isolation and crisis through the lack of information. The 
reason for this trend is the power and financial influence of managers and owners of in¬ 
formation. The same trend is obvious for the management and the participatory inclusion 
of coworkers in the corporation. The development of democratic processes globally, re¬ 
gionally in a country and in industry will be an important prerequisite in order to take 
countermeasures against possible negative tendencies and to intensify the use of positive 
impacts that new technologies may offer. 

Opportunities and Challenges for Multimedia in Business 

Technological innovations change products, production procedures, organizations, mate¬ 
rials, and markets. In the clothing industry measuring equipment will be used increas¬ 
ingly to scan a human body in three dimensions and to produce scans in true color. These 
scans are used to show the final result and to cut the cloth appropriately. A day later the 
customer will receive his clothes perfectly made-to-measure by a direct delivery service. 
The scanned anthropomorphic data will be written to a Personal Body Card by the cus¬ 
tomer and the salesman for later use. The advent of high tech in the world of sales and 
the textile industry requires new solutions from cyberspace. The Textile/Clothing Tech¬ 
nology Corporation (TC**2), a 200 firm consortium, demonstrated how a designer in 
New York cooperates with a customer using video-teleconferencing to design clothing 

Today it is not a technical problem for the customer to select fi'om huge computerized 
fashion collection, the fashion pattern, the fabric, colors and texture, even the buttons 
and to display the customer dressed in his personal composition of clothing on a digital 
photograph. There will arise a threatening competition for those businesses selling 
clothing not made-to-order and it will be necessary for my company to take this new de¬ 
velopment of made-to-measure clothing with great attention. 

The philosophy of buying will play an important role for the customer buying his cloth¬ 
ing. Shopping will be an entertainment, trying on and rummaging will be a means of joy 
and reduce efforts to find the right clothes to a minimum. The greatest challenges for 
industry and management resulting from closer links between industry, trade, and the 
customer will be soon visible. The processes in and around enterprise are becoming more 
and more complex. A special situation attained by the direct connection that is now pos¬ 
sible not only between trader and customer but also between producer and customer. For 
many production tasks basic strategic questions arise for businesses. 


Opportunities for the industry are in an increase of efficiency with respect to internal 
communication and informational processes. These opportunities can be enhanced using 
data exchange, videoconferencing, tele supply samplings, and education. Even external 
communication is an important future application area of multimedia. Offers can be 
made using online services, consumers can be attracted using CD-ROM catalogues, and 
direct exchange of information can be possible using e-mail or automatic exchange of 
documents. Decentralized firms based on telecommuting have the advantage to produce 
specific to the customer’s needs on schedule and be cost efficient. The market position of 
small and medium firms can be increased using state-of-the-art information and commu¬ 
nication media. 

The members of the “New Media“ study group, amongst whose members were delegates 
of the Federal Government of Germany, defined these particular requirements for the 

• More customer orientation by increasing the quality of offers, 

• Increased fulfillment of global market opportunities (e.g. by marketing using the 

• Increase of the ability for innovation and readiness for new services and better per¬ 
formance orientation through more efficient use of technological potentials and new 
forms of cooperation in networks, 

• Adaptation of the corporate identity to new technological opportunities such as de¬ 
centralized organization, a stronger orientation of staff towards corporate identity, re¬ 
duced number of management levels in the company management, lean and flat hier¬ 
archies, continuing education and qualification of staff in “learning enterprises**. 

Currently, the highest priority use of Multimedia is within the market-oriented manage¬ 
ment, a tendency called marketing-communication. 

In America, 25% of the biggest corporations are represented on the Internet. Online, each 
company has the same market opportunities. New service providers can effectively com¬ 
pete against established firms, if they are smart, innovative and offer their clientele inter¬ 
esting presentations. Using multimedia technologies, e.g. the Internet, results in many 
advantages for all parts of a company and many opportunities for the management: 

• Time effective processing of routine tasks, 

• Increase of productivity and at the same time reduction of staff, 

• More efficient use of resources and time saving, 

• Improvement of the basis of planning and decision taking through better access to 
external information, early warning when trends change, retrieval of market research 
data, knowledge of different industries and economic trends, 

• Enhancements in production by well-aimed selection of suppliers worldwide, direct 
communication with these suppliers, and online synchronization to reduce stock, 

• Changed aspects of corporations using global access to electronic business, open 
communications channels, and increased customer contacts, 

• More skillful product design using closer connections to designers and design con¬ 
sultants, cooperation with research establishments, continuous inclusion of new 
global trends, 

• Using expanded marketing or micro-marketing oriented strategies according to indi- 

vidual requiremeiits and preferences, fast feedback, market research data, and con¬ 
tinuous sales forecasts of trade companies worldwide, 

• Better customer orientation, customer following precise orders and deliveries using 
faster access of skilled knowledge, problem solutions and product information, 

• New chances for humanization of the world of labor, 

• Improvements in the field of personnel management, using electronic curricula vitae, 
access to differentiated education, self-learmng, information about jobs and careers 
in the corporations, hiring of self employed at-home workers and project oriented 

• More flexibility and at the same time more socially adequate organization of working 

• Reformed information handling, 

• Better and more cost effective communication, with new potential customers world¬ 

• Increase of professional knowledge offering chances for continuous exchange of ex¬ 

and others. 

Many corporations are starting to use the opportunities of the Internet by offering to their 
staff the chance to use this new medium, applying their own strategies and thus opening 
the net for important business processes. 

As soon as corporations become more flexible and smaller, the importance of alliances 
for the success of the corporation will rise. Information management will develop as a 
key factor. Everybody who wants to initiate alliances successfully needs frictionless, 
properly timed and unhindered flow of information and communication. Corporations 
require international presence, “global sourcing“. They also need development teams to 
cooperate worldwide using time differences to be innovative and creative 24 hours a day. 
There is an urgent need to define and optimize management information and information 
structures that are crucial for the business purpose and to reform process and informa¬ 
tional structures as well as data architecture. To guarantee efficiency in management of 
information there are further requirements than simply technological competence. Virtu¬ 
osity will be required for those who find synergies in knowledge resources with respect to 
a specific problem to derive innovative developments for the corporation. 

This means drastic changes for the top management as well as the employees. It also re¬ 
quires continuous learning and a lot of intuitive selecting of information. The Internet is 
not only a source of relevant information but also a huge ocean of banahties with singu¬ 
larities of great value (Martin, 1997b). 

Using Multimedia on the level of a corporation is more crucial to solving application 
problems than to technical aspects. In this respect, there are numerous new challenges for 
companies. Requirements relate to the development of worldwide new information 
sources, but also in designing a network of development and production stages. New 
virtual structures in organization and labor will develop in combination with new indus¬ 
trial processes that today cannot even be imagined. Integrated document management, 
diagnosis of technical systems using remote access and analysis systems will be even 
more effective using many new internal applications. Even now, there is definitely a fast 


reduction of hierarchies and a much faster flow of information using multimedia tech¬ 
niques, Management can be developed more efficiently and in a more process-oriented 
manner offering completely new possibilities in connection with multimedia customer 
service. Not only the introduction of computer oriented manufacturing, but especially 
computer supported, customer oriented, and interactive mass production-to-measure, will 
lead to the establishment of numerous new task areas. These applications within the cor¬ 
poration require permanent learning processes within industry. At the same time the de¬ 
velopment of intra-networks and changes in the overall communication structure require 
new efforts. The aim is to react in a timely, quality oriented, and predetermined way that 
is flexible to customer requirements. At the same time customers should be included in 
interactive design processes. This idea is already in preparation in the field of industrial 
mass confection made-to-measure. We can predict, especially in this professional area, 
most of the revolutionary consequences. At the same time there are new problems with 
security. These problems do not only relate to the increasing use of virtual money and 
digital signatures but also to new dangers of industry espionage and the potential break¬ 
down of complete information systems and databases. In response to this I would like to 
study the problem of computer viruses and hackers. I would also like to study the prob¬ 
lems of the intelligent theft encouraged by high tech systems that are not completely safe, 
especially since our industry is one of the most affected branches. 

Changes in Management Processes due to Communication and Information Tech¬ 

The possibilities for representing corporations and products on the Internet are plentiful 
and diverse. Company information and brands can be displayed and put forth. There are 
advertisements for many brands including promotional actions such as sponsored events, 
games, and price opportunities to attract customers. New ways of promotion are being 
developed using virtual showcases. Multimedia shops, video entertainment and shop 
radio. Shopping will become an “adventure joumey“, parts of the selection will be con¬ 
nected to product worlds or can be reorganized in new worlds by the customer himself. 
The Internet has received a degree of user-friendliness by its new graphical user inter¬ 
face, the World Wide Web. 

Entrepreneurs and top management have to ask themselves whether the corporation shall 
be developed to a virtual corporation, a cyber-enterprise, and whether it is possible. They 
will also need to decide whether there is a profound strategic vision and whether there are 
qualifications and skills - or competence in the enterprise altogether- to face these new 

New developments in information and communication technologies result in necessary 
changes for the corporation such as: 

• Fast reaction to events and - if necessary- to take a favorable turn into a new direc¬ 
tion of corporate strategy, 

• Continuous improvement of products and services and introduction of new products, 

• Ongoing development of applied strategies to permanently enhance and change 


speed of reaction in many situations will become a crucial factor in competition for. 

• Recognizing requirements of new products, 

• Bringing new products to market, 

• Establishing new services, 

• Improving products, 

• Reacting to trends in fashion, 

• Satisfying customer requirements, 

• Controlling stock and distribution of goods. 

Technical information and communication processes will put forth a significant contri¬ 
bution in these areas. We must be even more conscious of die fact that human communi¬ 
cation is a decisive part of our culture that will be critically influenced especially by the 
new information and communication techniques and the use of multimedia. Using the 
new media there is a completely different form of communication between individuals. 
The digital age is understood by many people as birth of a new confidence that is based 
on digital principles. The completely new cultural aspects of life in our so called “global 
village^, the new Internet culture, has not yet been researched sufficiently as of yet. In 
the “First Annual Report to the European Commission from the Information Society Fo- 
rum“ of June 1996 there was an introduction of 128 members of the Forum who were 
elected by the commission. Their background can be subsumed under five groups: 

• users of the new technologies: industry (banks, retail, maritime etc.), public services, 
consumer groups, small and medium-sized enterprises and the professions 

• social groups: academics, employers organizations, trade unions, youth groups, re¬ 
gional and city representatives 

• content and service providers: publishers and authors, film and TV producers, 
broadcasters, computer software producers and information service providers 

• network operators: telecommunications repair technicians, cable TV, mobile and 
satellite operators 

• Institutions: members for Parliament, of the Economic and Social Committee of the 
Committee of the Regions and the data protection Commissioner. 

In six working groups they dealt with the following questions 

• the impact on the economy and employment 

• basic social and democratic values in the „virtual community** 

• the influence on public services 

• education, training and learning in the Information Society 

• the cultural dimension and the future of the media 

• sustainable development, technology and infrastructure 

These tasks are probably correct. Yet it is not clear whether they address the really deci¬ 
sive question in appropriate depth: the evaluation of information and knowledge. 

Filtering the growing flood of information individually and converting information into 
applicable useful knowledge is, from my point of view, the primary purpose and presents 
the greatest challenge for today’s managers. New multimedia systems should not only 


offer information, but also explain complex contexts in an acceptable manner. At the 
same time any information of any content should be able to be produced and offered 

From my understanding it is a fatal error to believe that more information also improves 
the quality of information. We all know from our practical experience that more infor¬ 
mation guarantees by no means a qualitatively new level of being informed. Even in the 
information age we must not oversee that useless information, redundant information, 
false information, misinformation, and catastrophic information serve no purpose as 
people are frequently flooded by data and are still hungry for the right information. 
Evaluation of information is therefore frequently more important than the information 
itself. This subject is a task that cannot be easily formalized on the corporate level. In 
how far multimedia information processing enhances and facilitates the adoption of 
knowledge by offering a better emotional access to our brain or whether it hampers 
knowledge acquisition by sacrificing resources to redundant information, will be a ques¬ 
tion of continuous learning within the corporation. 

Consequences of Multimedia for Employment 

The political consequences with respect to the task of new multimedia developments are 
many and complex. There is hardly an industry that wiU not be influenced by new tech¬ 
nical innovations. The change of existing professions and the development of new ones 
will determine the organization of work. It will also be determined by changes of forms 
of cooperation within the corporations, between corporations, and in industiy. Neverthe¬ 
less, the potential for improved labor efficiency and effectiveness is highly controversial. 
One group understands Multimedia as the greatest hope for new workplaces, others de¬ 
fine it as a “job-killer“. As seen historically both sides are correct. Increases in produc¬ 
tion have lead to more growth and new jobs in the long run. Yet it is not clear how long it 
will take until production relevant increases of incomes will lead to a higher request for 
goods and as such compensate for prior effects of rationalization. Nobody can teU today 
which and where new products and services will develop and lead to a higher level of 
employment, Fourastid (Fourasti6, 1954) based his theories on his so called 3-sector- 
model in which he predicted employment and technical development in agriculture and 
forestry, industry, and services related to employment as 10:10:80. Initiated by the 
OECD in the 70’s, there was a reorganization of statistical measures taking a fourth sec¬ 
tor, the information sector, into account. This sector was defined using a very wide 
scope. It included all administrational professions, the complete educational world, even 
foresters and inn keepers, and other “information processing“ professions. 

Multimedia inspires the imagination of many but for most of it is still in a state of ex¬ 
perimentation. The most important purpose of the new information and communication 
technologies is to divert routine jobs to machines and then eliminate them. Computer 
technologies serve the purpose of increasing efficiency and quality in the corporation. 
Unskilled typists and low ranking managers who only conduct routine tasks can easily be 
replaced by cheap and reliable software products. The consequence of this is fear because 
finding a new job within the corporation is not likely. Everywhere, in production, in ad¬ 
ministration, or in the office, knowledge is used productively in a systematic way. Op¬ 
portunities for effecting change are used in an optimal way, and cost will be reduced. 
Many experts talk about a new ethic of efficiency that is applied to all branches of the 


business and which are directly or indirectly linked with modem information- and com¬ 
munication technology. 

Studies of the University of Wurzburg, Germany, prove that more than 6.7 million jobs 
of the current 22 million jobs in this country could be cleared away by introducing the 
new information and communication technology. One example for the amount of poten¬ 
tial workplace reduction is set out in the following table: 

Professional Domain 

Employees 1996 

(not part of the study) 

Estimated unemployed 


(after introduction of IT 














Health Services 






Public Administration 





















Sales Promotion 






This relates especially to branches such as banks and insurance companies, consulting, 
health services, commerce, public services, transport, logistics, and simple service pro¬ 
fessions such as in renting offices, which are subject to rationalization even more. The 
already mentioned “First Annual Report to the European Commission from the Informa¬ 
tion Society Fomm“ underlines the findings firom the point of view of the consequences 
with respect to employment using the following thesis: 

2 ^ 

“In order to maximize the job-creating benefits of the new technologies as quickly as 
possible, it is essential and urgent for companies and organizations of all sizes to adapt 
their organizations and structures. Until this process is well underway, the Information 
Society looks likely to destroy more jobs than it creates”. 

This strengthens the so-called “job-kilhng-thesis“. 

In the future the adapted principle of lifelong employment will be an unrealistic ideal. 
Outsourcing of processes in enterprise will increase. More and more work will be con¬ 
verted to self employed, free lancing and temporary jobs. A new highly competitive mar¬ 
ket will develop. Not only will telecommuting become more popular, but also improved 
flows of information will make work processes more flexible. It will become easier for 
firms to acquire external workers. The consequence will be that the corporations can 
work more effectively, be more decentralized, and can decrease in size. Telecommuting, 
teleservice, telecooperation, telecommunication, and even telefeeling will no doubt re¬ 
duce personal and direct human contact even more. It will depend a great deal on the 
management how far the personality of a human employee and his chance for creativity 
and self-development will be dealt with. And it will also depend on the management for 
which positive or negative consequences the increasing number of virtual enterprises will 
bring about, that can be transferred from one location to another and easily seize and 
start operation. 

A study of the German Institute for Work and Technology (Institut fUr Arbeit und Tech- 
nik, 1997 ) shows that telecommuting or telework has a growing importance in combina¬ 
tion with major shortcomings. Especially for women telecommuting offer new job op¬ 
portunities but it does result in some disdvantages. Housework and professional work 
have to be performed at the same time increasing workload. Telecommuting also tends to 
reduce chances of a career because career is closely linked to presence in the enterprise. 

Now I would like to talk about another consequence of Multimedia with respect to labor, 
which is the global competition for the company location. This is not only a competition 
between products and services but also a struggle for work places. Self-employed persons 
must be prepared for an intensified competition with respect to accomplishment and 
price for their products. Even developing countries possess highly specialized experts, 
who can access global markets without restrictions. From all these examples it is clear 
that the concern about the degree of employment is very justified. 

After talking in detail about the negative aspects of the employment situation I would 
like to contribute to the position of “Promising Multimedia“. We must not forget the 
positive employment tendencies. These tendencies promise to maintain employment and 
even define new workplaces using increased efficiency. Eveiything depends on the abil¬ 
ity of structural change in the corporations and the position in the international competi¬ 
tion, The optimistic attitude of many is based on the macroeconomic assumption, that 
earnings with respect to productivity cannot simply be ignored with respect to employ¬ 
ment. Increased buying power in different areas will strengthen demand and help to de¬ 
velop new workplaces. But it is only speculation as to what extent this will happen. 


Another thesis in the first annual report is the following: 

„Teleconimuting offers many job-creating possibilities and attractive improvements in 
working lifestyles, although, it raises many important issues for labor laws and collective 
bargaining. Public policy must facilitate, not obstruct, the development of teleconunut- 
ing. Promoting a mix of home and office telecommuting is an effective way of handling 
fears that telecommuting exclusively in the home may be too isolating.“ 

These sentences make clear that even the protagonists of Multimedia cannot contribute 
concretely to the blessings of new information and communication technologies. 

The brief effect of rationahzation cannot be avoided because the corporations are forced 
to produce as cost effectively and efficiently as possible to withstand international com¬ 
petition. Corporations must reduce product cycles and at the same time increase the vari¬ 
ety of products and models. They must also reduce development and delivery times, in¬ 
crease product quality and invent customer oriented system solutions. Currently, there is 
no reasonable alternative to the new technologies, or else all workplaces would be endan¬ 
gered. In how far information technologies are used for the development of new products 
or to improve these products is also dependent on management and motivation as well as 
skill of the employees. Discovery of new problem areas will be an important challenge 
for the management. 

To guarantee a good future standing of Germany as a major industrial player, the already 
mentioned working group ‘'New Media" was founded. In this working group delegates of 
the German Federal Government and those organizations were represented that are 
"members of the inner circle of the Bundeskanzler", the Chancellor of the Federal Re- 
pubhc of Germany. In this working group positive influences with respect to the level of 
employment were identified, especially in the information and communication markets. 
Despite the uncertainty of the prognoses in a series of different studies, quantitative ef¬ 
fects of employment were analyzed that could be initiated using new media. Results of 
an analysis of Prognos/DIW in Germany came to the conclusion that about 180.000 
workplaces in economical branches could develop that belong to the media- and commu¬ 
nication sector until the year 2010. The highest potentials are with so called “content- 
providers" and in the area of computer software as well as computer services. A smdy of 
the METIER- consortium for the European Union calculates that by expansion of new 
information and conununication technologies up by year 2010 six million additional 
workplaces could be generated. From these six million workplaces -on a calculated ba¬ 
sis- 1.5 milhon could be established in Germany. Such estimates suffer from a lot of 
methodological problems. As there are no concrete prognoses at the present time, we 
should concentrate on the fact that only the creation of mass markets will develop em¬ 
ployment effectively. The potential of technical innovations can only be fully utilized if 
they are accompanied by complementary organizational changes in work and production. 
Information society alone cannot solve the problem of employment. The solution of that 
problem will by no means be possible without lifelong education and social innovation. 

Electronic Shopping and Cybercash Unlimited 

A completely new challenge for the future is the increasing use of electronic shopping for 
all enterprises. All business transactions will radically change, not only from use of elec- 


tronic commerce and interactive data exchange, but even more by new digital worldwide 
shopping possibilities. 

Buying habits will drastically change. We do not yet know which will be the “killer ap- 
plications“ of information society. To identify these killer applications, pilot projects 
should be conducted in sample population as much as possible. Not only the technical 
aspects of transaction must be tested, but also the readiness of the customer to buy and to 

What will the achievements of online shopping be? Once a customer has developed a 
need for a particular item they may search online for that item that best matches the price 
range and specifications they desire. The order could then be taken and acknowledged 
online. The customer would receive immediate acknowledgement of purchase. Using 
direct data entry checked by the customers themselves, transaction errors could be re¬ 
duced and orders can be processed in a cost-effective way. The customer could make 
purchases easily and efficiently, without being restricted by business hours. Buying is 
made virtually stress fi*ee. In this way the customers needs are best served. 

Presently single enterprises in industry and commerce estimate the future of multimedia 
shopping very differently. Still it is very probable that radical changes of the structure of 
commerce will happen in the next decade. This change requires a timely orientation to¬ 
wards these new buying procedures. The basic credo of the information society is that 
everything is available everywhere and any time. 

Home shopping is not new to people. Mail order houses have guaranteed comfortable 
shopping for a long time. Yet, teleshopping will fundamentally change the presentation 
and display of the whole portfolio of goods. Buying will be possible 24 hours a day and 
worldwide. The inherent loss of sensual pleasure especially in fashion will establish some 
barriers but not change the basic trend. Until recently it could not be estimated what con¬ 
sequences this would have on small retail establishments or the larger commercial 
chains. However, with the development of online shopping, many chances emerge for 
commerce to manipulate consumer behavior. Individual shopping habits can be recorded, 
analyzed, and stored in databases able to generate complete buyer profiles for use in sub¬ 
sequent marketing decisions. With regards to global competition, borders between pro¬ 
ducers and commerce will change under the influence of these new technologies. Many 
producers plan to utilize direct sales, factory outlets, and new forms of electronic ware¬ 
housing to boost sales. 

New procedures in management such as efficient “consumer response“ or “category 
management“ are predecessors of this development. International regulations on product 
liability, advertisement, and consumer protection are becoming more and more necessary 
in a world of increasing electronic shopping. 

Multimedia and Simulation: Human Machine Interface Implications 
There is still another facet of Multimedia that I have not yet discussed. This aspect of 
Multimedia does not utilize this technology as an instrument for shopping or as a tool for 
today’s managers to operate their enterprise. Instead, it is the aspect of Multimedia that 
makes work at a complex technical plant acceptable, makes modem systems controllable 
and guarantees safe and efficient operation of complex systems. This facet includes the 


aspects of leisurely use, entertainment, and both useful and joyful implications for hu¬ 

Many aspects of our everyday life can only be accommodated through the use of techni¬ 
cal support. As those supporting systems become more and more sophisticated, they will 
require increasingly complex technical equipment. More complex technical systems re¬ 
quire an increase in the skills of the human operator to guarantee their safe and efficient 
use. Many studies with respect to reliability of human operators prove the great influence 
of the human factor. Reports of commissions to clear up accidents and malfunctions of 
complex technical systems contribute a great deal to these studies. Later in this chapter 
F11 try to develop a systematic approach to clarify in what areas of life and work such 
technical support is undeniable and something nobody should do without. More and 
more sophisticated technical support thus calls for a carefully optimized control station 
to reliably operate complex systems. As this optimization is not just a technical optimi¬ 
zation, but has to include the human operator as well, it will be an iterative design proc¬ 
ess comprising both the technical system components and the human factor. The human 
factor can be optimized by developing refined and adapted methods of training. Thus 
even a sub-optimal system can still be controllable. The far better approach, though, is 
the adaptation of the technical system to the requirements and characteristics of the hu¬ 
man operator. 

There is a new term called “Paradox of the Information Technology” (Brddner, 1997). 
This paradox explains the divergence of technical functions and human-machine per¬ 
formance. The more complex technical systems are, the more performance would be ex¬ 
pected. Frequently the contrary is happening. Making users believe that technical sys¬ 
tems are intelligent may result in inadequate confidence into those systems. Therefore it 
is crucially important that complex systems explain the way of their decision making to 
the human operator. Here again an important aspect of the human-machine interface is 

Adaptation of the system to man requires a careful analysis of human abilities and limi¬ 
tations, Multimedia is a more recent approach to displaying information at a control sta¬ 
tion in a more acceptable and multisensual manner. It is thus better suited to inform the 
responsible person(s) about the state of the technical system. Multimedia can also be a 
technology to broaden the channel of operator input to the system and thus facilitate in¬ 
teraction between human and technical systems components. 

There is a new term for the interaction of complex technical systems using Multimedia 
support at a control station for safe and efficient operation. This term is the “Human Ma¬ 
chine Interface“ for the control of human machine systems. As already mentioned, opti¬ 
mized Human Machine Interfaces will use techniques available to make system operation 
more efficient, safer, and more acceptable to human operators. Traditional interfaces 
predominantly use the human visual sense for information display. This is the historical 
technique because craftsmen have during all times in history judged their work processes 
by looking at their work. Today the display is mostly an electronic display, a screen dis¬ 
play offering graphics output rather than columns of figures. There are cyber helmets 
available today that literally offer a far closer view of remote or simulated environments. 
Using these Multimedia system components, a deeper navigation in the world of the 


Human Machine Interface is supported, A new term to describe these aspects of human 
control of technical systems is “immersion^ into the controlled process with the conse¬ 
quence of an ever more perfected “telepresence*' wherever required. Many modem ap¬ 
proaches offer auditory information in addition to visual data. From simple warmng 
horns in the past, to stereo headsets for pilots offering voice warnings from the direction 
in which a malfunction is located. Vestibular clues are presented to pilots in a moving 
base flight simulator enhancing the fidelity of operation. There are even approaches us¬ 
ing the haptic and olfactory senses of man. These multimedia information displays in 
conjunction with the data glove and other non traditional control devices complete the 
instrumentation of modem Human Machine Interfaces optimized from an ergonomic 
viewpoint operating with the support of Multimedia. 

Multimedia and the Human Machine Interface are tightly hnked. The optimization proc¬ 
ess is a topic for research establishments, universities, and ergonomic departments in 
industry to investigate. It is an interdisciplinary science including the disciplines of engi¬ 
neering, psychology, and work medicine. National and international organizations and 
associations are working in the field of Human Engineering or Human Factors Engi¬ 
neering in the research, development, and application as well as optimization and 
evaluation of Human Machine Interfaces. Beyond these are the GeseUschaft fur Arbeit- 
wissenschaft in Germany, the Ergonomics Society of Great Britain, the International Er¬ 
gonomics Association (IEA) and the Human Factors and Ergonomics Society in the 
United States (HFES). For example, the HFES, more specifically its European Section, 
is organizing a conference on exactly the topic of ‘Multimedia and Simulation. Human 
Machine Interface Implications" in my city Bochum, Germany, in November 1997 
(Holzhausen, 1997). 

Now we know that Human Machine Interfaces are crucial to the interaction between a 
human and a complex technical system respectively its underlying process. Yet, multi- 
media Human Machine Interfaces do not only serve the purpose of controlling “real" 
physical processes. Frequently they are used to control “simulated" processes. Many 
people do not realize this, but simulations are very important today and very helpful. Ex¬ 
perts can “model" a process even better if they know more about the process and describe 
it in a mathematical way. In this way the parameters influencing the processes will be 
recognized. The effect of these parameters and their changes on the process can then be 
determined. Important examples for such complex processes are the bloodflow in the 
human body, meteorology, or aerodynamic aircraft models. 

Researchers, engineers and other experts can “use" the simulated system much like the 
real one, try to improve the model and afterwards use it for diagnosis, weather forecast or 
as basis for constructing a plane. Sometimes a simulation will be run at the same control 
station that is used to control the real process. One such example would be a traffic 
simulator connected with a real control system controlling traffic lights in a city in order 
to improve the community traffic flow. 

One important criterion in a modern human engineered multimedia control station is the 
“factor time". The factor time can be very efficiently used to increase system perform¬ 
ance, and safety. Real processes perform in “real time". All system components including 
the human operator must react in the real time frame. Simulation though can use time 


compression methods effectively to arrive at simulation results much faster than the real 
system. Meteorologists compute weather models independently of the real time, produce 
a weather forecast much faster, and distribute a gale warning in time. A simulation can 
also be conducted in “slow motion^. This guarantees a much more detailed view for the 
human trying to understand turbulent effects of air behind an aircraft wing or in a blood 
vessel, especially if the Human Machine Interface offers adequate Multimedia support. 

I hope I could make clear that Human Engineering and Multimedia promise great sup¬ 
port to humans in a complicated world if it is properly designed and correctly applied. 

Let me try to describe possible application areas to Multimedia in connection with Hu¬ 
man Machine Interfaces through a short systematic approach. A lot of applications exist 
in traffic systems on land, in the water and in aeronautical applications. Many cockpit 
systems in modern cars, ships, and aircraft use multimedia support for navigation, diag¬ 
nosis, and communication. The same is true in aerospace applications. The space shuttle 
requires many human engineered control systems to safeguard its operation. 

Stationary control centers use Multimedia too. Examples are control of traffic systems, 
police forces or fire brigades, air traffic control and space systems. Operations of a space 
station, the Mars Rover, a Venus rocket, or the Hubble Telescope are unthinkable with¬ 
out simulation and Multimedia in the control room. 

In logistics ever increasing demands require Multimedia for applications, such as man¬ 
agement of truck fleets, the storage of goods in a warehouse or worldwide supply. Indus¬ 
try depends on Multimedia in aU the well known “Computer Aided Technologies*' such 
as CAD, CAE, CAD, and CAM or process control in a plant. 

As already mentioned modem medicine makes use of Multimedia in diagnosis (Ultra¬ 
sonic, tomography, data transmission between clinics and telepresentation of X-ray pho¬ 
tographs), 3D man models and stereotactics in surgery and 3D endoscopy. 

In research and development literature reviews using hypertext and hypermedia support 
are state of the art. Teleleaming and Videoconferencing are widely adopted. New didac¬ 
tic concepts are being developed and applied in universities, home learning. Even virtual 
universities are coming into existence. 

In architecture, applications such as urban development, reconstruction of antique sites, 
virtually real buildings are widely adopted. Navigation through such buildings as the 
famous Frauenkirche of Dresden, which was destroyed in World War n and now is under 
reconstmction, are possible. 

Terrain models for ecological protection, seismographic measuring campaigns, or satel¬ 
lite image processing require Multimedia and human machine interface optimization. 

Last but not least advertisement and entertainment are mass markets under heavy pres¬ 
sure and competition. Virtual reality, virtual actors, and tools for sophisticated merging 
of movies and computer graphics animation are practical examples for near perfect pres¬ 
entation of Multimedia techniques to humans. 


What are the consequences and benefits of applying Multimedia to the design and op¬ 
eration of Human Machine Interfaces? First of all safety aspects require every possible 
effort. These safety aspects include both work safety with respect to employees and to the 
population living in the area. More productivity and more transactions per time and em¬ 
ployee are achieved without increasing work strain or maybe even reducing it. Problem 
areas can be tackled that were never suited for man made solutions before. Employees 
can work with a higher degree of job satisfaction. They will be able to look upon their 
work as good, helpful, efficient, and ethical. 

Modem Technology for Information and Communicationf Pros and Cons of the 
Ecological Disaster 

Information technology is also frequently sited as a core element in technical environ¬ 
mental protection. Computers in conjunction with sensors and actuators reduce the emis¬ 
sion of vehicles, analyze dioxin concentrations, reduce power consumption of freezers, 
localize polluted areas, control heating systems, and detect illegal harvesting of the rain 
forest. Information networks help prevent environmental pollution and the abuse of valu¬ 
able resources. Telecommuting delivers an important ecological contribution, by reduc¬ 
ing the resources required for travelhng and driving considerably, especially in the serv¬ 
ice sector. Construction of new roads will be unnecessary, the collapse of the traffic sys¬ 
tems will be avoided. The people will not need to move, instead the data would travel. 
New information and communication techniques present an image that is ecologically 
acceptable: no malfunction, no air pollution and no exploitation of scarce resources. 

But the ecological impact is not solely positive. Here are two different opinions: the first 
opinion states that the information and communication technologies give us hope to be 
delivered from the problem of environmental destruction and the second opinion argues 
that industrial processes and western ways of life are supported and reinforced by infor¬ 
mation technology. From this point of view the new technologies accelerate the con¬ 
sumption of ecological resources on earth. On one hand the networking of partners using 
screens, e-mails and modems can drastically reduce the use of resources and nature. On 
the other hand the development of these techniques from an ecological point of view 
show considerable disadvantages. The ecological balance of the new information and 
communication technologies is not as favorable as often pretended. According to calcu¬ 
lations by the Institute for Climate Research, Environment and Energy based in Wup¬ 
pertal, Germany, the “ecological package“ of a 486 PC adds up to a total of 16 to 19 tons 
of resources. A state of the art production facility for semiconductor wafers consumes 
240.000 KWh energy per day. Additionally, it requires 6.8 million liters of ionized water 
and 70.000 cubic meters of pure nitrogen per day. The so-called dematerialization using 
electronic processing and storage proves to be more and more a “green mythos“. Thus 
the information market accelerates the waste of resources to a great extent. This can be 
shown even more drastically in connection with the use of paper. The paper consumption 
by a citizen of the Federal Republic of Germany increased from 126 kg in the year 1970 
to 232 kg now. The Frankfurt, Germany based magazine “Oko Test“ reports, that there 
would not be a single tree on earth once every Chinese person had the same paper con¬ 
sumption as a German, There are many other unsolved problems of ecology with respect 
to information and communication technology such as: emission of electro-smog, depo¬ 
sition and disposal of batteries, chassis and printed circuit boards, and the complete dis¬ 
posal of worn out computers. In this spirit I would like to state that the acceleration of 

technical and sociological developments initiated by the flow of information and com¬ 
munication on global scale contributes directly to the destruction of the environment and 
the creeping disintegration of society. As never before western ideals are distributed 
around the globe and lead to demands that revolutionize culture and way of life of other 
regions at a breathtaking speed. At the same time we recognize an increasing polarization 
of the rich and poor, especially with respect to information access. We must not overlook 
that today about 50% of mankind is not capable of using a telephone, even if they had 
access to one, much less, the capability of using an Internet account. A new ecology of 
media is required which would address the problem of disposal of the inevitable infor¬ 
mation waste“. 

Effortless Learning or New Competition for Education 

To receive or proliferate information, competence must be developed in the following 
areas: the ,Jaiow-how“ of information structure, the ability to evaluate information and to 
make it usable, the development of searching strategies, and the use of communication 
according to established rules. Multimedia development offers many new applications 
that require these qualifications: visualization of complex structures in science, new 
forms of work such as tele cooperation, telediagnosis in medicine and technical fields, 
telepublication, teleleaming and -teaching. 

There will be a global competition in the once national educational market. It is therefore 
important to start a comprehensive multimedia initiative for education and network ori¬ 
ented teaching. The competitiveness of single economies can only keep pace with global 
competition if network style thinking in school education and university teaching is es¬ 
tablished. In this manner, each economical location wiU be assessed as a scienctific site 
and as a place for information and communication. There will be a great new demand 
everyone to successfully export sufficient new knowledge into the global competition. 
This goal cannot be achieved by the industry alone. It requires a new and strategic coop¬ 
eration between industry and communities, the state, and private or social educational 
institutions. Multimedia and network oriented education will enrich everyday learning in 
many ways. It will by no means be restricted to subjects such as computer science as it 
was in the past. More than ever, new media must be accepted and established as a tool in 
all subjects, including the complete course of all subjects beginning with the scien¬ 
tific/technical and ending with the musical/visual arts. The currently practiced format of 
multimedia learning using e-mail will be insufficient in the future. It is more important to 
acquire expertise in the practical application of multimedia communication techniques, 
A basic requirement for this is the education of teachers, professors, and university staff. 
Another requirement is the establishment of the necessary technical equipment. Taking 
into account all of the investment that is required for the reorganization of education us¬ 
ing new multimedia learning there are considerable doubts that a fast implementation 
will be successful using public or community money alone. The universities are chal¬ 
lenged at the same time to reconsider their present curricula from the viewpoint of Mul¬ 
timedia and to develop pedagogic didactic concepts. Focus must be on software for edu¬ 
cation and training. The development of “learning on demand“ must be a subject for 
further research in order to understand its prerequisites and consequences better. One 
specific special danger is that, on different levels, at school, university, or administration, 
one waits for the other. A lot of time will be lost, as previously seen in many European 
states. Another challenge is the implementation of new technologies in the developing 

states, which requires even more effort. The 1996 initiative in Germany for “Internet ac¬ 
cess for aU schools“ must be further developed and fully implemented. The future costs 
for this initiative must also be taken into account. Along these lines, it should be pointed 
out again that transformation of our society into an information-based society should 
happen worldwide. On a global scope, there is a considerable lag in the developing 
countries, especially in the former Soviet Union and China. We clearly must see that in 
the next ten years more new knowledge will be processed and taught than in the previous 
2500 years. The faster and the more comprehensively we address this task the greater the 
chances are that it will be successfully completed. 

Using new information and communication technologies must become a general cultural 
tradition, including media education in professional education. Multimedia and network 
oriented education should not only be seen as only technical education but also as an 
opportunity to enrich learning playfully. Using these new technologies naturally, a new 
didactic and content-oriented creativity will be possible that allows more room for per¬ 
sonal activities and individual learning. As the teacher acts only as a coach to help ac¬ 
quire knowledge, independence and responsibility of the student become more and more 
important. This proves once more that only easy access to Multimedia will help achieve 
acceptance of new technological development and helps promote these opportunities. 
Industry is also responsible for promotion of media education. The development of edu¬ 
cational software should be designed for international use. As the problem of establishing 
multimedia techniques in industry is important in global competition, economics must 
maintain interest in encouraging educational initiatives so as not to fall behind the global 

Emotions and Virtual Reality 

The ability to utilize new multimedia opportunities depends solely on the motivation of 
the end user. This motivation can be promoted using different strategies: the opportunity 
can be presented as entertainment or it can facilitate work. Only these capabilities will 
result in user acceptance, where the user explores each new interactive possibility step by 
step. Personal motivation is -from my understanding- the key around which all of us 
should design applications. We should use new techniques in ways that make them emo¬ 
tionally acceptable. Emotional relations will play a crucial role in the cyberspace era. 
Several studies prove that cyberspace will change emotional relations of humans at work. 
In addition, new emotions will be created to replace missing real contacts. 

Traditional corporations very frequently ignore human feelings or even suppress them. 
But the work process is full of emotion. Enthusiasm, rage, fear, jealousy, pride, ambi¬ 
tion, hatred, decisiveness, fury, and delight are all possible emotions generated by the 
prospect of a successfully completed project. Negative emotions support “work-to-rules“ 
tendencies, positive emotions promote willingness to perform, productivity, quality, and 
finally customer satisfaction. It is important for companies to motivate their employees. 
To obtain this motivation, stimuli are required that are directly connected with custom¬ 
ers’ wishes, independent of the size of the office. Whether the cyber enterprise possesses 
better chances to develop its abilities in this area through more flexible structures has yet 
to be shown. We must remain aware that the customer will accept not everything that is 
technically practicable. 

^ 7 . 

The integration of human thinking, acting, and decision processes is one of the greatest 
challenges for the future of multimedia. Neither nature nor human senses utilize digit- 
ized“ information. Nearly everything is “analog^. It is not important to man that human 
feelings and sensual thrill such as smelling, tasting, viewing, listening, and loving can be 
implemented digitally. The effort of scientists to understand the human soul as an infor¬ 
mation processing system and to model it using mathematical equations may satisfy their 
ambition, but is of little help in eliminating human conflicts and contradictions of the 
presence. Not only the increasing number of religious, nationalistic, and racist wars sup¬ 
port this thesis, but also the inherent conflicts in everyday human life. The new ideology 
of formalizing, digitizing, and generaUy making everything technically controllable 
sweeps the world, while defining power under the cover of multimedia democracy and 
plurality. Man can as such become dependent on new artificial/ technical worlds to re¬ 
place the natural environment, forcing him to rely on new powers such as Bill Gates, Leo 
B^rch, Silvio Berlusconi and others. Along this line critics point out that Multimedia can 
lead to vicarious satisfaction and virtual reality without reference to the real world. 

The resulting consequences of Multimedia on mankind and the possible increase of cul¬ 
tural differences cannot be predicted today. Human relations through electronic channels 
will serve to change emotions and an effect, which we may call “vutualization“. Prede¬ 
cessors of “virtual children“ are already available as toys and for those who do not want 
real children. A proof of this is the “Tamagochi“-cult from Japan. Its purpose is to feed 
and nurse an artificial computer egg like a living creature. The development and imple¬ 
mentation of “soul-machines“ as described in the previously cited article “Emo lebt“ 
(Spiegel Spezial, 1996b) documents a possible trend. This trend makes me very thought¬ 
ful, it states that parents sooner or later may be satisfied with virtual children instead of 
real children. The blessings of Multimedia are Janus-faced. Even supporters of Multime¬ 
dia must concede that in the end virtual conference dinners have no taste. 

Change in Values by Multimedia 

The transformation of our society into an information-based society requires not only 
state or private regulations worldwide, but also self-obligation and responsibility. The 
development of Multimedia technology will essentially change political systems in all 
countries. This has already been seen by the influence of media on the breakdown of 
communism. Consequently we are obliged to support reforming nations and not con¬ 
struct new electronic walls. Many new ethic and moral questions must be investigated 
due to this ongoing virtualization. 

Because of conflict caused by lost values, changes of values, and inflation of values, per¬ 
sonal attitudes and ways of thinking gain increasing weight. This creates great new chal¬ 
lenges for politicians to establish regulations reaching much further than traditional laws 
which take morality absolutely seriously. The contradiction on all levels is that we mean 
well and hardly change our doings. 

Using new information and communication technologies, people must be taught new 
aims for their life, learning, and work. Nobody’s access to information must be hindered 
by another’s monopoly on information. In conjunction with the increasing erosion of 
states’ finances in all countries, a collapse would be inevitable because the planned de¬ 
velopments could not be paid for. In this scenario new workplaces would not develop and 

social peace could not exist. The unity between nations would not be promoted but, as 
presently described by many authors, countries would be further differentiated in a global 
competition. In this case the technological revolution caused by Multimedia and data 
highways could become “man-eating“ one day and eliminate their own children. This 
could happen in a way that we cannot foresee, as we could not foresee the catastrophes of 
the 20^ century. On the other hand, the technological revolution could also produce new 
values promoting living or working together, international responsibility, faimly or tradi¬ 
tional values, and regional affiliation in the best sense. An important question will be 
whether that the habits of man will change as quickly as media technologies and how 
many people will take part in this process, A pessimistic view on the development of the 
century will not help to make the best out of the inevitable evolution. For that reason 
all business and management relevant questions should be viewed with critical optimism. 


This contribution was developed from many personal talks, presentations, and discus¬ 
sions, the author conducted during the last years and on the basis of entrepreneural deci¬ 
sions he achieved. 

I am greatly obliged for the support given by Wolf D. Hartmann and my daughter 
Cornelia Steilmann of the Klaus Steilmann Institute for Innovation and Environment, 
Inc., Bochum-Wattenscheid and of the Witten-Herdecke University. I would like to ex¬ 
press special thanks to Heinz D. Haustein of the Institute for Innovation Management, 
Berlin and to Klaus-Peter Holzhausen of the Fachhochschule Bochum, University of 
Applied Sciences, Bochum, for their contributions and critical support. 


Auswirkungen neuer Informations- und Kommunikationstechnologien auf die 
Beschdftigungssituation von Frauen 
Institut fur Arbeit und Technik, 

August 1997 

Brddner, Peter. 

Der iiberlistete Odysseus 
Rainer Bohn Verlag, Berlin, 1997 

Edition 2, 02/09/1996 


Die grofie Hojfnung des 20. Jahrhunderts 
Kbln-Deutz, 1954 

Grabner, G.; Lang, W., 

Interaktive elektronische Systeme (IBS) in den Phasen des Marktes 
in: Hermanns, A.; Flegel, V. (Hrsg.) 

Handbuch des Electronic Marketing 
(Pages 761-775) 

Beck Verlag, Miinchen, 1992 

Holzhausen, Klaus-Peter (Ed.). 

Multimedia and Simulation, Human Machine Interface Implications. 

Proceedings of the 1997 Annual Conference of the Europe Chapter of the Human Factors 
and Ergonomics Society, Bochum, Germany. 

Fachhochschule Bochum, University of Applied Sciences, 1997. 

Hermanns, A.; Suckrow, C.. 

Aufbruch in das Multimedia-Zeitalter im Marketing 
in: Management & Computer 
Edition 2/1993 
(Pages 105-112) 

Hofling, S.. 

Informationszeitalter, Informationsgesellschaft- Wissensgesellschaft 
(Page 8) 

Hans-Seidel-Stiftung e. V., 1996 

Martin, James, 


(Page 35 and following pages), 1997 

Martin, James. 


(Page 67), 1997 

Riem, Ulrich; Wingert, Bemd. 

Multimedia, Mythen, Chancen und Herausforderungen 
Mannheim, 1996 

Spiegel Spezial 
Die Multimediazukunft 
Edition 3/1996 
(Page 21) 

Spiegel Spezial 
Die Multimediazukunft 
Edition 3/1996 
(Page 66) 

Stippel, P,. 

Multimedia! Was dndert sich im Marketing? 
in: Absatzwirtschaft 
Edition 6/1992 
(Pages 32-45) 

The paper w?as first published at the Club of Rome Conference, Smithonian Institution, 
October 23-25, 1997, Washington, D.C. 

Air Traffic Control Simulation and the Human Operator 

DipL-Ing. Ralf Beyer 
German Aerospace Research Establishment (DLR) 
Institute of Flight Guidance 
D-38108 Braunschweig 
Lilienthalplatz 7 
Fed, Rep, of Germany 


Mobility is one of the great societal demands and air traffic is one mode of transport to 
serve this need. Substantial growth rates are predicted for future air traffic but the natu¬ 
ral resources of real-estate, airspace, and human abilities required to accommodate much 
more air traffic become a limiting factor. A better utilization of given resources is 
needed, therefore, to increase air traffic capacity. One attempt in this direction is a more 
efficient air traffic management by the introduction of more automation and a better task 
sharing between the automated system elements and the human operator. A reconsidera¬ 
tion of the role of the human operator, the develop-ment of an appropriate automation 
strategy and appropriate support tools, and the demonstration of concepts for a future air 
traffic management system are essential steps to achieve the goal. In this context the 
particular aspects of air traffic control simulation including human operators for the test 
and demonstration of air traffic control concepts are discussed and recent experiences are 

1. Introduction 

Simulation in air traffic management research is one of the most used tools to achieve 
advances in air traffic management in Europe and world-wide. Different types of simu¬ 
lation exist. The fast-time simulation of air traffic, for instance, is an efficient means to 
assess future airspace and airport capacities, the expected flow of air traffic, and the de¬ 
lay of aircraft. Other topics are investigated best, however, in a real-time simulation of 
the respective air traffic scenario. These topics include - among others - the expected 
future demands on air traffic controllers and pilots or the performance and acceptability 
of advanced support tools onboard an aircraft and on the ground. Other areas are the use 
of a future air/ground data link and new procedures to control future air traffic more effi¬ 
ciently, Real-time simulation is the right choice in particular if human operators (air 
traffic controllers, pilots) take part in the simulation. 

In the past these simulations have provided administrations, airport authorities, air traffic 
control authorities, and airlines with a realistic view and an interactive experience of fu¬ 
ture air traffic management tools and procedures. Experiences have shown that the 
availability of a high-performance real-time simulation not only is an efficient means for 
the design, development, assessment, and demonstration of future air traffic management 
and flight guidance systems and procedures. It is also a prerequisite for a successful ac¬ 
quisition of attractive new research contracts in air traffic management regarding auto¬ 
mation and the future role of the human operator in air traffic management. 

Air traffic control (ATC) is that element of air traffic management (ATM) which exe¬ 
cutes control over aircraft en~route and in the terminal area of airports. In the following 
recent experiences in ATC simulation with particular respect to the human operator are 



2. Current role of human factors/engineering in ATC 

As more and more functions of the human operator can be implemented by machines 
today the question is not how much can be automated but what degree of automation is 
acceptable responsibly. This, in turn, determines the role of the human operator and of 
its interfaces to the automated system. For instance, the Concept Document for the fu¬ 
ture European Air Traffic Management System (EATMS) lists as one of the principles 
for its development (EUROCONTROL, 1992): 

„The concept introduces new methods of task sharing between the 
automated system elements and the human operator. That devel¬ 
opment shall be carefully balanced in particular in relation to the 
responsibilities for traffic monitoring and decision making; this 
may set limits to the degree of automation. The man/machine in¬ 
terface is a key element of successful application of automation 
and of overall system performance,“ 

The forecast of the potential future of the US Air Force (USAF, 1995) also lists among 
six identified essential capabilities „people“ which includes things like modeling of the 
human operator, training, education, and human/machine interaction. 

This much improved role of „people“ and associated designs and developments of hu¬ 
man/machine interactions and interfaces in comparison to the situation some years back 
cannot be overemphasized in times of severe cuts in funding and rearrangements of de¬ 
velopment priorities. 

3. Some fundamental considerations 

Human error will come, regardless of the quality of design and training. Suppressing 
human error is seen as a naive goal. A better approach would be to correct system ergo¬ 
nomics but keep the human in the loop (Amalberti, 1997). In recognition of this situa¬ 
tion aviation research organizations, air traffic services and airport authorities are in¬ 
creasingly interested in real-time simulations and demonstrations of new developments in 
air traffic control automation before any final decision on their possible introduction is 
made. Questions of expected system performance, human operator workload and ac¬ 
ceptance of a new system are to be answered. 

Recent reviews and studies on the human operator and automation in ATC made or con¬ 
tracted by DLR are reported by Beyer (1996). According to the findings of Lenorovitz, 
Olason, Krois, and Tobey (1991) and Jackson (1989) the tasks of air traffic controllers 
can be summarized as follows: 






what is the state of air traffic ? 
what happens and why ? 
what options are available ? 
what option is optimum ? 
what implements an option best ? 

A major issue of ATC automation is the transfer of cognitive functions from the human 
operator to machines. This approach to automation in ATC may lead to two self- 


contained lanes of functionality: one is present in the human operator and the other is 
implemented by the machine. Consequences are: 

• From a functional and procedural point of view the machine must 
be able to handle the tactical control of air traffic and the strategic 
management of the flow of air traffic automatically. 

• From an operational point of view there would be a partnership 
between the human operator and the machine: The human operator 
decides what type of machine support suits his momentary and 
foreseeable needs best and what tasks he can delegate to the ma¬ 
chine for the time being. 

This means in short: Automate as much as technically feasible but utilize machine func¬ 
tions only as far as responsibly acceptable. By this approach the human operator will be 
provided with a suite of tools to select from according to his needs and also with a 
chance to tune the tools in a way that their pace is matched to that of the human operator. 
Full authority is assumed by the human operator initially when new functions are intro¬ 
duced but this may change when the human operator gains better insight and more con¬ 
fidence into the new machine functions. 

4 . DLR^s ATC simulation suite 

The main target systems for air traffic management research and development are air¬ 
ports, aircraft, the airspace, and the air traffic. For obvious reasons it is next to impossi¬ 
ble to utilize some of the target systems directly as an experimental platform to investi¬ 
gate new concepts of human/machine interaction in ATC and to evaluate new products 
and procedures. In 1970, therefore, the DLR began to establish virtual worlds of air traf¬ 
fic control and aircraft guidance by means of a simulation of these target systems with a 
high degree of realism. Major elements of the simulation are: 

Airspace and air traffic simulation 

A data base provides the infrastructure of the airspace of the Federal Republic of Ger¬ 
many and in particular of the extended terminal area of the airport of Frankfurt/Main. 
The data base includes all navigation data defined by ARINC specification 424, aircraft 
performance and weight data for a number of aircraft types as well as several different 
wind profiles. An air traffic generator uses the data base to create air traffic scenarios 
with a selectable mix of aircraft types and speeds, routes, wind conditions, departures, 
and destinations. These scenarios can be run repetitively to investigate the influence of 
new air traffic structures, operator support tools and operational procedures on the flow 
of air traffic, the resulting air transport capacity, and the workload of air traffic control¬ 

Air traffic control simulation 

The DLR Air Traffic Management and Operations Simulator (ATMOS) employs the air¬ 
space and air traffic data bases and the air traffic scenarios created by the air traffic gen¬ 
erator to run the scenario and to feed the radar consoles of the air traffic controllers with 
realistically looking air traffic (fig. 1). ATMOS also accommodates new support tools 
for air traffic controllers to be investigated in a simulated operational scenario. A group 
of so-called pseudo-pilots (fig. 2) monitors the simulated aircraft, communicates with the 
air traffic controllers by voice or data link and implements advisories received from air 


traffic controllers by a modification of the state of aircraft under control. The air traffic 
controllers in turn monitor the aircraft reaction to a given advisory on their radar screens 
thereby closing the ATC control loop. 

Simulation of the airport of Frankfurt/Main 

A visual simulation of the airport of Frankfurt/Main provides views as seen from the 
working position of the tower controllers as well as from the working position of a pilot 
approaching at / departing from and taxiing on the airport of Frankfurt, The simulation 
includes the infrastructure of the airport (passenger buildings, hangars, navigation aids, 
runways, taxiways, etc.) as well as moving objects (approaching/ taxiing/departing air¬ 
craft, support vehicles). This photo-realistic representation of the airport is visualized by 
a 3-channel backprojection system. The three screens have a size of about 4x4 meters 
and provide a view of about 150 degrees horizontally by 40 degrees vertically. 

The DLR test aircraft 

The DLR operates the jet test aircraft ATT AS (Advanced Technologies Testing Aircraft 
System) with a rear experimental cockpit and a bi-directional air/ground data hnk. The 
test aircraft carries advanced avionic systems which co-operate with ATC in order to 
develop and negotiate an optimum space and time based flight path (4D trajectory) and 
to exchange situation information, sensor data, and planning information between the 
aircraft and the ground. 

5. Application ofDLR's ATC simulation suite 

The views, concepts, and tools described in the previous chapters form the background 
of research at DLR in air traffic control with particular emphasis on the role and integra¬ 
tion of the human operator in a future ATC system. A typical application is presented 

First, a more general remark on ATC simulations in a European context: The simulation 
of a future European ATC system lives (and sometimes suffers) from the fact that a dif¬ 
ferent education, training, and cultural background of air traffic controllers must be con¬ 
sidered. Air traffic controller teams participating in ATC simulations are selected by 
their national authorities and are made available for a very limited period of time only at 
the place of the ATC simulation. No homogeneous nor representative group of test sub¬ 
jects can be expected. An attempt is made, therefore, to reduce differences within and 
between the groups of test subjects by intensive instruction, training, and demonstration 
of the new ATC scenarios and support tools to be investigated. This requires a substan¬ 
tial effort and consumes a major part of the time budget available for an experimental 

5J An example: the PHARE demonstration PD/2 

A successful application of an ATC simulation including human operators was run 
within the Programme for Harmonized Air Traffic Management Research in 
EUROCONTROL (PHARE). The objective of PHAM is to organize, co-ordinate and 
conduct studies and experiments aimed at providing and demonstrating the feasibility 
and merits of a future air-ground integrated air traffic management system in all phases 
of flight. The results of the program should help to refine the description of the future 
Air Traffic System concepts needed to satisfy demand and to provide information on the 


best transition from the current to a new system (EURCKTONTROL, 1997). The program 
is currently planned to extend over a period from 1989 to 1998 with an overall cost of 
about 91 MECU. Participants are DERA and NATS in the UK, CENA and STNA in 
France, DFS and DLR in Germany, RLD/LVB and NLR in The Netherlands, and 
EUROCONTROL. DLR was in charge of the second PHARE Demonstration (PD/2) 
which focused on advanced tools for the management and control of arriving air traffic 
in the extended terminal area of an airport. 

5.2 Experimental set-up of PD/2 

The tools investigated in PD/2 were designed to transfer cognitive and planning func¬ 
tions of air traffic controllers to machines in an attempt to reduce the workload in a fu¬ 
ture air traffic scenario with much higher traffic volume than today (Schick and Tenoort, 
1997). This approach to more efficient man/machine systems in ATC addresses the fol¬ 
lowing general topics: 

• time-based tactical and strategic control of air traffic in the ex¬ 
tended terminal area of an airport, 

• support of the human operator by „intelligent“ machines in con¬ 
trolling arriving air traffic, 

• delegation of tasks to machines by the human operator while 
maintaining ultimate responsibility. 

An important assumption was that in the future more and more aircraft will be able to 
calculate and to implement space and time based flight trajectories onboard which fulfill 
the space and time constraints of a computer-based manager of arriving traffic on the 
ground. Future aircraft will be able to negotiate these trajectories with the arrival man¬ 
ager via an air/ground data link. Aircraft with this capability are called 4D-equipped 
aircraft. In PD/2 the ATTAS with its Experimental Flight Management System repre¬ 
sented such a 4D-equipped aircraft. 

The objectives of the experiment were: 

• Determine the effect of various automated support tools on air traf¬ 
fic controller workload and arriving air traffic flow. 

• Determine the effect of 4D-equipped aircraft on air traffic con¬ 
troller workload and arriving air traffic flow. 

• Determine the degree of air traffic controller acceptance of new 
automated support tools. 

Four test organizations were set-up as follows: 

1. A reference operating mode representing current tools and proce¬ 
dures with spatial separation of arriving aircraft (OrgO). 

2. An advanced operating mode with automatic space and time (4D) 
based planning of aircraft trajectories, detection/resolution of 
planning conflicts on the ground, and a novel human/machine in¬ 
terface (Orgl), 

3. Same as Orgl but with 30% 4D-equipped aircraft of all arriving 
aircraft (Org2/30). 


4. Same as Org2/30 but with 70% 4D-equipped aircraft of all arriv¬ 
ing aircraft (Org2/70), 

Each of the organizations was run with a medium and a high traffic load. The term , 4 iie- 
dium“ represented today’s traffic demand while „high“ corresponded to about 33% more 
traffic demand than ,^edium“. Data was gathered at four different air traffic controller 
working positions (ACC tactical, Approach Pickup/Planner/Feeder). 

Eight air traffic controller teams each consisting of 4 controllers from 7 European nations 
(France, Germany, Italy, The Netherlands, Romania, Sweden, United Kingdom) partici¬ 
pated in the investigations. 

Eight test runs per team - balanced by order - were recorded and analyzed. These in¬ 
cluded runs with medium and high traffic load for each of the four organizations OrgO to 
C)rg2/70. Quantitative statistical methods and non-parametric statistics based on 
matched pairs of observation were employed for data analysis. 

5J Data acquisition in PD/2 

Data was gathered in the following categories: 

- System performance 

Number of landings, flight time, inbound delays, precision of de¬ 
livery and flight accuracy, and aircraft separation. 

- Air traffic controller workload 

Objective workload indicators included the number of ATC in¬ 
structions issued, frequency of radio communications, and per¬ 
centage of time spent for radio communication. 

The subjective workload estimates were obtained by the Subjective 
Workload Assessment Technique (SWAT) and by the NASA Task 
Load Index (TLX). 

- Acceptance 

The air traffic controller acceptance of the simulation environment 
and training, human/machine interfaces, operational procedures, 
tools and functions was assessed by post-run debriefing interviews 
and questionnaires. 

In addition all experimental runs were video-taped. 

5,4 Representative results of PD/2 
General findings 

The results obtained showed that all teams were highly qualified and motivated, that they 
accepted their role in the experiment to test a simulated and not a pre-operational ATC 
system, and that they passed their training successfully and were able to participate in all 
phases of the experiment without problems. The teams rated the set-up of the experiment 
and the trials as being realistic and valid. 

System performance 

The results obtained showed that a medium traffic load could be handled almost equally 
well for all organizations Qrg0-0rg2/70. Under high traffic load, however, it was imme¬ 
diately evident that OrgO reached its limits and that air traffic was handled much more 


smoothly with the more advanced organizations Orgl-Orgl. While in QrgO about 62 
landings/hour were measured this value increased to about 64 landings/hour for Orgl- 
Org2 for the same traffic demand (for comparison: landing one B747 aircraft more per 
hour (average) at Frankfurt/Main in reality could result in an annual profit of about 10 
Mio. DM). Furthermore the variability of the number of landings per hour as well as of 
most other performance parameters like flight time, inbound delays, precision of deliv¬ 
ery, and aircraft separation was considerably reduced for the more advanced organiza¬ 
tions. This is because the advanced support tools provide a much improved predictabil¬ 
ity of the expected air traffic flow. 

Air traffic controller workload 

The objective workload indicators showed a high correlation with each other and a very 
uniform pattern of results. They indicated a workload reduction or no significant change 
of workload for the least advanced organization Orgl in comparison to the reference or¬ 
ganization OrgO for the air traffic controller working position with the highest load 
(Pickup). However, for the more advanced organizations Org2/30 and Org2/70 a signifi¬ 
cant reduction of workload was measured for all air traffic controller working positions. 

The subjective workload estimates collected by means of SWAT and TLX also correlated 
well with each other. The Pickup controllers rated their own workload considerably 
lower when they worked under Orgl in comparison to OrgO. The Feeder controllers did 
not indicate any difference whereas the ACC West controllers experienced a bit more 
workload when changing from OrgO to Orgl. However, the more advanced organiza¬ 
tions Org2/30 and Org2/70 led to a reduction of perceived workload for all controller 
working positions. 

The objective as well as the subjective measures of workload showed the same trend in 
workload reduction for the more advanced organizations together with some re¬ 
distribution of workload between the Approach and ACC working positions. But be¬ 
cause of the low overall workload level of the ACC working position the slightly greater 
workload for Orgl and Org2 was considered to be within acceptable limits. 


The human/machine interface characteristics (mouse as input device, availability of pop¬ 
up menus, display design such as color coding, display options like highlighting a se¬ 
lected trajectory) received a high degree of controller approval. Most operational aspects 
(support for safe traffic handling and „maintaining the picture“, concept of handling 4D- 
equipped and non-equipped aircraft, partitioning of tasks between working positions) 
were also very well accepted. 

From the results obtained by this experiment it was concluded that the concept of the 
more advanced organizations to control arriving traffic at an airport has proven its suit¬ 
ability. The concept has shown some potential to improve air traffic throughput and 
quality of service. Improvements can be achieved at acceptable levels of air traffic con¬ 
troller workload. Although the concept was approved by the participating air traffic 
controllers there is room for further improvements of the automated support tools. 


6. Further human/machine interface design experiences and observations 
The PHARE demonstration was only one of a number of experimental investiga¬ 
tions in ATC at DLR with particular respect to the human operator. Further ob¬ 
servations in similar experimental investigations showed that: 

• Concerns were raised that better automation tools designed to enhance 
the performance of air traffic controllers may take away interesting 
tasks from the human operator he/she is able and willing to perform. 

The new tools may render certain jobs redundant. 

• New monitoring tools provided information normally not available to 
the air traffic controller which was considered sensitive by the ATC 
authority. Selective access mechanisms had to be designed into the 
tools to control the availability of information to different parties. 

• A new monitoring tool documented encounters of the legal operating 
limits too close sometimes when air traffic controllers tried their best 
to resolve a demanding situation. This type of documentation was not 
available in the past. Adverse effects on human operator motivation 
and performance were expected. 

• When air traffic controllers were provided with the opportunity to se¬ 
lect alternative colors/color schemes for a graphical user interface, 
only a small amount of experimentation with colors was observed. 

The air traffic controllers stayed with the color scheme offered by the 

• In one case the color convergence and the focus of the monitors were 
not optimum. Therefore a light blue background of the screens was 
chosen to avoid too many different non-convergent colors and to 
achieve a certain level of apparent sharpness of the image. Knowing 
the background of this decision the air traffic controllers accepted the 
design very well. 

• A color scheme for ATC consoles developed with great effort and care 
by one European nation and accepted by their national air traffic con¬ 
trollers did not find the same degree of acceptance by air traffic con¬ 
trollers of other European nations. 

These and similar observations and experiences are going to be incorporated into human 
factors design guides for ATC systems. An example of such guide is the documentation 
„Human Factors in the Design and Evaluation of Air Traffic Control Systems^ by Car- 
dosi and Murphy (1995). 

7, Conclusions 

Despite the progress of automation in air traffic management and control, the role of the 
human operator in arriving at safe, economic, and acceptable ATC systems of high per¬ 
formance gains increasing importance. This is accompanied by a growing demand of 
real-time ATC simulations to test and to demonstrate the benefits and acceptance of new 
automated support tools with particular respect to the role and the work of air traffic 
controllers before a final decision about their possible introduction into operational use is 
made. Experience has shown that human factors/engineering knowledge, an advanced 
automation strategy and a real-time ATC simulation in combination are a sound basis to 


Figure 1: DLR Air Traffic Management and Operations Simulator (ATMOS). 

Figure 2: ATMOS pseudo-pilot working positions. 


8, References 

Amalberti, R. (1997) When Human Errors Serve Safety Goals. CSERIAC Gateway 
Vol. Vn, No. 3, 1997. 

ARINC (1987) ARINC Specification 424-7 Navigation System Data Base. Aeronautical 
Radio Inc., Annapolis, USA, 1987. 

Beyer, R. (1996) Integral Design of the Human/Machine Interface. Proceedings of the 
38th Conference of the Human Factors Group of the German Aerospace Society 
(DGLR), Braunschweig, Germany, 15-16 October 1996, DGLRNo. 96-02. 

Cardosi, K. and Murphy, E. (Eds.) (1995) Human Factors in the Design and Evaluation 
of Air Traffic Control Systems. Washington: Federal Aviation Administration. 

EUROCONTROL (1992) European Air Traffic Management System. Issue 1.1. 
EUROCONTROL Agency, June 1992. 

EUROCONTROL (1997) Programme for Harmonised Air Traffic Management Re¬ 
search in EUROCONTROL. EUROCONTROL Agency, May 1997. 

Jackson, A. (1989) The Role of the Controller in Future ATC Systems with Enhanced 
Information Processing Capability. EUROCONTROL, EEC Report No. 224, 
June 1989. 

Lenorovitz, D. R., Olason S. C,, Krois, P. A., and Tobey, W. K. (1991) Customizing the 
ATC computer-human interface via the use of controller preference sets. In R. 
S. Jensen (Ed.), "Proceedings of the Sixth International Symposium on Aviation 
Psychology" (pp. 454-459). Columbus, OH: Ohio State University. 

Schick, F. and Tenoort, S. (1997) PD/2 Results. Proceedings of the 10th Scientific 
Seminar of the DLR Institute of Flight Guidance, 23-24 June 1997, pp. 10-1 - 10- 
12. Braunschweig, Germany: German Aerospace Research Establishment (DLR). 

USAF Scientific Advisory Board (1995) New World Vistas - Air and Space Power for 
the 21 St Century. US Air Force, 15 December 1995. 


Horses for Courses - Simulation in Air Traffic Control 

Hugh David 
Eurocontrol Experimental Centre 
BP 15, Bretigny-sur-Orge 
F 91222 France CEDEX 


Simulation is the traditional method for the development of Air Traffic Control (ATC) 
systems. The Eurocontrol Experimental Centre houses an extremely flexible and 
advanced ‘Real-Time’ simulator, 'which is employed for the operational evaluation of 
developments in specific ATC systems, and for the development of generic future 
systems (ODID, PHARE etc.). Real-time simulations are expensive (about 100,000 ECU 
per hour), require skilled staff and participant controllers (up to 50 controllers and 20 
staff) and take about a year to prepare. Faster and cheaper alternatives have therefore 
been developed. ‘Fast-time’ simulation uses a computer-based simulation, where the 
human controller is replaced by pre-defined algorithms and data values, and exists at 
various levels of detail (RAMS, EAM, SIMMOD). A single-person Real-Time 
simulator, (TRACON/Pro) with Al-based synthetic ‘pilots’ and speech generation and 
recognition is used for the evaluation of measuring tools. ‘Rapid Prototyping’ is used for 
the development and initial investigation of new displays, ‘Part-task’ simulations which 
investigate specific aspects of human performance are used to provide data on the speed 
and accuracy of human operators in ATC-related tasks. These data are used in Fast-Time 
simulation, and in operator-oriented simulations using generic and special-purpose 
Micro-SAINT models, to investigate generic changes to existing methods - use of 
computer-based inter-sector transfers, or multi-sector planning. Finally, small simplified 
models are developed on PC-compatibles to demonstrate more radical approaches to 
ATC. (DEMON and DEMFAST, for example, show the effects of a radical re-structuring 
of the ATC system using data-links, computer-based conflict monitoring and pre¬ 
evaluation of conflict resolution strategies). 


Eurocontrol is the European Organisation for the Security of Air Navigation. It is an 
international organisation established by treaty, to which 24 nations are currently 
signatories. The purpose of the organisation is to improve Air Traffic Control procedures 
throughout Europe. 

The Eurocontrol Experimental Centre, at Bretigny-sur-Orge, near Paris, is concerned 
with the evaluation and development of ATC systems, and, to a lesser extent, with 
fundamental research into Air Traffic Control. Simulation has traditionally been, and 
remains, the major tool for these activities. The ways in which simulation is employed 
have diversified in recent years, so that the Centre now provides an instructive showcase 
for simulation methods for the investigation of complex multi-agent dynamic control 

“Horses for courses” is a traditional English proverb, meaning, literally, that you should 
choose a horse which suits the type of race you are riding in. In general, you should 

choose a tool that fits the task you are undertaking. In particular, in ATC research, you 
should choose the type of simulation method to suit the question you are asking. 

Real-Time Simulation 

The Eurocontrol Experimental Centre was originaUy devoted to, and literally built 
around, a Real-Time simulator. When this was originally built in 1967, it was the first 
digital Real-Time simulator in the world, based on a Telefunken TR4 digital computer. It 
was then, and probably remains, the only large simulator which is not tied to a particular 
real system. Each of the main components of the simulator: computer; piloting 
equipment; control room; video and recording systems has been replaced several times in 
the last thirty years, but, like Jack Hobbs’s cricket bat, after three new handles and five 
new blades it is just as good as new. The current simulator is capable of running two 
large-scale simulations simultaneously. Simulated control rooms are constructed for 
each simulation, corresponding to the physical layout of the existing or proposed future 
system. Up to 40 control positions can be simulated simultaneously, representing up to a 
dozen sectors in one or more ATC centres. These control positions resemble the existing 
positions, providing similar functions to the controller. Strip bays and strip printers can 
be provided where needed, although Electronic Data Displays are increasingly used. 
Monochrome or colour-coded radar displays can be provided, using various sizes of 
display up to 20-inch (50 cm ) square raster-scan displays showing 2,000 by 2000 pixel 
images in full colour. Additional Teed’ sectors are manned by EEC Controllers, who 
simulate the adjacent and underlying sectors. One or more ‘pilot-operators simulate the 
pilots of the aircraft in each sector. These pilot-operators, some of whom are qualified 
pilots in reality, translate the controller’s orders into instructions to the computer, and 
speak the computer-generated messages (position reports, etc.) to the controllers. Several 
hundred aircraft may be simulated simultaneously. Simulated RTF and ground 
communications are provided and can be recorded. Where needed, data-links can be 
simulated. Elaborate video-recording equipment is available, and a special system, Ergo- 
ISA (Instantaneous Stress Analysis) (Hering and Coatleven, 1996) can be used to provide 
a virtually ‘on-line’ self-assessment of controller strain from each working position, 
displayed at the central simulation control position. 

By any standards, this is a large system. The major costs lie, not in the hardware, but in 
the software of the system. Each Real-Time simulation consists of ten to forty exercises, 
each lasting one to two hours, spread over three to six weeks. For each simulation, route 
structures, sector layouts and traffic must be defined. Specific software functions must be 
programmed and communications and other links prepared and checked, (Our previous 
chief programmer, retiring after 25 years’ experience, told me that he had never received 
a specification which did not have at least one major omission, sufficient to make the 
system inoperable.) The programming of a large, distributed, Real-Time simulation of 
this sort is very specialised skill, and is only slightly less complex than programming a 
real system. Even when the software is as close to complete as can be managed, the data 
representing the traffic must be prepared with extreme care, since controllers are very 
sensitive to abnormal traffic, such as heavy aircraft programmed to land on grass 
airfields. Apparently trivial inconsistencies can greatly affect the willingness of 
controllers to accept the realism of the simulation. About fifteen skilled staff are involved 
in the programming of current simulations and anticipating future requirements. A 
further ten are employed in collecting and preparing traffic data. (This is never entirely 


routine, even when a simulation is concerned with current and near-future traffic. When 
‘future systems’ are being simulated, it is particularly difficult.) Each simulation is 
controlled by an experienced project leader, who is an experienced controller, with an 
assistant. Although the time-scale for a Real-Time simulation is about twelve to eighteen 
months, and we normally run about ten simulations a year, most project leaders are 
involved in several simulations at the same time,preparing one simulation, managing a 
second and reporting another. If we include the staff involved in procuring and 
maintaining equipment, in analysing the 50 megabytes of data from a routine simulation, 
producing training and briefing programs and documents and interim and final reports, 
finding accomodation for visiting controllers and for management visitors during the 
simulation, about a hundred staff are involved in the running of the Real-Time simulator. 

A Real-Time simulator exercise is very expensive. If a technician fails to switch on the 
recording system for a ninety-minute exercise, it costs the equivalent of totally destroying 
the Centre’s two best offical cars. 

A Real-Time simulation is not undertaken lightly. The yearly programs are negotiated 
several years in advance. It takes about six to nine months to prepare a simulation, one or 
two months to run it and three or four to analyse it, although the major results are 
usually obvious during the running, and the customer often puts them into application 
without waiting for the formal report. 

Real-time simulation is best used as a final verification of a well-researched system, or 
for revisions of the airspace of an existing system. It is, however, also used to investigate 
potential future systems. 

Two major series of simulations have been carried out at EEC to investigate some 
significantly different systems. The first series of simulations (ODID - Operational 
Displays and Input Devices) took place in 1987 (ODID I, Prosser and David, 1988a), 
1988 (ODID n,Prosser and David,1988b), 1990-1991 (ODID m,Prosser et al 1991) and 
1993 (ODID IV,Graham et al, 1991). These studied ways of presentation increasingly 
remote from the traditional. ODID IV, in fact, employed a widowing system using 
several innovative forms of display, and radar labels that could be expanded to provide 
full flight plans, and allowed the controller to record his instructions to the aircraft 
directly through the label. A great deal of attention was paid to the detailed ergonomic 
design of the display, which led to general recommendations (Jackson and Pichancourt, 
1995), the use of colour being based, for example, on the recommendations of Reynolds 
and Metcalfe (1992). 

The second set of simulations (PHARE - Program for Harmonised ATC Research in 
Europe ) began with PD/1 (Autumn 1995), has now reached PD/2 (Autumn 1996), and 
will culminate in PD/3 (Spring 1998). These simulations involve radical innovations, 
such as the linking of Real-Time simulators in different member states, and data-link 
transmissions to real aircraft in flight. The financing, organisation and administration of 
such simulations involve tremendous efforts, and take a very long time. 

From a research point-of-view, Real-Time simulation has other drawbacks. It is rarely 
possible to repeat an exercise, since the human variations introduced by the feed sector 
controllers and pilots introduce variations in the exact sequences of traffic, leading 

controllers to adopt different solutions to problems, so that the repeat exercise diverges 
from the original. Even where a close repetition is possible, the controller, being human, 
remembers his previous actions, particularly if he made mistakes, and does better the 
second time round. Comparisons between controllers are difficult, since each controller 
has a repertoire of preferred tactics, based on his working experience. Controllers tend to 
judge a system as a whole, so that a technically minor flaw may lead them to reject an 
otherwise promising system out of hand. On the other hand they often display a can-do 
attitude, finding ways around flaws in official procedures, and sharing duties within a 
sector so that expected overloads do not happen. Admirable as this trait may be in real- 
life, it can totally distort a carefully planned simulation. Finally, however much effort is 
devoted to training, controllers continue learning throughout a simulation. (In real life, it 
takes a controller up to six months to ieam’ a new sector - in simulation he rarely 
occupies the same position for as much as a week). Properly balanced experimental 
designs are extremely hard to achieve. 

Fast-Time Simulation 

Historically, the first alternative to Real-Time simulation was Fast-Time simulation. This 
technique is purely computer-based, and uses essentially the same data generation and 
analysis procedures, but replaces the actual simulation by a computer-based simulation. 
While the software of the Real-Time simulator calculates and presents successive images 
with a fixed cycle time (usually 5 or ten seconds), a Fast-Time simulator maintains a 
store of future events, and carries then out in succession as fast as it can, advancing its 
‘internal clock’ accordingly. If there is not much traffic, or if the level of detail required 
is low, this process may run much faster than Real-Time. On other occasions, ‘Fast- 
Time’ may run slower than ‘Real-Time’. The original Eurocontrol Fast-Time model 
(EAM) has been through almost as many modifications and revisions as the Real-Time 
simulator, and its current avatar, called RAMS (Revised Arithmetic Simulator), is 
designed to share data generation and analysis software with the Real-Time system. 
Since human intervention is essentially eliminated, repeated runs of the same exercise 
should give identical results. Estimates of the probable workloads on controllers can be 
obtained, using a post-processor, which allocated pre-defined quantities of effort or 
working time for specific events. Fast-time simulation is inherently cheaper and more 
consistent than Real-Time simulation, but it lacks the error-detecting facility and self- 
awareness of Real-Time simulation. Where a controller in Real-Time simulation will 
stream aircraft so that potential conflicts do not happen, the Fast-time simulator will 
‘solve’ individual conflicts as they occur, with no strategic insight. 

Practically, Fast-Time simulations take nearly as long as Real-Time simulations, since 
the preparation and analysis phases are essentially similar. More exercises can be carried 
out, since a few hours, or days, of computer time are now cheap, and simple methods can 
be developed for modifying traffic samples to generate minor variation. Fast-Time 
simulation may be used by itself, or as a preliminary to a Real-Time simulation to weed 
out the more unpromising alternatives. (Vergne and Tewes,1996) 

A Fast-Time simulation requires two or three months of preparation, about a month of 
running and about a month of analysis and reporting. About ten people are involved, 
mainly in data preparation, in liaison with the client and in analysis. Controllers are not 
required as such, although the team must contain, and is usually led by, staff with 


considerable control experience who can communicate effectively with the client. 


SIMMOD, developed by the US Federal Aviation Authority, is similar to the Eurocontrol 
Fast-Time simulator, except that it is more oriented towards airports and that it has some 
technical constraints on traffic behaviour. SIMMOD can be used to study the ground 
movement of aircraft, for which the EAM and RAMS are not suitable. It is not concerned 
with individual working positions, but with presenting a global appreciation of a set of 
alternative situations. 

SIMMOD studies are mainly concerned with the physical movement of aircraft, and the 
alterations to be expected if procedures or facilities are changed. Watkins et al (1997) 
investigated the reduction of separation that could be applied if a moderate cross-wind 
was blowing wake vortices away from the runway, for example. 

In general, SIMMOD studies take two or three months, and involve two or three staff. 


The TRACON/Pro simulator is a single-position autonomous Real-Time simulator. 
Initially designed as a training simulator in the USA, we use it as a test bed for 
measurement techniques for the main Real-Time simulator and for reality. It has some 
unusual features. Traffic can be generated relatively easily to produce representative 
samples, and can be made to behave in a more realistic manner than is usually the case in 
our main Real-Time simulator. Pilots can be ‘programmed’ to miss or misunderstand 
orders, and will refuse impossible or incorrect manoeuvres. Weather problems and 
communications failures can be simulated if desired. It can operate using voice 
recognition, provided the controller observes strict ICAO phraseology, and provided that 
he trains the system to recognise his voice. This task takes one to two hours, and leads to 
an average recognition rate of 94% allowing three repeats, in active control. The system 
generates a ‘score’ for each run, awarding points for correctly controlled aircraft, and 
deducting them for errors, and to a lesser extent for redundant orders. A first study 
(David and Pledger, 1995) showed that control using speech recognition was more 
efficient than control using keybord input of code instructions, possibly because the 
controller did not need to divert his attention from the screen whem using voice control. 
The same study showed that both the SWAT (Subjective Workload Assessment 
Technique) (Reid and Nygren 1988) and ISA (see above) produced a measurable 
decrease in control efficiency. We are currently using the TRACON/Pro system to 
evaluate some potential measures of ‘strain’, incuding the analysis of cortisol in saliva, 
and a test of the relative incidence of alpha-rhythm in the EEG with eyes open or shut. 
We intend to use it in future to evaluate other methods and to obtain base-line data on the 
time taken to perform common ATC operations. 

TRACON/Pro comes as a unit, including compiled software. We are not, therefore, able 
to modify the human-computer interface in line with modem developments, but, as a 
basic test-bed it is sufficiently close to current practice as to be accepted by controllers 

(A simple version of TRACON/Pro, lacking a separate screen for strips, strip printer and 
voice recognition facilities, is available as a computer game under the name TRACON H. 
We recommend this to students or experimenters needing a basic ATC simulation.) 


A TRACON/Pro study takes about six weeks of planning, one to three days of controller 
participation for each controller, and a month of analysis. 

Rapid Prototyping 

Rapid Prototyping can in principle be used for the development and initial investigation 
of new and unorthodox ATC displays and methods (Broadbent 1993). Although sonie 
studies can be done on static images, it is generally necessary to provide dynamic 
images, which implies a background ATC model. Since communications are an integral 
part of ATC, it becomes necessary to provide some form of ‘pilot-operator’ and some 
form of ‘feed controller’. Traffic must be more-or-less realistic, so that the rapid 
prototyping system becomes a miniature Real-Time simulator, with all the administrative 
and organisational problems that implies. For the moment, we are not pursuing this 
approach, although it may be necessary to meet some future problems. 


The ‘GENERIC’ model is a Fast-Time simulation model written in MicroSAINT, which 
is an event-based discrete-event simulation system. Micro-SAINT provides the 
underlying operating and analysis facilities and permits the user to define linked chains 
of events. GENERIC is essentially a mirror-image of RAMS and SIMMOD. Aircraft are 
defined only by their times of flight through successive sectors. Each sector is 
represented by a network which itself contains networks representing the activities of 
procedural and executive controllers. The tasks within the networks representing the 
individuals may require to joint activities of one or more individuals, and reqire times 
that may be fixed, normally or exponentially distributed. The probabilities that additional 
tasks will be required are defined by expressions reflecting the traffic density. Times 
befom which tasks must be performed are carefully noted, and sequential dependencies 
observed. (For example, the executive controller cannot alter the flight path of an aircraft 
before the procedural controller has treated it.) The number of occasions where a task is 
not completed in time is recorded, as are the total occupation times for each controller. 

We have used this model to investigate the probable workloads resulting from grouping 
several executive controllers under one procedural controller, and the effects of 
introducing a strategic ‘super-planner* (David, 1997b) 

Using this model requires a sound grasp of the principles of MicroSAINT, and close 
attention to the definitions of priority of tasks. It also requires reliable figures for the 
duration of standard ATC tasks. Subjective estimates, widely used for this purpose, have 
been shown to be unreliable, and the few available ‘objective’ measurements are, to put it 
mildly, obsolete (David 1972). 

We hope to obtain more reliable figures by analysis of video and audio recordings of 
TRACON exercises, and to adapt the GENERIC model to a more modem version of 

We will then use the model to complement Fast-time and other studies of aircraft 
movement to confirm that the corresponding workloads for controllers can be carried out 
in the available time, to make more detailed examinations of the consequences of 
assumptions about workload distributions, (For example, assuming that all activities take 
exactly the average time produces generally optimistic figures, while the times taken for 


most human ATC activities tend to be at least exponentially distributed) and to examine 
the consequences of changing operating methods. (For example, verbal coordination 
requires the simultaneous attention of both parties - computer-moderated coordination 
requires only sequential attention.) 

Part-Task Simulations 

Part-task simulations can be used to study specific aspects of ATC tasks, such as the 
choice of symbols for use in ATC labels (David and Thomas, 1996). Such studies must 
be taken with some reserve, since it is rarely clear how such items are used in practice, 
but they may be used to identify what subsequently appear to be obviously unsuitable 
choices. In the past, we have used standard IBM PCs with VGA colour screens as test¬ 
beds, writing the necessary display software in QuickBASIC, for speed and convenience. 
Part-task studies of this type should take from four to six weeks, and require the 
cooperation of controllers, or non-controllers for about one half day per individual. Since 
these part-tasks usually require basic perceptual abilities rather than integrated ATC 
skills, untrained volunteers can usually be substituted for controllers - always a scarce 


Demonstrators are small, self-contained programs developed to demonstrate new 
concepts, or to afford practice in unfamiliar methods. They are usually written in 
relatively high-level languages and presented on PC compatibles. Training versions can 
be sent to participant controllers before they come to a simulation (as was done with 
ODID rV), or copies can be distributed to provide the opportunity for ‘hands-on’ 
experience with EEC Reports. For example, a 3.5 inch disk containing three 
demonstrators is distributed with EEC Report No, 307 (David, 1997a). These allow 
readers to compare models of the current system (DEMOLD), a radically revised system 
(DEMON) and a more tightly-coupled predictive system (DEMFAST). 

More elaborate demonstrators are available at the Experimental Centre, for example, to 
demonstrate HIPS displays (Meckiff and Gibbs, 1994). 

Demonstrators are usually constructed by one or two persons during a few weeks. They 
may be employed in formal experiments taking some months, but they are, more often, 
used to demonstrate features of planned or actual systems. 


This brief outline shows the variety of simulation methods that are available at the 
Eurocontrol Experimental Centre. Some of these methods involve the active 
participation of skiUed controllers (Real-Time simulation, TRACON/Pro), others are 
purely computer based (Fart-Time Simulation, SIMMOD, MicroSAINT), complemented 
by measirements of task times from isolated real activities (Part-Task simulation, Rapid 
Prototyping.) Each type of simulation is appropriate to particular problems, and it is 
only by an intelligent use of different methods that a fair approximation to reality can be 

It should not be forgotten,however, that even the best simulation cannot guarantee that a 
system will pass the ultimate test, that of reality. 



(EEC Notes and Reports starred (*) below can be read at the Experimental Centre Web¬ 
site ‘’. Paper copies of all EEC publications are available on request 
to Mrs. J, Roelofsen, Eurocontrol Experimental Centre, BP 15 Bretigny-sur-Orge, 91222 
CEDEX France - 

Broadbent, S. (1993) An Experimental Evaluation of Traffic Filtering. EEC Report No. 

David, H. (1972) Data Collection for the Eurocontrol Ground Model. EEC Note No. 5. 

David, H. (1997a) Radical Revision ofEn-route Air Traffic Control. EEC Report No. 


David, H. (1997b) Use of MicroSAINTfor the simulation of ATC Activities in Multi¬ 
sector Evaluations in Contemporary Ergonomics 1997 ed Robertson S.A. Taylor and 
Francis London. 

David, H., Pledger, S. (1995) Intrusiveness of On-Line Self-assessment in ATC 
Simulation using Keyboard and Speech Recognition. EEC Report No. 275. 

David, H., Thomas, V. (1996) Choice Of Climb/Descent Symbols For ATC Displays. 

EEC Note No. 4/94. 

Graham, R .V., Young, D., Pichancourt, I., Marsden, A., Ddz, A. (1994) ODIDIV 
Simulation Report, EEC Report No, 269. 

*Hering H., Coatleven G. (1996) ERGO (Version 2) for Instantaneous Self-assessment of 
Workload in a Real-Time environment. EEC Note No, 10/96. 

*Jackson, A., Pichancourt, I. (1995) An HMI Reference System for En-Route ATC. EEC 
Report No. 292. 

*Meckiff, C., Gibbs, P. (1994) PHARE Highly Interactive Problem Solver. EEC Report 
No. 273. 

Prosser , M., David, H. (1988a) Real Time Simulation in support of the ODID Group, 

EEC Report No. 215. 

Prosser, M., David, H. (1988b) Second Real Time Simulation for the ODID Group, EEC 
Report No. 226. 

Prosser M., David, H., Clarke, L. (1991) ODID III Real-Time Simulation, EEC Report 
No. 242. 

Reid, G.B., Nygren, T.E. (1988) The Subjective Workload Assessment Technique: A 
Scaling procedure for measuring mental Workload. In Hancock and Meshati (eds) 
Human Mental Workload, North Holland Publishing, Amsterdam. 


Reynolds, L., Metcalfe, C. (1992) The Interim NATS Standard for the Use of Colour on 
ATC Displays. CS Report 9213 Issue 1.2, UK Civil Aviation Authority. 

*Vergne, F., Tewes, G. (1996) Airspace Model Simulation of German Airspace EEC 
Note 2/96 

♦Watkins, J., Brady, D., Day, C. (1997) Reduced Aircraft Separation SIMMOD Study. 
EEC Note 4/97. 


PC-based training for Air Traffic Controllers 

A feasibility study 

Bas Kuijpers, Herman Kuiper, and Jan Roessingh 
National Aerospace Laboratory NLR 
Amsterdam, The Netherlands 
e-mail: kuijpers @ nlr. nl 


To meet the needs of growing air traffic, new air traffic control (ATC) systems are being 
constantly developed. Training air traffic controllers to operate these systems requires a 
lot of time, because the controllers must be familiarised with both new human-machine 
interfaces and modified operational procedures. For that reason, the National Aerospace 
Laboratory (NLR) of the Netherlands decided to conduct a study into the feasibility of 
computer based training (CBT) programs running on PCs as a means of familiarisation 
training for air traffic controllers. By presenting the familiarisation training prior to high 
fidelity, operational ATC training, it was expected that air traffic controllers would be 
better prepared for the latter and thus require less operational training time. During the 
study a PC-based tutorial, as well as a PC-based simulation training, were developed. As 
a basis of the training modules the Converging Runways Display Aid was chosen. After 
performing a training analysis and designing the structure and human-machine interface, 
both modules were implemented using Authorware. Subsequently, the training modules 
were evaluated by operational air traffic controllers. From the study it can be concluded 
that application of PC-based familiarisation training for air traffic controllers is feasible 
and that the PC is an appropriate platform for this kind of training. 


To meet the needs of ever-growing air traffic and to restrict the workload of the air traffic 
controller to acceptable levels, new air traffic control (ATC) systems are being constantly 
developed. Before these systems can be put into use, air traffic controllers must be 
trained to operate them. Such training usually comprises familiarising air traffic 
controllers with new human-machine interfaces, changed task-allocation between human 
and machine (as well as between controllers themselves), and modified working 
procedures. This results in unacceptably long training periods. In addition, ATC training 
programs often require high fidelity ATC simulation equipment, which is in general 
costly, scarce, and complex to organize. 

Hence, NLR decided to conduct a study into the feasibility of a so-called computer based 
training (CBT) running on personal computers as a means of familiarisation training for 
air traffic controllers. This PC-based familiarisation training sought to introduce air 
traffic controllers to the new ATC system. By presenting the PC-based familiarisation 
training prior to the operational, high fidelity ATC simulator training, it was anticipated 
that air traffic controllers would be better prepared for the latter, thus reducing the time 
and costs associated with operational training. 


During the feasibility study a PC-based familiarisation training was built and evaluated. 
This training consisted of two modules: a PC-based tutorial^ and a PC-based simulation 
training^ As a basis of the familiarisation training an existing ATC system was chosen: 
the Converging Runways Display Aid which will be introduced at Amsterdam Airport. 

The Converging Runways Display Aid 

To increase the landing capacity of Amsterdam Airport during peak traffic hours, ATC 
switches from a single runway to a dual simultaneous runway configuration. Depending 
on the wind conditions (and thus landing directions) these two runways may converge. A 
disadvantage of the latter is that during poor visibility it can be dangerous to land aircraft 
on both runways simultaneously. Under such circumstances the controllers in the control 
tower cannot directly observe the in-trail separation between arriving aircraft. This in¬ 
trail separation is of vital importance, because in case two aircraft break off their landing 
one after the other and make a climb, the danger exists that they will collide at the 
intersection of both runways. To mitigate this danger, the Converging Runways Display 
Aid (CRDA) is used to set up a safety buffer between two aircraft. To do this, the 
concept of ghosting is used. Ghosting refers to replicating the on-screen image of an 
aircraft at a different spot on the same radar display. Such a copy of an aircraft is called 
“ghost” (see figure 1). The CRDA is the software tool that generates the ghosts and 
shows them on the radar display. 

One of each pair of converging runways is designated the “Master Runway”, whereas the 
other is designated the “Slave Runway”. The Master Runway is the runway from which 
ghosts are copied. The Slave Runway is the runway to which ghosts are copied. Of every 
aircraft that approaches the Master Runway, a copy is shown on the (extended centre line 
of the) Slave Runway. This is done by drawing an imaginary circle centred on the 
runway intersection from the actual aircraft to the Slave Runway. At the intersection of 
the circle and the extended centre line the ghost is displayed. As the actual aircraft 
approaches the Master Runway, the ghost approaches the Slave Runway as well; that is, 
the ghost and its original behave identically. 

Master Arrival and Slave Arrival controller 

Both the Master and Slave Runway have their own air traffic controllers. The air traffic 
controller of the Slave Runway is called “Slave Arrival”, whereas the air traffic controller 
of the Master Runway is called “Master Arrival”. On the Master Arrivars radar display 
only real aircraft are displayed, whereas the radar display of the Slave Arrival also shows 
ghosts. The Master Arrivars task is to direct air traffic to the Master Runway in such a 
way that the separation between the aircraft remains constant. To prevent collisions at the 
runway intersection, the Slave Arrival has to direct his / her traffic at a specific minimal 
distance behind the ghosts. This minimal distance is the above-mentioned safety buffer. 
Since estimating the distance between aircraft on a radar display is perceptually difficult, 
it has been decided to put the ghosts at the minimal distance behind their initial 
positions. In this case the Slave Arrival only has to direct his traffic onto or immediately 

^ PC-based tutorials present to be learned knowledge to the trainee, ask him questions about this knowledge, evaluate 
his answers, and present him feedback about the correctness of these answers. 

^ In a PC-based simulation training trainees acquire skills in solving problems and executing working procedures. This is 
done by simulating a part of the real world and letting the trainee interact with it. Each time the trainee interacts with the 
simulation, the latter responds by changing its state, so that the results of the trainee’s actions are presented to him. 


behind the ghosts to prevent collisions (see figure 2), The process of directing traffic onto 
or behind ghosts is called “tying”. 

Figure 1: ghosting Figure 2: ghosting with safety buffer 

Training analysis 

To develop a PC-based familiarisation training based on the CRDA, a training analysis 
was performed first. The analysis phase of the CBT development consisted of four steps: 

1. Select the air traffic controller position (Slave Arrival or Master Arrival) to 
be used as a basis for the training. 

2. Perform a scenario based task analysis (i.e. determine which tasks the 
selected air traffic controller position performs in which operational 

3. Determine the skills that are related to these tasks (i.e. determine what 
actually has to be trained). 

4. Formulate the learning objectives (i.e. describe what the trainee should have 
learned after the training). 

This top-down approach ensured that no necessary skills were missed, and also facilitates 
consultations with the customer (ATC Amsterdam Airport), as it relates closely to their 
current training practices. 

Selection of air traffic controller position 

The very first step in the training analysis was selection of a CRDA air traffic controller 
position as a basis for the CBT, Because the project was intended as a feasibility study, it 
was decided to base the CBT on the air traffic controller position that would benefit the 
most from the training. 

This resulted in the selection of the Slave Arrival controller position. There were three 
reasons for this decision: 

1. The tasks of the Slave Arrival are mainly “new” tasks (i.e. tasks that are not 
part of the regular operations of the controller). 

2. The correct functioning of the Slave Arrival is crucial for the success of the 
CRDA operation, 

3. The Slave Arrival’s function requires the highest effort from the controller. 


Task analysis 

The next step in the training analysis was to determine the tasks performed by the Slave 
Arrival. According to the air traffic control of Amsterdam Airport, there are 4 operational 
scenarios related to the Slave Arrivars function, each of which have implications for PC- 
based training. These four scenarios are: 

A. Traffic does not cause any problems: normal flow of traffiic. 

B. Missed approach above 750feet. 

A missed approach is a runway approach that is broken off. In this scenario, the 
aircraft performs the missed approach above an altitude of 750 feet. 

C. Worst-case situation with two missed approaches. 

A worst-case situation is a situation in which the minimal safety buffer would be 
jeopardized when two aircraft (one on the Master Runway, one on the Slave 
Runway) are at the runway intersection. If, during such a worst-case situation, 
both aircraft successively initiate a missed approach, unpredictable variations 
in aircraft behaviour could easily lead to a critical loss of separation. 

D. Excessive approach speed. 

If the approach speed of an aircraft is too high, the chance of a double missed 
approach (and hence the danger of a collision on the runway intersection) is no 
longer minimal. 

For each of the operational scenarios, a task analysis was performed. The tasks the Slave 
Arrival performs in these scenarios can be summarised as follows: 

• Monitoring whether an aircraft is under his / her control. 

• Issuing speed, altitude and heading instructions to the aircraft in order to 
direct it to the correct position for tying. 

• Monitoring whether the tie-process proceeds correctly. 

• When an aircraft initiates a missed approach, executing one of a number of 
so-called “default solutions” to prevent a collision on the runway intersection. 
The default solution to be chosen depends on the runway combination used. 

• Redirecting aircraft that had a missed approach back into the “main” stream 
of aircraft to approach the Slave Runway again. 

Training skills 

The third step in the analysis was to use the identified tasks to determine the skills to be 
trained. A training skill is defined as the product “task x task context x time pattern”. 
The context of the tasks at Amsterdam Airport was defined by the runway combinations 
used (which depend on wind conditions), and the type of aircraft being involved in the 
CRDA operation. Because the number of aircraft per hour is roughly constant when 
CRDA is in operation (as the CRDA is only used during peak traffic hours), the time 
pattern (the time available for each task) is constant. Therefore, the skiUs are a result of 
the product “task x task context”. For the four given scenarios, this resulted in 60 
different training skills (15 tasks, 4 different task contexts). 

Since the trainee population comprises skilled air traffic controllers, not all training skills 
identified need to be trained. Certain skills (e.g. ATC - aircraft communication, updating 
aircraft labels) would likely already have been mastered by the Slave Arrival. 


To determine which of the training skills would be suitable for training with CBT, it was 
decided to add each of these to the CBT one by one, starting with the simplest scenario, 
scenario A. 

Learning objectives 

Finally, the last step in the training analysis, was to write learning objectives based on the 
training skills. The formulation of the learning objectives was based on recommendations 
by Hannafm and Peck (1988), which include the following: 

• Each learning objective should contain the behaviour to be shown by the 

• Each learning objective should contain the circumstances under which the 
behaviour should be shown. 

• Each learning objective should contain the criteria the student behaviour 
should satisfy. 

An example learning objective for scenario C is the following: 

"’'The trainee is able, for runway combinations 06/0IR and 27/19R and air traffic 
consisting of heavy, medium and light aircraft, to check using radar display and 
aircraft labels, whether the tie-process is proceeding correctly, and whether a 
worst-case situation is caused'. 

However, it became clear that learning objectives like these are difficult to read, and not 
suitable for presentation to the trainees. Therefore, the objectives were reformulated. For 
the given example, this resulted in: 

""The trainee is able, for runway combinations 06/01R and 27/19R, to check 
whether traffic is causing a worst-case scenario". 

Selecting the type of computer based training 

The learning objectives could be classified into two categories: those related to the basic 
knowledge of CRD A, and those related to the procedures and tasks to be executed by the 
Slave Arrival during the identified scenarios. 

For the learning objectives on CRDA knowledge, it was decided to develop a PC-based 
tutorial module. Historically, CRDA knowledge training has been offered classroom style 
(with paper and pencil). Therefore, the PC-based tutorial uses a sequence of screens with 
CRDA information (e.g. when is CRDA used, under which circumstances), interspersed 
with “yes/no”-type questions. 

It has been known for well over ninety years that transfer of learned knowledge and skills 
from training to the real world is only achieved if the training situation and the real world 
contain identical elements (Thorndike & Woodworth, 1901). Therefore, to achieve a high 
positive transfer of the knowledge and skills acquired during the familiarisation training 
to the real world, it was decided to use simulation training to train the procedural 
learning objectives. 

By means of a simulation training the radar display as used by the Slave Arrival could be 
imitated. In order not to have the simulation training result in a high fidelity simulator of 
the Amsterdam Airport ATC system, only the elements relevant to the CRDA procedures 


of the Slave Arrival were included in it. The input of the instructions to the aircraft 
would have to be as identical to the real world as possible, except for the radio telephony 
inputs. The PC-based simulation training was intended to be offered after completion of 
the PC-based tutorial. 

The PC-based tutorial 

For the tutorial, the CRDA-manual was chosen as a starting point. From this manual, the 
information that should be known to the Slave Arrival before he / she is able operate the 
CRDA was extracted. This information was organized in a number of topics, like 
“CRDA terms and definitions”, and “Specific Schiphol Airport CRDA procedures”. 

The information on each of these topics was organized in a number of screens containing 
information. Where appropriate simple, non-interactive animations were added to 
explain some terms. For example, the difference between ghosting with and without 
safety buffer was demonstrated using two animations. In these animations, all 
unnecessary details were left out, so that the trainee could concentrate on the example 

The topics were interspersed with questions in which the trainee had to indicate whether 
the statement made in the question was correct (“yes/no” questions). A trainee could only 
proceed to the next topic if all questions in a block were answered. For each question that 
was answered incorrectly, the trainee was presented feedback indicating why the answer 
was incorrect. For some questions, there was additional clarifying feedback after correct 
answers as well. 

Additional requirements to the simulation training 

Following the decision to develop a PC-based tutorial, as well as a PC-based simulation 
training, additional requirements on the simulation training were defined. These were of 
the following categories: 

• Requirements with respect to the structure of the simulation training. It was 
decided that the simulation training should consist of a familiarisation lesson 
in which the trainee can become familiar with the radar maps of both CRDA 
runway combinations and with the procedures for issuing aircraft instructions. 
Further, a lesson was required in which trainees could practise tying air traffic 
on both runway combinations in each of the operational scenarios. 

• Requirements with respect to information shown to the trainee. For instance, 
update rate of aircraft positions, shape of aircraft position symbols, content of 
aircraft labels (altitude, speed, etc.), layout of radar maps, colours. 

• Requirements with respect to interactions between trainee and training. For 
example, how to select an aircraft in order to issue instructions to it, how to 
input aircraft instructions, how to update aircraft label data. 

• Requirements with respect to the behaviour of simulated aircraft. Such as 
rate of turn, rate of climb, rate of descent, runway approach routes, initial 
aircraft separation. 


Design of the simulation training 

Based on the learning objectives and additional requirements, a design of the simulation 
training was made. Figure 3 shows the design of the structure of the simulation training. 

Figure 3: design of the structure of the simulation training 

The simulation training took off with an introduction that briefly described the content of 
the training. After the introduction the trainee could follow the familiarisation or the 
tying lesson. On request of the trainee the familiarisation lesson presented explanations 
about elements of either of the radar maps. In this lesson the trainee could also practise 
issuing instructions to aircraft, hi the tying lesson the trainee could practise tying aircraft 
on both CRDA runway combinations (06/01R and 27/19R) and in each of the four 

In addition to the structure of the simulation training, a design of the interface between 
trainee and training was made. This interface design was based on the DenK architecture 
(Ahn et al., 1994). In accordance with the DenK architecture the interface between 
trainee and training can be illustrated as shown in figure 4. 

Figure 4: interface between trainee and simulation training 

The simulation training embodied two elements: a domain and a tutor. The purpose of 
the domain was to simulate the real world in which the Slave Arrival performs his / her 
tasks. Depending on the training’s state, the domain could be of different types. For 
instance, if the simulation training was in the state Radar map 06/01R (see figure 3), 


then the domain contained a picture of the radar map, as shown on the Slave Amval s 
radar display during CRD A operations for runway combination 06/0IR. When the 
training was in the state Familiarisation aircraft manipulation the domain held the 
aforementioned radar map, as well as simulated air traffic and an input field to issue 
aircraft instructions. The domain interacted with the tutor by for example supplying it 
with trainee performance data. The trainee interacted with the domain by requesting 
explanations about radar map elements and by issuing instructions in order to change 
aircraft behaviour. 

On the trainee’s request the tutor showed help-info and interacted with the domain to 
start and end lessons and domain types. Also, the tutor read in and verified aircraft 
instructions issued by the trainee. Based on performance data received from the domain 
the tutor also presented the trainee feedback about the correctness of his / her behaviour. 
Cohen (1985) and Sales (1988) say that such feedback is both motivating and 
informative. This means that the feedback attempts to alter the trainee’s behaviour by 
rewarding correct behaviour by praising him / her and by discouraging incorrect 
behaviour by telling him / her about mistakes and their consequences. It also means that 
the discouraging feedback offers the trainee information about where the mistakes were 
made and how they can be corrected. Furthermore, the training presented feedback about 
the degree to which the trainee was reaching the learning objectives. The training showed 
for instance the number of correctly tied aircraft and the utilised landing capacity. 
According to Latham and Locke (1979) this feedback type improves the trainee’s 
commitment to the learning objectives. 

Another educational concept that has been included in the PC-based simulation training 
is the concept of overtraining. Overtraining means presenting trainees additional 
learning opportunities after they have reached the learning objectives. When applied, 
overtraining improves the retention of acquired knowledge and skills (see Schendel & 
Hagman, 1982). The concept of overtraining was included in the simulation training by 
restarting the tying lesson whenever the trainee succeeded for the first time in tying all 
aircraft offered to him / her during the lesson. 

Implementation of the CBT 

For the implementation of the CBT modules, Authorware Professional was used. For the 
tutorial, no major problems were met. Nevertheless, for the simulation training it became 
clear that a real-time interactive simulation required a number of adjustments to be made, 
especially in order to keep the simulation running during trainee interactions. Further, it 
proved impossible to make a scaleable simulator; that is, a simulator in which the 
number of Master and Slave aircraft can be changed easily. For training purposes though, 
there would be a fixed maximum number of aircraft on the radar display at a given time, 
so this problem was more or less circumvented. 

Figure 5 shows a sample screen image containing the main radar screen of the simulation 

Evaluation of the PC-based tutorial and PC-based simulation training 

After they had been implemented, the PC-based tutorial and PC-based simulation 

training were evaluated by air traffic controllers from Amsterdam Airport. These 


controllers were asked to attend the tutorial and simulation training and to offer feedback 
on each of them. After correction of all shortcomings that were found in the modules, a 
questionnaire was developed in which controllers were requested to rate the tutorial and 
simulation training on a number of topics, including aircraft behaviour, feedback, and 
help-information. The intention was to deploy both modules during the official CRDA 
training of the Dutch air traffic control services and to hand out the questionnaire among 
the trainees attending the latter. However, due to circumstances beyond the control of 
NLR, dates have not yet been finalised for this empirical evaluation. 

Figure 5: sample screen image from the simulation training 


Even though the PC-based training modules have not been empirically evaluated yet, the 
software developed so far, and the remarks Ifom Amsterdam Airport air traffic 
controllers indicate that the application of PC-based training for familiarisation training 
of air traffic controllers is feasible, and that the PC is an appropriate platform for this 
type of training. 

During the study it was experienced too that a formal approach to training analysis 
assists in determining the skills and learning objectives to be satisfied with the training, 
as well as supports consultations between developer and customer. 

Furthermore, the feasibility study revealed that application of Authorware Professional to 
develop real-time simulations is not optimal. Instead, it might be better to create external 
simulation modules, which could be integrated into the Authorware training-software. 



The authors would like to thank the air traffic control services of Amsterdam Airport for 
their cooperation in conducting this study. 


Ahn, R.M.C., Beun, R.J., Borghuis, T., Bunt, H.C., Van Overveld, C.W.A.M. “The 
DenK-architecture: a fundamental approach to User-Interfaces”. Artificial 
Intelligence Review 8 (1994) 431-445 

Cohen, V.B. “A Reexamination of Feedback in Computer-Based Instruction: 

Implications for Instructional Design”. Educational Technology 1 (1985) 33-37 

Hannafm, M.J., Peck, K.L. (1988) The Design, Development, and Evaluation of 
Instructional Software Macmillan Publishing Company 

Latham, G.P., Locke, E.A. “Goal setting: A motivational technique that works”. 
Organizational Dynamics 8 (1979) 68-80 

Sales, G.C. “Designing Feedback for CBI: Matching Feedback to the Learner and 
Learner Outcomes”. Computers in the Schools 5 (1988) 605-610 

Schendel, J.D., Hagman, J.D. “On sustaining procedural skills over a prolonged retention 
interval”. Journal of Applied Psychology 61 (1982) 605-610 

Thorndike, E.L., Woodworth, R.S. 1: “The influence of improvement in one mental 
function upon the efficiency of other functions”; 2: “The estimation of 
magnitudes”; 3: “Functions involving attention, observation and discrimination”. 
Psychological Review 8 (1901) 247-261; 384-395; 553-564 


Deep Design - Beyond the Interface 

Hugh David 
Eurocontrol Experimental Centre 
BP 75, Bretigny-sur'Orge 
F 91222 France CEDEX 


Many large-scale dynamic control systems are moving from traditional equipment- 
determined control systems to computer-mediated wholly or semi-automated systems. 
There has been a tendency for computer-based systems to mimic traditional displays. 
This approach fails to make use of the enhanced information-handling capacity of the 
computer, and often produces displays that are in practice inferior to those they replace - 
occasionally with tragic results, 

A better approach is to begin with an qualitative functional analysis of the control task, 
with as little reference to the existing system as possible, then to consider the component 
tasks, selecting those for which the operators are best suited to compose an interesting 
but not overloading workload, and delegating the remaining tasks to the computer-based 
system. Where the workload varies with time, particularly if the variation is 
unpredictable or uncontrollable, it may be necessary to provide ‘fall-back’ automation, 
which will maintain safety without regard to efficiency. 

Only after the operators’ overall tasks have been defined should consideration be given to 
what and how to display information to the operators, and to the communication links to 
be used. In many instances graphical images will be preferable to tabular and symbohc to 
numeric forms. Displays need not be confined to current data - historical trends and 
projections of future situations can be displayed. The latter can be particularly valuable if 
linked with projected sequences of control actions, to verify their effectiveness before 
applying them. Where control is largely ‘by exception’ displays must be designed to 
enable the operator to obtain the necessary information for effective control in adequate 
time. Training may include drills for the allocation of tasks between operators, as with 
CRM (Cockpit Resource Management) in civil aviation. 


Dynamic control systems control processes that are operating in ‘real-time’. The events 
or processes controlled are happening now, and usually continuously. Examples are 
chemical plants, assembly plants, automated factories, water and sewage distribution 
systems, electricity generating plants (nuclear, coal, hydroelectric or others), electricity 
distribution systems, road, rail and air traffic control systems, ambulance services, and 
even military command systems. 

These systems usually have control rooms, where operators , usually in teams, receive 
information, (as verbal messages, by radio, telephone or data-link, TV images, readings 
on dials or chart-recorders, warning lights, gauges, or simply by looking out of the 
window) form a picture of the activity (which may be literally a picture, some form of 
semi-mechanical analog display, a computer record, or other display), decide on the 


necessary control actions required, and communicate these to the components of the 
system, by whatever means available (Button pushing, verbal messages, maneuvering 
controls etc.). Finally,(and often forgotten), they must check that their intended actions 
have actually happened, and that they have had the desired effect. 

Many of these systems are made up of more or less autonomous units. The degree of 
centralization or delegation of authority is usually the result of historical or technical 
accident rather than design. Although the general problems are similar, accidents of 
technical development have led to different styles of display in different fields of 
development. Ergonomists have spent much time and effort in the design of ‘knobs and 
dials’. The studies by Fitts and Jones(i961) are classic examples. 

There is, however, a contemporary trend that must inevitably affect the design of control 
rooms. More and more of these systems are computer-based. Rather than displaying raw 
radar data, the signals from several radars are smoothed and combined, weighted 
according to the reliability of the individual radar and used to construct a digital track for 
an aircraft, which is then displayed on a computer VDU. (My life’s experience is in Air 
Traffic Control, so I shall draw examples mainly from that field, but the message I wish 
to send applies to all the fields I have mentioned, and to many others. Hansen (1995) 
provides a similar example for a coal-fired power station. ) The circular display with 
rotating sweep so familiar from films is now unnecessary, and a standard CRT can be 
used. Ironically, some effort has made to produce a synthetic digital representation of the 
trail of decaying ‘blips’ behind the aircraft. 

The Horseless Carriage Method 

This is an example of the ‘horseless carriage’ effect. In order to ease perceptual problems 
in the transition from traditional electromechanical to digital systems, it is tempting to 
reproduce the traditional image on the CRT screen. This may have disastrous 
consequences. The image available on a flat CRT screen may be far less salient than that 
provided by a bank of electromechanical dials, so that minor but critical differences can 
be overlooked. A more subtle problem occurs in Air Traffic Control, where the 
traditional planner is provided with a strip, in a strip-holder, for each aircraft. He moves 
these strips around, marks the surface of the strip, moves it in its holder to signal to his 
partner, and finally discards it and rearranges his board after the aircraft has left his area 
of responsibility. Electronic strips lack these facilities, and, in some systems, may even 
remove themselves automatically when the aircraft leaves, so that when the controller 
next looks at his strip bay the aircraft are not where he left them. This is, to put it mildly, 
intolerable to a busy controller. 

Examples abound of similar short-sighted adaptations of pre-computer displays. They 
have some superficial attractions. They do not, in principle, require significant input 
from the users, and they are usually sufficiently close to the previous system that no 
training or transition period is apparently necessary. Often neither of these is true, but, by 
the time this becomes apparent, the system is in place. 

Beyond the Interface 

I propose a different approach to the development of these systems. None of the 
individual elements of this approach is unprecedented, and most of what I am saying can 

be derived from various studies, ranging from neurophysiology to anthropology to 
systems analysis, carried out in the last twenty-five to thirty years. 

Task Definition 

The starting point of interface design should be a careful, in-depth study of what are the 
real purposes of the control system being investigated. Most organizations have a 
standard definition, which will be produced in answer to the initial question. In Air 
Traffic Control, this is usually ‘The Safe, Orderly and Economic control of Air Traffic”, 
for example. If, however, you analyze the components of this statement, you find that the 
first is a constraint, the second is practical requirement of pre-computerized systems 
arising from the limits of human memory and i^ormation processing capacity, and the 
third is often a hope rather than a true goal. Although it is always possible to argue over 
abstract definitions, ATC’s purpose can be defined as to allow air traffic to fly from 
origin to destination as quickly and/or cheaply as the individual flights require, with the 
minimum disturbance necessary to maintain safety. 

Task Analysis 

The second stage of the process is to analyze the ‘real’ task. This may well not be closely 
related to the formal task as traditionally defined. It is particularly important to 
distinguish between a task ‘analysis’ and a task ‘description’. The construction of a task 
description involves observation of the existing system, often in extraordinary detail 
(Phillips et al, 1984), has the unfortunate consequence of producing a mental ‘set’ in 
favor of the existing method of control. For simplicity and convenience, I shall take as 
example ‘en-route’ ATC, which concerns aircraft in the major part of their flight, after 
they have left the region around their airport of origin, and before they arrive at their 
destination. The TMAs (Terminal Maneuvering Areas) around airports or groups of 
airports have their own problems, which will not be discussed here. 

A modem Upper Airspace Control centre, such as Maastricht UAC, where Eurocontrol 
controls the upper airspace of several busy states, and part of another, is organized into 
sectors, each defined by a geographical area and upper and lower flight levels. Within 
the sectors, routes are defined by sequences of beacons. Ground-air communication is 
by voice links. The data for each flight, stating its intentions, is abstracted from standard 
flight plans and combined with information from a network of radars to provide current 
aircraft positions. Although Flight Process Strips are still generated, in some sectors 
controllers prefer to refer to electronic data displays. Usually sectors are manned by two 
controllers. The Planning Controller (PC) examines the future traffic, and plans how it 
can be organized to minimize the disturbance to the aircraft while maintaining safe 
separations and satisfying entry and exit conditions. The Executive Controller (EC), 
referring mainly to the radar picture, makes short-term decisions, and communicates with 
the aircraft in the sector. Controllers communicate with aircraft by Radio-telephony 
(RTF), with adjacent sectors and centers by telephone or by computer-based digital 
messages, and with each other by voice, supplemented by gestures and computer or strip 
notations. In all modem ATC systems, the controllers are obliged to update the computer 
system when they take action, and in most modem aircraft, the pilots insert the changes 
in their flight path into their Flight Management System (FMS). 


Figure 1 (based on Dee 1996) sketches the tasks currently carried out by en-route Air 
Traffic Controllers. 

1) learn’sectors 

2) manipulate and mark strips 

3) plan future streams of aircraft entering their sectors, 

4) check that they will not conflict with each other within the sector, 

5) determine how to resolve conflicts 

6) match radar images to strips 

7) acknowledge aircraft coming on to the frequency when they enter, 

8) intervene to resolve conflicts 

9) coordinate their actions with the next sector 

10) monitor aircraft behavior for deviations from track 

11) hand aircraft over to the next sector 

12) attempt to comply v^th any special requests, 

(for example for direct routings or changed routings), 

13) handle unexpected emergencies. 

Figure 1 - Task List 

We do not have enough space and time to carry out a systematic analysis of these tasks 
(See David 1997), although it should be noted that the first mentioned task - ‘learning 
sectors’ is particularly interesting from an ergonomic point of view,. It takes a fully- 
trained controller about six months to ‘learn’ a sector. Too little attention has been paid 
to this process, which is generally one of apprenticeship. The controller accompanies an 
experienced controller on the sector, apparently absorbing a vast quantity of information 
about the sector, the usual traffic and its peculiarities, so that this information is in his 
permanent memory for recall. The necessity for this feat of memorization is a criticism of 
the current systems of training and information presentation. This information is a 
considerable burden to the controller, fading rapidly after a few days’ interruption of the 
task. Many ‘coping strategies’ are required to maintain it, even with the present level of 
traffic. (Stein and Bailey, 1994) 

Many of the tasks listed here are essentially routine, linked to the traditional methods of 
communication by RTF, occupying a considerable proportion of the controllers’ time. 
Others reflect the need to match the strip or tabular data on which planning is based, and 
the decisions arrived at during planning, to the image of the current situation presented 
by the Electronic Data Display (EDD) on which the annotated radar data is displayed. 
This operation is inherently wasteful, since the computer has already done it. Some 
modem stripless systems (Graham et al, 1994), eliminate this part of the workload by 
displaying the ‘strip’ data as part of an aircraft label. This approach has other drawbacks 
- it covers the screen with tabular data, or requires separate windows to be opened for 
each aircraft. 


The communication of information by RTF is inherently error-prone, and the attempt to 
achieve mechanical levels of reliability by drilling human operators is futile and 
inhumane. Cushing (1994) draws on reports of the (US) National Traffic Safety Board, 
and the Aviation Safety Reporting System, to show that this form of communication is 


no longer viable. He sketches a potential ‘data-link’ alternative. 

Human Tasks 

To summarize an extensive argument, the tasks particularly suited to, and enjoyed by 
human operators are conflict resolution (but NOT conflict detection), dealing with 
special requests from aircraft, and rectifying errors identified by computer monitoring of 
traffic, such as may arise from incorrectly set Flight Management Systems. Controllers 
are also good at imposing a strategic direction on the system, by developing and applying 
conventions to reduce workload and increase overall efficency. 

The human-computer interface should therefore be designed to facilitate these tasks, 
rendering the data necessary for these tasks easily accessible, while not prohibiting the 
controller from looking further into the data when opportunity offers. 

As Woods (1995) remarks, the now traditional windowing structure is not well adapted 
for dynamic control, since it requires the user to search out the correct window to find 
the data or formulate the instruction required. It is far better, if at all possible, to show all 
necessary information on one screen. This may at first seem a difficult task, but it is not 
impossible, if the nature of the data and the way they are used are taken into account. 

The data 

For design purposes, we can simplify the actual ATC situation by considering the 
trajectoiy of each aircraft as a series of linked straight-line segments. Each begins with a 
position in space, expressed as X, Y and H coordinates, and a (constant) rate of change 
of each of these. The position is true at a time T and the segment continues until one or 
more of the rates of change alters after an interval dT. Segments before the current time 
are fixed, and of no immediate interest for control purposes. Future segments describe 
the planned future path of the aircraft. These can be compared with the future segments 
of other aircraft to identify losses of separation at future times. The use of straight-Hne 
segments permits a fairly simple algorithm to be employed. Changes in future flight path 
can be specified simply, and converted into an alternative sequence of segments, which 
can be tested to identify new or remaining conflicts. 

Traditional displays 

There are two traditional means of displaying this data. The radar image displays the 
current X and Y dimensions as positions on the display surface, dX and dY by a ‘speed 
vector’ and codes height (H) as a three-digit numbers displayed close to the aircraft 
marker, in a ‘label’ containing the call-sign, and more or less other data expressed as 
letters and digits. The strip display (usually) reduces the X and Y dimensions to a series 
of named points (usually beacons where routes intersect) for which estimated times (T) 
and heights (H) are specified as numeric values. The strips themselves may be arranged 
in various ways, for example, under a label for the next beacon, arranged by height or 
estimated time, where the labels are arranged in a roughly geographical layout. It is very 
difficult to cope with aircraft which do not follow predefined routes using a strip-like 

Apart from the position and speed vector, virtually all the data are presented in letters 
and digits. To complicate matters further, beacons are usually shown by abbreviations, if 
they are labeled at all, and spoken as their full names. Commercial flight call-signs 


contain a company identifier such as PA for the (defunct) Pan-American Airways, which 
was spoken as ‘Clipper’. 

From the cognitive point of view, this is a mess. If the theories of cerebral asymmetry of 
cognitive function (Springer and Deutsch, 1993) have any validity, it would be hard to 
design a worse display system. Graphical and textual indicators are scrambled together 
unsystematically. In some systems, rates of climb, known to the computer system to a 
high degree of accuracy, are deducible only by watching the rate at which the last digit, 
representing hundreds of feet, changes. If, as is usually the case, the display is updated 
only at five or ten seconds intervals, it may take up to half a minute to obtain even an 
approximate idea of the climb rate. Controllers adapt to this by learning the trajectories 
to be expected of common types of aircraft in the hands of particular company pilots, and 
by allowing enormous safety margins (Lafon-Millet 1978). 

Conflicts, the main justification for en-route air traffic control are usually not shown to 
the controller, (Some systems have last-moment short-term conflict alerts, but many of 
these rely only on radar data, so that they cannot cope with aircraft which have been 
instructed to cease climbing below the level of the opposing traffic, for example, and 
generate many false alarms.) In the ODID (Prosser et A1 ,1991, Graham et al 1995) 
research simulations, a Conflict Risk Display was provided which consisted of: - 

• A rectangular display showing time-to-go as the horizontal and minimum separation 
as the vertical axis. Numbered points identified unsolved conflicts. A line represented 
the duration of separation loss. 

• A table showing the call-signs of the pairs of aircraft involved in each conflict. 

To resolve a conflict, the controller made a mental note of the number, referred to the 
table, made a mental note of the two call-signs, then searched the radar display to find 
the two aircraft involved- He then estimated their relative speeds, noted their heights, 
checked or recalled their future trajectories, estimated their relative positions at the time 
of conflict. (It is often easier to solve a conflict by maneuvering the second aircraft to 
pass the crossing point.) Experienced controllers would scan the table of call-signs to see 
if one aircraft was involved in several conflicts, and search for a maneuver which would 
solve them all. This display was greatly liked by controllers. It may be that this was 
because it was usually blank, assuring them that that they had not missed a conflict in 
their normal scanning of the display, a constant fear of all controllers. 

Revised display 

We will assume that we use a map-like display as our primary display, showing aircraft 
at positions by symbols. Figure 2 shows a representative contemporary and a revised 
aircraft symbol. 


BAW1 23 
273 / 29 


Radar Image 



Figure 2 - Aircraft image 

Figure 3 lists the aspects displayed, with brief notes justifying the choices of display 







X,Y position 

of symbol 

Not used 

X,Y Position 

select by mouse 
of wing/body 




Speed Vector 


direction of ion 
Size of symbol 





Vector length 
3-digit code 

-f- final level 

Wing sweep 



Figure 3 - Coding 

Height is indicated by colour coding, Eastbound cruising levels are shades of yellow, 
westbound are shades of blue. Aircraft in level flight are normally at a fixed level, and 
are coloured accordingly. Aircraft climbing or descending have the front end in the level 
they are approaching, and their rear in that they are leaving. The speed indication is 
highly approximate, but is rarely significant (see below). 

Figure 4 shows a representative screen, as displayed on a rather primitive VGA display. 
There are 40 aircraft in random flight on this display. Seventeen of these aircraft are 
involved in ten potential future conflicts. 


■ • At. , ' "■ 

i. .. , ■ ■ 

• ■ • /A; •:: . 

0 0-0 TleH 

Display Codes 

A '= All a/c * 

B = No H Sep 

C'= Conf1ict 

X . ■ ■ ■' : 

T = Trail , 

U = Uector 

I = Icon * 

...LA ■ ■ 

A.'-T.,"' v';' 

K - Colour » 

S = Sepn Cir 
. L .= 1 inks 

:N = A/c ID 

0 - Orders 

P = Profiles 

U = Wait * 

i X = eXit 


Speed up 1 

Figure 4 - Revised Display 

Figure 5 shows the potential conflicts present in this image. Conflicts are displayed by 
linking the aircraft involved. This leads the eye to the aircraft most involved in conflicts. 
Lines linking conflicts are made more salient as the time to go before loss of separation 
becomes shorter. Figure 5 also shows a logical extension of the simplification process, 
where aircraft that are not involved in conflicts are not shown to the controller. Many, 
possibly most aircraft would pass through the airspace without contacting the controller 
at all. 

(The reader is invited to try to identify the aircraft and their conflicts before looking at 
Figure 5.) 

,i. - yk' 

! 0 0 0 IM e i a 

Display Codes 
A = All a/c 
; B = No H Sep 
C = Coiif 1 ict * 
■■ ■ . 

■T = Trail 
0 = Mector 
' I = Icon « 

; K = Colour » 
S = Sepn Cir 
L = 1 iiiJts * 

. N = A/c ID 
i'O ,= Orders 

iV p > = Prof i les 

U = Wait 
X = eXit 

ii; ..Speed up 1 

Figure 5 - Potential Conflicts 

To resolve conflicts, instructions must be given to one or more aircraft. At present, these 

are given by the Executive Controller, by RTF, The EC, operating in the present, gives 
instructions to be executed directly by the pilot. He must monitor the pilot’s reply, to be 
sure that his order has been correctly received. He must check a minute or so later that 
the aircraft is actually doing what he asked. He must note, or remember, when he has 
altered the direction of an aircraft, in order to return it to track or height later. He must 
estimate a sufficient diversion from track to respect the required separation. For safety s 
sake, he overestimates. Where the separation minimum is five miles, he usually adds 
“two more for the wife and kids”. 

Conflicts are often detected by the Procedural controller, when comparing strips. He may 
annotate a strip to tell the EC to move the aircraft to a higher or lower level before a 
certain beacon. Since he cannot be certain exactly when the EC will make the move, both 
levels may be blocked for a considerable time. 

Our proposed display (Figure 5) allows the controller to choose which aircraft he will 
maneuver, but the system can be set up to choose the next aircraft to treat according to 
pre-defined criteria. In the currently available version, the pair of aircraft which will first 
lose separation is selected, then, within that pair: 

• The aircraft involved in most conflicts 

• The aircraft that not at the level it should leave at 

• The aircraft that has furthest to go before leaving - because this is easiest to return to 
track after a maneuver. 

Mr. ■ ' 

1 q = QUIT 

■ • 

1 ;Uaiting... 

0 0 0 "ffSTdf 

T>Tine (min) 
S>Speed (20kt) 
F>FL (Z0Fls) 
H>Hdcf (10deg) 
R>Res Nor Nau 

+ = Add 1 
- = Back 1 

X = Cancel 
? = try out 

Figure 6 - Aircraft Trajectory 

Having selected an aircraft, the system shows its future trajectory superimposed on the 
display, with the trajectories of conflicting aircraft, and their relative positions at closest 
approach. An auxiliary display shows the flight profile of the aircraft, with a horizontal 


scale representing distance to the same scale as the EDD, and a vertical scale showing 
the height (flight levels) available. Solid blocks represent aircraft which will be in 
conflict, and hollow blocks represent aircraft which pass over or under the flight, and 
therefore would be in conflict if the flight level is changed. (Figure 6) 

The controller can construct an order to solve the problems of this aircraft. (In the 
demonstration version DEMFAST, available with David (1997), keystrokes are used, 
because the software available does not support a mouse, although a mouse would be 
preferable.) As the order is constructed, the trajectory is revised, so that the consequences 
of each maneuver can be seen. The controller can instruct the aircraft to return to track 
after the problems have been solved. Once an acceptable order has been developed, the 
controller accepts it, and the system transmits it by data-link to the FMS of the aircraft, 
adding the call-sign. The message would be spoken to the controller and to the pilot and 
co-pilot of the aircraft addressed (and of any other aircraft in the vicinity) by voice 
synthesizers in the language of their choice, eliminating the classic Zagreb problem 
(Weston and Hxirst, 1982), The controller can now turn his attention to the next problem 
without needing to verify the correct reception of the order, or to remember to return the 
aircraft to track later. 

After a little practice, this system is considerably faster than spoken control. A non- 
controller can handle the demonstrator’s limit of 40 aircraft simultaneously, where each 
aircraft remains in the airspace for 12 minutes, corresponding to an entiy rate of 200 
aircraft per hour, at twice real-time rate. 

In a real system, this demonstrator would be expanded to include methods of monitoring 
deviations from track, or learning more about the aircraft. Controllers would learn to 
recognize particular ‘hot-spots’ in the traffic, and anticipate problems, as they do in the 
existing system. Additional displays might be designed to help the controllers cope with 
‘foreseeable emergencies’, such as emergency decompression or engine failure, where 
one controller would take charge of the aircraft in difficulties, while his partner would 
supervise the re-routing of other aircraft. 


Although controllers enjoy, and are good at, solving conflicts, there may be occasions 
where a temporary overload means they cannot solve all the problems in time. There 
should be a process triggered by the equivalent of a ‘short-term conflict alert’ to ensure 
safety if potential conflicts are not solved within a safety limit. The ACAS (Airborne 
Conflict Avoidance System) is in fact, such a system. The fail-back system should be 
essentially safe, while not as efficient as the human controller. Given the nature of human 
beings, the temptation to rely on the back-up system should not be offered. 

Organizational Repercussions 

The alternative display system designed here is not simply a cosmetic re-design of the 
existing system. It produces a considerable increase in capacity, reducing the number of 
sectors, which itself reduces the amount of coordination involved. It reduces the mental 
workload of the controller at the cost of his ability to maintain a continuous ‘picture’ of 
the present and future traffic. It can be argued, although it would be hard to prove, that 
he could not maintain a picture of this amount of traffic in direct flight in any case. The 


traditional distinction between the Planning and the Executive controller disappears. (In 
real Ufe ATC practice, the ‘official’ distinction between these two tasks is already 
considerably blurred.) The ‘style’ of control changes from a continuous ‘hands-on’ style 
to a ‘control by exception’, where the controller must react to the (future) problems 
detected by the system, and the immediate problems presented by emergencies. This 
would have considerable implications for initial and continued training. There are very 
good reasons not to reduce the control team to one man, as there are not to reduce 
aircrew, (Yerkes Law - “One chimpanzee is not a chimpanzee at all” (Yerkes 1945) also 
applies to controllers) As for aircraft, the trend will be to a pair of controllers, each 
capable of all the control tasks. Training may include drills for the allocation of tasks 
between operators (and the system), as with CRM (Cockpit Resource Management) in 
civil aviation. 


Finally, it is an illusion, induced by the linear nature of written communication, to see 
these stages as separate steps in a process. In reality, each step will interact with every 
other. To attempt to develop a complete system by following each step rigidly is rather 
like trying to teU a car driver how to drive from one place to another by providing a 
detailed, carefully timed set of instructions how to move the steering wheel, the clutch 
and the accelerator for an hour or so. The processes of task design and interface 
specification are inherently interdependent. Practically this means that crude models 
rapidly available are more valuable than elegant models that take six months to modify. It 
implies that the number of people involved in the design and development phase should 
be as small as possible. (Ideally, one.) This type of development requires the attitudes 
and commitment described in ''Skunk Works"' (Rich and Janos, 1994), although the 
physical resources need be no greater than a standard PC with a good display screen. 


Most large-scale real-time control systems have evolved, rather than been designed. 

The advent of fast, cheap and reliable computer-based information transfer and analysis 
systems presents an unprecedented opportunity to re-design such systems to provide a 
major improvement in the quality, reliability and efficiency of the system. 

It is not sufficient to imitate the existing display system on a CRT screen. The actual 
control task should be re-analyzed, and the human operator provided with a satisfying 
task appropriate to his natural abilities. Tasks the human fails to do should be taken over 
by the system, although it is important that the system should be safe, rather than as 
efficient as the human operator. 

These changes imply shifts in the location of information storage, in the means of 
transmission, and in the manner in which operators work. 



Cushing, S. (1994), Fatal Words: Communication Clashes and Aircraft Crashes, 
University of Chicago Press, ISBN 0 226 13200 5 

^David, H. (1997) Radical Revision ofEn-route Air Traffic Control. EEC Report No. 
307. Eurocontrol Experimental Centre, Bretigny-sur-Orge, France 

Dee, T.B. (1996), Ergonomic Re-design of Air Traffic Control for increased capacity 
and reduced stress, in Contemporary Ergonomics 1996, ed Robertson, S.A., 
Taylor and Francis ISBN 0 7484 0549 6 

Fitts, P.M., Jones, R.E, (1961) Analysis Of Factors Contributing To 460 'Tilot-ErroF' 
Experiences In Operating Aircraft Controls and Psychological Aspects Of 
Instrument Display: Analysis Of270 “Pilot-Error'* Experiences In Reading And 
Interpreting Aircraft Instruments in Selected papers on Human Factors in the 
Design and Use of Control Systems. Ed Sinaiko, H.W. Dover Publications Inc., 

Graham, R .V., Young, D., Pichancourt, L, Marsden, A,, Ddz, A. (1994) ODIDIV 
Simulation Report, EEC Report No. 269., Eurocontrol Experimental Centre, 
Bretigny-sur-Orge, France. 

Hansen, J.P., (1995) Representation of System Invariants by Optical Invariants in 

Configural Displays for Process Control in Local Applications of the Ecological 
approach to Human-Machine Systems, Eds Hancock, P,, Flach, J., Caird, J., 
Vicente, K. Lawrence Erlbaum Associates, Hillsdale New Jersey ISBN 0 8058 
1379 9 

Lafon-Millet, M-T., (1978) Observations en trafic reel de la resolution des conflits entre 
avions evolutives, Raport INRIA No CO/R/55, INRIA, Rocquencourt, France 

Phillips, M.D.,Tischer, K„ Ammermann, H.A, Jones, G.W., Kloster, G.V. (1984) 
Operations Concept for the Advanced Automation System Man-Machine 
Interface, Computer Technology Associates, Englewood, USA 

Prosser M., David, H., Clarke, L. (1991) ODID III Real-Time Simulation, EEC Report 
No. 242., Eurocontrol Experimental Centre, Bretigny-sur-Orge, France 

Rich, B.R., Janos, L. (1994) Skunk Works, Little, Brown and Company, New York, 

ISBN 0 7515 1503 5 

Springer, S.P., Deutsch, G. (1993) Left Brain, Right Brain, Revised 4th ed (originally 
published 1981), W.H. Freeman, New York ISBN 0 7167 2372 7 

Stein, E.S„ Bailey J., (1994) The Controller Memory Guide: Concepts from the Field. 
DOT/FAA Technical Center, Atlantic City, New York, 

Weston, R. Hurst, R. (1982) Zagreb One Four: Cleared to collide?, Granada Publishing, 
ISBN 0 246 11185 2 

^ This report includes a 3.5”disc with compiled code, source code and documentation for 
demonstrations of traditional (DEMOLD), intermediate (DEMON) and fast interfaces 


Woods, D.D, (1995) Towards a Theoretical Base for Representation Design in the 

Computer Medium in Global Perspectives on the Ecology of Human-Machine 
systems Ed Hach, J., Hancock, P., Caird, J., Vicente, J. Lawrence Earlbaum 
Associates, Hillsdale New Jersey ISBN 0 8058 1381 0 

Yerkes, R.M. (1945) Chimpanzees, a Laboratory Colony, Yale University Press, New 
Haven Connecticut. 


Advanced Manoeuvre Flight Training on PC’s 
and Transfer to the Real Aircraft 

Jan JM, Roessingh and Staszek F. Chlapowski 
Man-Machine Integration and Human Factors Department 
National Aerospace Laboratory NLR 
Amsterdam, The Netherlands 


Through ongoing progress in computer hardware, numerous improvements became pos¬ 
sible in flight simulation. In general, low-cost desktop simulators now provide possibili¬ 
ties that in the past could only be achieved on expensive and complex systems. In addi¬ 
tion to a quality improvement in low-cost desktop simulation, there is also a quantitative 
advantage. A wide range of complex tasks can nowadays be trained on relatively simple 
and inexpensive equipment, hi fact, a number of tasks can already be trained with ’State 
of the Art PC’s, providing the opportunity to procure a larger number of this type of 

This paper examines whether PC-based equipment can be used to support training of 
perceptual-motor skills, or more specific manual flying skills, i.e. skills that to date sup¬ 
posedly can only be acquired in more expensive simulators or the real aircraft. 

The motivation for the research described in this paper is lead by the basic question: 
what task-critical features are needed in simulation to achieve positive transfer to manual 
flying skills in real flight? 

To examine this question, a training experiment has been conducted. The objective of 
this experiment was to test the training-effectiveness of PC-based simulations used in 
advanced manoeuvre flight training. The experiment was set up such that transfer-of- 
training from two PC-based training devices- a standard (Commercial Off- The-Shelf, 
COTS) and a customized PC-based configuration - to the real aircraft could be exam¬ 

Three groups of trainee-pilots participated in the experiment. Each group went through 
identical in-flight training to learn to fly a complex sequence of aerobatics manoeuvres 
on a single-engine propeller aerobatics aircraft. The second and third group, though, re¬ 
ceived extra training: each in-flight lesson was preceded by a simulation session using 
the respective PC-based configurations. 

It was found that the group trained with customized PC-based simulation showed a clear 
advantage over the two other groups, trained with respectively standard PC-based simu¬ 
lation and no simulation. The use of customized PC-based simulation leads to a higher 
rate of learning in the aircraft. Most remarkable improvement of in-flight performance of 
the group trained with customized PC-based simulation takes place in the first lessons, 
giving them a headstart, causing an improvement that is existent throughout the remain¬ 
der of the training. It is discussed that this higher rate of learning observed by trainees 
using customized simulation can be accounted for by the following abilities learnt in the 
simulation that transfer to real flight: (1) the improved ability to extract visual cues from 


the outside world that are needed to perform the time-critical elements of aerobatics (2) 
speeded up learning of procedural elements of aerobatics in an earlier stage of training 
(3) an improved basis for in-flight development of “seat-of-the-pants skills”. 


This research applies in the first place to the training of perceptual-motor skills, more 
specific advanced manual flying skills for military operations. Military operations with 
jets and helicopters continuously depend on perceptual motor-skills. A real-life military 
scenario is likely to be a continuous sequence of very complex manoeuvres. Additional 
tasks, most often with cognitive and procedural aspects, must be performed in parallel 
with manoeuvering. Therefore, a high degree of skill automatization is desirable for suc- 
cesful completion of such a scenario. Such a high degree of automatization of perceptual- 
motor skills can only be obtained through extensive training. Unfortunately, shrinking 
military budgets and increasing public environmental awareness restrict the number of 
hours that military pilots can spend training advanced manoeuvres on real aircraft. 

PC-based systems have already proved useful in training of aircraft system knowledge, 
flight management system programming, radio-telephony phraseology, navigation, and 
many more areas. Thus far, such systems have not been used for teaching manual flying 
skills. These skills were usually taught on real aircraft or on training devices for which 
the design was most often guided by the desire for high engineering fidelity under the 
assumption that the more a training device is like the real aircraft, the better will be the 

Above restricting circumstances and advances in PC-based software and technology pro¬ 
vide a reason for the aviation community to investigate cost-effective use of advanced 
PC’s in part-task trainers for manual flying skills. Not with the intention to replace the 
real aircraft training or the full flight simulator but with the intention to support the 
training program, thus to use the real aircraft and the full flight simulator more effec¬ 
tively and more selectively. 

In our view, for a part of flight training, the term low-fidelity does not necessarily imply a 
sub-optimal training environment when compared to the real aircraft. A first condition for an 
effective low-fidelity device is the presence of identical elements that are critical to required 
task-performance, allowing the trainee pilot to develop the proper response strategy. This 
characteristic of the training device to allow the development of a proper response is more 
important than having an exact replica of the cues involved (the stimulus). 

Secondly, the low-fidelity training device may provide some instructional strategies superior 
to those attainable in the real aircraft. Examples of such instmctional strategies are part-task 
and adaptive training, extensive feedback, for example through Knowledge of Results (KR) 
followed by an explanation and demonstration of the correct response, provision of replay 
possibilities and cue augmentation, that is the use of cues that are not present or have insuf¬ 
ficient cueing effect in real flight, but may have a positive effect when they are introduced or 
amplified in the training environment. Lintem (1996) provides an overview of transfer- 
theories and existing research on flight simulation fidelity research to date. 

The current research is distinguishable from earlier research in the field of training effective¬ 
ness of low-cost/low-fidelity simulation because of its focus on the acquisition of perceptual 


motor skills (aerobatics skills), in-flight measurement of both learning curves and final per¬ 
formance, the optimal configuration for PC-based simulation and the use of augmented 
cues, and implications for selection of pilots. 


Experimental set-up and hypotheses 

The experiment entails a comparison between three groups of subjects during 10 flying 
lessons; the first 8 flying lessons were considered as training and the last 2 flying lessons 
considered as the transfer demonstration phase. The experimental set-up involving those 
three groups is depicted in table 1. Choices concerning duration and number of lessons 
were based on experience and common sense. 

Control Group (C) 

Exp. Group 1 (XI) 

Exp. Group 2 (X2) 

In-flight training 

10 lessons of 30 mins, 

id. to C-group. 

id. to C-group. 

Simulation training 

no simulation sessions 

10 simulation sessions of 
50 mins. each. 

10 simulation sessions of 
50 mins. each. 

Simulation configura¬ 

Table 1: Experimenta 


/ training set-up 

PC-based standard 


PC-based customized 

equipment + augmented 

From table 1 it can inferred that the experimental manipulations were: 

• the use of PC-based standard equipment in simulation sessions for group XI, 

• the use of PC-based customized equipment and augmented cues in simulation ses¬ 
sions for group X2. 

The average score (see following section on 
performance measurement and scoring) of 
each group in the transfer-demonstration 
phase will be mutually compared under the 
hypotheses that the average score of the 
XI-group is higher than that of the C-group 
and that the average score of the X2-group 
is higher than both those of XI- and C- 

The same hypotheses will be tested^ with 
respect to the average progress per lesson 
Further, the magnitude of the effects (if any) 
and those effects will 

(learning speed) during the training phase, 
resulting from the experimental manipulations wil be investigated 
be related to the shape of the learning curves for the three groups. 

^ The Statistica software package (StatSoft, 1994) will be used for statistical analysis. Significant results 
will have a p-level smaller than 0.05. 



The goal of the aerobatics training was to fly five aerobatics manoeuvres in a fixed order 
continuous sequence on an aircraft (see section ‘aircraft’). Those five aerobatics ma¬ 
noeuvres are the Loop, Slow Roll, Inverted Flight, Immelmann and Split-S. Each of those 
manoeuvres takes a skilled pilot approximately 20 seconds to complete. 

The order Loop-Slow-roll-Inverted-flight~Immelmann-Split-S was a logical choice: The 
loop and the slow-roll form the basis for both the Immelmann (which is a half-loop-half- 
roll) and the Split-S (which is a half-roll-half-loop). The Slow-roll is the basis for in¬ 
verted flight (which is a half-roll, level inverted for 15 seconds, and again a half roll). 
Moreover the Split-S was the last manoeuvre in the sequence since it is physically the 
most demanding, mainly caused by the transition from negative to positive g-force, and 
also the most critical manoeuvre with respect to loss of altitude. 

Performance measurement and scoring 

The manoeuvres were judged by the aerobatics instructors (instructor ratings) on a re¬ 
stricted number of criteria. The criteria were chosen and prioritized in agreement with the 
instructors on the basis of importance of a particular criterion for the manoeuvre and the 
ability to observe whether a criterion is fulfilled. For example, a g-force of 3.5 to 4 g, 
pulled at loop-entry, is very important for succesful completion of the loop and can be 
observed by both instructor and trainee and is therefore a suitable criterion. On the other 
hand, during a slow-roll manoeuvre it is very hard for the instructor to observe and judge 
the roll-rate in an absolute sense, although it is an important criterion. Therefore a roll 
rate of say, 20 degrees per second, is not a suitable criterion as a basis for instructor rat¬ 
ing. In this case a relative criterion (e.g. roll-in-rate equals roll-out-rate) is more suitable. 

A sufficient small number of criteria were chosen, such that the instructor, seated behind 
the subject on the back seat of the airplane, was able to fill out the pre-printed score- 
forms. Scoring was based on criteria of equal importance, that is aU criteria had an 
identical weight factor. All manoeuvres were considered as being of equal importance, 
that is were scored on an equal number of criteria. During the in-flight lessons, the 
trainee gained a point for each criterion that was fullfilled, that is when the associated 
parameter value fell within the acceptable range (see table 2 for an example). 



Phase in manoeuvre 

Ideal value 



Roll an¬ 

pull-up (phase 4) 

0 deg. Constant 

-5 tot +5 deg. 

Max g 

pull-up (phase 4) 

4g at entry speed of 145 mph 

3.5 to 4g 

Roll an¬ 

top (phase 7) 

0 deg. Constant 

-10 to -flO deg. 


top (phase 7) 


-5 to 4-5 deg. 


exit loop (phase 10) 

- 100 to - 1-100 feet 

Table 2: Example: criteria for scoring the quality of the loop 

Selection of subjects for the experiment 

In order to compose three groups of 8 subjects, 24 subjects were needed for the experi¬ 
ment. These were selected in three selection-rounds. 


In co-operation with a school for commercial pilots, approximately 60 trainee pilots at¬ 
tended a briefing and completed registration forms. The first selection-round took place 
on the basis of body weight (max. 80 kg, because of the restriction in take-off weight of 
the airplane), age (max. 27 yrs), fixed-wing flying experience (max. 250 hrs), previous 
aerobatics experience (no previous aerobatics experience allowed). 

With a remaining number of 31 trainee pilots, in a second selection round, the Aiming 
Screening Task (AST) was used to collect data on individual ability. The AST was de¬ 
veloped for the Learning Strategies Program (Donchin et al, 1989). This eye-hand co¬ 
ordination task was used since it is a well know selection task for research into percep¬ 
tual-motor skills. The objective of the AST is to destroy as many targets on a computer- 
screen as possible within a session of 2 minutes, using a joystick. The task consists of 
three sessions of which the highest score (number of targets destroyed) was taken as the 
selection measure. 

In the current research the AST was not used to remove potential subjects from the actual 
experiment but merely for balancing purposes between the three groups (conditions). On 

the basis of AST score, 4 
ability-levels were distin¬ 
guished: Low (L), Low- 
Medium (LM), High- 
Medium (HM) and High (H), 
corresponding with respec¬ 
tive quartiles of the distribu¬ 
tion depicted in figure 2. 
After assignment of subjects 
to groups, each group of 8 
subjects contained 2 L, 2 
LM, 2 HM and 2 H subjects. 
The remaining subjects were 
assigned to a spare group. 
The third selection round 
consisted of a 20 minutes test-flight on a single-engine propeller aircraft of make Fuji, a 
side-by-side seater with aerobatics capabilities. Deliberately, this aircraft was of a differ¬ 
ent type than was used for the actual experimental training to avoid confusion between 
the pre-training and training-phase. The subject had to perform a number of basic flight 
manoeuvres and a number of aerobatics manoeuvres, the latter deliberately being differ¬ 
ent than those included in the actual experimental training. The test-flight had two ob¬ 
jectives: firstly to collect data on the basic and advanced flying abilities of the subject 
and secondly to test and possibly remove subjects that had insufficient physical resis- 
tence against air sickness to complete the aerobatics training, A pre-printed score-form 
was used by the instructor. On the basis of test-flight results three subjects were removed 
from the experiment and replaced by subjects from the spare group. To avoid a potential 
bias in the behaviour of subjects, due to knowledge of the objectives of the experiment, 
the experimenters presented the project as a study of in-flight instructional strategies 
rather than a study into PC-based training. The three groups were trained separately. 

. Expected 

300 400 500 600 700 800 900 1000 1100 1200 Normal 

Aiming Screening Task scores 

Figure 2: Distribution of Aiming Screening Task Scores 


Training procedure 

Table 3 represents the total training structure for the different groups. The contents of 
the lessons in each phase will be briefly described, starting with the training elements 
that all groups had in common. 

In the pre-training phase, all subjects received the training manual. The theory in the 
manual had to be mastered before the start of the training. Before entering the flight 
training, the subjects had to go through a theoretical test on the contents of the manual. 


Theory and test 

Flight lesson 1 

Flight lesson 2 

Flight lesson 3-8 

Flight lesson 9 

Flight lesson 10 



Theory and test 









Fam. Phase 

Training phase 

Transfer-demonstration phase 

Table 3: Organization of training sessions for C group and XI/X2 groups 

Before and after each flight lesson the trainee received a briefing by the instructor. The 
first flight lesson (Familiarization phase, see table 3) was used to familiarize with the 
aircraft. Furthermore, the instructor demonstrated the complete sequence of five ma¬ 
noeuvres after which the trainee could try the sequence him/her self in a talk-through 

Figure 3: Organization of a flight lesson in the training phase 

In the training phase (flight lesson 2-8) a part-task instruction strategy (see Wightman 
and Lintem, 1985) was employed. The basic organization of each in-flight lesson is de¬ 
picted in the flowchart of figure 3. A logical choice was to segment the whole task (fly- 


ing the sequence) into its component manoeuvres 1-5 (loop =1, slow-roll = 2, etc.) and 
train those as part-tasks. Thus, the second in-flight lesson started with trials on the least 
complicated manoeuvre (the loop). If this manoeuvre could be flown according to the 
criteria, trials on the next manoeuvre (the slow-roll) were undertaken and so on . Once all 
separate manoeuvres were mastered, the same order of manoeuvres was repeated (outer 
loop of figure 3). 

In the transfer demonstration phase (lesson 9-10, see table 3), the trainee had to demon¬ 
strate the whole sequence of five manoeuvres in a continuous and errorless fashion. Em¬ 
phasis was on accuracy: “try to fly the sequence as accurate as possible”. 

Both XI and X2 group subjects received 50 minutes simulation sessions preceding each 
in-flight lesson. Simulation sessions for both groups were identically organized. 

The first simulation session (in the familiarization phase, see table 3) consisted of fa¬ 
miliarization with the simulation, a trial on each manoeuvre and a trial on the complete 

Simulation sessions 2-10 consisted of (1) structured repetition of problematic manoeu¬ 
vres as indicated by the flight instructor during the previous in-flight lesson and (2) 
preparation of new manoeuvres. No mediation by instructors was needed. 


The aerobatics aircraft used for the train¬ 
ing is of type Bellanca Super Decathlon 
depicted in Figure 4. This single-engine 
propeller aircraft is thrusted by a four 
cylinder 180 HP Lycoming piston engine. 

The aircraft can take g-loads in the range - 
5 to 6 G. Equipment on-board of the air¬ 
craft was used for recording 12 relevant 
flight parameters during the training. The 
recorded data is not yet taken into account 
in this article. 

Figure 4: The Bellanca Super Decathlon 

^ For didactical reasons, the instructor had the possibility to start with a next manoeuvre 
even though not all criteria were fulfilled. The rate with which trials were repeated was 
left to the decision of the instructor and his opinion on the skill-level and prospects of the 


PC‘based training equipment and software 

Both groups used the aerobatics training software package “Flight Unlimited” from 
Looking Glass Technologies (MS-DOS version. Looking Glass Technologies, 1995). 

Flight Unlimited is a 
PC flight simulation 
with relatively accu¬ 
rate aircraft flight 
models (including 
that of the aircraft 
used in the experi¬ 
ment) and 3D 
photo-realistic land- 

FigureS: Controls (throttle, pedals and stick) for the XI-group scapes. A view was 

selected in which on the lower half of the screen the instruments were depicted, while the 
upper half of the screen was used for out-of-the-window view. Furthermore the “ren¬ 
dered’ was set up such that terrain was in lowest detail, haze was on highest level, clouds 
were turned off, blending set on lowest level, g-effects were on and wind strength was set 
to a maximum. 

XI- and X2-groups used the same simulation software and the same PC (Pentium 166 
equipped with a sound card and speakers and a CD-ROM player). The differences in 
equipment are listed in table 4. 




17” colour monitor, 
used in VGA-mode 
(640x480 pixels) 

21”, further identical to XI 


Pro Throttle by CH- 
products (see figure 5) 

Throttle, rudder pedals and stick were custom¬ 
ized and mounted on a base such that their 
shape, position and displacement resembled 
those in the aircraft (see figure 6) 


Pro Pedals by CH- 
products (see figure 5) 


Flightstick Pro by CH- 
products (see figure 5) 


Standard bureau chair 

Mounted on base and resembled the chair in the 
aircraft such that trainees were able to occupy 
the same bodily posture as in the aircraft (see 
figure 6) 

Table 4: differences in equipment configurations between XI- and X2-groups 

Furthermore, subjects of the X2-group were presented with the following augmented 
cues during their simulation sessions: 


• guidance by a digitized instructor voice (feature of the Flight Unlimited software) during 
their simulation sessions (subjects were “talked through” the manoeuvre); 

• lesson arrows (feature of the Flight Unlimited software), those are indications for 
suggested stick and pedal movements were presented in real-time on the screen 


Invalid data and outliers 

For methodological reasons, data from three subjects had to be removed from the ex¬ 
periment. From the C-group, data from one subject was removed from the analysis since 
incorrect information had been received from the flying school. On the basis of new in¬ 
formation, it was decided that this subject could not be considered as representative for 
the trainee pilot population under study. From the XI-group, data from one subject was 
removed since this subject had a physical problem that hindered the subject to look 
sidewards under the high g-forces that occurred in most manoeuvres, and could therefore 
not complete the aerobatics training. From the X2-group, data from one subject had to be 
removed since this subject turned out to have significantly more fixed wing experience 
than initially reported. Furthermore, one outlier (unusual low-score score of a C-group 

subject) was removed on the basis of the 
3s-criterion used in residual analysis. 

Instructor ratings 

It must be noted that the results currently 
presented in this article are based on in¬ 
structor ratings. ^ The three instructors 
were randomly assigned over a total of 
240 flight-lessons and were instructed to 
score using pre-printed score forms with 
performance criteria in terms of objective 
and observable flight parameters (see ex¬ 
ample in table 2). To examine inter-rater 
reliability, stepwise regression analysis 
was conducted, in which the variables rep- 
Figure 6: Customized PC-based trainer for resenting the instructors (3 lev- 
the X2-group els/instructors coded in two dummy vari¬ 

ables) were two of several independent 
variables (simulation-configuration, ability-scores measured in the pre-training phase, 
etc.) and in which the score during the transfer-demonstration phase was the dependent 
variable. The dummy variables representing the instructors did not significantly contrib¬ 
ute to the model for regression, nor did those variables account for a substantial percent¬ 
age of the variance in scores. It is realised that these results do not formally prove inter¬ 
rater reliability. In view of time constraints and the availability of objective measure¬ 
ments more formal tests have been abandoned for this article. 

^ At the time of preparing this article the analysis of the recordings with the airborne 
measurement equipment is still ongoing and will be reported in the course of 1997. 


Representation of in-flight scores 

The instructional strategy employed in the training-phase was directed on leaming-by- 
doing and thereby flying as many manoeuvres as possible. A different approach was 
taken in the transfer-demonstration phase where emphasis was put on demonstrating a 
restricted number of sequences wiA the highest possible accuracy. For the training 
phase, we therefore calculated the in-flight score per lesson by summing the total number 
of points gained in that lesson, thus representing both speed (number of trials per lesson) 
and accuracy of the manoeuvres. For the transfer-demonstration phase however, we will 

calculate the in flight score by di¬ 
viding the points gained in flying the 
sequence by the number of se¬ 
quences flown in that phase, thus 
merely representing the accuracy of 
the sequence. For interpretation and 
comparison purposes, all scores will 
be scaled to 100%, through division 
by maximum attainable accuracy 
score and maximum number of ma¬ 
noeuvres per lesson observed. 

The objective of the transfer- 
demonstration phase was to fly con¬ 
tinuous sequences with maximum 
possible accuracy. Mean scores for the three groups are depicted in figure 7. The C- 
group scored 89.3% (M=89,3, s=13.6), the Xl-group scored 92.5% (M=:92,5, s=5,5) and 
the X2-group scored 98.2% (M=98.2 s=2.91). Wilcoxon matched pairs test was used to 
compare means. The difference in mean score between C and XI group is not significant, 
the difference between C and X2 group is significant (p=0.017), the difference between 
XI and X2 group is significant (p=0.015). 

Thus, while the Xl-group, that is the group using the standard simulation equipment, 
failed to show significant transfer, the X2-group proved to have a significant advantage 
from using the customized configuration. The differences in performance score in the 
transfer demonstration phase are relatively small (The X2-group scored only approx. 9% 
better that the C-group). However, it must be noted, that scores had a tendency towards 
100% in the transfer-demonstration phase. Since scores were based on a restricted num¬ 
ber of relatively simple criteria, near-maximum accuracy could relatively easy be 
achieved. This ceiling-effect in performance score (based on accuracy) is thought to 
mask the real difference in skill level in the transfer demonstration phase. Further analy¬ 
sis of in-flight data on the basis of a larger number of more sophisticated scoring criteria 
would reveal whether this assumption is true. 

Figure 7: Mean scores in transfer demo phase 


Learning speed 

The in-flight learning 
curves for the three 
groups are depicted in 
figure 8. Learning 
speed is evaluated in 
terms of average 
per lesson. It can be 
seen from figure 8 
that initial levels of 
performance (lesson 
1) are approximately 
equal. Looking at the 
training phase as a 
whole, the C-group 
had an average im- 
Figure 8: Learning curves provement of 4.6% 

per lesson, the XI- 

group had an average improvement of 5.3% per lesson and the X2-group had an average 
improvement of 6.9% per lesson. The differences in improvement per lesson between C- 
and X2-group and XI- and X2-group are significant (p resp. 0.00045 and 0.0025), while 
the difference between C - and XI-group is not significant (p=0,14). From figure 9 it 
can be seen that the most remarkable improvement of the X2-group (approx. 20%) is 
booked early in the training phase and is existent throughout the remainder of the train¬ 

Linear approximation of learning curves 

In order to assess aerobatics skill-level during the training phase of the three groups in 
terms of speed and accuracy under the assumption that performance wiU eventually 
level-off at an asymptotic level the scores per lesson were approximated by linear curves, 
using the same data as in figure 8 (see figure 9), Results of the analysis, using least 
square approximation are included in table 5. 

Model: Score = A * lesson no. + B 


Intercept B 
(begin level) 

Slope A (learn¬ 
ing speed) 

/F (coefficient of de¬ 

F-value (overall 
model utility) 



16.4 % 

4.6 % per lesson 





15,7 % 

5.3 % per lesson 





17.5 % 

6,9 % per lesson 




Table 5: results of linear approximation of the learning curves 

Figure 9: Linear approximation of learning curves 

These results show that all learning curves are approximately linear and do not curve to¬ 
wards an asymptote, indicating that skill level could be further improved and the training 
phase was too short to approach the maximum attainable skill level with the task given. 

Ability data and relation with aerobatics performance 

It was hypothesized that ability data collected in the pre-training phase (AST-scores, 
standard flying skills scores and advanced flying skill scores) are good predictors for 
aerobatics performance. To evaluate their relative predictive value ability scores were 
included as variables in a standard multiple regression analysis together with the inde¬ 
pendent variable of tlie experiment. For this purpose, the experimental manipulation of 
the experiment at three levels (no simulation, standard simulation, customized simula¬ 
tion) is coded with two qualititive dummy variables XI and X2. The dependent variable 
is the performance score (accuracy) obtained during the transfer demonstration phase. 
The results are summarized in table 6 (significant results are underlined). 



Score (transfer demonstration phase) = Bo+Bi*AST+B2*STD+B3*ADV+B4*Xl+B5*X2 

R = 0.64, R^=41%, F(5.35W.77. p<0.002___ 


Magnitude Bn of the effect 
scaled to a range [-1,1] 



Aimine Screenine Task f AST) 




Standard flying task (STD) 




Advanced flvine task (^ADV) 




Use of COTS simulation (XI) 




Use of customized simulation (X2) 




Table 6: Multiple regression analysis results, relating several variables to aerobatics 
performance in the transfer demonstration phase. 

The results from the analysis show that advanced flying ability (ADV, measured on three 
advanced manoeuvres: wing over, barrel roll and spiral dive), use of customized PC 
simulation (X2) and AST-score yield significantly to the prediction of aerobatics per¬ 
formance in the transfer demonstration phase, of which the latter one is of less impor¬ 
tance, in terms of the magnitude of the effect. The use of standard PC-simulation does 
not contribute significantly to the model, as could be expected from our earlier analysis. 
Standard flying ability, as measured during 9 standard manoeuvres, does not seem to 
bear a correlation with aerobatics performance. 

Conclusions and discussion 

Research has been carried out to evaluate transfer-of- training of perceptual motor skills 
(aerobatics skills) to the real aircraft from three different types of ground-training. It was 
found that the group that was trained with customized PC-based simulation (X2) showed 
a clear advantage over the two other groups, trained with respectively standard PC-based 
simulation (XI) and no simulation (C). 

The learning curves of the three groups show that those results apply to the initial skill 
acquisition phase, where the learning curve is approximately linear and skill-levels are 
one relatively low level of automaticity. In this phase, the use of customized PC-based 
simulation leads to a higher rate of learning. During the training phase, the group with 
no-simulation (C) was able to improve their performance by approx. 37%, the group with 
standard simulation (XI) by approx. 42% and the group with customized simulation 
(X2) by approx. 55%. Most remarkable improvement in score of the X2-group takes 
place in the first lessons, giving them a headstart, causing an improvement that is exis¬ 
tent throughout the remainder of the training. 

Concerning the validity of these conclusions, the question may be raised whether the 
flight performance differences between the X2- and respectively the XI- and C-group 
can be accounted for by other factors than the imposed experimental manipulations. A 
possibility that could give the X2-group an advantage over the XI- and C-group and not 
stenaming from the customized simulation itself is biased treatment by the flight instruc- 


tors, since it was practically impossible the keep the flight-instructors unaware of the 
ground-training that each group received. To minimize the likelyhood of this possibility, 
scoring criteria and scoring standards were used that left the least room for free interpre¬ 
tation and subjective judgement. Further validation of the results with the performance 
data logged with the airborne equipment and during simulation is needed to fully exclude 
this possibility and to gain more insight in the reliability of instructor ratings. 

Viewing the conclusions of this research, the key question is raised: Why does the X2 
group show significant positive transfer, where the XI group does not? What makes the 
difference in training effectiveness of perceptual motor skills when using customized PC- 
based simulation? 

A prominent and valued improvement (by expert aerobatics pilots) of the customized 
simulation over the COTS simulation was the realistic position and displacement of 
stick, rudder-pedals and thottle relative to the body of the pilot. Our interpretation of the 
results is that by providing a realistic design of the workspace in simulation, perceptual- 
motor strategies transfer better to real flight, thus decreasing the need for paying (visual) 
attention to the different configuration of stick, rudder-pedals and throttle during real 
flight. The observation that subjects in the XI-group noticeably payed visual attention to 
stick-handling during simulation-sessions further supports this view. Also, we hypothe¬ 
size on the basis of the results that realistic design of the simulation workspace, such that 
the trainee occupies a postural position that is identical to that in normal (straight and 
level) flight, serves a baseline cueing requirement for further development (in-flight) of 
seat-of-the-pants skills, i.e. skills that relate to the perception and processing of vestibu¬ 
lar, tactile and kinaesthetic cues. We further hypothesize on the basis of the results that 
the use of augmented cues, i.e. a digitized instructor voice during the simulation lessons 
and the use of the lesson arrows (visual indication of the direction in which to move the 
rudder pedals and stick) made learning of the procedural components of the manoeuvres 
easier, thereby decreasing the cognitive load, such that during simulation more attention 
could be paid to the visual components of the manoeuvres. The in-flight observation by 
instructors that X2-subjects had a better out-of-the-window visual orientation, while XI- 
subjects seemed to concentrate more on the cockpit instruments, supports this view. It is 
thought that the use of a larger monitor (21” rather than 17”) in the customized simula¬ 
tion has further contributed to positive transfer. The partial contributions of those cus- 
tomizations could not be established experimentally and is worthwile focussing upon in 
further research. 

The higher rate of learning observed during the in-flight lessons with the group that used 
customized simulation can thus be explained by (1) their improved ability to extract vis¬ 
ual cues from the outside world that are needed to perform the time-critical elements of 
aerobatics (2) speeded up learning of procedural elements of aerobatics in earlier stage 
in training (3) an improved basis for in-flight development of seat-of-the-pants skills. 

An additional factor that is hard to catch in figures or concrete observations, but may 
well be overlooked, is a possible higher level of motivation by the X2-group, i.e. to train 
with the customized configuration was notably more fun than to train with the standard 
configuration. Although all participating subjects were highly motivated, differential ex¬ 
trinsic motivation caused by the differences in ground training may have added to incre- 


mental transfer of the X2-group, 

This research has contributed to the fidelity database for flight simulation by focussing 
on some low-cost, state-of-the-art, multi-media solutions. Configurations such as those 
experimentally evaluated can be applied to several settings: (1) Customized PC-based 
simulations can be used to support the first stage of in-flight training and training on 
high-fidelity moving base simulators. (2) It was observed in the current training program 
that trainees in the XI- and X2-groups needed very little pre-flight briefing. Briefing 
times went down from approx. 15 minutes for the C-group subjects to approx. 5 minutes 
for the XI-group subjects to almost zero for the X2-group subjects. Thus, both simula¬ 
tion configurations could be applied as automatic briefing tools, therewith saving flight- 
instructor time. (3) It is a well-known fact from learning psychology that lower-abihty 
subjects benefit more from advanced instructional strategies than higher-ability subjects. 
Although not yet apparent from the current results, it is not unthinkable to employ cus¬ 
tomized PC-based simulation as a remedial instruction tool throughout initial training, 
therewith lowering wash-out rates in this stage of training. (4) The data in this experi¬ 
ment reveals that performance on standard flying tasks is by no means a good predictor 
for aerobatics performance, probably since aerobatics tasks appeal more to seat-of-the- 
pants flying. Further research could therefore be directed to the utility of customized PC- 
based simulations for the selection of military pilots. 


Donchin, E., Fabiani, M., and Sanders A. (1989), The training of complex task perform¬ 
ance: An examination of strategies in skill acquisition, Acta Psychologica, 71,1- 

Lintem, G. (1996), Simulation for flight training, in Flight Simulation Update 1996, 

State University of New York, Binghamton. 

Looking Glass Technologies (1995). Flight Unlimited, (Manual in Dutch, version 1.0 for 
MS-DOS Systems), Looking Glass Technologies, Cambridge, MA, US. 

StatSoft (1994), STATISTICA (Quick Reference), StatSoft Inc., Tulsa, OK, US. 

Wightman, D.C., Lintem, G. (1985). Part task training for tracking and manual control, 
Human Factors, Vol. 27(3), 267-283. 


We would like to thank Ronald van Gent (NLR) for providing us with some of the basic 
ideas for this research and Bert Huizenga (Wings over Holland) for the enthusiasm with 
which he co-operated with us in setting up and realizing the training-program. Further¬ 
more we are greatly indebted to all who helped us to realize the experiment: Hans Brug- 
man, Piet Werkhoven, Jaap-Jan van den Bosch, Rolf Zon, Martin Joosse, Bas Kuijpers, 
Esmyralda Eveleens, Hugo van Dorth (all NLR), Donald Kreuger, Rob Stamer (Wings 
Over Holland), staff and trainees from the Internationale Luchtvaartschool, Lelystad, and 
those we didn’t mention! 


A Systematic Approach For Applying Multimedia 

Techniques To Aviation 

ByMJ. Shaffer, Paradigm International, 
and R. Baldwin, Baldwin International Services 


For its central theme this paper will address some of the problems which arise because 
the human is often considered as the weakest link in the aviation system. By striving for 
a systematic integrated approach of the technical and human considerations the authors 
suggest that all the players in the aviation system can make a contribution to improving 
aviation safety. The paper addresses a number of human factors aspects need to be 
addressed in detail if they are to act as effective drivers in the efforts to improve aviation 

The advent of modem multimedia techniques suggests that a paradigm shift in 
capabilities has now occurred. This allows us to realistically model complex large-scale 
systems whilst also allowing individuals to specifically address their particular area at 
whatever level of complexity is appropriate to them. The paper will look at how 
multimedia techniques are particularly applicable to some of the aviation issues 
addressed earlier in the paper. This theme will be central to establishing the benefits 
which multimedia techniques can bring to the aviation system in order to create an 
effective and systematic integrated approach to the human performance interactions in 
the cockpit and air traffic control. 


Human-system interactions have recently been the focus of extensive study and 
discussion. A central point of this effort has been the recognition that the human is 
often the weakest link in the system framework. This is particularly true within the 
aviation community. Human error is the most cited single cause of aviation accidents. 
This is not surprising when one considers the myriad of aspects and stages of the aviation 
system development and implementation where the human is a major component: design 
and manufacture, operators, maintainers, managers and trainers. 

Human Factors in aviation 

The connections among some of the various aviation activities can be illustrated by the 
diagram shown in fig.l. 



As the model illustrates, the human is central to all aspects except the ambient weather 
conditions. The challenge is to address and optimize the interactions among all of these 


humans so that the system functions as effectively and efficiently as possible. 
Multimedia is often touted as the newest and most effective solution to all of the 
problems that plague the aviation human interactions. However, multimedia 
presentation of information both in the cockpit and on the ground has been in use for 

First consider the cockpit. Flight information is displayed on color LCD primary and 
secondary displays in many modem aircraft. The identification of some controls is 
presented as labels and in some cases, the controls are identifiable by shape. Digital 
AXIS messages are present in many cockpits. Voice communications are prevalent both 
between the air and ground, and in military environments, between aircraft. Checklists 
use printed media. Communication between the cockpit and the flight attendants is by 
telephone on most commercial airlines. Clearly, the modem commercial and military 
aircraft utilize multimedia to present and exchange information . 

The situation in Air Traffic Control is similar. Information on the location and 
identification of aircraft in the airspace is shown on a radar screen. Flight strips which 
contain information about aircraft are still present in printed format, Instmctions to 
aircraft and interactions with aircraft are imparted verbally. Interaction with a computer 
terminal is also involved. Again, multimedia are involved. 

Despite the fact that the use of multimedia is prevalent, it is not with out problems. 
Central to this problem is the fact that the end users of cockpits — the pilots, and the end 
users of ATC systems - the controllers, must interact. However, the systems which they 
are using have often been designed in isolation. That is, designers of ATC equipment 
and designers of cockpits are most often two independent entities. To further exacerbate 
the situation, even within the cockpit and within ATC systems design, the development 
and testing of the components of the systems have often been independent. The use of 
"off-the-shelf components is hailed by many to be a great time and money saving 
endeavor. However, each of the components, operating alone, may function quite 
adequately, incorporating it into a system configuration without utilising proper 
integration studies could prove disastrous. 

A number of human factors aspects need to be addressed in detail if they are to act as 
effective drivers in the efforts to improve aviation safety. In addition these drivers 
should be used in a proactive, rather than reactive way and the multimedia tools used to 
support the actions must be appropriately chosen for the particular activity. 

The first problem which arises is how to categorise the Human Factors actions as the air 
and ground aspects are often addressed in different ways. One of the earliest was the 
definition given by the military’s MANPRINT (MANpower and PeRsonnel INTegration) 
programme This essentially takes into consideration all system aspects which are 
concerned with Manpower, Personnel, Training, Human factors engineering, System 
safety. Health hazard assessment. 

Organisational and management aspects can be added as many aviation organisations 
have traditional mechanistic structures which need to be modernised, in order to meet 


modem task requirements and more effectively use the skills and expertise of the 

However, more appropriate recent conceptual models have been put forward in order to 
help classification of work. The one put forward by ICAO (Human Factors Digest No.l) 
is called the SHELL model from the terms Software, Hardware, Environment and 
Liveware. This notation is particularly useful in identifying the usefulness to aviation of 
HF studies as HF is essentially concerned with solving practical problems. Thus for 
Flight Operations the specific applications are categorised as but not limited to: 

reduction of human error 



human factors 


flight documentation 

flight deck design 

cabin design 

visual performance 

collision avoidance. 

Eurocontrol (1996) have built on the SHELL concept to specifically address ATC 
matters and consequently categorise specific applications as: 

customer focus 

rostering and working practices 
human factors techniques 
career and personnel development. 

Although further types could be developed it is important at this stage, to concentrate on 
the essential HF disciplines which impact on aviation safety. Thus the ICAO and 
Eurocontrol categories can be combined as follows: 


HF Category 

Meets ICAO 

Meets Eurocontrol 



Physical size 
and shape 







Rostering and 
working practices 










and training 



Human factors 



Career and 



Working practices 

Despite the fact that multimedia have been extensively involved both in cockpits and 
ATC, the problem of integration of all the information and required functions is 
pervasive. Even within a single display the problem of integrating all of the information 
and presentation issues can be problematic. To illustrate the problem, in a recent study 
by Shaffer and Mills (1993) to develop guidelines for a display avionics management 
unit for a military cargo aircraft, it was found that there had been no non-proprietary 
studies that addressed the interactions of menu development with its presentation, 
including font size, contrast, lighting, location and display density. There were 
numerous studies which addressed each of these elements in isolation, but none that 
addressed the issues as they interact. The require font size is dependent upon the 
location (in this case in the cockpit) which determines the lighting and contrast 
requirements. These also limit the amount of information that can be portrayed on the 
display given the display density guidelines. When one tries to make trade-offs, it is 
difficult, if not impossible to consider all of the interacting factors based upon today’s 
published literature. 

Subsequent to this effort, Toms, et al (1995) performed studies at Wright Laboratories in 
Ohio using simulated flat panel displays incorporating the best “guestimate” of 
combinations of display characteristics. The question being asked was “is the 
performance with the new display as good or better than performance with the existing 
display?” Since existing displays were being utilised by crews in the cockpit, this was 
considered to “validate” their usability. However, nothing could be said about 
optimising performance, or whether performance might have been even better using a 


slightly different configuration. While its scope was limited to the study of only one new 
configuration, the study was important as it is one of the few recent studies of display 
integration that is finding its way into the published literature. 

Most studies that address displays as an integrated part of a total system, if they exist at 
all, are considered to be the exclusive property of the researching company. As a result, 
each researcher must “reinvent the wheel,” and the body of literature addressing 
integration studies remains woefully small. Furthermore, the conduct of parametric 
studies of the myriad of display and system characteristics which interact to create the 
end product is, at best, an Herculean task. This task probably is too large, and cannot be 
entirely accomplished. Nevertheless, the industry and the traveling public would be 
better served if there were greater sharing of information among the researchers, 
particularly where integrated systems are being evaluated. Or perhaps a centre for 
technology exchange should be instituted. 

New solutions for existing and new problems 

Models and case studies have always suffered from looking backward and thus solving 
yesterday’s problems. The advent of modem multimedia techniques suggests that a 
paradigm shift in capabilities has now occurred. This allows us to realistically model 
complex l^ge scale system whilst allowing individuals to specifically to address their 
particular area at whatever level of complexity is appropriate to them. 

Obtaining relevant data 

Utilising models of complex systems to address complex issues depend upon accurate, 
relevant, and sufficient data. These data should be derived from an operational context 
for them to be maximally useful. Fortunately, the miniaturization of video cameras 
offers a unique opportunity to obtain such data. Empirical tools, using video offer more 
reliable and vaUd data since they do not rely upon self-report by operators on what they 
do and how they do it. Sometimes such self-report reflects what people think they do, or 
what they think they are supposed to do, not what they actually do. Models using such 
data, are therefore of limited utility. 

Subjective assessments translate into the “eye of the beholder”,, and there are times when 
such a view is not complete. A case in point is revealed in a study on the stressful 
conditions affecting pilots during aircraft carrier landings (Miller et. al„ 1970). The 
study found a three times higher level of serum cortisol levels (indicating stress) even 
among experienced pilots. This occurred despite pilots rating themselves, subjectively as 
being calm. Ergo, subjective analysis has its limits. 

An Empirical Methodology 

Video using Empirically Validated Task Analysis (EVTA) offers some realistic answers 
to this dilemma. It allows a presence in environments where observation has been 
impossible; it allows one to replay the segments to determine what is “really going on”; 
and it allows one to verify the tasks performed, to see how problems are faced and how 
they are successfully and sometimes unsuccessfully solved. Because the camera is 


unobtrusive (some cameras are smaller than a man’s thumb), the operators quickly forget 
that they are being taped. This means that they are more likely to behave as they 
normally would, not as they think they should. 

The camera captures images and conversation on videotape that may be viewed 
repeatedly and concurrently with the time consumed. Provide this information to a team 
of qualified observers who record tasks and time through consensus, and there is a 
considerable improvement in the quality of empirical data derived. 

To assist in the analysis process, Paradigm developed the proprietary EVTA software 
program. The EVTA process generates a task-descriptive data base from a detailed 
analysis of operator’s observable activities and audible communications. In some 
environments, it is possible to also record the information that is presented on a computer 
screen, so it is possible to determine the information that is present to which the operator 
is responding. The essential features of the EVTA process are the use of video/audio 
equipment to gather permanent records of operator functions from operational 
environments, and the generation of an empirical record of function times through a 
software package which builds, manages and analyses the resulting data base. The 
output from the EVTA software is intended to provide the data required by and 
compatible with simulation environments such as Micro SAINT (Anon., 1987, SWAS 
(Holley and Parkes, 1987,) and HOS (Lane et al, 1981). More recent models may also 
use such data. 

The focus of EVTA is on observable behaviours such as hand movements, body 
movements, internal and external communications and visible information presented on 
CRTs, LCDs or other idsplays. Cognitive activity is caputred only when 
communications between operators or an overt activity indicate that it has occurred, it is 
not inferred. 

Enhancements to the process 

One of the difficulties with using data obtained video analysis, rests with the analysis 
process itself: it is time consuming; tedious, and both boring and demanding. To 
address some of these problems, (primarily with the input process), a new system was 
designed by Monterey Technologies Inc. for the U.S. Navy: the Video Information 
Extraction Work Station (VIEWS). This system digitizes the audio/video tape which 
allows for random access anywhere on the audio/video stream without the time 
consuming, inaccuracies of rewinding of tapes. Start and stop times can be entered with 
a single keystroke, eliminating the necessity for entering the individual numbers in the 
time hack thereby reducing errors. Multiple sources of data can be entered. Digital 
information such as eye movments can be integrated with information from the 
audio/video stream on the same timeline It is possible to zoom into a portion of a video 
frame for closer examination. The audio/video stream can be viewed on a frame by 
frame basis, if very precise time information is required. One of the most useful features 
is the loop capability, which allows the analyst to identify the start and stop of an event 
and to replay the loop repeatedly until all of the information about the event is recorded. 
This is particularly useful in deciphering garbled communications which frequently 


occur in operational environments particularly in aircraft. Finally, future versions of the 
VIEWS system will include the ability to enhance images and audio quality. 

The only study performed to date using data obtained from VIEWS is a study for a major 
telecommunications firm which addressed a detailed job analysis of customer service 
representatives. That study used a prototype version of VIEWS and despite its pre- 
production bugs, it offered some assistance in translating multi-media source information 
into useable data. As the technology matures, these types of data can be useful in 
modeling complex systems that are becoming commonplace. 

Information processing 

In many cases it is not possible to engage in in-depth studies such as those involving 
EVTA and VIEWS and data must be provided in the most effective manner that expert 
opinion feels is appropriate. One such area is information processing. This is, of course 
a major activity at present, due to the immense amount of information which is available 
and relevant to every job. As a safety critical activity, the aviation field has always been 
in need of current information to allow for timely processing, as so many aspects of the 
flight environment change rapidly, and flying is a notoriously unforgiving environment 
for errors. Fortunately, multimedia techniques now give us an opportunity to provide 
instant communications of whatever information is available and appropriate to the 
situation. Because of advanced technology, it is possible to present information in a 
hierarchical fashion: the most important information provided first and foremost. This 
information can be presented in a myriad of ways. The most appropriate media can be 
selected for the particular application, with the ultimate goal of reducing errors. 

Application to training 

Applications to information processing are well understood in the operational context, 
however, the real challenge is to transfer the capability into the off-line activities such as 
training, where traditionally the demands have often not been so stringent. There are 
several aspects of these off-line activities that deserve special attention: ensuring up-to- 
date information in the airspace environment and maintanence of up-to-date instructors., 
training time, and controller education for new technologies. 

1. Recieving appropriate information on characteristics of new aircraft entering an 

Keeping up-to-date on all of the relevant information about new aircraft is traditionally a 
problem in developing countries, and can, because of volume, become a burden even to 
large counries. The ISO 9000 series stresses that as a part of the quality management 
process, it is critical that relevant information must be provided to the actual user and not 
left in the manager’s office. 

With current multimedia techniques this information can be placed directly with the 
actual user by using CD-ROM or via the Internet thus providing on the job training. 
Adding to this, the realistic use of video, there can be a reinforced connection between 
the type of aircraft and its performance. This can be provided at the level of detail 
required by the specific user. 


2. Maintaining up-to-date instructors. 

When the instructional staff must be removed from their operational capacities, it is is 
difficult, particularly over time, for them to maintain their currencies. Special skills can 
be lost, and up-to-date examples can be sacrificed. This problem applies particularly to 
controllers but also to airhne pilots. While simulations are useful in providing hands-on 
experience particularly with pilot training, the importance of having instructors with up- 
to-date, current, and extensive expertise is invaluable. With the availability of Web sites 
on the Internet, such information can be documented and available to trainees from 
diverse geographic localities. 

3. Training time 

Training time, particularly for controllers has always been considered a lengthy process, 
and many countries are making great efforts to reduce the time. New techniques include 
the use of PCs to match training presentation to individual student learning styles. 
Simulator exercises tailored both to the geographic area (with its accompanying level of 
complexity) and to the student learning styles provide a cost effective and timely training 

4. Contoller education for new technologies 

With the introduction of the Advance Automation Techniques (AAT) into ATC there are 
questions regarding the extent to which the present generation of controllers are 
adequately educated to understand the understanding principals of the new technology 
and the imphcations of its practical implementation. Both CD-ROM and the Internet 
offer appropriate and cost-effective training in these new techniques. Self-learmng can 
be a powerful tool for providing currencies, however it should be noted, that, given a 
variety of learning styles and management incentives, not all of the current controllers 
will respond appropriately to such techniques. A vehicle to identify those controllers 
who might “fall between Ae cracks” and not respond appropriately should be developed. 
For those controllers, more conventional techniques of classroom training may be 

Other applications 

There are many other applications which are reciving attention and the present situation 
will make for an excellent discussion. Among some of the topics must be mentioned: 

1. The ability to strengthen the data, information and knowledge links between 
regulatory authorities, licencing authorities and operational organisations (eg. for 
monitoring, control and recording of results). 

2. The abihty to link simulators to CBT and thus improve trainnig programmes for 
standard training or for re-validating licenses). 

3. The ability to link flying simulators and ATC simulators and thus practice 
realistic cockpit skills. Pilots, ATC skills and to produce CRM scenarios. 


Management and future challenges. 

Management must always be concerned with the competency and currency of its staff. 
Given the competitive marketplace present in the aviation community, this presents 
constant challenges. Adequate and appropriate communications are essential for all. 
Simulations (using PCs and more sophisticated computers), CD ROM, the Internet, 
Computer Based Training (CBT) all offer appropriate technological solutions to modem 
aviation problems. 

However, the challenge is for the multimedia models to stay current with the real systems 
and the real problems and thus to be continuously looking forward. This is the challenge 
for management. This then is central to establishing and ensuring the benefits which 
multimedia techniques can bring to the aviation system in order to create an effective 
and systematic integrated approach to the human performance interactions in the cockpit 
and air traffic control. This is a holistic approach that affords the opportunity for 
everyone to stay abreast of the latest technology. 

This is both the opportunity and the challenge for integrating technology and multimedia. 


Miller, R.G., Rubin,R.T., Clark, D.R., Poland, R.E; and Arthur, R.J., 1970. The Stress 
of Aircraft Carrier Landings in Cortecosteriod Responses in Naval Aviators. 
Psychosomatic Medicine, Vol 32, pps 581-588. 

Eurocontrol (1996). Series of Human Factors publications as part of the EATCHIP 
programme. Available from DGS Logistics and Support Department, 96 rue de la Fusee, 
B-1130, Brussels, Belgium. 

M.T, Schaffer and D. Mills, “human Factors Evaluation of the C-140 display Avionics 
Management Unit”, ARINC Research Corportation, prepared for Warner Robins Airm 
Logistics Center/LJLEB, Robins Air Force Base, Georgia, contract no.FO9603-89-G- 
0054, february 1993. 

M.LToms, S.M. Cone, G.R. Grier, G.S. Boucek, T.R. Brown, M.J. Patsek, “Full Mision 
Evaluation”, in C-141 Upgrade programme, vol H, for Flight Dynamics Directorate, 
Wright Laboratory, Wright Patterson Air Force Base, Ohio, contract no. F33615-93-D- 
3800, June 


Controller Work Position for 
future automatic dependent surveillance 

B. Ulbrich and A. Wattler, 
A. Nordwig and F. Posadny 

Berlin University of Technology 
Institute of Aeronautics and Astronautics 
Berlin, Germany 


The plannings of the ICAO (International Civil Aviation Organsiation) concerning the 
global implementation of future communication, navigation and surveillance, and air 
traffic management require worldwide operational satellite based systems (Global 
Navigation Satellite Systems, GNSS) [ICAO, 1988]. By the use of satellites for 
navigation and data communication purposes it will be possible to create surveillance 
(Automatic Dependent Surveillance, ADS) for air traffic control (ATC) independent of 
ground derived radar equipments. Based on a two-way air-ground data transmission a 
variety of onboard generated data, e.g. 4D data of flight profile or current weather data 
can be included in the planning and supervision process of air traffic control refering to 
its optimization. The improvement of quantity and quality of data however requires a 
redesign of the previous human-machine system ^controller work position^ (CWP) and a 
modification of the ATC procedures [ICAO 1995, EUROCONTROL 1991 und 1994]. 

In the context of a project promoted by Deutsche Forschungsgemeinschaft research was 
carried out to this at the Institute of Aeronautics and Astronautics at the Berlin 
University of Technology. The examinations reflect to operative and cognitive aspects of 
the air traffic controller's work using the application of ADS. The aim was to get a user 
oriented design of the human machine system, and thus to specify necessary 
modifications of modes of operation used at present by air traffic control. 

In a first approach the preliminary examinations were concentrated on the analysis of 
present air traffic control systems as well as on the task analysis of the controller's work. 
A complex task scheme of the radar controller, the coordinator and their team work was 
developed based on the Goals Operator's Methods Selection Rules - model [Card, 
1983] declared itself through software ergonomy. Furthermore a knowledge acquisition 
was carried out for the conflict management process to get information about the 
subjective theories of the controllers regarding their main task of conflict detectionand 
solution as well as their mental processing. The knowledge about cognitive processes of 
the controllers completed the results of the task analysis. The knowledge acquisition 
proved as meaningful and necessary with regard to the implementation of numerous new 
information particularly into the ADS CWP. This scheme (task analysis and knowledge 
acquisition) formed a basis for the design of the future ADS CWP. It was valid to answer 
the question how the technically possible availability of new data has been brought into 
harmony to the required data for previous work tasks as well as the construction of 


subjective theories of the controllers. 

The development of requirements for the specification of the human-machine interface of 
the CWP then was carried out in a second approach as well as the development of the 
graphical dialog interfaces and the implementation in a simulation environment. At least 
the complex human-machine interface of the ADS CWP was evaluted heuristically by 
experts of interface design and air traffic controllers. So, an evaluated ADS CWP for 
further experimental, comparative examinations of possible ADS modes of operation 

Preliminary Examination 

The analysis of the present situation covered documents such as ODID IV (Operator 
Display and Input Development) [EUROCONTROL, 1994] and the design guidelines for 
controller work position appropriate to COPS (Common Operational Performance 
Specification) [EUROCONTROL, 1991]. Observations and interviews of controllers at 
the ACC Tempelhof of the Deutsche Flugsicherung GmbH (DFS) were also carried out. 
The aim has been the analysis of different design concepts as well as the inquiry of the 
current state of development of air traffic controller work positions like DERD-XL 
(representation of extracted radar data/extended version). Potential design lacks and 
advantages of present air traffic control systems should be recognized. 

20 experienced radar controllers (EC) and coordinators (CC) took part in the interview. 
They could refer to an average professional experience of nine years and had admittances 
for at least two and maximum twelve air traffic control sectors. Ratings and open 
questions formed the main emphasis of the questionare. These were useful to judge the 
design of the single control units and informational displays as well as the expected benefit 
of possible innovations. 

The controllers named a variety of CWP shortages and estimated the general use of data 
link for the air-ground communication as very desirable. An expansion of the present 
information according to the aircrafts to be supervised seemed to be considered very 
positive. An absolutely required attention of design principles became in connection with 
this emphasis, too. Almost all of the controllers reported that they did not have sufficent 
support for the conflict detection and solution process by the present operational system. 
Need for action would particularly exists in the prediction of potential conflicts between 
aircrafts. One solution to a variety of the problems wOl be possible by the use of data as 
specified by ADS. By the use of data link safety in air transportation will increase by a 
more precise identification of flight targets, one improved supervision of airplanes in areas 
with no or less enlarged radar equipment as well as one enlarged quantity of information. 
A better u.sage of airspace capacity will be possible by reducing the separation standards. 
In addition, the correlation of the scheduled flight data with onboard generated position 
and flight intention data will be a relief for the execution of planning and coordination 

Further examination was carried out for completion of the analysis of present operative air 
traffic control systems regarding the current and future state of development to 
functionality and operability of ADS in air traffic control. This served to the inquiry of 
perhaps available specifications for the ADS functionality respectively its application into 
future air traffic control system. No standardization for ADS was carried out by 


the ICAO [ICAO, 1995] until today. The exchange of ADS data between air traffic 
control and aircrafts will be initiated by any of them. This can be earned out periodically 
or on request. This so called ADS report can contain up to eight blocks of data of various 
information. The composition varies in dependence of the avionic standard, the volume 
of air traffic and operational aspects. The transferred data refer e.g to the position of the 
aircraft (basic block), the current speed and heading (ground/air vector), the 4D-profile 
calculated by a flight management system (projected profiles) or actual flight weather 
data (met info). 

A detailed task analysis of controller's work in the context of the first work package 
became necessary because it was not available at the beginning of the project. However, 
the description of possible modifications of future ATC procedures given by the 
introduction of ADS and the redesign of an ADS CWP required some task analysis 
(Hollnagel, 1990). 

For this reason a document analysis has been conducted based on the BAFVK 
(operational procedures for air traffic service) and the EUROCONTROL Task Book 
AF51 (EUROCONTROL, 1995) as well as further observations and interviews with 
selected ATC experts at the ACC Tempelhof of the Deutsche Flugsicherung GmbH. The 
result was a clear structure of tasks to the present work of radar controllers and 
coordinators in connection with the used DERD-XL work station. To be able to better 
describe the distribution of tasks in the human-machine system, a customization of the 
determined tasks to the sooner mechanistic GOMS model, known from the software 
development discipline, was carried out on a functional level. 

As a matter of priority observable shares to the very complex task of air traffic 
controller's supervision were made visible with the task analysis already mentioned. It 
was valid to got a picture of this work as complete as possible including also the rules to 
the practice of the determined tasks being based and therefore subjective theories of the 
controllers. In an extensive, methodically complex examination with 20 subjects 
(controllers of ACC Tempelhof) the main task of the controllers, the conflict 
management, was analyzed. The methods used were an especially designed questionare, 
the method of network development and the method of constructive interaction. 

The subjects spread out as follows on the single methods: 





constructive interaction 

new info 

known info 





type of 
conflict 1-4 

type of 
conflict 5-9 















Table 2.1: Distribution of subjects 

The questionare used in the test consisted of so-called open questions and of ratings with 
main emphases concerning the demographic data, the conflict detection, and resolution. 

Using the method of network development the test participants had the task to group a 
set of data cards containing information will need at present and future for the practice of 
their job as air traffic controller. In a first step they had to sort all cards so that all 


information which has a similar importance for the conflict detection process were 
summarized in a group. Then they should try to structure these cards in a network, which 
represents individual mental information structure of their personal conflict detection 

Due to the number of subjects in comparison with the examined parameters (number of 
information to sort) an exclusively qualitative analysis of the obtained data in accordance 
with the network development seemed to be efficient [Liier, 1987]. A quantitative 
comparison did not seem meaningful here. Observations during the execution of the 
method showed that controllers had some difficulties to structure the information units of 
such a highly automated task like conflict detection. The controllers consequently 
emphasized the missing general validity of their networks. This was caused by the variety 
of conditions and situation variables having an effect on the process of conflict detection 
which were not checked explicitly. Nevertheless relatively similar results were produced 
within the respective random sample and the low number of subjects. This points out that 
there are generally pure coherences within the conflict detection process. 

Controllers were confronted with the method of constructive interaction (Nielsen, 1993) in 
the context of the examinations at the ACC Tempelhof. The potential conflict scenarios 
consisted of static variants designed as abstract radar pictures and control stripes. The 
conflict detectionand resolution was carried out during the series of experiments in a team 
of radar controllers and coordinators according to the organization in ATC. The task of 
the controller was to detect and solute potential conflict situations based on the 
information sources already mentioned (radar picture and control stripes). For the series 
of tests nine defined conflict types have been taken into account according to 
EUROCONTROL Task Book (EUROCONTROL, 1995). The conflict scenarios were 
formed on the radar sector UR 2 in the upper airspace of the UIR Berlin and checked in 
cooperation with active controllers with regard to its ecological validity. It had to be 
stated, that coordinators and radar controllers primary used flight level changes 
independently of the conflict type and secondarily measures affected the lateral profile like 
"parallel! offset" and "directs". The decision about the application of one of the actions 
mentioned was dependent on the flight performance, particularly the speed of the aircrafts 
concerned, and the accordance of the partial routings of the conflict partners concerned. 

By using an extensive catalog of methods a complex picture of the mental processes of 
conflict detection and resolution as well as a variety of specific controller"s knowledge had 
been determined. Because of the varying efficiency of the employed methods the single 
results were summarized with the intention to get extensive knowledge of information 
contents and their structuring at the human-machine interfaces of the CWP. 

The resulting specification of requirements (see chapter 3.1) became the guideline for the 
further development of the human-machine interfaces of the ADS CWP. Main emphases 
were made concerning the primary tasks of controllers e.g. conflict detection and 
resolution and the design of tools for controller assistance. 

Design of the HMI of a controller working position with ADS-functionality 
Specification of the HMI 

Resulting from the status analysis and the pre-studies, a catalog of requirements for 


designing the HMI of the ADS CWP was defined. The catalog is influenced by the 
guidelines of ICAO (1995), international guides for display design especially the ISO 
9241 part 10 and part 12, the DIN 66234, part 8, the results of the ODID-Group and 
COPS-Specification from 1991, and topical research results of design of GUIs like the 
study of Charwat (1996). 

Besides this analysis of documents, the results of described status analysis and the pre¬ 
study already mentioned (task analysis and knowledge acquisition) were used to develop 
the catalog. The results of knowledge acquisition and especially the conflict management 
in a team of controllers had have a great influence on the design process. Due to the 
collected results it was possible to fix a lot of important information have to be shown on 
the ADS CWP. Furthermore a number of conflict resolutions prefered by controllers 
could be recorded. 

The requirement catalogue defines following rules of designing the ADS CWP: 

1. Realization of technical requirements of air traffic control (ATC): 

• Realdisplaying of airspace (map data, airspace structure), 

. Real fli^tplan data of aircrafts with standard avionik and ADS specific avionics 
(plan data and/or radar data are at controllers disposal) oriented by the ICAO 
standard of transmission rates of ADS reports in high density areas (1995), 

• Exact conflict detections, 

• ADS specific service functions e.g. displaying of planned flight profile, ADS 
events, and meteorological data, 

• Calculation and development of complete conflict resolutions which are possible 
by automated techniques and be prefered by controllers. 

2, Support of typical job tasks of air traffic controllers as well as their ATC 

functions defined by COPS 

2.1. General requirements 

• Unique display for radar controllers and coordinators. 

• Possibility to adjust the interface to personal preferences. 

• Adjustments have to be saved by the system under a special personal code. 

• Possibility to execute all typically defined job tasks. 

The ADS CWP has to hold the following information and functionalities (part 2.2. - 2.6.) 
to the controllers disposal in view of the functions of controllers defined by COPS. 

2.2. Planning functions 

• Presentation of the complete flight information by electronical stripes (sorted by 
time and deleted automatically; the stripes have to show at least the following 
informations: AC code, AC type, sector entry point, entry time, and entry level, 
sector exit point, exit level, and exit time as well as the planned flight profile with 
at most five waypoints inside of the bordering sector. Graphical flight path 
elements (lateral and vertical) should be shown on the central radar screen, likewise 
outside the controlled sector,). 

• Facility of a simulation of flight path changes with a following-up check for 
conflict freedom, 

• A long term conflict alert with an exact visualization of conflict partners and 


• Holding possible conflict resolutions to controllers disposal, and displaying them in 
an understandable way. 

2.3. Surveillance functions 

• Presentation of the traffic situation of the controlled sector as well as the adjacent 
sectors (with a target symbol and the accompanying label, and at least the following 
informations: call sign, topical flight level, figure of merit, AC type, AC category). 

• Conflict warning with a preview of more than two minutes (containing the reason of 
conflict and the conflict partners). 

. Presentation of the following planned flight profile in a schedule and by means of 
graphical elements with the following information: sector exit point, exit time, and 
exit level). 

• Supporting the orientation of controllers in the airspace by a flexible measurement 
vector and an understandable presentation of the vertical profile. 

2.4. Monitoring and control functions 

. A workload display. 

• Status display of the particular modules e.g. conflict control. 

• Implementation of a zoom function and the possibility to adjust the picture. 

2.5. Coordination within the controlled sector, within the ACC, and between 
other ACCs (was not the topic of the project at present). 

2.6. Air-ground Communication (was not the topic of the project at present), 

2.7. With the help of ADS CWP the controllers should be able to detect and 
process the following conflict scenarios: 

. high conflict potential caused by the structure of sector, 

• potential conflicts resulting from aircraft movements, 

• conflicts caused by weather conditions, 

• emergencies. 

2.8. Observance of general software-ergonomic and cognitive-ergonomic 

2.9. System architecture 

• The simulation should have been implemented in a UNIX environment. 

• It should be designed according to the chent server model to bear sufficient 

• The simulation should have a minimum of dead times. 

• The controllers should be able to control a lot of traffic (approximately 60 AC s 
per hour) with ADS CWP. 

• Operations using a mouse as input device. 

. Little keyboard usage (e.g. for input of controller codes). 

Implementation into a simulation environment 

The software part of the ADS CWP discussed here is a development from an experimental 
ATC simulation produced by the DFG project “Fluglotsenarbeitsplatze” (AZ.: Fr375/45- 
l). It is implemented on Unix workstations using the client server model and X-Windows. 
The server simulates the radar turn, keeps all flight plans, flight profiles, ADS data, and 
controls inter process communication. 


The following figure 3.1 shows the communication scheme for the ADS CWP and other 

Air traffic 



Client ADS CWP 





Figure 3,1: Communication scheme for ADS CWP 

contains; ADS: ADS database, FPL: flight plan database, METEC: conflict detection, 
COREC: conflict solution 

To improve performance, the CWP client was divided into three subprocesses, separating 
the communication with the server from its radar driver (calculation of aircraft positions 
and zoom functions) and frontend (visualization of air traffic situation display and user 

Besides, in the ADS CWP, all mouse clicks can be recorded using log file protocols. This 
registration method allows immediate reproducability of all actions performed by the 
subjects, and thus makes it possible to conduct exact fault analysis, define reaction times, 
and to reconstruct all activity sequences. Log file protocolling is a substantial addition to 
interviews and video recordings as frequently used evaluation methods. 

Development of the Graphical User Interface 

At the beginning of this project there was no suitable ADS CWP prototype for simulation 
purposes. So the development of a new controller work position became a main part. 
This CWP then could be used as a test bed for the evaluation of the Computer Human 


Interface (CHI). 

The foUowing figure depicts the structure of the CHI and its essential features. The 
frontend was built following the rapid prototyping paradigm with Tcl/Tk. This interpreter 
language comes with a lot of widgets which are good suited for developing Graphical 
User Interfaces under the X windows system. 

Pull down menu contains: CWP setup, 
flight plan setup, ADS data setup, 
conflict module, communication fiel 
and library 

priority setup is used for preselection of 
special conflict solutions depending on 
sector load by the controller 

vertical window 
used to edit the 
vertical profile 

radar screen with: sector structure, 
restricted areas, air ways, way 
points, air craft symbols, extentable 
labels, context menu, graphical 
editor, etc. 

context sensitive multi function bar 
to enter the graphical editor, 
activating the vertical window and 
assigning or testing a modified 

Figure 3.2; Frontend of the ADS CWP 

The frontend is sub-structured in five main blocks: radar screen, pull down menu, context 
sensitive bar, info Box, and work load display. The radar screen with all its relevant 
information especially for short term management is placed in the center of the user 
interface. All additional information for instance flight plans or early detected conflicts will 
be accumulated in the Info Box. The work load widget displays the expected work load 
within next twenty minutes which is calculated by considering the given flight plans of 
the scenario. 

Setup functionality is provided by the pulldown menu. The status bar is placed at the 



bottom of the screen. It permanently shows a scale of 15 NM based on the actual zoom 
factor. Additionally it contains the buttons of the graphical editor during edit mode. This 
editor allows editing of 4D flight profiles. 

The CWP should be usable for aircrafts supporting different avionic standards. This is 
why context menus and labels of aircrafts contain different information. The following 
table summarizes this special information for the labels of ADS compatible and standard 
air crafts. 


aircraft label 

Explanation of Abbreviations 


ID, AC Type, 


AC code 

Ti. FI act. FI koor, NexWP, FI NexWP, At, TAS; 

AC Type 

AC type identifier 


clirab/descent indicator 


actual height 

FI coor 

coordinated height 


next (planned) way point 




height at next way point 

Ti, Rates, FI act, FI 

coor, NexWP, FI NexWP, At, TAS; 


time to next way point 

wind direction, wind speed, turbulence, DAT 


True Air Speed 



data quality of ADS avionic 

3 line is activable 


rate of climb/descent 

exclusivly for ADS 


Outer Air Temperature 


Table 3.1: Information of aircraft labels 

The ADS CWP frontend was extended by a graphical editor which was created in this 
project. It is partially integrated in the radar screen (lateral profile). The part of editing 
vertical profiles is implemented in a separate window which is activated on demand by 
the controller. 

The editor allows modification of profiles without intervention of keyboard inputs. The 
modified and acknowledged profiles will be passed to the air traffic simulation and to the 
conflict detection module. This higher dynamic behaviour of the system allows a real 
time management near to reality. 


Figure 3.3: Graphical editor for modification of lateral profiles 

The conflict management system is an additional client which is communicating with the 
ADS CWP. It consists of a conflict detection module (METEC), a conflict solution 
module (COREC) and a priority controller which is used for a user defined preselection of 
possible solutions. The detection module is configured by the pulldown menu (left part of 
following figure 3.4). The visualization of conflict regions and additional information are 
on the radar screen too (right part). The module for conflict solution is triggered 
automatically for the earliest conflict or manually for a specified one. 

Convict tioo| I ConyiHmicaibKin Librsffy 







M ^ IDtnsi 


Figure 3.4: Configuring the conflict detection (left) and visualization (right) of a conflict situation 

Heuristic Analysis of the ADS CWP Prototype 
Description of the used method 

After last system tests, a heuristic analysis of the interface of ADS CWP was conducted 
in May 1997. The subjects were experts of air traffic control as well as experts for 
interface design. The task was to judge the whole functionality of the working position, 


the monitoring of infomiation, and the ability to manipulate buttons and other input 
devices. The qualitative evaluation was oriented by heuristics which are well known in 
literature (compare with Nielsen and Molich, 1990). So it was interesting to analyse the 
dialog, the reduction of memory strain, and possible faults of graphical presentation like 
wrong font size, errors in layout or trouble with controling the interface. 

Before the test begun, the subjects had the possibility to learn about the interface and its 
functions and could „play“ with it. Therefore a specific task-oriented handbook was 
developed in the project. 

The aim of the heuristic analysis was the detection and exact description of all existing 
errors of interface design and the interaction of all components of the ADS CWP. 

The results of the heuristic analysis of the ADS CWP show a lot of interesting faults of 
design, which can help redesigning the interface more user friendly. They do not call the 
ADS CWP into question but they are quite good suggestions. 

Presentation of selected results 

The subjects could work very quickly with the ADS CWP in an correct manner. They did 
not take long to learn the system operations. This observation points to a compatible, 
consistent, and clear design of the ADS CWP interface. Although, a further experimental 
study to check the possibility of working on job-tasks in a justifiable time and with less 
mistakes, has to be conducted. 

The working on particular tasks was personally good valued by the subjects. The errors 
of design for instance overlapping of menus, to small types, and missed flexibility in 
colour design limit the fulfilling of tasks. 

Quite interesting in the results was the different judgement of working with the ADS 
CWP by both subject groups. For example, the tasks „Detection of potential conflicts“ 
and „Resolution of potential conflicts** were judged easier to solve by the experts of air 
traffic control. The experts of interface design had have a lot of problems. The following 
figure (Fig. 2.17, Page 13) will show these results more clearly. 


Take over of CWP 
AcfusI configuration via Seft4>-Briding 
Ac^usf configuralion via rrenues 
Observing traffic on radar screen 
Observing traffic in InfoBox 
Observe'whole traffic situation 
D^ection of potential conflicts 
Resolution of pot. conflicts 
. HandcverofCWP 

■ Median 1 2 3 4 5 

Figure 0-1: Evaluation of the quality of working with ADS CWP (experts of interface design); Evaluation 
standard: „The task is possible to execute in a good manner with tlie functionality of ADS 
CWP.“; l=is absolutely incorrect, to, 5=is absolutely correct. 

ffi Median 1 2 3 4 5 

Figure 0-2: Evaluation of tlie quality of working with ADS CWP (experts of air traffic control); 
Evaluation standard: ,,Thc task is possible to execute in a good manner with the functionality of ADS 
CWP.“; l=is absolutely incorrect, to, 5=is absolutely correct. 

A possible reason for the differences in evaluation is the amount of experience with the 


research topic, the air traffic control. To execute the given tasks it is necessary to use a 
clear interface, but also to have knowledge about the priorities of information and their 
meanings for instance in the solving process of conflicts. Which traffic constellation or 
information of the flight plans are indicators of potential conflicts? The controllers could 
use the showed information on screen in a good manner. Experts of interface design had 
much more problems with very complex scenarios compared with controllers. They could 
not control the whole traffic situation and could not distinguish between topical 
important information. 

The obtained results confirm the splitting of subjects and show the importance of 
evaluation of such a complex interface by experts of interface design and users in future. 


In the context of further research the ADS CWP will be completed by additional ADS 
events e.g. 'flight plan conformance monitoring', and 'clearance validation' as well as 
the revision of design lacks. 

The modified ADS CWP will be the experimental system of further examinations of 
ATC operations using ADS. Therefore, a future structure of ATC sectors based on 
PHARE (Program for Harmonization of ATM Research in EUROCONTROL) has to be 
implemented. Furthermore, the definition of special ADS trails will permit an efficient 
exploitation of the airspace. Main emphases will be the examination of transition 
scenarios between ADS trails and standard routes as well as the impact on operational 
procedures. Changes possible in controller team work between CC and EC and their 
specific tasks require also additional research. Increasing quality of data given by ADS 
win propably enable an optimization of long term planning functions for CC. 
Furthermore, the impact of automated assistance e.g. for conflict management has to be 

A further simulation phase will be the base for the experimental, comparative analysis of 
modifications mentioned above. The close co-operation with active controllers of the 
ACC Berlin will guarantee the user oriented approach. Finally, after the evaluation of the 
modified ADS CWP a styleguide has to be developed for controUer work position using 



ADS Europe. (1996). Automatic Dependent Surveillance trials pass significant milestone . 
London: ADS Europe Consortium. 

BAFVK. Betriebsanweisung fur den Flugverkehrskontrolldienst . Frankfurt am Main: DFS. 
im Anderungsdienst, 

Card, S.K., Moran, T.P., Newell, A. (1983) The psychology of human-computer 
interaction . Hillsdale N.J.; Erlbaum. 

Charwat, H.J. (1996). Zur Wahl von Farben auf Bildschirmen an Leitstanden, Zeitschrift 
flir Arbeitswissenschaften, 1, S.1-12. 

EUROCONTROL. (1991). Common Operational Performance Specifications for the 
future controller operating 

environment. Report Vers. 4. Brussel: EUROCONTROL. 
EUROCONTROL. (1994). Operational Display and Input Development . ODID IV 
Report. EEC Task AS08. Paris: EEC. 

EUROCONTROL. (1995). Task Book . AF 51. Paris: EEC. 

EUROCONTROL. (1996). Highly Interactive Problem Solver . Paris: EEC. 

DFS (1990). C ATM AC . Betriebskonzept fur die Durchflihrung der Flugsicherungsdienste 
im Bereich der Bundesrepublik Deutschland. Frankfurt/Main. 

Hollnagel, E. (1990). Die Komplexitat von Mensch-Maschine-Systemen. In C. Graf Hoyos 
& B. Zimolong (Hrsg ), Ingenieurpsychologie. Enzyklopadie der 
Psychologie, Themenbereich D, Serie III, Band 2, 501-510 
ICAO (1988). ICAO Special Committee on Future Air Navigation Systems . FANS 4/- 

Doc. 9524. Montreal: ICAO. 

ICAO (1995). Automatic Dependent Surveillance and Air Traffic Services Data Link 
Applications . 

ICAO Doc. 256-AN/152. Montreal: ICAO. 

Luer, G. (1987): Allgemeine Experimentelle Psychologie . (S.464-465). Stuttgart; Gustav 
Fischer Verlag. 

Nielsen, J. (1993): Usability Engineering . London: Academic Press, Inc. 

Nielsen, J. & Molich, R. (1990):Heuristic Evaluation of user interfaces; in; J.C. Chew & J. 

Whiteside (eds); Proceedings of CHI '90 Conference on Human Factors 
in Computing Systems, Seattle, Washington, April 1-5, New York:ACM; 

Ulich, E. (1986). Aspekte der Benutzerfreundlichkeit . In: Remmle, W. & Sommer, M. 

(Hrsg ). Arbeitsplatze morgen. Bericht des German Chapter of ACN. Bd. 
27. Stuttgart; Teubner. 

Ulich. E, (1992). Arbeitspsychologie . Stuttgart: C.E. Poeschel Verlag. 2.Auflage. 

Berlin University of Technology 
Institute of Aeronautics and Astronautics 
Marchstrabe 12-14 
10587 Berlin 

TEL: 030/314-22578; FAX: 030/314-22866 


The Diadem Software Development Methodology 
extended to Multimedia Interfaces 

Bemd-Burkhard Borys and Markus Tiermnn* 
Systems Engineering and Human-Machine Systems Laboratory 

University of Kassel • OiK 
Kassel, Germany 

(* naw at Section Flight Guidance and Control /Air Transportation 
Berlin University of Technology, Berlin, Germany) 


Diadem, created by THOMSON-CSF, is a methodology for specifying and developing 
user interfaces. It improves productivity of the interface development process as well as 
quality of the interface. The method provides support to user interface development in 
three aspects, (1) Diadem defines roles of people involved and their tasks and organises 
the sequence of activities. (2) It provides graphical formalisms supporting information 
exchange between people. (3) It offers a basic set of rules for optimum human-machine 
interfaces. The use of DIADEM in three areas (process control, sales support, and multi- 
media presentation) was observed and evaluated by our laboratory in the European proj¬ 
ect DIAMANTA (ESPRIT P20507), The method provides an open procedure that leaves 
room for adaptation to a specific application and environment. This paper gives an over¬ 
view of Diadem and shows how to extend formalisms for developing multimedia inter¬ 


From November 1995 to December 1996, the Systems Engineering and Human-Machine 
Systems Laboratory at the University of Kassel participated in the project DIAMANTA, 
supported by the European Union as project 20507 in the ESPRIT programme. Goal of 
this project was to confirm the suitability of the software development methodology 
Diadem (Dialogue Architecture and Design Method) as a means of developing graphical 
user interfaces that inherently satisfy user needs. Our group had a two-fold role. First, we 
provided one of the three teams that experimentally applied DiadEM to a software devel¬ 
opment problem. Second and independently fi*om the first, we designed and applied a 
formal method to observe the three teams, to determine their progress and experiences 
and, thus, to assess the benefits of using Diadem itself. We write this paper from the 
views of the manager (a DI AD EM role described later) of the development team as well as 
from the view of the main responsible person for the evaluation. 

The other partners in DIAMANTA were THOMSON-CSF (France), Sistemas y 
Tratamiento de Informacion S. A. (STI, Spain), and Informationssysteme fur computer- 
integrierte Automatisierung GmbH (ISA, Germany), THOMSON-CSF provided knowl¬ 
edge about Diadem and supervised its application. 

We will briefly outline the methodology itself and report our experiences gained during 
application of the methodology to the development of a process control interface. We will 
show how to extend DI AD EM to different and new applications and figure out how these 
adaptations may look like for multimedia applications in process control. 


The Diadem Methodology 

K process of user interface development becomes more and 

more an important economic factor for software development 
companies. Experience shows that the user interface software re¬ 
quires about 20% to 50% of the total software development costs. 
On the other hand, a well-designed user interface improves the usability of the product 
and, by this, the efficiency of its use and its acceptance by the users. 

Against this background, the Diadem methodology aims at giving software developers 
support for their development activities. Based on a dialogue model and an ergonomic 
approach to user interface design, THOMSON-CSF COMMUNICATIONS created 
Diadem in 1992 in conformation to the THOMSON-CSF software reference system. 
From the beginning, the method featured an adaptable multi-modal dialogue based on er¬ 
gonomic principles. Use in over 30 projects in the THOMSON group provided feedback 
for a new version, until from the end of 1995 the project DIAMANTA evaluated 

The objectives of Diadem are to enhance the efficiency in developing human-machine 
interfaces and to ensure the quality of the user interfaces to be developed. These objec¬ 
tives shall be achieved by providing a guide to structure, organise, and supervise all ac¬ 
tivities of the software development team. For the activities, DI AD EM defines develop¬ 
ment phases, for the team members, DI AD EM defines roles. We will describe both in the 
following sections along with our experience made during the DIAMANTA project. 

Diadem Roles 

Diadem suggests distinguishing the role of a manager, a specifier, a human factors spe¬ 
cialist, a programmer, and a user. It was a large industrial company that developed 
Diadem, and, therefore, the original users of the method were large teams. All teams in 
our project were smaller. The practical evaluation of Dl AD EM in our group as well as in 
the other development teams showed that it is difficult to have exactly these persons or 
group of persons covering these professions constantly available in a development team. 
Thus, it is better to speak of roles covered by different people in the development team 
while one person may cover different roles as well as one role may keep several persons 

By giving roles, Diadem describes the responsibilities and activities of the people in¬ 
volved in the software development. The manager is responsible for the overall organisa¬ 
tion and supervision of the development process. The specifier interacts closely with the 
end user during task analyses and develops comprehensive specifications for the user in¬ 
terfaces. These specifications formally describe the interaction of the user with the inter¬ 
face and build the basis for the software coding activities of the programmer. It appeared, 
that the original role of the human factors specialist should better be distributed to two 
roles: The cognitive ergonomist and the practical ergonomist. The cognitive ergonomist 
supports the specifier during the tasks analyses and ensures an appropriate breakdown of 
the user activities. The practical ergonomist generates the Man-Machine Interface (MMI) 
Handbook. This is an application-specific set of rules ensuring consistency and usability 
of the interface, based on a general set of ergonomic rules and guidelines. In addition to 
the task analyses at the beginning of the development, the ergonomists involve end users 
throughout the whole development process in usability tests. 


The development team at the University of Kassel involved five persons on the develop¬ 
ment side and two end users. One student acted as specifier while writing his diploma 
thesis on task breakdown and operation strategies in the apphcation. Two students 
worked as practical ergonomists, writing homework (Studienarbeit) on human factors 
guidelines in the application-specific MMI Handbook. The programmer was a mechani¬ 
cal engineer who just finished his degree. An engineer and a senior operator from a 
chemical plant supported the development team as end users of the system to be devel¬ 
oped. Finally, one of the authors covered the roles of manager, human factors specialist, 
and partly of the programmer, and he also assisted the specifier. This last combination of 
roles in one person contradicts some of the basic ideas of DI AD EM, in which the human 
factors specialist should defend the user’s needs against the manager. However, the two 
practical ergonomists counterbalanced this inadequacy during enthusiastic debates. 

Diadem Formalisms 

In the whole, DI AD EM focuses on having a common basis for conmiunication within the 
development team, and on being able to supervise and trace the development process. 
More than that, the comprehensive formal descriptions of the user interface make mainte¬ 
nance and further development of the software product easier. To co-ordinate the work 
within the development team, DIADEM provides several formal specifications. A task 
graph represents a first formal description of the users' task derived from a task analysis. 

The task graph shows a breakdown of the user’s goals in a hierarchical structure. So- 
called use relations connect the levels of the hierarchy. To achieve the specific goal, the 
user uses the tasks on the lower level. The task graph in our application showed the over¬ 
all goal "produce product" and lower-level goals like "start die system" and "correct er¬ 
ror". Figure 1 shows a more everyday example. 

The task graph enforces re-use of software components: On the lower levels, a small 
number of generic tasks were used to reach several goals on a higher level. In the process 
control application, the two tasks "change a pump status" and "set a valve position" alone 
could form the lowest level. For the implementation, this means only two software mod¬ 
ules, one to control a pump and another for operating a valve. 


Further formalisms provided by D I AD EM are technical sheets, one for each task in the 
task graph. The technical sheet comprises a strategy graph, giving an abstract description 
of the interaction between user and dialogue system. The strategy graphs formed the main 
source of information for the programmer. Using a window-based system, the program¬ 
mer created one dialogue window for each task. The strategy defined the logic of interac¬ 
tions in this window. It is important to note that the strategies only define the logic of in¬ 
teractions. They do not define the mechanism and media to be used. To keep use of 
mechanism and media consistent throughout the development, they are defined in the ap¬ 
plication’s MMI Handbook. 

This MMI Handbook is another formalism provided by Diadem. With the specifications 
recorded here the programmer can turn the strategies into algorithms driving the interface 
in a correct, economical sound and consistent way. Rules defining how to implement an 
interaction in a specific application enforce consistency. This starts with common pres¬ 
entation choices like colours of window border, window background, character fonts 
used, and it continues with window organisation and failure handling. The MMI Hand¬ 
book must cover the whole range of decisions to take when implementing the strategies. 
If not - it needs to be extended with the necessary additional decisions. Figure 2 shows 
rules from the handbook written for our application. 

lCOhi04 Number of icons 

tn sppifeaffofl, onjy the IbBowf bo toone ene 

used on buttons. 
wcN WMCtien 

representing the enctetf swa rf : • 

«n object 

Ml noa^lbepossibHfty to adjust 


J coflTiimeftoperetion 

-fr : 




. . P^rammafatefuntatoftkei^Bteuse^ 

Ptogramnieble imm keys nrf tn into flpjfllcetjw. 

TU use of jwogwmmeWe \m fe adapted to wrad e«ess 

iMuiraiitonta. Prea^ng by idatalw odn entai jmbodanloQosequenBes. 
tbwefertc«en$ beaeiwfetod, gtoupetf. tor example^ ioawtof 

birttone poafioned ^eng ifto screen, etc. 
fi£yS02 t!be TAB is used as fdtows: 

UteT/ifi key t? uead to move the cursor brfwew irf)}erfa to 9 fpnn. 

Figure 2 Definitions in the MMI Handbook (from Engel and, Quittkat, 1996) 

The Diadem handbook "Guidelines for Authors" (THOMSON 1995, 4) provides a table 
of topics to be treated in the MMI Handbook. A set of general rules is contained in a ge¬ 
neric MMI Handbook (THOMSON, 1995, 5) that can be extended and adapted to any 
kind of application to ensure the consistency of the user interface and the adherence to 
ergonomics rules. 


Diadem Activities 

Besides proposing a certain staff composition for the development team and providing 
formalisms to specify the user interface, Diadem also suggests how to organise the de¬ 
velopment process. In the ideal case, this process should follow the downward part of the 
V-model, covering system specification, software specification, preliminaiy design, de¬ 
tailed design, iterative modification, and finishing phase. Nevertheless, even if the devel¬ 
opment process will not follow the V-model, this will not affect the possibility to make 
use of the concepts and formalisms provided by the method. DI AD EM defines activities 
grouped in four development phases, called System Requirements, Software Require¬ 
ments, Preliminary Design, and Detailed Design, 

The System Requirements phase starts with a task analysis and defines requirements and 
capabilities of the user interface based on user goals and means necessary to achieve 
these goals. The Software Requirements phase defines descriptions of the user’s tasks and 
user objects. The MMI Handbook is written during this phase, defining the future appear¬ 
ance of the interface or, at least, of the first prototype. Diadem enforces the early genera¬ 
tion and evaluation of prototypes involving end users, leading to modifications of the 
definitions in several cycles. TTie graphical formalisms described above ease user com¬ 
ments to software designers. The software architecture is designed during the Preliminary 
Design phase while in the Detailed Design phase presentation and dialogue descriptions 
are completed. 

Our project covered a development process up to the preliminary design phase involving 
four prototypes of growing complexity and functionality. As with the roles, here again the 
problem of a small group compared to a large software development company shows up. 
It was not possible to follow strictly the sequence of development phases, as all the peo¬ 
ple in the development team must be kept continuously busy in the assigned roles 
throughout the project. Thus, phases were overlapping and not consecutive and activities 
were scheduled more regarding the availability of people than depending on the com¬ 
pleteness of required inputs. For example, MMI prototyping started when the program¬ 
mer was available and the first user task descriptions existed; it did not wait for a com¬ 
plete task analysis and a satisfactory MMI requirements definition. However, going 
through four prototyping cycles in four months step-wise compensated the early deficien¬ 

Evaluation of DiadeM 

The Diadem methodology for user interface development has been designed to be appli¬ 
cable to any kind of software apphcation. The Trial Application Project DIAMANTA 
was initiated to show that this goal could be achieved. Therefore, Diadem was used to 
develop three very different kinds of software applications. These three applications were 
a tourism information system, developed by STI; a sales support system for used cars, 
developed by ISA; and an interface to chemical process control, developed by our group. 

By applying DI AD EM to develop the applications, its overall suitability was evaluated and 
the specific costs and benefits of using DIADEM were pointed out. This was done by 
means of an evaluation procedure especially designed for the DIAMANTA project. 

The primary focus of the DIAMANTA evaluation activities was on the efficiency of the 
software development processes. However, the quality and usability of the user interfaces 


to be developed were a second important aspect. Therefore, we designed an evaluation 
and measurement procedure to cover these two criteria. This procedure, as shown in 

Figure 3, is based on three parts: Continual Evaluations, Periodical Evaluations, and Fi¬ 
nal Evaluations. 

Figure 3: The DIAMANTA evaluation procedure 

Weekly Continual Evaluations accompanied the development processes. They recorded 
the effort for development activities and subjective assessments by the developers in 
regular intervals. Periodical Evaluations followed each development phase completed. 
These aimed at recording the current state of the development process as well as the as¬ 
sessments by the developers referring to the respective development phase. Furthermore, 
we demanded to perform evaluations of the specifications and the interface prototypes in 
co-operation with real end users after finishing a development phase. The evaluation pro¬ 
cess ended with a Final Evaluation. It consisted of two parts: concluding assessments of 
the developers referring to the whole development process and a usability test of the de¬ 
veloped interface by the end user. 

All data recorded along the DIAMANTA trial application developments were processed, 
analysed and interpreted. Finally, the following conclusions were drawn: 

0 Diadem is sufficiently generic and flexible to use it in a large range of appli¬ 
cations and organisations, 

1 The adaptation of DI AD EM to the specific projects would have been much 
easier with the Customisation Guide now available. 

2 The generic MMI Handbook provided by Diadem is a valuable basis to de¬ 
velop a consistent interface. 

3 The formalisms provided by DI AD EM turned out to be a good basis for com¬ 
munication within the development team. They represent unambiguous speci¬ 
fications for the user interface, improve the efficiency of the development 


process, and make later modifications or extensions easier. 

4 The dialogue specifications are to a high degree independent from the kind of 
interface, thus, it will be possible to re-use existing specifications as the basis 
for development of future interfaces, including interfaces which use emerging 
interaction technologies. 

As pointed out above, the evaluation of the Diadem methodology within the DIA- 
MANTA project led to very satisfactory results. DIADEM increased the efficiency of all 
three application developments, and the user interfaces developed were of a good quality 
and usability by the end users of the respective systems. 

The good overall result is based on a comprehensive definition of user requirements at 
the very beginning of the development activities and on a constant involvement of end 
users in the development process. This leads to a reduced effort for expensive changes at 
a later stage. 

Furthermore, all requirements identified and all specifications become described in a 
formal way. This improves tracing the development process and communication between 
the people involved. Even validating the software developed is simplified by the formal 
descriptions. Finally, these specifications are invaluable means for developing future ver¬ 
sions of the software. 

One significant result of the evaluations performed in the framework of the DIAMANTA 
project is that the evaluations of the three very different applications led to the same con¬ 
clusions. Additionally, evaluation results of the development processes correspond with 
the results of a heuristic evaluation of Diadem from a theoretical point of view. This 
suggests that the conclusions drawn are valid even for a broad field of applications. 

Customising Diadem to Multimedia Interfaces 

The common benefits of using Diadem, as pointed out in the DIAMANTA project, are a 
result of the generic concept of the DI AD EM methodology. However, this generality also 
induces the need to customise the method to a specific application. 

Multimedia applications are a new field, providing features that are quite different from 
those of conventional software. Therefore, developing multimedia interfaces puts new re¬ 
quirements on the specification work. To meet these new requirements, the formalisms 
provided by DIADEM must be extended in an appropriate way. The significance of mul¬ 
timedia extensions to process control interfaces is already stressed in another contribution 
from our group in this conference (Borys and Johannsen, 1997), Backed by our good ex¬ 
perience during the trial application of Diadem, we now consider continuing the use of 
the skill gained in future software development projects. 

The main anchor point we see for extending Diadem to multimedia applications is the 
MMI Handbook. Task analyses deliver task graphs and strategy drawings describing dy¬ 
namics of interactions necessary to operate an application. They leave completely open 
by what means or media this interaction would take place. The MMI Handbook is based 
on the ’’General Rules and Guidelines” (THOMSON 1995, 5). First, this handbook needs 
extensions to cover the decision ’’what medium to use for what type of interaction". A be¬ 
ginning is already made in Chapter 5 Dialogue: Common Choices with rule DIAMOl: 
"The dialogue modes must be chosen taking into account the characteristics of the 
users and the task requirements: For example ... use frequency, ... time con- 


straints..." (THOMSON 1995, 5, pg. 59). 

Some new basic rules should cover ergonomics of additional media: Size of video pres¬ 
entation, level of acoustical output, speed of synthesised speech. An interesting research 
topic would relate to spatial sound sources, corresponding to Dl AD EM rules in Section 4.6 
of the handbook covering arrangement of elements on the screen. Rule SPGR03 defines 
the greatest distance for grouped objects, SPGR04 the minimum distance for separated 
objects, both in visual angel. Besides the angle, a rule for sound presentation must take 
care of differences in frequency, perceived distance, and loudness. 

Some amount of new rules will certainly extend Chapter 4 ("Presentation: Common 
Choices") and Chapter 9 ("Interaction Tools"), Especially the latter, which currently only 
mentions cursor and mouse, will undergo some extensions. It must cover additional input 
media like voice or the famous (but expensive) cyber glove, but it also must regard addi¬ 
tional more-dimensional mouse operations like pointing in space and rotations. Rules 
could be 

"when dialogue mode is question/answer and presentation mode is speech and an¬ 
swer is short, input medium should be voice" 

"when object moves, video bandwidth must permit a frame rate of... 

Finally, our aim is always to involve the user in the development. Thus, a basic guideline 
would be 

"when the user wants to hear it, provide sound, when the operator is used to feel it, 
provide vibrations". 

This will bring back the operator from the distant control room to the plant again, as it is 
also required in our other contribution to this conference. 


Diadem, a methodology for development of user interfaces, leads developers through the 
developing process. One of the last steps is turning strategies, derived from task analyses, 
into program code. When extending DI AD EM to multimedia interfaces, this last step 
needs to take into account the capabilities of new media available. Additional research, 
extending the mle set provided by DI AD EM, will provide a useful basis to develop multi- 
media interfaces. 



Borys, B.-B., Joharmsen, G. (1997), An Experimental Multimedia Process Control Room, 
Human Factors and Ergonomics Society Europe Chapter Annual Conference. 

Engel, M., Quittkat, M. (1996). Man-Machine Interface Handbook. Laboratory for Sys¬ 
tems Engineering and Human-Machine Systems, University of Kassel. 

THOMSON-CSF (1995,4): Software Development Guides: DIADEM - Dialogue Ar¬ 
chitecture and Design Method. Part 4: MMI Handbook Guidelines for Authors. 
partment of Technology. 

THOMSON-CSF (1995,5): Software Development Guides: DIADEM - Dialogue Ar¬ 
chitecture and Design Method. Part 5: General Rules and Guidelines. THOM¬ 
SON-CSF/SCTF/DLS - GDL 11-5 F, Issued by THOMSON-CSF, Department of 


WWW. ect/diamanta/DIADEM/diad_hom.htm 

Tiemann, M. (1996): Evaluation of a Methodology for Human-Machine Interface Devel¬ 
opment. XV. European Annual Conference on Human Decision Making and 
Manual Control, Soesterberg/NL. 

Tiemann, M., Averbukh, E., Johannsen, G. (1996a). Evaluation and Measurement Proce¬ 
dure. ESPRIT-Project DIAMANTA, Deliverable D4. Laboratoiy for Systems En¬ 
gineering and Human-Machine Systems, University of Kassel. 

Tiemann, M„ Averbukh, E., Johannsen, G. (1996b). Synthesis of Evaluation and Meas¬ 
urements. ESPRIT-Project DIAMANTA, Deliverable D5. Laboratory for Systems 
Engineering and Human-Machine Systems, University of Kassel. 


The European Union supported the work described above as ESPRIT project 20507 
DIAMANTA, Partners in this project were THOMSON-CSF (France), Sistemas y 
Tratamiento de Informacion SA (STI, Spain), the University of Kassel (UKS, 
Germany), and Informationssysteme fiir computerintegrierte Automatisierung 
GmbH (ISA, Germany). The development of the DIADEM method has been 
sponsored mainly by THOMSON-CSF COMMUNICATIONS. 


What is Multimedia? 

Proposal of a Taxonomy for Human-Machine Communication 

Georg Geiser 
Katholische Universitdt Eichstatt 
D-85071 Eichstatt, Germany 


That there is no accurate or standard definition of multimedia is a firequent statement in 
publications on this subject. After a survey on the resulting increasing variety of defini¬ 
tions, the existing terminology is characterised by discrepancy, inconsistency, incom¬ 
pleteness, and lack of task orientation. A proposal of a taxonomy for multimedia is 
developed with three categories: medium, representation and modality, which are part of 
a three-level concept being applicable to the input, and the output side of the human user 
as well. Thus, a clear separation of technical devices, modes of information 
representation, and of human sensory and effector channels is achieved. The importance 
of a task-related design of multimedia is finally shown by means of two examples which 
illustrate the relationship of the taxonomy to multimedia design tasks. 

Structure and Development of Human-Computer Interaction 

Before analysing multimedia human-computer interaction, the essential input and output 
channels of human information processing are briefly described. Figure 1 shows 
examples of the multimodality of human information input and output, i. e. there are 
multiple modalities for receiving and sending information. Sensoiy modalities on the 
input side are the classical five senses, however, actually only the eye, the ear, and the 
tactile sense are suitable for human-machine communication. For the output effector 
modalities such as mimic, voice, and gesture are used. 

Figure 1. Human information input and output channels. 


Graphics, Image 



Virtual Reality 


Figure 2. Development of human-machine communication. 

These means given by human nature have to be seen in relation to the technologies for 
presentation, processing, and input of information. It is interesting to look at the history 
of the corresponding technical devices. In Figure 2, important steps of the evolution of 
human-machine communication are presented. First, digits were the coding form for 
information to be exchanged between human and machine, which were represented by 
means of dials, digital displays, and rotary knobs. Then screen and keyboard allowed to 
include text. Recently, speech and image are additional coding forms for input and 
output. And actually expert systems are including knowledge representation. 
Furthermore, virtual reality allows the simulation of working and learning environments 
which, up to now, were not available in reality due to cost reasons or dangers. Today 
computer networks offer increasing conununication services for numerous professional 
and private applications. 

This development results in multimedia technology allowing increased adaptation of ma¬ 
chines to the properties of human information processing and even expanding human ca¬ 
pabilities. A multimedia computer is supposed to process not only columns of numbers 


and sequences of letters as in the beginning of computer era but also to handle sound, 
speech, graphics, and image sequences as input and output material or as transmitted net¬ 
work information. 

Various Definitions of Medium and Multimedia 

In various application areas multimedia moved into the focus of interest, e. g. in market¬ 
ing, telecommunication, and computer-based learning. Though used as a buzzword or 
collective noun, there are numerous efforts to find a taxonomy for this new technical 
area. In the following different definitions are compared by deliberately concentrating on 
the area of human-machine communication. Interpersonal communication as the other 
field of interest represented by publicity, television or arts is not considered here. 

Even for the term "medium" many definitions can be found, few of them are hsted in 
Table 1 in order to show the range of variance. In some references the definition of 
medium is omitted (e. g. Interactive Multimedia Association (IMA) 1996) or only an 
implicit definition is given by simply listing types of media (e. g. Heller, Martin 1995). 
Burger 1993 assigns medium to a transmission means in the sense of physics, e. g. air, 
water. ISO draft standard ISO/IEC CD 13522-1, 1993 (MHEG 1993) defines medium in 
a very comprehensive way as human and technical means for perception, representation, 
storage, transmission, and exchange of information. 

Weidenmann 1994 points out that definitions of medium are often vague and therefore 
he specifies: A medium is a technical means for presentation or storage of information 
for the human user. Blattner and Ghnert 1996 agree with him roughly when they 
designate media as information carriers. In their media taxonomy, Heller and Martin 
1995 define text, graphics, sound and motion as four types of media and additionally 
they assign three categories of expression to each medium. Khmsa 1995 uses a similar 
classification of media types. Additionally he distinguishes between time independent 
(text, graphics) and time dependent (video, audio, animation) media. 


Definition of „Medium“ 

Burger 1993 

Means for transmission for stimuli: air, water etc. 

MHEG 1993 

Means for perception, representation, presentation, storage, 
transmission and exchange of information 



Blattner, Glinert 

Means for presentation and storage of information (paper, 
screen etc.) 

Information carrier (paper, paint, video, CD-ROM, screen etc.) 

Heller, Martin 

Klimsa 1995 

Types of media: text, graphics, sound, motion 

Time independent (text, graphics) and time dependent media 
(video, audio, animation) 

Table 1. Vaiious definitions of "medium" by different authors. 


As there is no agreement on the term medium, the definitions of "multimedia" inevitably 
must show at least a comparable variety, A first example is given by Steinmetz 1993: A 
multimedia system is characterised by computer-controlled, integrated generation, 
manipulation, presentation, storage, and communication of independent information ele¬ 
ments being coded at least with one continuous (time dependent) and one discrete (time 
independent) medium. Whereas Koegel Buford 1994 calls the simultaneous use of 
different media (voice, video, text, animation etc.) multimedia. 

Weidenmann 1994 criticises available definitions as confounding the categories techni¬ 
cal devices, presentation modes, and sensory channels. He uses three terms for 
multimedia information presentation: medium, coding, and modality, hi addition to his 
already mentioned definition of medium, he declares coding as symbol systems for 
presentation of information. Following established definitions he assigns modality to 
human sensory channels. Blattner and Glinert 1996 end up with similar terms, however 
they leave an ambiguity of modality by allowing the two meanings sensory channel and 
interaction style. 

In his definition of multimedia, Klimsa 1995 claims the use and integration of several 
media, reference to the application and multimodahty as essential aspects. Multimodality 
is understood as parallel processing and simultaneous presentation of several media and 
interactivity, Hofstetter 1995 points out four essential components: computer-based in¬ 
formation processing, links that connect the information, navigation tools, and ways for 
user-specific interaction. He states:,Multimedia is the use of a computer to present and 
combine text, graphics, audio, and video with links that let the user navigate, interact, 
create, and communicate.“ 

The Interactive Multimedia Association (IMA) 1996 defines multimedia: „Delivery of in¬ 
formation, usually via a personal computer, that combines different content formats (text, 
graphics, audio, still images, animation, motion video, etc.) and/or storage media 
(magnetic disk, optical disc, video/audio tape, RAM).“ 

These and other definitions of multimedia are characterised by one or several of the fol¬ 
lowing deficiencies: 

• Discrepancy: There are considerable differences between most terminologies, 

• Inconsistency: Within some definitions terms are confounded, e. g. partially 
the definition of a single term relates to technical means and partially to 
presentation forms and sensory modalities. Furthermore, well established 
terms such as modality are redefined. 

• Incompleteness: Only partial aspects are dealt with, e. g. networking is mostly 
neglected and the information input and the output side of the human user are 
not both included. 

• Lack of task orientation: The user’s tasks of information input, processing, 

output are generally not considered. 

Multimedia Taxonomy 

In the following a more precise, comprehensive and user and task oriented taxonomy for 


the interaction of human with multimedia devices is proposed. In order to obtain such a 
definition human information input as well as information output and the user’s task 
have to be considered. Three terms as part of a three-level concept are used (Table 2): 

• Medium 

To begin with, a medium is a technical means for one or several of the following 
purposes: Presentation, storage, input, processing, transmission of information for and by 
the human user (example: screen). The media available are determined by the state of 
hardware and software technology. 

Looking at the media only is not sufficient to describe formally the information exchange 
between human and machine; two additional terms have to be introduced, representation 
and modality. 

• Representation 

Representation is the appearance of information, enabling human perception, processing 
or output of information (examples: text, displayed visually or spoken). 

• Modality 

Modality encompasses human sensor or effector channels for input or output of informa¬ 
tion (examples: visual system, gesture). 

On the first level of this approach technical requirements are addressed by medium and 
by the next two levels representation and modality, mainly the areas of human 
perception, cognition, and motoric are concerned. 



Areas concerned 


Means for presen¬ 
tation, storage, in¬ 
put, processing 
and transmission 
of information 

Screen, speaker, 
CD-ROM, network 





Appearance of 

Digit, text, colour, 

icon, table, bar 
graph, hypertext, 
tone, speech, vir¬ 
tual reality 


cognition, and 
motor systems of 
human user 



Sensor or effector 
channel for input 
and output of in¬ 

Visual, auditory, 
and tactile system; 
mimic, gesture, and 

Table 2. Definition of medium, representation, and modality. 

As Table 3 shows these three categories are apphcable to the human information input 
and output as well. Examples listed for illustration in Table 3 are not exhaustive. As an 


important aspect the task of the human has to be mentioned together with its result 
because use and therefore the design of these 2x3 levels depend strongly on the kind of 
task to be accomplished by the human user. 

These 2x3 levels may be used in a restrictive manner by using minimum numbers of 
media, representation forms, and modality channels. Multidimensional use offers 
abundant variety of interaction by multimedia and/or multirepresentation and/or 
multimodality. However, these three levels are not independent, e. g. the representation 
image can be displayed with the media paper or screen, but can only be perceived with 
the modality eye. 

Processing Levels Examples 


Input of 




Paper, screen, speaker 


Text; table 

Modality (Sensors) 

Eye, ear, tactile sense 



Modality and representation specific 



Output of 




Gesture, voice, mimic 


Text; formal language 

Medium (Effectors) 

Paper, key, microphone 

Table 3. Formal description of human information input and output by means of me¬ 
dium, representation, and modality. 

of Information 




System of symbols for single information 

Digit, text 


Structure for local, temporal, and se¬ 
mantic arrangement of several information 

Table, hypertext 

Table 4. Subcategories of representation of information: coding and organisation. 


The level representation can be divided in two subcategories (Table 4): 

• Coding 

Coding means the use of a system of symbols for single information elements, e. g. visual 
display of a measured variable such as time by digits (digital code) or by geometric pa¬ 
rameters (analog code). If there is a single information element to be presented merely 
coding is necessary, in the case of a set of information elements an additional mode of 
representation has to selected, namely 

• Organisation 

Organisation of information describes the representation of several information elements, 
e. g. arrangement of several numeric values by means of a table, bargraph, etc. Here the 
presentation of local, temporal and semantic structures are to be distinguished. Hypertext 
is an example for the last type of organisation allowing reading a text in a non-linear 
manner according to the contents. 

Thus, human-machine communication is not only determined by media but also by 
representation and modality. We need a three-level model in order to realise the 
extraordinary abundance of communication modes which the usual multimedia term tries 
to outline with a single notion. Also for a systematic design of human-machine 
communication this more detailed terminology seems to be necessary. So the following 
definition of multimedia is proposed: 

Utilisation of media, L e. several technical means for presentation, storage, input, proc¬ 
essing, transmission of information, for the human user in order to permit multiple 
representation (coding and organisation) of the information which meets the 
requirements of the human given by his modalities (sensors and effectors), cognition, 
and tasks. 

The proposed terminology is more comprehensive than the existing ones by including 
human information input, processing, and output as well. New developments can be 
easily included: new media such as computer networks, new organisation concepts such 
as hypertext or new interaction modes such as virtual reality. 

Figure 3 shows the components of today’s multimedia human-machine system. Sensor 
and effector modalities on the one hand, and displays (visual, auditory, tactile) and con¬ 
trols (manual, vocal) on the other hand, build the interface between human and machine. 
With these devices information with different forms of coding and organisation can be 
exchanged. Network connections and other devices like image scanners make additional 
information available. 


In the following the application of the proposed multimedia taxonomy is shown by 
means of two examples. In the first application (Table 5), which is based on the task to 
acquire ornithological knowledge, only the part of information presentation for the 
learner is considered. The second case concentrates on information output while learning 
a foreign language (Table 6), These simple examples illustrate the essential multimedia 


design tasks to be accomplished: choice of media, coding and organisation of 
information based on human properties of information input, cognition, and output in a 
task oriented manner. 

Figure 3. Multimedia human-machine system. 


Acquisition of 








Name, family 



Flight behaviour 





Image sequence 
Analog function 




Time scale 


Actual research 









Table 5. Example for the application of the multimedia taxonomy for human informa¬ 
tion input. 



Foreign Lan¬ 
guage Learning 











Keyboard ar- 










Record and play¬ 
back control 


Table 6, Example for the application of the multimedia taxonomy for human informa¬ 
tion output 

Relations to other taxonomies 

Based on this terminology the area of representation of information both on the input and 
the output side of the human user must be further differentiated. Several authors propose 
classification schemes for this level of the proposed multimedia taxonomy (Table 7). 


Classification Schemes for Representation (Coding and 

Arens et al, 1993 

Temporal endurance, granularity, baggage, detectability, 
and media type 

Bernsen 1993 

Linguistic/nonlinguistic, analog/nonanalog, 
arbitrary/nonarbitrarv, and static/dynamic 

Heller, Martin 1995 

Elaboration, representation, and abstraction 

Table 7. Classification Schemes for Representation (Coding and Organisation). 

Five categories are introduced by Arens et al. 1993: temporal endurance, granularity, 
baggage, and detectability. The first two describe time and resolution aspects, namely 
permanent or transient and continuous or discrete properties. Baggage is a measure of the 
user’s interpretation effort, and detectability characterises the intmsiveness of a 

Bemsen 1993 suggests a taxonomy of output modalities which should be called 
representation modes according to the taxonomy proposed in this paper. He applies the 
categories linguistic/nonlinguistic, analog/nonanalog, arbitraiy/nonarbitrary, and 

Heller and Martin 1995 describe a taxonomy which further differentiates the represen¬ 
tation level called media expression (Table 8). They allocate three types of expression 
namely elaboration, representation and abstraction to their four types of media text. 


graphics, sound, and motion. Elaboration is the unchanged original version of the 
information, e. g. the medium text may have the elaborations free text, sentences or para¬ 
graphs. Abbreviated, stylised modes are called representation (formatted text, drawings, 
etc.). Finally, abstraction as the third type of expression is based on metaphor and 
standardised forms. In summary, media expression denotes degree of abstraction, author 
control, and design effort at three levels. 

Media Type 


Media Expression 



Free text, sentences, 


Bold, italics, head¬ 

Shapes, icons 


Photographs, rende¬ 
rings, scanned images 

Blueprints, schema¬ 



Speech, audio 

Intensity, tone, inflec¬ 

Sound effects 


Raw film footage 

Animation, time- 
lapsed photography 

Animated models, 
highly edited video 

Table 8. Media expression (Heller and Martin 1995), 


It has been shown that the proposed taxonomy allows a more systematic description of 
multimedia human-machine interation. However, instead of the extensions described this 
taxonomy remains incomplete as the area of humian information processing was widely 
excluded. Obviously, representation and modality influence the ways how users process 
perceived information and how they generate output information. Therefore a main task 
in the future development of multimedia will not only be to consider the properties of 
human sensor and effector channels but also to support cognitive information processing 
by a suitable choice of medium, representation of information, and modality. 



Arens, Y.; Hovy, E.; Vossers, M. (1993) On the Knowledge Underlying Multimedia 

Presentations, In Intelligent Multimedia Interfaces, M. Maybuiy, ed., MIT Press, 
Cambridge, Mass,, 1993, pp. 470-506. 

Bemsen, N.O. (1993) The Structure of the Design Space. In Computer, Communication 
and Usability: Design Issues, Research, and Methods for Integrated Services, 
P.R Byerley, P.J. Barnard, and J, May, eds., North Holland, Amsterdam, 1993, 
pp, 221-244. 

Blattner, M.M.; Glinert, E.P. (1996) Multimodal Integration. IEEE Multimedia, Winter 
1996, pp. 14-24. 

Burger, J. (1993) Desktop Multimedia Bible. Addison-Wesley Publishing Company, 
Reading (Mass.) etc., 1993. 

Heller, R.S.; Martin, D.C. (1994) A Media Taxonomy. IEEE Multimedia, Winter 1995, 
pp. 36-45. 

Hofstetter, F.T. (1995) Multimedia Literacy. McGraw-Hill, New York etc., 1995. 

Interactive Multimedia Association (IMA) (1997) Glossary of Multimedia Terms. 

Interactive Multimedia Association,, 
visited 27,06.1997. 

Klimsa, P. (1995) Multimedia, Anwendungen, Tools und Techniken. Rowohlt, Reinbek, 

Koegel Buford, J.F, (1994) Uses of Multimedia Information. In Multimedia Systems, J.F. 
Koegel Buford, ed., Addison-Wesley Publishing Company, Reading, Mass, etc., 
1994, pp. 1-25. 

MHEG 1993, ISO/IEC JTC1/SC29AVG12 (1993) Information Technology - Coded 
Representation of Multimedia and Hypermedia Information Objects (MHEG). 
ISO Commitee Draft, ISO/IEC CD 13522-1, 1993, 

Steinmetz, R, (1993) Multimedia-Technologie. Springer-Verlag, Berlin etc., 1993. 

Weidenmann, B. (1994) Codes of instructional pictures. In Graphics and mental 
models, W. Schnotz and R.W. Kulhavy, eds., Elsevier, Amsterdam, 1994, pp. 29- 


Comparing Multimedia Concepts by Using Socio-oriented 

Modelling Methods 

Thomas Herrmann, Kai-Uwe Loser, Klaus Moysich and Thomas Walter 

University of Dortmund 
Computer Science and Society 
Dortmund, Germany 


This paper presents a method for modeling and comparing different multi-media con¬ 
cepts. The focus of our method lies on a social-oriented evaluation of these concepts. We 
identified and systematized elementary characteristics of multi-media systems and de¬ 
termined a set of frequently used building blocks to support the construction of models 
representing concepts of multi-media applications. The models can be used to visualize 
aspects of the working environment and to support early discussion between different 
interest groups. To support the evaluation of the different models we developed a proce¬ 
dure consisting of several if-then-rules which allows to find out the weakpoints of multi- 
media concepts referring to various social aspects. 


Expecting an enormous variety of multi-media applications, the question arises of how to 
compare different multi-media concepts and to select the most promising approaches. 
Multi-media comprises every kind of system which is used to generate, manipulate, dis¬ 

tribute and receive documents or transient data which include pictures (fixed as well as 
animated), graphics, sound, speech, text, applets, etc. Furthermore these systems can be 
used to support cooperation and communication; therefore groupware and multi-media 
will be increasingly integrated. Most important, these systems have to be embedded into 


an appropriate organizational structure as this is the decisive factor for success. It in¬ 
cludes mostly formal as well as informal social aspects. To anticipate the acceptance of 
multi-media applications, besides costs, their usability and usefulness related to human 
needs have to be considered. In this paper we present a modeling and evaluation method 
that is oriented towards those socio-technical aspects. 

Stereotypes and socio-oriented components for mulU-medm applications 
To identify and systematize elementary characteristics of multi-media systems we ana¬ 
lyzed concepts in the hterature as well as prototypes and existing apphcations. Unlike 
classifications which are based on the available types and classes of media and docu¬ 
ments we concentrated on blocks of typical functionality concerning social interaction. 
We call those abstract groupings of functionality “multi-media stereotypes”. The under¬ 
lying idea is to use these stereotypes in modeling various kinds of multi-media systems. 
The six multi-media stereotypes in figure 1 are therefore parts of our modeling notation. 

Through the refinement and synthesis of the inspected applications we determined a set 
of frequently found precise complexes of functionality. We call them building blocks. 
These “blocks” can repeatedly be used to “build” different multi-media applications. 
With this method, the set of building blocks can be enlarged if necessary. For example, a 
multi-media Group Decision Support System is “built” by “blocks” such as moderation, 
evaluation, whiteboard, email and merging of data. A subset of elements with social 
relevance (ref. fig, 2) was identified to supply a basis for the proposed method. 

As an example the building block Authentication is described in detail. Authentication is 
related to the social aspect authenticity of information. By authentication one can verify 
the assumed origin of a message or a document. Authentication is based on crypto tech¬ 
nology. In principle the sender (res. author) adds an authentication key to the information 
which can be verified by the addressee using the appropriate test key. The process of 










automatic Reply 



AV-Communicati on 


















Fig. 2: Social relevant building blocks 

authentication consists of the following steps which can be represented by blocks of our 
modeling method: exchanging the test key on a safe communication channel where its 
authenticity can be guaranteed by a trustworthy third party, adding an authentication key 
to the information to be transmitted and checking the authenticity after transmission. The 
authentication checking service is started by entering the test key and the pair of authen¬ 
tication key and information. As the result it reports the reliability of the inspected in- 

formation. It is sensible to describe the complex structure of authentication by a model. 
One can imagine different concepts being possible to organize authentication and to em¬ 
bed it into multi-media applications. These different concepts can also be modeled. 

Modeling multi-media systems for the socio-oriented evaluation 
The usage of models is especially intended to compare different concepts of multi-media 
applications. The comparison is intended to be socio-oriented which means, that aspects 
of the working environment of users are visualized with diagrams serving as a foundation 
of socio-oriented evaluation. The method for the socio-oriented evaluation of models is 
described below. 

It must be pointed out that modeling can be constructive and re-constructive. For the 
purpose of discussing different concepts for the realization of socio-technical systems, a 
modeling method must be especially designed for the construction of such a system. The 
representation of social aspects and requirements is helpful to support an early discussion 
between different interest groups. Therefore we have identified the presented set of socio- 
oriented building blocks. It is advantageous that the notation and semantics of the mod¬ 
els are based on standard software engineering or business process reengineering models 
to avoid multiple recording of the same facts: existing models can be used for the socio- 
oriented evaluation and socio-oriented models should be usable to implement a system. 

In the following paragraphs we describe which requirements for a socio-oriented model¬ 
ing notation exist and how to meet them with an object-oriented notation. Object- 
oriented technology is especially suitable for component-based systems like multi-media 

The representations created by modeling methods should comply with ergonomic crite¬ 
ria, This is especially relevant if the models are used for a participatory evaluation of the 
modeled concepts. Easiness of creation, comprehensibility, communicability for model¬ 
ers and for recipients must be as good as possible to support the discussion of different 
realization concepts for the multi-media system. 

One approach to develop a modeling method for our purposes is to extent existing meth¬ 
ods, However, it might prove that this extension leads to a new method. In this paper we 
start with a modeling notation which is based on the object-oriented methodology intro¬ 
duced by Ivar Jacobson [Jacobson et al. 1992] and on the extensions for process model¬ 
ing [Jacobson et al. 1995]. We adapted and extended the notation by adding stereotypes 
analogous to Jacobson’s extensions for business process modeling and using modeling 
concepts from UML [Booch et al. 1997]. UML is a rising standard for object-oriented 
modeling and there already exist a mapping from the notations of Jacobson to the UML. 
With this mappings it is possible to transform models from the usage oriented perspec¬ 
tive to an implementation-oriented perspective. 

Basically, in a socio-oriented modeling notation, objects, activities, roles and relations 
must be representable [Herrmann et al, 1997]. 

Objects means physical objects as well as virtual objects in an object-oriented sense. As 
multi-media systems are built from components as mentioned above, object-oriented 
modeling techniques fit properly because of the independence of objects representing 


these components. 

Activities represent the actions of users and participants or system based processes in the 
socio-oriented system. Therefore it is necessary for the modeling notation to be able to 
visualize the dynamic aspects of the system to catch the sequences of actions . 

Roles are a concept to abstract from real world participants. This is very helpful in mod- 
ehng socio-technical systems. 

Relations between these concepts must be representable to fix all the dependencies be¬ 
tween objects, activities and roles. An example of relations is the usage relation between 
different objects or objects and activities. 

The selected modeling methodology from Jacobson et al. (OOSE) includes different 
types of models. In Use Case Models it is possible to visualize roles, activities and rela¬ 
tions between them. The object-model focuses on objects (types) and their relations. 

As one extension to the modeling notation we use the presented stereotypes to visualize 
the abstract functionality of components of the multi-media application (see fig. 3). 

When modeling multi-media systems it is especially necessary to visualize the flow of 
information, as multi-media systems make communication channels available. This flow 
of information is fundamental for the socio-technical systems. It must be transparent 
which pieces of information flow over the recognized channels and where the source and 
destination of this information lie. For this requirement we propose, in addition to the 
OOSE-Notation, the concept of association classes from UML, which makes it possible 
to associate types/classes to relations. The associations in the object models are therefore 
limited to unidirectional relations to prevent the hiding of the flow of information. 

It is helpful to start the modeling process with modeling the business with the Use Case 
Method to figure out the functional requirements for the multi-media system. The design 
of the application with predefined building blocks can be based on such a model. With 
regard to the dynamic development in the field of multi-media systems, the presented 
building set is expandable by adding or modifying building blocks and evaluation rules. 
For the social evaluation it is even necessary to catch those elements with social rele¬ 
vance. We introduced the socio-oriented components to support that. The modeling pro¬ 
cess is driven by those elements. Models will be refined until these elements are recog¬ 
nized or until it is obvious that they are necessary or it can be stated that these elements 
are not necessary anyway. 

It is also necessary to note different socio-oriented attributes to the types in the model. 
They are used for the socio-oriented evaluation described in the following section and for 
testing the completeness of the model. Therefore we use rules also presented in the fol¬ 
lowing section. The rules for modeling include questions and additions to the current 
model which make the social necessity of functionality visible (see fig. 7). 

As an example we describe parts of the model for an insurance company. In the insur¬ 
ance organization new contracts from sales representatives are first checked by office 
workers. After the first checking and correction a new workflow process should be initi- 


ated. The communication and the transport of new contracts should be supported with a 
multi-media environment. This business system can be modeled with Use Cases. 

Fig. 3: Insurance Office Example - Workplace Object System 

The object model of one possible concept for the multi-media application is shown in fig. 
3. It represents the multi-media system from the perspective of the office worker. 

First it is necessary to describe the used elements from Jacobson. The notation consists of 
types visualized by a circle. There are three basic stereotypes for types: control types 
have controlling functionality on the object system, visualized by an arrowhead, bound¬ 
ary types represent interfaces, visualized by a „T‘-adomment and entity types have in¬ 
formation storage functionality, visualized by a baseline. The modeling language for 
business processes consists of different other stereotypes. Especially for participants in 
the business system Jacobson used special symbols like the case worker stereotype (Of- 
fice Worker). Arrows represent relations in the object-oriented sense. 

The object model contains several examples for the usage of building blocks and multi- 
media stereotypes. The AV-Communication building block is used for the communica¬ 
tion between the office worker and the sales representative, visualized by the synchro- 
nous-communication-stereotype. Another communication channel called File Transfer 
System is used for the transmission of files, especially for the application form. Trans¬ 
mitted forms are stored in a document image system. The document image system has 
database functionality, represented by the database-stereotype. If necessary the more spe¬ 
cialized stereotypes central database or distributed database could be used. For further 
processing in the organization a workflow management system, that is based on the 


document image system for the storage of forms, is used. The basic functionality of a 
workflow management system, the coordination of the cooperation, suggests using the 
coordinated cooperation stereotype. 

As an example for adding attributes to types, the attribute confidentiality is noted to the 
application form. Attributes are usually invisible in the graphical model. To show the 
usage of rules the following example may be impressive. We specified the following 
modeling rule to ensure the social aspect authenticity of information: 

If the document’s confidentiality is high AND its violation of identity has to be considered 
then check the model for existence of building block “authenticity checking”. _ 

Fig. 4: Insurance Office Example - Document Imaging System with Authentication 

The more detailed Object System DocImageSystem with Authentication (see fig. 4) con¬ 
sists of a user interface for the office worker (Office Worker UI) and the underlying 
document image system (DocImageSystem) that realizes the persistence of the incoming 
forms. As well we can identify the required building block Authenticity Checking Serv~ 
ice. Its functionality has been described above. 

Comparative Evaluation 

We derive a set of if-then-rules referring to the literature in the field of ergonomics and 
technology assessment [e.g. Henmann, 94; VDI, 91]. These rules exploit the social as¬ 
pects to make comparative evaluations between different multi-media application con¬ 
cepts and to find out which of them are more advantageous if various social aspects are 
taken into account. In addition to building blocks (see fig. 2), we identified structures 
(see fig. 5) with social relevance by referring to the experience we gained by modeling 
multi-media applications. By the term structures we refer to phenomena which cannot be 
represented by single building blocks because they are distributed over several blocks or 
expressed by the relationships between them. 

Examples for socially relevant structures: 

• Choice of communication partner __ 


• Choice of communication means 

• Attainability of roles 

• Transparency to the participants _____ 

Fig. 5: Examples for socially relevant structures 

To each socio-oriented building block we added attributes (see fig. 7) which are relevant 

Examples for attributes: 

• personal data included: (yes, no) 

• sensibility: (low, average, high) 

these three values mark the intensity of possible negative consequences for individual 
and for business concerns which can be the result of the publication of data 

• confidentiality: (low, average, high) 

• universal: (yes, no) 

This attribute relates to the availability of data for all participants. _ 

for the socio-oriented evaluation and which are also used in the if-then-rules. 

Fig. 6: Additional attributes for social evaluation 

Our method allows a - mostly comparative - evaluation of multi-media applications re¬ 
garding different social values, such as: 

• data protection 

• security 

• privacy 

• authenticity of information 

• chances for participation 

• reduction of the context 

• transparency 

There are too many rules to completely present them in this article. To give an example, 
figure 7 shows a subset of the if-then-rules to evaluate multi-media-applications in de¬ 
pendency of “data-protection”. 


Examples of if-then-rules for the social aspect data-protection: 

2. if “personal = yes” has been noted at the building block 

then check the existence of following building blocks in the model: 

• making data anonymous 

• coding 

• negotiation 

• tracing 

• identification 

4, if “sensibility = high” has been noted at the building block 

then check the existence of following building blocks in the model 

• checking the rights 

• coding 

• tracing 

9. if the building block “Control of costs and time” is part of the model 
then check the existence of following building blocks in the model: 

• making data anonymous 

• checking the rights 

Fig. 7: Examples of evaluation rules for the aspects "data protection" 

To evaluate a multi-media concept we suggest constructing a list of weakpoints for the 
multi-media concepts being regarded. This list allows a comparison of differently con¬ 
structed concepts on the basis of measurable values. The list of weakpoints can be used 
to serve three different purposes: it gives hints to improve a concept of a multi-media 
application, it might contain weakpoints showing that the concept is clearly violating 
certain social norms, it might become obvious (in relationship to certain aspects) that 
one concept is more advantageous than another one. 

To construct a list of weakpoints, we suggest using the following procedure : 

(1) Choose the aspect for the evaluation (e.g. data protection). 

(2) Start with the first evaluation rule for the chosen aspect. 

(3) For each object and its relationship to others investigate every rule: if it is violated, a hint to this 
weakpoint is added to the list. 

(4) For any further rules proceed the same way as described in step (3) _ 

With this procedure we can construct a values list for every social aspect , which we 
chose to be part of the evaluation or comparison of the regarded multi-media concepts. 

The evaluation rules and the procedure are independent of the used modeling language, if 
the modeling language supports the concept of building blocks and refinement. 


We presented expandable methods for the socio-oriented comparing of multi-media ap¬ 
plication concepts. Therefore, we introduced building blocks to support modeling of 
these multi-media concepts and to indicate the socio-oriented aspects. The relevance of 


the proposed building set has been demonstrated in this paper by an example. 

The modeling notation is based on standard methodologies and is independent from the 
building set concept. The shown object-oriented modeling language is appropriate for 
component-based systems like multi-media systems. However, additional extensions are 
still needed to make social aspects more comprehensible. This is especially necessary if 
the social aspects are related to relationships between objects or to the overall structure of 
parts of the model. 

Also, the evaluation rules and the methods for the evaluation are independent from the 
used modeling notation. The method for the evaluation leads to a list of ratings support¬ 
ing a comparison of multi-media concepts and to hints for improving them. 


The experiences we made on multi-media applications and the gathered data about multi- 
media concepts in real life are based on the work of a two-semester student project-group 
(KonMedia) at the University of Dortmund, Thanks to the members of this project- 
group. Especially we would like to thank Niels Lepperhoff who supported our work with 
his experience in modeling multi-media applications. 



Booch, Grady, Riimbaugh, James (1996): Unified Method for Object-Oriented Devel¬ 
opment. Documentation Set, Version 0.8. Santa Clara, CA: Rational Software 

Dix, Alan; Finlay, Janet; Abowd, Gregory; Beale, Russell (1995); Mensch, Maschine, 
Methodik. New York et. al.: Prentice Hall, 1995, 

Herrmann, Thomas (1994/1): Grundsdtze ergonomischer Gestaltung von Groupware. In: 
Hartmann, Anja; Herrmann, Thomas; Rohde, Markus; Wulf, Volker (Hrsg.) 
(1994): Menschengerechte Groupware - Software-ergonomische Gestaltung und 
partizipative Umsetzung, Stuttgart: Teubner Verlag. S. 65-107. 

Herrmann, Thomas (1994/2): Loss of Situative Context and its Relevance for Computer 
Mediated Communication and Cooperation. In: Clement, A. et al. (Hrsg,): Net- 
WORKing: connecting workers in and between organizations. Amsterdam et al.: 
North HoUand: 1994. S. 87-96, 

Herrmann, Thomas; Hoffmann, Marcel; Loser, Kai-Uwe (1997): Modellierungsnota- 
tionen fiir prospektive, gestaltungsorientierte Technikfolgenforschung. In: H. 

Paul (Hrsg.) (1997): Modellierung von Aufbau- und Ablauforganisation: von der 
Technozentrik zur Anthropozentrik, Projektberichte des lAT (97/3): Gelsen¬ 

Jacobson, Ivar; Christerson Magnus; Jonsson, Patrik; Overgaard, Gunnar (1992): Object- 
Oriented Software Engineering, A Use Case Driven Appwnc/i,Wokingham et al: 
Addison Wesley, 1992. 

Jacobson, Ivar; Ericsson, Maria; Jacobson, Agneta (1995): The object advantage. Busi¬ 
ness Process Reengineering with Object Technology .Wokingham et al.: Addison 
Wesley, 1995. 

RoBnagel, Alexander (1991): Vom informationellen zum kommunikativen Selbstbestim- 
mungsrecht. In: Kubicek, Herbert: Telekommunikation und Gesellschaft (Bd. 1). 
Karlsruhe, 86 - 111. 

VDI Report 15 (1991): Technikbewertung- Begrijfe und Grundlagen. Diisseldorf: VDI. 


Virtual Environment for the Simulation of a 
Tactical Situation Display 

M. Grandt, Th. Alexander & K.-P. Gartner 
FGAN - Research Institute for Electronics and Mathematics 
Ergonomics and Information Systems 
Wachtberg-Werthhoven, Germany 


Virtual Environment (VE) describes a computer-mediated experience by which an 
operator perceives a simulated environment by means of special human-computer inter¬ 
face equipment. The human operator interacts with virtual objects in the environment as 
if they were real. Several operators are able to interact and co-operate in a shared virtual 
environment, such as a synthetic Tactical Situation Display. 

The Research Institute for Electronics and Mathematics (FFM) carries out a research pro¬ 
ject which has the goal of acquiring methods for thi*ee dimensional visualisation of tacti¬ 
cal situation data and interaction between the officer in command and the units 
visualised on the display system. 

For fulfilling the operational task several operator look at a three dimensional representa¬ 
tion of a landscape using VE-devices like shutter glasses, etc. . The ergonomic research 
work is to design and evaluate the interactive functions. The important subject is the in¬ 
vestigation of different control input devices for simultaneous interaction of operators at 
a common virtual Tactical Situation Display unit (TSD), An interaction task is e.g. to 
move and position graphic objects as well as tactical symbols. 

The first step to be done is the development of principles for the design of a human-ma¬ 
chine-interface which enables the user to handle tactical situation data in a semi-immer¬ 
sive way and to interact with his own units. 

The long-term purpose of the realised experimental facility of an Electronic Sandtable 
(ELSA) is to perform research work with respect to development, performance, evalua¬ 
tion, and optimisation of compliant terrain database, real-time communications, and net¬ 
working of heterogeneous "Command & ControF-systems computing graphic situation 
display information. 

This paper describes technical challenges and discusses the issue of feasibility of a pro¬ 
ject started at the Research Institute for Electronics and Mathematics (FFM). 


Virtual Environment (VE), also popular under the synonyms Virtual Reality (VR) or 
Cyberspace, describes a computer- mediated experience by which an operator perceives a 
simulated environment by means of special human-computer interface equipment. The 
human operator interacts with virtual objects of the environment as if they were real. VE 


is conditioned by two factors, which are immersion and interaction (du Pont, 1994). The 
effect of immersion, the subjective "being there"-feeling of the user is increased by most 
complete stimulation of the human senses. Interaction is realised by a variety of 
interaction techniques which guarantee an intuitive behaviour of the operator in case of 
interacting with virtual objects. 

VE has become popular during the last years due to the rapid development and enhance¬ 
ment of computers, especially in the field of workstations with high graphic power. 
Powerful tools for generating synthetic environments have been developed and sold, too, 
and it is only a question of time, until new approaches for the design of human-machine- 
interfaces (HMI) will have grown up. 

Promising ideas of an immersive interaction between human beings and technical 
systems have been created for the civilian world, e.g. for medical purposes, architecture, 
education, systems design, etc., and for the military world. But it is well known from the 
history of technical development, that the promises of new approaches and ideas are 
often very high and easily made. But after critically studying the techniques differences 
between promises on the one hand and technique’s benefits and risks on the other 

Especially for applications connected with high risks for the operator and the 
environment the consequences of using these new techniques have to be examined very 
closely. Since the consequences of failures in using or handling such systems are risky or 
perilous it has to be ensured that the interaction techniques support reliable and safe 
function of the systems. An important influence on the handling-safety is given by the 
early involvement of ergonomics in the system design process. By determination of the 
human machine interface’s requirements it is be guaranteed that the system fits the 
operator which is an important precondition for systems design and handling-safety. 


The current state of military commander*s workplaces 

Before projecting the idea of a virtual Tactical Situation Display (TSD) it should be ex¬ 
plained what kind of functions are performed by means of such a device. TSDs are lo¬ 
cated in operation centres of command posts. They are used to display the current situa¬ 
tion of own and reconnoitred enemy troops and facilities, e.g. positions of units, head¬ 
quarters, bridges, mine barriers, etc, in the operation area to the commander of a unit, 
e.g. a commanding officer of a tank brigade. 

The quantity of displayed situation information depends upon the availability of com¬ 
mand and control. Especially communication systems like radio telephony which are 
used to receive and send situation data at present have high importance. Due to the 
danger of becoming reconnoitred by the enemy’s communications intelligence there are 
restrictions on the use of communication systems. Furthermore communication links are 
attentionally jammed by the enemy so that the transmission quality is limited. 

Moreover the quality of displayed situation data depends on the accuracy of the own 
troop’s reports about the own situation and the observations made by the soldiers or by 
technical reconnaissance systems. 

Moreover the TSD is used for tactical planning of intended future operations. Quantity 
and quality of situation data are essential for an adequate operation planning. 

At present the paper & pencil technique is used to display these pieces of information. 
Situation information data are manually noted on the Tactical Situation Display. There is 
a substantial set of symbology and colours used for the different kinds of information, 
e.g. for distinction between own and enemy troops, for kinds of vehicles (battle tank, 
infantry fighting vehicle, etc.), for kinds and states of barriers (prepared, permanent, 
time-limited) and so forth. Due to this technique it is a time-consuming work to hold this 
display up to date while receiving actual information from other units. Furthermore the 
transmission of orders for operations by means of radio telephony takes a lot of time. 
Due to these facts a situation of high information flow and high workload of the 
personnel requires the whole operator’s capacity and might even lead to states of 
overload. Figure 2 gives an impression of the situation in today’s operation centres. 

It is obvious that the form of 

• displaying situation information data at the commander’s workplace by 
means of paper & pencil TSD and 

• communication mainly made by radio telephony 
have not significantly changed for at least 50 years. 


Figure 2: Today’s army control centres 

Technical developments of C*I-Systems 

The technical environment has developed in recent years and will be improved in future 
with high speed. This development has taken and will take place in several fields which 
are usually joined within the expression ^-(Command, Control, Communications, 
Computers and Intelligence) systems: 

• Command/Control: Data hnks between operation centres attached to networks 
will improve amount and availability of situation information. Furthermore 
these means will reduce the necessity to use radio telephony for 

• Communication: Radio telephony by means of digital transmission reduces 
the vulnerability of jamming and spying by the enemy. 

• Computer: The use of computers for military applications will rise due to low 
costs and necessity to process huge amounts of data. 

• Intelligence: New sensors, e.g. FLIR (Forward Looking InlraRed), improve 
the quality and quantity of observations. 

These developments will influence two factors of the workplace at an operation centre: 

• increasing amount of available data, 

• non-verbal data reception via computer networks. 

Considering the information flow caused by technical developments it can be estimated 
that current techniques for handling and interpretation of tactical situation data will be¬ 
come insufficient within the next future. 

For this reason computer-based command and control information systems (CCIS) have 
been developed for the services (army, navy, and airforce). In the following CCIS 
developments for the German army will be discussed. The arguments are likely to be 
valid for the other services as well. 


Examples for CCIS-developments within the German army are the CCISs HER OS 
(Heeres-Fuhrungsinformationssystem fur rechnergestUtzte Operationsflihrung in Staben) 
for the army command, and ADLER (Artillerie Daten-, Lage- und Einsatz- 
Rechnerverbund) for the artillery. The current state of CCIS development has its focus on 
data transmission between units. It is planned and partly realised to connect the army’s 
units via networks so situation information data and orders can be exchanged in non¬ 
verbal manner. In order to make these systems "open" for communication with other 
services of the German Armed Forces, NATO and WEU partner nations protocols and 
interfaces have been standardised. Figure 3 illustrates the structure of such networks. 

Figure 3: CCIS-scenario (reprint with permission of SIEMENS AG - Sicherungstechnik) 

The hardware of the CCISs human-machine-interface has mostly been realized in a con¬ 
ventional way like civilian computer workplaces using commercial of the shelf (COTS) 
products, i.e. computer displays, keyboards, and additional input devices like mouse or 

Present CCISs visualise the situation information data in a two-dimensional way. The 
symbology is also similar to the one used with the paper & pencil TSDs. Tools like a 
march-planner or the display of lines of sight allow the user to take into account terrain 
shape, cruising speed, etc., when planning a march or checking whether an object is 
visible to another one. The march-planner is important in rough terrain, display of lines 
of sight in order to determine acceptable positions of own troops for intended operations. 
Current CCIS techniques bring along a significant improvement of the situation at the 
commanders’s workplace. 

Nevertheless it is obvious that the development will go on in future. This shall be 
explained by two examples: One disadvantage of two-dimensional representations of 
three-dimensional data is the necessarity of creating a mental image of the terrain. 
Although assisting tools are implemented it can be assumed that the availability of three- 
dimensional Geo-Information-Systems (GIS) with terrain databases like DTED or DHM, 
and feature vector-databases like DFAD, DCW, VMAP and others means improvement. 
Other improvements can be made by a perspective view from any location of the 
scenario into any direction. The operator would be able to see the terrain and objects in 
the same way a tank commander would do, which might become an important function 
in some situations. 


An important ergonomic requirement of human-machine-interface (HMI) design is to 
allow an intuitive interaction between the operator and the objects on the display. In pre¬ 
sent CCIS the interaction with objects of the scenario is not included in a satisfactory 
way. Instead the operator has to follow various procedures to manipulate objects. 
Although most of the systems are menu-driven, the procedures take an important amount 
of time and training for learning to use the systems. 

These two circumstances are only examples where lacks of the present systems appear 
and improvements can be achieved by applying new display and interaction techniques. 

Existing visions of future C"*! -Systems describe a combination of new sensors, 
communication lines and display techniques. The Swedish ’’Mobile Joint Command and 
Control System 2010" (ROLF), a vision of a future mobile Command & Control - 
System, will use a device called AQUARIUM as TSD. The AQUARIUM consists of a 
large semi-inunersive workbench (which will be described later) to enable discussion and 
co-operation between different operators. The underlying concept of ROLF is variable. It 
is principally military, but it can be used for civil applications, e.g. supporting the society 
under severe strain and stress in case of catastophy, too. (Sundin, 1996) 

The US-American system of the Naval Research Laboratory (NRL) or the system 
MIRAGE of the Institute for Simulation and Training (1ST) use exclusively workbenches 
as TSDs (NRL, 1997; 1ST, 1997). The systems were already tested in military exercises 
and found very useful and advantageous compared to conventional TSDs. Especially the 
possibility of displaying "natural features" of three-dimensional terrain in combination 
with abstract information data (visibility, range of weapon systems, etc.) was found to be 
very useful (Wilson, 1997). Another important result was that there is special need for 
ergonomic user interfaces and training to use the new technology and its benefits. 

Virtual Environment Systems - A brief overview 

In recent years there have been a lot of approaches and ideas for future displays and 
interaction techniques in the field of Virtual Environments. The devices available on the 
market are manifold. A good structure of VE-systems is given by use of the degree of 
immersion, which means "to what extent a user is physically tied in a virtual 
environment" (Bullinger, 1997). 

Simple VE-systems have nearly no immersive effect. Such systems use low-cost desktop- 
computers and usual monitor-displays. The virtual environment is presented only in two- 
dimensions. Real three-dimensional navigation or interaction is very limited. An example 
of these systems are commercial-of-the-shelf software (games, etc.) or VRML-browser 
on conventional PCs. 

Advanced (semi-immersive) systems create a higher degree of immersion. A monitor or 
projection-plane serves as display and by use of shutter glasses or other techniques three 
dimensional visualisation is achieved. Interaction devices with 6 degrees-of-freedom 
(DOF) are often used. Examples for semi-immersive systems are graphic high-end 
workstations with monitors, projectors or with a virtual workbench. AU these systems 
make the virtual environment and objects become a part of our real world (Bullinger, 
1997; Kruger & Frohlich, 1995). 


High-end (full-immersive) systems provoke total visual immersion of the user into the 
virtual environment. These systems create a virtual world and let the human operator 
participate in it. Navigation and interaction is possible in three dimensions and three 
directions. Examples for such systems are the head mounted displays (HMD), special 
1/0-devices or the CAVE (Cruz-Neira et al., 1993). 

Because of the rapid evolution and the variety of VE-systems and devices it does not 
seem to be useful to put in a summary at this point. To get an impression about the de¬ 
vices available at the market it is recommended to refer to further articles (Bullinger, 
1997; Kalawsky, 1993; Grandt & Gartner, 1997). 

The Idea of a Tactical Situation Display using VE-Technology 
Decision making is strongly influenced by the information available and perceivable. For 
this reason the human factor is the most important part to define the needs and goals of 
an advanced TSD, 

TSDs should include a variety of diverse information, which starts at terrain information 
and position of units and ends at social and political information of the area of interest. 
The information available should be as complete and actual as possible. These goals are 
technical and with ongoing improvement of sensors, communications and information- 
technology it is only a question of time until availability of information is sufficient. 

Because of the visualisation of huge amounts of information data the use of virtual en¬ 
vironment display techniques has been discussed and found very promising. The real-life 
appearance of the information is supposed to create a high situation awareness, which 
will make decision making easier for the human operator. Moreover VE has been found 
very useful for visualisation of huge amounts of complex data in several other research 

Additional benefits for using VE-technology as TSD is the possibility of including as¬ 
sisting functions for preliminary decision making and assessing the effects of an intended 
operation. Such functions might be the visualisation of the range of weapon systems, 
visualisation of the range of units for transportation, information about the current 
airspace situation above a terrain, lines of sight, visibility and others. Such functions 
would make the TSD an assisting tool for tactical situation assessment. 

Moreover co-operation is very important at operation centres. A battle commander 
should have the possibility to discuss his decisions and alternative ideas with his staff. 
Although there are some thoughts about "virtual staff centres", yet real co-operation 
cannot take place in full-immersive Virtual Environments due to unknown co-operation 
techniques and social problems. Instead the virtual world should become a part of the 
real world where staff meets as real persons, not as avatars or virtual agents. 

For this reason it was decided to use a virtual workbench. The workbench looks like a 
conventional cardtable and by using shutter glasses it is possible to create three- 
dimensional images on its horizontal projection plane. This way virtual objects become a 
part of the real world. Especially co-operation and interaction with the graphic objects is 
more natural than it would be by using full-immersive systems. 


The Electronic Sandtable (ELSA) - Facility at the FFM 

The Electronic Sandtable (ELSA) at the Research Institute for Electronics and Mathe¬ 
matics (FFM) has been designed as an advanced experimental three-dimensional display 
system for tactical information. It is planned to equip the system with additional 
functions to extend the passive display system to an assisting system for staff members in 
command centres (Alexander et al, 1997). 

To get a comparison to conventional TSDs the ELSA-project has been designed 
according to the following demands: 

• display of tactical information of existing CCIS, 

• use of standardised tactical symbols according to ZDV 1/11, 

• use of military geographic data (AMilGeo, 1996), 

• reference to the usual scale of military paper-cards. 

As it can be seen in figure 4 the structure of the display-system can be divided into two 
main parts. 

Part 1 works offline and creates the static scene-database to be displayed. Geographic 
data (terrain, feature and satellite data) and detailed objects (CAD-generated objects like 
houses, tanks, cars) are merged and converted into polygon data. At the following step 
tactical information of a CCIS is included giving information about number, kind, and 
starting position of military units at the field of interest. This information is linked to 
dynamic objects and included in the final scene-database for visualisation. 

The second part works online in real-time. Here the scene-database is displayed on a vir¬ 
tual workbench and interaction with the database is possible. For post-brief all the 
actions are written into a protocol file. As the dynamic objects are linked to information 
from the CCIS their position may change and the display has to be updated. 


Figure 4: Brief structure of the Electronic Sandtable (ELSA) 

The hardware of the systems consists of a graphic computer (Onyx2 Infinite Rality 2 of 
Silicon Graphics Inc.) and a virtual workbench with shutter glasses. The scene-database 
is created using EasyT-Software by Coryphaeus Software Inc. and DIMS (Digital Land- 
Mass System) data (AMilGeo, 1996). The visualisation software uses IRIS Performer 
graphics library (Silicon Graphics Inc.). With multichannel-option one channel is used 
for the display on a supervisor’s monitor and the second channel drives the workbench of 
the Electronic Sandtable. 

The virtual workbench is shown in figure 5. It is based on the ideas and studies of Kruger 
(Kruger & Frdhlich, 1995). A projector puts different frames for the left and the right eye 
time-multiplexed via a mirror onto the projection plane. The operators wear shutter 
glasses, which are synchronized with the projector. Synchronization works by using IR- 
emitters and IR-sensors on the glasses. An interface for connectiong different VO-devices 
to the computer system exists. 

The view geometry is calculated for a single operator. His eye-position is assumed fixed 
at the moment and perspective error grows with distance to this position. An 
achievement is expected by tracking the eye position and updating the displayed 
geometry if the position changes. 

Topics of Research 

Since applicability and limitations of VE-techniques in connection with CCIS are mostly 
unknown, there are questions arising about the human operator using VE-interaction de¬ 
vices and VE-TDS. 


Figure 5:Experimental facility of the Electronic Sandtable (ELSA) at 


Generally spoken, interaction devices have to fulfil several ergonomic requirements con¬ 
nected with the human operator’s abilities and limitations. These are: 

• Aspects of the human perception with different human senses. Especially if 
information is displayed in a three-dimensional way attention should be paid 
on depth perception of the visual sense; 

• Aspects of the usability of devices (e.g. weight, adaptation to all kinds of pro¬ 
tective clothes) in order to achieve a comfortable use for long duration. 

Software-ergonomic requirements have to be determined in order to assure high situation 
awareness, acceptable operator workload, and sufficient handling safety and reliability 
while taking advantage of a three-dimensional representation of the scenario. Questions 
on these deal with 

• adequate information presentation, 

• intuitive manipulation of objects embedded in the virtual scene, 

• ability for the operator to build up a mental model of the system’s 

• implementation of error-tolerant behaviour of the system, 

• implementation of user support. 

Applying VE-techniques to a TSD brings along a lot of questions. VE-techniques are 
defined as an "intuitive perceivable and experienceable scene of a natural or abstract 
environment" (Bullinger et al, 1997), The software-ergonomic requirements are still 
valid, but a simple adoption of the results for conventional two-dimensional computer- 


interaction will lead to the loss of nearly all benefits of VE-systems This means that new 
ways of visualisation and interaction have to be evaluated, tested and optimised. 

The main focus of research lies in the following areas: 

Visualisation: Human visual perception plays the important role for defining require¬ 
ments of future visual displays. Monocular and binocular depth cues like perspective, 
shadowing or stereoscopic parallax have different effects on individual’s depth 
perception. Studies about quantification of the effects of modelled depth cues in virtual 
environment scenarios with a large scale is not known. For visualisation of geographic 
data a simple model of the view geometry is not enough because of large scales. Instead 
view geometry has to be modified in order to create a better depth effect. This will be 
made by using un-real eye-to-eye distances. This procedure is also used for 
photogrametric or reconnaissance purposes. 

Interaction: A main goal of VE-Systems is the natural representation of virtual objects. 
This leads to natural interaction with virtual objects. Interaction in VE-systems means 
both: navigation in virtual worlds and manipulation of virtual objects. Procedures of 
navigation in virtual environments and databases are evaluated and hardware devices are 
tested with regard to their practical use. Because view geometry of the Electronic Sand- 
table is "god’s eye" from direct above, navigation is roughly two-dimensional (horizontal 
and vertical scrolling). This can be controlled by conventional two-dimensional I/O- 
devices. Manipulation of virtual objects (picking, moving, etc.) requires interaction with 
more degrees of freedom (normally 6 DOF). Because of this 6-DOF interaction devices 
like the space mouse or virtual pointers will be used. 

Co-operation: A multi-user VE-TSD offers the possibility of co-operative work and 
virtual discussions at military staffs. From the technical point of view multi-user systems 
bring along new problems. For example, view geometry has to be calculated for more 
than one eyepoint, which reduces the framerate of time-multiplexed display systems. 
Multi-user interaction techniques have only roughly been developed and simple problems 
like giving an object to another user brings along a lot of problems for software 
developers. Another question is whether all users should have the same interaction 
abilities. Is it better if each user has the same rights to manipulate objects or should there 
be a hierarchy of manipulation rights? These topics are of major interest for TSDs as 
there is a military hierarchy in reality which should have effects on virtual command 

Distribution: Yet co-operation is only possible if staff members meet at the same place. 
By use of distributed simulation this might change. Real persons will meet in a virtual 
environment as representations (so called avatars) and have their staff meeting virtually. 
This future vision sounds veiy promising, but do remember that co-operation, discussion 
and brainstorming meetings are events in the real world. There are sociologic, 
psychologic and individual factors involved in real world which cannot be simulated. 
Because of this the benefits of distributed TSDs have to be quantified and examined very 


Virtual Environment technology brings along a lot of new ideas for military TSDs. 


Especially the natural way of presenting complex data, the possibility of including assist¬ 
ing functions and the high actuality when connected with electronic CCIS are huge 
benefits for the use of such systems. 

Nevertheless the human factors have to become the main points of interest to define 
needs and requirements for an integration of VE-systems. If this does not happen, the 
benefits wiU change into disadvantages and human operators will not be assisted but 
confused by visualisation of the information data flow. 


Alexander, T., Wappenschmidt, A. & Gartner, K.-P. (1997): Ein Verfahren zur Dar- 
stellung von Geodaten in Echtzeit, In: R. Moller (ed.): Proceedings of the 5th 
Workshop: Sichtsysteme - Visualisierung in der Simulationstechnik, Produserv 
Springer Produktions-Gesellschaft, Berlin (to be published), 

AMilGeo (1996): DMG-Katalog: „MilGeo-Daten'\ Deutscher militargeographischer 
Dienst, Amt fUr militarisches Geowesen, Euskirchen. 

Bullinger, H.-J., Brauer, W. & Braun, M. (1997): Virtual Environments. In: Salvendy: 

Human Factors and Ergonomics, pp, 1725-1759, John Wiley & Sons, New York, 

Crux-Neira, C., Sandin, D. & DeFanti, T. (1993): Surround-screen Projection-based 
Virtual Reality: The Design and Implementation of the CAVE, Computer 
Graphics (SIGGRAPH '93 Proceedings), pp. 135-142. 

Grandt, M. & Gartner, K.-P, (1997): Bewertungsgrundlagen fur Simulationsverfahren - 
Simulationsverfahren „ Virtuelle Realitdt“ zur Erzeugung realistischer Stimuli, 
Zwischenbericht. Wachtberg: Forschungsinstitut fUr Funk und Mathematik der 
FGAN e.V.. VS-NfD. 

Institute for Simulation and Training (1ST) (1997): Mirage - A New Way o View the Vir¬ 
tual World. Internet: 

Kalawsky, R.S. (1993): The Science of Virtual Reality and Virtual Environments. 
Wokingham: Addision-Wesley, 

Kruger, W, & Frohlich, B. (1995): The Responsive Workbench (virtual work environ¬ 
ment), IEEE Computer Graphics and Applications, VoL 14 (3), pp. 12-15. 

Naval Research Laboratory (NRL) (1997): „Virtual Reality Responsive Workbench for 
Sea Dragon'". Internet: 

du Pont, P. (1994): Applied Virtual Reality. In: H.-J. Wamecke & H,-J. Bullinger (eds.): 
Virtual Reality *94 - Anwendungen & Trends. Berlin: Springer-Verlag. 

Silicon Graphics: IRIS Performer Programmers Guide, Silicon Graphics Inc. 

Sundin, C. (1996): ROLF - Mobile Joint C2 System for the year 2010. A vision under 
development and tesf. Proceedings of the 1996 Conunand and Control Research 
and Technology Symposium, pp. 176-187, Washington, D.C.: ACTIS. 

Wilson, J. R. (1997): Military Eyes new Fakespace virtual environment. Military aero¬ 
space & electronics 1/97, Penn Well Publishing Company. 


Multimedia Approaches in Management Information Systems 

Jurgen Raster 

Research Establishment of Applied Sciences 
Research Institute for Electronics and Mathematics 

Wachtberg, Germany 


Modem multimedia decision support systems are characterized by continuously chang¬ 
ing conditions regarding technology, task and user profiles. As a consequence of this 
heterogeneity a huge amount of information items of different data types has to be man¬ 
aged and processed. From a human factors point of view there is a need to bridge the gap 
between general ergonomic design principles (e.g., suitability for the problem, flexibility, 
controllability) and concrete implementations. Solutions have to be found to provide 
homogeneous user interfaces with consistent access and evaluation capabilities for dif¬ 
ferent data types. Another problem is the dynamic aspect of information flow. One has to 
take into consideration that - especially in hierarchical management information systems 
(MIS) - information items may change their data type during processing steps. E.g„ a 
broad range of textual input details may be aggregated and transformed to other presen¬ 
tation categories to support decision processes at a higher management level. To achieve 
those goals integrated information systems require problem solutions depending on 
document-oriented working styles instead of traditional program-oriented solutions. 

In client/server environments the data management concept should be developed inde¬ 
pendently from data types used in the operational system. It can be stated that good, 
clear process models are the key to design the core of an information system. Efficient 
CASE technology allows for breaking down the major processes into lower-level proc¬ 
esses step by step. During this top-down process software generators may be used to cre¬ 
ate applications and interfaces with respect to ergonomic styleguides to ensure consistent 
interaction techniques even in multimedia applications. Another approach to meet the 
ergonomic requirements in complex MIS is to make use of modem object-oriented con¬ 
cepts (e.g„ COM, CORBA). This takes into account that nowadays the ability to inte¬ 
grate different generic software components is of greater importance than developing 
specific applications. Several basic principles to achieve this integration will be ad¬ 
dressed in this paper. 


Our research efforts are directed towards the development and application of human en¬ 
gineering methods and criteria to the design and evaluation of complex man-machine 
systems in all phases of system development and acquisition. Multimedia communica¬ 
tion is ubiquitous in those domains. When humans converse one another, they utilize a 
wide range of media to interact. Some media of communication are more effective than 
others for certain situations, users or contexts. A multimedia document which includes 
text, tables, graphics, speech, video, may effect several human senses to process the in¬ 
coming information. This interaction occurs over time. Therefore, it is necessaiy to ac¬ 
count for the processing of discourse, context shifts, and changes over time. Whereas 
humans have a natural facility for managing and exploiting multiple input and output 


media, computers do not. Human abilities should be amplified by using computers, and 
the synergistic utilization of multiple media can support this amplification. If appropriate 
media are utihzed for human computer interaction, there is a potential to increase the 
bandwidth of information flow between human and machine. To achieve this goal re¬ 
quires a deep understanding of information characteristics, how they relate to character¬ 
istics of media, and how they relate to models of taisks, users, and environments. Under¬ 
standing these principles will not only result in better interactive devices, but also leads 
to new tools for context-sensitive multimedia interfaces, and intelligent agents for multi- 
media information retrieval, processing, presentation, and authoring. 

Two approaches dealing with multimedia information processing are covered in this pa¬ 
per. First, it is illustrated how actual tendencies in software development technologies 
can be utilized to design user- and task-oriented multimedia systems which allow for 
document-oriented working styles. Second, it is shown how integrated CASE-technology 
can be applied to meet general ergonomic requirements in complex database manage¬ 
ment systems. 

A modular workstation 

Modem workstations designed for complex information systems must support a wide 
range of functionalities. In our application domain these are military situation display, 
geographic information processing as well as text processing, spread sheet applications 
and business graphics. The incorporation of video and audio will be standard in the near 
future to realize multimedia presentations for situation reports. Those workstations are 
embedded in heterogeneous communication networks where the combined use of com¬ 
mercial, government, and military software products (CGM-software) is ubiquitous. 
Workstation design means to use and integrate specific characteristics of different prod¬ 
ucts which can be considered as independent „components“. In such a modular environ¬ 
ment state of the art is a program-oriented workstyle. This means that operators use dif¬ 
ferent tools for different tasks. The aim is - however - to construct homogeneous user 
interfaces based on complex documents which consist of different building blocks proc¬ 
essed by distinct tools. 

situation / ma p 
spread sheet 


business graphics ^^ 
text proems in g 


A modular workstation for command and control information systems 



In order to use efficient tools for the management of multimedia data the before men¬ 
tioned „components“ should not be responsible for data storage and organization. Ac¬ 
cording to the , 4 nodel“View-controller“-concept (MVC) known from Smalltalk, informa¬ 
tion container should be realized in separate databases. This separation of multimedia 
database (model) and user front ends (views) gives the chance to implement powerful 
algorithms for data evaluation and aggregation across different data types. These algo¬ 
rithms can be implemented independently from specific user interfaces and can be 
adapted to actual task requirements. On the other hand ,4ntelligent“ agents can be devel¬ 
oped to interpret user needs and control the information flow between database and user 
front end(s). E.g., a situation display can present data on a geographical map as well as in 
a textual form or in a tree structure if appropriate. While the model represents the actual 
situation in a formal description, different views can be applied to present the data in a 
task-oriented way. 

System architecture with separate modules: (1) views, (2) model, (3) controller 

Compound document 

The mentioned MVC-concept gives the freedom to implement different accesses to the 
various data sources. The aim is to design a task-oriented man-machine-interface without 
overloading the operator with a huge amount of functions of different programs, A solu¬ 
tion is the integration of objects in a single document to ensure that the user can concen¬ 
trate on his work. Object Linking and Embedding (OLE) is a method used by Micro¬ 
soft’s Windows products to integrate the output of one program (server task) as data into 
another (client task) - for example, a drawing into a word processing document. The ap¬ 
propriate server program is automatically run to edit the data when the OLE-object is 
activated. The newer version 2.0 of OLE supports in-place-editing, so that rather than 
opening or moving to a new window, it lets you edit the embedded object without hiding 
the rest of the document. Clicking starts the server application and changes only menus 
and toolbars. Competing approaches like Apple’s OpenDoc and IBM’s System Object 
Model SOM work in a similar manner. The difference between Object Linking and Em¬ 
bedding and conventional copy-and-paste-strategies is that with OLE, the data may be 
edited within the client document. Documents that contain links to the output from other 


applications are called ^compound documents“. The underlying „Componentware“- 
technique can be applied to configure the user interface according to actual situation 
specific needs. The figure shows such a compound document consisting of portions like 
map display, table, picture, graphics and embedded video sequence. 

Compound document as integrated interface to dijferent tools 

OLE allows to initiate a program-to-program-connection manually. Furthermore is it 
possible to create automatic connection between application by using, e.g., OLE custom 
controls (OCX). The link from one application to another is installed permanently and 
can be activated whenever the user wants to do this. These automatic links may be used 
to combine the features of different programs in a given context. E.g., a display system 
may be applied to present a situation in front of a geographic map. Using OCX- 
techniques an agent may be instructed to read additional background information from a 
database and present this text in word processing program. Other techniques called „vis¬ 
ual programming“ may be used to configure a graphical user interface independently 
from data processing components. Those components can be linked dynamically to inter¬ 
face commands such as menus items, toolbars and buttons to invoke the required func¬ 

Multimedia database applications 

In modem client/server environments the mass of information is stored in central ar¬ 
chives. CASE technology (computer aided systems engineering) can support the process 
of generating large databases and complex applications based on multimedia data sets. 
Graphical design tools assist developers defining the application domain by process and 
system models. The description is hold in a central repository. Automatic generators cre¬ 
ate basic prototypes with consistent look and feel as well as consistent behavior of the 
application system. The automatic generation process can be controlled by supporting 
components such as libraries, preferences, master forms, and templates. This can be used 
to direct the system generation in a way that ergonomic know-how is implicitly incorpo¬ 
rated in the design process. Libraries define the functionality (procedures) for different 
user interfaces in a consistent manner. Preferences specify general adjustments about the 
layout of applications, like window behavior. Templates hold exact information about 

common parts of the layout, color settings, etc. Master forms are used in an object- 
oriented way to develop application specific forms. 

Because routine work is performed by CASE tools, the system developer can concentrate 
on the design of the interactive user interface. According to the MVC-concept this inter¬ 
face is not directly influenced by the central data management system. Therefore it is 
possible to develop different interactive access components based on the same data 
model. The figure gives an example to illustrate the development process from „user re- 
quirements“ to , 4 nultimedia applications^ based on CASE technology: 

user requirements are analyzed and modeled in a formal language 
process and system model are managed in the central repository 
additional components control the system generation process in a predefined way 
interface prototypes are optimized to ensure an optimized presentation of data 
(e.g,, tabular views to display data records, graphic views to show maps, 

CAS E-technology used to bridge the gap between 
general ergonomic requirements and actual systems design 


Flexible and intelligent multimedia interfaces promise to enable systems and people to 
use media to their best advantage. They can increase the bandwidth of information flow 
between human and machine. New technologies can help to solve the structural problems 
coming along with the combined use of enriched media. They can also help that human 
abilities are amplified, not impeded, by using computers. The introduction of a docu¬ 
ment-oriented working style helps that users can concentrate on their goals. Controlling 
the software generation process enables new facilities to incorporate ergonomic know¬ 
how into the design procedure. 


Simulation-Supported Analysis and Development of 
Tactical and Lower Level Ground Battle Units 

Capt. M.Sc. Arne Worm 
Department of Operations 
National Defense College, Stockholm, Sweden 

Division of Industrial Ergonomics 
Department of Mechanical Engineering 
Linkoping University, Linkoping, Sweden 


Modem warfare requires a comprehensive understanding of effective decision making, 
clear-cut command and control procedures, and high-capacity communication systems. 
At the tactical and lower unit levels this understanding is decisive, because lack of 
insight in these issues can cause severe consequences for the intended outcome of future 
operations, and ultimately, unnecessary loss of life. 

In this work, we developed a framework for analysis, modeling, and evaluation of the 
combat skills of low-level ground forces, commander combat resource management, and 
overall unit performance. This framework originated from various sciences: control 
theory, quahty control, certain behavioral and human factors areas, operations research, 
and statistics. The integration of these well- proven research areas, together with 
advanced high-performance ground combat simulation, into a novel concept of Combat 
Efficiency Analysis (CEA), facihtated simulation-supported, unambiguous and 
comprehensive military unit evaluation and assessment. 

Two full-scale studies of mechanized infantry units against a realistic adversary were 
carried out. The subsequent CEA provided a powerful aid for understanding combat 
essentials and for requirements generation based upon hard facts. The described analysis 
and modeling approach made it possible to identify various limiting factors for combat 
success of ground forces. 


Unclassified research, regarding the specific kinds of skills and properties that is needed 
for optimum performance at the battalion, company and platoon levels of modem tactical 
forces, has been rarely published. The striking properties of such forces can be 
characterized in brief as constantly increasing mobility, lethality and level of protection. 
Corresponding enhancements are made concurrently in tactics and intelligence. These 
dynamic properties raise a demand for novel solutions to several of the eternal 
warfighting problems: Individual and team situation awareness and decision making, 
commander and soldier combat resource management, and combat endurance of the 
units and systems engaged in the mission. Determined and forward exploitation and 
control of these belligerent system dynamics are vital for success. Soldier/operator and 
commander skills in managing and mastering these dynamics have decisive impact on all 
decisions and selections of action, the battle course of events, logistics, the number of 
casualties, and many other vital components of war or severe crisis. 


Worm (1996) developed a set of methods and tools for modeling, analysis, and 
evaluation of ground forces and their abilities at the battalion and lower levels: Combat 
skills, commander combat resource management, and overall unit performance. The 
central point of this project was the integration of a number of methods and tools, used 
for many years in trade and industry as well as in military systems development, into a 
multi-discipline mission and unit evaluation and assessment technique. To facilitate this 
integration, a set of concepts were introduced in order to analyze and evaluate the 
accomplishments and shortcomings of various military units in an unambiguous and 
comprehensive way. The concepts were: 

• Control theory-based conceptual modeling of dynamic and complex combat 
systems and processes, and of their states and state changes. 

• Identification of mission and unit state variables, and of different action and 
decision making mechanisms as a combat process regulator. 

• Within-mission Combat Efficiency Analysis of fully manned and equipped 
company units against a realistic opposing force. 

The rest of this paper is set out as follows, hi the next four sections we describe the 
development of a dynamic system model of tactical missions. The following section 
elaborate on mission and unit state variables identification, and on regulating combat 
processes. The last sections describe the Combat Efficiency Analysis technique applied 
in a case study. Conclusions and an outline of future work concludes the paper, 

A Dynamic System Model of Tactical Missions 

By system is meant, in control theory, an object, driven by external input signals u(t) for 
every t and as a response produces a set of output signals y(t) for every t. From the work 
of Ashby (1956), Brehmer (1992), and Ljung and Glad (1995), it is well known that most 
complex systems have real-time, dynamic properties, i.e. the value of the output at a 
given time is not only dependent of the input value at this specific time, but also on 
earlier input values. The difficulties of maintaining situation awareness in such dynamic 
environments cause specific problems. Endsley (1995) developed a comprehensive 
theory of individual operator, commander, and team situation awareness in dynamic 
environments. In this work, an extended system terminology was used. In the domain of 
modem and future military combat, and military command and control. Worm (1996) 
defined three main system components: 

1. Technological Systems, e.g. weapons, vehicles, communication-, reconnaissance-, 
decision support-, or command support systems, along with system operators. 

2. Command and Control Systems, i.e. information structures, command structures, 
and decision stmctures, built up by technological systems, intelligence acquisition 
systems and directly involved decision makers, arranged in command hierarchies and 
chains of command. A few examples of command and control systems are: 

• Tactical Military Command and Control Systems 

• Fire Support Command and Control Systems 

• Emergency Management Command and Control Systems 

• Logistic Command and Control Systems 

• Transportation Command and Control Systems 


3. Units, i.e. organizational aggregates, whose structures consistof one or several 
technological systems, soldiers/operators, commanders, command and control 
systems, staff functions, support functions, and other special services. 

The Mission Control Problem: Definitions and Properties 

According to Ljung and Glad (1995), modem control theory formulated the control 
problem as follows: 

”Given a system (S), with available measurement (y), determine the controller output 
(u), so that a control signal (z) follows a reference(r), despite the influence of external 
disturbances (w), measurement errors (n), and system variability, while keeping 
controller output values within reasonable limits. ” 

This definition is illustrated in figure 1. 

The Basic Concepts of a Generic Combat Mission Model 
The System is the own military and/or civilian unit, defined above. 

The Controller Output to the system consists of a flow of information. The information 
components are orders, intelligence, and other control components, transmitted via 
available communication systems. 

The System Reference is constituted by the missions, objectives, goals, and the required 
system status of the unit. 

The System Process is in this case the battle which is strongly dynamic and non-linear, 
i.e. changes in the process can give non-proportional, unpredictable or even chaotic 

The Combat Environment (i.e. the enemy, his systems, the terrain, and other battlefield 
components), and its impact on the own unit and the combat conditions, are treated as 


The Controller compensates for the disturbances. The controller function consists of the 
planning, decision making, orders, and actions of commanders and soldiers within the 
own unit. 

Measurement are all values that can be registered, measured, and processed. There are 
also Measurement Error that must be handled. The Measurement Error consists of a 
number of components, of which two of the most important was identified this far 
(Worm, 1996). Those are Incorrect or Insufficient Situation Awareness^ and Insufficient 
or Misleading Intelligence. 

The System Output is the Influence on Battle Course of Events^ and Influence on Enemy 
Decisions, Actions, and Planning. 

Feedback: If you use the system output to determine the system input (closed-loop 
control, feedback control), there is only a limited need for knowledge on system 
dynamics. You can then continuously make necessary adjustments by measuring the 
deviation of the system output from the reference value. 

The Combat Mission Model 

Combat missions can be described as clusters of processes, arranged in a dynamic system 
model aggregate, depicted in figure 2, 

Figure 2: A simple combat model 

Description and Determination of Mission and Unit States 

With the System State at a given time is meant a set of information, e.g. variable values 
that makes it possible to determine future system output if future system input is known. 

Before the battle, the unit has an initial unit state that is continuously affected by all 
actions and events occurring in the combat environment. Correspondingly, the mission 
has an initial mission state that is influenced by all maneuvers and outcomes of the 


coming battle. When the activity, function, status or mission objective of the unit 
changes, due to any cause, a state change takes place. Initial unit and mission states are 
primarily determined by the following factors: 

1. The status of soldiers and operators concerning: 

• individual and team skills 

• combat experience 

• physical capacity and injuries 

• psychological resistance to trauma and stress 

• psychosocial capacity 

• motivation 

2. Commander’s support and improvement of unit’s status concerning: 

• confidence in subordinate commanders 

• fostering a good unit spirit 

• practicing and encouraging good leadership 

• good personal judgment 

• caring for subordinated personnel 

• evaluating experienced combat situations together with all personnel involved 

• assigning appropriate mission tasks 

• continuous training and adaptation of mission tactics and procedures 

• well-functioning maintenance, logistics, and medical services within the unit 

• sustaining good physical and psychological performance 

3. Materiel and ordnance status concerning 

• technical availability 

• tactical availability 

• damage assessment and maintenance requirements 

• depleted resources 

• replenished resources 

4. Information status concerning 

• operations orders and mission objectives 

• the reliability, availability and diagnosticity of intelligence and information 

• access to and utilization of intelligence/information acquisition resources 

• reporting from own and adjacent units 

Case Study: Mechanized Infantry Combat Efficiency Analysis 

The integration of the system models, concepts, and components above now leads to the 
definition of the combat efficiency measure, that will be applied in the case study in the 
following sections. The definition of combat efficiency and its determinants is depicted 
in figure 3, next page. 


Figure 3: Definition and determinants of the combat efficiency 

The Mission: Attack Enemy Airborne Assault 

In this case of unit evaluation and assessment, the unit at study was a reduced 
mechanized infantry company, transported in armored personnel earners, and equipped 
with man-portable anti-tank missile systems as its main armament. The unit also had 
access to indirect fire support, i.e, artillery and mortars, and anti-tank mines. The 
mission of the mechanized infantry company were to locate and attack enemy airborne 
landings in either of three geographical sectors, designated sector A, B, and C. 

The opposing force in this particular situation was a special unit, organized and 
composed by uniquely equipped and trained mechanized platoons, with tactics, mobility, 
and firepower equal to a hypothetical, modem, enemy air assault unit. The mission of 
the air assault forces were to seize and defend a number of vital crossing points in order 
to support following airborne landings. 

The combat environment and the effects of major weapon systems were simulated in the 
MIND simulation environment, (Jenvald, Morin, Worm, and Omberg, 1996). The 
MIND system, which is an integrated simulation and data collection system for 
instrumented foice-on-force battle training, is currently used by the Swedish Army in its 
Combat Training Center, at which the described case study took place. 

The Combat Efficiency Analysis was based on aggregation and evaluation of expert 
observations, registered combat events, and After Action Reviews (Rankin, Centner, and 
Crisscy (1995) of two consecutive battle training sessions, in which the mission and 
combat requirements were close to identical. The participating units were the same, and 
the operations orders were adjusted to compensate for learning and transfer effects. 
Information lost (if any) due to technical failure were not able to take into consideration. 


but judging from the consistency of the results of the two sessions, this fact did not affect 
the data compilation and analysis in any decisive way. 

Initial Mission and Unit States 

Status of Soldiers and Operators: Individual and team skills were limited because basic 
training was not completed; only 6,5 months of a total of 10 to 12 months were earned 
out. This induced limitations in the unit’s ability to correctly handle unforeseen 
situations, not yet experienced by the unit, and this was taken into consideration in the 
analysis. The fact that the unit was slightly reduced caused no significantly limited 
availability of personnel and materiel in battle training sessions as short as in this study. 

Commander Sustainment and Improvement of Unit's Status: Main limiting factors were 
that commanders had not yet performed any evaluation of experienced combat situations 
together with all personnel involved and, consequently, were not able to train and adapt 
mission tactics and procedures associated with the battle to come. 

Materiel and Ordnance Status: Unit had good logistic preparedness. No materiel 
lacked, and supplies of fuel and ammunition were full. 

Information Status: Tactical operations orders were understood and disseminated into 
clear, straightforward operations orders. Preparations orders were issued and executed. 
The limiting factor was terrain knowledge, and also intelligence, which was uncertain, 
unconfinned, and several hours old. 

Battle Course of Events 

The outline of the mission, preparations, enemy approach to landing zone and acquired 
intelligence prior to engagement are briefly depicted in the mission timeline in table 1. 
From the time of landing and engagement of the air assault forces the course of events 
were followed by expert observers and controllers, most of which were experienced 
combat training officers in active duty. They reported combat events supported by a 
computerized tool for structured activity reporting (Jenvald and Morin, 1997), which 
allowed automated processing of the very large quantity of information acquired. 

Battle course of events were violent and expeditious. Approximately sixty minutes from 
the landing of the assault forces at time T, the combat activity culminated and the session 
stopped sixty minutes later, in order to let both sides handle disengagement and 
preparations of reengagement, and to take care of the simulated casualties and wounded, 
thereby increasing workload of the logistics function. At the After Action Review, 
commanders and soldiers of both sides, together with the mission training officers, 
concluded that the opposing air assault forces, despite large losses at one flank, were able 
to break through or avoid the mechanized infantry units at the other flank, and 
accomplish their mission. 


Table 1: Mission Timeline 


Own unit events 

Opposing force events 



Session start. 

Tactical operations orders issued: 
Prepare location and attack of 
enemy airborne assault in sector 

A, B, or C. 

Session start. 

Tactical operations issued: 

Prepare airborne assault in sector 
A, B, or C. 



Company operations orders 
issued: Locate and attack enemy 
airborne assault in sector A, B, or 

Company operations orders 
issued: Commence airborne 
assault in sector A. 

Bomber sorties launched. 



Air defense warning issued: Air 
attacks east Skovde city, sector A 
and B. 

Assault preparations. Air strikes 
in sector AandB. Assault unit 


Air defense warning issued: Air 
attacks Karlsborg city westward, 
sector A and B. 

Last bomber sorties launched. 
Assault forces launched. 


Air defense warning issued: Air 
assault Karlsborg city heading 
Skovde city. 

Assault forces approach landing 
zone in sector A. 



Battlefield intelligence reports: 

Air assault landing in sector A. 

Time T: Assault units landing. 


Indirect fire support authorized. 

Assembly of units after landing. 



Maneuvering towards enemy 

Visual contact. Engagement. 

Time T -f 35 minutes: Advance 
towards target zone. 




Session stops. 

Preparations for After Action 

Session stops. 

Preparations for After Action 


Information processing during the battle 

The flow of information and intelligence within and between the units engaged during 
the session indicated grave inaccuracy and misconception. Intelligence was 
communicated slowly between the different levels of command, and was not properly 
acquired, compiled, and interpreted. This led to information time delays that caused the 
mechanized infantry company commander to have an incomplete intelligence support in 
his situation assessment, and hence, his situation awareness deteriorated. 
Communication systems caused information loss due to bad sound quality, units out of 
range or partial technical function losses. 

Decision making and actions during the battle 

Altogether, the insufficient intelligence support and the communications difficulties 
constituted the major part of the inconsistencies in the unit’s shared situation awareness, 
and the inaccuracies and misjudgments of the mechanized infantry company 
commander’s decisions. The company commander did not realize the need for 
reconnaissance patrols for potential enemy sector surveillance, nor did he identify the 
necessity of indirect fire forward observers in the potential landing sectors for immediate 


response to enemy actions. Both of which showed later to be of crucial importance to the 
fulfillment of the mission. This led to feedback delays in decision making and to actions 
that caused severe consequences for the unit’s ability to fight in a coordinated and timely 
manner. The results were consistent with Brehmer and Allard (1991), who addressed the 
issue of time delays in real time decision making. 

Utilization of available resources during the battle 

During the battle training sessions the mechanized infantiy company had an almost 
unlimited amount of simulated indirect fire support available, with capability of 
combating both soft and hard targets. Much of the indirect fire was unobserved, which 
led to low hit probabilities and waste of exclusive fire-and-forget anti-tank ammunition. 
A great number of anti-tank mines were hardly used at all in the battle, which caused 
difficulties in blocking terrain not possible to cover with troops. The units used 
unnecessary short firing distances of their medium-range anti-armor weapon systems. 
The units also primarily used firing positions towards the enemy’s front. All this caused 
low terminal weapon effects, low effects of surprise and of holding a well-prepared firing 
position, and high risks of detection and counter-attack, with large losses as a 
consequence. The mechanized infantry company did not accomplish its mission, to 
efficiently locate and attack air assault forces within its perimeter. The unit suffered 
from gravely reduced combat sustainment and reengagement capability after the first 
engagement. This rendered the mechanized infantry company low combat efficiency. 


The integration of various well-proven research domains, such as control theory, several 
behavioral and human factors areas, quality control, operations research, and statistics, 
together with extensive utilization of the MIND system, turned out to be a successful 
approach. Applied in a context of ground force battle training, combined with advanced 
medium and high fidelity combat simulations, and the systematic use of domain expert 
knowledge, this novel approach facilitated comprehensive militaiy unit evaluation and 
assessment. From the performed Combat Efficiency Analysis we concluded, that in this 
particular case, the following factors limited and constrained the unit’s ability to execute 
its mission with its available resources: 

• The ability to rapidly and accurately determine the identity and location of 
enemy targets and of own units, and to evaluate the battlefield terrain. 

• The access to and use of a battle information structure which supports and 
improves real time information and intelligence acquisition, and permits 
mission-relevant information and intelligence to reach the intended decision 
maker in a timely manner. 

• The access to and use of robust, wide-band communication systems that 
permit fast and accurate transfer of data and speech. 

To be able to objectively and reliably identify the hmiting factors of a specific unit, 
system, or operating procedure, and to assess the magnitude of influence of these factors 
on unit’s overall combat performance, a series of within-mission Combat Efficiency 
Analyses in will have to be performed in each typical case. In force-on-force battle 
training situations, most factors except the ones studied can be held constant to the 
greatest extent possible, which makes the validation of results easier. Also actual combat 


missions can be analyzed in the same way. However, the extreme risk exposure, and the 
reduced reliability, diagnosticity, and availability of information and intelligence 
obtained in armed conflict or other hostile situations, makes validation more 

Future Work 

There are several areas in which further development and refinement of the Combat 
Efficiency Analysis technique could be of vital importance. A few examples are: 
military training (especially training tactical decision making and combat resource 
management), battlefield intelligence (using the Combat Efficiency Analysis technique, 
a new perspective of the capabilities and limitations of the enemy can be obtained), and 
development of tactical units , procedures, and systems (particularly within the 
command and control domain). We also see possibilities to develop the Combat 
Efficiency Analysis technique into a generic mission evaluation tool for activities other 
than those of warfare, e.g. crisis management and emergency response. However, the 
potential risk of a workload increase has not yet been fully investigated. If workload 
actually increases significantly in a way that would endanger the outcome of an actual 
combat mission, this must be taken into consideration before launching a within-mission 
Combat Efficiency Analysis. If this should be the fact, the Combat Efficiency Analysis 
technique will perform very well in a post-mission debriefing situation as well, judging 
from the successful Army Combat Training Center achievements. 


This research was supported in equal parts by the Department of the Army and the 
National Defense Research Establishment, in the Army Combat Training Center project. 
Support was also given by the National Defense College and the University of 
Linkdping. Many thanks to Prof. Dr. Martin Helander, Linkbping University, and to 
Prof. Dr. Bemdt Brehmer, National Defense College, for helpful guidance and 
encouragement. We acknowledge the contribution of other members of the MIND 
system group: Mr. Goran Omberg, Mr. Johan Jenvald, and Mr. Magnus Morin, to the 
success of the MIND technology and methods for instrumented battle training. 



Ashby, W. R. (1956), An Introduction to Cybernetics, London: Chapman & Hall, 

Brehmer, B, Allard, R, (1991a). Real time dynamic decision making: The effects of task 
complexity and feedback delays. In J. Rasmussen, B. Brehmer, & J. Leplat (Eds.), 
Distributed Decision Making: Cognitive Models for Cooperative Work, 
Chichester: Wiley. 

Brehmer, B, (1992), Dynamic decision making: Human control of complex systems. 

Acta Psychologica 81, pp. 211-241. 

Endsley, M. R. (1995) Towards a theory for situation awareness in dynamic systems. 
Human Factors, 37, pp. 32-64. 

Jenvald, J., and Morin M. (1997). Multipurpose use of information from force-on-force 
battle training. In Proceedings of the 8th Training and Education Conference 
(rTEC97). Lausanne, Switzerland. 

Jenvald, J., Morin, M., Worm, A., and Omberg, G. (1996), MIND - Ett instrument for 
vdrdering, utveckling och utbildning av krigsforband, [MIND-An Instrument 
for Assessment, Development, and Training of Armed Forces.] Technical Report, 
Linkbping: National Defense Research Establishment. 

Rankin, W. J., Gentner, F. C., and Crissey, M. J. (1995). After Action Review and 
Debriefing Methods: Technique and Technology. In Proceedings of the 17th 
Interservice / Industry Training Systems and Education Conference (I/TTSEC). 
Albuquerque, NM. 

Worm, A. (1996). Metoder och verktygfor vdrdering och utveckling av krigsforband. 
[Methods and Tools for Evaluation and Assessment of Military Units.] Master's 
Thesis, Linkbping: University of Linkbping. 


Arne Worm (Captain, Army) is a member of the MIND system group at the National 
Defense Research Establishment, where he performs systems development and systems 
integration. Mr. Worm is also engaged in modeling, simulation, development, 
integration and design of future computerized command, control and intelligence 
systems of the armed forces. His research interests include command and control 
science, military and emergency management training, human factors, control theory and 
systems engineering issues. Mr. Worm holds a M.Sc. in Mechanical Engineering and 
Computerized Automation from Linkbping University. He is a Ph.D. candidate at the 
University of Linkbping and the National Defense College, at which he is currently 


Human Factors, an emergency situation, 
and the Automated Highway System 

Dick de Waard', Monique van derHulst', Marika Hoedemaeke/, Karel Brookhuis', 

& Hans Soeteman 
^ Centre for Environmental and Traffic Psychology 
University of Groningen 
^ Delft University of Technology 
^ Transport Research Centre 
Ministry of Transport & Public Works 
The Netherlands 


Twenty subjects drove a normal car and an Automated Highway System (AHS)-car in a 
high fidelity driving simulator. The AHS-vehicle was driven in three conditions; at 1 
second time headway, at 0.25 s. time headway and as platoon leader. In the final 
condition when they were driving as platoon leader, the front sensor of the automated 
vehicle failed to detect a merging car in time, leading to a critical incident (but not an 
accident). Ratings of mental effort, activation, risk, safety and acceptance were collected 
and the driver’s ECG was continuously registered. 

Results show that normal driving is more activating and effortful than automated driving. 
When driving in AHS mode drivers prefer a large time-headway to vehicles-in-front. 
Risk ratings are highest for the emergency situation, followed by driving at 0.25 seconds 
in automated mode. The emergency situation was surprising for most subjects, and the 
majority did not respond or responded late to it (drivers could avoid the critical situation 
by pressing the brake). 


In the Automated Highway System (AHS, e.g., Sussman, 1996) “intelligenf’cars will be 
guided by the road infrastmcture and will control their own lateral and longitudinal 
position on the road. Traffic on an automated lane will drive in platoons, i.e. in rows of 
cars driving at short headway. The ultimate aim of AHS is to increase road capacity and 
to eliminate driver error (Congress, 1994). However, for at least several years after initial 
AHS implementation. Safety Boards will require that the driver ‘stays in the loop’. The 
driver will be strongly encouraged or perhaps even legally requested to monitor the AHS 
and other vehicle systems during the journey, to ensure that all functions are working 
properly (Alicandri & Moyer, 1992). For reasons of liability the user has to stay alert in 
the AHS facility aU the time, just in case manual control needs to be resumed (Congress, 
1994), This is similar to an aeroplane pilot who, monitoring autopilot functioning, 
resumes control whenever deemed necessaiy. However, driving a car entails higher 
frequencies of actions and lower time constants, i.e. it will be much more difficult to take 
over car control than aeroplane control in case of acute emergency. 

Only after system reliability is ensured and driver liability is ruled out, driver attention to 
the system will become formally superfluous. One of the basic questions then in AHS, in 
due course, will be whether driver intervention with vehicle and system operation in an 
emergency situation should be allowed or not (Tsao et al., 1993). If the driver is expected 


to intervene, training will be required. If s/he is not supposed to intervene, one of the 
questions raised then is whether s/he will try to do so anyway (Tsao et al., 1993), Finally, 
will it be acceptable to the driver to be totally without control. The question whether 
drivers are going to give over control is the question whether AHS is accepted. Research 
indicates that drivers prefer warnings to actual take-over of control (Nilsson & Aim, 
1991, Hoedemaeker, 1996), with the exception of driver emergency situations (e.g., in 
case of an acute health problem or when falling asleep, Petica & Bekiaris, 1996). Shifting 
the driver ‘out-of-the-loop’ may lead to reduced responsiveness to critical events, which 
has been reported as pre-crash factor in aviation (e.g., Ward, 1996). Automation may also 
increase reaction time. In case of continuous monitoring, reaction time to events in a 
driving task can be restricted to something like one second, while if more than one 
functions have to be monitored and other tasks are attended to, awareness of the situation 
has to be refreshed with increased frequency and the malfunction and its origin have to be 
determined which might take many seconds. In this way, an attempt to reduce workload 
is actually very likely to lead to increased workload (Hancock & Parasuraman, 1992), 
The classic goal of automation is to replace human manual control, planning and problem 
solving by automatic devices. However, these systems still need human beings for 
supervision and adjustment. It has been suggested that the more advanced a control 
system, the more crucial is the contribution of the human operator (Bainbridge, 1983). 

The point made by Bainbridge (1983) is as follows: normal operation is performed 
automatically, abnormal conditions are to be dealt with manually. Unfortunately, as a 
result of automation, experience is limited, while in case of abnormal conditions (i.e., 
something is wrong with the process) unusual actions will be required. Also, human 
problem solving is not optimal under time-pressure. Monitoring of (present) automatic 
processes is based on skills that formerly manual operators have, and that future 
generations of operators (/drivers) cannot be expected to have (Bainbridge, 1983). Pilots 
also indicated that although automation reduced workload, it also had a negative effect on 
flying skills. They considered manually flying of a part of every trip important to 
maintain these skills (McClupha et al., 1991). 

The aim is free of faults functioning, but it is obvious that there is no way of guaranteeing 
that a car or any other part of the system will not fail in the AHS (Hitchcock, 1991). 
When a system fails to work or is in a state that failure is possible, feedback should be 
provided in order to let the driver know that s/he can not rely on the system. The main 
reason for this is that automated systems can and will lead to what has been called 
‘complacency’ (Wiener and Curry, 1980). Complacency is an attitude of (over)reliance 
on an automated system. This and other forms of behavioural adaptation or compensation 
as it is called in a wider field, are factors that should be taken into account when 
investigating the conditions for introduction of AHS (Verwey, Brookhuis & Janssen, 

The new AHS-driving task will bring the driver in an unknown situation, for the which 
the demands, limits and preferences are not yet determined. Questions of major 
importance in this respect are: 

• What is the driver’s opinion about this altered task, what is the level of acceptance? 

• Is there an effect of driver alertness level on task performance and acceptance? 


• Are there any differences in driver behaviour and risk ratings for different following 
distances in a platoon of cars? 

• Are there any differences in driver behaviour and risk ratings when driving as 
platoon-leader compared with driving inside the platoon of cars? 

• What happens in case of system failure in an emergency situation, will the driver (try 
to) take over control? 

Bringing the driver into the real AHS condition is not yet possible, reason to investigate 
driver’s behavioural changes and feelings in a driving simulator. Another reason to carry 
out AHS studies in a driving simulator is an ethical one, any emergency, any system or 
driver failure must be without any (physical and/or material) consequences. 


Twenty subjects drove the advanced driving simulator of the Centre for Environmental 
and Traffic Psychology (the former Traffic Research Centre, Van Wolffelaar & Van 
Winsum, 1995). All subjects drove the simulator in the automated mode where lateral and 
longitudinal position was regulated by the system, as well as in the ‘normal’ mode (as if 
in an ordinary hand-shifted car). Subjects completed two sessions in the simulator, one in 
the morning (between 9 an 12 AM) and one in the evening (between 9 and 11 PM), hi the 
evening, subjective driver alertness can be expected to be lower (Monk et al., 1983, De 
Waard et al., 1997) which could be a nonoptimal state for monitoring. The order of the 
conditions was balanced across subjects. 

In the AHS conditions drivers drove (or actually only sat in) a car in a platoon. The car 
drove at high speed. Three conditions were part of the experiment: driving in the platoon 
at 1 second time headway to the car-in-front, driving at 0.25 seconds headway, and 
driving as platoon leader. The experimental following times that were used have been 
proposed in the USA by Bloomfield (1995). The driver was not able to control lateral 
position, i.e., steering is not possible. The only way to overrule the AHS was by applying 
the brake. 

During the rides physiological measurements (ECG) and self-report ratings were 
collected. These measurements provide information on mental effort, activation and 
experienced risk and safety. Failure of the AHS can and was tested only once, in the final 
session at the end of the experiment when they were driving as platoon leader, in order to 
prevent negative effects on the rest of the experiment and the simulator in general. The 
emergency was a failure of the AHS front-sensor to detect a merging car. The car merged 
extremely close in front of the AHS vehicle (leaving 0.1 metres of space between the 
cars). Failure of the AHS did not lead to an accident, but the situation was such that in 
normal driving an alert subject would apply the brake. 

Before the actual experiment, subjects were asked to read a description of the AHS and 
were trained in driving the simulator, both in normal and in AHS mode. The order of 
these conditions was balanced across subjects. Per visit, subjects drove under the 
following conditions: 


N: Normal, conventional driving, distance to a lead car is self-chosen 

Al: AHS, supervisory, distance to car-in-front = 1 second, speed = 100 km/h 

A.25: AHS, supervisory, distance to car-in-front = 0.25 seconds, speed =100 km/h 
F: AHS platoon leader, supervisory. This condition was only part of the experiment 

during the first session (visit). 

F*: AHS failure to respond. In this condition the subject’s car was platoon leader, and 

a car merged from the emergency lane to subject’s path. This condition was only 
part of the experiment during the second session. 

In the middle part of the N, Al and A.25 conditions the lead car slowed down, in the 
AHS conditions the system reacted appropriately (i.e., also slowed down). The order of 
conditions was balanced over subjects. The following measures were sampled at 5 Hz: 
lateral position, steering wheel position, time headway, speed (aU for conventional 
driving) and brake pedal position. Changes in driver alertness are preferably measured 
from alertness related physiological measures (see Brookhuis & De Waard, 1993). As 
physiological measure subject’s heart rate was registered. From these measurements 
average heart rate and heart rate variability, in particular in the frequency domain, were 
calculated. Variability in the Blood pressure band (0.07-0.14 Hz, “the -0.10 Hz 
component”) reflects mental effort (see e.g., Mulder, 1992, Brookhuis & De Waard, 

Before each session subjects completed questionnaires on sleepiness (Stanford Sleepiness 
Scale and a rating of tiredness, see De Waard, Van der Hulst, & Brookhuis, 1997). Before 
the first test ride on the first day and after the last ride on the second day acceptance of 
the AHS was assessed by a standard ATT acceptance checklist (Van der Laan, Heino & 
De Waard, 1997). After each condition (i.e., after Al, A.25, N and F) a self-rating of 
mental effort (Rating Scale Mental Effort, RSME, Zijlstra, 1993), a rating of activation 
(Bartenwerfer, 1969), a risk rating (Heino, 1996) and a safety rating on a 5-point Likert 
scale were collected. Subject’s ECG while resting was measured at the start and end of 
each session. 

At the end of the second session subjects again completed the acceptance checklist under 
the instruction not to take into account the condition where the system failed. A preferred 
AHS-time headway was also indicated. 


Average age of the subjects was 29.8 {sd 6.0), 20 % was female. Their average mileage 
was 16 (XX) km/year and, on average, they had held a licence for 8.6 years (sd 5.7). 
Effects of time-of-day were limited to subjective tiredness, which was higher in the 
evening. As other effects of time-of testing (and subjective alertness level) were absend, 
no further attention will be given to this factor. 

Average speed in the ‘normal-driving condition’ was equal to the lead car, 100 km/h. 
Average time-headway was 2.2 seconds (sd 1.4), well above the AHS time headways of 1 
and 0.25 seconds. 

What happened in the emergency situation in the final trial is illustrated in. At t=0 the 
experimental AHS vehicle (E) is within viewing distance of the car that will merge (M), 
which is still standing still on the emergency lane. At t=7 seconds, the M-car accelerates. 


while at t=9.7 s the M-car blinks and starts to merge in front of E. At t=14.7 s the 
manoeuvre is completed, and if the subject did not brake, the distance between the two 
vehicles would be no more than 0.1 metres. 

IMU — 

Emergency lane 

t= 0 

t=7.0 t=9.7 t=14.7 

Figure 1. Emergency situation (F*) 

In Table 1 subject’s behaviour during this trial is classified on the basis of braking- 
response, the only means to interfere. As much as 50% of the subjects did not react at all, 
whereas 15% braked after the car had merged close in front. 

Table L Subject’s response during the emergency situation 


- -’O . . <3 

time (t =) 

Proportion subjects 

No reaction 





> 14 s 




9-14 s 




< 9 s 


After each conditions self-ratings of mental effort, activation, risk and safety were 
collected. In Table 2 averages of these ratings are summarised. Statistical tests (the reader 
is referred to the technical report, De Waard et al., 1997) indicate that normal driving is 
rated as more effortful and more activating. The short-headway AHS condition requires 
more mental effort, is more activating and riskier, and judged to be less safe. The 
emergency situation is considered to be the riskiest and the least safe. There were no 
significant effects of being inside or in front-position of the platoon on any of the 
subjective ratings. 


Table 2: Averages of self-ratings. The following ranges apply, from low(effort, risk etc) to high: Effort 
(RSME): 0-150, Activation 0-270, Risk 0-6 (7 point). Safety 1-5 {5-point). Session 1 is the first visit and 
test. Session 2 the second visit, thus on another day. _____ 


















AHS 1 s 











AHS 0,25 s 











Lead (F) 






(Lead F*) 






If the AHS conditions are taken together and are compared to normal driving, a 
significant effect on the 0.10 Hz component of heart rate variability is found. During 
normal driving the 0.10 Hz component is more supressed than during automated driving, 
which indicates increased mental effort during normal driving. All heart rate parameters 
differ significantly from rest during driving. More details on heart rate can be found in 
the technical report (De Waard et al., 1997). 

Driver acceptance of AHS was measured using standard ATT acceptance items (Van der 
Laan, Heino & De Waard, 1997). Before the actual tests, drivers were asked to judge the 
AHS system, and after the second session the same was done with the explicit instmction 
not to take into account the emergency trial. Reason for this is the interest in general AHS 
acceptance, and the unlikeliness of acceptance of an ill-functioning system. Usefulness of 
the AHS is slightly positive (0,5 on a scale from -2 to -i-2), in terms of being satisfying 
the overall opinion is slightly negative (-0,3), Acceptance ratings did not change 
significantly after experience with the system and are comparable to the evaluation of an 
autonomous intelligent cruise control (see Van der Laan, Heino & De Waard, 1997), i.e. 
relatively low scores due to take-over of control. 

75% of the subjects preferred the long (1 s. time headway) following condition to the 
short (0.25 s. time headway) following condition. The other 25% did not prefer one 
condition to the other. 

In addition to driver acceptance in terms of usefulness and satisfaction, questions were 
asked whether the driver agreed with various positive and negative aspects of the 
systems, which were explained in one or two sentences. Table 3 shows the percentages of 


the drivers who agree on four of the statements about the Automated Highway System 
(AHS) before and after the tests. 

Table 3. Percentages of AHS opinions. 




It enhances traffic flow 




I don’t have to be attentive 




No control of driving 




No fun of driving 



The Wilcoxon Signed Ranks test shows that the increase in ‘No control of driving’ is 
significant. Much more drivers disliked the fact that they had no control of driving when 
driving in the AHS system after experience with the system. On the other three items 
drivers did not change their opinion after experience with the system, or in the direction 
of a less positive opinion. 


In the present experiment differences between automated driving and normal driving 
were found on mental effort and activation. More effort is required for normal driving, as 
is indicated by self-reports of invested effort and heart rate parameters. Driver activation 
is also higher during normal driving compared to automated driving. The results on these 
self-report scales are in line with expectations with respect to an altered driving task; 
during conventional driving the driver has to be active, during automated driving the task 
is supervisory. Risk and safety ratings differ to a lower extent and depend upon the time¬ 
headway of the AHS. At the close headway of 0.25 s, the average risk rating is 
comparable to normal driving whereas safety during the automated drive is judged to be 
lower. At the larger headway of 1 second these differences disappear, and risk is actually 
rated lower during the automated ride compared with the normal ride. When asked to 
indicate a preferred AHS following distance the majority chooses the large headway, 
while none of the subjects prefers the short headway (25% does not have a preference). 
One of the advantages of AHS is an increased road capacity per driving lane. This effect 
is enhanced by shorter time-headways, but, driver acceptance of this is a prerequisite. In 
addition to the preference as indicated directly by the subjects, the higher risk ratings for 
the short following conditions also give a clue that introduction of short time headways in 
platoon driving may be stopped by low user acceptance. Similar effects were found in 
studies by De Vos (De Vos et al, 1996, De Vos & Hoekstra, 1997), drivers dislike short 
time headway, De Vos and coworkers expected increased ratings of comfort when 
driving at very close distances from the vehicle-in-front, but did not find them. Their 
hypothesis was that even though at short headways the chance of collision may be high, 
the collision energy (and thus damage) is low due to small speed differences when 
colliding. Perhaps for most people this positive implication is difficult to imagine. 

The emergency situation at the end of the experiment was a surprise for most, as many 
did not react or responded fairly late. In most instances where the front-sensor failed to 
work, the tested situation would have led to a collision. Complacency may be the reason 
for no response, after the experiment a subject stated that “you either trust the system and 


sit and relax or don’t trust the system and are continuously alert and stressed”. One 
should be aware of the fact that subjects were tested in a driving simulator, which may 
have had an effect on their behaviour. The effects on motivation of driving in a simulator 
opposed to on-the-road tests have been described in the literature (e.g., Smiley & 
Brookhuis, 1987). The actual risk of collision in the present experiment was nil and may 
have made subjects less inclined to take over control. 

Driver opinion on different Automated Vehicle Guidance systems indicates that drivers 
expect the systems to have as much negative as positive effects when introduced into the 
traffic system. In general it can be concluded that drivers are positive about the effects on 
traffic safety, but they see also negative effects like loosing control of driving and 
attention reduction. This result is in accordance with the results of an earlier study with 
this questionnaire (Hoedemaeker, 1996). Driver acceptance of a well-functioning AHS is 
neutral in terms of satisfying and usefulness. This means that such a system is not refused 
after experience in a simulator. It is, however, obvious that an AHS should be fail-safe 
and that drivers should be able to trust on accurate functioning of the system. 


Alicandri, E., Moyer, M.J. (1992), Human Factors and the Automated Highway System, 
Proceedings of the Human Factors Society 36th annual meeting, pp. 1064-1067. 
HFES: Santa Monica, CA, U.S.A. 

Bainbridge, L. (1983). Ironies of automation. Automatica, 19, 775-779. 

Bartenwerfer, H. (1969), Einige praktische Konsequenzen aus der Aktivierungstheorie. 

Zeitschriftfur experimentelle und angewandte Psychologic, 16, 195-222. 
Bloomfield, J.R. (1995). Human Factors in the automated highway system: transferring 
control to the driver. Report FHWA-RD-95-189 / FHWA-RD-W-114, Also 
available at: humanfac/rd95189.htm 
Brookhuis, K.A., Waard, D. de (1993). The use of psychophysiology to assess driver 
status. Ergonomics, 36, 1099-1110. 

Congress, N. (1994).The Automated Highway System: an idea whose time has come. 

Public Roads On-Line, Summer 1994. http: // / pubrds / summer94 
/ p94sul,htm. 

De Vos, A.P., Theeuwes, J. & Hoekstra, W. (1996). Behavioural aspects of Automatic 
Vehicle Guidance (AVG): The relationship between headway and driver comfort. 
Report TM-96-C022. Soesterberg, The Netherlands: TNO Human Factors 
Research Institute. 

De Vos, A.P. & Hoekstra, W. (1997). Behavioural aspects ofAuto,matic Vehicle 

Guidance (AVG); Leaving the automated lane. Report TM-97-C010. Soesterberg, 
The Netherlands: TNO Human Factors Research Institute. 

De Waard, D., Van der Hulst, M., & Brookhuis, K,A. (1997), The detection of driver 

inattention and breakdown. Proceedings International Seminar human Factors in 
Road Traffic n, pp, 62-67. Braga, Portugal: Universidade do Minho. 

De Waard, D., Van der Hulst, M., Hoedemaeker, M, & Brookhuis, K,A. (1997). Human 
control in an emergency situation in the Automated Higway System (AHS). Report 
COV 97-04. Groningen, The Netherlands: Centre for Environmental and Traffic 
Psychology, University of Groningen. 

Hancock, P.A, & Parasuraman, R. (1992). Human Factors and safety in the design of 
Intelligent Vehicle-Highway Systems (FVHS). Journal of Safety Research, 23, 


Heino, A. (1996). Risk taking in car driving; perceptions, individual differences and 
effects of safety incentives. PhD thesis, University of Groningen. 

Hitchcock, A. (1991). Intelligent Vehicle-Highway System safety: problems of 

requirement specification and hazard analysis. Transportation Research Record 
1318, 98-103. 

Hoedemaeker, M. (1996). Behoeften, rijstijlen en meningen ten aanzien van 

Automatische Voertuig Besturing (Needs, driving styles and opinion towards 
Automatic Driving). Report. Delft, The Netherlands, Technical University Delft. 

McClumpha, A.J., James, M., Green, R.G., & Belyavin, A.J. (1991). Pilots' attitudes to 
cockpit automation. Proceedings of the Human Factors Society 35th annual 
meeting, pp. 107-111. HFES: Santa Monica, CA, U.S.A. 

Monk, T,H., Leng, V.C., Folkard, S„ & Weitzman, E.D. (1983). Circadian rhythms in 
subjective alertness and core body temperature. Chronobiologia, 10, 49-55. 

Mulder, L.J.M, (1992). Measurement and analysis methods of heart rate and respiration 
for use in applied environments. Biological Psychology, 34, 205-236. 

Nilsson, L. & Aim, H. (1991). Collision Avoidance Systems -Effects of different levels of 
task allocation on driver behaviour. Report DRIVE V1041/GIDS/MAN3. Haren, 
The Netherlands: University of Groningen, Traffic Research Centre. 

Petica, S., Bekiaris, E. (1996) Driver Needs and Public Acceptance of Emergency 
Control Aids. SAVE TR1047 EU Project Deliverable 3.1. Arcueil, France: 

Smiley, A. & Briookhuis, K.A. (1987). Alcohol, drugs and traffic safety, hi J.A. 

Rothengatter & R.A. de Bruin (Eds.) Road users and traffic safety (pp, 83-105). 
Assen, the Netherlands: Van Gorcum. 

Sussman, J.M. (1996). ITS: A short history and perspective on the future. Transportation 
Quarterly, 50 (4), 115-127. 

Tsao, H,-S.J., Hall, R.W., Shladover, S.E. (1993). Design options for operating 

Automated Highway Systems. Proceedings of the lEEE-IEE Vehicle Navigation & 
Information Systems Conference, Ottawa, pp. 494-500. Piscataway, NY, U.S.A.: 

Van der Laan, J., Heino, A., De Waard, D. (1997). A simple procedure for the assessment 
of acceptance of advanced transport telematics. Transportation Research - C, 5, 1- 

Van Wolffelaar, P.C. & Van Winsum, W. (1995). Traffic simulation and driving simulation 
—an integrated approach. In Proceedings of the Driving Simulator Conference 
(DSC'95). Toulouse, France: Teknea. 

Verwey, W.B., Brookhuis, K,A. (1996). Safety effects of in-vehicle information systems. 
Report TM-96-C002. Soesterberg, the Netherlands: TNO Human Factors 
Research Institute. 

Ward, N.J. (1996). Interactions with Intelligent Transport Systems (ITS): effects of task 
automation and behavioural adaptation. ITS Focus Workshop ‘Intelligent 
Transport Systems and Safety’, Februari 29th 1996, London. 

Wiener, E.L. & Curry, R.E. (1980). Flight deck automation: promises and problems. 
Ergonomics, 23, 995-1011 

Zijlstra, F.R.H. (1993). Efficiency in work behavior. A design approach for modem tools. 
PhD thesis, Delft University of Technology. Delft, The Netherlands: Delft 
University Press. 


A Model of Individually Acting Drivers with Cognitive Capabilites for 

Microscopic Traffic Simulation 

Marita Irmscher, Thomas Jurgensohn 
InstitutfUr Strafien- und Schienenverkehr 
Technische Universitdt Berlin, Germany 


When assessing novel car technologies like driver assistance systems or route guidance 
systems it is desirable to predict the effects of these technologies on vehicle-driver- 
environment interaction already during the development phase. Due to the complexity of 
the issue computer simulation is the most appropriate means for such an evaluation. As 
human factors play an important role in the performance of the entire system, the need of 
reliable conclusions implies an appropriate modelling of human decision making and 
driving behaviour. 

In this paper a model of individually acting drivers which have cognitive capabilities like 
planning, decision making and goal directed acting is outlined. The model presented is 
based on a production system, which can be extended by the instrumentality of Artificial 
Neural Networks and Fuzzy Techniques to describe psychological knowledge about 
motivation, information processing capabilities and motor control. The behaviour of one 
or more drivers in typical traffic situations is simulated. This includes also the modelling 
of inter social relations. 

The driver model is implemented and run in a driving simulation environment. Some 
selected examples are presented and discussed. 


Traffic accidents are frequently caused by perceptual flaws or mutual misunderstandings 
of drivers in quickly emerging critical situations, especially at high speeds. Microscopic 
traffic simulation that focuses on individual behaviour of the human as well as the 
driver-vehicle interaction could help to gain more insight into the origination of 
accidents and the means how to prevent them as well as predict the effects of driver 
assistance systems and influence their development in an early stage. We therefore 
concentrate on modeling and simulating of human cognitive capabilities and 
performance in a typical dynamic situation considering social interaction with other road 

Driver behaviour is determined in a decisive way by accessible information about the 
current situation, experience and skill as well as motivation. The necessary information 
about the environment is gathered by means of the human sensory apparatus which 
cannot reproduce quantities like relative velocities or distances with great accuracy. 
Experience and skill in car driving manifests in the capability to judge the significance 
of available information about the actual driving situation and the existence of highly 
trained complex motor programs like parking or overtaking. Together with motivation 
which is a central factor in the entire cognitive process, these factors influence the 
internal setting of goals, judgment of a situation, the decision process and the way of 


executing actions. Goal directed acting in an environment populated by other agents 
implies moreover the evaluation of the consequences of one’s decisions and actions and 
hence anticipation of the behaviour of other traffic participants. 

We conclude that a driver model for microscopic traffic simulation which can be used to 
simulate individual behaviour in critical situations has to embody human perception 
characteristics and motor processes as well as motivation, decision making and goal 
directed acting. 

Other Driver Models 

To simulate driver behaviour it is most common and often sufficient to model the driver 
by means of control theory where driver behaviour is described by a single, basically 
linear differential equation with time shifting operations and possibly a nonlinear part. 
The individual attributes of a driver are encoded in the structure of the differential 
equation and its parameters that can be adjusted such that „carefur‘ or „daring driver 
behaviour can be simulated in a given situation (Wiedemann, 1974). Intrinsically human 
characteristics like goal directed action that is not a mere stimulus reaction, but a result 
of planning and decision making influenced by particular motivational factors cannot be 
modelled that way, though. To achieve the objective to model individual behaviour, 
psychological and physiological knowledge about human cognition has to be 
incorporated into the model. 

In cognitive psychology production systems in which knowledge and behaviour are 
represented in the form of IF - THEN rules are commonly used to model task 
performance and knowledge representation. Proved program systems like Soar und ACT- 
R (Newell, 1990; Anderson, 1993) which are based on psychologically motivated 
problem solving procedures and algorithms have already been used for the 
implementation of cognitive driver models (Aasman, 1995). As the process of problem 
solving which is equivalent to the decision process in our case is part of the program 
design of the above mentioned program systems, we believe that a specially designed 
program better serves the purpose of getting some insight particularly into the relevance 
of motivation and social interaction on decision making. 

Another approach of to model driver behaviour is that of hybrid modelling (JUrgensohn, 
1997). The idea is to combine different modelling tools like Fuzzy Techniques, Artificial 
Neural Networks, production systems or control theory in order to take advantage of the 
particular qualities each method can contribute to the modelling of human behaviour 
(Wolter et al„ 1997; JUrgensohn & Willumeit 1997). A comprehensive survey of 
existing driver models was presented by (Willumeit & JUrgensohn, 1996). 

Description of the driver model 

Individual behaviour is defined through motivational factors, overall goals, specific 
conflict resolution strategies, estimation qualities, driving skills and given declarative 
and procedural knowledge about traffic rules, vehicle operation as well as driver specific 
motor programs. 

Perception, decision making and the execution of movements or motor programs all 
involve processing times which can play an important role especially in dynamic 
situations at high speeds. 


The program is structured in the sense of object oriented design by means of data 
structures and corresponding methods. Driver behaviour is then described by element 
functions which can contain rules like production systems but also parametric functions 
and programs. This conception can be found especially for multiagent systems (Huhn, 

In figure 1 the realized model structure is sketched in principle. It should be noted that 
the separate blocks with the denoted functionalities cannot be interpreted in a way that 
these functions are independent from each other, although from the viewpoint of 
computer implementation it makes sense to treat them as seperate modules. 

external world 

Figure 1: Model Structure of the Cognitive Driver Model 

Coordination of Subtasks 

The driving task in an environment populated by a number of other agents can be 
considered to be composed of several subtasks which have to be to be handled 
simultaneously as well as consecutively during driving. Some obvious subtasks of car 
driving which are relevant for the present investigation are listed below : 

Sensation of environment (expressed by state variables ) 

Identification and interpretation of traffic situations 

Decision among several alternatives of action through evaluating of resulting 

Motoric execution of an action 

Regulation of the action after comparison with estimated outcome 
Communication with other traffic participants 

Coordination of subtasks 

For the coordination of the subtasks some assumptions about human resources in 
information processing and motoric executions are necessary. It is presupposed that 
perception, decision making and motoric action can be executed simultaneously, 
bottlenecks occur only when similar tasks are to be performed. Therefore the main 
criterion for subtask coordination is chronological order and consideration of time 
constants for the respective execution. This simple approach is certainly not valid for a 
comprehensive cognitive modelling, but simulation results of multiple-task performance 
in the field of human-computer interaction assumptions match well with observed 
behaviour (Kieras & Meyer, 1992). 


Drivers have only limited access to important state variables of the environment stored in 
the public database, limits being due to human sensory characteristics. The state 
variables listed below are presumed to describe the driver’s world and therefore are made 
available to the model: 

• Width and course of the road 

• own velocity and position 

• type of neighbouring objects 

• distance, relative velocity and acceleration to neighbouring objects 

• steering movements and movement characteristics (smoothness) of 
neighbouring objects 

• Braking, blinking, flash of neighbouring objects 

• traffic rules 

• additional information from guidance or driver assistance systems 

In the model important characteristics of human sensation should be reflected. This was 
realized by processing the quantities read from the database in a particular way to 
account for the following features: 

• limitation to visual information gathering only 

• limited visual field (objects are eventually not visible) 

• erroneous estimation of relative distances, velocities and accelerations 

• finite processing time for object perception 

• objects perceived peripherally imply information insufficiency and further 
delay in perception due to required eye movements 

Errors in estimating distances etc. are affected by the driver’s expertise and by 
motivational factors. The resulting vagueness of perceived values is modelled by adding 
noise to the database quantitites. 


Reported processing times for visual perception vary between 50 and 250 ms. These 
differences reflect specific processing speeds for different visual resolution (peripheral vs 
foveal) on the one hand and the influence of the test environment on the other hand. 

Identification of Situation 

For the driver model the state of the world is represented by the aggregation of a limited 
number of state variables relevant for the driving task like position or velocity, which are 
continuously changing with time. In cognitive psychology the term „situation“ is 
understood as a kind of category which encompasses a certain range within the state 
space and is closely related with a corresponding action. This is modelled in such a way 
that the identification of the situation is performed by a classification of the current 
variables of state. Classification here means that a continuous state vector is mapped 
onto a number of discrete situational categories. 

The identification of a situation can also be considered as a kind of pattern recognition, a 
typical field of application for Artificial Neural Networks (ANNs). Previous work shows 
that ANNs are well suited to model humanlike identification of a situation with 
subsequent estimation of action urgency (Jiirgensohn et al., 1994; Jurgensohn & 
Willumeit, 1997; Jurgensohn 1997). 

We chose to classify a situation by providing the driver model with declarative 
knowledge about traffic rules and individual safety margins that are influenced by 
motivational factors. The model compares the perceived variables of state with its rule 
base and the identified situation with an internally desired state. If the deviations are 
above a certain margin, a drive to change the current situation a specific way to approach 
the desired state is created. As there may be several action alternatives that are able to 
transfer the present state into a more favourable one a decision process becomes 
necessary to evaluate the alternatives and make a choice. 

Decision making 

Concurring action alternatives require a decision. In order to evaluate distinctive actions 
the action sequences have to be planned first, in a next step consequences due to 
reactions of other traffic participants have to be taken into account and evaluated with 
respect to own goals and security. Goals are evaluated through estimation of benefit and 
cost (risk). For this purpose the individual needs some knowledge about implications of 
other traffic participants’ behaviour and signals they possible convey to him. 
Additionally the model will have to make some assumptions about the reactions of other 
traffic participants on own behaviour, which are deduced from observing the driving 
style of other drivers or from their communications. 

The presented model has only two action alternatives to choose from: following and 
overtaking. The decision will be largely influenced by motivation factors, that is how the 
benefit of overtaking compares with the risk of an accident due to the driver approaching 
from behind. The risk estimation itself will also depend on motivation and driving skills 
of the model. 

If a decision has been made, the situation changes and thus the possible actions the 
driver can perform. In the state of following there are the possibilities to approach or 
brake, in the state of overtaking there are the possibilities of regulating the manoeuvre 

e.g. by accelerating or stopping the manoeuvre. 

To prevent the system from moving between different actions very often, the currently 
selected action is given high priority which can be overrun only by very urgent 
situational requirements. 

Motoric Execution of an Action 

The chosen action sequence can either initiate an operation like steering, braking, 
stepping on the gas pedal or handle blinker and flash or a more complex motor program 
according to the theory of generalized motor programs (Schmidt, 1976, 85), Motor 
programs like lane change or overtaking are actions composed of a well defined sequence 
of discrete operations. Usually these motor programs have been internalized by the 
driver through long time training and are executed in such a way that movements are 
time optimal. Nevertheless the driver controls execution and regulates if the result of the 
action does not conform to his expectations. In fig. 2 a typical diagram of the lateral 
deviation for a lane change with corresponding steering wheel angle and steering wheel 
rate of a well trained driver is shown (Jiirgensohn, 1997), 

Motor programs and discrete actions have a finite duration and cannot be interrupted 
arbitrarily. If there is the need to switch to an action that is more urgent than the current 
action, though, the preferred action cannot be started immediately, but only after a 
somewhat delayed interruption of the actual motor program due to neuromuscular lag. 
While in general actions are selected by explicit considering the alternatives, some 
actions are executed skipping the decision process, for example emergency braking in 
panic (reflex action). 

Figure 2: Schematic motor program of steering wheel angleX, steering wheel rate 
VX and the resulting lateral deviation q for lane change 

In order to have the model act in a way similar to human drivers, the driver model is 
supplied with simplified parametric motor schemes which have been observed at real 
drivers. Parameters are current speed, acceleration and relative distances. Both of the two 
action sequences available to the driver model are complex motor programs composed of 
other motor programs and several single vehicle operations. „Following“ includes 
„approaching“, “braking“ and „keeping the distance" as sub-procedures. „Overtaking" is 
composed of „change to left lane", „acceleration" and „change to right lane". 



As already mentioned, the psychological state of the driver is relevant for every stage of 
the cognitive process. In the presented driver model only a limited number of 
motivational factors are accounted for, namely 

• hurry 

• competence 

• daring 

While plausible assumptions how these attributes are connected with human behaviour 
are quite self-evident, valid quantitative interdependences will be almost impossible to 
formulate. Here a fuzzy-like description using linguistic variables could be considered, 
but it is also possible to presume simple functional relationships. (Domer & Hille, 1995) 
employ very elementary functions to simulate the influence of emotion and motivation 
on human acting in „artificial souls“ and are able to produce reasonable behaviour. 

For example, the desired speed (the overall goal) will increase with the individual’s 
hurry, its estimate of own driving skills, the degree of boldness and last but not least the 
type of vehicle. It is further assumed that estimation of safety margins and even 
perception of e.g. relative speeds are dependent on motivational factors. Because the 
evaluation of benefit and risk is also subjective, the decision process is thus very 
sensitive to the disposition of the individual. 

Intersocial Behaviour 

Intersocial behaviour can show itself in communication activities like operating blinker 
or flash or driving characteristics like jostling. In any case intersocial behaviour will give 
drivers hints about other driver’s intentions and a means to communicate own plans. 
Usually drivers observe other traffic participants carefully in order to predict their future 
actions and be able to react in an appropriate manner. This need for anticipation requires 
that the driver model be able to infer other drivers’ current goals and intentions from 
observed actions, in other words, it should be able to generate an internal model about 
the mental state of other road users. As a typical example of internal models the frequent 
classification according to distinctive features into categories like „speedster“, 
„slowcoach“ or „dare devir‘ which possess typical driving patterns can be mentioned. 

The Simulation Environment 

The drivers move in an artificial environment consisting of the road and fixed or moving 
objects herein. The course of the road is constructed by joining discrete street segments 
together. Objects can be positioned along the road and be assigned velocities and 
trajectories to move along. The whole environment can be configured for different 
driving situations by defining the course of the road and the participating objects with 
their characteristic movements. In addition it is possible to incorporate a human driver 
into the situation. The simulation environment becomes then a driving simulator and can 
serve to validate the driver model. 

Vehicles are manipulated by the driver model by operating gas, brake, steering wheel, 
blinker and flash, whose current values are fed into a single track model of the vehicle to 


calculate relevant variables of state like position, velocities and acceleration (longitudinal 
and transversal) for each time step. 

Figure 3 : Schematic representation of the simulation program 

Example Driving Situation 

The driver model is specified for the investigation of the situation „overtaking on a dual 
track highway**, see figure 4, At least 4 vehicles are necessary to describe this situation, 
which do not need the same degree of model accuracy, though. The following 
assumptions and simplifications are made for the described driving situation: 

• the leading vehicle (FI) drives at fixed speed on the rightmost lane, 

• the second vehicle follows FI, but starts overtaking with a given probability, 

• the third vehicle (EGO) contains the discussed cognitive driver model. EGO decides 
according to the perceived traffic situation, its motivation and its estimation of the 
behaviour of the other drivers between two cardinal driving programs, following or 
overtaking. Both driving programs consist of a complex set of action sequences (e.g. 
lane change or keeping a constant distance). If a situation rated as dangerous emerges 
(e.g. sudden swerve out of the previous vehicle or close drive up of the car behind) the 
overtaking manoeuvre can be eventually interrupted. There is social interaction with 
vehicle F4 that approaches quickly from behind on the left lane. 

• the vehicle on the left lane (F4) approaches the column from behind at high speed. 
According to its motivation the model cooperates with EGO or not. Decisions are 
possible only with respect to speed choice and communication, lane change has not to 
be considered. 



Figure 4 Investigated traffic situation 

The described driving situation allows the simulation of many traffic situations due to 
different initial conditions and driver characteristics. The following constellations were 
considered quite illustrative for the behaviour of the driver model: 

(1) large distance between F2 and EGO, EGO not motivated to drive fast 

(2) medium distance between F2 and EGO, F2 is driving slowly, both EGO and F4 
are motivated to drive at maximum speed in sport cars 

(3) same as (2), but EGO driving a slow car 
Discussion of Results 

In constellation (1) EGO chooses not to overtake, but first accelerates until the safety 
distance is almost reached, then the regulation program „follow in constant distance“ is 
started and continued. 

Figure 5 shows the simulated following behaviour plotted in the Ax-Ax- plane. The 
parabolic parts of the curve reflect the rule that whenever the model chooses to approach 
or to brake, it does so with constant acceleration or deceleration of the vehicle 
respectively. The irregular distribution of turning points is due to the stochastically 
varying perception of relative distance and velocity. Observed car following behaviour 
results in curves like those displayed in figure 6, which are in principle similar to the 
calculated ones. 


Figure 5: Simulated Following Behaviour in the Ax-Ax- plane 

Figure 6: Observed limit cycles of car following in the Ax - Ax- plane (modified after 
Hartwich, 1971) 

Constellation (2) is more interesting with respect to decision making. EGO first 
approaches and then decides to accelerate with maximum gas for overtaking in order to 
be able to achieve the desired speed and avoid the conflict with F4, who is approaching 
quickly and operating the flash. Due to the stochastic component which is introduced 
through perception errors further development of the situation is not definitely 
determined. If the estimation of relative speed and distance to F4 was on the safe side, 


overtaking can be finished. If not so, or if F2 suddenly swerves out, an accident can 

The third case demonstrates the anticipation of other traffic participants’ behaviour. 
Although the circumstances are equal to that of the previous example, F4 brakes much 
earlier and more vigourously supposing that EGO shows no inclination to yield and that 
the maximum speed of EGO’S vehicle is very much under F4’s speed. 


A cognitive driver model that accounts for motivation, decision making and goal directed 
acting as well as human perception characteristics and motor processes with emphasis on 
decision making has been developed in order to simulate critical traific situations. The 
assertion that the motivational disposition of drivers affects traific situations to a great 
extent could be confirmed. First results show that the chosen modelling method using a 
modified production system is suitable for the purpose to simulate individual car driving 

For an assessment of novel technologies like driver assistance systems, however, a more 
realistic modelling of perception and information processing will be necessary. For this 
objective modelling tools like ANNs or Fuzzy could be employed. 



Aasman, J. (1995): "Modelling Driver Behaviour in Soar", Proefschrift, Rijksuniversiteit 
Groningen, Leidschendam: KPN Research 

Anderson, J.R. (1993): ,Jlules of the mind“, Hillsdale, New Jersey, Lawrence Erlbaum 
Domer, D. & Hille, K. (1995): "Artificial Souls: Motivated Emotional Robots", IEEE 
Conference Proceedings, International Conference on Systems, Man, and 
Cybernetics; Intelligent Systems for the 21st Century. Vancouver, Volume 4, 1995, 
S. 3828-3832. 

Hartwich, E. (1971): ,J.angsdynamik und Folgebewegung des StraBenfahrzeugs und ihr 
EinfluB auf das Verhalten der Fahrzeugschlange“, Dissertation, TH Darmstadt 
Huhn, A. (1991): „DIPLOMA - Ein System fur verteilte Multiagentenplanung in einer 
Echtzeitumgebung“, Dissertation Universitdt Karlsruhe 
Jiirgensohn, T. (1997): „Hybride Fahrermodelle“ (Hybnd Driver Models, ZMMS- 
Spektrum, Band 4}, Sinzheim: Pro Universitate Verlag 
Jurgensohn, T.; Raupach, H.-C.; Willumeit, H.-P. (1994): "A Model of the Driver Based 
on Neural Networks", 25th FISITA Congress, Beijing, SAE - Paper, No. 945082, 

S. 161-166. 

Jurgensohn, T. & Willumeit, H.-P. (1997) "Modellierung von Mikro-Steuerstrategien mit 
Hilfe kunstlicher neuronaler Netze", A7Z, 99. Jahrgang/Nr.6, S. 348-352 
Kieras, D.E. & Meyer, D.E. (1995): „An overview of the EPIC architecture for cognition 
and performance with application to human-computer interaction^, University of 
Michigan, EPIC Report No. 5 (TR-95/ONR-EPIC-5 
Opwis, K. (1992); „Kognitive Modellierung“, Bern: Hans Huber 
Newell, A. (1990): „Unified Theories of Cognition, Cambridge^, Harvard University 

Schmidt, R.A. (1975): „A schema theory of discrete motor-skill leaming“. Psychological 
Review, 82, S. 225-260 

Schmidt, R.A. (1986): „A search for invariance in skilled motor behavior“, Research 
Quarterly for Exercise and Sport, 56, S. 225-260 
Willumeit, H.-P. 8l JUrgensohn, T. (1996): „Driver-models: A critical survey“, 26th 
FISITA Congress, paper. No. 08.01. Prague. 

Wolter, T.-M.; JUrgensohn, T.;Willumeit, H.-P. (1997): "Ein auf Fuzzy-Methoden 
basierendes Situations-Handlungsmodell des Fahrerverhaltens", ATZ, 99. 
Jahrgang/Nr.3, S. 142-147 


Online Traffic Simulation Based on Cellular Automata 

M. Schreckenberg and J. Esser 
Fachbereich Physik-Technologie 
Gerhard Mercator Universitdt 
47048 Duisburg / Germany 


Saturated capacities in traffic systems evoke increasing interest in simulations of complex 
networks serving as laboratory environment for developing management strategies. 
Especially for urban areas questions concerning overall traffic control have to be 
considered with regard to their impacts on the whole network. Modelling traffic flow 
dynamics using cellular automata allows to run large network traffic simulations with 
only comparative low computational efforts. We present a traffic simulation tool for 
urban road networks which is based on the Nagel-Schreckenberg Model. Arbitrary kinds 
of roads and crossings are modelled as combinations of only a few basic elements. 
Furthermore parking capacities are considered as well as circulations of public transports. 
The vehicles are driven corresponding to route plans or at random depending on the 
available data. The application of this network simulation covers investigations on the 
field of traffic planning as well as online simulations based on real-time traffic data as 
basis for dynamic traffic management systems. 


While for global aimed questions in the field of traffic management like dynamic vehicle 
routing it is self-evident to be considered on the whole network, even local aspects like 
changes in priority regulation at single crossings have to be investigated on large scales 
to avoid undesired global effects due to mutual correlations. Testing control strategies in 
realtiy is usually infeasible or at least extremely demanding in time and costs; therefore 
the application of simulation tools as laboratory environment is desired. Designing a 
simulation tool for network traffic requires to find an overall compromise: To enable 
collection of individual vehicle data like travel times or number of stops due to priority 
rules a microscopic approach is appropriate. In addition one is engaged to minimize 
running times, since e.g. for checking out different vehicle routing or traffic light control 
strategies complex scenarios have to be simulated again and again such that large 
computation times are very cumbersome and inefficient. Furthermore for online 
simulations based on real-time traffic data run speeds of at least real-time are a necessary 

We present a microscopic simulation tool for vehicular traffic in urban road networks [1], 
which was developed in the framework of the Northrhine Westfalia Cooperative FVU 
(Forschungsverbund Verkehrssimulationen und Umweltwirkungen) [2]. The underlying 
traffic flow dynamics is based on the cellular automaton introduced by Nagel and 
Schreckenberg [3]. This approach has proved as very efficient: Meanwhile it is possible 
to simulate even the whole German autobahn network in real-time [4,5] and within the 
project TRANSIMS [6] microscopic traffic simulations are carried out for the Dallas / 
Fort Worth area [7], 


The Cellular Automaton Model 

The Nagel-Schreckenberg model [3] was originally defined on a single-lane road. The 
road is subdivided into cells, which can be either empty or occupied by one vehicle. 
Every vehicle has a non-negative integer velocity. For one update of the road the 
following four steps are performed simultanously for all vehicles: 

1. Acceleration: v = min(v + 1, Vmax) 

2. Avoiding crashes: v = min(v, gap) 

3. Randomization: if rand() < p then v = max(v -1,0) 

4. Update: Each vehicle is advanced v cells. 

Here gap denotes the number of empty cells in front of a vehicle and Vmax the maximum 
velocity. The randomization (Step 3) takes into account, that individual driving 
behaviours for different vehicles result in non-deterministic dynamics of vehicle motions 
in reality. The resolution is taken to be 7.5m per cell representing the average space a 
vehicle takes in a jam, but can be suitably adjusted with regard to the considered problem 
[8]; using this cell width each time step in the simulation corresponds to one second in 
reality. For modelling traffic on multi-lane roads a set of rules for lane changes is added 
to the fundamental rules [9,10,11]. 

Detailed investigations showed that despite its simplicity the cellular automaton model is 
capable to reproduce macroscopic traffic flow features including realistic lane changing 
behaviours [9,12]. Also a continuous extension [13] of the basic discrete model and a 
modified set of rules resulting in more realistic breaking and accelerating behaviour [14] 
was developed to enable modelling of additional more detailed vehicle features (e.g. 
exhaust emission). 

Network Simulation Tool 

The road network is described as composition of nodes and edges representing crossings 
and roads. To avoid misunderstandings: Whenever we use the expression "edge" we refer 
to directed edges representing one direction of motion on a road; i.e. one road usually 
consists of two (oppositely directed) edges. The basic idea is to define different types of 
edges. This approach causes arbitrary complex crossing types to be represented as 
combinations of at most three basic edge types: On single-lane edges simply the basic 
cellular automaton is applied. In addition there can be connected turn pockets for each 
direction at the end of the edge. Especially at the end of multi-lane roads the lane 
changing behaviour strongly depends on the desired driving direction. For this purpose, 
multi-lane edges are subdivided into different regions. Finally also highways play an 
important role for a realistic description of urban traffic. Therefore an additional element 
is incorporated to simulate highway drive-ups and also crossings with large spatial 
extension. Transfer edges are one- or multi-lane edges which merge in a destination edge. 


I t 


Figure 1: Representation of a node as combination of different edge types. 

Fig. 1 shows a crossing as combination of different edge types. The numbers at the end of 
the edges represent signal groups of the corresponding traffic lights and will be used 
below. Optionally the first cells on a edge can be defined as crossing section (shaded in 
the figure) to model the spatial extension roughly. In this case for each possible direction 
it is predefined if vehicles entering the edge are put onto the first cell of the edge or 
beyond the crossing section. This also allows to consider cases of jammed crossing 
regions in combination with priority rules. 

One striking feature of urban road networks is the high density of intersections, which 
causes essential correlations among traffic streams on different edges. It is important for 
the simulation to have reahstic throughputs at the crossings, which again requires to 
consider complex priority rules and especially traffic lights in a realistic manner. Each 
edge in the network has a driving-direction-dependent leave flag at its end, by which 
vehicles may be prevented from advancing any further. Traffic lights are reahzed by 
switching these flags corresponding to predefined time tables. In addition a hierarchy can 
be defined to consider crossings where special directions are controlled by more than one 
light (e.g. occasionally active green-light arrows for right turns). To facilitate handling of 
traffic lights the simulation tool contains an editor for setting green and red phases within 
a switch matrix. 

Priority rules are realized by switching leave flags depending on vacant cells on other 
edges. In reality turning vehicles are often hindered in driving further e.g. by pedestrians 
crossing the destination road, which can reduce the throughput at certain crossings 
essentially. This effect is in case considered by blocking turning vehicles, for which 
driving on is allowed by traffic lights and priority rules, with predefined probabilities. 


Furthermore no-stop and stop priority rules are distinguished; for the latter it is 
additionally checked if vehicles had zero velocity standing on the last cell of the edge for 
at least one time step before they are allowed to drive on. 

One of the fundamental questions with regard to network traffic simulations concerns the 
question how the vehicles are guided through the network. In a randomly driven 
simulation at each crossing destination directions for the vehicles are choosen at random 
corresponding to turn counts. Running the simulation route plan driven means that each 
vehicle makes its turning decision corresponding to individual route plans, which are 
derived from origin-destination information. In the simulation the guidance mode is 
specified for each vehicle separately. Combination of both for example allows to check 
routing strategies on random background traffic representing typical traffic states. 

The vehicles are characterized by their maximum speed, length (number of occupied 
cells) and a probability for carrying out risky lane changes (i.e. lane changes without 
considering safety distances). In the actual version the deceleration probabilities are not 
individual, but attributed to the edges to tune throughputs at the crossings. There are two 
special kinds of vehicles: Vehicles can be guided periodically along predefined routes 
following time tables to simulate public transports. Additionaly bus respectively tram 
stops are defined, where these have to wait for a certain time and in case are taken out of 
the network for that time. The second type covers hindrances like accidents or road 
works. These are represented by vehicles of special length and zero maximum velocity. 

An essential characteristic of urban traffic is the fact that there are no predefined sources 
and sinks; rather vehicles are allowed to enter or leave the network at nearly every 
arbitrary position. For that reason sources and sinks can be linked to every cell in the 
network. At sources vehicles are characterized by type, guidance mode (see section 
Vehicle Guidance) and - if necessary -- additional information like if the vehicle follows 
a dynamic routing system or not. They are inserted corresponding to source rates or trip 
plans; the latter means that at the beginning a list of vehicles with departure times, 
vehicle characteristics and route plans is prepared. In addition flow check points as 
source-sink combination can be defined. These can be located everywhere in the network 
to adjust the number of vehicles to counting loop data. For these check points statistics 
covering number of added and deleted vehicles are collected separately. Route plan 
driven vehicles are taken out of the network after reaching the destination node. 
Probabilities for leaving the network at the next node are attributed to every edge for 
randomly driven vehicles. 

Especially for urban areas the consideration of parking capacities is desired. In the 
simulation these are considered as special source-sink pairs, which are fiUed up with 
regard to predefined capacities and from which — as far as they are not empty — vehicles 
can enter the network. This allows to develop strategies for parking control systems. It is 
also possible to define parking-lanes on the roads, which are used as usual lanes and in 
addition vehicles can be put on them marked as parking. This feature allows to consider 
capacity reduction of roads due to parking vehicles. 

A necessary requirement for traffic management systems is information about the traffic 
state. For this purpose, three instruments are incorporated in the simulation: At measure 
points local data like number and average speed of vehicles are collected separately for 
different vehicle types and averaged over predefined time intervals. Edges can also be 


grouped together representing measure regions , by which global measurings are carried 
out. The individual travel time is measured for every vehicle in the network. For more 
detailed information it is possible to guide vehicle probes through the network. This 
allows to collect additional point to point data for special routes (e.g, data with regard to 
the driving comfort like number and duration of stops due to priority rules and traffic 
lights). Furthermore overall network data like number of vehicles, average speed and 
edge usage are stored. 

The overall simulation tool consists of two principal processes. The master controller 
maintains the overall coordination: it checks static and dynamic network data read from 
the database for consistence and initializes the scenarios. During the simulation it 
receives and updates time dependent data like turn counts, handles the simulation output 
including in case updates of the graphics and if necessary provides dynamic data to the 
routing module, which updates route plans for the vehicle guidance system. The actual 
network dynamics is carried out by the micro simulation process; i.e. vehicle motions, 
source and traffic light updates and data collection for statistics.The advantage of this 
subdivision is twofold: On the one hand it speeds up the simulation, since in addition the 
micro simulation can be parallelized independently. Furthermore in practice it facilitates 
overall handling, because it is possible to rearrange e.g, graphical output or data formats 
without caring about the complex structure of the micro simulation. 

Simulation of the Duisburg road network 

Currently traffic simulations for the inner city of Duisburg are developed applying the 
simulation tool. Here we focus on presenting the present traffic state using online traffic 
data stemming from induction counting loops provided by the traffic control center of the 
municipality and updated every minute. 


Figure 2: The simulated road network of Duisburg. 

Fig,2 shows the considered network covering 107 nodes, 280 edges; the total lane length 
amounts to about 165 kilometers (22059 cells). Inclusive online source and turn count 
updates, priority regulation and data collection for statistics it is possible to simulate in 20 
minutes on a PC (Pentium P133) a whole day of typical traffic. 


Figure 3: Number and average speed of vehicles in the Duisburg network during an 

online simulation. 

Fig.3 shows the number of vehicles and the average speed during an online simulation 
for the time interval from 12.23am on the 1st until 12.22am on the 2nd of April in 1997. 
The extended minimum in the night and the two maxima at about Sam and 5pm are well 
dicemible. Due to missing origin-destination information with appropriate time resolution 
the simulation is driven randomly, while turn counts are derived from real-time traffic 
data. The present traffic state for the city of Duisburg is presented in the Internet [15]. 


We presented a simulation tool for urban traffic. The microscopic dynamics based on the 
Nagel-Schreckenberg cellular automaton allows to simulate large networks in multiple 
real-time. Within the network model complex crossings inclusive realistic traffic lights 
and priority rules are considered as well as parking capacities and circulations of public 


transports. In combination with real-time traffic counts this tool serves as useful 
laboratory environment for designing and checking dynamic traffic management systems. 


[1] J. Esser, Microscopic Simuation of Urban Traffic based on Cellular Automata, to 

appear in hit. J. Mod. Phys. C 

[2] NRW'FVU Home Page, 


[3] K. Nagel and M. Schreckenberg, A cellular automaton for freeway traffic, J. Physique 

1,2 ,2221 (1992) 

[4] M. Rickert and P. Wagner, Parallel real-time implementation of large-scale, route- 

plan-driven traffic 

Simulation, Int. J. Mod. Phys. C 7(2), 133 (1996) 

[5] PAMINA Home Page, 

[6] TRANSMS Home Page, http://studguppy.tsasa, 

[7] M. Rickert and K. Nagel, Experiences with a simphfied microsimulation for the 

Dallas / Fort Worth area, Int. J. Mod. Phys. C 8 , June 1997 

[8] Ch. L. Barrett, S, Eubank, K. Nagel, S, Rasmussen, J. Riordan and M. Wolinsky, 

Issues in the Representation of Traffic Using Multi-Resolution Cellular Automata, 
Los Alamos National Laboratory Technical Report: LA/UR 95-2658 

[9] M. Rickert, K. Nagel, M. Schreckenberg and A. Latour, Two Lane Traffic Simulation 

using Cellular Automata, Physica A 231,534 (1996) 

[10] K. Nagel, P. Wagner and D. Wolf, Lane-changing rules in two-lane traffic 

simulation using cellular automata: II. A systematic approach, in preparation 

[11] P, Wagner, K. Nagel and D. Wolf, Realistic multi-lane traffic rules for cellular 

automata, Physica A 234,687 (1996) 

[12] P. Wagner, Traffic Simulations Using Cellular Automata: Comparison with 

Reality, in Traffic and Granular Flow edited by D. E. Wolf, M. Schreckenberg and A. 
Bachem, World Scientific Singapore 1996 

[13] S. KrauB, P, Wagner and C. Gawron, Continuous limit of the Nagel-Schreckenberg 

model, Phys. Rev. E , 54,3707 (1996) 

[14] S. KrauB, P, Wagner, C. Gawron, Metastable states in a microscopic model of traffic 

flow, to appear in Phys. Rev. E, 55,5 (1997) 

[15] Duisburg Onhne Simulation Home Page, http://www.comphys.uni- 


Intelligent Speed Adaptor (ISA) 


Karel Brookhuis 
Dick de Waard 

Centre for Environmental and Traffic Psychology 
University of Groningen, The Netherlands 

This manuscript concerns a study on the effects of a prototype intelligent speed adaptor 
(ISA) in actual traffic. Twenty-four subjects were included in a test of effects of feedback 
on speed behaviour, mental workload and acceptance. Subjects drove an instrumented 
vehicle in normal traffic on various types of roads with different speed restrictions and in 
a driving simulator, interacting with other traffic as well as under different- speed 
restrictions. Subjects completed both test parts twice, half of the subjects receiving 
feedback in the second trial (experimental group), half of the subjects not (control 
group). The groups differed in several ways, the most important being adaptation of their 
behaviour under feedback. Subjects in the experimental group behaved more according 
to traffic rules, in particular speed limits, than subjects in the control group. No 
differences in workload were found. Several types of feedback were tested to acceptance 
and rated differentially. 


The Dutch Ministery of Transport and PubUc Works (‘AVV’, Transport Research 
Centre) has initiated a project (‘Automatisering Rijtaak’, automatizing the driving task) 
whose objective is to delineate standardized test methodologies for assessing safety 
effects of advanced transport telematics (ATT). This report presents a study in which the 
general methodology, as proposed by Verwey, Brookhuis and Janssen (1996), is followed 
to investigate the effects of feedback on speed behaviour, mental workload and 
acceptance of telematics applications developed to regulate speed. 

As put forward by Kuiken and Heijer (1995) in their theoretical considerations regarding 
driver support systems and traffic safety in general, this type of systems aims to facilitate 
driving performance. One of the functionalities of such a system may be to facilitate the 
task pc rformance of drivers by providing real-time advice, instruction and warnings. The 
latter type of systems are usually also described by the term ‘co-driver systems’. Co¬ 
driver systems may operate in advisory, semi-automatic or automatic mode (e.g. 
Rosengren, 1995), which concern, for instance, speed regulation systems. Speed 
regulation as driver support system per se is different from speed regulation in (fully) 
automatic form, such as in autonomous intelligent cruise control (AICC) which is 
especially developed for and useful on motorways. In advisory mode a speed regulation 
system is suitable for any environment and situation. 

According to Kuiken and Heijer (1995) the underlying assumption behind presenting 


drivers with specific information about the driving environment or supporting them in 
performing certain aspects of their tasks, is that their behaviour will change for the 
better. For example, if you warn a driver that his/her speed is too high, s/he will decrease 
the speed (see De Waard et al., 1994, De Waard and Brookhuis, 1995a), Thus far, little is 
known about the effects of driver information and support systems on traffic safety. 
Kuiken and Heijer (1995) recommended to develop, at the first stage, guidelines that will 
allow a systematical assessment of current and future in-vehicle support systems. In 
assessment procedures developed so far (Verwey, Brookhuis and Janssen, 1996) the 
human driver is considered in relation to the traffic environment, in terms of their 
capacities and limitations, terms of the tasks that need to be performed, and in terms of 
the errors (i.e. traffic rule violations) they might make. 

Violation of traffic rules is obviously unwanted because of its relationship to accident 
causation. Well-known deviations from the norm, related to accident causation are close 
following, driving while intoxicated, speeding, running red traffic lights and neglecting 
stop signs (cf. Smiley and Brookhuis, 1987, Rothengatter, 1991). Speeding has a 
relatively well-known relationship to accident involvement. Salusjarvi (1981) reported an 
almost linear relationship between change in number of accidents and change of mean 
speed in Finland. Joksch (1993) found that the probability that a driver is killed increases 
with increased speed fits regression models with exponents as high as four, meaning that 
each km/h faster exponentially increases fatality risk. An average reduction of as little as 
2 km/h to 5 km/h could lead to a reduction of 10% up to 30% in injury accidents (for a 
more elaborate overview, see Rothengatter, 1993). However, these effects seem less 
evident on motorways where the maximum speed (120 km/h in the Netherlands) is 
allowed than on, for instance, rural roads where a speed limit of 80 km/h is imposed 
(Wegman et al., 1991). One approach to influencing unwanted traffic rule violations, in 
particular speeding, is embedded in a project in the European DRIVE (Dedicated Road 
Infrastructure for Vehicle safety in Europe) initiative, the DETER (Detection, 
Enforcement & Tutoring for Error Reduction) project. DETER is concerned with the 
development and testing of prototypes of on-site, in-vehicle and integrated monitoring, 
tutoring and enforcement systems, among which Intelligent Speed Adaptation (ISA, see 
also De Waard and Brookhuis, 1995b). 

Intelligent Speed Adaptation (ISA) 

Through recent legislation in the Netherlands, the maximum driving speed is restricted 
by a speed limiter in the heavier types of lorries and coaches. As a consequence of this 
legislation the number of heavy vehicles in which the maximum driving speed is 
restricted will only increase in the coming years (Alink, 1992). The effect of these 
devices on fuel consumption, noise, air pollution, wearing of the tires and traffic safety is 
expected to be mainly positive (e.g., Almqvist et al., 1991, Van der Mede, 1992). The 
obvious restriction of the speed limiter as mandatory now, is that it only prevents driving 
above ^ maximum allowed driving speed of heavy goods haulage vehicles, and is 
independent of local limit in a specific road environment. An intelligent speed adapter 
(ISA) takes into account local restrictions, and adjusts the maximum driving speed to the 
posted maximum speed. When it comes to restriction of driving speed of private 
vehicles, the use of intelligent speed limiters is to be preferred due to further 


differentiation of speed limits for private cars compared to heavy goods vehicles. A non- 
intelligent speed limiter is set at the maximum allowed driving speed for motorways (120 
km/h), while the majority of a speed limiting system’s safety benefits can be attained on 
‘A’-class roads (limit 80 km/h) and in built-up areas (50 km/h). 

Intelligent speed limiters require vehicle-infrastmcture communication. For exchange of 
information about the road class or local restrictions, one option is to equip traffic signs 
with transmitters, beacons or tags. At the moment passes such a sign, the new speed 
limit is to be conveyed to the vehicle in one way or the other. Examples of experimental 
car-infrastructure communication can be found in Sweden (Nilsson and Berlin, 1992, 
Palmquist, 1993, Persson et al., 1993) and the Netherlands (De Waard and Brookhuis, 

In general, a standard speed limiter is an intrusive system that restricts speed control, i.e, 
the device sets the maximum possible driving speed. An intelligent speed limiter is able 
to set this maximum speed in accordance with local posted legal limits. A less intrusive 
device is a system that provides the driver with feedback about local limits, for instance, 
on the accelerator. An active gas pedal increases the counterforce if the driver is driving 
too fast (Godthelp and Schumann, 1991). In principle, such a speed limiter leaves the 
driver in control, while the feedback provided in case of a speed violation is highly 
compelling. Moreover, the feedback is provided in the tactile modality, i.e. the same 
modality through which action has to be undertaken to observe the rules again. Feedback 
could also be presented in the visual modality, e.g. a warning light or message in the 
dashboard, or auditory, an acoustic signal or vocal message. On the one hand this type of 
feedback seems less intrusive than the feedback an active gas pedal provides because 
these warnings can easily be ignored. On the other hand, it might be that the social effect 
of being warned in the presence of other passengers is a more severe chastisement and 
therefore less preferred. Anyway, acceptance of the feedback type systems can be 
expected to be higher than of a strict, standard speed limiter, because behaviour is less 
restrained. Results from a questionnaire survey demonstrate that a slight majority of 
people consider an indicator of speed-limit violations useful, while only 35% of the 
respondents were of that opinion with respect to a speed limiter (Hagen and Fokkema, 
1990). Another advantage of the feedback systems opposed to the speed restricters, is 
that speed violations can sometimes be advantageous for traffic safety, e.g,, there are 
instances in which a (perhaps misjudged) critical overtaking manoeuvre is faster and 
safer performed if the limit is exceeded. 

Development of a prototype intelligent speed adaptor 

An effort is now undertaken in the Netherlands to develop a prototype intelligent speed 
adaptor that leaves the driver in control. For a start, this resulted in the development of a 
continuous feedback display in close proximity of the speedometer indicating the current 
speed limit, quite similar to the CAROSI system (Nilsson and Berlin, 1992). Central part 
of the CAROSI (CAr ROadside Signalling) system is the instrument panel, which 
includes not only standard displays such as the speedometer, but also contains sections 
on which roadside information is displayed. Amongst these is the posted speed limit, 
which is displayed below the speedometer. Major advantage of giving feedback by 


displaying the speed limit inside the car is that this information remains continuously 
visible instead of only being visible at the moment a sign is passed. This might reduce 
speeding because of general unawareness of the limit, which is not unconunon in the 
Netherlands (e.g., Steyvers et al., 1992, De Waard et al., 1995). 

A special version of the latter type of feedback display is developed for implementation 
in an experimental test-vehicle. Whenever the speed , limit is exceeded the colour in 
which the speed limit is displayed changes from green (‘normal/neutral’) to amber, or 
yellow, (‘warning*). In case the speed limit is exceeded by 10% the colour changes from 
amber to red (‘violation^), and then an additional, auditory warning message is issued 
(see also De Waard et al., 1994, De Waard and Brookhuis, 1995ab). The systems are 
integrated in the existing DETER system (see De Waard and Brookhuis, 1995a), which 
is developed as an open system to integrate driver monitoring and feedback 
(sub)systems. In the present experiment this set-up is tested, letting subjects drive the 
test-vehicle with and without the feedback systems. 

Additionally, an active gas pedal is tested as a medium for haptic feedback in case of 
speed limit violations, exceeding by 10%, in a driving simulator with the same subjects, 
in a cross-over design. All modes of feedback are studied to effects on behaviour, mental 
workload and acceptance. 



Twenty-four subjects were selected from the TRC subjects’ pool. They were paid for their 
participation in the test on effects of feedback concerning speed restrictions and 
violations in the institute’s instrumented test vehicle and driving simulator. 

Experimental design and procedure 

Upon arrival at the institute, subjects were first informed about the purpose of the study 
in general terms, and more specifically what was expected from them. Heart rate 
electrodes were fixed at appropriate places on the chest. Then a general questionnaire 
concerning personal data, such as driving experience and their ideas about speed 
restriction systems, was completed after which half of the subjects drove the 
instrumented test vehicle over a fixed route and then performed the simulator test, half of 
the subjects vice versa. Each of the test-rides consisted of two parts, first the baseline 
measurement, then after a short break, either the test ride with feedback or the control 
ride. Half of the subjects received feedback, half were in the control condition. The 
subjects were instructed to drive as they would normally do, in their own cars, and were 
told in the second ride that behaviour with respect to law compliance was monitored and 
feedback was possibly given. 

The test rides in the instrumented test vehicle were in normal traffic, under various 
conditions. Subjects were guided by sampled vocal route guidance messages that were 
triggered by the investigator for reasons of proper timing. They were led over a varied 
route that included sections of motorways, A-roads and built-up areas, with speed 
restrictions of 50, 70, 80, 100 and 120 km/h. The test ride in the simulator was a copy of 

the situation used for the DETER experiment (De Waard, 1995a), other traffic being 
present, speed restrictions being 50, 80, 100 and 120 km/h. 

After each of the (four) test rides, subjects were requested to complete questionnaires 
concerning perceived workload and subjective driving quality. At the end of the whole 
test, subjects completed a general questionnaire again, asking for their ideas with respect 
to speed resticting systems again. 

Test vehicle 

For the test rides on the road, the instrumented test vehicle (a Renault 19 RT) was used 
(for a description of the instrumentation, see Kok and Brookhuis, 1995), Data regarding 
speed and steering wheel movements were sampled on-line at 10 Hz by an industrial PC 
with a Pentium™ processor and were stored on harddisk. Information regarding speed 
Limits and stop signs were conveyed to the car by means of a microwave system adapted, 
not especially developed, for this specific purpose, by an electronics manufacturer. An 
antenna (MIDS SAA-4S) / reader (MIDS VI) combination was instaUed on top 
respectively inside the vehicle, while traffic signs were equipped, at the back side, with 
tags (labels, MIDS ML20). This microwave system operates at a frequency of 2.4-2.5 
GHz. Positioning of the tags at the back side of traffic signs ensured that information 
regarding local speed limits did not enter the vehicle too early, i.e. not before a speed 
limit zone actually was entered. In this way it was avoided that drivers were ‘accused’ of 
speeding by the system when the new restrictions had not taken effect yet in geographical 
sense. Feedback about local speed limits was provided only to subjects in the 
experimental condition during feedback trials (i.e., the second series of trials). On the 
road the display indicating the speed limit in green (when keeping to the limit), amber 
(when exceeding the limit but less than 10%) or red (when exceeding the limit by more 
than 10%). The vocal message ‘You are driving too fast, the local limit is was given 
at the instance the display’s colour changed from amber to red. Headway feedback was 
not given in this experiment. 


Subjects performed two rides in the institute’s driving simulator (Van Wolffelaar and 
Van Winsum, 1992, Van Winsum and Van Wolffelaar, 1993). Subjects drove in the 
same condition as in the on-the-road test, i.e. subjects in the control condition did not 
receive feedback regarding detected violations. Subjects in the experimental condition 
were provided with haptic feedback about the (posted) speed limit when exceeding the 
limit, but this feedback was given in the second trial only. Haptic feedback consisted of a 
continuous counterforce on the gas pedal. The counterforce was dependant upon the 
extent to which the limit was exceeded, and was equal to: 4 x (speed - speed limit (both 
in m/s)) Newton. If the speed limit was exceeded by more than 10% the same vocal 
message was given as in the on-the-road trials. The same testbed as described in De 
Waard et al. (1994) was used with one exception: in the previous simulator experiment 
the last 3 minutes of driving of each trial consisted of tedious dual-carriageway driving. 

Subjective measures 

During the whole test, performance parameters and subjects' heart rate were registered as 


measures of workload, whereas after each ride the self-report scale RSME (Rating Scale 
Mental Effort, see Zijlstra and Meijman, 1989) was completed. In previous tests, both 
performed in the simulator (De Waard et al., 1994) and on-the-road (De Waard and 
Brookhuis, 1995a,b), results showed that the tutoring messages increased driver mental 
load in a moderate way. Both reduced heart rate variance and increased self-report scores 
indicated increased effort. 

A subjective driving quality score (driving quality scale, see Brookhuis et al, 1985) was 
also derived to check on subject’s personal impression about driving circumstances. 

Before entering the experiment and after the whole session, a score on nine items 
regarding acceptance of tutoring and enforcement systems (Van der Laan et al., 1997) 
was derived from the subjects. After the test rides more detailed questions about 
acceptance of the different feedback systems were posed. 


A total of 24 subjects participated in the test rides. Eleven subjects were female, average 
age 34.6 {sd 8.7), average mileage 10.000 km/year {sd 9.000), thirteen were male, 
average age 34.7 {sd 10.7), mileage 21.000 km/year {sd 19.000). On average, both males 
and females had held a drivers’ licence for 15 years. Five subjects did not (fully) 
complete the test due to simulator sickness, but were as much as possible included in the 

Although the number of detected violations during the second (feedback) series of trials 
is lower in the experimental group, this effect does not attain statistical significance 
(Group X Trial interaction: F(l,18) = 1.33, NS). The difference in the number of speed 
violations between the two test facilities is significant (F(l,18) = 25.4,p< 0.001), drivers 
more frequently violated the speed limit on-the-road. The extent to which the speed limit 
was exceeded was higher in the simulator (F(l,18) = 24.3, p< 0.001). During the 
feedback-trial the extent to which the limit was exceeded was on average lower (with the 
exception of the second on-the-road trail for the control group), but this effect is not 
significant (Group x Trial interaction F(l,18) < 1, NS, Group x Trial x Testfacihty 
interaction, F(l,18) = 2.7, NS), 

During the test rides it was noted that some of the subjects in the experimental group 
obviously ‘played’ with the feedback display to test its behaviour or use it to stay in the 
marginal area (amber). It was therefore decided to determine a new parameter, the 
propordon of time violating the limit. This parameter does not discretely sum up the 
number of times the limit is exceeded, but reflects the time the driver is not complying. 
Two parameters were determined, the proportion of time driving above the limit, i.e. the 
time the display was or would have been amber or red, and the proportion time driving 
above the limit -i- 10%, i.e. the time the display was or would have been red and an 
auditory message was or would have been issued. The "would have been'-condition is for 
the control group, the experimental group actually received the described feedback. From 
figure 1 it is clear that as much as 20 to 25% drivers are speeding in the strict juridical 
sense. Between 5 and 10% of the time they are driving faster than the speed limit plus a 
10% margin. The effect of the feedback system is only significant for the latter parameter 

(Group X trial interaction: F(l,22) = 1.50, NS for amber/red display, F(l,22) = 9.39, 
/ 7 < 0.01 for red-display feedback). 

Percentage time driving above limit 

1 2T 1 2T 

AMBER or RED (> LIMIT) RED (> LIMIT 4- 10%) 

Figure 1 Time driving above the limit and above the limit plus a 10% margin per group 
and Trial (on~the-road data only). 1 = Trial 1, 2T = Trial 2 (T= Tutoring feedback, 
provided to subjects in the experimental group only). 


Task performance was measured by the vehicle parameters SD of the steering wheel 
movements (on straight sections in the road), average lateral position on the road and SD 
of the lateral position on the road. During the second simulator trial swerving as 
measured by the SD of the lateral position increased in both groups (F(l,18) = 7.59, 
p<0.05). No significant changes in task performance parameters as a result of the 
feedback system were found. 

Self-report ratings on the Rating Scale Mental Effort (Zijlstra and Meijman, 1989) show 
a clear effect of test environment (F(l,17) = 26.6, p< 0.001). Previous comparison of 
trials between on-the-road driving and simulator tests had also shown increased mental 
effort for task performance in the simulator (De Waard and Brookhuis, 1995a). There is a 
tendency to evaluate the amount of invested effort in the second trial as lower (F(l,17) = 
4.04, /?<0.10). The interaction Group x Trial approaches significance for the car data 
only (F(l,22)= 3.98,p<0.06). 


One physiological measure was registerd, the subjects’ heart rate, or ECG. Data were off¬ 
line checked for artifacts. Different problems varying from simulator sickness to cable 
problems led to incomplete data sets, four subjects had to be deleted from the car data 
set, eight subjects from the simulator set, ECG analysis resulted in parameters: IBI (inter¬ 
beat-interval) and variation coefficient (normalized heart rate variability in the time 
domain). Both parameters are sensitive to general arousal and effort. There were no 
effects of feedback on any of the heart rate parameters. Only self-reported mental effort 
during the trials performed on-the-road are indicative for a slight increase in mental 
workload. Physiology and task performance did not reflect any changes as a result of 

Control variables 

Control variables for driving performance are self-report ratings about driving quality 
and average and sd of driving speed. After each trial subjects were asked to compare 
their performance to normal driving on the driving quality scale (Brookhuis et al., 1985). 
The scale has a range from -100 (T have driven exceptionally poorly’) to -i-lOO (T have 
driven exceptionally well’), while a rating of ‘0’ stands for T have driven normally’. 
Analysis shows an interaction between trial and testfacility (F(l,17) = 9.1, p < 0.01). 
Rated driving quality decreases slightly in the second on-the-road trial, while it improves 
during the second simulator trial. Probably due to relatively low familiarity driving 
quality in the simulator is rated lower than in a real car (F(l,17) = 17,8 ,p < 0.001). 

Speed was determined on the motorway (speed limit 120 km/h), on a rural 50 km/h 
speed-limit road (both on-the-road) and in the simulator on a dual carriageway (speed 
limit 100 km/h). The difference between trial 1 and 2 is significant for both average 
speed (F(l,17) = 6.0, p < 0,05) and variability in driving speed (F(l,17) = 10.4, p < 
0 . 01 ). 

The control variables show that rated task performance as well as average driving speed 
on high speed roads increase with time-on-task (i.e., trials). Driving speed and variability 
(sd) in driving speed decrease mostly in the second trial, in particular in the experimental 

Ratings on the acceptance items (Van der Laan et al, 1997) did not differ significantly 
between the control and experimental group (F(l,22) < 1, NS), nor did they differ 
between before and after-test measurements (F(l,22) < 1, NS). Usefulness of a speed 
compliance feedback system is rated negatively, -1.04, on a scale from -2 to 4-2. Such a 
system is rated slightly positive in terms of being satisfying (-1-0,16) on a scale with the 
same range. 

After the test rides, subjects from the experimental group were asked to rate the effect the 
feedback system had had on their speed choice. Evaluation in terms of good/bad and 
pleasant/not pleasant are given in table 1. All drivers had been asked whether they would 
accept the different types of systems The display-only turned out to be the best accepted 
system, followed by haptic pedal feedback (experimental group) and auditory feedback 
(control group). 


Table 1. Evaluation of the different types of feedback systems (N^ll). 


‘Good’ ‘Not Good’ ‘Pleasant’ ‘Not pleasant’ 

Auditory (car) 73% 

Visual (Display) (car) 91% 

Auditory (simulator) 73% 

Haptic (simulator) 40% 

In table 2 the ‘preferred syst<— - c.- . ^ . 

experienced the systems (the experimental group). The feedback display is clearly 
favourite, followed by pedal counterforce. 













is given in 

order of preference 

of the group who had 

Table 2. Preferred feedback system 






Pedal counterforce 



Voice + pedal 



Voice + display 






Display + pedal 




In fact this is the third experiment on 

the road in which feedback in one way or the other 

was given to subjects about violations of traffic rules, in this case primarily speed. 
Similar to the first two experiments (De Waard et al., 1994, De Waard and Brooldiuis, 
1995a), effects of feedback are present again, but not as large as in previous experiments 
and slightly different. Specifically the effect on number of speed violations was much 
lower in the present study. The main difference, and probably largely explaining the 
lower effects, in driving condition for the subjects this time was a continuous feedback 
on (posted) speed limits and rule compliance, as opposed to incidental warnings in case 
of violations in the first two studies, A clear reduction in speed in the experimental 
condition in those cases where subjects clearly tended to violate the limit, i.e. in the 50 
km/h zones, was found. Furthermore, from the data on percentage of time driving above 
limit (see figure 1) it is clear that, at least some subjects used the continuous feedback to 
keep their speed in the margin of ‘limit to limit4-10%’ (amber). 

From the acceptance data (see tables 1 and 2) it follows that acceptance very much 
depends upon feedback system, continuous feedback was accepted best of all means of 
feedback by far. The ratings for the continuous visual feedback were unusually high and 
can maybe even considered as (highly) appreciated. The system that is most proximate to 
an actual speed-restrictor, haptic feedback, is according to the subjects, surprisingly 
enough not the most influential on behaviour. Differences between the two test 
environments (simulator <==> car) make a comparison of effects on actual behaviour of 


haptic versus visual display feedback difficult (coufounded). 

An new, unexpected effect of the compound feedback was found, a significant reduction 
in speed variation. One of the reasons is the earlier mentioned use of the amber to stay in 
the margin of ‘limit to limit+10%’. The implication of this finding is that less variation 
in driving speed could help to harmonize traffic, which is one of the candidate tools to 
reduce the number of accidents (see also Brookhuis and Brown, 1992). 

No effects on workload were found in this study, again contrary to the first two 
experiments as mentioned. However, in the latter studies the (slight) effects were 
marginally significant, while in the present data the (slight) effects demonstrated in either 
of the two measures of mental load did not attain significance. The implication of these 
findings, in line with Verwey, Brookhuis and Janssen (1996), is that before 
implementing telematics systems, in principle workload effects should be measured, just 
to be sure, but the type of systems tested so far are not implying alarming effects. 

Finally, the conclusions of this study can be summarized succinctly in three statements: 

Giving feedback about speed limit compliance/violation is effective 

Giving continuous including compliance feedback is effective and appreciated 

Giving continuous including compliance feedback could help harmonizing traffic 


Alink, G.M.M. (1992). Snelheidsbegrenzer voor de vrachtauto. Verkeerskunde, 3,22- 24. 

Almqvist, S, Hyd6n, C. and Risser, R. (1991). Use of speed limiters in cars for increased 
safety and a better environment, Transportation Research Record 1318. Washington 
D.C.: Transportation research Board. 

Brookhuis, K.A., de Vries, G., Prins van Wijngaarden, P. and O’Hanlon, J.F. (1985). The 
effects of increasing doses of meptazinol (100, 200,400 mg) and glafenine (200 mg) on 
actual driving performance. Haren, The Netherlands: Traffic Research Centre, 
University of Groningen. Report VK 85-16. 

Brookhuis, K.A., Brown, I.D. (1992). Ergonomics and road safety. Impact of science on 
society, 165, 35-40. 

De Waard, D., Brookhuis, K.A., Van der Hulst, M. and Van der Laan, J.D. (1994). 

Behaviour comparator prototype test in a driving simulator (Report 

2009/DETER/Deliverable 10 (32IB)). Haren, The Netherlands: University of Groningen, 

Traffic Research Centre. 

De Waard, D. and Brookhuis, K.A. (1995a) Experimental test of the integrated DETER 
system. In K.A. Brookhuis (Ed), Integrated systems. Results of experimental test, 
recommendations for introduction (Report V2009/ DETER/Deliverable 18 (331 A, 
440AB)). Haren, The Netherlands: University of Groningen, Traffic Research Centre. 


De Waard, D. and Brookhuis, K.A. (1995b). Een intelligent toezichtsysteem in de auto. 
Verkeerskunde, 11, 16-20, 

De Waard, D,, Jessurun, M., Steyvers, F.JJ.M,, Raggatt, P.T.F., Brookhuis, K.A. (1995). 
Effect of road layout and road environment on driving performance, drivers’ physiology 
and road appreciation. Ergonomics, 38, 1395 - 1407. 

Godthelp, H. and Schumann, J. (1991). The use of an intelligent accelerator as an 
element of a driver support system. In Proceedings of the 24th IS AT A International 
symposium on automotive technology and automation, p, 615 - 622. Croydon, 
England: Automotive Automation Limited. 

Hagen, J.A.M. and Fokkema, H.J. (1990). Snelheidsbegrenzers in personenauto*s. 
Rapport TT 90-4. Veenendaal, Traffic Test. 

Joksch, H.C. (1993) Velocity change and fatality risk in a crash - a rule of thumb. 
Accident Analysis & Prevention, 25, 103-104. 

Kok, K. and Brookhuis, K.A. (1995), Specifications of the integrated system. Haren, The 
Netherlands: Traffic Research Centre, University of Groningen. Report 
V2009/DETER/Dehverable 19 (331B, 340B, 410A, 420A, 430A) to the commission of 
the European Union. 

Kuiken, M.J., Heijer, T. (1995) Driver support systems and trajfic safety: theoretical 
considerations. Report R-95-68, Leidschendam: Stichting Wetenschappelijk Onderzoek 
V erkeers veiligheid. 

Nilsson, L. and Berhn, M, (1992). Driver attitudes and behavioural changes related to 
presentation of roadside information inside the car. A pilot study of the CAROSI 
system. Report 698A, Linkoping, Sweden: Swedish Road and Traffic Institute (VTl), 

Palmquist, U. (1993). Intelligent Cruise Control and roadside information, IEEE Micro, 

February 1993, 20 - 28. 

Persson, H., Towliat, M., Almqvist, S., Risser, R. and Magdeburg, M. (1993), 
Hastighetsbegrdnsare i bil. Fdltstudie av hastigheter, beteenden, konflikter och 
fdrarkommentarer vid horning i tdtort. Technical report. Lund, Sweden: University of 
Lund, Lund institute of technology. Department of traffic planning and engineering. 

Rosengren, L.G. (1995), Driver assitance and co-operative driving, hi: ERTICO (Ed.) 
Towards an intelligent transport system. Proceedings of the First World Congress on 
Advanced Transport Telematics and Intelligent Vehicle Highway Systems. London: 
Artech House, 1613-1622. 


Rothengatter, J.A. (1991) Automatic policing and information systems for increasing 
traffic law compliance. Journal of Applied Behavior Analysis, 24, 85-87. 

Rothengatter, J.A. (1993) De risico's van rijplezier (The risks of driving pleasure). 
Inaugural lecture, University of Groningen. 

Salusjarvi, M. (1981) Speed limits and traffic accidents in Finland. Proceedings of the 
OECD Symposium on The Effects of Speed Limits on Traffic Accidents & Transport 
Energy Use, An Foras Forbartha, Dublin. 

Smiley, A. and Brookhuis, K.A. (1987) Alcohol, drugs and traffic safety. In J.A. 
Rothengatter and R.A. de Bruin (Eds.) Road users and traffic safety. Van Gorcum, 

Steyvers, F.J.J.M., De Waard, D., Jessurun, M., Rooijers, A.J. and Brookhuis, K.A. 
(1992). Een voorstudie naar de effecten van snelheidsbeperkende maatregelen op het 
rijgedrag op 80-km wegen (Rapport VK 92-05). Haren: Rijksuniversiteit Groningen, 
Verkeerskundig Studiecentnim. 

Van der Laan, J.D., Heino, A. and De Waard, D. (1997), A simple procedure for the 
assessment of acceptance of advanced transport telematics. Transportation Research - 
PartC: 5, 1-10. 

Van der Mede, P.H.J. (1992). Snelheidsbegrenzer voor vrachtauto's in 
bedrijfseconomisch perspectief. Verkeerskunde, 5, 24 - 27. 

Van Winsum, W. and Van Wolffelaar, P.C. (1993). GIDS Small World simulation. In 
J.A. Michon (Ed.) Generic Intelligent Driver Support System. London: Taylor & 
Francis Ltd. 

Van Wolffelaar, P.C. and Van Winsum, W. (1992). A new driving simulator including 
an interactive intelligent traffic environment. In L. Olausen (Ed.), Vehicle navigation, 
information systems. Oslo: Moberg & Helli, 499 - 506. 

Verwey, W.B,, Brookhuis, K,A., Janssen, W.H, (1996). Safety effects of in-vehicle 
information systems. TM-96-C002. Soesterberg: TNO Human Factors Research 

Wegman, F. et al. (1991) Voor alle veiligheid (For all safety / To be safe). SDU, The 

Zijlstra, F.R.H. and Meijman, T.F. (1989). Het meten van mentale inspanning met 
behulp van een subjectieve methode. In T.F. Meijman (Ed.), Mentale belasting en 
werkstress. Een arbeidspsychologische benadering, p. 42 - 61. Assen, The 
Netherlands: Van Gorcum. 


Simulation and Control of Traffic Flow 
on Large Motorway Networks 

Sonke Schoof 
Fachbereich Elektrotechnik 
Fachhochschule Hannover 
Hannover, Germany 


Motorway networks are important facilities to handle high demand of traffic flow and 
transportation. In overloaded situations or in cases of incidents efficient, operative 
control means are necessary to save the operability of the motorway network. The most 
effective control means are ramp metering at the entries of the motorway network, 
variable speed limitation on individual sections of the motorway and diversion of traffic 
flow using variable message signs. 

In this paper a macroscopic simulation model for traffic flow is used to find optimal 
control strategies for traffic responsive control means. First the traffic model allows a 
forecast of the traffic evalution on the motorway network. Further it is used for an 
offline-optimization of the control means in certain traffic load situations. Therefore an 
optimization problem for the control variables is formulated with a suitable performance 
criterion for the traffic flow. The ojfline optimization results are compared with the 
developed online traffic responsive control law in some simulation studies. 


Congestion on German motorway networks have become a common phenomenon 
especially within the holiday period. It leads to considerable delays, reduced traffic 
security, increased fuel consumption and air pollution. 

Traffic control systems are important facilities to preserve the capacity of the motorway 
network and to avoid congestion. They require intelligent traffic responsive control 

In this paper a traffic responsive control concept in motorway networks is presented. As 
control inputs we take into consideraton: 

• ramp metering at the on-ramps of the network by traffic signalization, 

• variable speed limitation imposed on individual sections of the motorway, 

• traffic flow diversion to alternative routes by variable message signs. 

The concept of a traffic responsive closed loop control is illustrated in Fig. 1. 

Actual traffic data are collected fi*om appropiate locations. These sample data are 
extended by estimation techniques ( e.g, as proposed by Cremer, 1976 ). The values of 
the state variables are then mapped by a short control algorithm into the actual control 
inputs which respond best to the given traffic load. 


Figure I: Structure of a feedback control system for a small network 

This paper is organized as follows: In the following chapter a dynamic, macroscopic 
model for the traffic flow in a motorway network is introduced. In chapter 3, an optimal 
control problem for open loop control of traffic flow is formulated. This nonlinear, large 
scale optimization problem is solved by particular heuristic optimization algorithms. 
Using tese results a closed loop control law is designed in chapter 4. The efficiency of 
the control law is underlined in chapter 5 by a simulation of a critical case. Finally a 
summary with some conclusions is given. 

The simulation model of traffic flow 

To simulate traffic flow in a large network a macroscopic, time discrete model is used 
which was first introduced by Payne (1971) and later improved by Cremer (1979). 

To present the model equations briefly, let us consider a section of a motorway, which is 
spatially divided into subsections of a length between 400 and 600 meters. 

.-A, --- A, -. -- A .-1 -^" 

Figure 2: Section of a multilane motorway with subsections and model variables 


Let T denote the time step width. Then for each subsection j (j-1, n ) the following 
variables of the time discrete model are introduced: 

Cj (k) trajfic density in subsection j at time kT [veh/km] 

Vj(k) mean speed of the vehicles within subsection j at time kT [km/h] 

q^(k) trajfic volume from subsection j into subsection j+1 during the time 

interval kT < t < (k + 1)T [veh/h] 

qr (k), (k) entering or leaving ramp volumes during kT < t < (k + 1)T [veh/h] 

Now we can formulate difference equations for the state variables Cj(k) and Vj(k): 

Cj(k + 1 > = Cj (k) + (k) -qj(k) + 5^jq,(k) - KjqJk)] 

( 1 ) 

Vj(k + l) = Vjik)+^[v{cj(k),U:,(k))-v^(k)]+f-[vj(k){vj_i(k)-Vj(k))'\ 

V T 


X Ay 


( 2 ) 

Equation (1) is a simple balance of vehicles and equation (2) was derived from empirical 
considerations. The first term in brackets on the right hand side describes a retarded 
adaptation of the average speed to the stationary speed-density characteristic V(c,U 2 ). 
The second term represents convection of spatial speed profile in the downstream 
direction and the third term models drivers anticipation of a density variation as seen in 
downstream direction. 

For the volumes between the subsections the following approximate expression taken 
from hydrodynamical relations is meaningful: 

qj(k) = acj(k)vj(k) + ('l-a) Cj^i(k)Vj^,(k) (3) 

The model parameters cx, k, x, v and the speed-density-characteristic ( see below) have to 
be calibrated by real measurements. For German motorways the calibration was 
performed in Cremer and Papageorgiou (1981). For the time step T a value of 10 sec has 
shown to be a suitable value. Care has to be taken that the maximum speed does not 
exeed the ratio Aj/T. 

The influence of speed limitation is modelled by the control variable U 2 . Based on 
investigations of the effect of speed limitations on German motorways (Zackor, 1972) 
the following relationship for a stationary speed-density characteristic has been 


V(C,U2)~VfU2' 1 - 




where is the average free flow velocity, is the maximum density at full congestion 
and 1 and m are positive real numbers. The control variable M 2 has the following 

= 1: 

unlimited free speed 

= 0.82: 

speed limitation to 100 km/h 

= 0.65: 

speed limitation to 80 km/h 

= 0.5: 

speed limitation to 60 km/h 

Figure 3: Speed-density characteristic and volume-density characteristic for different 
speed limitations 

Ramp metering by control input m^ is modelled by the following equation: 
qr 'f(c^(k)) 

q^(k) = min' q^-u^ik) (5) 

^d(k) + qjk) 

Thus, the entering volume is either limited by congestion on the motorway ( function 
f(ci)) or by the green time quota u^ of the ramp metering signal or by the number of 
waiting ( q^) and arriving {q^) vehicles. 

To model the leaving ramp volumes at intersections or off-ramp some knowledge about 
the origin destination pattern is necessary. For simplifying reasons we assume constant 
local OD-quotas (0 < e < 1): 


( 6 ) 

q^jk) = (l-e)^qjk) 

If diversion of traffic flow due to congestion or an incident on the main route is 
recommended by the control variable e { 0 , 1 } > the off-ramp volume is increased by an 

additional trafficstream qjk): 
qJk) = u^(k)‘E^^(\-e) qJk) 


denotes the divertable portion of the traffic flow. The diverted traffic flow has to be 
balanced separately on the alternative route in order to secure correct origin destination 
flow. For this reason density and volume have to be calculated separately for the diverted 
flow in Eq. (1) and (3), The average speed (Eq. (2)) has to be calculated by taking the 
total density. 

The control variables Ui,U 2 and 1/3 are defined at several places in the network and can 
be changed after a period of seconds which should be chosen not too small for reason 
of acceptance. We have chosen a value of = 5 min. 

Optimal control problem 

In order to determine optimal controls for a given traffic demand pattern we need a 
suitable performance criterion by which the impact of the control variables on the traffic 
flow can be quantified. Several objective functions may be found in literature (e.g. in 
Isaksen and Payne ( 1973) or Cremer and Heischmann (1987)). A meaningful objective 
is to minimize overall travel time within the network during the time period under 
consideration. The only disadvantage is that the effects of the control actions can reach 
beyond the chosen period and are therefore not accounted properly. 

The total delay time in the network is closely related to the overall travel time and avoids 
the mentioned disadvantage (Cremer and Schoof (1989)). Total delay time is calculated 
as a function of the model variables by taking a sum of three different terms: 

• delay time from the waiting queues on the i-th on-ramp: 

I\{.k) = TUl{k) ( 8 ) 

• momentary delay time in subsection j of the motorway: 

li(k) = AjCj{k)T (9) 

• delay time caused by driving on the deviation route: 

L,(k)=Tq,(k)^^ ( 10 ) 

where /„ and 4 are the lenghts of the main route and the alternative route. 
Total delay time is then the sum over time and space of the momentary terms: 


P = Z Zl^(k) + tli{k) + L,{k) 

k:=l L 1=1 >1 

With this performance criterion we can now formulate an optimization problem: 

Given a time discrete model for the traffic flow through a motorway network as nonlinear 
differential equation system: 

x(k^-\) = x(k) + f{x(kX u(k), qo(k)) (12) 

with known initial state x(0) and known entry flows qo(k), k = X...,K, 

Find sequences for all control variables |mJ (k),u{(k), u^(kjjover a chosen time period K 
which minimize the total delay time P ( 11 ) under the following constraints: 


0.5<ui(k)<l (13) 

u^(k)G {0,1} 

Values can be changed after a period of r„ seconds (e.g. 5 min,), 

This problem can be classified as a large scale, nonlinear, mixed integer optimization 
problem. To solve such a complex problem, several optimization techniques have been 
tested. The best results were achieved by the method of „Threshold Accepting**, which 
was developed by Dueck and Scheuer (1990). 

In one optimization step a complete simulation of the traffic evolution in the network is 
necessary, so the performance of the simulation model is very important. 

Traffic responsive control law 

The optimization procedure is only suitable for an offline-control of the traffic flow, 
because of the complexity and the computational effort. 

For an online-control-system as shown in Figure 1 a simple control law is needed, which 
maps the actual state variables into new control decisions. From typical case studies and 
analysis of the static traffic control problem a new control law for the online traffic 
control problem in urban traffic corridors was derived (Schoof (1991)). 

We present the control law briefly adapted for motorway networks: 

7. Determination of the on-ramp control values u\(k) for the i-th section 

In stationary state of traffic flow the ramp metering control variables can be calculated by 
a simple linear optimization scheme, hi case of congestion a special algorithm is 
necessary to reduce the congestion. 

In the i-th section the following constraint has to be fulfilled ( see Fig. 4): 



The incoming static traffic volumes are given at the network boundarys or can be 
calculated successively. 


Figure 4: Necessary constraint for congestion reduction 

From this the values u\ can be determined as follows: 

f Qmp congestion (e.g. = 4000 veh / h for 2 lanes) 

~ I min \q )} in case of congestion ( max {c} } > ) 

0 for9-^-r‘<0 

“I = {(log «otherwise 

1 toxq^^-r-'^qr 

If the reduction of the incoming traffic volume is not sufficient, other upstream onramps 
have to be closed. 

2. Determination of the speed limitation values u\(k) for the i-th section 

The influence of speed limitation to the stationary speed-density characteristic V(c,M 2 ) 

is formulated in Eq. (4). With Q(c,U 2 )-c^V(c,U 2 ) and ^~(c,U 2 ) = 0 the optimal 

speed limitation for a stationary density ccan be calculated. The functional 
correspondence u^ = U 2 (c)is depicted in Fig. 5. 

To obtain a characteristic density value in the section, a weighted average density c' is 
calculated with heigher weight to the heigher density values: 

‘ =YiWj-c)(k), X)w';=l and U 2 =U 2 {c') 

If congestion is detected ( max{c} }> c^^it) > speed limitation of W 2 = 0.65 ( 80 km/h) is 


recommended. If other upstream onramps must be closed, speed limitaion in the 
upstream sections are set to 80 km/h respectively. 

Figure 5: Stationary control characteric U 2 (c) 

3. Determination of the route diversion decision u^(k) 

For calculation of the route diversion decision it is necessary, to estimate the influence of 
the decision to the performance on the main route and on the alternative route. In 
stationaiy state of traffic flow the delay time in the i-th section can be determined by: 

and /j- denotes the length of the i-th section. 

The delay time caused by driving on the deviation route is in this case: 


If ^< ql^for all sections i, recommendation of the route diversion can be determined 
by calculating: 

A, = (i9) 

16 /,„ 16 /^ 

If Aj > 0, set u^(k) = !.(/„ denotes the set of sections on the main route, 4 of 

sections on the alternative route respectively). 

The revocation of the route diversion can be determined by calculating: 

If A 2 > 0, set uJk) = 0. 

If q' >^^^^for a section i, the route diversion is determined with the aim to avoid 

congestion. For a detailed algorithm see Schoof (1991). 


To demonstrate the efficieny of the traffic responsive control law, a topology like Fig.l 
was chosen. The length of the main route is 5 km to node 1, then 15 km to node 2 and 
another 5 km. The length of the alternative route between node 1 and 2 is 20 km. 

The simulation period is 1 hour, and on the main route an incident occurs from min. 17 
to 33 by which one of two lanes is blocked. 

Fig. 6-8 show the evaluation of the density at the main route in case of no control, traffic 
responsive control and optimal control. In case 2 the route diversion is recommended 
from min, 20 to 40, in case 3 from min. 15 to 30. 

t [min} 

Figure 6: Time space diagram of density on the main route without control 

Figure 7: Time space diagram of density on the main route with traffic responsive 
control law 


Figure 8: Time space diagram of density on the main route with optimal control 

The performance index for the overall network in the three cases is shown in the 
following table: 

performance index [veh h] 

no control 


traffic responsive control 


optimal control 



In this paper a traffic responsive control law is presented to determine optimal control 
sequences for traffic flow in a motorway network. The control means are ramp metering, 
speed limitation and route diversion. 

First a dynamic macroscopic model for the traffic flow in a motorway network as a 
simulation tool is introduced. Second an optimal control problem was formulated. To 
solve this large scale, nonlinear, mixed integer optimization problem special optimization 
techniques, e.g. the method of „Threshold Accepting^, are necessary. 

For an online-control-system a traffic responsive control law is necessary, which was 
derived by investigation of the stationary state of traffic flow. The efficieny of the control 
concept is demonstrated by the results of a case study. 

Future work is directed to extend the control law for multiple alternative routes with 
conflicting requirements. Other research tasks are a sensitivity analysis or the reduction 
of the measured information. 



Cremer, M. (1976) A new scheme oftrajfic flow estimation and control with a two 

component model. Control in Transporation Systems, Proc, of the IF AC / IMP / 
IFORS 3-rd Mt. Symposium, Columbus, Ohio. 

Cremer, M. (1979) Der Verkehrsflufi auf Schnellstrafien. Springer Verlag Berlin 
Heidelberg New York. 

Cremer, M., Papageorgiou, M. (1981) Parameter identification for a traffic flow model 
Automatica, Vol. 17, No, 6, pp, 837-843. 

Cremer, M., Heischmann, S. (1987) Entwicklung eines regelungstechnischen Konzepts 
zum verkehrabhdngingen Einsatz von Wechselverkehrszeichen und Zufahrts- 
dosierungen in Schnellstrafiennetzen. Forschung Strafienbau und 
StraBenverkehrstechnik, No.505, Bonn-Bad Godesberg. 

Cremer, M., Schoof, S. (1989) On Control Strategies for Urban Traffic Corridors. Proc. 
of the CCCT’ 89, International Federation of Automatic Control; Paris, pp. 287- 

Dueck, G„ Scheuer, T. (1990) A General Purpose Optimization Algorithm Appearing 
Superior to Simulated Annealing. Journal of Computational Physics, Vol. 90, 

No. l,pp. 161-175 

Isaksen, L., Payne, H.J. (1973) Freeway Traffic Surveillance and Control Proc. IEEE 
61 pp. 526-536. 

Payne, H.J. (1971) Model of freeway traffic control Simualtion Council Proc., Vol. 1, 
pp. 51-61. 

Schoof, S, (1991) Modellierung, Optimierung und Regelung von VerkehrsflUssen in 

Korridoren mit Strafien unterschiedlicherLeistungsfdhigkeit. Schriftenreihe der 
AG Automatisierungstechnik, Technische Universitat Hamburg Harburg, Heft 7. 

Zackor, H. (1972) Beurteilung verkehrabhdngiger Geschwindigkeitsbegrenzungen auf 
Autobahnen. Forschung StraBenbau und StraBenverkehrstechnik, No. 128, Bonn- 
Bad Godesberg, 


False yield and false go decisions at signalized left-turn intersections: 

A driving simulator study 

Andrea Szymkowiak, Donald L. Fisher, & Karen A. Connerney 
Department of Mechanical and Industrial Engineering 
University of Massachusetts, Amherst, MA 01003, USA 


The possible errors drivers can make interpreting different traffic signal/sign 
combinations at signalized intersections fall into two error categories: False yield and 
false go responses. In order to understand better the possible reasons for false yield and 
false go responses, younger and older drivers had to interpret different signal/sign 
combinations while operating a driving simulator. The combinations of traffic signals 
and signs were selected such that the elements in those combinations either conveyed a 
yield response, a go response, or both, to different degrees. The percentage of errors 
drivers made was the dependent variable. We predicted that the percentage of errors 
would vary systematically with the compatibility of the messages implied by the sign and 
signal. The predictions were consistent with the observed errors in the majority of 
combinations. Thus, we are able to predict which particular signs will be most effective 
with a given signal. Theoretical and practical gains clearly follow from this capability. 


Drivers have to react to different signals, one of which is the traffic signal at 
intersections. Oftentimes the design and location of these traffic signals does not allow 
their efficient interpretation by drivers (Bonneson and McCoy, 1994; Williams, 
Ardekani, and Asante, 1992). This leads to errors, where drivers sometimes go when 
they should yield (false go), and sometimes yield when they should go (false yield), 
Williams et al. (1992), for instance, conducted a mail survey, in which different 
pictographic designs of signalized intersections were sent to hundreds of drivers in Texas. 
Drivers had to decide whether a left mm was allowed during the presented signal interval, 
and if so, whether it was permitted or protected. Based on a sample of close to 900 
responses, the authors concluded that the simultaneous presentation of a red globe and a 
green arrow on a horizontal five section head should be avoided since this combination 
produced a great many false yield decisions. False yield responses were also found in a 
similar survey conducted by Bonneson and McCoy (1994), In their survey, the authors 
tested how well motorists understood whether a signal design indicated either a protected 
left mm (green arrow and red globe), a permitted left turn (green globe, i.e„ drivers can 
mm left if there is no oncoming traffic), or an overlap condition (i.e., left mrn green 
arrow for left mm indication and green globe for through indication). They found that 
the overlap indication was least understood (i.e., false yield responses for about half of 
the respondents), and permitted and protected indications were understood equally well. 
Regarding sign use, a significantly higher correct response rate occurred without the use 
of signs. The sign used was “left mm yield on green (symbolic green globe)”. It 
improved driver understanding only in the permitted indication (green globe) whereas 
during overlap and protected indication it appeared to confuse drivers, a result also 
consistent with other smdies (Hummer, Montgomery, and Sinha, 1990; see also Williams 
et al., 1992), 

In addition to false yield responses, false go responses have also been observed. For 

example, Staplin and Fisk (1991) ran an experiment in which subjects were shown a 
series of still frames representing the approach to an intersection. Cars were positioned in 
the opposite lane of the intersection. Upon the onset of an auditory signal subjects had to 
press either the brake or accelerator pedal to indicate either a no go or go decision as a 
response to the depicted traffic signal and sign. Staplin and Fisk found that if the signal 
was a green globe the percentage of false go responses ranged from 13.1% to 30.4% for 
younger adults, and from 44% to 64% for older adults depending on the accompanying 
sign. For example, if a green globe was presented with the sign “left turn yield on green 
(symbolic green globe)”, 13.1% of younger, but 44% of older adults indicated that it is 
safe to mm left. If the signal was a green globe and the sign was “protected left on green 
arrow” 30.4% of younger, but 64% of older adults made a false go response. 

The reviewed results indicate that the presence of an auxiliary sign can help as well as 
impair driver understanding. Furthermore, it appears that understanding of the traffic 
signal and sign depends on whether both elements convey the same message. For 
example, in Staplin and Fisk's (1991) experiment, more errors were made when the green 
globe was accompanied by the sign “protected left on green arrow”, than when the sign 
was “left mm yield on green (symbolic green globe)”. Since the latter sign and signal 
both imply that the driver should yield if making a left mm, whereas the former sign and 
signal imply different actions (go and yield, respectively), drivers may respond correctly 
more often to the combination for green globe and “left mm yield on green (symbolic 
green globe)” sign than they do to the combination of a green globe and “protected left on 
green arrow” sign. 

In an attempt to investigate the possible reasons for incorrect responses as well as to 
replicate the results of the previous smdies we tested drivers under realistic driving 
conditions on a driving simulator. Drivers approached a 4-way signalized intersection 
and had to decide whether it is safe to mm left in the presence of oncoming traffic in the 
opposite lane of the given intersection. We expected that, in general, fewer errors would 
be made in our smdy compared to the reviewed smdies because subjects could interact 
with and take into account approaching traffic when making a response. We further 
theorized that errors would vary with the degree of compatibility between the conveyed 
messages of the traffic signal and the sign. Thus, in conditions in which a protected left 
mm stams is conveyed by the signal (i.e., a green arrow with or without a green globe), a 
compatible sign that conveys verbally the same stams [e.g., “protected left on green 
arrow”] should allow better understanding than an incompatible sign that conveys a 
“yield” message [e.g., “left turn yield on green (symbolic green globe)”]. The possible 
errors drivers can make for those signal/sign combinations implying a protected left turn 
stams are false yield responses. Conversely, in conditions in which a permitted left turn 
stams is conveyed by the signal (i.e., green globe), a compatible sign that conveys 
verbally the same stams [e.g., “left mm yield on green (symbolic green globe)”] should 
allow better understanding than an incompatible sign that conveys a “protected left turn” 
message. The possible errors drivers can make interpreting those signals which imply a 
permitted left mm stams are false go responses. For combinations with a red globe or 
arrow no predictions can be made with respect to the compatibility between signal and 
sign, since no sign used referred to a red arrow or globe. However, we would expect that 
a red light indicates a yield or stop response and thus should benefit from a sign that 
implies a yield response. 



Subjects. 32 drivers were tested; 16 of which were between 20 to 32 years of age 
(average 23) and had at least four years of driving experience. The other 16 drivers were 
between the age of 63 and 84 (average 71), Subjects were reimbursed with $10 for each 
session in which they participated. 

Subjects were tested on the two cognitive tests of the Wechsler Adult Intelligence Scale 
(1955), the digit span test and the vocabulary test. On average, older adults had a 
vocabulary score of 38.25 and a digit span score of 11.31; younger adults had an average 
vocabulary score of 37.38 and a digit span score of 12,31. These tests were used to 
ensure that both age groups had equivalent educational backgrounds. Older and younger 
adults did not differ significantly in both test scores as established by a tow-tailed t-test at 
a = 0.05 (r = 1.069 for the vocabulary test, and r = 1.54 for the digit span test). 

Apparatus. The driving simulator consisted of a sedan (Saturn) positioned in front of a 
screen. A virtual driving scenario was projected onto the screen using a Sony Multiscan 
Projector VPH-1272Q/1272QM, 1993©. The car was equipped with the usual devices 
installed in a car with an automatic gear shift. Steering wheel, brake and accelerator 
pedals, as well as the gear shifting stick, allowed realistic driving maneuvers. The 
scenarios were designed using a real-time 3D modeling environment (Designer’s 
Workbench 3.1) developed by Coryphaeus Software, Inc, 1995©. Driving within a built 
scenario and interaction with other cars were accomplished using another software 
package developed by Monterey Technologies, Inc. 1995©. 

Visual Database. One scenario consisted of four 4-way signalized intersections which 
were built continuously, one intersection after the other. There were two through lanes 
and one separate left-turn lane for the driver with a white stop line at the end of this lane. 
The length of the left turn lane was about 33.5 meters. The intersections were built 
according to MUTCD (Manual on Uniform Traffic Control Devices, 1988, 4b-12/4b-13) 
recommendations. One intersection was 31 meters across and 31 meters wide. The 
traffic signal was visible within a 40 degree visual angle from the position of the driver at 
the white stop line at a distance of 14.5 meters to the traffic light. The height of the lower 
part of the casing of the traffic signal was 4.9 meters with the sign positioned to its right. 

The relevant traffic signals consisted of either a 2x3 signal head- (Signals 4, 6, and 8) or a 
1x5 signal head (Signals 1, 2, 3, 5, and 7) arrangement as depicted in Figure 1. The size 
of the signal and signs differed from MUTCD recommendations in that they were bigger 
than real signs to allow for better legibility. Signs were legible at a distance of about 25 
meters. The signals 1 to 8 were combined with each of the signs A to D depicted in 
Figure 1 resulting in 32 relevant combinations. Not shown are eight additional 
signal/sign combinations that were chosen as “fillers” to induce variation in the scenarios 
and to convey the impression of an “realistic driving” task, resulting in a total of 40 
signalized intersections. “Fillers” included two additional signs, i.e., “left on arrow only” 
or “left turn not protected”. All signs are currently in use in the United States. Signals in 
combination with signs D and B allowed us to compare directly the effects of 
compatibility between the signals and the sign. Sign A (no sign) represents a control 
condition in which only the effects of the shown signal are tested, whereas sign C, “left 
lane must turn left”, allows us to assess the distracting effects of signs that do not relate to 
driver actions with respect to the depicted signal. 


12 3 4 

5 6 7 8 

Figure 1: Signal (1-8) and signs (A, B, C, D). Each of the eight signals is combined with 
each of the signs. The dimensions are not drawn to scale. 

Scenario Dynamics. Some of the signals included a red light (red globe and/or red arrow) 
which induced the driver to stop at the intersection. If the driver slowed down and 
continued to slowly approach the intersection the red light would change to a green light, 
i.e., a green arrow and/or a green globe. When the driver approached the intersection 
there was approaching traffic (2 to 4 cars) in the opposite lane for through traffic. The 
velocity of the approaching cars was set around 100% relative to that of the driver. 
Movement onset of the approaching cars was triggered at a distance of about 100 to 300 
meters to the driver. At a certain distance (between 40 and 80 meters) from the 
intersection the approaching cars drove independently of the velocity of the driver and 
continued through the intersection or stopped at the traffic light depending on the traffic 
signal shown to the driver. If the driver was shown a protected left turn status (signal- 
sign combinations 2A-D, 5A-D, 7A-D) the approaching cars stopped at the intersection. 
If the driver saw a permitted left-turn status displayed by the traffic signal and sign 
(combinations lA-D) the approaching cars proceeded to drive through the intersection. If 
the driver had to come to a stop because of a red light (red globe and/or red arrow) the 
approaching traffic stopped also. After the signal changed and the driver could continue, 
the approaching cars proceeded accordingly, i.e., dependent on the traffic signal status 
shown to the driver. 


Design. Within a scenario the four intersections were arranged such that no signal or sign 
occurred twice in a row. Since there were 40 traffic signal and sign combinations, 10 
scenarios were constructed. The order of the 10 scenarios presented to each subject 
within an age group was different. 

Procedure. Before the experiment drivers were given instructions and familiarized with 
the driving simulator in two practice sessions. In the first practice session the driver sat 
on the passenger seat and the experimenter drove through one scenario to show the driver 
the dynamics of the car and to demonstrate steering and braking behavior. After that, the 
driver drove through one scenario in a second practice session. The practice scenarios 
consisted of four signalized intersections, however, no signals or signs were visible. The 
practice scenarios were otherwise identical to the experimental scenarios. Drivers were 
instructed to drive between 10 and 15 miles per hour (16 to 24 km/h) and to make no 
abrupt steering or braking movements to avoid potential simulator discomfort (e.g., 
dizziness, nausea). Controlled driving was emphasized. 

Subsequent to the practice sessions the experiment began. The driver was informed that 
he/she would see a signalized intersection. There would be approaching traffic in the 
opposite lane of the intersection. Drivers were supposed to approach the intersection and 
turn into the left-turn lane. Their task was to turn left at the intersection if they thought it 
was safe to do so given the depicted traffic signal. To ensure that drivers read the sign 
they were informed at the beginning of the experiment that they would be given a sign 
comprehension test at the end of the study. If drivers decided it was safe to turn left they 
should turn across the intersection. If they decided it was not safe they should stop at the 
white stop line or yield at the intersection, and then continue the left turn when permitted 
either by changes in the traffic or by changes in the status of the traffic signal, finally 
driving to the next intersection. Drivers were further instructed to indicate verbally their 
decision by saying, for instance, “I have to yield” or “I can go”. Drivers were informed 
that the other cars would drive according to traffic regulations. After four intersections, 
the driver stopped and took a break for about 2 minutes during which time the next 
scenario was loaded. After completing a pair of scenarios the driver could take a break of 
5 minutes if he/she wished to do so. If drivers drove 15 miles per hour (24 km/h) they 
needed about 2 minutes to drive through the entire scenario. The entire experiment lasted 
about one hour. 

Dependent Variables. A driver error was recorded if the driver falsely stopped (yielded) 
or falsely went through the intersection. Driver behavior (yield or stop vs. go) and errors 
were recorded manually. The velocity of the driver’s car (meters/second) and the x, y, 
and z coordinates of the car were recorded at a frequency of 33 Hertz. The velocity data 
were analyzed over data points within a 2-second window before and after subjects 
passed the white stop line at the intersection to establish the validity of the manual coding 


Three older subjects canceled the experiment due to experienced nausea and were 
replaced. All drivers were able to give a correct definition of the used signs. A repeated 
measures ANOVA was computed over the velocity data with driver behavior (yield or 
stop vs. go) and age group (young vs, older adults) as factors. The velocity data are 
shown in Table 1. Depicted are the average velocities (in km/h) for correct responses. 
Subjects drove significantly slower when they correctly yielded than when they correctly 

drove through the intersection, F(l,30)= 91. 28, p < 0.001, and older drivers, in general, 
drove slower than younger drivers, F(l,30) = 65.32, p < 0.001. The interaction between 
age and driver behavior was also significant, F(l,30) = 37.29,/? < 0.001, indicating that 
the difference between yield and go responses was more pronounced for younger adults. 

Table 1: Average velocity in km/h for correct go and correct yield responses. Velocity 
data are averaged over data points within a 2-second window before and after drivers 
passed the white stop line. 

Correct Yield 

Correct Go 













Pair-wise comparisons were made between the error percentages of selected signal and 
sign combinations. The selected test statistic was z* = (pi-p 2 )/[p(i"p)(l/^+l/w)] , 
where pi and p 2 are the respective proportions and m and n the selected sample sizes. 
Parameter p is the pooled proportion value, p = [{ml{m+n))p\+{nl{m+n))p/\. The 
proportion differences were tested at a =0,01. Summary data are depicted in Table 2. 
The overall error rate was 3.9%, There were significantly more false yield than false go 
responses, z* = 6,99, There was no age effect reflected by the error data, z* = 0.64. 
The error data collected for the different traffic signal and sign combinations are depicted 
in Figure 2 for the “false yield” responses, and Figure 3 for “false go” responses. In 
Figures 2 and 3, the signal/sign combinations are depicted in a matrix-like arrangement. 
The number of the signal and the letter of the sign, as well as error percentages are 
depicted in the “cells”. For signals displaying a protected left turn status Sign B (16.7%) 
produced significantly more “false yield” responses than sign D (4.2%), z* = 2.83. Sign 
C (13.5%) produced significantly more false yield responses than no sign (sign A, 3.2%), 
z* = 2.61 (see last row of Figure 2). No significant differences between the signs were 
obtained for “false go” responses (see last row of Figure 3). 

Table 2: Error Percentages for false yield and false go responses for the different age 

False Yield 




















No Sign 

2B 12.5% 

old 6.2% 
young 18.8% 

2D 3.1% 

old 6.2% 

2A 0% 


2C 62% 
young 12.5% 

5B 28.1% 
old 31.2% 

young 25.0% 

5D 6.2% 

old 6.2% 

young 6.2% 

5A 6.2% SC 25.0% 

old 6.2% old 25% 

young 6.2% young 25% 

7B 9.4% 

old 6.2% 
young 12.5% 

7D 3.1% 

young 6.2% 

7A 3.1% 
old 6.2% 

7C 9.4% 

old 12.5% 
young 6.2% 

Error 16.7% 4.2% 

Percentage old 14.6% old 4.2% 

per Sign young 18.6% young 4.2% 

3.2% 13.5% 

old 2.1% old 14.5% 

young 4.2% young 12.5% 


EZ] green 

Figure 2: Percentage of false yield responses for signal and sign combinations for 
younger and older drivers. Each of the signals shown in the left column is paired with 
each of the signs in the first row. The respective signal/sign combinations and the 
percentage of errors are depicted in the cells'". 






No Sign 


ID 3.1% 

old 6.2% 

1C 3.1% 

young 6.2% 

3C 3.1% 

young 6.2& 

4A 0% 4C 

6A 0% 6C 3.1% 

young 6.2% 

8B 0% 

8C 0% 

per Sign 

old 1.2% 

red i-y-j green 

young 3.8% 

Figure 3: Percentage of false go responses for signal and sign combinations for younger 
and older adults. Each of the signals shown in the left column is paired with each of the 
signs in the first row. The respective signal/sign combinations and the percentage of 
errors are depicted in the **cells". 


The velocity data support the results of the manual coding procedure: Participants drove 
slower when they had to yield than when they drove through the intersection. The low 
overall error rate (3,9%) indicates that subjects understood the task well. Interestingly, 
no difference in errors between the age groups over false yield and false go responses 
(Table 2) was found. The low error rate indicates that subjects can make efficient use of 


the presence of and interaction with approaching traffic to respond to traffic signals. For 
a comparison, Staplin and Fisk (1991) obtained 22% and 23% false yield responses for 
younger and older adults, respectively, whereas the corresponding percentages were 
9.89% and 8.85% in our study. Their percentage for false go responses was 24% and 
46% for younger and older adults, whereas we obtained 0.94% and 0.31%, respectively. 
Although Staplin and Fisk’s subjects saw approaching cars on the still frames of the 
upcoming intersection one might speculate that subjects might benefit from an even more 
realistic, that is dynamic, environment as in the current study. 

Regarding a particular signal and sign combination, a closer look at the error rates for 
signal combinations with sign B and D over all drivers reveals that the error rates are 
consistent with the compatibility hypothesis described earlier in the introduction. As 
expected, for signals implying a protected left turn status (Figure 2) significantly more 
errors were produced when the sign implied a permitted left turn - all combinations in 
which both signal and sign imply a protected left turn status (2D, 5D, 7D) are related to 
lower error rates than combinations in which the sign implies a “yield” or permitted left 
turn status (2B, 5B, and 7B). For combinations implying a permitted left turn (Figure 3), 
errors occurred only in condition ID (green globe combined with “protected left on green 
arrow”), and in combinations IC, 3C and 6C. Note that no errors occurred with Sign A 
and B combinations. Since Sign B is compatible with a yield or stop response whereas 
sign D implies a go response, the result is consistent with the compatibility hypothesis, 
however, overall error rates are too low to allow reasonable conclusions. 

The lowest error rate overall was found when no sign accompanied the signal (see last 
rows of Figures 2 and 3). This is also in agreement with the results by Bonneson and 
McCoy (1994) and Hummer et al. (1988). The data clearly indicate that in almost all 
cases participants benefited from the absence of a sign. An unexpected result was that 
significantly more errors were committed if sign C was presented with a signal than when 
no sign was presented with the protected left turn signals (Figure 2). Sign C did not 
relate to the status of the traffic signal, however, almost as many errors were produced for 
combination 5C as for 5B which produced the highest error rates for a particular 
signal/sign combination. It appears that if the meaning of the sign does not directly 
correspond to the meaning of the signal, its presence is harmful for interpreting the 


The following conclusions can be drawn from the above study: 

• False go responses are potentially much more serious than false yield errors. 
Since no false go responses were made when the sign was “left turn yield on 
green (symbolic green globe)” it can be concluded that this sign is beneficial if 
used with a green globe by itself. 

• If a protected left turn status is conveyed by the signal, the sign should state so 
explicitly, too. Thus, a green arrow by itself can be accompanied by the sign 
“protected left on green arrow”. 

• In the case that both signals occur alternately, the use of variable message sign 
technology seems obvious: If the signal is a green arrow the sign should 
convey a “protected” left turn status; if the signal is a green globe it could 
convey a “yield” status. 


• If a protected left turn is conveyed by the signal in combination with a 
permitted left turn (green arrow and green globe), the sign should not 
emphasize a “yield” message, i.e., the sign should not be “left turn yield on 
green (symbolic green globe)”, A great many false yield errors were obtained 
if this combination was used (Figure 2, combination 5B). False yield 
responses might lead to an increased risk of rear-end collisions; and 

• Since aU signals were best interpreted when no sign was present it can be 
concluded that the absence of a sign helps driver understanding. This was 
also supported by the fact that sign C which did not relate to the status of the 
signal was associated with many errors. 


This research was supported by grant 5-25656 from the National Institute on Aging to 
Donald Fisher. 


Bonneson, J. A., & McCoy, P. T. (1994). Driver understanding of protected and 
permitted left-turn signal displays. Transportation Research Record, TRB, 
National Research Council. Washington D. C., 1464,42-50. 

Hummer, J. E., Montgomery, R. E., & Sinha, K. C. (1990). Motorist understanding of 
and preferences for left-turn signals. Transportation Research Record, TRB, 
National Research Council Washington D, C., 1281,136-147. 

Manual on Uniform Traffic Control Devices (1988), US Department of Transportation, 
Federal Highway Administration. 

Staplin, L., & Fisk, A. D. (1991). A cognitive engineering approach to improving 
signalized left turn intersections. Human Factors, 559-511. 

Wechsler, D. (1955). Wechsler Adult Intelligence Scale. Psychological Corporation, 
New York. 

Williams, J. C,, Ardekani, S. A., & Asante S. A. (1992). Motorist understanding of left- 
turn signal indications and auxiliary signs. Transportation Research Record, 
TRB, National Research Council Washington D. C., 1376, 57-63. 


Validation of an Economical Fast Method to Evaluate 
Situation Specific Parameters of Traffic Safety 

Katharina Dahmen-Zimmer, Kilian Ehrl, Alf Zimmer 
University of Regensburg 
Experimental Applied Psychology Unit 
Regensburg, Germany 

The need for efficient simulators for training and investigating complex and especially 
risky tasks is obvious. Accordingly simulators for as diverse tasks as flying a plane, 
driving a car, flight control and operating nuclear reactors have been developed. While 
the benefits of simulator training are easy to see, for instance, dangerous situations can 
be trained without any risk for personnel of machinery, the question how valid a 
simulation is, that is, how similar is the behavior in the simulator to the behavior in real- 
life situations quite often is left open or only answered by appealing to the obvious 
simularity in the layout of instruments plus the physical characteristics of controls and 
the realism of the depictured visual scene. Especially, the lacking degree of realism in 
computer generated visual scenes has been criticized: the lack of non-geometric 
perspective cues as haze or blue-shift, the unnatural regularity of buildings, the crispness 
of contures, and, in general, the lack of realistic clutter starting from pedestrians on 
seemingly random courses to debris and discolorations of the surfaces. This situation has 
led to the development of video-simulators (2D and 3D) at the Experimental Applied 
Psychology Unit at the University of Regensburg, among others (for an overview, 
especially for research in the US, see MacAdam, 1993); in this simulation methodology 
videos from real traffic scenes are used. 

The simulator developed in the Experimental Applied Psychology Unit at the University 
in Regensburg consists of a BMW limousine, where all the controls and displays are 
linked to a computer, and a video projector producing for the driver a visual scene with a 
visual angle of about 45°. It is obvious, that such a kind of simulation is sensitive only to 
the skills on the basic level of Janssen's (1979) hierarchy of the driving task (van der 
Molen & Botticher, 1988): Subjects can accelerate and decelerate the speed of the video, 
and steering control can be simulated by keeping a target in the middle of the lane. Even 
on this level the task is not as interactive as in full fledged computer simulations because 
deceleration and acceleration influence not only the simulated car in relation to the stable 
environment but also influences all other moving objects in the video. Despite these 
draw-backs, the advantage of video simulation lies in the realism of the depicted scenes. 
In our experiments we were especially interested in drivers' speeding behavior, for the 
adjusted speed is the main cause of traffic car accidents. 

In order to determine the validity of this simulation technique the correspondences with 
real-life driving have been determined on three different levels corresponding to three 
separable research questions: 


(i) Do individually different driving styles induce a corresponding regulation of velocity 
in the simulation and in the real-life driving? 

(ii) Do different driving tasks (velocity maintenance vs. self paced driving) lead to the 
same effects in those situations? and 

(iii) Do corresponding situations in the video and in the real world give rise to the same 
pattern of acceleration or deceleration? 

Additionally, subjective evaluations of both tasks have been elicited with questionnaires 
in order to check if they are rated in such a different way that the experiences cannot be 


The general experimental plan is shown in Table 1, 

Table 1: __ . 




speed maintenance 


real life driving styie 

si mu latord riving styie 


real life driving style 

simulator driving style 

free driving 


real life driving style 

simulator driving style 


real life driving styie 

simulator driving style 

Additionally, in a questionnaire data about the subjects" driving style were obtained. As 
in Assmann (1985) nearly all variance could be attributed to three factors, namely. 

Factor 1: following the traffic flow. 

Factor 2: attitudes towards car driving, and 

Factor 3: judging once own driving ability. 

Using factor scores to classify subjects it was possible to determine the influence of the 
driving style in the two experimental situations. 

Table 2 gives an overview over the route used in the experiments; numbers indicate the 
corresponding situations. In order to determine if accidental influences in the real-life 
driving influence the choice of velocity any of the following observations were 
timecoded: oncoming traffic, following traffic, slow traffic ahead, passing or being 
passed, cyclists on the lane, pedestrians on the lane or immediately beside the lane, and 


children close to the road. The real-life driving was done in a BMW 730i with an 
automatic transmission. During the field experiment all the relevant driving parameters 
were entered into an on-board computer. 

Table 2: 


Speed limit 



Segment of a county road 

60 km/h 

Straight, flat 


Entering a village with a reduced speed 

zone (Oberisling) 

60 km/h 

30 km/h 

slightly curved, incline 


30 km/h 

slight decline, straight 


Leaving the reduced speed zone 


30 km/h 

60 km/h 

right turn, approximately 60° 


60 km/h 

30 km/h 

straight ahead, flat 


30 km/h 

60 km/h 

straight ahead, flat 


Entering the reduced speed zone 

60 km/h 

30 km/h 

straight ahead, priority of 
the turning road 


First T-intersection in GraB 
(without traffic signs) 

30 km/h 


Second T-intersection in GraB 

30 km/h 


Free fane road 

60 km/h 

50 km/h 

straight ahead, slight 


..Sleeping policemen" with following cross¬ 
walk for pedestrians 

30 km/h 

straight ahead, slight 


..Sleeping policemen" 

30 km/h 

straight ahead, slight 


Connecting road 

30 km/h 

straight ahead, slight 

52 subjects participated in the experiment, 26 female and 26 male, in the age range from 
19 to 26 years (mean 24,1 years). Subjects had their driving licences between 1.8 and 
11.8 years (mean 6 years). 26 of the subjects were driving more than 10 000 km/year, 26 
less. Half of the subjects have had experience with an automatic transmission. Subjects 
got a fee of 15 DM. Two subjects missed either the simulator or the real-life driving 
situation, therefore their data could not be used in the comparison of the corresponding 

The simulator consists of a BMW 325i, where by means of the above-described 
projection technique driving can be simulated. Figure 1 shows what has been simulated 
for the speed regulation and what variables have been skipped. 


During simulation a virtual velocity is shown on the speedometer in the car. In order to 
influence the speed of the video realistically, the pressure on the effectors in the car 
(brakes and accelerator) are fed into a computer where by means of a formal car model 
the virtual speed is determined according to the following formula: 

Vpsd ='l (u^*(g-b/u)-u*v^*y)dt 






= 1 - (vpsd / vmax ) 
= go* 1^5 
= bo * 4,0 
= yo * 0,15 = 0,0 03 

acceleration depending on the given velocity 
pressure on the accelerator 
pressure on the brake 
rolling and air resistance 


Due to the fact that the available videorecorder (Panasonic AG 7330) can not increase 
and decrease continually the following relation between the virtual velocity as shown on 
the speedometer and the projection velocity is as in Figure 2. 

Velocity as shown 
on hie speedometer 

Speed of the videorecorder 

Figure 2: Relation between pseudospeed and recorder speed 


In general, subjects regard driving in real-life situations as easier (median answer: 
„easy“) than in the simulator (median answer: , 4 ’ather diffucult“). Furthermore, the 
simulator driving is regarded as needing more attention. Despite these differences, 
subjects when asked if the simulator influences their driving behavior, describe this 
influence as negligable (modal answer: „slightly if at air‘). For this reason it can be 
assumed that the driving experience is comparable for both the field and the simulator. 

Three analyses of variance regarding driving styles with the factor scores of the 
questionnaire on driving styles as the dependent variable, reveal no significant 
differences between real-life driving and simulator-driving with one exception: There is 
an interaction between the values of the factor 1 („going with the traffic flow“) and the 
driving situation. The Scheffd-Test reveals that subjects who „go with the flow“ increase 
the speed in the simulator in contrast to the subjects with negative factor scores who 
drive faster in the real-hfe situation. 

An analysis of variance of the task („speed maintenance'* vs. self-paced driving) shows 
significant main effects for the speed maxima and the average speed. The task „speed 


maintenance** induces slower driving (53,5 km/h) with less variance (1,62) than self- 
paced driving (67,8 km/h and 2,71 respectively) but no significant interaction with the 
driving situation (real-life vs. simulator). Furthermore, there is no significant influence of 
sex on the velocity regulation. 

Figure 3 shows the variability of velocity during one test drive. 

Figure 3: Curve of the driven speed in real-life driving (subject female, 21 years old 
with the task „ speed maintenance “) in relation to the position on the driving 


Figure 4 shows the corresponding curve for the simulator condition. 

Figure 4: Curve of driven speed in the simulator in relation to the driven course. 

If one determines the average maximal velocity for the 13 situations (see Table 2) the 
following curves for real-life driving (upper curve) and simulator driving (lower curve) 
result (see Figure 5) 

Figure 5: Mean values of the maximal velocities for the 13 situations 

For all 13 situations (see Table 2) and all subjects the data for the real-life driving and for 
the simulator driving results of the scattergram in Figure 6. 


Virtual velocity in the simulation 

Figure 6: Scattergram of driven velocities per situation in real-life driving and in 
simulator driving for all subjects fr = .7158; Regression (real-life) 
speed = 12.983 + .852 * simulatorspeed 

If one reduces the noise and uses only the mean velocities for the 13 situations the 
scattergram of Figure 7 results. 


20 30 40 50 60 70 

Velocity in the simulation 

Figure 7: Real-life driving velocities vs, simulator driving velocities (r= ,982; 
Regression: (real-life) speed - 6,48 + 1.02 * simulatorspeed) 


The data from the questionnaire where the subjects describe their experiences in the real- 
life driving situation and the simulator situation reveal that both situations are regarded 
as equally interesting and not influencing ones^ driving behavior. However, driving in 
the simulator is regarded as more difficult and requiring more attention. The reason for 
this might be that the simulator situation is novel and that the velocities in the simulator 
are not changed continuously but in steps. The fact that driving in the simulator is 
experienced as difficult and requiring special attention indicates that the motivation of 
the subjects is high even in the simulator situation. That is, when driving in the simulator 
subjects tried to exhibit „normal“ driving behavior. This can be seen not only from the 
subjective reports but also from the fact that the observed driving style does not influence 
the driving in the two situations differently except for the interaction between „going 
with the flow“ and average speed regulation, but also this influence is very slight. The 
conclusion can be drawn that in the two situations there is no differentiating influence of 
driving attituedes on the real-live vs. simulator driving. 

Most important for the validation of the simulator are the situation dependent 
correlations of situation specific velocities in the simulator and in the real-life situation. 
Accelerating and decelerating behavior in real-life and simulator driving correspond 
nearly perfectly. The main influence on the driving behavior are the situational 
characteristics independently from the fact whether subjects experience real-life driving 
or simulator driving. 


If one analyzes the individual correlations between the simulator data and the real-life 
data for the 13 situations, it turns out that only 4 subjects show correlations less than .5 
but more than 75 % of the subjects show correlations above .85. The effect that the 
regression coefficient does not differ significantly from 1 shows that there is nearly a 
one-to-one correspondence between the driving behavior in real-life and in the simulator, 
only the additive constant of about 6 km/h shows the influence of a „tunnel effect in the 


Standard approaches for estimating the validity of measures start from the theoretical 
assumption that the „true validity“ only depends on the correlation between the „true 
values** in the measure and the „true values** in the criterion. These „true values** cannot 
be measured directly and are practically always confounded with the covariation of 
systematic errors. For these reasons the selection of situations, measures etc. has to be 
planned exactly: For instance, in the case of the evaluation of our video simulator it was 
not important that individual characteristics of drivers („speeding‘* vs, „dawdling** etc.) 
show in both situations but that the same situational factors elicit corresponding kinds of 
behavior, that is, for validating a simulator aimed at the improvement of traffic 
conditions the decisive carrier of information is not the individual driver but the specific 
situation, Generalizability of results therefore depends on the representativity of the 
situations. Insofar the video simulator can be regarded as a valid system for 
determinating situational characteristics of traffic safety. 

Since finishing this field and simulator study the video simulator has been further 
improved, (i) it is now possible to increase and decrease the projection velocity 
continually, (ii) steering control can be simulated additionally by shifting the video 
picture to the left or right in accordance with a computerbased model of steering. 


Janssen, W.H. (1979) Routeplanning en geleiding: een literatuurstudie (Rapport IZF 
1979-C13). Soesterberg (NL): Institute for perception. 

MacAdam, C.C., Green, P.A. & Reed, M.P. (1993) An overview of current UMTRI 
driving simulators. VMTRl Research Review, 24(1), 1-8. 
van der Molen, H.H., Bdtticher, A.M.T, (1988) A hierarchical risk-model for traffic 
participants. Ergonomics, 31, 537-555. 


New Multimedia Components Inside a Car-Cockpit 

Dipl. - Ing. Jens Ohler, Leopold Kostal GmbH & Co. KG, LUdenscheid 
Dipl.- Ing. Roland Drees, Leopold Kostal GmbH <& Co. KG, LUdenscheid 
Prof. Dr.-Ing. Klaus Peter Holzhausen, Fachhochschule Bochum 


Modem cars will be equipped with an increasing number of complex technical systems. 
Many of these systems require the driver to interact. Such an interaction is a process of 
controlling the subsystems and taking its response into account reading a display. The 
more subsystems will be integrated into a car, the more control devices and displays have 
to be placed in a car-cockpit. In the paper a variety of multifunction controls are de¬ 
scribed, that can be used together with screen displays for multipurpose input/output for 
automotive communication. Experiences are included. A resume is given for known as 
well as novel control concepts 


The development of modem automobiles leads to more and more complex technical sys¬ 
tems. Vehicles are equipped with more sophisticated systems onboard. Traditional mod¬ 
ules in a car will be electrically controlled (windows, sunroofs, convertible roofs etc.). All 
these systems require controls to operate them and in many cases displays to inform the 
driver about their state of operation (door locking state, radio frequency, accessibility of 
a cellular telephone channel etc.). CAN Bus and other digital busses are used in the elec¬ 
trical automotive system. They are capable of reducing the increasing number of discrete 
controls as well as the increasing cost and effort of wiring. Ultimately the vehicle will be 
a “drive-by-wire” system with less mechanical parts with even steering wheels, switches 
to operate the headlights or direction indicators replaced by computers and integrated 

The operation of traditional and more recent onboard systems using discrete mechanical 
controls may no longer be feasible because spatial restrictions and operational short¬ 
comings as well as marginal designs with respect to Human Factors aspects are con¬ 
cerned. Considerations to use these controls not only in the vehicle standing still but also 
during operation do even increase difficulties designing a control concept for a modem 

Future Car-Cockpits 

Flat screens are increasingly installed as multifunction displays in cars. Presently they are 
mainly used as front ends for dedicated components, mostly for automobile navigation 
systems. The potential for flat screens as a modem user interface for automobiles can 
only be fully explored when the screen will be used as a universal front end for a larger 
number of electronically operated automotive subsystems (car radio, air conditioning, on 
board computer, telephone and functions to comfort the driver and the passengers). One 
or more flat screen displays in connection with a carefully designed concept of control 
devices will be capable to replace dedicated traditional controls by a multifunctional, 


flexible, and powerful Human Engineered automotive cockpit. 

Figure 1: The BMW navigation system Figure 2: The Blaupunkt navigation 


New displays and controls require space in the automotive cockpit. Especially smaller 
modern cars offer even less cockpit space than more traditional designs. Especially 
scarce is the space for displays in the so called secondary field of vision. This field is 
located in the central dashboard area behind the steering wheel. It will be called secon¬ 
dary field of vision because the driver mainly monitors traffic conditions outside the ve¬ 
hicle through the front window, the true primary field of vision. The secondary filed of 
vision refers to the cockpit area that the driver can monitor safely while the vehicle is in 
motion. This area of the cockpit accommodates speedometer and other important dis¬ 
plays crucial to the function of the automobile. Tertiary space is available in the center 
console, mostly next to the dashboard controls and displays. The ever increasing number 
and complexity of automotive systems require more display space, especially in the sec¬ 
ondary field of viewing area. The adjacent controls must be installed in those areas of the 
cockpit as closely related to the displays as possible. The position called primary control 
space is in the steering wheel (or in the future position of the main steering device, which 
may be a displacement side stick mounted in an arm rest to be reached at ease by the 
driver). The secondary display area is practically unreachable for the driver because it is 
obscured by the steering wheel. Even in a future automobile this space will not be part of 
the secondary control and reach area because restraining harness systems will hinder the 
operator to access that area easily. The tertiary visual range is really the secondary con¬ 
trol area, that is the space in the middle console. It is quite obvious that major automotive 
systems will be operated using controls in the primary reach area and correspond to dis¬ 
plays in the primary or secondary display space. 

The described space restrictions make it evident tliat it is not feasible to increase the 


number of discrete controls and displays in a car. Integration of controls and displays is a 
vital interest for both the manufacturers and the users of modem automobiles. Yet the 
integration has implications from the technical side (aspects of production, installation, 
maintenance, and safety of operation), operability ( function and ease of operation, ade¬ 
quate night design, gerontotechnical aspects), and acceptance by the customer (Human 
Factors concept, self explaining user interface, little instruction effort required). 

Thus an integrated concept for an advanced control and display system is required. Such 
a concept will have to include multisensory aspects for such an Input/Output system. 
Human senses such as the visual channel, the auditory system, speech, and the hap¬ 
tic/proprioceptive capabilities should be included. The visual and the auditory senses 
lend themselves for display purposes, the speech and haptic as well as proprioceptive 
channels can both be used controlling automotive components and systems. An inte¬ 
grated concept is far beyond the scope of this paper. 

Multipurpose Input/Output for Automotive Communication 

In the experimental study on which this paper is based a number of control devices were 
selected and tested to control an automotive subsystem similar to a car navigation system 
but not restricted to the functions of such a system. 

Speech input and acoustic output alone were not considered because of the complexity of 
the task. Yet, such an approach is very appealing to operate a cellular phone in a car or 
other dedicated subsystems of comparable complexity, hi this application there is a need 
to input single commands. Phone operation requires only 15 different commands for 
comfortable operation. For a navigation system, the acoustic system for information in¬ 
put is not flexible enough. Moreover, the acoustic systems may not always work prop¬ 
erly if it is very noisy during speech input. Acoustic output alone with no visual display 
involved may also unacceptably restrict communication. 

Existing navigation systems in automobiles use different approaches for the Human Ma¬ 
chine Interface. Mercedes Benz (Figure 4), for example, uses function keys next to the 
screen. The BMW systems uses a twist button (Figure 1), and the Blaupunkt navigation 
systems (Figure 2) are controlled with a so called four way module. Some products can 
be mounted as add ons, not being integrated in the dashboard thus facilitating later in¬ 
stallation in older cars. All designs require hand and arm movements towards the screen, 
use cockpit space in the secondary control area respectively in the tertiary display space. 

Control Devices for Use in a Car 

In the following chapter, control devices for mechanical information input using hap¬ 
tic/proprioceptive senses will be described and compared. They are always referred to as 
input devices in combination with a flat screen display that requires interaction such as 
moving a cursor around or selecting from a screen menu. 

There are four main groups of such devices: first there are touch-screens and touch-pads 
with or without function-keys next to the screen. The next group are rotating knobs or 
switches, also called twist systems. The third group are the so called pressure systems 
generating inputs by pressing on a control device surface. The fourth group are four-way- 
switches, a device called EN-Joy device and trackball systems. 


A touch-screen (Figure 3) is basically an overlay mounted in front of the screen. The user 
touches the screen on the desired location. There are three different kinds of touch¬ 
screens: The capacitance screen is based on a glass plat with a layer of conductive trans¬ 
parent material. Because of the capacity of the human hand touching the screen, the ca¬ 
pacity between the sensors on the layer will change. The infrared touch-screen works 
with IR- photoelectric barriers witch are arranged in a horizontal and vertical position on 
the edges of the screen. A hand touching the screen interrupts at least one x- and one y- 
ray of light. A controller locates the position. The resistive touch-panel is also based on 
two transparent layers. In contrast to the capacitance technology, resistance between the 
layers changes under pressure. 

A touch-pad does not work in front of a display like the touch-screen. The Versa touch- 
pad, for example, detects the touch position using two semiconductor layers. One layer 
records the x-position, the other one the y-position. Touching the surface changes the 
resistance between the two layers. It is also possible to register the intensity of the touch. 
This allows data input in a “third dimension”. There is also a capacitance touch-panel 
that works like a capacitance touch-screen, yet it is not transparent. 

Function keys, used for example with the Mercedes-Benz navigation system, are me¬ 
chanical switches mounted next to the screen. They are used as multifunction keys. Their 
present functions will be displayed on screen. They cannot be used for analog control. 

Twist systems or rotating knobs can be distinguished by three different physical effects. 
The paddle-system (Figure 5) transfers it’s position to the computer changing the resis¬ 
tance between two conductors. It has been used as controller for the first video games. 
Paddles are analogous devices. Their resistance tends to change with time. For this rea¬ 
son they cannot be used for precision input. An additional problem is the missing op¬ 
erator feedback. 

Mechanical Incremental Resolvers (Figure 6) produce two digital impulses when being 
rotated. The direction of the spin can be determined through the chronological order of 
both output signals. The advantage of the mechanical devices is that they do not need 
much space. Yet, they tend to wear out when used frequently. 

Optical Incremental Systems (Figure 7) produce the same output as the mechanical de¬ 
vices. They need a small tension and a little more space. The big advantage is, that they 
have nearly no attrition. The output is produced by a light barrier. 

Pressure systems (Figure 8) are based on two different physical effects: the Hall effect 
and the Force Sensing Resistor (FSR). Combining four such sensors, a position sensing 
system can be realized. 

Four-way-switches belong to the group to multidimensional sensors. They are applied to 
adjust outside mirrors in a car. Blaupunkt uses them in their navigation systems as well. 
The technical principle is based on a button, that actuates four or more switches. 


Figure 3: The touch-screen 

Figure 4: Function keys 

Figure 5: paddle system Figure 6: mechanical incre¬ 
mental system 

Figure 7: optical 
incremental systems 

The EN-Joy element, also known as the Stick Switch from Matsushita (Figure 9), is a 
device that accommodates different elements in one system. It consists of a mechanical 
incremental system, a confirmation button and a four-way-switch. 

A trackball (Figure 10) uses a sphere rotating in all directions. The sphere drives two 
rollers, one to measure the motion of the sphere in one direction, the x-position, one to 
take the y-position. Photoelectric barriers measure the spin-information and turn them, 
with the help from a controller, into orientation signals. 

The Joy-Three-Dim device is a small joystick accommodating x-, y- inputs plus a z- co¬ 
ordinate when moved up and down as well as an angular information when rotated. It is 
available as a prototype and can be moved with thumb and forefinger when installed in a 
steering wheel. 

Based on the construction size and the function range, elements like the Interlink Point¬ 
ing device, the FID from Fujitsu and the GHImouse enable efficient input strate^es. 
These devices also include mirror operation in cars. If equipped with noticeable switch¬ 
ing points these device have a wide range of automotive applications. 


The before mentioned control devices where all installed in an experimental setup similar 
to a car cockpit. The experimental task was to operate a subsystem similar to a car navi¬ 
gation system which was simulated on a computer controlled screen. A cursor had to be 
precisely positioned on the screen and menu items had to be selected. The experiments 
were not conducted and evaluated following a strict experimental design. The task cho¬ 
sen could not be standardized at this early point in the development in integrated control 
in a vehicle. The primary aim was no direct ranking of the different products but a 
clearer understanding of their pros and cons with respect to a task similar to the task cho¬ 
sen. A quantitative ranking of the different control setups did not seem to make sense 

Some input elements, like switches, have a mechanical feedback (switching point). This 
allows the user to make inputs without a direct optical control. So he can devote his vis¬ 
ual attention to the traffic. Most of the analog elements do not produce noticeable control 
response. It was difficult to realize whether an appropriate input action was taken. This 
effect makes it particularly difficult to use such control devices while the car is moving. 
This disadvantage could be in part compensated by allowing to use the full range of the 
functions only when the car was not in motion. With the car moving the available input 
functions should be reduced. User friendly control devices, such as touch screens could 
lose their importance in the future. The reason is that screen displays could be installed 
in the secondary vision range replacing the speedometer. 

Certainly those multidimensional sensors will be in demand, which can be integrated in 
the steering wheel. A device with many functions as the joy-three-dim lead to problems 
during operation. Most car owners may not be prepared to work with such a complicated 
device. A four dimensional device will require precise instruction before use to avoid 
wrong input. Pointing devices could do more than just mirror adjustments. They are user 
friendly and easy to apply. 


Figure 11: push trackball 

The push trackball is a patented trackball system (Ohler, 1997). The push trackball was 
installed in the control section a normal steering wheel (Figure 11). In the figure the push 
trackball is installed as the small sphere in the second location from the bottom. It works 
much like a normal trackball when operated by the driver’s thumb. The system works as 
a zero-order or position system. That means, that the screen cursor will move when the 
track ball is rotated and stand still as soon as the trackball is halted. The only difference 
is a switching function built into the device. Pressing the ball down is like pressing a 
confirmation button. The push trackball will then send a signal to the controller. Though 
a bigger push trackball could be installed in the middle console, the miniaturized push 
trackball in the steering wheel was accepted and worked very well. The prototype showed 
little haptic feedback to the driver. A later version should be built that has a noticeable 
stepping effect when rotated. Another version of the trackball was tried out. This track¬ 
ball only works in two directions, up-down and left-right. Such a system gives less flexi¬ 
bility but may sometimes prevent wrong inputs. 


Dependent on different applications, users found numerous the pros and cons using dif¬ 
ferent input devices in an experimental setup similar to operating a car navigation sys¬ 
tem. S)'stems installed in the steering wheel, that is in the primary control range, may 
have advantages over devices installed elsewhere. Screen displays for multifunctional use 
in a car will probably be mounted in the dashboard, in the secondary visual range. Input 
systems with many degrees of freedom can induce input errors due to complicated mode 
of operation and lack of instruction or experience on the driver’s side. A two dimensional 
device such as the push trackball is easy to learn and use, the push function is acceptable 
and effective to be used as confirmation. Devices with no proprioceptive feedback seem 
to be less acceptable. Controls with no clear haptic characteristics such as a missing 
feedback during operation are less favored. Combining more than one input technology 
can be a great help but was not tested. One possible solution is a combination of speech- 
input and the push-trackball. In this case the trackball will be used to interact on the map 
which is displayed on the screen, while the speech-input can be used for commands. 

Complex interaction should be conducted while the vehicle stands still. Yet, there is a 
chance of safe interaction while the vehicle is in motion if a control device is installed in 
the primary control range being operable with a finger while the hand is securely rested. 
A multifunctional control strategy can be established to control a wide variety of on¬ 
board systems if connected properly to a multifunctional screen display and an efficient 
optimized Human-Machine-Interface. 

Jens Ohler(1997) 

Bedienelemente zur Meniiorientierung und -steuerung auf einem Bildschirm im 

Diplomarbeit, Fachhochschule Bochum, University of Applied Sciences, 

Bochum, Germany 


Changing Attitudes of Speed-Limit Offenders 
Using a Multimedia Programme 

Frank JJ.M. Steyvers, Anneke J. Menting & Karel A. Brookhuis 
Centre for Environmental and Traffic Psychology 
University of Groningen 
Grote Kruisstraat 2/1 
9712 TS Groningen 
The Netherlands 


The Traffic Research Centre developed, in co-operation with digiTAAL inc., an 
interactive multi-media computerprogramme to change speed-limit offenders’ attitudes 
with respect to speeding. The computer programme is meant to be used during speed 
controls; the offender may be remitted a part of the fine by completing the programme. 
Objective of the programme is to make speeders aware of the negative implications of 
their behaviour and to change their attitude negatively towards offending speed limits. 
To attain this goal, offenders are confronted with possible negative consequences of their 
behaviour while their arguments for speeding are refuted, using small video-clips, 
demonstrations of counter-arguments and short verbal stories. The effects of this 
multi-media programme were studied in a laboratory evaluation, in terms of knowledge 
and attitudes, compared with two information conditions, a general leaflet about traffic, 
and a specific leaflet about speeding. One week after participation in the study subjects 
were sent a questionnaire again, to measure whether changes in knowledge and attitudes 
were retained afterwards. It appeared that the general attitude towards speeding was 
changed most in the multi-media programme condition, subjects became more negative 
towards offending speed limits and various related aspects. The specific speeding leaflet 
appeared to influence the attitude towards driving fun positively and obeying traffic rules 
negatively, which are unwanted directions. With regard to knowledge of speeding and its 
consequences the computer programme did not do better than the other conditions. 
However, the subjects considered the programme more impressive than the leaflet 
conditions and indicated that they would consent to participate when being stopped in 
real speeding conditions. 


In assignment of the Regional Traffic Safety Council Groningen (ROG) and the 
Ministerial department of Waterworks and Road Maintenance (RWS) the Centre for 
Environmental and Traffic Psychology developed together with digiTAAL inc, a multi- 
media computer programme for driving speed enforcement by police and justice. Drivers 
who violate the speed limit are halted and can choose between the normal fine or 
participating in the programme and a reduced fine. Firstly this paper will summarise the 
social-psychological models that are the basis for the programme. Secondly, an 
experiment will be described that evaluated the programme in terms of its ability to 
change an attitude towards speeding in a laboratory set-up, compared with more 
traditional leaflets. The paper will be finished with some conclusions and 


Behavioural change with multi-media - a social-psychological perspective 

A hterature review was done to find out to what extent multi-media applications may 
teach a different driving speed choice (De Rooij, Werda, Rooijers & Steyvers, 1994). It 
appeared that nothing was found, and this did not change since then. There are, 
howerver, other ways to change traffic behaviour, and there are other behavioural goals 
that were subject to change by multi-media applications. Concerning traffic behaviour it 
appeared that driver improvement courses do not (immediately) change the behaviour of 
problem drivers, because 1) it is difficult to identify these drivers (Veling, 1986), and 2) 
the driver improvement programmes evaluated had serious flaws (Kaestner, 1981). These 
flaws are: the programmes are far and foremost verbal, whereas the goal behaviour is 
sensory-motoric, only attitude was the most important goal, thus underestimating 
external factors, accident involvement is not a personality trait, there were no controls for 
experimenter effects of placebo effects, and although a large attitude change was 
estabhshed, there was no change in behaviour. In spite of this, attempting to influence 
behaviour by influencing attitude is the only way to try to establish a "change from 
within", since other and proven more effective methods are politically, technically and/or 
financially unviable. Heavy enforcement and fining campaigns may also bring about a 
behavioural change, but this is not "from within", and will end as soon as the campaigns 
come to an end (see e.g, Rooijers, De Waard & Soder, 1992). 

The theoretical basis for a multi-media application to change (attitudes towards) 
speeding behaviour comes from the "Theory of reasoned action" (Fishbein & Ajzen, 
1975). Behaviour is thought to originate from a behavioural intention, which stems from 
attitudes and subjective norms. Attitudes are modulated by beliefs about the 
consequences of the behaviour in question, and evaluations about these consequences, in 
this case of speed in terms of driving fun, risks, travel time etc. The subjective norm is 
the expected judgement of the social environment and is modulated by normative 
believes about speed and by a personality factor that may be called tendency to comply. 
Some remarks are in place here. This model implies that drivers constantly have 
conscious control over their behaviour, which is not so. Furthermore, the model imphes 
that behaviour is always the consequence of an intention, which is not so. And attitudes 
are, in this view, rational and objective, only to be changed by rational activities, which 
is too narrow-minded. 

To counteract these objections the model had to be extended: Ronis, Yates & Kirscht 
(1989) expanded the ways behaviour is created. There are two ways: reasoned and 
unreasoned influences. Intentions are part of the reasoned influences, whereas habits are 
part of the unreasoned. Both these influences are subject to modulation by internal and 
external stimuli (such as infrastructure, weather, time of day, reason for vehicle use etc,), 
and behaviour that originated from them may be influenced by facilitating factors (such 
as the availability of a car, driving skill and knowledge, etc). Hence this model 
incorporates the possibility for other than rational-conscious factors to influence 
behaviour. Driving habits are very strong behavioural modulators. Furthermore one may 
argue that before a trip is taken various social factors may influence behavioural 
intentions, but once in the "wheeled cage" the possibilities for influencing drivers by 
social factors diminish strongly (ways to communicate with drivers by other drivers are 
minimal), and hence within a certain task and environmental context habits take over. 


Ronis et al. however maintain that habit change only will take place when repeatedly 
conscious information processing takes place. The driver, however, will resist reasons for 
other behaviour. Petty & Cacioppo (1986) incorporate this resistance to change in their 
"Elaboration likelihood model". This model has two routes by which behaviour may 
change. When the driver is willing and able to do conscious and rational information 
processing, persuasion of the driver towards the wanted behaviour is possible by a direct 
or central route of rational information. When the driver is not willing and/or able to 
consciously process rational information a kind of detour, a peripheral route has to be 
taken. In stead of a rational an emotional and associative approach has to be taken. 

In case of speeding behaviour all attempts to change behaviour can be expected to meet 
certain resistance. Speeders, halted in an enforcement campaign, are expecting 
beforehand attempts of influence and arm themselves mentally with arguments in favour 
of their (unwanted) behaviour. Influencing them by taking the peripheral route may be 
more successful, using associations, and emotionally loaded arguments. Using a 
computerised multi-media programme it may be an advantage that the speeder may be 
actively involved in handling the application (interactive approach), that more or less 
individual courses through the programme are possible, an that the emotional impact 
may be augmented by using various media (sound bites, short films, pictures of the 
actual situation of the offence etc.). 

The basis of the actual programme is formed by eleven reasons speed offenders may 
think of for their behaviour: I was in a hurry, speeding is fun, I was behind a snail, this 
speed is best for my car, I was not aware of my speeding, I did not know you the actual 
limit, the speedometer was broken, I don’t accept the local hmit, I decide my driving 
speed myself, all drivers driver more than the limit here, and there is never a speed 
control. Each of these is counteracted by both rational and emotional arguments, by short 
and hefty pictures, films or sounds. Furthermore the offender is confronted with the 
negative outcome of speeding behaviour by him/herself, by engaging him/her in various 
tasks, such as a reaction time task, brake distance estimation task and the like. 

A laboratory evaluation 

To find out whether the programme is better in changing attitude than traditional leaflets 
an evaluation was performed. A situation was invented, a 50-km/h road running through 
a suburban village. The situational specific frames in the programme were equipped with 
pictures and accident information of the location. There were three conditions for the 
between-subject study: the programme, a specific speed-related leaflet, and a general 
traffic-safety leaflet. In each condition 20 subjects were scheduled: experienced drivers 
that once or even more often were confronted with a fine for speeding, 57 subjects 
actually participated in the study (33 male, 24 female): 18 in the programme condition, 

19 in the specific leaflet condition, and 20 in the general leaflet condition. After an oral 
introduction subjects had to finish a questionnaire for assessing their attitude. Then they 
were confronted with their condition (doing the programme or reading the leaflets). Then 
they had to finish a questionnaire again. One week after participation in the lab a 
questionnaire was mailed to them, for assessing their attitude again. 

The attitude questionnaire consisted of three bi-polar five-point scales (from -2 to +2) 
anchoring with good - bad, fun - no fun, attractive - unattractive. The composite 


(summated) score is interpreted as the attitude towards speeding. Table 1 shows these 
summated scores, separately for each condition and the pre- and post-treatment 
assessment. As can be seen the attitude in the programme group towards speeding is 
significantly decreased (speeding became worse, less fun and less attractive), whereas for 
the two other groups there was no (significant) change. Furthermore, the difference 
between the programme group and the specific leaflet group was significant in the post¬ 
treatment assessment: the programme group became more negative about speeding, 
whereas the specific leaflet group did not. 

Table 1: Summative attitude scores for each of three treatment groups, separately for pre- 
treatment and post-treatment assessment. Scores range from -2 to +2. * = significant 
difference between pre- and post-treatment assessment. 

Treatment group pre-treatment 


I Multimedia Programme 0.0 

n Specific Leaflet 0.4 

in General Leaflet -0.6 



- 1.1 (*) 


- 0.2 

There were also questions about motives. The following aspects were assessed: agreeable 
driving speed for the vehicle, driving fun, having self-control of the speed, the vehicle 
remains "speedy", endangering other drivers, traffic rule compliance, lacking time to 
react on other drivers’ behaviour, probability of a speeding fine, incorrect estimation by 
other drivers of the subject’s speed, gaining time, being quickly at the destination. Each 
of these aspects was expressed as a short statement. Subjects had to indicate their 
measure of agreement with these statements (from -2 to +2) and their measure of 
considering this important (also from -2 to +2). The product of these scores provides a 
score for behavioural motive. Table 2 shows these scores, for each of the treatment 
groups, separately for pre- and post-treatment assessments. 

Table 2: Mean scores for the behavioural motives of the three treatment groups, 
separately for pre- and post-treatment assessment. Scores range is from -4 to -i-4. A high 
score means an advantage for speeding. 

Subjects in the various treatment groups appeared to differ significantly in their opinion 
about various aspects. Since this was an indicative study the significance level was 
placed at 0.10 in stead of the usual 0.05. The specific-treatment group in the pre¬ 
treatment assessment considered driving fun more negative than the programme group 
(t=1.70, df=35, p < 0.10). The general-leaflet group was more negative about probability 
of a speeding fine (t = 2.39, df = 37, p < 0.05) and about the vehicle remaining more 
"speedy" (t = 1.70, df = 37, p < 0.10). Subjects of the programme group considered the 
aspect being quickly at the destination more positive than subjects from the general 
leaflet group (t = 2.05, df = 36, p < 0.05). In the post-treatment assessment subjects of 
the specific leaflet group considered the vehicle remaining more "speedy" a more 
positive aspect of speeding than the subjects of the general leaflet group (t = 1.80, df = 
36, p< 0.10). 


Pre-treatment assessment 
post-treatment assessment 

Treatment group I 







Best speed 

for the car 0.7 






Having driving 







Self determination 
of speed 1.1 






Car remains more 
’speedy’ 1.2 






Endangering other 
drivers -0.7 






Compliance of 
traffic rules -1,4 






Lacking time to react 
to others’ behaviour-0.2 






Probability of speeding 







Incorrect speed estimation 
by others -0.8 














Being at destination 
more quickly 1.6 






Then the change between pre- and post-treatment assessments were compared. Subjects 
in the programme group became less positive about the aspect of gaining time (t = 1.80, 
df = 17, p < 0.10). Subjects of the specific leaflet group considered the aspect diving fun 
in speeding more positive (t = 2.09, df = 18, p < 0,10) and traffic rule compliance more 
negative (t = 2.52, df = 18, p < 0.05)!. They became more negative about the time to 
react to other drivers’ behaviour (t = 2.51, df = 18, p < 0.05), and about the incorrect 
estimation by other drivers of the subject’s speed (t = 2.05, df = 18, p < 0.10). Subjects of 
the general leaflet group considered driving fun more positive (t = 2.01, df = 19, p < 
0.10), as well as probability of a speeding fine (t = 3.01, df = 19, p < 0,01) and gaining 
time by speeding (t = 1,76, df = 19, p< 0.10). 

There were questions concerning the specific situation that was well known to all 
subjects. They considered the speed limit of 50 km/h quite sensible, and they thought 


that traffic safety would increase when all drivers would comply to that limit. The 
statement "I comply to a limit earlier when accompanied in the vehicle than when alone 
gave varied levels of agreement, hi the post-treatment assessment subjects of the specific 
leaflet group more often were totally opposed the statement (chi-square= 13,85, df=8, p < 
0.10), compared to the other groups. Subjects from the general leaflet group and the 
programme group considered police speeding control more sensible than subjects from 
the specific leaflet group. This was found both in the pre- and in the post-treatment 
assessment (respectively chi-square 15.11, df 6, p < 0.05 and chi-square 11.44, df = 6, p 
< 0.10), hi the post-treatment assessment the difference became smaller. All subjects 
estimated the probability of a speeding fine on the specific location rather small. 
Unnoticed one tends to drive faster than allowed. A speeding fine was considered very 
unpleasant. Subjects reacted variously when asked wether on would driver faster if speed 
controls were certainly not to be held. 

In pre- and post-treatment assessments also knowledge was tested by asking questions 
about the following aspects: proportion of drivers that complied to the speed limit at the 
specific (fake) location, liability in case of an accident with a bicyclist, insurance policies 
in this case (loss of no-claim reductions), braking distance at 50 km/h, braking distance 
at 80 km/h, physical impact of a collision with 50 km/h in terms of a free-fall jump, 
meaning of traffic sign "end of built-up area" for speed limit, estimation of time gained 
by a speed of 20 km above the limit at a stretch of 10 km, magnitude of the fine in case 
of a speed of 15 km above the limit, speed over the limit that may cause the drivers 
licence to be withdrawn. In the pre-treatment assessment there were no differences in 
knowledge between the three treatment groups. In the post-treatment assessment there 
were two differences. The subjects of the specific leaflet group and (less pronounced) of 
the programme group responded more often that the drivers licence might be withdrawn 
at speed limit violations between 31 and 60 km/h (chi-square 13.24, df = 6, p < 0.05). 
Forty-five percent of the respondents in the general leaflet group thought (correctly) that 
a non-guilty collision with a bicycle would cause the loss of the no-claim insurance 
reduction, whereas in the programme group and the specific-leaflet group this was 
seventeen and thirty three percent respectively (chi-square 8.23, df = 4, p < 0.10). All 
respondents underestimated in both pre- and post-treatment assessments the proportion 
of drivers complying to the speed limit. Most drivers were aware of the liability in case 
of a non-guilty collision with a bicycle, but that this would mean the loss of the no-claim 
reduction of the insurance was not common knowledge. The meaning of the sign "end of 
built-up area" in terms of a speed limit of 80 km/h unless otherwise stated was common 
knowledge. And subjects estimated the time gained by speeding, and the fine in case of a 
speed limit violation of 15 km/h quite accurately Fur&ermore there was a systematic 
underestimation of braking distances for speeds of 50 and 80 km/h. Finally subjects 
underestimated the speed limit violation at which police officers may withdraw the 
drivers licence. 

To gain information about the medium, especially the multi-media programme, 
immediately after the treatment a questionnaire was filled in. Subjects had to rate on a 
five-point scale (good-bad) the following aspects of the topics treated: educational-not 
educational, understandable-not understandable, clear-unclear, making sense-not making 
sense, good-bad, boring-exiting, realistic-unreahstic, fun-no fun. And they had to rate the 
way the topics were presented with: fun-no fun, understandable-not understandable, 


making sense - not making sense, good-bad, real-artificial. Subjects were more positive 
about the programme than about the leaflets (chi-square 12.30, df = 6, p < 0.10). 
Furthermore subjects of the programme group tended towards a more positive judgement 
about the contents of the programme. They considered it more understandable, exiting, 
and realistic. On other aspects the programme scores as good as the leaflets, with the one 
exception that the general leaflet was more "making sense" than the other treatments. 
This positive rating of the programme was also found for the aspects of treatment of the 
topics. The specific leaflet however, was found more understandable than the other two 
treatments. In general, the multi-media programme was appealing to the subjects. About 
the length subjects were diverted. Two considered it too long, nine neither too long nor 
too short, and seven too short, one remarking "a punishment in terms of time is horrible 
for speeders". Executing the computer was no problem. Given the choice between the 
programme and the full fine most (16 of 18) subjects would take the programme. Finally 
details of the programme were given ratings and subjects gave open-end remarks about 
anything they would like to comment on. 

Discussion and conclusions 

The results of this evaluation study may be summarised as follows. The multi-media 
computer programme for educational enforcement of speed limit offenders did bring 
about a change in attitude towards driving too fast. It became significantly more 
negative, whereas subjects treated with a more traditional medium - specific or general 
leaflets - did not show such an attitude change. Subject in the general leaflet group even 
showed a slightly more positive attitude towards speeding, and subjects in the specific 
leaflet condition remained more or less positive. Furthermore opinions about various 
aspects of speeding were assessed. The programme caused subjects to be less positive 
about the gain of time by speeding. Subjects are now aware that speeding does not 
automatically result in gain of time. The specific leaflet brought about that subjects 
became less negative about driving fun by speeding, and less positive about complying to 
traffic rules. This is in contrast with what one would want to achieve in such treatments. 

The programme group did not gain more knowledge about the consequences of speeding, 
compared to the other two treatment groups. It appears that subjects already possess quite 
some knowledge in this respect. However the way the various topics are treated was 
judged more positive for the programme than for the other treatments. This was the case 
in this laboratory experiment. In a real-life setting the advantage of the programme over 
leaflets may become even larger, hi this experiment subjects were actually reading the 
leaflets, whereas in real life people take the leaflet, put it away in order to drive on, and 
forget its existence. 

Therefore, on may conclude that the programme may contribute to a change in attitude 
towards driving speed violations in general, and about the "myth" of time gained by 
speeding in particular. The question whether this treatment not only will cause a change 
in attitude but also in driving behaviour is the next step in the evaluation of the 
programme. Field trials are planned on both 50 and 80 km/h roads. 



De Rooij, A.MJ., Wierda, M., Rooijers, A.J,, & Steyvers, FJ.J.M. (1994), Een 

verkennende studie naar computerleerprogramma's als altematieve straf voor 
snelheidsovertreders, [An exploration towards computer training programmes as 
an alternative punishment procedure for speeders] Haren: Traffic Research 
Centre, University of Groningen, The Netherlands. Report VK 94-01. 

Fishbein, M, & Ajzen, L (1975). Belief attitude, intention and behavior: an introduction 
to theory and research, Reading, Massachusetts: Addison-Wesley. 

Kaestner, (1981), In K,B. Joscelyn & R.K. Jones (Eds.), Managing the traffic risk. Ann 
Arbor, Michigan: The University of Michigan, Highway Safety Research 

Petty, R.E„ & Cacioppo, J.T. (1986). Communication and persuasion - central and 
peripheral routes to attitude change. New York: Springer Verlag. 

Ronis, D.L., Yates, J.R, & Kirscht, J.R (1989). Attitudes, decisions, and habits as 
determinants of repeated behavior. In A.R. Pratkanis, S.J. Breckler, & A.G 
Greenwald (Eds,), Attitude structure and function. Hillsdale, NJ: Lawrence 
Erlbaum Associates. 

Rooijers, A.J., De Waard, D., & Soder, J.C.M, (1992). De effectiviteit van maatregelen 
ter beheersing van de rijsnelheid - een overzicht. [The effectiveness of measures 
to control driving speed - an overview] Haren: Traffic Research Centre, 
University of Groningen, The Netherlands. Report VK 92-09. 

Veling, LH. (1986), Verkeersvorming van probleemchauffeurs in Nederland. [Traffic 
training of problem drivers in The Netherlands] Veenendaal: Traffic Test. TT86- 


Driving Simulation Systems as fast Tools to evaluate 
different Types of Head-Up Displays in a Vehicle 

Andreas Penka, Rolf Gengenbach and Heiner Bubb 

Institute of Ergonomics 
Technical University of Munich 
Garching, Germany 


A Head-Up Display (HUD) is an optical control which is mirrored in the windscreen of a 
car. The apparent distance of this kind of display can be varied with the help of the 
optical system used. HUDs can be used to offer information to the driver without forcing 
him to accommodate to the instmment panel. This reduces the time the driver needs to 
perceive the presented information. Experiments with Head-Up speedometers show that 
subjects prefer using the HUD. After a short period of habituation the still available 
normal speedometer is used very seldom. 

HUDs can also be used to display additional information: For example the distance a 
vehicle moves during the reaction period of the driver. 

Actual experiments examine the use of HUDs for new navigation assistance systems. In 
this case the information, normally presented on a small screen in the car or as acoustical 
information, is presented directly in the environment and is perceived similar to road 
signs or traffic-line markings. 

Because of reproducible traffic situations, short implementation time for new display 
types (they must simply be added to the simulation scene) and the possibility of 
examination of even dangerous situations with normal subjects, statements based on 
statistics can be made faster than in not simulated experiments. 

In this presentation the actual research and results will be presented with an additional 
short video demonstration. 


Looking at the control panel to read the actual speed or other information about the car is 
something the driver of a car does very often. Normally the driver has to look away from 
the outside traffic situation in order to perceive the indicated information. To reduce the 
time a driver is turning away his attention from the outside some car manufacturers made 
studies whith HUDs - which were used in air planes for a long time - in their vehicles. 
Although the first prototypes for car HUD systems were realized more than ten years ago, 
only a few companies integrated simple HUDs in their cars on the market (e.g. Buick 
ParkAvenue, Nissan Bluebird and Pontiac Bonneville). Many of the other companies 
regard the HUD technique as a topic of basic research for future developments. They 
don’t believe that the HUD offers a security enhancement even if there are a lot of studies 
which show the advantages of the HUD technique. Some of these mentioned by 
Schneider et. al. (1992) are: 


• The driver perceives the indicated information in a smaller viewing angle to 
the traffic situation. 

• There is less or in some cases even nearly none accommodation between near 
and far distances needed to read the indicated information. 

• Adaptation to different lightning conditions on the board instruments and the 
outside of the car are no longer necessary. 

All these items lead to shorter periods of time in which the driver is turning his attention 
from the traffic situation. For example Bartolomai (1990) found that the time to read the 
speedometer in different conditions can be reduced from a range of 100 to 2000 ms to a 
range between 100 and 800 ms by using a HUD speedometer. As an interesting detail 
about HUDs Bubb (1992) mentioned the fact that air planes with this technique have to 
pay less insurance fee, because of the improved security level. 

How an HUD works 

Figure 1: The basic implementation of a HUD 

The basic implementation of a HUD as shown in figure 1 is always the same: An 
ordinary display consisting of any source of light (e.g. an arrangement of LEDs or an 
TFT display) is mirrored in the windscreen of a car by an optical system. The optical 
system must be adjustable to the viewpoint of the driver, so that the user perceives the 
HUD at the right position behind the windscreen. The distance of the virtual image of the 
HUD depends on the optical system used. For simple HUDs often a distance between 2.5 
and 3.5 meters is used. This distance is easy to realize and works for all standard 
information such as speed, rounds per minute, remaining fuel and motor temperature 
warnings. An disadvantage of these simple HUD systems is that the HUD seems to move 
in relation to the outside traffic situation whenever the driver moves his head. Therefor 
this simple system has to be replaced by another with a variable or a larger apparent 
distance of the HUD to indicate information that should be combined with the outside 
traffic situation by the driver. A HUD that offers such information is called a contact 
analogue HUD. Some applications of such HUD systems will be described later. 

Our own experiences using HUDs 

In our institute there where a lot of studies related to HUDs. Most of them where done 
with HUDs in real vehicles. These studies concerned with HUDs which had a fixed 
apparent distance to the driver. Most of the studies like those of Assmann (1985) and 
StUrzer (1983) were about a new HUD concept that allows to indicate the distance a car 


moves during the reaction period of the driver. This HUD with an apparent distance of 
eight meters consisted of a rectangle that could be varied in si 2 e and height to simulate a 
green bar lying on the road in the desired distance. The apparent distance of eight meters 
was enough to achieve a sufficient realistic impression. As a base of these studies 
Bierbrauer already did studies in an other simulator with this concept in 1980. 

Other studies at our institute dealt with the habituation of subjects to a HUD speedometer 
similar to the one shown in figure 2, In these studies our line of sight detection system 
JANUS was used to get information about how often and how long a subject looks at the 
HUD and the still available ordinary speedometer, Gengenbach (1997) showed that all 
subjects in this study preferred using the HUD speedometer. He also confirmed the 
shorter time that is needed to read a HUD with his evaluation. 

Figure 2: Current implementation of a HUD speedometer in our simulator. 

The simulator as a tool to evaluate new HUD concepts 

A lot of our studies about HUD systems tried to show the advantages of contact analogue 
HUDs. To obtain the best possible quality for this kind of display there were many 
experiments with adjustable optical systems. Gengenbach (1993) even made studies with 
a holographic HUD. A result of all these studies was, that there is a lot of engineering 
necessary to realize a HUD system that is able to indicate an contact analogue HUD for a 
special purpose. In addition an optical system that worked good with one HUD concept 
often couldn’t be used with others. The free programmable TFT display used by many 
vehicle manufacturers, which is very flexible to examine new standard HUD concepts are 
also not suitable for these kind of studies. One way to generate a tool that makes it 
possible to examine the mammachine-interface of new contact analogue HUDs without 
having to develop a complicated optical system that is able to indicate this HUDs in the 
desired way was to use virtual HUDs in a simulator. 

By using a virtual HUD it is also much easier to examine concepts that normally would 
cause a lot of expense. For example a HUD prototype for a navigation assistance system 
that normally should be linked to an on board GPS (Global Positioning System) or for a 
driver assistance system that indicates warnings if the vehicle gets to near to the roadside 


or if it gets to close to the car ahead can be implemented by using the already available 
information about the positions of all objects in the simulated scene (including other cars) 
and an additional database that includes the rights of way at road junctions or road 
crossings, the speed restrictions, information about traffic signs and the suggested courses 
for a navigation assistance system. It is also much easier to realize completely new ideas 
like an assistance system for parking between two other cars where a HUD could be used 
to visualize the distances to other cars, traffic-line markings and other obstacles in all 
directions of a car without having to integrate a lot of measurement instruments in a 
prototype vehicle. 

The simulator offers a wide palette of possibilities to implement HUD systems. The 
virtual HUDs can be moved in relation to the car by using DCSs (Dynamic Coordinate 
Systems), the indicated HUD can be modified in size, color, transparency and brightness 
by using morph nodes and last not least it is possible to implement HUDs with different 
states such as the digits of a speedometer by using switch nodes. For more information 
about the graphical abilities of a simulation system it’s recommendable to have a look at 
a handbook for a simulation software like the IRIS Performer library by SGI. 

Examples of contact analogue HUDs implemented in a simulator 
The first contact analogue HUD that was integrated in our new simulator was the already 
mentioned security distance indicator. To be able to compare the virtual HUD with the 
one we examined in a real car we did two different implementations. The first was a 
green rectangle with an fixed distance of eight meters that could be varied in height and 
size to get nearly the same impression as with the display that was implemented in the 
real car. The second implementation was a simple rectangle (shown in figure 3) which 
wasn’t modified in height and size, but was lying on the road and moved to the point of 
the road where the car would be after the reaction period of the driver. 

Figure 3:Contact analogue security distance HUD 

The main advantages of the second implementation were that head movements of the 
driver didn’t result in apparent movements of the indicated HUD in relation to the road 
surface and that the HUD didn’t had to be adjusted to the viewpoint of the driver. 


Another HUD project in our simulator is about navigation assistance systems. These 
systems are expected to show the driver where he should turn left or right in order to get 
to a spotted point of the scenery. A simple and quite good possibility for such a system 
would be another car which leads you to the selected location. As it is not possible to 
indicate a leading vehicle in a HUD, because this display would hide to much of the 
surrounding and would take to much of the attention of the driver, an more simple HUD 
had to be implemented which offered nearly the same comfort. A first implementation as 
shown in figure 4 was a simple HUD which indicated the navigation systems advises in 
an symbolic way. 

Figure 4: Simple HUD for navigation control information 
This HUD system was sufficient in most traffic situations. Only if there is a short 
succession of road junctions the indicated information might not be clear. To improve 
this HUD a contact analogue HUD was realized which consisted of a set of arrows which 
had a similar appearance like traffic-line markings and which were virtually laid down on 
the road surface just a few meters in front of the road crossing, where the driver had to 
turn left or right. This HUD which is shown on figure 5 was easy to interpret by the 
driver but couldn’t be read from a distance from the crossing. As a result of these studies 
both of the two displays were combined. The simple HUD was used to indicate that the 
driver has to turn left or right in the next seconds and the contact analogue HUD was used 
to indicate the exact position where he had to leave his present road. 


Figure 5: Contact analogue HUD for a navigation control system. 

Driving simulators are not in the first place a tool to adjust existing HUD concepts to a 
special vehicle, but a good tool to examine the man-machine-interface of new kinds of 
HUDs without the hmitations of acmal technical practicabilities. By doing some studies 
with a new HUD concept in a simulator the expense to inte^ate a prototype in a vehicle 
can be reduced, because it is possible to limit the indicated information to those parts of 
the concept that were accepted by subjects. 

As an additional field for simulator studies it is possible to examine if the HUD technique 
improves the security in different traffic situations. For example it would be possible to 
evoke a special event sometimes the driver reads the speedometer. This event - for 
example a ball that suddenly rolls on the road - should cause the driver to reduce his 
speed immediately. By comparison of the periods of times needed to reduce the speed by 
5 km/h for example, or by comparison of the part of events where an accident could be 
avoided an objective security judgement of the HUD technique in vehicles would be 
possible. In order to measure the influence of the HUD technique and not just typical 
simulator effects a kinesthetic feedback to the driver is absolutely necessary to avoid the 
loss of control while having no optical feedback about the actual driving situation. To be 
able to do these kind of studies in our simulator, too, we’re actually examine different 
possible kinesthetic feedback systems for our simulator. 



Assmann, Ernst (1985): Untersuchung uberden Einflufi einer Bremsweganzeige aufdas 
Fahrverhalten. Dissertation am Institut fur Ergonomie der TU Miinchen. 

Bartholomai, G,; Becker, O.; Schneider, W und Walter, W. (1990): Ergonomische 
Bewertung der Ablesbarkeit digitaler Instrumente bei verschiedenen 
Lichtverhdltnissen, In: Derkum, H. (Hrsg.)* Sicht und Sicherheit im 
StraBenverkehr: Beitrage zur interdisziplinaren Diskussion, Kdln, 

Bierbrauer, Franz-Josef (1980): Der Einflufi der Bremsweganzeige auf die Leistung des 
Kraftfahrers, untersucht an einem Simulator. Diplomarbeit am Institut fiir 
Ergonomie der TU MUnchen. 

Bubb, H.; Reichart, G. (1992): Braucht man HUD's im Kraftfahrzeug? Ergebnisse einer 
Literaturrecherche und eigener Versuche. Das Mensch-Maschine-System im 
Verkehr, VDI Bericht Nr. 948, Dusseldorf: VDI-Verl. 

Gengenbach, Rolf (1993): Konzept und Untersuchung eines holografischen Head-Up 
Displays zur Visualisierung des Bremsweges bei Pkws. Diplomarbeit am 
Lehrstuhl fiir Ergonomie, TU Miinchen. 

Gengenbach, Rolf (to be published in 1997): Fahrerverhalten im Pkw mit Head-Up 

Display - Gewbhnung und visuelle Aufmerksamkeit, Dusseldorf: VDI-Verlag. 

Schneider, W.; Beeck, M.-A.; Thomas, J. und Rackow, L (1992): Digitale 

Geschwindigkeitsanzeige im HUD, ein Beitrag zur Verkehrssicherheit?, Das 
Mensch-Maschine-System im Verkehr: VDI Berichte Nr. 948,Dusseldorf: VDI- 

Silicon Graphics International (publisher): IRIS Performer Programmer’s Guide. Online 
Help System on SGI workstations, 

Stiirzer, Walter (1983): Einflufi des Head-Up Displays des Bremsweges aufdas 
Abstands- und Reaktionsverhalten des Kraftfahrers, untersucht an einem 
Simulator. Diplomarbeit am Lehrstuhl fiir Ergonomie der TU Miinchen. 


Analysis of Hand Position in Unstructured Environments 

Lutz Geisen, Otmar Bock 
Physio logisches Institut 

Deutsche Sporthochschule, Carl-Diem~Weg 6, Koln, 50933 Koln Germany 
Email: geisen@hrz.dshs-koeln,de, 


We are developing a system for the marker-free analysis of hand position in real-time, 
which performs in undefined, non-stationary environments. The system is based on a 
frame-to-frame SSD correlation of greyscale video images, and currently provides the 
two-dimensional coordinates of thumb, index fingertip and hand at a frame-rate of 14 
Hz. This system can be applied as a man-machine interface(e.g. replacing the mouse), 
and for the recording of human motor performance in complex, uncontrolled scenarios 
(e.g. clinics, military, aerospace). 


Several systems for the determination of 3-dimensional limb positions are available on 
the commercial market. They all require that „markers“ are affixed to various locations 
on the limb, and track the marker positions through space. Some systems employ active 
markers such as light sources (e.g. Selspot®, Optotrack®) or induction coils (Fastrak®), 
while others use passive markers such as fluorescent balls (Macreflex®). Unfortunately, 
however, all markers obstruct the free movement of the limb, due to their physical size, 
weight, and in case of active markers the leads to operate them. Thus, markers modify 
what they are supposed to measure. 

As a second problem, marker-based systems require special environments for error-free 
operation, such as rooms free of metal or glossy surfaces. This excludes their application 
in many real-life scenarios, where environment is not controlled. 

As a consequence, we decided to develop a marker-free systems which can operate in 
undefined, nonstationary and nonhomogenious environments. To be of practical use, the 
system should have a spatial accuracy of about 1 mm and a temporal resolution of 100 
Hz. Such a system could be deployed in scenarios such as virtual reality, man-machine 
interaction, testing of human performance in clinical, military, and aerospace settings, or 
quality control. The long-term objective is to achieve 3 dimensional measurements by 
means of 2 cameras. 


In the following we describe the actual state of the development and the methods used. 
Test setup 

The test semp of this system is shown in figure 1, 

Simple graphical objects varying in shape and position are displayed on a computer 
monitor. The subjects task is to point onto the targets with his index finger or to grasp 
them with his index finger and thumb Fig 1, These movements are recorded by a camera. 


the signal of which is digitised, is sent to the CPU of a PC, and evaluated to yield 2D 
hand position coordinates in real-time. 

The working range for the hand is 50 by 50 cm with a distance of 90 cm between 
camera and screen. 


The actual hardware used is one camera DALSA CAD 1-256 with 8 bit greyscale and 
256*256 pixel resolution, a lens COSMICAR 8mm, one framegrabber CORECO 
OCULUS F64 (with signal processor TMS 320C40 and special image processing 
functions) and one standard Pentium MMX 200 MHz PC. 

Possible ways of image analysis 


Fig, 1 Test setup. 

To determine hand positions from the recorded frames, we could use several techniques: 

1. Edge detection with fixed or adaptive threshold level. 

2. Filtering with different kind of operators (Laplace, Sobel, Prewitt...) and subsequent 
edge detection with fixed or adaptive threshold level.[1,3] 

3. Knowledge based methods which fit descriptive models direct in the greyscale or 
filtered image. [1,2,6,7,8,9,10] 

4. Transformational methods, e.g., Gabor filter, FFT, Hough transformation to change 
from the spatial space into a transformation space. [1,2] 

5. Neural networks which work on greyscale, filtered images, or following an 
transformation method.[l 1] 

6. Direct pattern searching in the greyscale image with pattern matching 
algorithms. [1,2,4] 

Methods 1 and 2 are highly dependent on in the quality of the image background and the 


structure of illumination and their result is not an distinct position. Their benefit lies in 
the field of image pre processing, and augmented by simple knowledge based rules. 

Methods 3,4 and 5 are cumbersome and time consuming, and may therefore be 
unsuitable for high speed real-time applications. 

We therefore decided to use method 6, based on a fast pattern matching algorithm similar 
to theSSDalgorithms[4]. 

The pattern matching method 

The presented method is based on a combination of pattern matching and knowledge 
based evaluation of the image. A modified SSD algorithm is used with dynamically 
updated search pattern and greyscale discrimination to determine the required positions. 
The knowledge based component is used to determine the search areas, and the greyscale 
discrimination parameters to refine the determined finger positions. 

To start the analysis, the subject has to bring his hand in a pre specified starting position 
in front of the camera. The software then determines the pixel pattern of the known 
locations of thumb, index finger and the area where thumb and index finger meet (which 
we will call "hand'* in future) as shown in Fig.2. these patterns serve as templates for the 
subsequent search algorithm. The size of these templates is 14*14 pixel for the index 
finger and thumb, and 22*40Pixel for the hand. The system determines the mean grey- 
value from a 6x6 window of the centre of the hand template. 

As a next step, the pattern matching algorithm searches in the second frame for a pixel 
pattern that is maximally similar to that of the hand template from the first frame. This 
search is limited to a search area of 39x57 pixels and the template is displaced within 
this area in horizontal and vertical steps of 3 pixels. Once a best position is found, the 
search is refined locally displacing the template in steps of 1 pixel. 

As a further step, the algorithm searches in the second frame for a pixel pattern that 
resembles the thumb and index finger templates, respectively, within a search area of 
23x23 pixels. ITie templates are displaced in this search in steps of one pixel. To prevent 
that the found pattern "runs" towards the hand pattern [5], the algorithm searches not for 
the maximum similarity, but rather for a location as far as possible to the left of the 
maximum (for right handed subjects) which still yields at least 90% of the maximum 


index finger pattern 

Thumb pattern 

Fi^, 2 Search templates. 

Once the hand and the finger pattern are found in the second frame, they are used as 
templates in the third frame, etc. in an iterative procedure. 

To determine the grade of similarity between a template and a pixel pattern in a image, 
we use the modified SSD algorithm[4] shown in function (1). This function gives a value 
MR(sx,sy) that becomes 0 at the location sx,sy in the image with best coincidence 
between template and the pixel pattern. The factor k provides the greyscale 
discrimination. If the grey value of one pixel at x,y in the template or the corresponding 
position x+sx,y+sy in the image lies not within ± 25 % of the mean grey value, k 
becomes 0, else k is set to 1. Nx and Ny represent the size of the template N = Nx*Ny is 
the normalisation faktor, with each k = 1 N becomes decreased by 1 to ensure a proper 
normalisation, Ixy and P^y are the grey values of the template and image respectively. The 
result of the search Function (1) is shown in Fig. 3 for a large search area and an index 
fingertip as search template. 

( 1 ) 

Nx Ny, , 

L £ k*(Px,y- 

MR(sx,sy) = 




Old handposition Atx \ 

New handposition 

Fig 4 Determination of finger search areas. 

Further features of the algorithm 

In order to increase the speed of the search algorithm, we displace the finger search areas 
in each frame along with the displacement of the handposition. To achieve this we use 
the relative distances of the finger positions to the hand position (Afx,Afy) and (Atx,Aty) 
in the previous frame and use these with the hand position in the actual frame to 
determine the search area for the finger positions Fig 4. So we could reduce the finger 
search area shift to 23 pixels in x and y direction. Due to the fact that the template and 
the search area are now overlapping with about 50% of template size, the algorithm 
becomes more robust against locations in the image background with a high grade of 
coincidence to the searched template.. 


With the system presented in this paper, we achieve an recognition rate of 15 frames per 

second, or 67 ms per frame, yielding 3 coordinates per frame. The frame time consists of 
20 ms for image analysis, and 47 ms for transferring the images from the frame grabber 
to the PC. 

Hand and finger positions are tracked successfully and independent from image 
background, while the hand executes transitional or rotational movements, or changes its 

The accuracy of tracking the finger positions is ± 3 mm from frame to frame, and ± 6 
mm across 30 frames. 

The maximal trackable actual velocity is ± 25 cm per sec for the hand, and ± 12 cm per 
sec for the finger. 

With an increase of the recognition rate to 100 frames per second the maximal trackable 
velocities should increase to ± 166 cm per sec for the hand and ± 80cm per sec for the 
finger. These values correspond to very fast real life hand movements. 

Next steps 

Our future work will address the following issues. 

• Improvement of accuracy for the finger positions by improvement of the search 
technique which actually determines the leftmost finger position (for right handed 

• Resolving problems with overlapping fingers by adding shape-orientated components 
into the matching algorithm 

• Increase of processing speed by migrating and parallelisation of the algorithm from 
the PC to the signal processor and the coprocessing system, and by reducing the 
search areas by means of a prediction algorithm for the next finger and hand position. 

• Introduction of three dimensional processing with 2 cameras to achieve three 
dimensional coordinates. The extension of the system to two cameras will provide real 
3 dimensional coordinates and the possibility to dynamically adapt the size of search 
templates and search areas. While the algorithm gives only 2-D coordinates it is a 
relative simple task to transform the two camera outputs. This task can be simplified 
by a dextrous camera arrangement. 

• Automatic determination of the positions in the very first frame by means of neural 
networks which recognise the hand shape irrespective of their position. This approach 
which has recently started in our lab, will eliminate the need pre-specified starting 


The presented preliminary system provides a encouragingly fast and accurate, marker- 
free algorithm to determine the position of 3 interconnected objects with almost no 
constraints on image background or light conditions. Greyscale cameras are sufficient 
for our system and faster in processing than colour cameras would be. The algorithm is 
applicable to objects with a narrow banded histogram. The field of application is not 
limited to physiological applications and rather could include, e.g., the localisation of 


any moving objects in an unstructured environment. The system can be used as a 

interface in virtual environment generators. 


Voss/Su6e Praktische Bildverarbeitung, Hanser-Studienbiicher, I Aufl.1991 

Peter Haberacker Praxis der digitalen Bildverarbeitung und Mustererkennung, Hanser 
Verlag, 1. Auflage 1995 

John C. Russ The Image Processing Handbook, CRC Press Inc., 2 Auflage 1995 

I, Bamea and H. F, Silvermann, ” A class of Algorithms For Fast Digital Image 
Registration", IEEE Transactions On Computers C-21, pp 179-186, 1972. 

Nikolaos P. Papanikolopoulos, Padreep K. Khosla Takeo Kanade, "Visual Tracking of a 
Moving Target by a Camera Mounted on a Robot: A Combination of Control and 
Vision", IEEE Transactions on Robotics and Automation Vol. 9 No 1 February 

Wachter H.H, Nagel, "Tracking of Persons in Monocular Image Sequences" Fraunhofer 
Institut fiir Informations- und Datenverarbeitung IH'B. 

Grobel, H. Hienz, "Videobased Handshape Recognition Using a Handshape Structure 
Model in Real Time", 13th conference ICPR 1996 Viena Austria 

Kashi Rao, Ram Nevita, "Describing and Segmenting Scenes from Imperfect and 

Incomplete Data", CVIGP: Image Understanding, Vol. 57, No. 1, January, pp 1- 

(((Ross T. Withaker, "Geometry-Limited Diffusion in the Characterization of Geometric 
Patches in Images", CVGIP: Image Understanding, Vol. 57, No. 1, January, pp 
111-120, 1993))) 

Sei-Wang Chen, Anil K. Jain, "Strategies of Multi-view and Multi-matching for 3D 

Object Recognition", CVGIP: Image Understanding,Vol. 57, No. 1, January, pp 
121-130, 19934 

Steven J. Nowlan, John C. Platt, "A Convolutional Neural Network Hand Tracker", 
Advances in neural information processing systems 1995 pp 901-908 


An Experimental Multimedia Process Control Room 

Bemd-Burkhard Borys and Gunnar Johannsen 
Systems Engineering and Human-Machine Systems Laboratory 

University of Kassel • GhK 
D-34109 Kassel, Germany 


The centralised control rooms of large industrial plants have separated people from the 
processes they should control. Perception is restricted mainly to the visual sense. Only 
telephone or radio links provide narrow-band voice communication with maintenance 
personnel down in the plant. Multimedia equipment can perceptionally bring back the 
operator into the plant while bodily keeping him the comfortable and safe control room. 
TTiis involves video and audio transmission from process components as well as sights 
and sounds artificially generated from measurements. Groupware systems support inter¬ 
action between operators, engineers, and managers in different plants. With support from 
the German government, the state of Hessen, and industrial companies the Laboratory for 
Systems Engineering and Human-Machine Systems at the University of Kassel estab¬ 
lishes an Experimental Multimedia Process Control Room. Core of this set-up are two 
high-performance graphics workstations linked to one of several process or vehicle 
simulators. Multimedia periphery includes video and teleconferencing equipment and a 
vibration and sound generation system. 


Today, operators in few or even only one central control room are separated from the 
process control large industrial plants. Increasingly sophisticated control systems shift the 
role of the human operator from manual to supervisory control, and research for autono¬ 
mous operation of large plants is in progress. However, the presence of the operator in the 
control room of a plant is still necessary. Thus, the interface to the process and its control 
system remains an important component. Process operators, seeing only selected physical 
measurements of large systems on their small displays, and physically separated from the 
equipment loose contact to the process. 

Before introduction of CRT technology, process control had walls showing mimic dia¬ 
grams, electro-mechanical instruments, and mechanical switches. Still today, mimic dia¬ 
grams and instrument walls influence the design of CRT displays. While the number of 
process variables increased, the task of deriving the necessaiy information about the pro¬ 
cess state out of all these variables is still mainly left to the operator. The operator has to 
perceive a vast amount of information. Except for acoustical alarms, the visual channel 
has to convey the majority of this information while conventional interfaces do not yet 
take advantage of other human senses and new, multimedia-based technologies. 

Experience in nuclear power industry shows, that plant operators do not need support 
systems during 95% to 98% of operation. In the remaining cases, support systems do not 
always provide the support necessary and the operators depend on their own knowledge. 
Although highly trained and motivated, operators are unable to cope with situations when 
they cannot access the necessary information about system state and performance. In the 
2% of operational cases, that are unusual, complex, and mostly new, the operator needs 
all available information to solve the problems. This information may comprise phenom- 


ena not transferred by usual measurements and unsuited to conventional displa^ys. Addi¬ 
tional media, integrated in the control room, may be able to convey the information. 

Extensive task and knowledge analyses performed by our laboratory in a variety of in¬ 
dustrial processes showed that an experienced operator uses more information from a 
plant than just the measurements displayed in the control room. The noise spectrum, 
sounds, vibration, temperatures, direct vision, even odour adds information to form the 
overall picture. It may be helpful to bring this information back, specially to the operator 
who is more and more separated (physically as well as concepmally) from the process. 
Opposed to hardware and software used in process control rooms today, equipment and 
procedures used in computer graphics show immense improvements. This offers the gen¬ 
eration of virtual images based on physical measurements, abstract data, and process 
structure. Multimedia technology can bring the process closer to the operator in a distant 
control room by adding some of the lost sensory channels. Instead of showing the icon of 
a tank besides a level measurement, the computer-generated image can show the tank it¬ 
self, along with the correct liquid level inside and a sound depicting liquid flow. Using 
computer graphics and virtual reality algorithms, other measurements can enhance the 
image, for example component colour for temperature, surface structure for efficiency. 
Using an image of a process component instead of physical measurements, showing live 
videos along with stereo sound from the process can again establish the close contact to 
the process. Simulating a reality sufficiently close to the operator’s view of the process 
will enable the operator to identify problems quickly. A live video image can show proc¬ 
ess failures not captured by measurements. In parallel, video conferencing enhances con¬ 
tact between process control operators and maintenance personnel in the plant. Beyond 
this, groupware and co-operative interfaces for multi-user interaction in plant-wide con¬ 
trol and communication can support problem solving on aU levels, such as the opera¬ 
tional, the maintenance, or the management level. 

We know from own experience, how much the background sound in a control room next 
to the turbine room and boiler changes with slight variations in the plant state. Experi¬ 
enced operators will benefit from good reproduction of these phenomena. Besides giving 
back information to the operators they had in conventional systems it is also possible to 
use these additional media to transmit new information and to support collaborative work. 

The Future Control Room: Co-Operative Work and Multimedia Support 
Because of the growing complexity of systems, more operators, even in different loca¬ 
tions in the plant need to co-operate. Telephone or radio communication provides only 
poor means to exchange process measurements and drawings from operation manuals 
along with views and ideas for co-operative problem solving. The co-operation between 
different human user classes (e.g., operators, engineers and managers) can be facilitated 
by the combination of several multimedia representations on different abstraction levels. 
The different information needs of these user classes are considered in the multi-human 
interface design by providing them with dedicated windows or screens and audio infor¬ 
mation. Engineers and managers may more often want to use goals-means hierarchies and 
multilevel flow model presentations whereas operators can use the less abstract presenta¬ 
tions of ecological and topological displays. However, free navigational access to all dis¬ 
play options must be allowed for all user classes, based on the different focuses of their 
individual preferences. The concepts of visual momentum and cognitive layouts should 
be implemented in such a way that they support the integrated view among team mem- 


bers of multi-user groups across all different forms of graphical representations. Related 
video and audio information further support this. 

A large number of pictures in a complex display network is characteristic for many in¬ 
dustrial applications. In such cases, only some of these pictures may be individualised for 
different user classes. Tani et al. (1994) suggested to use a large screen as a shared dis¬ 
play for all group members, together with detailed personal displays on individual work¬ 
station screens for each of the group members. Multimedia presentations of live video 
can be combined with computer graphics presentations. The visual momentum is imple¬ 
mented by means of highlighting manipulated objects as well as by corresponding 
movements of individual cursors for each group member in the shared and detailed 

Relatively new communication technologies for human-machine interaction have been 
developed with the field of multimedia. The main idea of multimedia communication is 
to combine and integrate different visual and auditory media for the display and visuali¬ 
sation of information about tasks to be performed with a machine or a computer, hi par¬ 
ticular, the following media are combined v^th each other: computer-generated visual 
displays, video recordings, and auditory information such as recordings of noises and 
synthetic speech (Steinmetz and Herrtwich, 1991). In addition, three-dimensional stereo¬ 
scopic video scenes and stereoscopic computer graphics can be superimposed upon one 
another (Milgram, et al, 1990). Even music and haptic information as well as smell may 
be used for multimedia communication. 

A connection of multimedia objects with an information network can be used interac¬ 
tively, for example, in travel agencies and libraries (Jerke, et al. 1990), In this pure hu¬ 
man-computer interaction, the querying technique is combined with the method of 
browsing in a hypertext environment. A hypertext system is a network of information 
nodes that are connected by links, in a non-sequential way, to arbitrary non-hnear infor¬ 
mation stmctures (Bogaschewsky, 1992). A hypertext network has been combined with 
an automatic fault-diagnosis module in order to support the problem-solving activities 
during trouble-shooting (particularly, motor diagnosis) tasks of human technicians in car- 
repair shops (Hollender, 1995). 

In general, the human user can navigate through a hypertext network by analysing the 
connections between the nodes. Conceptual connections between alternative interaction 
procedures lead to a hypermedia information network. The separate nodes of this network 
are related to the separate types of information, such as text, graphics, video or audio, 
which are different forms of multimedia. Thus, hypermedia represent an information 
concept connecting several media (Begoray, 1990), whereas multimedia encompass a 
combination of different presentation media. The terms hypermedia and multimedia 
are, thus, distinguishable from each other in a similar way as the dialogue and the pres¬ 
entation layers are, in a user interface management system. 

Multimedia communication systems will be introduced in the future in several application 
domains. We mentioned libraries and travel agencies already above. In addition, the en¬ 
tertainment industry is highly interested in this new technology. Particular applications 
will be possible in medicine, for co-operative conferencing between physicians (Klein- 
holz and Ohly, 1994) and for surgery. Industrial applications may be possible in glass 
production, the paper and pulp industries, the chemical industries (Heuer, et al., 1994), 

the power industry (Zinser, 1993), and the maintenance of networks such as those for dis¬ 
tribution of electricity (Akiyoshi et al,, 1995). Tanaka et al. (1988) suggested a tutoring 
system for the latter case. 

Alty and Bergan (1992) emphasise that many questions still exist with respect to the ap¬ 
plication of multimedia communication in dynamic industrial process-control tasks. The 
general problem of the interface design remains: Which information is needed by the 
human user, when, in which form and why? This problem becomes more severe with a 
larger number of technological options, which increase further with the multimedia do¬ 
main. Therefore, the information needs of the later human users have to be investigated 
by means of task analyses. This requirement becomes even more important in the cases of 
co-operation among several human users in co-operative work situations. 

Expert analyses (performed in a cement plant, as a kind of unstructured task analysis) in¬ 
dicated some aspects of the co-operative work situations in that particular application do¬ 
main. The face-to-face communication is absolutely mandatory. In addition, the audio 
channel, e.g., telephone communication, is very important. Multimedia technologies can 
be used for integrating the video information from some of the equipment, which is now 
available in the control room on separate video screens. Otherwise, video observations 
are rejected as a spy system in this application domain. Consequently, the important 
aspects of multimedia communication for cement planp comprise: (1) computer graphics 
and related video presentations with interaction facilities for human-machine and human- 
human communication, as well as (2) teleconferencing with highly improved face-to-face 
video presentations and screen-based audio communication. Both aspects need to be re¬ 
lated to one another and integrated in a task-oriented manner. Overlapping windows need 
to be avoided as much as possible, as usual in process control applications. 

The important aspects for travel agencies do not include the teleconferencing part. How¬ 
ever, a much more substantial search for related multimedia objects in an information 
network is required (Jerke, et al, 1990). The human user classes are travel agents and 
customers, who are possibly also co-operating with one another for more efficient cus¬ 
tomer support. Special computer skills cannot be expected from the average customer. 
This leads to the necessity of easy-to-use multimedia interfaces with related text, picture, 
graphics, video and audio information. 

The same general tendency exists with the important aspects of multimedia communica¬ 
tion in the entertainment sector. Everybody wants to participate and enjoy in an easy, di¬ 
rect and interactive way, comparable to strolling through a garden or singing a song, e.g., 
in a Karaoke environment (Tamura, 1995). This most intuitive computer use required in 
the entertainment domain will probably contribute strongly to new powerful and highly 
user-friendly multimedia systems. Although they will appear more as game instruments 
rather than as conventional computers, they may become the next generation of multime¬ 
dia computers also in industrial and service applications. 

The research into computer supported co-operative work (CSCW, 1994) and groupware 
deals with theories, design concepts, architectures, prototype systems, and empirical re¬ 
sults for co-operative work situations in application domains such as offices, classrooms 
and factories. Electronic mail systems, computer and videoconference systems, office in¬ 
formation systems, organisational knowledge bases, shared window systems and other 
communication and co-operation systems are being investigated. Some of these systems 


do not allow the human users to interact or co-operate with each other directly in space 
and time. New approaches for the integration of action space and time have been sug¬ 
gested, e.g., by Ishii, et al (1992). Such support for co-operative work seems to be par¬ 
ticularly required in industrial human-machine systems. 

As the results from the expert analyses in the cement plant indicate, co-operative work 
should be organised as far as possible using face-to-face communication. Large projec¬ 
tion screens are not welcome, because they are very soon too overloaded, and are not 
adaptable enough. However, they have already been implemented in some other applica¬ 
tion domains, but the concept of overlapping information for different human user classes 
has not been taken into account in the information content of these projection screens. 
Thus, their use for co-operative work needs to be rethought. Display screens for group 
meetings in different offices and the control room are welcomed as multi-human machine 
interfaces in the cement plant. They will also be accepted as dedicated human-machine 
interfaces in a network, and for discussions of smaller problems over the telephone. 

The display screens for the group meetings may consist of one screen with four to five 
windows. They allow access to all pictures in the control room, rather than having just 
printouts, as presently available. Different, most favoured pictures may be selected by 
different user-group representatives. All the selected pictures need to be seriously consid¬ 
ered by all members of the meeting, because co-operation rather than ego-centred views 
are required, where each user-group representative contributes. Modifications of control- 
room pictures are foreseeable for the display screens in group meetings. Quick-change 
and easy-to-use editing facilities may allow the selection of important lines or variables 
from a table, qualitative zooming-in and selection of subareas of component flow dia¬ 
grams, and manoeuvring or selection by sliders or text menus through different levels of 
abstraction. The latter range from physical forms, such as scanned-in photos, e.g., taken 
from databases or just of broken components (inside a pump, etc.), to goal hierarchies via 
multilevel flow-modelling representations. 

The consistency and coherence across selected and edited multimedia has to be guaran¬ 
teed, This will support the visual momentum, which is now already available, when a 
trend curve is selected by the cursor for a particular variable in the component flow dia¬ 
gram. The consistency will be increased when the computer completes the other selected 
pictures shown in parallel, e.g., consistent with the information reduction in the just- 
edited picture. The information filtering, reduction and qualitative modifications may be 
supported by the computer, or can be done solely by the group members. Further, com¬ 
puter-supported drawing facilities, e.g., for straight lines or for rapid prototyping 
(sketching) of new pictures and ideas or for modifying existing ones, are possible. How¬ 
ever, they may be more suitable for exploratory purposes rather than for normal group 
meetings, because the latter might become too long with too much computer interaction. 

The overlapping information for the different user classes has already been considered in 
the logbook, now available on a PC in the cement plant. This information has still to be 
implemented further in the presentation, the dialogue and the explanation facilities of the 
human-machine interfaces, particularly of the display screen for group meetings. 
Thereby, the visual momentum between different windows, which relate primarily to dif¬ 
ferent user-group representatives, has to be supported. 

All the suggested designs of human-machine interfaces for co-operative work also have 

to consider face-to-face communication. Otherwise, the social contacts would not be im¬ 
proved if this face-to-face contact were to disappear. Tele-co-operation is often not feasi¬ 
ble because the contact with the production will be lost, e.g., the feeling for clinker qual¬ 
ity will disappear. In addition, the work climate will deteriorate and, thus, there will 
eventually be no co-operation. 

Design work is often accomplished in teams where human users interact and co-operate 
with each other directly in space and time. New approaches for the integration of action 
space and time have been suggested, e.g., by Ishii et al. (1994), as mentioned above. Two 
people work on the same drawing for design and, at the same time, see the co-operative 
design partner through the transparent digitiser sheet. The two designers can physically 
work in remote places from each other. The basic metaphor of this concept is 'Talking 
through and drawing on a big transparent glass board.” Tlius, it is believed to be very im¬ 
portant to have face-to-face conununication as well as audio communication with each 
other in a team. 

Similar design principles were used for a co-operative work support system, which allows 
the supervisory control as well as the maintenance of a power plant (Muraoka, Ohi, 
1995). Not only image data but also audio and drawing data are combined with each 
other. Several windows on the display screen allow to visualise in parallel topological 
component-oriented presentations and video scenes from the plant or, alternatively, from 
the face of the communication partner, e.g., of the maintenance person in the field work. 
Additionally, a communication board can be shown in a further window. The communi¬ 
cation board can be used as a white board or a pin board on which selected information, 
cut out from any other window, can be pasted. This white board is shown to both com¬ 
munication partners in the remote sites. They can circle and mark or write anything they 
wish on top or besides the pasted information, again visible also for the remote site. Thus, 
a direct telecommunication can be used for co-operative problem solving. 

The already mentioned multimedia means of video conferencing with audio and face-to- 
face communication are even more important on the management and the marketing lev¬ 
els, As the facial communication is regarded as very important in many cultures, it seems 
to be worthwhile trying to reduce the huge amounts of data to be transferred in telecom¬ 
munication. It is feasible to simulate computer images of a person in the remote partner’s 
place. Then, only minimal parameter sets, which characterise the perceived impressions 
of a human face, e.g., fierceness or gentleness of a face, have to be transferred. An exam¬ 
ple of such investigations of facial features was presented by Kato et al. (1995), 

Advances in virtual reality technology allow to explore freely the virtual environment, 
e.g., of a maintenance situation by manipulating 3D graphical objects. Liquid-crystal 
glasses and data gloves have to be used for viewing and manipulating the stereoscopic 
display. Currently, the possibilities for exploiting such virtual reality and multimedia 
technologies for learning and training environment and for marketing are investigated 
(Akiyoshi et al,, 1995). 

Integrating the above approaches across aU levels in an appropriate human user- and task- 
oriented manner will be a major challenge of the next coming years. It will be important 
to think about desired work organisations, first, and to pursue corresponding cognitive 
task analyses in order, then, to build plant-wide control and communication systems 
which will be welcomed with high user acceptance. It is also very likely that remarkable 


cultural and socio-political differences with respect to optimal or, at least, satisfying so¬ 
lutions may exist even within individual countries between different companies. 

Even in the next future, process control rooms will need the presence of human operators. 
Those cases, when pre-programmed automatic systems would fail to control the process 
specially demand for sldll and knowledge of the operators. In these rare cases, aU avail¬ 
able information on the process and the combined skills and knowledge of operators, 
technicians, engineers, and managers is needed for co-operative problem solving. The 
integration of new media for information presentation and co-operative work to support 
humans in these situations is a main research topic of the new Multimedia Process Con¬ 
trol Room established in the Laboratory for Human-Machine Systems of the University 
of Kassel, The next sections describe the desirable capabilities of such an experimental 
instrument and the current implementation. 

Essential Capabilities of a Multimedia Process Control Room 

For technical (and financial) reasons, the multimedia process control room in its current 
configuration addresses the auditory and the visual sense with some support for percep¬ 
tion of vibrations. Visual and acoustic information is presented supporting perception of 
directions and depth. Artificial, fully computer-generated information, play-back of pre¬ 
recorded information, as well as life presentation is foreseen. In detail, a multimedia con¬ 
trol room should provide 

• Computer-generated and animated visual displays, showing process information and 
incorporating textual, pictorial, and graphical objects, combined with Hve or pre¬ 
recorded video; 

• life video image display for plant supervision as well as visual inspection of equip¬ 
ment or products and video presentations for operator and maintenance support; 

• life, reproduced, or computer-generated audio, vibration, and speech for supporting 
visual perception, alerting, and informing operators; 

• 3D stereoscopic and holographic presentation of the generated visual scene and spa¬ 
tial presentation of audio information; 

• face-to-face video communication and conferencing, incorporating document ex¬ 
change, drawing and annotation facilities; screen-operated telephone and radio com¬ 
munication; and, 

• maybe in the future haptic information, motion, music, odour, and more. 

For research and experimentation in this area, an experimental set-up needs in addition 
facilities for development of software, scenes and environments. Some additional prepa¬ 
rations necessary are 

• recording equipment to take audio and video samples in existing plants, import filters 
for CAD data from existing environments; 

• modelling software to generate 3D objects and assembling those to scenes together 
with audio and video objects; 

• various means for manual interaction like mouse, joystick, keyboard, together with 
related analogue and digital input channels; 

• software to generate realistic behaviour of the interface, like plant, process, and com¬ 
munication simulators; and 

• support for supervision and control of experiments, data collection, and evaluation. 

The following section describes some functions of a multimedia process control room 


with examples assuming a live process environment. 

Computer Graphics and Video Support 

Today, computer graphics in process control rooms are still too much copying the tradi¬ 
tional means of walls with control instruments to the computer screen, while separate 
monitors provide video information. Current research shows the value of displays devel¬ 
oped into two directions. One is creating more abstract views that are appropriate to the 
operators’ task, behaviour, and knowledge (Ali, 1997; Ali and Heuer, 1995; Johannsen et. 
al, 1997). The other direction supports the objective to bring back the operator into tiie 
plant by generating views close to reality (Wittenberg, 1997). hi both cases, integrating 
video information can support the view. 

The demand for further enhancement of a computer generated view leads to stereoscopic 
and holographic displays. Stereoscopic displays extend the view from the flat screen into 
the third dimension without taking account of the observer’s position. The common tech¬ 
nology generates separate images for each eye of the observer, displayed alternately on 
one screen or in parallel on two screens. Head-mounted devices providing one screen for 
each eye support the second solution while the observer needs to wear special glasses for 
the first version. 

Stereoscopic views look abnormal to the observer when moving around the scene, as a 
3D stereo view presents the image seen from one specific position only. The advantage is 
that the image is the same for every viewer fi:om every position. The disadvantage is that 
the viewer cannot just walk closer or around the image. Even shifting the head from side 
to side or forward-backward does not change the view as expected from natural scenes 
and creates an annoying effect. 

Holographic views take into account the observer’s eye-point by tracking the position of 
the 3D-glasses. A holographic projection adapts to shifts in all directions (left-right, up- 
down, to-from) within the range of the tracking device. The observer can see the scene 
from different directions as long as he looks towards the screen surface. This feature eas¬ 
ily adds to stereoscopic views with additional tracking hardware and software compo¬ 
nents. The display generation, however, supports only the view of one person; other 
persons need separate sets of display, glasses and tracker. Using head-mounted screens 
and sensing position and orientation provides means for full virtual reality that also shows 
views from the reverse side. The drawback is the weight of the helmet to carry and the 
isolation from the real world. The latter also acoustically, as common headsets moreover 
provide headphones. With this set is possible to walk around the display in the virtual 
world but difficult to avoid at the same time falling over the objects in the real world. 

Besides video integration to enhance personal communication and conferencing, we see 
the necessity of two different video applications in process control: Live video for 
equipment or product observation, and video sequences for operator support. Live images 
from the plant, integrated into computer generated diagrams enable operators to inspect 
critical equipment visually. Cameras either placed in strategic position down the plant or 
hand-held (or even helmet-mounted) by maintenance personnel may provide these sig¬ 
nals. In case of failures, video clips can instruct operators as well as maintenance person¬ 
nel on correct procedures. 


Audio Support 

In a conventional control room, the audio channel provides the operator mainly with 
alarms. A multimedia environment adds acoustic information from the process as sounds 
and noises that provide the operator with relevant information on process state. These 
need to cover the full range of audible frequencies and include vibrations. Acoustic in¬ 
formation from the process can be collected live in the plant or generated from process 
measurements. Process sounds can give a static acoustic^ picture of the process or focus 
on specific sections of the plant according to operator needs or current operator focus and 
actions. Individual microphones, placed at strategic points in the plant area can collect a 
live acoustic picture. The individual signals need appropriate mixing taking care of dif¬ 
ferent sound intensities, thresholds and masking effects. In contrast, it is also possible to 
generate a synthetic acoustic picture artificially from plant data using stored sound sam¬ 
ples or from noise and sound generators. In all cases, the spatial arrangement of sound 
sources must meet the layout of the control room. 

As an example, the control room of a fossil-fired power plant shows a spatial separation 
between furnace and steam generation and turbine and power generation. Instruments and 
controls for steam generation cover the left, those for power generation the right side of 
the control room. Important parameters are the continuity of the combustion process, the 
balance of heat and water on the left side and the turbine revolution rate and vibration on 
the other. In the past, operators were able to feel harmful vibrations of the turbine in the 
control room as long as it was sufficiently close to the turbine. The current load as well as 
the revolution rate provides a continuos background sound in the turbine building. Mi¬ 
crophones can pick up the sound down in the plant and speakers on the right side of the 
control room play it to the operators. Special transducers or low frequency speakers pro¬ 
vide vibration information. As operators have to keep turbine speed very close to the 
nominal rate, an artificial generated signal, providing more contrast in deviations from set 
points, may be of more use. In this example, acoustic information provided on the left 
side of the control room relates to the steam generation system. As an example, sounds of 
flowing water with different intensity can provide state information. Besides spatial ar¬ 
rangement, such sounds show good contrast to the noise from the turbine in frequency 
and complexity. 

Acoustic information from the process control system includes alarms and attention get¬ 
ters, These can be generated or pre-recorded tones, sounds, or voice messages. The sys¬ 
tem should provide means to spatially separate attention-getting audio on and arrange it 
according the general control room layout. 

System Components 

Following a market evaluation in 1996, we selected the equipment and implementation 
begins in late summer 1997. The Hardware selected includes high performance graphics 
workstations as well as high-end consumer electronics. The main software component is 
a set of programs for generating, animating, and displaying virtual worlds. 


Central components of the hardware are two graphics workstations, a SGI Octane/MXI 
with two 195 MHz R10000 processors, 256 MB memory, 4 MB texture memory, and a 
SGI 02 with 180 MHz R5000PC processor, 96 MB memory, Video I/O and camera. 
Both workstations are equipped with identical 20“ monitors capable of displaying stereo 


and holographic images. 

Hardware for holographic presentation of images consists of active stereo glasses with 
liquid crystal shutters, an infrared emitter synchronising the shutters with the monitors 
frame rate, and a tracking system to determine the actual eye point. The system from 
CrystalEYES will be used in combination with the software. 

As an input device, the Space Mouse will be used. This device, developed by the German 
aerospace research establishment DLR, is basically a force stick. However, along with the 
X- and y-axis, it senses translation force in the third dimension (z) and rotational forces in 
three dimensions (pitch, roU, and yaw). A 3D audio system will be added in the future. A 
possible candidate is the Acoustreon. This system, hosted on a separate workstation, ap¬ 
plies signal processing to given audio data. Given the distance and direction from the ob¬ 
server to the sound source the system filters the audio data such that the observer has the 
impression of a distant sound source in this position. 

The conventional audio periphery centres on a 6-channel amplifier and equaliser. 

Figure 1: System Components 

The two graphics workstations, a DAT recorder, and microphones provide audio inputs 
signals in stereo. For output, four speakers are placed around the workstations as well as 
headphones for privacy. The DAT recorder will be used to take high-quality samples of 
process noise and sounds during field studies. Integration of a special low-frequency am¬ 
plifier and speaker (subwoofer) or a vibration device is under consideration. Such a vi¬ 
bration device, the T.A,N. Vihrofloor System is a disk with 20-cm diameter, coupled to the 
floor or the operator’s chair. It can provide vibrations with a power of 100-Watt RMS 
with frequencies down to five Hz. 


The video option of the 02 workstation provides Essential equipment for videoconfer¬ 
encing. This includes two input-, one output-channel and the monitor-mounted digital 
camera. A camcorder will be used for recordings in plants. We also imagine the use of 
portable, helmet-mounted cameras (for, e.g., maintenance personnel) coupled to the con¬ 
trol room by radio links. Systems with a range suitable for in-house use are available at a 
price below 1.000 DEM. Besides this local video equipment, support exists by the uni¬ 
versities Media Centre providing professional video cutting and editing facilities. 

Both workstations will have access to the laboratories local area network, the universities 
future ATM network, and the local ISDN telephone exchange. The laboratory’s network 
provides access to several simulators, among &ose are a high fidelity simulator of a dis¬ 
tillation column, cement plant and power plant simulators, a small business jet and a sim¬ 
plified helicopter. 

Software Tools 

The major software are five components from the virtual reality package of the German 
company REALAX. The modeller enables to create 3D volumes or to import shapes from 
conventional CAD data. Objects are than grouped and combined to scenes in the editor, 
which also provides material properties (colour, reflection, transmission) and surface 
textures. The scene generated this way is displayed using the real-time component. Addi¬ 
tional software in combination with the stereo glasses described above generates the 
holographic view. Features of this software important in our applications are level of de¬ 
tail switching, morphing, and callback functions. 

Level of Detail switching saves limited computer capacity. Instances of an object are cre¬ 
ated with different complexity. The close-up view shows the object with all its details. 
When the distance between object and observer increases, displaying the less complex 
version keeps the number of polygons in the scene smaller. 

With morphing, it is possible to use several (slightly) different shapes of an object and 
display a weighted average of these shapes. This feature is impressively demonstrated in 
animated sequence of car crashes. However, we can imagine giving an operator hints on 
pressure and stress by displaying deformations in boilers, pipes, and joints. 

Inevitable in research, when own software extends existing tools or when simulation data 
will be integrated is a means to access and manipulate of the visual scene. Callback func¬ 
tions hand control to user programmes periodically or on defined events. They also pass 
pointers to scene data. When control is transferred and data is accessible, the scene can 
manipulated or information extracted for other applications. Thus, measurements pro¬ 
vided from an external simulator can change dials in the scene and information on eye 
point position can be used for correctly generating 3D audio. 

Intended Research Topics 

Some of the features wanted for a multimedia control room are already build into or de¬ 
livered with the computer equipment, such as the InPerson teleconferencing tool. This 
provides a shared whiteboard, on which the local and different remote users simultane¬ 
ously view and manipulate images and video windows displaying the conference partici¬ 
pants. However, the multimedia process control room is a research vehicle. Existing 
software provide only tools and existing solutions from other areas provide only pointers 
to own developments. Microphones, speakers, and the camera are standard equipment of 


modem workstations as they become to be for home computers. Now it will be our task 
to evaluate existing capabilities of hardware and software as well as known solutions 
from, for example, the entertainment industry for their use in process control. 

Examples for possible use of multimedia in control rooms have been given already 
above. In all research performed in our laboratory we include the final user in the early 
phases. Considering the growing awareness of multimedia in present-day Ufe we expect 
useful proposals from future users. Intended research topics include multimedia- 
supported co-operation between plant operation, maintenance, and management, man¬ 
agement of information, user-support for large multimedia data bases. 


The experimental Multimedia Process Control Room is a new vehicle to support future 
research projects of the Laboratory for Human-Machine Systems in cooperation with in¬ 
dustrial partners. It will also enhance lectures and support diploma- and PhD theses. 


Akiyoshi, M., Miwa, S., Ueda, T., Nishida, S. (1995). A learning environment for main¬ 
tenance of power equipment using virtual reality. In: Image Processing and its 
Application, Conf. Publication No. 410, lEE, Edinborough, pp. 331-335. 

Ali, S, (1997), Wissensbasierte Prozefivisualisierung mit Fuzzy-Logik. 39. 

FachausschuBsitzung Anthropotechnik der DGLR Uber menschliche Zuverlassig- 
keit, Beanspruchung und benutzerorientierte Automatisierung. Karlsruhe 21.-22. 
Oktober 1997. 

Ali, S., Heuer, J. (1995). Knowledge-Based Multimodel as a Basis for the Design of 

HUMAN-Machine Systems in Process Control XIV European Annual Conference 
on Human Decision Making and Manual Control, Juni 14-16, 1995, TU Delft. 

Alty, J.L., Bergan, M. (1992), The design of multimedia interfaces for process control In: 
H.G, Stassen (Ed.), Analysis, Design and Evaluation of Man-Machine Systemes 
(Preprints, The Hague). Pergamon Press, Oxford, 

Begault, D. R. (1994): 3-D Sound for Virtual Reality and Multimedia. London, England, 
Academic Press. 

Begoray, J.A. (1990). An introduction to hypermedia issues, systems and application ar¬ 
eas. In: Int. J. Man-Machine Studies, Vol. 33. pp. 121-147. 

Boff, K. R., Lincoln, J. E. (1988): Section 11,4: Attentional Directors. Engineering Data 
Compendium: Human Perception and Performance, Ohio, AAMRL, 
Wright-Patterson Airforce Base. 

Bogaschewsky, R. (1992). Hypertext-ZHypermedia-Systeme: Ein Uberblick. Informatik- 
Spektrum, Vol. 15, pp, 127-143. 

Calhoun, G. L., Janson, W. P., et al. (1988): The Effectiveness of Three Dimensional 
Auditory Directional Cues. Proc. Human Factors Society 32: 68. 

CSCW (1994). Proc. ACM Conference on Computer Supported Co-operative Work, 
ACM, Chapel Hill, NC. 

Edworthy, J. (1994): The design and implementation of non-verbal auditory warnings. 
Applied Ergonomics 25(4): 202. 


Heuer, J„ Ali, S., Hollender, M„ Ranh, J. (1994). Vermittlmg von Mentalen Modellen in 
einer chemischen Destillationskolonne mit Hilfe einer Hypermedia- 
Bedienoberfldche. Hypermedia in der Aus- und Weiterbildung. Dresdner Sympo¬ 
sium zum computerunterstutzten Lemen, Dresden. 

Hollender, M. (1995). Elektronische Handbiicher zur UnterstUtzung der wissensbasierten 
Fehlerdiagnose. Doctoral Dissertation, University of Kassel. 

Ishii, H., Kobayashi, M., Arita, K. (1994). Iterative design ofseamles collaboration me¬ 
dia. CACM 37 (8), PP 83-97. 

Ishii, H., Kobayashi, M., Grudin, J, (1992). Integration of inter-personal space and 
shared work space: Clearboard design and experiments. In: Proc. CSCW’92- 
Sharing Perspectives. ACM Press, Toronto, pp. 33-42. 

Jerke, K.-H., Szabo, P., Lesch, A., RdBler, H., Schwab, T., Herczeg, J. (1990). Combin¬ 
ing hypermedia browsing with formal queries. In: D. Diaper, D. Gilmore, G. 
Cockton, B. Shackel (Eds.), Human-Computer Interaction - INTER-ACT’90, 
North-Holland, Amsterdam, pp. 593-598. 

Johannsen G. (1997). Conceptual design of multi-human machine interfaces. Control En¬ 
gineering Practice, Vol. 5 (1997) No. 3, pp, 349-361. 

Johannsen, G., Ali, S., van Paassen, R. (1997). Intelligent Human-Machine Systems. In: 
Methods and Applications of Intelligent Control, S. Tzafestas (Ed.), Dordrecht, 
The Netherlands, Kluwer Academic Publishers, (in print). 

Kato, T., Yamaguchi, M. K., Akamatsu, S. (1995). Facial features and configurations 
effecting impressions of faces. In: Y. Anzai, K, Ogawa and H. Mori (Eds.), Sym¬ 
biosis of Human and Artifact, Vol. 20B. Elsevier Science, Amsterdam, pp. 559- 

Kleinholz, L., Ohly, M, (1994). Supporting co-operative medicine: The Bermedproject. 
IEEE MultiMedia, Winter, pp. 44-53. 

Milgram, P., Drascic, D„ Grodski, J. (1990). A virtual stereographic pointer for a real 

three dimensional video world. In: D. Diaper, D. Gilmore, G. Cockton, B. Shackel 
(Eds.), Human-Computer Interaction-INTERACT’90. North-Holland, Amster¬ 
dam, pp. 695-700. 

Muraoka, S., Ohi T. (1995). Co-operative work support system for a power plant. In: Post 
HCr95 Conference Seminar on Human-Machine Interface in Process Control, 
Hieizan, Japan, pp. 87-96. 

Numano et al.. Plant Operation Fitted with Human Sense and Perception. HCI Int. Conf,, 
Tokyo 1995. 

Rupert, A. H., Guedry, F. E., et al, (1993): The use of a tactile interface to convey posi¬ 
tion and motion perceptions. Virtual Interfaces: Research and Applications, Lis¬ 
bon, Portugal, AGARD, NATO, Paper 20 pg, 1-7. 

Steinmetz, R., Herrtwich, R. G. (1991). Integrierte verteilte Multimedia-Systeme. Infor- 
matik-Spektrum, Vol. 14, pp. 249-260. 

Tamura, H. (1995). Information technology in support of living oriented society. In: Pre¬ 
prints 6th IFAC/IFIP/IFORS/IEA Symposium on Analysis, Design and Evalua¬ 
tion of Man-Machine Systems. Cambridge, Mass., pp. 157-162. 


Tanaka, H., Muto, S., Yoshizawa, J., Nishida, S., Ueda, T., Sakaguchi, T. (1988). 

ADVISOR: A learning environment formaintenance with pedagogical interfaces 
to enhance student* s understanding. In: H.-J. Bullinger, E.N. Protonatorios, D. 
Bouwhuis, F. Reim (Eds,), Information Technology for Organisational Systems 
(EURINFO’88). North-Holland, Amsterdam, pp. 886-891. 

Tani, M., Horita, M., Yamaashi, K., Tanikoshi, K., Futakawa, M. (1994). Courtyard: In¬ 
tegrating shared overview on a large screen and per-user detail on individual 
screens. In: Proc. ACM CHr94 Conference on Human Factors in Computing 
Systems (Boston, Mass. April). ACM, New York, pp. 44-50, 

Wittenberg, Carsten (1997): Unterstutzung der menschlichen Informationsaufnahme 
durch Prozefivisualisierung mittels virtue Her Prozefielemente. 39. 
FachausschuBsitzung Anthropotechnik der Deutschen Gesellschaft fiir Luft- und 
Raumfahrt, Karlsruhe, 21 to 22 October 1997. 

Zinser, K, (1993). Integrated multi-media and visualization techniques for process S&C. 
In: Proc, IEEE Intern, Conf. SMC, Le Touquet. Vol 1, pp 367-372. 


A Multimedia Decision Support System 
for Work Groups in Flexible Manufacturing 

Dietmar Gude, Andreas Bauerle, Andreas Stiegler, and Wolfgang Laurig 
Institut fUr Arbeitsphysiologie an der Universitdt Dortmund 

Dortmund, Germany 


In flexible manufacturing, the shop-floor staff is increasingly organized in semi- 
autonomous work groups with an enriched task spectrum. This suggests to provide 
assistance in the form of decision support systems. However, there are few decision 
support systems specifically designed for work groups, which moreover do not yet have 
received a widespread acceptance. Therefore, a new support concept, focusing on the 
universal problem solving strategy of case-based reasoning, has been developed. This 
concept was realized as an application program for Personal Computers, in close 
cooperation with the shop-floor staff of a CIM pilot plant. Tlie program allows to record 
cases in a format similar to a news report, including multimedia illustrations, to ensure a 
vivid representation. The resulting case database can be searched for records that are 
similar to the current task, which then serve as models. In addition, the system provides 
functions to integrate the case data, to support the work group in its strategic decisions. 


The increasing complexity of flexible manufacturing systems requires an integration of 
the human resources, which takes into account the interactions between previously 
separated task domains. Correspondingly, there is a growing number of manufacturers 
who integrate duties formerly performed by their specialized departments, like quality 
control and maintenance, into the task spectrum of the shop-floor staff. Such an 
enrichment of the task spectrum has considerable positive effects on the performance of 
the employees (Lovdn, Helander, 1997). As a supplementation, the shop-floor staff is 
frequently organized in semi-autonomous work groups, coordinating the activities and 
pursuing a continuous improvement process. As an example, in the German machine 
building industry the proportion of companies practicing group work in their 
manufacturing departments increased from 29% to 43% between 1991 and 1993 (SFB 
187, 1994, p. 3). That is, substantial expert decision competencies are transferred to the 

Due to the complexity of flexible manufacturing systems, it seems to be advisable to 
provide the shop-floor staff with appropriate decision support tools, as an assistance in 
exercising its new competencies. However, most of the currently available decision 
support systems are primarily designed to fit the requirements of specialized 
departments. Presupposing a sophisticated educational background, these tools represent 
the work system in an abstract, generalized way. In addition, they are rather incompatible 
with an integrated task spectrum, because of their frequently narrow field of application. 

Lately, a number of decision support systems specifically designed for a shop-floor 
environment have been introduced, to assist work groups in short-term scheduling 


(Mertins, Carbon, 1996), quality management (Triebe, Falter, Krings, 1996; Wasserman, 
1995), or failure diagnosis and repair (Engel, 1996; Timpe, Rothe, GaBner, 1997). But 
there are indications that these systems meet the practical requirements on the shop-floor 
only to a certain extent. As a current example, in the research project RE-INST, 
concerned with 'Job Enrichment by Reintegration of Maintenance Tasks, the 
participating companies had to resort to internal developments, which were more or less 
ad-hoc solutions, because of the limited software development resources (lAW, 1997). 

Therefore, in the following section weak points of the currently available decision 
support systems and potential alternative design concepts will be discussed. Subsequent 
to this discussion a prototypical realization of the proposed design alternatives is 
outlined. The final section addresses possible shortcomings of this prototype. 

Current Systems and Alternative Design Concepts 

Currently available decision support systems for work groups are frequently 
characterized by three weak points, a restricted scope of application, an insufficient 
flexibility, and a tutorial support concept. Generally, the systems are designed to assist a 
single task, for example maintenance, whereas the enriched task spectrum of the shop- 
floor staff requires a more comprehensive support. Moreover, they are frequently 
restricted to specific equipment, for example a certain manufacturing center, which 
seriously limits the set of potential users and provides difficulties for the integration of 
the system in the information infrastructure of the factory. 

A solution to the problem of a restricted scope demonstrates SAP, for example in its 
latest product R/3. Here, the software is a rather general frame that the users tailor to 
their needs and requirements by entering the equipment and company data relevant for 
their purposes. 

However, experts are necessary to adapt such an installation to modifications of the 
equipment or the organization. This limited flexibility carries a lot of weight in the 
present context, as work groups, in their continuous improvement process, are expected 
to identify weak points of their work system and to eliminate them rather autonomously, 
whenever possible. Therefore, it would be more coherent if the shop-floor staff could 
represent these improvements in the corresponding databases of their assistance system. 

Finally, the support concept of currently available assistance software is generally based 
on the expert system approach. That is, the knowledge of external experts about the 
equipment and the task domains is implemented in databases, which the shop-floor staff 
uses as more or less intelligent computerized manuals. However, such a tutorial support 
concept is rather incompatible with the motivation to introduce group work, to tap the 
staffs practical knowledge and make this expertise available to the colleagues. 

An alternative support concept can be derived from the finding that expert employees 
frequently accomplish a task in analogy to a previous, similar case (Rothe, Timpe, 1997). 
According to Riesbeck and Schank (1989) this principle is a ubiquitous problem solving 
strategy, or as they say, 'the essence how human reasoning works' (p. 7). 

The concept of case-based reasoning is outlined in Figure 1. Supposing that an employee 
realizes that the fill-level of an oil gauge is only 75% (if-component of the current case). 


Then the question arises how he or she should react to this finding (then-component of 
the current case). To clarify this point, one can refer to episodic memory, in which 
numerous previous cases are stored, each as an if-then-relation. First he or she tries to 
recall the case whose if-component approximates to the if-component of the current case 
(Best Match). Subsequently, the employee has to adapt the then-component of the 
recalled case to the one under consideration (Analogy Formation). In the scenario in 
Figure 1, for example, he or she would refill a smaller amount of oil, as the fill-level is 
now a little bit higher than that in the recalled case. 

Figure 1: Conceptual outline of case-based reasoning 

According to the outline in Figure 1 an assistance system supporting case-based 
reasoning should have at least three components. First, the shop-floor staff needs a 
database containing previous cases, as a reduction of the memory demands. Secondly, 
the system should provide a function allowing to retrieve a case whose if-component fits 
the current task, as a support of the best match. Thirdly, the system ought to assist in the 
adaptation of the then-component of the retrieved case to the current conditions, the 
analogy formation. 

As the case database represents the practical knowledge of the shop-floor staff it cannot 
be provided by external experts. Rather, the shop-floor staff should record the cases they 
accomplished, which requires an equipment- and task-independent data recording 
interface. In addition, the cases are to be represented as illustrative as possible, to allow 
that they can be reconstructed by those colleagues who were not directly involved, as 


well (Proctor, Dutta, 1995). This requirement of a vivid representation leads to the 
application of multimedia (Hasebrook, 1995). That is, cases should be illustrated by 
graphic, sound, or digital video files, if necessary. 

However, the work group needs not only support in the execution of specific tasks, but 
also in its strategic decisions concerning, for example, quality control procedures or 
preventive maintenance intervals. With regard to the requirements of such a support, 
Lockamy and Cox (1995) claim that ’at the plant level, performance on the strategic 
objectives must be evaluated with a minimal passage of time to provide an appropriate 
level of control’ (p. 233). Thus, the decision support should also supply functions for the 
integration of the case data, allowing to perform weak point analyses. 

Prototypical Realization 

The alternative design concepts outlined above have been realized in a Windows 
application program for Personal Computers. The program, called Behavior-Outcome 
Feedback (BOF), has been developed in a CIM pilot plant, in close cooperation with the 
shop-floor staff, which in addition tested different versions in practice. BOF is installed 
directly at the equipment and thus is integrated in the activities on the shop-floor. The 
program provides a facility for the recording and administration of cases. These cases are 
merged by feedback functions, presenting the underlying behavior-outcome 
contingencies in a graphic way. In the following, the use of the program is illustrated by 
referring to data that were recorded at the flexible manufacturing center Huller-Hille nb-h 

The desktop is designed as a Multiple Document Interface (MDI), the case 
administration and feedback charts are child windows within the main program window. 
Figure 2 shows the child window of the case administration. Its structure is based on text 
grammars for news reports (Dijk, 1983), to provide a format that is general enough to 
document cases from different task domains. The field Condition describes the state of 
the object under consideration before and after the event or activity. These values do not 
necessarily differ, in an inspection, for example, they are identical. Field Cost allows to 
record aspects relevant for the evaluation of the event or activity, which typically are 
economic variables. Beyond that, they may include features like the cognitive or physical 
strain, if one advocates a more general cost concept (Fiy, 1995; Gelders, Manaerts, Maes, 

The values for the fields Status, Who, What, Where, Why, Condition, and Cost are 
entered using dialogs in which the alternatives are organized in hierarchical selection 
lists, similar to the Explorer in Windows 95 and NT. These selection lists can be edited, 
permitting to adapt the case administration to arbitraiy equipment, task spectra, to be 
controlled conditions, and cost aspects. For example, one might record not only 
maintenance activities, as shown in Figure 2, but also quality control measures, 
specifying the corresponding items. 

The section Illustration allows to attach multimedia files depicting a specific aspect of 
the case in a more vivid way. The program processes a fairly wide spectrum of 
multimedia data, be it audio, bitmap, or digital video files. Such a file might be a 
scanned technical drawing showing the construction of a component or a tutorial video 


about how to perform a specific measure. Pushing a button in the Illustration section, the 
multimedia file is displayed in its original size in a separate child window, which 
provides some extra presentation functions. 

^"'{Behavioi-Outcome Feedback 

file £dil £ases ^election Inlegralion Mu^^imedia jgindow Help 



1 1^; Cases - CABOF\EXAMPLE\NBH90.DBF 

ime I Roirtine cleaning of our 'home-made'fitter 
r Event 






mishit Ilij When 103.02.97 11:06:55 ^ 

StaffManufacturing/Shop floor 


nb-h 90/Chip scavBi^ing s^em/Filter 



. .. 


Instruction as video 


Condition I Component/Soiling [%|.../47 => 0 

Cost Sum fSliWorking 

Sum [SliSubj^ive^^^^ i%j...G5 


Filter mats are a lidle bit 



^Condition ... analy... 

Creates a new table with cases. 

Figure 2: Main window of EOF with the child window for the administration of cases, 
presenting a maintenance activity 

Case-based reasoning is supported particularly by a table function. In such a table the 
previous cases relevant for accomplishing the current task are listed. Figure 3 shows a 
table for the question with which soiling the filter of the chip scavenging system has 
been cleaned so far. In addition, one of these cases provides an Illustration When to 
clean’, showing the cleaning criterion in a digital photo. For an easy access of the 
complete case data, tables provide a navigation facility - clicking on an item the 
corresponding record is presented in the child window of the case administration. 

Other functions perform an integration of the cases, to support strategic decisions of the 
work group. One aspect is to aggregate the cost values, in order to determine weak points 
of the strategies. An example of a cost analysis is shown in Figure 4, presenting the 
working time expenditures for the maintenance of the coolant system components during 
a specified period of time. According to this bar chart, exceptional working time values 
were recorded for maintaining the tank, as compared to the other components of the 
machine unit. 


Again, BOF is rather flexible, as it allows to analyze arbitrary subsets of the selection 
lists. Moreover, the case data can be analyzed with regard to different levels of the cost 
hierarchy. For example, the program permits to specify a weighted sum of costs, rather 
than a single item, as the outcome variable. 

When to clean? - CAB0F\EXAMPLEANBH90.DBF 

When to clean? 

Where = nb-h QO/Chip scavenging systemFilter, When = Last Year, Editor = BOFA5L)DE/, State = 25.08.9715;. 

1 Wlien i 

1 Status 

f Condition 

fwhat 1 

: 19.09.9614:45:05 

21.10.96 09:16:32 
08.11.96 08:06:22 

10.12.96 09:11:52 









Component/Soiling [%].../44 => 0 
Component/Soiling [%].../15 => 15 
Component/Soiling i%i.../38 38 

Component/Soiling [%].../57 => 0 
Compcnent/Soiling [%]...>27 27 

Component/Soiling [%].../47 => 0 
Component/Soiling [%].../20 20 






Maintain/Clean 1 




Instruction as text 

When to clean 

Instruction a 

Figure 3: Table of previous, similar cases for the problem when to clean the filter of 
the chip scavenging system; the case in the last row represents the corresponding 
current case 

Cost analysis ot coolant system maintenance - C:\B0F\EXAMPLE\NBH90.DBF 

Cost anah/sis of coolant system maintenance 

When = 01.07.94 00:00:00 - 30.06.95 00:00:00, Editor = BOF/GUDE/, State = 26.08.9716:29:39 

Sum [$] Working time [min].../ 






Fiow measurin... Gauge Tank 

nb-h 90/Coolant system/ 

Wire sieve 

Figure 4: Cost analysis presenting working times for maintenance activities at the 
components of the coolant system during a specified period of time 

If a weak point has been determined, the next step is to identify potential reasons. This is 
supported by providing time and condition analyses. A time analysis allows to determine 
whether an event or activity occurred periodically, after a constant time interval. For this 
purpose, the interval of successive cases is plotted as a function of time, expecting that 
they form a more or less parallel line to the time axis. Figure 5 shows the corresponding 
scatter plot for the events at the coolant system tank, which are obviously aperiodic. As 
in the tables, such a time analysis includes a navigation facility; pushing a circle one gets 


the corresponding complete record in the child window of the case administration. This 
would reveal that the events in Figure 5 were concerned with the inspection of the fill- 
level and the topping up of the tank. Therefore, a potential cause of the identified weak 
point is an inconsistently performed time-based strategy. 

Time analysis of lank mainlenance - C:\BOF'vEXAMPLE\NBH90.DBF gElB 

Time ana^is of tank maintenance 

Cost = Sum [$]/Working time [min].../, When = 01.07.94 00:00:00 - 30.06.95 00:00:00, Editor = BOF/GUDE/, State = 26.08.97 ... 
■ Parallel Cases: None 

# Intervall Cases: VMiere = nb-h 90/Coolant system/Tank 

Figure 5: Time analysis for the maintenance of the coolant system tank during a 
specified period of time 

However, an alternative explanation is that the maintenance of the tank was not 
performed as a routine task, but rather dependent on its condition. The shop-floor staff of 
the CIM pilot plant preferred such condition-based strategies, as can be seen from an 
analysis of their recorded maintenance activities. According to this, 52% of the cases 
were inspections. Moreover, condition-based service and condition-based preventive 
repair dominated over their time-based variants (26% vs. 6% and 2% vs. 1%, 
respectively). Finally, 11% of the records were corrective repairs. The preference of 
condition-based strategies seems to be rather general in nature, as the above finding is in 
accord with results from more controlled laboratory settings (Gude, Psaralidis, Stiegler, 
Seeber, 1996; Kerstholt, 1995). 

Consequently, the assistance system permits to create so-called condition analyses. Such 
an analysis visualizes whether an event or activity occurred with specific values of the 
condition under consideration. Thus, for a consistently performed condition-based 
strategy one would expect that the cases line up on a parallel to the time axis. 

Figure 6 shows the corresponding plot for the fill-level of the coolant system tank. A 
circle represents the condition before an activity, the top of the arrow depicts the 
resulting condition. According to the figure, during the first six months there was no well 
defined criterion when to top up the tank, occasionally this activity was performed rather 
late. During the last six months the behavior was much more consistent, with the 
criterion beyond a fill-level of 50%. 


This pattern suggests that the identified weak point is based on an initial inconsistent 
condition-based maintenance. To test this hypothesis, the staff can create separate cost 
analyses for the first and last six months. These analyses reveal that during the first 
period of time the working time expenditure was 182 min, which then decreased to 87 
min. That is, the strategy shift in 1995 had actually a beneficial effect on the outcome 
variable analyzed here. 

Condiliun analysis ol lank maintenance - C:SB0F\EXAMPLE\NBH90.DBF _ 

Condition analysis of tank maintenance 

Cost = Sum [$)AVorkingtime [rnlnj.../, When = 01.07.94 00:00:00 - 30.06.95 00:00:00, Editor = BOFA3UDE/, State = 26.08.97 ... 
■ Parallel Cases: None 

# Condition Cases: Where = nb-h 90/Coolant system/Tank 






Figure 6: Condition analysis for the maintenance of the coolant system tank during a 
specified period of time; the three cases in Figure 5 for the period of time between 
23.11.94 and 04.02.95 are not displayed here, as the required filldevel data are missing 
in these records 

01.07.94 11.09.94 23.11.94 04.02.95 18.04.95 30.06.95 


Condition analyses do not only assist in the evaluation of preventive maintenance 
strategies but can be applied in quality control as well. The corresponding plot is similar 
to a Shewhart control chart in Statistical Process Control (SPC), indicating required 
adjustments or replacements of tools. Moreover, examining the quality control 
procedures in cost analyses permits to optimize the sampling frequency as well as the 
sample size. 


As a solution to the problems of a restricted scope of application and an insufficient 
flexibility, EOF provides a framework that the work groups can adapt to their specific 
requirements. As an alternative to the commonly realized tutorial support concept, the 
shop-floor staff is asked to record its practical knowledge in a case database, which 
supports the universal problem solving strategy of case-based reasoning. The database 
reduces, inter alia, the memory demands, which have been identified as a critical 
performance variable in manufacturing (Stanislaw, 1995). Moreover, less experienced 
employees can utilize their colleagues* practical knowledge, especially if the cases are 
illustrated by multimedia files. Finally, the program allows to integrate the case data for 
weak point analyses, which support the strategic decisions during the continuous 
improvement process. 


The multimedia files are not provided by the development team, as the specific field of 
application is not known beforehand. Rather, the users have to get or produce those files 
that are relevant for their purposes. However, taking into account the current technical 
state-of-the-art, one should not expect that complex multimedia presentations can be 
produced by the work groups, because specific expert knowledge is still required. On the 
contrary, this is a task for specialized departments or the manufacturer of the equipment. 
Nevertheless, with the increasing application of multimedia in the office and at home, it 
is very likely that the required knowledge will be generally available in the near future. 

In the current version of the assistance system the execution of the best match and the 
analogy formation is left to the shop-floor staff. The program allows to flexibly create a 
selection of previous cases. However, the user has to decide which of these cases 
optimally matches the present task and how to adapt it to the current conditions. For the 
moment, an automation of these processes has been deferred, as it presupposes 
considerable knowledge about the specific field of application (Reimann, 1997). 

Although data recording was held at a minimum, it still requires some effort. Therefore, 
the users should try to achieve an additional return on investment. In particular, the 
exchange of case data ought not to be restricted to the members of a single work group 
but rather is to be extended across groups and companies, including the manufacturers of 
the equipment. Such an integration would be especially valuable for small and medium- 
size companies, which frequently have only restricted opportunities to obtain the 
practical knowledge required to develop optimized manufacturing strategies. 


Dijk, T.A. van (1983) Discourse analysis: its development and application to the 
structure of news. Journal of Communication, 33,20-43. 

Engel, J. (1996) Diagnose-Informations-System - Ein System zur UnterstUtzung der 
Maschinenfuhrer bei der Instandhaltung. In D. Gude, A. Seeber, B. Zimolong 
(Hrsg.), Maschinenfuhrer in der flexiblen Fertigung (S. 167-181). VDI-Verlag. 

Fry, T.D. (1995) Japanese manufacturing performance criteria. International Journal of 
Production Research, 33, 933-954. 

Gelders, L., Mannaerts, P., Maes, J. (1994) Manufacturing strategy, performance 
indicators, and improvement programs. International Journal of Production 
Research, 32, 797-805. 

Gude, D., Psaralidis, E., Stiegler, A., Seeber, A. (1996) Economical and psychological 
aspects in the selection of preventive maintenance strategies. In A. Mital, H. 
Krueger, S. Kumar, M. Menozzi, J.E. Fernandez (Eds.), Advances in 
Occupational Ergonomics and Safety I (Volume 2, pp. 732-737). International 
Society for Occupational Ergonomics and Safety. 

Hasebrook, J, (1995) Multimedia-Psychologie. Spektrum, 

lAW (1997) Abschlufiprdsentation RE-INST Arbeitsbereicherung durch Reintegration 
von Instandhaltungsaufgaben\ Lehrstuhl und Institut fur Arbeitswissenschaften 
der RWTH Aachen. 

Kerstholt, J.H. (1995) Decision making in a dynamic situation: the effect of false alarms 

and time pressure. Journal of Decision Making, 8,181-200. 

Lockamy m, A„ Cox m, J.R (1995) An empirical study of division and plant 
performance measurement systems in selected world class manufacturing firms: 
linkages for competitive advantage. International Journal of Production 
Research, 33,221-236. 

Lov6n, E.M., Helander, M.G. (1997) Effect of operator competence on assessment of 
quality control in manufacturing. International Journal of Industrial 
Ergonomics, 19, 307-316. 

Mertins, K., Carbon, M. (1996) Shop floor control systems suitable for teamwork. In R.J. 
Koubek, W. Karwowski (Eds.), Manufacturing Agility and Hybrid Automation I 
(pp. 299-302). lEA Press. 

Proctor, R.W., Dutta, A. (1995) Skill acquisition and human performance. Sage. 
Reimann, P. (1997) Lernprozesse beim Wissenserwerb aus Beispielen. Huber. 

Riesbeck, C.K,, Schank, R.C. (1989) Inside case-based reasoning. Erlbaum. 

Rothe, H.-J., Timpe, K.-P. (1997) Wissensanforderungen bei der Stdrungsdiagnose an 
CNC-Werkzeugmaschinen. In Kh. Sonntag, N. Schaper (Hrsg,), 
Storungsmanagement und Diagnosekompetenz (S. 137-172). Vdf 


SFB 187 (1994) Mitteilung fur den Maschinenbau (Ausgabe 7). 

Sonderforschungsbereich 187 an der Ruhr-Universitat Bochum. 

Stanislaw, H. (1995) Effect of type of task and number of inspectors on performance of 
an industrial inspection-type task. Human Factors, 37, 182-192. 

Timpe, K.-P., Rothe, H.-J., GaBner, K. (1997) Entwicklung eines wissensbasierten 
Diagnosesystems zur Stdrungsdiagnose bei CNC-Werkzeugmaschinen. In Kh. 
Sonntag, N. Schaper (Hrsg.), Storungsmanagement und Diagnosekompetenz (S. 
279-298). Vdf Hochschulverlag. 

Triebe, T., Falter, H., Krings, K. (1996) Qualitatsinformationssysteme ftir Gruppenarbeit 
Oder Was will die Gruppe wirklich wissen? REFA-Nachrichten, 49, 30-35. 

Wasserman, G.S. (1995) An adaptation of the EWMA chart for short run SPC. 
International Journal of Production Research, 33, 2821-2833, 


Incorporating Ergonomic Considerations into Models 

of Manufacturing Systems 

Henry Herper 

Otto-von-Guericke- University of Magdeburg 
Department of Computer Simulation and Graphics 

P.0, BOX4120 
D~39016 Magdeburg, Germany 

Hansjurgen Gebhardt 
Institute for Industrial Medicine, Safety and Ergonomics (ASER) 

Comeliusstr. 31 
D-42329 Wuppertal, Germany 


Today the consideration of worker, costs and environmental factors becomes more and 
more important in designing of technical systems. Diverse criteria, like qualification, 
work organisation, compensation of employees and work structuring, need to be more 
integrated into the planning process and for that reason also into the planning tools. 

The qualification of workers is very important for reengineering manufacturing systems. 
Material flow simulators generate often only results concerning the utilization of work¬ 
ers. The developed evaluation module takes into consideration to what extent a worker is 
qualified for a given activity. It compares the qualification of the worker with the re¬ 
quirements of the working process. The interpretation of the results allows the determi¬ 
nation of necessary training for the workers already in the planning stage. 

The project is supported by the BMBF (German Federal Ministry of Education, Science, 
Research and Technology) in its program „Work and Technology*'. 


In the last years many companies underwent a change from traditional structures, like 
productions in bulk, toward customers' order production. This requires new concepts of 
production organisation and flexible working structures. Planning and simulation tools 
must be able to describe these structures. In these kind of systems and their models, 
workers become more important. 

Classic simulators are able to model technical processes in manufacturing and material 
handling systems with a variable degree of detail. They are used by factory planners to 
check and improve plans of technical systems. During the last years these tools were ex¬ 
tended with components to support the modelling of flexible manufacturing systems, 
automated production lines and automated material flow processes. 

One of the main application areas of factory simulation is the modelling of technical 
systems with their production processes and material flow. But today the consideration of 
worker, costs and environmental factors becomes more and more important in the design 
of technical systems. Diverse criteria, like qualification, work organisation, compensa¬ 
tion of employees and work structuring, need to be more integrated into the planning 
process and for that reason also into the planning tools. 



Today’s simulation tools are equipped with extensive components for the assessment of 
the behavior of manufacturing and material flow systems. The simulation results are de¬ 
livered in a processed form as compressed result data. They include, for instance, infor¬ 
mation on the utilization rate of transport devices and manufacturing facilities, the utili¬ 
sation of storage space, or the throughput time of items or orders. These data allow the 
materials flow planner an effective assessment of technical systems, their performance 
and their performance limits. Besides, most simulators offer primary result data. These 
are for instance trace protocols. The animation trace file is a special form of primary re¬ 
sult data which allows a post-run animation. With the help of the animation components 
in today’s simulation tools, the experienced user makes use of various possibilities in 
order to assess the modelled processes and/or to put the system states in a proper place 
due to their history. 

Different levels of description and modelling for the workers are possible. In traditional 
simulation models workers are often only components required for the operation of the 
manufacturing system. This kind of description now is not sufficient to fulfil the ex¬ 
tended requirements of simulation studies. 

A team of the Association of German Engineers (German abbreviation: VDI) is devel¬ 
oping a VDI Guideline dealing with the „Simulation of material flow and production 
systems“. It is known as VDI 3633. Part 8 of this Guideline deals with the workers in 
simulation models. Two terms are introduced: worker-integrated and worker-oriented 
simulation. The main aim of this part is to define the minimum requirements for worker- 
integrated as well as worker-oriented simulation of manufacturing and material handhng 

Worker-integrated simulation requires in addition to the modelling of the production and 
material handling system also the modelling of the workers. 

For the modelling of the workers the following minimum requirements have been de¬ 

• description of each single worker or type of worker, 

• description of teamwork, 

• consideration of the qualification level of each worker or type of worker and 

• separation between running time of technical system and working hours of the work¬ 

hi addition to the classic simulation results, the planning team is provided with more 
information about the system for their decision making, e.g. they can estimate how many 
workers are necessary and what qualification they should have. This kind of modelling of 
workers only requires an appropriate detailed modelling of the technical system. 

We use the term worker-oriented simulation in case that the simulation study primarily 
deals with worker related aims. Again, the basis is the model of the production and mate¬ 
rial handling system. Normally, the level of detail is higher than for worker-integrated 


Aims of the simulation study 
are to get results about impacts 
of different working forms on 
workers. Therefore, the model¬ 
ling of additional properties is 
necessary. These are, for exam¬ 

• physical and non-physical 
stress and strain factors for 

• changes in qualification due 
to learning and unlearning, 

• modelling of worker's reli- 

The modelling of these proper¬ 
ties requires additional input 
data and evaluation algorithms. 

Additional input data and 
evaluation components are ex¬ 
pensive and so only properties 
relevant for the aims of the 1: Data Flow in Advanced Simulation 

simulation study are included. Models 

In both cases, the worker-integrated and the worker-oriented simulation, the results about 
the workers have to be seen in combination with the results about the technical systems. 

Extension of Simulation Tools 

The Otto-von-Guericke University of Magdeburg and the Institute for Industrial Medi¬ 
cine, Safety and Ergonomics (ASER) of Wuppertal have been cooperating in the field of 
worker-oriented simulation for about 6 years now. Main aspects are the modelling of 
stress and strain as well as of qualification aspects. We prototypically develop and test 
modules to complement simulation tools with defined interfaces. As these modules are 
useful for a wide area of simulation languages and simulation packages, a high user ac¬ 
ceptance can be achieved. 

The precondition for getting additional information is a simulation model that describes 
the analysed system in a very detailed way. It must have a clear connection of all manu¬ 
facturing sequences to machines, items and workers. The series of manufacturing se¬ 
quences (processes) are to be recorded in trace protocols. Also the writing of trace proto¬ 
cols of changes in states of system components must be possible. Both forms of trace 
protocols constitute as primary result data the basis for the extended data evaluation. 

In principle, two approaches are possible: Additional functions may be integrated in 
simulation tools, as is done, for example, in simulators with special economic evaluation 
components. Such an approach requires a special tool for each simulator. 


A second approach is based on post-run data interpretation. The unprocessed trace data 
are connected with additional input data. Special modules allow the extended evaluation 
of the system (see Figure 1), Different views for different user groups are possible. 

We used the second approach. As an environment for the simulator, the evaluation mod¬ 
ules and the database we use a help and documentation system. The following basic 
functions are part of the system: 

• collection and interpretation of input data 

• evaluation of the results 

• help and documentation 

• database connection 

The evaluation modules can be chosen according to the aims of the simulation study. 
Figure 2 shows the implemented modules and their interfaces. 

Help and documentation system as an environment for simulation 
Normally the users of simulation-models are engineers of planning divisions, e.g, mate¬ 
rial flow engineers. This group of users is often not so familiar either with the developing 
of a model or with the interpretation of ergonomic results. Therefore it is necessary to 
give assistance in all states of the simulation process. 

The prototypicaUy developed help and documentation system was designed in Tool- 
book™ as an environment for supporting a planning team during the process of changes 
in working structures. The aim is not to replace the planning team, but to give them ad¬ 
vice in decision-making. Additional specific data and information can be integrated by 
the user. 

The help and documentation system may be used in each phase of restructuring. In the 
first stage experiences from similar projects in other companies are demonstrated so that 
the planning team can put their aims in concrete terms. They are then asked for necessary 
input data to give a rough estimate of the planned system. 

The system demonstrates to the users the possibilities and the limits of a simulation study 
for a special field of application. Based on this information, the planning team can de¬ 
cide whether a simulation study is necessary or not. 

If more detailed information of the dynamic process is necessai^, input data that have 
been introduced until this stage can be transferred into the simulation model. Examples 
for such data are the description of the working cycles and their characteristics, types of 
workers and their qualification as well as specifications of the expected order situation. 

After the simulation runs, the help and documentation system supports the interpretation 
of the results. Another feature of the system is to document the restructuring process. 


Figure 2: Components and Interfaces of the System 

Collection, preparation and management of input data 

In addition to the data collection supported by the help and documentation system, spe¬ 
cial modules can be used. Evaluated data for stress factors of working sequences can be 
directly imported from the stress documentation system (German abbreviation: BDS) 
which was developed for risk assessment in enterprises. Also special input data modules 
as a part of simulators can be used. 

The data are transferred to a common database, in our case we use the PARADOX™- 
format. For the data exchange between database and simulator we use the ASCII-Format. 
This format is supported by nearly every database and simulation tool independent from 
the hardware and operation system. 

For the extended evaluation of the simulation results, it is necessary to find a reference 
parameter the additional properties can be assigned to. 

The application of this concept requires a trace file generated by the simulator with the 
contents starting and closing time, working location, working item and number of worker 
for each working sequence. The combination of working sequences and primary simula¬ 
tion results is realized by a connection file which is represented by a matrix. 

Another kind of data connection is the extension of state protocols generated by the 
simulator with additional data. These state protocols are necessary for the evaluation of 
data which cannot be collected from working sequences, e.g. environmental conditions. 

One of the main aspects of our work is to get more information on the required qualifi¬ 
cation of the workers. Questions that arise often during the restructuring process are: 

• Are the workers qualified for the new tasks ? 


• Which tasks must a worker be qualified for? 

It is difficult to generate results about the required qualification of the workers. Normally 
the simulator assigns a worker to a work task only if he has an adequate qualification. 
However, simulators do not consider the effects of qualification on task duration and 
product quality. If no worker with the required qualification is available, the process is 
suspended until such a worker is idle. 

Figure 3: Input data collection using a help and documentation system 
here: Input of worker's qualification level 

In our modules two approaches are realized: The first approach is to give each worker a 
very high qualification. During the post-run evaluation, we analyze to what extent each 
single qualification was used. Specialists interpret the results and determine the neces¬ 
sary qualification demands for the workers. 

In the second approach, the model control of the simulator gets the qualification data by 
a number for each worker and each task in the range from „0“ (not able to perform the 
task) to „1“ (well qualified for this task) (see Figure 3). Different levels of qualification 
can be used for the assignment of workers. If the chosen worker is not so well qualified, 
this may influence the duration of performance in the model- The interpretation of the 
results allows the determination of necessary training for the workers already in the 
planing stage. 

Integration of physiological aspects 

In particular in case of heavy work load or aggravated environmental conditions physio¬ 
logical aspects should be incorporated into the simulation model to provide for the 
evaluation of the workers’ stress and strain factors already in the planning phase and the 


prevention of overstrain. 

Therefore different stress factors must be taken into consideration. They result from the 
task, on the one hand, and the environmental conditions, on the other. Stress factors 
caused by the task are, for example, dynamic and static muscular workload as well as the 
required body posture. Stress factors caused by environmental conditions are, for exam¬ 
ple, noise, vibrations, heat or cold. They may primarily depend on the working location, 
but also on the working item and the task itself. 

The level of detail should allow the definite assignment of working sequences to any 
workers, together with the indication of duration and working location. A working se¬ 
quence may be composed of several activities, but may not be interrupted in the simula¬ 
tion model. The required data may be derived from risk assessment studies, which has to 
be done due to European regulations. If no data are available, instruments for risk as¬ 
sessment studies (e.g. BDS) can be used in a prospective way. 

As a result of a simulation study we get exposure times for each worker. By combining 
the stress intensity and the exposure times, we can evaluate the stress factors for a normal 
shift. Physiological models for the prediction of heart rate are available to take synergetic 
effects into account. 

By incorporating such ergonomic considerations, the planning team gains information on 
potential risks in an early stage of planning. So it becomes able to improve both the tech¬ 
nical system and the working conditions and is always informed about the expected 
stress and potential risks in combination with the economic data. 


With the presented approach we intended to show new application fields for simulation 
models. Additional knowledge is integrated into the additional modules. Results are pre- 
processed for the user. Of course, the user has to draw his own conclusions and take his 
own decisions as ever. Simulation is not able to optimize the system, but is able to evalu¬ 
ate planning. The supporting level for the planning team is increasing. 

The shown modular approach concerning the post-run data processing allows to use clas¬ 
sical simulation tools under various conditions in the diverse fields of application. 



/!/ Lorenz, P., H. Dorwarth, L Ehrhardt, Hj. Gebhardt, H. Herper, B. Holzki, A, 
Kuhn, B.H. Muller, C. Vomholt (1995). Entwicklung eines Modells zur Be- 
schreibung der Belastung des Menschen und des Einflusses manueller Tatigkei- 
ten im innerbetrieblichen MaterialfluB. AbschluBbericht 2 nim Forschungsvorha- 

/2/ Ehrhardt, L, Henry Herper, and HansjUrgen Gebhardt. 1994. Modelling Strain of 
Manual Work in Manufacturing Systems. In: 1994 Winter Simulation Conference 
Proceedings, ed. by J. D. Tew, S. Manivnnan, D. A. Sadowski and A. F. Sei- 
la, 1044-1049. Institute of Electrical and Electronics Engineers, Piscataway, New 

73/ Herper, H., Thomas Kagerl and Dietrich Ziems. 1996, Incorporating Financial 

and Ergonomic Considerations into Models of Manufacturing Systems. In : 1996 
Winter Simulation Conference Proceedings, ed. J.M. Chames, D. J. Morrice, D. 

T. Brunner and J. J. Swain, 1149-1154. Institute of Electrical and Electronics En¬ 
gineers, Piscataway, New Jersey, 

14/ BAB - Beurteilung arbeitsbedingter Belastungen und Gefahrdungen. 1996. Das 
Analyse- und Dokumentationssystem fur die betriebliche Praxis. ASER 

75/ Banks, J., John S Carson, and Berry L. Nelson. 1996, Discrete-event system 
simulation. Upper Saddle River, New Jersey: Prentice Hall. 


Concept of an Object-Oriented Multimedia System 
for Teleservice - Applications 

Reinhard Mdller and Achim Sixtus 
Dept, of Electrical Engineering, Institute of Automation 
Graphics and Simulation Group 
University of Wuppertal, Germany 


Multimedia communication is a technology of growing importance for user interfaces in 
automation and process control. We are investigating and developing methods and utili¬ 
ties, that provide an object-oriented integration of multimedia aspects into modem dis¬ 
tributed process control systems. A general approach of a multimedia-based teleservice- 
concept will be proposed. We introduce an object-oriented multimedia communication 
system based on our implementation of the CORBA standard, which will also be de¬ 
scribed briefly. A first prototype of the communication system allows a periodic ex¬ 
change of stiU images between a supervising camera at one computer and a remote com¬ 
puter via a small-band ISDN telephone line. 


Multimedia conununication is a technology of growing importance for user interfaces in 
automation and process control. Some actual keywords related to multimedia are tele¬ 
working, teleleaming and teleservice, and these also influence concepts of process control 
engineering. Think of acoustic and visual data obtained from microphones or cameras 
applied to a certain automation system. Simply used and understood as a new brand of 
sensors or actors they can improve automation concepts significantly. Supervising and 
monitoring of certain process states is simplified, if the operator can "see" and "hear" the 
real process at the same control station, where the process is visualized. The user inter¬ 
face is much more intuitive, if the operator can interact on images of the real things (for 
example switch on/off a motor with a virtual button on a real image). A large impact and 
added value can be expected fi'om the integration of multimedia concepts in teleservice 
applications. The mean time for service and diagnosis of complex machines, installed far 
away from the supplier, can be significantly reduced by an intelligent teleservice concept. 

We are investigating and developing methods and utilities, that provide an object- 
oriented integration of multimedia aspects into modem distributed process control sys¬ 
tems. The main fundament is a strong and efficient concept of object-oriented communi¬ 
cation. Based on the CORBA-standard we have developed an object management system, 
the MiniORB, which promises for this. Another goal, which speaks for an implementa¬ 
tion of multimedia concepts into automation systems, is cost efficiency, i.e. low-band¬ 
width communication lines, efficient data compression and low-cost equipment. We will 
describe our experiments with a simple object-oriented multimedia communication sys¬ 
tem based on our implementation of the CORBA standard. This prototype allows a peri¬ 
odic exchange of still images between a supervising camera at one computer and a re¬ 
mote computer via a small-band ISDN phone line with adequate quahty and speed. 

The object-oriented approach 

In control engineering a hierarchical concept of four layers is generally used (Tab. 1). 

Each different layer has its own responsibilities. It defines different types of interacting 
processes and the necessary communication between them, concerning human users and 
machines. AU participants can be modeled as objects. This matches obviously for sensors 
and actors in the process field, as well as for any computing system. Even all the different 
processes can be seen as communicating objects. Especially the human-computer- 
interaction consists of interacting objects, but with different levels of detail, depending on 
the model layer, where they live. 

Tab. 1: Levels of control 



Administrative functions, i.e. in managerial-economics and 
marketing (administrative management) 

Production control 

Logistics control, quality control, controlling, operations 
scheduling (economical, technical and disposing tasks) 

Process control 

Plant control and supervisory, machine control 

0 supervision, starting and stopping of processes 
(operative tasks) 

1 measurement, command, control, visualization 
(elementary functions of process control) 


Sensors, actors, wires 

(data acquisition and set point control) 

The object-modeling-concept can be explained with the following example: Figure I 
shows a part of a chemical process. We can derive three different classes of objects, con¬ 
tainer, valve and reactor. The reactor looks like a specialized container with a heater and 
a mixer. Valves are of types input control and output control. Obviously there are hierar¬ 
chical relations. The whole technical process can be abstracted step by step until the ob¬ 
ject-model is complete. 








Figure 1: A chemical process and its abstraction 

Considering an object-model, which is universal and common for all layers we further 


deal with abstractions of all different real-world objects and comnaunication. Therefor we 
mark several kinds of communication-media (i.e. speech, text, images), each of them 
instantiated as objects from different classes (i.e. reports, tables or process values). The 
communication-media are transmitted via special communication-channels like 
ETHERNET, field bus or telephone lines, according to special communication- 
protocols. A universal model must take this into account and provide the necessary 
mechanisms and concepts. 

How does the object-model match a distributed automation system? A mechanism is nec¬ 
essary, which, based on a unique protocol, provides a network-transparent communica¬ 
tion between principally heterogeneous computing platforms. An object should not need 
to have any knowledge about its computing environment, nor should it know, whether 
another object it communicates with, is local or remote. Also a fixed client-server relation 
is not possible, because the role of client and server can change in the context of an on¬ 
going process. This demands a object- and communication manager - a broker. 

Several broker concepts were introduced. OLE (object Unking and embedding) and 
CORB A (common object request broker architecture) are the most important. OLE uses a 
fixed and binary compatible protocol, i.e. it demands equal operating systems and com¬ 
puting platforms. The implementation of a CORBA object broker is platform dependant, 
but the protocol is dear-text coded and independent of any hardware. This guarantees 
interoperability in any heterogeneous distributed computing system. Due to these evident 
advantages we chose CORBA as the basis for our developments. 

A communication concept for distributed objects 

The CORBA specification is part of the "Object Management Architecture" (OMA) de¬ 
veloped by the OMG^ and describes, how an "Object Request Broker" (ORB) has to be 
built. The ORB is the central component within an OMA environment. It supervises and 

^OMG = Object Management Group 


manages any messages between the distributed objects. The architecture of a CORBA 
ORB is shown in Figure 2. 

A communication happens always between two members: a client object (CO), that needs 
a service and an object implementation (01) that offers a service. It is neither platform-, 
nor implementation-dependent. The client object demands a certain service with a mes¬ 
sage called "request", that it sends to the ORB. The ORB knows, which object imple¬ 
mentation is able to satisfy the client object’s demands, locates the 01 within the network 
and forwards the request message to it. The object implementation processes the request 
and produces some results, which it puts into a "reply"-message, that is then sent the way 
back to the client object. 

The MiniORB 

The MiniORB is our implementation of a CORBA-ORB (Heinz, 1996). Divergently from 
the original specification we have not yet implemented dynarmc requests and a CORBA 
conformant exception handling, i.e. the dynamic invocation interface and the diverse re¬ 
positories (Figure 2) are object of our ongoing work. The MiniORB is actually imple¬ 
mented in C-i-i- under Unix, the interprocess communication between any hosts is based 
on Berkeley sockets. 

1 lokder Host 


R&nofe Host 

> CUertf 

f I 1 • 







.■: >< . > ? 

'"s'! ' • N { ^ 

_^ ; i 

1 I i 

^ . 1 




Figure 3 : Components of the MiniORB 

Figure 3 shows the components of the MiniORB, It is a client- and object implementa¬ 
tion- resident ORB, i.e. the ORB-functions are realized as library functions, which are 
linked to the respective code of the client object and the object implementation. These 
libraries contain 

0 the ORB-core, that implements the inter-process communication (IPC) be¬ 
tween the distributed computing systems, where the communicating objects 
reside on 

1 the ORB-interface, that offers some general services to the client object and 
the object implementation 


2 the object adapter, that coordinates the cooperation between the object imple¬ 
mentation and the ORB. 

The components "stub" and "skeleton" are generative components, which originate from 
the interface specification between a client object and the respective object implementa¬ 
tion. They are generated as code-appendixes for any new object (see Figure 4). A client 
object is directly connected to a local "proxy object" (the stub), with which it exchanges 
any messages. The main task of a proxy object is to encode or decode the passing mes¬ 
sages to or from the locally implemented ORB. A server-agent task (not shown here), re¬ 
siding on the remote system, resolves demanded object-to-object paths, initializes the cor¬ 
rect object implementations and activates the inter-process ORB-to-ORB communication. 
The remote ORB then exchanges the message with the selected object implementation. 
The object implementation contains a special skeleton method, that decodes the received 
encoded data and calls the demanded service. The results are transmitted back to the cli¬ 
ent object by the opposite way. 

The definition of interfaces 

The interfaces between client objects and object implementations within a CORBA- 
environment are always well defined. The communication between any objects is trans¬ 
parent, i.e. an object can never determine, if its partner is a remote or local object. This is 
provided by a special interface definition language (IDL), which allows a neutral defini¬ 
tion of all services a certain object implementation offers. The interface definition lan¬ 
guage is part of the CORBA specification. 

The IDL description of an object implementation interface consists of the following com¬ 

0 the name of the interface 

1 the name of all services the object implementation offers 

2 the data that are necessary for the execution of each service 

3 the data that results from the execution of each service 

4 the needed data types 

IDL is a purely declarative language. The implementation of an IDL-defined interface 
requires the use of a conventional programming language. Therefore a mapping of the 
IDL constructs to the constructs of the implementation language is necessary. The 
CORBA-specification contains mapping rules for the languages C, and Smalltalk. 
We have developed an IDL-compiler, which automatically generates a C-i-f-implemen- 
tation from IDL-defined interfaces (Sixtus, 1997). 

The development of a distributed multimedia-application 

With the MiniORB and our IDL compiler it is quite simple to develop distributed appli¬ 
cations: For each certain object implementation an interface has to be declared in IDL. 
From each of these interfaces the IDL compiler automatically generates four sets of C-I-+- 
files, each consisting of a header- and a implementation-file {Figure 4): 

0 a global file set, that contains the implementation of code relevant for both the 
client object and die object implementation 

1 a stub file set, that contains the implementation of the proxy object 

2 a skeleton file set, that contains the implementation of the skeleton methods 

3 the object implementation file set, that frames the real implementation of the 
services the object implementation provides 

The several header files include each other. 

Figure 4: The development of a distributed environment 

The developer has to add the implementation code for any services the object implemen¬ 
tation provides to the implementation file set. Furthermore he has to write a server pro¬ 
gram that activates the object implementation at runtime and a client program which calls 
the services of the object implementation via the proxy object. After all the source code is 
compiled and linked, the client object can transparently communicate to the object im¬ 
plementation, regardless where this resides in the distributed system. 

Figure 5: First-step prototype of a teleservice system 


The prototype 

Our first prototype implementation of a distributed multimedia system for process control 
and teleservice applications is shown in Figure 5. It is a low cost communication system 
based on our MiniORB, that provides the ability to exchange image data from a camera 
between a local and a remote computer via a simple ISDN telephone line. We use a Con- 
nectix QuickCam™, which is wide spread in low-cost video applications because it works 
with any common Centronics parallel-port. Beside the CORBA standard we also use 
shared-memory and semaphore concepts for inter-process communication. The comput¬ 
ing platform of our prototype are personal computers with a LINUX operating system. 

We evaluated the three parts of transmission, i.e. image acquisition, data communication 
and image presentation, to find out the overall possible performance (AmshofT, 1997). 
Evaluation concerned processing speed, image resolution and size, image compression 
and refresh rates. We use JPEG compression/decompression and found a compression 
factor of 60 to be fully satisfying for diagnostic or monitoring tasks. The following tables 
show some results. 

Tab. 2 shows, that with our environment an average image flow of 1/2 image/s is possi¬ 
ble (including compression), with a maximum acquisition speed of 193 kBytes/s, This 
can be eventually advanced by using the ECP/EPP transmission mode of modem parallel 
ports, which promises a speedup-factor of 5.. 10. 

Tab. 2 : Image acquisition, Size 320^240 pixel (full color), JPEG quality-factor 60 

486 DX 66 
parallel port 

Pentium 133 
parallel port 

Pentium 133 
parallel port 

Compressed-image size 




Delay before first image 

6.03 s 

2.32 s 

1.6 s 

Time between two images 

4.37 s 

1.88 s 

1.19 s 

Tab. 3 shows, that an average communication speed of about 5900 Bytes/s is possible 
with our MiniORB. This is slightly below the approximate maximum of 6500 Bytes/s on 
a ISDN B-channel, due to the communication overhead caused by the object manage¬ 

Tab. 3: Image transmission performance 

File size 

Transmission time 





1992 Byte 













Tab, 4 represents the image presentation including decompression and local communica¬ 
tion. It shows, that with our prototype a frame rate of 5 images/s is simply realizable. 

Tab, 4: Image presentation, Size 320*240 pixel, JPEG quality-factor 60 _ 

--- O'' r- -..r- 


486DX 66 
lMB/16 Bit Graphics 

Pentium 133 

2MB/16 Graphics 


1024 * 768 pixel 

Image size 

12637 Byte 

Time between two images 

918 ms 

191 ms 


640 * 480 pixel 

Image size 

12637 Byte 

Time between two images 

860 ms 

182 ms 


The concept of an object-oriented multimedia communication system for teleservice ap¬ 
plications was introduced. We presented the state of development of our object-oriented 
environment, the MiniORB and the IDL-compiler. It was shown, that the grade of inte¬ 
gration of video (and audio) data into a process control system should depend on the real¬ 
time demands of the process. A regular ISDN-connection (one channel, 64 KBit) allows a 
periodical exchange of still images and packed audio data every few seconds. This is suf¬ 
ficient for many teleservice applications. Online process monitoring with audio- and 
video- components or teleconferencing requires higher bandwidths. But, for example, the 
online diagnosis of erroneous parts of a manufacturing machine works fine with low im¬ 
age rates or still images. 

The next step of evaluation will be the transmission of audio data and process values 
between a technical process and its control system via the same communication channel 
at the same time, eventually by using a second ISDN B-channel. A main focus of interest 
will be on the synchronization of audio and video streams within our object management 



Amshoff, W. (1997) Bildiibertragung in einer CORBA-Umgebung auf Basis von ISDN. 
Study, University of Wuppertal, Dept, of Electrical Engineering. 

Heinz, O. (1996) Entwurf and Implementierung eines Object Request Brokers nach der 
CORBA-Spezifikation. Diploma thesis. University of Wuppertal, Dept, of Electri¬ 
cal Engineering. 

Object Management Group (1995) The Common Object Request Broker: Architecture 
and Specification. Rev. 2.0, July 1995. OMG Document Number 96-03-04. 

Sixtus, A. (1997) Implementierung eines IDL-Compilers zur Umsetzung von OMG IDL in 
C+-f. Diploma thesis, University of Wuppertal, Dept, of Electrical Engineering. 

Architecture of the "Robotic Tele Lab" 

Amin B. Cremers, Wolfram Burgard and Dirk Schulz 
Department of Computer Science III 
University of Bonn 
Bonn, Germany 
Email: (wolfram, abc, schulz)@ 


With the growth of the Internet the ability to share remote laboratory resources and to 
monitor and control remote devices becomes more and more important. Since the Inter¬ 
net makes no guarantees of the communication link’s bandwidth, tele-operation systems 
have to deal with possible lags in communication. This paper describes the architecture of 
the ‘ ‘Robotic Tele Lab”, a tele-operation system allowing multiple researchers in differ¬ 
ent locations to observe experiments on our task-level controlled mobile robot RHINO 
over the Internet. To cope with the poor communication properties of the Internet, our 
system uses a distributed virtual environment, predictive simulations and IP-Multicasting. 
This approach allows the monitoring of complex experiments even over low bandwidth 
connections with varying transmission delays. We further describe an experiment with 
our task-level controlled mobile robot RHINO. During this experiment RHINO operated 
for six days as a tourguide in a populated museum and was tele-operated over the Internet 
by more than 600 virtual visitors. 

1 Introduction 

A well known application area for robots, and especially tele-operated autonomous ro¬ 
bots, is the exploration of dangerous environments or of environments whose direct 
observation is in some way difficult. The mars pathfinder mission [Sto96] is perhaps the 
most popular example of this type of application. However, we are concerned with a dif¬ 
ferent research direction in autonomous robotics, namely the design of service robot 
systems. Such are robots capable of carrying out tasks like delivery jobs autonomously in 
ordinary environments like hospitals and offices. It is a common occurrence that these 
systems are being jointly developed by researchers working in different countries. Test¬ 
ing new robot control systems is a time consuming and expensive process under these 
circumstances because the developers have to meet in one location and have to assemble 
the whole system before they can test it. 

This paper introduces the “Robotic Tele Lab” (RTL), a tele-operation system allowing 
researchers in different locations to test and demonstrate new control software on a robot 
over the Internet. We believe that this system will help to improve the utilisation of ex¬ 
pensive robots and will lead to a more efficient development process. Software 
components can be tested frequently and new results can be demonstrated without the 
need to travel. 

RTL is different in many respects from tele-operation interfaces used for exploration 
tasks. The operating environment is almost known but changes dynamically and since 
RTL is using the Internet as the communication Hnk, it has to cope with low bandwidth 
and varying communication delays. Traditional methods of tele-operation, such as video 
transmissions and direct remote control of robots, are not feasible under these circum- 


stances. We use the task-level control system of our mobile robot RHINO and a 3D com¬ 
puter graphics visualisation of the operating environment instead, A sophisticated scheme 
of synchronised simulations, adopted from the field of distributed interactive simulation 
[MBZ95-I-], bridges transnussion delays and helps to further reduce bandwidth require¬ 
ments. For the transmission of state changes we employ IP-Multicasting, the most 
efficient method for transferring information to many destinations over the Internet. As 
the robot’s environment is changing dynamically (people walk around, doors are 
opened/closed, etc.), these changes in the real world have to be integrated into the world 
model, which forms the basis of the 3D visualisation. RTL can be easily extended by 
specialised sensor interpretation modules, each of them performing particular subtasks as 
part of the overall system goal. 

The strength of these concepts has been demonstrated during a museum tourguide proj¬ 
ect. During this experiment, RHINO operated for six days as a tourguide in a populated 
museum and was tele-operated over the Internet by more than 600 virtual visitors. The 
Web interface of the tourguide system was the first application of RTL technology, which 
will be introduced in the second half of this paper. Due to limitations of the Internet 
communication facilities of the WWW, some differences between the RTL architecture 
and the tourguide interface exist and these will also be explained. 

2 Related Work 

So far, research on tele-operated autonomous robots has focused mainly on robots ex¬ 
ploring dangerous environments [HHF+95, FPW-i-95] and on the design of autonomous 
robot systems for space missions [HBJJ94, Sto96]. In contrast to RTL, these systems use 
leased tines and satellite links for communication. High bandwidth is permanently avail¬ 
able and transmission delays are nearly constant. Visualisation is carried out mainly by 
video transmission and is supported by 3D computer graphics to bridge the veiy large 
transmission delays of up to several minutes. The environment of the robot is assumed to 
be static in these systems. 

The Internet communication architecture of RTL is based on NPSNET-IV [MBZ-i-95], a 
distributed, interactive virtual 3D multi-user environment, connecting different training 
simulators over the Internet, From the NPSNET-IV system RTL adopts the use synchro¬ 
nous predictive simulations and the use of IP-Multicasting. NPSNET-IV is a pure virtual 
environment. The RTL system on the other hand is designed to keep its virtual environ¬ 
ment synchronised with the actual environment of a real robot. 

3 Components of RTL 

From an abstract point of view, RTL has a client-server architecture. The server resides 
on a computer in the robotics laboratory and its task is to integrate all information avail¬ 
able from the robot (e.g. position, velocity, state of all actuators and sensor data) into its 
internal world model. Significant changes in the state of the world will be transferred to 
all connected RTL clients. The clients will update their own copy of the world model ac¬ 
cording to the incoming state information and visualise the actual laboratory environment 
using 3D computer graphics. In the following sections we will briefly explain the main 
components of the RTL system: Internet communication, clients, servers and the syn¬ 
chronisation of the virtual robot with the real robot. Please note that actual control of the 
robot is not part of the system itself because its main purpose is to test and demonstrate 
new robot control software. 


3A Internet Communication 

A big difference between RTL and typical client-server systems, for example the 'WWW, 
is the network communication technology used. In typical client-server systems, the 
server sends information to a single client only on request. The RTL server broadcasts 
state changes to all connected clients at once. Broadcasting is achieved using IP- 
Multicasting over the multicast backbone (MBone) [Dee89] of the Internet. Clients reg¬ 
ister at the so-called multicast address of the server. Together the server and clients form 
a multicast group. In multicast groups every message sent by one group member is re¬ 
ceived by all the other group members. The use of IP-Multicasting has at least two 

1. The communication expenditure of the server is independent of the number of clients. 
For this reason, the architecture is scalable to a large number of clients listening to the 
server at the same time. 

2. Since the clients register at a multicast address and not at the server directly, the 
server can be split up into separate modules, each handling distinct parts of the com¬ 
plex world update task. 

3,2 Clients and Servers 

We will now describe the basic building blocks of RTL clients and servers: the world 
database, the predictive simulations, the visualisations on the client side and the synchro¬ 
nisation of the virtual robot with the real robot. In addition we will explain how they 



Figure 1.1: Block diagram of an RTL client 

Figure 1.2: Block diagram of an RTL server 

World database: The central part of the RTL server and the chents is the world data¬ 
base. This object oriented database contains the 3D world model of the RTL system. The 
same model is used for the clients and the servers. Every object in this model refers to a 
real object in the operating environment, for example a piece of furniture or RHINO it¬ 
self. It is the job of the RTL server to integrate changes in the state of the world into the 
database. These changes are either recognised by dedicated sensor interpretation mod¬ 
ules, for example in the case where a door has been closed, or are announced directly by 
the object whose state has changed. One example of this type is the robot’s navigation 
system. Every object inside the server’s database contains a method which transfers 
changes of its attribute values to its twin objects inside the clients’ databases. This way 
all databases will agree upon their state over time. Internet communication is completely 
hidden from the remaining parts of the system. 


The world database distinguishes active and passive objects. Passive objects are furniture, 
doors and other parts of the world which do not change their state by themselves. Attrib¬ 
ute values of passive objects are only changed by the sensor interpretation modules every 
time they detect a change of the real object’s state. Every change is transferred to all twin 
objects in the clients* databases. 

Examples of active objects are the robot itself and humans. In contrast to passive objects 
active objects change their states by themselves. Inside the world database these changes 
are carried out by special simulation procedures, one for each type of active object. 

Simulation: A simulation procedure changes the state of the active object, every few 
milliseconds. Simulation takes place in the server’s database as well as in the clients’ da¬ 
tabases. The simulation fulfils two purposes. (1) It is able to predict the behaviour of the 
robot for several seconds, thus bridging the varying transmission delays of the Internet. 
(2) As simulation takes place inside the server and inside the clients, database updates for 
active objects are only sent if the predicted state of the object differs significantly from 
the real state of the object. This dead reckoning simulation keeps all the databases nearly 
synchronised over time and it also considerably reduces the bandwidth requirements of 
the system. Watching the robot move in a static environment produces a maximum Inter¬ 
net traffic of only 480 bytes per second and the average traffic is less than 200 bytes per 

Figure 2: The 2D visualisation showing 
the robot’s position 

Visualisation: An RTL client contains a 2D and a 3D visualisation of the laboratory en¬ 
vironment. These two visualisations form the tele-presence interface of the RTL system. 
The 2D visualisation shows a top view of the whole operating environment including the 
actual position of the robot (fig. 2). Since a robot marker is animated by the simulation 
according to the movements of the real robot, the user is able to observe the real robot’s 

Using an OpenGL based 3D visualisation, users of the real robot can define virtual cam¬ 
eras, enabling them to view the virtual environment from various perspectives (fig. 3). A 
virtual camera can also be bound to the position of one of the robot’s real cameras (fig. 4). 
Since the simulations are synchronised, the user will see the virtual environment from the 
perspective of the real robot, even while it is moving. 


Figure 3.1: Virtual view of the 

Figure 3.2: Real view of nearly 


the same position 

Synchronisation: In this paragraph we will briefly explain how the RTL server is con¬ 
nected to the RHINO system [BBC95-I-, TBB-J-] and how the virtual robot’s position is 
synchronised with RHINO’S position. The RHINO system consists of the robot itself, an 
RWIB21 robot, and several software modules, each performing a specific navigation or 
planning task. The modules exchange information asynchronously via TCP/IP using the 
message passing library TCX [Fed93]. Two modules are employed for the synchronisa¬ 
tion tasks:- (1) the navigation and collision avoidance module [FBT97] and (2) the 
localisation module [BFHS96]. The navigation module sends robot state information, po¬ 
sition, velocities and accelerations, to the RTL server up to 4 times a second. These 
values are used by the RHINO interface of the server to re-initialise the simulation if the 
simulated values differ significantly from the real values. However, due to dead reckon¬ 
ing errors of the robot, the position information of the navigation module may not be 
correct. It is the task of the localisation module to find the robot’s absolute position and 
to keep track of it. It is accurate to within 10 square cm. The RHINO interface of the 
RTL server uses correction parameters sent by the localisation module to compensate for 
the dead reckoning error of ihe navigation module. 

As the RTL system is still in its early stages, only a first prototype implementation of the 
system as described above exists. We only carried out some small tests over our local 
department network. Although the system gave us a good impression about what was 
really going on in the laboratory, these tests can not be compared to tele-operation over 
the Internet, because network communication is much better over a local area network. 
However, the museum tourguide project proved that the underlying concepts scale to the 


Internet as the communication link for tele-operation systems. Before we go on to de¬ 
scribe this project, we want to suggest another possible application of the RTL system. 

3J Simulation of multi-robot experiments 

We want to explain the use of RTL as a simulation environment for multi-robot experi¬ 
ments. An RTL server can be used as a pure RHINO simulator. In this configuration the 
modules of the RHINO system are connected to the simulation inside the RTL server in¬ 
stead of to the real robot. As the world database is not restricted to only one robot, several 
servers can be started as simulators, thus leading to a multi-robot simulation environment. 
Please note that every simulated robot is able to detect every other simulated robot with 
its simulated sensors, as soon as it comes within reach. The multicasting based communi¬ 
cation architecture makes it easy to distribute the whole simulation environment over 
several workstations to gain the computation power needed. Thus experiments with con¬ 
trol software for co-operating autonomous robots can be easily carried out. 

For the museum tourguide project, a facility based on RTL’s world database has been de¬ 
veloped, allowing the navigation system of the real robot to detect objects with virtual 
sensors which could not be directly detected with its real sensors. Using this facility, 
simulated and real robots can also be used together in one experiment. 

4 The museum tourguide project 

From May 29. to June 1st 1997 and on the first two days of July, RHINO operated as a 
tourguide in the “Deutsches Museum Bonn”. The goal of this experiment was to demon¬ 
strate the potential of the RHINO system to a large audience. Our tourguide can be used 
in two different modes: local mode and Internet mode. If it is operated in local mode, 
visitors to the museum can attend a guided tour to selected exhibits. The robot will move 
to the exhibits, by-passing other visitors, and will play back pre-recorded information 
about particular exhibits when they are reached. If the robot is operated in Internet mode, 
WWW users are able to send it to one or more of the exhibits listed on the “Tell RHINO 
where to go” page (fig. 5) and the tourguide system schedules the incoming requests into 
a tour through the museum. Independent of the mode of operation, new pictures from 
one of the robot's cameras, and from a fixed camera observing the exhibition, are placed 
on a WWW server every 5 seconds. 

The WWW interface of the tourguide employs techniques developed for the RTL system. 
However the communication architecture had to be considerably changed because of 
some deficiencies of WWW technology. The main purpose of the WWW is still to make 
static information available. Current Web browsers only have restricted facilities for dis¬ 
playing dynamic Web pages. We will first discuss these opportunities before we go on to 
describe the actual design of the Web interface of the museum tourguide. 

Client pull: An annotation in the head of the HTML page informs the browser, that it has 
to reload the page after a given time interval. However, it has to be taken into account, 
that the server has no possibility of synchronising these updates with external events such 
as the robot reaching an exhibit. 


Tell RHINO where to go 

Please enter your name or email adress -and^regteter Itto monitor that your requests are scheduled. 

Tlia numbers mark the exhibits of 

\i(/hiqh RHINO cari be sent 
are sensitive to the clicking. : 
allows you to view the exhitjjt 
frqilrl perspectives 

Pleas? Ch.r on. H« h^v^hPtyk 

According to the requests of the VlrtiMl wsftWB RHIISIQ'S lour you tathefoHowing exhibits 
1. Airbus ruddpi^S)for schulz 

Figure 5: The "Tell RHINO where to go” page 

Server push: This facility makes use of multi-part documents. A multi-part document 
consists of the concatenation of a sequence of simple documents. A multi-part JPEG 
document, for example, consists of a sequence of JPEG images. The browser displays 
every part of a multi-part document as soon as it arrives. Using server push, the server 
has control over the point in time when an update is sent to the client. However, it gets 
no feedback about when the new content has actually arrived at the client. On slow Inter¬ 
net connections this may lead to old information accumulating and, as a result, the virtual 
visitor will get outdated information. We did not use server push for the Web interface of 
the tourguide for these reasons, 

Java Applets: A Java applet is a Java program, running in a Web browser. Using Java 
we are not bound to proprietary client server communication facilities, but are able to im¬ 
plement our own communication scheme. However, there are two restrictions. (1) For 
security reasons Java applets are only allowed to contact the computer from which they 
have been loaded. (2) At the time of the tourguide project, most Web browsers were not 
capable of multicasting. For these reasons it was not possible to simply adopt the com¬ 
munication scheme used by RTL. 


ti3s>* if/mm iste . 'Htmi 






!,: v.Kv ® j 

Cr C-nree refng^ator 

• . v.* V* • 

Figure 6: Pictures from the exhibition 

We employed client pull and Java applets for the Web interface. Client pull is used to re¬ 
load the images on the “Online Picture Page” at regular intervals. Together with these 
images a GIF picture is displayed, containing a top view of the exhibition and the robot's 
position at the time of image capture (fig. 6). Every time these three images are reloaded, 
approximately ISCKX) bytes have to be transmitted and, because of this, only low update 
rates are possible. The update rate is individually configurable by the virtual visitors and 
most of them tuned it to 30 seconds. Obviously it is not possible to observe tlie robot's 
behaviour under these conditions and so we implemented a simplified version of the 2D 
visualisation of an RTL client in Java which could be viewed on a separate Web page. 
This applet achieves a smooth animation of the robot and additionally explains the robot's 
actions in one line of scrolled text (fig. 7). Instead of IP-Multicasting, the applet commu¬ 
nicates over a simple unicast UDP connection with the server. 

More than 630 different virtual visitors used this Web interface during the experiment. 
They sent RHINO to 1264 target points. In only three cases, was the robot unable to 
complete its job. In all three cases failure was caused by minor hardware problems like 
empty batteries. The large number of completed jobs demonstrates the high reliability of 
the whole system. The tourguide experiment proves that, thanks to contemporary AI 
technology, autonomous robots can safely be tele-operated over the Internet and even 
over the WWW. 

ss Wctscape; RHWOt -RMifgufcfc (appiel) 

Find out where I am (Java appictveralon) 

This is an cuOine of RHINO’S environment. The numbers marie the esehibits to v/hich RHINO can be 
imt. The yeflow circle Is RHINO’s current position. 

:-..j V«™^™.V...5.WA%^. 

Figure 7 ; The Java applet. The trace of the robot’s trajectory 
shows the smoothness of the animation. 

5 Conclusions 

This paper introduced the “Robotic Tele Lab”, a tele-operation system allowing re¬ 
searchers all over the world to test and demonstrate robot control systems via Internet. 
RTL demonstrates, that the Internet is a suitable communication medium for tele¬ 
operation systems, despite its poor communication properties. We managed to overcome 
these problems using the combination of the three key concepts of RTL: a distributed 
virtual environment, synchronised predictive simulations and a task-level controlled robot 
with advanced sensor interpretation capabilities. 

Development of the RTL system is still at an early stage: work has until now mainly been 
focused on the network communication and visualisation part of the system and on the 
tourguide interface. The environment is still assumed to be static in the current imple¬ 
mentation, One of the next steps will therefore be the development of sensor 
interpretation modules. Modules which detect the state of doors and the position of fur¬ 
niture have to developed as well as a module, which detects humans and computes their 
walking direction and walking speed. 

In addition we want to extend the RTL system so that different researchers can carry out 
experiments with new robot control software over the Internet together, each researcher 


checking their own part of the new software. For this goal to be achieved, we want to im¬ 
plement an access control mechanism on top of the RHINO system because we believe, 
that such experiments are particularly prone to access errors, such as two control compo¬ 
nents trying to drive the robot to two different destinations at the same time. 


[BBC-(-95] Joachim Buhmann, Wolfram Burgard, Armin B. Cremers, Dieter Fox, Tho¬ 
mas Hofmann, Frank Schneider, Jiannis Strikes, and Sebastian Thrun. The 
mobile robot Rhino. AI Magazine, 16(2):31-38, Summer 1995 

[BFHS96] Wolfram Burgard, Dieter Fox, Daniel Hennig, and Thimo Schmidt. Estimat¬ 
ing the absolute position of a mobile robot using position probability grids. In 
Proc. Of the Fourteenth National Conference on Artificial Intelligence, pages 

[Dee89] Steve Deering. Host extensions for ip multicasting. Request for Comments 
(RFC) 1112, Internet Engineering Task Force (IETF), August 1989 

[FBT97] Dieter Fox, Wolfram Burgard, and Sebastian Thrun. The dynamic window 
approach to collision avoidance, IEEE Robotics & Automation Magazine, 
4(l):23-33, March 1997 

[Fed93] C. Fedor. TCX. An interprocess communication system for building robotic 
architectures. Programmers guide to version lO.xx. Carnegie Mellon Univer¬ 
sity, Pittsburgh, PA, December 1993 

[FPW-i-95] Terrence Fong, Henning Pangels, David Wettergreen, Erik Nygren, Butler 
Hine, Phil Hontalas, and Christopher Fedor, Operator interfaces and net¬ 
work-based participation for Dante 11, In SAE 25*^ International Conference 
on Environmental systems, 1995 

[HBJJ94] G. Hirzinger, B. Brunner, Dietrich J., and Heindl J. ROTEX-the first re¬ 
motely controlled robot in space. In IEEE Conference on Robotics and 
Automation, San Diego, May 1994 

[HHF+95] Butler Hine, Phil Hontalas, Terrence Fong, Laurent Piguet, Erik Nygren, and 
Aaron Kline. VEVl: A virtual environment teleoperations interface for 
planetary exploration. In SAE 25^*^ International Conference on Environ¬ 
mental Systems, July 1995 

[MBZ-<-95] Michael R. Macedonia, Donald P. Brutzman, Michael J. Zyda, David R. Pratt, 
Paul T. Barham, John Falby, and John Locke. NPSNET: A multi-player 3D 
virtual environment over the internet. In Proc. of the ACM 1995 Symposium 
on Interactive 3D graphics, 1995 

[Sto96] H. W. Stone. Mars pathfinder microrover: A low-cost, low-power spacecraft. 

In Proc. of the 1996 AIAA Forum on Advanced Developments in Space Ro¬ 
botics, Madison, WI, August 1996 

[TBB+] S. Thrun, A. Bticken, W. Burgard, D. Fox, T. Frdhhnghaus, D, Hennig, T. 

Hofmann, M. Krell, and T. Schmidt, Map learning and high-speed navigation 
in RHINO. In D. Kortenkamp, R. P, Bonasso, and R, Murphy, editors, Al- 
based Mobile Robots: Case studies of successful robot systems. MIT Press, to 
appear, Cambridge, MA 


A 3D Interface for a mobile robot using standard Internet tools 

Eric Colon, Yvan Baudoin, Bernard Heggermont 
Signal and Image Center / Cell Robotics 
Royal Military Academy 
Brussels Belgium 
colon @ mapp. rma. ac, he 


Because of their relative simplicity, ultrasonic sensors have been widely used in the 
robotics community for measuring distances. With the development of always more 
accurate video cameras and powerful image processing systems, ultrasonic sensors have 
become less popular. However, in nuclear applications for example, ultrasonic sensors 
have proved to resist very well to radiations and are thus still widely used. 

But these sensors have a lot of shortcomings and sophisticated algorithms have to be used 
to obtain reliable results. We present in this paper several algorithms that temporally filter 
the data and fuse them to build a map of the environment. 

Furthermore, as video cameras cannot be used due to poor resistance of electronics 
against high level of radiations, an alternative solution is needed to help the user to 
remotely control the robot or the arm for manipulation. In this case a 3D representation of 
the environment is needed. We propose here a solution based on today Internet tools: 
VRML and Java. We demonstrate the application on a mobile robot Nomad200, 

Keywords: robotics, sensors, VRML, Java. 


Maintenance, repair, and dismantling operations in nuclear facilities must be performed 
remotely to minimize contamination risks and occupational doses to the operators. 
Sensor-based teleoperation enhances safety, reliability and performance by helping the 
operator in difficult tasks when remote perception is difficult. Mobile robots can be used 
to explore contaminated sites for maintenance purposes or after an incident. 

One major problem is the poor resistance of common sensors to radiations. The video 
cameras generally used in robotics applications do not resist very well to high radiation 
doses. In this case, special radiation-hardened sensors have to be used: specific ultrasonic, 
force and optical fiber optics sensors have been studied and developed for many years.[1] 
Ultrasonic sensors are used to detect obstacles and to measure distances; they can be used 
on a mobile robot or on a teleoperated arm. They offer a good resistance in such 
conditions but they return very limited information, such as distance to a point, unlike 
video cameras which deliver high-level very comprehensive (for a human) information. 
Ultrasonic sensors have a relatively good precision but offer a lot of uncertainties and 
we need intelligent data processing to extract useful information. 

When looking at the raw data acquired during the motion of a mobile robot, we can see 
that some measurements are completely wrong. It is difficult to detect these erroneous 
values when the data are spatially comWed, but if a temporal sequence is recorded, we 
can more easily find and reject it. 


In our laboratory, we use an electrical mobile robot Nomad200 to develop and test the 
algorithms. This system can be controlled remotely via a HP-UNIX or PC-Linux 
workstation. The data acquired by the sensors can be viewed in different windows but the 
user can only see a 2D representation of the world. This is perhaps enough in a well- 
known laboratory but not in a real world. We have then developed a 3D virtual 
representation of the robot and of its environment. This application is based on the new 
tools offered by the Internet technology VRML and Java. 

We will begin with a short presentation of the robot Nomad and the sensors we use. We 
will then present the solution we have developed to improve the measurements of the 
ultrasonic sensors. The tools and the 3D application will then be presented. 

A robot called ^^Nomad*^ 

General description 

The Nomad200 is an integrated mobile robot 
system with four sensing modules: tactile, 
infrared, ultrasonic and 2D laser (Figure 1). 
The Nomad200 has an on-board multiprocessor 
system consisting of a 80486 computer and 
multiple slave microcontrollers. This system 
performs sensor and motor control, host 
computer communication, and supports on¬ 
board programming. The Nomad200 offers an 
integrated software development package for 
the host computer including a graphic interface 
and a robot simulator. 

Programming theNomad200 

Figure 1: The NomadlOO 

The Nomad200 can be programmed and controlled in two ways: directly via the on-board 
PC card or remotely via a UNIX workstation. 

In the first case, no communications are possible between the robot and the remote station 
and the user has no idea of what its program is doing. 


In the late case, commands and data are transmitted by an Ethernet radio link. The 
application program called ‘client program’ runs on the workstation and sends 
commands, via the host interface (Nserver), to a communication program running on the 
robot (Robot Demon or robotd)^. This program interprets the packets sent by the station 
and sends itself the data of the sensors to the workstation. The programmer is able to see 
not only what its program is doing but also the movements of the robots and the values of 
the sensors. 

Figure 2: The Nserver GUI 
The ultrasonic sensors 

The system called “Sensus 200” is a ring of 16 Polaroid 6500 sonar ranging modules. The 
Polaroid 6500 is an acoustic range finding device that has been widely used in the mobile 
robotics community. It can measures distances from 6 inches to 35 feet, with a typical 
absolute accuracy of +/- 1 percent over the entire range. 

The transducer do not emits energy homogeneously in all directions, but instead form 
lobes of decreasing intensity, as illustrated in figure 3. 

These sensors generate a lot of 
uncertainties in the measurements. The 
main sources of errors are the 
characteristics and the position of the 
target. Most of the indoor surfaces act 
as mirrors with respect to sound waves. 
One consequence of the reflective 
properties of surfaces is the multiple 
echo effect: the sound wave bounces 
around, and eventually reaches the 
Figure 3: A typical Emission diagram of a sonar sensor receiver after several reflections. 

^ A distributed control mode have been developed and presented in [2] 


A good representation of sonar data is the sonar 
scan: a dot representing the measured distance is 
drawn on the sonar axis. Repeated measurements 
over a given angular range give a sonar scan. 
Sonar scans are the basic data that can be used for 
map building or navigation purposes. 

You can see on figure 4 a typical map build using 
sonar data. 

The mapping of the environment 

In order to obtain a usable map of the environment, several sensors are used together and 
successive measurements are accumulated, or better fused, in a spatial 2D map. Several 
methods exist to fused sonar data: probabilistic or statistical combination, heuristics rules, 
fuzzy logic, etc. 

The idea here is to rejects wrong data as soon as possible, with other words before the 
spatial mapping. Two different methods have been developed: the first one uses the well- 
known statistical median function [3], the second one is based on linear regression and 
the estimation of future readings [4]. The processing is made for each sensor 

The time sequence median filtering 

The idea is to remove fast measurements variations by regarding the mean evolution of 
the data. This processing has to be done during the motion of the robot and in real-time. 

If we look at a typical time sequence recorded during a motion of the robot we see that 
some values are completely different from their neighbor values. The median function is 
a statistical function that is commonly used in image processing to smooth the image. It 
acts like a low-pass filter and has the property to remove some noise while preserving 

In our case we record a time sequence and the values are then ordered. The measured 
value is then replaced by the middle value of the sequence. You can see several examples 
here below. 

Time sequence 

Ordered sequence 






















false data rejected 











preservation of 


15 '35 35 


Value to be replaced 




55 35 

. f'“ 

Middle value 




New value 


In a spatial filtering, there is no difficulties to consider the values surrounding the center 
value (the value to which we applied the filter). But in a time filtering we must process 
the data recorded before and after the value we want to replace. The consequence is that 
the corrected data are only available after a certain delay. The duration of the delay 


depends on the firing rate of the sensors and on the length of the sequence you consider 
for the filtering. 

The length of the sequence also influences the size of the wrong data we can reject. If we 
consider a sequence of N elements, we can eliminate wrong sequence which size is < 
(N+l)/2. Otherwise, the value will be replace by a wrong value. 

In the figures 5 we can see the results of a processing with a size of 7 elements. 

donn^es percues 


Figure 5a and 5b: original and processed data 

The best results were generally obtained with this length. In this case we can reject a 
sequence of 3 successive false measurements and the delay introduced is not to 

We see on the next figure a complete map built with corrected data. 


In order to suppress the delay we developed the method explained in the next paragraph. 

The time sequence filtering by linear regression and estimation of future readings 
With this processing we want to reject the wrong data generated by false echoes. The 
goal is to correct the data in real time and with no delay. To achieve this goal, we 
consider the three last measurements and we compute the straight line that best fits these 
points (Least square method). We can now compute one hypothesis for the next reading 
and compare with the real value. If the next measurement differs too much from the 
hypothesis, we cannot just drop it; several cases have to be considered because this new 
point could perhaps own to a new obstacle. So we have to compute new hypothesis and 

compare the next data 
with these hypothesis. 
We repeat this 

procedure three times 
and if the last point 
does not match one 
hypothesis we stop the 
processing and we 
restart with new data. 
You can see on figure 7 
one example of 

corrected data. 

before processing —W—after processing ] 

Figure 7: Raw and corrected data for one sensor 


We can also compare a map built with raw and corrected data: 

Figure 8; Without correction 

Figure 9: With correction 

We see that some noise remains. These isolated points can now be eliminated (if wanted) 
by a simple image processing (low pass filtering or something like that.. 

The 3D visualization 

As explained in the introduction, video cameras offer a poor resistance to high radiations 
level and an altemati^'e solution is needed to help the user to remotely control the robot or 
the arm for manipulation, A 3D representation of the environment of the robot is certainly 
required. The data acquired by the ultrasonic sensors can be fused with existing 3D 
representation of the installations to indicate parts and obstacles. 

Other requirements for this application were to use low cost computer to perform the 
visualization and to get rapidly an usable system. The solution we propose is based on 
today Internet tools: VRML^ and Java. The best motivation was the existence of 
viewer/browser and the portability of the applications written in Java. 

^ Similar developments are made at the kiss-lab [5] 



The Virtual Reality Modeling Language (VRML) is a file format for describing 3D 
interactive worlds and objects^. It may be used in conjunction with the World Wide Web. 
It may be used to create three-dimensional representations of complex scenes such as 
illustrations, product definition and virtual reality presentations. 

VRML is capable of representing static and animated objects and it can have hyperlinks 
to other media such as sound, movies and images. Interpreters (browsers) for VRML are 
widely available for many different platforms as well as authoring tools for the creation 
of VRML files. 

VRML is a new standard that has known a rapid evolution since its apparition two years 
ago. VRMLl.O provided a means of creating and viewing 3D static worlds; VRML 2.0 
adds interaction, animation and extension capabilities. 

You can find viewers for all platforms and operating system (OS) but each of them runs 
on a limited number of computers. Some viewers can run as standalone application and 
others like plug-in in your favorite browser. At present day, no one viewer implements 
the complete VRML specifications. 

In VRML2.0 you can use scripts to animate objects in a world or give them a semblance 
of intelligence. This script can be written in any programming language that the browser 
supports. In practice it is Java or Javascript, 

JA VA - Applets and Applications 

Java is an object-oriented programming language developed by Sun Microsystems. The 
main purposes of the developers were to create a language that let programmers write 
programs that can be executed on every computer whatever its processor or OS. 

Sun introduced the concept of Java Virtual Machine. When you compile a program 
written in Java, you produce bytecode that will be interpreted by the Java interpreter 
during its execution. Another goal was to create a language that will permit to write 
distributed and network applications more easily. Java can be used to write serious big 
applications (see Corel WordPerfect suite for Java) or simple applet to enhance WEB 
pages. Applets are Java programs that are embedded in Web pages. The code is 
downloaded with the page and executed on the computer of the netsurfer. Of course to do 
this you must have a compatible Java browser. 

Javascript is totally different from Java. It is a scripting language that is interpreted by the 
browser and was introduced by Netscape; conunands are embedded in HTML pages as 

In the following table you can see a sununary of the principal viewer/plug-ins for VRML 
that will run with Linux 


Works as: 


on. platform: 






Vrwave (beta) 





^ See the complete specifications at http.7/www.vrml.orgArRML97/DIS 
^ For a complete review see: http://www.sdsc.edii/vrml 


We opted for the LiquidReality (Lr) package because of existing versions for Linux and 
Windows95 and the possibility to write standalone applications. 

Liquid Reality is a set of Java classes that implement the VRML2.0 specifications. It adds 
the capabilities of 3D technology to the power of Java. It provides a rich set of function 
calls from Java that allow the developers to create and manipulate simple or complex 3D 
scenes as an integral part of their applications. 

With the 3D programming interface for Java, the developer has the ability to create 
applications that can read in and allow the user to view and interact with VRML2.0 
worlds. In addition, the developer has a new level of control over these worlds. Nodes 
can be crated, deleted and modified at mntime. Complex behaviors can be triggered when 
VRML or standard window events are generated. 

The visualization application: 3DApp 

The main functionality of our application is to visualize the motion of our robot in 3D. 
This program conununicates with the Nserver program (see $2.2) and continuously reads 
the coordinates of the robot and its orientation; the drawing is updated at each iteration. 
Normally, to command the robot via the Nserver, we have to write a C program. The Java 
language offers the possibility to call native functions, that is, functions written in an 
other language (C,C++,... ). So the developer can use optimized rendering functions or 
existing specialized API (loosing the portability). 

The solution we adopted was to implement the function that permits the communication 
with the Nserver as a library and to call it from a Java program. 

Java offers different solution for the implementation: the 3DApp can be executed as a 
stand-alone application or as an applet in a browser (or in Appletviewer). Due to security 
and access permissions, an application is easier to write. 

An usual client program for the Nserver communicates with it via sockets, consequently, 
it can run on the same or on a different computer (but with Linux) than the Nserver. So 
the 3Dapp application can run on a remote computer (if this computer has a Java Virtual 




Java + C 

Figure 10: Simple communication: the 2 applications 
can run on the same or on different machines. 

If we want to let our program run on different computers and different operating systems, 
we can implement one of these solutions: 

On the Linux server machine we can write a mix Java/C program that wiU act as a server 
for the 3DApp program or 3D Applet applet. 





Java + C 


Browser + Plug-in 


Figure 11; Communication via a server 

With this configuration we can chose an other viewer/plug-in, if we want for example 
make the visualization on a MS-Windows computer. In this case the speed of the drawing 
could be improve using a JIT^ compiler (not yet available for Linux) 

The 3Dapp can also be used as a standalone application (not connected to the Nserver). 
The user has the possibility to drive the robot with a virtual joystick. You can see a screen 
capture of the interface in the figure 12. 

Future developments 

The developments we have planned are the following: 

• the implementation of the second communication solution (see above), 

• the command of the real robot via the 3D GUI, 

^ A Just-in-Time compiler compiles the Java bytecode into machine code. 

• The VRML standard does not handle collisions between objects but only between the 
viewer (called avatar) and objects. In order to use this application as simulator this 
behavior should be added to the objects. 

• At this stage, the user can navigate freely into the scene. An interesting option should 
be the automatic following of objects and the automatic selection of the best 


Today we see very sophisticated user-interfaces with multi cameras, HUD, etc... In 
nuclear applications quite simple sensors are used because they resist well to radiations 
doses. This work addresses two problems introduced by this restriction: the improvement 
of ultrasonic data by time filtering and the representation of the 3D environment of the 

The developed algorithms for the data filtering have proved to be robust and improve the 
quality of returned data. 

The 3D representation is at the beginning of its development. It is based on today Internet 
standards and runs on several platforms. 


[1] Decrdton, M., De Geeter J., Coenen S. (1997) Implementing radiation hardened 
sensors for computer aided teleoperation in a nuclear environment ISIE’97 

[2] Colon, E., Baudoin, Y.(1996) Development and evaluation of distributed control 

algorithms for the mobile robot Nomad200 SPIE Conference Boston 

[3] Heggermont, B. (1996) Amelioration des mesures renvoyispar les capteurs a ultra- 

sons du robot Nomad200: traitement des faux-echos. Rapport de stage ISIB 

[4] Heggermont, B. (1997) Lissage temporel en temps reel de mesures par ultra-sons: 

application a la robotique mobile Travail de fin d’dtudes ISIB Bruxelles 

[5] Holzhausen,K.-P., Colon, E., Baudoin, Y. (1997) Experimental data fusion for path¬ 

planning of mobile wheeled robots IMEKO Topic 17 Tempere Finland 


Performance Shaping Factors in Teleoperation 
Using a Six-Axis Active Hand Controller 

Research on Efficient System Parameters 
in Bilateral Teleoperation 

Gordon Gillet and Herbert Rausch 
Institute of Ergonomics 
Technical University Munich 
Garching, Germany 


A six-axis hand controller designed to use the high sensitivity of the human hand-fmger- 
system is presented. The applied input force is measured by a force/torque-sensor and 
proportionally transformed to control the motion and the output force of a six-axis in¬ 
dustrial robot. The motor feedback of the robot position to the hand controller gives the 
operator the feeling of handling the robot’s load with his own hand. By means of a sim¬ 
ple task which is matching a cylindrical peg into nine holes with different diameters and 
chamfers the operator’s performance is evaluated. The operator’s forces and the peg po¬ 
sitions are recorded while the task is carried out. The time required to complete the task 
(sequential match into each one of the holes) is the main indicator of the operator’s per¬ 

Research has been made on the influence of some parameters like force / torque resolu¬ 
tion (by time and value) on force measurement as well as translation and rotation scales 
on the position / orientation feedback. Results show best performance on scales that cor¬ 
respond to the finger workspace if the hand is supported. For the examined task, the ori¬ 
entation feedback comes out to be more important than the position feedback. 

The force / torque transmission parameters might not be necessarily as precise as the 
human fingers performing the same task under optimum circumstances since the dy¬ 
namics of the controlled manipulator is more restricting. Complementary inquiries have 
been made on knob size and shapes. 


The optic, acoustic, kinesthetic and haptic senses have evolved to constitute the most 
important information channels for people to perceive the environment. On controlling 
machines, some information can be perceived without technical displays (e.g. accelera¬ 
tion, noise, track while driving vehicles), others must be provided by those displays (e.g. 
velocity and height in airplanes). Conventional displays mostly use the optical and 
sometimes the acoustical channel (e.g. for speech output and alert signals). 

The haptic channel is essentially used for recognition of forms and surfaces as well as to 
determine the force which is applied by or to the surroundings. Tasks like inserting 
screws into threads, cutting soft things on hard surfaces, or digging in non-homogenous 
soils are much harder to complete without the haptic information (= force and position 

When controlling machines hke cranes and power shovels, this haptic sense usually gives 
feedback on the information transmitted to the machine, but not about its internal state 
(e.g. position and forces). For effective appHcation of the highly sophisticated haptic 
sense, whose reaction time and perception thresholds are significantly lower compared to 
the other senses, these machine states are displayed on active hand controllers. Thus, the 
operator handles his tool intuitively like if the tool was in his own hands. By the use of 
highly trained habituation patterns, learning time and mental work load are reduced, and 
at the same time dangerous situations can be overcome safer and faster. 

Regarding the high motorial precision and sensitivity of the hand-finger-system for the 
control of many kinds of manipulators, near or distant, the Active Hand Controller „Spi- 
der“ (Fig. 1) has been developed, constructed and tested at the Institute of Ergonomics. 
In comparison to typical manual input devices (joysticks, wheels, pedals) it can provide 
full haptical feedback. This is achieved by a force/torque-sensor which measures the ap¬ 
plied force and a positioning unit which displays positions and orientations. It is de¬ 
signed as a 6-degrees-of-freedom device to be a universal platform for experiments and 

Figure 1: Active Hand Controller „SpideC 

The advantages of force and position display in comparison to conventional controllers 
are well known. The aim is now to quantify the influence of single system parameters on 
human performance in bilateral teleoperation with haptic feedback. In this way, the ex¬ 
tensive control and hardware needs can be reduced to a cost effective minimum without 
decrement of human control capabilities. 

System Description 

The used teleoperation setup consists of the Active Hand Controller „Spider“ as the 
Master in combination with a Puma 560 industrial robot as the Slave. The setup involves 
principally two independent control chains: 

• Proportional transfer of hand-/finger forces to the manipulator (Slave). 

• Position / velocity feedback of the manipulator's action to the fingers of the 

The environment of the manipulator and the operator close the control chains to a control 
loop (Fig. 2). 

nrwrriinatP ] _ 

4 ^ ( JransformationJ [ 



f Workspace ) 
[ Sun/ey J 


Active Hand Controller 

Co^rol j 


1 1 
* * ^ 

^ I 

Robot ^ 




Encoder . 





(The dotted lines serve for keeping velocity limits and workspace) 

Figure 2: Control scheme of the teleoperation system 

Transmission of Forces from the Hand Controller to the Manipulator: 

Forces and torques of the hand and fingers are measured by a 6-axis force/torque-sensor. 
They have to be output by the manipulator in two different manners: 

• as a velocity (without environmental contact) 

• as a force (in constrained motion) 

The key point in the transmission control is to choose between velocity and force output. 
Therefore, the default velocity control is replaced by force control if tlie motor currents of 
the robot (which are proportional to the motor torque) exceed the value that the input 
force allows. The application of these two schemes provides controlled movement 
through the free space as well as relatively sensitive handling when in contact with ob¬ 


Position Feedback from the Manipulator to the Hand Controller 

The position of the robot^s tool center point is calculated from the built-in angle encod¬ 
ers. TTie joint positions are transformed separately by translation and rotation into Carte¬ 
sian coordinates, which can be displayed proportionally by the hand controller. 

Hand Controller Mechanics 

The positioning unit of the hand controller is a complete robot with 6 degrees of freedom. 
The essential difference to typical industrial robots is the kinematics which is chosen 
following the mobility and sensitivity of the human hand-finger-system. It consists of 
linear units with universal joints at both sides, connecting a base plateau with a head 
plateau (Fig. 1). Special care has been taken to construct joints without play and wide 
angular limits at the head plateau. 

The construction is based on a geometry optimized and checked by computer calcula¬ 
tions. A postulation was a rotatory mobility of 30 degrees around any axis. Thus, the 
maximum workspace of the positioning unit is 2,3*10^ mm^ which contents a symmetri¬ 
cal workspace of 1,3*10^ mm^ (Fig. 3). This workspace is greater than the area a sup¬ 
ported hand can reach. 

Figure 3: Maximum symmetrical workspace of the ^Spider"' (Mobility 30° around any 


The actuators are stepping motors. Damping elements suppress excessive noise and high 
frequency oscillations. The position accuracy is strongly depending from the actual posi¬ 
tion, but always better than 10 pm. Horizontal maximum speed is 125 mm/s, maximum 
acceleration and deceleration values are 5.1 m/s^ and 41 m/s^. Vertical rates are about the 
half of this figures. This high deceleration makes even hard impacts well recognizable 

Experimental Setup 

[Hannaford et al (1991)] presented a frequently cited experiment for teleoperation. The 
task is to insert a peg (diameter 25.35mm) into 9 cylindrical holes. The holes are placed 
in a 3x3 matrix and differ by diameter (25.4lmm to 25.47mm) and chamfer (0.4mm*45° 
to 1.6mm*45°). The task has a Fitts difficulty from 7.8 to 9. The time needed to insert 
the peg sequentially into the holes (twice for each pass) is an indicator for the operator's 


performance. It also depends on the system's parameters and indicates the ergonomic 
aptitude of the parameters set. 

For the experiments, an arm and hand support which is free movable horizontally is pro¬ 
vided. The subjects have to train the task a certain number of times so that learning ef¬ 
fects disappear. The holes to be inserted are indicated by LEDs. The task can not proceed 
if the desired insertion is not complete, so that missing or incomplete insertions never 
occur. The time needed to perform the whole cycle is reduced to a fit-in time, which is 
the remainder when the time from leaving the last hole to approaching (1cm) the next 
one is eliminated. This reduction (amounts an average of only 18% of the complete pass) 
is made to ignore unconstrained movements and single mistakes in the hole order made 
by the subjects. 

Experiment 1: Scales 

In this experiment, the influence of the scale in which robot movements are transformed 
into movements of the hand controller are examined. It can be distinguished between the 
scales of position and orientation. Four different scale combinations are tested: 

• A: position 1:1, orientation 1:1 

• B: position 3:1, orientation 3:1 

• C: position 6:1 orientation 1:1 

• D: position 16:1, orientation 16:1 

Hereby, at combination A the workspace of the hand controller is widely utilized, while 
D represents almost an isometric controller with very small movements. 

Results of Experiment 1: 

There are 50 subjects with an average age of 27 years (a=6.4) performing the task. Af¬ 
terwards they are presented a questionnaire to judge the different scale combinations. 

Fit-in times: The following times are achieved at the four combinations: 

Scale combination 

Fit-in time t [s] 

Std.-Dev. a [s] 

A(Pos. 1:1, Or. 1:1) 



B(Pos. 3:l,Or.3:l) 



C(Pos.6:l,Or. 1:1) 



D(Pos.l6:l Or. 16:1) 



Table 2: Fit-in times for experiment 1 

The null hypothesis „A does not differ from B“ must be accepted by a probability of 
0.75, All other combinations are different by a very high degree of statistical signifi¬ 



The judgement on the order of preference constitutes the most relevant item in the ques¬ 
tionnaire. 38 of 50 subjects vote for combination C as the most favorable one, 26 judge 
D as the least preferable one. The question about the necessary concentration at each one 
of the combinations show up similar results. Most concentration is needed at D, least at 
C. In both cases, the combinations A and B receive average judgements without signifi¬ 
cant differences. 

Experiment 2: Force Parameters 

This experiment covers some parameters of the force / torque measurement at the hand 
controller. As the implemented sensor is able to work at high sampling rates (up to 520 
Hz) and a resolution of 10 bit, the point of interest is if lower precision would reduce the 
system's overall performance. As it has been shown by some physiological experiments, 
the human fingers' sensitivity can beat the above benchmarks under ideal circumstances. 
The following four force parameters are chosen for this experiment: 

• E: (Ideal) Sampling rate 520 Hz, Force resolution 0.02N, Torque resolution 

• F: Sampling rate 65 Hz, Force resolution 0.02N, Torque resolution 0.0002Nm 

• G: Sampling rate 22 Hz, Force resolution 0.02N, Torque resolution 0.0002Nm 

• H: Sampling rate 520 Hz, Force resolution IN, Torque resolution 0. INm 
The position scale is set according to the result of the previous experiment (6:1 for posi¬ 
tion, 1:1 for orientation). That is why the maximum speed of the manipulator is raised in 
this experiment in order to adjust it to the speed range of the hand controller. 

Results of Experiment 2: 

There are 24 subjects with an average age of 32 years (a=10.5) performing the task. Af¬ 
ter the experiment, a similar questionnaire to experiment 1 has to be completed. 

Fit-in times: 

The subjects achieve the following fit-in times with the different force parameters: 

Force Parameter 

Fit-in time t [s] 

Std.-Dev. a [s] 

E (Ideal) 



F (Sampling Rate 65 Hz) 



G (Sampling rate 22 Hz) 



H (Force/Torque resolution 1 N / 
0.1 Nm) 



Table 2: Fit-in times for experiment 2 

The null hypothesis must be accepted between the parameter sets E and H as well as F 
and H, and there is a very high significant difference between G and the other three pa¬ 
rameter sets. 


According to the objective data, subjects are able to judge G significantly as the worst 
parameter set, but can not state differences between the other ones. 

Knob Judgement 

During the two experiments mentioned above an accompanying inquiry concerning knob 
characteristic is made. The two utilized knobs are: 

• A 43mm plastic ball with regular hollows (golf ball) in experiment 1 

• A 60 nun wooden bowl in experiment 2 

Questions are asked about the size and the surface of the knob. The judging criteiia is a 
scale with 5 items ranging from too small to too big and too smooth to too rough, re¬ 

The results are as follows: 

Experiment 1 


too small 

a bit too small 



a bit to big 


too big 


43mm diame¬ 


too smooth 

a bit to smooth 



a bit too rough 

too rough 


Experiment 2 


too small 

a bit too small 



a bit to big 

too big 


60mm diame¬ 




too smooth 

a bit to .smooth 



a bit too rough 

too rough 


Table 3: Judgements on knob sizes and surfaces 

Regarding the form of the knob, there are two opposite judgements: about the half of the 
subjects would prefer some kind of cylindrical knob shape, the other half does not. Inde¬ 
pendently, most subjects would prefer some bigger grip-holes and a more anatomic shape 
to augment the perception of orientation. 


In Experiment 1, scale combination C, the only one with scales different by position and 
orientation (6:1 resp. 1:1), is evaluated subjectively as well as objectively as the best 
combination. Following factors contribute to achieve this result: 

1) Orientation feedback is more important for the performed task than position feedback, 
that is why combination B (3:1,3:1) and combination D (16:1,16:1) are less advanta¬ 


2) The workspace of the human hand-fmger-system is well utilized. The arm support can 
be put under the ball of the hand so that full advantage can be taken of the finger sen¬ 
sitivity (not so in combination A(l:l,l*l))- 

3) High velocities at the hand controller, especially translatory ones (like at A), interfere 
with the haptical feedback by out-of-round action of some linear units. These vibra¬ 
tions of about 15 Hz are transferred to the entire hand controUer. Furthermore, the 
motion of the six units is not coordinated but each one is heading for its desired posi¬ 
tion as fast as possible. Thus, a maximum straightness of the movements is not guar¬ 

On comparing the scale combinations A, B and D the following conclusions can be 

• The factor 1) balances the factors 2) and 3) comparing A with B 

• The almost isometric combination D provides too less haptic information 
so that the superiority of hand controllers that reflect force and position 
has once more been proven. 

Experiment 2 shows clearly that the force/torque parameters may not meet the require¬ 
ments of the human sensitivity under optimum circumstances. The dynamics of the con¬ 
trolled manipulator and the control loop is supposed to be the benchmark for force pa¬ 
rameters. In every case, a stable control of the system implies lowpassing of the input 
force so that even rough quantizations by time (65 Hz, „F‘) and value (1 N , 0.1 Nm, 
„H“) do not impair human overall performance. Only a very slow sampling rate (22 Hz, 
„G“) has a perceivable damping effect on manipulator control so that a difference in fit- 
in times and subjective rating can be measured. 

Regarding the two experiments, some recormnendations on the design of an active hand 
controller for teleoperation purposes with force input and position feedback can be for¬ 

• The scale for rotation and translation should be chosen to fill the hand-finger work¬ 
space so that the hand can be supported. The separation of position and orientation 
scales is not problematic, 

• The resolution (by time and value) of the force/torque sensor does not have to meet 
highest requirements. It can be oriented by the system^s overall dynamics, 

• A knob size of 60mm (bowl) was judged convenient in average by all subjects. Some 
grip-hole like structures are desirable. 

Further research is planned to be made on other force parameters (offset & hysteresis) 
and minimum requirements to the hand controller's mechanical components to be func¬ 
tional without loss of human performance. 


Bolte U. (1991): DasAktive Stellteil - Ein ergonomisches Bedienkonzept Dissertation, 
Technische Universitat MUnchen 

Bubb H. Schmidtke H. (1993): Ergonomie, Kap, Systemergonomk, Hanser, Munchen 

Oriels M., Spain H., Aviles W. (1992): The Effect of Sensory Feedback on the Identifi¬ 
cation of Objects Using a Teleoperator, Automation in Construction 1, p.l67 

Garcia K. D., Chapel J. D., Spofford J. R. (1990): Effect of Hand Controller Character¬ 
istics on Operator Performance, Proceedings: ISMCR "90, International Meas¬ 
urement Confederation, Houston, Tx, USA, 20.-22.6.1990, Vol.l p. B3.2.1 

Goble A. K., Collins A. A., Cholewiak R. W. (1991): Vibrotactile Threshold in Young 
and Old Observers - The Effects of Spatial Summation and the Presence of a 
Rigid Surround, Journal of the Acoustical Society of America, Vol.99, Iss.4, p. 

Hannaford B., Wood L., McAffee D. A., Zak H. (1991): Performance Evaluation of a 
Six-Axis Generalized Force-Reflecting Teleoperator, IEEE Transactions on Sys¬ 
tems, Man, and Cybernetics, Vol.21 No.3, p. 620 

Kazerooni H. (1991): Achievable Performance in Bilateral Teleoperation, Proceedings: 
Robotics Research 1990, The Winter Annual Meeting of the ASME, Band DSC- 
26 p. 225 

Klatzky R. L., Loomis J. M., Lederman S. J., Wake H., Fujita N. (1993): Haptic Identifi¬ 
cation of Objects and Their Depictions, Perception and Psychophysics, VoL54, 
Iss.2, p. 170 

Leung G. M, H., Francis B. A., Apkarian J. (1995): Bilateral Controller for Teleopera¬ 
tors with Time Delay via p-Synthesis, IEEE Transactions on Robotics and Auto¬ 
mation, Vol.l 1, p. 105 

Lumelsky V. (1991): On Human Performance in Telerobotics, IEEE Transactions on 
Systems, Man and Cybernetics, Vol.21 Iss.5, p. 971 

Maeda S., Griffin M. J. (1994): A Comparison of Vibrotactile Thresholds on the Finger 
Obtained with Different Equipment, Ergonomics, Vol. 37, No. 8, p.l391 

Massimino M, J., Sheridan T, B. (1994): Teleoperator Performance with Varying Force 
and Visual Feedback, Human Factors 36(1), p.l45 

Rausch H. (1996): Einflufi der haptischen Ruckmeldung von Stellteilen auf die 

Leistungsfdhigkeit bei Kransteuerungen, Untersuchungsbericht, Lehrstuhl fur 
Ergonomic, TU Munchen 

Schmidtke H., Riihmann H. (1989): Handbuch der Ergonomie, Hanser, Munchen, p. 

Thurecht F., Miicke S-K.M., Rohmert W., Kubbat W., Landau K. (1996): Entwicklung 
eines aktiven Steuerkniippels zur Untersuchung der Mensch-Maschine- 
Schnittstelle „ Cockpit, Jahrbuch 1996 II der Deutschen Gesellschaft fur Luft- u. 
Raumfahrt - Lilienthal-Oberth e. V., p. 923 

Virokannas, H. (1992): Vibration Perception Thresholds in Workers Exposed to Vibra¬ 
tion, International Archives of Occupational and Environmental Health 64, p. 377 

Virtual Reality Usability Evaluation Techniques: 


Pedro Z Caldeiraf and Nuno Oterot* 
Desaflo Total - Realidade Virtual e Multimedia, Ida 
finstituto Superior de Psicologia Aplicada 
tinstituto Superior de Economia e Gestdo 
Lisbon, Portugal 


This paper is based on two case studies which gave the authors a great deal of insight into 
the development of Virtual Reality (VR) systems. The authors have chosen to 
diiferentiate two types of observations: those related to development process, and those 
related to design issues. 

RECMUVI and TILE VIZ are VR systems. The first focuses on the environmental 
education of 10-12 year olds on urban solid waste recycling. The second focuses on the 
selection and 3D visualization of floor and wall ceramic tiles. 

In both projects two different usability techniques were employed: heuristic evaluations, 
made by the authors, and participative evaluations, made by end-users. Each technique 
raised different (but sometimes overlapping) usability problems. The use of the ‘thinking- 
aloud’ technique was questioned, as the possible demands on cognitive resources affect 
subject’s explorations of the virtual world (RECMUVI). Group interviews with end-users 
focusing on a checklist of usability evaluation criteria themes were made. These meetings 
proved useful because they enable end-users to report their experiences and discuss the 
central aspects of the interactions. 

The main design problems were related to the need to generate virtual environments with 
cues that orientate the navigation and representation of the virtual space. The need to 
develop navigation systems that do not present cognitive overhead and the possibility of 
the co-existence of different navigation types, each one with different motor and 
cognitive requirements, was also raised. 


Human computer interaction evaluation is common practice in computer systems design. 
Through the use of evaluation tests designers want to assure systems adaptation to end- 
users. And this is true not only of ordinary computer systems but also of more innovative 
technology (Virtual Reality - VR - is only an example). These evaluations gain a 
growing importance when designers want to generalize innovative technology for non¬ 
expert users. 

VR systems have special features very different from those of other computer systems, 
demanding complex interaction styles. First of all, one of the main goals of VR systems is 
the transmission to users of a feeling of presence in a computer-generated (virtual) world. 

' The authors wish to thanks to Prof. Antdnio Sousa Camara, Head of Environmental Systems Analysis Group (GASA) 
of Faculdade de Ciencias e Tecnologia (New University of Lisbon), as RECMUVI was one of GASA’s projects. We 
would like to express our gratitude for the valuable help of Celeste Duque and Angelique Chrisafis. 


That feeling of immersion is the computer generated illusion of the user being inside a 
virtual environment. According to Hayward (1993), the brain is the last artifice of 
immersion. Furthermore, Varela (1995) argued that VR can be thought of as a process of 
excess of signification and the ability of the mind as a neuronal “narratives” generator. 
Pimentel and Teixeira (1993) consider "'The question isn't whether the created virtual 
world is as real as the physical world, but whether the created world is real enough for 
you to suspend your disbelief for a period of time.'' (p. 15), a concept shared by Slater and 
Usoh (1993). These last authors stated that the human partner builds the world through 
those displays and acts in an extended virtual space generated through the interaction 
between perceptual system and computer-generated displays. 

Second, and as a way to assure the feeling of presence, VR systems generally have 
complex user interfaces. Immersion, or the feeling of presence, in a virtual world almost 
always requires a visualization tool (e.g., a Head Mounted Display - HMD), coupled 
with a navigation and/or object manipulation tool. 

Therefore, the need to better understand VR systems characteristics and their cognitive 
demands are essencial to design better VR systems, i.e. systems adapted to end-users. In 
this paper, we present two case-studies we followed during the past two years, where VR 
systems characteristics were analysed. 

RECMUVI background 

RECMUVI was a project to develop a VR game applied to environmental education on 
urban solid waste recycling, aiming to generate and develop skills in 10-12 years old on 
collecting and selection of recyclable materials (Caldeira, Dias, Otero, & Silva, 1995). 
The evaluation of the interface was made for die indoors situation, were the players 
should collect recyclable objects and put them in the respective boxes. 

Tools. This game was developped in a Pentium 90, using WorldToolKit of SenseS. Due 
to technical restrictions the house was broken down into six models (each model 
representing a room - corridor and five other rooms). 

The game used a HMD (Virtual Research) with a visual field of 90 degrees horizontal 
and 35 degrees vertical, a navigation tool (Trigger, Figure 1) with a button for object 
manipulation, and one (or two, depending on the prototype) Polhemus track sensors. 



Figure 1. Trigger 

Prototypes. Prototype 1 representes three rooms in the virtual home: the kids’ room, 
bathroom and corridor. The viewpoint is given through head movements. Navigation was 
controlled by two tools: the trigger control the navigation velocity (only forward 
movements), and the HMD control the navigation direction. 

Object manipulation included a visual cue - a virtual hand - and two different actions: 


approaching the object and touching it, or the surrounding area, with the virtual hand. 
This last action makes a yellow wire frame appear around the object. Only then should 
the user press the trigger button selecting the object (which became attached to the virtual 
hand). In order to drop the object, the user must press the same button a second time. 

The corridor was the only model connected to the other models, the entrance and exit of 
rooms was made through a portal, with a waiting time filled with a fixed white image. 

Prototype 2 includes all rooms: parents’ room, kids’ room, dining room, kitchen, 
bathroom, and corridor. This prototype differs fi'om the first one in the following aspects: 
introduction of lights that simulate shadows and contrasts; different height-width ratio; 
introduction of various object and boxes in blue, green, yellow, and red colours; the white 
in portals changed into a less aggressive blue. 

In prototype 3 the corridor doors that where closed on the first two prototypes are now 
opened, with the visualization of each room interior. 

The navigation scheme changed. The navigation direction is controlled by the trigger. 
Hand movement corresponds to changes in the visualization of an arrow - that replaced 
the virtual hand. This functionality was introduced with the help of another Polhemus 
sensor (one in the HMD - to the viewpoint - and the other one on the top of the trigger - 
to the navigation direction), 

TILE VIZ background 

TILE VIZ is the code name adopted for Virtual Decor™ 1.0^ during the development 
process and is the name used throughout this paper, TILE VIZ is a tool to help users in 
selecting and visualizing ceramic tile patterns. Therefore, TILE VIZ is a design 
application of WC’s and Kitchens where users can place typical objects and select and 
visualize these objects and wall/floor ceramic tiles. 

The evolution of the project demanded the development of three interdependent modules. 
In the first one users can design the WC and/or the kitchen and place various typical 
objects in the designed room/s (doors, windows, lights, bathtub...). 

In the second module users can select tile patterns. They can visualize the design rooms 
and selected objects and tiles in the last module, the real VR model. In this model users 
can navigate and do various tile manipulations. 

Tools, TILE VIZ was developped in a Pentium 120, using a toolkit from Microsoft. The 
visualization tool (similar to die one used on RECMUVI) was a HMD from Virtual i-O. 

In the first prototype, the navigation (with 6 degrees of freedom) was controled by two 
tools: the HMD controling the navigation direction; and the trigger controling the 
navigation velocity. 

In the second prototype the trigger button is active and users can do several tile 
manipulations (e.g. change of patterns). 


Virtual Decor™ 1.0 is a trademark of Soexporta, Lda. 


In the third prototype the trigger was replaced by a pad (Figure 2), with an extended set 
of functionalities: users can not only change tile patterns but also move bars and wall tiles 
upwards and downwards. 

Prototypes. The first prototype (very priihitive), was only a part of the final VR model: 
one room with a door, a panoramic window and four bare walls. The ceiling was the 
source of a diffuse light. 

The second prototype again includes just the VR model (with kitchen and WC connected 
by a corridor and with several typical objects - bathtub, light, kitchen-sink - 
automatically generated by the system). This prototype had two different types of light: 
natural and artificial. Three types of navigation were implemented: free; fixed height; and 
separate view points.. 

The third prototype evaluations included all three modules. As the first two modules are 
typically Windows (point-and-click), this paper only reports the evaluations made of the 
VR module. The virtual world is identical to the last prototype (richer in objects) and the 
users are the ones that design the rooms, place the typical objects and select tile patterns 
that are going to be visualized in the VR model. The functionality of navigating 
backwards was added, in this prototype. 

Development process 

Design team. The design team constitution and ergonomist role in the team influenced 
the design and development processes of both projects. RECMUVI design team, with 
three computer scientists, three ergonomists, and two environmental engineers developed 
the project starting from computer solutions. 

The lack of comprehension of the computer scientist ergonomists role on the design team 
led them to minimize the importance of heuristic and participative evaluations. In the 
heuristic evaluations (always a posteriori regarding computer implementation) 
suggestions made by ergonomists were followed only when they were easy to implement. 
Problems with software architecture were systematical ignored, as changes in structure 
pointed to a radicaly different design philosophy. For computer scientists interface was 
secondary regarding to programing problems and solutions. 

In the participative evaluations, it was very interesting to observe the computer scientists 
change of behaviour regarding evaluation sessions. From the moment they saw subjects 
fighting with their design, several of them enthusiasticaly adhered to the evaluation 

TILE VIZ design process followed a completely different philosophy. One of the authors 

was the manager of the design team, and that radicaly change the ergonomists role. Their 
participation started from the beginning of process and the interface was the first thing to 
be defined. Therefore the system design followed the interface design. The goal was to 
simplify and potencialize users’ performance. 

Almost all changes proposed by ergonomists to other team members (three computer 
scientists) were accepted and implemented in the system. Participative evaluations were 
prepared by the design team and followed with attention by all of its members. 

Methodology, The first question faced by the evaluation team was related to the validity 
of transfer to VR systems of traditional methodology of interface evaluation. Before the 
option for traditional methodology, the hypothesis of drama-based evaluations was 
considered as suggested by Brenda Laurel (1993), but those dramatizations were not 
chosen due to lack of resources. 

Therefore, both projects were subject to usability iterative tests using three vertical 
prototypes (Hix & Hartson, 1993). In both projects heuristic evaluations were carried out 
by two ergonomists (2 evaluation points in RECMUVI and a continuous evaluation on 
TILE VIZ) and two tests with nine non-expert users. Subject numbers per session are 
those advised in literature for the two evaluation types (Nielsen, 1992; Virzi, 1990, 

In TILE VIZ there was a third evaluation session in real context with 103 subjects testing 
the software during three days. 

Design evolutions 

Figure 3. RECMUVI - Evaluations Scheme. 

On heuristic evaluations Ravden and Johnson (1989) usability criteria were used as 
guidance. These criteria were also used to formulate questions during the evaluation 
sessions, especially in the 3'“'^ prototype, where a group interview took place, with open 

Subjects had 15 minutes to explore the game. Then there was an individual interview (2"^^ 
prototype) or group interview (3'^^ prototype). 

Some objective performance indicators were chosen and developed (e.g. number of 
picked up and dropped objects, execution times). Users were asked to draw the virtual 
home plan. 


The instruction for the participative evaluations was simple: find the objects and drop 
them into the correct recipients. All the interactions were taped on video then analyzed by 
the authors. 

On the participative evaluation made to the 2”^* prototype some questions were asked 
during the interactions, as a means of promoting the thinking-aloud process. But both the 
thinking-aloud process and the questions revealed to be concurrent stimuli that interfere 
with users performance (e.g. interactions stops). 

On the evaluation of the prototype the thinking-aloud technique and the questions 
during the interactions were substituted by a group interview a less intrusive technique. 


Figure 4. TELE VIZ — Evaluations Scheme. 

In this project, authors used the techniques that they found most valid in RECMUVI. The 
interactions on the r‘ participative evaluation were coded in a grid and taped on video for 
further analysis; after the interactions users were interviewed in group. 

The criteria of Ravden and Johnson (1989) were again used for extracting useful 
information fi:om users. Group interviews showed to be a good means of explaining 
clearly the problems that the subjects found, and precious ideas generators. 

Some improvements were introduced. First, the heuristic evaluations were performed 
continuously, the ergonomists were always present when decisions were made regarding 
the interface style and interactions type. 

Second, tasks were given to users. On the T* participative evaluation, those tasks were 
specific, closely related to vision criteria (accuracy, color, distance measure) and 
locomotion (execution styles and times). On the 2"^^ evaluation, tasks were simply 
suggested, since the context required such an approach, and object manipulation criteria 
were added. 

It should be noted that the 2"*^ evaluation was extremely useful, since the context where it 
took place was similar to the one of its final use. Authors were able to see the impact of 
context on system and users performance. 

The final participative evaluation was a confirmation for the choice of the navigation and 
manipulation tool, following the template of the first evaluation, with individual 


interviews and without video recording. 

Design Evolutions 

The 1 considerations to be drawn were related to simple indicators, such as luminance, 
height/width ratio, that can be considered to be basic problems on the construction of 
virtual worlds. Although those problems seem rather evident, one should not take into 
account just psychophysiologic criteria, but also consider aesthetic aspects. 

The next issue is the necessity to implement a medium height of navigation. One of the 
central elements of different difficulty stages could be the development of different types 
of navigation. So, in TILE VIZ, the second prototype includes three navigation types, that 
cover all system needs. In general, it seems that fixed height navigation and the free 
navigation require different cognitive resources, and this can be applied to all VR 

The search for a better navigation system can be viewed with the change introduced in 2"^^ 
to 3^^* prototype of RECMUVI. The dissociation between viewpoint and moving direction 
implied the existence of two different sets of visual cues. On one hand we have the 
images actualization in response to head movements. On the other hand, we have the 
modifications on direction arrow in response to hand movements. 

This solution introduced a greater complexity level, since users would have to control 
two position referentials, each one with their visual cues. Furthermore, the direction 
arrow is “linked” to the viewpoint since subject is always seeing the arrow, whatever the 
direction he is looking to; finally, the arrow takes the relative position of the direction 
(hand movement) and coordinates with the eyeing direction. So they are not completely 

The complexity of such solution should have been studied, introducing his characteristics 
gradually, allowing an evaluation of the cognitive capacities required. 

The next point reports the inclusion of visual cues to guide the user in the virtual world. 
One first question concerns the reduction of the visual field and the discontinuity on the 
space apprehension, requiring bigger memory resources to its representation. A simple 
corridor with 6 different doors demands that the subject explore systematically the 
different parts that constitute that corridor, not apprehending it with one glance. The 
objects that go out of the limited visual field have to be retained in memory, especially 
their locations. These affirmations are, in part, supported by data obtained through plant 
drawings in RECMUVI project. After the corridor doors are opened, subjects present 
more correct plans, although the navigation is much more difficult and the exploring is 
then restricted (note that the subjects in the 3*^^ prototype enter less in the different 


Table L Comparative results of RECMUVI i 

orototypes 2 e 3. 


Frequency of 


Frequency of 

Mean Number 
of Correct 

Number of 

Average of 















The other issue in visual cues has to do with the difficulty to calculate distances in the 
virtual environment. The nature of the TILE VIZ itself solved the problem, since the 
simulation of tile application introduces a squad in the space that facilitates the 

Finally, the last thing to report is the change of the navigation tool, from the trigger to a 
pad in TILE VIZ. On one hand, this decision was taken due to the end increase system 
functions. On the other, it was essential to implement the functionality of walking 

Walking backwards is cmcial since users had difficulties in executing a rotation of 180 
degrees, necessary for driving away from an object, for instance. The pad allowed the 
existence of one button to move forward and another to move backwards. 

Finally, since this pad is an infrared one, users showed less navigation difficulties, as 
wires, that were a constant embarassment were reduced. 


The approaches concerned with the usability iterative tests are accepted to be the most 
effective, as suggested by Landauer (1988), Jeffries, Miller, Wharton, and Uyeda (1991), 
Bayley, Allan and Raiello (1992) and Virzi, Sorce e Herbert (1993). The application of 
heuristic evaluations and end users tests, as complementary methodologies, enables the 
correction of the gross majority of system problems. TILE VIZ project experience was 
truly helpful in clarifying this issue. 

Barnard et al. (1986, in Barnard, 1991) raise doubts about the use of thinking-aloud 
techniques. On a first contact with a virtual environment, the interaction developed by the 
user is centered on action itself. Users search for correlations between body movements 
and the system states, paying less attention to verbal behaviors. 

As the thinking-aloud technique demands cognitive resources needed for the interaction, 
it affects users ability to explore the virtual world (RECMUVI). This technique may be 
admitted as a good double task indicator. 

Task definitions and group interviews revealed to be more productive techniques. On one 
hand, navigation and objects manipulation problems are clarified with task performances, 
evaluated with objective criteria. On the other, group interviews enable a sharing of 
individual experiences, a discussion about the central aspects of the interaction and 
proposals of solutions for recognized problems. 


Another important problem raised by Bailey et al. (1992), is concerned with the abilities 
that should characterize the evaluator. This does not seems to restrict to different levels of 
technical knowledge about user interfaces, or systems, in spite of the importance it surely 
has (Nielsen, 1992). Barnard (1991) when referring that a significant share of die 
exploratory development methodologies depends on the evaluators intuition and expertise 
qualifications also sustain this fact. 

Future research studies will focus on a VR interfaces development model with a 
sequential implementation of the characteristics and a recording of users* behaviors in 
minimal units. 

Let us now look at VR issues. Following some ideas proposed by Varela (1995) it can be 
accepted that a cognitive system experience implies a linkage between a structure and an 
environment. Therefore, there is a context where the user, as a cognitive entity with a 
body, possess one perspective and relates to the VR system through a specific dynamic. 

As is viewed by Steuer (1992, in Slater & Usoh, 1993) VR is created by individual 
consciousness mechanisms, revealing the importance of studying subjective variables and 
their effects on the interaction of the subject with the virtual world (e.g., presence 

It is easy to create a virtual world sensation of presence because human cogmtive system 
generates the illusion of immersion. This implies that, in a VR system, a careful 
conception of all aspects related with the interaction (which comprises vision, navigation 
and object manipulation) should be developed, especially when deahng with non-expert 
users. It follows that, if the feeling of presence is easy to obtain in a first stage, helped by 
the technology natural attractiveness, it is clear that the following interactions, if not 
intuitive and easy to learn, may kill the first impression. 

Keeping the exciting aspect of VR demands intuitive and easy to learn systems able to 
generate comfort and safety feelings corresponding to users expectations. To accomphsh 
that, users should get an added value from the interactions, and well defined objectives 
should be attained regarding learning, entertainment and decision-making. 



Bailey, R. W., Allan, R. W„ & Raiello, P, (1992). Usability Testing vs. Heuristic 
Evaluation: a head-to-head comparison. Proceedings of the Human Factors 
Society 36th Annual Meeting-1992, 409-413. 

Barnard, P. (1991). The Contribution of Applied Cognitive Psychology to the Study of 
Human-Computer Interaction. In B. Shackel & S. Richardson (Eds.), Human 
Factors for Informatics Usability (pp. 151-182), Cambridge: Cambridge 
University Press. 

Caldeira, P, Z., Dias, I„ Otero, N., & Silva, J. P. (1995). RECMUVI - Recycling in a 
Virtual World. Demonstration presented at ED~MEDIA 95 - World Conference 
on Educational Multimedia and Hypermedia. 

Hayward, T. (1993). Adventures in virtual reality. Carmel: Que. 

Hix, D., & Hartson, H. R, (1993). Developing User Interfaces: Ensuring usability 
through product & process. New York: John Wiley & Sons. 

Jeffries, R., Miller, J. R., Wharton, C., & Uyeda, K. M. (1991). User Interface Evaluation 
in the Real World: A comparison of four techniques. Proceedings of the CHr9I 
Conference on Human Factors in Computing Systems, 119-124. 

Landauer, T. K. (1988), Relations Between Cognitive Psychology and Computer System 
Design. In J. Preece & L. Keller (Eds.) Human-Computer Interaction (pp. 141- 
146), Cambridge: Prentice Hall. 

Laurel, B. (1993). Computers as Theatre. New-York: Addison-Wesley Publishing Co. 

Nielsen, J. (1992). Finding Usability Problems Trough Heuristic Evaluation. Proceedings 
of the CHr92 Conference on Human Factors in Computing Systems-1992, 373- 

Pimentel, K., & Teixeira, K. (1993). Virtual Reality: Through the new looking glass. 
New-York: Windcrest McGraw Hill. 

Ravden, S., & Johnson, G. (1989). Evaluating Usability of Human-Computer Interfaces: 
A practical method. New York: John Wiley & Sons. 

Slater, M., & Usoh, M. (1993). Representations Systems, Perceptual Position, and 
Presence in Immersive Virtual Environments. Presence, 2(3), 221-233. 

Varela, F. (1992). L'Incription Corporelle de VEsprit. Sciences cognitives et experience 
humaine. Paris: Editions du Seuil. 

Varela, F, (1994). Conhecer. As ciSncias cognitivas tendencias e perspectivas. Lisbon: 
Instituto Piaget. 

Varela, F. (1995). Sobre a Competencia Etica. Lisbon: Edi^oes 70. 

Virzi, R. A. (1990), Streamlining the Design Process: Running fewer subjects. 

Proceedings of the Human Factors Society 34th Annual Meeting-1990, 291-294. 

Virzi, R. A., Sorce, J. F., & Herbert, L. B. (1993). A Comparison of Three Usabihty 
Evaluation Methods: Heuristic, think-aloud, and performance testing. 

Proceedings of the Human Factors and Ergonomics Society 37th Annual 
Meeting-1993, 309-313. 


A Multimedia Archaeological Museum 

Claus Diefienhacher, Ernst Rank 

Numerische Methoden und Informationsverarbeitung 

Fakultdt Bauwesen 
Universitdt Dortmund, D-44221 Dortmund 


This paper will present a project, which was first initated in 1994 as a graduate 
students seminar and is now being continued as a research project in a cooperation of 
computer scientists, architects and archaeologists. An ancient roman city (Colonia 
Ulpia Traiana near todays Xanten in Germany) has been reconstructed, using various 
levels of abstraction. On the coarsest level, a 3D-model of the whole city was 
established, distinguishing between different historical periods of the city. The second 
level picks places of special interest (temples, the forum, the amphitheater, the town- 
baths etc.) and reconstructs these buildings or groups of buildings. On the finest level 
important interior parts or functional details like the Hypocaustae in the town-baths 
are modelled. All econstructions are oriented as close as possible to results from 
excavations or other available documents. 

All levels of the 3D-model have been visualized using photorealistic images and 
sequences of video animations. The 3D model is integrated into a multimedia 
environment, augmenting the visualization elements with plans of the city and 
individual buildings and with text documents. It is intended, that parts of the outlined 
system will be available at the site of the ancient city, where today a large public 
archaeological park is located. 


Museums are in many aspects a very suitable field of application for multimedia 
techniques. This contibution will focus on public information systems, which can be 
used as multimedia guides on CD-ROM or in a communication network as well as in 
the museum itself. There are already a couple of examples available in internet, e.g. a 
guide through the National Air and Space Museum in Washington D.C. (NASM 
1995) with many image and text documents or through the Museum of History of 
Science, Florence (Bemi and Guidi 1995), Applications in the museum itself range 
from orientation guides (Where can I find which department, which exhibit 70 to 
multimedia informations at individual objects. Significant advantages over standard 
information charts and audio tapes arise especially, if visualization of dynamic 
processes (e.g. function or manufacturing of technical exhibits) support the visitors 
understanding. Moreover, an interested user can be supplied with nearly arbitrarily 
much information on details or on background. 

Using the opportunity to individually select information, a visitor can thus play a 
much more active role in a multimedia supported exhibition than in classical 
concepts, where he is usually restricted to pure passive examination. The most 
attractive field of application of multimedia systems in museums may yet be all those 
areas, where objects being otherwise invisible can be shown. 


An example for this application, a prototype of a multimedia archaeological museum, 
is presented in this paper. 

In a graduate students seminar with the title ’Seeing the invisible reconstruction of an 
antique city by CAD" 16 students of the Department of Architecture and Civil 
Engineering at the University 

of Dortmund created, in cooperation with computer scientists and archaeologists, a 
huge 3D model of the ancient roman city 'Colonia Ulpia Traiana’ near todays Xanten 
in Germany. A prototype for this seminar was Koobs pioneering reconstruction of the 
Cathedral of Cluny 

(Cramer and Koob 1993). The primary goal of the seminar was to teach students in 
3D-CAD modelling and visualization. They were challenged by the required 
modelling of very complex geometric objects and were, on the other hand, given 
enough freedom to use and to develop individual knowledge (and intuition) about 
classical culture and buildings. In examining processes of construction, students had 
to study thoroughly principles and techniques of anchient master builders. Thus, the 
use of CAD contributed much more to the architectural education of the students than 
only to the development of technical skills. 

Our CAD model uses different levels of abstraction. On the coarsest level, a model of 
the whole city was established, giving the frame for the second level, where places of 
special interest (several temples, the amphitheater, the town-baths etc.) were 
elaborated. On the third level, important interior parts or functional details were 
modelled. All reconstructions were orientated as close as possible to results of 
excavations or to other available historical documents. 

The integration of the CAD-model in a multimedia document and an example 
session of a 'tour' through the document will be outlined in the next sections. 

System design and implementation 

The implemented 'multimedia museum’ (Rank 1995) is based on the 3D CAD-model, 
having been created using a commercially available CAAD-system (Nemetschek 
1994) with powerful solid modelling capabilities. 

Within this system, a large number of photorealistic images, using standard shading 
or ray-tracing techniques, was created and stored as gif- or jpeg-files. It was also 
possible to create video-sequences of selected parts of the model and to store these 
clips as mpeg-files. Images and videos 

are integrated in a multimedia system, using html-version2-standard and public 
domain tools for browsing and visualisation. The overall structure of the document is, 
according to the structure of the underlying CAD-model hierachical, with some 
additional horizontal links between particular parts of the document. 

We consider it especially important to design the structure of a multimedia document 
ver>' carefully, so that a user of the document is able to understand this structure 
quickly and intuitively. For our application it is very natural to use images of the 
model as basic ordering elements, applying the concept of 'clickable maps' to select 

information on the next level of hierarchy. Considering for example the root -level of 
the document, a birds eye view of the city is presented, and the user can select 
information on different buildings, clicking on these objects in the images. 

Moreover, a unique box with available documents (floor plans, index, videos) is 
available on every layer of the document, preventing the user from ’getting lost in the 
large amount of informations. 


Since its opening in 1977 more than 4.5 million people have visited the 
Archaeological Park Xanten. The ancient monument Colonia Ulpia Traiana is open to 
scientific investigation, but in particular is presented to the public. Methods and 
results of archaeological research are displayed at the spot where history took place 
(from Rieche 1994). The design of the multimedia guide, being aligned to the general 
goals of the APX, is intended to serve as well as a first introduction to the Park and 
also as a system to provide background information. 

Both ways of use start entering the system in a page shown in Figure 1, being 
composed of first textual information and a perspective view of the city. 

l»>K«»>Mhw«< •«rh >h1m >r«wv i 


->(+ftW|''»•*' it I.lW >1 5I. J 

Figure 1 Clickable image of the whole Roman city 
Selecting e.g. the amphitheater or the town-baths in the clickable map leads to 
branches with images and additional text information partially shown in Figures 2 
and 3, 


Figure 2 Amphitheater with image of construction details 


Figure 3 The town-baths with technical informations about the Hypocaustae 

Discussing the branch assoziated to the capitoline temple it will be shown that a 
multimedia system based on a 3D CAD-model can easily provide a lot of additional 
information, being only implicitly related to the spacial model. The entrance point 
into this object is again a photorealistic image of the temple providing options to 
select more images in details (Figure 4). Also available is a floor plan (Figure 5) 
together with a scanned picture of the (very small) part of the fully excavated site 
(Figure 6), This part is marked in the floor plan (shaded area) and thus gives the user 
a very immediate idea of the size of the whole site of this temple. Other details are 
shown in Figure 7, where, derived from the 3D-model, construction parts of the front 
side and of the columns are plotted and explained in more detail. This image is used 


again as a clickable map to obtain perspective views of constitutional elements like a 
part of the roof construction with capitals, shown in Figure 8 as CAD-model and as a 
1:1 reconstruction. 

Mit -ww dw -Mtonh* dHtwy 

viw( « >(*•«* wnm 

l>er CspltDlsternpeJ 

Figure 5 The capitoline temple: Floor plan with shaded excavated area 

Conclusions and future work 

Based on a huge 3D CAD-model a prototype for a multimedia system to be used as a 
guide through an archaeological site has been outlined. The system offers opportunities 
to retrieve information, which go far beyond the possibilities of classical media. Up to 


now, the system is composed of text, high resolution photorealistic images, plans, 
photographs and sketches. 

Figure 8 A detail of the CAD-model (left) and Photo of a 1:1 reconstruction (right) 

With the availability of sufficiently powerful hardware it will be possible to allow a user 
an online animation of the full 3D-model, giving him the opportunity of a Virtual walk' 
through the ancient city. Additionally, parts of the model have already been 
implemented on a '3D workbench' (Krueger and Froelich, 1994), presenting the model 
with the possibility of online interaction on a 'table' in stereo view. The most attractive 
perspective of the multimedia system is yet a presentation at the site of the ancient city 
itself, where today a large public archaeological park is located. The viscinity of'real' 
excavation and 'virtual' reconstruction will give the visitor a much better chance to 
understand the site and the roman culture as a whole than any classical way of 
presentation of archaeology. 

A cknowledgement: 

We would like to thank the APX, especially Dr.Precht and Dr.Zieling as well as the 
Nemetschek Programm GmbH for their support to this project. 


NASM 1995. National Air and Space Museum, 

Berni, M., Guidi, F. 1995. Home page of the Institute and Museum of History of 
Science, Florence ITALY, 

Rank, E. 1995. Ein multimediales archaeologisches Museum. 
http: //www. bau wesen/forschung/xanten/xanten. html. 


Cramer, H., Koob, M., 1993, Cluny - Architektur als Vision, Edition Braus, 

Nemetschek Programmsystem GmbH, 1994, Allplan Version JO. Muenchen, 

Rieche, A, 1994. Guide Through the Archaeolocical ParkXanten. Rheinland 
Verlag GmbH, Kdln. 

Krueger, W„ Froelich, B., 1994. The Responsive Workbench. IEEE Computer 
Graphics and Applications. 


Subjective Realism in a Simulated Squash Game 

Heiko Hecht 

Zentrumfur interdisziplindre Forschung 
Universitdt Bielefeld 
D-33619 Bielefeld 


Subjective realism is thought to play an important role in computer simulation. The 
purpose of several experiments was to assess to what degree subjective realism can be 
captured by well defined, task-relevant information such as time delays or resolution, and 
to what degree it has to be considered a psychological concept in its own right. We 
investigated immersive variables that were manipulated above and beyond the visual 
information necessary to carry out the task. Three experiments were conducted to 
compare performance measures with ratings of subjective realism within an engaging 
and demanding task. Observers were confronted with a simulated squash court. 
Accuracy of racket placement and timing were assessed as a function of gratuitous cues 
for compellingness (such as depth cueing, cast shadows) and as a function of objective 
disturbances in the ball’s trajectory. Subjective realism proved to be a psychological 
variable in its own right that cannot be predicted by task performance. Pragmatic, task- 
specific criteria were found to be particularly important determinants of subjective 


Subjective realism has been acknowledged to be a major aspect in the evaluation of 
visual simulations. The goal of this paper is to explore status of subjective realism form 
a psychological point of view. 1 demonstrated that subjective realism cannot be reduced 
to performance measures, that stimulus properties influencing compellingness should not 
be studied in isolation, and that subjective realism bears a complex, non-intuitive 
relationship with stimulus parameters of the simulation. 

A taxonomy for testing reality or realism has been proposed by Kruse and Stadler (1990), 
who have attempted to establish a catalog of criteria that contribute to phenomenal 
realism. They suggest three broad categories of criteria for reality: (1) Syntactic criteria, 
such as contrast, starkness of object contours, richness of texture, three-dimensionality, 
intermodal congruence, motion, and spatial location. All primary and secondaiy depth 
cues would fall into this category. (2) Semantic criteria: Significance, expressiveness, 
and context congruity are addressed here. For example, a perfectly weU-rendered object 
may not look compelling because it is placed in an inappropriate context. (3) Pragmatic 
criteria are the possibility of the object to become part of an action, the degree to which 
the event can be anticipated, and inter-subjective agreement on the object. An artificial 
situation, even if it does not attempt to replicate a maximal number of aspects associated 
with a real-world experience may be very compelling if it becomes a meaningful part of 
an action. To be able to differentiate between and separately manipulate syntactic, 
semantic, and pragmatic aspects of subjective realism, a squash game was simulated on a 


Experimental Setup 

A simulation of a squash game was chosen and reduced to the following action sequence 
during the experimental trials: A ball was simulated to fly toward the far wall of the court 
(as if hit by an invisible opponent), rebound, and then move toward the observer. By 
manipulating the mouse, the observer could move a simulated racket and "hit" the 
oncoming ball. Independent variables were shadow, amount of detail, texture, fog, and 
sound effects. Pragmatic variables were success rate as manipulated by racket size, 
random path deflections and delayed mouse action. On all trials, positioning of the 
racket and timing of the action were used as performance measures, a compellingness 
rating was added to directly assess subjective realism. 

The stimuli were generated on a Silicon Graphics Indigo 2 XZ workstation with a 
resolution of 1280 xl024 pixels and a refresh rate of 72 Hz (non-interlaced). The 
animation update rate was 18 frames/second. Observers sat with their eye-point 27 cm 
away from the screen such that their line-of-sight was centered on the 20" display screen 
The monitor was 38 cm wide and 29 cm high resulting in a visual field of 70° by 56°. 
The stimulus displays consisted of a 3-D rendition of a squash court. A ball would fly 
toward the back wall as if hit by an invisible opponent. After it rebounded it moved 
toward the observer, whose task was to intercept it with her/his racket. The racket could 
only be moved on the invisible front wall but not in depth (2-D movement) by using the 
computer mouse. The viewpoint of the observer was fixed thus simulating a stationary 
observer with a very long arm. A thin line was drawn from the edge of the racket to the 
floor to provide additional visual support for the fact that the racket never moved into the 
court but remained on the front wall. 

Various Experimental Manipulations 

First we probed into some parameters that are critical for a hitting task as well as into 
some parameters that offer more gratuitous embellishment, such as sound and lighting. 
To provide a standard to anchor judgments of subjective realism, some clearly unrealistic 
changes were introduced in some trials: The ball could deviate from its motion path in 
mid air, as if deflected by an unseen force. A large number of syntactic and semantic 
criteria were varied, while the task remained always identical. Surprisingly, some 
manipulations that were thought to have a strong influence on performance (e. g. the cast 
shadow of the ball) only affected judged realism, and vice versa. 

Also, the consequences of the action were manipulated. Not the whole racket, but only a 
cross hair indicating its center was visible. At the same time the actual size of the 
invisible racket was varied. Thus, the number of successful hits could be changed 
independently from parameters of the simulation. The success of the action, above and 
beyond all other manipulations, had a strong positive effect on judged realism. 

Both sound and the shadow cast by the ball produced an informative asymmetry between 
performance and realism. Sound effects were not exploited to fine-tune action, not even 
to reduce timing errors. However, trials with sound effects were judged to be more 
reahstic than those without. Shadow had the opposite effect: It increased the racket 
positioning error but had no effect on judged realism. The fact that shadow distracted in 
the proper positioning of the racket without affecting hit rates or timing of the ball 
suggests that shadow was used to fine tune the racket position, albeit with negative 


effects. Shadows are known to exert a strong influence on the perceived depths of 
objects and may even override other depth cues and create powerful illusions (Kersten, 
Tarr, & BUlthoff, 1995). Thus, the negative shadow effect is in accordance with the 

Observers did not receive explicit feedback, but they could use the effect of their action 
(hit versus miss) to judge the positioning accuracy of their action. Tuning errors, on the 
other hand, could not be perceived at all. We considered making a successful hit and 
rebound dependent and correct positioning and timing, but pilot subjects, including the 
experimenter, found it very frustrating. Also, it should be avoided that the ball flew 
through the racket in order not to create another variable affecting realism. The fact that 
variability in performance was carried by positioning was a function of focusing the 
observer’s attention on hitting the ball, which was much harder than in Experiment 1. 
This effect was desirable since timing errors might have been hard to interpret given the 
rather slow update rate of 13 Hz. 

Changing the light transmittance inside the squash court (fog) produced no significant 
effects, neither did it affect performance nor judged realism. This indicates that 
observers could do without the information contained in the initial phase of the balls 
trajectory after leaving the rear wall. They might have extrapolated the ball's motion 
during the period where it practically merged with the dark background. It seems 
plausible, that observers considered fog as a natural effect while they treated pervasive 
effects like sound as necessary for a realistic rendition. 

In sum, the decoupling of performance and judged realism in the case of sound and 
shadow shows that the former interact in non-trivial ways. It is thus unhkely, that any 
given performance measure is able to capture reahsm. 

In another experiment, the pragmatic factor was more thoroughly explored by making it 
independent of parameters of the simulation. This was achieved by making the racket 
invisible except for a cross-hair indicating its center. The size of the invisible racket was 
changed unpredictably from trial to trial. That is, a well executed placement of the racket 
could lead to failure to hit the ball (as well as to success); and a sloppily executed racket 
placement could lead to a success (as well as to a miss). Success rate was thus decoupled 
from quality of the animation, precision of the action, and other possible confounds. In 
all other respects the experimental setup remained unchanged with the exception that the 
shadow manipulation was dropped, mouse action was always instantaneous, and 
visibility was changed by introducing fog to the display on some trials. 

Manipulating the success rate of the motor response had the hypothesized effects (see 
Figure 1), The pragmatic factor of success exerted a strong influence on judged reahsm. 
Trials in which balls rebounded from the invisible racket looked more natural and 
realistic to observers compared to trials where the ball went past the racket. This effect 
cannot be attributed to any syntactic or context effects of the simulation but solely on the 
outcome of the action. A roughly linear trend between judged realism and racket size 
indicates that, ceteris paribus, reahsm is a more or less linear function of success. 



judged Realism 






Figure 1. Mean judged realism as a function of racket size. Note that the racket was 
invisible and displays were identical in all cases. The rackets center was marked by a 
cross hair and could be 10 %, 200 %, or 600 % of its standard size. Timing and 
positioning accuracy were identical in these conditions. Thus, identical actions based 
on identical visual stimulation yield different realism as a function of their outcome. 

0.1 1 6 

Sze of Invi^ble Racket 


The results support the notion that judged realism can neither be confined to pure 
performance measures nor to display parameters of the simulation. In addition to these 
factors, the observer’s action context and the outcome of the event have been shown to be 
critical for the degree of subjective realism in the simulated scene. In hindsight, these 
results justify the unified measure of subjective realism that was employed. 

When confronted with the task to compare two pictures or two computer simulations, it 
is hard to imagine a difference between them that does not also reflect a difference in 
realism. For instance, if an observer is confronted with two displays that vary in only 
one aspect, let’s say stereo, and if he/she is then asked which display was nicer or more 
realistic, the display with stereo is usually preferred (e. g. Hendrix & Barfield, 1996). 
However, by itself, this judgment is rather worthless because a different reference object 
or reference class can completely change the judgment. Moreover, if stimuli vary on one 
dimension only then the observer is maximally constrained and will therefore accept 
almost any label for the difference. One could probably replace realistic with words like 
cozy, strong, or heavy and still get the same results. Thus, to interpret the results as a 
difference in judged realism an additional argument has to be made to connect the rating 
to a content. 

As Gestalt psychologists have shown, this depends on the hierarchical nesting of the 
stimuli in their context. From our results it is clear, that factors such as the success of the 
action are part of this context. It is therefore questionable if much is learned if only one 
or two parameters are varied simultaneously. The concept of realism appears to be too 
complex to be fully grasped by studying aspects of it in isolation. Thus, in the case of 
subjective realism, complex experimental situations may well be a virtue rather than a 


vice. At the same time, results obtained with small parameter sets have to be interpreted 
very carefully. The rating scales that were used in the present experiments did not rely 
on a small set of parameters, but they reflect considerable variability in syntactic, 
semantic, and pragmatic aspects of the action and its simulated environment. The fact 
that they produced interpretable results justifies the choice of a rating scale that required 
a single rating after each trial. Moreover, the simultaneous significant effects of several 
independent variables (sound, success, mouse delay, and path deflection) on judged 
realism indicates that observers do not just focus on one salient aspect of the simulation, 
but that they do integrate - knowingly or not - many aspects of the display into one 
meaningful judgment. The lack of significant interactions between the different 
independent variables revealed that the resolution of the compound judgment was limited 
to main effects. 

Possible alternatives to the realism ratings could have been (1) cue trade-off studies and 
(2) successive addition models. Trade-off studies place two cues in conflict. The cue 
that wins is the stronger one. Underlying this model is a static view of cue strength that 
is very situation specific. As a function of this model typical empirical results are that, 
depending on situation and observer, different cues win. However, these methods could 
not be used here for two reasons. First, too httle is known about variables or cues that 
contribute to subjective realism. Second, compellingness or realism are integral concepts 
that have to be assessed in a complex (non-additive) fashion. This was achieved in the 
present study, since no explicit understanding of the concept of realism was necessary for 
the task. The required rating scale of compellingness or realism was anchored in the first 
trial blocks relative to the variability in the stimuli 

In sum, the importance of pragmatic criteria for subjective realism has been 
demonstrated. Time delay of consequences caused by the motor action (mouse 
manipulation) had a strong negative effect on judged realism. Likewise did outcome of 
the action (hit or miss) influence judged realism. This entails that it is not very useful to 
define subjective realism, or presence for that matter, as a dimension that is independent 
of interactivity or that does not play a role in standard displays, as was suggested by 
Zeltzer (1992). 


The findings suggest that the taxonomy devised by Kruse & Stadler (1990) is very useful 
when evaluating subjective realism of visual simulations. Moreover, the results are in 
agreement with hypotheses by Loomis (1992) that distal attribution of events is crucial in 
subjective realism. The results also are compatible with and extend findings by Welch et 
al. (1996). The particular implications for our thinking about subjective realism can be 
summarized as follows: 

1. Subjective realism cannot be reduced to the fidelity of the 
graphical rendition 

2. Subjective realism cannot be reduced to performance measures 

3. Violations of physical laws often do not affect subjective realism 


4. Subjective realism is context specific 

5. Subjective realism is highly dependent on the action 

the observer is involved in as well as the consequences of this action 


Hendrix, C., & Barfield, W. (1996). Presence within virtual environments as a function 
of visual display parameters. Presence: Teleoperators and Virtual 
Environments, 5,274-289. 

Kersten, D., Tarr, M., & Bulthoff, H. H. (1995, March). Object recognition depends on 
illumination. Investigative Ophthalmology & Visual Science, 36, S474. 

Kruse, P., & Stadler, M. (1990). Wirklichkeit E: Psychologische Wirklichkeitskriterien. 
In H. J. Sandkiihler (Ed.), Europdische Enzyklopddie zu Philosophie und 
Wissenschaften.. Hamburg: Felix Meiner. 

Loomis, J. M. (1992). Distal attribution and presence. Presence: Teleoperators and 
Virtual Environments, 1, 113-119. 

Welch, R. B„ Blackmon, T. T., Liu, A., Mellers, B. A., & Stark, L. W, (1996). The 

effects of pictorial realism, delay of visual feedback, and observer interactivity on 
the subjective sense of presence. Presence: Teleoperators and Virtual 
Environments . 5, 263-273. 

Zeltzer, D. (1992). Autonomy, interaction, and presence. Presence: Teleoperators and 
Virtual Environments, 1, 127-132. 


Virtual Reality as an Aid to Document and Inventorize 

Historical Buildings 

Martin Trautwein 
Labor fur Bauinformatik 
Fachhochschule Lippe 
Detmold, Germany 


Working with models and simulations has a long tradition in the building industry. The 
models are used for visualising the drawings of the architect and to discuss possible 
problems during construction. 

Due to the introduction of CAD-Systems visualisation has been expanded potentially. 

The geometry of an architectural project is digitised to create a three dimensional model 
within the computer. Virtual Reality and it’s possibility to compute 3D-data in real time, 
interactive control mechanisms and free movability inside Cyberspace opens new per¬ 
spectives in architectural modelling. 

By using the techniques of virtual reality it is possible to document different stages of a 
buildings history, enabling the viewer to interactively walk through time. This could be 
used for research and reconstruction of historic buildings, with the possibility to show 
different stages of completion and changes through time. 

"House Moven" in the Westfalian Outdoor Museum at Detmold has been modelled in 
this way and is to show the potential of virtual reality. It is possible to walk through the 
three dimensional model and to pick out certain aspects of construction to analyse more 

Furthermore, changes throughout different historic periods and building stages since it’s 
construction in the 16th century are made visible. The analysis of the user interface be¬ 
tween man and machine which is to be used by academically trained and by people with 
little or no knowledge of computer modelling is of great interest. 


The task of combining two veiy different disciplines has a particular attraction for an 
engineer. On the one hand there is the preservation and documentation of historic 
monuments - a very old discipline with roots in Historicism and in Enlightenment (Ger- 
mann 1980). Virmal Reality or Cyberspace on the other hand are rather new terms. Their 
exact meaning has not yet been fixed. Cyberspace is a word that was first used in Science 
Fiction Literature (Gibson 1987). The objective of combining those disciplines is to look 
for a man computer interface that facilitates research of building historians by allowing a 
simple interactive and immersive navigation through historic buildings. 


Documentation of historic monuments 

The way generations of historians documented their work is measuring the building with 
the conventional measuring methods of the surveyors (Cramer 1993) and making draw¬ 
ings of them - usually drawn to a scale of 1: 20, 1:50 or 1:100 and in four different 
grades of precision (GebeBler 1986), Results of research were marked with different col¬ 

Figure 1: Manually Drawn Plan of House Moven 

New techniques of measuring like for instance photogrammetry were naturally adopted. 
As they supplied three dimensional data in x-y-z co-ordinates it was obvious to use them 
for building a three dimensional computer model. 



Figure 2: Detail of Manually Drawn Plan 

There are excellent examples to be visited in the Internet as for instance the Amiens Ca¬ 
thedral Media Centre For Art History of the Columbia University in New York (Colum¬ 
bia 1997). Another excellent example of the possibilities is the reconstruction of the 
Archaeological Park - Xanten by the "Instimt fur Numerische Methoden und Morma- 
tionsverarbeitung" - University of Dortmund (Rank 1995) 

Virtual Reality 

The term "Virtual Reality" often seems to be woolly and vague. Even a dictionary is not 
very helpful, as virtual there is defined as "almost" , "nearly" or "seeming". So I will try 
to give an example. 

Man's left and right eye have different angles of view so they produce two different 
"pictures". Our brain is able to blend two 2-dimensional pictures into one 3-dimensional 
image of our surroundings. What we can see is a 3-dimensional image of the space - 
there brain forms a "virtual space". This virtual space is shaped by our natural experi¬ 
ence. Our brain adds the information of our sense of touch, taste and hearing extending 
our image of the space. Since we are bom we are living in a virtual space. 

Our image of space seems natural to us because we are living in it from birth. All our 
experience is based on the information this image of space provides. The painting of 
Ren6 Magritte, the surrealistic painter, entitled "c’ est ne pas une pipe" shows us the di¬ 
lemma of our mind: What we see on the painting is a pipe but the title tells us it’s only 
the image of a item of practical use as the artist saw it. And - to bring this idea to an end 
- the pipe I am smoking in the evening is also a virtual image of a pipe that is created by 
my sense of sight, taste and touch. 

The term "Virtual Space" or "Virtual Reality" changes it’s meaning when used in context 
with computers. It describes a new way to create a man - computer interface. In this 
context interface means all units that let man manipulate the computer. 


The object of research on virtual reality is to improve man - computer interface even 
more. Many problems and tasks we have to solve with computers are located in the 3- 
dimensional space. An architect for example wants to show the planned building to his 
client, so that he can judge proportions, light and materials. 

Virtual Reality with computers means the simultaneous existence of 

3D-input and -output, 
real time behaviour, 
autonomous objects and 

More detailed: 

3D-input and -output means that the representation as well as the sensors of the 
VR system are designed for three dimensions. Suitable equipment is for in¬ 
stance a data glove, space mouse, head mounted displays and tracker, stereo 

Real time behaviour means that the VR-System reacts on user’s input or move¬ 
ment with a delay less than 10 to 50 milliseconds. 

Autonomous objects are characterised by owning properties that are independent 
of the current reaction of the user and by the possibility to interact with them. 
Immersion is the effect that the user is mentally completely immersed in the vir¬ 
tual world. 

The Task 

A historic building is rarely found in the state it was built centuries ago. The buildings 
we consider have changed their use several times. They have partly been broken down 
and extended in other parts. With changing fashion they got new facades - up to seven 
times in six centuries. Therefore it is extremely difficult to decide which historic state of 
the building is to be reconstructed. The earlier and later states of this building can only 
be documented. There seems to be a need for a system that supports the decision by visu¬ 
alising the different states of the building in an easily understood way. We are looking 
for an economic solution for this problem as a compromise between costs and perform¬ 
ance. We suppose that Virtual Reality can serve as a tool do visualise different stages of a 
building and can be used even by inexperienced computer users. The aims of the project 

• to find a way to document and inventarize the different states of a historical 
building for research in a 3-dimensional way, 

• to make visible the different states of a building to visitors of the museum 
that are not experienced in using computers, 

• to test different man - computer interfaces for their suitability 
The Tools 

As we had to regard the economic efficiency we chose a PC-based System as a worksta¬ 

• Dual processor Pentium 200 MHz, 

• 256 MB memory, 

• 4 GB harddisk. 


• Diamond Fire GL 1000 SMB graphic card, Diamond Monster 3Dfx graphics 

• Windows NT 4.0 operating system. 

Space mouse and stereo glasses will complete the '’virtual” equipment. 

Although all used to design with the CAD system MicroStation and Triforma we decided 
to do all construction work with 3D Studio MAX by Kinetix. Mainly designed as a pro¬ 
gram for visualisation and animation it has the capability to export the animated model 
to VRML 2.0, the common description language for Virtual Reality. Where necessary we 
improved and optimised the VR model with the help of the program Realimation by 

Users can immerse in the Model using the viewers of Realimation or Superscape even 
within the WWW. For this purpose one will to have to install the Cosmo Player plug-in 
for Netscape Navigator by Silicon Graphics or the World View plug-in for Netscape 
Navigator by Intervista. 

Figure 3: Virtual View of House Moven 

An Example 

House Moven is part of the "Paderbom Village” in the Westfalian Outdoor Museum at 
Detmold. hi 1980 it was pulled down at it’s original site in Bruchhausen and rebuilt 1982 
to 1986 in Detmold. It is a typical half timbered construction of this region. House 
Moven was built in 1651 by Peter Moven, a farmer. He used the foundations of a former 

The original ground plan of 1651 is to make the history easier to understand. 


Grouncf Floor UpK^oor 

Figure 4: Ground-plan 1651 

First changes were made in the 18th century: 

The original cellar was placed below the living - room. Owing to frequent flooding by the 
silver stream, it was given up about 1800. In place of the old one a new cellar was built 
below the back chamber. It could be accessed from the kitchen next to it. As the fireplace 
was exactly where the staircase to the cellar was to be built, the fireplace was moved to 
the opposite comer. 

Figure 5: Virtual View into the Cellar of the 18th Century 

In the mid-18th-century the value of the house was increased by replacing the old roof 
thatched with straw by a roof of sandstone slabs. Additional rafters were built in order to 
take the additional weight of the stone slabs. 


Figure 6: Roof with additional rafters 

Substantial changes took place in the third phase of rebuilding in the 19th century. 

The filling of the gable that originally was made of straw and loam was replaced by 
wooden boards. The inhabitants obviously wanted to reduce maintenance of the house. 
The living room was expanded by building a bedroom on the ground floor behind the 
kitchen. A half timbered wall was built to separate the kitchen from the hall. As smoke 
from the fireplace could no longer pass through the hall and the roof, a chimney had to 
be built. At the back gable an extension for a cowshed was added to the house. The place 
won in the old cowshed was used to install a staircase instead of the old ladder to the 
grain loft. 

All those changes were reconstructed out of the observations made when taking down 
the house. The other sources were old documents, letters and bills. They are documented 
in drawings and sketches true to scale but without dimensions. 

Working Method 

In a first step the existing building that was reconstructed in the state of 1860 was redes¬ 
igned using 3D Studio MAX. Dimensions were taken from the documentation of the 
museum and whenever there was a contradiction they were taken in place. 

The textures of wood, loam, stone and the other material were gathered from digitised 
photographs. They were rectified and colour corrected. The reachable speed of movement 
in the VR model strongly depends on the size of the texture files. So rather a big effort 
had to be made to process the textures. Finally they were stored in JPEG format and 
linked to the geometry to get a realistic impression. The figures in this paper were gener¬ 
ated synthetically by means of this 3D-model. 

Figure 7: Texture of a Board and Geometry of the Roof 

The reconstruction of the other stages required intensive discussion with the building 
historians because their findings when taking down the house needed interpretation. This 
interpretation was facilitated as designer and historian could refer to the VR model. 
Contradictions in the conventional plans could be resolved by consulting the 3D-model 

Elements that were lost during the centuries like for instance windows were replaced by 
elements of other houses which are typical for the period to be shown. 


We succeeded in building VR models of the most important stages of house Moven. One 
can walk through the house, look at details, climb up to the upper floor, leave the house 
and walk around, using mouse and screen. The model was transferred to a SGI equipped 
with space mouse and shutter glasses and provided a rather realistic impression. 

Links in the VR model lead to explaining text about the history of the house and it’s in¬ 
habitants, They also lead to animations of important details. As other links also lead back 
to the VR model, all the information is connected to a Hypermedia document. 


The example of house Moven proves that it is possible to reproduce an old half timbered 
house and his different stages through the time as a VR model. Even if the workflow for 
producing the model should be improved for further projects, the described method of 
working is a suitable way to generate a VR model out of conventional documents. 

With about 70 000 polygons in the model the PC based workstation has reached it’s limit 
so it is not yet possible to reach a grade of precision that allows to consider the deforma¬ 
tions of a building. Whereas a PC-based workstation is sufficient for designing the VR 
model and doing historical research, one should consider to use faster computers for 


Columbia University (1997) Amiens Cathedral 


Cramer, J. (1993) Handbuch der Bauaufnahme, DVA 

GebeBler, A. (1986) Empfehlungen fUr Bauaufnahmen, Kohlhammer Verlag. 

Germann, G. (1987) EinfUhrung in die Geschichte der Architekturtheorie, Wissen- 
schaftliche Buchgesellschaft 

Gibson, W. (1995) Neuromancer. Ace books 

Rank (1995) Archaeological Park - Xanten, http://www.bauwesen.uni- 


Designing for Telecommunication on the Internet 

Expanding Human Factors to Community Research 

Marita Franzke, Anne McClard, Carrie Rudman, and Pat Somers 

U S WEST Advanced Technologies 
4001 Discovery Dr. 
Boulder, Colorado USA 


We describe a series of related research projects investigating the adoption of Internet 
technologies in a small town. In one project we used an ethnographic framework to 
understand communication patterns in the town and how knowledge about and use of the 
Internet was disseminated. The ethnographic project set the stage for two other projects 
we did in the same town. Here, we reflect on the advantages of conducting long-term 
qualitative research in the area of communications technologies, how the ethnographic 
project informed and influenced the shape and findings from the two other projects. 


The Winona Projects were undertaken by U S WEST Advanced Technologies in an effort 
to understand how the technology of the Internet disseminates into a community, to 
understand what factors and what technologies facilitate and impede Internet adoption by 
individuals, families, schools, businesses, governments, and other institutions. We also 
hoped to cultivate a well-understood testbed for numerous products and services that we 
imagined would be delivered by way of the Internet in the coming years. The projects 
spanned a period of 18 months during 1995 and 1996. They encompassed several 
different research and development efforts, three of which are discussed in this paper: 1) 
Networked Community Ethnographic Project; 2) Virtual School Project; and 3) 
Communications Rich Web Site Project. 

The three projects can be seen on a continuum of complexity and level of detail. The 
Networked Community Ethnographic Project was an ethnographic study of technology 
diffusion in the town of Winona, providing insights into the complex organizational and 
individual factors contributing to the genesis of an on-line community. The Virtual 
School Project focused on the use of the Internet by a school and its families. The 
Communications Rich Web Site project introduced Internet collaboration technology into 
a maturing Web environment for use by community leaders in town meetings. 

We discuss both the findings from the research we did in Winona and the importance of 
the ethnographic component of the work. The ethnographic project, which was first, 
informed and influenced focus, data analysis, and evaluation of the other projects we 
carried out in Winona. The data collected in the Virtual School and Communications 
Rich Web Site studies revealed their full meaning only in the context of our involvement 
with the community as a whole. 

Identification of early adopter individuals using traditional marketing criteria, such as 
demographic and consumer characteristics in an organization did not adequately predict 
adoption of the technology. While individual early adopters played key roles in Winona, 
it was critical to understand how Winona’s various social institutions supported and 


influenced each other. An assumed value of the Internet was the access it gives to non¬ 
local information resources. We were surprised to find that Winonans were drawn to the 
Internet initially for its ability to facilitate communication of local information within the 
community, and secondarily for access to remote resources, commerce, and long-distance 
communication. We hold that the study of conamunication technologies, like the Internet, 
necessitates an approach that takes social complexity into account. Using an 
ethnographic, whole-community, approach provided us with a framework for 
understanding what and who was communicated with, and how adoption proceeded, and 
would be likely to proceed under similar circumstances. 

The Winona projects and their interrelations are discussed in greater detail below. We 
begin with a discussion of the history of the projects before launching into each project 

The Winona Projects 
Background: Why Winona? 

“Breaking Ground” is an apt metaphor for what happened in Winona, Minnesota, when, 
in 1994, U S WEST supported a grass-roots initiative to install a fiber-optic network 
throughout the town of Winona, Minnesota, linking eight of the town's most prominent 
social institutions. The initiative began when a prominent business leader offered a seed 
grant to put Winona on the Internet, the “information superhighway.” Years before, 
Winona had been passed over for a place on an Interstate highway, and they wanted to 
make sure that they did not miss another opportunity to enhance their economic base. 

Early in 1995, after the fiber installation had been completed by U S WEST, but before 
WWW/Intemet service was commercially and locally available in the town, we learned 
about Winona, We saw this as an opportunity to watch adoption of a new technology 
from the ground up. 

Winona is a small town with approximately 25,000 residents. It is located about two-and- 
a-half hours south of Minneapolis on the Mississippi River. We made our first foray into 
our new fieldsite two months prior to widespread availability of the Internet in Winona. 

In May of 1995, Luminet, a local Internet Service Provider (ISP), began offering service 
to residents, businesses and other organizations. Within one year, about ten percent of 
private households had gained Internet access\ and community organizations and 
businesses had embraced the WWW as a new publication and communication medium 
[ 1 ]. 

In the fall of 1995 an educational initiative called The Virtual SchooF^ (TVS) began, 
bringing 240 families from a private high school on-line [2]. TVS now links more than 
500 families whose children attend parochial schools in Winona. Two goals of the TVS 
initiative were to make communication between parents and school staff easier, and to 
free the educational enterprise from the constraints of the school building and extend it 
into homes and the broader community. 

^ This number only represents subscribers to Luminet; it does not include those who had Internet access 
through an on-line service, one of the universities, K-12 educational institutions, work, or public 
computing facilities. 


Breaking New Ground 

Although ethnographic techniques have been used in the field of human computer 
interaction (HCI) since the 1980s [3, 4, 5], only a few HCI researchers have attempted, or 
seen a need for conducting long-term ethnographic research of a more anthropological 
nature in complex communities comprised of multiple, interrelated, and interacting social 
organizations. The Networked Community Ethnographic Project was such an effort. 

The Networked Community Ethnographic Project set the stage for the other Winona 
Projects. Initially, we wanted to understand how information about the Internet is 
disseminated in the context of a town, what beliefs, if any, people in Winona held about 
the Internet, and how the use of the technology itself would proceed. We were interested 
in understanding this at both the organizational and individual levels. 

One of the concerns we had in doing our research in Winona was its representativeness. 
After all, it is a small rural town that is relatively homogenous demographically 
compared with many of the more urban areas served by our company. Although there 
were no equivalent smdies with which we could compare our findings, we kept a close 
watch on the various data collection activities surrounding the Blacksburg Electronic 
Village (BEV)[6], the HomeNet Project [7], and the Boulder VaUey Internet Project [8, 
9]. This helped us discern idiosyncratic findings from those findings that were more 
generalizeable. With few exceptions, our findings in Winona were consistent with what 
researchers in these other contexts were finding. 

At the outset, we felt that if we were later going to be able to understand any changes in 
communication patterns related to the Internet, we needed to first understand modes of 
communication used in the town without it. To this end, we documented public forms of 
communication of local information, including bulletin boards, flyers, newspapers, radio, 
television, and so forth. We documented the form of the communication, the content, the 
location, and the intended audience for the information (e.g,, students, women, tourists, 
etc.). This data collecting activity was fruitful, because it gave us a good overview of the 
town and the issues and events that were of public importance. The two local newspapers 
were invaluable. From them we learned critical political information, as well as received 
an introduction to a cast of characters with whom we would later form important 
relationships. We knew in advance where to tread lightly, and where we would likely be 
welcomed with open arms. Information we collected during this phase also gave us 
plenty of conversational content, so that when we were talking with Winonans we could 
understand tangential references and subtle nuances of what people were saying. 

During the early phase of the ethnographic study we wanted to learn how aware of the 
Internet people in Winona were: if they had heard of it, if they had used it, if they had any 
beliefs about it or technology in general. To collect data on this we informally 
“interviewed” people we met on the street, people we met in restaurants and bars, shop 
owners, and whatever Winonan crossed our path. These interviews were more like 
directed conversations than interviews. Typically they would begin as any conversation 
between strangers, with an exchange of pleasantries about the weather, a conmient on 
how nice the town was, leading into the fact that we were visiting, working for U S 
WEST on a project having to do with the Internet. From here, the conversation would 
diverge in many directions depending on the interests of the person we were talking to. 


When we first arrived in Winona in the Spring of 1995, awareness of the Internet was 
limited. The majority of the people we spoke with had heard of it through local media, 
but had never used it in any way. For most people the Internet, the World Wide Web, and 
Luminet, the local ISP, were synonymous. A small percentage of Winona households 
subscribed to on-line services (AOL, CompuServe). Most of those people who had used 
the Internet to send email or had used an on-line service had never seen the World Wide 
Web. There were already a couple of Web sites hosted in the town. The most extensive 
site was at St. Mary’s University of Minnesota (then St, Mary’s College). Winona State 
University did not have a web site, but they were using the email capacities of the 
Internet, and were heavily involved in the Luminet initiative. They also hosted a gopher 
server for the campus. In both universities, Internet access was Hmited primarily to 
faculty, but both schools had plans for expansion to their entire student bodies, as well as 
expansion of their networks into the residential halls. Only two larger businesses in 
Winona had Web sites, but these were hosted in large metropolitan areas outside of 
Winona. TTiere was also a private not-for-profit technology company, Vanguard 
Technology Group, that had just begun selling its services for Web hosting and page 
development, but initially their only client was Cotter High School (discussed in detail 

As the Luminet launch date loomed nearer, public awareness of the Internet grew. The 
newspapers ran articles almost daily on Luminet developments and activities surrounding 
it. The radio stations publicized these events as well. At this point, we concentrated our 
efforts on understanding what visions, if any, people in the town had for the technology, 
and whether they were acting on their visions in any way. As the technology matured, we 
tracked developments so that we might gain a better understanding of barriers and 
catalysts of adoption. We formally interviewed individuals in all of the major social 
institutions with direct drops into the network (hospital, city hall, Winona Technical 
College, Winona State University, Winona Middle School, Winona High School, Cotter 
High School) as well as individuals from social institutions without connections (e.g., the 
chamber of commerce, public library, and the senior center), proprietors of small 
businesses, and employees of medium, and larger businesses. 

Adoption of the Internet 

We learned that having direct connectivity to the fiber-optic network did not necessarily 
predispose organizational adoption of the technology, nor did technical knowledge. For 
example, in the months preceding the launch of Luminet, there was much talk about the 
benefits of the network to the social institutions with direct drops to the network. 
Everyone assumed that these institutions would be the first to adopt the technology and to 
use it in innovative ways. This did not turn out to be the case. For example, numerous 
articles were published in the local papers on how telemedicine would enhance the 
hospital’s ability to serve the health needs of rural patients; the hospital would have a 
direct link to the Mayo Clinic in neighboring Rochester, using the two-way video 
capabilities provided by Luminet. Other visions included on-line patient registration, and 
consultations. Today, these visions have still not been realized. When we interviewed 
various stakeholders at the hospital, we discovered these visions emanated from a single 
technology champion. The hospital administrator and systems support staff, however, 
were skeptical about the Web and the role it would play for them. Lack of security was a 
concern as well as the expense of equipment that would be required to make these visions 
a reality. 


On the other hand, at the Winona Senior Friendship Center (a city department without 
direct connectivity), the director and a senior volunteer had more modest visions of what 
the Internet could do for the aging population of Winona. They envisioned a scenario 
where senior citizens would serve as mentors to young people over the Internet. The 
director of the center did not need previous technology exposure to understand the great 
value that the Internet could bring to the seniors in her center , She collaborated with a 
volunteer at the center who had more technical experience. Together, they went to local 
businesses to raise money for their computer lab. Local businesses immediately perceived 
the value of the project and were willing to donate to the cause. Also, because the senior 
center fell under the auspices of the city government. City Hall’s support was a critical 
factor in making the Sr. Friendship Center computer lab a reality. They installed a small 
computer lab with dial-up Luminet service, and had made arrangements for members of 
the senior center to get free email through a state university. Seniors literally lined up 
outside the door before the center opened to claim their spot at a terminal, and 
membership at the center doubled in a short period. Visionaries and champions of the 
technologies emerged from unlikely places. 

What happened at the Sr. Friendship Center illustrates that adoption of the Internet not 
only occurs in technologically prepared organizations, but also in places where individual 
visions are met with the support of the community. We quickly understood that adoption 
of the technology was not going to occur in institutions that lacked individual champions 
of the technology, or institutions where political and bureaucratic barriers were difficult 
to overcome. 

Another institution that was critical to raising Internet awareness was the Winona Daily 
News, which launched an experimental “on-line newspaper” linking to a plethora of 
community and regional information. City Hall, and some of its associated organizations 
also played an important role in fostering Internet adoption. Although City Hall had 
expansive visions of how to use the Internet to facilitate community involvement in city 
government, it has not been able to bring its visions to light yet. The city manager, and a 
number of other city officials constantly championed technology efforts in the town. The 
visions and accomplishments of both the newspaper and City Hall are described in 
greater detail in our discussion of the Communications Rich Web Site below. 

It was the synergy of efforts in three main sectors-the public, businesses, and the 
educational sector-that turned Winona into an active on-line community during the 
groundbreaking phase. Businesses expected revenue and therefore invested not only in 
their own systems but provided financial help to public institutions. Schools 
disseminated access to the Internet into the families and thus created an audience and 
active participants. 

Schools as Sources of Technology Diffusion 

The Virtual School Project was a natural extension of the ethnographic study of the town, 
and indeed overlapped. The goal of the Virtual School Project was to understand Internet 
adoption at a more detailed level; to focus on a single organization~the school-and, also 
to understand some of the dynamics of Internet adoption at the family level. From the 
ethnographic research that preceded the Virtual School Project we knew that schools in 
Winona were key points of dissemination of information about the Internet, and also 
provided Winonans with their first hands-on experiences with the WWW. This was also 


apparent nationally; there had been an almost exponential growth of school entries in the 
on-line school registry Web66, and Clinton’s administration had issued a challenge to 
businesses to provide on-line access in every classroom by the year 2000, 

In Winona, educators in the public school system showed an early interest in the Internet, 
mainly because it promised access to unlimited teaching materials. The main concerns 
were that most of the public schools had outdated systems, and many teachers felt they 
lacked the preparation and skills required to use the Internet. This is not unlike the 
situation at many public schools, at that time [11]. Another Winona project, not 
described here, provided educators at the public middle school with the needed resources. 

During the summer of 1995, while we continued ethnographic data collection, we 
established a good working relationship with individuals at the Vanguard Technology 
Group, a non-profit business that was founded to support technology efforts at the 
Catholic Cotter High School. Cotter was ahead of most other schools, both inside and 
outside of Winona, with respect to the use of technology. Even at the beginning of the 
study, the school boasted broadcast cable connectivity into every classroom, a computer 
lab with 30-1- well-equipped Apple Macintoshes, a classroom equipped for collaborative 
work on Apple workstations, ethemet connectivity across campus into the dorms, and its 
own multimedia production lab, run by Vanguard. Clearly, “access” itself was a non¬ 
issue at this institution. 

Integrating Learning with Community Citizenship 

Vanguard’s vision for using the new town network was more far-reaching than what had 
been installed into the school. It was to extend Cotter’s educational reach from the 
campus into the students’ homes, and thus, into the community by providing all student 
families with in-home Internet connectivity. The educational goal was to contextualize 
education, to provide access to educational means to the whole community, and to teach 
the students in the rich context of that community instead of the bounds of the physical 
campus. During our first six months of work in Winona, this project was bom out as the 
‘Virtual SchooP’^ Project” (TVS), 

In more concrete terms, this translated into the goal of equipping all four-hundred 
families of Cotter smdents with Macintosh Performa’s'*'^ and modems, and all family 
members with individual email accounts. The families would also have access to the 
World Wide Web through Netscape'^^ browsers. The Vanguard staff had started 
developing the TVS Website that would serve as the “virtual campus.” These pages 
included a school calendar published weekly with daily curricular and extra-curricular 
activities, and the school menu, also updated weekly. Faculty and student directories 
were planned, with pictures, some personal information and email addresses. Teachers 
would be able to publish assignments and class materials on-line, and students could post 
their assignments even from their home computers. A forum would provide opportunity 
to discuss topics of interest to the school or community in an open fashion. Finally, there 
was an area for “Hot News” where school cancellations, or announcements of PTA 
meetings and the like could be posted. There was also an area showing exceptional 
student work, and a student art exhibition. In essence, TVS Web site was planned to be 
the communication switchboard for Cotter High School, see Figure 1. 

During the first year, participation in this trial was seen as voluntary; however, eventually 
in-home connectivity would become a requisite for becoming a Cotter student. Parent 


meetings to kick off the trial were full of controversy: Would families with older 
computers, and especially those with PC’s, be able to participate, and who would provide 
technical support for those? Was the lease for the Performa’s provided by the school 
really a good deal? How would parents protect their children from accessing unwanted 
material, and would this program eventually discriminate against children from less 
affluent households? Finally, was the school’s heritage-to provide good, basic Catholic 
education to all families who wanted it-be threatened by this new turn towards 
technology? However, after all was said and done, about 250 families (over 60 percent of 
all Cotter families) had signed up to be members of the trial in the first year alone! 

The research project 

It was through our work with Vanguard, during the Networked Community Ethnographic 
Project phase that we succeeded at recruiting fifty families (155 individuals) to 
participate in an in-depth study of Internet adoption and use. The Virtual School Project, 
unlike the ethnographic project, relied more heavily upon quantitative data collection 
techniques, but we used qualitative approaches as well. This project focused on the 
experience, attitudes, and use of the Internet of fifty families as they came on-line. 

Our plan was to roll out a questionnaire, tracking family members’ attitudes toward 
technology, their interests, self-reported Internet activities, use of other media, and 
attitudes toward their community several times during the trial, starting a few weeks after 
the official kick-off in the fail of 1995. These data were to be used as predictors of the 
on-line tracking data we planned to collect. We collected data on email activities, as well 
as URL accesses of each individual family member. Finally, we planned to do in-home 
interviews with the families participating in our study, to follow up on any interesting 
data patterns that we observed, and to learn more about in-home dynamics of Internet 


Figure 1: Cotter's Virtual School home page. Printed with permission of The Virtual SchooP'^ of Winona 

Connecting 250families 

To facilitate family participation in our study, and to collect observational data on 
parents’ first experiences, we helped out in eight parent training sessions held by 
Vanguard. In these sessions parents were introduced to the Internet, the use of email and 
the Webbrowser. They were guided through everything from taking their equipment out 
of the boxes to connecting the modem. Modem configurations were pre-loaded on all 
machines that were leased to the families. A few months into the trial, however, we still 
had not recruited many families to participate in our study. Also, email accounts had not 
been set up in a manner that allowed us to track individual use; we could only track 
family use. We finally resorted to calling the 250 families by phone to elicit 
participation, and found that a large majority of families had not yet established a reliable 
connection to the Internet. 

Technical problems abounded, and while family member