The History of the Interface

in Interactive Art

Söke Dinkla

At the moment the catchword "interactivity" is common talk. Most often it is

mentioned in connection with a revolution in television. Techno-prophets

anticipate more than 200 TV channels for the near future in each home. Thus,

viewers will not only be able to choose from an almost unlimited offer, they

will also be able to determine the course and outcome of individual programs

[1]. Proponents of these new opportunities are already praising

interactivity as a means to change the passive reception of the viewer into

an active one [2]. Thus, it seems as if Bertolt Brecht's Radio Theory [3],

which he developed in the late twenties, is now to become reality. Brecht

envisioned the transformation of broadcasting from a distribution machine

into a communication device that offers listeners the opportunity to help

create its content. And actually this development has been actively persued

for years by groups such as the Ponton Media Art Lab, by persons such as

Myron Krueger, and by the communication structure of the internet.

This slightly anarchistic approach was notably absent from this year's

Siggraph computer trade show in Orlando, Florida. The trade show showed that

besides interactive TV games the US-American entertainment industry is

concentrating on the employment of interactive technologies in the scope of

big theme parks. While the well-known Virtuality games by W-lndustries

individualize the player, the theme parks stress cooperation and team

spirit. The company Evans & Sutherland, for example, presented at Siggraph

the game Virtual Adventures, in which six players search together for the

eggs of the Loch Ness monster. The game appeals to typically athletic

characteristics such as ambition and team spirit. It offers alternative

experiences of pleasure and frustration, that is the classic features of a

game.

Computer games like this have a more than 20 year old history of

technological development, which remarkably took place at the same time in

military research and in art. In these years Interactive Art supplied many

alternatives to the above mentioned Loch Ness game and is essentially

characterized by the attempt to "humanize" the interface between system and

player. On top of this, the use of technologies that address the viewer

directly and involve her or him in a dialogue, constitutes a decisive change

in the tradition of the image. Therefore my main attention focuses in the

following on the reciprocal dialogue between user and system and on the

design of the interface. In the following text I will distinguish six

important implications of interactivity.

0. Historical Background and some Conflicts of Interactive Art

Its background in art consists of participational art forms from the late

sixties like for example Happenings and reactive kinetic environments.

Theoretical works like Umberto Ecos Opera aperta (1962) [4] contributed to a

reinterpretation of the part played by the spectator. In German aesthetics

this view was developed further especially by Wolfgang Kemp in the middle of

the eighties. His book Der Betrachter ist im Bild (The Viewer is Inside the

Picture) [5], in which he describes the method of receptional aesthetics

(RezeptionsSsthetik), seems to anticipate the perception principle we are

experiencing today in virtual reality. But this line of tradition is not

unbroken, as will be shown later on in this paper.

In a way Interactive Art builds on the traditions of participational art

forms by allowing the viewer to intervene in the action. However, in most

works, unlike in Happenings, this interaction is not meant as an attack

against the established art audience. Instead, it meets the needs of a

mediaeducated public. The implications of Interactive Art, though, go even

further: this art also reflects the role played by computer technology [6].

This may seem complicated, because Interactive Art uses the same technology

it comments upon, meaning, there is a certain lack of distance. The

situation of Interactive Art is therefore comparable with Video Art, which

had to gain certain independence from the language of television. Both art

forms demonstrate that today the role of the artist is changing

significantly. Instead of being a commentator standing outside society, the

artist now decides to take part in the socio-technological change and judge

from within.

1. Power and Play

With the American Myron Krueger the development of computer-controlled

Interactive Art started. He began as early as 1969 to conceive spaces in

which actions of visitors set off effects. In co-operation with Dan Sandin,

Jerry Erdman and Richard Veneszky he conceived the work Glowflow in 1969.

Glowflow is a space with pressure sensitive sensors on its floor,

loudspeakers in the four corners of the room and tubes with coloured

suspensions on the walls. The visitor who steps on one of the sensors sets

off either sound or light effects. In the scope of the Art & Technology

movement in the late sixties artists like Robert Rauschenberg and James

Seawright created similar 'responsive environments'. But at that time no one

in the 'art world' thought of creating a more complex computer-controlled

dialogue and focussing the interaction itself.

In the computer sciences this situation was different. Almost simultaneously

with Glowflow Ivan Sutherland developed at the Universiy of Utah the

precursor of today's head-mounted-display (HMD). This display was worn like

a pair of glasses and contained two small monitors, each of which showed a

stereoscopical sight to the eyes. Sensors registrate the headmovements and

transmit the information to a computer which then calculates the perspective

and gives the viewer the impression to move within the image.

Thus, at the end of the sixties two trends emerged independently of each

other, which have significantly influenced the present situation of

Interactive Art and computer technology in general:

1. the development of 'responsive environments' in the scope of the

US-American Art & Technology movement and

2. the development of the head-mounted-display

Krueger's work cannot be assigned to either of these trends. Neither did he

participate in the projects of the Art & Technology movement, nor did he

regard the head-mounted-display as a suitable interface. He thus used a

different variant, which was also developed at the end of the sixties, but

in the scope of Video Art: it was the closed-circuit installation in which

visitors are confronted by their own camera image. Krueger now combined this

principle with computer technology.

In Videoplace, a work Krueger has been constantly developing since 1974, the

visitors find themselves faced with their own projected video image that can

be changed by the computer program. In Videoplace there are a number of

different interactions, in which Krueger subverts the rules of narcissistic

self-reflection and self-control of the traditional video closed-circuit and

lets the user play with constantly changing versions of themselves. In the

most famous interaction called Critter a green figure appears on the screen

and tries to make contact with the visitor. It steers towards an exposed

part of the visitor's body and lands there. Then Critter begins to climb up

the arm, shoulder and neck until it reaches the highest point of the head.

Once there, it performs a joyful dance. Since Critter is programmed to reach

the highest point of the visitor's outline, the aim of the players is to

outwit Critter, that is, to subvert the program and develop their own rules.

Thus, the interactions of Videoplace are not only a joyful game but are also

concerned with the probing of power distribution between user and system.

Krueger's attitude towards the interface shows that he is opposed to the

isolation of the user caused by the head-mounted-display. Instead, he

creates an open space where it is the interaction and not the instrument

that causes the proximity to the system. This has important consequences for

the understanding of the interface. The technical interface - in this case

the video camera - is, in a way, invisible and loses significance. It is

substituted by the application itself.

In Europe the approach to Interactive Art and also the use of the interface

was quite different. The situation at the beginning of the 80's could be

characterized by the catchwords "Participation versus Interaction".

2. Participation versus Interaction

In Amsterdam in 1983 the Australian Jeffrey Shaw produced his first

interactive installation. He transferred his participational concept of art,

which he developed during the sixties to computer installations. In his

first interactive installation Points of View Shaw takes up the joystick,

the interface that is still customary for video games. Sitting on a chair

the spectator can move the projected video image of a stage with Egyptian

Hieroglyphs. With a second joystick she or he can steer sound traces. In

Points of View the spectator turns into the director who individually

selects the picture and sound material. The intended reception of Points of

View is described by Shaw as following: "It is the particular audio visual

journey made by a spectator who operates the joystick which constitutes a

'performance' of this work. For the other spectators that performance

becomes 'theater'."[7]

Although in Points of View Shaw dispenses with the physical performance of

the spectator, he still keeps his familiar terminology. The term movement

does not any longer signify the movement of the performer in space, like in

the former Happenings, but the movement of the image caused by the joystick.

The projected scene can be changed in its perspective with only very small

physical expenditure. Thus, the computer-controlled system inverts the

reception situation of the earlier Happenings. Formerly the spectator had to

change her or his position to perceive differently; now she or he induces

the computer image to change its perspectives. Thus, the movement of the

spectator is substituted by the movement of the image.

By means of the development in Shaw's oeuvre the above mentioned break in

the tradition reaching from participational art forms to Interactive Art

becomes clear. New points of view are not formed by physical experience but

with the help of new interactive media strategies. As I presumed at the

beginning, artists like Shaw adress in their Interactive Art a

media-educated audience, but nevertheless formulate an opposite position to

the passive reception of technically produced moving images. At the same

time Shaw is also criticizing certain potentials of interactive technology

itself. He decides against the video camera as the interface with the

system, probably because he considers it too unvisible. Instead, he uses a

bicycle in his most famous work called The Legible City, begun in 1988. With

the familiar pedaling and steering movements the cyclist can move through a

projected city of letters. The choice of this specific interface on the one

hand aims at providing the visitors with familiar patterns of behaviour, on

the other hand the bicycle as interface constitutes a refusal to do without

physical activity altogether.

3. Proximity and Manipulation

At the same time as Points of View - in 1983 - the Canadian David Rokeby

began to develop his interactive sound installation Very Nervous System,

which in the beginning he exhibited with changing titles and changing

technical equipment [8]. After Rokeby had experimented for a short time with

light sensors as interface and with analogous electronics, he decided -

without knowing the earlier works of Krueger - to use the video camera as

interface. Rokeby's Very Nervous System has a much more suggestive effect

than the works of Krueger and Shaw because he works with nonvisual system

effects as well as with an invisible interface. If one reacts intuitively to

the sound, a closed-circuit is created, in which music and movement are

slowly becoming a unity.

There is, however, a basic restriction: Krueger's Videoplace requires a

contrasted background to absorb distinguishably the persons in space; Rokeby

on the other hand is working only with a strong spotlight to achieve the

same effect. Therefore, the causal relations between an actual movement and

the sound are ambiguous. Although Rokeby employs the same interface as

Krueger, their positions differ from each other. Krueger's dissatisfaction

with the 'responsive environment' Glowflow was caused primarily by the fact

that the visitors interpreted chance events as the response to their

actions. While Krueger attempts a precise attribution of cause and effect to

reveal the reactions of the system, Rokeby is playing with the irritation of

the visitor. He hugely reduces the distance between visitor and system.

This is shown by his newest installation so far, titled Silicon remembers

Carbon from 1993. In this installation the visitor is even allowed to enter

the image that is projected on the floor and change it with her or his

movements. Infrared sensors and cameras are used as interface. With this

concept of reducing the distance Rokeby attempts a tightrope walk: on the

one hand the visitor assumes that she or he can control the image or the

sound, on the other hand the visitor is manipulated by these effects. This

suggestive power of interactive correlation is only disturbed by the fact

that Rokeby, as well as Shaw and Krueger, creates environments which allow

the presence of more than one visitor.

The works of Shaw, Rokeby and Krueger are conceived as environments. This is

not the case with the works which were created in the United States at the

same time or a bit later. Most of them are conceived as installations, that

is, the surrounding space is involved less strongly and the user often has

direct access to the input instruments. The most common input instruments

are the touchscreen and the mouse. As the works of Krueger, Shaw and Rokeby

have shown, the description of the interface is not restricted to its

technology. The same holds true for videodisc installations.

4. Strategies of Seduction

Nearly around the same time as similar works by the group around Glorianna

Davenport at the Media Lab at MIT [9] Lynn Hershman from San Francisco

developed her first interactive installation Lorna , finished in 1984. Lorna

as well as Hershman's second installation Deep Contact (1990) both work with

verbal requests like "Help Lorna Leave Her Home!". The picture sequences and

the texts depict women in the world of media as passive objects of male

desire. In Deep Contact changes in a projected video image are triggered by

touching a screen. The touching of body parts of the character Marion on the

touch-screen sets off different strands of narration and, according to

Hershman 'entangles the viewers into meeting their own voyeurism'[10.]

Her newest work so far, A Room of One's Own (1992), also attributes this

part to the spectator: the visitor looks through a little periscope into a

small bedroom on whose back wall sequences of images are projected. The

interaction in Hershman's work is being sexualized by the tactility of the

touch screen (in Deep Contact ) as well as by the intimacy of the observed

situation (in A Room of One's Own ). At the same time a fatal situation

ensues. As soon as the spectator acts he or she is caught in his or her role

as voyeur. Hershman does not use interactivity to free the user from

passivity, but to expose him or her as a voyeur. Put differently, the

desires of the audience are the cause for the repressive depiction of women

in media. Not even interactive technology can change that fact.

5. Nonlinear Narration

The New Yorker Grahame Weinbren produced his first interactive installation

The Erlking in 1986. In this installation the interaction is mainly

initiated and born by mysterious, almost static images. Weinbren - in

co-operation with Roberta Friedman - works with distinctly cinematographic

sequences.

The first picture shows the soprano Elisabeth Arnold who sings Schubert's

song Der Erlksnig. This picture functions as a leitmotif and guideline

assistance to which the user can return again and again. The other pictures

are partly based an Goethe's ballad, in which an old man narrates the saga

of the Erlking. Originating from the basic sequence the structure of the

narration branches out. It goes not only into detail but also into

additional aspects which are only loosely associated with the main plot or

the backup picture. In addition to this storyline Weinbren uses Freud's 1918

case study "From the History of an Infantile Neurosis" to try out a

nonlinear access to the sequence of images.

Narration and song in The Erlking are being quoted as historical examples of

oral tradition and are confronted with the nonlinear interactive form of

narration. As a result the interactive system takes over the role of

pictorial memory. The user occupies the role of the director and cutter

respectively, similar to Shaw's Points of View. Weinbren hopes that

interactive technologies are a more appropriate means to tell these old

stories [11]. This hope is problematic, since with originally linear

storylines the fragmentation of content doesn't necessarily lead to a better

understanding. Only if the stories were very well known today - that means

if they had a kind of social significance - the interactive access could

possibly add new points of view.

6. Remembering, Forgetting, Reconstructing - The 'Surrogate Travel'

The New Yorker Ken Feingold is the first who uses a touchscreen as interface

without integrating a second monitor. In his first interactive installation

The Surprising Spiral from 1991 the surface susceptible to touch is set in

the cover of a book. Fingerprints and two open hands inside the book

indicate that this object may be touched. Thus, the book functions as

interface to the pictorial action of The Surprising Spiral. A second contact

point, depicting a mouth with a light source, makes sound manipulations

possible.

On the videodisc of The Surprising Spiral pictures and sound are stored that

Feingold recorded in India, Japan, Argentina, Thailand, Scotland and the

United States over a period of 12 years. The documentary pictures are

contrasted with fast-moving Japanese TV advertisings and coloured computer

animations. Feingold makes a collage out of disparate film material from

different contexts, such as ethnographical, cultural, historical, religious,

aesthetic and medial contexts. Thus he combines nearly all the approaches

that until now made the reconstruction of historical facts possible. It

becomes clear that despite the partly documentary film material and the

mostly photorealistic video pictures the aspect of documentary truth in the

The Surprising Spiral is of no importance [12].

Because of the missing mise-en-sc*ne it is only the interaction, or to be

more precise, the filling up of empty positions, which creates a new context

for the user. Thus, her or his part in the reconstruction of reality seems

to be autonomous to a large extent Although Feingold - with the book as

interface - is quoting the reading culture, his position differs

fundamentally from Shaw's, who in his Legible City tries to mediate between

reading culture and interactive perception. In Feingold's Surprising Spiral

the book is a relict of times past - auratically charged, but nevertheless

hollow and robbed of its original function. What today is preserved or

forgotten as history, does not follow the laws of written culture anymore,

but is determined by the technological memory media. The reconstruction of

the stored material is determined by the perception strategies of these new

media.

The Surprising Spiral does not allow the purposeful approach of certain

places, which is still possible in Shaw's Legible City. Its place is taken

by the nondirectional, intuitive exploration of images and texts. This

gliding through the picture sequences is similar to the images of Feingold's

travel impressions - short moments which are unrepeatable, which are always

remembered, or reconstructed differently or sometimes even forgotten.

On the basis of this sketch showing the beginnings of Interactive Art one

can see that critical concepts about the role of interactivity in society

are not missing. By discussing the works of Myron Krueger, Jeffrey Shaw,

David Rokeby, Lynn Hershman, Grahame Weinbren and Ken Feingold I have

distinguished six important implications of interactivity:

1. Power and Play

2. Participation versus Interaction

3. Proximity and Manipulation

4. Strategies of Seduction

5. Nonlinear Narration and

6. Remembering, Forgetting, and Reconstructing

7. The Second Generation

In the past years the second generation of interactive artists has emerged.

Like with every second generation things are both easier and more difficult

for them. On the one hand the artists are able to build on what has already

been achieved, on the other hand they have to fulfill expectations of new

developments. This young generation shows a clear geographical separation

concerning the technologies used. While North-American and Canadian artists

like Bill Seaman and Luc Courchesne are working with interactive

installations and are using a touchscreen as interface, in Europe and

especially in Germany the environment is asserting itself. The group Supreme

Particles from Germany, for example, is working with the video camera as

interface like Krueger and Rokeby. In Architexture the recorded image of the

visitor is reproduced as a metallic-organic colour pattern on a moving

projection screen. The computer graphical alienation of the image is so

pronounced that a recognition is not easy. The fascination of the game is

created primarily by the inner life of the image that pulsates between its

own morphology and the representation of the visitor.

The sea animals in A-Volve by Christa Sommerer and Laurent Mignonneau have

an autonomous existence, too. In A-Volve the visitors create little sea

creatures with which they can then interact in a large water basin. The

individual virtual creatures react very differently to the hand movements of

the visitors. Some can be attracted, others try to flee from the hands. As

their modes of behaviour are very difficult to find out, they create free

play for the visitors who start to ascribe individual characteristics to the

various animals. The interface Sommerer and Mignonneau worked with has

completely lost its technoid character. This idea was already employed by

the artists in their 1992 work Interactive Plant Growing. Here the reaching

for real plants causes the growth of computer-generated plants on a

projection screen. Sommerer and Mignonneau draw the consequences from the

increasing control computer technology has over our environment. To them the

so-called artificial and the natural world do not oppose each other, but are

closely interconnected areas. In dealing with these areas a sensibility is

required that has to be partly re-learned, partly found anew.

Agnes Heged×s' work Handsight requires a similar sensibility. The

externalised eye - as the interface with the system - gives the viewer

access to the virtual world which in the end is to be explored by using the

sense of touch. In Joachim Sauter's and Dirk L×sebrink's "Zerseher", too,

the eye acquires tactile qualities. Only through the eye movements that are

recorded by an eye tracker can a monitor image be destroyed and newly

generated.

These few examples already show that the concepts for designing the

interface and with it the design of the interaction are getting more and

more subtle and diverse. The feedback-loop, which was most conspicuous in

David Rokeby's Very Nervous System, is getting closer in the works of the

young generation. The group Otherspace - that by the way, like the Supreme

Particles and Sommerer & Mignonneau worked at the Institute for New Media in

Frankfurt - uses brainwaves' to set little beetle-like beings into motion.

Only if the test person manages to relax, do the solar-powered beetles start

to move. Their movement in turn soothes the visitor so much that the result

is a very intimate relationship. The debate on Artificial Life - or A-Life -

that took place at last year's Ars Electronica seems to have created a sort

of 3. Frankfurt School that is decisively influencing the development of

Interactive Art. The question about the crucial differences between the

first and the second generation of interactive artists makes clear various

aspects:

1. Through institutions such as the Institute for New Media in Frankfurt,

the Media Art Academy in Cologne and the Karlsruhe Centre of Art and

Media Technologies SGI workstations are available to young artists

especially in Germany. This is one reason for the fact that the second

generation favours interactive environments (and invisible or 'natural'

interfaces) over installation work.

2. While in the work of the first generation a story or metaphors often

influenced the content of the work, the content of the newer works is

the interaction itself, which works without any form of traditional

narration. Because of this new meaning of the interaction the design of

the interface becomes increasingly important.

3. At the same time the antagonism between computer system and human being

is overcome. It is not so much the antagonism but the forms of future

co-existence that are being reflected. That is, to put it shortly, the

affirmation of interactive technology prevails over a critical

distance, but this does not result in an unreflected use of technology.

Concerning this general affirmation of technology the first generation

does not differ greatly from the second.

All in all the multi-layered, encoded levels of meaning in early interactive

works, which disclose their actual content only after a sort of decoding,

contributed to a certain acceptance of Interactive Art in the 'art world'.

However, this strategy had its price: the narrational contents often do not

come from contemporary social contexts, but from the safe context of

history. With this, some artists of the first generation adressed the

'reading-habits' of the artcritic's establishment. They negated the

achievements of the avantgarde, which clearly saw that art only has a chance

when talking to the masses and not only to a small bourgeois elite.

This trend is starting to change with the new generation. If they will

pursue this direction Interactive Art will fulfill its promise of being the

beginning of a new dialogue between the two ideologically separated sections

of art and technology.

References

[1] Uwe Jean Heuser: Der Computer ×bernimmt (The Computer Takes Over),

in: Die Zeit, 29.10.1993, pp.41 -42

[2] See for example Lynn Hershman: Art-ificial Sub-versions,

Inter-action, and the New Reality, in: Camerawork. A Journal of

Photographic Arts, Vol.20, Nr.1, 1993, pp.20-25, p.22

[3] Bertolt Brecht, Radiotheorie (Radio Theory), in: Gesammelte

Schriften, Vol.18, Frankfurt/M. 1967, pp.119-134

[4] Umberto Eco, Das offene Kunstwerk (The Open Work, Opera apena),

Frankfurt/M.[2]. 21973

[5] Wolfgang Kemp, Der Betrachter ist im Bild. Kunstwissenschaft und

Rezeptionsästhetik (The Viewer is Inside the Picture. Sciences of Art

and Receptional Aesthetics), Köln 1985

[6] See Erkki Huhtamo: Commentaries on Metacommentaries on

Interactivity, in: CAD Forum, 4th International Conference on

Development and Use of Computer Systems, MediaScape, Zagreb 1993, pp

229-236

[7] Jeffrey Shaw, information on Points of View1,2,3, dated 1983, with

letter to the author, 24.6.1991

[8] For a detailed discussion of the development of Rokeby's oeuvre and

its dating see Sske Dinkla: Interaktive computergest×tzte

Installationen. Eine exemplarische Analyse (Interactive

Computer-Controlled Installations. A Study of Some Examples),

unpublished MA thesis, University of Hamburg, 1992, pp.73-78. Here you

will find a more detailed discussion of Shaw's Legible City and

Krueger's Videoplace, too. A kind of summary of the MA thesis is

published under the title 'Interactive Computer-Supported

Installations. Examples of a New Art Form', in: CAD Forum, 5th

International Conference on Development and Use of Computer Systems,

MediaScape, Zagreb 1994, pp.29-36 (originally published in:

K×nstlerischer Austausch. Artistic Exchange. Conference Proceedings of

the XXVIII. International Congress for Art History, Berlin 1992,

pp.283-294, ill., german)

[9] The former Film/Video Group (now the Interactive Cinema Group)

produced the Videodisc Elastic Movies Disc with pieces by Bill Seaman,

Luc Courchesne, Russell Sasnett and Rosalyn Gerstein in 1984. They

worked at that time with Benjamin Bergery and Glorianna Davenport in

the work-shop in Elastic Movie Time. This information is based on

interviews with Bill Seaman in Karlsruhe (8./9.2.1994) and with

Glorianna Davenport (11.7.1994) in Cambridge and on the viewing of the

Elastic Movies Disc at the Media Lab at MIT.

[10] See Lynn Hershman: (note 2), pp.23,24 and Lynn Hershman: Some

Thoughts on Deep Contact, unpublished statement, 1991

[11] Interview with Grahame Weinbren, 18.7.1994, New York City

[12] Timothy Druckrey made a similar observation concerning Feingold's

work Childhood/Hot and Cold Wars/The Appearance of Nature in his

article 'Revisioning Technology', in: Iterations. The New Image, ed. by

Timothy Druckrey, International Center of Photography New York City,

Cambridge/London 1993, pp.17-38, p.35

----------------------------------------------------------------------------