The Cycle of Social Interaction as a Framework for Immaterialising Social Interaction Satisfiers

David Leevers, Sept 2001

This section of the ASSIST State of the Art report explores the current state of the art in the area conventionally known as HCI and is written from the designer’s point of view. Thus it is confined to practical tools and to the scientific research that can help in the design of future network supported systems.

It is no longer viable to design a self-contained product of service for the current market. It is necessary to plan a total system made up of what the designer can deliver and what they can anticipate designers of complementary products can deliver on the same timescale. Thus the community of designers is effectively planning a total system on the same timescale. As such, every designer is playing their own part in changing the culture of a complete community in which the immediate users are the interface between the community and its network infrastructure.

This perspective is already important for the designers of networked games for children. Such games can be played without regard for the conventional barriers of nation, race and culture. If played when young enough they add an emerging world-wide belief system to the belief systems acquired from family and local community. As these children mature so such applications will mature from games to business tools. Just as the battle of Waterloo was won on the playing fields of Eton so the battle of global sustainability will be won on the Playstations of Sony.

By changing the name from HCI to CNI, Community Network Interaction, it is hoped that the enormous impact of future Mature IST, MIST, on global society is highlighted. As the user interface has improved to include more and more of the context needed to understand the information so the scope of the network has broadened to represent the rest of the community. No longer is it one person in front of one responsive computer, it is one person with their own place in society, interfacing to the rest of society through the network. Thus CNI, Community Network Interaction, is an important perspective for replacing material-intensive social interaction with IST-intensive immaterial alternatives.

Over the last 30 years the increasing logical power if the computer has been used to throw new light on the way brain and body work together. Initially the insights were confined to the rational areas – grammatical speech, hand-eye coordination and problem solving. However affective issues started to become relevant as costs of computer equipment dropped to a level which allowed their use for entertainment and meetings as well as for clerical office activities.

MUDs (text based fantasy based chat groups) were the first indication, (Roy Trubshaw, Essex University, 1979), of the unique affordances of computer-mediated interaction between people. Like the mask in a classical Greek play, the text exchange sanctions a greater level of fantasy than is credible in face-to-face interaction, or even over the telephone.

The State of the Art in the Relevant Human Sciences

The computer started to have a substantial effect on the human sciences about 30 years ago. More recently work on disembodied artificial intelligence, embodied artificial life and evolutionary psychology has led to an enormous increase in our understanding of body and brain and encouraged a more holistic approach in both the human and the computer sciences.

Brain scanning has emerged as a new and very powerful tool for exploring what is really going on in the working brain. Until about 15 years ago the only insights on the relationship between brain and mind were obtained from the very crude procedure of studying the link between damaged brains and damaged minds. The fatal flaw was that all that could be proved was that certain brain areas are necessary for certain types of brain activity, not that they are sufficient. Thus most of the integrating activities of the brain, those closest to consciousness, were missed in early work.

The computer has also helped with the recent growth in longitudinal studies of large numbers of individuals over periods of many years. These are helping to define the effects of early environment on later behaviour at a quantitative and statistical level. Much of this work has been funded by US organisations anxious to understanding the reasons for the underperformance of blacks in society. Whilst the philosophers have been arguing over nature versus nurture the scientists have been quantifying the complex and time dependent interactions between the two.

Perhaps one of the most relevant social issues for ASSIST is the multigenerational effect of any cultural pattern of behaviour. Immigrant families do not get fully integrated till the third generation. Concentration camp trauma can be observed in the grandchildren of survivors, and, if a slave heritage of over 150 years ago is a daily reminder to everyone who meets a black, then it will be a very long time before they can escape the demeaning nature of this heritage.

The Network Society has the potential to reduce the chance of the damage from extreme influences in early life. A closed community cannot be maintained if there is ready access to television, radio and the web. South Africa prevented television until just before the collapse of Apartheid and the Taliban still ban television. Although such early effects of access to global culture appear beneficial, little work has been done to predict longer-term effects. Statistics on the correlation between fragmented families and subsequent alienation, suicide and crime have not yet been assimilated.

Although there appears to be a strong ethical instinct in humans, as part of their ability to function as a socially sophisticated animal little is known about the way in which this ethical sense can be damaged by comprehensive exposure to the future information infrastructure. This infrastructure is becoming so comprehensive and powerful that it will be as essential to future generations as the servant class was to the middle class of Victorian London.

In particular the effects of multiple cultural influences on the very young needs to be explored. This reduces the power of any particular formal belief system and removes threats such as hell that many have assumed to be necessary in order to ensure good behaviour. We are relying more and more on the basic instinct of this social and collaborative species to respect other members of the same species – dangerous position to be in before we understand the scope and limits of this moral instinct.

Attachment theory in particular has come a long way since Bowlby’s work and the Harlow experiments on socially deprived monkeys. The processes from conception to childhood have now been mapped out in great detail. The stages in structuring the growing brain and emerging mind are now well understood. It is now possible to measure the effectiveness of a mother by using objective attachment tests. As mentioned above the validity of these tests has been established by observation many years later in large scale longitudinal studies. This is highly relevant to any new technology aimed at very young children. Indeed the design of most intelligent toys is guided by the results from such attachment work. The new generation does not ask the question, “can toys be intelligent” but “ how intelligent is this toy”. The responsiveness of baby toys is a totally new phenomenon; until now the only things that responded to the very young child were people and pets, and each are far more troublesome than the average intelligent toy.

Robert E. Lane in “The Loss of Happiness in Market Democracies” provides the social and factual framework that justifies the potential shift to IST supported immaterialisation. Although citizens of advanced market democracies are satisfied with their material progress, many are haunted by a spirit of unhappiness. Drawing on extensive research in such fields as quality of life, economics, politics, sociology, psychology, and biology, he shows that the main sources of well-being in advanced economies are friendships and a good family life and that, once one is beyond the poverty level, a larger income contributes almost nothing to happiness. As prosperity increases, there is a tragic erosion of family solidarity and community integration, and individuals become more and more distrustful of each other and their political institutions. Lane urges that we alter our priorities so that we increase our levels of companionship even at the risk of reducing our income. This is an ideal opportunity for IST based alternatives to material consumption. The precise nature of which services are required at what stages in a persons life can be explored and clarified by using the Cycle of Social Interaction.

The Cycle of Social Interaction

A Systems Framework for implementing Immaterialisation

In ACTS project CICC the Cycle of Collaboration was one part of what was called the Cycle of Cognition, a framework for designing the wide range of IST-intensive tools needed to support individual information work and distributed collaboration, all within the construction sector. The cycle evolved during the 1990’s, starting with the Virtual Meeting Room implemented in RACE project DIMUN in 1991. Multiple video windows were included in RACE project BRICC in 1994. Finally familiar metaphors for all the stages in the cycle were added in ACTS project CICC in 1996. RACE project MITS (Metaphors for Integrated Telecommunications Services) helped with fundamental issues of the usability and consistency of a range of metaphors. The approach owes a lot to George Lakoff’s work in his book “Metaphors we live by” and his body-mind bridging “Philosophy in the Flesh”. Although very comprehensive compared with most attempts at a framework for local and tele- interaction there were two major omissions from the Cycle of Collaboration.

1, Representations of the bodies of the participants. This is not very relevant when every participant is an able bodied adult (or, if disabled, is using the new affordances of IST to represent themselves in as abled a way as possible) and interaction is confined to what is needed during the working day. What is missing are the physical and more emotional aspects of family life with children, partners and the elderly

2, Representations of conflict and competition were not included.

This section is a first attempt to broaden the Cycle of Collaboration to become a more inclusive Cycle of Social Interaction. A future exercise is to broaden the other components of the overall Cycle of Cognition; the Cycle of Information, the People and Information Finder and the Hyperbola of Synchronisation.

In ASSIST the primary purpose of the Cycle of Social Interaction is to address Immaterialisation at a holistic or systems level. As has been emphasised elsewhere, Immaterialisation Switches are never a one-for-one substitution of a material intensive satisfier by an IST intensive one. The affordances of the new are so different from those of the old that it is necessary to adopt a holistic point of view and explore the implication in all aspects of a particular human need such as social interaction. Since very sophisticated social interaction is the most distinctive feature of the human species it seems to be an appropriate one to start with.

The Cycle of Social Interaction is the temporal framework within which the different types of communication take place, both locally and across the network. By focusing on the most likely transitions from one mode to another this cycle provides a way of mapping the diversity and richness of social activities on to a wide variety of emerging voice, data, video and multimodal services.

The stages of the cycle; Territory, Map, Landscape, Room, Table and Theatre, describe a paradigmatic route through different communications stages. Each stage can be made up of any combination of material-intensive and IST-intensive components. In the following description the stages are interpreted in terms of the working day.

1.1.1. Territory

Territory is the non-communicating stage, when the individual is safely inside their personal fortress of home or workplace; working, relaxing or asleep. Every workplace has some area within which the individual’s tasks are performed: the wall the bricklayer has been asked to build or the office where a paper is written. Others respect this area and avoid interrupting unless invited. At home this territory is that quiet corner where ideas are formulated and plans are made.

There is an equivalent mental territory, that part of the unconscious mind which reviews what has been learnt recently and adjusts existing mental models to fit in with new experiences. This process includes rehearsing the implications of new ideas, perhaps in dreams or in play. Any unanswered questions are identified and the need to resolve them drives us forward into the next stage, the Map.

1.1.2. Map

The Map is the stable information that is effectively a signpost to up-to-date but not necessarily as reliable information. When we start to venture out from the security of our own Territory we are not immediately ready to confront and question others. The Map might be the morning paper, Yellow Pages, web pages or a database - anything that provides a reliable starting point for addressing the real challenges of the day. These challenges are clarified through social interaction in the Landscape.

1.1.3. Landscape

As issues become clearer it becomes necessary to track down the most up-to-date information. This is carried out in a physical or virtual Landscape of people and the documents and databases that they are actively working on. One physical implementation of the Landscape is the open-plan office. Actions are visible, conversations are overheard and it is easy to interrupt, ask advice or share documents. The networked version can be a shared virtual environment in which remote people are represented as avatars in a perspective virtual landscape and information is represented as icons. When two or three members of a networked team get into conversation video from their Webcams can be embedded in the virtual landscape, thus implementing the casual and continually changing conversational clusters that are one of the main benefits of open plan team offices.

1.1.4. Room

Contentious or unresolved issues require more privacy and more concentration than is possible in the public open plan Landscape. The ritual of going from a shared area of the Landscape to the private area of the Table is represented by the Room metaphor. It supports rapport and trust building before starting the meeting proper. It is where first impressions are made and the real or metaphoric handshake takes place. Real “Rooms” are often impressive reception areas in which the embodied minds of the meeting participants can size each other up before proceeding into the meeting room itself. A comparable impact can be achieved after the immaterialisation switch by confining the use of a high-quality high-bandwidth video to this trust building stage only.

1.1.5. Table

The Table is the real or virtual space where the shared objects (common artefacts) required for collaboration are displayed and manipulated. In real life, the “Table” could be a desktop, building site or factory. On the network it is the shared window or whiteboard. Each person also needs a private area, a notebook in a co-located meeting or a private part of the screen in a distributed one.

The solid walls and closed door of a real room convey a feeling of privacy and encourage the occupants to share confidences. There is a need for an equivalent privacy in the networked meetings. As yet there are no good examples, plain virtual walls look meaningless on the screen and quickly get covered by foreground material; shared documents and video of participants.

In the CICC factory prototype (1996) the Virtual Table has occupied the lower part of the computer screen. Above the Table were pairs of views; a video glance and a screen glance for each participant. These miniatures serve the same function as looking across a real table to see what the others are doing in a co-located meeting. The level of privacy is the same as in the co-located meeting. The miniatures are not clear enough to read text but they do indicate the extent to which the others are focusing on the shared task. These views are important in maintaining rapport and trust, even when their occupant is not contributing to the conversation.

CICC Factory Prototype

Networked meeting with 5 members and 2 active participants

The Table is a very rich area for current design work. There are many other features that should be supported in a production system; a quick zoom in to one video view or a call to an outsider who is on standby, ready to participate when their expertise is required. As the desktop functions have stabilised and PCs have reached adequate speed so these features have started to appear in commercial shared virtual environment systems such as NetMeeting, BT’s Passepartout, Blaxxun, etc.

1.1.6. Theatre

Meetings are held to achieve results. These results are conveyed to the relevant audience; customers, colleagues or students in the Theatre stage. In project work the theatre is the metaphor for any process that conveys results to a wider audience; minutes of meetings, a project database, multimedia presentations and giving instructions to project implementation staff at the start of the day.

Communication in the Landscape or around the Table is informal, sometimes barely structured, often using the abbreviated or specialist language of a group who know and understand each other well; sometimes dependent as much on body language as on speech.

However presenting outside the immediate group requires a performance with some dramatic structure and language that can be understood by the audience. We need to emphasise and reinforce some points and compress or discard others and we need to choreograph the relationship of words and pictures. We need presence; we need timing; and we need to experience the individuality of applause and the anonymity of booing.

More than that, we need to create an atmosphere of community and trust in the audience. We may have to convey difficult concepts or unwelcome truths. So, in behavioural terms, we need to weld a disparate collection of individuals into a single resonant group before we get to the hard bit.

An effective performance changes the way the audience will think in future. The performance can be said to have killed the previous personality and given birth to a wiser one. This is one reason for the rituals of trust associated with going to the real theatre; checking reviews before buying a ticket, studying the mood of others in the foyer and being aware of their presence during the performance. The loss of these rituals in conventional broadcasting has had some unfortunate social repercussions. Their disappearance in the early Internet was one reason for excessive scepticism about the quality of the information on it.

Conventional TV has taken over many of the functions of the physical theatre while adding the new element of a global shared experience. What it cannot do is re-create the atmosphere of excitement and the sense of togetherness experienced by an audience when they share the real physical space of an auditorium. It is to be hoped that the interactive two-way features of digital television will restore this feeling of being part of a vast and responsive audience. Early steps in this direction include telephone voting as in Big Brother and voting for particular music or episodes.

1.2. Fractal Communications

In the above the Cycle of Social Interaction can be seen can be seen to cover the daily cycle: waking at home, setting off using a map, actively browsing the office landscape in the morning, negotiating over the midday meal and collaborating in the hazy glow of the afternoon, then taking a seat in the theatre as the sun sets to surrender the mind to the persuasive powers of playwright and actor, finally going home with new ideas teeming inside the head, ideas that will have slightly altered us when we wake up next morning.

The Cycle of Social Interaction can also be seen to represent our journey through life: emerging from the home of the womb, spending a few months in a Map of clarifying sensations and an attachment figure, then learning from the social Landscape of other children and adults. Adolescence is spent in a Room of continually changing personalities, working towards an enduring rapport with the rest of society. Then the individual settles down to more focused activities at the Table of career and family. Finally the respect of the community is gained and cultural memes are passed on to the next generation in the Theatre of grandparents stories.

At the micro extreme a member of a meeting will go through a complete cycle inside their head in reacting to what someone has said and then presenting a response. Thus the Cycle of Social Interaction can be applied to an enormous range of time-scales from the formulation and presentation of a single statement to the lifecycle of a person or even a civilisation. As such it may have the potential to become a robust and flexible framework for taking account of the affordances changes in the many different Immaterialisation Switches involved in reducing the material intensity of today’s patterns of social interaction.