In Wood, Davidge & Costa eds., (1992) The Fifth White House Papers: Graduate Research in the Cognitive and Computing Sciences at Sussex. University of Sussex, School of Cognitive and Computing Sciences, Brighton U.K. Research Paper CSRP 251

INFORMAL INTERFACES

Ian H.S. Cullimore

ianc@cogs.susx.ac.uk

I am currently researching the area of Informal Interfaces, human-computer interfaces in which some or all of the conventional formalities of input-output are relaxed to various degrees. For instance, we are used to being constrained in our choice of input to a system through the standard input devices of, say, a keyboard or a mouse. Such input devices show intolerance to a certain degree; the choices on a keyboard are finite, and a mouse or other pointing device input is itself finitely constrained by the menu choices available in, say, the windowing system. Similarly, we are also used to the restrictions imposed on us in the output mechanism of a system, such as finite menu choices and constrained graphics. Such mechanisms exhibit invariance in their output.

What is of interest, instead, is the notion of relaxing the conventional bounds of formal interface and allowing tolerance in input and variance in output. The third, and perhaps most crucial, part of the concept is in being able to encapsulate the gist of an idea, action or representation in the interaction, rather than its explicit enunciation.

What is required, then, is a more thorough understanding of the concept of informality itself as applied to HCI, and what this might mean both in theory and in practice. There are several possible avenues of research which open themselves up to examination, such as a study from the psychological viewpoint as to the contrasting natures of formality and informality, and how this applies to humans in social as well as computer interaction. Instead I am concentrating more on a computer science-oriented approach. Study certainly needs to be made of the underlying concepts and theories behind informalism in order to provide a concrete foundation for further work, but I also hope to be able to take the essence of that foundation and apply it, at least better than simplistically, to a practical implementation of informalism.

It is important to distinguish between superficial informality (existent only on the level of interaction itself), and embedded informality in which the notions and mechanics of informality extend deeper beneath the surface. An informal interface could be on top of an underlying formal engine, and similarly an informal engine could have a formal interface on top of it.

It will be interesting, too, to analyse existing applications of human-computer interfaces as to whether they do in fact already, to some degree, exhibit some of these concepts of informalism. Even more interesting will be the possibilities inherent with applying new concepts and translations to existing systems to see what may result.

I have researched a good number of papers already, of course, but there are a few of particular interest which I will briefly detail here. In the paper The Design Space of Input Devices (Card, Mackinlay & Robertson, 1990), the authors propose a useful method for physically mapping the properties of input devices in the form of a formal two-dimensional chart. Thus devices such as a keyboard, mouse, touch screen, trackball, joystick and so forth are seen to occupy independent positions in this space. This construction is naturally useful to possibly be able to predict, or construct, unique new input devices. This type of work has been continued by other authors, in papers such as A Comparison of Input Devices in Elemental Pointing and Dragging Tasks (MacKenzie, Sellen, Buxton, 1991).

In The Use of Complexity Theory in Evaluating Interfaces (Kiss and Pinder, 1991), the single notion of user effort is examined to determine the quality of a user interface, and in turn the authors identify computational work with the size complexity of the algorithm through which the task is accomplished at the interface. In Generalized Fisheye Views (Furnas, 1986) the author constructs a system based on the premise that there is usefulness in displaying informational images in such a way that immediate (local) information is shown in great detail, whereas distant (or global) information is shown in much less detail. Similar concepts are being worked on elsewhere, and there are several papers coming out of Xerox PARC, such as The Perspective Wall: Detail and Context Smoothly Integrated (Mackinlay, Robertson and Card, 1991).

As I mentioned previously, it has been of benefit to explore other, quite different, areas of research in order to study the principle of applying informality to previously formal concepts. In art and design, for instance, there has been some work done in the principles behind sketching, as in Amplifying the Mind's Eye: Sketching and Visual Cognition (Fish and Scrivener, 1990). In the paper the authors explain how Leonardo da Vinci advocated the use of "untidy indeterminacies" for working out compositions, because he believed that sketches stimulated visual invention.

In How to Handle an Image (Scrivener, 1983), the author discusses the problems associated with applying the conventional approaches of interactive computer graphics to certain kinds of two-dimensional design in art, and argues that, in fact, these conventional approaches are inappropriate for dealing with images and the synthesis of images. The author's concern is that the user is not really interfacing with the image of some model on the screen of a computer, but rather with the system's internal image of that model; this leads to possibly unwanted side-effects and constraints.

"Informal" logic from mathematics, as in An Introduction to Possibilistic and Fuzzy Logics (Dubois and Prade), may prove to be of some use too as a well-understood tool from another discipline to be applied to informal interfaces. The same may turn out to be true in the area of fractals, as in A Better Way to Compress Images (Barnsley & Sloan, 1988).

In Non-Determinism as a Paradigm for Understanding the User Interface (Dix, 1990, in Formal Methods in Human-Computer Interaction, Eds. Harrison & Thimbleby), the interesting and possibly relevant concept of non-determinism in computer systems is explored. The rather surprising result proposed by the author is that deliberately introducing non-determinism can actually 'help' in the system sometimes.

In Comprehending and Manipulating Complex Information Structures (Green, 1991a) the author further discusses his notion of Cognitive Dimensions, and the need for a descriptive vocabulary to more accurately describe relevant interface qualities. The author introduces notions such as viscosity (resistance to change), role-expressiveness and premature commitment. These concepts are further explored in The Cognitive Dimension of Viscosity: a Sticky Problem for HCI (Green, 1990) and Describing Information Artifacts with Cognitive Dimensions and Structure Maps (Green, 1991b).

Several papers of interest can be found in the Proceedings of the Workshop on Informal Computing (Defense Advanced Research Projects Agency, 1991). In What is Informalism? (Fisher, 1991) the author studies formalism and informalism in computer systems, and discusses the need to exploit informalism to computational advantage. Formal methods are seen to succeed in some cases, but fail in others. In Informalism in Interfaces (Reeker, 1991), the author studies some examples of adaptive interfaces, and analyses concepts such as representations of visual knowledge, and projecting cognitive representational structures onto computational representations.

In Constraint Programming Languages: Their Specification and Generation (Wm Leler, 1998), the potentially useful concept of programming languages constructed from a very different viewpoint than conventional procedural programming languages is studied. It may be that this is the best way in which to construct an informal processing engine. Constraint systems have been studied before; ThingLab - A Constraint-Oriented Simulation Laboratory (Borning, 1979) is one such example. Similar literature is detailed in Constraint-Based Tools for Building User Interfaces (Borning & Duisberg, 1987), Fabrik, A Visual Programming Language (Ingalls et al., 1988) and Planning with Constraints (Stefik, 1980).

References

Barnsley, M.F. & Sloan, A.D. (1988). A Better Way to Compress Images. Byte Magazine, January 1988, pp 215-223.

Borning, A. (1979). ThingLab - A Constraint-Oriented Simulation Laboratory. Xerox PARC paper and Stanford Computer Science Department Report STAN-CS-79-746.

Borning, A., & Duisberg, R. (1986). Constraint-Based Tools for Building User Interfaces. ACM Transactions on Graphics, Vol. 5, No. 4, October 1986. pp 345-374.

Card, S.K., Mackinlay, J.D. & Robertson, G.G. (1990). The Design Space of Input Devices. CHI '90 Conference Proceedings, Seattle April 1-5 1990. ACM Press.

Dix, A. (1990). Non-Determinism as a Paradigm for Understanding the User Interface. Cambridge University Press. Paper in Formal Methods in Human-Computer Interaction. Harrison & Thimbleby (eds.) 1990.

Dubois, D. & Prade, H. (1988). An Introduction to Possibilistic and Fuzzy Logics. In Readings in Uncertain Reasoning. Morgan Kaufman, 1990.

Fish, J. & Scrivener, S.A.R. (1990). Amplifying the Mind's Eye: Sketching and Visual Cognition. Leonardo, Vol 23, No. 1. pp 117-126, 1990.

Fisher, D.A. (1991). What is Informalism? Paper presented at the Workshop on Informalism, Santa Cruz, California, May 28-30 1991. Incremental Systems Corp.

Furnas, G.W. (1986). Generalized Fisheye Views. CHI '86 Proceedings, ACM Press.

Green, T.R.G. (1990). The Cognitive Dimension of Viscosity: a Sticky Problem for HCI. In Human-Computer Interaction - INTERACT '90. Diaper, D., Gilmore, G., Cockton, G. & Shackel, B. (eds.). Elsevier.

Green, T.R.G. (1991a). Comprehending and Manipulating Complex Information Structures. Queen Mary & Westfield College, London 1991. Summer School on "Theory & Methodology of Cognitive Science Applied to HCI Problems".

Green, T.R.G. (1991b). Describing Information Artifacts with Cognitive Dimensions and Structure Maps. MRC Applied Psychology Unit. Paper in HCI '91: Usability Now. Diaper, D. & Hammond, N.V. (eds.). Cambridge University Press.

Ingalls, D., Wallace, S., Chow, Y-Y., Ludolph, F. & Doyle, K. (1988). Fabrik: A Visual Programming Environment. Apple Computer, OOPSLA '88 Proceedings, September 25-30 1988. ACM Press.

Kiss, G. & Pinder, R. (1986). The Use of Complexity Theory in Evaluating Interfaces. In People & Computers: Designing for Usability (1986).

Leler, W. (1988). Constraint Programming Languages: Their Specification and Generation. Addison-Wesley.

MacKenzie, S, Sellen, A., & Buxton, W. (1991). A Comparison of Input Devices in Elemental Pointing and Dragging Tasks. CHI '91 Proceedings, ACM Press.

Mackinlay, J.D., Robertson, G.G. & Card, S.K. (1990). The Perspective Wall: Detail and Context Smoothly Integrated. CHI '91 Proceedings, ACM Press.

Reeker, L. (1991). Informalism in Interfaces. Paper presented at the Workshop on Informalism, Santa Cruz, California, May 28-30 1991. Incremental Systems Corp.

Scrivener, S.A.R. (1983): How to Handle an Image. In Design Studies, Vol 4 No. 1 January 1983. Butterworth & Co. (Publishers) Ltd.

Stefik, M. (1980). Planning with Constraints. Artificial Intelligence 16 (1981), pp 111-140.