By: Fanny Ljungman
Materiality has an important part in design. Everything we work with is materials. In the article by Gross, Bardzell and Bardzell, they even brought up the question if information can be materials [3]. But if everything can be materials it might be hard to understand where the concept starts and ends. I understand materiality as the use of physical materials in the world when integrating information technology. Not removing or design around physical obstacles but by making the physical and technical world work together. This is, e.g., brought up in the article by Lorenzo and Redström [1]. They study the possibility to incorporate drones to the delivery of mail and packages to people who live in rural areas. In the rural areas, the delivery guy delivers the packages to a store or a mail agent and then the receiver needs to go and pick up the package at that location. Here, the big problem is the distance. The delivery guy cannot deliver to all peoples doorsteps since they live so far apart in the rural areas. There is not enough time. To solve this problem, instead of minimising the distance, they use the distance and incorporates technology to it [1]. This helps people to be able to live in rural areas as well. Instead of just keep working for solutions that are only helpful for city living, we make use of the distance and add the technology of drones to be able to reach cities and homes that are further away. This is a great way of taking advantage of materiality. By using physical artefacts and aspects in information technology design, we use the stuff we already have and try to solve problems in the physical space. This example looks specifically at physical materials, but after reading the articles for this assignment, I now understand that non-physical objects, that are not perceived by the human sensory system, also can be materials [3].
It has become more and more common in the HCI field to try to understand materiality, to understand the material ground of which devices and systems are created [3]. HCI- and design- professionals try to understand how devices and systems are built but they also try to understand what is actually important to incorporate when it comes to materials. What are the most vital parts of this device? Could there be any other way of using materials for this interaction? There are several approaches to try and understand materiality and how to incorporate it in design. One approach, mentioned in Murer, Jacobsson, Skillgate, and Sundströms article is deconstruction [2]. They take a device apart to see how it looks inside and try to understand how it works. This method is useful for exploring and understanding the material in-depth and to kick-start design processes in the early stages. Gross, Bardzell and Bardzell also lift three approaches of understanding materiality, in their article [3]. One of these approaches is the Tangible User Interface (TUI)-perspective which focuses on observing the functionalities of tangible interfaces. In this approach, they mention the use of physical objects when creating interfaces. The second approach is the Computation as Materials-perspective which tries to understand computation itself as structured with materials. In this approach, the concept of material is not limited to only referring to physical objects. Though, we can only observe the non-physical materials indirectly by observing how the physical material get affected by non-physical material. The third approach is the Crafts as HCI-perspective which looks at how materials generate valuable information to us, clarifying how creation works and how information technology can be integrated with traditional crafts.
The perspectives that I have mentioned here are all presented as separate approaches as if you would choose between them when doing an analysis. They look at materials in different ways and therefore you will get different information about the materials depending on which approach that is used. I’m now gonna describe an input device on which I am going to try to apply all three of the perspectives form Gross, Bardzell and Bardzell’s article [3].
- Figure 1. An example of a graphics tablet. The tablet is connected to your computer so when you draw something on the tablet it shows on the computer monitor. Source: https://www.pexels.com/search/graphics%20tablet/
A graphics tablet is an input device for your computer that looks like a tablet with a blank, plastic or metal screen (Figure 1). This is most commonly used when you want to draw digital art. I used it a lot when I was studying animations. For this device, you also have a stylus, a pen-like object, that you use to draw on the graphics tablet and then the drawing shows up on your computer monitor. When I used the graphics tablet for animation, I also used an animation software program that would help me to save all the drawings and to lay them on top of each other to create the illusion of movement.
When applying the TUI-perspective to the graphics tablet it is clear that the tablet is a good example of a TUI. A TUI is recognised as an interface that is displayed in the physical space and combined with the digital aspects of a system [3]. With the example of the graphics tablet, drawing with the stylus on the tablet reassembles how you draw with a pencil on a piece of paper. This is the tangible interaction since it is in the physical space. The graphics tablet works as an input device, and the information that is put in when you draw something on the tablet is then displayed in digital form on the monitor, which is the digital aspect of the interaction. Something that Gross, Bardzell and Bardzell mentioning in their article is that with TUI:s there may be a collision between how the designer thinks the interface should be used and how the end-user interpret how it should be used [3]. Especially if the input devices reminds of some other object that the end-user has pre-assumptions about. The stylus that I used during my animation studies had some clear references to a physical pencil. When you turned the stylus around and put the back end of the stylus against the graphics tablet, you were able to erase the drawings with the stylus as if you were using an eraser. This was, of course, a reference to pencils that have an eraser on the back end (Figure 2). With this, the user could use their pre-assumptions about pencils when using the graphics tablet, which made it easy for new users to understand the device.
Though some other functions were not as intuitive, for example, instead of using the back end of the stylus on the graphics tablet to erase something, you could also change the controls on the digital interface (Figure 3) so that the tip of the stylus would be the eraser instead. This made it very confusing because now when you wanted to change back to the pen-function again, intuitively, you tried to turn the stylus around again (as you would with an ordinary pen) to get back to the tip of the pencil but instead you got an eraser on that side of the stylus as well. This feature was not as intuitive since it did not resemble the features of an ordinary pencil. When working with tangible interaction, it is important to look at how the physical properties of the object work to see how efficiently it communicate affordances to the end-user [3]. In the case of the stylus, I can understand that the controls on the digital interface were there to make it possible to use the software even though you did not have a graphics tablet, but for those who did, it was very confusing. It might have been better if they just had made one interface for tablet-users and one interface for computer-mouse-users.
If we look at the graphics tablet through the second perspective; Computation as Materials, we will be focusing more on the information rather than the physical materials [3]. In this example, that would be the information that is sent between the graphics tablet and the computer. We can’t really see it directly, which makes it hard to understand how it can be material but since materials are known as the ground components of design, we should not limit ourselves to only talk about physical objects. The information in computers have the ability to affect other materials and therefore it is materials as well [3]. To analyze the computation of a device you need to look at the relationship between what you put into the system and what comes out as the actual functionalities or output from the system. By drawing something on the graphics tablet and see what shows up on the computer monitor, we can observe the computation of this device. We do the actual drawings on the graphics tablet which, then, are shown as information on the monitor. The information is displayed through the physical objects and therefore we can observe the information as material. The primary role is the actual information that is being sent between the input device and the output device but the physical objects still play an important role in displaying the computational material to the user [3].
Figure 3. The digital interface of the software Pencil which is used to create 2-dimensional animations. Circled with red, you can see the control of which you can change the function of the stylus to work as an eraser instead. Source: https://www.thewindowsclub.com/pencil-animation-software-windows
When looking at the graphics tablet through the perspective of Crafts as HCI, I quickly realise that there may not be a complete connection, but I still feel that there are some traditional and communicative aspects in there. For example, the tradition of drawing is combined with digital opportunities. This is something that they mention in the article, a connection between traditional crafts and the digital world [3], which I think the graphics tablet fulfils. Since the illustrations are displayed through your computer, you can easily use functions like delete, copy and undo, actions that are not possible in the same way in the traditional pen and paper drawing-technique. The connection between physical interaction and digital aspects allows the user to communicate their creativity through the digital world with all the benefits of that at the same time as the traditional act of drawing is still present. In the article, they talk a lot about that the communicative roles of materials help to strengthen materials of the traditional crafts [3]. And I guess that the graphics tablet does not create anything with traditional materials since it does not directly create a traditional drawing on a paper but rather a digital drawing. But there is a lot that can be communicated through the digital drawings as well. When animating, your creative mind is communicating a story, which then will give some sort of meaning to the observer. I would not say that this is exactly what Gross, Bardzell and Bardzell mean with the communicational roles, but the creation of animation through the graphics tablet still have communicative value. In the article, their focus lies more on what is communicated by traditional materials, rather than through the materials [3]. The graphics tablet and stylus may not be made of traditional materials that would be used in the traditional craft of drawing, but they imitate a pencil and paper and some of their functions and therefore it still has some traditional aspect in it, I would say. It is a traditional practise that is implemented with new properties. Though, to make the graphics tablet more connected to this perspective it would be better if it actually would be able to communicate traditional craft by using traditional materials such as a canvas or a paper.
When looking at the approaches mentioned in this essay, it is not about looking at which one that is the superior approach, but more about which one that works best for the desired purpose. As I showed, it is possible for example to use all the three perspectives from Gross, Bardzell and Bardzell’s article when analysing the same input device. The difference is that you get different information about the materials. The TUI-perspective shows that the graphics tablet has the requirements of TUI but there are some questions about if the end-users view on the affordances is respected in the design. The Computation as Materials-perspective shows that the information that is sent between the graphics tablet and the computer is also materials and it shows us how we can acknowledge and understand that information by observing it through physical materials. The Crafts as HCI-perspective show that the graphics tablet might not communicate traditional materials but it shows that the tablet still has some aspects of communication and traditional crafts. By looking at the graphics tablet through all three of these perspectives we can observe and understand the materials of it in different ways to get different views of the materials. But I also think that a combination of several approaches would create a much deeper understanding of materials. By deconstructing a device, you get an initial understanding of the used material which gives you a good start for a design process, but then you might need to look more in-depth at the different materials for creating a TUI. Or maybe, you need to look at the computation of the system, something which the deconstruction-approach might not be able to show you. Our understanding of materials is very important in the design process, and it might take a combination of these approaches to understand it properly. If we fully understand the material and if we analyse the components that the object is built upon, we might see new ways to design objects or even new ways of using it. My point is that these approaches might not fully inform us about material individually but together they give a broader understanding of materiality, how to understand it and how to use it to find new creative design ideas.
References
[1] Lorenzo Davoli and Johan Redström. 2014. Materializing Infrastructures for Participatory Hacking. In Proceedings of the 2014 Conference on Designing Interactive Systems (DIS ’14), 121–130. DOI:https://doi.org/10.1145/2598510.2602961
[2] Martin Murer, Mattias Jacobsson, Siri Skillgate, and Petra Sundström. 2014. Taking Things Apart: Reaching Common Ground and Shared Material Understanding. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’14), 469–472. DOI:https://doi.org/10.1145/2556288.2557267
[3] Shad Gross, Jeffrey Bardzell, and Shaowen Bardzell. 2014. Structures, Forms, and Stuff: The Materiality and Medium of Interaction. Personal Ubiquitous Comput. 18, 3 (March 2014), 637–649. DOI:https://doi.org/10.1007/s00779-013-0689-4
Embodied Interaction 