Putting the social back in social robotics

Much research on the interaction of humans and robots focuses on dydadic interaction, or at best a few humans with one robot. Below a photo from the FRACTOS study where a robot and a virtual tutor create a triad with a child in a learning task on fractions.

Sooraj Krishna, Catherine Pelachaud, and Arvid Kappas. 2020. FRACTOS: Learning to be a Better Learner by Building Fractions. In Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’20). Association for Computing Machinery, New York, NY, USA, 314–316. DOI:https://doi.org/10.1145/3371382.3378318

In social psychology, much research deals with how groups interact with one another. The us vs them, that is at the heart of much conflict. This is not yet a topic of much social robotics research and theory. However, this is of course a function of the small number of robots in the wild. In a science fiction movie it is easy to have hordes of robots being part of a busy street scene, such as in the classic I, Robot movie. But which research group possesses enough robots to create real groups? Who has interacted with large quantities of robots in their place of work? However, it is only a question of time that robots will cease to be novelties and intergroup psychologiy of mixed human/robot society will be a thing. I am convinced that in this decade it will become important to foreshadow how robots might become part of human groups, or how groups of humans and robots will interact in collaboration or competition.

Some of these issues are dealt with in a recent paper with Eric Vanman.


Society’s increasing reliance on robots in everyday life provides exciting opportunities for social psychologists to work with engineers in the nascent field of social robotics. In contrast to industrial robots that, for example, may be used on an assembly line, social robots are designed specifically to interact with humans and/or other robots. People tend to perceive social robots as autonomous and capable of having a mind. As such, they are also more likely to be subject to social categorization by humans. As social robots become more human like, people may also feel greater empathy for them and treat robots more like (human) ingroup members. On the other hand, as they become more human like, robots also challenge our human distinctiveness, threaten our identity, and elicit suspicion about their ability to deceive us with their human‐like qualities. We review relevant research to explore this apparent paradox, particularly from an intergroup relations perspective. We discuss these findings and propose three research questions that we believe social psychologists are ideally suited to address.

Vanman, EJ, Kappas, A. “Danger, Will Robinson!” The challenges of social robots for intergroup relations. Soc Personal Psychol Compass. 2019; 13:e12489. https://doi.org/10.1111/spc3.12489

Looking for a PhD position in psychology/affective computing?

I am looking for candidates with a psychology background for the new ANIMATAS (Advancing human machine interaction with human-like social capabilities for education in schools) project. ANIMATAS (MSCA – ITN – 2017 – 765955 2) is a H2020 Marie Sklodowska Curie European Training Network funded by Horizon 2020 (the European Union’s Framework Programme for Research and Innovation), coordinated by Université Pierre et Marie Curie (Paris, France). The person working with me would be based at Jacobs University Bremen. The topic is

Social context effects on expressive behaviour of embodied systems

More here

Please note that ANIMATAS offers in total 15 funded PhD positions and information about all of these, and the instructions for applications can be found here.

Explain the Uncanny Valley

A friend alerted me on Facebook to this Youtube video which is on the IEEE Spectrum page. Roboticists struggling to define the Uncanny Valley in one minute. (Thanks Astrid!)

Somehow it is really amazing which role the Uncanny Valley has taken in pop culture – and in research as well. I remember when I started to get interested in the phenomenon it was neither well known, not taken very seriously. A colleague who I had asked said “… well it is not really something that is much taken serious among roboticists — perhaps you joke about it after a few glasses of Sake.”

I was not so sure about whether this is true because it was interesting that there seemed to be a taboo to create robots that were too human … obviously the early work of David Hanson and of course Hiroshi Ishiguro were the exceptions. Otherwise robots would have blank plates as faces or would display only the most symbolic facial features.

So listening to the video I am fascinated that people do not primarily focus on the fact that the Uncanny Valley is first and foremost an hypothesis which was presented by Masahiro Mori in 1970. It is thus a theoretical construct, not a thing, or a phenomenon. Whether the phenomenon exists, for whom, for how long and all of that are empirical questions. At this point the evidence is mixed. What do you think?

(trying to get a poll here …)

IEEE Transactions on Affective Computing Seeks New Editor-in-Chief for 2015-2016 Term


The IEEE Computer Society seeks nominations and applications for the volunteer position of Editor-in-Chief (EIC) for the IEEE Transactions on Aff ective Computing (TAC) serving a two-year term starting 1 January 2015.

IEEE TAC is a cross disciplinary and international archive journal aimed at disseminating results of research on the design of systems that can recognize, interpret, and simulate human emotions and related affective phenomena. The journal publishes original research on the principles and theories explaining why and how affective factors condition interaction between humans and technology, on how affective sensing and simulation techniques can inform our understanding of human affective processes, and on the design, implementation and evaluation of systems that carefully consider affect among the factors that influence their usability.

The EIC will be responsible for the day-to-day volunteer leadership of TAC, including coordinating and overseeing the peer review process; recommending candidates for the editorial board; chairing the editorial board; developing editorial plans for the publication; serving as a non-voting, exofficio member of the Publications Board; serving as a non-voting ex-officio member of the Steering Committee for TAC, and working in general with volunteers and staff to ensure and maintain the timely publication of an exceptionally high-quality transactions.

Applications should include a complete curriculum vita, a brief plan for the future of TAC, and a letter of support from the candidate’s institution or employer. The plan should include (1) the candidate’s perspective on the challenges and opportunities for the enhancement and improvement of the quality of TAC, (2) specific tasks to be undertaken if appointed EIC, (3) objective milestones associated with each task, and (4) a proposed schedule.
The search committee prefers electronic submissions in Microsoft Word or PDF. Please direct all questions and submit completed applications to Kimberly Sperka,


The due date for nominations and applications is 7 February 2014.

Additional information about the IEEE Computer Society, the co-sponsoring societies, and the IEEE Transactions on Affective Computing is available at http://www.computer.org and www.computer.org/tac


Empathy with Military Robots?

US Navy 090207-N-9610C-007 A remote-controlled...

US Navy 090207-N-9610C-007 A remote-controlled robot used by Explosive Ordnance Disposal Expeditionary Support Unit (EODESU) 1 called (Photo credit: Wikipedia)

The Interwebz are buzzing with reports regarding the PhD work of Dr. Julie Carpenter at the College of Education at the University of Washington. She has interviewed Explosive Ordnance Disposal (EOD)  personnel who work with robots to clear dangerous explosives. These machines are very purpose-built for difficult terrain and manipulating particular types of objects.


Talon IV (QinetiQ)

Not very cute, not very pretty – and yet, according to Dr. Carpenter’s work, handlers may feel quite strongly about their machines. They might assign them gender, they give them names – and they feel not well when they are destroyed in the call of duty. Specifically, there appears to be a trend to attribute human traits to these machines (some more details here).

English: INDIAN HEAD, Md. (Feb. 26, 2009) Warr...

English: INDIAN HEAD, Md. (Feb. 26, 2009) Warren Tibbs, a robot operator for Navy Explosive Ordnance Disposal (EOD) Technical Division, Indian Head, Md., shows the many different robots that are developed on the base and used by EOD technicians and civilian police department SWAT team members. (U.S. Navy photo by Mass Communication Specialist 2nd Class Jhi L. Scott/Released) (Photo credit: Wikipedia)

I still have not read the thesis itself, as the news regarding this just broke, and I do not want to go into details regarding the potential military implications if the handlers of such machines feel empathy towards them – there is apparently a discussion already out there. Instead, I want to focus briefly on the importance of time spent and experience with machines and the development and expression of empathic sentiments and the implications for research on social relationships with robots.

Many studies on the uncanny valley, or others that deal with the relationship of humans and machines confront relatively unprepared participants with devices of various complexity in the laboratory, or laboratory-like contexts. Perhaps this is really not a great idea. Possibly, uncanny valley effects disappear quickly with machines that approach human perfection. This is an idea that Astrid Rosenthal von der Pütten, who is working on empathic responses to robots, proposed when we discussed such phenomena. Perhaps, the other side of the coin is that very non-human machines may become very close to us if we work with them day-in, day-out. If this is so, then we should take results of research that studies exclusively responses to unfamiliar stimuli with a grain of salt. With this I mean either responses of people who are generally unfamiliar with robots, or even with people who have some familiarity with robots but not a particular type of robot. I do not want to question the usefulness of such studies in general – in my own laboratory, for example, we are investigating responses to artificial entities of various kinds in the laboratory and measure behavioral, physiological, and subjective responses of student participants who have little experience with the types of stimuli we use. However, we and other researchers in this field should keep the influence of experience and familiarity in mind.

Repliee Q2. Taken at Index Osaka Note: The mod...

Repliee Q2. Taken at Index Osaka Note: The model of Repliee Q2 is probably same as Repliee Q1expo, Ayako Fujii, announcer of NHK. (Photo credit: Wikipedia)

Remember how people responded first to cars, or the famous report of how people responded to early films of the brothers Lumiere? Apparently, people were shocked, scared, ran away (the extreme versions of this story are likely to be urban legend). These responses are difficult for us to imagine – so much have we gotten used to the highly mechanized and mediatized environments we grow up in. Possibly, if we would grow up in a robotized environment, there would be no odd sensation to almost-human machines and we might feel deeply for machines that are very much not humanoid. For me the fictional version of this is the relationship of the character Freeman Lowell played by Bruce Dern in the movie Silent Running with the three little robot drones Huey, Dewey, and Louie, As he spends more and more time with them, his emotional bond becomes stronger and stronger – and this means not just feeling warm and fuzzy, but displaying a whole range of emotions toward them and reacting very strongly to their actions.


Poster from the movie Silent Running, claiming fair use (does not detract from original work).

Anthropomorphic machines … from the 19th century



The idea to create anthropomorphic robots is today often associated with the uncanny geminoids of Hiroshi Ishiguro. However, the idea to build anthropomorphic automatons is of course much older. Most famous is probably the chess player automaton The Turk by von Kempelen in 1769.

An illustration of the workings of the model. ...

An illustration of the workings of the model. The various parts were directed by a human via interior levers and machinery. This is a distorted measurement based on Racknitz’s calculations, showing an impossible design in relation to the actual dimensions of the machine. Standage, 88 (Photo credit: Wikipedia)

Less known is the French tradition to use anthropomorphic automatons for advertising purposes. A recent BBC piece shows some rare and interesting clips from an exhibition at the Musée de l’automate de Souillac. Recommended! Here is a different version of the video.


… and if you cannot get enough of the early automatons … check this site out.



Traveller at Intelligent Digital Games for Empowerment and Inclusion


Serious games have come quite some way. In their introduction to the first International Workshop on Intelligent Digital Games for Empowerment and Inclusion on May 14, 2013, Schuller, Palettam, and Sabouret summarize:

Digital Games for Empowerment and Inclusion possess the
potential to change our society in a most positive way by
preparing selected groups in a playful and fun way for their
everyday life’s social and special situations. The current
generation of such games thereby increasingly demands for
computational intelligence algorithms to help analyze players’ behavior and monitor their motivation and interest to adapt  game progress. Intelligent Digital Games for Empowerment and Inclusion (IDGEI) focus in particular on such games in connection with machine intelligence and its inclusion in digital serious games.


My colleagues and I from the eCUTE project are very thrilled to participate in the workshop, where we will describe Traveller, an intercultural training tool for young adults.

Traveller is based on an original theoretical
framework which focuses on key concepts of intercultural
training. By progressing through a creative story, users are able to engage via a novel interaction paradigm with intelligent virtual characters that incorporate different simulated cultures which can lead to misunderstandings and sometimes conflicts. Through the use of an innovative evaluation approach, users will gain a greater understanding of the behavioural differences between these
characters, and thereby learn to become more effective at dealing with misunderstandings due to differences in culture.

The contributions of the workshop will be published in open access proceedings with the Society for the Advancement of the Science of Digital Games (SASDG), (see below – I will post a link when this becomes available). In the meanwhile you can learn more about TRAVELLER and see some video material on the eCUTE web page. We have spent almost 3 years developing TRAVELLER and a second application, MIXER in the pursuit of technology enhanced training in cultural understanding. A demonstration of TRAVELLER was also given this week at AAMAS 2013 (12th International Conference on Autonomous Agents and Multiagent Systems, Saint Paul, Minnesota, USA, May 6 – 10, 2013).


Degens, N., Hofstede, G. J., Mascarenhas, S., Silva, A., Paiva, A., Kistler, F., André, E. Swiderska, A., Krumhuber, E., Kappas, A., Hume, C., Hall, L., & Aylett, R. (2013). Traveller – Intercultural training with intelligent agents for young adults. Presented at the First International Workshop on Intelligent Digital Games for Empowerment and Inclusion (IDGEI 2013) held in conjunction with the 8th Foundations of Digital Games 2013 (FDG), Chania, Greece, May 2013.

Schuller, B.,Paletta, L. & Sabouret, N. (Eds.)(2013) Proceedings 1st International Workshop on Intelligent Digital Games for Empowerment and Inclusion (IDGEI 2013) held in conjunction with the 8th Foundations of Digital Games 2013 (FDG), Chania, Greece, May 2013. SASDG.

The Happy Vacuum Cleaner

One of several autonomous vacuum cleaners - the Kobold VR100

Kobold VR100

In 2005, I organized a symposium at the 14th Conference of the International Society for Research on Emotions (ISRE), which was held in July (11-15) in Bari Italy. In this context I gave a talk, which I had given the title MY HAPPY VACUUM CLEANER.

If I remember correctly, a considerable number of my dear colleagues initially stared at me as if I was nuts. The talk was a conceptual piece that traced issues in creating robots as helpers or companions from a point of view of affective processes and the complexity of the role such artificial entities might take on for the social and affective life of their humans. I started the presentation with reading a short piece of (my own) fiction – basically a no-no at a serious scientific conference. Having said that, ISRE is a rather special group of people that agree that emotions are processes of such complexity that it requires serious interdisciplinarity – and this bridges not only the social, natural, and behavioral sciences, but also engineering and the humanities. In that spirit I jumped into the cold water and read the vacuum diaries – which are here republished for the first time:

Arvid Kappas – The Vacuum Cleaner Diaries (2005)

Tuesday: I return late from work and find a note – the package has finally arrived! It has been three weeks that I waited for my new VacBot. The post man left the box with the neighbors. It is too late to try everything out – I guess I will just open the box and charge my new high tech gadget, a RoboVac 2400 intelligent vacuum cleaner with EmoTronics. The cat throws a skeptical look at the box! The robot is a shiny red disk with wheels at the bottom, diameter that of a large pizza. Oh yes, Pizza. I must get something to eat …

Wednesday: Today is the day … I had started reading the instruction documents, of which there were very few. The RoboVac has all of the documentation built in and negotiated installation of all software with the wireless house net – so some of it actually lives in my house PCs. First came the general greeting session where RoboVac gets to know me, what I look like and what I sound like. In turn I am greeted with a series of happy and not so happy bleeping and whistling to familiarize myself with the basic messages. As far as I remember, the company did some pretests and the last generation of talking VacBots was a major flop. OK – the setup is complete and the info sounds all rather clear, I guess it is off for the ‘bot to go and explore the apartment …

Thursday: I must say, it was really fun to watch the little thing explore the place. Cleaning should start in earnest today. When I came home today, the little guy came out of the kitchen and upon seeing me spun a few times around shooting off a cheerful set of bleeps and beeps and then returned to his work. Fantastic! I could not resist spying a little bit. Oh yes, that soup spot on the kitchen floor. Up and down, left and right he goes. This is determination I would say.  I remember that part of the EmoTronics set-up is a basic set of motivations and emotions – for example, if something is particularly difficult to clean, it is particularly rewarding to get rid of the dirt – another powerful reinforcer is if I praise him. “Well done VacBot! Well done! … of course a little dance and song and he continues. I must say that I start to enjoy the occasional beeping in the background underlining success in the fight against some dust bunnies while I correct some student papers in my office.

Friday: Odd. The little guy was spending some time again with the soup spot in the kitchen which seems to have long disappeared. I believe this is where I praised him. Mmmh.  “VacBot!” I tell the spinning disc “Well done – now move on” – of course, he responds with the sing and dance. I wonder how the cat is dealing with all of this. She has been a bit quiet in the last few days.

I also saw that if a place is associated with some problems, the VacBot will be very careful – in the corner over there I had leaned an umbrella which had fallen over when the ‘bot bumped against it … now I can see that this is a corner that is negotiated with utmost care – even if I have taken the umbrella away. Anyway … now he is recharging at his station.

Saturday: Today I will spend some more time with my new gadget. Where is it? Curious! “VacBot!! Identify your location!” I yell. A low beep bleep beep is coming out of the bathroom. What is going on? Did the wheels get stuck? When I checked everything out I saw that the little guy was repeatedly approaching the door from the inside, but turned away every time approaching the threshold. Interesting. I swear – the pattern looks like an approach – avoidance conflict. I guess I have to have a look at the diagnostics. After breakfast however! Ah, upon hearing me dropping corn flakes on the floor he makes it out of the bathroom. I will check the diagnostics in any case.

*** I can’t believe it! After trying all of the diagnostics and not finding a fault, I checked the surveillance video the VacBot constantly records and sends to the house network. I have found the problem. At 4.32 last night the ‘bot was attacked by my cat. Beaten up so to say. So that was the problem. The little machine was scared to leave the bathroom. This is silly. I understand that it is functional for my vacuum cleaner to be careful when something … ahem … dangerous could happen, but this is silly. And now. No. He is in the kitchen again – the soup spot. I guess I should have been careful with my praises. Argh. And yet. When he sees me he does a little dance routine and beeps and whistles a little tune. Do you think he is maybe relieved because in my presence the cat won’t attack?

Sunday: After some more interesting side effects of the EmoTronics which would take too long now to write down I have come to the conclusion that I am better off resetting the software to the previous version. Autonomous, Intelligent. But no Emotions! Who needs a vacuum cleaner that is afraid of your cat?

Monday: This is the end of spending my first week with the VacBot and I got used to him. No sing and dance when I get home now. No rushing into the kitchen to get praise for the non-existent soup spot. Sad. Sort of.

Tuesday: I have given up and reactivated the EmoTronics. We watched a movie tonight together. Terminator V! I know that Marvin – I had to give my ‘bot  a name after all – does not enjoy the movie itself, but he likes to be near me and I drop some popcorn on the floor from time to time which he sucks up because he likes it clean and he likes to please me. “Good boy. Good boy.” He utters some happy bleeps and beeps. The cat is sitting resigned on my lap. I am smiling.

Cover of "Wall-E (Single-Disc Edition)"

Cover of Wall-E (Single-Disc Edition)

In 2005, Roombas and similar devices only started to appear, but had not yet reached a large degree of public attention. But of course, the talk was not about little disc-shaped vacuum cleaners, but a complex tale of empathy and interaction. It was important for me to underline that intelligent artificial companions would not need to be humanoid. They would not even need a face, such as Pixar’s Wall-E (2008). As I said in the Nautilus Interview (see previous post)

It is not surprising that we have empathy to robots. Everybody understands that they might react strongly to R2-D2 being trashed with a sledgehammer.

One of the key research issues that my group and many colleagues in the field of affective computing are trying to illuminate today is, what are the minimal conditions for human attachment to artificial entities? In my case I am interested in this question because I want to understand how humans link to each other and why and how they interact with each other. This is important, because demonstrating that very simple system properties give us the sense that the other is sentient and links with us in some way has the capacity to change the way we look at human interaction. I will give an example from social psychology: One of the earliest topics in social psychology was the study of the phenomenon of social facilitation. The fact that in the presence of others many behaviors are accelerated or amplified. To explain the effects that had been demonstrated in many empirical studies, complex theories were developed. However, in an attempt to demonstrate that the explanation might not be so complicated after all, the famous social psychologist Robert Zajonc demonstrated in 1965 related social effects with cockroaches (!). Of course, if cockroaches show social phenomena, it is not necessary to use much brain to compare yourself with representations of others, in the light of public expectations yada yada yada. While the experiment and its interpretation are not without critics, it is resarch like this that causes a shift in thinking how we can conceive of social phenomena. In a similar way, it might be that the feeling of being connected with a human is not due to complex cognitive processes, but a handful of simple features, such as behavioral synchronicity. These were the candidates I touched upon in the vacuum cleaner talk:

Candidates for minimal requirements for human attachment to robots (Kappas, 2005)

  • Morphological features?
  • Emotional expressions?
  • Intelligence?
  • Reactivity?
  • Synchronicity?

What makes this question so interesting is that there are issues that complicate the matter – in procedural terms, they interact with the features of the artificial systems, such as properties of the human (age, socio-cultural aspects of how they were brought up), experience, and of course the specific situational context. These are psychological aspects that make life complicated for the engineer who wants to design a machine and they make life interesting for the behavioral scientist.


Kappas. A. (2005). My happy vacuum cleaner. Presented at the XIVth Conference of the International Society for Research on Emotions, Bari, Italy (July 2005).

Zajonc, R.B., Heingartner, A., & Herman, E.M. (1969). Social enhancement and impairment of performance in the cockroach. Journal of Personality and Social Psychology, 13, 83-92.