CIMON Gets Started as First AI-Powered Astronaut Assistant

CIMON Demonstrates Its Assistance Capabilities on the ISS

November 16, 2018
 

CIMON, the world’s first artificial intelligence-enabled astronaut assistant, has made its debut aboard the International Space Station.

The ISS’s newest crew member, developed and built in Germany, was called into action on Thursday with the command, “Wake up, CIMON!,” by German ESA astronaut Alexander Gerst, who has been living and working on the ISS since June 8.

CIMON (which stands for Crew Interactive Mobile Companion), responded to Commander Gerst by saying, “What can I do for you?” The five-kilogram, plastic orb—about the size of a medicine ball—then went through a series of operational steps, including weightless navigation, to demonstrate its capabilities.

Airbus developed and built CIMON on behalf of the German Aerospace Center (DLR) and supplied the robot’s autonomous navigation capabilities for motion planning and object recognition. IBM Watson allows CIMON to recognize faces and respond to Gerst’s spoken commands and requests with the aid of voice-controlled AI hosted in the IBM Cloud.

CIMON was able to look for Gerst’s face and make eye contact. Two cameras provide CIMON with the data it needs to perform facial recognition, while five other cameras are used for orientation and video documentation.

Demonstrating Assistance Skills

CIMON’s commissioning is a milestone two and a half years in the making for the operation team at the BIOTESC ground control centre at the University of Applied Sciences and Arts of Lucerne, Switzerland. After intensive preparation and countless training hours, CIMON emerged from the ISS’s Columbus module, where it’s been in storage since arriving on June 8.

The AI robot demonstrated its assistance skills and, on its display “face,” showed instructions for a student experiment on crystallization. The robot also flew some maneuvers, recorded a video and photos with its integrated camera, and played music.

The human aspects of the assistance system were co-developed and supervised by scientists from the Ludwig Maximilian University Hospital in Munich. CIMON’s “ears” consist of 8 microphones for directional recognition plus a directional microphone for good speech recognition. Its mouth is a loudspeaker through which it can speak and play music.

Twelve internal fans enable CIMON to move and rotate freely in all directions. In this way, CIMON can turn to a particular astronaut when addressed, nod or shake its head and follow the astronaut spatially, independently or on command.

A 50-person project team from DLR, Airbus, IBM and LMU, with funding from the German Federal Ministry of Economics and Energy, began work on CIMON in August 2016. Nearly two years later CIMON arrived at the ISS along with a crew led by Gerst, a German ESA astronaut on his second mission to the space station.