Emojis from the Industrial World

A user-friendly app could be used on the smartphone in future to check if machines and systems are working optimally. Siemens' researchers are combining Augmented Reality (AR) with the Industrial Internet of Things (IIOT) in a decentralized network. What's special is the display: the current operating status – all ok or alarm – is expressed with Emojis. The prototype called SIEMoji was developed by Siemens Corporate Technology together with the Technical University of Munich.


by Katrin Nikolaus

It is some 36 years since Scott Fahlman, at the time a researcher in computer science at the Carnegie Mellon University, discovered the Smiley – the first Emoji. Now the small and colorful stickers could be set to find their way into industrial warehouses and production halls. According to Jochen Nickles, this is illustrative of a trend, which looks set to gain wide-scale momentum. Systems will be tailored more toward the user and less toward the machines. Augmented Reality technology (AR) is likewise really user friendly. Digital contents are loaded into images of the actual environment, which help users to get along  better in certain situations, for instance for operating complex devices, in a computer game or in a museum.

"Augmented Reality will enable employees to close the gap between the physical and virtual world more quickly."

Closing the Gap between the Physical and Virtual World

Jochen Nickles is convinced that "Augmented Reality will enable employees to close the gap between the physical and virtual world more quickly". He works for the Company Core Technology field (CCT) Connectivity and Edge Devices (see box alongside). What is involved here is adapting systems and components to conditions that are continually changing at a rapid pace in the Industrial Internet of Things (IIoT). "A large proportion of machines in complex industrial facilities are meanwhile equipped with their own devices for connecting to the IIoT," explains Nickles. This is where the app comes in, providing status information about the devices more quickly and easily in addition to the displays on the devices, which are often only legible for experts. Application-specific communication has to be developed on an ad hoc basis in the background for this type of approach. Corporate Technology is working on a collaborative IoT framework in this respect, which was also used for the SIEMojis.

Everyone always has a Smartphone to Hand anyway

The students and Siemens researchers therefore quickly realized that the smartphone is the most suitable device "since everyone has one to hand always anyway," says the CCT expert. A typical scenario could be: when the employee enters the machine hall, the app searches independently for the local IIoT network and connects to it. The employee then directs the smartphone's camera at the system. The software detects all IoT devices – for example on pumps, motors or sensors – and identifies them on the basis of a QR code or using LEDs for instance.


It then reads the data from the devices and analyzes it. "The app uses preinstalled data models for this purpose," says Nickles. The results of the analysis are presented visually by the app – directly on the display of the smartphone whose camera is currently recording the device in question. If the Emoji is smiling, the employee knows that the motor is working properly. If there is a need for action, an Emoji with the corners of the mouth turned down is displayed. This can then imply, for example, that power consumption is exceptionally high. If the Emoji is looking aghast, this means that a device is no longer working at all.

Exploiting the Benefits of the IIoT

Rapid evaluation of the data and clearly laid out display of information: only when this is the case can the benefits of IIoT actually also be exploited according to the experts. "Information has to be customized without the engineering effort required to do this exploding," explains Franz Josef Menzl, CTO of the Factory Automation Business Unit in the Digital Factory Division. He therefore regards apps like SIEMojis as having a promising future. "The complexity of systems will continue to increase steadily," says Menzl. Despite this, employees need to know the current status at all times. "The idea of visualizing this with Emojis is clever. Other conceivable applications could include traffic signs or such like, the main thing is that everyone understands the symbols.” The underlying technology is critical, in other words the connection between the IIoT devices, the local network and the analysis software.


Nickles and the students feel vindicated by Menzl to continue developing the app. Nickles is convinced that "Our prototype offers a highly promising means of bringing man and machine closer together".


Katrin Nikolaus

Subscribe to our Newsletter

Stay up to date at all times: everything you need to know about electrification, automation, and digitalization.