06-Mar-2020 - Technische Universität Wien

Neural Hardware for Image Recognition in Nanoseconds

Ultra-fast image sensor with a built-in neural network can be trained to recognize certain objects

Automatic image recognition is widely used today: There are computer programs that can reliably diagnose skin cancer, navigate self-driving cars, or control robots. Up to now, all this has been based on the evaluation of image data as delivered by normal cameras - and that is time-consuming. Especially when the number of images recorded per second is high, a large volume of data is generated that can hardly be handled.

Scientists at TU Wien therefore took a different approach: using a special 2D material, an image sensor was developed that can be trained to recognize certain objects. The chip represents an artificial neural network capable of learning. The data does not have to be read out and processed by a computer, but the chip itself provides information about what it is currently seeing - within nanoseconds. The work has now been presented in the scientific journal "Nature".

Learning Hardware

Neural networks are artificial systems that are similar to our brain: Nerve cells are connected to many other nerve cells. When one cell is active, this can influence the activity of neighbouring nerve cells. Artificial learning on the computer works according to exactly the same principle: A network of neurons is simulated digitally, and the strength with which one node of this network influences the other is changed until the network shows the desired behaviour.

"Typically, the image data is first read out pixel by pixel and then processed on the computer," says Thomas Mueller. "We, on the other hand, integrate the neural network with its artificial intelligence directly into the hardware of the image sensor. This makes object recognition many orders of magnitude faster."

The chip was developed and manufactured at the TU Vienna. It is based on photodetectors made of tungsten diselenide - an ultra-thin material consisting of only three atomic layers. The individual photodetectors, the "pixels" of the camera system, are all connected to a small number of output elements that provide the result of object recognition.

Learning through variable sensitivity

"In our chip, we can specifically adjust the sensitivity of each individual detector element - in other words, we can control the way the signal picked up by a particular detector affects the output signal," says Lukas Mennel, first author of the publication. "All we have to do is simply adjust a local electric field directly at the photodetector." This adaptation is done externally, with the help of a computer program. One can, for example, use the sensor to record different letters and change the sensitivities of the individual pixels step by step until a certain letter always leads exactly to a corresponding output signal. This is how the neural network in the chip is configured – making some connections in the network stronger and others weaker.

Once this learning process is complete, the computer is no longer needed. The neural network can now work alone. If a certain letter is presented to the sensor, it generates the trained output signal within 50 nanoseconds - for example, a numerical code representing the letter that the chip has just recognized.

Object detection when things have to go fast

"Our test chip is still small at the moment, but you can easily scale up the technology depending on the task you want to solve," says Thomas Mueller. "In principle, the chip could also be trained to distinguish apples from bananas, but we see its use more in scientific experiments or other specialized applications.”

The technology can be usefully applied wherever extremely high speed is required: "From fracture mechanics to particle detection - in many research areas, short events are investigated," says Thomas Mueller. "Often it is not necessary to keep all the data about this event, but rather to answer a very specific question: Does a crack propagate from left to right? Which of several possible particles has just passed by? This is exactly what our technology is good for."

Facts, background information, dossiers

  • artificial neural networks

More about TU Wien

  • News

    New Biochip Technology for Pharma Research

    In pharmaceutical research, small tissue spheres are used as mini-organ models for reproducible tests. TU Wien has found a way to develop a reliable standard for these tissue samples. Before drugs are tested in clinical trials, they must be tested either by animal experiments or, more recen ... more

    Nanoparticles: The Complex Rhythm of Chemistry

    Most of commercial chemicals are produced using catalysts. Usually, these catalysts consist of tiny metal nanoparticles that are placed on an oxidic support. Similar to a cut diamond, whose surface consists of different facets oriented in different directions, a catalytic nanoparticle also ... more

    Tracking down the tiniest of forces: how T cells detect invaders

    T-cells play a central role in our immune system: by means of their so-called T-cell receptors (TCR) they make out dangerous invaders or cancer cells in the body and then trigger an immune reaction. On a molecular level, this recognition process is still not sufficiently understood. Intrigu ... more

  • q&more articles


    The aim of personalized medicine (or precision medicine) is to take patients’ personal features into consideration as much as possible for their medical treatment, thereby going beyond the functional diagnosis of the disease. A promising concept that is gaining ever more attention and showi ... more

  • Authors

    Sarah Spitz

    Sarah Spitz, born in 1993, studied biotechnology at the University of Natural Resources and Applied Life Sciences (BOKU) in Vienna, graduating with an engineering diploma degree. While studying, she was employed for two years as a research assistant at the Department of Biotechnology (DBT) ... more

    Prof. Dr. Peter Ertl

    Peter Ertl, born in 1970, studied food and biotechnology at the University of Natural Resources and Applied Life Sciences, Vienna. He obtained a PhD in chemistry from the University of Waterloo, Canada, and subsequently spent several years as a postdoc at the University of California at Ber ... more

    Dr. Kurt Brunner

    Kurt Brunner, born in 1973 graduated in Technical Chemistry from TU Vienna before obtaining his doctorate from the University’s Institute of Chemical Engineering in 2003. While preparing his thesis, he worked on the molecular biology of fungi. Following research work conducted at the Univer ... more

q&more – the networking platform for quality excellence in lab and process

The q&more concept is to increase the visibility of recent research and innovative solutions, and support the exchange of knowledge. In the broad spectrum of subjects covered, the focus is on achieving maximum quality in highly innovative sectors. As a modern knowledge platform, q&more offers market participants one-of-a-kind networking opportunities. Cutting-edge research is presented by authors of international repute. Attractively presented in a high-quality context, and published in German and English, the original articles introduce new concepts and highlight unconventional solution strategies.

> more about q&more

q&more is supported by: