Simultaneous broadband image sensing and convolutional processing using van der Waals heterostructures

Efficiently processing broadband signals using convolutional neural networks (CNNs) could enhance the performance of machine learning tools for a wide range of real-time applications, including image recognition, remote sensing and environmental monitoring. However, past studies suggest that performing broadband convolutional processing computations directly in sensors is challenging, particularly when using conventional complementary metal-oxide-semiconductor (CMOS) technology, which underpins the functioning of most existing transistors.

Researchers at Huazhong University of Science and Technology and Nanjing University have recently investigated the possibility of achieving the convolutional processing of broadband signals using an alternative platform, namely van der Waals heterostructures. Their paper, published in Nature Electronicscould ultimately inform the development of better performing image recognition algorithms.

“Our paper was inspired by some our previous research works,” Tianyou Zhai, Xing Zhou and Feng Miao, three of the researchers who carried out the study, told TechXplore. “In studies published in Advanced Materials and Advanced Functional Materialswe realized type-III and type-II band-alignments in different heterostructures. Furthermore, we published a paper in Science Advanceswhere we realized a reconfigurable neural network vision sensor based on WSe₂.”

Building on their previous research efforts, Zhai, Zhou, Miao and their colleagues decided to fabricate heterostructures that could be used to process broadband signals using convolutional neural networks. They specifically created heterostructures based on PdSe₂/MoTe₂using a mechanical transfer method.

The heterostructures created by the researchers exhibit gate-tunable positive and negative photoresponses, as well as a broadband linear gate-dependent photoresponsivity. Due to their advantageous characteristics, the team was able to use their heterostructures to create photovoltaic sensors, with which they could implement different types of broadband convolutional processing.

“Under a fixed gate voltage, the photoresponse modulation under different photon energies determines the wavelength-dependent convolutional characteristics,” Zhai, Zhou and Miao explained. “In addition, the photoresponse depends on the gate voltage, and modulation of the gate voltage can be used to realize the configuration of different convolution kernels to achieve different operations on remote sensing images.”

As an initial proof-of-concept, Zhai, Zhou, Miao and their colleagues used a single device based on their heterostructures to receive pixel images individually and then perform broadband convolutional processing on these images. In the future, however, they could also test their proposed system using two or more devices.

The researchers are among the first to perform broadband image recognition directly in-sensor. Their findings are highly promising, as their solution significantly outperformed conventional convolutional networks that are only capable of single-band processing.

Notably, the device created by Zhai, Zhou, Miao and their colleagues simultaneously achieved photodetection and broadband information processing. In the future, it could thus be used to develop more advanced image recognition tools, as well as alternative solutions to monitor remote environments.

“Using our newly devised, special ambipolar vdW heterostructure, we developed a new approach to realize in-sensor computing multifunctional optoelectronic devices,” Zhai, Zhou and Miao added. “We now plan to realize in-sensor-memory computing at hardware level for more complex functional requires. Meanwhile, we also plan to achieve large-scale integration of these devices to implement practical in-sensor convolutional processing.”

© 2022 Science X Network