Recently, we are now witnessing the emergence of unprecedented services and applications using artificial intelligence (AI) such as the autonomous vehicles, drone-based deliveries, smart cities and factories, remote medical diagnosis and surgery, to name just a few. AI-based approaches are data-driven in nature, so applications using visual and audio/speech data are popular among others. In particular, computer vision (CV) technique, a field of AI that enables computers to derive meaningful information from visual data such as image and video, has achieved a remarkable success in various tasks such as the image classification, object detection, image captioning, and saliency detection. In the perspective of future wireless systems, benefits of CV are twofold: First, physical characteristics of wireless signals (in particular, mmWave and THz radio waves) are very close to the sensing signal (e.g., visible light in 400∼790 THz) in that the transmit energy is mostly concentrated in the line of sight (LoS) path. Second, recent advances of the CV techniques have made a gigantic improvement in various tasks, from which we can infer that the CV techniques can dramatically reduce the complicated control process of the wireless communication systems since the essential operation of CV-aided wireless systems is to capture the image and use AI in performing the desired task. In this tutorial, we present CV-aided future wireless systems equipped with the visual sensing mechanism (e.g., RGB, LiDAR, laser, infrared). After discussing basics of sensing devices and deep learning (DL) mechanism, we will explain the state-of-the-art CV-techniques and its applications to 6G wireless communication systems. We will also discuss the practical issues such as multi-modal sensor fusion, dataset acquisition, model training, and integration with communication systems. From our discussion, we will show that the CV technique is effective in improving the reliability and capacity, reducing the end-to-end latency and power consumption, and also operation cost of wireless systems.
Byonghyo Shim received the B.S. and M.S. degree in Control and Instrumentation Engineering (currently Electrical Eng.) from Seoul National University (SNU), Seoul, Korea, in 1995 and 1997, respectively, and the M.S. degree in Mathematics and the Ph.D. degree in Electrical and Computer Engineering from the University of Illinois at Urbana-Champaign (UIUC), Urbana, in 2004 and 2005, respectively. From 1997 and 2000, he was with the Department of Electronics Engineering at the Korean Air Force Academy as an Officer (First Lieutenant) and an Academic Full-time instructor. He also had a short time research position in the Texas Instruments and Samsung Electronics in 1997 and 2004, 2019, respectively. From 2005 to 2007, he was with the Qualcomm Inc., San Diego, CA as a Staff Engineer working on CDMA systems. From 2007 to 2014, he was with the School of Information and Communication, Korea University, Seoul, Korea, as an associate professor. Since September 2014, he has been with the Dept. of Electrical and Computer Engineering, Seoul National University, where he is currently a Professor. His research interests include signal processing for wireless communications, statistical signal processing, machine learning, compressed sensing, and information theory. Dr. Shim was the recipient of the M. E. Van Valkenburg Research Award from the ECE Department of the University of Illinois (2005), the Hadong Young Engineer Award from IEIE (2010), the Irwin Jacobs Award from Qualcomm and KICS (2016), the Shinyang Research Award from the Engineering College of SNU (2017), the Okawa Foundation Research Award (2020), and the IEEE COMSOC Asia Pacific Outstanding Paper Award (2021). He was a technical committee member of Signal Processing for Communications and Networking (SPCOM), and currently serving as an associate editor of IEEE Transactions on Signal Processing (TSP), IEEE Transactions on Communications (TCOM), IEEE Transactions on Vehicular Technology (TVT), IEEE Wireless Communications Letters (WCL), Journal of Communications and Networks (JCN), and a guest editor of IEEE Journal of Selected Areas in Communications (location awareness for radios and networks).