Cited 0 times in

Pseudo Multi-Modal Approach to LiDAR Semantic Segmentation

Authors
 Kyungmin Kim 
Citation
 SENSORS, Vol.24(23) : 7840, 2024-12 
Journal Title
SENSORS
Issue Date
2024-12
Keywords
LiDAR semantic segmentation ; knowledge distillation
Abstract
To improve the accuracy and reliability of LiDAR semantic segmentation, previous studies have introduced multi-modal approaches that utilize additional modalities, such as 2D RGB images, to provide complementary information. However, these methods increase the cost of data collection, sensor hardware requirements, power consumption, and computational complexity. We observed that multi-modal approaches improve the semantic alignment of 3D representations. Motivated by this observation, we propose a pseudo multi-modal approach. To this end, we introduce a novel class-label-driven artificial 2D image construction method. By leveraging the close semantic alignment between image and text features of vision-language models, artificial 2D images are synthesized by arranging LiDAR class label text features. During training, the semantic information encoded in the artificial 2D images enriches the 3D features through knowledge distillation. The proposed method significantly reduces the burden of training data collection and facilitates more effective learning of semantic relationships in the 3D backbone network. Extensive experiments on two benchmark datasets demonstrate that the proposed method improves performance by 2.2-3.5 mIoU over the baseline using only LiDAR data, achieving performance comparable to that of real multi-modal approaches.
Files in This Item:
T992024988.pdf Download
DOI
10.3390/s24237840
Appears in Collections:
1. College of Medicine (의과대학) > Dept. of Neurology (신경과학교실) > 1. Journal Papers
Yonsei Authors
Kim, Kyung Min(김경민) ORCID logo https://orcid.org/0000-0002-0261-1687
URI
https://ir.ymlib.yonsei.ac.kr/handle/22282913/202480
사서에게 알리기
  feedback

qrcode

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Browse

Links