0 22

Cited 0 times in

Improving Medical Speech-to-Text Accuracy using Vision-Language Pre-training Models

Authors
 Jaeyoung Huh  ;  Sangjoon Park  ;  Jeong Eun Lee  ;  Jong Chul Ye 
Citation
 IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, Vol.28(3) : 1692-1703, 2024-03 
Journal Title
IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS
ISSN
 2168-2194 
Issue Date
2024-03
MeSH
Humans ; Language ; Speech* ; Voice*
Abstract
Automatic Speech Recognition (ASR) is a technology that converts spoken words into text, facilitating interaction between humans and machines. One of the most common applications of ASR is Speech-To-Text (STT) technology, which simplifies user workflows by transcribing spoken words into text. In the medical field, STT has the potential to significantly reduce the workload of clinicians who rely on typists to transcribe their voice recordings. However, developing an STT model for the medical domain is challenging due to the lack of sufficient speech and text datasets. To address this issue, we propose a medical-domain text correction method that modifies the output text of a general STT system using the Vision Language Pre-training (VLP) method. VLP combines textual and visual information to correct text based on image knowledge. Our extensive experiments demonstrate that the proposed method offers quantitatively and clinically significant improvements in STT performance in the medical field. We further show that multi-modal understanding of image and text information outperforms single-modal understanding using only text information.
Full Text
https://ieeexplore.ieee.org/document/10372102
DOI
10.1109/jbhi.2023.3345897
Appears in Collections:
1. College of Medicine (의과대학) > Dept. of Radiation Oncology (방사선종양학교실) > 1. Journal Papers
Yonsei Authors
Park, Sang Joon(박상준)
URI
https://ir.ymlib.yonsei.ac.kr/handle/22282913/199157
사서에게 알리기
  feedback

qrcode

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Browse

Links