Title: | Transfer Learning of Transformers for Spoken Language Understanding |
Authors: | Švec, Jan Frémund, Adam Bulín, Martin Lehečka, Jan |
Citation: | ŠVEC, J. FRÉMUND, A. BULÍN, M. LEHEČKA, J. Transfer Learning of Transformers for Spoken Language Understanding. In Text, Speech, and Dialogue 25th International Conference, TSD 2022, Brno, Czech Republic, September 6–9, 2022, Proceedings. Cham: Springer International Publishing, 2022. s. 489-500. ISBN: 978-3-031-16269-5 , ISSN: 0302-9743 |
Issue Date: | 2022 |
Publisher: | Springer International Publishing |
Document type: | konferenční příspěvek ConferenceObject |
URI: | 2-s2.0-85139041956 http://hdl.handle.net/11025/50928 |
ISBN: | 978-3-031-16269-5 |
ISSN: | 0302-9743 |
Keywords in different language: | Wav2Vec model;Speech recognition;T5 model;Spoken language understanding |
Abstract in different language: | Pre-trained models used in the transfer-learning scenario are recently becoming very popular. Such models benefit from the availability of large sets of unlabeled data. Two kinds of such models include the Wav2Vec 2.0 speech recognizer and T5 text-to-text transformer. In this paper, we describe a novel application of such models for dialog systems, where both the speech recognizer and the spoken language understanding modules are represented as Transformer models. Such composition outperforms the baseline based on the DNN-HMM speech recognizer and CNN understanding. |
Rights: | Plný text je přístupný v rámci univerzity přihlášeným uživatelům. © Springer Nature Switzerland AG |
Appears in Collections: | Konferenční příspěvky / Conference papers (NTIS) Konferenční příspěvky / Conference Papers (KKY) OBD |
Files in This Item:
File | Size | Format | |
---|---|---|---|
Svec_Fremund_Bulin_Lehecka-Transfer_Learning_of_Transformers_TSD_2022.pdf | 286,91 kB | Adobe PDF | View/Open Request a copy |
Please use this identifier to cite or link to this item:
http://hdl.handle.net/11025/50928
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.