Comparison and Adaptation of Automatic Evaluation Metrics for Quality Assessment of Re-Speaking - Wołk, Krzysztof; Koržinek, Danijel (Computer Science, 2017)
Re-speaking is a mechanism for obtaining high quality subtitles for use in livebroadcast and other public events. Because it relies on humans performing theactual re-speaking, the task of estimating the quality of the results is non-trivial.Most organisations rely on humans to perform the actual quality assessment,but purely automatic methods have been developed for other similar problems,like Machine Translation. This paper will try to compare several of thesemethods: BLEU, EBLEU, NIST, METEOR, METEOR-PL, TER and RIBES.These will then be matched to the human-derived NER metric, commonly usedin re-speaking.