Translation Memory Retrieval Methods

Translation Memory Similarity Metrics

所有的评价指标都获取两个输入(M与C),M是待翻译的句子,C是来自TM库中的源端候选

Percent Match

percent match.png

Weighted Percent Match

上面方法的缺点在于对所有的unigram都同等的看待,但对于译者来说,并不是所有词对他所起的帮助都是等同的,那些不确定如何翻译的部分才是最有价值的
Weighted percent match (WPM) uses inverse document frequency(IDF) as a proxy for trying to weight words based on how much value their translations are expected to provide to translators.
那些出现频率高的词往往容易翻,而那些出现频率低的词更难翻好,因此其价值也就越大


image.png

Edit Distance

上面两种方法的缺点在于都只考虑了词的覆盖度,而没有考虑词的上下文,而相同的词在不同的上下文的语境下翻译出来的结果会有很大的差别。
Thus, a TM metric that matches sentences on more than just (weighted) percentage coverage of lexical items can be expected to perform better for TM bank evaluation and retrieval.


image.png

N-Gram Precision

Although ED takes context into account, it does not emphasize local context in matching certainhigh-value words and phrases as much as metrics that capture n-gram precision between the MTBT workload sentence and candidate source-side sentences from the TMB.
有点类似BLEU值的评测
Perhaps the most important is that TM fuzzy matching has to be able to operate at a sentence-to-sentence level where as automated MT evaluation metrics such as BLEU score are intended to operate over a whole corpus.


image.png

选择算术平均会比几何平均更好,更加贴合当前的任务。
对Z值的设定是一个召回率与准确率的trade off,Z值设成1会倾向于得到更长的句子(考虑召回率),而设成0会倾向于得到更短的句子(考虑精确率),作者最后实现设置的值为0.75

Weighted N-Gram Precision

与提高PM到WPM的考虑相似,对NGP也做相同的处理


image.png

Modified Weighted N-Gram Precision

对n-gram也根据这个n做一个加权
Note that in Equation 6 each wpn contributes equally to the average. Modified Weighted NGram Precision (MWNGP) improves on WNGP by weighting the contribution of each wpn so that shorter n-grams contribute more than longer ngrams. The intuition is that for TM settings, getting more high-value shorter n-gram matches atthe expense of fewer longer n-gram matches willbe more helpful since translators will get relatively more assistance from seeing new high-value vocabulary.


image.png
©著作权归作者所有,转载或内容合作请联系作者
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

推荐阅读更多精彩内容

  • rljs by sennchi Timeline of History Part One The Cognitiv...
    sennchi阅读 12,151评论 0 10
  • 172days 今天…可闹了~ baba说怎么妳会有需要私人空间这么高级的东西~ 好吧! 厉害。
    sueva阅读 910评论 0 0
  • 八只锤子四种颜色解读征人怨
    明朗说阅读 2,801评论 0 0
  • (13)仙境一月,凡间一年 一连几天,白灵除了修炼,按照观音菩萨的吩咐,每天都要带着猫猫兔兔们练习一...
    宇小天阅读 1,241评论 0 0
  • 静坐小楼的院里 四周蝉鸣一片 伴我,吟诵如诗的夏夜 一轮孤月 映入微醉的眼帘 凄清的光,洒落在 爱人灰尘未曾沾染的...
    伶子风阅读 1,241评论 0 0