实体链接杂记

单实体链接,打分排序模型

源文档AFP_ENG_20090605.036内容:



文档AFP_ENG_20090605.036里有15个待链接实体。



15个待链接实体中,其中有很多同名的,去重过后只有9个待链接实体:



在同一篇文章内,进行待链接实体扩展,扩展如下:

Brown------扩展为--------Gordon Brown

Johnson------扩展为--------Alan Johnson

扩展之后,剩下的待链接实体还有7个:



待链接实体基于文档,进行实体扩展后,可以减少待链接实体


———————————————————————————————————————

候选集生成:

①字符串匹配查找--------------分数1----------最小编辑距离----

Gordon Brown    2个

Alan Johnson    2个

Sky News    7个

Jacqui Smith    1个

Labour     119个

British   1081个,并且字符匹配找不到答案,一般的做法是基于经验知识,构建同义词字典。所以Britsh可以等价于United Kingdom  United Kingdom在知识库在字符匹配查找可找到272个

LONDON   1114个


②语义查找--------------分数2-----------余弦相似度----

[if !supportLists]1) [endif]构建同义词字典的方法

British   1081个,并且字符匹配找不到答案,一般的做法是基于经验知识,构建同义词字典。所以Britsh可以等价于United Kingdom  United Kingdom在知识库在字符匹配查找可找到272个

2)词向量的方法



假设通过最小编辑距离、词向量余弦相似度等方法,每种方法设定一个阈值,筛选或加入一部分实体进来。(那两篇CNN的论文只对能链接上知识库的实体进行了实验,对于NIL空实体没有考虑,所以保证了每个待链接实体都生成30个候选实体)


假设每个实体保证5个候选集

Gordon Brown    5个

Gordon Brown----->E0455080  Gordon Brown  

分数1=1.000;分数2=

Gordon Brown----->E0416691  Gordon Brown (rugby player)

分数1=1.000;分数2=

Gordon Brown----->E0437484  Alan Brown (Worcestershire cricketer)

分数1=0.167;分数2=

Gordon Brown----->E0479533  Alan Brown (cricketer)

分数1=0.167;分数2=

Gordon Brown----->E0707815  Alan Brown (footballer)

分数1=0.167;分数2=


Alan Johnson    5个

Alan Johnson----->E0421495  Alan Johnson

分数1=1.000;分数2=

Alan Johnson----->E0180326  Alan Johnson (footballer)

分数1=1.000;分数2=

Alan Johnson----->E0303825  Gordon Johnson (musician)

分数1=0.167;分数2=

Alan Johnson----->E0379343  Jesse Johnson (actor)

分数1=0.167;分数2=

Alan Johnson----->E0384090  Ken Johnson (football player)

分数1=0.250;分数2=


Sky News    5个

Sky News----->E0716249  Sky News

分数1=1.000;分数2=

Sky News----->E0222711  Sky News Sunrise

分数1=0.111;分数2=

Sky News----->E0363541  Sky News Australia

分数1=0.091;分数2=

Sky News----->E0020143  Sky News Today

分数1=0.143;分数2=

Sky News----->E0468133  Sky News Ireland

分数1=0.111;分数2=


Jacqui Smith    5个

Jacqui Smith----->E0474577  Jacqui Smith

分数1=1.000;分数2=

Jacqui Smith----->E0476371  Jacqui Lait

分数1=0.250;分数2=

Jacqui Smith----->E0560356  Jacqui Dean

分数1=0.167;分数2=

Jacqui Smith----->E0592854  Jacqui Ainsley

分数1=0.125;分数2=

Jacqui Smith----->E0735952  Jacqui Jacoby

分数1=0.143;分数2=


Labour     5个

Labour----->E0172063  Labour Party (UK)

分数1=0.143;分数2=

Labour----->E0272874  Norwegian Labour Party

分数1=0.059;分数2=

Labour----->E0300315  Young Labour (New Zealand)

分数1=0.143;分数2=

Labour----->E0310728  Socialist Labour Party (UK)

分数1=0.059;分数2=

Labour----->E0499297  Labour Day (album)

分数1=0.200;分数2=


British   5个  

United Kingdom----->E0018102  United Kingdom

分数1=1.000;分数2=

United Kingdom----->E0037310  United Kingdom Independence Party

分数1=0.050;分数2=

United Kingdom----->E0146010  United Kingdom Special Forces

分数1=0.063;分数2=

United Kingdom----->E0547203  United Kingdom (Epcot)

分数1=1.000;分数2=

United Kingdom----->E0690051  United Kingdom (album)

分数1=1.000;分数2=


LONDON   5个

LONDON----->E0397283  London

分数1=1.000;分数2=

LONDON----->E0418367  London (2005 film)

分数1=1.000;分数2=

LONDON----->E0416741  London borough

分数1=0.111;分数2=

LONDON----->E0420961  London Rail

分数1=0.167;分数2=

LONDON----->E0477529  London Eye

分数1=0.200;分数2=


———————————————————————————————————————


知识库向量化----TransE

TransE模型是将<头实体>----<关系>----<尾实体>看作h+r=t的翻译模型



Gordon Brown    5个

Gordon Brown----->E0455080  Gordon Brown                          450423

Gordon Brown----->E0416691  Gordon Brown (rugby player)              412425     

Gordon Brown----->E0437484  Alan Brown (Worcestershire cricketer)       433013

Gordon Brown----->E0479533  Alan Brown (cricketer)                    474622

Gordon Brown----->E0707815  Alan Brown (footballer)                   700720



Gordon Brown----->E0455080  Gordon Brown                          450423



关系三元组整理有:

Gordon Brown<------class----->Prime Minister

Gordon Brown<----name----->Gordon Brown

Gordon Brown<----type----->PER

Gordon Brown<-----honorific-prefix----->The Right Honourable

Gordon Brown<-----honorific-suffix----->MP

Gordon Brown<-----order----->Prime Minister of the United Kingdom

Gordon Brown<-----monarch----->Elizabeth II

Gordon Brown<-----term_start----->27 June 2007

Gordon Brown<-----predecessor----->Tony Blair

Gordon Brown<-----order2----->Chancellor of the Exchequer

Gordon Brown<-----term_start2----->2 May 1997

Gordon Brown<-----term_end2------>27 June 2007

Gordon Brown<-----primeminister2----->Tony Blair

Gordon Brown<-----predecessor2----->Kenneth Clarke

Gordon Brown<-----successor2----->Alistair Darling

Gordon Brown<-----constituency_MP3----->Kirkcaldy and Cowdenbeath

Gordon Brown<-----constituency_MP3----->Dunfermline East

Gordon Brown<-----term_start3----->9 June 1983

Gordon Brown<-----majority3----->18,216 (43.6%)

Gordon Brown<-----predecessor3----->New constituency

Gordon Brown<-----birth_date----->20 February 1951 (1951-02-20) (age 58)

Gordon Brown<-----birth_place----->Govan

Gordon Brown<-----birth_place----->Glasgow

Gordon Brown<-----birth_place----->Scotland

Gordon Brown<-----nationality----->British

Gordon Brown<-----party----->Labour

Gordon Brown<-----spouse----->Sarah Brown

Gordon Brown<-----children----->Jennifer Jane (deceased), John Macaulay and James Fraser

Gordon Brown<-----residence----->10 Downing Street (official) North Queensferry (private)

Gordon Brown<-----alma_mater----->University of Edinburgh

Gordon Brown<-----occupation----->Politician

Gordon Brown<-----profession----->Academic

Gordon Brown<-----profession----->Journalist

Gordon Brown<-----religion----->Church of Scotland

Gordon Brown<-----website----->Prime Minister's Office Website





Gordon Brown----->E0416691  Gordon Brown (rugby player)              412425     



关系三元组整理有:

Gordon Brown<------class----->Rugby biography

Gordon Brown<------name----->Gordon Brown

Gordon Brown<------type----->PER

Gordon Brown<------"birthname----->Gordon Lamont Brown

Gordon Brown<------nickname----->Broon frae Troon

Gordon Brown<------birthyear----->1947

Gordon Brown<------birthmonth----->11

Gordon Brown<------birthday----->01

Gordon Brown<------placeofbirth----->Troon, Scotland

Gordon Brown<------deathyear----->2001

Gordon Brown<------deathmonth----->03

Gordon Brown<------deathday----->19

Gordon Brown<------placeofdeath----->Troon, Scotland

Gordon Brown<------ru_position----->Lock

Gordon Brown<------ru_nationalteam----->ScotlandBritish and Irish Lions

Gordon Brown<------ru_nationalyears----->1969-1976,1971,1974,1977

Gordon Brown<------ru_nationalcaps----->30

Gordon Brown<------ru_ntupdate----->5 March 2007

Gordon Brown<------ru_amateurclubs----->West of Scotland

Gordon Brown<------ru_amateurclubs----->Marr College FP

Gordon Brown<------ru_amupdate----->5 March 2007



Gordon Brown----->E0437484  Alan Brown (Worcestershire cricketer)       433013

阿兰·布朗(伍斯特郡板球队员)


Alan Brown<-----class----->Recent cricketer

Alan Brown<-----name----->Alan Brown

Alan Brown<-----type----->UKN

Alan Brown<-----nationality----->English

Alan Brown<-----country----->England

Alan Brown<-----country abbrev----->Eng

Alan Brown<-----batting style----->Right-handed batsman

Alan Brown<-----bowling style----->N/A

Alan Brown<-----FCs----->1

Alan Brown<-----FC runs----->0

Alan Brown<-----FC bat avg----->-

Alan Brown<-----FC 100s/50s----->0/0

Alan Brown<-----FC top score----->-

Alan Brown<-----FC balls----->0

Alan Brown<-----FC wickets----->0

Alan Brown<-----FC bowl avg----->-

Alan Brown<-----FC 5s----->0

Alan Brown<-----FC 10s----->0

Alan Brown<-----FC best bowling----->-

Alan Brown<-----FC catches/stumpings----->2/0

Alan Brown<-----List As----->1

Alan Brown<-----List A runs----->0

Alan Brown<-----List A bat avg----->-

Alan Brown<-----List A 100s/50s----->0/0

Alan Brown<-----List A top score----->-

Alan Brown<-----List A balls----->0

Alan Brown<-----List A wickets----->0

Alan Brown<-----List A bowl avg----->-

Alan Brown<-----List A 5s----->0

Alan Brown<-----List A 10s----->0

Alan Brown<-----List A best bowling----->-

Alan Brown<-----List A catches/stumpings

Alan Brown<-----debut date----->3/0

Alan Brown<-----debut year----->16 June

Alan Brown<-----last date----->1979

Alan Brown<-----last year----->19 June






Gordon Brown----->E0479533  Alan Brown (cricketer)                    474622

艾伦·布朗(板球运动员)



Alan Brown<-----class----->Historic Cricketer

Alan Brown<-----name----->Alan Brown

Alan Brown<-----type----->UKN

Alan Brown<-----nationality----->English

Alan Brown<-----country----->England

Alan Brown<-----country abbrev----->ENG

Alan Brown<-----batting style----->Right-hand bat

Alan Brown<-----bowling style----->Right-arm fast-medium

Alan Brown<-----tests----->2

Alan Brown<-----test run----->3

Alan Brown<-----test bat avg----->-

Alan Brown<-----test 100s/50s----->-/-

Alan Brown<-----test top score----->3*

Alan Brown<-----test balls----->323

Alan Brown<-----test wickets---->3

Alan Brown<-----test bowl avg----->50.00

Alan Brown<-----test 5s----->-

Alan Brown<-----test 10s----->-

Alan Brown<-----test best bowling----->3/27

Alan Brown<-----test catches/stumpings----->1/-

Alan Brown<-----FCs----->251

Alan Brown<-----FC runs----->2189

Alan Brown<-----FC bat avg----->9.72

Alan Brown<-----FC 100s/50s----->-/3

Alan Brown<-----FC top score----->81

Alan Brown<-----FC balls----->40603

Alan Brown<-----FC wickets----->743

Alan Brown<-----FC bowl avg----->24.66

Alan Brown<-----FC 5s----->26

Alan Brown<-----FC 10s----->4

Alan Brown<-----FC best bowling----->8/47

Alan Brown<-----FC catches/stumpings----->104/-

Alan Brown<-----debut date----->21 October

Alan Brown<-----debut year----->1961

Alan Brown<-----last date----->11 November

Alan Brown<-----last year----->1961

Alan Brown<-----source----->http://content-aus.cricinfo.com/ci/content/player/9249.html






Gordon Brown----->E0707815  Alan Brown (footballer)                   700702



Gordon Brown<-----class----->Football biography

Gordon Brown<-----name----->Alan Brown (footballer)

Gordon Brown<-----type----->PER

Gordon Brown<-----playername----->Alan Brown

Gordon Brown<-----fullname----->Alan Winston Brown

Gordon Brown<-----dateofbirth----->26 August 1914 (1914-08-26)

Gordon Brown<-----cityofbirth----->Corbridge

Gordon Brown<-----countryofbirth----->England

Gordon Brown<-----dateofdeath----->8 March 1996 (aged 81)

Gordon Brown<-----cityofdeath----->Barnstaple

Gordon Brown<-----countryofdeath----->England

Gordon Brown<-----height----->5 ft 11 in (1.80 m)

Gordon Brown<-----position----->Centre half

Gordon Brown<-----years----->1933–1946 1946–1948 1948–1949

Gordon Brown<-----clubs----->Huddersfield Town

Gordon Brown<-----clubs----->Burnley

Gordon Brown<-----clubs----->Notts County

Gordon Brown<-----caps(goals)----->57 (0) 88 (0) 13 (0)

Gordon Brown<-----manageryears----->1954–1957 1957–1964 1964–1968 1968–1972

Gordon Brown<-----managerclubs----->Burnley

Gordon Brown<-----managerclubs----->Sunderland

Gordon Brown<-----managerclubs----->Sheffield Wednesday

Gordon Brown<-----managerclubs----->Sunderland





源文档去重:

johnson british interior minister london june afp alan johnson confirmed friday secretary reshuffle british prime minister gordon brown wave ministerial resignations johnson former postman possible successor brown media reports jacqui smith office deals issues borders immigration brown sky news television people secretary labour government serving prime minister supporting gordon brown backing gordon brown ambition leader six ministers four cabinet ministers quit brown government

(6, 'brown')

(3, 'minister')

(3, 'johnson')

(3, 'gordon')

(2, 'secretary')

(2, 'prime')

(2, 'ministers')

(2, 'government')

(2, 'british')

保留名词的,词袋子相似度计算,使用注意力机制模型,词袋的维度会非常小,且针对每一篇文章有不同的词袋子


关系可以表示一个实体,在哪个关系数量级上,就足以区分出实体与实体之间的区别????假定10个关系


共有的关系都要保留,因为虽然都有这种关系,但是关系的值可能是不一样的,这样就具有消歧作用。且与其他实体构成关系的关系,也要保留。关系留足10个即可。剩余的关系只用关系词来代表这个实体,因为h+r=t,头实体+关系已经可以表示尾实体了。



CNN去特征提取,去对比:----------------------分数3-----------------CNN语义相似度计算--------

gordon brown

源文档抽取句子:secretary reshuffle british prime minister  wave ministerial resignations johnson former

知识库中的实体表示:

Gordon Brown----->E0455080  Gordon Brown

(Prime Minister) (PER) (Prime Minister of the United Kingdom) (Elizabeth II) (Tony Blair) (Chancellor of the Exchequer) (Alistair Darling).....


Gordon Brown----->E0455080  Gordon Brown

Gordon Brown<------class----->Prime Minister

Gordon Brown<----name----->Gordon Brown

Gordon Brown<----type----->PER

Gordon Brown<-----order----->Prime Minister of the United Kingdom

Gordon Brown<-----monarch----->Elizabeth II

Gordon Brown<-----predecessor----->Tony Blair

Gordon Brown<-----order2----->Chancellor of the Exchequer

Gordon Brown<-----primeminister2----->Tony Blair

Gordon Brown<-----predecessor2----->Kenneth Clarke

Gordon Brown<-----successor2----->Alistair Darling

Gordon Brown<-----birth_place----->Govan

Gordon Brown<-----birth_place----->Glasgow

Gordon Brown<-----birth_place----->Scotland

Gordon Brown<-----nationality----->British

Gordon Brown<-----party----->Labour

Gordon Brown<-----spouse----->Sarah Brown

Gordon Brown<-----children----->Jennifer Jane (deceased), John Macaulay and James Fraser

Gordon Brown<-----residence----->10 Downing Street (official) North Queensferry (private)

Gordon Brown<-----alma_mater----->University of Edinburgh

Gordon Brown<-----occupation----->Politician

Gordon Brown<-----profession----->Academic

Gordon Brown<-----profession----->Journalist

Gordon Brown<-----religion----->Church of Scotland    


Gordon Brown----->E0416691  Gordon Brown (rugby player)  

Gordon Brown<------class----->Rugby biography

Gordon Brown<------name----->Gordon Brown

Gordon Brown<------type----->PER

Gordon Brown<------"birthname----->Gordon Lamont Brown

Gordon Brown<------nickname----->Broon frae Troon

Gordon Brown<------placeofbirth----->Troon, Scotland

Gordon Brown<------placeofdeath----->Troon, Scotland

Gordon Brown<------ru_position----->Lock

Gordon Brown<------ru_nationalteam----->ScotlandBritish and Irish Lions

Gordon Brown<------ru_ntupdate----->5 March 2007

Gordon Brown<------ru_amateurclubs----->West of Scotland

Gordon Brown<------ru_amateurclubs----->Marr College FP




Gordon Brown----->E0437484  Alan Brown (Worcestershire cricketer)  

Alan Brown<-----class----->Recent cricketer

Alan Brown<-----name----->Alan Brown

Alan Brown<-----type----->UKN

Alan Brown<-----nationality----->English

Alan Brown<-----country----->England

Alan Brown<-----country abbrev----->Eng

Alan Brown<-----batting style----->Right-handed batsman





Gordon Brown----->E0479533  Alan Brown (cricketer)   

Alan Brown<-----class----->Historic Cricketer

Alan Brown<-----name----->Alan Brown

Alan Brown<-----type----->UKN

Alan Brown<-----nationality----->English

Alan Brown<-----country----->England

Alan Brown<-----country abbrev----->ENG

Alan Brown<-----batting style----->Right-hand bat

Alan Brown<-----bowling style----->Right-arm fast-medium




Gordon Brown----->E0707815  Alan Brown (footballer)   

Gordon Brown<-----class----->Football biography

Gordon Brown<-----name----->Alan Brown (footballer)

Gordon Brown<-----type----->PER

Gordon Brown<-----playername----->Alan Brown

Gordon Brown<-----fullname----->Alan Winston Brown

Gordon Brown<-----dateofbirth----->26 August 1914 (1914-08-26)

Gordon Brown<-----cityofbirth----->Corbridge

Gordon Brown<-----countryofbirth----->England

Gordon Brown<-----dateofdeath----->8 March 1996 (aged 81)

Gordon Brown<-----cityofdeath----->Barnstaple

Gordon Brown<-----countryofdeath----->England

Gordon Brown<-----position----->Centre half

Gordon Brown<-----clubs----->Huddersfield Town

Gordon Brown<-----clubs----->Burnley

Gordon Brown<-----clubs----->Notts County

Gordon Brown<-----managerclubs----->Burnley

Gordon Brown<-----managerclubs----->Sunderland

Gordon Brown<-----managerclubs----->Sheffield Wednesday

©著作权归作者所有,转载或内容合作请联系作者
【社区内容提示】社区部分内容疑似由AI辅助生成,浏览时请结合常识与多方信息审慎甄别。
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

相关阅读更多精彩内容

  • **2014真题Directions:Read the following text. Choose the be...
    又是夜半惊坐起阅读 11,499评论 0 23
  • rljs by sennchi Timeline of History Part One The Cognitiv...
    sennchi阅读 7,920评论 0 10
  • 9th,February,2017 Nancy Chineseyoyo Before the 1950...
    Chineseyoyo阅读 1,231评论 0 1
  • 不是 所有的花朵 都能成为 果实 不是 所有的付出 都能得到 回报 热烈绽放的 过程 心甘情愿的 付出 本身就是 ...
    端阳香草阅读 215评论 0 1
  • 刚刚看到这个新闻的时候,我觉得特别的可笑,什么时候人与人之间的相处竟然处到了如斯境界。负能量总是充斥在我们的周围,...
    陶子的西顾阅读 564评论 1 3

友情链接更多精彩内容