实体链接杂记

单实体链接,打分排序模型

源文档AFP_ENG_20090605.036内容:



文档AFP_ENG_20090605.036里有15个待链接实体。



15个待链接实体中,其中有很多同名的,去重过后只有9个待链接实体:



在同一篇文章内,进行待链接实体扩展,扩展如下:

Brown------扩展为--------Gordon Brown

Johnson------扩展为--------Alan Johnson

扩展之后,剩下的待链接实体还有7个:



待链接实体基于文档,进行实体扩展后,可以减少待链接实体


———————————————————————————————————————

候选集生成:

①字符串匹配查找--------------分数1----------最小编辑距离----

Gordon Brown    2个

Alan Johnson    2个

Sky News    7个

Jacqui Smith    1个

Labour     119个

British   1081个,并且字符匹配找不到答案,一般的做法是基于经验知识,构建同义词字典。所以Britsh可以等价于United Kingdom  United Kingdom在知识库在字符匹配查找可找到272个

LONDON   1114个


②语义查找--------------分数2-----------余弦相似度----

[if !supportLists]1) [endif]构建同义词字典的方法

British   1081个,并且字符匹配找不到答案,一般的做法是基于经验知识,构建同义词字典。所以Britsh可以等价于United Kingdom  United Kingdom在知识库在字符匹配查找可找到272个

2)词向量的方法



假设通过最小编辑距离、词向量余弦相似度等方法,每种方法设定一个阈值,筛选或加入一部分实体进来。(那两篇CNN的论文只对能链接上知识库的实体进行了实验,对于NIL空实体没有考虑,所以保证了每个待链接实体都生成30个候选实体)


假设每个实体保证5个候选集

Gordon Brown    5个

Gordon Brown----->E0455080  Gordon Brown  

分数1=1.000;分数2=

Gordon Brown----->E0416691  Gordon Brown (rugby player)

分数1=1.000;分数2=

Gordon Brown----->E0437484  Alan Brown (Worcestershire cricketer)

分数1=0.167;分数2=

Gordon Brown----->E0479533  Alan Brown (cricketer)

分数1=0.167;分数2=

Gordon Brown----->E0707815  Alan Brown (footballer)

分数1=0.167;分数2=


Alan Johnson    5个

Alan Johnson----->E0421495  Alan Johnson

分数1=1.000;分数2=

Alan Johnson----->E0180326  Alan Johnson (footballer)

分数1=1.000;分数2=

Alan Johnson----->E0303825  Gordon Johnson (musician)

分数1=0.167;分数2=

Alan Johnson----->E0379343  Jesse Johnson (actor)

分数1=0.167;分数2=

Alan Johnson----->E0384090  Ken Johnson (football player)

分数1=0.250;分数2=


Sky News    5个

Sky News----->E0716249  Sky News

分数1=1.000;分数2=

Sky News----->E0222711  Sky News Sunrise

分数1=0.111;分数2=

Sky News----->E0363541  Sky News Australia

分数1=0.091;分数2=

Sky News----->E0020143  Sky News Today

分数1=0.143;分数2=

Sky News----->E0468133  Sky News Ireland

分数1=0.111;分数2=


Jacqui Smith    5个

Jacqui Smith----->E0474577  Jacqui Smith

分数1=1.000;分数2=

Jacqui Smith----->E0476371  Jacqui Lait

分数1=0.250;分数2=

Jacqui Smith----->E0560356  Jacqui Dean

分数1=0.167;分数2=

Jacqui Smith----->E0592854  Jacqui Ainsley

分数1=0.125;分数2=

Jacqui Smith----->E0735952  Jacqui Jacoby

分数1=0.143;分数2=


Labour     5个

Labour----->E0172063  Labour Party (UK)

分数1=0.143;分数2=

Labour----->E0272874  Norwegian Labour Party

分数1=0.059;分数2=

Labour----->E0300315  Young Labour (New Zealand)

分数1=0.143;分数2=

Labour----->E0310728  Socialist Labour Party (UK)

分数1=0.059;分数2=

Labour----->E0499297  Labour Day (album)

分数1=0.200;分数2=


British   5个  

United Kingdom----->E0018102  United Kingdom

分数1=1.000;分数2=

United Kingdom----->E0037310  United Kingdom Independence Party

分数1=0.050;分数2=

United Kingdom----->E0146010  United Kingdom Special Forces

分数1=0.063;分数2=

United Kingdom----->E0547203  United Kingdom (Epcot)

分数1=1.000;分数2=

United Kingdom----->E0690051  United Kingdom (album)

分数1=1.000;分数2=


LONDON   5个

LONDON----->E0397283  London

分数1=1.000;分数2=

LONDON----->E0418367  London (2005 film)

分数1=1.000;分数2=

LONDON----->E0416741  London borough

分数1=0.111;分数2=

LONDON----->E0420961  London Rail

分数1=0.167;分数2=

LONDON----->E0477529  London Eye

分数1=0.200;分数2=


———————————————————————————————————————


知识库向量化----TransE

TransE模型是将<头实体>----<关系>----<尾实体>看作h+r=t的翻译模型



Gordon Brown    5个

Gordon Brown----->E0455080  Gordon Brown                          450423

Gordon Brown----->E0416691  Gordon Brown (rugby player)              412425     

Gordon Brown----->E0437484  Alan Brown (Worcestershire cricketer)       433013

Gordon Brown----->E0479533  Alan Brown (cricketer)                    474622

Gordon Brown----->E0707815  Alan Brown (footballer)                   700720



Gordon Brown----->E0455080  Gordon Brown                          450423



关系三元组整理有:

Gordon Brown<------class----->Prime Minister

Gordon Brown<----name----->Gordon Brown

Gordon Brown<----type----->PER

Gordon Brown<-----honorific-prefix----->The Right Honourable

Gordon Brown<-----honorific-suffix----->MP

Gordon Brown<-----order----->Prime Minister of the United Kingdom

Gordon Brown<-----monarch----->Elizabeth II

Gordon Brown<-----term_start----->27 June 2007

Gordon Brown<-----predecessor----->Tony Blair

Gordon Brown<-----order2----->Chancellor of the Exchequer

Gordon Brown<-----term_start2----->2 May 1997

Gordon Brown<-----term_end2------>27 June 2007

Gordon Brown<-----primeminister2----->Tony Blair

Gordon Brown<-----predecessor2----->Kenneth Clarke

Gordon Brown<-----successor2----->Alistair Darling

Gordon Brown<-----constituency_MP3----->Kirkcaldy and Cowdenbeath

Gordon Brown<-----constituency_MP3----->Dunfermline East

Gordon Brown<-----term_start3----->9 June 1983

Gordon Brown<-----majority3----->18,216 (43.6%)

Gordon Brown<-----predecessor3----->New constituency

Gordon Brown<-----birth_date----->20 February 1951 (1951-02-20) (age 58)

Gordon Brown<-----birth_place----->Govan

Gordon Brown<-----birth_place----->Glasgow

Gordon Brown<-----birth_place----->Scotland

Gordon Brown<-----nationality----->British

Gordon Brown<-----party----->Labour

Gordon Brown<-----spouse----->Sarah Brown

Gordon Brown<-----children----->Jennifer Jane (deceased), John Macaulay and James Fraser

Gordon Brown<-----residence----->10 Downing Street (official) North Queensferry (private)

Gordon Brown<-----alma_mater----->University of Edinburgh

Gordon Brown<-----occupation----->Politician

Gordon Brown<-----profession----->Academic

Gordon Brown<-----profession----->Journalist

Gordon Brown<-----religion----->Church of Scotland

Gordon Brown<-----website----->Prime Minister's Office Website





Gordon Brown----->E0416691  Gordon Brown (rugby player)              412425     



关系三元组整理有:

Gordon Brown<------class----->Rugby biography

Gordon Brown<------name----->Gordon Brown

Gordon Brown<------type----->PER

Gordon Brown<------"birthname----->Gordon Lamont Brown

Gordon Brown<------nickname----->Broon frae Troon

Gordon Brown<------birthyear----->1947

Gordon Brown<------birthmonth----->11

Gordon Brown<------birthday----->01

Gordon Brown<------placeofbirth----->Troon, Scotland

Gordon Brown<------deathyear----->2001

Gordon Brown<------deathmonth----->03

Gordon Brown<------deathday----->19

Gordon Brown<------placeofdeath----->Troon, Scotland

Gordon Brown<------ru_position----->Lock

Gordon Brown<------ru_nationalteam----->ScotlandBritish and Irish Lions

Gordon Brown<------ru_nationalyears----->1969-1976,1971,1974,1977

Gordon Brown<------ru_nationalcaps----->30

Gordon Brown<------ru_ntupdate----->5 March 2007

Gordon Brown<------ru_amateurclubs----->West of Scotland

Gordon Brown<------ru_amateurclubs----->Marr College FP

Gordon Brown<------ru_amupdate----->5 March 2007



Gordon Brown----->E0437484  Alan Brown (Worcestershire cricketer)       433013

阿兰·布朗(伍斯特郡板球队员)


Alan Brown<-----class----->Recent cricketer

Alan Brown<-----name----->Alan Brown

Alan Brown<-----type----->UKN

Alan Brown<-----nationality----->English

Alan Brown<-----country----->England

Alan Brown<-----country abbrev----->Eng

Alan Brown<-----batting style----->Right-handed batsman

Alan Brown<-----bowling style----->N/A

Alan Brown<-----FCs----->1

Alan Brown<-----FC runs----->0

Alan Brown<-----FC bat avg----->-

Alan Brown<-----FC 100s/50s----->0/0

Alan Brown<-----FC top score----->-

Alan Brown<-----FC balls----->0

Alan Brown<-----FC wickets----->0

Alan Brown<-----FC bowl avg----->-

Alan Brown<-----FC 5s----->0

Alan Brown<-----FC 10s----->0

Alan Brown<-----FC best bowling----->-

Alan Brown<-----FC catches/stumpings----->2/0

Alan Brown<-----List As----->1

Alan Brown<-----List A runs----->0

Alan Brown<-----List A bat avg----->-

Alan Brown<-----List A 100s/50s----->0/0

Alan Brown<-----List A top score----->-

Alan Brown<-----List A balls----->0

Alan Brown<-----List A wickets----->0

Alan Brown<-----List A bowl avg----->-

Alan Brown<-----List A 5s----->0

Alan Brown<-----List A 10s----->0

Alan Brown<-----List A best bowling----->-

Alan Brown<-----List A catches/stumpings

Alan Brown<-----debut date----->3/0

Alan Brown<-----debut year----->16 June

Alan Brown<-----last date----->1979

Alan Brown<-----last year----->19 June






Gordon Brown----->E0479533  Alan Brown (cricketer)                    474622

艾伦·布朗(板球运动员)



Alan Brown<-----class----->Historic Cricketer

Alan Brown<-----name----->Alan Brown

Alan Brown<-----type----->UKN

Alan Brown<-----nationality----->English

Alan Brown<-----country----->England

Alan Brown<-----country abbrev----->ENG

Alan Brown<-----batting style----->Right-hand bat

Alan Brown<-----bowling style----->Right-arm fast-medium

Alan Brown<-----tests----->2

Alan Brown<-----test run----->3

Alan Brown<-----test bat avg----->-

Alan Brown<-----test 100s/50s----->-/-

Alan Brown<-----test top score----->3*

Alan Brown<-----test balls----->323

Alan Brown<-----test wickets---->3

Alan Brown<-----test bowl avg----->50.00

Alan Brown<-----test 5s----->-

Alan Brown<-----test 10s----->-

Alan Brown<-----test best bowling----->3/27

Alan Brown<-----test catches/stumpings----->1/-

Alan Brown<-----FCs----->251

Alan Brown<-----FC runs----->2189

Alan Brown<-----FC bat avg----->9.72

Alan Brown<-----FC 100s/50s----->-/3

Alan Brown<-----FC top score----->81

Alan Brown<-----FC balls----->40603

Alan Brown<-----FC wickets----->743

Alan Brown<-----FC bowl avg----->24.66

Alan Brown<-----FC 5s----->26

Alan Brown<-----FC 10s----->4

Alan Brown<-----FC best bowling----->8/47

Alan Brown<-----FC catches/stumpings----->104/-

Alan Brown<-----debut date----->21 October

Alan Brown<-----debut year----->1961

Alan Brown<-----last date----->11 November

Alan Brown<-----last year----->1961

Alan Brown<-----source----->http://content-aus.cricinfo.com/ci/content/player/9249.html






Gordon Brown----->E0707815  Alan Brown (footballer)                   700702



Gordon Brown<-----class----->Football biography

Gordon Brown<-----name----->Alan Brown (footballer)

Gordon Brown<-----type----->PER

Gordon Brown<-----playername----->Alan Brown

Gordon Brown<-----fullname----->Alan Winston Brown

Gordon Brown<-----dateofbirth----->26 August 1914 (1914-08-26)

Gordon Brown<-----cityofbirth----->Corbridge

Gordon Brown<-----countryofbirth----->England

Gordon Brown<-----dateofdeath----->8 March 1996 (aged 81)

Gordon Brown<-----cityofdeath----->Barnstaple

Gordon Brown<-----countryofdeath----->England

Gordon Brown<-----height----->5 ft 11 in (1.80 m)

Gordon Brown<-----position----->Centre half

Gordon Brown<-----years----->1933–1946 1946–1948 1948–1949

Gordon Brown<-----clubs----->Huddersfield Town

Gordon Brown<-----clubs----->Burnley

Gordon Brown<-----clubs----->Notts County

Gordon Brown<-----caps(goals)----->57 (0) 88 (0) 13 (0)

Gordon Brown<-----manageryears----->1954–1957 1957–1964 1964–1968 1968–1972

Gordon Brown<-----managerclubs----->Burnley

Gordon Brown<-----managerclubs----->Sunderland

Gordon Brown<-----managerclubs----->Sheffield Wednesday

Gordon Brown<-----managerclubs----->Sunderland





源文档去重:

johnson british interior minister london june afp alan johnson confirmed friday secretary reshuffle british prime minister gordon brown wave ministerial resignations johnson former postman possible successor brown media reports jacqui smith office deals issues borders immigration brown sky news television people secretary labour government serving prime minister supporting gordon brown backing gordon brown ambition leader six ministers four cabinet ministers quit brown government

(6, 'brown')

(3, 'minister')

(3, 'johnson')

(3, 'gordon')

(2, 'secretary')

(2, 'prime')

(2, 'ministers')

(2, 'government')

(2, 'british')

保留名词的,词袋子相似度计算,使用注意力机制模型,词袋的维度会非常小,且针对每一篇文章有不同的词袋子


关系可以表示一个实体,在哪个关系数量级上,就足以区分出实体与实体之间的区别????假定10个关系


共有的关系都要保留,因为虽然都有这种关系,但是关系的值可能是不一样的,这样就具有消歧作用。且与其他实体构成关系的关系,也要保留。关系留足10个即可。剩余的关系只用关系词来代表这个实体,因为h+r=t,头实体+关系已经可以表示尾实体了。



CNN去特征提取,去对比:----------------------分数3-----------------CNN语义相似度计算--------

gordon brown

源文档抽取句子:secretary reshuffle british prime minister  wave ministerial resignations johnson former

知识库中的实体表示:

Gordon Brown----->E0455080  Gordon Brown

(Prime Minister) (PER) (Prime Minister of the United Kingdom) (Elizabeth II) (Tony Blair) (Chancellor of the Exchequer) (Alistair Darling).....


Gordon Brown----->E0455080  Gordon Brown

Gordon Brown<------class----->Prime Minister

Gordon Brown<----name----->Gordon Brown

Gordon Brown<----type----->PER

Gordon Brown<-----order----->Prime Minister of the United Kingdom

Gordon Brown<-----monarch----->Elizabeth II

Gordon Brown<-----predecessor----->Tony Blair

Gordon Brown<-----order2----->Chancellor of the Exchequer

Gordon Brown<-----primeminister2----->Tony Blair

Gordon Brown<-----predecessor2----->Kenneth Clarke

Gordon Brown<-----successor2----->Alistair Darling

Gordon Brown<-----birth_place----->Govan

Gordon Brown<-----birth_place----->Glasgow

Gordon Brown<-----birth_place----->Scotland

Gordon Brown<-----nationality----->British

Gordon Brown<-----party----->Labour

Gordon Brown<-----spouse----->Sarah Brown

Gordon Brown<-----children----->Jennifer Jane (deceased), John Macaulay and James Fraser

Gordon Brown<-----residence----->10 Downing Street (official) North Queensferry (private)

Gordon Brown<-----alma_mater----->University of Edinburgh

Gordon Brown<-----occupation----->Politician

Gordon Brown<-----profession----->Academic

Gordon Brown<-----profession----->Journalist

Gordon Brown<-----religion----->Church of Scotland    


Gordon Brown----->E0416691  Gordon Brown (rugby player)  

Gordon Brown<------class----->Rugby biography

Gordon Brown<------name----->Gordon Brown

Gordon Brown<------type----->PER

Gordon Brown<------"birthname----->Gordon Lamont Brown

Gordon Brown<------nickname----->Broon frae Troon

Gordon Brown<------placeofbirth----->Troon, Scotland

Gordon Brown<------placeofdeath----->Troon, Scotland

Gordon Brown<------ru_position----->Lock

Gordon Brown<------ru_nationalteam----->ScotlandBritish and Irish Lions

Gordon Brown<------ru_ntupdate----->5 March 2007

Gordon Brown<------ru_amateurclubs----->West of Scotland

Gordon Brown<------ru_amateurclubs----->Marr College FP




Gordon Brown----->E0437484  Alan Brown (Worcestershire cricketer)  

Alan Brown<-----class----->Recent cricketer

Alan Brown<-----name----->Alan Brown

Alan Brown<-----type----->UKN

Alan Brown<-----nationality----->English

Alan Brown<-----country----->England

Alan Brown<-----country abbrev----->Eng

Alan Brown<-----batting style----->Right-handed batsman





Gordon Brown----->E0479533  Alan Brown (cricketer)   

Alan Brown<-----class----->Historic Cricketer

Alan Brown<-----name----->Alan Brown

Alan Brown<-----type----->UKN

Alan Brown<-----nationality----->English

Alan Brown<-----country----->England

Alan Brown<-----country abbrev----->ENG

Alan Brown<-----batting style----->Right-hand bat

Alan Brown<-----bowling style----->Right-arm fast-medium




Gordon Brown----->E0707815  Alan Brown (footballer)   

Gordon Brown<-----class----->Football biography

Gordon Brown<-----name----->Alan Brown (footballer)

Gordon Brown<-----type----->PER

Gordon Brown<-----playername----->Alan Brown

Gordon Brown<-----fullname----->Alan Winston Brown

Gordon Brown<-----dateofbirth----->26 August 1914 (1914-08-26)

Gordon Brown<-----cityofbirth----->Corbridge

Gordon Brown<-----countryofbirth----->England

Gordon Brown<-----dateofdeath----->8 March 1996 (aged 81)

Gordon Brown<-----cityofdeath----->Barnstaple

Gordon Brown<-----countryofdeath----->England

Gordon Brown<-----position----->Centre half

Gordon Brown<-----clubs----->Huddersfield Town

Gordon Brown<-----clubs----->Burnley

Gordon Brown<-----clubs----->Notts County

Gordon Brown<-----managerclubs----->Burnley

Gordon Brown<-----managerclubs----->Sunderland

Gordon Brown<-----managerclubs----->Sheffield Wednesday

©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 204,921评论 6 478
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 87,635评论 2 381
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 151,393评论 0 338
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 54,836评论 1 277
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 63,833评论 5 368
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 48,685评论 1 281
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 38,043评论 3 399
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 36,694评论 0 258
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 42,671评论 1 300
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 35,670评论 2 321
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 37,779评论 1 332
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 33,424评论 4 321
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 39,027评论 3 307
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 29,984评论 0 19
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 31,214评论 1 260
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 45,108评论 2 351
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 42,517评论 2 343

推荐阅读更多精彩内容

  • **2014真题Directions:Read the following text. Choose the be...
    又是夜半惊坐起阅读 9,372评论 0 23
  • rljs by sennchi Timeline of History Part One The Cognitiv...
    sennchi阅读 7,289评论 0 10
  • 9th,February,2017 Nancy Chineseyoyo Before the 1950...
    Chineseyoyo阅读 1,056评论 0 1
  • 不是 所有的花朵 都能成为 果实 不是 所有的付出 都能得到 回报 热烈绽放的 过程 心甘情愿的 付出 本身就是 ...
    端阳香草阅读 194评论 0 1
  • 刚刚看到这个新闻的时候,我觉得特别的可笑,什么时候人与人之间的相处竟然处到了如斯境界。负能量总是充斥在我们的周围,...
    陶子的西顾阅读 514评论 1 3