PostgreSQL 源码解读(18)- 查询语句#3(SQL Parse)

本文简单介绍了PG执行SQL的流程,重点介绍了查询语句的解析(Parse)过程。

一、SQL执行流程

PG执行SQL的过程有以下几个步骤:
第一步,根据输入的SQL语句执行SQL Parse,进行词法和语法分析等,最终生成解析树;
第二步,根据解析树,执行查询逻辑/物理优化、查询重写,最终生成查询树;
第三步,根据查询树,生成执行计划;
第四步,执行器根据执行计划,执行SQL。

二、SQL解析

如前所述,PG的SQL Parse(解析)过程由函数pg_parse_query实现,在exec_simple_query函数中调用。
代码如下:

 /*
  * Do raw parsing (only).
  *
  * A list of parsetrees (RawStmt nodes) is returned, since there might be
  * multiple commands in the given string.
  *
  * NOTE: for interactive queries, it is important to keep this routine
  * separate from the analysis & rewrite stages.  Analysis and rewriting
  * cannot be done in an aborted transaction, since they require access to
  * database tables.  So, we rely on the raw parser to determine whether
  * we've seen a COMMIT or ABORT command; when we are in abort state, other
  * commands are not processed any further than the raw parse stage.
  */
 List *
 pg_parse_query(const char *query_string)
 {
     List       *raw_parsetree_list;
 
     TRACE_POSTGRESQL_QUERY_PARSE_START(query_string);
 
     if (log_parser_stats)
         ResetUsage();
 
     raw_parsetree_list = raw_parser(query_string);
 
     if (log_parser_stats)
         ShowUsage("PARSER STATISTICS");
 
 #ifdef COPY_PARSE_PLAN_TREES
     /* Optional debugging check: pass raw parsetrees through copyObject() */
     {
         List       *new_list = copyObject(raw_parsetree_list);
 
         /* This checks both copyObject() and the equal() routines... */
         if (!equal(new_list, raw_parsetree_list))
             elog(WARNING, "copyObject() failed to produce an equal raw parse tree");
         else
             raw_parsetree_list = new_list;
     }
 #endif
 
     TRACE_POSTGRESQL_QUERY_PARSE_DONE(query_string);
 
     return raw_parsetree_list;
 }
 
 /*
  * raw_parser
  *      Given a query in string form, do lexical and grammatical analysis.
  *
  * Returns a list of raw (un-analyzed) parse trees.  The immediate elements
  * of the list are always RawStmt nodes.
  */
 List *
 raw_parser(const char *str)
 {
     core_yyscan_t yyscanner;
     base_yy_extra_type yyextra;
     int         yyresult;
 
     /* initialize the flex scanner */
     yyscanner = scanner_init(str, &yyextra.core_yy_extra,
                              ScanKeywords, NumScanKeywords);
 
     /* base_yylex() only needs this much initialization */
     yyextra.have_lookahead = false;
 
     /* initialize the bison parser */
     parser_init(&yyextra);
 
     /* Parse! */
     yyresult = base_yyparse(yyscanner);
 
     /* Clean up (release memory) */
     scanner_finish(yyscanner);
 
     if (yyresult)               /* error */
         return NIL;
 
     return yyextra.parsetree;
 }
 

重要的数据结构:SelectStmt结构体

/* ----------------------
  *      Select Statement
  *
  * A "simple" SELECT is represented in the output of gram.y by a single
  * SelectStmt node; so is a VALUES construct.  A query containing set
  * operators (UNION, INTERSECT, EXCEPT) is represented by a tree of SelectStmt
  * nodes, in which the leaf nodes are component SELECTs and the internal nodes
  * represent UNION, INTERSECT, or EXCEPT operators.  Using the same node
  * type for both leaf and internal nodes allows gram.y to stick ORDER BY,
  * LIMIT, etc, clause values into a SELECT statement without worrying
  * whether it is a simple or compound SELECT.
  * ----------------------
  */
 typedef enum SetOperation
 {
     SETOP_NONE = 0,
     SETOP_UNION,
     SETOP_INTERSECT,
     SETOP_EXCEPT
 } SetOperation;
 
 typedef struct SelectStmt
 {
     NodeTag     type;
 
     /*
      * These fields are used only in "leaf" SelectStmts.
      */
     List       *distinctClause; /* NULL, list of DISTINCT ON exprs, or
                                  * lcons(NIL,NIL) for all (SELECT DISTINCT) */
     IntoClause *intoClause;     /* target for SELECT INTO */
     List       *targetList;     /* the target list (of ResTarget) */
     List       *fromClause;     /* the FROM clause */
     Node       *whereClause;    /* WHERE qualification */
     List       *groupClause;    /* GROUP BY clauses */
     Node       *havingClause;   /* HAVING conditional-expression */
     List       *windowClause;   /* WINDOW window_name AS (...), ... */
 
     /*
      * In a "leaf" node representing a VALUES list, the above fields are all
      * null, and instead this field is set.  Note that the elements of the
      * sublists are just expressions, without ResTarget decoration. Also note
      * that a list element can be DEFAULT (represented as a SetToDefault
      * node), regardless of the context of the VALUES list. It's up to parse
      * analysis to reject that where not valid.
      */
     List       *valuesLists;    /* untransformed list of expression lists */
 
     /*
      * These fields are used in both "leaf" SelectStmts and upper-level
      * SelectStmts.
      */
     List       *sortClause;     /* sort clause (a list of SortBy's) */
     Node       *limitOffset;    /* # of result tuples to skip */
     Node       *limitCount;     /* # of result tuples to return */
     List       *lockingClause;  /* FOR UPDATE (list of LockingClause's) */
     WithClause *withClause;     /* WITH clause */
 
     /*
      * These fields are used only in upper-level SelectStmts.
      */
     SetOperation op;            /* type of set op */
     bool        all;            /* ALL specified? */
     struct SelectStmt *larg;    /* left child */
     struct SelectStmt *rarg;    /* right child */
     /* Eventually add fields for CORRESPONDING spec here */
 } SelectStmt;

重要的结构体:Value

 /*----------------------
  *      Value node
  *
  * The same Value struct is used for five node types: T_Integer,
  * T_Float, T_String, T_BitString, T_Null.
  *
  * Integral values are actually represented by a machine integer,
  * but both floats and strings are represented as strings.
  * Using T_Float as the node type simply indicates that
  * the contents of the string look like a valid numeric literal.
  *
  * (Before Postgres 7.0, we used a double to represent T_Float,
  * but that creates loss-of-precision problems when the value is
  * ultimately destined to be converted to NUMERIC.  Since Value nodes
  * are only used in the parsing process, not for runtime data, it's
  * better to use the more general representation.)
  *
  * Note that an integer-looking string will get lexed as T_Float if
  * the value is too large to fit in an 'int'.
  *
  * Nulls, of course, don't need the value part at all.
  *----------------------
  */
 typedef struct Value
 {
     NodeTag     type;           /* tag appropriately (eg. T_String) */
     union ValUnion
     {
         int         ival;       /* machine integer */
         char       *str;        /* string */
     }           val;
 } Value;
 
 #define intVal(v)       (((Value *)(v))->val.ival)
 #define floatVal(v)     atof(((Value *)(v))->val.str)
 #define strVal(v)       (((Value *)(v))->val.str)
 

实现过程本节暂时搁置,先看过程执行的结果,函数pg_parse_query返回的结果是链表List,其中的元素是RawStmt,具体的结构需根据NodeTag确定(这样的做法类似于Java/C++的多态)。
测试数据

testdb=# -- 单位信息
testdb=# drop table if exists t_dwxx;
ues('Y有限公司','1002','北京市海淀区');
insert into t_dwxx(dwmc,dwbh,dwdz) values('Z有限公司','1003','广西南宁市五象区');
NOTICE:  table "t_dwxx" does not exist, skipping
DROP TABLE
testdb=# create table t_dwxx(dwmc varchar(100),dwbh varchar(10),dwdz varchar(100));
CREATE TABLE
testdb=# 
testdb=# insert into t_dwxx(dwmc,dwbh,dwdz) values('X有限公司','1001','广东省广州市荔湾区');
INSERT 0 1
testdb=# insert into t_dwxx(dwmc,dwbh,dwdz) values('Y有限公司','1002','北京市海淀区');
INSERT 0 1
testdb=# insert into t_dwxx(dwmc,dwbh,dwdz) values('Z有限公司','1003','广西南宁市五象区');
INSERT 0 1
testdb=# -- 个人信息
testdb=# drop table if exists t_grxx;
NOTICE:  table "t_grxx" does not exist, skipping
DROP TABLE
testdb=# create table t_grxx(dwbh varchar(10),grbh varchar(10),xm varchar(20),nl int);
CREATE TABLE
insert into t_grxx(dwbh,grbh,xm,nl) values('1002','903','王五',43);
testdb=# 
testdb=# insert into t_grxx(dwbh,grbh,xm,nl) values('1001','901','张三',23);
INSERT 0 1
testdb=# insert into t_grxx(dwbh,grbh,xm,nl) values('1002','902','李四',33);
INSERT 0 1
testdb=# insert into t_grxx(dwbh,grbh,xm,nl) values('1002','903','王五',43);
INSERT 0 1
testdb=# -- 个人缴费信息
testdb=# drop table if exists t_jfxx;
NOTICE:  table "t_jfxx" does not exist, skipping
DROP TABLE
testdb=# create table t_jfxx(grbh varchar(10),ny varchar(10),je float);
CREATE TABLE
testdb=# 
testdb=# insert into t_jfxx(grbh,ny,je) values('901','201801',401.30);
insert into t_jfxx(grbh,ny,je) values('901','201802',401.30);
insert into t_jfxx(grbh,ny,je) values('901','201803',401.30);
insert into t_jfxx(grbh,ny,je) values('902','201801',513.30);
insert into t_jfxx(grbh,ny,je) values('902','201802',513.30);
insert into t_jfxx(grbh,ny,je) values('902','201804',513.30);
insert into t_jfxx(grbh,ny,je) values('903','201801',372.22);
insert into t_jfxx(grbh,ny,je) values('903','201804',372.22);
testdb=# insert into t_jfxx(grbh,ny,je) values('901','201801',401.30);
INSERT 0 1
testdb=# insert into t_jfxx(grbh,ny,je) values('901','201802',401.30);
INSERT 0 1
testdb=# insert into t_jfxx(grbh,ny,je) values('901','201803',401.30);
INSERT 0 1
testdb=# insert into t_jfxx(grbh,ny,je) values('902','201801',513.10);
INSERT 0 1
testdb=# insert into t_jfxx(grbh,ny,je) values('902','201802',513.30);
INSERT 0 1
testdb=# insert into t_jfxx(grbh,ny,je) values('902','201804',513.30);
INSERT 0 1
testdb=# insert into t_jfxx(grbh,ny,je) values('903','201801',372.22);
INSERT 0 1
testdb=# insert into t_jfxx(grbh,ny,je) values('903','201804',372.22);
INSERT 0 1
testdb=# -- 获取pid
testdb=# select pg_backend_pid();
 pg_backend_pid 
----------------
           1560
(1 row)
-- 用于测试的查询语句
testdb=# select t_dwxx.dwmc,t_grxx.grbh,t_grxx.xm,t_jfxx.ny,t_jfxx.je
testdb-# from t_dwxx,t_grxx,t_jfxx
testdb-# where t_dwxx.dwbh = t_grxx.dwbh 
testdb-# and t_grxx.grbh = t_jfxx.grbh
testdb-# and t_dwxx.dwbh IN ('1001','1002')
testdb-# order by t_grxx.grbh
testdb-# limit 8;
   dwmc    | grbh |  xm  |   ny   |   je   
-----------+------+------+--------+--------
 X有限公司 | 901  | 张三 | 201801 |  401.3
 X有限公司 | 901  | 张三 | 201802 |  401.3
 X有限公司 | 901  | 张三 | 201803 |  401.3
 Y有限公司 | 902  | 李四 | 201801 |  513.1
 Y有限公司 | 902  | 李四 | 201802 |  513.3
 Y有限公司 | 902  | 李四 | 201804 |  513.3
 Y有限公司 | 903  | 王五 | 201801 | 372.22
 Y有限公司 | 903  | 王五 | 201804 | 372.22
(8 rows)

结果分析

[xdb@localhost ~]$ gdb -p 1560
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-100.el7
Copyright (C) 2013 Free Software Foundation, Inc.
...
(gdb) b pg_parse_query
Breakpoint 1 at 0x84c6c9: file postgres.c, line 615.
(gdb) c
Continuing.

Breakpoint 1, pg_parse_query (
    query_string=0x1a46ef0 "select t_dwxx.dwmc,t_grxx.grbh,t_grxx.xm,t_jfxx.ny,t_jfxx.je\nfrom t_dwxx inner join t_grxx on t_dwxx.dwbh = t_grxx.dwbh\ninner join t_jfxx on t_grxx.grbh = t_jfxx.grbh\nwhere t_dwxx.dwbh IN ('1001','100"...) at postgres.c:615
615     if (log_parser_stats)
(gdb) n
618     raw_parsetree_list = raw_parser(query_string);
(gdb) 
620     if (log_parser_stats)
(gdb) 
638     return raw_parsetree_list;
(gdb) p *(RawStmt *)(raw_parsetree_list->head.data->ptr_value)
$7 = {type = T_RawStmt, stmt = 0x1a48c00, stmt_location = 0, stmt_len = 232}
(gdb) p *((RawStmt *)(raw_parsetree_list->head.data->ptr_value))->stmt
$8 = {type = T_SelectStmt}
#转换为实际类型SelectStmt 
(gdb)  p *(SelectStmt *)((RawStmt *)(raw_parsetree_list->head.data->ptr_value))->stmt
$16 = {type = T_SelectStmt, distinctClause = 0x0, intoClause = 0x0, targetList = 0x1a47b18, 
  fromClause = 0x1a48900, whereClause = 0x1a48b40, groupClause = 0x0, havingClause = 0x0, windowClause = 0x0, 
  valuesLists = 0x0, sortClause = 0x1afd858, limitOffset = 0x0, limitCount = 0x1afd888, lockingClause = 0x0, 
  withClause = 0x0, op = SETOP_NONE, all = false, larg = 0x0, rarg = 0x0}
#设置临时变量
(gdb) set $stmt=(SelectStmt *)((RawStmt *)(raw_parsetree_list->head.data->ptr_value))->stmt
#查看结构体中的各个变量
#------------------->targetList 
(gdb) p *($stmt->targetList)
$28 = {type = T_List, length = 5, head = 0x1a47af8, tail = 0x1a48128}
#targetList有5个元素,分别对应t_dwxx.dwmc,t_grxx.grbh,t_grxx.xm,t_jfxx.ny,t_jfxx.je
#先看第1个元素
(gdb) set $restarget=(ResTarget *)($stmt->targetList->head.data->ptr_value)
(gdb) p *$restarget->val
$25 = {type = T_ColumnRef}
(gdb) p *(ColumnRef *)$restarget->val
$26 = {type = T_ColumnRef, fields = 0x1a47a08, location = 7}
(gdb) p *((ColumnRef *)$restarget->val)->fields
$27 = {type = T_List, length = 2, head = 0x1a47a88, tail = 0x1a479e8}
(gdb) p *(Node *)(((ColumnRef *)$restarget->val)->fields)->head.data->ptr_value
$32 = {type = T_String}
#fields链表的第1个元素是数据表,第2个元素是数据列
(gdb) p *(Value *)(((ColumnRef *)$restarget->val)->fields)->head.data->ptr_value
$37 = {type = T_String, val = {ival = 27556248, str = 0x1a47998 "t_dwxx"}}
(gdb) p *(Value *)(((ColumnRef *)$restarget->val)->fields)->tail.data->ptr_value
$38 = {type = T_String, val = {ival = 27556272, str = 0x1a479b0 "dwmc"}}
#其他类似

#------------------->fromClause 
(gdb) p *(Node *)($stmt->fromClause->head.data->ptr_value)
$41 = {type = T_JoinExpr}
(gdb) set $fromclause=(JoinExpr *)($stmt->fromClause->head.data->ptr_value)
(gdb) p *$fromclause
$42 = {type = T_JoinExpr, jointype = JOIN_INNER, isNatural = false, larg = 0x1a484f8, rarg = 0x1a48560, 
  usingClause = 0x0, quals = 0x1a487d0, alias = 0x0, rtindex = 0}

#------------------->whereClause 
(gdb)  p *(Node *)($stmt->whereClause)
$44 = {type = T_A_Expr}
(gdb)  p *(FromExpr *)($stmt->whereClause)
$46 = {type = T_A_Expr, fromlist = 0x1a48bd0, quals = 0x1a489d0}

#------------------->sortClause 
(gdb)  p *(Node *)($stmt->sortClause->head.data->ptr_value)
$48 = {type = T_SortBy}
(gdb)  p *(SortBy *)($stmt->sortClause->head.data->ptr_value)
$49 = {type = T_SortBy, node = 0x1a48db0, sortby_dir = SORTBY_DEFAULT, sortby_nulls = SORTBY_NULLS_DEFAULT, 
  useOp = 0x0, location = -1}

#------------------->limitCount 
(gdb)  p *(Node *)($stmt->limitCount)
$50 = {type = T_A_Const}
(gdb)  p *(Const *)($stmt->limitCount)
$51 = {xpr = {type = T_A_Const}, consttype = 0, consttypmod = 216, constcollid = 0, constlen = 8, 
  constvalue = 231, constisnull = 16, constbyval = false, location = 0}

以上为简单的数据结构介绍,下一节将详细解析parseTree的Tree结构。

三、小结

1、SQL执行流程:简单介绍了SQL的执行流程,简单分为解析、查询优化、执行计划生成和执行这四步;
2、SQL解析:解析执行后,结果存储在解析树中,分为distinctClause、intoClause、targetList等多个部分。

最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 212,029评论 6 492
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 90,395评论 3 385
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 157,570评论 0 348
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 56,535评论 1 284
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 65,650评论 6 386
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 49,850评论 1 290
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 39,006评论 3 408
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 37,747评论 0 268
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 44,207评论 1 303
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 36,536评论 2 327
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 38,683评论 1 341
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 34,342评论 4 330
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 39,964评论 3 315
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 30,772评论 0 21
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 32,004评论 1 266
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 46,401评论 2 360
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 43,566评论 2 349

推荐阅读更多精彩内容

  • 关于Mongodb的全面总结 MongoDB的内部构造《MongoDB The Definitive Guide》...
    中v中阅读 31,912评论 2 89
  • 一、背景 为了分析postgresql代码,了解其执行查询语句的过程,我采用eclipse + gdb集成调试环境...
    hemny阅读 12,168评论 2 6
  • 一、正则表达式的作用: 给定的字符串是否符合正则表达式的过滤逻辑(称作“匹配”) 可以通过正则表达式,从字符串中获...
    伯恩的遗产阅读 4,634评论 18 81
  • 早上起床,我们就开始了抱怨的一天。我们抱怨卫生间总是被隔壁住户占用着,我们抱怨没有漂亮衣服可以好好装扮自己, 我们...
    北斗夜行阅读 190评论 0 0
  • 一周过去了 可是什么都没干 感觉自己像一个保镖一样 每天跟着就好 唉 不知道做什么 什么都不会 要学的还很多 时间...
    洛维阅读 240评论 0 0