大师兄的Python源码学习笔记(十三）: Python虚拟机中的一般表达式(二)

大师兄的Python源码学习笔记(十二）: Python虚拟机中的一般表达式(一)
大师兄的Python源码学习笔记(十四）: 虚拟机中的控制流（一）

三、创建非空dict

  1           0 LOAD_CONST               0 ('pp')
              2 LOAD_CONST               1 ('qq')
              4 LOAD_CONST               2 (('puppy1', 'puppy2'))
              6 BUILD_CONST_KEY_MAP      2
              8 STORE_NAME               0 (d)

比对可以看出，与创建空dict的BUILD_MAP相比，非空dict使用了BUILD_CONST_KEY_MAP字节码。
字节码对应的虚拟机代码如下：

ceval.c

        TARGET(BUILD_CONST_KEY_MAP) {
            Py_ssize_t i;
            PyObject *map;
            PyObject *keys = TOP();
            if (!PyTuple_CheckExact(keys) ||
                PyTuple_GET_SIZE(keys) != (Py_ssize_t)oparg) {
                PyErr_SetString(PyExc_SystemError,
                                "bad BUILD_CONST_KEY_MAP keys argument");
                goto error;
            }
            map = _PyDict_NewPresized((Py_ssize_t)oparg);
            if (map == NULL) {
                goto error;
            }
            for (i = oparg; i > 0; i--) {
                int err;
                PyObject *key = PyTuple_GET_ITEM(keys, oparg - i);
                PyObject *value = PEEK(i + 1);
                err = PyDict_SetItem(map, key, value);
                if (err != 0) {
                    Py_DECREF(map);
                    goto error;
                }
            }

            Py_DECREF(POP());
            while (oparg--) {
                Py_DECREF(POP());
            }
            PUSH(map);
            DISPATCH();
        }

这段代码首先创建了一个PyDictObject。
将运行时栈中的key和value弹出，在PyDictObject中设置为键值对。
最后将PyDictObject压入运行时栈中。

四、其它一般表达式

demo.py

a=1
b=a
c=a+b
print(c)

pycparser.py

code
   argcount 0
   nlocals 0
   stacksize 2
   flags 0040
  1           0 LOAD_CONST               0 (1)
              2 STORE_NAME               0 (a)

  2           4 LOAD_NAME                0 (a)
              6 STORE_NAME               1 (b)

  3           8 LOAD_NAME                0 (a)
             10 LOAD_NAME                1 (b)
             12 BINARY_ADD
             14 STORE_NAME               2 (c)

  4          16 LOAD_NAME                3 (print)
             18 LOAD_NAME                2 (c)
             20 CALL_FUNCTION            1
             22 POP_TOP
             24 LOAD_CONST               1 (None)
             26 RETURN_VALUE
   consts
      1
      None
   names ('a', 'b', 'c', 'print')
   varnames ()
   freevars ()
   cellvars ()
   filename D:\pythonProject\parser_learn\demo.py
   name <module>
   firstlineno 1

1. 变量之间赋值运算

demo.py中第二行指令b=a变量之间赋值运算的机器码如下：

  2           4 LOAD_NAME                0 (a)
              6 STORE_NAME               1 (b)

与给变量赋值不同，变量间的赋值字节码是LOAD_NAME，对应的代码如下：

ceval.c

        TARGET(LOAD_NAME) {
            PyObject *name = GETITEM(names, oparg);
            PyObject *locals = f->f_locals;
            PyObject *v;
            if (locals == NULL) {
                PyErr_Format(PyExc_SystemError,
                             "no locals when loading %R", name);
                goto error;
            }
            if (PyDict_CheckExact(locals)) {
                v = PyDict_GetItem(locals, name);
                Py_XINCREF(v);
            }
            else {
                v = PyObject_GetItem(locals, name);
                if (v == NULL) {
                    if (!PyErr_ExceptionMatches(PyExc_KeyError))
                        goto error;
                    PyErr_Clear();
                }
            }
            if (v == NULL) {
                v = PyDict_GetItem(f->f_globals, name);
                Py_XINCREF(v);
                if (v == NULL) {
                    if (PyDict_CheckExact(f->f_builtins)) {
                        v = PyDict_GetItem(f->f_builtins, name);
                        if (v == NULL) {
                            format_exc_check_arg(
                                        PyExc_NameError,
                                        NAME_ERROR_MSG, name);
                            goto error;
                        }
                        Py_INCREF(v);
                    }
                    else {
                        v = PyObject_GetItem(f->f_builtins, name);
                        if (v == NULL) {
                            if (PyErr_ExceptionMatches(PyExc_KeyError))
                                format_exc_check_arg(
                                            PyExc_NameError,
                                            NAME_ERROR_MSG, name);
                            goto error;
                        }
                    }
                }
            }
            PUSH(v);
            DISPATCH();
        }

在上面的代码中，虚拟机会在当前活动的PyFrameObject对象中所维护的多个名字空间中进行搜索，顺序是f_locals -> f_globals -> f_builtins。
如果搜索到与符号对应的元素，就将该元素压入运行时栈中，否则抛出异常。
从源码中可以清楚的看出变量在不同命名空间中的调用顺序。

2. 数值运算

第三行指令c=a+b数值运算的机器码如下：

  3           8 LOAD_NAME                0 (a)
             10 LOAD_NAME                1 (b)
             12 BINARY_ADD
             14 STORE_NAME               2 (c)

在读取变量值后，机器码BINARY_ADD指令负责加法运算，对应的代码如下：

ceval.c

        TARGET(BINARY_ADD) {
            PyObject *right = POP();
            PyObject *left = TOP();
            PyObject *sum;
            /* NOTE(haypo): Please don't try to micro-optimize int+int on
               CPython using bytecode, it is simply worthless.
               See http://bugs.python.org/issue21955 and
               http://bugs.python.org/issue10044 for the discussion. In short,
               no patch shown any impact on a realistic benchmark, only a minor
               speedup on microbenchmarks. */
            if (PyUnicode_CheckExact(left) &&
                     PyUnicode_CheckExact(right)) {
                sum = unicode_concatenate(left, right, f, next_instr);
                /* unicode_concatenate consumed the ref to left */
            }
            else {
                sum = PyNumber_Add(left, right);
                Py_DECREF(left);
            }
            Py_DECREF(right);
            SET_TOP(sum);
            if (sum == NULL)
                goto error;
            DISPATCH();
        }

这段代码首先从运行时栈弹出值，并创建了两个对应的PyObject。
同时，创建了一个空的PyObject用于记录结果。
之后判断对象是否是字符串，如果是字符串，则执行字符串的拼接unicode_concatenate。
如果不是字符串，则进行加法操作PyNumber_Add，并返回结果。
具体实现加法的底层代码如下：

Objects\abstract.c

PyObject *
PyNumber_Add(PyObject *v, PyObject *w)
{
    PyObject *result = binary_op1(v, w, NB_SLOT(nb_add));
    if (result == Py_NotImplemented) {
        PySequenceMethods *m = v->ob_type->tp_as_sequence;
        Py_DECREF(result);
        if (m && m->sq_concat) {
            return (*m->sq_concat)(v, w);
        }
        result = binop_type_error(v, w, "+");
    }
    return result;
}

Objects\abstract.c

static PyObject *
binary_op1(PyObject *v, PyObject *w, const int op_slot)
{
    PyObject *x;
    binaryfunc slotv = NULL;
    binaryfunc slotw = NULL;

    if (v->ob_type->tp_as_number != NULL)
        slotv = NB_BINOP(v->ob_type->tp_as_number, op_slot);
    if (w->ob_type != v->ob_type &&
        w->ob_type->tp_as_number != NULL) {
        slotw = NB_BINOP(w->ob_type->tp_as_number, op_slot);
        if (slotw == slotv)
            slotw = NULL;
    }
    if (slotv) {
        if (slotw && PyType_IsSubtype(w->ob_type, v->ob_type)) {
            x = slotw(v, w);
            if (x != Py_NotImplemented)
                return x;
            Py_DECREF(x); /* can't do it */
            slotw = NULL;
        }
        x = slotv(v, w);
        if (x != Py_NotImplemented)
            return x;
        Py_DECREF(x); /* can't do it */
    }
    if (slotw) {
        x = slotw(v, w);
        if (x != Py_NotImplemented)
            return x;
        Py_DECREF(x); /* can't do it */
    }
    Py_RETURN_NOTIMPLEMENTED;
}

可以看出，Python计算加法的步骤还是很多的。
除了加法外，还有以下机器码对应不同的二元操作指令:

指令	功能
BINARY_POWER	乘方
BINARY_MULTIPLY	乘法
BINARY_MATRIX_MULTIPLY	矩阵乘法
BINARY_FLOOR_DIVIDE	除法，结果向下取整
BINARY_TRUE_DIVIDE	除法
BINARY_MODULO	取余
BINARY_ADD	加法
BINARY_SUBTRACT	减法
BINARY_SUBSCR	数组取下标，栈顶为下标
BINARY_LSHIFT	左移操作符（乘2）
BINARY_RSHIFT	右移操作符（除2向下取整）
BINARY_AND	按位与
BINARY_XOR	异或
BINARY_OR	按位或
STORE_SUBSCR	列表下标存储
DELETE_SUBSCR	按下标删除元素

3. 信息输出

最后一句print(c)的字节码如下：

  4          16 LOAD_NAME                3 (print)
             18 LOAD_NAME                2 (c)
             20 CALL_FUNCTION            1
             22 POP_TOP

这段代码首先通过LOAD_NAME依次将指令字符串print和变量c依次塞入运行时栈。
之后的CALL_FUNCTION指令对应的代码如下：

ceval.c
     
    ...
    PyObject **stack_pointer;  /* Next free slot in value stack */
    ...
    int oparg;         /* Current opcode argument, if any */
    ...
        PREDICTED(CALL_FUNCTION);
        TARGET(CALL_FUNCTION) {
            PyObject **sp, *res;
            sp = stack_pointer;
            res = call_function(&sp, oparg, NULL);
            stack_pointer = sp;
            PUSH(res);
            if (res == NULL) {
                goto error;
            }
            DISPATCH();
        }

在这里，我们首先看PREDICTED(CALL_FUNCTION):

ceval.c

#define PREDICT(op) \
   do{ \
       _Py_CODEUNIT word = *next_instr; \
       opcode = _Py_OPCODE(word); \
       if (opcode == op){ \
           oparg = _Py_OPARG(word); \
           next_instr++; \
           goto PRED_##op; \
       } \
   } while(0)
#endif
#define PREDICTED(op)           PRED_##op:

这个宏判定下一条指令是否为op, 如果是则跳转到该 op 分支的地方执行，用于提升代码效率。

在这之后，程序将print方法作为参数调用了call_function函数。

ceval.c

Py_LOCAL_INLINE(PyObject *) _Py_HOT_FUNCTION
call_function(PyObject ***pp_stack, Py_ssize_t oparg, PyObject *kwnames)
{
   PyObject **pfunc = (*pp_stack) - oparg - 1;
   PyObject *func = *pfunc;
   PyObject *x, *w;
   Py_ssize_t nkwargs = (kwnames == NULL) ? 0 : PyTuple_GET_SIZE(kwnames);
   Py_ssize_t nargs = oparg - nkwargs;
   PyObject **stack = (*pp_stack) - nargs - nkwargs;

   /* Always dispatch PyCFunction first, because these are
      presumed to be the most frequent callable object.
   */
   if (PyCFunction_Check(func)) {
       PyThreadState *tstate = PyThreadState_GET();
       C_TRACE(x, _PyCFunction_FastCallKeywords(func, stack, nargs, kwnames));
   }
   else if (Py_TYPE(func) == &PyMethodDescr_Type) {
       PyThreadState *tstate = PyThreadState_GET();
       if (nargs > 0 && tstate->use_tracing) {
           /* We need to create a temporary bound method as argument
              for profiling.

              If nargs == 0, then this cannot work because we have no
              "self". In any case, the call itself would raise
              TypeError (foo needs an argument), so we just skip
              profiling. */
           PyObject *self = stack[0];
           func = Py_TYPE(func)->tp_descr_get(func, self, (PyObject*)Py_TYPE(self));
           if (func != NULL) {
               C_TRACE(x, _PyCFunction_FastCallKeywords(func,
                                                        stack+1, nargs-1,
                                                        kwnames));
               Py_DECREF(func);
           }
           else {
               x = NULL;
           }
       }
       else {
           x = _PyMethodDescr_FastCallKeywords(func, stack, nargs, kwnames);
       }
   }
   else {
       if (PyMethod_Check(func) && PyMethod_GET_SELF(func) != NULL) {
           /* Optimize access to bound methods. Reuse the Python stack
              to pass 'self' as the first argument, replace 'func'
              with 'self'. It avoids the creation of a new temporary tuple
              for arguments (to replace func with self) when the method uses
              FASTCALL. */
           PyObject *self = PyMethod_GET_SELF(func);
           Py_INCREF(self);
           func = PyMethod_GET_FUNCTION(func);
           Py_INCREF(func);
           Py_SETREF(*pfunc, self);
           nargs++;
           stack--;
       }
       else {
           Py_INCREF(func);
       }

       if (PyFunction_Check(func)) {
           x = _PyFunction_FastCallKeywords(func, stack, nargs, kwnames);
       }
       else {
           x = _PyObject_FastCallKeywords(func, stack, nargs, kwnames);
       }
       Py_DECREF(func);
   }

   assert((x != NULL) ^ (PyErr_Occurred() != NULL));

   /* Clear the stack of the function object. */
   while ((*pp_stack) > pfunc) {
       w = EXT_POP(*pp_stack);
       Py_DECREF(w);
   }

   return x;
}

这段代码比较复杂，在这里他检查了关键字，并调用了PyFuncitonObject对象，实际上就是调用了函数。

最后CALL_FUNCTION会在结束后弹出栈顶对应参数数量的元素，但是函数不会被弹出栈，因此最后有一个POP_TOP用于弹出栈顶元素。

4. 更多一般指令

指令	作用
NOP	占位操作
POP_TOP	弹出栈顶元素
LOAD_CONST	将读取的值推入栈
LOAD_GLOBAL	将全局变量对象压入栈顶
STORE_FAST	将栈顶指令存入对应局部变量
COMPARE_OP	比较操作符
CALL_FUNCTION	调用函数
BUILD_SLICE	调用切片，跟的参数为切片的值的个数
JUMP_ABSOLUTE	向下跳转几句操作符，变量为跳转偏移量
UNARY_POSITIVE	实现 Val1 = +Val1
UNARY_NEGATIVE	实现 Val1 = -Val1
UNARY_NOT	实现 Val1 = not Val1
UNARY_INVERT	实现 Val1 = ~Val
FOR_ITER	for 循环
GET_ITER	获取迭代器（一般后面跟循环）
GET_YIELD_FROM_ITER	获取 yield 生成器