大师兄的Python源码学习笔记(二十）: 虚拟机中的函数机制(二)
大师兄的Python源码学习笔记(二十二）: 虚拟机中的类机制(一)

四、函数中局部变量的访问

• 在Python中，函数的参数和局部变量的实现机制是完全一致的。

demo.py

def f(a,b):
    c = a + b

  2           0 LOAD_FAST                0 (a)
              2 LOAD_FAST                1 (b)
              4 BINARY_ADD
              6 STORE_FAST               2 (c)
              8 LOAD_CONST               0 (None)
             10 RETURN_VALUE

• 可以看出，与函数的参数一样，局部变量也是通过LOAD_FAST和STORE_FAST来操作的。

ceval.c

        PREDICTED(STORE_FAST);
        TARGET(STORE_FAST) {
            PyObject *value = POP();
            SETLOCAL(oparg, value);
            FAST_DISPATCH();
        }

• 所以与函数的参数一样，局部变量c将被复制到PyFrameObject对象的f_localsplus中。
• 在访问局部变量时，虚拟机可以通过索引来访问f_localsplus中存储的符号对应的值对象。

五、闭包的实现

• 闭包的实现与局部变量都使用了f_localsplus。

demo.py

def outer():
    value = 'inner'
    def inner():
        print(value)
    return inner

f = outer()
f()

  1           0 LOAD_CONST               0 (<code object outer at 0x000002B59EC39150, file "demo.py", line 1>)
              2 LOAD_CONST               1 ('outer')
              4 MAKE_FUNCTION            0
              6 STORE_NAME               0 (outer)

  7           8 LOAD_NAME                0 (outer)
             10 CALL_FUNCTION            0
             12 STORE_NAME               1 (f)

  8          14 LOAD_NAME                1 (f)
             16 CALL_FUNCTION            0
             18 POP_TOP
             20 LOAD_CONST               2 (None)
             22 RETURN_VALUE

  2           0 LOAD_CONST               1 ('inner')
              2 STORE_DEREF              0 (value)

  3           4 LOAD_CLOSURE             0 (value)
              6 BUILD_TUPLE              1
              8 LOAD_CONST               2 (<code object inner at 0x000002B59EB2D150, file "demo.py", line 3>)
             10 LOAD_CONST               3 ('outer.<locals>.inner')
             12 MAKE_FUNCTION            8
             14 STORE_FAST               0 (inner)

  5          16 LOAD_FAST                0 (inner)
             18 RETURN_VALUE
         
  4           0 LOAD_GLOBAL              0 (print)
              2 LOAD_DEREF               0 (value)
              4 CALL_FUNCTION            1
              6 POP_TOP
              8 LOAD_CONST               0 (None)
             10 RETURN_VALUE

1. 创建闭包

• 在虚拟机执行CALL_FUNCTION指令时，会最终辗转进入_PyEval_EvalCodeWithName:

ceval.c

PyObject *
_PyEval_EvalCodeWithName(PyObject *_co, PyObject *globals, PyObject *locals,
           PyObject *const *args, Py_ssize_t argcount,
           PyObject *const *kwnames, PyObject *const *kwargs,
           Py_ssize_t kwcount, int kwstep,
           PyObject *const *defs, Py_ssize_t defcount,
           PyObject *kwdefs, PyObject *closure,
           PyObject *name, PyObject *qualname)
{
    PyCodeObject* co = (PyCodeObject*)_co;
    PyFrameObject *f;
    PyObject *retval = NULL;
    PyObject **fastlocals, **freevars;
    PyThreadState *tstate;
    PyObject *x, *u;
    const Py_ssize_t total_args = co->co_argcount + co->co_kwonlyargcount;
    Py_ssize_t i, n;
    PyObject *kwdict;
... ...

    /* Allocate and initialize storage for cell vars, and copy free
       vars into frame. */
    for (i = 0; i < PyTuple_GET_SIZE(co->co_cellvars); ++i) {
        PyObject *c;
        Py_ssize_t arg;
        /* Possibly account for the cell variable being an argument. */
        if (co->co_cell2arg != NULL &&
            (arg = co->co_cell2arg[i]) != CO_CELL_NOT_AN_ARG) {
            c = PyCell_New(GETLOCAL(arg));
            /* Clear the local copy. */
            SETLOCAL(arg, NULL);
        }
        else {
            c = PyCell_New(NULL);
        }
        if (c == NULL)
            goto fail;
        SETLOCAL(co->co_nlocals + i, c);
    }
... ...
}

• 我们知道co_cellvars是Code Block中内部嵌套函数所引用的局部变量名集合。
• 在这里通过SETLOCAL将co_cellvars中的东西拷贝到新创建的PyFrameObject的f_localsplus中。
• 并且会通过PyCell_New创建一个PyCellObject:

Objects\cellobject.c

PyObject *
PyCell_New(PyObject *obj)
{
    PyCellObject *op;

    op = (PyCellObject *)PyObject_GC_New(PyCellObject, &PyCell_Type);
    if (op == NULL)
        return NULL;
    op->ob_ref = obj;
    Py_XINCREF(obj);

    _PyObject_GC_TRACK(op);
    return (PyObject *)op;
}

Include\cellobject.h

typedef struct {
    PyObject_HEAD
    PyObject *ob_ref;       /* Content of the cell or NULL when empty */
} PyCellObject;

• PyCellObject只维护一个ob_ref指向一个PyObject对象。
• 而这个PyObject对象是在函数中的变量被赋值时确定指向对象：

demo.py

value = 'inner'

  2           0 LOAD_CONST               1 ('inner')
              2 STORE_DEREF              0 (value)

• 这里首先通过LOAD_CONST指令将PyStringObject对象"inner"压入运行时栈。
• 随后通过STORE_DEREF从f_localsplus中取得PyCellObject对象，并设置ob_ref。

ceval.c

case STORE_DEREF:
        {
            PyObject **freevars = (f->f_localsplus +
                                   f->f_code->co_nlocals);
            PyObject *c = freevars[oparg];
            if (PyCell_GET(c) ==  v) {
                PyCell_SET(c, NULL);
                Py_DECREF(v);
            }
            break;
        }

Include\cellobject.h

#define PyCell_SET(op, v) (((PyCellObject *)(op))->ob_ref = v)

• 这么一来, f_localsplus就有了新的变量名与变量值的对应关系。
  
• 接下来在执行inner的创建：

demo.py 

def inner():
        print(value)

  3           4 LOAD_CLOSURE             0 (value)
              6 BUILD_TUPLE              1
              8 LOAD_CONST               2 (<code object inner at 0x000002B59EB2D150, file "demo.py", line 3>)
             10 LOAD_CONST               3 ('outer.<locals>.inner')
             12 MAKE_FUNCTION            8
             14 STORE_FAST               0 (inner)

• 在这里LOAD_CLOSURE将刚刚放置好的PyCellObject对象取出并压入运行时栈：

ceval.c

        TARGET(LOAD_CLOSURE) {
            PyObject *cell = freevars[oparg];
            Py_INCREF(cell);
            PUSH(cell);
            DISPATCH();
        }

• 接着BUILD_TUPLE指令将PyCellObject对象打包进一个PyTupleObject对象中：

ceval.c

        TARGET(BUILD_TUPLE) {
            PyObject *tup = PyTuple_New(oparg);
            if (tup == NULL)
                goto error;
            while (--oparg >= 0) {
                PyObject *item = POP();
                PyTuple_SET_ITEM(tup, oparg, item);
            }
            PUSH(tup);
            DISPATCH();
        }

• 随后虚拟机通过LOAD_CONST将inner对应的PyCodeObject对象压入到运行时栈中。
• 接下来的指令MAKE_FUNCTION的参数为8，将PyCodeObject存在了PyFunctionObject中：

ceval.c

        TARGET(MAKE_FUNCTION) {
            PyObject *qualname = POP();
            PyObject *codeobj = POP();
            PyFunctionObject *func = (PyFunctionObject *)
                PyFunction_NewWithQualName(codeobj, f->f_globals, qualname);
... ...

            if (oparg & 0x08) {
                assert(PyTuple_CheckExact(TOP()));
                func ->func_closure = POP();
            }
... ...
        }

• 通过以上操作，将函数和嵌套函数间的对应关系进行了冻结, 使得在嵌套函数inner_func被调用时也能使用这层对应关系。

2. 使用闭包

  4           0 LOAD_GLOBAL              0 (print)
              2 LOAD_DEREF               0 (value)
              4 CALL_FUNCTION            1
              6 POP_TOP
              8 LOAD_CONST               0 (None)
             10 RETURN_VALUE

• 这里的关键指令是LOAD_DEREF:

ceval.c

        TARGET(LOAD_DEREF) {
            PyObject *cell = freevars[oparg];
            PyObject *value = PyCell_GET(cell);
            if (value == NULL) {
                format_exc_unbound(co, oparg);
                goto error;
            }
            Py_INCREF(value);
            PUSH(value);
            DISPATCH();
        }

Include\cellobject.h

#define PyCell_GET(op) (((PyCellObject *)(op))->ob_ref)

• 如上面的代码所示，LOAD_DEREF从PyCellObject中获取了ob_ref，并将其塞入到运行时栈中。
  

六、装饰器的实现

• 首先设定一个简单的装饰器

demo.py 

def outer(func):
    value = 'inner'
    def inner(*args):
        print(value)
        func(*args)
    return inner

@outer
def f():
    print("func")

f()

  1           0 LOAD_CONST               0 (<code object outer at 0x000001F456B590C0, file "demo.py", line 1>)
              2 LOAD_CONST               1 ('outer')
              4 MAKE_FUNCTION            0
              6 STORE_NAME               0 (outer)

  8           8 LOAD_NAME                0 (outer)
             10 LOAD_CONST               2 (<code object f at 0x000001F456B591E0, file "demo.py", line 8>)
             12 LOAD_CONST               3 ('f')
             14 MAKE_FUNCTION            0
             16 CALL_FUNCTION            1
             18 STORE_NAME               1 (f)

 12          20 LOAD_NAME                1 (f)
             22 CALL_FUNCTION            0
             24 POP_TOP
             26 LOAD_CONST               4 (None)
             28 RETURN_VALUE

  2           0 LOAD_CONST               1 ('inner')
              2 STORE_DEREF              1 (value)

  3           4 LOAD_CLOSURE             0 (func)
              6 LOAD_CLOSURE             1 (value)
              8 BUILD_TUPLE              2
             10 LOAD_CONST               2 (<code object inner at 0x000001F456A4D8A0, file "demo.py", line 3>)
             12 LOAD_CONST               3 ('outer.<locals>.inner')
             14 MAKE_FUNCTION            8
             16 STORE_FAST               1 (inner)

  6          18 LOAD_FAST                1 (inner)
             20 RETURN_VALUE

  4           0 LOAD_GLOBAL              0 (print)
              2 LOAD_DEREF               1 (value)
              4 CALL_FUNCTION            1
              6 POP_TOP

  5           8 LOAD_DEREF               0 (func)
             10 LOAD_FAST                0 (args)
             12 CALL_FUNCTION_EX         0
             14 POP_TOP
             16 LOAD_CONST               0 (None)
             18 RETURN_VALUE

 10           0 LOAD_GLOBAL              0 (print)
              2 LOAD_CONST               1 ('func')
              4 CALL_FUNCTION            1
              6 POP_TOP
              8 LOAD_CONST               0 (None)
             10 RETURN_VALUE

• 将装饰器改为闭包方式实现：

demo.py 

def outer(func):
    value = 'inner'
    def inner(*args):
        print(value)
        func(*args)
    return inner

def f():
    print("func")

outer(f)()

  1           0 LOAD_CONST               0 (<code object outer at 0x0000022135AA90C0, file "demo.py", line 1>)
              2 LOAD_CONST               1 ('outer')
              4 MAKE_FUNCTION            0
              6 STORE_NAME               0 (outer)

  8           8 LOAD_CONST               2 (<code object f at 0x0000022135AA91E0, file "demo.py", line 8>)
             10 LOAD_CONST               3 ('f')
             12 MAKE_FUNCTION            0
             14 STORE_NAME               1 (f)

 11          16 LOAD_NAME                0 (outer)
             18 LOAD_NAME                1 (f)
             20 CALL_FUNCTION            1
             22 CALL_FUNCTION            0
             24 POP_TOP
             26 LOAD_CONST               4 (None)
             28 RETURN_VALUE

  2           0 LOAD_CONST               1 ('inner')
              2 STORE_DEREF              1 (value)

  3           4 LOAD_CLOSURE             0 (func)
              6 LOAD_CLOSURE             1 (value)
              8 BUILD_TUPLE              2
             10 LOAD_CONST               2 (<code object inner at 0x000002213599D8A0, file "demo.py", line 3>)
             12 LOAD_CONST               3 ('outer.<locals>.inner')
             14 MAKE_FUNCTION            8
             16 STORE_FAST               1 (inner)

  6          18 LOAD_FAST                1 (inner)
             20 RETURN_VALUE

  4           0 LOAD_GLOBAL              0 (print)
              2 LOAD_DEREF               1 (value)
              4 CALL_FUNCTION            1
              6 POP_TOP

  5           8 LOAD_DEREF               0 (func)
             10 LOAD_FAST                0 (args)
             12 CALL_FUNCTION_EX         0
             14 POP_TOP
             16 LOAD_CONST               0 (None)
             18 RETURN_VALUE

  9           0 LOAD_GLOBAL              0 (print)
              2 LOAD_CONST               1 ('func')
              4 CALL_FUNCTION            1
              6 POP_TOP
              8 LOAD_CONST               0 (None)
             10 RETURN_VALUE

• 可以看出，主要区别就是MAKE_FUNCTION后面直接CALL_FUNCTION:

  8           8 LOAD_NAME                0 (outer)
             10 LOAD_CONST               2 (<code object f at 0x000001F456B591E0, file "demo.py", line 8>)
             12 LOAD_CONST               3 ('f')
             14 MAKE_FUNCTION            0
             16 CALL_FUNCTION            1
             18 STORE_NAME               1 (f)

• 这个区别只是装饰器强制执行了函数，这也验证了装饰器是func=decorator(func)的一种包装形式。