KGP源码浅析一

AGP并不负责kotlin的代码编译，工程项目里的kotlin代码是由KGP负责编译的，本文主要是分析一下KGP的代码编译部分逻辑。本文分析的KGP插件源码版本为1.3.72

KotlinCompile

在AS里面负责编译kotlin代码的任务是compile${variant}Kotlin，执行下这个task便会开始编译项目里面的kotlin代码，这个任务本质上是个KotlinCompile对象，代码在org.jetbrains.kotlin.gradle.tasks，KotlinCompile是继承了AbstractKotlinCompile，当我们执行compile${variant}Kotlin任务时，首先会跑到AbstractKotlinCompile类的execute方法去，代码如下

@TaskAction
fun execute(inputs: IncrementalTaskInputs) {
    //省略部分代码...
    try {
        executeImpl(inputs)
    } catch (t: Throwable) {
        if (outputsBackup != null) {
            kotlinLogger.logTime("Restoring previous outputs on error") {
                outputsBackup.restoreOutputs()
            }
        }
        throw t
    }
}

execute方法做的事情十分的少，逻辑都丢给了executeImpl，executeImpl主要是负责做一些简单判断逻辑，获取工程项目里的kotlin代码以及创建编译时所需要的参数配置对象等等，代码如下

 private fun executeImpl(inputs: IncrementalTaskInputs) {
    //获取工程项目里的kotlin代码
    val sourceRoots = getSourceRoots()
    val allKotlinSources = sourceRoots.kotlinSourceFiles
    
    //省略部分代码...
    sourceRoots.log(this.name, logger)
    //创建出编译时所需要的配置对象信息.
    val args = prepareCompilerArguments()
    taskBuildDirectory.mkdirs()
    //调用callCompilerAsync准备开始编译代码
    callCompilerAsync(args, sourceRoots, ChangedFiles(inputs))
}

callCompilerAsync是个抽象方法，KotlinCompile实现了这个抽象方法，主要是负责了创建compilerRunner，初始化编译时环境以及调用compilerRunner的方法来执行代码编译，代码如下

 override fun callCompilerAsync(args: K2JVMCompilerArguments, sourceRoots: SourceRoots, changedFiles: ChangedFiles) {
    sourceRoots as SourceRoots.ForJvm

    //省略部分代码...
    val compilerRunner = compilerRunner()

    //创建编译时所需要的一些环境配置.
    val icEnv = if (incremental) {
        logger.info(USING_JVM_INCREMENTAL_COMPILATION_MESSAGE)
        IncrementalCompilationEnvironment(
            if (hasFilesInTaskBuildDirectory()) changedFiles else ChangedFiles.Unknown(),
            taskBuildDirectory,
            usePreciseJavaTracking = usePreciseJavaTracking,
            disableMultiModuleIC = disableMultiModuleIC(),
            multiModuleICSettings = multiModuleICSettings
        )
    } else null

    val environment = GradleCompilerEnvironment(
        computedCompilerClasspath, messageCollector, outputItemCollector,
        outputFiles = allOutputFiles(),
        buildReportMode = buildReportMode,
        incrementalCompilationEnvironment = icEnv,
        kotlinScriptExtensions = sourceFilesExtensions.toTypedArray()
    )
    //调用compilerRunner开始编译.
    compilerRunner.runJvmCompilerAsync(
        sourceRoots.kotlinSourceFiles,
        commonSourceSet.toList(),
        sourceRoots.javaSourceRoots,
        javaPackagePrefix,
        args,
        environment
    )
}

GradleCompilerRunner

runJvmCompilerAsync方法比较简单，它只负责一些编译参数的初始化，然后调用内部方法runCompilerAsync编译代码，代码如下

fun runJvmCompilerAsync(
    sourcesToCompile: List<File>,
    commonSources: List<File>,
    javaSourceRoots: Iterable<File>,
    javaPackagePrefix: String?,
    args: K2JVMCompilerArguments,
    environment: GradleCompilerEnvironment
) {
    args.freeArgs += sourcesToCompile.map { it.absolutePath }
    args.commonSources = commonSources.map { it.absolutePath }.toTypedArray()
    args.javaSourceRoots = javaSourceRoots.map { it.absolutePath }.toTypedArray()
    args.javaPackagePrefix = javaPackagePrefix
    runCompilerAsync(KotlinCompilerClass.JVM, args, environment)
}

runCompilerAsync内部又会创建GradleKotlinCompilerWorkArguments编译时环境配置对象，最后会调用重载runCompilerAsync方法，内部又会创建GradleKotlinCompilerWork来进行代码的编译，代码如下：


 private fun runCompilerAsync(
        compilerClassName: String,
        compilerArgs: CommonCompilerArguments,
        environment: GradleCompilerEnvironment
    ) {
    
    //省略部分代码...
    val workArgs = GradleKotlinCompilerWorkArguments(
        projectFiles = ProjectFilesForCompilation(project),
        compilerFullClasspath = environment.compilerFullClasspath,
        compilerClassName = compilerClassName,
        compilerArgs = argsArray,
        isVerbose = compilerArgs.verbose,
        incrementalCompilationEnvironment = incrementalCompilationEnvironment,
        incrementalModuleInfo = modulesInfo,
        outputFiles = environment.outputFiles.toList(),
        taskPath = task.path,
        buildReportMode = environment.buildReportMode,
        kotlinScriptExtensions = environment.kotlinScriptExtensions
    )
    TaskLoggers.put(task.path, task.logger)
    runCompilerAsync(workArgs)
}

protected open fun runCompilerAsync(workArgs: GradleKotlinCompilerWorkArguments) {
    val kotlinCompilerRunnable = GradleKotlinCompilerWork(workArgs)
    kotlinCompilerRunnable.run()
}

GradleKotlinCompilerWork

GradleKotlinCompilerWork的主要工作是根据预先的配置，来选择编译的方式，主要有三种编译方式：守护进程的增量编译、本进程的全量编译、外部进程的全量编译。代码如下：

override fun run() {
    //省略部分没用代码
    val exitCode = compileWithDaemonOrFallbackImpl(messageCollector)
    throwGradleExceptionIfError(exitCode)
}

private fun compileWithDaemonOrFallbackImpl(messageCollector: MessageCollector): ExitCode {
    //省略部分代码...
    val executionStrategy = kotlinCompilerExecutionStrategy()
    if (executionStrategy == DAEMON_EXECUTION_STRATEGY) {
        //守护进程的增量编译
        val daemonExitCode = compileWithDaemon(messageCollector)

        if (daemonExitCode != null) {
            return daemonExitCode
        } else {
            log.warn("Could not connect to kotlin daemon. Using fallback strategy.")
        }
    }

    val isGradleDaemonUsed = System.getProperty("org.gradle.daemon")?.let(String::toBoolean)
    return if (executionStrategy == IN_PROCESS_EXECUTION_STRATEGY || isGradleDaemonUsed == false) {
        //内部进程的全量编译
        compileInProcess(messageCollector)
    } else {
        //外部进程的全量编译
        compileOutOfProcess()
    }
}

值得注意的是，只有守护进程才会有增量编译的能力，默认是增量编译方式，无论是增量还是全量，最终都会调用到K2JVMCompiler的内部方法来编译kotlin代码，不一样的就是增量编译的话会做了很多文件修改的检测以及一些依赖关系的处理等等。

compileInProcess

顾名思义的，就是在本进程内完成代码的编译工作，方法代码如下：

 private fun compileInProcessImpl(messageCollector: MessageCollector): ExitCode {
    val stream = ByteArrayOutputStream()
    val out = PrintStream(stream)
    // todo: cache classloader?
    val classLoader = URLClassLoader(compilerFullClasspath.map { it.toURI().toURL() }.toTypedArray())
    val servicesClass = Class.forName(Services::class.java.canonicalName, true, classLoader)
    val emptyServices = servicesClass.getField("EMPTY").get(servicesClass)
    val compiler = Class.forName(compilerClassName, true, classLoader)

    val exec = compiler.getMethod(
        "execAndOutputXml",
        PrintStream::class.java,
        servicesClass,
        Array<String>::class.java
    )

    val res = exec.invoke(compiler.newInstance(), out, emptyServices, compilerArgs)
    val exitCode = ExitCode.valueOf(res.toString())
    processCompilerOutput(
        messageCollector,
        OutputItemsCollectorImpl(),
        stream,
        exitCode
    )
    log.logFinish(IN_PROCESS_EXECUTION_STRATEGY)
    return exitCode
}

代码比较简单，就是通过反射的方式创建出一个compiler，它是K2JVMCompiler类对象，参数compilerClassName是在调用GradleCompilerRunner的runJvmCompilerAsync方法时被赋值

runCompilerAsync(KotlinCompilerClass.JVM, args, environment)

这个KotlinCompilerClass.JVM的定义是在KotlinCompilerClass里，代码如下：

object KotlinCompilerClass {
    const val JVM = "org.jetbrains.kotlin.cli.jvm.K2JVMCompiler"
    const val JS = "org.jetbrains.kotlin.cli.js.K2JSCompiler"
    const val METADATA = "org.jetbrains.kotlin.cli.metadata.K2MetadataCompiler"
}

最后会调用了K2JVMCompiler对象的execAndOutputXml方法来进行kotlin代码的编译，execAndOutputXml方法内部会通过几次调用，最终会跟增量编译一样，走到了同一个方法入口进行kotlin代码的编译，这里暂不分析execAndOutputXml方法，等下面介绍增量编译的时候再做分析。

compileOutOfProcess

compileOutOfProcess就是在外部进程编译kotlin代码，它的实现更加简单，大概就是拼接参数，最终调起外部进程，通过调用compile tool来完成kotlin代码的编译工作，代码如下：

private fun compileOutOfProcess(): ExitCode {
    //省略部分代码...
    return try {
        runToolInSeparateProcess(compilerArgs, compilerClassName, compilerFullClasspath, log)
    } finally {
        //省略部分代码...
        }
    }
}

runToolInSeparateProcess方法内部做了一些command拼接的工作，然后拉起新进程来执行编译工作，最终通过errorStream inputStream等读取到编译结果，代码如下：

internal fun runToolInSeparateProcess(
    argsArray: Array<String>,
    compilerClassName: String,
    classpath: List<File>,
    logger: KotlinLogger
): ExitCode {
    val javaBin = System.getProperty("java.home") + File.separator + "bin" + File.separator + "java"
    val classpathString = classpath.map { it.absolutePath }.joinToString(separator = File.pathSeparator)
    val builder = ProcessBuilder(javaBin, "-cp", classpathString, compilerClassName, *argsArray)
    val messageCollector = createLoggingMessageCollector(logger)
    val process = launchProcessWithFallback(builder, DaemonReportingTargets(messageCollector = messageCollector))

    // important to read inputStream, otherwise the process may hang on some systems
    val readErrThread = thread {
        process.errorStream!!.bufferedReader().forEachLine {
            logger.error(it)
        }
    }

    if (logger is GradleKotlinLogger) {
        process.inputStream!!.bufferedReader().forEachLine {
            logger.lifecycle(it)
        }
    } else {
        process.inputStream!!.bufferedReader().forEachLine {
            println(it)
        }
    }

    readErrThread.join()

    val exitCode = process.waitFor()
    logger.logFinish(OUT_OF_PROCESS_EXECUTION_STRATEGY)
    return exitCodeFromProcessExitCode(logger, exitCode)
}

最后也附上command的一份参数值列表如下图：

compileWithDaemon

这是默认的编译方式，也是唯一一种支持增量的编译方式，compileWithDaemon会通过进程通讯的方式让守护进程来编译kotln代码，过程比较复杂，后面会单独开一篇文章来分析。

总结

compile${variant}Kotlin任务本质上是个KotlinCompile对象，编译时它首先会初始化一些参数，接着会创建GradleCompilerRunner对象，GradleCompilerRunner本质上也只是做了一些参数赋值跟初始化工作，最后会创建出GradleKotlinCompilerWork，后者会根据一些预选配置来选择走进程内的全量编译(compileInProcess)还是进程外的全量编译(compileOutOfProcess)，又或者是守护进程的增量编译等(compileWithDaemon)，最后附上一份调用琏，方便读者们来追踪源码

->.AbstractKotlinCompile.execute()
->.AbstractKotlinCompile.executeImpl()
->.KotlinCompile.callCompilerAsync()
->.GradleCompilerRunner.runJvmCompilerAsync()
->.GradleCompilerRunner.runCompilerAsync()
->.GradleCompilerRunner.runCompilerAsync(workArgs: GradleKotlinCompilerWorkArguments) 
->.GradleKotlinCompilerWork.run()
->.GradleKotlinCompilerWork.compileWithDaemonOrFallbackImpl()
->.GradleKotlinCompilerWork.compileWithDaemon()
->.GradleKotlinCompilerWork.compileInProcess()
->.GradleKotlinCompilerWork.compileOutOfProcess()