（基于android12分析和测试）
一、现存问题
Android早期是aot的方式先编译成机器码，然后再运行的，这样会导致安装时间变长，后面的版本改成jit编译方式，在运行时编译，这样会导致运行速度较慢。android7.0之后的版本支持jit+oat的方式编译，支持app配置baseline-profile的方式设置热点代码。
二、理论基础
1、ART 执行方式
- 最初安装应用时不进行任何 AOT 编译。应用前几次运行时，系统会对其进行解译，并对经常执行的方法进行 JIT 编译。
- 当设备闲置和充电时，编译守护程序会运行，以便根据在应用前几次运行期间生成的配置文件对常用代码进行 AOT 编译。
-
下一次重新启动应用时将会使用配置文件引导型代码，并避免在运行时对已经过编译的方法进行 JIT 编译。在应用后续运行期间经过 JIT 编译的方法将会添加到配置文件中，然后编译守护程序将会对这些方法进行 AOT 编译。
1695197336986.png
2、编译选项
dex2oat工具可以将dex文件生成vdex，odex或者.art文件。其中：
.dex
.java->.class->.dex （apk解压后的问题）
java被编译成.class之后，使用d8工具（以前是dx）将class文件合成dex文件，dex一般是jar的50%大小。然后被打包成单个.apk文件。.dex文件可以通过自动转换用 Java 编程语言编写的编译应用程序来创建。.odex一种文件格式
.java->.class->.dex->.oat
过 AOT 编译的方法代码,ART可以直接用的机器码。
- .vdex
dex->.vdex
对dex文件进行初步优化，调用dexOpt方法，转成vdex文件（文件名后缀依然是.dex），只是小小的优化了操作码，
其中odex的文件是可以直接被运行的。生成那种类型的文件依赖dex2Oat工具，dex2Oat依赖一个核心参数“编译过滤器”
编译过滤器：(android 官方sdk android8.0之后没有再更新)
-
verify：仅运行 DEX 代码验证。 -
quicken：（从 Android 12 开始已移除）运行 DEX 代码验证，并优化一些 DEX 指令，以获得更好的解译器性能。（我在12还是看到有这样配置） -
speed：运行 DEX 代码验证，并对所有方法进行 AOT 编译。 -
speed-profile：运行 DEX 代码验证，并对配置文件中列出的方法进行 AOT 编译。
实际上最新的android 12代码中增加了一些。
//android_12/art/libartbase/base/compiler_filter.h
enum Filter {
kAssumeVerified, // Skip verification but mark all classes as verified anyway.
kExtract, // Delay verication to runtime, do not compile anything.
kVerify, // Only verify classes.
kSpaceProfile, // Maximize space savings based on profile.
kSpace, // Maximize space savings.
kSpeedProfile, // Maximize runtime performance based on profile.
kSpeed, // Maximize runtime performance.
kEverythingProfile, // Compile everything capable of being compiled based on profile.
kEverything, // Compile everything capable of being compiled.
};
//android_12/art/libartbase/base/compiler_filter.cc
std::string CompilerFilter::NameOfFilter(Filter filter) {
switch (filter) {
case CompilerFilter::kAssumeVerified: return "assume-verified";
case CompilerFilter::kExtract: return "extract";
case CompilerFilter::kVerify: return "verify";
case CompilerFilter::kSpaceProfile: return "space-profile";
case CompilerFilter::kSpace: return "space";
case CompilerFilter::kSpeedProfile: return "speed-profile";
case CompilerFilter::kSpeed: return "speed";
case CompilerFilter::kEverythingProfile: return "everything-profile";
case CompilerFilter::kEverything: return "everything";
}
UNREACHABLE();
}
从这里可以得知，quicken被移除了，如果配置了重定向到verify 。配置speed可以将所有方法进行oat，从而加速代码的运行速度。配置speed-profile，可以选择性的让配置的方法进行AOT编译。
3、强制编译命令
系统支持用命令执行odex编译
命令：
adb shell cmd package compile
基于配置文件：
adb shell cmd package compile -m speed-profile -f my-package
全面编译：
adb shell cmd package compile -m speed -f my-package
代码执行流程 ：
1695197534122.png
命令执行后会生成odex和vdex文件，放置在data/app/package/oat/arm[64]/xxx
重启进程后可以使用命令查：
/proc/pid/maps/ |grep "odex"
这样就看到应用加载了odex加载到了内存。
4、手动执行dex2Oat
android系统是自带dex2oat工具的，直接在平台执行dex2oat 命令可以直接生成对应的文件，默认位speed编译
dex2oat --dex-file=a.dex --oat-file=./oat/arm64/base.odex
不过我企图生成odex去覆盖原来的odex失败了。简单看了ART执行的逻辑，应该是校验不通过导致。（没有仔细研究，只是简单看看代码+推测）。
后面发现如果用PackageManagerService（pkms）去生成一个odex就可以用。于是我加了点log查看一样的命令参数，使用pkms同款编译参数后就可以用了。放上研究了一天的命令参数
dex2oat32 --dex-file=./data/app/xxx/xxx.dex --oat-file=/data/app/xxx/oat/arm/xxx.odex --classpath-dir=/data/app/xxx --class-loader-context=PCL[]{PCL[/system/framework/org.apache.http.legacy.jar]} --instruction-set=arm --instruction-set-features=default --instruction-set-variant=cortex-a7 --compiler-filter=speed --compilation-reason=cmdline --max-image-block-size=524288 --resolve-startup-const-strings=true --generate-mini-debug-info --runtime-arg -Xdeny-art-apex-data-files --runtime-arg -Xtarget-sdk-version:31 --runtime-arg -Xhidden-api-policy:enabled -j4 --runtime-arg -Xms64m --runtime-arg -Xmx512m --compile-individually
三、PKMS 代码执行逻辑
1、pkms执行逻辑
1695200019816.png
int performDexOpt(AndroidPackage pkg, @NonNull PackageSetting pkgSetting,
String[] instructionSets, CompilerStats.PackageStats packageStats,
PackageDexUsage.PackageUseInfo packageUseInfo, DexoptOptions options) {
if (PLATFORM_PACKAGE_NAME.equals(pkg.getPackageName())) {
throw new IllegalArgumentException("System server dexopting should be done via "
+ " DexManager and PackageDexOptimizer#dexoptSystemServerPath");
}
if (pkg.getUid() == -1) {
throw new IllegalArgumentException("Dexopt for " + pkg.getPackageName()
+ " has invalid uid.");
}
if (!canOptimizePackage(pkg)) {//过滤不允许oat的
return DEX_OPT_SKIPPED;
}
synchronized (mInstallLock) {
final long acquireTime = acquireWakeLockLI(pkg.getUid());
try {
return performDexOptLI(pkg, pkgSetting, instructionSets,
packageStats, packageUseInfo, options);
} finally {
releaseWakeLockLI(acquireTime);
}
}
}
// 进入performDexOptLI(pkg, pkgSetting, instructionSets, packageStats, packageUseInfo, options)
private int performDexOptLI(AndroidPackage pkg, @NonNull PackageSetting pkgSetting,
String[] targetInstructionSets, CompilerStats.PackageStats packageStats,
PackageDexUsage.PackageUseInfo packageUseInfo, DexoptOptions options) {
...
int result = DEX_OPT_SKIPPED;
for (int i = 0; i < paths.size(); i++) {
// Skip paths that have no code.
if (!pathsWithCode[i]) {
continue;
}
if (classLoaderContexts[i] == null) {
throw new IllegalStateException("Inconsistent information in the "
+ "package structure. A split is marked to contain code "
+ "but has no dependency listed. Index=" + i + " path=" + paths.get(i));
}
// Append shared libraries with split dependencies for this split.
String path = paths.get(i);
if (options.getSplitName() != null) {
// We are asked to compile only a specific split. Check that the current path is
// what we are looking for.
if (!options.getSplitName().equals(new File(path).getName())) {
continue;
}
}
String profileName = ArtManager.getProfileName(
i == 0 ? null : pkg.getSplitNames()[i - 1]); //找profile文件
...
final String compilerFilter = getRealCompilerFilter(pkg,
options.getCompilerFilter(), isUsedByOtherApps);//对一些过滤编译器做调整
...
for (String dexCodeIsa : dexCodeInstructionSets) {
int newResult = dexOptPath(pkg, pkgSetting, path, dexCodeIsa, compilerFilter,
profileAnalysisResult, classLoaderContexts[i], dexoptFlags, sharedGid,
packageStats, options.isDowngrade(), profileName, dexMetadataPath,
options.getCompilationReason());
...
return result;
}
//进入dexOptPath
private int dexOptPath(AndroidPackage pkg, @NonNull PackageSetting pkgSetting, String path,
String isa, String compilerFilter, int profileAnalysisResult, String classLoaderContext,
int dexoptFlags, int uid, CompilerStats.PackageStats packageStats, boolean downgrade,
String profileName, String dexMetadataPath, int compilationReason) {
...
String oatDir = getPackageOatDirIfSupported(pkg,
pkgSetting.getPkgState().isUpdatedSystemApp());
...
mInstaller.dexopt(path, uid, pkg.getPackageName(), isa, dexoptNeeded, oatDir,
dexoptFlags, compilerFilter, pkg.getVolumeUuid(), classLoaderContext,
seInfo, false /* downgrade*/, pkg.getTargetSdkVersion(),
profileName, dexMetadataPath,
getAugmentedReasonName(compilationReason, dexMetadataPath != null)); //调用Installer 执行dexOpt过程
...
return DEX_OPT_PERFORMED;
} catch (InstallerException e) {
Slog.w(TAG, "Failed to dexopt", e);
return DEX_OPT_FAILED;
}
}
调用performDexOpt需要传入以下参数
performDexOpt(AndroidPackage pkg, @NonNull PackageSetting pkgSetting,
String[] instructionSets,
CompilerStats.PackageStats packageStats,
PackageDexUsage.PackageUseInfo packageUseInfo,
DexoptOptions options)
其中DexoptOptions 定义了编译的可选项，其构造方法如下，有个比较重要的属性compilationReason，构造方法2也是通过reason获取compilerFilter的。
//构造1
public DexoptOptions(String packageName, String compilerFilter, int flags) {
this(packageName, /*compilationReason*/ -1, compilerFilter, /*splitName*/ null, flags);
}
//构造2
public DexoptOptions(String packageName, int compilationReason, int flags) {
this(packageName, compilationReason, getCompilerFilterForReason(compilationReason),
/*splitName*/ null, flags);
}
//构造3
public DexoptOptions(String packageName, int compilationReason, String compilerFilter,
String splitName, int flags) {
int validityMask =
DEXOPT_CHECK_FOR_PROFILES_UPDATES |
DEXOPT_FORCE |
DEXOPT_BOOT_COMPLETE |
DEXOPT_ONLY_SECONDARY_DEX |
DEXOPT_ONLY_SHARED_DEX |
DEXOPT_DOWNGRADE |
DEXOPT_AS_SHARED_LIBRARY |
DEXOPT_IDLE_BACKGROUND_JOB |
DEXOPT_INSTALL_WITH_DEX_METADATA_FILE |
DEXOPT_FOR_RESTORE;
if ((flags & (~validityMask)) != 0) {
throw new IllegalArgumentException("Invalid flags : " + Integer.toHexString(flags));
}
mPackageName = packageName;
mCompilerFilter = compilerFilter;
mFlags = flags;
mSplitName = splitName;
mCompilationReason = compilationReason;
}
通过reason获取compilerFilter的。也就是从这个通过prop定义的值完成设置编译过滤器。
public static String getCompilerFilterForReason(int reason) {
return getAndCheckValidity(reason);
}
/ Load the property for the given reason and check for validity. This will throw an
// exception in case the reason or value are invalid.
private static String getAndCheckValidity(int reason) {
String sysPropValue = SystemProperties.get(getSystemPropertyName(reason));
if (sysPropValue == null || sysPropValue.isEmpty()
|| !(sysPropValue.equals(DexoptOptions.COMPILER_FILTER_NOOP)
|| DexFile.isValidCompilerFilter(sysPropValue))) {
throw new IllegalStateException("Value \"" + sysPropValue +"\" not valid "
+ "(reason " + REASON_STRINGS[reason] + ")");
} else if (!isFilterAllowedForReason(reason, sysPropValue)) {
throw new IllegalStateException("Value \"" + sysPropValue +"\" not allowed "
+ "(reason " + REASON_STRINGS[reason] + ")");
}
return sysPropValue;
}
PKMS 中定义了14中编译原因，对应了每种触发dexOpt的原因，和执行dexOpt定义的编译过滤器。
// Compilation reasons.
public static final int REASON_FIRST_BOOT = 0;
public static final int REASON_BOOT_AFTER_OTA = 1;
public static final int REASON_POST_BOOT = 2;
public static final int REASON_INSTALL = 3;
public static final int REASON_INSTALL_FAST = 4;
public static final int REASON_INSTALL_BULK = 5;
public static final int REASON_INSTALL_BULK_SECONDARY = 6;
public static final int REASON_INSTALL_BULK_DOWNGRADED = 7;
public static final int REASON_INSTALL_BULK_SECONDARY_DOWNGRADED = 8;
public static final int REASON_BACKGROUND_DEXOPT = 9;
public static final int REASON_AB_OTA = 10;
public static final int REASON_INACTIVE_PACKAGE_DOWNGRADE = 11;
public static final int REASON_CMDLINE = 12;
public static final int REASON_SHARED = 13;
2、BackgroundDexOptService
BackgroundDexOptService 是一个JobService，用于定期执行任务。在SystemServer#startOtherService()方法中启动，启动后执行两个任务。
任务一：监听开机广播，在开机10分钟-60分钟内完成post-boot的dex优化
任务二：每隔一天执行一次idle 场景下dex优化。
1695200395519.png
//SystemServer.java
private void startOtherServices(@NonNull TimingsTraceAndSlog t) {
...
t.traceBegin("StartBackgroundDexOptService");
try {
BackgroundDexOptService.schedule(context);
} catch (Throwable e) {
reportWtf("starting StartBackgroundDexOptService", e);
}
t.traceEnd();
...
}
// BackgroundDexOptService.java
public static void schedule(Context context) {
if (isBackgroundDexoptDisabled()) {//读属性"pm.dexopt.disable_bg_dexopt" ,目前是false
return;
}
final JobScheduler js = context.getSystemService(JobScheduler.class);
// Schedule a one-off job which scans installed packages and updates
// out-of-date oat files. Schedule it 10 minutes after the boot complete event,
// so that we don't overload the boot with additional dex2oat compilations.
context.registerReceiver(new BroadcastReceiver() {
@Override
public void onReceive(Context context, Intent intent) {
js.schedule(new JobInfo.Builder(JOB_POST_BOOT_UPDATE, sDexoptServiceName)//BackgroundDexOptService 的第一个任务
.setMinimumLatency(TimeUnit.MINUTES.toMillis(10)) //最短执行时间10min
.setOverrideDeadline(TimeUnit.MINUTES.toMillis(60)) //最迟执行时间：60mins
.build());
context.unregisterReceiver(this);
if (DEBUG) {
Slog.i(TAG, "BootBgDexopt scheduled");
}
}
}, new IntentFilter(Intent.ACTION_BOOT_COMPLETED));//监听开机启动广播
// Schedule a daily job which scans installed packages and compiles
// those with fresh profiling data.
js.schedule(new JobInfo.Builder(JOB_IDLE_OPTIMIZE, sDexoptServiceName) //BackgroundDexOptService 的第二个任务
.setRequiresDeviceIdle(true) //设备在idle状态
.setRequiresCharging(true) //充电中
.setPeriodic(IDLE_OPTIMIZATION_PERIOD) //执行周期 一天
.build());
if (DEBUG) {
Slog.d(TAG, "BgDexopt scheduled");
}
}
//关注第一个job：到点执行进入onStartJob方法
public boolean onStartJob(JobParameters params) {
if (DEBUG) {
Slog.i(TAG, "onStartJob");
}
// NOTE: PackageManagerService.isStorageLow uses a different set of criteria from
// the checks above. This check is not "live" - the value is determined by a background
// restart with a period of ~1 minute.
PackageManagerService pm = (PackageManagerService)ServiceManager.getService("package");
if (pm.isStorageLow()) {//Environment.getDataDirectory().getUsableSpace() < getMemoryLowThreshold(); 为data空间的20%或者低于500MB
Slog.i(TAG, "Low storage, skipping this run");
return false;
}
final ArraySet<String> pkgs = pm.getOptimizablePackages();
if (pkgs.isEmpty()) {
Slog.i(TAG, "No packages to optimize");
return false;
}
mThermalStatusCutoff =
SystemProperties.getInt("dalvik.vm.dexopt.thermal-cutoff", THERMAL_CUTOFF_DEFAULT);//2
boolean result;
if (params.getJobId() == JOB_POST_BOOT_UPDATE) {
result = runPostBootUpdate(params, pm, pkgs);//进入这里
} else {
result = runIdleOptimization(params, pm, pkgs);
}
return result;
}
//执行post-boot更新
private boolean runPostBootUpdate(final JobParameters jobParams,
final PackageManagerService pm, final ArraySet<String> pkgs) {
if (mExitPostBootUpdate.get()) {
// This job has already been superseded. Do not start it.
return false;
}
new Thread("BackgroundDexOptService_PostBootUpdate") {//新起一个线程执行postBootUpdate
@Override
public void run() {
postBootUpdate(jobParams, pm, pkgs);
}
}.start();
return true;
}
//子线程运行执行所有可优化包的dex优化
private void postBootUpdate(JobParameters jobParams, PackageManagerService pm,
ArraySet<String> pkgs) {
final BatteryManagerInternal batteryManagerInternal =
LocalServices.getService(BatteryManagerInternal.class);
final long lowThreshold = getLowStorageThreshold(this);
mAbortPostBootUpdate.set(false);
ArraySet<String> updatedPackages = new ArraySet<>();
for (String pkg : pkgs) {
if (mAbortPostBootUpdate.get()) {
// JobScheduler requested an early abort.
return;
}
if (mExitPostBootUpdate.get()) {
// Different job, which supersedes this one, is running.
break;
}
if (batteryManagerInternal.getBatteryLevelLow()) {//低电量不执行
// Rather bail than completely drain the battery.
break;
}
long usableSpace = mDataDir.getUsableSpace();
if (usableSpace < lowThreshold) {//存储不足
// Rather bail than completely fill up the disk.
Slog.w(TAG, "Aborting background dex opt job due to low storage: " +
usableSpace);
break;
}
if (DEBUG) {
Slog.i(TAG, "Updating package " + pkg);
}
// Update package if needed. Note that there can be no race between concurrent
// jobs because PackageDexOptimizer.performDexOpt is synchronized.
// checkProfiles is false to avoid merging profiles during boot which
// might interfere with background compilation (b/28612421).
// Unfortunately this will also means that "pm.dexopt.boot=speed-profile" will
// behave differently than "pm.dexopt.bg-dexopt=speed-profile" but that's a
// trade-off worth doing to save boot time work.
int result = pm.performDexOptWithStatus(new DexoptOptions(//进入PKMS
pkg,
PackageManagerService.REASON_POST_BOOT,//原因
DexoptOptions.DEXOPT_BOOT_COMPLETE));
if (result == PackageDexOptimizer.DEX_OPT_PERFORMED) {
updatedPackages.add(pkg);
}
}
notifyPinService(updatedPackages);
notifyPackagesUpdated(updatedPackages);
// Ran to completion, so we abandon our timeslice and do not reschedule.
jobFinished(jobParams, /* reschedule */ false);
}
进入PKMS 的performDexOptWithStatus方法。
/* package */ int performDexOptWithStatus(DexoptOptions options) {
return performDexOptTraced(options);
}
private int performDexOptTraced(DexoptOptions options) {
Trace.traceBegin(TRACE_TAG_PACKAGE_MANAGER, "dexopt");
try {
return performDexOptInternal(options);
} finally {
Trace.traceEnd(TRACE_TAG_PACKAGE_MANAGER);
}
}
private int performDexOptInternal(DexoptOptions options) {
AndroidPackage p;
PackageSetting pkgSetting;
synchronized (mLock) {
p = mPackages.get(options.getPackageName());
pkgSetting = mSettings.getPackageLPr(options.getPackageName());
if (p == null || pkgSetting == null) {
// Package could not be found. Report failure.
return PackageDexOptimizer.DEX_OPT_FAILED;
}
mPackageUsage.maybeWriteAsync(mSettings.getPackagesLocked());
mCompilerStats.maybeWriteAsync();
}
final long callingId = Binder.clearCallingIdentity();
try {
synchronized (mInstallLock) {
return performDexOptInternalWithDependenciesLI(p, pkgSetting, options); //进入这里
}
} finally {
Binder.restoreCallingIdentity(callingId);
}
}
private int performDexOptInternalWithDependenciesLI(AndroidPackage p,
@NonNull PackageSetting pkgSetting, DexoptOptions options) {
// System server gets a special path.
if (PLATFORM_PACKAGE_NAME.equals(p.getPackageName())) {
return mDexManager.dexoptSystemServer(options);//android 走系统
}
// Select the dex optimizer based on the force parameter.
// Note: The force option is rarely used (cmdline input for testing, mostly), so it's OK to
// allocate an object here.
PackageDexOptimizer pdo = options.isForce()
? new PackageDexOptimizer.ForcedUpdatePackageDexOptimizer(mPackageDexOptimizer)
: mPackageDexOptimizer; //如果有携带f就要强制编译，无其他逻辑
// Dexopt all dependencies first. Note: we ignore the return value and march on
// on errors.
// Note that we are going to call performDexOpt on those libraries as many times as
// they are referenced in packages. When we do a batch of performDexOpt (for example
// at boot, or background job), the passed 'targetCompilerFilter' stays the same,
// and the first package that uses the library will dexopt it. The
// others will see that the compiled code for the library is up to date.
Collection<SharedLibraryInfo> deps = findSharedLibraries(pkgSetting);
final String[] instructionSets = getAppDexInstructionSets(
AndroidPackageUtils.getPrimaryCpuAbi(p, pkgSetting),
AndroidPackageUtils.getSecondaryCpuAbi(p, pkgSetting));
if (!deps.isEmpty()) {
DexoptOptions libraryOptions = new DexoptOptions(options.getPackageName(),
options.getCompilationReason(), options.getCompilerFilter(),
options.getSplitName(),
options.getFlags() | DexoptOptions.DEXOPT_AS_SHARED_LIBRARY);
for (SharedLibraryInfo info : deps) {
AndroidPackage depPackage = null;
PackageSetting depPackageSetting = null;
synchronized (mLock) {
depPackage = mPackages.get(info.getPackageName());
depPackageSetting = mSettings.getPackageLPr(info.getPackageName());
}
if (depPackage != null && depPackageSetting != null) {
// TODO: Analyze and investigate if we (should) profile libraries.
pdo.performDexOpt(depPackage, depPackageSetting, instructionSets, //先对依赖库进行dex优化
getOrCreateCompilerPackageStats(depPackage),
mDexManager.getPackageUseInfoOrDefault(depPackage.getPackageName()),
libraryOptions);
} else {
// TODO(ngeoffray): Support dexopting system shared libraries.
}
}
}
return pdo.performDexOpt(p, pkgSetting, instructionSets,//进入PackageDexOptimizer#performDexOpt流程，上面已经分析过。
getOrCreateCompilerPackageStats(p),
mDexManager.getPackageUseInfoOrDefault(p.getPackageName()), options);
}
