As I said in last posting "Crack Android apk with Apktool & Apksigner", I've faced some troubles in the rebuilding phase.
A) No resource identifier found for attribute "XXX" in package "com.XXX.XXX"
B) No resource found that matches the given name (at "XXX" with value "XXX")
As you can see, these problems are caused by custom attribute in a private namespace of a resource file.
For case A, Let's have a closer look at res\menu\a.xml
The attribute name in question is ktv:showAsAction.
Let's check out its internal id by aapt.
AAPT is an Android Asset Packaging Tool, which is able to dump or package the compiled Zip-compatible archive.
Let's try to dump the designated res\menu\a.xml as which is refered by r\p\a.xml before being unarchived.
aapt d xmltree anonymity.apk r/p/a.xml
As the result showed above, the internal id of attribute "showAsAction" is 0x7f01020a. Pay attention to the path which is composed of slash "/", not backslash "\".
Search the very id 0x7f01020a in the global resource id table -- res\values\public.xml -- and we'll find that it represents an attribute named "jt" which is obviously obfuscated.
<public type="attr" name="jt" id="0x7f01020a" />
Replace the so-called "showAsAction" which is definte incorrect into "jt", then recompile. As a result, the errors relating to "ktv:showAsAction" disappear.
It works. But we can't certainly do such a tedious work in which we change the erroneous attribute name one by one.
We're gonna plunge into the source code of Apktool and find out an elegant and efficient way to substantially change all the false attribute names or values.
Apktool recommend that we use IntelliJ IDEA to manage the project for convenient building and dependencies deployment with gradle.
But it's a new tool for me. Who feels complicated would be better looking to the official tutorials for how to leverage it to build and export a usable jar archive rather than searching out the Internet inefficiently.
Let's get going and unveil the code.
Apktool start decoding from brut.androlib.ApkDecoder.
public void decode() throws AndrolibException, IOException, DirectoryException {
try {
File outDir = getOutDir();
AndrolibResources.sKeepBroken = mKeepBrokenResources;
if (!mForceDelete && outDir.exists()) {
throw new OutDirExistsException();
}
if (!mApkFile.isFile() || !mApkFile.canRead()) {
throw new InFileNotFoundException();
}
try {
OS.rmdir(outDir);
} catch (BrutException ex) {
throw new AndrolibException(ex);
}
outDir.mkdirs();
LOGGER.info("Using Apktool " + Androlib.getVersion() + " on " + mApkFile.getName());
// 1. Decode resources.
if (hasResources()) {
switch (mDecodeResources) {
case DECODE_RESOURCES_NONE:
mAndrolib.decodeResourcesRaw(mApkFile, outDir);
if (mForceDecodeManifest == FORCE_DECODE_MANIFEST_FULL) {
setTargetSdkVersion();
setAnalysisMode(mAnalysisMode, true);
// done after raw decoding of resources because copyToDir overwrites dest files
if (hasManifest()) {
mAndrolib.decodeManifestWithResources(mApkFile, outDir, getResTable());
}
}
break;
case DECODE_RESOURCES_FULL:
setTargetSdkVersion();
setAnalysisMode(mAnalysisMode, true);
if (hasManifest()) {
mAndrolib.decodeManifestWithResources(mApkFile, outDir, getResTable());
}
mAndrolib.decodeResourcesFull(mApkFile, outDir, getResTable());
break;
}
} else {
// if there's no resources.arsc, decode the manifest without looking
// up attribute references
if (hasManifest()) {
if (mDecodeResources == DECODE_RESOURCES_FULL
|| mForceDecodeManifest == FORCE_DECODE_MANIFEST_FULL) {
mAndrolib.decodeManifestFull(mApkFile, outDir, getResTable());
} else {
mAndrolib.decodeManifestRaw(mApkFile, outDir);
}
}
}
// 2.1. Decode dex files.
if (hasSources()) {
switch (mDecodeSources) {
case DECODE_SOURCES_NONE:
mAndrolib.decodeSourcesRaw(mApkFile, outDir, "classes.dex");
break;
case DECODE_SOURCES_SMALI:
mAndrolib.decodeSourcesSmali(mApkFile, outDir, "classes.dex", mBakDeb, mApi);
break;
}
}
// 2.2. Merge multiple dexes.
if (hasMultipleSources()) {
// foreach unknown dex file in root, lets disassemble it
Set<String> files = mApkFile.getDirectory().getFiles(true);
for (String file : files) {
if (file.endsWith(".dex")) {
if (!file.equalsIgnoreCase("classes.dex")) {
switch (mDecodeSources) {
case DECODE_SOURCES_NONE:
mAndrolib.decodeSourcesRaw(mApkFile, outDir, file);
break;
case DECODE_SOURCES_SMALI:
mAndrolib.decodeSourcesSmali(mApkFile, outDir, file, mBakDeb, mApi);
break;
}
}
}
}
}
// 3. Handle files of miscellaneous types
mAndrolib.decodeRawFiles(mApkFile, outDir, mDecodeAssets);
mAndrolib.decodeUnknownFiles(mApkFile, outDir, mResTable);
mUncompressedFiles = new ArrayList<String>();
mAndrolib.recordUncompressedFiles(mApkFile, mUncompressedFiles);
mAndrolib.writeOriginalFiles(mApkFile, outDir);
writeMetaFile();
} catch (Exception ex) {
throw ex;
} finally {
try {
mApkFile.close();
} catch (IOException ignored) {
}
}
}
We can see that there are three major steps:
- Decode resources.
- Decode dex files.
- Handle files of miscellaneous types
I'll merely expand the first step in order to find out the solotion to our faults in the process of res-decoding.
Let's scoot ahead.
1. Check if there is an achive named "resources.arsc"
public boolean hasResources() throws AndrolibException {
try {
return mApkFile.getDirectory().containsFile("resources.arsc");
} catch (DirectoryException ex) {
throw new AndrolibException(ex);
}
}
The whole of resource files are originally packaged in the .arsc archive. We have to make sure it is being ready to be extracted.
2. Build the resource table and extract the sdk version
public void setTargetSdkVersion() throws AndrolibException, IOException {
// Build a resource table
if (mResTable == null) {
mResTable = mAndrolib.getResTable(mApkFile);
}
// Retrieve sdk version
Map<String, String> sdkInfo = mResTable.getSdkInfo();
if (sdkInfo.get("targetSdkVersion") != null) {
mApi = Integer.parseInt(sdkInfo.get("targetSdkVersion"));
}
}
The procedure for building resource table is one of the most important steps. A resource table is a collection of the header information of each resource file. We're gonna decrypt and reorganize target files by this table.
Let's check out how to build it.
public ResTable getResTable(ExtFile apkFile, boolean loadMainPkg)
throws AndrolibException {
ResTable resTable = new ResTable(this);
if (loadMainPkg) {
//.arsc file may be made of a certain number
// (which is basically one) of packages.
loadMainPkg(resTable, apkFile);
}
return resTable;
}
In this method, apktool create a new ResTable and bind it with the AndrolibResources singleton which manages all the resource affairs.
The main logic is execute loadMainPkg method in which we will inflate the newly generated ResTable.
Let's move on to explore what loadMainPkg would have done.
public ResPackage loadMainPkg(ResTable resTable, ExtFile apkFile)
throws AndrolibException {
LOGGER.info("Loading resource table...");
// 1. Retrieve details of ResTable
// including ones of Packages, Types, Resources in it.
ResPackage[] pkgs = getResPackagesFromApk(apkFile, resTable, sKeepBroken);
ResPackage pkg = null;
switch (pkgs.length) {
case 1:
pkg = pkgs[0];
break;
case 2:
if (pkgs[0].getName().equals("android")) {
LOGGER.warning("Skipping \"android\" package group");
pkg = pkgs[1];
break;
} else if (pkgs[0].getName().equals("com.htc")) {
LOGGER.warning("Skipping \"htc\" package group");
pkg = pkgs[1];
break;
}
default:
pkg = selectPkgWithMostResSpecs(pkgs);
break;
}
if (pkg == null) {
throw new AndrolibException("arsc files with zero packages or no arsc file found.");
}
// 2. Put the main package in the ResTable.
resTable.addPackage(pkg, true);
return pkg;
}
The main idea of loadMainPkg is retrieving details of ResTable including ones of Packages, Types, Resources in the package.
A Table has got a tree-like internal structure.
- Package 1 (com.XXX.XXX)
- Type 1 (layout, drawable, menu, anim, ...)
- Res 1 (a.xml, b.png, ResAttr, ResStringValue, ...)
- Res 2
...
- Type 2
...
- Type 1 (layout, drawable, menu, anim, ...)
- Package 2
...
A package is a unit represented by a package id stored in AndroidManifest.xml looks like "package='com.XXX.XXX'"
A Type is actually a folder name under "res" directory like "layout, drawable, menu, anim, ...".
A Res is a resource under respect Type folder like "a.xml, b.png, ResAttr, ResStringValue, ..." which is in different formats.
No matter a Type or a Res has a header information called Spec which may be the abbreviation of "specification", involves its id, file name, type (aka folder name), etc.
Here is the involking chain of loadMainPkg method.
Apktool read table, read packages in table recursively, read types which is the "folder" in a package recursively, read entry which is the "Res" in a type recursively. Apktool puts raw bytes of every resource into a particular class like "ResAttr, ResStringValue, ..." to be waited to process.
At last, getResPackagesFromApk method will return all the packages read in the ResTable after traversing and recording all of their resources. loadMainPkg method puts the main package of these packages into current ResTable instance.
Actually, there is often only one package which is also the main package.
As of now, the ResTable has been built over.
3. Decode AndroidManifest.xml
mAndrolib.decodeManifestWithResources(mApkFile, outDir, getResTable());
I won't expand this step since it's simply decoding the manifest file and do some adjusted work which we will cover in the next step.
4. Decode the whole of resources and reorganize them.
mAndrolib.decodeResourcesFull(mApkFile, outDir, getResTable());
This is the most important step which involves the key to our recompiling errors.
public void decode(ResTable resTable, ExtFile apkFile, File outDir)
throws AndrolibException {
// 1. Prepare the key XML Decoder & Parser to process xml files.
Duo<ResFileDecoder, AXmlResourceParser> duo = getResFileDecoder();
ResFileDecoder fileDecoder = duo.m1;
ResAttrDecoder attrDecoder = duo.m2.getAttrDecoder();
// 2. Bind the main package with AXmlResourceParser.
attrDecoder.setCurrentPackage(resTable.listMainPackages().iterator().next());
Directory inApk, in = null, out;
try {
// 3. Custom particular input and output directory
out = new FileDirectory(outDir);
inApk = apkFile.getDirectory();
out = out.createDir("res");
if (inApk.containsDir("res")) {
in = inApk.getDir("res");
}
if (in == null && inApk.containsDir("r")) {
in = inApk.getDir("r");
}
if (in == null && inApk.containsDir("R")) {
in = inApk.getDir("R");
}
} catch (DirectoryException ex) {
throw new AndrolibException(ex);
}
// 4. Prepare an xml serializer for storing xml elements.
ExtMXSerializer xmlSerializer = getResXmlSerializer();
for (ResPackage pkg : resTable.listMainPackages()) {
attrDecoder.setCurrentPackage(pkg);
LOGGER.info("Decoding file-resources...");
for (ResResource res : pkg.listFiles()) {
// 5. Handle every resource file.
fileDecoder.decode(res, in, out);
}
LOGGER.info("Decoding values */* XMLs...");
for (ResValuesFile valuesFile : pkg.listValuesFiles()) {
// 6. Put every value-type resource into a specified file, ex. value.xml.
generateValuesFile(valuesFile, out, xmlSerializer);
}
// 7. List the header information of every resource in the public.xml
// including its id, type, name, etc.
generatePublicXml(pkg, out, xmlSerializer);
}
AndrolibException decodeError = duo.m2.getFirstError();
if (decodeError != null) {
throw decodeError;
}
}
I have concluded 7 substeps in the brut.androlib.res.AndrolibResources.decode method. The most crucial step is step 5. It traverse all the resource objects we've got in the loadMainPkg method and generate generate resource file one by one.
Let's go and dig into it.
public void decode(ResResource res, Directory inDir, Directory outDir)
throws AndrolibException {
// 1. Prepare the input and output path.
ResFileValue fileValue = (ResFileValue) res.getValue();
String inFileName = fileValue.getStrippedPath();
String outResName = res.getFilePath();
String typeName = res.getResSpec().getType().getName();
String ext = null;
String outFileName;
int extPos = inFileName.lastIndexOf(".");
if (extPos == -1) {
outFileName = outResName;
} else {
ext = inFileName.substring(extPos).toLowerCase();
outFileName = outResName + ext;
}
// 2. Decode resource files according to their types.
try {
if (typeName.equals("raw")) {
decode(inDir, inFileName, outDir, outFileName, "raw");
return;
}
if (typeName.equals("font") && !".xml".equals(ext)) {
decode(inDir, inFileName, outDir, outFileName, "raw");
return;
}
if (typeName.equals("drawable") || typeName.equals("mipmap")) {
if (inFileName.toLowerCase().endsWith(".9" + ext)) {
outFileName = outResName + ".9" + ext;
// check for htc .r.9.png
if (inFileName.toLowerCase().endsWith(".r.9" + ext)) {
outFileName = outResName + ".r.9" + ext;
}
// check for raw 9patch images
for (String extension : RAW_9PATCH_IMAGE_EXTENSIONS) {
if (inFileName.toLowerCase().endsWith("." + extension)) {
copyRaw(inDir, outDir, outFileName);
return;
}
}
// check for xml 9 patches which are just xml files
if (inFileName.toLowerCase().endsWith(".xml")) {
decode(inDir, inFileName, outDir, outFileName, "xml");
return;
}
try {
decode(inDir, inFileName, outDir, outFileName, "9patch");
return;
} catch (CantFind9PatchChunk ex) {
LOGGER.log(
Level.WARNING,
String.format(
"Cant find 9patch chunk in file: \"%s\". Renaming it to *.png.",
inFileName), ex);
outDir.removeFile(outFileName);
outFileName = outResName + ext;
}
}
// check for raw image
for (String extension : RAW_IMAGE_EXTENSIONS) {
if (inFileName.toLowerCase().endsWith("." + extension)) {
copyRaw(inDir, outDir, outFileName);
return;
}
}
if (!".xml".equals(ext)) {
decode(inDir, inFileName, outDir, outFileName, "raw");
return;
}
}
// *Key entrance*
decode(inDir, inFileName, outDir, outFileName, "xml");
} catch (AndrolibException ex) {
LOGGER.log(Level.SEVERE, String.format(
"Could not decode file, replacing by FALSE value: %s",
inFileName), ex);
res.replace(new ResBoolValue(false, 0, null));
}
}
What decode method do is to decode resource file as to its specific type.
I have marked a "key entrance" to decode an xml file in which our errors occured.
By the way, the first step of this code passage will generate the input path (like r/p/a.xml) and the output path (like res/menu/a.xml), by which we can sumarize a mapping between them.
That's why I knew res\menu\a.xml is refered by r\p\a.xml above when I leveraged aapt to inquire the attribute name id.
@Override
public void decode(InputStream in, OutputStream out)
throws AndrolibException {
try {
XmlPullWrapperFactory factory = XmlPullWrapperFactory.newInstance();
XmlPullParserWrapper par = factory.newPullParserWrapper(mParser);
final ResTable resTable = ((AXmlResourceParser) mParser).getAttrDecoder().getCurrentPackage().getResTable();
XmlSerializerWrapper ser = new StaticXmlSerializerWrapper(mSerial, factory) {
boolean hideSdkInfo = false;
boolean hidePackageInfo = false;
@Override
public void event(XmlPullParser pp)
throws XmlPullParserException, IOException {
int type = pp.getEventType();
if (type == XmlPullParser.START_TAG) {
if ("manifest".equalsIgnoreCase(pp.getName())) {
try {
hidePackageInfo = parseManifest(pp);
} catch (AndrolibException ignored) {}
} else if ("uses-sdk".equalsIgnoreCase(pp.getName())) {
try {
hideSdkInfo = parseAttr(pp);
if (hideSdkInfo) {
return;
}
} catch (AndrolibException ignored) {}
}
} else if (hideSdkInfo && type == XmlPullParser.END_TAG
&& "uses-sdk".equalsIgnoreCase(pp.getName())) {
return;
} else if (hidePackageInfo && type == XmlPullParser.END_TAG
&& "manifest".equalsIgnoreCase(pp.getName())) {
super.event(pp);
return;
}
super.event(pp);
}
......
};
par.setInput(in, null);
ser.setOutput(out, null);
// Each xml is wrapped by a AXmlResourceParser
while (par.nextToken() != XmlPullParser.END_DOCUMENT) {
ser.event(par);
}
ser.flush();
} catch (XmlPullParserException ex) {
throw new AndrolibException("Could not decode XML", ex);
} catch (IOException ex) {
throw new AndrolibException("Could not decode XML", ex);
}
}
Apktool creates a inner class StaticXmlSerializerWrapper which override the same class of org.xmlpull.v1.wrapper.classic.StaticXmlSerializerWrapper which is in the xpp3 jar. Xpp3 is a fast, lightweight xml parser that comforms to the "Pull Stream API"
The main course is so clear that tokens are read from input raw xml data and are wrapped with an AXmlResourceParser object. This parser will be delivered to the serializer which is the inner class which is just created.
ser.event(par);
Then the inner class will call outer StaticXmlSerializerWrapper by
super.event(pp);
public void event(XmlPullParser pp) throws XmlPullParserException, IOException {
int eventType = pp.getEventType();
switch (eventType) {
case XmlPullParser.START_DOCUMENT:
//use Boolean.TRUE to make it standalone
Boolean standalone = (Boolean) pp.getProperty(PROPERTY_XMLDECL_STANDALONE);
startDocument(pp.getInputEncoding(), standalone);
break;
case XmlPullParser.END_DOCUMENT:
endDocument();
break;
// XML token entrance
case XmlPullParser.START_TAG:
writeStartTag(pp);
break;
case XmlPullParser.END_TAG:
endTag(pp.getNamespace (), pp.getName ());
break;
case XmlPullParser.IGNORABLE_WHITESPACE:
//comment it to remove ignorable whtespaces from XML infoset
String s = pp.getText ();
ignorableWhitespace(s);
break;
case XmlPullParser.TEXT:
if(pp.getDepth() > 0) {
text(pp.getText ());
} else {
ignorableWhitespace(pp.getText ());
}
break;
case XmlPullParser.ENTITY_REF:
entityRef (pp.getName ());
break;
case XmlPullParser.CDSECT:
cdsect( pp.getText () );
break;
case XmlPullParser.PROCESSING_INSTRUCTION:
processingInstruction( pp.getText ());
break;
case XmlPullParser.COMMENT:
comment (pp.getText ());
break;
case XmlPullParser.DOCDECL:
docdecl (pp.getText ());
break;
}
}
writeStartTag(pp); will be invoked.
private void writeStartTag(XmlPullParser pp) throws XmlPullParserException, IOException {
// Set prefix resembling <xmlns:ns_name="ns_value">
if (!pp.getFeature (XmlPullParser.FEATURE_REPORT_NAMESPACE_ATTRIBUTES)) {
int nsStart = pp.getNamespaceCount(pp.getDepth()-1);
int nsEnd = pp.getNamespaceCount(pp.getDepth());
for (int i = nsStart; i < nsEnd; i++) {
String prefix = pp.getNamespacePrefix(i);
String ns = pp.getNamespaceUri(i);
setPrefix(prefix, ns);
}
}
// Set start tag of an item resembling
// <ns_name:tag_name ...>
startTag(pp.getNamespace(), pp.getName());
// Set following attribute resembling
// <... ns_name:attr_name="attr_value" ...>
for (int i = 0; i < pp.getAttributeCount(); i++) {
attribute
(pp.getAttributeNamespace (i),
pp.getAttributeName (i),
pp.getAttributeValue (i));
}
}
Here we can locate where our errors happened:
- Case A occured at
pp.getAttributeName (i), so that the attribute name under a private namespace is invalid. - In the same way, Case B occured at
pp.getAttributeValue (i)and the corresponding values are faulty.
We'll have to examine the exact name and value in accordance with the id of a ResSpec
StaticXmlSerializerWrapper class is a facade of the real AXmlResourceParser of which is actually taken advantage.
I have modified 3 methods relating to the logic of xml parsing.
/////////////////////////////////////////////////
//
// brut.androlib.res.decoder.AXmlResourceParser
//
/////////////////////////////////////////////////
public String getAttributeName(int index) {
int offset = getAttributeOffset(index);
int nameIndex = m_attributes[offset + ATTRIBUTE_IX_NAME];
if (nameIndex == -1) return "";
String name = m_strings.getString(nameIndex);
String namespace = getAttributeNamespace(index);
// If attribute name is lacking or a private namespace emerges,
// retrieve the exact attribute name by its id.
if (name == null || name.length() == 0) {
try {
name = mAttrDecoder
.decodeManifestAttr(getAttributeNameResource(index));
if (name == null) name = "";
} catch (AndrolibException e) {name = "";}
} else if (! namespace.equals(android_ns)) {
try {
String obfuscatedName = mAttrDecoder
.decodeManifestAttr(getAttributeNameResource(index));
if (! (obfuscatedName == null || obfuscatedName.equals(name))) {
name = obfuscatedName;
}
} catch (AndrolibException e) {}
}
return name;
}
public String getAttributeValue(int index) {
int offset = getAttributeOffset(index);
int valueType = m_attributes[offset + ATTRIBUTE_IX_VALUE_TYPE];
int valueId = m_attributes[offset + ATTRIBUTE_IX_VALUE_DATA];
int valueRaw = m_attributes[offset + ATTRIBUTE_IX_VALUE_STRING];
if (mAttrDecoder != null) {
try {
String value = valueRaw == -1 ? null : ResXmlEncoders
.escapeXmlChars(m_strings.getString(valueRaw));
String obfuscatedValue = mAttrDecoder
.decodeManifestAttr(valueId);
if (! (value == null || obfuscatedValue == null)) {
int slashPos = value.lastIndexOf("/");
if (slashPos != -1) {
// Handle a value with a format of "@yyy/xxx"
String dir = value.substring(0, slashPos);
value = dir + "/"+ obfuscatedValue;
} else if (! value.equals(obfuscatedValue)) {
value = obfuscatedValue;
}
}
return mAttrDecoder.decode(
valueType,
valueId,
value,
getAttributeNameResource(index));
} catch (AndrolibException ex) {
setFirstError(ex);
LOGGER.log(Level.WARNING, String.format("Could not decode attr value, using undecoded value "
+ "instead: ns=%s, name=%s, value=0x%08x",
getAttributePrefix(index),
getAttributeName(index),
valueId), ex);
}
}
return TypedValue.coerceToString(valueType, valueId);
}
/////////////////////////////////////////////////
//
// brut.androlib.res.decoder.ResAttrDecoder
//
/////////////////////////////////////////////////
public String decodeManifestAttr(int attrResId)
throws AndrolibException {
if (attrResId != 0) {
int attrId = attrResId;
// See also: brut.androlib.res.data.ResTable.getResSpec
if (attrId >> 24 == 0) {
ResPackage pkg = getCurrentPackage();
int packageId = pkg.getId();
int pkgId = (packageId == 0 ? 2 : packageId);
attrId = (0xFF000000 & (pkgId << 24)) | attrId;
}
// Retrieve the ResSpec in a package by its id
ResID resId = new ResID(attrId);
ResPackage pkg = getCurrentPackage();
if (pkg.hasResSpec(resId)) {
ResResSpec resResSpec = pkg.getResSpec(resId);
if (resResSpec != null) {return resResSpec.getName();}
}
}
return null;
}
The basic ideas:
- Get the attribute id in m_attributes by offset
- Invoke brut.androlib.res.decoder.ResAttrDecoder.decodeManifestAttr method in order to inquire the exact attribute value by id
- Replace the invalid one in a sensible condition.
In the end, we decode and rebuild the apk.
Thanks goodness, it works anyway.
