项目作者: obfusk

项目描述 :
apksigcopier - copy/extract/patch android apk signatures & compare apks
高级语言: Python
项目地址: git://github.com/obfusk/apksigcopier.git
创建时间: 2021-03-25T01:20:33Z
项目社区:https://github.com/obfusk/apksigcopier

开源协议:GNU General Public License v3.0

下载


GitHub Release
PyPI Version
Python Versions
CI
GPLv3+


Packaging status


Packaging status

apksigcopier

copy/extract/patch android apk signatures & compare apks

apksigcopier is a tool that enables using an android APK
signature
as a
build input (by
copying it from a signed APK to an unsigned one), making it possible to create a
(bit-by-bit identical)
reproducible build from the source code
without having access to the private key used to create the signature. It can
also be used to verify that two APKs with different signatures are otherwise
identical. Its command-line tool offers four operations:

  • copy signatures directly from a signed to an unsigned APK
  • extract signatures from a signed APK to a directory
  • patch previously extracted signatures onto an unsigned APK
  • compare two APKs with different signatures

Extract

  1. $ mkdir meta
  2. $ apksigcopier extract signed.apk meta
  3. $ ls -1 meta
  4. 8BEA2A77.RSA
  5. 8BEA2A77.SF
  6. APKSigningBlock
  7. APKSigningBlockOffset
  8. MANIFEST.MF

Patch

  1. $ apksigcopier patch meta unsigned.apk out.apk

Copy (Extract & Patch)

  1. $ apksigcopier copy signed.apk unsigned.apk out.apk

Compare (Copy & Verify)

Compare two APKs by copying the signature from the first to a copy of the second
and checking if the resulting APK verifies.

This command requires apksigner.

  1. $ apksigcopier compare foo-from-fdroid.apk foo-built-locally.apk
  2. $ apksigcopier compare --unsigned foo.apk foo-unsigned.apk

NB: copying from an APK v1-signed with signflinger to an APK signed with
apksigner works, whereas the reverse fails; see the FAQ.

Help

  1. $ apksigcopier --help
  2. $ apksigcopier copy --help # extract --help, patch --help, etc.
  3. $ man apksigcopier # requires the man page to be installed

Environment Variables

The following environment variables can be set to 1, yes, or
true to override the default behaviour:

  • set APKSIGCOPIER_EXCLUDE_ALL_META=1 to exclude all metadata files
  • set APKSIGCOPIER_COPY_EXTRA_BYTES=1 to copy extra bytes after data (e.g. a v2 sig)
  • set APKSIGCOPIER_SKIP_REALIGNMENT=1 to skip realignment of ZIP entries

Python API

  1. >>> from apksigcopier import do_extract, do_patch, do_copy, do_compare
  2. >>> do_extract(signed_apk, output_dir, v1_only=NO)
  3. >>> do_patch(metadata_dir, unsigned_apk, output_apk, v1_only=NO)
  4. >>> do_copy(signed_apk, unsigned_apk, output_apk, v1_only=NO)
  5. >>> do_compare(first_apk, second_apk, unsigned=False)

You can use False, None, and True instead of NO, AUTO, and
YES respectively.

The following global variables (which default to False), can be set
to override the default behaviour:

  • set exclude_all_meta=True to exclude all metadata files
  • set copy_extra_bytes=True to copy extra bytes after data (e.g. a v2 sig)
  • set skip_realignment=True to skip realignment of ZIP entries

FAQ

What is the purpose of this tool?

This is a tool for reproducible builds
only. Its purpose is to allow verifying that different builds from the same
source code produce identical results, to prove that two APKs — one built and
signed by the upstream developer, another one built by you (or some trusted
third party) from the published source code — are identical. Since you
cannot create an identical signature without the private key, you need to copy
it (and nothing else) as part of the build process instead to be able to create
a bit-by-bit identical APK.

The motivation behind the Reproducible Builds project is […] to allow
verification that no vulnerabilities or backdoors have been introduced during
this compilation process. By promising identical results are always generated
from a given source, this allows multiple third parties to come to a consensus
on a “correct” result, highlighting any deviations as suspect and worthy of
scrutiny.

Modified APKs

Copying a signature to a modified APK will not work (i.e. it cannot possibly be
valid even if the copying itself seems to work) and this is not a tool for doing
anything of the sort.

Copying a signature will succeed even if the signature is not valid for the
target APK — as long as the target APK is unsigned and not larger than the
source APK it can be inserted successfully. But a signature that is not valid
for the target APK will never verify.

What kind of signatures does apksigcopier support?

It currently supports v1 + v2 + v3 (which is a variant of v2).

It should also support v4, since these are stored in a separate file
(and require a complementary v2/v3 signature).

When using the extract command, the v2/v3 signature is saved as
APKSigningBlock + APKSigningBlockOffset.

How does patching work?

First it copies the APK exactly like apksigner would when signing it,
including re-aligning ZIP entries and skipping existing v1 signature files.

Then it adds the extracted v1 signature files (.SF, .RSA/.DSA/.EC,
MANIFEST.MF) to the APK, using the correct ZIP metadata (either the same
metadata as apksigner would, or from differences.json).

And lastly it inserts the extracted APK Signing Block at the correct offset
(adding zero padding if needed) and updates the central directory (CD) offset in
the end of central directory (EOCD) record.

For more information about the ZIP file format, see e.g. the Wikipedia
article
.

What does the “APK Signing Block offset < central directory offset” error mean?

It means that apksigcopier can’t insert the APK Signing Block at the required
location, since that offset is in the middle of the ZIP data (instead of right
after the data, before the central directory).

In other words: the APK you are trying to copy the signature to is larger than
the one the signature was copied from. Thus the signature cannot be copied (and
could never have been valid for the APK you are trying to copy it to).

In the context of verifying reproducible builds,
getting this error almost certainly means the build was not reproducible.

What does the “Unexpected metadata” error mean?

It almost always means the target APK was signed; you can only copy a signature
to an unsigned APK.

What about signatures made by apksigner from build-tools >= 35.0.0-rc1?

Since build-tools >= 35.0.0-rc1, backwards-incompatible changes to apksigner
break apksigcopier as it now by default forcibly replaces existing alignment
padding and changed the default page alignment from 4k to 16k (same as Android
Gradle Plugin >= 8.3, so the latter is only an issue when using older AGP).

Unlike zipalign and Android Gradle Plugin, which use zero padding, apksigner
uses a 0xd935 “Android ZIP Alignment Extra Field” which stores the alignment
itself plus zero padding and is thus always at least 6 bytes.

It now forcibly replaces existing padding even when the file is already aligned
as it should be, except when --alignment-preserved is specified, in which case
it will keep existing (non)alignment and padding.

This means it will replace existing zero padding with different padding for each
and every non-compressed file. This padding will not only be different but also
longer for regular files aligned to 4 bytes with zero padding, but often the
same size for .so shared objects aligned to 16k (unless they happened to
require less than 6 bytes of zero padding before).

Unfortunately, supporting this change in apksigcopier without breaking
compatibility with the signatures currently supported would require rather
significant changes. Luckily, there are 3 workarounds available:

First: use apksigner from build-tools <= 34.0.0 (clearly not ideal).

Second: use apksigner sign from build-tools >= 35.0.0-rc1 with the
--alignment-preserved option.

Third: use zipalign.py --page-size 16 --pad-like-apksigner --replace on the
unsigned APK to replace the padding the same way apksigner now does before
using apksigcopier.

What about APKs signed by gradle/zipflinger/signflinger instead of apksigner?

Compared to APKs signed by apksigner, APKs signed with a v1 signature by
zipflinger/signflinger (e.g. using gradle) have different ZIP metadata —
create_system, create_version, external_attr, extract_version,
flag_bits — and compresslevel for the v1 signature files (.SF,
.RSA/.DSA/.EC, MANIFEST.MF); they also usually have a 132-byte virtual
entry at the start as well.

Recent versions of apksigcopier will detect these ZIP metadata differences and
the virtual entry (if any); extract will save them in a differences.json
file (if they exist), which patch will read (if it exists); copy and
compare simply pass the same information along internally.

CAVEAT for compare

NB: because compare copies from the first APK to the second, it will fail when
only the second APK is v1-signed with zipflinger/signflinger; e.g.

  1. $ compare foo-signflinger.apk foo-apksigner.apk # copies virtual entry; works
  2. $ compare foo-apksigner.apk foo-signflinger.apk # only 2nd APK has virtual entry
  3. DOES NOT VERIFY
  4. [...]
  5. Error: failed to verify /tmp/.../output.apk.

What are these virtual entries?

A virtual entry is a ZIP entry with an empty filename, an extra field filled
with zero bytes, and no corresponding central directory entry (so it should be
effectively invisible to most ZIP tools).

When zipflinger deletes an entry it leaves a “hole” in the archive when there
remain non-deleted entries after it. It later fills these “holes” with virtual
entries.

There is usually a 132-byte virtual entry at the start of an APK signed with a
v1 signature by signflinger/zipflinger; almost certainly this is a default
manifest ZIP entry created at initialisation, deleted (from the central
directory but not from the file) during v1 signing, and eventually replaced by a
virtual entry.

Depending on what value of Created-By and Built-By were used for the default
manifest, this virtual entry may be a different size; apksigcopier supports
any size between 30 and 4096 bytes.

Installing

Debian

Official packages are available in
Debian and
Ubuntu.

  1. $ apt install apksigcopier

You can also manually build a Debian package using the debian/sid
branch, or download a pre-built .deb via GitHub releases.

NixOS & Arch Linux

Official packages are also available in
nixpkgs and
Arch Linux
(and derivatives).

Using pip

  1. $ pip install apksigcopier

NB: depending on your system you may need to use e.g. pip3 --user
instead of just pip.

From git

NB: this installs the latest development version, not the latest
release.

  1. $ git clone https://github.com/obfusk/apksigcopier.git
  2. $ cd apksigcopier
  3. $ pip install -e .

NB: you may need to add e.g. ~/.local/bin to your $PATH in order
to run apksigcopier.

To update to the latest development version:

  1. $ cd apksigcopier
  2. $ git pull --rebase

Dependencies

  • Python >= 3.7 + click.
  • The compare command also requires apksigner.

Debian/Ubuntu

  1. $ apt install python3-click
  2. $ apt install apksigner # only needed for the compare command

License

GPLv3+