AOSP - Setup

reference: https://source.android.com/setup/build/requirements


Android is an open source software stack created for a wide array of devices with different form factors.

The core Android source code is known as Android Open Source Project (AOSP).
The AOSP code can be found without modification on select devices, mainly the Nexus and Pixel series of devices.
The source code is, in turn, customized and adapted by original equipment manufacturers (OEMs) to run on their hardware.

Establishing a Build Environment

To set up your local work environment to build the Android source files.

Setting up a Linux build environment

• Installing required packages

$sudo apt-get install git-core gnupg flex bison gperf build-essential zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z-dev libgl1-mesa-dev libxml2-utils xsltproc unzip

• Configuring USB access
• Using a separate output directory
If your machine has multiple storage devices, builds are faster when storing the source files and the output on separate volumes. By default, the output of each build is stored in the out/ subdirectory of the matching source tree. To set this up, export the OUT_DIR_COMMON_BASE variable to point to the location where your output directories will be stored.

export OUT_DIR_COMMON_BASE=

Codelines, Branches, and Releases

• An upstream project is an open source project (such as the Linux kernel and WebKit) from which the Android stack pulls code.
Generally, these projects are developed entirely in the public tree.
• A release corresponds to a formal version of the Android platform.
A release of the platform corresponds to the version in the SdkVersion field of AndroidManifest.xml files and defined within frameworks/base/api in the source tree.
• Experimental codelines are established to capture changes from the community so they can be iterated on with an eye toward stability.
• Changes that prove stable are eventually be pulled into a release branch. This applies only to bug fixes, application improvements, and other changes that do not affect the APIs of the platform.

Codenames, Tags, and Build Numbers

Android development happens around families of releases that use code names ordered alphabetically after tasty treats.
The code names match the following version numbers, along with API levels and NDK releases provided for convenience:

Code name	Version	API level
Oreo	8.1.0	27
Oreo	8.0.0	26
Nougat	7.1	25

In Android 8.0 (Oreo) and higher, individual builds are identified with the build ID format PVBB.YYMMDD.bbb[.Cn], where:

• P represents the first letter of the codename of the platform release, for example, O is Oreo.
• V represents a supported vertical. By convention, P represents the primary platform branch.
• BB is an alphanumeric code that allows Google to identify the exact code branch the build was made from.
• YYMMDD identifies the date when the release is branched from or synced with the development branch. It's not always the exact date at which a build was made as it's common for minor variations added to an existing build to re-use the same date code as the existing build.
• bbb identifies individual versions related to the same date code, sequentially starting with 001.
• Cn is an optional, alphanumeric that identifies a hotfix on top of an existing PVBB.YYMMDD.bbb build, starting from A1.

Build	Branch	Version	Supported devices
OPM4.171019.016.C1	android-8.1.0_r29	Oreo	Pixel C
OPM4.171019.016.B1	android-8.1.0_r28	Oreo	Pixel XL, Pixel, Pixel 2 XL, Pixel 2

Source code tags and builds

Both tags and branches point to a commit, they are thus aliases for a specific hash and will save you time by not requiring to type in a hash.

The difference between tags and branches are that a branch always points to the top of a development line and will change when a new commit is pushed whereas a tag will not change. Thus tags are more useful to "tag" a specific version and the tag will then always stay on that version and usually not be changed.

In practice, tags are used to designate the software versioning, and are named with numbers (ex: v1.0.2).

Downloading and Building

Hardware requirements

• A 64-bit environment is required
• At least 100GB of free disk space to checkout the code and an extra 150GB to build it.
• If you are running Linux in a virtual machine, you need at least 16GB of RAM/swap.

Software requirements

Downloading the Source

• install Repo
Repo is a tool that makes it easier to work with Git in the context of Android. Repo comes in two parts: One is a launcher script you install, and it communicates with the second part, the full Repo tool included in a source code checkout. Repo helps manage many Git repositories. The repo command is an executable Python script that you can put anywhere in your path. Make sure you have a bin/ directory in your home directory and that it is included in your path:

$ mkdir ~/bin
$ PATH=~/bin:$PATH

Download the Repo script and ensure that it is executable:

$ curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
$ chmod a+x ~/bin/repo

• Create an empty directory to hold your working files

$ mkdir aosp
$ cd aosp

• Download the source
Run repo init to get the latest version of the full Repo tool with its most recent bug fixes:

$ repo init -u https://android.googlesource.com/platform/manifest

Or, to check out a specific branch other than master, specify it with -b :

$ repo init -u https://android.googlesource.com/platform/manifest -b android-4.0.1_r1  

A successful initialization will end with a message stating that "Repo is initialized in" your working directory. Your client directory should now contain a .repo directory where files such as the manifest(.repo/manifests/default.xml) will be kept. To pull down the Android source tree to your working directory from the repositories as specified in the default manifest, run:

$ repo sync

Downloads new changes and updates the working files in your local environment. If you run repo sync without arguments, it synchronizes the files for all projects. When you run repo sync, this is what happens:
• If the project has never been synchronized, then repo sync is equivalent to git clone. All branches in the remote repository are copied to the local project directory.
• If the project has been synchronized before
then repo sync is equivalent to:

git remote update
git rebase origin/branch

After a successful run of repo sync,

Checking out projects: 100% (609/609), done.
repo sync has finished successfully.

the code in specified projects is up to date and synced with the code in the remote repository. The Android source files are located in your working directory.

If Linux clients experience connectivity issues, getting stuck in the middle of downloads (typically during receiving objects). Modify the setting:

$ sudo sysctl -w net.ipv4.tcp_window_scaling=0

If you see the following error while syncing :

fatal: The remote end hung up unexpectedly

You can use the following to work around it:

$ git config --global http.postBuffer 2428000

The Manifest

A manifest file is an XML file, describing a list of repositories to sync our working directory with.
We can take repositories from different git servers, use different branches for each one of them, gather them into groups for easy management and more.
Here’s an example for a manifest xml file:

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
  <remote  name="aosp"
           fetch=".."
           review="https://android-review.googlesource.com/" />
  <default revision="refs/tags/android-o-mr1-iot-release-1.0.14"
           remote="aosp"
           sync-j="4" />
  <!-- BEGIN open-source projects -->
...
<!-- END open-source projects -->
  <!-- BEGIN Android Things open-source projects -->
  <project path="external/u-boot" name="platform/external/u-boot" groups="iot" />
  <project path="hardware/bsp/bootloader/nxp/uboot-imx" name="platform/hardware/bsp/bootloader/nxp/uboot-imx" groups="imx7d" />
  <project path="hardware/bsp/kernel/common/v4.4" name="platform/hardware/bsp/kernel/common/v4.4" />
  <project path="hardware/bsp/kernel/nxp/imx-v4.9" name="platform/hardware/bsp/kernel/nxp/imx-v4.9" groups="imx7d" />
  <project path="hardware/bsp/kernel/raspberrypi/pi-v4.4" name="platform/hardware/bsp/kernel/raspberrypi/pi-v4.4" groups="rpi" />
  <!-- END Android Things open-source projects -->
</manifest>

Let’s review its structure:

• <remote>
The remote tag is describing the remote git servers we support pulling repositories from.
• <project>
Defines a single repository. These are the main attributes:
• path
Where the repository will be checked out into, relative to the current working directory.
• name
The name of the project on our git server.
• groups
The group name(s) for this project. We can declare a set of projects as members of a group, then sync only them. Omitting this attribute makes the project a member of the “default” group.
• <default>
Defines a default values for attributes we are using, when syncing the system. If we omit those attributes when defining a project, the defaults defined here will be used. These are the main attributes:
• sync-j
The value for this will be the number of threads to use when syncing the system. Parallelization helps get the job done quicker, but could also get the computer or the network stuck. Using 4 threads is what commonly used for syncing.

Build toolchain

Android 8.0 and higher support only Clang/LLVM for building the Android platform.
For the Native Development Kit (NDK) and legacy kernels, GCC 4.9 included in the AOSP master branch (under prebuilts/) may also be used.

Preparing to Build

• Clean up
Delete the existing output of any previous build using:

make clobber

• Set up environment
Initialize the environment with the envsetup.sh script.

source build/envsetup.sh

• Choose a target
Choose which target to build with lunch. If run with no arguments lunch will prompt you to choose a target from the menu.

lunch build-buidtype

All build targets take the form build-buidtype, where:
• build
is a codename referring to the particular feature combination. Here's a partial list:

Build name	Device	Notes
aosp_arm	ARM emulator	AOSP, fully configured with all languages, apps, input methods
aosp_maguro	maguro	AOSP, running on Galaxy Nexus GSM/HSPA+ ("maguro")
aosp_panda	panda	AOSP, running on PandaBoard ("panda")

• buildtype
is one of the following:
• user
limited access; suited for production
• userdebug
like user but with root access and debuggability; preferred for debugging
• eng
development configuration with additional debugging tools
• Build the code
GNU make can handle parallel tasks with a -jN argument, and it's common to use a number of tasks N that's between 1 and 2 times the number of hardware threads on the computer being used for the build. "lscpu" can show you all the related information. For ex.: 2 CPUs, 4 cores per CPU, 2 threads per core. The fastest builds are made with commands between make -j16 and make -j32.

make -j8

• Run it!
You can either run your build on an emulator or flash it on a device. Please note that you have already selected your build target with lunch, and it is unlikely to run on a different target than it was built for.
• Flash with fastboot
To flash a device, you will need to use fastboot, which should be included in your path after a successful build.
• Emulate an Android device
The emulator is added to your path automatically by the build process. To run the emulator, type:

emulator

Compiling with Jack

https://source.android.com/setup/build/jack

Jack is an Android toolchain that compiled Java source into Android dex bytecode.
Jack has its own .jack file format that contains the pre-compiled dex code for the library

The Jill tool translates the existing .jar libraries into the new library format

The Jack server's goal is to handle a pool of Jack compiler instances in order to limit memory usage and benefit from already warm instances.

Using Reference Boards

https://source.android.com/setup/build/devices

Google supports HiKey960 and HiKey certified 96Boards as Android reference boards.

• Compiling userspace
Download the Android source:

repo init -u https://android.googlesource.com/platform/manifest -b master
repo sync -j24

Build the code:

source ./build/envsetup.sh
lunch hikey960-userdebug
make -j32

• Installing initial images
Select fastboot mode(for details, refer to the HiKey960 user guide) on the reference board then boot it. Flash initial images:

cd device/linaro/hikey/installer/hikey960
./flash-all.sh

• Building the kernel
Build:

git clone https://android.googlesource.com/kernel/hikey-linaro
cd hikey-linaro
git checkout -b android-hikey-linaro-4.9 origin/android-hikey-linaro-4.9
make ARCH=arm64 hikey960_defconfig
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-android- -j24

Update the built kernel and device tree in the boot image:

cp arch/arm64/boot/dts/hisilicon/hi3660-hikey960.dtb hikey-kernel/hi3660-hikey960.dtb-4.9
cp arch/arm64/boot/Image.gz-dtb  hikey-kernel/Image.gz-dtb-hikey960-4.9

Make the boot image:

make bootimage -j24

Running Builds

Fastboot is a bootloader mode in which you can flash a device.

• Booting the devce into fastboot mode
• Flashing a device

Building Kernels

After the building is successful, the built kernel is under android/out/target/product/xxx .

• Identifying kernel version

dd if=kernel bs=1 skip=$(LC_ALL=C grep -a -b -o $'\x1f\x8b\x08\x00\x00\x00\x00\x00' kernel | cut -d ':' -f 1) | zgrep -a 'Linux version'

• Downloading kernel sources
A full list of the kernel projects can be found in the Kernel directory "android/kernel". Below are some of the commonly used kernels:
• exynos
The kernel sources for Nexus 10, and can be used as a starting point for work on Samsung Exynos chipsets.

git clone https://android.googlesource.com/kernel/exynos

• goldfish
The kernel sources for the emulated platforms.

git clone https://android.googlesource.com/kernel/goldfish

• omap
can be used as a starting point for work on TI OMAP chipsets.

git clone https://android.googlesource.com/kernel/omap

• tegra
can be used as a starting point for work on NVIDIA Tegra chipsets.

git clone https://android.googlesource.com/kernel/tegra

Note: Kernel names differ by device. To locate the correct filename for your kernel, refer to device/VENDOR/NAME in the kernel source.
• Building the kernel
After downloadeding the kernel source and prebuilt gcc, you are ready to build the kernel:

export ARCH=arm64
export CROSS_COMPILE=aarch64-linux-android-
cd kernelProject
git checkout -b android-kernelProject-4.1 origin/android-kernelProject-4.1
make vendor_defconfig
make

This will generate:
• arch/arm64/boot/Image
• arch/arm64/boot/dts/VENDOR/*.dtb
Copy these to device/VENDOR/NAME directory.

Developing

Repo unifies Git repositories when necessary, performs uploads to the Gerrit revision control system, and automates parts of the Android development workflow.

Workflow

• Start a new topic branch using repo start.
• Edit the files.
• Stage changes using git add.
• Commit changes using git commit.
• Upload changes to the review server using repo upload.

Common tasks

• repo init
Initializes a new client.
• repo sync
Syncs client to repositories.
• repo start
Starts a new branch.
• repo status
Shows status of current branch.
• repo upload
Uploads changes to the review server.
• git add
Stages files.
• git commit
Commits staged files.
• git branch
Shows current branches.
• git branch [branch]
Creates new topic branch.
• git checkout [branch]
Switches HEAD to specified branch.
• git merge [branch]
Merges [branch] into current branch.
• git diff
Shows diff of unstaged changes.
• git diff --cached
Shows diff of staged changes.
• git log
Shows history of current branch.
• git log m/[codeline]..
Shows commits that are not pushed.