BuildAOSP10Kernel

Jan 29, 2022
Technology

为模拟器emulator编译aosp10的内核.

内核源码获取

需要获取模拟器自身的内核代码，方法是通过adb shell进入到shell下运行uname -a。

尽量找和默认内核差不多的版本的内核进行编译，否则有可能编译出的内核无法启动系统。这里选择的是4.14.112+.

$ git clone https://android.googlesource.com/kernel/goldfish.git
$ git checkout -b android-goldfish-4.14-gchips remotes/origin/android-goldfish-4.14-gchips
$ git branch 
* android-goldfish-4.14-gchips
  master

获取的内核源码架构如下：

# ls
arch                          build-kernel.sh  CREDITS        firmware  ipc      lib          modules.builtin  README    sound       verity_dev_keys.x509
block                         built-in.o       crypto         fs        Kbuild   MAINTAINERS  modules.order    samples   System.map  virt
build.config.goldfish.arm64   certs            Documentation  include   Kconfig  Makefile     Module.symvers   scripts   tools       vmlinux
build.config.goldfish.x86_64  COPYING          drivers        init      kernel   mm           net              security  usr         vmlinux.o

内核编译链获取

内核编译链在编译好的aosp10源码下，编译方法不再详细讲解。

# ls prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.9/
bin  COPYING  COPYING3  COPYING3.LIB  COPYING.LIB  COPYING.RUNTIME  lib  libexec  MODULE_LICENSE_GPL  NOTICE  OWNERS  repo.prop  x86_64-linux-android
# ls prebuilts/qemu-kernel/build-kernel.sh
prebuilts/qemu-kernel/build-kernel.sh

编译

android-goldfish-4.14-gchips内核中需要做以下修改以通过编译:

# vim /security/selinux/include/classmap.h
添加:
    #include <linux/socket.h>
# vim scripts/selinux/mdp/mdp.c 
# scripts/selinux/genheaders/genheaders.c
去掉头文件包含的 #include <sys/socket.h>

编译配置:

export PATH=$PATH:/home/xxxxx/Code/aosp10/prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.9/bin
export ARCH=x86_64
export CROSS_COMPILE=x86_64-linux-android-
export REAL_CROSS_COMPILE=x86_64-linux-android-

创建内核定义文件：

# cp arch/x86/configs/x86_64_ranchu_defconfig  arch/x86/configs/x86_64_emu_defconfig

编译:

# /home/xxxx/Code/aosp10/prebuilts/qemu-kernel/build-kernel.sh --arch=x86_64

最终编译成品:

# cp /tmp/kernel-qemu/x86_64-4.14.112/kernel-qemu  ~/

测试内核

emulator启动的时候显式指定所需的内核:

# emulator -show-kernel -kernel ~/kernel-qemu

检测内核的方式同样是通过adb shell进入到命令行界面下查看内核。

RunWaydroidUbportsOnPixel3a

Jan 25, 2022
Technology

最近会研究一些安卓虚拟化的知识，这里记录下来在Pixel 3a上刷Ubports后，在Ubports中启用waydroid的过程。

1. 准备工作/相关概念

Pixel 3a一支，淘宝很容易买到，注意要买可解锁BootLoader的，价格大概是四五百块，取决于成色。

选择Pixel 3a是因为它是Ubports官方支持中比较完善的几个之一，官方的设备支持列表如下：
https://devices.ubuntu-touch.io/?pk_vid=5294f9f7280f826e164306913156a7ee

当然如果手头有别的可以刷的手机也可以尝试下, Pixel 3a的刷机体验是比较好的。

Ubports: Ubuntu touch.
Waydroid: 以前称为Anbox-Halium，是Anbox 的重建版本，旨在使用比Anbox 更新更流畅的安卓算力体验。

2. 操作指南

2.1 刷入安卓9

参考https://devices.ubuntu-touch.io/device/sargo/

全翻墙的情况下，访问:

https://flash.android.com/release, 按提示操作, 记得选择图示为PQ3B.190801.002的安卓9版本。

/images/2022_01_25_09_06_50_621x486.jpg

看到此提示则代表刷入已经成功:

/images/2022_01_25_07_58_49_612x183.jpg 刷入成功后会提示需要lock bootloader，按照提示先将bootloader锁定后，进入到安卓9中执行初始化操作. 接着进入到解锁bootloader以便刷Ubports.

2.2 解锁bootloader:

首先要做的是开启Usb调试 关于手机->版本号上狂点，直至打开开发者模式，而后系统->开发者选项->调试->USB调试.

# adb devices
List of devices attached
xxxxxx5	device

# adb reboot bootloader

此时手机会停在fastboot阶段，在终端中继续敲入

# fastboot flashing unlock
OKAY [  0.338s]
Finished. Total time: 0.338s

此时在手机上按音量-键选择到unlock bootloader的选项后，继续按电源键确定。bootloader解锁成功后需重新启动并重新设置手机。此时需要重新开启usb调试等。

2.3 刷入ubports

再次强调: ubports刷入的先决条件是pixel 3a上运行的安卓版本为PQ3B.190801.002。

ArchLinux上通过snap安装ubports刷机软件, 参考网址如下:

https://snapcraft.io/install/ubports-installer/arch

开启刷机软件开始刷ubports:

sudo ubports-installer

/images/2022_01_25_08_25_47_803x441.jpg

确认手机型号:

/images/2022_01_25_08_26_04_657x371.jpg

选择devel版(如果不是为了体验waydroid也可以选择为stable版), 一定要选择wipe userdata，否则刷机将一直卡顿不会成功:

/images/2022_01_25_08_26_38_621x462.jpg

这里提示在fastboot菜单中需要选择为Recovery Mode才可以继续安装，在手机上按音量-键结合电源键选择，进入到恢复模式后点下一步:

/images/2022_01_25_08_27_53_716x365.jpg

现在ubports-installer将自动下载镜像并进行刷机，刷机速度取决于下载速度, 推荐在空闲时段操作，在早上7、8点时大概几分钟时间刷好:

/images/2022_01_25_08_29_20_554x581.jpg

刷机软件提示出以下界面的时候，手机将重启，这时需要耐心等待手机上的刷机进程完成写入操作，刷机过程中会有个黄色的小标志在不停转动，而后将重启:

/images/2022_01_25_08_31_21_651x492.jpg

经过一系列设置之后，Ubports将启动成功并进入到系统中。

2.4 安装waydroid

确认自己安装的是开发者版本，如果是stable的话，在关于->检查更新下的配置按钮中，切换成开发者版本就可以了:

/images/2022_01_25_08_47_25_478x309.jpg

waydroid中也是可以使用adb命令的，使用adb命令连接到手机上后，运行waydroid的安装操作, adb命令的一个参考如下(以下参考演示了如何连接到手机、如何从手机取回截屏):

$ adb devices
List of devices attached
92UAY04L95	device

$ adb shell
phablet@ubuntu-phablet:~$ ls
Desktop  Documents  Downloads  Music  Pictures  Public  Templates  Videos
phablet@ubuntu-phablet:~$ cd Pictures/
phablet@ubuntu-phablet:~/Pictures$ ls
Screenshots
phablet@ubuntu-phablet:~/Pictures$ cd Screenshots/
phablet@ubuntu-phablet:~/Pictures/Screenshots$ ls
screenshot20220125_084411555.png
phablet@ubuntu-phablet:~/Pictures/Screenshots$ exit

$ adb pull /home/phablet/Pictures/Screenshots/screenshot20220125_084411555.png
/home/phablet/Pictures/Screenshots/screenshot20220125_084411555.png: 1 file pulled, 0 skipped. 2.7 MB/s (76661 bytes in 0.027s)

可以检查一下磁盘占用情况, 确保安装waydroid具备足够的磁盘空间:

dfphablet@ubuntu-phablet:~$ df -h
Filesystem       Size  Used Avail Use% Mounted on
/dev/root        3.0G  2.2G  804M  73% /
devtmpfs         1.7G  604K  1.7G   1% /dev
tmpfs            1.8G  1.3M  1.8G   1% /run
/dev/loop0       268M  267M     0 100% /android
/dev/mmcblk0p72   49G   27M   47G   1% /userdata
none             4.0K     0  4.0K   0% /sys/fs/cgroup
cgmfs            100K     0  100K   0% /run/cgmanager/fs
tmpfs            1.8G   32K  1.8G   1% /tmp
none             5.0M     0  5.0M   0% /run/lock
none             1.8G  172K  1.8G   1% /run/shm
none             100M     0  100M   0% /run/user
tmpfs            1.8G     0  1.8G   0% /media
tmpfs            1.8G     0  1.8G   0% /var/lib/openvpn/chroot/tmp
tmpfs            1.8G     0  1.8G   0% /var/lib/sudo
/dev/mmcblk0p71  755M  512M  228M  70% /android/vendor
tmpfs            1.8G     0  1.8G   0% /mnt/vendor
/dev/mmcblk0p48   35M  3.8M   30M  12% /mnt/vendor/persist
/dev/mmcblk0p66   12M   80K   11M   1% /android/metadata
/dev/mmcblk0p22   80M   72M  8.8M  90% /android/vendor/firmware_mnt
tmpfs            356M   40K  356M   1% /run/user/32011
tmpfs            356M     0  356M   0% /run/user/0

adb shell下输入如下命令安装waydroid:

sudo -s
sudo mount -o remount,rw /
apt update
apt install waydroid -y
waydroid init

值得注意的是，waydroid init需要从sourceforge拉取700MB左右的镜像，由于众所周知的原因，国内网络一直不是很好，这一步可能会耗费很长的时间。个人建议是在网络空闲时段(例如起个大早来跑)运行该操作。当然具备动手能力的可以用全翻墙网路来跑，速度会快很多。

waydroid命令实际上是一个python脚本，理论上是可以改一下调整它的镜像初始化机制以使用本地包用于离线安装的, 有兴趣的可以更改源码。

root@ubuntu-phablet:~# which waydroid
/usr/bin/waydroid
root@ubuntu-phablet:~# cd /usr/lib/waydroid/
root@ubuntu-phablet:~# grep -i "http" ./ -r | grep channel
./tools/config/__init__.py:    "system_channel": "http://ota.waydro.id/system",
./tools/config/__init__.py:    "vendor_channel": "http://ota.waydro.id/vendor",

waydroid初始化过程:

root@ubuntu-phablet:~# waydroid init
[08:55:02] Download https://sourceforge.net/projects/waydroid/files/images/system/lineage/waydroid_arm64/lineage-17.1-20211021-VANILLA-waydroid_arm64-system.zip/download
[09:04:37] Validating system image
[09:04:38] Extracting to /var/lib/waydroid/images
[09:05:22] Download https://sourceforge.net/projects/waydroid/files/images/vendor/waydroid_arm64/lineage-16.0-20211020-HALIUM_9-waydroid_arm64-vendor.zip/download
[09:05:42] Validating vendor image
[09:05:42] Extracting to /var/lib/waydroid/images

此时需要重启手机以继续安装。

2.5 启动waydroid

启动完毕后，首先启动waydroid容器:

$ adb shell
phablet@ubuntu-phablet:~$ sudo waydroid container start
[sudo] password for phablet:

然后在另一个adb shell中启动waydroid session，此时第一个shell里的命令会退出，接着就可以点开waydroid了:

$ waydroid session start
[09:15:30] XDG Session is not "wayland"
[09:15:32] Failed to start Clipboard manager service, check logs
[09:15:51] Android with user 0 is ready

接着在手机里点开wayroid则可以开启waydroid界面.

启动这一步一直没太搞明白应该怎样才是正确的操作。但经历以上操作后，再次关机重启后，waydroid都可以直接在ubports下直接点开

如果遇到初始化问题不成功需要重新开始的话，可以使用以下命令清零waydroid安装:

apt remove waydroid
apt purge waydroid
sudo rm -rf /var/lib/waydroid /home/.waydroid ~/waydroid ~/.share/waydroid ~/.local/share/applications/*aydroid* ~/.local/share/waydroid

2.6 体验waydroid

安装应用宝:

$ adb push MobileAssistant_1.apk /home/phablet/
MobileAssistant_1.apk: 1 file pushed, 0 skipped. 6.1 MB/s (10925622 bytes in 1.720s)
$ apks adb shell
phablet@ubuntu-phablet:~$ sudo waydroid app install ~/MobileAssistant_1.apk
[sudo] password for phablet:

/images/2022_01_25_09_25_18_406x838.jpg 接下来的操作就是常规的安装apk, 使用apk。

鲁大师跑快40万分:

/images/2022_01_25_09_49_20_405x834.jpg

LinuxTips14

Jan 10, 2022
Technology

1. ubuntu20.04 use python2

Via:

$ sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1

this solves some aosp building issues when the aosp building tree requires python2.7 to be the default python version.

2. ubports issue

via:

B ) Manual installation

#Open Ubuntu Touch terminal (or a shell session on your host computer) and run:

sudo -s
sudo mount -o remount,rw /
apt update
apt install waydroid -y
waydroid init 
ubports-qa install xenial_-_fixwindowreopen
alternatively, to install Google Apps, replace the line

waydroid init
with:

waydroid init -s GAPPS
#For Vollaphone, add:

waydroid prop set persist.waydroid.height_padding 70

3. lock/unlock bootloader

via:

fastboot flashing lock
fastboot flashing unlock

4. reflashing pixel 3a

visit https://flash.android.com/release/10.0.0.

5. flashing kernel module

via:

# disable-verity on the phone
adb root
adb disable-verity
adb shell sync
adb reboot

# push module
adb root
adb remount
adb push out/android-msm-pixel-4.9/dist/*.ko /vendor/lib/modules/
# 重启设备以后可以设备正常使用 查看内核版本 的确已经是自编译的内核
adb reboot

6. Disable oh-my-zsh upgrade

By adding following lines in ~/.zshrc :

+ DISABLE_UPDATE_PROMPT=true
+ DISABLE_AUTO_UPDATE=true
source $ZSH/oh-my-zsh.sh

7. Install chromium on mips64el

Via:

wget http://okapps.oukan.online/lroapps/deb-loongson/chromium-browser-beta_82.0.4051.0-1_mips64el.deb
wget http://ftp.us.debian.org/debian/pool/main/libi/libindicator/libindicator3-7_0.5.0-4_mips64el.deb
wget http://ftp.us.debian.org/debian/pool/main/liba/libappindicator/libappindicator3-1_0.4.92-7_mips64el.deb
apt-get install -y ./libappindicator3-1_0.4.92-7_mips64el.deb ./libindicator3-7_0.5.0-4_mips64el.deb
apt-get install -y ./chromium-browser-beta_82.0.4051.0-1_mips64el.deb

8. grub configuration(Ubuntu)

Changed back to origin kernel version:

# vim /etc/default/grub
GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 5.4.0-99-generic"
# update-grub2
# update-grub
# reboot

9. Install docker-ce on arm64

First remove conflict packages and import gpg:

sudo apt-get remove docker docker-engine docker.io
sudo apt-get install apt-transport-https ca-certificates curl gnupg2 software-properties-common
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -

Then import repository and install docker-ce:

# vim /etc/apt/sources.list
....
deb [arch=arm64] https://mirrors.aliyun.com/docker-ce/linux/ubuntu focal stable

# apt-get update && apt-get install -y docker-ce

10. run redroid with amd

Command:

$ sudo docker run -itd --name redroid9 --memory-swappiness=0 --privileged   -p 5556:5555 redroid/redroid:9.0.0-latest redroid.fps=60 ro.sf.lcd_density=150 redroid.gpu.mode=host redroid.gpu.node=/dev/dri/renderD128
$ sudo docker run -itd --name redroid12 --memory-swappiness=0 --privileged   -p 5555:5555 redroid/redroid:12.0.0-latest redroid.fps=120 ro.sf.lcd_density=240 redroid.gpu.mode=host redroid.gpu.node=/dev/dri/renderD128

Or with virtio:

sudo docker run -itd --name redroid9 --memory-swappiness=0 --privileged   -p 5555:5555 redroid/redroid:9.0.0-latest redroid.fps=120 ro.sf.lcd_density=240 redroid.width=1080 redroid.height=1920

11. opengl issue

via virgl issue:

https://archived.forum.manjaro.org/t/qemu-kvm-vm-in-virt-manager-is-running-with-a-black-screen-when-opengl-acceleration-is-turned-on/138632/8

12. content of rpm

via:

# rpm -ql {packageName}
# rpm -ql htop
# rpm -qlp /path/to/nginx.rpm

13. apt-fast

install via:

sudo add-apt-repository ppa:apt-fast/stable
sudo apt-get update
sudo apt-get -y install apt-fast

14. ubuntu device driver

Install via:

$ ubuntu-drivers devices

15. yum groupremove

via:

yum groupremove "X Window System"

16. linux-zen

In archlinux via:

# sudo pacman -S linux-zen linux-zen-headers

Then update the entry:

# vim /etc/default/grub
#GRUB_DEFAULT=0
GRUB_DEFAULT="Advanced options for Arch Linux>Arch Linux, with Linux linux-zen"
GRUB_TIMEOUT=3
# grub-mkconfig -o /boot/grub/grub.cfg
# reboot
# uname -a
Linux archsg1 5.16.14-zen1-1-zen #1 ZEN SMP PREEMPT Fri, 11 Mar 2022 17:40:33 +0000 x86_64 GNU/Linux

17. Build linux-zen

Steps:

$ sudo pacman -S base-devel asp
$ asp checkout linux-zen
Copy to some place, and modify the config
$ makepkg --clean --syncdeps --rmdeps

gpg checksum failed, solved via:

==> Verifying source file signatures with gpg...
    archlinux-linux git repo ... FAILED (unknown public key 3B94A80E50A477C7)
==> ERROR: One or more PGP signatures could not be verified!

# vim /etc/pacman.d/gnupg/gpg.conf
no-greeting
no-permission-warning
lock-never
keyserver-options timeout=10
keyserver-options import-clean
keyserver-options no-self-sigs-only
+ keyserver hkps://keys.openpgp.org

# curl -s https://keybase.io/heftig/pgp_keys.asc/\?fingerprint\=a2ff3a36aaa56654109064ab19802f8b0d70fc30 | gpg --import
# gpg --lsign 3B94A80E50A477C7

Rebuild the kernel and install

18. Change password in single

/images/2022_03_18_16_40_49_526x166.jpg

19. find which file belong to rpm

via:

$ rpm -qf file_name

20. linux-zen

Install on ubuntu via:

sudo add-apt-repository ppa:damentz/liquorix && sudo apt-get update
sudo apt-get install linux-image-liquorix-amd64 linux-headers-liquorix-amd64

20. arp-ignore

arp-ignore should be studied

21. Install vmplayer

On Ubuntu or other linux distribution:

$ sudo apt install build-essential linux-headers-generic
$ wget --user-agent="Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0" https://www.vmware.com/go/getplayer-linux
$ chmod +x getplayer-linux
$ sudo ./getplayer-linux --required --eulas-agreed

22. Install mate(Ubuntu)

via:

# apt  install tasksel
# tasksel install ubuntu-mate-desktop

16. rpm2cpio

Generating the working tar.gz files Steps:

# cd workingdirectory/
# cp ../kmod-ukmd-4.19.112-20212.el7.centos.src.rpm .
# rpm2cpio kmod-ukmd-4.19.112-20212.el7.centos.src.rpm| cpio -idmv
#  vim kmod.spec
Change 4.19.112->4.19.12
# rm -f kmod-ukmd-4.19.112.tar.gz 
# cp /root/rpmbuild/SOURCES/kmod-ukmd-4.19.12.tar.gz .
# mv workingdirectory/ kmod-ukmd-4.19.12.cpio
# tar czvf kmod-ukmd-4.19.12.cpio.tar.gz kmod-ukmd-4.19.12.cpio

17. limit bandwidth

via:

https://unix.stackexchange.com/questions/34116/how-can-i-limit-the-bandwidth-used-by-a-process

18. amdgpu centos howto(5700)

Download the rpms from:

https://www.amd.com/en/support/graphics/amd-radeon-5700-series/amd-radeon-rx-5700-series/amd-radeon-rx-5700

Installation Steps:

yum localinstall ./amdgpu-install-21.50.2.50002-1.el7.noarch.rpm 
# change the repo definition to something like: baseurl=https://repo.radeon.com/amdgpu/21.50.2/rhel/7.9/main/x86_64
yumdownloader dkm
rpm -i --nodeps ./dkms-3.0.3-1.el7.noarch.rpm 
yum install gcc elfutils-libelf-devel
amdgpu-install -y --usecase=graphics
# Then install graphical desktop environment, then
systemctl set-default graphical.target

19. virgl On CentOS

Steps:

# yum -y groupinstall Development --skip-broken
# rpm -ivh /mnt/Packages/systemtap-3.3-3.el7.x86_64.rpm /mnt/Packages/systemtap-client-3.3-3.el7.x86_64.rpm /mnt/Packages/systemtap-devel-3.3-3.el7.x86_64.rpm --nodeps
# yum-builddep mesa
# wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
# yum makecache 
# yum install -y meson

Then:

yum-builddep libvirt

20. amd driver in ubuntu

Get the deb and Install via:

sudo apt-get install ./amdgpu-install_21.50.2.50002-1_all.deb
dpkg --add-architecture i386
amdgpu-install -y --usecase=graphics

21. zen kernel in ubuntu

Install via:

sudo add-apt-repository ppa:damentz/liquorix && sudo apt-get update
64-bit:
sudo apt-get install linux-image-liquorix-amd64 linux-headers-liquorix-amd64

22. convert vdi to img

via:

root@lucky:/home/dash# cd /home/dash/manjavoLinux/
root@lucky:/home/dash/manjavoLinux# VBoxManage clonehd --format RAW manjavoLinux.vdi manjaro.img
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Clone medium created in format 'RAW'. UUID: 8187b47d-86bb-4156-9996-48e921a53e7

23. ssh push to github

via adding following to ~/.ssh/config:

Host github.com
        Hostname ssh.github.com
        Port 443

24. Disable gdm sleeping

via:

sudo -u gdm dbus-launch gsettings set org.gnome.settings-daemon.plugins.power sleep-inactive-ac-type 'nothing'

25. last crash log

via:

journalctl --since=today

26. check virt host

via:

# virt-host-validate qemu
  QEMU: Checking if device /dev/kvm exists                                   : PASS
  QEMU: Checking if device /dev/kvm is accessible                            : PASS
  QEMU: Checking if device /dev/vhost-net exists                             : PASS
  QEMU: Checking if device /dev/net/tun exists                               : PASS
  QEMU: Checking for cgroup 'cpu' controller support                         : PASS
  QEMU: Checking for cgroup 'cpuacct' controller support                     : PASS
  QEMU: Checking for cgroup 'cpuset' controller support                      : PASS
  QEMU: Checking for cgroup 'memory' controller support                      : PASS
  QEMU: Checking for cgroup 'devices' controller support                     : PASS
  QEMU: Checking for cgroup 'blkio' controller support                       : PASS
WARN (Unknown if this platform has IOMMU support)
  QEMU: Checking for secure guest support                                    : WARN (Unknown if this platform has Secure Guest support)

27. file belongs to apk

via:

$ apt-file search filename

28. screenshot

via:

xwd -root -out screenshot.xwd

29. build ampere kernel

Steps:

unzip ampere-lts-kernel-linux-5.4.y.zip 
cd ampere-lts-kernel-linux-5.4.y/
cp arch/arm64/configs/altra_5.4_defconfig .config
make olddefconfig
make menuconfig
make -j128 deb-pkg LOCALVERSION=-amp54-lts

But the kernel boot failed and let the system hang.

30. virtio-gpu On Arm

Via following command:

DISPLAY=:61 qemu-system-aarch64  -M virt-4.2 -machine gic-version=max -m 15192M -enable-kvm -cpu host -smp 8,sockets=1,cores=8,threads=1   \
	-netdev user,id=vnet,hostfwd=:127.0.0.1:2278-:22,hostfwd=tcp::5555-:5555,hostfwd=tcp::5556-:5556,hostfwd=tcp::5557-:5557,hostfwd=tcp::5800-:5800,hostfwd=tcp::3389-:3389,hostfwd=tcp::14000-:4000 -device virtio-net-pci,netdev=vnet \
	-drive file=ubuntu-image.img,if=none,id=drive0,cache=writeback -device virtio-blk,drive=drive0,bootindex=0 -display gtk,gl=on  \
	-drive file=ubuntu-20.04.3-live-server-arm64.iso,if=none,id=drive1,cache=writeback -device virtio-blk,drive=drive1,bootindex=1 \
	-drive file=flash0.img,format=raw,if=pflash -drive file=flash1.img,format=raw,if=pflash  -vga none -device virtio-gpu-pci,virgl=on -usb  -device usb-ehci,id=ehci -device usb-mouse -device usb-kbd

31. Manually add pool

Via:

 virsh pool-create-as --name sda --type dir --target /media/sda/images

32. no-machine issue

Change the configuration then restart the service, needn’t confirm on logged session:

grep DesktopAuthorization /usr/NX/etc/server.cfg
sudo /etc/NX/nxserver --restart

33. centos 8 docker installation

via:

# dnf remove -y podman
# REPO=https://mirrors.aliyun.com/docker-ce
# dnf config-manager --add-repo $REPO/linux/centos/docker-ce.repo
# sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
# sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*
# yum update
# dnf install --allowerasing -y containerd.io docker-ce docker-ce-cli
# docker -v

34. before build aosp

via:

apt install openssh-server screen  git openjdk-8-jdk android-tools-adb bc bison build-essential curl flex g++-multilib gcc-multilib gnupg gperf imagemagick lib32ncurses-dev lib32readline-dev lib32z1-dev  liblz4-tool libncurses5-dev libsdl1.2-dev libssl-dev libxml2 libxml2-utils lzop pngcrush rsync schedtool squashfs-tools xsltproc yasm zip zlib1g-dev libtinfo5 libncurses5

35. Install golang for ubuntu20.04

via:

sudo add-apt-repository ppa:longsleep/golang-backports 
sudo apt update
sudo apt install golang-go

36. Ubuntu driver install

via:

$ ubuntu-drivers devices
== /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0 ==
modalias : pci:v000010DEd00001287sv00001642sd00003F86bc03sc00i00
vendor   : NVIDIA Corporation
model    : GK208B [GeForce GT 730]
driver   : nvidia-driver-390 - distro non-free
driver   : nvidia-driver-470-server - distro non-free
driver   : nvidia-driver-450-server - distro non-free
driver   : nvidia-driver-418-server - distro non-free
driver   : nvidia-driver-470 - distro non-free recommended
driver   : xserver-xorg-video-nouveau - distro free builtin
$ sudo ubuntu-drivers autoinstall

37. cuttlefish render mode

via:

# launch_cvd --gpu-mode=drm_virgl
# launch_cvd --gpu-mode=gfxstream

38. forward to vm

via iptables:

# iptables -I FORWARD -o virbr1 -d 10.17.18.2 -j ACCEPT
# iptables -t nat -I PREROUTING -p tcp --dport 2422 -j DNAT --to 10.17.18.2:22

39. Recompile Ubuntu Linux

via:

Clone the repo:

 git clone git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/focal
Checkout the branch:

 cd focal
 git checkout origin/hwe-5.8
If you want to disable the nvidia driver, add this patch:

diff --git a/debian/rules b/debian/rules
index 8c3c71a..8742713 100755
--- a/debian/rules
+++ b/debian/rules
@@ -100,6 +100,8 @@ ifeq ($(do_mainline_build),true)
        skipretpoline=true
 endif
 
+do_dkms_nvidia=false
+
 # Disable tools build and packaging if do_tools != true
 ifneq ($(do_tools),true)
        do_linux_tools=
Install the build dependencies:

 sudo apt-get build-dep linux
Build the kernel:

 fakeroot debian/rules clean
 fakeroot debian/rules binary-headers binary-generic

40. iptables forbid only one ip

only one ip to specific port running ss server via:

sudo iptables -I INPUT \! --src 121.8.254.210 -m tcp -p tcp --dport 1080 -j DROP

41. yarn issue

via:

sudo apt remove cmdtest
sudo apt remove yarn
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt-get update
sudo apt-get install yarn -y
then

yarn install

42. efi grub issue

via:

#GRUB_DEFAULT=0
GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 5.11.0-46-generic"
#GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT_STYLE=menu
GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT=""
GRUB_CMDLINE_LINUX=""

43. disable auto-update ubuntu

via:

# vim /etc/apt/apt.conf.d/20auto-upgrades
Change content:
FROM:

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
TO:

APT::Periodic::Update-Package-Lists "0";
APT::Periodic::Download-Upgradeable-Packages "0";
APT::Periodic::AutocleanInterval "0";
APT::Periodic::Unattended-Upgrade "1"

Then:

systemctl stop apt-daily.timer;systemctl disable apt-daily.timer ; systemctl stop apt-daily-upgrade.timer ; systemctl disable apt-daily-upgrade.timer; systemctl stop apt-daily.service;  systemctl mask apt-daily.service; systemctl daemon-reload
systemctl stop unattended-upgrades.service && systemctl disable unattended-upgrades.service

44. start redroid

command:

sudo docker run -itd --name redroid9 --memory-swappiness=0 --privileged   -p 5556:5555 redroid:9officialhoudini redroid.fps=120 ro.sf.lcd_density=240 redroid.gpu.mode=host redroid.gpu.node=/dev/dri/renderD128 ro.sf.lcd_density=240 redroid.width=1080 redroid.height=1920

45. Upgrade Kernel

In centos via:

# 查看可用内核版本及启动顺序
$ sudo awk -F\' '$1=="menuentry " {print i++ " : " $2}' /boot/grub2/grub.cfg

# 查看启动顺序
$ yum install -y grub2-pc
$ grub2-editenv list

# 设置开机启动
$ grub2-set-default 0
或者
$ grub2-mkconfig -o /boot/grub2/grub.cfg

# 重启生效
$ reboot

# 重启后查看内核版本
$ uname -sr

46. print photos

via:

https://www.phoyosystem.com/photo-booth-software/#the-id-photos-printing-tool-only

47. untar deb

command:

mkdir tmp
dpkg-deb -R abcde.deb  tmp

48. linux-zen for ArchLinux

Build via:

# vim trunk/config
Added something.
# makepkg --skipinteg --syncdeps

Install via:

# sudo pacman -U linux-zen-5.19.zen1-1-x86_64.pkg.tar.zst linux-zen-headers-5.19.zen1-1-x86_64.pkg.tar.zst linux-zen-docs-5.19.zen1-1-x86_64.pkg.tar.zst
# sudo grub-mkconfig -o /boot/grub/grub.cfg
# sudo reboot

Verify its kernel version:

# uname -r

49. disable gdm sleep

via:

sudo -u gdm dbus-launch gsettings set org.gnome.settings-daemon.plugins.power sleep-inactive-ac-type 'nothing'

50. redroid 12 issue

on linux-zen kernel, 5.19, should enter the adb shell and run:

setprop sys.use_memfd 1

51. ArchLinux Configuration IP

via:

ip addr add 10.17.18.3/24 dev enp1s0
ip link set enp1s0 up
ip route add default via 10.17.18.1 dev enp1s0
Then set resolv.conf

52. ubuntu install zen kernel

via:

sudo add-apt-repository ppa:damentz/liquorix && sudo apt-get update
sudo apt-get install linux-image-liquorix-amd64 linux-headers-liquorix-amd64

53. ipmi reset lan

via:

ipmitool raw 0x32 0x76 0x08

53. efi grub recover

Type normal after you insert following commands:

/images/2022_08_18_09_26_33_688x238.jpg

normal boot will get you into the normal boot:

/images/2022_08_18_09_27_08_815x579.jpg

54. Ubuntu vfio sg1

commands:

docker run -itd --name redroidkkk --memory-swappiness=0 --privileged   -p 5555:5555 redroid12:latest redroid.fps=120 ro.sf.lcd_density=240 redroid.width=1080 redroid.height=1920 redroid.gpu.mode=host redroid.gpu.node=/dev/dri/renderD128 androidboot.use_memfd=1

Upgrade ubuntu mesa drivers:

sudo add-apt-repository ppa:oibaf/graphics-drivers
sudo apt update
sudo apt upgrade

55. Build linux-zen

via:

sudo INSTALL_MOD_STRIP=1 CONCURRENCY_LEVEL=X fakeroot make-kpkg --initrd kernel_image kernel_headers -j 16

56. find pkg in archlinux

via:

# pacman -Fy
# pacman -F genfstab(get the pkg which provides genfstab)

57. qemu-nbd

via:

modprobe nbd max_part=8

Step 2 - Connect the QCOW2 as a network block device

qemu-nbd --connect=/dev/nbd0 /var/lib/vz/images/100/vm-100-disk-1.qcow2
Step 3 - List partitions inside the QCOW2

fdisk /dev/nbd0 -l
Step 4 - Mount the partition from the VM

mount /dev/nbd0p1 /mnt/somepoint/
You can also mount the filesystem with normal user permissions, ie. non-root:

mount /dev/nbd0p1 /mnt/somepoint -o uid=$UID,gid=$(id -g)
Step 5 - After you're done, unmount and disconnect

umount /mnt/somepoint/
qemu-nbd --disconnect /dev/nbd0
rmmod nbd

58. arm64 mesa upgrade

checked via:

# dpkg -l | grep -i mesa
ii  libegl-mesa0:arm64                            21.2.6-0ubuntu0.1~20.04.2             arm64        free implementation of the EGL API -- Mesa vendor library
ii  libgl1-mesa-dri:arm64                         21.2.6-0ubuntu0.1~20.04.2             arm64        free implementation of the OpenGL API -- DRI modules
ii  libglapi-mesa:arm64                           21.2.6-0ubuntu0.1~20.04.2             arm64        free implementation of the GL API -- shared library
ii  libglu1-mesa:arm64                            9.0.1-1build1                         arm64        Mesa OpenGL utility library (GLU)
ii  libglx-mesa0:arm64                            21.2.6-0ubuntu0.1~20.04.2             arm64        free implementation of the OpenGL API -- GLX vendor library
ii  mesa-utils                                    8.4.0-1build1                         arm64        Miscellaneous Mesa GL utilities
ii  mesa-va-drivers:arm64                         21.2.6-0ubuntu0.1~20.04.2             arm64        Mesa VA-API video acceleration drivers
ii  mesa-vdpau-drivers:arm64                      21.2.6-0ubuntu0.1~20.04.2             arm64        Mesa VDPAU video acceleration drivers
ii  mesa-vulkan-drivers:arm64                     21.2.6-0ubuntu0.1~20.04.2             arm64        Mesa Vulkan graphics drivers

Upgrade to oibaf:

59. print message of last boot

via:

 journalctl --boot=-1

60. new xorg configure

via:

Xorg -configure
cat /root/xorg.conf.new | grep -i busid

61. build glmark2 on centos7

Steps:

$ git clone https://github.com/glmark2/glmark2.git

$  INSTALL_APP ="epel-release centos-release-scl hwdata libX11 libX11-common libXau libXdamage libudev-devel \
libXext libXfixes libXxf86vm libdrm libjpeg-turbo libpciaccess libpng12 libxcb libxshmfence mesa-libEGL \
mesa-libGL mesa-libGLES mesa-libgbm mesa-libglapi libXmu-devel libXi-devel libGL-devel libjpeg-turbo-devel \
libpng12-devel mesa-libEGL-devel mesa-libGLES mesa-libgbm-devel mesa-libGLES-devel mesa-dri-drivers \
devtoolset-4-toolchain"

$  flag=1; for pkg in $INSTALL_APP; do echo "INFO: installing $flag package $pkg ..."; yum install -y "$pkg" &> /dev/null || echo -e "\tERROR: cannot install $pkg ...."; flag=$((flag+1)); done
$ PATH="/opt/rh/devtoolset-4/root/bin:$PATH"
$  ./waf configure --with-flavors="drm-gl,drm-glesv2,x11-gl,x11-glesv2"
$ ./waf  -v
$ ./waf install

$ glmark2
=======================================================
    glmark2 2017.07
=======================================================
    OpenGL Information
    GL_VENDOR:     Intel Open Source Technology Center
    GL_RENDERER:   Mesa DRI Intel(R) Haswell Mobile 
    GL_VERSION:    3.0 Mesa 17.0.1
=======================================================
[build] use-vbo=false: FPS: 2567 FrameTime: 0.390 ms
=======================================================
                                  glmark2 Score: 2567 
=======================================================

change the devtoolset-4 to devtoolsset-8.

62. nvidia-smi slow on ampere 1804

solved via:

sudo nvidia-persistenced --persistence-mode

63. minimum xorg

installed via:

apt install -y xserver-xorg-core

64. repo init aosp_master

via:

repo init -u git://mirrors.ustc.edu.cn/aosp/platform/manifest -b master
repo sync -j4

65. git search

search the commit via somebody:

git log --author='intel.com' --all --grep='SRIOV'

66. lspci tree view

via:

$ lspci -tvv

67. disable lock in gnome

via:

Found couple of ways finally.
First method : is to run in terminal the command

gsettings set org.gnome.desktop.lockdown disable-lock-screen true
That should remove the lock option from the tray. And it can be re-ebabled using reset instead of set in the same command and removing true attribute

gsettings reset org.gnome.desktop.lockdown disable-lock-screen

68. android sdk on aarch64

Steps:

docker run -d -p 5901:5901 -p 2222:22 android-sdk:latest
c38e2d3b3a730b34dc40dc632aa5fa7c6cd32f0631c18d3a965671f1d8115aa1
root@feiteng:/home/ctctest# docker ps
CONTAINER ID   IMAGE                COMMAND                  CREATED         STATUS        PORTS                                                                                        NAMES
c38e2d3b3a73   android-sdk:latest   "/usr/bin/supervisord"   2 seconds ago   Up 1 second   5037/tcp, 0.0.0.0:5901->5901/tcp, :::5901->5901/tcp, 0.0.0.0:2222->22/tcp, :::2222->22/tcp   mystifying_sammet
root@feiteng:/home/ctctest# docker exec -it c38e2d3b3a73 bash
root@c38e2d3b3a73:/# JAVA_HOME=/usr/lib/jvm/java-1
java-1.11.0-openjdk-arm64/ java-11-openjdk-arm64/     
root@c38e2d3b3a73:/# JAVA_HOME=/usr/lib/jvm/java-11-openjdk-arm64/
root@c38e2d3b3a73:/# sdkmanager
[=======================================] 100% Computing updates...             
root@c38e2d3b3a73:/# cd /opt/
android-sdk/         gradle/              kotlinc/             license_accepter.sh  
root@c38e2d3b3a73:/# cd /opt/android-sdk/
root@c38e2d3b3a73:/opt/android-sdk# ls
cmdline-tools  licenses
root@c38e2d3b3a73:/opt/android-sdk# sdkmanager  "platform-tools"
[=======================================] 100% Unzipping... platform-tools/lib64
root@c38e2d3b3a73:/opt/android-sdk# sdkmanager "platforms;android-31"
[=======================================] 100% Unzipping... android-12/framework

Copy from host to docker:  

root@feiteng:/home/ctctest/20220915# docker cp arm64-v8a c38e2d3b3a73:/opt/android-sdk/system-images/android-31/default && docker cp emulator c38e2d3b3a73:/opt/android-sdk/

69. remount in single user

via:

mount -o ro,remount /
Then you could do passwd or other staffs.

WorkingTipsOnAndroidDocker2

Dec 29, 2021
Technology

目的

记录aosp avd 11上启动容器的操作事项.

步骤

因为内核经过修改，故启动的时候需要点击ok后进入系统:

/images/2021_12_29_11_56_51_356x415.jpg

确保wifi连接:

/images/2021_12_29_11_58_07_347x482.jpg

进入到adb shell:

$ adb root
restarting adbd as root
$ adb shell
generic_x86_64:/ # ip addr

添加路由规则:

ip rule add pref 1 from all lookup main
ip rule add pref 2 from all lookup default
ip route add default via 192.168.91.254 dev wlan0
ip rule add from all lookup main pref 30000

启动dockerd进程:

dockerd  --dns=223.5.5.5 --data-root=/data/var/ --ip=192.168.89.153 & >/data/dockerd-logfile 2>&1

启动docker:

# docker run -d --privileged -p 8888:5555 redroid/redroid:8.1.0-latest
# docker run -d --privileged -p 8888:5555 redroid/redroid:9.0.0-latest
# docker run -d --privileged -p 8888:5555 redroid/redroid:10.0.0-latest

AndroidInAosp11AVDWorkingTips

Dec 23, 2021
Technology

0. 目的

本文目的是为了记录如何在基于aosp11的avd开启redroid容器实例。

1. 准备aosp源码并运行avd

下载tsinghua的repo用于同步代码, 由于repo需要使用python3来同步代码，故需要安装python-is-python3包:

$ curl https://mirrors.tuna.tsinghua.edu.cn/git/git-repo -o repo
$ chmod a+x repo
$ sudo apt-get install -y python-is-python3

repo的运行过程中会尝试访问官方的git源更新自己，指定使用tuna的镜像源进行更新:

$ vim ~/.bashrc
export REPO_URL='https://mirrors.tuna.tsinghua.edu.cn/git/git-repo'

创建目录并开始同步代码(具体时间取决于网络状态), 如果需要同步别的分支的源码，可以参考这里(https://source.android.com/setup/start/build-numbers#source-code-tags-and-builds):

$ mkdir aosp11
$ cd aosp11
$ repo init -u https://mirrors.tuna.tsinghua.edu.cn/git/AOSP/platform/manifest -b android-11.0.0_r48
$ repo sync -j8

安装需要的依赖:

$ sudo apt-get install -y libncurses5
$ sudo apt-get install openjdk-8-jdk
$ sudo apt-get install git-core gnupg flex bison gperf build-essential zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z-dev ccache libgl1-mesa-dev libxml2-utils xsltproc unzip

更改aosp11源代码，在build/target/product/AndroidProducts.mk中加入关于编译时lunch的选项并开始编译:

COMMON_LUNCH_CHOICES := \
.......
     sdk_phone_x86_64-userdebug \

开始编译源码:

$ m -j120

看到下面类似的画面代表编译成功:

/images/2021_12_24_09_47_41_564x156.jpg

在当前目录下运行emulator可以直接打开模拟器:

/images/2021_12_24_09_50_39_527x644.jpg

指定分区大小及内存大小:

$ emulator -partition-size 61240 -qemu -cpu host -m 16535M

登入模拟器shell的方法:

$ adb root
$ adb shell
generic_x86_64:/ # exit

2. 编译aosp 11内核并集成至avd

上传内核参数检测脚本以获取当前内核运行docker缺失参数：

$ adb push check-config.sh /data
check-config.sh: 1 file pushed. 1.4 MB/s (11990 bytes in 0.008s)
$ adb shell
generic_x86_64:/ # chmod 777 /data/check-config.sh
generic_x86_64:/ # /data/check-config.sh

类似如下:

/images/2021_12_24_14_26_49_358x314.jpg

主机上获取内核树:

$BRANCH=common-android11-5.4-lts
$ROOTDIR=AVD-kernel-$BRANCH
$mkdir $ROOTDIR && cd $ROOTDIR
$which repo
$repo init --depth=1 -u https://android.googlesource.com/kernel/manifest -b $BRANCH && repo sync --force-sync --no-clone-bundle --no-tags -j$(nproc)
$ ls
build  common  common-modules  hikey-modules  kernel  prebuilts  prebuilts-master  repo  tools

配置内核:

$ BUILD_CONFIG=common-modules/virtual-device/build.config.goldfish.x86_64 \
FRAGMENT_CONFIG=common/arch/x86/configs/gki_defconfig \
build/config.sh

将出现以下界面:

/images/2021_12_24_14_43_28_555x562.jpg

按照前面内核检测时docker所缺失参数的情况，依次打开所需开启的内核选项.

Generic setup -> POSIX Message Queues
Generic setup -> Controller Group support -> PIDs controller
Generic setup -> Controller Group support -> Device controller
Generic setup -> Controller Group support -> CPU controller -> Group sheduling for SCHED_OTHER
Generic setup -> Controller Group support -> CPU controller -> CPU bandwidth provisioning for FAIR-GROUP_SCHED
Generic setup -> Controller Group support -> CPU controller -> Group sheduling for SCHED_RR/FIFO
Generic setup -> Controller Group support -> Perf controller
Generic setup -> Namespaces support -> User namespace
Generic setup -> Namespaces support -> PID namespace
Networking support -> Networking options -> Network packet filtering framework (Netfilter) -> Bridged IP/ARP packets fiiltering
Networking support -> Networking options -> Network packet filtering framework (Netfilter) -> IP virtual server support
Networking support -> Networking options -> Network packet filtering framework (Netfilter) -> Core Netfilter configuration ->  "addrtype" address type match support
Networking support -> Networking options -> Network packet filtering framework (Netfilter) -> Core Netfilter configuration ->  "control group" address type match support
Networking support -> Networking options -> Network packet filtering framework (Netfilter) -> Core Netfilter configuration ->  "control group" address type match support
File Systems -> Overlay filesystem support

其中Overlay filesystems suppport的配置如下:

/images/2021_12_24_15_02_04_681x191.jpg

编译内核:

$ BUILD_CONFIG=common-modules/virtual-device/build.config.goldfish.x86_64 \
build/build.sh -j$(nproc)

拷贝内核及内核模块至安卓源码树下并重新编译安卓镜像:

$ cd prebuilts/qemu-kernel/x86_64/
$ mv 5.4 5.4back
$ mkdir -p 5.4/ko
$ cp /media/sdb/aosp11kernel/AVD-kernel-common-android11-5.4-lts/out/android11-5.4/dist/bzImage kernel-qemu2
$ cp /media/sdb/aosp11kernel/AVD-kernel-common-android11-5.4-lts/out/android11-5.4/dist/*.ko ./ko
$ cd ../../../../
$ m -j80
$ emulator

重新启动后的界面中启动会出现报错，点OK忽略:

/images/2021_12_24_15_12_39_335x346.jpg

重新检测后，大部分参数会就绪，cgroupv2下的某些参数会显示红色，但不妨碍使用:

/images/2021_12_24_15_14_13_304x219.jpg

3. aosp11更改/合并docker

下载docker-20.10.8二进制版本并解压到aosp源码下:

$ wget https://download.docker.com/linux/static/stable/x86_64/docker-20.10.8.tgz
// 切换到安卓源码下
$ cd prebuilts
$ tar xzvf ~/docker-20.10.8.tgz -C .

添加docker打包到system.img中的方式如下，将docker二进制文件添加到/system/bin中以便开箱即用:

$ vim  ./build/make/target/board/generic_x86_64/device.mk
// 文件末尾添加
PRODUCT_COPY_FILES += \
    prebuilts/docker/containerd:system/bin/containerd \
    prebuilts/docker/containerd-shim:system/bin/containerd-shim \
    prebuilts/docker/containerd-shim-runc-v2:system/bin/containerd-shim-runc-v2 \
    prebuilts/docker/ctr:system/bin/ctr \
    prebuilts/docker/docker:system/bin/docker \
    prebuilts/docker/dockerd:system/bin/dockerd \
    prebuilts/docker/docker-init:system/bin/docker-init \
    prebuilts/docker/docker-proxy:system/bin/docker-proxy \
    prebuilts/docker/runc:system/bin/runc \
$ vim build/target/product/sdk_phone_x86_64.mk
// 文件末尾添加
PRODUCT_ARTIFACT_PATH_REQUIREMENT_ALLOWED_LIST := \
    system/bin/containerd \
    system/bin/containerd-shim \
    system/bin/containerd-shim-runc-v2 \
    system/bin/ctr \
    system/bin/docker \
    system/bin/dockerd \
    system/bin/docker-init \
    system/bin/docker-proxy \
    system/bin/runc \

更改sepolicy权限以创建docker运行时需要的路径:

$ vim system/sepolicy/prebuilts/api/30.0/private/file_contexts
// 在# Symlinks下添加关于/var, /run, /system/etc/docker软链接定义
# Symlinks
/bin                u:object_r:rootfs:s0
/bugreports         u:object_r:rootfs:s0
/charger            u:object_r:rootfs:s0
/d                  u:object_r:rootfs:s0
/etc                u:object_r:rootfs:s0
/sdcard             u:object_r:rootfs:s0
/var                u:object_r:rootfs:s0
/run                u:object_r:rootfs:s0
/system/etc/docker                u:object_r:system_file:s0
$ vim system/sepolicy/private/file_contexts
// 约42行处: 
 42 /sdcard             u:object_r:rootfs:s0
 43 /var             u:object_r:rootfs:s0
 44 /run             u:object_r:rootfs:s0
 45 /system/etc/docker             u:object_r:system_file:s0
 46 
 47 # SELinux policy files
$ vim system/core/rootdir/Android.mk
// 约84行处:
83     ln -sf /system/etc $(TARGET_ROOT_OUT)/etc; \
 84     ln -sf /data/var $(TARGET_ROOT_OUT)/var; \
 85     ln -sf /data/run $(TARGET_ROOT_OUT)/run; \
 86     ln -sf /data/user_de/0/com.android.shell/files/bugreports $(TARGET_ROOT_OUT)/bugreports; 
// 约143行处:
144 # Since init.environ.rc is required for init and satisfies that requirement, we hijack it to create the symlink.
145 LOCAL_POST_INSTALL_CMD += ; ln -sf /system/bin/init $(TARGET_ROOT_OUT)/init
146 LOCAL_POST_INSTALL_CMD += ; ln -sf /data/docker $(TARGET_OUT)/etc/
147 LOCAL_POST_INSTALL_CMD += ; ln -sf /data/resolv.conf $(TARGET_OUT)/etc/resolv.conf

m -j 50重新编译aosp镜像后，手动生成/data目录下所需预创建的文件后，重新生成userdataimage:

$ mkdir -p out/target/product/generic_x86_64/data/run
$ mkdir -p out/target/product/generic_x86_64/data/var
$ mkdir -p out/target/product/generic_x86_64/data/docker
$ echo "nameserver 223.5.5.5" > out/target/product/generic_x86_64/data/resolv.conf
$ make userdataimage -j50

重新启动emulator后，此时docker已经就绪了。

4. 启动redroid容器实例

手动启动dockerd进程:

$ adb root
$ adb shell
# dockerd --iptables=false --dns=223.5.5.5 --data-root=/data/var/

如果需要无Log输出（后台运行模式)，可以使用以下命令:

# dockerd --iptables=false --dns=223.5.5.5 --data-root=/data/var/ &> /data/dockerd-logfile &

在另一个adb shell进程里，运行以下命令, 可以看到docker正常运行:

# docker version

启动redroid 9版本的容器实例并检查运行情况:

# docker run -d --privileged  -p 15589:5555 --memory-swappiness=0 redroid/redroid:9.0.0-latest
# docker ps
CONTAINER ID   IMAGE                          COMMAND                  CREATED          STATUS         PORTS                                         NAMES
a900b38c0fb1   redroid/redroid:9.0.0-latest   "/init qemu=1 androi…"   12 seconds ago   Up 9 seconds   0.0.0.0:15589->5555/tcp, :::15589->5555/tcp   gallant_mcclintock
# docker exec -it a900b38c0fb1 sh
# getprop | grep boot | grep complete                                         
[dev.bootcomplete]: [1]
[sys.boot_completed]: [1]
[sys.logbootcomplete]: [1]

5. 配置网络

主机上配置tap0设备并桥接到br0(tunctl is installed via apt-get install -y uml-utilities):

$ sudo tunctl -u intel
$ sudo brctl addif br0 tap0
$ sudo ip link set dev tap0 up

启动avd的时候使用以下参数:

$ emulator -show-kernel -partition-size 61240 -net-tap tap0 -qemu -cpu host -m32768

在avd中，手动设置wifi的地址为br0同网段固态IP地址, 而后可以直接Ping通该设置的地址。

Docker启动的命令需更改为:

$ dockerd --iptables=false --dns=223.5.5.5 --data-root=/data/var/ --ip=192.168.xx.xxx & >/data/dockerd-logfile 2>&1

此时接近于成功，但是仍然无法与容器实例通过转发后的adbd端口通信并截图。

很惭愧，就做了一点微小的工作

BuildAOSP10Kernel

内核源码获取

内核编译链获取

编译

测试内核

RunWaydroidUbportsOnPixel3a

1. 准备工作/相关概念

2. 操作指南

2.1 刷入安卓9

2.2 解锁bootloader:

2.3 刷入ubports

2.4 安装waydroid

2.5 启动waydroid

2.6 体验waydroid

LinuxTips14

1. ubuntu20.04 use python2

2. ubports issue

3. lock/unlock bootloader

4. reflashing pixel 3a

5. flashing kernel module

6. Disable oh-my-zsh upgrade

7. Install chromium on mips64el

8. grub configuration(Ubuntu)

9. Install docker-ce on arm64

10. run redroid with amd

11. opengl issue

12. content of rpm

13. apt-fast

14. ubuntu device driver

15. yum groupremove

16. linux-zen

17. Build linux-zen

18. Change password in single

19. find which file belong to rpm

20. linux-zen

20. arp-ignore

21. Install vmplayer

22. Install mate(Ubuntu)

16. rpm2cpio

17. limit bandwidth

18. amdgpu centos howto(5700)

19. virgl On CentOS

20. amd driver in ubuntu

21. zen kernel in ubuntu

22. convert vdi to img

23. ssh push to github

24. Disable gdm sleeping

25. last crash log

26. check virt host

27. file belongs to apk

28. screenshot

29. build ampere kernel

30. virtio-gpu On Arm

31. Manually add pool

32. no-machine issue

33. centos 8 docker installation

34. before build aosp

35. Install golang for ubuntu20.04

36. Ubuntu driver install

37. cuttlefish render mode

38. forward to vm

39. Recompile Ubuntu Linux

40. iptables forbid only one ip

41. yarn issue

42. efi grub issue

43. disable auto-update ubuntu

44. start redroid

45. Upgrade Kernel

46. print photos

47. untar deb

48. linux-zen for ArchLinux

49. disable gdm sleep

50. redroid 12 issue

51. ArchLinux Configuration IP

52. ubuntu install zen kernel

53. ipmi reset lan

53. efi grub recover

54. Ubuntu vfio sg1

55. Build linux-zen