Kobo Clara HD Custom Linux Distro/RootFS 2021-07-22 ======================================== These are just some notes I made when creating my own mini-distro after wanting something more custom than just using buildroot or making the official firmware more slim. For people other than me, I suggest looking through (C)LFS or running postmarketOS instead once this reader's pull request[1] gets integrated into upstream. Two things that'll greatly help with this is having serial terminal access with the four uart pins near the top right in the back of the reader, near the uSD card slot (I don't connect the 5V pin as my reader doesn't really turn on anything other than the power LED). I suggest maybe soldering female pin headers to there to make your life easier (you can later cut out a hole in the back cover or desolder the headers once you're done). Other than that, I suggest installing QEMU with ARM userspace to test programs that you have built or running them on a separate ARM device like a Raspberry Pi. Prelude ------- Ever since I learnt that the official firmware for the Clara was just using a modified Linux kernel with busybox as coreutils and many other libraries, I just knew that I had to minimize it. I also saw that it was using glibc for it's libc, which I really dislike as statically linking C programs against it was a pain in my experience, compared to something like musl and uclibc. It's also much larger than them and I don't use any of glibc extensions so it seemed like a waste of space to me. Initially when I replaced Nickel with Plato, I was able to shave about 100 MiB after I removed /usr/local (which contains Nickel, Qt and a few other things), from 189 MiB to 74 MiB, but I still wanted to make it smaller. Using buildroot, I was able to get it under 2 MiB (!!) which was a little less than half the size of an uncompressed armhf Alpine Linux minirootfs (4.9M for 3.14). With Busybox, it was pretty much working out of the box, with serial terminal access! But waiting around 15 minutes for the toolchain to build each time I wanted to change something in the rootfs took way too long, although it could've been minimized if I used ccache with a fairly large cache size. I still found that it compiled and installed a lot of things I wouldn't be using (particularly in /usr) even after disabling almost all of the third-party packages. I've uploaded the config file and the resulting rootfs for buildroot 2021.05. The root password by default is changeme. EDIT 2022-10-21: gone, build it yourself Of course the rootfs I got from buildroot nor me making the official firmware smaller is the point of this article, and the actual point is making one yourself! (or rather what I did to make my own) Cross-toolchain --------------- For now as of July 22, 2021, I'm using my distro (Void Linux)'s packaged cross toolchain for armhf musl, but eventually I would be using my own. I'm not compiling off of the device itself as it would be somewhat slow for bigger programs, which is currently primarily the Linux kernel, U-Boot, and the toolchain itself, considering that the ereader's CPU (Freescale i.MX 6SLL) is a single core running up to 1 GHz. Including the development tools and headers would also take up more space on the device itself, and since the terminal can currently only be accessed through it's serial/uart pins, I don't think it's ideal. TODO: include steps to create own toolchain (probably based off of gcc 4.7.3 as that doesn't require c++) Building the rootfs ------------------- Assuming you made a new filesystem on your rootfs's partition, it'll likely be empty with no directories you'd expect to find on a regular distro. So you'll just have to make them. cd /path/to/rootfs mkdir bin dev etc proc sbin Your binaries would usually go in /bin, the uSD card, ttymxc0, and other devices would go in /dev, felker init's default program/script to execute is usually in /etc/rc, /proc is optional but I have it mounted to see what is currently mounted through /proc/mounts (or mount(1) without any arguments) as well as to see my disk usage through df(1). /sbin is there to place the init in as /sbin/init is the default init path the kernel looks at. toybox ------ Now on to the main part of the distro, the userspace. I intend to keep it fairly minimal so I've chosen to use toybox along with a slightly modified version of felker (musl dev)'s init[2], as well as dash[3] as the main shell since toybox doesn't include one as of 0.8.5 (though it'll probably be there by 1.0). I'll also be statically linking all the programs that'll be used so I wouldn't have to worry about shared libraries not being included/copied over, and also including LTO for slightly faster binaries. Originally, I tried going with sinit, sbase, and ubase but I was having trouble getting serial terminal access with getty to /dev/ttymxc0 (the default serial tty, at least with the vendor kernel). I didn't have this problem with busybox's and toybox's getty however. My config for toybox was also about 81K smaller than my trimmed sbase-box and ubase-box (352K compared to 267K+166K) where I removed programs that I won't use from ${BIN} in their respective Makefiles. EDIT 2022-10-21: also gone First I suggest exporting some environment variables to set the toolchain used as well as enabling static linking and LTO. export CROSS_COMPILE="arm-linux-musleabihf-" # change to your cross-tc export CC="${CROSS_COMPILE}gcc" export LDFLAGS="--static" export CFLAGS="-flto -static" export ARCH=arm # for compiling the linux kernel To compile toybox, get the source from https://landley.net/toybox/downloads/ (or clone the upstream repo). Then run make menuconfig (optionally with make defconfig before it) and change it as you see fit. Personally, I disabled most of the programs I wouldn't use and kept only the ones that'll help with fixing a problem. Finally, make sure to run make. make defconfig make menuconfig make To move it to your rootfs and set it's symlinks, you could probably run make install after setting PREFIX to your rootfs's /bin directory, but I did it manually. # automatic (didn't test, check README) make PREFIX=/path/to/rootfs/bin/ install # (semi?) manual cp toybox /path/to/rootfs/bin # add symlinks if doing manual and you want them cd /path/to/rootfs/bin for prog in $(qemu-arm ./toybox); do ln -s toybox "$prog"; done dash ---- Also as of toybox 0.8.5, a shell still isn't included (probably would be included by 1.0 according to scripts/install.sh as well as a few other programs like gzip), so a separate shell would need to be built. Any can be used but dash would be shown as an example as I was able to get a static binary without too much trouble. First obtain the source[3] and cd into its untarred directory. Assuming your CC and CFLAGS are set, you can run these steps: autoreconf -fiv ./configure --host=$CROSS_COMPILE --with-libedit make ${CROSS_COMPILE}strip src/dash As this is going to be used as the main shell, I've decided to just copy it to /bin/sh in the rootfs directory, though copying it there but as /bin/dash and /bin/sh being symlinked to dash is also an option. cp src/dash /path/to/rootfs/bin/sh # or cp src/dash /path/to/rootfs/bin cd /path/to/rootfs/bin ln -s dash sh felker's init ------------- The init is just a single file that you can get from felker's site[2] or the gist on github[7]. I haven't had a good experience with the default startup program (/etc/rc) as a shell script with execve() run on it so I'd change it to execvp() and remove the third (specifies environment). To compile and install the init, all you need to do is run: $CC $CFLAGS -o init init.c cp init /path/to/rootfs/sbin Instead of /etc/rc being a shell script, you can also make a C program that does whatever you think is needed for a proper startup. I'll still use a shell script though which is linked here. EDIT 2022-10-21: you get the idea, it's gone. /etc/passwd ----------- Copying the rootfs's contents to your device's/uSD card's root partition and then turning the device on should now work with a login prompt shown in the serial terminal. However, you probably wouldn't be able to login to any user. So you'll have to create a file at /path/to/rootfs/etc/passwd. For an empty password to root, you can use this, though I suggest setting a password as soon as you login: # in rootfs's /etc/passwd root::0:0:root:/root:/bin/sh With the passwd file created/updated, you should now be able to login to root after the rootfs is copied to your uSD card. Your rootfs so far should now be around 550-560K, which is much much smaller than the original firmware's, though it'll likely be much larger to maybe a few megabytes once a proper reader software is added. Custom Linux Kernel ------------------- WARNING: I haven't actually gotten the kernel to load in u-boot yet. It just hangs in the "Starting kernel ..." step and the init doesn't get loaded, so I'm assuming the kernel itself isn't either. If anyone out there has gotten a custom kernel working in the Kobo Clara HD, please send me an email or message on xmpp. UPDATE Jul 28, 2021: Gave up on it as I just couldn't get any kernels I built (both vendor and akemnade's mainline) to boot. But neither did postmarketOS boot beyond the initial initramfs messages without the log file being created. So I'll revisit this for later. EDIT 2022-10-21: I have gotten this working, but have been unable to get Plato build for musl, so I will have to either continue fighting with the crab or create my own with fbink, as that still works. Separate article on this later. My next big step is compiling my own kernel for the Clara HD. With the default configuration built for the vendor kernel, it appears to be about 3M, so my goal is to build a kernel that is smaller than that while retaining only the functionality that I need. I'm also not going to include networking support as that is unneeded for my purposes, but I suggest just keeping it if you're unsure. The wifi driver for the Kobo Clara HD is available as an out-of-tree driver[8]. You should first obtain the kernel source, with two main options, the vendor kernel[9] and the mainline kernel (with akemnade's patches)[10]. For the latter, you need to clone the repo and switch to the latest kobo/drm-merged branch (kobo/merged-5.13 as of July 25, 2021). After you've got them and assuming the CROSS_COMPILE and ARCH environment variables are set, you'd want to configure the kernel. I had a hard time compiling the vendor kernel with many things disabled, so I've kept my config somewhat similar to the default config. The config I used is available here (EDIT: dead). make menuconfig make zImage Assuming it compiles properly and arch/arm/boot/zImage exists, all that's needed to is to write it to your uSD card at the 1M offset. dd if=/path/to/kernel/zImage of=/path/to/uSDdev bs=512 seek=2048 Custom U-Boot ------------- I have not done this yet, nor really plan to, but if you do manage to compile the Kobo's vendored u-boot source, then all you'd have to do to install it is: dd if=u-boot-file of=/dev/mmcblk0 bs=128k count=1 seek=6 If I remember correctly, this command was included in an older firmware's startup script/rcS for updating udev, and it should still work. Links ----- [1]: https://gitlab.com/postmarketOS/pmaports/-/merge_requests/2334 [2]: https://ewontfix.com/14 [3]: https://git.kernel.org/pub/scm/utils/dash/dash.git [4]: https://github.com/akemnade/linux/tree/kobo/merged-5.13 [5]: https://misc.andi.de1.cc/kobo/uboot-env.txt [6]: https://misc.andi.de1.cc/kobo/ [7]: https://gist.github.com/rofl0r/6168719/raw/183525e0f0007169a49392b21ceee5b507e3aee8/init.c [8]: https://github.com/jwrdegoede/rtl8189ES_linux/tree/rtl8189fs [9]: https://github.com/kobolabs/Kobo-Reader/blob/master/hw/imx6sll-clara/kernel.tar.bz2 [10]: https://github.com/akemnade/linux/tree/kobo/drm-merged-5.12