Documentation: teaching: Add ARM kernel development lab

This is an introduction to the ARM world. Starting with a simple
overview of SoCs, will jump into how to setup a development environment
by installing a crosscompiling.

Then will go into how to compile the linux kernel and boot it using
qemu.

Finally, we will have a look at how ARM uses Device tree files to
describe the hardware.

Keywords: SoCs, BSP, toolchain, crosscompiler, i.MX6UL.

Signed-off-by: Daniel Baluta <daniel.baluta@nxp.com>

Author: dbaluta

Documentation/teaching/index.rst

@@ -73,6 +73,7 @@ tools/labs/docs/Dockerfile for dependencies):
    labs/filesystems_part1.rst
    labs/filesystems_part2.rst
    labs/networking.rst
+   labs/arm_kernel_development.rst
    labs/memory_mapping.rst
    labs/device_model.rst
 

Documentation/teaching/labs/arm_kernel_development.rst

@@ -0,0 +1,387 @@
+=========================
+Kernel Development on ARM
+=========================
+
+Lab objectives
+==============
+
+* get a feeling of what System on a Chip (SoC) means
+* get familiar with embedded world using ARM as support architecture
+* understand what a Board Support Package means (BSP)
+* compile and boot an ARM kernel with Qemu using i.MX6UL platform as an example
+* get familiar with hardware description using Device Trees
+
+System on a Chip
+================
+
+A System on a Chip (**SoC**) is an integrated circuit (**IC**) that integrates an entire system onto it. The components
+that can be usually found on an SoC include a central processing unit (**CPU**), memory, input/output ports, storage devices
+together with more sophisticated modules like audio digital interfaces, neural processing units (**NPU**) or graphical
+processing units (**GPU**).
+
+SoCs can be used in various applications most common are:
+ - consumer electronics (TV sets, mobile phones, video game consoles)
+ - industrial computers (medical imaging, etc)
+ - automotive
+ - home appliances
+
+The leading architecture for SoCs is **ARM**. Worth mentioning here is that there are also x86-based SoCs platforms. Another thing
+we need to keep an eye on is **RISC-V** an open standard instruction set architecture.
+
+A simplified view of an **ARM** platform is shown in the image below:
+
+.. image:: ../res/schematic.png
+   :align: center
+
+We will refer as a reference platform at NXP's `i.MX6UL <imx6ul>`_ platform, but in general all SoC's contain the following building blocks:
+
+  - one or more CPU cores
+  - a system bus
+  - clock and reset module
+
+    - PLL
+    - OSC
+    - reset controller
+
+ - interrupt controller
+ - timers
+ - memory controller
+ - peripheral controllers
+
+   - `I2C <https://en.wikipedia.org/wiki/I%C2%B2C>`_
+   - `SPI <https://en.wikipedia.org/wiki/Serial_Peripheral_Interface>`_
+   - `GPIO <https://en.wikipedia.org/wiki/General-purpose_input/output>`_
+   - `Ethernet <https://en.wikipedia.org/wiki/Network_interface_controller>`_ (for network)
+   - `uSDHC <https://en.wikipedia.org/wiki/MultiMediaCard>`_ (for storage)
+   - USB
+   - `UART <https://en.wikipedia.org/wiki/Universal_asynchronous_receiver-transmitter>`_
+   - `I2S <https://en.wikipedia.org/wiki/I%C2%B2S>`_ (for sound)
+   - eLCDIF (for LCD Panel)
+
+Here is the complete block diagram for i.MX6UL platform:
+
+.. image::  https://www.nxp.com/assets/images/en/block-diagrams/IMX6UL-BD.jpg
+   :alt: IMX6UL-BD
+   :width: 60 %
+   :align: center
+
+i.MX6UL Evaluation Kit board looks like this:
+
+.. image:: https://www.compulab.com/wp-content/gallery/sbc-imx6ul/compulab_sbc-imx6ul_single-board-computer.jpg
+   :alt: imx6ul-evk
+   :width: 60 %
+   :align: center
+
+Other popular SoC boards:
+
+        * `Broadcom Raspberry Pi <https://en.wikipedia.org/wiki/Raspberry_Pi>`_
+        * `Texas Instruments Beagle board <https://en.wikipedia.org/wiki/BeagleBoard>`_
+        * `Odroid Xu4 <https://wiki.odroid.com/odroid-xu4/odroid-xu4>`_
+        * `Nvidia Jetson Nano <https://developer.nvidia.com/embedded/jetson-nano-developer-kit>`_
+
+Board Support package
+=====================
+
+A board support package (**BSP**) is the minimal set of software packages that allow to demonstrate the capabilities of a certain hardware platform. This includes:
+
+  - toolchain
+  - bootloader
+  - Linux kernel image, device tree files and drivers
+  - root filesystem
+
+Semiconductor manufacturers usually provide a **BSP** together with an evaluation board. BSP is typically bundled using `Yocto <https://www.yoctoproject.org/>`_
+
+Toolchain
+=========
+Because our development machines are mostly x86-based we need a cross compiler that can produce executable
+code for ARM platform.
+
+We can build our own cross compiler from scratch using https://crosstool-ng.github.io/ or we can install one
+
+.. code-block:: bash
+
+  $ sudo apt-get install gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf # for arm32
+  $ sudo apt-get install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu     # for arm64
+
+There are several of toolchain binaries depending on the configuration:
+
+  - With "arm-eabi-gcc" you have the Linux system C library which will make calls into the kernel IOCTLs, e.g. for allocating memory pages to the process.
+  - With "arm-eabi-none-gcc" you are running on platform which doesn't have an operating system at all - so the C library is different to cope with that.
+
+Compiling the Linux kernel on ARM
+---------------------------------
+
+Compile the kernel for 32bit ARM boards:
+
+.. code-block:: bash
+
+  # select defconfig based on your platform
+  $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make imx_v6_v7_defconfig
+  # compile the kernel
+  $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make -j8
+
+Compile the kernel for 64bit ARM boards:
+
+.. code-block:: bash
+
+  # for 64bit ARM there is a single config for all supported boards
+  $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make defconfig
+  # compile the kernel
+  $ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- make -j8
+
+Linux kernel image
+==================
+
+The kernel image binary is named ``vmlinux`` and it can be found in the root of the kernel tree. Compressed image used for booting can be found under:
+
+- ``arch/arm/boot/Image``, for arm32
+- ``arch/arm64/boot/Image``, for arm64
+
+.. code-block:: bash
+
+    $ file vmlinux
+      vmlinux: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, not stripped
+
+    $ file vmlinux
+      vmlinux: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), statically linked, not stripped
+
+Rootfs
+======
+
+The root filesystem (``rootfs``) is the filesystem mounted at the top of files hierarchy (``/``). It should contain at least
+the critical files allowing the system to boot to a shell.
+
+.. code-block:: bash
+
+   root@so2$ tree -d -L 2
+   ├── bin
+   ├── boot
+   ├── dev
+   ├── etc
+   ├── home
+   │   └── root
+   ├── lib
+   │   └── udev
+   ├── mnt
+   ├── proc
+   ├── sbin
+   │   └── init
+   ├── sys
+   ├── usr
+   │   ├── bin
+   │   ├── include
+   │   ├── lib
+   └── var
+
+As for ``x86`` we will make use of Yocto rootfs images. In order to download an ``ext4`` rootfs image for ``arm32`` one needs to run:
+
+.. code-block:: bash
+
+   $ cd tools/labs/
+   $ ARCH=arm make core-image-minimal-qemuarm.ext4
+
+Device tree
+===========
+
+Device tree (**DT**) is a tree structure used to describe the hardware devices in a system. Each node in the tree describes a device hence it is called **device node**. DT was introduced
+to provide a way to discover non-discoverable hardware (e.g a device on an I2C bus). This information was previously stored inside the source code for the Linux kernel. This meant that
+each time we needed to modify a node for a device the kernel needed to be recompiled. This no longer holds true as device tree and kernel image are separate binaries now.
+
+Device trees are stored inside device tree sources (*.dts*) and compiled into device tree blobs (*.dtb*).
+
+.. code-block:: bash
+
+   # compile dtbs
+   $ make dtbs
+
+   # location for DT sources on arm32
+   $ ls arch/arm/boot/dts/
+     imx6ul-14x14-evk.dtb imx6ull-14x14-evk.dtb bcm2835-rpi-a-plus.dts
+
+   # location for DT source on arm64
+   $ ls arch/arm64/boot/dts/<vendor>
+     imx8mm-evk.dts imx8mp-evk.dts
+
+The following image is a represantation of a simple device tree, describing board type, cpu and memory.
+
+.. image:: ../res/dts_node.png
+   :align: center
+
+Notice that a device tree node can be defined using ``label: name@address``:
+
+   - ``label``, is an identifier used to reference the node from other places
+   - ``name``, node identifier
+   - ``address``, used to differentiate nodes with the same name.
+
+A node might contain several properties arranged in the ``name = value`` format. The name is a string
+and the value can be bytes, strings, array of strings.
+
+Here is an example:
+
+.. code:: c
+
+   / {
+        node@0 {
+             empty-property;
+             string-property = "string value";
+             string-list-property = "string value 1", "string value 2";
+             int-list-property = <value1 value2>;
+
+             child-node@0 {
+                     child-empty-property;
+                     child-string-property = "string value";
+                     child-node-reference = <&child-node1>;
+             };
+
+             child-node1: child-node@1 {
+                     child-empty-property;
+                     child-string-property = "string value";
+             };
+      };
+   };
+
+Qemu
+====
+
+We will use ``qemu-system-arm`` to boot 32bit ARM platforms. Although, this can be installed from official distro repos, for example:
+
+.. code:: bash
+
+   sudo apt-get install -y qemu-system-arm
+
+We strongly recommend using latest version of ``qemu-system-arm`` build from sources:
+
+.. code:: bash
+
+   $ git clone https://gitlab.com/qemu-project/qemu.git
+   $ ./configure --target-list=arm-softmmu --disable-docs
+   $ make -j8
+   $ ./build/qemu-system-arm
+
+Exercises
+=========
+
+.. include:: ../labs/exercises-summary.hrst
+.. |LAB_NAME| replace:: arm_kernel_development
+
+.. warning::
+
+  The rules for working with the virtual machine for ``ARM`` are modified as follows
+
+  .. code-block:: shell
+
+     # modules build
+     tools/labs $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make build
+     # modules copy
+     tools/labs $ ARCH=arm make copy
+     # kernel build 
+     $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make -j8
+
+0. Intro
+--------
+
+Inspect the following locations in the Linux kernel code and identify platforms and vendors using
+ARM architecture:
+
+ - 32-bit: ``arch/arm/boot/dts``
+ - 64-bit: ``arch/arm64/boot/dts``
+
+
+Use ``qemu`` and look at the supported platforms:
+
+.. code-block:: bash
+
+   ../qemu/build/arm-softmmu/qemu-system-arm -M ?
+
+.. note:: We used our own compiled version of ``Qemu`` for ``arm32``. See `Qemu`_ section for more details.
+
+1. Boot
+-------
+
+Use ``qemu`` to boot ``i.MX6UL`` platform. In order to boot, we first need to compile the kernel.
+Review `Compiling the Linux kernel on ARM`_ section.
+
+Successful compilation will result in the following binaries:
+
+   - ``arch/arm/boot/Image``, kernel image compiled for ARM
+   - ``arch/arm/boot/dts/imx6ul-14x14-evk.dtb``, device tree blob for ``i.MX6UL`` board
+
+Review `Rootfs`_ section and download ``core-image-minimal-qemuarm.ext4`` rootfs.
+Run ``qemu`` using then following command:
+
+.. code-block:: bash
+
+   ../qemu/build/arm-softmmu/qemu-system-arm -M mcimx6ul-evk -cpu cortex-a7 -m 512M \
+     -kernel arch/arm/boot/zImage -nographic  -dtb arch/arm/boot/dts/imx6ul-14x14-evk.dtb \
+     -append "root=/dev/mmcblk0 rw console=ttymxc0 loglevel=8 earlycon printk" -sd tools/labs/core-image-minimal-qemuarm.ext4
+
+.. note:: LCDIF and ASRC devices are not well supported with ``Qemu``. Remove them from compilation.
+
+.. code-block:: bash
+
+   $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make menuconfig
+   # set FSL_ASRC=n and DRM_MXSFB=n
+   $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make -j8
+
+Once the kernel is booted check kernel version and cpu info:
+
+.. code-block:: bash
+
+   $ cat /proc/cpuinfo
+   $ cat /proc/version
+
+2. CPU information
+------------------
+
+Inspect the CPU configuration for ``NXP i.MX6UL`` board. Start with ``arch/arm/boot/dts/imx6ul-14x14-evk.dts``.
+
+  - find ``cpu@0`` device tree node and look for ``operating-points`` property.
+  - read the maximum and minimum operating frequency the processor can run
+
+   .. code:: bash
+
+      $ cat /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq
+      $ cat /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_max_freq
+
+3. I/O memory
+-------------
+Inspect I/O space configuration for ``NXP i.MX6UL`` board. Start with ``arch/arm/boot/dts/imx6ul-14x14-evk.dts`` and identify each device mentioned below.
+
+.. code:: bash
+
+   $ cat /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq
+     00900000-0091ffff : 900000.sram sram@900000
+     0209c000-0209ffff : 209c000.gpio gpio@209c000
+     021a0000-021a3fff : 21a0000.i2c i2c@21a0000
+     80000000-9fffffff : System RAM
+
+Identify device tree nodes corresponding to:
+
+  -  ``System RAM``, look for ``memory@80000000`` node in ``arch/arm/boot/dts/imx6ul-14x14-evk.dtsi``. What's the size of the System RAM?
+  -  ``GPIO1``, look for ``gpio@209c000`` node in ``arch/arm/boot/dts/imx6ul.dtsi``. What's the size of the I/O space for this device?
+  -  ``I2C1``,  look for ``i2c@21a0000`` node in ``arch/arm/boot/dts/imx6ul.dtsi``. What's the size of the I/O spaces for this device?
+
+4. Hello World
+--------------
+
+Implement a simple kernel module that prints a message at load/unload time. Compile it and load it on ``i.MX6UL`` emulated platform.
+
+.. code-block:: shell
+
+   # modules build
+   tools/labs $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make build
+   # modules copy
+   tools/labs $ ARCH=arm make copy
+   # kernel build
+   $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make -j8
+
+5. Simple device
+----------------
+
+Implement a driver for a simple platform device. Find ``TODO 1`` and notice how ``simple_driver`` is declared and register as a platform driver.
+Follow ``TODO 2`` and add the ``so2,simple-device-v1`` and ``so2,simple-device-v2`` compatible strings in the simple_device_ids array.
+
+Create two device tree nodes in ``arch/arm/boot/dts/imx6ul.dtsi`` under ``soc`` node with compatible strings ``so2,simple-device-v1`` and
+``so2,simple-device-v2`` respectively. Then notice the behavior when loading ``simple_driver`` module.
+
+.. _imx6ul: https://www.nxp.com/products/processors-and-microcontrollers/arm-processors/i-mx-applications-processors/i-mx-6-processors/i-mx-6ultralite-processor-low-power-secure-arm-cortex-a7-core:i.MX6UL