arch/arm64/kernel/head.S
arch/arm64/kernel/hyp-stub.S
arch/arm64/include/asm/assembler.h
init/main.c
virt/kvm/arm/arm.c
virt/kvm/kvm_main.c
virt/kvm/arm/mmu.c
arch/arm64/kvm/hyp-init.S
arch/arm64/include/asm/virt.h
arch/arm64/include/asm/kvm_host.h
arch/arm64/kvm/hyp.S
arch/arm64/include/asm/kvm_mmu.h
arch/arm64/kernel/vmlinux.lds.S
arch/arm64/hypsec_proved/el1.c
arch/arm64/include/asm/hypsec_host.h
arch/arm64/kvm/hyp/hyp-entry.S
arch/arm64/hypsec_proved/el2.c
arch/arm64/hypsec_proved/hypsec.h
arch/arm64/include/asm/hypsec_mmu.h
arch/arm64/hypsec_proved/s2-setup.c
arm64/hypsec_proved/MemManager.c
When the machine boots up, the kernel is loaded into the memory by the BIOS or the bootloader. According to the Linux Arm boot protocol, the Program Counter (PC) is set to where the symbol _head
is loaded and the kernel takes control of the hardware from there. The platform we use boots from EL2 so the kernel initially runs at EL2. It configures necessary hardware features and then deprivileges itself into EL1. The first few instructions of the kernel are straightforward. Remember we are currently at EL2.
arch/arm64/kernel/head.S:
/* arch/arm64/kernel/head.S */
_head:
/* ... */
**b stext**
/* ... */
_head
does nothing but jumps to stext
in the same file:
ENTRY(stext)
bl preserve_boot_args
**bl el2_setup** // Drop to EL1, w0=cpu_boot_mode
/* ... */
ENDPROC(stext)
preserve_boot_args
loads the boot arguments passed from the bootloader, which we don't care about for now. Then it jumps to el2_setup
also in the same file.
There are three vector tables used as part of initializing EL2. A vector table basically defines the handlers to execute when an exception occurs. The first vector table is setup when the kernel initial runs at EL2 and its purpose is to basically define an EL2 handler so that it is possible to go back to EL2 later after already running in EL1. Otherwise, there would be no way for the hardware to know what to do when trapping to EL2. This initial vector table’s only purpose is to allow the kernel to return to EL2. The second vector table is setup when the kernel returns to EL2, after performing various initialization steps for EL1. The second table installs a handler to actually perform EL2 initialization required for supporting virtualization. There is only one EL2 vector table active at any point in time, so the second table replaces the first table. After EL2 initialization is done, a third vector table is setup, which replaces the second table. This third table installs the actual handlers used during BlackBox execution for handling exceptions that trap to EL2.
el2_stub_vector
The kernel is now at el2_setup
as part of initializing EL2. This function gets the kernel ready for running at EL1 by giving EL1 control of most hardware functions. We omit most of the tedious configuration details here but focus on critical instructions:
ENTRY(el2_setup)
/* ... */
#ifdef CONFIG_ARM64_VHE
/* ... */
#else
**mov x2, xzr**
#endif
/* ... */
**cbz x2, install_el2_stub**
/* ... */
install_el2_stub:
/* ... */
ENDPROC(el2_setup)
In our system, VHE (Virtualization Host Extension, when the kernel runs at EL2) is not enabled so x2
is set to zero(xzr
means “Zero Register”. Any read from this register gets 0). Later on, the kernel jumps to another label install_el2_stub
. The instruction cbz
means “Compare and Branch on Zero”.
install_el2_stub:
/* ... */
7: adr_l x0, __hyp_stub_vectors
msr vbar_el2, x0
msr elr_el2, lr
mov w0, #BOOT_CPU_MODE_EL2 // This CPU booted in EL2
eret
install_el2_stub
contains some tedious configuration as well. We will focus on the instruction we care about most.
The first and second instructions above load the address of __hyp_stub_vectors
into vbar_el2
register. vbar_el2
means "Vector Base Address Register" and the msr
instruction, meaning “Move General Purpose Register to System Register”, set vbar_el2
to the value of x0
. When an exception is taken to EL2, the CPU will find a corresponding entry in the exception vector table and jump to that entry.
<aside>
ℹ️ At this point, the first exception vector table, __hyp_stub_vectors
, at EL2 is installed.
</aside>
The third instruction loads lr
into elr_el2
. elr_el2
means “Error Link Register”. Usually, when a function/procedure call returns, the ret
instruction is used and the CPU sets the PC to the value of the Linker Register, i.e. lr
. Similarly, if we use the eret
instruction, instead of setting the PC to lr
, the CPU sets the PC to elr
. Apart from changing the value of PC, eret
also brings the current exception level down by one.
So the third and last instructions make the CPU return from the current function and deprivilege to EL1.