Linux booting 串口

在kernel booting的早期,肯定是希望尽快把一些信息输出的串口,能够尽早知道kernel boot 到什么阶段。 所以一般在u-boot中,在FVP平台上有如下bootargs

bootargs=console=ttyAMA0 earlycon=pl011,0x1c090000 root=/dev/vda1 rw ip=dhcp debug user_debug=31 loglevel=9

这里 earlycon=pl011,0x1c090000 就是让Linux boot的早期就可以知道串口的相关信息,尽早输出boot的关键信息。

在FVP平台上,可以看到使用的是PL011串口,它的基地值可以参考FVP的用户文档,可以看到FVP里面有四个串口,信息如下:

FVP 串口文件

因为这里使用的是PL011,对应的驱动代码在 drivers/tty/serial/amba-pl011.c:

static int __init pl011_early_console_setup(struct earlycon_device *device,
                                            const char *opt)
{
        if (!device->port.membase)
                return -ENODEV;

        device->con->write = pl011_early_write;
        device->con->read = pl011_early_read;

        return 0;
}

OF_EARLYCON_DECLARE(pl011, "arm,pl011", pl011_early_console_setup);

OF_EARLYCON_DECLARE(pl011, "arm,sbsa-uart", pl011_early_console_setup);

earlycon的IO地址是如何映射的?

在Linux boot 的早期,即启动到start_kernel,这个时候就不能直接使用物理地址访问设备和内存,因为MMU已经enable了。

而Linux只映射了自己image和device tree所在的位置,而这里要尽可能早的能个输出关键boot信息,所以这个串口的IO地址也要在早期给映射好。

在mem_init之前,系统的内存管理并没有准备好,所以并不能直接使用ioremap 来给串口进行映射。所以这里就会使用到fixed map来映射这个早期的map。

也就是 FIX_EARLYCON_MEM_BASE ,这也是在arm64的系统里面,必须要使能: CONFIG_FIX_EARLYCON_MEM 的原因:

static void __iomem * __init earlycon_map(resource_size_t paddr, size_t size)
{
        void __iomem *base;
#ifdef CONFIG_FIX_EARLYCON_MEM
        set_fixmap_io(FIX_EARLYCON_MEM_BASE, paddr & PAGE_MASK);
        base = (void __iomem *)__fix_to_virt(FIX_EARLYCON_MEM_BASE);
        base += paddr & ~PAGE_MASK;
#else
        base = ioremap(paddr, size);
#endif
        if (!base)
                pr_err("%s: Couldn't map %pa\n", __func__, &paddr);

        return base;
}

如何不使能这个 CONFIG_FIX_EARLYCON_MEM 的话,会出现如下错误:

[    0.000000] ------------[ cut here ]------------
[    0.000000] WARNING: CPU: 0 PID: 0 at mm/ioremap.c:23 generic_ioremap_prot+0xf0/0x12c
[    0.000000] Modules linked in:
[    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 6.10.0-rc1-00013-g2bfcfd584ff5-dirty #4
[    0.000000] Hardware name: FVP Base RevC (DT)
[    0.000000] pstate: 800003c9 (Nzcv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[    0.000000] pc : generic_ioremap_prot+0xf0/0x12c
[    0.000000] lr : generic_ioremap_prot+0x28/0x12c
[    0.000000] sp : ffff800082623b40
[    0.000000] x29: ffff800082623b40 x28: 0000000000000000 x27: ffff8000829ab000
[    0.000000] x26: 0000000000000000 x25: ffff800081e11398 x24: 0000000000000000
[    0.000000] x23: ffff800081e13608 x22: ffff800080033710 x21: 0068000000000713
[    0.000000] x20: 000000001c090000 x19: 0000000000000040 x18: fffffffffffe0198
[    0.000000] x17: 00000000242dc565 x16: 0000000000000000 x15: 0000000000000028
[    0.000000] x14: ffffffffffffffff x13: 0000000000000007 x12: ffff80008168a740
[    0.000000] x11: 0101010101010101 x10: ffff800101d38846 x9 : 0fffffffffffffff
[    0.000000] x8 : 0000000000000010 x7 : 0000000000000018 x6 : ffff800082b20500
[    0.000000] x5 : ffff800082b20500 x4 : 0000000000000000 x3 : 0000000000000000
[    0.000000] x2 : 0068000000000713 x1 : 0000000000000040 x0 : 0000000000000000
[    0.000000] Call trace:
[    0.000000]  generic_ioremap_prot+0xf0/0x12c
[    0.000000]  ioremap_prot+0x50/0x78
[    0.000000]  earlycon_map+0x4c/0x94
[    0.000000]  setup_earlycon+0x254/0x314
[    0.000000]  param_setup_earlycon+0x30/0x50
[    0.000000]  do_early_param+0xb8/0xf8
[    0.000000]  parse_args+0x144/0x3ac
[    0.000000]  parse_early_param+0x60/0x78
[    0.000000]  setup_arch+0x14c/0x60c
[    0.000000]  start_kernel+0x70/0x708
[    0.000000]  __primary_switched+0x80/0x88
[    0.000000] ---[ end trace 0000000000000000 ]---
[    0.000000] earlycon: earlycon_map: Couldn't map 0x000000001c090000
[    0.000000] earlycon: pl11 at MMIO 0x000000001c090000 (options '')
[    0.000000] Malformed early option 'earlycon'

在运行完函数earlycon_map 之后,即:

#0 register_earlycon(buf = (char*) 0x0, match = <Value optimised away by compiler>) at earlycon.c:151
#1 setup_earlycon(buf = 0xFFFF800081C78847 "0x1c090000") at earlycon.c:221
#2 param_setup_earlycon(buf = <Value not available : Undefined value in stack frame for register X1>) at earlycon.c:245
#3 do_early_param(param = 0xFFFF800081C78838 "earlycon", val = 0xFFFF800081C78841 "pl011,0x1c090000", unused = <Value not available : Undefined value in stack frame for register X2>, arg = <Value not available : Undefined value in stack frame for register X3>) at main.c:753
#4 parse_one(param = <Value optimised away by compiler>, val = <Value optimised away by compiler>, doing = <Value optimised away by compiler>, params = <Value optimised away by compiler>, num_params = <Value optimised away by compiler>, min_level = <Value optimised away by compiler>, max_level = <Value optimised away by compiler>, arg = <Value optimised away by compiler>, handle_unknown = <Value optimised away by compiler>) at params.c:153
#5 parse_args(doing = 0xFFFF8000819075B0 "early options", args = <Value not available : Undefined value in stack frame for register X1>, params = (const struct kernel_param*) 0x0, num = 0, min_level = <Value not available : Undefined value in stack frame for register X4>, max_level = <Value not available : Undefined value in stack frame for register X5>, arg = <Value not available : Undefined value in stack frame for register X6>, unknown = init/main.c#int do_early_param( char* param, char* val, const char* unused, void* arg ) = 0xffff800081b700b8) at params.c:188
#6 parse_early_param() at main.c:779
#7 local_daif_restore(flags = <Value optimised away by compiler>) at setup.c:298
#8 cpu_set_reserved_ttbr0_nosync() at daifflags.h:117
#9 get_boot_config_from_initrd(_size = <Value optimised away by compiler>) at main.c:906
#10 setup_command_line(command_line = <Value optimised away by compiler>) at main.c:643
#11 __primary_switched() at head.S:244

就会发现mmu页表里面就多了串口的映射,即0x000000001C090000 --> 0xFFFFFFFFFF5FD000

>mmu memory-map
Virtual Range                              | Physical Range                           | Type         | AP | C     | S     | X
---------------------------------------------------------------------------------------------------------------------------------
EL2N:0x0000000000000000-0x000000008020FFFF | <unmapped>                               |              |    |       |       |
EL2N:0x0000000080210000-0x0000000081E6FFFF | NP:0x0000000080210000-0x0000000081E6FFFF | Normal       | RO | True  | True  | True
EL2N:0x0000000081E70000-0x0000000082C9FFFF | NP:0x0000000081E70000-0x0000000082C9FFFF | Normal       | RW | True  | True  | False
EL2N:0x0000000082CA0000-0x00000000FD6BDFFF | <unmapped>                               |              |    |       |       |
EL2N:0x00000000FD6BE000-0x00000000FD8BDFFF | NP:0x00000000FD6BE000-0x00000000FD8BDFFF | Normal       | RW | True  | True  | False
EL2N:0x00000000FD8BE000-0x0000FFFFFFFFFFFF | <unmapped>                               |              |    |       |       |
EL2N:0xFFFF000000000000-0xFFFF80008000FFFF | <unmapped>                               |              |    |       |       |
EL2N:0xFFFF800080010000-0xFFFF80008105FFFF | NP:0x0000000080210000-0x000000008125FFFF | Normal       | RO | True  | True  | True
EL2N:0xFFFF800081060000-0xFFFF800081B6FFFF | NP:0x0000000081260000-0x0000000081D6FFFF | Normal       | RW | True  | True  | False
EL2N:0xFFFF800081B70000-0xFFFF800081C6FFFF | NP:0x0000000081D70000-0x0000000081E6FFFF | Normal       | RO | True  | True  | True
EL2N:0xFFFF800081C70000-0xFFFF800082A9FFFF | NP:0x0000000081E70000-0x0000000082C9FFFF | Normal       | RW | True  | True  | False
EL2N:0xFFFF800082AA0000-0xFFFFFFFFFF5FCFFF | <unmapped>                               |              |    |       |       |
EL2N:0xFFFFFFFFFF5FD000-0xFFFFFFFFFF5FDFFF | NP:0x000000001C090000-0x000000001C090FFF | Device-nGnRE | RW | False | False | False
EL2N:0xFFFFFFFFFF5FE000-0xFFFFFFFFFF600FFF | NP:0x00000000FD6BE000-0x00000000FD6C0FFF | Normal       | RO | True  | True  | False
EL2N:0xFFFFFFFFFF601000-0xFFFFFFFFFFFFFFFF | <unmapped>                               |              |    |       |       |

有了这些IO的映射之后,就可以调用pl011_early_console_setup 对PL011串口进行初始化,然后再把这个串口设备通过函数register_console 注册到输出设备上,就可以开始输出信息了。

printk调用PL011驱动函数的调用stack如下:

#0 pl011_early_write(con = (struct console*) 0xFFFF8000828B4878, s = 0xFFFF8000829F3678 "[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd0f0]
", n = 71) at amba-pl011.c:2541
#1 console_emit_next_record(con = <Value optimised away by compiler>, handover = <Value optimised away by compiler>, cookie = <Value optimised away by compiler>) at printk.c:2915
#2 __srcu_read_unlock_nmisafe(ssp = <Value optimised away by compiler>, idx = <Value optimised away by compiler>) at srcu.h:301
#3 console_unlock() at printk.c:3049
#4 vprintk_emit(facility = 0, level = -1, dev_info = (const struct dev_printk_info*) 0x0, fmt = 0xFFFF800081917728 "6printk: %s%sconsole [%s%d] enabled
", args = {__stack = (void*) 0x8, __gr_top = (void*) 0x2, __vr_top = (void*) 0x0, __gr_offs = 0, __vr_offs = 0}) at printk.c:2348
#5 vprintk_default(fmt = <Value not available : Undefined value in stack frame for register X0>, args ERROR(CMD367-TAB183):
! Error computing the variable "args"
! Undefined value in stack frame for register X5
) at printk.c:2364
#6 vprintk(fmt = 0xFFFF800081917728 "6printk: %s%sconsole [%s%d] enabled
", args = {__stack = (void*) 0xFFFF800082533B90, __gr_top = (void*) 0xFFFF800082533B90, __vr_top = (void*) 0xFFFF800082533B50, __gr_offs = -56, __vr_offs = 0}) at printk_safe.c:46
#7 _printk(fmt = <Value not available : Undefined value in stack frame for register X0>) at printk.c:2374
#8 register_console(newcon = (struct console*) 0xFFFF8000828B4878) at printk.c:3570
#9 register_earlycon(buf = (char*) 0x0, match = <Value optimised away by compiler>) at earlycon.c:162
#10 setup_earlycon(buf = 0xFFFF800081C78847 "0x1c090000") at earlycon.c:221
#11 param_setup_earlycon(buf = <Value not available : Undefined value in stack frame for register X1>) at earlycon.c:245
#12 do_early_param(param = 0xFFFF800081C78838 "earlycon", val = 0xFFFF800081C78841 "pl011,0x1c090000", unused = <Value not available : Undefined value in stack frame for register X2>, arg = <Value not available : Undefined value in stack frame for register X3>) at main.c:753
#13 parse_one(param = <Value optimised away by compiler>, val = <Value optimised away by compiler>, doing = <Value optimised away by compiler>, params = <Value optimised away by compiler>, num_params = <Value optimised away by compiler>, min_level = <Value optimised away by compiler>, max_level = <Value optimised away by compiler>, arg = <Value optimised away by compiler>, handle_unknown = <Value optimised away by compiler>) at params.c:153
#14 parse_args(doing = 0xFFFF8000819075B0 "early options", args = <Value not available : Undefined value in stack frame for register X1>, params = (const struct kernel_param*) 0x0, num = 0, min_level = <Value not available : Undefined value in stack frame for register X4>, max_level = <Value not available : Undefined value in stack frame for register X5>, arg = <Value not available : Undefined value in stack frame for register X6>, unknown = init/main.c#int do_early_param( char* param, char* val, const char* unused, void* arg ) = 0xffff800081b700b8) at params.c:188
#15 parse_early_param() at main.c:779
#16 local_daif_restore(flags = <Value optimised away by compiler>) at setup.c:298
#17 cpu_set_reserved_ttbr0_nosync() at daifflags.h:117
#18 get_boot_config_from_initrd(_size = <Value optimised away by compiler>) at main.c:906
#19 setup_command_line(command_line = <Value optimised away by compiler>) at main.c:643
#20 __primary_switched() at head.S:244

在串口初始化之前,也会调用printk,但是这些信息会被buffer住,等到串口设备注册好之后,才会输出到串口。 所以看到如下信息:

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd0f0]
[    0.000000] Linux version 6.9.0-dirty (qixxu01@a015921) (aarch64-none-linux-gnu-gcc (Arm GNU Toolchain 13.2.rel1 (Build arm-13.7)) 13.2.1 20231009, GNU ld (Arm GNU Toolchain 13.2.rel1 (Build arm-13.7)) 2.41.0.20231009) #37 SMP PREEMPT Fri May 31 16:16:32 CST 2024
[    0.000000] KASLR disabled due to lack of seed
[    0.000000] Machine model: FVP Base RevC
[    0.000000] earlycon: pl11 at MMIO 0x000000001c090000 (options '')
[    0.000000] printk: legacy bootconsole [pl11] enabled

earlycon是如何切换到正常串口?

在前面的bootargs里面,这里有给出最后console的device console=ttyAMA0

bootargs=console=ttyAMA0 earlycon=pl011,0x1c090000 root=/dev/vda1 rw ip=dhcp debug user_debug=31 loglevel=9

这里指定了console对应的device,这个是在函数console_setup 里面进行解析,然后调用__add_preferred_console 来设置正常串口的信息.

static int __init console_setup(char *str)
{
        //...
        __add_preferred_console(buf, idx, options, brl_options, true);
        return 1;
}

__setup("console=", console_setup);

参考调用栈:

#0 console_setup(str = 0xFFFF800081D4DF28 "ᅢ깨ヘᅡチ") at printk.c:2488
#1 obsolete_checksetup(line = <Value optimised away by compiler>) at main.c:215
#2 unknown_bootoption(param = 0xFFFF00087F801140 "console=ttyAMA0", val = 0xFFFF00087F801148 "ttyAMA0", unused = <Value not available : Undefined value in stack frame for register X2>, arg = <Value not available : Undefined value in stack frame for register X3>) at main.c:561
#3 parse_one(param = <Value optimised away by compiler>, val = <Value optimised away by compiler>, doing = <Value optimised away by compiler>, params = <Value optimised away by compiler>, num_params = <Value optimised away by compiler>, min_level = <Value optimised away by compiler>, max_level = <Value optimised away by compiler>, arg = <Value optimised away by compiler>, handle_unknown = <Value optimised away by compiler>) at params.c:153
#4 parse_args(doing = 0xFFFF800081907650 "Booting kernel", args = <Value not available : Undefined value in stack frame for register X1>, params = (const struct kernel_param*) 0xFFFF800081B4E1B8, num = 477, min_level = <Value not available : Undefined value in stack frame for register X4>, max_level = <Value not available : Undefined value in stack frame for register X5>, arg = <Value not available : Undefined value in stack frame for register X6>, unknown = init/main.c#int unknown_bootoption( char* param, char* val, const char* unused, void* arg ) = 0xffff800081b703f8) at params.c:188
#5 start_kernel() at main.c:917
#6 __primary_switched() at head.S:244

而在device tree, 中,有:

v2m_serial0: serial@90000 {
        compatible = "arm,pl011", "arm,primecell";
        reg = <0x090000 0x1000>;
        interrupts = <5>;
        clocks = <&v2m_clk24mhz>, <&v2m_clk24mhz>;
        clock-names = "uartclk", "apb_pclk";
};

当Linux解析device tree 的时候,会触发PL011的driver:

static struct console amba_console = {
        .name           = "ttyAMA",
        .write          = pl011_console_write,
        .device         = uart_console_device,
        .setup          = pl011_console_setup,
        .match          = pl011_console_match,
        .flags          = CON_PRINTBUFFER | CON_ANYTIME,
        .index          = -1,
        .data           = &amba_reg,
};

当解析到上面的device tree 会调用pl011_probe ,在这个函数中,又会调用register_console 来注册新的console来替换boot阶段的console, 调用栈:

#8 register_console(newcon = (struct console*) 0xFFFF8000828B4B10) at printk.c:3572
#9 uart_configure_port(drv = <Value optimised away by compiler>, state = <Value optimised away by compiler>, port = <Value optimised away by compiler>) at serial_core.c:2668
#10 serial_core_add_one_port(drv = <Value optimised away by compiler>, uport = <Value optimised away by compiler>) at serial_core.c:3229
#11 serial_ctrl_register_port(drv = <Value not available : Undefined value in stack frame for register X0>, port = <Value not available : Undefined value in stack frame for register X1>) at serial_ctrl.c:42
#12 uart_add_one_port(drv = <Value not available : Undefined value in stack frame for register X0>, port = <Value not available : Undefined value in stack frame for register X1>) at serial_port.c:136
#13 pl011_register_port(uap = (struct uart_amba_port*) 0xFFFF000800A41480) at amba-pl011.c:2764
#14 pl011_probe(dev = (struct amba_device*) 0xFFFF000800A41000, id = <Value not available : Undefined value in stack frame for register X1>) at amba-pl011.c:2829
#15 amba_probe(dev = (struct device*) 0xFFFF000800A41000) at bus.c:308
#16 call_driver_probe(dev = <Value optimised away by compiler>, drv = <Value optimised away by compiler>) at dd.c:578

在函数register_console 就完成了从earlycon到正式的console的切换,这个时候串口会有如下输出:

[    0.196646] Serial: AMBA PL011 UART driver
[    0.236878] 1c090000.serial: ttyAMA0 at MMIO 0x1c090000 (irq = 16, base_baud = 0) is a PL011 rev2
[    0.237034] printk: legacy console [ttyAMA0] enabled
[    0.237034] printk: legacy console [ttyAMA0] enabled
[    0.237160] printk: legacy bootconsole [pl11] disabled
[    0.237160] printk: legacy bootconsole [pl11] disabled

这里的log打印两次是因为这个时候老的还没有解注册,所以就会有两个console存在,会打印两次。

运行到register_console的最后,earlycon也会被unregister_console_locked给删除掉。 这样就完成了从earycon到正式的console的切换。

Note:别的系统可能会在start_kernel-> console_init里面完成console的转换。参考:

console_initcall(con_init);

Comments !