mbox series

[v3,00/46] pxe: Support read_all() for extlinux and PXE

Message ID 20241206023626.2456858-1-sjg@chromium.org
Headers show
Series pxe: Support read_all() for extlinux and PXE | expand

Message

Simon Glass Dec. 6, 2024, 2:35 a.m. UTC
This series implements read_all() so that it is possible to read all the
files relating to a bootflow, make adjustments and then boot.

Unfortunately quite a few things stand in the way, so this series
finishes off several pending items: zboot without CONFIG_CMDLINE,
required support in pxe_utils and the differing code in the extlinux and
PXE bootmeths. There is very little new code, but quite a lot of
refactoring.

The bootm, booti and bootz commands have all been refactored previously,
so that they can operate without needing CONFIG_CMDLINE to be enabled.
At the, time, zboot was left alone, since it is x86-specific and a bit
more trouble.

However it turns out that the booti support doesn't work with compressed
booti images, so this series resolved that problem too.

This series adds a programatic API for zboot and uses the forthcoming
bootstd 'image list' to collect information from an extlinux file. It is
therefore possible to parse the file, load the resulting images and then
examine/adjust the resulting bootflow, before booting.

The addition of options to extlinux resulted in the PXE and extlinux
bootmeth having slightly different features. This is tidied up in this
series, with common functions for both. This allows the same features
(loading images as a separate step) to be provided for PXE.

Changes in v3:
- Add new patch to convert IMAGE_FORMAT into an enum
- Add new patch to add a format for the booti file
- Add new patch to support booti format in bootm
- Add new patch to support compressed booti images in bootm
- Pass through the bootfile since extlinux and PXE are different
- Pass through the bootfile since extlinux and PXE are different
- Add new patch to detect booti format properly in pxe_utils
- Pass through the bootfile since extlinux and PXE are different

Changes in v2:
- Split out strlcpy() change into new patch
- Move strlcpy() change into an earlier patch
- Fix 'initaddr' typo'
- Rebase on earlier change
- Add new patch to simplify the code reading fdtcontroladdr and fdt_addr

Simon Glass (46):
  x86: Make do_zboot_states() static
  x86: Rename zboot_run() to zboot_run_args()
  x86: Drop duplicate definition of zimage_dump()
  x86: Move x86 zboot state into struct bootm_info
  x86: Rename state to bmi
  x86: Move the bootm state for zimage into cmd/
  bootstd: Correct display of kernel version
  x86: Drop the unnecessary base_ptr argument to zboot_dump()
  boot: Use strlcpy() in label_boot()
  boot: Split pxe label_boot() into two parts
  boot: Split pxe label_run_boot() into two parts
  boot: Pass just the FDT argument to label_process_fdt()
  bootm: Allow building bootm.c without CONFIG_SYS_BOOTM_LEN
  boot: Convert IMAGE_FORMAT into an enum
  boot: arm: riscv: sandbox: Add a format for the booti file
  boot: Support booti format in bootm
  boot: Support compressed booti images in bootm
  boot: pxe: Use bootm_...() functions where possible
  pxe_utils: Simplify default fdt in label_run_boot()
  boot: pxe: Refactor label_run_boot() to avoid cmdline
  net: Keep the bootstage functions together
  net: Tidy up the comments to parse_args()
  net: Simplify parse_args()
  net: Return the load address from parse_args()
  net: Return the address and size from parse_addr_size()
  net: Return the size from parse_args()
  net: Refactor part of netboot_common() into a function
  net: Drop #ifdef in parse_args()
  net: Provide a function to run network operations
  boot: Avoid using the cmdline in bootmeth_pxe and pxe cmd
  pxe: Drop the cmdline parameter
  pxe: Record the bootflow in the PXE context
  pxe: Allow skipping the boot
  test: Update bootm test to restore silent_console
  bootmeth_extlinux: Move extlinux_info into plat
  bootmeth_extlinux: Move pxe_context into plat
  bootmeth: Refactor to put options in a common file
  bootmeth_pxe: Implement the fallback option
  bootmeth_pxe: Drop the driver-private data
  bootmeth_extlinux: Move boot code into common file
  bootstd: Update extlinux and pxe to allow boot interruption
  pxe: Collect the FDT in the bootflow
  pxe: Deal with a missing FDT in the bootflow
  boot: Detect booti format properly in pxe_utils
  pxe_utils: Refactor to separate reading from booting
  bootmeth_extlinux: Refactor extlinux to split boot

 arch/arm/lib/image.c          |   9 +-
 arch/riscv/lib/image.c        |   9 +-
 arch/sandbox/lib/bootm.c      |   5 +
 arch/x86/include/asm/zimage.h |  57 +---
 arch/x86/lib/zimage.c         | 102 ++++---
 boot/Makefile                 |   6 +-
 boot/bootm.c                  |  76 ++++-
 boot/bootmeth_cros.c          |   6 +-
 boot/bootmeth_extlinux.c      |  98 +------
 boot/bootmeth_pxe.c           |  50 ++--
 boot/ext_pxe_common.c         | 139 +++++++++
 boot/image-board.c            |  36 +--
 boot/pxe_utils.c              | 513 +++++++++++++++++++++-------------
 cmd/bootflow.c                |   8 +-
 cmd/net.c                     |  92 +++---
 cmd/pxe.c                     |  30 +-
 cmd/sysboot.c                 |   4 +-
 cmd/x86/zboot.c               |  33 ++-
 include/bootm.h               |  63 ++++-
 include/bootmeth.h            |   8 +-
 include/extlinux.h            |  60 +++-
 include/image.h               |  48 +++-
 include/net-common.h          |  30 ++
 include/pxe_utils.h           |  56 +++-
 net/net.c                     |  44 +++
 test/boot/bootflow.c          |  71 +++++
 test/boot/bootm.c             |   7 +
 27 files changed, 1127 insertions(+), 533 deletions(-)
 create mode 100644 boot/ext_pxe_common.c

Comments

Tom Rini Jan. 16, 2025, 11:28 p.m. UTC | #1
On Thu, Dec 05, 2024 at 07:35:39PM -0700, Simon Glass wrote:

> This series implements read_all() so that it is possible to read all the
> files relating to a bootflow, make adjustments and then boot.
> 
> Unfortunately quite a few things stand in the way, so this series
> finishes off several pending items: zboot without CONFIG_CMDLINE,
> required support in pxe_utils and the differing code in the extlinux and
> PXE bootmeths. There is very little new code, but quite a lot of
> refactoring.
> 
> The bootm, booti and bootz commands have all been refactored previously,
> so that they can operate without needing CONFIG_CMDLINE to be enabled.
> At the, time, zboot was left alone, since it is x86-specific and a bit
> more trouble.
> 
> However it turns out that the booti support doesn't work with compressed
> booti images, so this series resolved that problem too.
> 
> This series adds a programatic API for zboot and uses the forthcoming
> bootstd 'image list' to collect information from an extlinux file. It is
> therefore possible to parse the file, load the resulting images and then
> examine/adjust the resulting bootflow, before booting.
> 
> The addition of options to extlinux resulted in the PXE and extlinux
> bootmeth having slightly different features. This is tidied up in this
> series, with common functions for both. This allows the same features
> (loading images as a separate step) to be provided for PXE.

To sync up with what I said on IRC, this breaks
test/py/tests/test_net_boot.py likely due to changes in the output.
Simon Glass Jan. 20, 2025, 7:20 p.m. UTC | #2
Hi Tom,

On Thu, 16 Jan 2025 at 16:28, Tom Rini <trini@konsulko.com> wrote:
>
> On Thu, Dec 05, 2024 at 07:35:39PM -0700, Simon Glass wrote:
>
> > This series implements read_all() so that it is possible to read all the
> > files relating to a bootflow, make adjustments and then boot.
> >
> > Unfortunately quite a few things stand in the way, so this series
> > finishes off several pending items: zboot without CONFIG_CMDLINE,
> > required support in pxe_utils and the differing code in the extlinux and
> > PXE bootmeths. There is very little new code, but quite a lot of
> > refactoring.
> >
> > The bootm, booti and bootz commands have all been refactored previously,
> > so that they can operate without needing CONFIG_CMDLINE to be enabled.
> > At the, time, zboot was left alone, since it is x86-specific and a bit
> > more trouble.
> >
> > However it turns out that the booti support doesn't work with compressed
> > booti images, so this series resolved that problem too.
> >
> > This series adds a programatic API for zboot and uses the forthcoming
> > bootstd 'image list' to collect information from an extlinux file. It is
> > therefore possible to parse the file, load the resulting images and then
> > examine/adjust the resulting bootflow, before booting.
> >
> > The addition of options to extlinux resulted in the PXE and extlinux
> > bootmeth having slightly different features. This is tidied up in this
> > series, with common functions for both. This allows the same features
> > (loading images as a separate step) to be provided for PXE.
>
> To sync up with what I said on IRC, this breaks
> test/py/tests/test_net_boot.py likely due to changes in the output.

I am taking a look at this, but I'm really not sure how to set up
test_net_pxe_boot_config test

I have this in my py/ellesmere/u_boot_boardenv_rpi_3_32b.py file:

env__pxe_boot_test_skip = False
env__net_pxe_bootable_file = {
    'fn': 'default.boot',
    'addr': 0x10000000,
    # 'size': 74,
    'timeout': 50000,
    'pattern': 'Linux',
    'valid_label': '1',
    'invalid_label': '2',
    'exp_str_invalid': 'Skipping install for failure retrieving',
    'local_label': '3',
    'exp_str_local': 'missing environment variable: localcmd',
    'empty_label': '4',
    'exp_str_empty': 'No kernel given, skipping boot',
    'check_type': 'boot_error',
    'check_pattern': 'ERROR',
}

This is what I see when running test_net_pxe_boot_config :

test/py/tests/test_net_boot.py +u-boot-test-reset rpi_3_32b rpi3
{lab mode}
Building U-Boot in pytest-source dir for rpi_3_32b
Bootstrapping U-Boot from dir /tmp/b/rpi_3_32b
Writing U-Boot using method rpi3
cat: /sys/class/block/sdz1/size: No such file or directory
cat: /sys/class/block/sdz1/size: No such file or directory

{lab ready in 10.4s: U-Boot 2025.01-rc6-00508-gd6da3dbaef57-dirty (Jan
20 2025 - 12:11:05 -0700)}
skip False
U-Boot> setenv autoload no
U-Boot> U-Boot> dhcp
Waiting for Ethernet connection... done.
BOOTP broadcast 1
DHCP client bound to address 192.168.4.50 (4 ms)
U-Boot> U-Boot> echo $bootfile

U-Boot> U-Boot> setenv pxefile_addr_r 10000000
U-Boot> U-Boot> pxe get
missing environment variable: pxeuuid
Retrieving file: pxelinux.cfg/01-b8-27-eb-b4-f9-f2
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/01-b8-27-eb-b4-f9-f2'.
Load address: 0x10000000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
Retrieving file: pxelinux.cfg/C0A80432
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/C0A80432'.
Load address: 0x10000000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
Retrieving file: pxelinux.cfg/C0A8043
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/C0A8043'.
Load address: 0x10000000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
Retrieving file: pxelinux.cfg/C0A804
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/C0A804'.
Load address: 0x10000000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
Retrieving file: pxelinux.cfg/C0A80
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/C0A80'.
Load address: 0x10000000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
Retrieving file: pxelinux.cfg/C0A8
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/C0A8'.
Load address: 0x10000000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
Retrieving file: pxelinux.cfg/C0A
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/C0A'.
Load address: 0x10000000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
Retrieving file: pxelinux.cfg/C0
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/C0'.
Load address: 0x10000000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
Retrieving file: pxelinux.cfg/C
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/C'.
Load address: 0x10000000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
Retrieving file: pxelinux.cfg/default-arm-bcm283x-rpi
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/default-arm-bcm283x-rpi'.
Load address: 0x10000000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
Retrieving file: pxelinux.cfg/default-arm-bcm283x
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/default-arm-bcm283x'.
Load address: 0x10000000
Loading: ##################################################  64 Bytes
4.9 KiB/s
done
Bytes transferred = 64 (40 hex)
Config file '<NULL>' found
U-Boot> U-Boot> pxe boot 10000000
Retrieving file: pxelinux.cfg/default-arm
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/default-arm'.
Load address: 0x10000044
Loading: ##################################################  349 Bytes
30.3 KiB/s
done
Bytes transferred = 349 (15d hex)
Retrieving file: pxelinux.cfg/default
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/default'.
Load address: 0x100001a4
Loading: ##################################################  108 Bytes
9.8 KiB/s
done
Bytes transferred = 108 (6c hex)
Linux boot selections
1: Boot kernel
2: Invalid boot
3: Local boot
4: Empty boot
Enter choice: 3
3: Local boot
missing environment variable: localcmd
U-Boot> pxe boot 10000000
Retrieving file: pxelinux.cfg/default-arm
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/default-arm'.
Load address: 0x10000044
Loading: ##################################################  349 Bytes
19.5 KiB/s
done
Bytes transferred = 349 (15d hex)
Retrieving file: pxelinux.cfg/default
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'pxelinux.cfg/default'.
Load address: 0x100001a4
Loading: ##################################################  108 Bytes
9.8 KiB/s
done
Bytes transferred = 108 (6c hex)
Linux boot selections
1: Boot kernel
2: Invalid boot
3: Local boot
4: Empty boot
Enter choice: 4
4: Empty boot
No kernel given, skipping boot
1: Boot kernel
Retrieving file: Image
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'Image'.
Load address: 0x80000
Loading: ##################################################  6 Bytes
0 Bytes/s
done
Bytes transferred = 6 (6 hex)
Retrieving file: rootfs.cpio.gz.u-boot
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'rootfs.cpio.gz.u-boot'.
Load address: 0x2700000
Loading: ##################################################  30.6 MiB
995.1 KiB/s
done
Bytes transferred = 32123014 (1ea2886 hex)
Retrieving file: system.dtb
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'system.dtb'.
Load address: 0x2600000
Loading: ##################################################  11 Bytes
1000 Bytes/s
done
Bytes transferred = 11 (b hex)
zimage: Bad magic!
2: Invalid boot
Retrieving file: kernels/install.bin
Waiting for Ethernet connection... done.
Using smsc95xx_eth device
TFTP from server 192.168.4.1; our IP address is 192.168.4.50
Filename 'kernels/install.bin'.
Load address: 0x80000
Loading: *
TFTP error: 'File not found' (1)
Not retrying...
Skipping install for failure retrieving kernel
3: Local boot
missing environment variable: localcmd
U-Boot> F

Do you have a passing run I could compare against, and/or a boardenv
file that works?

Thanks,
Simon
Tom Rini Jan. 20, 2025, 11:44 p.m. UTC | #3
On Mon, Jan 20, 2025 at 12:20:06PM -0700, Simon Glass wrote:
> Hi Tom,
> 
> On Thu, 16 Jan 2025 at 16:28, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Thu, Dec 05, 2024 at 07:35:39PM -0700, Simon Glass wrote:
> >
> > > This series implements read_all() so that it is possible to read all the
> > > files relating to a bootflow, make adjustments and then boot.
> > >
> > > Unfortunately quite a few things stand in the way, so this series
> > > finishes off several pending items: zboot without CONFIG_CMDLINE,
> > > required support in pxe_utils and the differing code in the extlinux and
> > > PXE bootmeths. There is very little new code, but quite a lot of
> > > refactoring.
> > >
> > > The bootm, booti and bootz commands have all been refactored previously,
> > > so that they can operate without needing CONFIG_CMDLINE to be enabled.
> > > At the, time, zboot was left alone, since it is x86-specific and a bit
> > > more trouble.
> > >
> > > However it turns out that the booti support doesn't work with compressed
> > > booti images, so this series resolved that problem too.
> > >
> > > This series adds a programatic API for zboot and uses the forthcoming
> > > bootstd 'image list' to collect information from an extlinux file. It is
> > > therefore possible to parse the file, load the resulting images and then
> > > examine/adjust the resulting bootflow, before booting.
> > >
> > > The addition of options to extlinux resulted in the PXE and extlinux
> > > bootmeth having slightly different features. This is tidied up in this
> > > series, with common functions for both. This allows the same features
> > > (loading images as a separate step) to be provided for PXE.
> >
> > To sync up with what I said on IRC, this breaks
> > test/py/tests/test_net_boot.py likely due to changes in the output.
> 
> I am taking a look at this, but I'm really not sure how to set up
> test_net_pxe_boot_config test
> 
> I have this in my py/ellesmere/u_boot_boardenv_rpi_3_32b.py file:
> 
> env__pxe_boot_test_skip = False
> env__net_pxe_bootable_file = {
>     'fn': 'default.boot',
>     'addr': 0x10000000,
>     # 'size': 74,
>     'timeout': 50000,
>     'pattern': 'Linux',
>     'valid_label': '1',
>     'invalid_label': '2',
>     'exp_str_invalid': 'Skipping install for failure retrieving',
>     'local_label': '3',
>     'exp_str_local': 'missing environment variable: localcmd',
>     'empty_label': '4',
>     'exp_str_empty': 'No kernel given, skipping boot',
>     'check_type': 'boot_error',
>     'check_pattern': 'ERROR',
> }
> 
> This is what I see when running test_net_pxe_boot_config :
> 
> test/py/tests/test_net_boot.py +u-boot-test-reset rpi_3_32b rpi3
> {lab mode}
> Building U-Boot in pytest-source dir for rpi_3_32b
> Bootstrapping U-Boot from dir /tmp/b/rpi_3_32b
> Writing U-Boot using method rpi3
> cat: /sys/class/block/sdz1/size: No such file or directory
> cat: /sys/class/block/sdz1/size: No such file or directory
> 
> {lab ready in 10.4s: U-Boot 2025.01-rc6-00508-gd6da3dbaef57-dirty (Jan
> 20 2025 - 12:11:05 -0700)}
> skip False
> U-Boot> setenv autoload no
> U-Boot> U-Boot> dhcp
> Waiting for Ethernet connection... done.
> BOOTP broadcast 1
> DHCP client bound to address 192.168.4.50 (4 ms)
> U-Boot> U-Boot> echo $bootfile
> 
> U-Boot> U-Boot> setenv pxefile_addr_r 10000000
> U-Boot> U-Boot> pxe get
> missing environment variable: pxeuuid
> Retrieving file: pxelinux.cfg/01-b8-27-eb-b4-f9-f2
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/01-b8-27-eb-b4-f9-f2'.
> Load address: 0x10000000
> Loading: *
> TFTP error: 'File not found' (1)
> Not retrying...
> Retrieving file: pxelinux.cfg/C0A80432
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/C0A80432'.
> Load address: 0x10000000
> Loading: *
> TFTP error: 'File not found' (1)
> Not retrying...
> Retrieving file: pxelinux.cfg/C0A8043
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/C0A8043'.
> Load address: 0x10000000
> Loading: *
> TFTP error: 'File not found' (1)
> Not retrying...
> Retrieving file: pxelinux.cfg/C0A804
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/C0A804'.
> Load address: 0x10000000
> Loading: *
> TFTP error: 'File not found' (1)
> Not retrying...
> Retrieving file: pxelinux.cfg/C0A80
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/C0A80'.
> Load address: 0x10000000
> Loading: *
> TFTP error: 'File not found' (1)
> Not retrying...
> Retrieving file: pxelinux.cfg/C0A8
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/C0A8'.
> Load address: 0x10000000
> Loading: *
> TFTP error: 'File not found' (1)
> Not retrying...
> Retrieving file: pxelinux.cfg/C0A
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/C0A'.
> Load address: 0x10000000
> Loading: *
> TFTP error: 'File not found' (1)
> Not retrying...
> Retrieving file: pxelinux.cfg/C0
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/C0'.
> Load address: 0x10000000
> Loading: *
> TFTP error: 'File not found' (1)
> Not retrying...
> Retrieving file: pxelinux.cfg/C
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/C'.
> Load address: 0x10000000
> Loading: *
> TFTP error: 'File not found' (1)
> Not retrying...
> Retrieving file: pxelinux.cfg/default-arm-bcm283x-rpi
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/default-arm-bcm283x-rpi'.
> Load address: 0x10000000
> Loading: *
> TFTP error: 'File not found' (1)
> Not retrying...
> Retrieving file: pxelinux.cfg/default-arm-bcm283x
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/default-arm-bcm283x'.
> Load address: 0x10000000
> Loading: ##################################################  64 Bytes
> 4.9 KiB/s
> done
> Bytes transferred = 64 (40 hex)
> Config file '<NULL>' found
> U-Boot> U-Boot> pxe boot 10000000
> Retrieving file: pxelinux.cfg/default-arm
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/default-arm'.
> Load address: 0x10000044
> Loading: ##################################################  349 Bytes
> 30.3 KiB/s
> done
> Bytes transferred = 349 (15d hex)
> Retrieving file: pxelinux.cfg/default
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/default'.
> Load address: 0x100001a4
> Loading: ##################################################  108 Bytes
> 9.8 KiB/s
> done
> Bytes transferred = 108 (6c hex)
> Linux boot selections
> 1: Boot kernel
> 2: Invalid boot
> 3: Local boot
> 4: Empty boot
> Enter choice: 3
> 3: Local boot
> missing environment variable: localcmd
> U-Boot> pxe boot 10000000
> Retrieving file: pxelinux.cfg/default-arm
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/default-arm'.
> Load address: 0x10000044
> Loading: ##################################################  349 Bytes
> 19.5 KiB/s
> done
> Bytes transferred = 349 (15d hex)
> Retrieving file: pxelinux.cfg/default
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'pxelinux.cfg/default'.
> Load address: 0x100001a4
> Loading: ##################################################  108 Bytes
> 9.8 KiB/s
> done
> Bytes transferred = 108 (6c hex)
> Linux boot selections
> 1: Boot kernel
> 2: Invalid boot
> 3: Local boot
> 4: Empty boot
> Enter choice: 4
> 4: Empty boot
> No kernel given, skipping boot
> 1: Boot kernel
> Retrieving file: Image
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'Image'.
> Load address: 0x80000
> Loading: ##################################################  6 Bytes
> 0 Bytes/s
> done
> Bytes transferred = 6 (6 hex)
> Retrieving file: rootfs.cpio.gz.u-boot
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'rootfs.cpio.gz.u-boot'.
> Load address: 0x2700000
> Loading: ##################################################  30.6 MiB
> 995.1 KiB/s
> done
> Bytes transferred = 32123014 (1ea2886 hex)
> Retrieving file: system.dtb
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'system.dtb'.
> Load address: 0x2600000
> Loading: ##################################################  11 Bytes
> 1000 Bytes/s
> done
> Bytes transferred = 11 (b hex)
> zimage: Bad magic!
> 2: Invalid boot
> Retrieving file: kernels/install.bin
> Waiting for Ethernet connection... done.
> Using smsc95xx_eth device
> TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> Filename 'kernels/install.bin'.
> Load address: 0x80000
> Loading: *
> TFTP error: 'File not found' (1)
> Not retrying...
> Skipping install for failure retrieving kernel
> 3: Local boot
> missing environment variable: localcmd
> U-Boot> F
> 
> Do you have a passing run I could compare against, and/or a boardenv
> file that works?

I didn't test this on 32bit ARM, only 64bit. For Pi 3:

env__net_uses_usb = True

env__net_dhcp_server = True

env__tftp_boot_test_skip = False

env__net_tftp_bootable_file = {
    'fn': 'v6.6/image.fit.nocomp',
    'addr': 0x00200000,
    'size': 85984256,
    'crc32': '754c839a',
    'pattern': 'Linux',
    'config': 'conf-852',
}

# Details regarding a file that may be read from a TFTP server. This variable
# may be omitted or set to None if PXE testing is not possible or desired.
env__net_pxe_bootable_file = {
    'fn': 'default',
    'addr': 0x00200000,
    'size': 64,
    'timeout': 50000,
    'pattern': 'Linux',
    'valid_label': '1',
    'invalid_label': '2',
    'exp_str_invalid': 'Skipping install for failure retrieving',
    'local_label': '3',
    'exp_str_local': 'missing environment variable: localcmd',
    'empty_label': '4',
    'exp_str_empty': 'No kernel given, skipping boot',
}

# False or omitted if a PXE boot test should be tested.
# If PXE boot testing is not possible or desired, set this variable to True.
# For example: If pxe configuration file is not proper to boot
env__pxe_boot_test_skip = False

And:
$ cat /srv/tftp/pxelinux.cfg/default
label Linux
    menu label Boot kernel
    kernel Image
    fdt system.dtb
    initrd rootfs.cpio.gz.u-boot
$ cat /srv/tftp/pxelinux.cfg/default-arm
menu title Linux boot selections
menu include pxelinux.cfg/default

label install
    menu label Invalid boot
    kernel kernels/install.bin
    append console=ttyAMA0,38400 debug earlyprintk
    initrd initrds/uzInitrdDebInstall

label local
    menu label Local boot
    append root=/dev/sdb1
    localboot 1

label boot
    menu label Empty boot
$ cat /srv/tftp/pxelinux.cfg/01-b8-27-eb-fc-64-a6
menu include pxelinux.cfg/default-arm
timeout 50

default Linux

And kernels/install.bin, etc, do not exist. Only v6.6/image.fit.nocomp
exists.
Simon Glass Jan. 23, 2025, 2:37 p.m. UTC | #4
-Bin

Hi Tom,

On Mon, 20 Jan 2025 at 16:45, Tom Rini <trini@konsulko.com> wrote:
>
> On Mon, Jan 20, 2025 at 12:20:06PM -0700, Simon Glass wrote:
> > Hi Tom,
> >
> > On Thu, 16 Jan 2025 at 16:28, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Thu, Dec 05, 2024 at 07:35:39PM -0700, Simon Glass wrote:
> > >
> > > > This series implements read_all() so that it is possible to read all the
> > > > files relating to a bootflow, make adjustments and then boot.
> > > >
> > > > Unfortunately quite a few things stand in the way, so this series
> > > > finishes off several pending items: zboot without CONFIG_CMDLINE,
> > > > required support in pxe_utils and the differing code in the extlinux and
> > > > PXE bootmeths. There is very little new code, but quite a lot of
> > > > refactoring.
> > > >
> > > > The bootm, booti and bootz commands have all been refactored previously,
> > > > so that they can operate without needing CONFIG_CMDLINE to be enabled.
> > > > At the, time, zboot was left alone, since it is x86-specific and a bit
> > > > more trouble.
> > > >
> > > > However it turns out that the booti support doesn't work with compressed
> > > > booti images, so this series resolved that problem too.
> > > >
> > > > This series adds a programatic API for zboot and uses the forthcoming
> > > > bootstd 'image list' to collect information from an extlinux file. It is
> > > > therefore possible to parse the file, load the resulting images and then
> > > > examine/adjust the resulting bootflow, before booting.
> > > >
> > > > The addition of options to extlinux resulted in the PXE and extlinux
> > > > bootmeth having slightly different features. This is tidied up in this
> > > > series, with common functions for both. This allows the same features
> > > > (loading images as a separate step) to be provided for PXE.
> > >
> > > To sync up with what I said on IRC, this breaks
> > > test/py/tests/test_net_boot.py likely due to changes in the output.
> >
> > I am taking a look at this, but I'm really not sure how to set up
> > test_net_pxe_boot_config test
> >
> > I have this in my py/ellesmere/u_boot_boardenv_rpi_3_32b.py file:
> >
> > env__pxe_boot_test_skip = False
> > env__net_pxe_bootable_file = {
> >     'fn': 'default.boot',
> >     'addr': 0x10000000,
> >     # 'size': 74,
> >     'timeout': 50000,
> >     'pattern': 'Linux',
> >     'valid_label': '1',
> >     'invalid_label': '2',
> >     'exp_str_invalid': 'Skipping install for failure retrieving',
> >     'local_label': '3',
> >     'exp_str_local': 'missing environment variable: localcmd',
> >     'empty_label': '4',
> >     'exp_str_empty': 'No kernel given, skipping boot',
> >     'check_type': 'boot_error',
> >     'check_pattern': 'ERROR',
> > }
> >
> > This is what I see when running test_net_pxe_boot_config :
> >
> > test/py/tests/test_net_boot.py +u-boot-test-reset rpi_3_32b rpi3
> > {lab mode}
> > Building U-Boot in pytest-source dir for rpi_3_32b
> > Bootstrapping U-Boot from dir /tmp/b/rpi_3_32b
> > Writing U-Boot using method rpi3
> > cat: /sys/class/block/sdz1/size: No such file or directory
> > cat: /sys/class/block/sdz1/size: No such file or directory
> >
> > {lab ready in 10.4s: U-Boot 2025.01-rc6-00508-gd6da3dbaef57-dirty (Jan
> > 20 2025 - 12:11:05 -0700)}
> > skip False
> > U-Boot> setenv autoload no
> > U-Boot> U-Boot> dhcp
> > Waiting for Ethernet connection... done.
> > BOOTP broadcast 1
> > DHCP client bound to address 192.168.4.50 (4 ms)
> > U-Boot> U-Boot> echo $bootfile
> >
> > U-Boot> U-Boot> setenv pxefile_addr_r 10000000
> > U-Boot> U-Boot> pxe get
> > missing environment variable: pxeuuid
> > Retrieving file: pxelinux.cfg/01-b8-27-eb-b4-f9-f2
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/01-b8-27-eb-b4-f9-f2'.
> > Load address: 0x10000000
> > Loading: *
> > TFTP error: 'File not found' (1)
> > Not retrying...
> > Retrieving file: pxelinux.cfg/C0A80432
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/C0A80432'.
> > Load address: 0x10000000
> > Loading: *
> > TFTP error: 'File not found' (1)
> > Not retrying...
> > Retrieving file: pxelinux.cfg/C0A8043
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/C0A8043'.
> > Load address: 0x10000000
> > Loading: *
> > TFTP error: 'File not found' (1)
> > Not retrying...
> > Retrieving file: pxelinux.cfg/C0A804
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/C0A804'.
> > Load address: 0x10000000
> > Loading: *
> > TFTP error: 'File not found' (1)
> > Not retrying...
> > Retrieving file: pxelinux.cfg/C0A80
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/C0A80'.
> > Load address: 0x10000000
> > Loading: *
> > TFTP error: 'File not found' (1)
> > Not retrying...
> > Retrieving file: pxelinux.cfg/C0A8
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/C0A8'.
> > Load address: 0x10000000
> > Loading: *
> > TFTP error: 'File not found' (1)
> > Not retrying...
> > Retrieving file: pxelinux.cfg/C0A
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/C0A'.
> > Load address: 0x10000000
> > Loading: *
> > TFTP error: 'File not found' (1)
> > Not retrying...
> > Retrieving file: pxelinux.cfg/C0
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/C0'.
> > Load address: 0x10000000
> > Loading: *
> > TFTP error: 'File not found' (1)
> > Not retrying...
> > Retrieving file: pxelinux.cfg/C
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/C'.
> > Load address: 0x10000000
> > Loading: *
> > TFTP error: 'File not found' (1)
> > Not retrying...
> > Retrieving file: pxelinux.cfg/default-arm-bcm283x-rpi
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/default-arm-bcm283x-rpi'.
> > Load address: 0x10000000
> > Loading: *
> > TFTP error: 'File not found' (1)
> > Not retrying...
> > Retrieving file: pxelinux.cfg/default-arm-bcm283x
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/default-arm-bcm283x'.
> > Load address: 0x10000000
> > Loading: ##################################################  64 Bytes
> > 4.9 KiB/s
> > done
> > Bytes transferred = 64 (40 hex)
> > Config file '<NULL>' found
> > U-Boot> U-Boot> pxe boot 10000000
> > Retrieving file: pxelinux.cfg/default-arm
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/default-arm'.
> > Load address: 0x10000044
> > Loading: ##################################################  349 Bytes
> > 30.3 KiB/s
> > done
> > Bytes transferred = 349 (15d hex)
> > Retrieving file: pxelinux.cfg/default
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/default'.
> > Load address: 0x100001a4
> > Loading: ##################################################  108 Bytes
> > 9.8 KiB/s
> > done
> > Bytes transferred = 108 (6c hex)
> > Linux boot selections
> > 1: Boot kernel
> > 2: Invalid boot
> > 3: Local boot
> > 4: Empty boot
> > Enter choice: 3
> > 3: Local boot
> > missing environment variable: localcmd
> > U-Boot> pxe boot 10000000
> > Retrieving file: pxelinux.cfg/default-arm
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/default-arm'.
> > Load address: 0x10000044
> > Loading: ##################################################  349 Bytes
> > 19.5 KiB/s
> > done
> > Bytes transferred = 349 (15d hex)
> > Retrieving file: pxelinux.cfg/default
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'pxelinux.cfg/default'.
> > Load address: 0x100001a4
> > Loading: ##################################################  108 Bytes
> > 9.8 KiB/s
> > done
> > Bytes transferred = 108 (6c hex)
> > Linux boot selections
> > 1: Boot kernel
> > 2: Invalid boot
> > 3: Local boot
> > 4: Empty boot
> > Enter choice: 4
> > 4: Empty boot
> > No kernel given, skipping boot
> > 1: Boot kernel
> > Retrieving file: Image
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'Image'.
> > Load address: 0x80000
> > Loading: ##################################################  6 Bytes
> > 0 Bytes/s
> > done
> > Bytes transferred = 6 (6 hex)
> > Retrieving file: rootfs.cpio.gz.u-boot
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'rootfs.cpio.gz.u-boot'.
> > Load address: 0x2700000
> > Loading: ##################################################  30.6 MiB
> > 995.1 KiB/s
> > done
> > Bytes transferred = 32123014 (1ea2886 hex)
> > Retrieving file: system.dtb
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'system.dtb'.
> > Load address: 0x2600000
> > Loading: ##################################################  11 Bytes
> > 1000 Bytes/s
> > done
> > Bytes transferred = 11 (b hex)
> > zimage: Bad magic!
> > 2: Invalid boot
> > Retrieving file: kernels/install.bin
> > Waiting for Ethernet connection... done.
> > Using smsc95xx_eth device
> > TFTP from server 192.168.4.1; our IP address is 192.168.4.50
> > Filename 'kernels/install.bin'.
> > Load address: 0x80000
> > Loading: *
> > TFTP error: 'File not found' (1)
> > Not retrying...
> > Skipping install for failure retrieving kernel
> > 3: Local boot
> > missing environment variable: localcmd
> > U-Boot> F
> >
> > Do you have a passing run I could compare against, and/or a boardenv
> > file that works?
>
> I didn't test this on 32bit ARM, only 64bit. For Pi 3:
>
> env__net_uses_usb = True
>
> env__net_dhcp_server = True
>
> env__tftp_boot_test_skip = False
>
> env__net_tftp_bootable_file = {
>     'fn': 'v6.6/image.fit.nocomp',
>     'addr': 0x00200000,
>     'size': 85984256,
>     'crc32': '754c839a',
>     'pattern': 'Linux',
>     'config': 'conf-852',
> }
>
> # Details regarding a file that may be read from a TFTP server. This variable
> # may be omitted or set to None if PXE testing is not possible or desired.
> env__net_pxe_bootable_file = {
>     'fn': 'default',
>     'addr': 0x00200000,
>     'size': 64,
>     'timeout': 50000,
>     'pattern': 'Linux',
>     'valid_label': '1',
>     'invalid_label': '2',
>     'exp_str_invalid': 'Skipping install for failure retrieving',
>     'local_label': '3',
>     'exp_str_local': 'missing environment variable: localcmd',
>     'empty_label': '4',
>     'exp_str_empty': 'No kernel given, skipping boot',
> }
>
> # False or omitted if a PXE boot test should be tested.
> # If PXE boot testing is not possible or desired, set this variable to True.
> # For example: If pxe configuration file is not proper to boot
> env__pxe_boot_test_skip = False
>
> And:
> $ cat /srv/tftp/pxelinux.cfg/default
> label Linux
>     menu label Boot kernel
>     kernel Image
>     fdt system.dtb
>     initrd rootfs.cpio.gz.u-boot
> $ cat /srv/tftp/pxelinux.cfg/default-arm
> menu title Linux boot selections
> menu include pxelinux.cfg/default
>
> label install
>     menu label Invalid boot
>     kernel kernels/install.bin
>     append console=ttyAMA0,38400 debug earlyprintk
>     initrd initrds/uzInitrdDebInstall
>
> label local
>     menu label Local boot
>     append root=/dev/sdb1
>     localboot 1
>
> label boot
>     menu label Empty boot
> $ cat /srv/tftp/pxelinux.cfg/01-b8-27-eb-fc-64-a6
> menu include pxelinux.cfg/default-arm
> timeout 50
>
> default Linux
>
> And kernels/install.bin, etc, do not exist. Only v6.6/image.fit.nocomp
> exists.

Thanks very much for that. I have got it running and bisected the
problem to the penultimate patch, so will figure it out and send v4.

Regards,
Simon
Tom Rini Jan. 24, 2025, 7:18 p.m. UTC | #5
On Thu, Dec 05, 2024 at 07:35:39PM -0700, Simon Glass wrote:

> This series implements read_all() so that it is possible to read all the
> files relating to a bootflow, make adjustments and then boot.
> 
> Unfortunately quite a few things stand in the way, so this series
> finishes off several pending items: zboot without CONFIG_CMDLINE,
> required support in pxe_utils and the differing code in the extlinux and
> PXE bootmeths. There is very little new code, but quite a lot of
> refactoring.
> 
> The bootm, booti and bootz commands have all been refactored previously,
> so that they can operate without needing CONFIG_CMDLINE to be enabled.
> At the, time, zboot was left alone, since it is x86-specific and a bit
> more trouble.
> 
> However it turns out that the booti support doesn't work with compressed
> booti images, so this series resolved that problem too.
> 
> This series adds a programatic API for zboot and uses the forthcoming
> bootstd 'image list' to collect information from an extlinux file. It is
> therefore possible to parse the file, load the resulting images and then
> examine/adjust the resulting bootflow, before booting.
> 
> The addition of options to extlinux resulted in the PXE and extlinux
> bootmeth having slightly different features. This is tidied up in this
> series, with common functions for both. This allows the same features
> (loading images as a separate step) to be provided for PXE.

With v4 of 45/46, this fails with lwIP on Pi in the PXE tests now. You
likely don't need more than CONFIG_NET_LWIP=y being added but for
completeness here's everything:
CONFIG_UNIT_TEST=y
# CONFIG_CMD_EFIDEBUG is not set
CONFIG_CMD_BOOTMENU=y
CONFIG_CMD_LOG=y
# CONFIG_CMD_BOOTEFI_SELFTEST is not set
CONFIG_IPV6=y
CONFIG_IPV6_ROUTER_DISCOVERY=y
CONFIG_CMD_TFTPPUT=y
CONFIG_CMD_WGET=y
CONFIG_FIT=y
CONFIG_FIT_SIGNATURE=y
CONFIG_FIT_BEST_MATCH=y
CONFIG_SYS_BOOTM_LEN=0x4000000
CONFIG_NET_LWIP=y
CONFIG_BOOTSTAGE_STASH_ADDR=0x02400000
CONFIG_BOOTSTAGE=y
CONFIG_BOOTSTAGE_STASH=y
CONFIG_CMD_BOOTSTAGE=y
on top of rpi_3_defconfig.