| Message ID | 20240611200156.2245525-1-sjg@chromium.org |
|---|---|
| Headers | show |
| Series | labgrid: Provide an integration with Labgrid | expand |
Hi, On Tue, 11 Jun 2024 at 14:02, Simon Glass <sjg@chromium.org> wrote: > > Labgrid provides access to a hardware lab in an automated way. It is > possible to boot U-Boot on boards in the lab without physically touching > them. It relies on relays, USB UARTs and SD muxes, among other things. > > By way of background, about 4 years ago I wrong a thing called Labman[1] > which allowed my lab of about 30 devices to be operated remotely, using > tbot for the console and build integration. While it worked OK and I > used it for many bisects, I didn't take it any further. > > It turns out that there was already an existing program, called Labgrid, > which I did not know about at time (thank you Tom for telling me). It is > more rounded than Labman and has a number of advantages: > > - does not need udev rules, mostly > - has several existing users who rely on it > - supports multiple machines exporting their devices > > It lacks a 'lab check' feature and a few other things, but these can be > remedied. > > On and off over the past several weeks I have been experimenting with > Labgrid. I have managed to create an initial U-Boot integration (this > series) by adding various features to Labgrid[2] and the U-Boot test > hooks. > > I hope that this might inspire others to set up boards and run tests > automatically, rather than relying on infrequent, manual test. Perhaps > it may even be possible to have a number of labs available. > > Included in the integration are a number of simple scripts which make it > easy to connect to boards and run tests: > > ub-int <target> > Build and boot on a target, starting an interactive session > > ub-cli <target> > Build and boot on a target, ensure U-Boot starts and provide an interactive > session from there > > ub-smoke <target> > Smoke test U-Boot to check that it boots to a prompt on a target > > ub-bisect > Bisect a git tree to locate a failure on a particular target > > ub-pyt <target> <testspec> > Run U-Boot pytests on a target > > Some of these help to provide the same tbot[4] workflow which I have > relied on for several years, albeit much simpler versions. > > The goal here is to create some sort of script which can collect > patches from the mailing list, apply them and test them on a selection > of boards. I suspect that script already exists, so please let me know > what you suggest. > > I hope you find this interesting and take a look! > > [1] https://github.com/sjg20/u-boot/tree/lab6a > [2] https://github.com/labgrid-project/labgrid/pull/1411 > [3] https://github.com/sjg20/uboot-test-hooks/tree/labgrid > [4] https://tbot.tools/index.html > > > Simon Glass (42): > trace: Update test to tolerate different trace-cmd version > binman: efi: Correct entry docs > binman: Regenerate nxp docs > binman: ti: Regenerate entry docs > binman: Update the entrydocs header > buildman: Make mrproper an argument to _reconfigure() > buildman: Make mrproper an argument to _config_and_build() > buildman: Make mrproper an argument to run_commit() > buildman: Avoid rebuilding when --mrproper is used > buildman: Add a flag to force mrproper on failure > buildman: Retry the build for current source > buildman: Add a way to limit the number of buildmans > dm: core: Enhance comments on bind_drivers_pass() > initcall: Correct use of relocation offset > am33xx: Provide a function to set up the debug UART > sunxi: Mark scp as optional > google: Disable TPMv2 on most Chromebooks > meson: Correct driver declaration for meson_axg_gpio > test: Allow signaling that U-Boot is ready > test: Make bootstd init run only on sandbox > test: Use a constant for the test timeout > test: Pass stderr to stdout > test: Release board after tests complete > log: Allow tests to pass with CONFIG_LOGF_FUNC_PAD set > test: Allow connecting to a running board > test: Decode exceptions only with sandbox > test: Avoid failing skipped tests > test: dm: Show failing driver name > test: Check help output > test: Create a common function to get the config > test: Introduce the concept of a role > test: Move the receive code into a function > test: Separate out the exception handling > test: Detect dead connections > test: Tidy up remaining exceptions > test: Introduce lab mode > test: Improve handling of sending commands > test: Fix mulptiplex_log typo > test: Avoid double echo when starting up > test: Try to shut down the lab console gracefully > test: Add a section for closing the connection > CI: Allow running tests on sjg lab > > .gitlab-ci.yml | 151 +++++++++++++++++ > arch/arm/dts/sunxi-u-boot.dtsi | 1 + > arch/arm/mach-omap2/am33xx/board.c | 18 +- > configs/chromebook_link64_defconfig | 1 + > configs/chromebook_link_defconfig | 1 + > configs/chromebook_samus_defconfig | 1 + > configs/chromebook_samus_tpl_defconfig | 1 + > configs/nyan-big_defconfig | 4 +- > configs/snow_defconfig | 1 + > drivers/core/lists.c | 16 ++ > drivers/pinctrl/meson/pinctrl-meson-axg-pmx.c | 2 +- > drivers/pinctrl/meson/pinctrl-meson-axg.c | 4 +- > drivers/pinctrl/meson/pinctrl-meson-axg.h | 2 +- > drivers/pinctrl/meson/pinctrl-meson-g12a.c | 4 +- > lib/initcall.c | 6 +- > test/py/conftest.py | 86 ++++++++-- > test/py/tests/test_dm.py | 5 +- > test/py/tests/test_help.py | 6 +- > test/py/tests/test_log.py | 11 +- > test/py/tests/test_trace.py | 6 +- > test/py/tests/test_ut.py | 1 + > test/py/u_boot_console_base.py | 154 ++++++++++++------ > test/py/u_boot_console_exec_attach.py | 31 +++- > test/py/u_boot_console_sandbox.py | 2 +- > test/py/u_boot_spawn.py | 123 ++++++++++++-- > test/test-main.c | 16 +- > tools/binman/entries.rst | 115 +++++++++---- > tools/binman/entry.py | 2 +- > tools/binman/etype/efi_capsule.py | 40 ++--- > tools/binman/etype/efi_empty_capsule.py | 22 +-- > tools/binman/etype/ti_secure.py | 45 ++--- > tools/buildman/builder.py | 18 +- > tools/buildman/builderthread.py | 44 +++-- > tools/buildman/buildman.rst | 8 +- > tools/buildman/cmdline.py | 6 +- > tools/buildman/control.py | 141 +++++++++++++++- > tools/buildman/pyproject.toml | 6 +- > tools/buildman/test.py | 121 ++++++++++++++ > tools/u_boot_pylib/terminal.py | 7 +- > 39 files changed, 1003 insertions(+), 226 deletions(-) > > -- > 2.34.1 > Here is an example run (scroll to the right): https://source.denx.de/u-boot/custodians/u-boot-dm/-/pipelines/21081 Regards, Simon
Hi Simon On Tue, 11 Jun 2024 at 23:02, Simon Glass <sjg@chromium.org> wrote: > > Labgrid provides access to a hardware lab in an automated way. It is > possible to boot U-Boot on boards in the lab without physically touching > them. It relies on relays, USB UARTs and SD muxes, among other things. > > By way of background, about 4 years ago I wrong a thing called Labman[1] > which allowed my lab of about 30 devices to be operated remotely, using > tbot for the console and build integration. While it worked OK and I > used it for many bisects, I didn't take it any further. > > It turns out that there was already an existing program, called Labgrid, > which I did not know about at time (thank you Tom for telling me). It is > more rounded than Labman and has a number of advantages: > > - does not need udev rules, mostly > - has several existing users who rely on it > - supports multiple machines exporting their devices > > It lacks a 'lab check' feature and a few other things, but these can be > remedied. > > On and off over the past several weeks I have been experimenting with > Labgrid. I have managed to create an initial U-Boot integration (this > series) by adding various features to Labgrid[2] and the U-Boot test > hooks. > > I hope that this might inspire others to set up boards and run tests > automatically, rather than relying on infrequent, manual test. Perhaps > it may even be possible to have a number of labs available. > > Included in the integration are a number of simple scripts which make it > easy to connect to boards and run tests: > > ub-int <target> > Build and boot on a target, starting an interactive session > > ub-cli <target> > Build and boot on a target, ensure U-Boot starts and provide an interactive > session from there > > ub-smoke <target> > Smoke test U-Boot to check that it boots to a prompt on a target > > ub-bisect > Bisect a git tree to locate a failure on a particular target > > ub-pyt <target> <testspec> > Run U-Boot pytests on a target > > Some of these help to provide the same tbot[4] workflow which I have > relied on for several years, albeit much simpler versions. > > The goal here is to create some sort of script which can collect > patches from the mailing list, apply them and test them on a selection > of boards. I suspect that script already exists, so please let me know > what you suggest. > > I hope you find this interesting and take a look! Thanks this is interesting! I only got cc'ed on the cover letter and I'll slowly have a look on the rest. A naive question -- I saw you did integrate this on gitlab with your internal lab. How secure is this? Could we schedule weekly builds that run on various remote labs and get results on actual hardware? Or do we have to rely on users for that ? Thanks /Ilias > > [1] https://github.com/sjg20/u-boot/tree/lab6a > [2] https://github.com/labgrid-project/labgrid/pull/1411 > [3] https://github.com/sjg20/uboot-test-hooks/tree/labgrid > [4] https://tbot.tools/index.html > > > Simon Glass (42): > trace: Update test to tolerate different trace-cmd version > binman: efi: Correct entry docs > binman: Regenerate nxp docs > binman: ti: Regenerate entry docs > binman: Update the entrydocs header > buildman: Make mrproper an argument to _reconfigure() > buildman: Make mrproper an argument to _config_and_build() > buildman: Make mrproper an argument to run_commit() > buildman: Avoid rebuilding when --mrproper is used > buildman: Add a flag to force mrproper on failure > buildman: Retry the build for current source > buildman: Add a way to limit the number of buildmans > dm: core: Enhance comments on bind_drivers_pass() > initcall: Correct use of relocation offset > am33xx: Provide a function to set up the debug UART > sunxi: Mark scp as optional > google: Disable TPMv2 on most Chromebooks > meson: Correct driver declaration for meson_axg_gpio > test: Allow signaling that U-Boot is ready > test: Make bootstd init run only on sandbox > test: Use a constant for the test timeout > test: Pass stderr to stdout > test: Release board after tests complete > log: Allow tests to pass with CONFIG_LOGF_FUNC_PAD set > test: Allow connecting to a running board > test: Decode exceptions only with sandbox > test: Avoid failing skipped tests > test: dm: Show failing driver name > test: Check help output > test: Create a common function to get the config > test: Introduce the concept of a role > test: Move the receive code into a function > test: Separate out the exception handling > test: Detect dead connections > test: Tidy up remaining exceptions > test: Introduce lab mode > test: Improve handling of sending commands > test: Fix mulptiplex_log typo > test: Avoid double echo when starting up > test: Try to shut down the lab console gracefully > test: Add a section for closing the connection > CI: Allow running tests on sjg lab > > .gitlab-ci.yml | 151 +++++++++++++++++ > arch/arm/dts/sunxi-u-boot.dtsi | 1 + > arch/arm/mach-omap2/am33xx/board.c | 18 +- > configs/chromebook_link64_defconfig | 1 + > configs/chromebook_link_defconfig | 1 + > configs/chromebook_samus_defconfig | 1 + > configs/chromebook_samus_tpl_defconfig | 1 + > configs/nyan-big_defconfig | 4 +- > configs/snow_defconfig | 1 + > drivers/core/lists.c | 16 ++ > drivers/pinctrl/meson/pinctrl-meson-axg-pmx.c | 2 +- > drivers/pinctrl/meson/pinctrl-meson-axg.c | 4 +- > drivers/pinctrl/meson/pinctrl-meson-axg.h | 2 +- > drivers/pinctrl/meson/pinctrl-meson-g12a.c | 4 +- > lib/initcall.c | 6 +- > test/py/conftest.py | 86 ++++++++-- > test/py/tests/test_dm.py | 5 +- > test/py/tests/test_help.py | 6 +- > test/py/tests/test_log.py | 11 +- > test/py/tests/test_trace.py | 6 +- > test/py/tests/test_ut.py | 1 + > test/py/u_boot_console_base.py | 154 ++++++++++++------ > test/py/u_boot_console_exec_attach.py | 31 +++- > test/py/u_boot_console_sandbox.py | 2 +- > test/py/u_boot_spawn.py | 123 ++++++++++++-- > test/test-main.c | 16 +- > tools/binman/entries.rst | 115 +++++++++---- > tools/binman/entry.py | 2 +- > tools/binman/etype/efi_capsule.py | 40 ++--- > tools/binman/etype/efi_empty_capsule.py | 22 +-- > tools/binman/etype/ti_secure.py | 45 ++--- > tools/buildman/builder.py | 18 +- > tools/buildman/builderthread.py | 44 +++-- > tools/buildman/buildman.rst | 8 +- > tools/buildman/cmdline.py | 6 +- > tools/buildman/control.py | 141 +++++++++++++++- > tools/buildman/pyproject.toml | 6 +- > tools/buildman/test.py | 121 ++++++++++++++ > tools/u_boot_pylib/terminal.py | 7 +- > 39 files changed, 1003 insertions(+), 226 deletions(-) > > -- > 2.34.1 >
On Wed, Jun 12, 2024 at 09:08:32AM +0300, Ilias Apalodimas wrote: > Hi Simon > > On Tue, 11 Jun 2024 at 23:02, Simon Glass <sjg@chromium.org> wrote: > > > > Labgrid provides access to a hardware lab in an automated way. It is > > possible to boot U-Boot on boards in the lab without physically touching > > them. It relies on relays, USB UARTs and SD muxes, among other things. > > > > By way of background, about 4 years ago I wrong a thing called Labman[1] > > which allowed my lab of about 30 devices to be operated remotely, using > > tbot for the console and build integration. While it worked OK and I > > used it for many bisects, I didn't take it any further. > > > > It turns out that there was already an existing program, called Labgrid, > > which I did not know about at time (thank you Tom for telling me). It is > > more rounded than Labman and has a number of advantages: > > > > - does not need udev rules, mostly > > - has several existing users who rely on it > > - supports multiple machines exporting their devices > > > > It lacks a 'lab check' feature and a few other things, but these can be > > remedied. > > > > On and off over the past several weeks I have been experimenting with > > Labgrid. I have managed to create an initial U-Boot integration (this > > series) by adding various features to Labgrid[2] and the U-Boot test > > hooks. > > > > I hope that this might inspire others to set up boards and run tests > > automatically, rather than relying on infrequent, manual test. Perhaps > > it may even be possible to have a number of labs available. > > > > Included in the integration are a number of simple scripts which make it > > easy to connect to boards and run tests: > > > > ub-int <target> > > Build and boot on a target, starting an interactive session > > > > ub-cli <target> > > Build and boot on a target, ensure U-Boot starts and provide an interactive > > session from there > > > > ub-smoke <target> > > Smoke test U-Boot to check that it boots to a prompt on a target > > > > ub-bisect > > Bisect a git tree to locate a failure on a particular target > > > > ub-pyt <target> <testspec> > > Run U-Boot pytests on a target > > > > Some of these help to provide the same tbot[4] workflow which I have > > relied on for several years, albeit much simpler versions. > > > > The goal here is to create some sort of script which can collect > > patches from the mailing list, apply them and test them on a selection > > of boards. I suspect that script already exists, so please let me know > > what you suggest. > > > > I hope you find this interesting and take a look! > > Thanks this is interesting! > I only got cc'ed on the cover letter and I'll slowly have a look on > the rest. A naive question -- I saw you did integrate this on gitlab > with your internal lab. How secure is this? Could we schedule weekly > builds that run on various remote labs and get results on actual > hardware? Or do we have to rely on users for that ? That's where this certainly gets a little tricky. Ideally, and part of why I've been hoping to get Labgrid going with our pytest suite, since it's good enough for kernelci (and people to be reasonably confident about allowing not-exactly-random-user-access), it should be good enough for us too. So my long term hope is that we can: - Have Labgrid based labs + something know to monitor trees+branches at X location (like I know today some people poll and test my WIP branches). - Have it be reasonably clear that if you maintain a lab for kernelci, you can add testing U-Boot in too. In the medium term, I am hopeful we can at least expose this to all custodian trees on source.denx.de. And I'm saying "we" here as I have two much smaller labs also going, one with Labgrid, and both a much smaller set of targets currently.
Hi Tom, On Wed, 12 Jun 2024 at 10:03, Tom Rini <trini@konsulko.com> wrote: > > On Wed, Jun 12, 2024 at 09:08:32AM +0300, Ilias Apalodimas wrote: > > Hi Simon > > > > On Tue, 11 Jun 2024 at 23:02, Simon Glass <sjg@chromium.org> wrote: > > > > > > Labgrid provides access to a hardware lab in an automated way. It is > > > possible to boot U-Boot on boards in the lab without physically touching > > > them. It relies on relays, USB UARTs and SD muxes, among other things. > > > > > > By way of background, about 4 years ago I wrong a thing called Labman[1] > > > which allowed my lab of about 30 devices to be operated remotely, using > > > tbot for the console and build integration. While it worked OK and I > > > used it for many bisects, I didn't take it any further. > > > > > > It turns out that there was already an existing program, called Labgrid, > > > which I did not know about at time (thank you Tom for telling me). It is > > > more rounded than Labman and has a number of advantages: > > > > > > - does not need udev rules, mostly > > > - has several existing users who rely on it > > > - supports multiple machines exporting their devices > > > > > > It lacks a 'lab check' feature and a few other things, but these can be > > > remedied. > > > > > > On and off over the past several weeks I have been experimenting with > > > Labgrid. I have managed to create an initial U-Boot integration (this > > > series) by adding various features to Labgrid[2] and the U-Boot test > > > hooks. > > > > > > I hope that this might inspire others to set up boards and run tests > > > automatically, rather than relying on infrequent, manual test. Perhaps > > > it may even be possible to have a number of labs available. > > > > > > Included in the integration are a number of simple scripts which make it > > > easy to connect to boards and run tests: > > > > > > ub-int <target> > > > Build and boot on a target, starting an interactive session > > > > > > ub-cli <target> > > > Build and boot on a target, ensure U-Boot starts and provide an interactive > > > session from there > > > > > > ub-smoke <target> > > > Smoke test U-Boot to check that it boots to a prompt on a target > > > > > > ub-bisect > > > Bisect a git tree to locate a failure on a particular target > > > > > > ub-pyt <target> <testspec> > > > Run U-Boot pytests on a target > > > > > > Some of these help to provide the same tbot[4] workflow which I have > > > relied on for several years, albeit much simpler versions. > > > > > > The goal here is to create some sort of script which can collect > > > patches from the mailing list, apply them and test them on a selection > > > of boards. I suspect that script already exists, so please let me know > > > what you suggest. > > > > > > I hope you find this interesting and take a look! > > > > Thanks this is interesting! > > I only got cc'ed on the cover letter and I'll slowly have a look on > > the rest. A naive question -- I saw you did integrate this on gitlab > > with your internal lab. How secure is this? Could we schedule weekly > > builds that run on various remote labs and get results on actual > > hardware? Or do we have to rely on users for that ? > > That's where this certainly gets a little tricky. Ideally, and part of > why I've been hoping to get Labgrid going with our pytest suite, since > it's good enough for kernelci (and people to be reasonably confident > about allowing not-exactly-random-user-access), it should be good enough > for us too. So my long term hope is that we can: > - Have Labgrid based labs + something know to monitor trees+branches at > X location (like I know today some people poll and test my WIP > branches). > - Have it be reasonably clear that if you maintain a lab for kernelci, > you can add testing U-Boot in too. > > In the medium term, I am hopeful we can at least expose this to all > custodian trees on source.denx.de. And I'm saying "we" here as I have > two much smaller labs also going, one with Labgrid, and both a much > smaller set of targets currently. That sounds good to me. Regards, Simon
Hi Ilias, On Wed, 12 Jun 2024 at 00:09, Ilias Apalodimas <ilias.apalodimas@linaro.org> wrote: > > Hi Simon > > On Tue, 11 Jun 2024 at 23:02, Simon Glass <sjg@chromium.org> wrote: > > > > Labgrid provides access to a hardware lab in an automated way. It is > > possible to boot U-Boot on boards in the lab without physically touching > > them. It relies on relays, USB UARTs and SD muxes, among other things. > > > > By way of background, about 4 years ago I wrong a thing called Labman[1] > > which allowed my lab of about 30 devices to be operated remotely, using > > tbot for the console and build integration. While it worked OK and I > > used it for many bisects, I didn't take it any further. > > > > It turns out that there was already an existing program, called Labgrid, > > which I did not know about at time (thank you Tom for telling me). It is > > more rounded than Labman and has a number of advantages: > > > > - does not need udev rules, mostly > > - has several existing users who rely on it > > - supports multiple machines exporting their devices > > > > It lacks a 'lab check' feature and a few other things, but these can be > > remedied. > > > > On and off over the past several weeks I have been experimenting with > > Labgrid. I have managed to create an initial U-Boot integration (this > > series) by adding various features to Labgrid[2] and the U-Boot test > > hooks. > > > > I hope that this might inspire others to set up boards and run tests > > automatically, rather than relying on infrequent, manual test. Perhaps > > it may even be possible to have a number of labs available. > > > > Included in the integration are a number of simple scripts which make it > > easy to connect to boards and run tests: > > > > ub-int <target> > > Build and boot on a target, starting an interactive session > > > > ub-cli <target> > > Build and boot on a target, ensure U-Boot starts and provide an interactive > > session from there > > > > ub-smoke <target> > > Smoke test U-Boot to check that it boots to a prompt on a target > > > > ub-bisect > > Bisect a git tree to locate a failure on a particular target > > > > ub-pyt <target> <testspec> > > Run U-Boot pytests on a target > > > > Some of these help to provide the same tbot[4] workflow which I have > > relied on for several years, albeit much simpler versions. > > > > The goal here is to create some sort of script which can collect > > patches from the mailing list, apply them and test them on a selection > > of boards. I suspect that script already exists, so please let me know > > what you suggest. > > > > I hope you find this interesting and take a look! > > Thanks this is interesting! Thanks! > I only got cc'ed on the cover letter and I'll slowly have a look on > the rest. A naive question -- I saw you did integrate this on gitlab > with your internal lab. How secure is this? Could we schedule weekly > builds that run on various remote labs and get results on actual > hardware? Or do we have to rely on users for that ? I believe this could be done. My understanding is that any custodian can push to a tree which runs on all available runners. If you have any ideas on the script I mentioned, let me know! Regards, Simon > > Thanks > /Ilias > > > > > [1] https://github.com/sjg20/u-boot/tree/lab6a > > [2] https://github.com/labgrid-project/labgrid/pull/1411 > > [3] https://github.com/sjg20/uboot-test-hooks/tree/labgrid > > [4] https://tbot.tools/index.html > > > > > > Simon Glass (42): > > trace: Update test to tolerate different trace-cmd version > > binman: efi: Correct entry docs > > binman: Regenerate nxp docs > > binman: ti: Regenerate entry docs > > binman: Update the entrydocs header > > buildman: Make mrproper an argument to _reconfigure() > > buildman: Make mrproper an argument to _config_and_build() > > buildman: Make mrproper an argument to run_commit() > > buildman: Avoid rebuilding when --mrproper is used > > buildman: Add a flag to force mrproper on failure > > buildman: Retry the build for current source > > buildman: Add a way to limit the number of buildmans > > dm: core: Enhance comments on bind_drivers_pass() > > initcall: Correct use of relocation offset > > am33xx: Provide a function to set up the debug UART > > sunxi: Mark scp as optional > > google: Disable TPMv2 on most Chromebooks > > meson: Correct driver declaration for meson_axg_gpio > > test: Allow signaling that U-Boot is ready > > test: Make bootstd init run only on sandbox > > test: Use a constant for the test timeout > > test: Pass stderr to stdout > > test: Release board after tests complete > > log: Allow tests to pass with CONFIG_LOGF_FUNC_PAD set > > test: Allow connecting to a running board > > test: Decode exceptions only with sandbox > > test: Avoid failing skipped tests > > test: dm: Show failing driver name > > test: Check help output > > test: Create a common function to get the config > > test: Introduce the concept of a role > > test: Move the receive code into a function > > test: Separate out the exception handling > > test: Detect dead connections > > test: Tidy up remaining exceptions > > test: Introduce lab mode > > test: Improve handling of sending commands > > test: Fix mulptiplex_log typo > > test: Avoid double echo when starting up > > test: Try to shut down the lab console gracefully > > test: Add a section for closing the connection > > CI: Allow running tests on sjg lab > > > > .gitlab-ci.yml | 151 +++++++++++++++++ > > arch/arm/dts/sunxi-u-boot.dtsi | 1 + > > arch/arm/mach-omap2/am33xx/board.c | 18 +- > > configs/chromebook_link64_defconfig | 1 + > > configs/chromebook_link_defconfig | 1 + > > configs/chromebook_samus_defconfig | 1 + > > configs/chromebook_samus_tpl_defconfig | 1 + > > configs/nyan-big_defconfig | 4 +- > > configs/snow_defconfig | 1 + > > drivers/core/lists.c | 16 ++ > > drivers/pinctrl/meson/pinctrl-meson-axg-pmx.c | 2 +- > > drivers/pinctrl/meson/pinctrl-meson-axg.c | 4 +- > > drivers/pinctrl/meson/pinctrl-meson-axg.h | 2 +- > > drivers/pinctrl/meson/pinctrl-meson-g12a.c | 4 +- > > lib/initcall.c | 6 +- > > test/py/conftest.py | 86 ++++++++-- > > test/py/tests/test_dm.py | 5 +- > > test/py/tests/test_help.py | 6 +- > > test/py/tests/test_log.py | 11 +- > > test/py/tests/test_trace.py | 6 +- > > test/py/tests/test_ut.py | 1 + > > test/py/u_boot_console_base.py | 154 ++++++++++++------ > > test/py/u_boot_console_exec_attach.py | 31 +++- > > test/py/u_boot_console_sandbox.py | 2 +- > > test/py/u_boot_spawn.py | 123 ++++++++++++-- > > test/test-main.c | 16 +- > > tools/binman/entries.rst | 115 +++++++++---- > > tools/binman/entry.py | 2 +- > > tools/binman/etype/efi_capsule.py | 40 ++--- > > tools/binman/etype/efi_empty_capsule.py | 22 +-- > > tools/binman/etype/ti_secure.py | 45 ++--- > > tools/buildman/builder.py | 18 +- > > tools/buildman/builderthread.py | 44 +++-- > > tools/buildman/buildman.rst | 8 +- > > tools/buildman/cmdline.py | 6 +- > > tools/buildman/control.py | 141 +++++++++++++++- > > tools/buildman/pyproject.toml | 6 +- > > tools/buildman/test.py | 121 ++++++++++++++ > > tools/u_boot_pylib/terminal.py | 7 +- > > 39 files changed, 1003 insertions(+), 226 deletions(-) > > > > -- > > 2.34.1 > >
Hi, On Tue, 11 Jun 2024 14:01:14 -0600, Simon Glass wrote: > Labgrid provides access to a hardware lab in an automated way. It is > possible to boot U-Boot on boards in the lab without physically touching > them. It relies on relays, USB UARTs and SD muxes, among other things. > > By way of background, about 4 years ago I wrong a thing called Labman[1] > which allowed my lab of about 30 devices to be operated remotely, using > tbot for the console and build integration. While it worked OK and I > used it for many bisects, I didn't take it any further. > > [...] Thanks, Applied to https://source.denx.de/u-boot/custodians/u-boot-amlogic (u-boot-amlogic-next) [18/42] meson: Correct driver declaration for meson_axg_gpio https://source.denx.de/u-boot/custodians/u-boot-amlogic/-/commit/96e1a156e90a0674b21c56e7121b17a9211176ec
Labgrid provides access to a hardware lab in an automated way. It is possible to boot U-Boot on boards in the lab without physically touching them. It relies on relays, USB UARTs and SD muxes, among other things. By way of background, about 4 years ago I wrong a thing called Labman[1] which allowed my lab of about 30 devices to be operated remotely, using tbot for the console and build integration. While it worked OK and I used it for many bisects, I didn't take it any further. It turns out that there was already an existing program, called Labgrid, which I did not know about at time (thank you Tom for telling me). It is more rounded than Labman and has a number of advantages: - does not need udev rules, mostly - has several existing users who rely on it - supports multiple machines exporting their devices It lacks a 'lab check' feature and a few other things, but these can be remedied. On and off over the past several weeks I have been experimenting with Labgrid. I have managed to create an initial U-Boot integration (this series) by adding various features to Labgrid[2] and the U-Boot test hooks. I hope that this might inspire others to set up boards and run tests automatically, rather than relying on infrequent, manual test. Perhaps it may even be possible to have a number of labs available. Included in the integration are a number of simple scripts which make it easy to connect to boards and run tests: ub-int <target> Build and boot on a target, starting an interactive session ub-cli <target> Build and boot on a target, ensure U-Boot starts and provide an interactive session from there ub-smoke <target> Smoke test U-Boot to check that it boots to a prompt on a target ub-bisect Bisect a git tree to locate a failure on a particular target ub-pyt <target> <testspec> Run U-Boot pytests on a target Some of these help to provide the same tbot[4] workflow which I have relied on for several years, albeit much simpler versions. The goal here is to create some sort of script which can collect patches from the mailing list, apply them and test them on a selection of boards. I suspect that script already exists, so please let me know what you suggest. I hope you find this interesting and take a look! [1] https://github.com/sjg20/u-boot/tree/lab6a [2] https://github.com/labgrid-project/labgrid/pull/1411 [3] https://github.com/sjg20/uboot-test-hooks/tree/labgrid [4] https://tbot.tools/index.html Simon Glass (42): trace: Update test to tolerate different trace-cmd version binman: efi: Correct entry docs binman: Regenerate nxp docs binman: ti: Regenerate entry docs binman: Update the entrydocs header buildman: Make mrproper an argument to _reconfigure() buildman: Make mrproper an argument to _config_and_build() buildman: Make mrproper an argument to run_commit() buildman: Avoid rebuilding when --mrproper is used buildman: Add a flag to force mrproper on failure buildman: Retry the build for current source buildman: Add a way to limit the number of buildmans dm: core: Enhance comments on bind_drivers_pass() initcall: Correct use of relocation offset am33xx: Provide a function to set up the debug UART sunxi: Mark scp as optional google: Disable TPMv2 on most Chromebooks meson: Correct driver declaration for meson_axg_gpio test: Allow signaling that U-Boot is ready test: Make bootstd init run only on sandbox test: Use a constant for the test timeout test: Pass stderr to stdout test: Release board after tests complete log: Allow tests to pass with CONFIG_LOGF_FUNC_PAD set test: Allow connecting to a running board test: Decode exceptions only with sandbox test: Avoid failing skipped tests test: dm: Show failing driver name test: Check help output test: Create a common function to get the config test: Introduce the concept of a role test: Move the receive code into a function test: Separate out the exception handling test: Detect dead connections test: Tidy up remaining exceptions test: Introduce lab mode test: Improve handling of sending commands test: Fix mulptiplex_log typo test: Avoid double echo when starting up test: Try to shut down the lab console gracefully test: Add a section for closing the connection CI: Allow running tests on sjg lab .gitlab-ci.yml | 151 +++++++++++++++++ arch/arm/dts/sunxi-u-boot.dtsi | 1 + arch/arm/mach-omap2/am33xx/board.c | 18 +- configs/chromebook_link64_defconfig | 1 + configs/chromebook_link_defconfig | 1 + configs/chromebook_samus_defconfig | 1 + configs/chromebook_samus_tpl_defconfig | 1 + configs/nyan-big_defconfig | 4 +- configs/snow_defconfig | 1 + drivers/core/lists.c | 16 ++ drivers/pinctrl/meson/pinctrl-meson-axg-pmx.c | 2 +- drivers/pinctrl/meson/pinctrl-meson-axg.c | 4 +- drivers/pinctrl/meson/pinctrl-meson-axg.h | 2 +- drivers/pinctrl/meson/pinctrl-meson-g12a.c | 4 +- lib/initcall.c | 6 +- test/py/conftest.py | 86 ++++++++-- test/py/tests/test_dm.py | 5 +- test/py/tests/test_help.py | 6 +- test/py/tests/test_log.py | 11 +- test/py/tests/test_trace.py | 6 +- test/py/tests/test_ut.py | 1 + test/py/u_boot_console_base.py | 154 ++++++++++++------ test/py/u_boot_console_exec_attach.py | 31 +++- test/py/u_boot_console_sandbox.py | 2 +- test/py/u_boot_spawn.py | 123 ++++++++++++-- test/test-main.c | 16 +- tools/binman/entries.rst | 115 +++++++++---- tools/binman/entry.py | 2 +- tools/binman/etype/efi_capsule.py | 40 ++--- tools/binman/etype/efi_empty_capsule.py | 22 +-- tools/binman/etype/ti_secure.py | 45 ++--- tools/buildman/builder.py | 18 +- tools/buildman/builderthread.py | 44 +++-- tools/buildman/buildman.rst | 8 +- tools/buildman/cmdline.py | 6 +- tools/buildman/control.py | 141 +++++++++++++++- tools/buildman/pyproject.toml | 6 +- tools/buildman/test.py | 121 ++++++++++++++ tools/u_boot_pylib/terminal.py | 7 +- 39 files changed, 1003 insertions(+), 226 deletions(-)