| Message ID | 20240611-docker-image-v1-14-51472eb70357@flygoat.com |
|---|---|
| State | Changes Requested |
| Delegated to: | Tom Rini |
| Headers | show |
| Series | New CI image and fixes | expand |
On Tue, Jun 11, 2024 at 10:04:13PM +0100, Jiaxun Yang wrote: > Current build_world task runs for too long on public gitlab > runner. > > Split the job as what we've done to azure pipeline. > > Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> > --- > .gitlab-ci.yml | 103 +++++++++++++++++++++++++++++++++------------------------ > 1 file changed, 59 insertions(+), 44 deletions(-) I don't like this. The list in Azure is because of the time limits there and in turn we: a) Have to tweak it periodically to keep things from running too long b) Have to tweak it to ensure that we don't miss some new SoC/etc
在2024年6月12日六月 下午5:01，Tom Rini写道： > On Tue, Jun 11, 2024 at 10:04:13PM +0100, Jiaxun Yang wrote: > >> Current build_world task runs for too long on public gitlab >> runner. >> >> Split the job as what we've done to azure pipeline. >> >> Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> >> --- >> .gitlab-ci.yml | 103 +++++++++++++++++++++++++++++++++------------------------ >> 1 file changed, 59 insertions(+), 44 deletions(-) > > I don't like this. The list in Azure is because of the time limits there > and in turn we: > a) Have to tweak it periodically to keep things from running too long > b) Have to tweak it to ensure that we don't miss some new SoC/etc Then it will render running CI test on gitlab.com impossible again :-( Thanks > > -- > Tom > > 附件: > * signature.asc
On Wed, Jun 12, 2024 at 05:14:46PM +0100, Jiaxun Yang wrote: > > > 在2024年6月12日六月 下午5:01，Tom Rini写道： > > On Tue, Jun 11, 2024 at 10:04:13PM +0100, Jiaxun Yang wrote: > > > >> Current build_world task runs for too long on public gitlab > >> runner. > >> > >> Split the job as what we've done to azure pipeline. > >> > >> Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> > >> --- > >> .gitlab-ci.yml | 103 +++++++++++++++++++++++++++++++++------------------------ > >> 1 file changed, 59 insertions(+), 44 deletions(-) > > > > I don't like this. The list in Azure is because of the time limits there > > and in turn we: > > a) Have to tweak it periodically to keep things from running too long > > b) Have to tweak it to ensure that we don't miss some new SoC/etc > > Then it will render running CI test on gitlab.com impossible again :-( Yeah, it's not something I'm the happiest with. Looking around a bit, I see a blog post that talks about dealing with dynamic variables, in Azure. So we could, I think, figure out some logic to have each build stage say what platforms it covers. And then have a final step that compares all of the platforms built vs the global list (just tools/buildman/buildman --dry-run -v) to make sure nothing was missed. With something like that, and assuming GitLab can do it too (it probably can), I'm OK with having the world build be broken down to 10 groups (maximum number of parallel jobs in Azure CI for free) since we'll know if we miss something too. So lets set this patch (and the doc update) aside for now, unless you want to look at the above. I'll look at the above soon.
Hi Tom, On Wed, 12 Jun 2024 at 11:07, Tom Rini <trini@konsulko.com> wrote: > > On Wed, Jun 12, 2024 at 05:14:46PM +0100, Jiaxun Yang wrote: > > > > > > 在2024年6月12日六月 下午5:01，Tom Rini写道： > > > On Tue, Jun 11, 2024 at 10:04:13PM +0100, Jiaxun Yang wrote: > > > > > >> Current build_world task runs for too long on public gitlab > > >> runner. > > >> > > >> Split the job as what we've done to azure pipeline. > > >> > > >> Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> > > >> --- > > >> .gitlab-ci.yml | 103 +++++++++++++++++++++++++++++++++------------------------ > > >> 1 file changed, 59 insertions(+), 44 deletions(-) > > > > > > I don't like this. The list in Azure is because of the time limits there > > > and in turn we: > > > a) Have to tweak it periodically to keep things from running too long > > > b) Have to tweak it to ensure that we don't miss some new SoC/etc > > > > Then it will render running CI test on gitlab.com impossible again :-( > > Yeah, it's not something I'm the happiest with. Looking around a bit, I > see a blog post that talks about dealing with dynamic variables, in > Azure. So we could, I think, figure out some logic to have each build > stage say what platforms it covers. And then have a final step that > compares all of the platforms built vs the global list (just > tools/buildman/buildman --dry-run -v) to make sure nothing was missed. > With something like that, and assuming GitLab can do it too (it probably > can), I'm OK with having the world build be broken down to 10 groups > (maximum number of parallel jobs in Azure CI for free) since we'll know > if we miss something too. > > So lets set this patch (and the doc update) aside for now, unless you > want to look at the above. I'll look at the above soon. Could we hook up one of our machines as a public runner somehow? Regards, Simon
On Wed, Jun 12, 2024 at 02:24:24PM -0600, Simon Glass wrote: > Hi Tom, > > On Wed, 12 Jun 2024 at 11:07, Tom Rini <trini@konsulko.com> wrote: > > > > On Wed, Jun 12, 2024 at 05:14:46PM +0100, Jiaxun Yang wrote: > > > > > > > > > 在2024年6月12日六月 下午5:01，Tom Rini写道： > > > > On Tue, Jun 11, 2024 at 10:04:13PM +0100, Jiaxun Yang wrote: > > > > > > > >> Current build_world task runs for too long on public gitlab > > > >> runner. > > > >> > > > >> Split the job as what we've done to azure pipeline. > > > >> > > > >> Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> > > > >> --- > > > >> .gitlab-ci.yml | 103 +++++++++++++++++++++++++++++++++------------------------ > > > >> 1 file changed, 59 insertions(+), 44 deletions(-) > > > > > > > > I don't like this. The list in Azure is because of the time limits there > > > > and in turn we: > > > > a) Have to tweak it periodically to keep things from running too long > > > > b) Have to tweak it to ensure that we don't miss some new SoC/etc > > > > > > Then it will render running CI test on gitlab.com impossible again :-( > > > > Yeah, it's not something I'm the happiest with. Looking around a bit, I > > see a blog post that talks about dealing with dynamic variables, in > > Azure. So we could, I think, figure out some logic to have each build > > stage say what platforms it covers. And then have a final step that > > compares all of the platforms built vs the global list (just > > tools/buildman/buildman --dry-run -v) to make sure nothing was missed. > > With something like that, and assuming GitLab can do it too (it probably > > can), I'm OK with having the world build be broken down to 10 groups > > (maximum number of parallel jobs in Azure CI for free) since we'll know > > if we miss something too. > > > > So lets set this patch (and the doc update) aside for now, unless you > > want to look at the above. I'll look at the above soon. > > Could we hook up one of our machines as a public runner somehow? It's a pool of free runners, ala Azure. I don't think we can, no, and I don't want to make all of our runners available on public CI either, they're a bit over-whelmed as it is I think once custodians start working.
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 4c17abea468a..efb84c3b119f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -91,56 +91,71 @@ stages: - "*.css" expire_in: 1 week -.world_build: +.world_build_template: &world_build_dfn stage: world build rules: - when: always - -build all 32bit ARM platforms: - extends: .world_build - script: - - ret=0; - git config --global --add safe.directory "${CI_PROJECT_DIR}"; - ./tools/buildman/buildman -o /tmp -PEWM arm -x aarch64 || ret=$?; - if [[ $ret -ne 0 ]]; then - ./tools/buildman/buildman -o /tmp -seP; - exit $ret; - fi; - -build all 64bit ARM platforms: - extends: .world_build - script: - - virtualenv -p /usr/bin/python3 /tmp/venv - - . /tmp/venv/bin/activate - - ret=0; - git config --global --add safe.directory "${CI_PROJECT_DIR}"; - ./tools/buildman/buildman -o /tmp -PEWM aarch64 || ret=$?; - if [[ $ret -ne 0 ]]; then - ./tools/buildman/buildman -o /tmp -seP; - exit $ret; - fi; - -build all PowerPC platforms: - extends: .world_build script: - ret=0; git config --global --add safe.directory "${CI_PROJECT_DIR}"; - ./tools/buildman/buildman -o /tmp -P -E -W powerpc || ret=$?; - if [[ $ret -ne 0 ]]; then - ./tools/buildman/buildman -o /tmp -seP; - exit $ret; - fi; - -build all other platforms: - extends: .world_build - script: - - ret=0; - git config --global --add safe.directory "${CI_PROJECT_DIR}"; - ./tools/buildman/buildman -o /tmp -PEWM -x arm,powerpc || ret=$?; - if [[ $ret -ne 0 ]]; then - ./tools/buildman/buildman -o /tmp -seP; - exit $ret; - fi; + if [[ "${BUILDMAN}" != "" ]]; then + ret=0; + tools/buildman/buildman -o /tmp -PEWM ${BUILDMAN} ${OVERRIDE} || ret=$?; + if [[ $ret -ne 0 ]]; then + tools/buildman/buildman -o /tmp -seP ${BUILDMAN}; + exit $ret; + fi; + fi + +am33xx_at91_kirkwood_mvebu_omap: + variables: + BUILDMAN: "am33xx at91_kirkwood mvebu omap -x siemens" + <<: *world_build_dfn + +amlogic_bcm_boundary_engicam_siemens_technexion_oradex: + variables: + BUILDMAN: "amlogic bcm boundary engicam siemens technexion toradex -x mips" + <<: *world_build_dfn + +arm_nxp_minus_imx: + variables: + BUILDMAN: "freescale -x powerpc,m68k,imx,mx" + <<: *world_build_dfn + +imx: + variables: + BUILDMAN: "mx imx -x boundary,engicam,technexion,toradex" + <<: *world_build_dfn + +rk: + variables: + BUILDMAN: "rk" + <<: *world_build_dfn + +sunxi: + variables: + BUILDMAN: "sunxi" + <<: *world_build_dfn + +powerpc: + variables: + BUILDMAN: "powerpc" + <<: *world_build_dfn + +arm_catch_all: + variables: + BUILDMAN: "arm -x aarch64,am33xx,at91,bcm,ls1,kirkwood,mvebu,omap,rk,siemens,mx,sunxi,technexion,toradex" + <<: *world_build_dfn + +aarch64_catch_all: + variables: + BUILDMAN: "aarch64 -x amlogic,bcm,engicam,imx,ls1,ls2,lx216,mvebu,rk,siemens,sunxi,toradex" + <<: *world_build_dfn + +everything_but_arm_and_powerpc: + variables: + BUILDMAN: "-x arm,powerpc" + <<: *world_build_dfn .testsuites: stage: testsuites
Current build_world task runs for too long on public gitlab runner. Split the job as what we've done to azure pipeline. Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> --- .gitlab-ci.yml | 103 +++++++++++++++++++++++++++++++++------------------------ 1 file changed, 59 insertions(+), 44 deletions(-)