| Message ID | 20180813063652.20534-2-joel@jms.id.au |
|---|---|
| State | Accepted |
| Headers | show |
| Series | CI updates for make check | expand |
| Context | Check | Description |
|---|---|---|
| snowpatch_ozlabs/apply_patch | success | master/apply_patch Successfully applied |
| snowpatch_ozlabs/build-multiarch | success | Test build-multiarch on branch master |
On Mon, 2018-08-13 at 16:06 +0930, Joel Stanley wrote: > Update travis config to point to the new location. > > Add a symlink so other CI still works, we will remove this once it > has > been moved over. > > Signed-off-by: Joel Stanley <joel@jms.id.au> Can we also add "make check" to build.sh? For build.sh, can we just assume it's going to be run on x86_64 platform? That should avoid the need for --host=x86_64-linux-gnu. Also, simplifies adding "make check" to build.sh (we don't have to worry about where to run "make check"). > --- > .build.sh | 59 +-------------------------------- > ---- > .travis.yml | 2 +- > .build.sh => utils/build.sh | 0 > 3 files changed, 2 insertions(+), 59 deletions(-) > mode change 100755 => 120000 .build.sh > copy .build.sh => utils/build.sh (100%) > > diff --git a/.build.sh b/.build.sh > deleted file mode 100755 > index 59fc6c3a524b..000000000000 > --- a/.build.sh > +++ /dev/null > @@ -1,58 +0,0 @@ > -#!/bin/bash > - > -set -e > - > -CONTAINER=pdbg-build > - > -Dockerfile=$(cat << EOF > -FROM ubuntu:18.04 > -RUN apt-get update && apt-get install --no-install-recommends -yy \ > - make \ > - gcc-arm-linux-gnueabi \ > - libc-dev-armel-cross \ > - gcc-powerpc64le-linux-gnu \ > - libc-dev-ppc64el-cross \ > - autoconf \ > - automake \ > - libtool \ > - git \ > - device-tree-compiler > -RUN groupadd -g ${GROUPS} ${USER} && useradd -d ${HOME} -m -u ${UID} > -g ${GROUPS} ${USER} > -USER ${USER} > -ENV HOME ${HOME} > -RUN /bin/bash > -EOF > -) > - > -docker pull ubuntu:18.04 > -docker build -t ${CONTAINER} - <<< "${Dockerfile}" > - > -RUN="docker run --rm=true --user=${USER} -w ${PWD} -v > ${HOME}:${HOME} -t ${CONTAINER}" > - > -${RUN} ./bootstrap.sh > - > -# Out-of-tree build, arm > -# TODO: clean up when the build fails > -SRCDIR=$PWD > -TEMPDIR=`mktemp -d ${HOME}/pdbgobjXXXXXX` > -RUN_TMP="docker run --rm=true --user=${USER} -w ${TEMPDIR} -v > ${HOME}:${HOME} -t ${CONTAINER}" > -${RUN_TMP} ${SRCDIR}/configure --host=arm-linux-gnueabi > -${RUN_TMP} make > -rm -rf ${TEMPDIR} > - > -# In-tree build, arm > -${RUN} ./configure --host=arm-linux-gnueabi > -${RUN} make > -${RUN} make clean > - > -# In-tree build, powerpc64le > -${RUN} ./configure --host=powerpc64le-linux-gnu > -${RUN} make > -${RUN} make clean > - > -# In-tree build, amd64 > -# TODO: work out how to install a amd64 compiler if we are building > on a eg. > -# ppc machine in a way that still works when we're on amd64 > -${RUN} ./configure --host=x86-64-linux-gnu > -${RUN} make > -${RUN} make clean > diff --git a/.build.sh b/.build.sh > new file mode 120000 > index 000000000000..d96834636319 > --- /dev/null > +++ b/.build.sh > @@ -0,0 +1 @@ > +utils/build.sh > \ No newline at end of file > diff --git a/.travis.yml b/.travis.yml > index e6efef64657d..9e1a86bc3140 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -2,4 +2,4 @@ services: > - docker > > script: > - - ./.build.sh > + - ./utils/build.sh > diff --git a/.build.sh b/utils/build.sh > similarity index 100% > copy from .build.sh > copy to utils/build.sh > -- > 2.17.1 > Amitay.
On Mon, 13 Aug 2018 at 16:51, Amitay Isaacs <amitay@ozlabs.org> wrote: > > On Mon, 2018-08-13 at 16:06 +0930, Joel Stanley wrote: > > Update travis config to point to the new location. > > > > Add a symlink so other CI still works, we will remove this once it > > has > > been moved over. > > > > Signed-off-by: Joel Stanley <joel@jms.id.au> > > Can we also add "make check" to build.sh? > > For build.sh, can we just assume it's going to be run on x86_64 > platform? That should avoid the need for --host=x86_64-linux-gnu. > Also, simplifies adding "make check" to build.sh (we don't have to > worry about where to run "make check"). We were doing builds on ppc64le hosts. Since adding the arm cross compile I had to disable it, as Ubuntu ppc64le doesn't build the ARM cross compiler. I'd like to get this going again. Make check as it's written only uses the host compiler, so it will work on ppc64le today. The reason we have broken out make check into a different script is so it shows up as a separate check in patchwork's 'checks' section. We will run the build script first, and the check second, so any build errors are not confused with failing tests.
On Mon, 2018-08-13 at 17:13 +0930, Joel Stanley wrote: > On Mon, 13 Aug 2018 at 16:51, Amitay Isaacs <amitay@ozlabs.org> > wrote: > > > > On Mon, 2018-08-13 at 16:06 +0930, Joel Stanley wrote: > > > Update travis config to point to the new location. > > > > > > Add a symlink so other CI still works, we will remove this once > > > it > > > has > > > been moved over. > > > > > > Signed-off-by: Joel Stanley <joel@jms.id.au> > > > > Can we also add "make check" to build.sh? > > > > For build.sh, can we just assume it's going to be run on x86_64 > > platform? That should avoid the need for --host=x86_64-linux-gnu. > > Also, simplifies adding "make check" to build.sh (we don't have to > > worry about where to run "make check"). > > We were doing builds on ppc64le hosts. Since adding the arm cross > compile I had to disable it, as Ubuntu ppc64le doesn't build the ARM > cross compiler. I'd like to get this going again. > > Make check as it's written only uses the host compiler, so it will > work on ppc64le today. > > The reason we have broken out make check into a different script is > so > it shows up as a separate check in patchwork's 'checks' section. We > will run the build script first, and the check second, so any build > errors are not confused with failing tests. Ah. I thought build.sh was for Jenkins/Patchwork and travis.yml was for CI. Does that mean we run CI on pdbg-master (not just patches)? And where does it run? Amitay.
On Mon, 13 Aug 2018 at 17:22, Amitay Isaacs <amitay@ozlabs.org> wrote: > > On Mon, 2018-08-13 at 17:13 +0930, Joel Stanley wrote: > > On Mon, 13 Aug 2018 at 16:51, Amitay Isaacs <amitay@ozlabs.org> > > wrote: > > > > > > On Mon, 2018-08-13 at 16:06 +0930, Joel Stanley wrote: > > > > Update travis config to point to the new location. > > > > > > > > Add a symlink so other CI still works, we will remove this once > > > > it > > > > has > > > > been moved over. > > > > > > > > Signed-off-by: Joel Stanley <joel@jms.id.au> > > > > > > Can we also add "make check" to build.sh? > > > > > > For build.sh, can we just assume it's going to be run on x86_64 > > > platform? That should avoid the need for --host=x86_64-linux-gnu. > > > Also, simplifies adding "make check" to build.sh (we don't have to > > > worry about where to run "make check"). > > > > We were doing builds on ppc64le hosts. Since adding the arm cross > > compile I had to disable it, as Ubuntu ppc64le doesn't build the ARM > > cross compiler. I'd like to get this going again. > > > > Make check as it's written only uses the host compiler, so it will > > work on ppc64le today. > > > > The reason we have broken out make check into a different script is > > so > > it shows up as a separate check in patchwork's 'checks' section. We > > will run the build script first, and the check second, so any build > > errors are not confused with failing tests. > > Ah. I thought build.sh was for Jenkins/Patchwork and travis.yml was > for CI. > > Does that mean we run CI on pdbg-master (not just patches)? And where > does it run? Currently we have Travis enabled, which means whenever a branch is pushed to Github (and the user has Travis enabled), it will run against that branch. This can be useful for testing before submission: https://travis-ci.org/shenki/pdbg/builds/415311411 I got Stewart to turn this on for the main repository too, so builds should start showing up here: https://travis-ci.org/open-power/pdbg We also have snowpatch building patches that are sent to the mailing list, and updating the "tested" status on patchwork. I think will also email you if your patch is broken: https://patchwork.ozlabs.org/patch/956833/ There's a different job that will build master. This one doesn't report anywhere yet: https://openpower.xyz/job/openpower/job/pdbg-master/ I will add some documentation to the repository explaining all of this. The missing piece is to run this on real hardware somewhere. Cheers, Joel
Thanks. > I will add some documentation to the repository explaining all of > this. The missing piece is to run this on real hardware somewhere. Agreed. Now that we have Amitay's test driver and I'm trying to push a release that actually works I am going to add some basic tests for running on actual hardware. The trick will be getting the HW in a "good" state for the test, but even if manual intervention is required prior to running tests it is still a good start. Do we currently run "make check" on any 32-bit (ie. ARM) platforms? Is that hard to do? We have had the odd 32-bit vs. 64-bit bug, although they were mostly picked up at compile time. - Alistair > Cheers, > > Joel >
On Tue, 14 Aug 2018 at 09:28, Alistair Popple <alistair@popple.id.au> wrote: > > Thanks. > > > I will add some documentation to the repository explaining all of > > this. The missing piece is to run this on real hardware somewhere. > > Agreed. Now that we have Amitay's test driver and I'm trying to push a release > that actually works I am going to add some basic tests for running on actual > hardware. The trick will be getting the HW in a "good" state for the test, but > even if manual intervention is required prior to running tests it is still a > good start. Having something in the utils directory that anyone can run on hardware would be useful. From there we can do some best-effort running of that with CI. > Do we currently run "make check" on any 32-bit (ie. ARM) platforms? Is that hard > to do? We have had the odd 32-bit vs. 64-bit bug, although they were mostly > picked up at compile time. We don't yet. I'll do some experiments, I think qemu-user will be our best bet. Easier might be to do a 32-bit x86 test.
diff --git a/.build.sh b/.build.sh deleted file mode 100755 index 59fc6c3a524b..000000000000 --- a/.build.sh +++ /dev/null @@ -1,58 +0,0 @@ -#!/bin/bash - -set -e - -CONTAINER=pdbg-build - -Dockerfile=$(cat << EOF -FROM ubuntu:18.04 -RUN apt-get update && apt-get install --no-install-recommends -yy \ - make \ - gcc-arm-linux-gnueabi \ - libc-dev-armel-cross \ - gcc-powerpc64le-linux-gnu \ - libc-dev-ppc64el-cross \ - autoconf \ - automake \ - libtool \ - git \ - device-tree-compiler -RUN groupadd -g ${GROUPS} ${USER} && useradd -d ${HOME} -m -u ${UID} -g ${GROUPS} ${USER} -USER ${USER} -ENV HOME ${HOME} -RUN /bin/bash -EOF -) - -docker pull ubuntu:18.04 -docker build -t ${CONTAINER} - <<< "${Dockerfile}" - -RUN="docker run --rm=true --user=${USER} -w ${PWD} -v ${HOME}:${HOME} -t ${CONTAINER}" - -${RUN} ./bootstrap.sh - -# Out-of-tree build, arm -# TODO: clean up when the build fails -SRCDIR=$PWD -TEMPDIR=`mktemp -d ${HOME}/pdbgobjXXXXXX` -RUN_TMP="docker run --rm=true --user=${USER} -w ${TEMPDIR} -v ${HOME}:${HOME} -t ${CONTAINER}" -${RUN_TMP} ${SRCDIR}/configure --host=arm-linux-gnueabi -${RUN_TMP} make -rm -rf ${TEMPDIR} - -# In-tree build, arm -${RUN} ./configure --host=arm-linux-gnueabi -${RUN} make -${RUN} make clean - -# In-tree build, powerpc64le -${RUN} ./configure --host=powerpc64le-linux-gnu -${RUN} make -${RUN} make clean - -# In-tree build, amd64 -# TODO: work out how to install a amd64 compiler if we are building on a eg. -# ppc machine in a way that still works when we're on amd64 -${RUN} ./configure --host=x86-64-linux-gnu -${RUN} make -${RUN} make clean diff --git a/.build.sh b/.build.sh new file mode 120000 index 000000000000..d96834636319 --- /dev/null +++ b/.build.sh @@ -0,0 +1 @@ +utils/build.sh \ No newline at end of file diff --git a/.travis.yml b/.travis.yml index e6efef64657d..9e1a86bc3140 100644 --- a/.travis.yml +++ b/.travis.yml @@ -2,4 +2,4 @@ services: - docker script: - - ./.build.sh + - ./utils/build.sh
Update travis config to point to the new location. Add a symlink so other CI still works, we will remove this once it has been moved over. Signed-off-by: Joel Stanley <joel@jms.id.au> --- .build.sh | 59 +------------------------------------ .travis.yml | 2 +- .build.sh => utils/build.sh | 0 3 files changed, 2 insertions(+), 59 deletions(-) mode change 100755 => 120000 .build.sh copy .build.sh => utils/build.sh (100%) diff --git a/.build.sh b/utils/build.sh similarity index 100% copy from .build.sh copy to utils/build.sh