diff mbox series

[2/2] utils: Add script to run make check

Message ID 20180813063652.20534-3-joel@jms.id.au
State Accepted
Headers show
Series CI updates for make check | expand

Checks

Context Check Description
snowpatch_ozlabs/apply_patch success master/apply_patch Successfully applied
snowpatch_ozlabs/build-multiarch success Test build-multiarch on branch master

Commit Message

Joel Stanley Aug. 13, 2018, 6:36 a.m. UTC
This builds the software for the host architecture and runs make check
inside a container.

Added to the travis configuration.

Signed-off-by: Joel Stanley <joel@jms.id.au>
---
 .travis.yml   |  1 +
 utils/test.sh | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100755 utils/test.sh

Comments

Amitay Isaacs Aug. 13, 2018, 7:26 a.m. UTC | #1
On Mon, 2018-08-13 at 16:06 +0930, Joel Stanley wrote:
> This builds the software for the host architecture and runs make
> check
> inside a container.
> 
> Added to the travis configuration.
> 
> Signed-off-by: Joel Stanley <joel@jms.id.au>
> ---
>  .travis.yml   |  1 +
>  utils/test.sh | 33 +++++++++++++++++++++++++++++++++
>  2 files changed, 34 insertions(+)
>  create mode 100755 utils/test.sh
> 
> diff --git a/.travis.yml b/.travis.yml
> index 9e1a86bc3140..596b50ff4e1e 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -3,3 +3,4 @@ services:
>  
>  script:
>      - ./utils/build.sh
> +    - ./utils/test.sh
> diff --git a/utils/test.sh b/utils/test.sh
> new file mode 100755
> index 000000000000..e2bd25893910
> --- /dev/null
> +++ b/utils/test.sh
> @@ -0,0 +1,33 @@
> +#!/bin/bash
> +
> +set -e
> +
> +CONTAINER=pdbg-check
> +
> +Dockerfile=$(cat << EOF
> +FROM ubuntu:18.04
> +RUN apt-get update && apt-get install --no-install-recommends -yy \
> +	make \
> +	gcc \
> +	autoconf \
> +	automake \
> +	libtool \
> +	git \
> +	device-tree-compiler
> +RUN groupadd -g ${GROUPS} ${USER} && useradd -d ${HOME} -m -u ${UID}
> -g ${GROUPS} ${USER}
> +USER ${USER}
> +ENV HOME ${HOME}
> +RUN /bin/bash
> +EOF
> +)
> +
> +docker pull ubuntu:18.04
> +docker build -t ${CONTAINER} - <<< "${Dockerfile}"
> +
> +RUN="docker run --rm=true --user=${USER} -w ${PWD} -v
> ${HOME}:${HOME} -t ${CONTAINER}"
> +
> +${RUN} ./bootstrap.sh
> +${RUN} ./configure
> +${RUN} make clean

Does travis cache the older image?  Or it's always created from
scratch?  If it's created from scratch, then we don't really need "make
clean".  If not, we also should add "git clean -x -d -f" before running
"./bootstrap.sh"

The same logic applies to build.sh.

> +${RUN} make
> +${RUN} make check
> -- 
> 2.17.1
> 

Amitay.
Joel Stanley Aug. 13, 2018, 7:41 a.m. UTC | #2
Hi Amitay,

On Mon, 13 Aug 2018 at 16:56, Amitay Isaacs <amitay@ozlabs.org> wrote:

> > +docker pull ubuntu:18.04
> > +docker build -t ${CONTAINER} - <<< "${Dockerfile}"
> > +
> > +RUN="docker run --rm=true --user=${USER} -w ${PWD} -v
> > ${HOME}:${HOME} -t ${CONTAINER}"
> > +
> > +${RUN} ./bootstrap.sh
> > +${RUN} ./configure
> > +${RUN} make clean
>
> Does travis cache the older image?  Or it's always created from
> scratch?  If it's created from scratch, then we don't really need "make
> clean".  If not, we also should add "git clean -x -d -f" before running
> "./bootstrap.sh"

Travis doesn't appear to cache the containers. We do cache them on
openpower.xyz's Jenkins. When I use them locally, it is cached.

We don't want to do a git clean in the script as that might have
unexpected results for people using it in their local tree.

For our CI (Travis, Jenkins) the workspace will be clean each time, so
the make clean is unnecessary. This these files are not just for
travis, but is intended for anyone wanting to do a build. Hence the
make clean which is useful if the previous build was a cross-compile.
Without the clean a user might get this:

   CCLD     pdbg
/usr/bin/x86_64-linux-gnu-ld: unknown architecture of input file
`fake.dtb.o' is incompatible with i386:x86-64 output
collect2: error: ld returned 1 exit status

Cheers,

Joel

>
> The same logic applies to build.sh.
>
> > +${RUN} make
> > +${RUN} make check
> > --
> > 2.17.1
> >
>
> Amitay.
> --
>
> Everyone is a genius at least once a year. A real genius has his ideas
> closer together.
>
Amitay Isaacs Aug. 13, 2018, 8 a.m. UTC | #3
On Mon, 2018-08-13 at 17:11 +0930, Joel Stanley wrote:
> Hi Amitay,
> 
> On Mon, 13 Aug 2018 at 16:56, Amitay Isaacs <amitay@ozlabs.org>
> wrote:
> 
> > > +docker pull ubuntu:18.04
> > > +docker build -t ${CONTAINER} - <<< "${Dockerfile}"
> > > +
> > > +RUN="docker run --rm=true --user=${USER} -w ${PWD} -v
> > > ${HOME}:${HOME} -t ${CONTAINER}"
> > > +
> > > +${RUN} ./bootstrap.sh
> > > +${RUN} ./configure
> > > +${RUN} make clean
> > 
> > Does travis cache the older image?  Or it's always created from
> > scratch?  If it's created from scratch, then we don't really need
> > "make
> > clean".  If not, we also should add "git clean -x -d -f" before
> > running
> > "./bootstrap.sh"
> 
> Travis doesn't appear to cache the containers. We do cache them on
> openpower.xyz's Jenkins. When I use them locally, it is cached.
> 
> We don't want to do a git clean in the script as that might have
> unexpected results for people using it in their local tree.
> 
> For our CI (Travis, Jenkins) the workspace will be clean each time,
> so
> the make clean is unnecessary. This these files are not just for
> travis, but is intended for anyone wanting to do a build. Hence the
> make clean which is useful if the previous build was a cross-compile.
> Without the clean a user might get this:
> 
>    CCLD     pdbg
> /usr/bin/x86_64-linux-gnu-ld: unknown architecture of input file
> `fake.dtb.o' is incompatible with i386:x86-64 output
> collect2: error: ld returned 1 exit status

More confusion.  So docker is only for access to cross-compilers?
I assumed that the build was in a directory in the docker container.
Didn't realize that it's actually the directory on the host.

Amitay.
Joel Stanley Aug. 13, 2018, 8:21 a.m. UTC | #4
On Mon, 13 Aug 2018 at 17:30, Amitay Isaacs <amitay@ozlabs.org> wrote:
>
> On Mon, 2018-08-13 at 17:11 +0930, Joel Stanley wrote:
> > Hi Amitay,
> >
> > On Mon, 13 Aug 2018 at 16:56, Amitay Isaacs <amitay@ozlabs.org>
> > wrote:
> >
> > > > +docker pull ubuntu:18.04
> > > > +docker build -t ${CONTAINER} - <<< "${Dockerfile}"
> > > > +
> > > > +RUN="docker run --rm=true --user=${USER} -w ${PWD} -v
> > > > ${HOME}:${HOME} -t ${CONTAINER}"
> > > > +
> > > > +${RUN} ./bootstrap.sh
> > > > +${RUN} ./configure
> > > > +${RUN} make clean
> > >
> > > Does travis cache the older image?  Or it's always created from
> > > scratch?  If it's created from scratch, then we don't really need
> > > "make
> > > clean".  If not, we also should add "git clean -x -d -f" before
> > > running
> > > "./bootstrap.sh"
> >
> > Travis doesn't appear to cache the containers. We do cache them on
> > openpower.xyz's Jenkins. When I use them locally, it is cached.
> >
> > We don't want to do a git clean in the script as that might have
> > unexpected results for people using it in their local tree.
> >
> > For our CI (Travis, Jenkins) the workspace will be clean each time,
> > so
> > the make clean is unnecessary. This these files are not just for
> > travis, but is intended for anyone wanting to do a build. Hence the
> > make clean which is useful if the previous build was a cross-compile.
> > Without the clean a user might get this:
> >
> >    CCLD     pdbg
> > /usr/bin/x86_64-linux-gnu-ld: unknown architecture of input file
> > `fake.dtb.o' is incompatible with i386:x86-64 output
> > collect2: error: ld returned 1 exit status
>
> More confusion.  So docker is only for access to cross-compilers?
> I assumed that the build was in a directory in the docker container.
> Didn't realize that it's actually the directory on the host.

We use docker's "mount volume" option to mount the source directory at
the same path as it exists on the host machine. This allows the user
(or CI) to build the tool without necessarily having the tools
installed on their host machine (or the same versions).

I first used this technique when developing a systemd application for
openbmc, when openbmc's libsystemd was newer than the one on my
machine. You can still use your host's editor and what not, but the
build process is isolated.

I find it useful even for small tools without many external
dependencies like pdbg, as it lets me test on a released version of
the distro, and you find out when a new dependency has been added
pretty fast.

Cheers,

Joel
diff mbox series

Patch

diff --git a/.travis.yml b/.travis.yml
index 9e1a86bc3140..596b50ff4e1e 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -3,3 +3,4 @@  services:
 
 script:
     - ./utils/build.sh
+    - ./utils/test.sh
diff --git a/utils/test.sh b/utils/test.sh
new file mode 100755
index 000000000000..e2bd25893910
--- /dev/null
+++ b/utils/test.sh
@@ -0,0 +1,33 @@ 
+#!/bin/bash
+
+set -e
+
+CONTAINER=pdbg-check
+
+Dockerfile=$(cat << EOF
+FROM ubuntu:18.04
+RUN apt-get update && apt-get install --no-install-recommends -yy \
+	make \
+	gcc \
+	autoconf \
+	automake \
+	libtool \
+	git \
+	device-tree-compiler
+RUN groupadd -g ${GROUPS} ${USER} && useradd -d ${HOME} -m -u ${UID} -g ${GROUPS} ${USER}
+USER ${USER}
+ENV HOME ${HOME}
+RUN /bin/bash
+EOF
+)
+
+docker pull ubuntu:18.04
+docker build -t ${CONTAINER} - <<< "${Dockerfile}"
+
+RUN="docker run --rm=true --user=${USER} -w ${PWD} -v ${HOME}:${HOME} -t ${CONTAINER}"
+
+${RUN} ./bootstrap.sh
+${RUN} ./configure
+${RUN} make clean
+${RUN} make
+${RUN} make check