From patchwork Fri Oct 11 11:53:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ales Musil X-Patchwork-Id: 1996057 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=AL/sPwqd; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XQ4lx5bw1z1xt1 for ; Fri, 11 Oct 2024 22:54:05 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id AB95F415CD; Fri, 11 Oct 2024 11:54:02 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id Xp8eElW_09Sm; Fri, 11 Oct 2024 11:54:00 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 585534045B Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=AL/sPwqd Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 585534045B; Fri, 11 Oct 2024 11:54:00 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 09156C08A6; Fri, 11 Oct 2024 11:54:00 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 21D1BC08A3 for ; Fri, 11 Oct 2024 11:53:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 0CA3E42451 for ; Fri, 11 Oct 2024 11:53:59 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id sjET7l4TDGTP for ; Fri, 11 Oct 2024 11:53:58 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amusil@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org C2EAD42441 Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org C2EAD42441 Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=AL/sPwqd Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id C2EAD42441 for ; Fri, 11 Oct 2024 11:53:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728647636; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=UIo5wn0LzUDDcRt/Tn3sJYJfxjCWo3bF7gNzaAlaiDk=; b=AL/sPwqdFEEs64FdS0BtaKG0Ef7zT15DE2hCBLJReqOmO56w7kLv5vB3r5QRaBl+/9o6TO xbT+2ISy7sLIwLOGnQidY2YVut+KwXrIu6hQ90ucowbDg1LafS6Tq5+XJe37D1LknWsQ12 nn8uiqpNdd0g7OQKgCT39V+TCctRO34= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-201-l_llCmwVOS-ss2EN3PSW9Q-1; Fri, 11 Oct 2024 07:53:55 -0400 X-MC-Unique: l_llCmwVOS-ss2EN3PSW9Q-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C5AD31956096 for ; Fri, 11 Oct 2024 11:53:54 +0000 (UTC) Received: from amusil.redhat.com (unknown [10.45.224.197]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 841D41956089; Fri, 11 Oct 2024 11:53:53 +0000 (UTC) From: Ales Musil To: dev@openvswitch.org Date: Fri, 11 Oct 2024 13:53:51 +0200 Message-ID: <20241011115351.1199364-1-amusil@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH ovn v2] ci: Add timeout command to test execution. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: dceara@redhat.com Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The timeout command allows us to specify a timeout for each job in CI. This is handy because there might be a test that is not able to finish for various reasons, last time it was forking nc process. At the same time having timeout over the make command allows us to preserve artifacts even if the timeout is reached so it might be easier to debug what is happening. Signed-off-by: Ales Musil --- v2: Rebase on top of current main. Address comments from Dumitru: - Fix typo in commit message. - Add the timeout command grace period of 5m before sending SIGKILL. - Add verbose option to timeout. - Increase the GH timeout to 2 hours. --- .ci/ci.sh | 12 +++++++++--- .ci/linux-build.sh | 9 ++++++--- .github/workflows/test.yml | 4 ++-- 3 files changed, 17 insertions(+), 8 deletions(-) diff --git a/.ci/ci.sh b/.ci/ci.sh index f543967dc..13df3db34 100755 --- a/.ci/ci.sh +++ b/.ci/ci.sh @@ -20,6 +20,7 @@ CONTAINER_CMD=${CONTAINER_CMD:-podman} CONTAINER_WORKSPACE="/workspace" CONTAINER_WORKDIR="/workspace/ovn-tmp" IMAGE_NAME=${IMAGE_NAME:-"ovn-org/ovn-tests"} +TIMEOUT=${TIMEOUT:-"0"} # Test variables ARCH=${ARCH:-$(uname -m)} @@ -100,7 +101,8 @@ function run_tests() { && \ ARCH=$ARCH CC=$CC LIBS=$LIBS OPTS=$OPTS TESTSUITE=$TESTSUITE \ TEST_RANGE=$TEST_RANGE SANITIZERS=$SANITIZERS DPDK=$DPDK \ - RECHECK=$RECHECK UNSTABLE=$UNSTABLE ./.ci/linux-build.sh + RECHECK=$RECHECK UNSTABLE=$UNSTABLE TIMEOUT=$TIMEOUT \ + ./.ci/linux-build.sh " } @@ -115,7 +117,7 @@ function check_clang_version_ge() { } options=$(getopt --options "" \ - --long help,shell,archive-logs,jobs:,ovn-path:,ovs-path:,image-name:\ + --long help,shell,archive-logs,jobs:,ovn-path:,ovs-path:,image-name:,timeout:\ -- "${@}") eval set -- "$options" while true; do @@ -142,11 +144,15 @@ while true; do --archive-logs) archive_logs="1" ;; + --timeout) + shift + TIMEOUT="$1" + ;; --help) set +x printf "$0 [--shell] [--help] [--archive-logs] [--jobs=] " printf "[--ovn-path=] [--ovs-path=] " - printf "[--image-name=]\n" + printf "[--image-name=] [--timeout=]\n" exit ;; --) diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh index 75a9480f9..82fcf7b80 100755 --- a/.ci/linux-build.sh +++ b/.ci/linux-build.sh @@ -10,6 +10,7 @@ OVN_CFLAGS="" OPTS="$OPTS --enable-Werror" JOBS=${JOBS:-"-j4"} RECHECK=${RECHECK:-"no"} +TIMEOUT=${TIMEOUT:-"0"} function install_dpdk() { @@ -105,7 +106,8 @@ function configure_clang() function run_tests() { - if ! make distcheck CFLAGS="${COMMON_CFLAGS} ${OVN_CFLAGS}" $JOBS \ + if ! timeout -k 5m -v $TIMEOUT make distcheck \ + CFLAGS="${COMMON_CFLAGS} ${OVN_CFLAGS}" $JOBS \ TESTSUITEFLAGS="$JOBS $TEST_RANGE" RECHECK=$RECHECK \ SKIP_UNSTABLE=$SKIP_UNSTABLE then @@ -147,8 +149,9 @@ function run_system_tests() local type=$1 local log_file=$2 - if ! sudo make $JOBS $type TESTSUITEFLAGS="$TEST_RANGE" \ - RECHECK=$RECHECK SKIP_UNSTABLE=$SKIP_UNSTABLE; then + if ! sudo timeout -k 5m -v $TIMEOUT make $JOBS $type \ + TESTSUITEFLAGS="$TEST_RANGE" RECHECK=$RECHECK \ + SKIP_UNSTABLE=$SKIP_UNSTABLE; then # $log_file is necessary for debugging. cat tests/$log_file return 1 diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index f7a184c13..82e09fbef 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -168,11 +168,11 @@ jobs: - name: build if: ${{ startsWith(matrix.cfg.testsuite, 'system-test') }} - run: sudo -E ./.ci/ci.sh --archive-logs + run: sudo -E ./.ci/ci.sh --archive-logs --timeout=2h - name: build if: ${{ !startsWith(matrix.cfg.testsuite, 'system-test') }} - run: ./.ci/ci.sh --archive-logs + run: ./.ci/ci.sh --archive-logs --timeout=2h - name: upload logs on failure if: failure() || cancelled()