From patchwork Sat Mar 24 01:57:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Darrick Wong X-Patchwork-Id: 890370 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.b="BsQQUBPF"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 407NpN331Cz9s08 for ; Sat, 24 Mar 2018 12:57:48 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751823AbeCXB5r (ORCPT ); Fri, 23 Mar 2018 21:57:47 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:51108 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751758AbeCXB5r (ORCPT ); Fri, 23 Mar 2018 21:57:47 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w2O1jYkC024935; Sat, 24 Mar 2018 01:57:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2017-10-26; bh=w+RAVpbrY6cffeOVjCKK2UpYpBLPhHPDO96/BUIIvHk=; b=BsQQUBPFlV7uCXhhAHjhAIF7O9oAD6OGxdwJCdlDDoa//yzQFl1ocWoBR4uzQ/hzqOA+ LmKZKjT1eM6j6JWWnGhXBksuB6IOxAATZjD6MiEQEAaCCYtnBJeM1R+fFMbJiydXpHSa FyMKnjRGCTjJjXg3EyzQlVl6lpo3ThfS4Tozn9L1auvxmdFWQGCZFSLUyDhJ4fb1zSyQ nLgHw0Qx3MIH8fHKok8SVdDJulLMMBbL2LD1C5A8ECmGnh8O0EdAnOlOHn9S5U/7PKf9 SMLSUJjXPoYd0MXXuDmdizG8A6gANR23boeYbjW1Dd9rdVF+U3OukiAm+EBEhP/IvSew bw== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2gwd8rg0fe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:57:45 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1vhTV025272 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:57:44 GMT Received: from abhmp0012.oracle.com (abhmp0012.oracle.com [141.146.116.18]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w2O1vhHm024101; Sat, 24 Mar 2018 01:57:43 GMT Received: from localhost (/10.159.132.121) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 23 Mar 2018 18:57:43 -0700 Subject: [PATCH 1/5] misc: enable link time optimization, if requested From: "Darrick J. Wong" To: tytso@mit.edu, darrick.wong@oracle.com Cc: linux-ext4@vger.kernel.org Date: Fri, 23 Mar 2018 18:57:42 -0700 Message-ID: <152185666252.10571.8114279325912132495.stgit@magnolia> In-Reply-To: <152185665629.10571.4584627052866250686.stgit@magnolia> References: <152185665629.10571.4584627052866250686.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8841 signatures=668695 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=9 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1803240010 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Darrick J. Wong Enable link time optimization (LTO) if the builder requests it. The extra link optimization results in smaller binaries. Signed-off-by: Darrick J. Wong --- MCONFIG.in | 19 +++++--- configure | 138 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ configure.ac | 34 ++++++++++++++ debian/rules | 2 - 4 files changed, 184 insertions(+), 9 deletions(-) diff --git a/MCONFIG.in b/MCONFIG.in index ee83554..f06933d 100644 --- a/MCONFIG.in +++ b/MCONFIG.in @@ -81,21 +81,24 @@ SYSTEMD_SYSTEM_UNIT_DIR = @systemd_system_unit_dir@ @ifNotGNUmake@ CHECK_CMD=true @ifNotGNUmake@ CPPCHECK_CMD=true +SANITIZER_CFLAGS = @lto_cflags@ +SANITIZER_LDFLAGS = @lto_ldflags@ + CC = @CC@ BUILD_CC = @BUILD_CC@ -CFLAGS = @CFLAGS@ -CFLAGS_SHLIB = @CFLAGS_SHLIB@ -CFLAGS_STLIB = @CFLAGS_STLIB@ +CFLAGS = $(SANITIZER_CFLAGS) @CFLAGS@ +CFLAGS_SHLIB = $(SANITIZER_CFLAGS) @CFLAGS_SHLIB@ +CFLAGS_STLIB = $(SANITIZER_CFLAGS) @CFLAGS_STLIB@ CPPFLAGS = @INCLUDES@ ALL_CFLAGS = $(CPPFLAGS) $(CFLAGS) $(CFLAGS_WARN) @DEFS@ $(LOCAL_CFLAGS) ALL_CFLAGS_SHLIB = $(CPPFLAGS) $(CFLAGS_SHLIB) $(CFLAGS_WARN) @DEFS@ $(LOCAL_CFLAGS) ALL_CFLAGS_STLIB = $(CPPFLAGS) $(CFLAGS_STLIB) $(CFLAGS_WARN) @DEFS@ $(LOCAL_CFLAGS) -LDFLAGS = @LDFLAGS@ -LDFLAGS_SHLIB = @LDFLAGS_SHLIB@ +LDFLAGS = $(SANITIZER_LDFLAGS) @LDFLAGS@ +LDFLAGS_SHLIB = $(SANITIZER_LDFLAGS) @LDFLAGS_SHLIB@ ALL_LDFLAGS = $(LDFLAGS) @LDFLAG_DYNAMIC@ -LDFLAGS_STATIC = @LDFLAGS_STATIC@ -BUILD_CFLAGS = @BUILD_CFLAGS@ -BUILD_LDFLAGS = @BUILD_LDFLAGS@ +LDFLAGS_STATIC = $(SANITIZER_LDFLAGS) @LDFLAGS_STATIC@ +BUILD_CFLAGS = $(SANITIZER_CFLAGS) @BUILD_CFLAGS@ +BUILD_LDFLAGS = $(SANITIZER_LDFLAGS) @BUILD_LDFLAGS@ RDYNAMIC = @RDYNAMIC@ LINK_BUILD_FLAGS = @LINK_BUILD_FLAGS@ LINK_INSTALL_FLAGS = @LINK_INSTALL_FLAGS@ diff --git a/configure b/configure index f6a0dea..5947614 100755 --- a/configure +++ b/configure @@ -653,6 +653,11 @@ E2SCRUB_CMT UNIX_CMT CYGWIN_CMT LINUX_CMT +lto_ldflags +lto_cflags +have_lto +gcc_ranlib +gcc_ar UNI_DIFF_OPTS SEM_INIT_LIB FUSE_CMT @@ -903,6 +908,7 @@ with_libiconv_prefix with_included_gettext with_libintl_prefix enable_fuse2fs +enable_lto with_multiarch with_udev_rules_dir with_crond_dir @@ -1581,6 +1587,7 @@ Optional Features: --disable-threads build without multithread safety --disable-rpath do not hardcode runtime library paths --disable-fuse2fs do not build fuse2fs + --enable-lto enable link time optimization Optional Packages: --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] @@ -13727,6 +13734,137 @@ $as_echo "#define HAVE_EXT2_IOCTLS 1" >>confdefs.h ;; esac +# Check whether --enable-lto was given. +if test "${enable_lto+set}" = set; then : + enableval=$enable_lto; +else + enable_lto=probe +fi + +if test "$enable_lto" = "yes" || test "$enable_lto" = "probe"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: checking if C compiler supports LTO" >&5 +$as_echo_n "checking if C compiler supports LTO... " >&6; } + OLD_CFLAGS="$CFLAGS" + OLD_LDFLAGS="$LDFLAGS" + LTO_FLAGS="-flto -ffat-lto-objects" + CFLAGS="$CFLAGS $LTO_FLAGS" + LDFLAGS="$LDFLAGS $LTO_FLAGS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + lto_cflags=$LTO_FLAGS + lto_ldflags=$LTO_FLAGS + # Extract the first word of "gcc-ar", so it can be a program name with args. +set dummy gcc-ar; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_gcc_ar+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $gcc_ar in + [\\/]* | ?:[\\/]*) + ac_cv_path_gcc_ar="$gcc_ar" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_gcc_ar="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +gcc_ar=$ac_cv_path_gcc_ar +if test -n "$gcc_ar"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $gcc_ar" >&5 +$as_echo "$gcc_ar" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + + # Extract the first word of "gcc-ranlib", so it can be a program name with args. +set dummy gcc-ranlib; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_path_gcc_ranlib+:} false; then : + $as_echo_n "(cached) " >&6 +else + case $gcc_ranlib in + [\\/]* | ?:[\\/]*) + ac_cv_path_gcc_ranlib="$gcc_ranlib" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then + ac_cv_path_gcc_ranlib="$as_dir/$ac_word$ac_exec_ext" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + + ;; +esac +fi +gcc_ranlib=$ac_cv_path_gcc_ranlib +if test -n "$gcc_ranlib"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $gcc_ranlib" >&5 +$as_echo "$gcc_ranlib" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + if test -x "$gcc_ar" && test -x "$gcc_ranlib"; then + have_lto=yes + AR="${gcc_ar}" + RANLIB="${gcc_ranlib}" + fi + CFLAGS="${OLD_CFLAGS}" + LDFLAGS="${OLD_LDFLAGS}" + + + +fi +if test "$enable_lto" = "yes" && test "$have_lto" != "yes"; then + as_fn_error $? "LTO not supported by compiler." "$LINENO" 5 +fi LINUX_CMT="#" CYGWIN_CMT="#" UNIX_CMT= diff --git a/configure.ac b/configure.ac index 5a2c8be..a605aad 100644 --- a/configure.ac +++ b/configure.ac @@ -1295,6 +1295,40 @@ linux*) ;; esac dnl +dnl Enable LTO for all packages +dnl +AC_ARG_ENABLE([lto], +[ --enable-lto enable link time optimization],, +enable_lto=probe) +if test "$enable_lto" = "yes" || test "$enable_lto" = "probe"; then + AC_MSG_CHECKING([if C compiler supports LTO]) + OLD_CFLAGS="$CFLAGS" + OLD_LDFLAGS="$LDFLAGS" + LTO_FLAGS="-flto -ffat-lto-objects" + CFLAGS="$CFLAGS $LTO_FLAGS" + LDFLAGS="$LDFLAGS $LTO_FLAGS" + AC_LINK_IFELSE([AC_LANG_PROGRAM([])], + [AC_MSG_RESULT([yes])] + [lto_cflags=$LTO_FLAGS] + [lto_ldflags=$LTO_FLAGS] + [AC_PATH_PROG(gcc_ar, gcc-ar,,)] + [AC_PATH_PROG(gcc_ranlib, gcc-ranlib,,)], + [AC_MSG_RESULT([no])]) + if test -x "$gcc_ar" && test -x "$gcc_ranlib"; then + have_lto=yes + AR="${gcc_ar}" + RANLIB="${gcc_ranlib}" + fi + CFLAGS="${OLD_CFLAGS}" + LDFLAGS="${OLD_LDFLAGS}" + AC_SUBST(have_lto) + AC_SUBST(lto_cflags) + AC_SUBST(lto_ldflags) +fi +if test "$enable_lto" = "yes" && test "$have_lto" != "yes"; then + AC_MSG_ERROR([LTO not supported by compiler.]) +fi +dnl dnl OS-specific uncomment control dnl LINUX_CMT="#" diff --git a/debian/rules b/debian/rules index b6df023..176ea78 100755 --- a/debian/rules +++ b/debian/rules @@ -164,7 +164,7 @@ endif BACKTRACE_CONF_FLAGS ?= $(shell if ${debdir}/scripts/test-backtrace ; then echo --disable-backtrace ; fi) -COMMON_CONF_FLAGS = --disable-e2initrd-helper \ +COMMON_CONF_FLAGS = --enable-lto --disable-e2initrd-helper \ --infodir=/usr/share/info --enable-symlink-install \ --with-multiarch=$(DEB_HOST_MULTIARCH) \ $(BACKTRACE_CONF_FLAGS) $(UTIL_CONF_FLAGS) From patchwork Sat Mar 24 01:57:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Darrick Wong X-Patchwork-Id: 890371 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.b="U5HGRz/K"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 407NpW5cNnz9s08 for ; Sat, 24 Mar 2018 12:57:55 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751832AbeCXB5z (ORCPT ); Fri, 23 Mar 2018 21:57:55 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:33972 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751758AbeCXB5y (ORCPT ); Fri, 23 Mar 2018 21:57:54 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w2O1vqR0047733; Sat, 24 Mar 2018 01:57:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2017-10-26; bh=KdQeAX5z6b4LCD0jnrVWz4xwd0S5QiGCokozXwtRX4A=; b=U5HGRz/KYTlGtYpPOOPjTVKUdmIlT22fJQFA575ZWkzaDQXJrptvRWOewhXsY8ekh3Mv UqwuFJS9GUVOLqBzqAz4mYGncyCCV+mjJNlYlY/v4r3FCpWJ0P9EwXU90oUORY1lT7gn Vi0H6N7gb5SobcQLFN82ROTj4UpP68qYipJRVPifCzFDitNyw+5qqRX7o6kRd9fYCCki FejFCeq3uyDUgOFImTawV9UX+cmyMWU7Re1vADON6GEDEoTEbp0chqOkGzurxgRdsSsg 4kgH8LEjG5Y3yejwEjUeek8MNmqwRCsZurHDusFoC8nVXuO74dQo2UVjP583ZAlaoUm1 WQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp2120.oracle.com with ESMTP id 2gwd8rr0gu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:57:51 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1voto007449 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:57:51 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1vo78023785; Sat, 24 Mar 2018 01:57:50 GMT Received: from localhost (/10.159.132.121) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 23 Mar 2018 18:57:49 -0700 Subject: [PATCH 2/5] misc: enable UBSAN if the builder wants it From: "Darrick J. Wong" To: tytso@mit.edu, darrick.wong@oracle.com Cc: linux-ext4@vger.kernel.org Date: Fri, 23 Mar 2018 18:57:48 -0700 Message-ID: <152185666885.10571.2566613183664269112.stgit@magnolia> In-Reply-To: <152185665629.10571.4584627052866250686.stgit@magnolia> References: <152185665629.10571.4584627052866250686.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8841 signatures=668695 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=9 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=969 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1803240010 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Darrick J. Wong Enable the undefined behavior sanitizer (ubsan) if the builder requests it and it's available. Signed-off-by: Darrick J. Wong --- MCONFIG.in | 4 ++-- config/ltmain.sh | 1 + configure | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++ configure.ac | 28 ++++++++++++++++++++++++++++ debian/rules | 2 +- 5 files changed, 84 insertions(+), 3 deletions(-) create mode 120000 config/ltmain.sh diff --git a/MCONFIG.in b/MCONFIG.in index f06933d..4234ff0 100644 --- a/MCONFIG.in +++ b/MCONFIG.in @@ -81,8 +81,8 @@ SYSTEMD_SYSTEM_UNIT_DIR = @systemd_system_unit_dir@ @ifNotGNUmake@ CHECK_CMD=true @ifNotGNUmake@ CPPCHECK_CMD=true -SANITIZER_CFLAGS = @lto_cflags@ -SANITIZER_LDFLAGS = @lto_ldflags@ +SANITIZER_CFLAGS = @lto_cflags@ @ubsan_cflags@ +SANITIZER_LDFLAGS = @lto_ldflags@ @ubsan_ldflags@ CC = @CC@ BUILD_CC = @BUILD_CC@ diff --git a/config/ltmain.sh b/config/ltmain.sh new file mode 120000 index 0000000..8cff389 --- /dev/null +++ b/config/ltmain.sh @@ -0,0 +1 @@ +/usr/share/libtool/build-aux/ltmain.sh \ No newline at end of file diff --git a/configure b/configure index 5947614..7632a4a 100755 --- a/configure +++ b/configure @@ -653,6 +653,9 @@ E2SCRUB_CMT UNIX_CMT CYGWIN_CMT LINUX_CMT +ubsan_ldflags +ubsan_cflags +have_ubsan lto_ldflags lto_cflags have_lto @@ -909,6 +912,7 @@ with_included_gettext with_libintl_prefix enable_fuse2fs enable_lto +enable_ubsan with_multiarch with_udev_rules_dir with_crond_dir @@ -1588,6 +1592,7 @@ Optional Features: --disable-rpath do not hardcode runtime library paths --disable-fuse2fs do not build fuse2fs --enable-lto enable link time optimization + --enable-ubsan enable undefined behavior sanitizer Optional Packages: --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] @@ -13865,6 +13870,53 @@ fi if test "$enable_lto" = "yes" && test "$have_lto" != "yes"; then as_fn_error $? "LTO not supported by compiler." "$LINENO" 5 fi +# Check whether --enable-ubsan was given. +if test "${enable_ubsan+set}" = set; then : + enableval=$enable_ubsan; +else + enable_ubsan=no +fi + +if test "$enable_ubsan" = "yes" || test "$enable_ubsan" = "probe"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: checking if C compiler supports UBSAN" >&5 +$as_echo_n "checking if C compiler supports UBSAN... " >&6; } + OLD_CFLAGS="$CFLAGS" + OLD_LDFLAGS="$LDFLAGS" + UBSAN_FLAGS="-fsanitize=undefined" + CFLAGS="$CFLAGS $UBSAN_FLAGS" + LDFLAGS="$LDFLAGS $UBSAN_FLAGS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + ubsan_cflags=$UBSAN_FLAGS + ubsan_ldflags=$UBSAN_FLAGS + have_ubsan=yes +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + CFLAGS="${OLD_CFLAGS}" + LDFLAGS="${OLD_LDFLAGS}" + + + +fi +if test "$enable_ubsan" = "yes" && test "$have_ubsan" != "yes"; then + as_fn_error $? "UBSAN not supported by compiler." "$LINENO" 5 +fi LINUX_CMT="#" CYGWIN_CMT="#" UNIX_CMT= diff --git a/configure.ac b/configure.ac index a605aad..981c6b0 100644 --- a/configure.ac +++ b/configure.ac @@ -1329,6 +1329,34 @@ if test "$enable_lto" = "yes" && test "$have_lto" != "yes"; then AC_MSG_ERROR([LTO not supported by compiler.]) fi dnl +dnl Enable UBSAN for all packages +dnl +AC_ARG_ENABLE([ubsan], +[ --enable-ubsan enable undefined behavior sanitizer],, +enable_ubsan=no) +if test "$enable_ubsan" = "yes" || test "$enable_ubsan" = "probe"; then + AC_MSG_CHECKING([if C compiler supports UBSAN]) + OLD_CFLAGS="$CFLAGS" + OLD_LDFLAGS="$LDFLAGS" + UBSAN_FLAGS="-fsanitize=undefined" + CFLAGS="$CFLAGS $UBSAN_FLAGS" + LDFLAGS="$LDFLAGS $UBSAN_FLAGS" + AC_LINK_IFELSE([AC_LANG_PROGRAM([])], + [AC_MSG_RESULT([yes])] + [ubsan_cflags=$UBSAN_FLAGS] + [ubsan_ldflags=$UBSAN_FLAGS] + [have_ubsan=yes], + [AC_MSG_RESULT([no])]) + CFLAGS="${OLD_CFLAGS}" + LDFLAGS="${OLD_LDFLAGS}" + AC_SUBST(have_ubsan) + AC_SUBST(ubsan_cflags) + AC_SUBST(ubsan_ldflags) +fi +if test "$enable_ubsan" = "yes" && test "$have_ubsan" != "yes"; then + AC_MSG_ERROR([UBSAN not supported by compiler.]) +fi +dnl dnl OS-specific uncomment control dnl LINUX_CMT="#" diff --git a/debian/rules b/debian/rules index 176ea78..600b751 100755 --- a/debian/rules +++ b/debian/rules @@ -164,7 +164,7 @@ endif BACKTRACE_CONF_FLAGS ?= $(shell if ${debdir}/scripts/test-backtrace ; then echo --disable-backtrace ; fi) -COMMON_CONF_FLAGS = --enable-lto --disable-e2initrd-helper \ +COMMON_CONF_FLAGS = --enable-lto --disable-ubsan --disable-e2initrd-helper \ --infodir=/usr/share/info --enable-symlink-install \ --with-multiarch=$(DEB_HOST_MULTIARCH) \ $(BACKTRACE_CONF_FLAGS) $(UTIL_CONF_FLAGS) From patchwork Sat Mar 24 01:57:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Darrick Wong X-Patchwork-Id: 890372 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.b="ZYhfi+Hv"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 407Npd4sv5z9s08 for ; Sat, 24 Mar 2018 12:58:01 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751843AbeCXB6B (ORCPT ); Fri, 23 Mar 2018 21:58:01 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:34018 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751758AbeCXB6A (ORCPT ); Fri, 23 Mar 2018 21:58:00 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w2O1hEgN040224; Sat, 24 Mar 2018 01:57:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2017-10-26; bh=CCKta4Bo+thXY3Nr0tGhXdvw4UQvmBSg+78T8bK9dKU=; b=ZYhfi+Hvtq1/3TkLVU8r0GySwwA5tJO99BbIVBaHoYENrJdUdaVqVmMhKOdLFomuVPrv PPUBx7hN53CVjo+QXZWVj4+SWL4OUmAYTxxjtf/5sd5afa4uYGHUk0K4iTvE2/nQA/pv +EfpWYOf/zDIYXtjahsalwb+znVC/0JPukBnqFcz1KfdWaIzsM//IwBZ5xQxscM6hG83 +/FRgEuW0bwq2adcSxRbgjNrgoTBpv4LhKgsqOvxQCYRx9fVSyo6ukKNyFJ726VjGQig X+n9wyBlLtgzF/hQzeCRuoDM4RWFtcUzoqxNcP1EA0EtrKB4/eMkm04ILlBi0zXgVyVr Ug== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp2120.oracle.com with ESMTP id 2gwd8rr0gy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:57:57 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1vumZ006142 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:57:57 GMT Received: from abhmp0013.oracle.com (abhmp0013.oracle.com [141.146.116.19]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1vuDK020369; Sat, 24 Mar 2018 01:57:56 GMT Received: from localhost (/10.159.132.121) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 23 Mar 2018 18:57:56 -0700 Subject: [PATCH 3/5] misc: enable gcc/clang address sanitizer if the builder wants it From: "Darrick J. Wong" To: tytso@mit.edu, darrick.wong@oracle.com Cc: linux-ext4@vger.kernel.org Date: Fri, 23 Mar 2018 18:57:55 -0700 Message-ID: <152185667509.10571.4425225168076417085.stgit@magnolia> In-Reply-To: <152185665629.10571.4584627052866250686.stgit@magnolia> References: <152185665629.10571.4584627052866250686.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8841 signatures=668695 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=9 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=922 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1803240010 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Darrick J. Wong Enable AddressSanitizer to look for memory usage errors if the builder asks for it and it's available. Signed-off-by: Darrick J. Wong --- MCONFIG.in | 4 ++-- configure | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++ configure.ac | 28 ++++++++++++++++++++++++++++ debian/rules | 3 ++- 4 files changed, 84 insertions(+), 3 deletions(-) diff --git a/MCONFIG.in b/MCONFIG.in index 4234ff0..1344511 100644 --- a/MCONFIG.in +++ b/MCONFIG.in @@ -81,8 +81,8 @@ SYSTEMD_SYSTEM_UNIT_DIR = @systemd_system_unit_dir@ @ifNotGNUmake@ CHECK_CMD=true @ifNotGNUmake@ CPPCHECK_CMD=true -SANITIZER_CFLAGS = @lto_cflags@ @ubsan_cflags@ -SANITIZER_LDFLAGS = @lto_ldflags@ @ubsan_ldflags@ +SANITIZER_CFLAGS = @lto_cflags@ @ubsan_cflags@ @addrsan_cflags@ +SANITIZER_LDFLAGS = @lto_ldflags@ @ubsan_ldflags@ @addrsan_ldflags@ CC = @CC@ BUILD_CC = @BUILD_CC@ diff --git a/configure b/configure index 7632a4a..31fe8ca 100755 --- a/configure +++ b/configure @@ -653,6 +653,9 @@ E2SCRUB_CMT UNIX_CMT CYGWIN_CMT LINUX_CMT +addrsan_ldflags +addrsan_cflags +have_addrsan ubsan_ldflags ubsan_cflags have_ubsan @@ -913,6 +916,7 @@ with_libintl_prefix enable_fuse2fs enable_lto enable_ubsan +enable_addrsan with_multiarch with_udev_rules_dir with_crond_dir @@ -1593,6 +1597,7 @@ Optional Features: --disable-fuse2fs do not build fuse2fs --enable-lto enable link time optimization --enable-ubsan enable undefined behavior sanitizer + --enable-addrsan enable address sanitizer Optional Packages: --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] @@ -13917,6 +13922,53 @@ fi if test "$enable_ubsan" = "yes" && test "$have_ubsan" != "yes"; then as_fn_error $? "UBSAN not supported by compiler." "$LINENO" 5 fi +# Check whether --enable-addrsan was given. +if test "${enable_addrsan+set}" = set; then : + enableval=$enable_addrsan; +else + enable_addrsan=no +fi + +if test "$enable_addrsan" = "yes" || test "$enable_addrsan" = "probe"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: checking if C compiler supports ADDRSAN" >&5 +$as_echo_n "checking if C compiler supports ADDRSAN... " >&6; } + OLD_CFLAGS="$CFLAGS" + OLD_LDFLAGS="$LDFLAGS" + ADDRSAN_FLAGS="-fsanitize=address" + CFLAGS="$CFLAGS $ADDRSAN_FLAGS" + LDFLAGS="$LDFLAGS $ADDRSAN_FLAGS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + addrsan_cflags=$ADDRSAN_FLAGS + addrsan_ldflags=$ADDRSAN_FLAGS + have_addrsan=yes +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + CFLAGS="${OLD_CFLAGS}" + LDFLAGS="${OLD_LDFLAGS}" + + + +fi +if test "$enable_addrsan" = "yes" && test "$have_addrsan" != "yes"; then + as_fn_error $? "ADDRSAN not supported by compiler." "$LINENO" 5 +fi LINUX_CMT="#" CYGWIN_CMT="#" UNIX_CMT= diff --git a/configure.ac b/configure.ac index 981c6b0..e02bdea 100644 --- a/configure.ac +++ b/configure.ac @@ -1357,6 +1357,34 @@ if test "$enable_ubsan" = "yes" && test "$have_ubsan" != "yes"; then AC_MSG_ERROR([UBSAN not supported by compiler.]) fi dnl +dnl Enable ADDRSAN for all packages +dnl +AC_ARG_ENABLE([addrsan], +[ --enable-addrsan enable address sanitizer],, +enable_addrsan=no) +if test "$enable_addrsan" = "yes" || test "$enable_addrsan" = "probe"; then + AC_MSG_CHECKING([if C compiler supports ADDRSAN]) + OLD_CFLAGS="$CFLAGS" + OLD_LDFLAGS="$LDFLAGS" + ADDRSAN_FLAGS="-fsanitize=address" + CFLAGS="$CFLAGS $ADDRSAN_FLAGS" + LDFLAGS="$LDFLAGS $ADDRSAN_FLAGS" + AC_LINK_IFELSE([AC_LANG_PROGRAM([])], + [AC_MSG_RESULT([yes])] + [addrsan_cflags=$ADDRSAN_FLAGS] + [addrsan_ldflags=$ADDRSAN_FLAGS] + [have_addrsan=yes], + [AC_MSG_RESULT([no])]) + CFLAGS="${OLD_CFLAGS}" + LDFLAGS="${OLD_LDFLAGS}" + AC_SUBST(have_addrsan) + AC_SUBST(addrsan_cflags) + AC_SUBST(addrsan_ldflags) +fi +if test "$enable_addrsan" = "yes" && test "$have_addrsan" != "yes"; then + AC_MSG_ERROR([ADDRSAN not supported by compiler.]) +fi +dnl dnl OS-specific uncomment control dnl LINUX_CMT="#" diff --git a/debian/rules b/debian/rules index 600b751..f4135e8 100755 --- a/debian/rules +++ b/debian/rules @@ -164,7 +164,8 @@ endif BACKTRACE_CONF_FLAGS ?= $(shell if ${debdir}/scripts/test-backtrace ; then echo --disable-backtrace ; fi) -COMMON_CONF_FLAGS = --enable-lto --disable-ubsan --disable-e2initrd-helper \ +COMMON_CONF_FLAGS = --enable-lto --disable-ubsan --disable-addrsan \ + --disable-e2initrd-helper \ --infodir=/usr/share/info --enable-symlink-install \ --with-multiarch=$(DEB_HOST_MULTIARCH) \ $(BACKTRACE_CONF_FLAGS) $(UTIL_CONF_FLAGS) From patchwork Sat Mar 24 01:58:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Darrick Wong X-Patchwork-Id: 890373 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.b="hSVXYARC"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 407Npl2T2Jz9s08 for ; Sat, 24 Mar 2018 12:58:07 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751848AbeCXB6G (ORCPT ); Fri, 23 Mar 2018 21:58:06 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:51454 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751758AbeCXB6G (ORCPT ); Fri, 23 Mar 2018 21:58:06 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w2O1hLmZ023619; Sat, 24 Mar 2018 01:58:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2017-10-26; bh=JRgsWVguvaS9NhGDRiPexSRR3Q5zq2dQI4vd6ZU1I5Y=; b=hSVXYARCq/L7BRlqGTd1OFX3ngYMeMPed+ZXNr0JKRmjxerNn8o6Pds+nQGH/kW2kveH bqujJIYf/9iKZRG9tY018xk50uqD3JaewKRWVmKLBa8r6M/HYnSA/f7W+8yj3n0WTR0q ZBugxDvTHjmSK2GeHxwqrsnMUxaWewWckpx644JpoQcvQPTp5FR9X+7uRmGXStGrq/bE FYzN/pypi7WCWwypfrXedt/HZdSD9d1sGACT6CIyzk/a9yBHINQ/qj0jvYTQ6qkUZt/f A6G6Mg7UXEUSMGP/jXKMnHSsQ+cb5xSS84Yxaca9XqQOVYHU60YuFqfty1ArGnFS7wrY 5w== Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp2130.oracle.com with ESMTP id 2gwd8rg0g4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:58:04 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1w2Wj026378 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:58:03 GMT Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w2O1w2UJ024143; Sat, 24 Mar 2018 01:58:02 GMT Received: from localhost (/10.159.132.121) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 23 Mar 2018 18:58:02 -0700 Subject: [PATCH 4/5] misc: enable thread sanitizer if the builder wants it From: "Darrick J. Wong" To: tytso@mit.edu, darrick.wong@oracle.com Cc: linux-ext4@vger.kernel.org Date: Fri, 23 Mar 2018 18:58:01 -0700 Message-ID: <152185668144.10571.4093176467557555312.stgit@magnolia> In-Reply-To: <152185665629.10571.4584627052866250686.stgit@magnolia> References: <152185665629.10571.4584627052866250686.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8841 signatures=668695 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=9 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1803240010 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Darrick J. Wong Enable the gcc/clang thread data corruption sanitizer if the builder requests it and it's available. Signed-off-by: Darrick J. Wong --- MCONFIG.in | 4 ++-- configure | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ configure.ac | 31 +++++++++++++++++++++++++++++++ debian/rules | 2 +- 4 files changed, 90 insertions(+), 3 deletions(-) diff --git a/MCONFIG.in b/MCONFIG.in index 1344511..c3b3740 100644 --- a/MCONFIG.in +++ b/MCONFIG.in @@ -81,8 +81,8 @@ SYSTEMD_SYSTEM_UNIT_DIR = @systemd_system_unit_dir@ @ifNotGNUmake@ CHECK_CMD=true @ifNotGNUmake@ CPPCHECK_CMD=true -SANITIZER_CFLAGS = @lto_cflags@ @ubsan_cflags@ @addrsan_cflags@ -SANITIZER_LDFLAGS = @lto_ldflags@ @ubsan_ldflags@ @addrsan_ldflags@ +SANITIZER_CFLAGS = @lto_cflags@ @ubsan_cflags@ @addrsan_cflags@ @threadsan_cflags@ +SANITIZER_LDFLAGS = @lto_ldflags@ @ubsan_ldflags@ @addrsan_ldflags@ @threadsan_ldflags@ CC = @CC@ BUILD_CC = @BUILD_CC@ diff --git a/configure b/configure index 31fe8ca..c278ddb 100755 --- a/configure +++ b/configure @@ -653,6 +653,9 @@ E2SCRUB_CMT UNIX_CMT CYGWIN_CMT LINUX_CMT +threadsan_ldflags +threadsan_cflags +have_threadsan addrsan_ldflags addrsan_cflags have_addrsan @@ -917,6 +920,7 @@ enable_fuse2fs enable_lto enable_ubsan enable_addrsan +enable_threadsan with_multiarch with_udev_rules_dir with_crond_dir @@ -1598,6 +1602,7 @@ Optional Features: --enable-lto enable link time optimization --enable-ubsan enable undefined behavior sanitizer --enable-addrsan enable address sanitizer + --enable-threadsan enable thread sanitizer Optional Packages: --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] @@ -13969,6 +13974,57 @@ fi if test "$enable_addrsan" = "yes" && test "$have_addrsan" != "yes"; then as_fn_error $? "ADDRSAN not supported by compiler." "$LINENO" 5 fi +# Check whether --enable-threadsan was given. +if test "${enable_threadsan+set}" = set; then : + enableval=$enable_threadsan; +else + enable_threadsan=no +fi + +if test "$enable_threadsan" = "yes" || test "$enable_threadsan" = "probe"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: checking if C compiler supports THREADSAN" >&5 +$as_echo_n "checking if C compiler supports THREADSAN... " >&6; } + OLD_CFLAGS="$CFLAGS" + OLD_LDFLAGS="$LDFLAGS" + THREADSAN_FLAGS="-fsanitize=thread" + CFLAGS="$CFLAGS $THREADSAN_FLAGS" + LDFLAGS="$LDFLAGS $THREADSAN_FLAGS" + cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ + + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + threadsan_cflags=$THREADSAN_FLAGS + threadsan_ldflags=$THREADSAN_FLAGS + have_threadsan=yes +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + CFLAGS="${OLD_CFLAGS}" + LDFLAGS="${OLD_LDFLAGS}" + + + +fi +if test "$enable_threadsan" = "yes" && test "$have_threadsan" != "yes"; then + as_fn_error $? "THREADSAN not supported by compiler." "$LINENO" 5 +fi +if test "$have_threadsan" = "yes" && test "$have_addrsan" = "yes"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: WARNING: ADDRSAN and THREADSAN are not known to work together." >&5 +$as_echo "$as_me: WARNING: ADDRSAN and THREADSAN are not known to work together." >&2;} +fi LINUX_CMT="#" CYGWIN_CMT="#" UNIX_CMT= diff --git a/configure.ac b/configure.ac index e02bdea..1cc7420 100644 --- a/configure.ac +++ b/configure.ac @@ -1385,6 +1385,37 @@ if test "$enable_addrsan" = "yes" && test "$have_addrsan" != "yes"; then AC_MSG_ERROR([ADDRSAN not supported by compiler.]) fi dnl +dnl Enable THREADSAN for all packages +dnl +AC_ARG_ENABLE([threadsan], +[ --enable-threadsan enable thread sanitizer],, +enable_threadsan=no) +if test "$enable_threadsan" = "yes" || test "$enable_threadsan" = "probe"; then + AC_MSG_CHECKING([if C compiler supports THREADSAN]) + OLD_CFLAGS="$CFLAGS" + OLD_LDFLAGS="$LDFLAGS" + THREADSAN_FLAGS="-fsanitize=thread" + CFLAGS="$CFLAGS $THREADSAN_FLAGS" + LDFLAGS="$LDFLAGS $THREADSAN_FLAGS" + AC_LINK_IFELSE([AC_LANG_PROGRAM([])], + [AC_MSG_RESULT([yes])] + [threadsan_cflags=$THREADSAN_FLAGS] + [threadsan_ldflags=$THREADSAN_FLAGS] + [have_threadsan=yes], + [AC_MSG_RESULT([no])]) + CFLAGS="${OLD_CFLAGS}" + LDFLAGS="${OLD_LDFLAGS}" + AC_SUBST(have_threadsan) + AC_SUBST(threadsan_cflags) + AC_SUBST(threadsan_ldflags) +fi +if test "$enable_threadsan" = "yes" && test "$have_threadsan" != "yes"; then + AC_MSG_ERROR([THREADSAN not supported by compiler.]) +fi +if test "$have_threadsan" = "yes" && test "$have_addrsan" = "yes"; then + AC_MSG_WARN([ADDRSAN and THREADSAN are not known to work together.]) +fi +dnl dnl OS-specific uncomment control dnl LINUX_CMT="#" diff --git a/debian/rules b/debian/rules index f4135e8..6f694dd 100755 --- a/debian/rules +++ b/debian/rules @@ -165,7 +165,7 @@ endif BACKTRACE_CONF_FLAGS ?= $(shell if ${debdir}/scripts/test-backtrace ; then echo --disable-backtrace ; fi) COMMON_CONF_FLAGS = --enable-lto --disable-ubsan --disable-addrsan \ - --disable-e2initrd-helper \ + --disable-threadsan --disable-e2initrd-helper \ --infodir=/usr/share/info --enable-symlink-install \ --with-multiarch=$(DEB_HOST_MULTIARCH) \ $(BACKTRACE_CONF_FLAGS) $(UTIL_CONF_FLAGS) From patchwork Sat Mar 24 01:58:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Darrick Wong X-Patchwork-Id: 890374 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=oracle.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.b="TOYlTB3/"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 407Npv0hQVz9s08 for ; Sat, 24 Mar 2018 12:58:15 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751866AbeCXB6O (ORCPT ); Fri, 23 Mar 2018 21:58:14 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:48334 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751758AbeCXB6M (ORCPT ); Fri, 23 Mar 2018 21:58:12 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w2O1tu2J189385; Sat, 24 Mar 2018 01:58:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : from : to : cc : date : message-id : in-reply-to : references : mime-version : content-type : content-transfer-encoding; s=corp-2017-10-26; bh=jWOcbSViZc0n81BGqqDpNhQfO/sS9yxgygk4UvxnBK0=; b=TOYlTB3/0uptToNvleHrj/3kN1+PDnJMJTMQikNEN4ROKUrrqmZSC0r4QBaLWMYFxKNA z6tVTbDp36LjXsCQe4rjlzOEfFxsXEtNBBdb2VEKaUj+9xDpg5nLeVylo9IIdh+RHB7F X8XT+QaggzOx2H0GQQxVDsSIT86stmqvjaE+AjBgQOPsUY4o1zftrfvbmMJfEK9elGsN MNvSN9LchIiabvhUQpJpSCS3ZJZ0DeES0f6LoT99xHmXTxoO02qV5HATCkULx7lZEku+ x7xuldQNcUUHR3HaYPrnfYzQMeXJ0sWcNAoYkYQzigyKILjeRS1eZv1iGeZA1zNGPrNs eQ== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2130.oracle.com with ESMTP id 2gwded0046-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:58:10 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1w9L6006966 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 24 Mar 2018 01:58:10 GMT Received: from abhmp0003.oracle.com (abhmp0003.oracle.com [141.146.116.9]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w2O1w9rD020401; Sat, 24 Mar 2018 01:58:09 GMT Received: from localhost (/10.159.132.121) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 23 Mar 2018 18:58:08 -0700 Subject: [PATCH 5/5] e2fsck: adjust quota counters when clearing orphaned inodes From: "Darrick J. Wong" To: tytso@mit.edu, darrick.wong@oracle.com Cc: linux-ext4@vger.kernel.org Date: Fri, 23 Mar 2018 18:58:07 -0700 Message-ID: <152185668775.10571.9106551147583686858.stgit@magnolia> In-Reply-To: <152185665629.10571.4584627052866250686.stgit@magnolia> References: <152185665629.10571.4584627052866250686.stgit@magnolia> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8841 signatures=668695 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1803240010 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Darrick J. Wong If e2fsck encounters a filesystem that supports internal quotas, it is responsible for adjusting the quota counters if it decides to clear any orphaned inodes. Therefore, we must read the quota files, adjust the counters, and write the quota files back out when we are done. Signed-off-by: Darrick J. Wong --- e2fsck/super.c | 121 +++++++++++++++++++++++++++++++++++++------- lib/support/mkquota.c | 12 +++- tests/f_orphquot/expect | 10 ++++ tests/f_orphquot/image.bz2 | Bin tests/f_orphquot/script | 25 +++++++++ 5 files changed, 144 insertions(+), 24 deletions(-) create mode 100644 tests/f_orphquot/expect create mode 100644 tests/f_orphquot/image.bz2 create mode 100644 tests/f_orphquot/script diff --git a/e2fsck/super.c b/e2fsck/super.c index 5e29b64..80b67f3 100644 --- a/e2fsck/super.c +++ b/e2fsck/super.c @@ -72,6 +72,7 @@ struct process_block_struct { int abort; errcode_t errcode; blk64_t last_cluster; + struct ext2_inode_large *inode; }; static int release_inode_block(ext2_filsys fs, @@ -168,6 +169,8 @@ static int release_inode_block(ext2_filsys fs, retval |= BLOCK_CHANGED; } + if (ctx->qctx) + quota_data_sub(ctx->qctx, pb->inode, 0, ctx->fs->blocksize); ext2fs_block_alloc_stats2(fs, blk, -1); ctx->free_blocks++; return retval; @@ -179,15 +182,16 @@ static int release_inode_block(ext2_filsys fs, * not deleted. */ static int release_inode_blocks(e2fsck_t ctx, ext2_ino_t ino, - struct ext2_inode *inode, char *block_buf, + struct ext2_inode_large *inode, char *block_buf, struct problem_context *pctx) { struct process_block_struct pb; ext2_filsys fs = ctx->fs; + blk64_t blk; errcode_t retval; __u32 count; - if (!ext2fs_inode_has_valid_blocks2(fs, inode)) + if (!ext2fs_inode_has_valid_blocks2(fs, EXT2_INODE(inode))) return 0; pb.buf = block_buf + 3 * ctx->fs->blocksize; @@ -196,6 +200,7 @@ static int release_inode_blocks(e2fsck_t ctx, ext2_ino_t ino, pb.errcode = 0; pb.pctx = pctx; pb.last_cluster = 0; + pb.inode = inode; if (inode->i_links_count) { pb.truncating = 1; pb.truncate_block = (e2_blkcnt_t) @@ -220,15 +225,17 @@ static int release_inode_blocks(e2fsck_t ctx, ext2_ino_t ino, return 1; /* Refresh the inode since ext2fs_block_iterate may have changed it */ - e2fsck_read_inode(ctx, ino, inode, "release_inode_blocks"); + e2fsck_read_inode_full(ctx, ino, EXT2_INODE(inode), sizeof(inode), + "release_inode_blocks"); if (pb.truncated_blocks) - ext2fs_iblk_sub_blocks(fs, inode, pb.truncated_blocks); + ext2fs_iblk_sub_blocks(fs, EXT2_INODE(inode), + pb.truncated_blocks); - if (ext2fs_file_acl_block(fs, inode)) { - retval = ext2fs_adjust_ea_refcount3(fs, - ext2fs_file_acl_block(fs, inode), - block_buf, -1, &count, ino); + blk = ext2fs_file_acl_block(fs, EXT2_INODE(inode)); + if (blk) { + retval = ext2fs_adjust_ea_refcount3(fs, blk, block_buf, -1, + &count, ino); if (retval == EXT2_ET_BAD_EA_BLOCK_NUM) { retval = 0; count = 1; @@ -240,15 +247,68 @@ static int release_inode_blocks(e2fsck_t ctx, ext2_ino_t ino, return 1; } if (count == 0) { - ext2fs_block_alloc_stats2(fs, - ext2fs_file_acl_block(fs, inode), -1); + if (ctx->qctx) + quota_data_sub(ctx->qctx, inode, 0, + ctx->fs->blocksize); + ext2fs_block_alloc_stats2(fs, blk, -1); ctx->free_blocks++; } - ext2fs_file_acl_block_set(fs, inode, 0); + ext2fs_file_acl_block_set(fs, EXT2_INODE(inode), 0); } return 0; } +/* Load all quota data in preparation for orphan clearing. */ +static errcode_t e2fsck_read_all_quotas(e2fsck_t ctx) +{ + ext2_ino_t qf_ino; + enum quota_type qtype; + errcode_t retval = 0; + + if (!ext2fs_has_feature_quota(ctx->fs->super)) + return retval; + + retval = quota_init_context(&ctx->qctx, ctx->fs, 0); + if (retval) + return retval; + + for (qtype = 0 ; qtype < MAXQUOTAS; qtype++) { + qf_ino = *quota_sb_inump(ctx->fs->super, qtype); + if (qf_ino == 0) + continue; + + retval = quota_update_limits(ctx->qctx, qf_ino, qtype); + if (retval) + break; + } + if (retval) + quota_release_context(&ctx->qctx); + return retval; +} + +/* Write all the quota info to disk. */ +static errcode_t e2fsck_write_all_quotas(e2fsck_t ctx) +{ + struct problem_context pctx; + enum quota_type qtype; + + if (!ext2fs_has_feature_quota(ctx->fs->super)) + return 0; + + clear_problem_context(&pctx); + for (qtype = 0 ; qtype < MAXQUOTAS; qtype++) { + pctx.num = qtype; + pctx.errcode = quota_write_inode(ctx->qctx, 1 << qtype); + if (pctx.errcode) { + fix_problem(ctx, PR_6_WRITE_QUOTAS, &pctx); + break; + } + } + + quota_release_context(&ctx->qctx); + return pctx.errcode; +} + /* * This function releases all of the orphan inodes. It returns 1 if * it hit some error, and 0 on success. @@ -257,13 +317,20 @@ static int release_orphan_inodes(e2fsck_t ctx) { ext2_filsys fs = ctx->fs; ext2_ino_t ino, next_ino; - struct ext2_inode inode; + struct ext2_inode_large inode; struct problem_context pctx; char *block_buf; if ((ino = fs->super->s_last_orphan) == 0) return 0; + clear_problem_context(&pctx); + pctx.errcode = e2fsck_read_all_quotas(ctx); + if (pctx.errcode) { + fix_problem(ctx, PR_0_QUOTA_INIT_CTX, &pctx); + return 1; + } + /* * Win or lose, we won't be using the head of the orphan inode * list again. @@ -276,15 +343,18 @@ static int release_orphan_inodes(e2fsck_t ctx) * list, since the orphan list can't be trusted; and we're * going to be running a full e2fsck run anyway... */ - if (fs->super->s_state & EXT2_ERROR_FS) + if (fs->super->s_state & EXT2_ERROR_FS) { + if (ctx->qctx) + quota_release_context(&ctx->qctx); return 0; + } if ((ino < EXT2_FIRST_INODE(fs->super)) || (ino > fs->super->s_inodes_count)) { clear_problem_context(&pctx); pctx.ino = ino; fix_problem(ctx, PR_0_ORPHAN_ILLEGAL_HEAD_INODE, &pctx); - return 1; + goto err_qctx; } block_buf = (char *) e2fsck_allocate_memory(ctx, fs->blocksize * 4, @@ -292,10 +362,11 @@ static int release_orphan_inodes(e2fsck_t ctx) e2fsck_read_bitmaps(ctx); while (ino) { - e2fsck_read_inode(ctx, ino, &inode, "release_orphan_inodes"); + e2fsck_read_inode_full(ctx, ino, EXT2_INODE(&inode), + sizeof(inode), "release_orphan_inodes"); clear_problem_context(&pctx); pctx.ino = ino; - pctx.inode = &inode; + pctx.inode = EXT2_INODE(&inode); pctx.str = inode.i_links_count ? _("Truncating") : _("Clearing"); @@ -307,13 +378,15 @@ static int release_orphan_inodes(e2fsck_t ctx) (next_ino > fs->super->s_inodes_count))) { pctx.ino = next_ino; fix_problem(ctx, PR_0_ORPHAN_ILLEGAL_INODE, &pctx); - goto return_abort; + goto err_buf; } if (release_inode_blocks(ctx, ino, &inode, block_buf, &pctx)) - goto return_abort; + goto err_buf; if (!inode.i_links_count) { + if (ctx->qctx) + quota_data_inodes(ctx->qctx, &inode, ino, -1); ext2fs_inode_alloc_stats2(fs, ino, -1, LINUX_S_ISDIR(inode.i_mode)); ctx->free_inodes++; @@ -321,13 +394,21 @@ static int release_orphan_inodes(e2fsck_t ctx) } else { inode.i_dtime = 0; } - e2fsck_write_inode(ctx, ino, &inode, "delete_file"); + e2fsck_write_inode_full(ctx, ino, EXT2_INODE(&inode), + sizeof(inode), "delete_file"); ino = next_ino; } ext2fs_free_mem(&block_buf); + pctx.errcode = e2fsck_write_all_quotas(ctx); + if (pctx.errcode) + goto err; return 0; -return_abort: +err_buf: ext2fs_free_mem(&block_buf); +err_qctx: + if (ctx->qctx) + quota_release_context(&ctx->qctx); +err: return 1; } diff --git a/lib/support/mkquota.c b/lib/support/mkquota.c index e65c95b..efc37cb 100644 --- a/lib/support/mkquota.c +++ b/lib/support/mkquota.c @@ -516,6 +516,7 @@ struct scan_dquots_data { dict_t *quota_dict; int update_limits; /* update limits from disk */ int update_usage; + int check_consistency; int usage_is_inconsistent; }; @@ -533,8 +534,9 @@ static int scan_dquots_callback(struct dquot *dquot, void *cb_data) print_dquot("dsk", dquot); /* Check if there is inconsistency */ - if (dq->dq_dqb.dqb_curspace != dquot->dq_dqb.dqb_curspace || - dq->dq_dqb.dqb_curinodes != dquot->dq_dqb.dqb_curinodes) { + if (scan_data->check_consistency && + (dq->dq_dqb.dqb_curspace != dquot->dq_dqb.dqb_curspace || + dq->dq_dqb.dqb_curinodes != dquot->dq_dqb.dqb_curinodes)) { scan_data->usage_is_inconsistent = 1; fprintf(stderr, "[QUOTA WARNING] Usage inconsistent for ID %u:" "actual (%lld, %lld) != expected (%lld, %lld)\n", @@ -568,8 +570,9 @@ static errcode_t quota_read_all_dquots(struct quota_handle *qh, struct scan_dquots_data scan_data; scan_data.quota_dict = qctx->quota_dict[qh->qh_type]; - scan_data.update_limits = update_limits; - scan_data.update_usage = 0; + scan_data.check_consistency = 0; + scan_data.update_limits = 0; + scan_data.update_usage = 1; return qh->qh_ops->scan_dquots(qh, scan_dquots_callback, &scan_data); } @@ -659,6 +662,7 @@ errcode_t quota_compare_and_update(quota_ctx_t qctx, enum quota_type qtype, scan_data.quota_dict = qctx->quota_dict[qtype]; scan_data.update_limits = 1; scan_data.update_usage = 0; + scan_data.check_consistency = 1; scan_data.usage_is_inconsistent = 0; err = qh.qh_ops->scan_dquots(&qh, scan_dquots_callback, &scan_data); if (err) { diff --git a/tests/f_orphquot/expect b/tests/f_orphquot/expect new file mode 100644 index 0000000..90a7813 --- /dev/null +++ b/tests/f_orphquot/expect @@ -0,0 +1,10 @@ +Clearing orphaned inode 12 (uid=0, gid=0, mode=0100644, size=3842048) +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information + +test_filesystem: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesystem: 11/512 files (9.1% non-contiguous), 1070/2048 blocks +Exit status is 0 diff --git a/tests/f_orphquot/image.bz2 b/tests/f_orphquot/image.bz2 new file mode 100644 index 0000000000000000000000000000000000000000..44c831852b0740019705c1ffbafa3d37cc561da8 GIT binary patch literal 1327 zcmV+~1 zXg~na8Z^}8U|8G00004z$O3y z044%334j3500004!fB%+r=TVP002c2OoE=KN2!uz$l4&tVjiPRnKBvyqaX}`(?Osb zX{LiiMuvbI00ThDrkI9-gsStIa}90{i-o$^7K0&Fnl9v$_;F03eab6_f5x#K3=KaF zQ*~*u%QiQ&Qe^9>Ziq-jW<+WByf<$9k6!f8P~E+8+J^C$TbAECr+Q!unGTl)FSayf zd4mzQ(W!iAD$TM%gJAb=^|Tg6pp0z^MJZp}M z1JcvVyY18apMiIyk;CoH8RNGpg0 zAfO#HnNBxN;}K4G%$Qds2I7IBU_&6n&|q=LNaHxIL!4n-3NnK7D+rdz1poj7+YMn` zDQF;2p#n0%jbu~}V+@Ki*fP<9LstkF7*(Lc2T20mVH<1=u&WFRLkfkVg>&pR)gZ9K zx5!yx1%^2i6+i$0lBbAqGtI4FMl%5{A*d?8(UYiniqs~V`5gmccz_%3*oI0FhE^Y^ zf=$*6?1_B)*(yI?9!zK-mYBF+YEZ+4pineT4oo3ze#iM90hJqVVupqAU5iJSgu`LN zV~q%9&HVncsrB4Kii9xstOGE?02!?5xFDl;FN`hog)Oup9HHOqYC;XQXfubk_YG6Q#l1RKVbf!O&f?0|I>j>Z_eUk!0*UG z0q`KG-U56NoMHGFn$Rtk<~qL`^DECnE8gi$Q8>RDhNJ3$I7t;W?+fA|1#AW`;+cz1PgL $TMPFILE + +$FSCK -f -y -N test_filesystem $TMPFILE > $OUT.new 2>&1 +status=$? +echo Exit status is $status >> $OUT.new +sed -f $cmd_dir/filter.sed $OUT.new >> $OUT +rm -f $OUT.new + +cmp -s $OUT $EXP +status=$? + +if [ "$status" = 0 ] ; then + echo "$test_name: $test_description: ok" + touch $test_name.ok +else + echo "$test_name: $test_description: failed" + diff $DIFF_OPTS $EXP $OUT > $test_name.failed + rm -f tmp_expect +fi + +unset IMAGE FSCK_OPT OUT EXP