From patchwork Sun Sep 7 19:33:46 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Basile X-Patchwork-Id: 386740 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from hemlock.osuosl.org (hemlock.osuosl.org [140.211.166.133]) by ozlabs.org (Postfix) with ESMTP id 27AD71400BB for ; Mon, 8 Sep 2014 05:31:43 +1000 (EST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 6287094057; Sun, 7 Sep 2014 19:31:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FR8xeKRY8k+b; Sun, 7 Sep 2014 19:31:39 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id EA0BD93D0C; Sun, 7 Sep 2014 19:31:38 +0000 (UTC) X-Original-To: uclibc@lists.busybox.net Delivered-To: uclibc@osuosl.org Received: from fraxinus.osuosl.org (fraxinus.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 832891C2DDD for ; Sun, 7 Sep 2014 19:31:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 7D0A2A0F93 for ; Sun, 7 Sep 2014 19:31:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hpn5UIwaO-Q6 for ; Sun, 7 Sep 2014 19:31:37 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from virtual.dyc.edu (mail.virtual.dyc.edu [67.222.116.22]) by fraxinus.osuosl.org (Postfix) with ESMTP id 63805A0F89 for ; Sun, 7 Sep 2014 19:31:35 +0000 (UTC) Received: from yellow.dyc.edu (unknown [192.168.3.7]) by virtual.dyc.edu (Postfix) with ESMTP id D29F37E00E5; Sun, 7 Sep 2014 15:31:28 -0400 (EDT) Received: by yellow.dyc.edu (Postfix, from userid 1000) id 36D723198057; Sun, 7 Sep 2014 15:33:53 -0400 (EDT) From: basile@opensource.dyc.edu To: uclibc@uclibc.org Subject: [PATCH] libc: add fallocate() and fallocate64() Date: Sun, 7 Sep 2014 15:33:46 -0400 Message-Id: <1410118426-20163-1-git-send-email-basile@opensource.dyc.edu> X-Mailer: git-send-email 1.8.5.5 Cc: blueness@gentoo.org X-BeenThere: uclibc@uclibc.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion and development of uClibc \(the embedded C library\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: uclibc-bounces@uclibc.org Sender: uclibc-bounces@uclibc.org From: "Anthony G. Basile" We add the Linux-specific function fallocate() which allows the user to directly manipulate allocate space for a file. fallocate() can operate in different modes, but the default mode is equivalent to posix_fallocate() which is specified in POSIX.1. Recent releases of e2fsprogs 1.42.11 and above expect fallocate64() to be available. Signed-off-by: Anthony G. Basile --- extra/Configs/Config.in | 4 +- include/fcntl.h | 31 +++++ libc/sysdeps/linux/common/Makefile.in | 11 ++ libc/sysdeps/linux/common/fallocate.c | 51 ++++++++ libc/sysdeps/linux/common/fallocate64.c | 43 +++++++ libc/sysdeps/linux/common/posix_fallocate.c | 23 +--- libc/sysdeps/linux/common/posix_fallocate64.c | 11 +- test/.gitignore | 2 + test/unistd/Makefile.in | 18 ++- test/unistd/tst-fallocate.c | 163 ++++++++++++++++++++++++++ test/unistd/tst-fallocate64.c | 2 + 11 files changed, 328 insertions(+), 31 deletions(-) create mode 100644 libc/sysdeps/linux/common/fallocate.c create mode 100644 libc/sysdeps/linux/common/fallocate64.c create mode 100644 test/unistd/tst-fallocate.c create mode 100644 test/unistd/tst-fallocate64.c diff --git a/extra/Configs/Config.in b/extra/Configs/Config.in index cdb2cb1..32180f7 100644 --- a/extra/Configs/Config.in +++ b/extra/Configs/Config.in @@ -1014,8 +1014,8 @@ config UCLIBC_LINUX_SPECIFIC default y help accept4(), bdflush(), - capget(), capset(), eventfd(), fstatfs(), - inotify_*(), ioperm(), iopl(), + capget(), capset(), eventfd(), fallocate(), + fstatfs(), inotify_*(), ioperm(), iopl(), madvise(), modify_ldt(), pipe2(), personality(), prctl()/arch_prctl(), pivot_root(), modify_ldt(), ppoll(), readahead(), reboot(), remap_file_pages(), diff --git a/include/fcntl.h b/include/fcntl.h index c749ad5..eb72ce4 100644 --- a/include/fcntl.h +++ b/include/fcntl.h @@ -239,6 +239,37 @@ extern int posix_fallocate64 (int __fd, __off64_t __offset, __off64_t __len); # endif #endif +#if (defined __UCLIBC_LINUX_SPECIFIC__ && defined __USE_GNU) || defined _LIBC +/* Reserve storage for the data of a file associated with FD. This function + is Linux-specific. For the portable version, use posix_fallocate(). + Unlike the latter, fallocate can operate in different modes. The default + mode = 0 is equivalent to posix_fallocate(). + + Note: These declarations are used in posix_fallocate.c and posix_fallocate64.c, + so we expose them internally. */ + +/* Flags for fallocate's mode. */ +# define FALLOC_FL_KEEP_SIZE 1 /* Don't extend size of file + even if offset + len is + greater than file size. */ +# define FALLOC_FL_PUNCH_HOLE 2 /* Create a hole in the file. */ + +# ifndef __USE_FILE_OFFSET64 +extern int fallocate (int __fd, int __mode, __off_t __offset, __off_t __len); +# else +# ifdef __REDIRECT +extern int __REDIRECT (fallocate, (int __fd, int __mode, __off64_t __offset, + __off64_t __len), + fallocate64); +# else +# define fallocate fallocate64 +# endif +# endif +# ifdef __USE_LARGEFILE64 +extern int fallocate64 (int __fd, int __mode, __off64_t __offset, __off64_t __len); +# endif +#endif + __END_DECLS #endif /* fcntl.h */ diff --git a/libc/sysdeps/linux/common/Makefile.in b/libc/sysdeps/linux/common/Makefile.in index dc3a4b7..e8a9e75 100644 --- a/libc/sysdeps/linux/common/Makefile.in +++ b/libc/sysdeps/linux/common/Makefile.in @@ -62,6 +62,17 @@ CSRC-$(UCLIBC_LINUX_SPECIFIC) += \ vmsplice.c CSRC-$(if $(findstring yy,$(UCLIBC_LINUX_SPECIFIC)$(UCLIBC_HAS_LFS)),y) += \ sendfile64.c +# posix_fallocate() needs the internal function _fallocate() defined in fallocate.c +# and posix_fallocate64() needs _fallocate64() in fallocate64.c if LFS is enabled. +# So we do not remove them even if we want just posix_fallocate() or posix_fallocate64(). +ifeq ($(findstring y,$(UCLIBC_LINUX_SPECIFIC)$(UCLIBC_HAS_ADVANCED_REALTIME)),y) +ifneq ($(findstring y,$(UCLIBC_HAS_LFS)),y) +CSRC- += fallocate64.c +endif +else +CSRC- += fallocate.c +CSRC- += fallocate64.c +endif # NPTL needs these internally: madvise.c CSRC-$(findstring y,$(UCLIBC_LINUX_SPECIFIC)$(UCLIBC_HAS_THREADS_NATIVE)) += madvise.c ifeq ($(UCLIBC_HAS_THREADS_NATIVE),y) diff --git a/libc/sysdeps/linux/common/fallocate.c b/libc/sysdeps/linux/common/fallocate.c new file mode 100644 index 0000000..c16cd17 --- /dev/null +++ b/libc/sysdeps/linux/common/fallocate.c @@ -0,0 +1,51 @@ +/* vi: set sw=4 ts=4: */ +/* + * fallocate() for uClibc - Based off of posix_fallocate() by Erik Andersen + * http://www.opengroup.org/onlinepubs/9699919799/functions/posix_fallocate.html + * + * Copyright (C) 2000-2006 Erik Andersen + * + * Licensed under the LGPL v2.1 or later, see the file COPYING.LIB in this tarball. + */ + +#include +#include +#include +#include + +#if defined __NR_fallocate +int attribute_hidden _fallocate(int fd, int mode, __off_t offset, __off_t len) +{ + int ret; + +# if __WORDSIZE == 32 + uint32_t off_low = offset; + uint32_t len_low = len; + /* may assert that these >>31 are 0 */ + uint32_t zero = 0; + INTERNAL_SYSCALL_DECL(err); + ret = (int) (INTERNAL_SYSCALL(fallocate, err, 6, fd, mode, + __LONG_LONG_PAIR (zero, off_low), + __LONG_LONG_PAIR (zero, len_low))); +# elif __WORDSIZE == 64 + INTERNAL_SYSCALL_DECL(err); + ret = (int) (INTERNAL_SYSCALL(fallocate, err, 4, fd, mode, offset, len)); +# else +# error your machine is neither 32 bit or 64 bit ... it must be magical +# endif + if (unlikely(INTERNAL_SYSCALL_ERROR_P (ret, err))) + return INTERNAL_SYSCALL_ERRNO (ret, err); + return 0; +} + +# if defined __UCLIBC_LINUX_SPECIFIC__ && defined __USE_GNU +int fallocate(int fd, int mode, __off_t offset, __off_t len) +{ + return _fallocate(fd, mode, offset, len); +} +# if defined __UCLIBC_HAS_LFS__ && __WORDSIZE == 64 +strong_alias(fallocate,fallocate64) +# endif +# endif + +#endif diff --git a/libc/sysdeps/linux/common/fallocate64.c b/libc/sysdeps/linux/common/fallocate64.c new file mode 100644 index 0000000..e81c69a --- /dev/null +++ b/libc/sysdeps/linux/common/fallocate64.c @@ -0,0 +1,43 @@ +/* vi: set sw=4 ts=4: */ +/* + * fallocate() for uClibc - based off posix_fallocate() by Erik Andersen + * http://www.opengroup.org/onlinepubs/9699919799/functions/posix_fallocate.html + * + * Copyright (C) 2000-2006 Erik Andersen + * + * Licensed under the LGPL v2.1 or later, see the file COPYING.LIB in this tarball. + */ + +#include + +#include +#include +#include + +#if defined __NR_fallocate + +# if __WORDSIZE == 64 +/* Can use normal fallocate() */ +# elif __WORDSIZE == 32 +int attribute_hidden _fallocate64(int fd, int mode, __off64_t offset, __off64_t len) +{ + int ret; + INTERNAL_SYSCALL_DECL(err); + ret = (int) (INTERNAL_SYSCALL(fallocate, err, 6, fd, mode, + OFF64_HI_LO (offset), OFF64_HI_LO (len))); + if (unlikely(INTERNAL_SYSCALL_ERROR_P (ret, err))) + return INTERNAL_SYSCALL_ERRNO (ret, err); + return 0; +} + +# if defined __UCLIBC_LINUX_SPECIFIC__ && defined __USE_GNU +int fallocate64(int fd, int mode, __off64_t offset, __off64_t len) +{ + return _fallocate64(fd, mode, offset, len); +} +# endif + +# else +# error your machine is neither 32 bit or 64 bit ... it must be magical +# endif +#endif diff --git a/libc/sysdeps/linux/common/posix_fallocate.c b/libc/sysdeps/linux/common/posix_fallocate.c index 9aaa6ce..9e9b09c 100644 --- a/libc/sysdeps/linux/common/posix_fallocate.c +++ b/libc/sysdeps/linux/common/posix_fallocate.c @@ -13,29 +13,12 @@ #include #include +extern __typeof(fallocate) _fallocate attribute_hidden; + #if defined __NR_fallocate int posix_fallocate(int fd, __off_t offset, __off_t len) { - int ret; - -# if __WORDSIZE == 32 - uint32_t off_low = offset; - uint32_t len_low = len; - /* may assert that these >>31 are 0 */ - uint32_t zero = 0; - INTERNAL_SYSCALL_DECL(err); - ret = (int) (INTERNAL_SYSCALL(fallocate, err, 6, fd, 0, - __LONG_LONG_PAIR (zero, off_low), - __LONG_LONG_PAIR (zero, len_low))); -# elif __WORDSIZE == 64 - INTERNAL_SYSCALL_DECL(err); - ret = (int) (INTERNAL_SYSCALL(fallocate, err, 4, fd, 0, offset, len)); -# else -# error your machine is neither 32 bit or 64 bit ... it must be magical -#endif - if (unlikely(INTERNAL_SYSCALL_ERROR_P (ret, err))) - return INTERNAL_SYSCALL_ERRNO (ret, err); - return 0; + return _fallocate(fd, 0, offset, len); } # if defined __UCLIBC_HAS_LFS__ && __WORDSIZE == 64 strong_alias(posix_fallocate,posix_fallocate64) diff --git a/libc/sysdeps/linux/common/posix_fallocate64.c b/libc/sysdeps/linux/common/posix_fallocate64.c index 76dc9b8..d5875ca 100644 --- a/libc/sysdeps/linux/common/posix_fallocate64.c +++ b/libc/sysdeps/linux/common/posix_fallocate64.c @@ -13,20 +13,15 @@ #include #include -#if defined __NR_fallocate +extern __typeof(fallocate64) _fallocate64 attribute_hidden; +#if defined __NR_fallocate # if __WORDSIZE == 64 /* Can use normal posix_fallocate() */ # elif __WORDSIZE == 32 int posix_fallocate64(int fd, __off64_t offset, __off64_t len) { - int ret; - INTERNAL_SYSCALL_DECL(err); - ret = (int) (INTERNAL_SYSCALL(fallocate, err, 6, fd, 0, - OFF64_HI_LO (offset), OFF64_HI_LO (len))); - if (unlikely(INTERNAL_SYSCALL_ERROR_P (ret, err))) - return INTERNAL_SYSCALL_ERRNO (ret, err); - return 0; + return _fallocate64(fd, 0, offset, len); } # else # error your machine is neither 32 bit or 64 bit ... it must be magical diff --git a/test/.gitignore b/test/.gitignore index 85161da..8f32031 100644 --- a/test/.gitignore +++ b/test/.gitignore @@ -320,6 +320,8 @@ unistd/getcwd unistd/getopt unistd/getopt_long unistd/tstgetopt +unistd/tst-fallocate +unistd/tst-fallocate64 unistd/tst-posix_fallocate unistd/tst-posix_fallocate64 unistd/tst-preadwrite diff --git a/test/unistd/Makefile.in b/test/unistd/Makefile.in index cfef22e..0aaefb5 100644 --- a/test/unistd/Makefile.in +++ b/test/unistd/Makefile.in @@ -1,12 +1,28 @@ # uClibc unistd tests # Licensed under the LGPL v2.1, see the file COPYING.LIB in this tarball. +# If LFS is not set, get rid of all *64 tests up front ifeq ($(UCLIBC_HAS_LFS),) -TESTS_DISABLED := tst-preadwrite64 tst-posix_fallocate64 +TESTS_DISABLED := tst-preadwrite64 tst-posix_fallocate64 tst-fallocate64 endif + +# If we don't have LINUX_SPECIFIC, then get rid of tst-fallocate, and check to +# make sure we didn't miss removing tst-fallocate64 above because LFS is enabled +ifeq ($(UCLIBC_LINUX_SPECIFIC),) +TESTS_DISABLED += tst-fallocate +ifeq ($(UCLIBC_HAS_LFS),y) +TESTS_DISABLED += tst-fallocate64 +endif +endif + +# The logic is similar for HAS_ADVANCED_REALTIME and tst-posix_fallocate/tst-posix_fallocate64 ifeq ($(UCLIBC_HAS_ADVANCED_REALTIME),) TESTS_DISABLED += tst-posix_fallocate +ifeq ($(UCLIBC_HAS_LFS),y) +TESTS_DISABLED += tst-posix_fallocate64 +endif endif + OPTS_getopt := -abcXXX -9 OPTS_getopt_long := --add XXX --delete YYY --verbose ifeq ($(UCLIBC_HAS_GNU_GETOPT),y) diff --git a/test/unistd/tst-fallocate.c b/test/unistd/tst-fallocate.c new file mode 100644 index 0000000..2b15596 --- /dev/null +++ b/test/unistd/tst-fallocate.c @@ -0,0 +1,163 @@ +#include +#include + +#ifndef TST_FALLOCATE64 +# define stat64 stat +# define fstat64 fstat +# else +# ifndef O_LARGEFILE +# error no O_LARGEFILE but you want to test with LFS enabled +# endif +#endif + +static void do_prepare (void); +#define PREPARE(argc, argv) do_prepare () +static int do_test (void); +#define TEST_FUNCTION do_test () +#include + +static int fd; +static void +do_prepare (void) +{ + fd = create_temp_file ("tst-fallocate.", NULL); + if (fd == -1) + { + printf ("cannot create temporary file: %m\n"); + exit (1); + } +} + + +static int +do_test (void) +{ + struct stat64 st; + + if (fstat64 (fd, &st) != 0) + { + puts ("1st fstat failed"); + return 1; + } + + if (st.st_size != 0) + { + puts ("file not created with size 0"); + return 1; + } + + /* This is the default mode which is identical to posix_fallocate(). + Note: we need a few extra blocks for FALLOC_FL_PUNCH_HOLE below. + While block sizes vary, we'll assume eight 4K blocks for good measure. */ + if (fallocate (fd, 0, 8 * 4096, 128) != 0) + { + puts ("1st fallocate call failed"); + return 1; + } + + if (fstat64 (fd, &st) != 0) + { + puts ("2nd fstat failed"); + return 1; + } + + if (st.st_size != 8 * 4096 + 128) + { + printf ("file size after 1st fallocate call is %llu, expected %u\n", + (unsigned long long int) st.st_size, 8u * 4096u + 128u); + return 1; + } + + /* Without FALLOC_FL_KEEP_SIZE, this would increaste the size of the file. */ + if (fallocate (fd, FALLOC_FL_KEEP_SIZE, 0, 16 * 4096) != 0) + { + puts ("2nd fallocate call failed"); + return 1; + } + + if (fstat64 (fd, &st) != 0) + { + puts ("3rd fstat failed"); + return 1; + } + + if (st.st_size != 8 * 4096 + 128) + { + printf ("file size changed in 2nd fallocate call to %llu, expected %u\n", + (unsigned long long int) st.st_size, 8u * 4096u + 128u); + return 1; + } + + /* Let's fill up the first eight 4k blocks with 'x' to force some allocations. */ + int c; + char garbage[4096]; + memset(garbage, 'x', 4096); + for(c=0; c < 8; c++) + if(write(fd, garbage, 4096) == -1) + { + puts ("write failed"); + return 1; + } + + if (fstat64 (fd, &st) != 0) + { + puts ("4th fstat failed"); + return 1; + } + + blkcnt_t blksb4 = st.st_blocks; + + /* Let's punch a hole in the entire file, turning it effectively into a sparse file. */ + if (fallocate (fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, 0, 8 * 4096 + 128) != 0) + { + puts ("3rd fallocate call failed"); + return 1; + } + + if (fstat64 (fd, &st) != 0) + { + puts ("5th fstat failed"); + return 1; + } + + if (st.st_size != 8 * 4096 + 128) + { + printf ("file size after 3rd fallocate call is %llu, expected %u\n", + (unsigned long long int) st.st_size, 8u * 4096u + 128u); + return 1; + } + + /* The number of allocated blocks should decrease. I hope this works on + all filesystems! */ + if (st.st_blocks >= blksb4) + { + printf ("number of blocks after 3rd fallocate call is %lu, expected less than %lu\n", + (unsigned long int) st.st_blocks, blksb4); + return 1; + } + +#ifdef TST_FALLOCATE64 + /* We'll just do a mode = 0 test for fallocate64() */ + if (fallocate64 (fd, 0, 4097ULL, 4294967295ULL + 2ULL) != 0) + { + puts ("1st fallocate64 call failed"); + return 1; + } + + if (fstat64 (fd, &st) != 0) + { + puts ("6th fstat failed"); + return 1; + } + + if (st.st_size != 4097ULL + 4294967295ULL + 2ULL) + { + printf ("file size after 1st fallocate64 call is %llu, expected %llu\n", + (unsigned long long int) st.st_size, 4097ULL + 4294967295ULL + 2ULL); + return 1; + } +#endif + close (fd); + + return 0; +} diff --git a/test/unistd/tst-fallocate64.c b/test/unistd/tst-fallocate64.c new file mode 100644 index 0000000..720428f --- /dev/null +++ b/test/unistd/tst-fallocate64.c @@ -0,0 +1,2 @@ +#define TST_FALLOCATE64 +#include "tst-fallocate.c"