From patchwork Thu Mar 5 17:17:58 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrea Arcangeli X-Patchwork-Id: 446826 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 176E114016B for ; Fri, 6 Mar 2015 04:24:32 +1100 (AEDT) Received: from localhost ([::1]:53202 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YTZVe-0006zS-0f for incoming@patchwork.ozlabs.org; Thu, 05 Mar 2015 12:24:30 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40472) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YTZQX-00049I-IG for qemu-devel@nongnu.org; Thu, 05 Mar 2015 12:19:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YTZQW-0007fd-JR for qemu-devel@nongnu.org; Thu, 05 Mar 2015 12:19:13 -0500 Received: from mx1.redhat.com ([209.132.183.28]:59768) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YTZQW-0007f6-CR for qemu-devel@nongnu.org; Thu, 05 Mar 2015 12:19:12 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t25HIDJJ031707 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Thu, 5 Mar 2015 12:18:13 -0500 Received: from mail.random (ovpn-116-22.ams2.redhat.com [10.36.116.22]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t25HIBiW008109 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 5 Mar 2015 12:18:12 -0500 From: Andrea Arcangeli To: qemu-devel@nongnu.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-api@vger.kernel.org, Android Kernel Team Date: Thu, 5 Mar 2015 18:17:58 +0100 Message-Id: <1425575884-2574-16-git-send-email-aarcange@redhat.com> In-Reply-To: <1425575884-2574-1-git-send-email-aarcange@redhat.com> References: <1425575884-2574-1-git-send-email-aarcange@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: Robert Love , Dave Hansen , Jan Kara , Neil Brown , Stefan Hajnoczi , Andrew Jones , Sanidhya Kashyap , KOSAKI Motohiro , Michel Lespinasse , Taras Glek , zhang.zhanghailiang@huawei.com, Pavel Emelyanov , Hugh Dickins , Mel Gorman , Sasha Levin , "Dr. David Alan Gilbert" , "Huangpeng \(Peter\)" , Andres Lagar-Cavilla , Christopher Covington , Anthony Liguori , Paolo Bonzini , "Kirill A. Shutemov" , Keith Packard , Wenchao Xia , Juan Quintela , Andy Lutomirski , Minchan Kim , Dmitry Adamushko , Johannes Weiner , Mike Hommey , Andrew Morton , Linus Torvalds , Peter Feiner Subject: [Qemu-devel] [PATCH 15/21] userfaultfd: UFFDIO_COPY and UFFDIO_ZEROPAGE X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org These two ioctl allows to either atomically copy or to map zeropages into the virtual address space. This is used by the thread that opened the userfaultfd to resolve the userfaults. Signed-off-by: Andrea Arcangeli --- fs/userfaultfd.c | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 6b31967..6230f22 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -798,6 +798,100 @@ out: return ret; } +static int userfaultfd_copy(struct userfaultfd_ctx *ctx, + unsigned long arg) +{ + __s64 ret; + struct uffdio_copy uffdio_copy; + struct uffdio_copy __user *user_uffdio_copy; + struct userfaultfd_wake_range range; + + user_uffdio_copy = (struct uffdio_copy __user *) arg; + + ret = -EFAULT; + if (copy_from_user(&uffdio_copy, user_uffdio_copy, + /* don't copy "copy" and "wake" last field */ + sizeof(uffdio_copy)-sizeof(__s64)*2)) + goto out; + + ret = validate_range(ctx->mm, uffdio_copy.dst, uffdio_copy.len); + if (ret) + goto out; + /* + * double check for wraparound just in case. copy_from_user() + * will later check uffdio_copy.src + uffdio_copy.len to fit + * in the userland range. + */ + ret = -EINVAL; + if (uffdio_copy.src + uffdio_copy.len <= uffdio_copy.src) + goto out; + if (uffdio_copy.mode & ~UFFDIO_COPY_MODE_DONTWAKE) + goto out; + + ret = mcopy_atomic(ctx->mm, uffdio_copy.dst, uffdio_copy.src, + uffdio_copy.len); + if (unlikely(put_user(ret, &user_uffdio_copy->copy))) + return -EFAULT; + if (ret < 0) + goto out; + BUG_ON(!ret); + /* len == 0 would wake all */ + range.len = ret; + if (!(uffdio_copy.mode & UFFDIO_COPY_MODE_DONTWAKE)) { + range.start = uffdio_copy.dst; + ret = wake_userfault(ctx, &range); + if (unlikely(put_user(ret, &user_uffdio_copy->wake))) + return -EFAULT; + } + ret = range.len == uffdio_copy.len ? 0 : -EAGAIN; +out: + return ret; +} + +static int userfaultfd_zeropage(struct userfaultfd_ctx *ctx, + unsigned long arg) +{ + __s64 ret; + struct uffdio_zeropage uffdio_zeropage; + struct uffdio_zeropage __user *user_uffdio_zeropage; + struct userfaultfd_wake_range range; + + user_uffdio_zeropage = (struct uffdio_zeropage __user *) arg; + + ret = -EFAULT; + if (copy_from_user(&uffdio_zeropage, user_uffdio_zeropage, + /* don't copy "zeropage" and "wake" last field */ + sizeof(uffdio_zeropage)-sizeof(__s64)*2)) + goto out; + + ret = validate_range(ctx->mm, uffdio_zeropage.range.start, + uffdio_zeropage.range.len); + if (ret) + goto out; + ret = -EINVAL; + if (uffdio_zeropage.mode & ~UFFDIO_ZEROPAGE_MODE_DONTWAKE) + goto out; + + ret = mfill_zeropage(ctx->mm, uffdio_zeropage.range.start, + uffdio_zeropage.range.len); + if (unlikely(put_user(ret, &user_uffdio_zeropage->zeropage))) + return -EFAULT; + if (ret < 0) + goto out; + /* len == 0 would wake all */ + BUG_ON(!ret); + range.len = ret; + if (!(uffdio_zeropage.mode & UFFDIO_ZEROPAGE_MODE_DONTWAKE)) { + range.start = uffdio_zeropage.range.start; + ret = wake_userfault(ctx, &range); + if (unlikely(put_user(ret, &user_uffdio_zeropage->wake))) + return -EFAULT; + } + ret = range.len == uffdio_zeropage.range.len ? 0 : -EAGAIN; +out: + return ret; +} + /* * userland asks for a certain API version and we return which bits * and ioctl commands are implemented in this kernel for such API @@ -855,6 +949,12 @@ static long userfaultfd_ioctl(struct file *file, unsigned cmd, case UFFDIO_WAKE: ret = userfaultfd_wake(ctx, arg); break; + case UFFDIO_COPY: + ret = userfaultfd_copy(ctx, arg); + break; + case UFFDIO_ZEROPAGE: + ret = userfaultfd_zeropage(ctx, arg); + break; } return ret; }