From patchwork Tue Aug 10 09:28:53 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Bader X-Patchwork-Id: 61353 X-Patchwork-Delegate: stefan.bader@canonical.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from chlorine.canonical.com (chlorine.canonical.com [91.189.94.204]) by ozlabs.org (Postfix) with ESMTP id 93FA5B70CE for ; Tue, 10 Aug 2010 19:29:17 +1000 (EST) Received: from localhost ([127.0.0.1] helo=chlorine.canonical.com) by chlorine.canonical.com with esmtp (Exim 4.69) (envelope-from ) id 1Oil8x-00055a-Vf; Tue, 10 Aug 2010 10:29:12 +0100 Received: from adelie.canonical.com ([91.189.90.139]) by chlorine.canonical.com with esmtp (Exim 4.69) (envelope-from ) id 1Oil8v-00054F-9t for kernel-team@lists.ubuntu.com; Tue, 10 Aug 2010 10:29:09 +0100 Received: from hutte.canonical.com ([91.189.90.181]) by adelie.canonical.com with esmtp (Exim 4.69 #1 (Debian)) id 1Oil8v-00017H-7r; Tue, 10 Aug 2010 10:29:09 +0100 Received: from p5b2e54bf.dip.t-dialin.net ([91.46.84.191] helo=canonical.com) by hutte.canonical.com with esmtpsa (TLS-1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.69) (envelope-from ) id 1Oil8u-00053B-T8; Tue, 10 Aug 2010 10:29:09 +0100 From: Stefan Bader To: kernel-team@lists.ubuntu.com, stable@kernel.org Subject: [PATCH 05/16] writeback: fix writeback completion notifications Date: Tue, 10 Aug 2010 11:28:53 +0200 Message-Id: <1281432544-20831-6-git-send-email-stefan.bader@canonical.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1281432544-20831-1-git-send-email-stefan.bader@canonical.com> References: <1281432544-20831-1-git-send-email-stefan.bader@canonical.com> Cc: axboe@kernel.dk X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.9 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: kernel-team-bounces@lists.ubuntu.com Errors-To: kernel-team-bounces@lists.ubuntu.com From: Christoph Hellwig BugLink: http://bugs.launchpad.net/bugs/585092 commit 7f0e7bed936a0c422641a046551829a01341dd80 upstream The code dealing with bdi_work->state and completion of a bdi_work is a major mess currently. This patch makes sure we directly use one set of flags to deal with it, and use it consistently, which means: - always notify about completion from the rcu callback. We only ever wait for it from on-stack callers, so this simplification does not even cause a theoretical slowdown currently. It also makes sure we don't miss out on the notification if we ever add other callers to wait for it. - make earlier completion notification depending on the on-stack allocation, not the sync mode. If we introduce new callers that want to do WB_SYNC_NONE writeback from on-stack callers this will be nessecary. Also rename bdi_wait_on_work_clear to bdi_wait_on_work_done and inline a few small functions into their only caller to make the code understandable. Signed-off-by: Christoph Hellwig Signed-off-by: Jens Axboe Signed-off-by: Stefan Bader --- fs/fs-writeback.c | 65 ++++++++++++---------------------------------------- 1 files changed, 15 insertions(+), 50 deletions(-) diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index 478dec1..1538e44 100644 --- a/fs/fs-writeback.c +++ b/fs/fs-writeback.c @@ -62,24 +62,16 @@ struct bdi_work { }; enum { - WS_USED_B = 0, - WS_ONSTACK_B, + WS_INPROGRESS = 0, + WS_ONSTACK, }; -#define WS_USED (1 << WS_USED_B) -#define WS_ONSTACK (1 << WS_ONSTACK_B) - -static inline bool bdi_work_on_stack(struct bdi_work *work) -{ - return test_bit(WS_ONSTACK_B, &work->state); -} - static inline void bdi_work_init(struct bdi_work *work, struct wb_writeback_args *args) { INIT_RCU_HEAD(&work->rcu_head); work->args = *args; - work->state = WS_USED; + __set_bit(WS_INPROGRESS, &work->state); } /** @@ -94,43 +86,16 @@ int writeback_in_progress(struct backing_dev_info *bdi) return !list_empty(&bdi->work_list); } -static void bdi_work_clear(struct bdi_work *work) -{ - clear_bit(WS_USED_B, &work->state); - smp_mb__after_clear_bit(); - /* - * work can have disappeared at this point. bit waitq functions - * should be able to tolerate this, provided bdi_sched_wait does - * not dereference it's pointer argument. - */ - wake_up_bit(&work->state, WS_USED_B); -} - static void bdi_work_free(struct rcu_head *head) { struct bdi_work *work = container_of(head, struct bdi_work, rcu_head); - if (!bdi_work_on_stack(work)) - kfree(work); - else - bdi_work_clear(work); -} - -static void wb_work_complete(struct bdi_work *work) -{ - const enum writeback_sync_modes sync_mode = work->args.sync_mode; - int onstack = bdi_work_on_stack(work); + clear_bit(WS_INPROGRESS, &work->state); + smp_mb__after_clear_bit(); + wake_up_bit(&work->state, WS_INPROGRESS); - /* - * For allocated work, we can clear the done/seen bit right here. - * For on-stack work, we need to postpone both the clear and free - * to after the RCU grace period, since the stack could be invalidated - * as soon as bdi_work_clear() has done the wakeup. - */ - if (!onstack) - bdi_work_clear(work); - if (sync_mode == WB_SYNC_NONE || onstack) - call_rcu(&work->rcu_head, bdi_work_free); + if (!test_bit(WS_ONSTACK, &work->state)) + kfree(work); } static void wb_clear_pending(struct bdi_writeback *wb, struct bdi_work *work) @@ -146,7 +111,7 @@ static void wb_clear_pending(struct bdi_writeback *wb, struct bdi_work *work) list_del_rcu(&work->list); spin_unlock(&bdi->wb_lock); - wb_work_complete(work); + call_rcu(&work->rcu_head, bdi_work_free); } } @@ -184,9 +149,9 @@ static void bdi_queue_work(struct backing_dev_info *bdi, struct bdi_work *work) * Used for on-stack allocated work items. The caller needs to wait until * the wb threads have acked the work before it's safe to continue. */ -static void bdi_wait_on_work_clear(struct bdi_work *work) +static void bdi_wait_on_work_done(struct bdi_work *work) { - wait_on_bit(&work->state, WS_USED_B, bdi_sched_wait, + wait_on_bit(&work->state, WS_INPROGRESS, bdi_sched_wait, TASK_UNINTERRUPTIBLE); } @@ -233,10 +198,10 @@ static void bdi_sync_writeback(struct backing_dev_info *bdi, struct bdi_work work; bdi_work_init(&work, &args); - work.state |= WS_ONSTACK; + __set_bit(WS_ONSTACK, &work.state); bdi_queue_work(bdi, &work); - bdi_wait_on_work_clear(&work); + bdi_wait_on_work_done(&work); } /** @@ -898,7 +863,7 @@ long wb_do_writeback(struct bdi_writeback *wb, int force_wait) * If this isn't a data integrity operation, just notify * that we have seen this work and we are now starting it. */ - if (args.sync_mode == WB_SYNC_NONE) + if (!test_bit(WS_ONSTACK, &work->state)) wb_clear_pending(wb, work); wrote += wb_writeback(wb, &args); @@ -907,7 +872,7 @@ long wb_do_writeback(struct bdi_writeback *wb, int force_wait) * This is a data integrity writeback, so only do the * notification when we have completed the work. */ - if (args.sync_mode == WB_SYNC_ALL) + if (test_bit(WS_ONSTACK, &work->state)) wb_clear_pending(wb, work); }