From patchwork Mon Oct 15 10:38:14 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hans de Goede X-Patchwork-Id: 191531 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 178162C00A7 for ; Mon, 15 Oct 2012 22:15:06 +1100 (EST) Received: from localhost ([::1]:59347 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TNi4u-0007X9-1B for incoming@patchwork.ozlabs.org; Mon, 15 Oct 2012 06:39:20 -0400 Received: from eggs.gnu.org ([208.118.235.92]:43978) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TNi3r-0005Oc-6R for qemu-devel@nongnu.org; Mon, 15 Oct 2012 06:38:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TNi3l-0003rE-2C for qemu-devel@nongnu.org; Mon, 15 Oct 2012 06:38:14 -0400 Received: from mx1.redhat.com ([209.132.183.28]:25414) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TNi3k-0003r5-QN for qemu-devel@nongnu.org; Mon, 15 Oct 2012 06:38:08 -0400 Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id q9FAc8T5027789 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Mon, 15 Oct 2012 06:38:08 -0400 Received: from localhost.localdomain.com (vpn1-5-26.ams2.redhat.com [10.36.5.26]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id q9FAc0ET008943; Mon, 15 Oct 2012 06:38:07 -0400 From: Hans de Goede To: qemu-devel@nongnu.org Date: Mon, 15 Oct 2012 12:38:14 +0200 Message-Id: <1350297511-25437-6-git-send-email-hdegoede@redhat.com> In-Reply-To: <1350297511-25437-1-git-send-email-hdegoede@redhat.com> References: <1350297511-25437-1-git-send-email-hdegoede@redhat.com> X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 209.132.183.28 Cc: Hans de Goede , Gerd Hoffmann Subject: [Qemu-devel] [PATCH 05/22] ehci: Improve latency of interrupt delivery and async schedule scanning X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org While doing various performance tests of reading from USB mass storage devices I noticed the following:: 1) When an async handled packet completes, we don't immediately report an interrupt to the guest, instead we wait for the frame-timer to run and report it from there 2) If 1) has been fixed and an async handled packet takes a while to complete, then async_stepdown will become a high value, which means that there will be a large latency before any new packets queued by the guest in response to the interrupt get seen 1) was done deliberately as part of commit f0ad01f92: http://www.kraxel.org/cgit/qemu/commit/?h=usb.57&id=f0ad01f92ca02eee7cadbfd225c5de753ebd5fce Since setting the interrupt immediately on async packet completion was causing issues with Linux guests, I believe this recently fixed Linux bug explains why this is happening: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commitdiff;h=361aabf395e4a23cf554cf4ec0c0c6963b8beb01 Note that we can *not* count on this fix being present in all Linux guests! I was hoping that the recently added support for Interrupt Threshold Control would fix the issues with Linux guests, but adding a simple ehci_commit_irq() call to ehci_async_bh() still caused problems with Linux guests. The problem is, that when doing ehci_commit_irq() from ehci_async_bh(), the "old" frindex value is used to calculate usbsts_frindex, and when the frame-timer then runs possibly very shortly after ehci_async_bh(), it increases the frame-timer, and thus any interrupts raised from that frame-timer run, will also get reported to the guest immediately, rather then being delayed to the next frame-timer run. Luckily the solution for this is simple, this means that we need to increase frindex before calling ehci_commit_irq() from ehci_async_bh(), which in the end boils down to simple calling ehci_frame_timer() instead of ehci_async_bh() from the bh. This may seem like it causes a lot of extra work to be done, but this is not true. Any work done from the frame-timer processing the periodic schedule is work which then does not need to be done the next time the frame timer runs, also the frame-timer will re-arm itself at (possibly) a later time then it was armed for saving a vmexit at that time. As an additional advantage moving to simply calling the frame-timer also fixes 2) as the packet completion will set async_stepdown to 0, and the re-arming of the timer with an async_stepdown of 0 ensures that any newly queued up packets get seen in a reasonable amount of time. This improves the speed (MB/s) of a Linux guest reading from a USB mass storage device by a factor of 1.5 - 1.7 with input pipelining disabled, and by a factor of 1.8 with input pipelining enabled. Signed-off-by: Hans de Goede --- hw/usb/hcd-ehci.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/hw/usb/hcd-ehci.c b/hw/usb/hcd-ehci.c index d9d4918..bbfa441 100644 --- a/hw/usb/hcd-ehci.c +++ b/hw/usb/hcd-ehci.c @@ -1244,7 +1244,7 @@ static void ehci_opreg_write(void *ptr, target_phys_addr_t addr, s->usbcmd = val; /* Set usbcmd for ehci_update_halt() */ ehci_update_halt(s); s->async_stepdown = 0; - qemu_mod_timer(s->frame_timer, qemu_get_clock_ns(vm_clock)); + qemu_bh_schedule(s->async_bh); } break; @@ -2509,12 +2509,6 @@ static void ehci_frame_timer(void *opaque) } } -static void ehci_async_bh(void *opaque) -{ - EHCIState *ehci = opaque; - ehci_advance_async_state(ehci); -} - static const MemoryRegionOps ehci_mmio_caps_ops = { .read = ehci_caps_read, .valid.min_access_size = 1, @@ -2743,7 +2737,7 @@ static int usb_ehci_initfn(PCIDevice *dev) } s->frame_timer = qemu_new_timer_ns(vm_clock, ehci_frame_timer, s); - s->async_bh = qemu_bh_new(ehci_async_bh, s); + s->async_bh = qemu_bh_new(ehci_frame_timer, s); QTAILQ_INIT(&s->aqueues); QTAILQ_INIT(&s->pqueues); usb_packet_init(&s->ipacket);