From patchwork Thu Mar 5 15:56:20 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Freimann X-Patchwork-Id: 446763 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 9A1441400F1 for ; Fri, 6 Mar 2015 02:58:04 +1100 (AEDT) Received: from localhost ([::1]:52676 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YTY9y-0006xg-Ug for incoming@patchwork.ozlabs.org; Thu, 05 Mar 2015 10:58:02 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48750) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YTY8b-00046T-Dr for qemu-devel@nongnu.org; Thu, 05 Mar 2015 10:56:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YTY8V-0007fE-Fm for qemu-devel@nongnu.org; Thu, 05 Mar 2015 10:56:37 -0500 Received: from e06smtp11.uk.ibm.com ([195.75.94.107]:50313) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YTY8V-0007f1-4e for qemu-devel@nongnu.org; Thu, 05 Mar 2015 10:56:31 -0500 Received: from /spool/local by e06smtp11.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 5 Mar 2015 15:56:29 -0000 Received: from d06dlp01.portsmouth.uk.ibm.com (9.149.20.13) by e06smtp11.uk.ibm.com (192.168.101.141) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 5 Mar 2015 15:56:27 -0000 Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by d06dlp01.portsmouth.uk.ibm.com (Postfix) with ESMTP id C4C7617D805A for ; Thu, 5 Mar 2015 15:56:46 +0000 (GMT) Received: from d06av06.portsmouth.uk.ibm.com (d06av06.portsmouth.uk.ibm.com [9.149.37.217]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id t25FuR6I10289408 for ; Thu, 5 Mar 2015 15:56:27 GMT Received: from d06av06.portsmouth.uk.ibm.com (localhost [127.0.0.1]) by d06av06.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id t25ApdVS017737 for ; Thu, 5 Mar 2015 05:51:40 -0500 Received: from tuxmaker.boeblingen.de.ibm.com (tuxmaker.boeblingen.de.ibm.com [9.152.85.9]) by d06av06.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id t25Apd9b017728; Thu, 5 Mar 2015 05:51:39 -0500 Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 1122) id 536151224459; Thu, 5 Mar 2015 16:56:26 +0100 (CET) From: Jens Freimann To: Christian Borntraeger , Alexander Graf , Cornelia Huck Date: Thu, 5 Mar 2015 16:56:20 +0100 Message-Id: <1425570981-40609-2-git-send-email-jfrei@linux.vnet.ibm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1425570981-40609-1-git-send-email-jfrei@linux.vnet.ibm.com> References: <1425570981-40609-1-git-send-email-jfrei@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15030515-0041-0000-0000-00000397508A X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 195.75.94.107 Cc: Jens Freimann , qemu-devel@nongnu.org, "Jason J. Herne" , "Jason J. Herne" Subject: [Qemu-devel] [PATCH 1/2] s390x/kvm: Guest Migration TOD clock synchronization X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: "Jason J. Herne" Synchronizes the guest TOD clock across a migration by sending the guest TOD clock value to the destination system. If the guest TOD clock is not preserved across a migration then the guest's view of time will snap backwards if the destination host clock is behind the source host clock. This will cause the guest to hang immediately upon resuming on the destination system. Reviewed-by: David Hildenbrand Signed-off-by: Jason J. Herne Signed-off-by: Jens Freimann --- hw/s390x/s390-virtio-ccw.c | 4 ++++ hw/s390x/s390-virtio.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++ linux-headers/linux/kvm.h | 1 - target-s390x/cpu.h | 34 +++++++++++++++++++++++++++++ target-s390x/kvm.c | 39 ++++++++++++++++++++++++++++++++++ 5 files changed, 130 insertions(+), 1 deletion(-) diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index 8f0ae59..078371a 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -171,6 +171,10 @@ static void ccw_init(MachineState *machine) /* Create VirtIO network adapters */ s390_create_virtio_net(BUS(css_bus), "virtio-net-ccw"); + + /* Register savevm handler for guest TOD clock */ + register_savevm(NULL, "todclock", 0, 1, + gtod_save, gtod_load, kvm_state); } static void ccw_machine_class_init(ObjectClass *oc, void *data) diff --git a/hw/s390x/s390-virtio.c b/hw/s390x/s390-virtio.c index 412e49b..fb0ac64 100644 --- a/hw/s390x/s390-virtio.c +++ b/hw/s390x/s390-virtio.c @@ -39,6 +39,8 @@ #include "hw/s390x/s390_flic.h" #include "hw/s390x/s390-virtio.h" +#include "cpu.h" + //#define DEBUG_S390 #ifdef DEBUG_S390 @@ -53,6 +55,9 @@ #define ZIPL_FILENAME "s390-zipl.rom" #define TYPE_S390_MACHINE "s390-machine" +#define S390_TOD_CLOCK_VALUE_MISSING 0x00 +#define S390_TOD_CLOCK_VALUE_PRESENT 0x01 + static VirtIOS390Bus *s390_bus; static S390CPU **ipi_states; @@ -196,6 +201,51 @@ void s390_create_virtio_net(BusState *bus, const char *name) } } +void gtod_save(QEMUFile *f, void *opaque) +{ + uint64_t tod_low; + uint8_t tod_high; + int r; + + r = s390_get_clock(&tod_high, &tod_low); + if (r) { + fprintf(stderr, "WARNING: Unable to get guest clock for migration. " + "Error code %d. Guest clock will not be migrated " + "which could cause the guest to hang.\n", r); + qemu_put_byte(f, S390_TOD_CLOCK_VALUE_MISSING); + return; + } + + qemu_put_byte(f, S390_TOD_CLOCK_VALUE_PRESENT); + qemu_put_byte(f, tod_high); + qemu_put_be64(f, tod_low); +} + +int gtod_load(QEMUFile *f, void *opaque, int version_id) +{ + uint64_t tod_low; + uint8_t tod_high; + int r; + + if (qemu_get_byte(f) == S390_TOD_CLOCK_VALUE_MISSING) { + fprintf(stderr, "WARNING: Guest clock was not migrated. This could " + "cause the guest to hang.\n"); + return 0; + } + + tod_high = qemu_get_byte(f); + tod_low = qemu_get_be64(f); + + r = s390_set_clock(&tod_high, &tod_low); + if (r) { + fprintf(stderr, "WARNING: Unable to set guest clock value. " + "s390_get_clock returned error %d. This could cause " + "the guest to hang.\n", r); + } + + return 0; +} + /* PC hardware initialisation */ static void s390_init(MachineState *machine) { @@ -253,6 +303,9 @@ static void s390_init(MachineState *machine) /* Create VirtIO network adapters */ s390_create_virtio_net((BusState *)s390_bus, "virtio-net-s390"); + + /* Register savevm handler for guest TOD clock */ + register_savevm(NULL, "todclock", 0, 1, gtod_save, gtod_load, NULL); } void s390_nmi(NMIState *n, int cpu_index, Error **errp) diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h index 84a4c10..ed80e49 100644 --- a/linux-headers/linux/kvm.h +++ b/linux-headers/linux/kvm.h @@ -1094,7 +1094,6 @@ struct kvm_s390_ucas_mapping { #define KVM_SET_DEVICE_ATTR _IOW(KVMIO, 0xe1, struct kvm_device_attr) #define KVM_GET_DEVICE_ATTR _IOW(KVMIO, 0xe2, struct kvm_device_attr) #define KVM_HAS_DEVICE_ATTR _IOW(KVMIO, 0xe3, struct kvm_device_attr) - /* * ioctls for vcpu fds */ diff --git a/target-s390x/cpu.h b/target-s390x/cpu.h index 60325b8..be53b5a 100644 --- a/target-s390x/cpu.h +++ b/target-s390x/cpu.h @@ -400,6 +400,8 @@ void kvm_s390_vcpu_interrupt(S390CPU *cpu, struct kvm_s390_irq *irq); void kvm_s390_floating_interrupt(struct kvm_s390_irq *irq); int kvm_s390_inject_flic(struct kvm_s390_irq *irq); void kvm_s390_access_exception(S390CPU *cpu, uint16_t code, uint64_t te_code); +int kvm_s390_get_clock(uint8_t *tod_high, uint64_t *tod_clock); +int kvm_s390_set_clock(uint8_t *tod_high, uint64_t *tod_clock); #else static inline void kvm_s390_virtio_irq(int config_change, uint64_t token) { @@ -407,11 +409,40 @@ static inline void kvm_s390_virtio_irq(int config_change, uint64_t token) static inline void kvm_s390_service_interrupt(uint32_t parm) { } +static inline int kvm_s390_get_clock(uint8_t *tod_high, uint64_t *tod_low) +{ + return -ENOSYS; +} +static inline int kvm_s390_set_clock(uint8_t *tod_high, uint64_t *tod_low) +{ + return -ENOSYS; +} static inline void kvm_s390_access_exception(S390CPU *cpu, uint16_t code, uint64_t te_code) { } #endif + +static inline int s390_get_clock(uint8_t *tod_high, uint64_t *tod_low) +{ + if (kvm_enabled()) { + return kvm_s390_get_clock(tod_high, tod_low); + } + /* Fixme TCG */ + *tod_high = 0; + *tod_low = 0; + return 0; +} + +static inline int s390_set_clock(uint8_t *tod_high, uint64_t *tod_low) +{ + if (kvm_enabled()) { + return kvm_s390_set_clock(tod_high, tod_low); + } + /* Fixme TCG */ + return 0; +} + S390CPU *s390_cpu_addr2state(uint16_t cpu_addr); unsigned int s390_cpu_halt(S390CPU *cpu); void s390_cpu_unhalt(S390CPU *cpu); @@ -421,6 +452,9 @@ static inline uint8_t s390_cpu_get_state(S390CPU *cpu) return cpu->env.cpu_state; } +void gtod_save(QEMUFile *f, void *opaque); +int gtod_load(QEMUFile *f, void *opaque, int version_id); + /* service interrupts are floating therefore we must not pass an cpustate */ void s390_sclp_extint(uint32_t parm); diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c index 6c4360b..6ef714c 100644 --- a/target-s390x/kvm.c +++ b/target-s390x/kvm.c @@ -441,6 +441,45 @@ int kvm_arch_get_registers(CPUState *cs) return 0; } +int kvm_s390_get_clock(uint8_t *tod_high, uint64_t *tod_low) +{ + int r; + struct kvm_device_attr attr = { + .group = KVM_S390_VM_TOD, + .attr = KVM_S390_VM_TOD_LOW, + .addr = (uint64_t)tod_low, + }; + + r = kvm_vm_ioctl(kvm_state, KVM_GET_DEVICE_ATTR, &attr); + if (r) { + return r; + } + + attr.attr = KVM_S390_VM_TOD_HIGH; + attr.addr = (uint64_t)tod_high; + return kvm_vm_ioctl(kvm_state, KVM_GET_DEVICE_ATTR, &attr); +} + +int kvm_s390_set_clock(uint8_t *tod_high, uint64_t *tod_low) +{ + int r; + + struct kvm_device_attr attr = { + .group = KVM_S390_VM_TOD, + .attr = KVM_S390_VM_TOD_LOW, + .addr = (uint64_t)tod_low, + }; + + r = kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attr); + if (r) { + return r; + } + + attr.attr = KVM_S390_VM_TOD_HIGH; + attr.addr = (uint64_t)tod_high; + return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attr); +} + /* * Legacy layout for s390: * Older S390 KVM requires the topmost vma of the RAM to be