From patchwork Tue Jul 31 16:51:45 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robert Wang X-Patchwork-Id: 174285 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 19EB72C00A0 for ; Wed, 1 Aug 2012 02:52:36 +1000 (EST) Received: from localhost ([::1]:59965 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SwFgP-0001Qz-Pg for incoming@patchwork.ozlabs.org; Tue, 31 Jul 2012 12:52:33 -0400 Received: from eggs.gnu.org ([208.118.235.92]:35260) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SwFgI-0001Qf-BT for qemu-devel@nongnu.org; Tue, 31 Jul 2012 12:52:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1SwFgE-0004Ic-5G for qemu-devel@nongnu.org; Tue, 31 Jul 2012 12:52:26 -0400 Received: from mail-yx0-f173.google.com ([209.85.213.173]:51739) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SwFgD-0004IN-VT for qemu-devel@nongnu.org; Tue, 31 Jul 2012 12:52:22 -0400 Received: by yenl1 with SMTP id l1so6109195yen.4 for ; Tue, 31 Jul 2012 09:52:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:x-mailer; bh=SGyAsiJIAwoQRRGMvUQ/XgG8Avz7t5D3+xKzO7IsxBY=; b=bAxQRP9iCf6AncXGZXVysIokrGh31qVebOd5RjjLqkjwQj4x4ltn99peIItO62bdy/ NGZiAd+4m6PAJ0nWPUMVhEFbEd01EMyJc7jgb33ywQ42faf5Oh8nUA+Yb9mNQgqf/Zy2 yECEXtxXQsNjrLQkY3O9K8d9oecys4EOp5yne1jeShnx/pESZbqb2LqHiWknNO5VVSDS E7OhMZfXGBi1Txv048npfN10n94kNBLEXNc/fKGWdiltwgerwlFxe+96Pj0qfY67XvLF XeAtcZJdwFB3vyiynzYmnZaHFGHoSAx73TNiNIv9S9fO49JzRq5NmJTA1H+vzNqdR5Km CdtA== Received: by 10.50.104.228 with SMTP id gh4mr1223263igb.71.1343753540403; Tue, 31 Jul 2012 09:52:20 -0700 (PDT) Received: from localhost.localdomain ([202.108.130.138]) by mx.google.com with ESMTPS id uq6sm3025753igb.14.2012.07.31.09.52.17 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 31 Jul 2012 09:52:19 -0700 (PDT) From: Dong Xu Wang To: qemu-devel@nongnu.org Date: Wed, 1 Aug 2012 00:51:45 +0800 Message-Id: <1343753510-24661-1-git-send-email-wdongxu@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.1 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 209.85.213.173 Cc: kwolf@redhat.com, Dong Xu Wang Subject: [Qemu-devel] [PATCH 1/6 v11] docs: spec for add-cow file format X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Introduce a new file format:add-cow. The usage can be found at this patch. Signed-off-by: Dong Xu Wang --- Now add-cow is still using QEMUOptionParameter, not QemuOpts, I will send a seperate patch series to convert. docs/specs/add-cow.txt | 128 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 128 insertions(+), 0 deletions(-) create mode 100644 docs/specs/add-cow.txt diff --git a/docs/specs/add-cow.txt b/docs/specs/add-cow.txt new file mode 100644 index 0000000..4793a3e --- /dev/null +++ b/docs/specs/add-cow.txt @@ -0,0 +1,128 @@ +== General == + +Raw file format does not support backing file and copy on write feature. +The add-cow image format makes it possible to use backing files with raw +image by keeping a separate .add-cow metadata file. Once all sectors +have been written into the raw image it is safe to discard the .add-cow +and backing files, then we can use the raw image directly. + +While using add-cow, procedures may like this: +(ubuntu.img is a disk image which has been installed OS.) + 1) Create a raw image with the same size of ubuntu.img + qemu-img create -f raw test.raw 8G + 2) Create an add-cow image which will store dirty bitmap + qemu-img create -f add-cow test.add-cow \ + -o backing_file=ubuntu.img,image_file=test.raw + 3) Run qemu with add-cow image + qemu -drive if=virtio,file=test.add-cow + +test.raw may be larger than ubuntu.img, in that case, the size of test.add-cow +will be calculated by the size of ubuntu.img, test.raw will be used from the +1st byte, the rest part can be used for other purpose. + +=Specification= + +The file format looks like this: + + +---------------+-------------+-----------------+ + | Header | Reserved | COW bitmap | + +---------------+-------------+-----------------+ + +All numbers in add-cow are stored in Little Endian byte order. + +== Header == + +The Header is included in the first bytes: +(#define HEADER_SIZE (4096 * header_pages_size)) + Byte 0 - 7: magic + add-cow magic string ("ADD_COW\xff"). + + 8 - 11: version + Version number (only valid value is 1 now). + + 12 - 15: backing file name offset + Offset in the add-cow file at which the backing file + name is stored (NB: The string is not null terminated). + If backing file name does NOT exist, this field will be + 0. Must be between 80 and [HEADER_SIZE - 2](a file name + must be at least 1 byte). + + 16 - 19: backing file name size + Length of the backing file name in bytes. It will be 0 + if the backing file name offset is 0. If backing file + name offset is non-zero, then it must be non-zero. Must + be less than [HEADER_SIZE - 80] to fit in the reserved + part of the header. + + 20 - 23: image file name offset + Offset in the add-cow file at which the image file name + is stored (NB: The string is not null terminated). It + must be between 80 and [HEADER_SIZE - 2]. + + 24 - 27: image file name size + Length of the image file name in bytes. + Must be less than [HEADER_SIZE - 80] to fit in the reserved + part of the header. + + 28 - 35: features + Currently only 3 feature bit is used: + Feature bits: + The image uses a backing file: + * ADD_COW_F_BACKING_FILE = 0x01. + The image uses a image file: + * ADD_COW_F_IMAGE_FILE = 0x02. + All bits in bitmap have been set to 1, add-cow wrapper + can be discarded. + * ADD_COW_F_All_ALLOCATED = 0x04. + + 36 - 43: optional features + Not used now. Researved for future use. + + 44 - 47: header pages size + The header field is variable-sized. This field indicates + how many pages(4k) will be used to store add-cow header. + In add-cow v1, it is fixed to 1, so the header size will + be 4k * 1 = 4096 bytes. + +Image file name and backing file name must NOT be the same, we prevent this +while creating add-cow files. + +Image file and backing file are interpreted relative to the qcow2 file, not +to the current working directory of the process that opened the qcow2 file. + +== Reserved == + + Byte 48 - 63: backing file format + format of backing file. It will be filled with 0 if + backing file name offset is 0. If backing file name + offset is none-zero, it must be non-zero. + + 64 - 79: image file format + format of image file. It must be non-zero. + + 80 - [HEADER_SIZE - 1]: + It is used to make sure COW bitmap field starts at the + HEADER_SIZE byte, backing file name and image file name + will be stored here. + +== COW bitmap == + +The "COW bitmap" field starts at the 4096th byte, stores a bitmap related to +backing file and image file. The bitmap will track whether the sector in +backing file is dirty or not. + +Each bit in the bitmap indicates one cluster's status. One cluster includes 128 +sectors, then each bit indicates 512 * 128 = 64k bytes. the size of bitmap is +calculated according to virtual size of backing file. In each byte, bit 0 to 7 +will track the 1st to 7th cluster in sequence, bit orders in one byte look like: + +----+----+----+----+----+----+----+----+ + | b7 | b6 | b5 | b4 | b3 | b2 | b1 | b0 | + +----+----+----+----+----+----+----+----+ + +If the bit is 0, indicates the sector has not been allocated in image file, data +should be loaded from backing file while reading; if the bit is 1, indicates the +related sector has been dirty, should be loaded from image file while reading. +Writing to a sector causes the corresponding bit to be set to 1. + +If raw image is not an even multiple of cluster bytes, bits that correspond to +bytes beyond the raw file size in add-cow will be 0.