Message ID | 1418347746-15829-2-git-send-email-liang.z.li@intel.com |
---|---|
State | New |
Headers | show |
* Liang Li (liang.z.li@intel.com) wrote: > Give some details about the multiple compression threads and > how to use it in live migration. > > Signed-off-by: Liang Li <liang.z.li@intel.com> > Signed-off-by: Yang Zhang <yang.z.zhang@intel.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> > --- > docs/multi-thread-compression.txt | 141 ++++++++++++++++++++++++++++++++++++++ > 1 file changed, 141 insertions(+) > create mode 100644 docs/multi-thread-compression.txt > > diff --git a/docs/multi-thread-compression.txt b/docs/multi-thread-compression.txt > new file mode 100644 > index 0000000..3bbc641 > --- /dev/null > +++ b/docs/multi-thread-compression.txt > @@ -0,0 +1,141 @@ > +Use multiple thread (de)compression in live migration > +====================================================== > +Copyright (C) 2014 Intel Corporation > +Author: Li Liang <liang.z.li@intel.com> > + > +This work is licensed under the terms of the GNU GPLv2 or later. See > +the COPYING file in the top-level directory. > + > +Contents: > +========= > +* Introduction > +* When to use > +* Performance > +* Usage > +* TODO > + > +Introduction > +============ > +Instead of sending the guest memory directly, this solution will > +compress the ram page before sending; after receiving, the data will > +be decompressed. Using compression in live migration can help > +to reduce the data transferred about 60%, this is very useful when the > +bandwidth is limited, and the migration time can also be reduced about > +70% in a typical case. > + > +The process of compression will consume additional CPU cycles, and the > +extra CPU cycles will increase the migration time. On the other hand, > +the amount of data transferred will reduced, this factor can reduce > +the migration time. If the process of the compression is quick > +enough, then the total migration time can be reduced, and multiple > +thread compression can be used to accelerate the compression process. > + > +The decompression speed of zlib is at least 4 times as quick as > +compression, if the source and destination CPU have equal speed, > +keeping the compression thread count 4 times the decompression > +thread count can avoid CPU waste. > + > +Compression level can be used to control the compression speed and the > +compression ratio. High compression ratio will take more time, level 0 > +stands for no compression, level 1 stands for the best compression > +speed, and level 9 stands for the best compression ratio. Users can > +select a level number between 0 and 9. > + > + > +When to use the multiple thread compression in live migration > +============================================================== > +Compression of data will consume extra CPU cycles; so in a system with > +high overhead of CPU, avoid using this feature. When the network > +bandwidth is very limited and the CPU resource is adequate, use of > +multiple thread compression will be very helpful. If both the CPU and > +the network bandwidth are adequate, use of multiple thread compression > +can still help to reduce the migration time. > + > +Performance > +=========== > +Test environment: > + > +CPU: Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz > +Socket Count: 2 > +Ram: 128G > +NIC: Intel I350 (10/100/1000Mbps) > +Host OS: CentOS 7 64-bit > +Guest OS: Ubuntu 12.10 64-bit > +Parameter: qemu-system-x86_64 -enable-kvm -m 1024 > + /share/ia32e_ubuntu12.10.img -monitor stdio > + > +There is no additional application is running on the guest when doing > +the test. > + > + > +Speed limit: 32MB/s > +--------------------------------------------------------------- > + | original | compress thread: 8 > + | way | decompress thread: 2 > + | | compression level: 1 > +--------------------------------------------------------------- > +total time(msec): | 26561 | 7920 > +--------------------------------------------------------------- > +transferred ram(kB):| 877054 | 260641 > +--------------------------------------------------------------- > +throughput(mbps): | 270.53 | 269.68 > +--------------------------------------------------------------- > +total ram(kB): | 1057604 | 1057604 > +--------------------------------------------------------------- > + > + > +Speed limit: No > +--------------------------------------------------------------- > + | original | compress thread: 15 > + | way | decompress thread: 4 > + | | compression level: 1 > +--------------------------------------------------------------- > +total time(msec): | 7611 | 2888 > +--------------------------------------------------------------- > +transferred ram(kB):| 876761 | 262301 > +--------------------------------------------------------------- > +throughput(mbps): | 943.78 | 744.27 > +--------------------------------------------------------------- > +total ram(kB): | 1057604 | 1057604 > +--------------------------------------------------------------- > + > +Usage > +===== > +1. Verify both the source and destination QEMU are able > +to support the multiple thread compression migration: > + {qemu} info_migrate_capablilites > + {qemu} ... compress: off ... > + > +2. Activate compression on the souce: > + {qemu} migrate_set_capability compress on > + > +3. Set the compression thread count on source: > + {qemu} migrate_set_paramter compress_threads 12 > + > +4. Set the compression level on the source: > + {qemu} migrate_set_parameter compress_level 1 > + > +5. Set the decompression thread count on destination: > + {qemu} migrate_set_parameter decompress_threads 3 > + > +6. Start outgoing migration: > + {qemu} migrate -d tcp:destination.host:4444 > + {qemu} info migrate > + Capabilities: ... compress: on > + ... > + > +The following is the default setting: > + compress: off > + compress_threads: 8 > + decompress_threads: 2 > + compress_level: 1 (which means best speed) > + > +So, only the first two steps are required to use the multiple > +thread compression in migration. You can do more if the default > +setting is not appropriate. > + > +TODO > +==== > +Some faster (de)compression method such as lz4 and quicklz can help > +to reduce the CPU consumption when doing (de)compression. Less > +(de)compression threads are needed when doing the migration. > -- > 1.8.3.1 > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
On 12/11/2014 06:28 PM, Liang Li wrote: > Give some details about the multiple compression threads and > how to use it in live migration. > > Signed-off-by: Liang Li <liang.z.li@intel.com> > Signed-off-by: Yang Zhang <yang.z.zhang@intel.com> > --- > docs/multi-thread-compression.txt | 141 ++++++++++++++++++++++++++++++++++++++ > 1 file changed, 141 insertions(+) > create mode 100644 docs/multi-thread-compression.txt > > diff --git a/docs/multi-thread-compression.txt b/docs/multi-thread-compression.txt > new file mode 100644 > index 0000000..3bbc641 > --- /dev/null > +++ b/docs/multi-thread-compression.txt > @@ -0,0 +1,141 @@ > +Use multiple thread (de)compression in live migration > +====================================================== > +Copyright (C) 2014 Intel Corporation You may want to include 2015 now. > +Introduction > +============ > +Instead of sending the guest memory directly, this solution will > +compress the ram page before sending; after receiving, the data will s/ram/RAM/ > +be decompressed. Using compression in live migration can help > +to reduce the data transferred about 60%, this is very useful when the > +bandwidth is limited, and the migration time can also be reduced about > +70% in a typical case. Of course, that's all dependent on memory having compressible contents. An application that reads /dev/hwrng into a large amount of memory will not be that compressible :) > + > +When to use the multiple thread compression in live migration > +============================================================== off-by-one on the divider length > +Usage > +===== > +1. Verify both the source and destination QEMU are able > +to support the multiple thread compression migration: > + {qemu} info_migrate_capablilites s/capablilites/capabilities/ > + {qemu} ... compress: off ... > + > +2. Activate compression on the souce: s/souce/source/ > + {qemu} migrate_set_capability compress on > + > +3. Set the compression thread count on source: > + {qemu} migrate_set_paramter compress_threads 12 s/paramter/parameter/ > + > +The following is the default setting: s/is...setting/are...settings/ > + compress: off > + compress_threads: 8 > + decompress_threads: 2 > + compress_level: 1 (which means best speed) > + > +So, only the first two steps are required to use the multiple > +thread compression in migration. You can do more if the default > +setting is not appropriate. s/setting is/settings are/
diff --git a/docs/multi-thread-compression.txt b/docs/multi-thread-compression.txt new file mode 100644 index 0000000..3bbc641 --- /dev/null +++ b/docs/multi-thread-compression.txt @@ -0,0 +1,141 @@ +Use multiple thread (de)compression in live migration +====================================================== +Copyright (C) 2014 Intel Corporation +Author: Li Liang <liang.z.li@intel.com> + +This work is licensed under the terms of the GNU GPLv2 or later. See +the COPYING file in the top-level directory. + +Contents: +========= +* Introduction +* When to use +* Performance +* Usage +* TODO + +Introduction +============ +Instead of sending the guest memory directly, this solution will +compress the ram page before sending; after receiving, the data will +be decompressed. Using compression in live migration can help +to reduce the data transferred about 60%, this is very useful when the +bandwidth is limited, and the migration time can also be reduced about +70% in a typical case. + +The process of compression will consume additional CPU cycles, and the +extra CPU cycles will increase the migration time. On the other hand, +the amount of data transferred will reduced, this factor can reduce +the migration time. If the process of the compression is quick +enough, then the total migration time can be reduced, and multiple +thread compression can be used to accelerate the compression process. + +The decompression speed of zlib is at least 4 times as quick as +compression, if the source and destination CPU have equal speed, +keeping the compression thread count 4 times the decompression +thread count can avoid CPU waste. + +Compression level can be used to control the compression speed and the +compression ratio. High compression ratio will take more time, level 0 +stands for no compression, level 1 stands for the best compression +speed, and level 9 stands for the best compression ratio. Users can +select a level number between 0 and 9. + + +When to use the multiple thread compression in live migration +============================================================== +Compression of data will consume extra CPU cycles; so in a system with +high overhead of CPU, avoid using this feature. When the network +bandwidth is very limited and the CPU resource is adequate, use of +multiple thread compression will be very helpful. If both the CPU and +the network bandwidth are adequate, use of multiple thread compression +can still help to reduce the migration time. + +Performance +=========== +Test environment: + +CPU: Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz +Socket Count: 2 +Ram: 128G +NIC: Intel I350 (10/100/1000Mbps) +Host OS: CentOS 7 64-bit +Guest OS: Ubuntu 12.10 64-bit +Parameter: qemu-system-x86_64 -enable-kvm -m 1024 + /share/ia32e_ubuntu12.10.img -monitor stdio + +There is no additional application is running on the guest when doing +the test. + + +Speed limit: 32MB/s +--------------------------------------------------------------- + | original | compress thread: 8 + | way | decompress thread: 2 + | | compression level: 1 +--------------------------------------------------------------- +total time(msec): | 26561 | 7920 +--------------------------------------------------------------- +transferred ram(kB):| 877054 | 260641 +--------------------------------------------------------------- +throughput(mbps): | 270.53 | 269.68 +--------------------------------------------------------------- +total ram(kB): | 1057604 | 1057604 +--------------------------------------------------------------- + + +Speed limit: No +--------------------------------------------------------------- + | original | compress thread: 15 + | way | decompress thread: 4 + | | compression level: 1 +--------------------------------------------------------------- +total time(msec): | 7611 | 2888 +--------------------------------------------------------------- +transferred ram(kB):| 876761 | 262301 +--------------------------------------------------------------- +throughput(mbps): | 943.78 | 744.27 +--------------------------------------------------------------- +total ram(kB): | 1057604 | 1057604 +--------------------------------------------------------------- + +Usage +===== +1. Verify both the source and destination QEMU are able +to support the multiple thread compression migration: + {qemu} info_migrate_capablilites + {qemu} ... compress: off ... + +2. Activate compression on the souce: + {qemu} migrate_set_capability compress on + +3. Set the compression thread count on source: + {qemu} migrate_set_paramter compress_threads 12 + +4. Set the compression level on the source: + {qemu} migrate_set_parameter compress_level 1 + +5. Set the decompression thread count on destination: + {qemu} migrate_set_parameter decompress_threads 3 + +6. Start outgoing migration: + {qemu} migrate -d tcp:destination.host:4444 + {qemu} info migrate + Capabilities: ... compress: on + ... + +The following is the default setting: + compress: off + compress_threads: 8 + decompress_threads: 2 + compress_level: 1 (which means best speed) + +So, only the first two steps are required to use the multiple +thread compression in migration. You can do more if the default +setting is not appropriate. + +TODO +==== +Some faster (de)compression method such as lz4 and quicklz can help +to reduce the CPU consumption when doing (de)compression. Less +(de)compression threads are needed when doing the migration.