[RFC,6/6] vhost-user: document migration log

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 docs/specs/vhost-user.txt | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

On Thu, Jul 23, 2015 at 03:36:43AM +0200, Marc-André Lureau wrote:
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>

for some reason I didn't get 5/6.

> ---
>  docs/specs/vhost-user.txt | 40 ++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 40 insertions(+)
> 
> diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
> index 0062baa..c2d2e2a 100644
> --- a/docs/specs/vhost-user.txt
> +++ b/docs/specs/vhost-user.txt
> @@ -120,6 +120,7 @@ There are several messages that the master sends with file descriptors passed
>  in the ancillary data:
>  
>   * VHOST_SET_MEM_TABLE
> + * VHOST_SET_LOG_BASE (if VHOST_USER_PROTOCOL_F_LOG_SHMFD)
>   * VHOST_SET_LOG_FD
>   * VHOST_SET_VRING_KICK
>   * VHOST_SET_VRING_CALL
> @@ -135,6 +136,11 @@ As older slaves don't support negotiating protocol features,
>  a feature bit was dedicated for this purpose:
>  #define VHOST_USER_F_PROTOCOL_FEATURES 30
>  
> +Protocol features
> +-----------------
> +
> +#define VHOST_USER_PROTOCOL_F_LOG_SHMFD 0
> +
>  Message types
>  -------------
>  
> @@ -301,3 +307,37 @@ Message types
>        Bits (0-7) of the payload contain the vring index. Bit 8 is the
>        invalid FD flag. This flag is set when there is no file descriptor
>        in the ancillary data.
> +
> +Migration
> +---------
> +
> +During live migration, the master may need to track the modifications
> +the slave makes to the memory mapped regions. The client should mark
> +the dirty pages in a log. Once it complies to this logging, it may
> +declare VHOST_F_LOG_ALL has a vhost feature.
> +
> +All the modifications to memory pointed by vring "descriptor" should
> +be marked. Modifications to "used" vring should be marked if
> +VHOST_VRING_F_LOG is part of ring's features.

It's device's features I think.

> +
> +Dirty pages are of size:
> +#define VHOST_LOG_PAGE 0x1000
> +
> +The log memory fd is provided in the ancillary data of
> +VHOST_USER_SET_LOG_BASE message when the slave has
> +VHOST_USER_PROTOCOL_F_LOG_SHMFD protocol feature.
> +
> +The size of the log may be computed by using all the known guest
> +addresses. The log covers from address 0 to the maximum of guest
> +regions. In pseudo-code, to mark page at "addr" as dirty:
> +
> +page = addr / VHOST_LOG_PAGE
> +log[page / 8] |= 1 << page % 8

Pls note it must be done atomically.


> +
> +VHOST_USER_SET_LOG_FD is an optional message with an eventfd in
> +ancillary data, it may be used to inform the master that the log has
> +been modified.
> +
> +Once the source has finished migration, VHOST_USER_RESET_OWNER message
> +will be sent by the source. No further update must be done before the
> +destination takes over with new regions & rings.
> -- 
> 2.4.3

Hi

----- Original Message -----
> On Thu, Jul 23, 2015 at 03:36:43AM +0200, Marc-André Lureau wrote:
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> 
> for some reason I didn't get 5/6.
> 

strange: http://lists.nongnu.org/archive/html/qemu-devel/2015-07/msg04640.html

> > ---
> >  docs/specs/vhost-user.txt | 40 ++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 40 insertions(+)
> > 
> > diff --git a/docs/specs/vhost-user.txt b/docs/specs/vhost-user.txt
> > index 0062baa..c2d2e2a 100644
> > --- a/docs/specs/vhost-user.txt
> > +++ b/docs/specs/vhost-user.txt
> > @@ -120,6 +120,7 @@ There are several messages that the master sends with
> > file descriptors passed
> >  in the ancillary data:
> >  
> >   * VHOST_SET_MEM_TABLE
> > + * VHOST_SET_LOG_BASE (if VHOST_USER_PROTOCOL_F_LOG_SHMFD)
> >   * VHOST_SET_LOG_FD
> >   * VHOST_SET_VRING_KICK
> >   * VHOST_SET_VRING_CALL
> > @@ -135,6 +136,11 @@ As older slaves don't support negotiating protocol
> > features,
> >  a feature bit was dedicated for this purpose:
> >  #define VHOST_USER_F_PROTOCOL_FEATURES 30
> >  
> > +Protocol features
> > +-----------------
> > +
> > +#define VHOST_USER_PROTOCOL_F_LOG_SHMFD 0
> > +
> >  Message types
> >  -------------
> >  
> > @@ -301,3 +307,37 @@ Message types
> >        Bits (0-7) of the payload contain the vring index. Bit 8 is the
> >        invalid FD flag. This flag is set when there is no file descriptor
> >        in the ancillary data.
> > +
> > +Migration
> > +---------
> > +
> > +During live migration, the master may need to track the modifications
> > +the slave makes to the memory mapped regions. The client should mark
> > +the dirty pages in a log. Once it complies to this logging, it may
> > +declare VHOST_F_LOG_ALL has a vhost feature.
> > +
> > +All the modifications to memory pointed by vring "descriptor" should
> > +be marked. Modifications to "used" vring should be marked if
> > +VHOST_VRING_F_LOG is part of ring's features.
> 
> It's device's features I think.

Hmm, it's part of both, not sure why: see vhost_virtqueue_set_addr() and vhost_dev_set_features()

Not sure it's correct in device features, it doesn't seem to be check in kernel vhost.c either.

There is also some dead definitions like VHOST_MEMORY_F_LOG there

> 
> > +
> > +Dirty pages are of size:
> > +#define VHOST_LOG_PAGE 0x1000
> > +
> > +The log memory fd is provided in the ancillary data of
> > +VHOST_USER_SET_LOG_BASE message when the slave has
> > +VHOST_USER_PROTOCOL_F_LOG_SHMFD protocol feature.
> > +
> > +The size of the log may be computed by using all the known guest
> > +addresses. The log covers from address 0 to the maximum of guest
> > +regions. In pseudo-code, to mark page at "addr" as dirty:
> > +
> > +page = addr / VHOST_LOG_PAGE
> > +log[page / 8] |= 1 << page % 8
> 
> Pls note it must be done atomically.

ok

> 
> 
> > +
> > +VHOST_USER_SET_LOG_FD is an optional message with an eventfd in
> > +ancillary data, it may be used to inform the master that the log has
> > +been modified.
> > +
> > +Once the source has finished migration, VHOST_USER_RESET_OWNER message
> > +will be sent by the source. No further update must be done before the
> > +destination takes over with new regions & rings.
> > --
> > 2.4.3
>