[4/6] progressmeter: protect with a mutex

Progressmeter is protected by the AioContext mutex, which
is taken by the block jobs and their caller (like blockdev).

We would like to remove the dependency of block layer code on the
AioContext mutex, since most drivers and the core I/O code are already
not relying on it.

The simplest thing to do is to add a mutex to be able to provide
an accurate snapshot of the progress values to the caller.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
---
 blockjob.c                    | 33 +++++++++++++++++++++++++--------
 include/qemu/progress_meter.h | 31 +++++++++++++++++++++++++++++++
 job-qmp.c                     |  8 ++++++--
 job.c                         |  3 +++
 qemu-img.c                    |  9 ++++++---
 5 files changed, 71 insertions(+), 13 deletions(-)

10.05.2021 11:59, Emanuele Giuseppe Esposito wrote:
> Progressmeter is protected by the AioContext mutex, which
> is taken by the block jobs and their caller (like blockdev).
> 
> We would like to remove the dependency of block layer code on the
> AioContext mutex, since most drivers and the core I/O code are already
> not relying on it.
> 
> The simplest thing to do is to add a mutex to be able to provide
> an accurate snapshot of the progress values to the caller.
> 
> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
> ---
>   blockjob.c                    | 33 +++++++++++++++++++++++++--------

[..]

> --- a/include/qemu/progress_meter.h
> +++ b/include/qemu/progress_meter.h
> @@ -27,6 +27,8 @@
>   #ifndef QEMU_PROGRESS_METER_H
>   #define QEMU_PROGRESS_METER_H
>   
> +#include "qemu/lockable.h"
> +
>   typedef struct ProgressMeter {
>       /**
>        * Current progress. The unit is arbitrary as long as the ratio between
> @@ -37,21 +39,50 @@ typedef struct ProgressMeter {
>   
>       /** Estimated current value at the completion of the process */
>       uint64_t total;
> +
> +    QemuMutex lock;
>   } ProgressMeter;
>   
> +static inline void progress_init(ProgressMeter *pm)
> +{
> +    qemu_mutex_init(&pm->lock);
> +}
> +
> +static inline void progress_destroy(ProgressMeter *pm)
> +{
> +    qemu_mutex_destroy(&pm->lock);
> +}


Could we instead add a c file and add the structure private? Then we'll have progress_new() and progress_free() APIs instead.

This way, it would be a lot simpler to control that nobady use structure fields directly.



> +
> +static inline void progress_get_snapshot(ProgressMeter *pm,
> +                                         uint64_t *current, uint64_t *total)
> +{
> +    QEMU_LOCK_GUARD(&pm->lock);
> +
> +    if (current) {
> +        *current = pm->current;
> +    }
> +
> +    if (total) {
> +        *total = pm->total;
> +    }
> +}

We don't have caller that pass only one pointer. So we can just do

*current = pm->current;
*total = pm->total;

implicitly requiring both pointers to be non NULL.

On 10/05/2021 13:28, Vladimir Sementsov-Ogievskiy wrote:
> 10.05.2021 11:59, Emanuele Giuseppe Esposito wrote:
>> Progressmeter is protected by the AioContext mutex, which
>> is taken by the block jobs and their caller (like blockdev).
>>
>> We would like to remove the dependency of block layer code on the
>> AioContext mutex, since most drivers and the core I/O code are already
>> not relying on it.
>>
>> The simplest thing to do is to add a mutex to be able to provide
>> an accurate snapshot of the progress values to the caller.
>>
>> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
>> ---
>>   blockjob.c                    | 33 +++++++++++++++++++++++++--------
> 
> [..]
> 
>> --- a/include/qemu/progress_meter.h
>> +++ b/include/qemu/progress_meter.h
>> @@ -27,6 +27,8 @@
>>   #ifndef QEMU_PROGRESS_METER_H
>>   #define QEMU_PROGRESS_METER_H
>> +#include "qemu/lockable.h"
>> +
>>   typedef struct ProgressMeter {
>>       /**
>>        * Current progress. The unit is arbitrary as long as the ratio 
>> between
>> @@ -37,21 +39,50 @@ typedef struct ProgressMeter {
>>       /** Estimated current value at the completion of the process */
>>       uint64_t total;
>> +
>> +    QemuMutex lock;
>>   } ProgressMeter;
>> +static inline void progress_init(ProgressMeter *pm)
>> +{
>> +    qemu_mutex_init(&pm->lock);
>> +}
>> +
>> +static inline void progress_destroy(ProgressMeter *pm)
>> +{
>> +    qemu_mutex_destroy(&pm->lock);
>> +}
> 
> 
> Could we instead add a c file and add the structure private? Then we'll 
> have progress_new() and progress_free() APIs instead.
> 
> This way, it would be a lot simpler to control that nobady use structure 
> fields directly.
> 
Makes sense, I'll do it.
> 
> 
>> +
>> +static inline void progress_get_snapshot(ProgressMeter *pm,
>> +                                         uint64_t *current, uint64_t 
>> *total)
>> +{
>> +    QEMU_LOCK_GUARD(&pm->lock);
>> +
>> +    if (current) {
>> +        *current = pm->current;
>> +    }
>> +
>> +    if (total) {
>> +        *total = pm->total;
>> +    }
>> +}
> 
> We don't have caller that pass only one pointer. So we can just do
> 
> *current = pm->current;
> *total = pm->total;
> 
> implicitly requiring both pointers to be non NULL.

Is it so performance critical that we need to skip these safety checks?
IMHO we can keep it as it is.

Thank you,
Emanuele
> 
> 
>

10.05.2021 19:52, Emanuele Giuseppe Esposito wrote:
> 
> 
> On 10/05/2021 13:28, Vladimir Sementsov-Ogievskiy wrote:
>> 10.05.2021 11:59, Emanuele Giuseppe Esposito wrote:
>>> Progressmeter is protected by the AioContext mutex, which
>>> is taken by the block jobs and their caller (like blockdev).
>>>
>>> We would like to remove the dependency of block layer code on the
>>> AioContext mutex, since most drivers and the core I/O code are already
>>> not relying on it.
>>>
>>> The simplest thing to do is to add a mutex to be able to provide
>>> an accurate snapshot of the progress values to the caller.
>>>
>>> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
>>> ---
>>>   blockjob.c                    | 33 +++++++++++++++++++++++++--------
>>
>> [..]
>>
>>> --- a/include/qemu/progress_meter.h
>>> +++ b/include/qemu/progress_meter.h
>>> @@ -27,6 +27,8 @@
>>>   #ifndef QEMU_PROGRESS_METER_H
>>>   #define QEMU_PROGRESS_METER_H
>>> +#include "qemu/lockable.h"
>>> +
>>>   typedef struct ProgressMeter {
>>>       /**
>>>        * Current progress. The unit is arbitrary as long as the ratio between
>>> @@ -37,21 +39,50 @@ typedef struct ProgressMeter {
>>>       /** Estimated current value at the completion of the process */
>>>       uint64_t total;
>>> +
>>> +    QemuMutex lock;
>>>   } ProgressMeter;
>>> +static inline void progress_init(ProgressMeter *pm)
>>> +{
>>> +    qemu_mutex_init(&pm->lock);
>>> +}
>>> +
>>> +static inline void progress_destroy(ProgressMeter *pm)
>>> +{
>>> +    qemu_mutex_destroy(&pm->lock);
>>> +}
>>
>>
>> Could we instead add a c file and add the structure private? Then we'll have progress_new() and progress_free() APIs instead.
>>
>> This way, it would be a lot simpler to control that nobady use structure fields directly.
>>
> Makes sense, I'll do it.
>>
>>
>>> +
>>> +static inline void progress_get_snapshot(ProgressMeter *pm,
>>> +                                         uint64_t *current, uint64_t *total)
>>> +{
>>> +    QEMU_LOCK_GUARD(&pm->lock);
>>> +
>>> +    if (current) {
>>> +        *current = pm->current;
>>> +    }
>>> +
>>> +    if (total) {
>>> +        *total = pm->total;
>>> +    }
>>> +}
>>
>> We don't have caller that pass only one pointer. So we can just do
>>
>> *current = pm->current;
>> *total = pm->total;
>>
>> implicitly requiring both pointers to be non NULL.
> 
> Is it so performance critical that we need to skip these safety checks?
> IMHO we can keep it as it is.


Not performance. It's just less code. You propose more complex interface (pointers may be valid pointers or NULL), implemented by more complex code (extra if blocks). And there no users for this. So, I don't see any reason for extra logic and code lines.

What kind of safety you want? All current callers pass non-zero pointers, it's obvious. If someone will create new call, he should look first at function itself, not blindly pass NULL pointers. And if not, he will get clean crash on zero pointer dereference.

>>>
>>> We don't have caller that pass only one pointer. So we can just do
>>>
>>> *current = pm->current;
>>> *total = pm->total;
>>>
>>> implicitly requiring both pointers to be non NULL.
>>
>> Is it so performance critical that we need to skip these safety checks?
>> IMHO we can keep it as it is.
> 
> 
> Not performance. It's just less code. You propose more complex interface 
> (pointers may be valid pointers or NULL), implemented by more complex 
> code (extra if blocks). And there no users for this. So, I don't see any 
> reason for extra logic and code lines.
> 
> What kind of safety you want? All current callers pass non-zero 
> pointers, it's obvious. If someone will create new call, he should look 
> first at function itself, not blindly pass NULL pointers. And if not, he 
> will get clean crash on zero pointer dereference.

Ok, makes sense. Will remove the ifs.

Thank you,
Emanuele

On 10/05/21 13:28, Vladimir Sementsov-Ogievskiy wrote:
> 
> 
> Could we instead add a c file and add the structure private? Then we'll 
> have progress_new() and progress_free() APIs instead.
> 
> This way, it would be a lot simpler to control that nobady use structure 
> fields directly.

I don't know...  I prefer embedding structs to heap allocation.

Paolo

11.05.2021 15:28, Paolo Bonzini wrote:
> On 10/05/21 13:28, Vladimir Sementsov-Ogievskiy wrote:
>>
>>
>> Could we instead add a c file and add the structure private? Then we'll have progress_new() and progress_free() APIs instead.
>>
>> This way, it would be a lot simpler to control that nobady use structure fields directly.
> 
> I don't know...  I prefer embedding structs to heap allocation.
> 


I agree that embedding looks better from allocation point of view.. But IMHO encapsulation guarantee of private structure is better feature. Especially when we have getter/setter methods. Even the patch itself is example: to review this carefully, I should somehow check that every use of the structure is updated (grepping the code, or maybe change field name and recompile, to catch every occurrence of it).. And if patch makes structure private and adds setters/getters, it can be reviewed without applying.

And we have to call _init/_destroy anyway, so it's not more complex to call _new/_free instead.

On Mon, May 10, 2021 at 10:59:39AM +0200, Emanuele Giuseppe Esposito wrote:
> Progressmeter is protected by the AioContext mutex, which
> is taken by the block jobs and their caller (like blockdev).
> 
> We would like to remove the dependency of block layer code on the
> AioContext mutex, since most drivers and the core I/O code are already
> not relying on it.
> 
> The simplest thing to do is to add a mutex to be able to provide
> an accurate snapshot of the progress values to the caller.
> 
> Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
> ---
>  blockjob.c                    | 33 +++++++++++++++++++++++++--------
>  include/qemu/progress_meter.h | 31 +++++++++++++++++++++++++++++++
>  job-qmp.c                     |  8 ++++++--
>  job.c                         |  3 +++
>  qemu-img.c                    |  9 ++++++---
>  5 files changed, 71 insertions(+), 13 deletions(-)

Unlike the next two patches where I had comments, here adding the lock
makes sense to me - the point of the progress meter is to report numbers
to the monitor thread so thread-safety is a key part of the API's value.

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>