| Message ID | 20170801063942.29591-1-nikunj@linux.vnet.ibm.com |
|---|---|
| State | Superseded |
| Headers | show |
On 01.08.2017 08:39, Nikunj A Dadhania wrote: > Found that virtio-net is using a around 200K receive buffer per device, if we > connect more than 40 virtio-net devices the heap(8MB) gets over. Because of > which allocation starts failing and the VM does not boot. > > Use common receive buffer for all the virtio-net devices. This will reduce the > memory requirement per virtio-net device in slof. Not sure whether this is a really good idea... I think theoretically an OF client could open multiple network devices and use them in parallel - and then the devices would destroy their shared receive buffers mutually. Maybe you could change the code so that the buffers are only allocated when the device is opened, and released again when the device is closed again? Thomas
Thomas Huth <thuth@redhat.com> writes: > On 01.08.2017 08:39, Nikunj A Dadhania wrote: >> Found that virtio-net is using a around 200K receive buffer per device, if we >> connect more than 40 virtio-net devices the heap(8MB) gets over. Because of >> which allocation starts failing and the VM does not boot. >> >> Use common receive buffer for all the virtio-net devices. This will reduce the >> memory requirement per virtio-net device in slof. > > Not sure whether this is a really good idea... I think theoretically an > OF client could open multiple network devices and use them in parallel > - True, but looking at the virtio-net driver, i dont think it supports multiple network devices. I see the global struct virtio_device being used throughout the driver. > and then the devices would destroy their shared receive buffers mutually. > Maybe you could change the code so that the buffers are only allocated > when the device is opened, and released again when the device is closed > again? This should be possible by re-designing the driver to handle multiple device open scenario. Let me have a quick look if this can be done. Regards Nikunj
diff --git a/lib/libvirtio/virtio-net.c b/lib/libvirtio/virtio-net.c index 2573031..a93ad7a 100644 --- a/lib/libvirtio/virtio-net.c +++ b/lib/libvirtio/virtio-net.c @@ -43,6 +43,7 @@ struct virtio_device virtiodev; static struct vqs vq_rx; /* Information about receive virtqueues */ static struct vqs vq_tx; /* Information about transmit virtqueues */ +static void *rx_buf_mem; /* See Virtio Spec, appendix C, "Device Operation" */ struct virtio_net_hdr { @@ -137,12 +138,19 @@ static int virtionet_init(net_driver_t *driver) virtio_set_guest_features(&virtiodev, 0); } - /* Allocate memory for one transmit an multiple receive buffers */ - vq_rx.buf_mem = SLOF_alloc_mem((BUFFER_ENTRY_SIZE+net_hdr_size) - * RX_QUEUE_SIZE); - if (!vq_rx.buf_mem) { - printf("virtionet: Failed to allocate buffers!\n"); - goto dev_error; + /* Use common buffer to save memory space per device */ + if (!rx_buf_mem) { + /* Allocate memory for one transmit and multiple receive + * buffers + */ + rx_buf_mem = SLOF_alloc_mem((BUFFER_ENTRY_SIZE+net_hdr_size) + * RX_QUEUE_SIZE); + if (!rx_buf_mem) { + printf("virtionet: Failed to allocate buffers!\n"); + goto dev_error; + } + } else { + vq_rx.buf_mem = rx_buf_mem; } /* Prepare receive buffer queue */
Found that virtio-net is using a around 200K receive buffer per device, if we connect more than 40 virtio-net devices the heap(8MB) gets over. Because of which allocation starts failing and the VM does not boot. Use common receive buffer for all the virtio-net devices. This will reduce the memory requirement per virtio-net device in slof. Signed-off-by: Nikunj A Dadhania <nikunj@linux.vnet.ibm.com> --- lib/libvirtio/virtio-net.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-)