| Message ID | 1559808003-1030-1-git-send-email-yanjun.zhu@oracle.com |
|---|---|
| State | Accepted |
| Delegated to: | David Miller |
| Headers | show |
| Series | [1/1] net: rds: fix memory leak in rds_ib_flush_mr_pool | expand |
On 6/6/19 1:00 AM, Zhu Yanjun wrote: > When the following tests last for several hours, the problem will occur. > > Server: > rds-stress -r 1.1.1.16 -D 1M > Client: > rds-stress -r 1.1.1.14 -s 1.1.1.16 -D 1M -T 30 > > The following will occur. > > " > Starting up.... > tsks tx/s rx/s tx+rx K/s mbi K/s mbo K/s tx us/c rtt us cpu > % > 1 0 0 0.00 0.00 0.00 0.00 0.00 -1.00 > 1 0 0 0.00 0.00 0.00 0.00 0.00 -1.00 > 1 0 0 0.00 0.00 0.00 0.00 0.00 -1.00 > 1 0 0 0.00 0.00 0.00 0.00 0.00 -1.00 > " > From vmcore, we can find that clean_list is NULL. > > From the source code, rds_mr_flushd calls rds_ib_mr_pool_flush_worker. > Then rds_ib_mr_pool_flush_worker calls > " > rds_ib_flush_mr_pool(pool, 0, NULL); > " > Then in function > " > int rds_ib_flush_mr_pool(struct rds_ib_mr_pool *pool, > int free_all, struct rds_ib_mr **ibmr_ret) > " > ibmr_ret is NULL. > > In the source code, > " > ... > list_to_llist_nodes(pool, &unmap_list, &clean_nodes, &clean_tail); > if (ibmr_ret) > *ibmr_ret = llist_entry(clean_nodes, struct rds_ib_mr, llnode); > > /* more than one entry in llist nodes */ > if (clean_nodes->next) > llist_add_batch(clean_nodes->next, clean_tail, &pool->clean_list); > ... > " > When ibmr_ret is NULL, llist_entry is not executed. clean_nodes->next > instead of clean_nodes is added in clean_list. > So clean_nodes is discarded. It can not be used again. > The workqueue is executed periodically. So more and more clean_nodes are > discarded. Finally the clean_list is NULL. > Then this problem will occur. > > Fixes: 1bc144b62524 ("net, rds, Replace xlist in net/rds/xlist.h with llist") > Signed-off-by: Zhu Yanjun <yanjun.zhu@oracle.com> > --- Thanks. Acked-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>
From: Zhu Yanjun <yanjun.zhu@oracle.com> Date: Thu, 6 Jun 2019 04:00:03 -0400 > When the following tests last for several hours, the problem will occur. ... > When ibmr_ret is NULL, llist_entry is not executed. clean_nodes->next > instead of clean_nodes is added in clean_list. > So clean_nodes is discarded. It can not be used again. > The workqueue is executed periodically. So more and more clean_nodes are > discarded. Finally the clean_list is NULL. > Then this problem will occur. > > Fixes: 1bc144b62524 ("net, rds, Replace xlist in net/rds/xlist.h with llist") > Signed-off-by: Zhu Yanjun <yanjun.zhu@oracle.com> Applied and queued up for -stable.
diff --git a/net/rds/ib_rdma.c b/net/rds/ib_rdma.c index d664e9a..0b347f4 100644 --- a/net/rds/ib_rdma.c +++ b/net/rds/ib_rdma.c @@ -428,12 +428,14 @@ int rds_ib_flush_mr_pool(struct rds_ib_mr_pool *pool, wait_clean_list_grace(); list_to_llist_nodes(pool, &unmap_list, &clean_nodes, &clean_tail); - if (ibmr_ret) + if (ibmr_ret) { *ibmr_ret = llist_entry(clean_nodes, struct rds_ib_mr, llnode); - + clean_nodes = clean_nodes->next; + } /* more than one entry in llist nodes */ - if (clean_nodes->next) - llist_add_batch(clean_nodes->next, clean_tail, &pool->clean_list); + if (clean_nodes) + llist_add_batch(clean_nodes, clean_tail, + &pool->clean_list); }
When the following tests last for several hours, the problem will occur. Server: rds-stress -r 1.1.1.16 -D 1M Client: rds-stress -r 1.1.1.14 -s 1.1.1.16 -D 1M -T 30 The following will occur. " Starting up.... tsks tx/s rx/s tx+rx K/s mbi K/s mbo K/s tx us/c rtt us cpu % 1 0 0 0.00 0.00 0.00 0.00 0.00 -1.00 1 0 0 0.00 0.00 0.00 0.00 0.00 -1.00 1 0 0 0.00 0.00 0.00 0.00 0.00 -1.00 1 0 0 0.00 0.00 0.00 0.00 0.00 -1.00 " From vmcore, we can find that clean_list is NULL. From the source code, rds_mr_flushd calls rds_ib_mr_pool_flush_worker. Then rds_ib_mr_pool_flush_worker calls " rds_ib_flush_mr_pool(pool, 0, NULL); " Then in function " int rds_ib_flush_mr_pool(struct rds_ib_mr_pool *pool, int free_all, struct rds_ib_mr **ibmr_ret) " ibmr_ret is NULL. In the source code, " ... list_to_llist_nodes(pool, &unmap_list, &clean_nodes, &clean_tail); if (ibmr_ret) *ibmr_ret = llist_entry(clean_nodes, struct rds_ib_mr, llnode); /* more than one entry in llist nodes */ if (clean_nodes->next) llist_add_batch(clean_nodes->next, clean_tail, &pool->clean_list); ... " When ibmr_ret is NULL, llist_entry is not executed. clean_nodes->next instead of clean_nodes is added in clean_list. So clean_nodes is discarded. It can not be used again. The workqueue is executed periodically. So more and more clean_nodes are discarded. Finally the clean_list is NULL. Then this problem will occur. Fixes: 1bc144b62524 ("net, rds, Replace xlist in net/rds/xlist.h with llist") Signed-off-by: Zhu Yanjun <yanjun.zhu@oracle.com> --- net/rds/ib_rdma.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)