| Message ID | 20100825110049.F3C9.A69D9226@jp.fujitsu.com |
|---|---|
| State | Accepted, archived |
| Delegated to: | David Miller |
| Headers | show |
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Date: Wed, 25 Aug 2010 11:05:48 +0900 (JST) > This issue come from ruby language community. Below test program > hang up when only run on Linux. > > % uname -mrsv > Linux 2.6.26-2-486 #1 Sat Dec 26 08:37:39 UTC 2009 i686 > % ruby -rsocket -ve ' > BasicSocket.do_not_reverse_lookup = true > serv = TCPServer.open("127.0.0.1", 0) > s1 = TCPSocket.open("127.0.0.1", serv.addr[1]) > s2 = serv.accept > s2.close > s1.write("a") rescue p $! > s1.write("a") rescue p $! > Thread.new { > s1.write("a") > }.join' > ruby 1.9.3dev (2010-07-06 trunk 28554) [i686-linux] > #<Errno::EPIPE: Broken pipe> > [Hang Here] > > FreeBSD, Solaris, Mac doesn't. because Ruby's write() method call > select() internally. and tcp_poll has a bug. In your opinion. > SUS defined 'ready for writing' of select() as following. > > | A descriptor shall be considered ready for writing when a call to an output > | function with O_NONBLOCK clear would not block, whether or not the function > | would transfer data successfully. > > That said, EPIPE situation is clearly one of 'ready for writing'. How Linux should behave is defined by many things, and often it is simply defined by how we've behaved for a very long time. This is because changing behavior can often break as many applications as it can fix. Standards don't necessarily tell us how we must behave, since often is it impractical to follow their definions. And in this case here, I call into question the behavior of Ruby and the application from two perspectives: 1) Unlike all of the other conditions signalled by poll() this is one the application explicitly created and therefore knows about. If the application calls close() or shutdown() with the send flag set, IT KNOWS what is going to happen on a write() attempt. 2) Ruby and this script will have to deal with the past 13 years worth of Linux kernels. Even if I were to apply this fix now it is not going to propagate onto a user's system any time soon. Many systems would never ever get this fix. Therefore it behooves Ruby and this script to make a very reasonable change, which is to track when close() or send shutdown() calls occur and behave appropriately on a write() call. I'm therefore not applying this patch, because not only can applications handle this properly with information they already have, the change has the potential to break applications. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, 2010-08-25 at 15:34 -0700, David Miller wrote: > From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> > Date: Wed, 25 Aug 2010 11:05:48 +0900 (JST) > > > This issue come from ruby language community. Below test program > > hang up when only run on Linux. > > > > % uname -mrsv > > Linux 2.6.26-2-486 #1 Sat Dec 26 08:37:39 UTC 2009 i686 > > % ruby -rsocket -ve ' > > BasicSocket.do_not_reverse_lookup = true > > serv = TCPServer.open("127.0.0.1", 0) > > s1 = TCPSocket.open("127.0.0.1", serv.addr[1]) > > s2 = serv.accept > > s2.close > > s1.write("a") rescue p $! > > s1.write("a") rescue p $! > > Thread.new { > > s1.write("a") > > }.join' > > ruby 1.9.3dev (2010-07-06 trunk 28554) [i686-linux] > > #<Errno::EPIPE: Broken pipe> > > [Hang Here] [...] > And in this case here, I call into question the behavior of Ruby and > the application from two perspectives: > > 1) Unlike all of the other conditions signalled by poll() this is > one the application explicitly created and therefore knows about. > > If the application calls close() or shutdown() with the send flag > set, IT KNOWS what is going to happen on a write() attempt. [...] This example seems to have both server (serv, s2) and client (s1) in the same process for simplicity. The server socket (s2) is closed and the client cannot be expected to know that. Of course the client ought to drop the connection after the first EPIPE, but it's reasonable to expect that this is a sticky condition just as it would be for a pipe. Here's a similar test case in C: #include <assert.h> #include <signal.h> #include <stdio.h> #include <sys/select.h> #include <sys/socket.h> #include <sys/types.h> #include <unistd.h> int main(void) { struct sockaddr sa; struct timeval tv; int serv, s1, s2; socklen_t len; fd_set fds; signal(SIGPIPE, SIG_IGN); serv = socket(AF_INET, SOCK_STREAM, 0); assert(serv >= 0); assert(!listen(serv, 1)); len = sizeof(sa); assert(!getsockname(serv, &sa, &len)); s1 = socket(AF_INET, SOCK_STREAM, 0); assert(s1 >= 0); assert(!connect(s1, &sa, len)); len = sizeof(sa); s2 = accept(serv, &sa, &len); assert(s2 >= 0); close(s2); for (;;) { printf("write: %d\n", write(s1, "a", 1)); FD_ZERO(&fds); FD_SET(s1, &fds); tv.tv_sec = 1; tv.tv_usec = 0; printf("select: %d\n", select(s1 + 1, NULL, &fds, NULL, &tv)); } return 0; } Ben.
Hi Thank you for quick responce! >> SUS defined 'ready for writing' of select() as following. >> >> | A descriptor shall be considered ready for writing when a call to an output >> | function with O_NONBLOCK clear would not block, whether or not the function >> | would transfer data successfully. >> >> That said, EPIPE situation is clearly one of 'ready for writing'. > > How Linux should behave is defined by many things, and often it is > simply defined by how we've behaved for a very long time. This is > because changing behavior can often break as many applications as it > can fix. Standards don't necessarily tell us how we must behave, > since often is it impractical to follow their definions. > > And in this case here, I call into question the behavior of Ruby and > the application from two perspectives: > > 1) Unlike all of the other conditions signalled by poll() this is > one the application explicitly created and therefore knows about. > > If the application calls close() or shutdown() with the send flag > set, IT KNOWS what is going to happen on a write() attempt. Umm?? Probably my example is not so good. That's not my point. In the example application, client and server socket is in the same process. But it's NOT generic. usually, client and server are another process. then, client can't expect when server close socket. The most big matter is, this is can't be avoided in userland. In addition, EVERY application don't want application hang up. we don't hesitate userland change. > 2) Ruby and this script will have to deal with the past 13 years > worth of Linux kernels. Even if I were to apply this fix now > it is not going to propagate onto a user's system any time soon. > > Many systems would never ever get this fix. > > Therefore it behooves Ruby and this script to make a very reasonable > change, which is to track when close() or send shutdown() calls occur > and behave appropriately on a write() call. > > I'm therefore not applying this patch, because not only can applications > handle this properly with information they already have, the change has > the potential to break applications. At first, I was thinking two fix plan. 1) this patch 2) adding POLLWRHUP as POLLRDHUP. However I couldn't find any regression rick in (1). then I did choice (1). So, Can you please tell us what rick you worry? My thinking is, If select(writefds) returned, an application naturally call to write. (why not? If not, why do you call select?) and write return EPIPE. every network application have EPIPE error checking. But, there is any rick. I can remake a patch as (2). Thanks. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Date: Thu, 26 Aug 2010 12:09:55 +0900 > Probably my example is not so good. That's not my point. > In the example application, client and server socket is in the same process. > But it's NOT generic. usually, client and server are another process. then, > client can't expect when server close socket. > > The most big matter is, this is can't be avoided in userland. In addition, > EVERY application don't want application hang up. we don't hesitate > userland change. Ok, you and Ben Hutchings have convinced me to reconsider. And this matches what even BSD4.4-Lite does (I checked yesterday before my initial reply), so I will apply this patch. Thanks for your patience. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> On Wed, 2010-08-25 at 15:34 -0700, David Miller wrote: > > From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> > > Date: Wed, 25 Aug 2010 11:05:48 +0900 (JST) > > > > > This issue come from ruby language community. Below test program > > > hang up when only run on Linux. > > > > > > % uname -mrsv > > > Linux 2.6.26-2-486 #1 Sat Dec 26 08:37:39 UTC 2009 i686 > > > % ruby -rsocket -ve ' > > > BasicSocket.do_not_reverse_lookup = true > > > serv = TCPServer.open("127.0.0.1", 0) > > > s1 = TCPSocket.open("127.0.0.1", serv.addr[1]) > > > s2 = serv.accept > > > s2.close > > > s1.write("a") rescue p $! > > > s1.write("a") rescue p $! > > > Thread.new { > > > s1.write("a") > > > }.join' > > > ruby 1.9.3dev (2010-07-06 trunk 28554) [i686-linux] > > > #<Errno::EPIPE: Broken pipe> > > > [Hang Here] > [...] > > And in this case here, I call into question the behavior of Ruby and > > the application from two perspectives: > > > > 1) Unlike all of the other conditions signalled by poll() this is > > one the application explicitly created and therefore knows about. > > > > If the application calls close() or shutdown() with the send flag > > set, IT KNOWS what is going to happen on a write() attempt. > [...] > > This example seems to have both server (serv, s2) and client (s1) in the > same process for simplicity. The server socket (s2) is closed and the > client cannot be expected to know that. Of course the client ought to > drop the connection after the first EPIPE, but it's reasonable to expect > that this is a sticky condition just as it would be for a pipe. > > Here's a similar test case in C: > > #include <assert.h> > #include <signal.h> > #include <stdio.h> > > #include <sys/select.h> > #include <sys/socket.h> > #include <sys/types.h> > #include <unistd.h> > > int main(void) > { > struct sockaddr sa; > struct timeval tv; > int serv, s1, s2; > socklen_t len; > fd_set fds; > > signal(SIGPIPE, SIG_IGN); > > serv = socket(AF_INET, SOCK_STREAM, 0); > assert(serv >= 0); > assert(!listen(serv, 1)); > len = sizeof(sa); > assert(!getsockname(serv, &sa, &len)); > > s1 = socket(AF_INET, SOCK_STREAM, 0); > assert(s1 >= 0); > assert(!connect(s1, &sa, len)); > len = sizeof(sa); > > s2 = accept(serv, &sa, &len); > assert(s2 >= 0); > close(s2); > > for (;;) { > printf("write: %d\n", write(s1, "a", 1)); > FD_ZERO(&fds); > FD_SET(s1, &fds); > tv.tv_sec = 1; > tv.tv_usec = 0; > printf("select: %d\n", select(s1 + 1, NULL, &fds, NULL, &tv)); > } > return 0; > } Cool! Ben, I think your code is cleaner than mine. If you allow me, I hope to include this one into my patch description. Thanks. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> > Date: Thu, 26 Aug 2010 12:09:55 +0900 > > > Probably my example is not so good. That's not my point. > > In the example application, client and server socket is in the same process. > > But it's NOT generic. usually, client and server are another process. then, > > client can't expect when server close socket. > > > > The most big matter is, this is can't be avoided in userland. In addition, > > EVERY application don't want application hang up. we don't hesitate > > userland change. > > Ok, you and Ben Hutchings have convinced me to reconsider. > > And this matches what even BSD4.4-Lite does (I checked yesterday before > my initial reply), so I will apply this patch. > > Thanks for your patience. Great! Thank you!! -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, 2010-08-26 at 12:55 +0900, KOSAKI Motohiro wrote: [...] > Cool! > > Ben, I think your code is cleaner than mine. If you allow me, I hope to > include this one into my patch description. You can do that if you like, but it seems a bit long for a commit message. By the way, I was able to reproduce this bug in RHEL 3 (kernel based on 2.4.21) so it seems to have been around for a while. Ben.
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 176e11a..2497e48 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -451,7 +451,8 @@ unsigned int tcp_poll(struct file *file, struct socket *sock, poll_table *wait) if (sk_stream_wspace(sk) >= sk_stream_min_wspace(sk)) mask |= POLLOUT | POLLWRNORM; } - } + } else + mask |= POLLOUT | POLLWRNORM; if (tp->urg_data & TCP_URG_VALID) mask |= POLLPRI;
This issue come from ruby language community. Below test program hang up when only run on Linux. % uname -mrsv Linux 2.6.26-2-486 #1 Sat Dec 26 08:37:39 UTC 2009 i686 % ruby -rsocket -ve ' BasicSocket.do_not_reverse_lookup = true serv = TCPServer.open("127.0.0.1", 0) s1 = TCPSocket.open("127.0.0.1", serv.addr[1]) s2 = serv.accept s2.close s1.write("a") rescue p $! s1.write("a") rescue p $! Thread.new { s1.write("a") }.join' ruby 1.9.3dev (2010-07-06 trunk 28554) [i686-linux] #<Errno::EPIPE: Broken pipe> [Hang Here] FreeBSD, Solaris, Mac doesn't. because Ruby's write() method call select() internally. and tcp_poll has a bug. SUS defined 'ready for writing' of select() as following. | A descriptor shall be considered ready for writing when a call to an output | function with O_NONBLOCK clear would not block, whether or not the function | would transfer data successfully. That said, EPIPE situation is clearly one of 'ready for writing'. We don't have read-side issue because tcp_poll() already has read side shutdown care. | if (sk->sk_shutdown & RCV_SHUTDOWN) | mask |= POLLIN | POLLRDNORM | POLLRDHUP; So, Let's insert same logic in write side. - reference url http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/31065 http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/31068 Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> --- net/ipv4/tcp.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)