From patchwork Mon Oct 10 12:21:39 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liping Zhang X-Patchwork-Id: 680384 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3sszlH4YYlz9s3s for ; Mon, 10 Oct 2016 23:22:03 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b=sHguwtw2; dkim-atps=neutral Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752078AbcJJMVm (ORCPT ); Mon, 10 Oct 2016 08:21:42 -0400 Received: from mail-ua0-f169.google.com ([209.85.217.169]:33541 "EHLO mail-ua0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750708AbcJJMVk (ORCPT ); Mon, 10 Oct 2016 08:21:40 -0400 Received: by mail-ua0-f169.google.com with SMTP id p102so84010155uap.0; Mon, 10 Oct 2016 05:21:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=D4GIb5M8KHYWL4cnDSIie0BFimLI8f1sgQGcMhVb2DU=; b=sHguwtw2QSbaXbG6QDwjmXOrFLFCjUDqF1u3LJMP9vOe4WOP3YozkTS5yuOugJ07BM v1nj4UJT9A+nSqRXfsv1nOyMV1/MQTYHWGiqZgfels5v0BnLkjJsFImhTB+EPJ4V4oiY l/l0elGq2nd9R9i9yGDUrV9KN3OvAISxz9fmMJCnEdS1dif/N52bku01gokYWCbnmeid +ONfpcREFLki5N2GDgcTbOLohOHF4/GQUU5KiYhK89238KtjdrSnIfbLnreUu9nVfejo VXxIa6y5GEcQFHObqAtHg2aXi/xP4PPNn3L5MxunBsSSMPzvs3doMxnpzJpGvVOseL86 jEKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=D4GIb5M8KHYWL4cnDSIie0BFimLI8f1sgQGcMhVb2DU=; b=m7QQSd/+FODq0UYVsK3kfTMdNBasOZjAaFKZ0/6s1rtLG+358mfa6dBDq/u64P4qzE XIf73p1zF33Atq4Rman3VhldPLw3vUY2hCZeE8eYogXZ6jWrGyy/0QgBrYvesB9SKkq3 xtYin1ASXR9LXmfzbeIUqyKV+EeUIQzaJ1u24f3E2hEG+udpI7rCpai8IsK2PiZjnrMZ 36pazgNIim3sdX5ILyxaevzNiRaGahVDxnAtpWxbOvTbgAuGVMWoFd0rmQvhkE/ZrxG/ Gzr6S1Yzdeq6lGDT0QhAY5xiZYkxjp+xKZ2AJJuZ+X2F+Gg9P0X1iKA2EylaWk3Y7iFl Yarg== X-Gm-Message-State: AA6/9Rn4QWroD7yn0ZMeYMNZYPP78udx1uT4rwywRqfKWyHDtzjLqvxX5Zc4PbZxNg5IPvOctLLuP0/tjLwRcA== X-Received: by 10.176.4.102 with SMTP id 93mr6337053uav.143.1476102099661; Mon, 10 Oct 2016 05:21:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.56.21 with HTTP; Mon, 10 Oct 2016 05:21:39 -0700 (PDT) In-Reply-To: References: From: Liping Zhang Date: Mon, 10 Oct 2016 20:21:39 +0800 Message-ID: Subject: Re: kernel v4.8: iptables logs are truncated with the 4.8 kernel? To: Chris Caputo Cc: Vishwanath Pai , Pablo Neira Ayuso , Justin Piszcz , linux-kernel@vger.kernel.org, Linux Kernel Network Developers Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Hi Chris, 2016-10-10 15:02 GMT+08:00 Chris Caputo : > On Tue, 4 Oct 2016, Justin Piszcz wrote: >> kernel 4.8 with ulogd-2.0.5- IPs are no longer logged: >> >> Oct 4 17:51:30 atom INPUT_BLOCK IN=eth1 OUT= >> MAC=00:1b:21:9c:3b:fa:3e:94:d5:d2:49:1e:08:00 LEN=0 TOS=00 PREC=0x00 >> TTL=0 ID=0 PROTO=0 MARK=0 >> Oct 4 17:51:31 atom INPUT_BLOCK IN=eth1 OUT= >> MAC=00:1b:21:9c:3b:fa:3e:94:d5:d2:49:1e:08:00 LEN=0 TOS=00 PREC=0x00 >> TTL=0 ID=0 PROTO=0 MARK=0 >> Oct 4 17:51:32 atom INPUT_BLOCK IN=eth1 OUT= >> MAC=00:1b:21:9c:3b:fa:3e:94:d5:d2:49:1e:08:00 LEN=0 TOS=00 PREC=0x00 >> TTL=0 ID=0 PROTO=0 MARK=0 >> >> (reboot back to kernel 4.7, works fine) >> >> kernel 4.7 with ulogd-2.0.5: >> Oct 4 17:56:44 atom INPUT_BLOCK IN=eth1 OUT= >> MAC=00:1b:21:9c:3b:fa:3e:94:d5:d2:49:1e:08:00 SRC=74.125.22.125 >> DST=1.2.3.4 LEN=397 TOS=00 PREC=0x00 TTL=48 ID=58093 PROTO=TCP >> SPT=5222 DPT=19804 SEQ=2032644254 ACK=2273184383 WINDOW=55272 ACK PSH >> URGP=0 MARK=0 >> Oct 4 17:56:45 atom INPUT_BLOCK IN=eth1 OUT= >> MAC=00:1b:21:9c:3b:fa:3e:94:d5:d2:49:1e:08:00 SRC=74.125.22.125 >> DST=1.2.3.4 LEN=397 TOS=00 PREC=0x00 TTL=48 ID=58725 PROTO=TCP >> SPT=5222 DPT=19804 SEQ=2032644254 ACK=2273184383 WINDOW=55272 ACK PSH >> URGP=0 MARK=0 >> >> Looks like there were some changes in the 4.8 kernel regarding ulogd, >> has anyone else run into this problem? > > For me, kernel 4.8.1 results in segfaults in ulogd-2.0.5 at: > > Program received signal SIGSEGV, Segmentation fault. > 0x00007ffff65fd18a in _interp_iphdr (pi=0x617f50, len=0) at ulogd_raw2packet_BASE.c:720 > > 715 static int _interp_iphdr(struct ulogd_pluginstance *pi, uint32_t len) > 716 { > 717 struct ulogd_key *ret = pi->output.keys; > 718 struct iphdr *iph = > 719 ikey_get_ptr(&pi->input.keys[INKEY_RAW_PCKT]); > 720 void *nexthdr = (uint32_t *)iph + iph->ihl; > > I believe 7643507fe8b5bd8ab7522f6a81058cc1209d2585 changed previous > behavior by not always copying IP header data to user space. > > On my machine IPv4 log packets result in a ulogd segfault while IPv6 > packets do not. I'm not sure of the cause of the difference. > > The corresponding userspace commit for the 209d2585 kernel change is: > > https://git.netfilter.org/iptables/commit/?id=7070b1f3c88a0c3d4e315c00cca61f05b0fbc882 > > This adds --nflog-size to iptables. When --nflog-size is used with my > iptables NFLOG lines, the ulogd-2.0.5 segfaults cease. What numbers did you specify after --nflog-size option? --nflog-size 0 or ...? If you want log the whole packet to the ulogd, please do not specify this nflog-size option. > > I'm surprised to see a kernel change cause unexpected userspace segfaults, > so further investigation into a kernel fix would seem a good idea. According to the original user's manual, nflog-range option was designed to be the number of bytes copied to userspace, but unfortunately there's a bug from the beginning and it never works, i.e. in kernel, it just ignored this option. Try to change the current nflog-range option's semantics may cause unexpected results(maybe like this ulogd crash) ... In order to keep compatibility, Vishwanath introduce a new nflog-size option and keep nflog-range unchanged. If you just upgrade the kernel, and do not change iptables rules, this problem will not happen. So I think this is ulogd's bug, in _interp_iphdr, it try to dereference the iphdr pointer before validation check, meanwhile this problem does not exist in ipv6 path. Can you try this patch: return ULOGD_IRET_OK; @@ -734,6 +734,7 @@ static int _interp_iphdr(struct ulogd_pluginstance *pi, uint32_t len) okey_set_u16(&ret[KEY_IP_ID], ntohs(iph->id)); okey_set_u16(&ret[KEY_IP_FRAGOFF], ntohs(iph->frag_off)); + nexthdr = (uint32_t *)iph + iph->ihl; switch (iph->protocol) { case IPPROTO_TCP: _interp_tcp(pi, nexthdr, len); Thanks > Having to add the likes of "--nflog-size 200" (200 simply being what I am > using) to every NFLOG line in firewall configs is a significant burden for > many. > > Putting out a new release of iptables may help ease this transition if the > kernel is not patched to fix this. I had to use the git code since 1.6.0 > doesn't have it. > > Chris diff --git a/filter/raw2packet/ulogd_raw2packet_BASE.c b/filter/raw2packet/ulogd_raw2packet_BASE.c index 8a6180c..fd2665a 100644 --- a/filter/raw2packet/ulogd_raw2packet_BASE.c +++ b/filter/raw2packet/ulogd_raw2packet_BASE.c @@ -717,7 +717,7 @@ static int _interp_iphdr(struct ulogd_pluginstance *pi, uint32_t len) struct ulogd_key *ret = pi->output.keys; struct iphdr *iph = ikey_get_ptr(&pi->input.keys[INKEY_RAW_PCKT]); - void *nexthdr = (uint32_t *)iph + iph->ihl; + void *nexthdr; if (len < sizeof(struct iphdr) || len <= (uint32_t)(iph->ihl * 4))