From patchwork Wed Jun 11 10:33:12 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Siddhesh Poyarekar X-Patchwork-Id: 358596 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 98EDB1400E0 for ; Wed, 11 Jun 2014 20:32:00 +1000 (EST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:date:from:to:cc:subject:message-id:references :mime-version:content-type:in-reply-to; q=dns; s=default; b=nHTy +vbxi/lW3/+bLXlNUupf4DZt2UtVpxBqKCdKpMf+cfnIpCktfU78MSED/7OorNFr 2V1kJppa13PU7OYCSgkYLP7sTdU/qS3XnKVZ01v7MerXApf0EMvJFSkP4gfPaeOu b3qgVAi7ycAqIFWIgJMqgCLiZF6nVpahbexqu0E= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:date:from:to:cc:subject:message-id:references :mime-version:content-type:in-reply-to; s=default; bh=LHAC+Ky7gp fN/iB1+SpQ63ghppg=; b=dxTNr5LFM6IuqdfTQdKinA4u0aXsAdaSVId7v4CCIU 2g2Qh1V2z2rfbSUZ2cYzNIRQLJFB0IbiqE3NJrx4pFkE2aHzEeGiqkrZ/ZuTF2tg PoiIT7/NCboCDH1+ecAF1z9+WPLCCkpBmmXdVsIV+M0C70P9QIdxqTqhmosyF4YQ Y= Received: (qmail 17389 invoked by alias); 11 Jun 2014 10:31:53 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 17379 invoked by uid 89); 11 Jun 2014 10:31:52 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-3.9 required=5.0 tests=AWL, BAYES_00, RP_MATCHES_RCVD, SPF_HELO_PASS, SPF_PASS autolearn=ham version=3.3.2 X-HELO: mx1.redhat.com Date: Wed, 11 Jun 2014 16:03:12 +0530 From: Siddhesh Poyarekar To: Pedro Alves Cc: "Joseph S. Myers" , libc-alpha@sourceware.org Subject: Re: Patch not showing in patchwork Message-ID: <20140611103312.GE24193@spoyarek.pnq.redhat.com> References: <53976045.8050602@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <53976045.8050602@redhat.com> User-Agent: Mutt/1.5.22.1-rc1 (2013-10-16) On Tue, Jun 10, 2014 at 08:45:09PM +0100, Pedro Alves wrote: > On 06/10/2014 05:39 PM, Joseph S. Myers wrote: > > I noticed that my patch > > does not > > appear in the list of patches in patchwork (it appears not to have a patch > > entry in patchwork at all, rather than having been mistakenly marked as > > committed). Is there some way to tell why it didn't get recognised by > > patchwork? > > > > I'll guess due to: > > Content-Type: text/plain; charset="X-UNKNOWN" This was the problem. I have pushed the patch manually: http://patchwork.sourceware.org/patch/1430/ I have also applied the following patch to patchwork to allow it to fall back on alternate encodings if the charset is unknown. I'll post it to the patchwork mailing list. Siddhesh --- a/apps/patchwork/bin/parsemail.py 2014-06-11 15:53:12.685666812 +0530 +++ b/apps/patchwork/bin/parsemail.py 2014-06-11 15:53:03.991667186 +0530 @@ -147,6 +147,13 @@ return match.group(1) return None +def try_decode(payload, charset): + try: + payload = unicode(payload, charset) + except UnicodeDecodeError: + return None + return payload + def find_content(project, mail): patchbuf = None commentbuf = '' @@ -157,15 +164,27 @@ continue payload = part.get_payload(decode=True) - charset = part.get_content_charset() subtype = part.get_content_subtype() - # if we don't have a charset, assume utf-8 - if charset is None: - charset = 'utf-8' - if not isinstance(payload, unicode): - payload = unicode(payload, charset) + charset = part.get_content_charset() + + # If there is no charset or if it is unknown, then try some common + # charsets before we fail. + if charset is None or charset == 'x-unknown': + try_charsets = ['utf-8', 'windows-1252', 'ascii', 'iso-8859'] + else: + try_charsets = [charset] + + for cset in try_charsets: + decoded_payload = try_decode(payload, cset) + if decoded_payload is not None: + break + payload = decoded_payload + + # Could not find a valid decoded payload. Fail. + if payload is None: + return (None, None) if subtype in ['x-patch', 'x-diff']: patchbuf = payload