From patchwork Mon Jul 6 03:41:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1323284 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4B0WdN3X4mz9sRN for ; Mon, 6 Jul 2020 13:43:52 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=apzA3zi5; dkim-atps=neutral Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 4B0WdN2GSjzDqTT for ; Mon, 6 Jul 2020 13:43:52 +1000 (AEST) X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=chromium.org (client-ip=2607:f8b0:4864:20::d41; helo=mail-io1-xd41.google.com; envelope-from=sjg@chromium.org; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=apzA3zi5; dkim-atps=neutral Received: from mail-io1-xd41.google.com (mail-io1-xd41.google.com [IPv6:2607:f8b0:4864:20::d41]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4B0WbZ28v4zDqRZ for ; Mon, 6 Jul 2020 13:42:15 +1000 (AEST) Received: by mail-io1-xd41.google.com with SMTP id f6so22314755ioj.5 for ; Sun, 05 Jul 2020 20:42:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pkISjmh7lrMt/2QE0p4NbqcMjbl3sgvPTtSAJPZh/UE=; b=apzA3zi5Mz2qlOs5obO5r2Ptq+e6LwHu30BLyva1SlUIJ6U43J5np1CaOdXa+dP3Vk tO/T2joypE+HCNZTz7JWAY+1jechPMp++SODO+vaH3CkSJMxBhQfPhGa9CulISYJvvlc GJf9PVLYasaBu0gOaSkk8MzmSQYbQZFCtsDYM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pkISjmh7lrMt/2QE0p4NbqcMjbl3sgvPTtSAJPZh/UE=; b=tDhjISfhahbvW78Zngca+lqPvhYdoLfk8gNq6FMhvDO52O/Jzs0sCWsd3N4nmZp7kW T+TMn/YFRgXVqxFLNDXD/6rpkqnVjUZIJ9c4hs8lPwRkbQhMdWqEfRfZKSXcGgN9voIF rotcZSj/mlyiy08+h6AS8kF0M7HSZEQOnwCfpxg0LrGMHekmX4fteK9eOcXzBkIpAcid XaJ4YHJMFcVLI4EkxbeIHuTr5KBetTOS8Kqj2Hs8QE17UHEZso+7Q0tZ/ySnQYHn6s8I thCCqo0vjTowwNqlQxlOuvHVW/WeXQgxNPSHRzRNP5frAoa4zh528F2KSXZP0P138gc/ 4NxA== X-Gm-Message-State: AOAM530Qlo6go3nXNg8Qh/Fcd+tagTrkKRXPza0+CjBHQa9/fVJ0t0gp FBh7q0+gEUfF26qTU1vaUA3KYw== X-Google-Smtp-Source: ABdhPJyE3RHu33SbOuTP55o/3ARN1FoEyMd2dO0fVo0xpJbTwaXT5moHHlDPcFg9JZ9wSZ+tx9hq5Q== X-Received: by 2002:a5d:8417:: with SMTP id i23mr23572138ion.132.1594006932239; Sun, 05 Jul 2020 20:42:12 -0700 (PDT) Received: from localhost.localdomain (c-73-14-175-90.hsd1.co.comcast.net. [73.14.175.90]) by smtp.gmail.com with ESMTPSA id r15sm1664958ilh.86.2020.07.05.20.42.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 05 Jul 2020 20:42:11 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Subject: [RFC PATCH 02/16] patman: Move main code out to a control module Date: Sun, 5 Jul 2020 21:41:49 -0600 Message-Id: <20200706034203.2171077-3-sjg@chromium.org> X-Mailer: git-send-email 2.27.0.212.ge8ba1cc988-goog In-Reply-To: <20200706034203.2171077-1-sjg@chromium.org> References: <20200706034203.2171077-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Tom Rini , Simon Glass , patchwork@lists.ozlabs.org Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" To make testing easier, move the code out from main into a separate 'control' module and split it into four parts: setup, preparing patches, checking patches and emailing patches. Add comments and fix a few code-style issues while we are here. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- tools/patman/checkpatch.py | 6 ++ tools/patman/control.py | 170 +++++++++++++++++++++++++++++++++++++ tools/patman/gitutil.py | 4 +- tools/patman/main.py | 57 +------------ tools/patman/series.py | 2 +- 5 files changed, 182 insertions(+), 57 deletions(-) create mode 100644 tools/patman/control.py diff --git a/tools/patman/checkpatch.py b/tools/patman/checkpatch.py index 07c3e2739a..263bac3fc9 100644 --- a/tools/patman/checkpatch.py +++ b/tools/patman/checkpatch.py @@ -41,6 +41,12 @@ def FindCheckPatch(): def CheckPatch(fname, verbose=False, show_types=False): """Run checkpatch.pl on a file. + Args: + fname: Filename to check + verbose: True to print out every line of the checkpatch output as it is + parsed + show_types: Tell checkpatch to show the type (number) of each message + Returns: namedtuple containing: ok: False=failure, True=ok diff --git a/tools/patman/control.py b/tools/patman/control.py new file mode 100644 index 0000000000..a896c924b5 --- /dev/null +++ b/tools/patman/control.py @@ -0,0 +1,170 @@ +# SPDX-License-Identifier: GPL-2.0+ +# +# Copyright 2020 Google LLC +# +"""Handles the main control logic of patman + +This module provides various functions called by the main program to implement +the features of patman. +""" + +import os +import sys + +from patman import checkpatch +from patman import gitutil +from patman import patchstream +from patman import terminal + +def setup(): + """Do required setup before doing anything""" + gitutil.Setup() + +def prepare_patches(col, count, start, ignore_binary): + """Figure out what patches to generate, then generate them + + The patch files are written to the current directory, e.g. 0001_xxx.patch + 0002_yyy.patch + + Args: + col (terminal.Color): Colour output object + count (int): Number of patches to produce, or -1 to produce patches for + the current branch back to the upstream commit + start (int): Start partch to use (0=first / top of branch) + ignore_binary (bool): Don't generate patches for binary files + + Returns: + Tuple: + Series object for this series (set of patches) + Filename of the cover letter as a string (None if none) + patch_files: List of patch filenames, each a string, e.g. + ['0001_xxx.patch', '0002_yyy.patch'] + """ + if count == -1: + # Work out how many patches to send if we can + count = (gitutil.CountCommitsToBranch() - start) + + if not count: + sys.exit(col.Color(col.RED, + 'No commits found to process - please use -c flag')) + + # Read the metadata from the commits + to_do = count + series = patchstream.GetMetaData(start, to_do) + cover_fname, patch_files = gitutil.CreatePatches( + start, to_do, ignore_binary, series) + + # Fix up the patch files to our liking, and insert the cover letter + patchstream.FixPatches(series, patch_files) + if cover_fname and series.get('cover'): + patchstream.InsertCoverLetter(cover_fname, series, to_do) + return series, cover_fname, patch_files + +def check_patches(series, patch_files, run_checkpatch, verbose): + """Run some checks on a set of patches + + This santiy-checks the patman tags like Series-version and runs the patches + through checkpatch + + Args: + series (Series): Series object for this series (set of patches) + patch_files (list): List of patch filenames, each a string, e.g. + ['0001_xxx.patch', '0002_yyy.patch'] + run_checkpatch (bool): True to run checkpatch.pl + verbose (bool): True to print out every line of the checkpatch output as + it is parsed + + Returns: + bool: True if the patches had no errors, False if they did + """ + # Do a few checks on the series + series.DoChecks() + + # Check the patches, and run them through 'git am' just to be sure + if run_checkpatch: + ok = checkpatch.CheckPatches(verbose, patch_files) + else: + ok = True + return ok + + +def email_patches(col, series, cover_fname, patch_files, process_tags, its_a_go, + ignore_bad_tags, add_maintainers, limit, dry_run, in_reply_to, + thread, smtp_server): + """Email patches to the recipients + + This emails out the patches and cover letter using 'git send-email'. Each + patch is copied to recipients identified by the patch tag and output from + the get_maintainer.pl script. The cover letter is copied to all recipients + of any patch. + + To make this work a CC file is created holding the recipients for each patch + and the cover letter. See the main program 'cc_cmd' for this logic. + + Args: + col (terminal.Color): Colour output object + series (Series): Series object for this series (set of patches) + cover_fname (str): Filename of the cover letter as a string (None if + none) + patch_files (list): List of patch filenames, each a string, e.g. + ['0001_xxx.patch', '0002_yyy.patch'] + process_tags (bool): True to process subject tags in each patch, e.g. + for 'dm: spi: Add SPI support' this would be 'dm' and 'spi'. The + tags are looked up in the configured sendemail.aliasesfile and also + in ~/.patman (see README) + its_a_go (bool): True if we are going to actually send the patches, + False if the patches have errors and will not be sent unless + @ignore_errors + ignore_bad_tags (bool): True to just print a warning for unknown tags, + False to halt with an error + add_maintainers (bool): Run the get_maintainer.pl script for each patch + limit (int): Limit on the number of people that can be cc'd on a single + patch or the cover letter (None if no limit) + dry_run (bool): Don't actually email the patches, just print out what + would be sent + in_reply_to (str): If not None we'll pass this to git as --in-reply-to. + Should be a message ID that this is in reply to. + thread (bool): True to add --thread to git send-email (make all patches + reply to cover-letter or first patch in series) + smtp_server (str): SMTP server to use to send patches (None for default) + """ + cc_file = series.MakeCcFile(process_tags, cover_fname, not ignore_bad_tags, + add_maintainers, limit) + + # Email the patches out (giving the user time to check / cancel) + cmd = '' + if its_a_go: + cmd = gitutil.EmailPatches( + series, cover_fname, patch_files, dry_run, not ignore_bad_tags, + cc_file, in_reply_to=in_reply_to, thread=thread, + smtp_server=smtp_server) + else: + print(col.Color(col.RED, "Not sending emails due to errors/warnings")) + + # For a dry run, just show our actions as a sanity check + if dry_run: + series.ShowActions(patch_files, cmd, process_tags) + if not its_a_go: + print(col.Color(col.RED, "Email would not be sent")) + + os.remove(cc_file) + +def send(options): + """Create, check and send patches by email + + Args: + options (optparse.Values): Arguments to patman + """ + setup() + col = terminal.Color() + series, cover_fname, patch_files = prepare_patches( + col, options.count, options.start, options.ignore_binary) + ok = check_patches(series, patch_files, options.check_patch, + options.verbose) + its_a_go = ok or options.ignore_errors + if its_a_go: + email_patches( + col, series, cover_fname, patch_files, options.process_tags, + its_a_go, options.ignore_bad_tags, options.add_maintainers, + options.limit, options.dry_run, options.in_reply_to, options.thread, + options.smtp_server) diff --git a/tools/patman/gitutil.py b/tools/patman/gitutil.py index 5189840eab..29444bf8e9 100644 --- a/tools/patman/gitutil.py +++ b/tools/patman/gitutil.py @@ -261,8 +261,10 @@ def CreatePatches(start, count, ignore_binary, series): Args: start: Commit to start from: 0=HEAD, 1=next one, etc. count: number of commits to include + ignore_binary: Don't generate patches for binary files + series: Series object for this series (set of patches) Return: - Filename of cover letter + Filename of cover letter (None if none) List of filenames of patch files """ if series.get('version'): diff --git a/tools/patman/main.py b/tools/patman/main.py index 03668d1bb8..2432d31871 100755 --- a/tools/patman/main.py +++ b/tools/patman/main.py @@ -18,10 +18,9 @@ if __name__ == "__main__": sys.path.append(os.path.join(our_path, '..')) # Our modules -from patman import checkpatch from patman import command +from patman import control from patman import gitutil -from patman import patchstream from patman import project from patman import settings from patman import terminal @@ -128,56 +127,4 @@ elif options.full_help: # Process commits, produce patches files, check them, email them else: - gitutil.Setup() - - if options.count == -1: - # Work out how many patches to send if we can - options.count = gitutil.CountCommitsToBranch() - options.start - - col = terminal.Color() - if not options.count: - str = 'No commits found to process - please use -c flag' - sys.exit(col.Color(col.RED, str)) - - # Read the metadata from the commits - if options.count: - series = patchstream.GetMetaData(options.start, options.count) - cover_fname, args = gitutil.CreatePatches(options.start, options.count, - options.ignore_binary, series) - - # Fix up the patch files to our liking, and insert the cover letter - patchstream.FixPatches(series, args) - if cover_fname and series.get('cover'): - patchstream.InsertCoverLetter(cover_fname, series, options.count) - - # Do a few checks on the series - series.DoChecks() - - # Check the patches, and run them through 'git am' just to be sure - if options.check_patch: - ok = checkpatch.CheckPatches(options.verbose, args) - else: - ok = True - - cc_file = series.MakeCcFile(options.process_tags, cover_fname, - not options.ignore_bad_tags, - options.add_maintainers, options.limit) - - # Email the patches out (giving the user time to check / cancel) - cmd = '' - its_a_go = ok or options.ignore_errors - if its_a_go: - cmd = gitutil.EmailPatches(series, cover_fname, args, - options.dry_run, not options.ignore_bad_tags, cc_file, - in_reply_to=options.in_reply_to, thread=options.thread, - smtp_server=options.smtp_server) - else: - print(col.Color(col.RED, "Not sending emails due to errors/warnings")) - - # For a dry run, just show our actions as a sanity check - if options.dry_run: - series.ShowActions(args, cmd, options.process_tags) - if not its_a_go: - print(col.Color(col.RED, "Email would not be sent")) - - os.remove(cc_file) + control.send(options) diff --git a/tools/patman/series.py b/tools/patman/series.py index b7eef37d03..9f885c8987 100644 --- a/tools/patman/series.py +++ b/tools/patman/series.py @@ -244,7 +244,7 @@ class Series(dict): add_maintainers: Either: True/False to call the get_maintainers to CC maintainers List of maintainers to include (for testing) - limit: Limit the length of the Cc list + limit: Limit the length of the Cc list (None if no limit) Return: Filename of temp file created """