From patchwork Mon Jun 17 22:18:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adam Hassick X-Patchwork-Id: 1948883 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=iol.unh.edu header.i=@iol.unh.edu header.a=rsa-sha256 header.s=unh-iol header.b=iqJFQbxQ; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W347q3Jg4z20XQ for ; Tue, 18 Jun 2024 08:20:07 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=iol.unh.edu header.i=@iol.unh.edu header.a=rsa-sha256 header.s=unh-iol header.b=iqJFQbxQ; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4W347p2s67z3gHR for ; Tue, 18 Jun 2024 08:20:06 +1000 (AEST) X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=iol.unh.edu Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=iol.unh.edu header.i=@iol.unh.edu header.a=rsa-sha256 header.s=unh-iol header.b=iqJFQbxQ; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=iol.unh.edu (client-ip=2607:f8b0:4864:20::72c; helo=mail-qk1-x72c.google.com; envelope-from=ahassick@iol.unh.edu; receiver=lists.ozlabs.org) Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com [IPv6:2607:f8b0:4864:20::72c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4W346z4Bb9z3g9K for ; Tue, 18 Jun 2024 08:19:21 +1000 (AEST) Received: by mail-qk1-x72c.google.com with SMTP id af79cd13be357-7965d034cedso311757885a.3 for ; Mon, 17 Jun 2024 15:19:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; t=1718662757; x=1719267557; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kxDWOJyQXXXt1wvS/by2LOlfKjKJxSno6B4HHVHhD1o=; b=iqJFQbxQs9pW9rKvo9H4zuiMkn7AQpDxjwvWjEuAqhrgDvW+pSe3Vm8JoYnCINz/Y+ ay0L+anucNLHbZzhnAi2vqzYCRGML7EcRuRzFrZzUqh21ly88SCsKSIliSuZJmfQgLCN q1dcfEv7XKwoQVc2LvGMXs67a6AdugKvtI30s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718662757; x=1719267557; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kxDWOJyQXXXt1wvS/by2LOlfKjKJxSno6B4HHVHhD1o=; b=DCK1XMw3PKimkg26Sul0B1OPt4a0hB3/COd3ClJmOBhNA92ED+JK3Cu4GmvQtqvlLE ttIMGTLkImyN0zJR1SkDFMMRMO+KuUZ0+WhbBDCkOl9xsq6U6aKnu6vLmFZuhuuCj/JO zjWAdhkacp8QHSKITbWUZWvIpcLeJ5QndxGq1CRGGAv5RU5CwIs+x8bUfiCEGIt2bHKQ zR8Jg/pIaPuwXG+XJgmwNbTM+p2QaYiWrgLkWWtnSj+5ZNLCIZvAM0dmIPH+da++AEv7 RQXwVfx/bJiOUzDGCDkOjXpZqKbz2AcRi08I2Qh1OqBVt+l8drgKkisy5m2hxaLVHz6c gXtQ== X-Gm-Message-State: AOJu0YwIRMQuvcgp038g+7p23Qhlke6KoAU2FFJE63na+FXVhaWUwH+a Umjryb8Y9ddId0+MZPQtUSh9G5t9RYW+TUQQeKFZOLZbsAjRilqaBLXdgPhdMSReUhGAmR/ds6y xQR0Kwu77m+MwXuwUyMXMHJE8NyFJ6/KH62Hwc5dB7mHBTRI1iUPIAjV1YTbUxfB9s7ON3bopew q14gbfUWOsmy+xhTkom+MBIJABkha7gsGj97HNude5LU/zkg== X-Google-Smtp-Source: AGHT+IEtFOrsux88GzN261bzVZOaagOSBJaN/FpP9N6IEl1Pi3Gufkks19Lg1FMzOLJvb+HaVEcRUQ== X-Received: by 2002:a05:620a:318a:b0:796:100f:2536 with SMTP id af79cd13be357-798d23c56bamr1301706085a.9.1718662756905; Mon, 17 Jun 2024 15:19:16 -0700 (PDT) Received: from bubs.loudonlune.net ([71.254.3.64]) by smtp.gmail.com with ESMTPSA id af79cd13be357-798aaee0e52sm469808185a.37.2024.06.17.15.19.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jun 2024 15:19:16 -0700 (PDT) From: Adam Hassick To: patchwork@lists.ozlabs.org Subject: [PATCH v1 3/9] parser: Parse "Depends-on" tags in emails Date: Mon, 17 Jun 2024 18:18:49 -0400 Message-ID: <20240617221900.156155-3-ahassick@iol.unh.edu> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240617221900.156155-1-ahassick@iol.unh.edu> References: <20240617221900.156155-1-ahassick@iol.unh.edu> MIME-Version: 1.0 X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" Add a new function to parse "Depends-on" tags to the parser. The value may either be a "reference" to a patch or series object or the web URL to the object. For example, a reference may look like "patch-1234" or "series-5678". When this tag is found, the parser will add the series (or the series the patch belongs to) as a dependency to the patch series it is creating. Signed-off-by: Adam Hassick --- patchwork/parser.py | 109 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 108 insertions(+), 1 deletion(-) diff --git a/patchwork/parser.py b/patchwork/parser.py index 09a53a0..90ec63b 100644 --- a/patchwork/parser.py +++ b/patchwork/parser.py @@ -14,11 +14,14 @@ from email.errors import HeaderParseError from fnmatch import fnmatch import logging import re +from urllib.parse import urlparse from django.contrib.auth.models import User from django.db.utils import IntegrityError from django.db import transaction from django.utils import timezone as tz_utils +from django.urls import resolve, Resolver404 +from django.conf import settings from patchwork.models import Cover from patchwork.models import CoverComment @@ -32,7 +35,6 @@ from patchwork.models import Series from patchwork.models import SeriesReference from patchwork.models import State - _msgid_re = re.compile(r'<[^>]+>') _hunk_re = re.compile(r'^\@\@ -\d+(?:,(\d+))? \+\d+(?:,(\d+))? \@\@') _filename_re = re.compile(r'^(---|\+\+\+) (\S+)') @@ -1054,6 +1056,102 @@ def parse_pull_request(content): return None +def find_series_from_url(url): + """ + Get series or patch from URL. + """ + + parse_result = urlparse(url) + + # Resolve the URL path to see if this is a patch or series detail URL. + try: + result = resolve(parse_result.path) + except Resolver404: + return None + + if result.view_name == 'patch-list' and parse_result.query: + # Parse the query string. + # This can be replaced with something much friendlier once the + # series detail view is implemented. + series_query_param = next( + filter( + lambda x: len(x) == 2 and x[0] == 'series', + map(lambda x: x.split('='), parse_result.query.split('&')), + ) + ) + + if series_query_param: + series_id = series_query_param[1] + + try: + series_id_num = int(series_id) + except ValueError: + return None + + return Series.objects.filter(id=series_id_num).first() + elif result.view_name == 'patch-detail': + msgid = Patch.decode_msgid(result.kwargs['msgid']) + patch = Patch.objects.filter(msgid=msgid).first() + + if patch: + return patch.series + + +def find_series_from_ref(match): + (obj_type, obj_id_str) = match + + try: + object_id = int(obj_id_str) + except ValueError: + return None + + if obj_type == 'series': + series = Series.objects.filter(id=object_id).first() + elif obj_type == 'patch': + patch = Patch.objects.filter(id=object_id).first() + + if not patch: + return None + + series = patch.series + + return series + + +def parse_depends_on(content): + """Parses any dependency hints in the comments.""" + depends_patterns = [ + ( + re.compile( + r'^Depends-on: ((?:patch)?(?:series)?)-([\d]+)(?: \("[^\n\r"]+"\))?\s*$', + re.MULTILINE | re.IGNORECASE, + ), + find_series_from_ref, + ), + ( + re.compile( + r'^Depends-on: (http[s]?:\/\/[\w\d\-.\/=&@:%?_\+()]+)\s*$', + re.MULTILINE | re.IGNORECASE, + ), + find_series_from_url, + ), + ] + + dependencies = list() + + for pat, mapper in depends_patterns: + matches = pat.findall(content) + + # Produce a list of tuples containing type and ID of each dependency. + # Eliminate elements where we could not parse the ID as an integer. + dependencies.extend( + filter(lambda series: series is not None, map(mapper, matches)) + ) + + # Return list of series objects to depend on. + return dependencies + + def find_state(mail): """Return the state with the given name or the default.""" state_name = clean_header(mail.get('X-Patchwork-State', '')) @@ -1171,6 +1269,12 @@ def parse_mail(mail, list_id=None): pull_url = parse_pull_request(message) + # Only look for "Depends-on" tags if the setting is enabled. + if settings.ENABLE_DEPENDS_ON_PARSING: + dependencies = parse_depends_on(message) + else: + dependencies = [] + # build objects if not is_comment and (diff or pull_url): # patches or pull requests @@ -1308,6 +1412,8 @@ def parse_mail(mail, list_id=None): # always have a series series.add_patch(patch, x) + series.add_dependencies(dependencies) + return patch elif x == 0: # (potential) cover letters # if refs are empty, it's implicitly a cover letter. If not, @@ -1374,6 +1480,7 @@ def parse_mail(mail, list_id=None): logger.debug('Cover letter saved') series.add_cover_letter(cover_letter) + series.add_dependencies(dependencies) return cover_letter