From patchwork Tue Sep 3 10:15:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 1156907 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46N2sF1kN7z9s4Y for ; Tue, 3 Sep 2019 20:15:45 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="FxvDR/Hu"; dkim-atps=neutral Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46N2sF06H4zDqYV for ; Tue, 3 Sep 2019 20:15:45 +1000 (AEST) X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (mailfrom) smtp.mailfrom=that.guru (client-ip=199.181.239.189; helo=relay0189.mxlogin.com; envelope-from=stephen@that.guru; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=that.guru Authentication-Results: lists.ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="FxvDR/Hu"; dkim-atps=neutral Received: from relay0189.mxlogin.com (relay0189.mxlogin.com [199.181.239.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46N2s94vDLzDqP5 for ; Tue, 3 Sep 2019 20:15:41 +1000 (AEST) Received: from filter001.mxrelay.co (unknown [94.130.183.33]) by relay0189.mxlogin.com (Postfix) with ESMTP id 06A6ECC70314; Tue, 3 Sep 2019 05:15:37 -0500 (CDT) Received: from one.mxroute.com (one.mxroute.com [195.201.59.211]) by filter001.mxrelay.co (Postfix) with ESMTPS id 26E701001C9; Tue, 3 Sep 2019 10:15:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=Content-Transfer-Encoding:MIME-Version:Message-Id:Date:Subject: Cc:To:From:Sender:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=muygNmLV2ZMZo9k4R+pC7J/cM23j1QEyAKzanVVKNAA=; b=FxvDR/Hu2iYq5Y+/z3XzuTXfUO rl2UpDOeddjx78t1V4pFU23ZRw7jBKtuFCdTJwKN7zciiUN97H2QD2fnRYh+oKs5UTT0IG3UFrAKa kYgrBI+79+HswUvTyUsAoj8Gkz/R283nWKp0FYWm+EYFa3yzjnaaaDSZdVqYkOIY5eF/dUfuMlhOe Or2Z/fXScKoKHrKASM1abbo2viHo2ijBEPGo/FMnMAV1zg1yNsQQUJmI1CIBSy48sDq72nk52acuD GL/sE6tXeHVMioEsMYO6iw3BwFv7bSIGjKmxUOz1OGLdkUCQrUtCkxB5WBUnvI1/kfJ5yunAmxDjP ed7SHuFw==; From: Stephen Finucane To: patchwork@lists.ozlabs.org Subject: [PATCH v4] tox: Integrate tox-docker Date: Tue, 3 Sep 2019 11:15:28 +0100 Message-Id: <20190903101528.15657-1-stephen@that.guru> X-Mailer: git-send-email 2.21.0 MIME-Version: 1.0 X-AuthUser: stephen@that.guru X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" This eliminates the need to use docker-compose for most use cases. Instead, we can now do: tox -e py27-django111-postgres If you're using a locally configured PostgreSQL or MySQL instance, you simply omit the last factor and things behave as before: tox -e py27-django111 We removed the 'venv' environment, since it was never actually that useful and is even less so now (you'd need to have a local DB set up) and add the 'skip_missing_interpreters' flag for folks on an OS that doesn't provide all the Python versions under the sun like Fedora does. Signed-off-by: Stephen Finucane Cc: Daniel Axtens --- v4: - Don't enable DB-based toxenvs by default --- docs/development/contributing.rst | 35 +++++++++++++------------------ docs/development/installation.rst | 6 ------ patchwork/settings/dev.py | 12 +++++++++++ tox.ini | 23 ++++++++++++++++---- 4 files changed, 45 insertions(+), 31 deletions(-) diff --git a/docs/development/contributing.rst b/docs/development/contributing.rst index 5089bba8..aa3b58c1 100644 --- a/docs/development/contributing.rst +++ b/docs/development/contributing.rst @@ -31,29 +31,15 @@ Testing ------- Patchwork includes a `tox`_ script to automate testing. This requires a -functional database and some Python requirements like *tox*. Refer to -:doc:`installation` for information on how to configure these. - -You may also need to install *tox*. If so, do this now: +functional database and some Python requirements like *tox*. These can be +installed using :command:`pip`: .. code-block:: shell - $ pip install --user tox - -.. tip:: - - If you're using Docker, you may not need to install *tox* - locally. Instead, it will already be installed inside the - container. For Docker, you can run *tox* like so: - - .. code-block:: shell - - $ docker-compose run --rm web tox [ARGS...] + $ pip install --user tox tox-docker - For more information, refer to :ref:`installation-docker`. - -Assuming these requirements are met, actually testing Patchwork is quite easy -to do. To start, you can show the default targets like so: +Once installed, actually testing Patchwork is quite easy to do. To start, you +can show the default targets like so: .. code-block:: shell @@ -66,7 +52,14 @@ parameter: .. code-block:: shell - $ tox -e py27-django18 + $ tox -e py36-django21 + +You can also include a ``postgres`` or ``mysql`` factor which avoids the need to +configure a database locally: + +.. code-block:: shell + + $ tox -e py36-django21-mysql In the case of the unit tests targets, you can also run specific tests by passing the fully qualified test name as an additional argument to this @@ -74,7 +67,7 @@ command: .. code-block:: shell - $ tox -e py27-django18 patchwork.tests.SubjectCleanUpTest + $ tox -e py36-django21-mysql patchwork.tests.SubjectCleanUpTest Because Patchwork support multiple versions of Django, it's very important that you test against all supported versions. When run without argument, tox will do diff --git a/docs/development/installation.rst b/docs/development/installation.rst index 0ab755f4..433c3a41 100644 --- a/docs/development/installation.rst +++ b/docs/development/installation.rst @@ -86,12 +86,6 @@ To run unit tests against the system Python packages, run: $ docker-compose run --rm web python manage.py test -To run unit tests for multiple versions using ``tox``, run: - -.. code-block:: shell - - $ docker-compose run --rm web tox - To reset the database before any of these commands, add ``--reset`` to the command line after ``web`` and before any other arguments: diff --git a/patchwork/settings/dev.py b/patchwork/settings/dev.py index e110e745..f77dbbf0 100644 --- a/patchwork/settings/dev.py +++ b/patchwork/settings/dev.py @@ -19,6 +19,18 @@ try: except ImportError: debug_toolbar = None +# +# tox-docker settings +# + +if 'POSTGRES_5432_TCP' in os.environ: + os.environ['PW_TEST_DB_HOST'] = os.environ['POSTGRES_HOST'] + os.environ['PW_TEST_DB_PORT'] = os.environ['POSTGRES_5432_TCP'] +elif 'MYSQL_3306_TCP' in os.environ: + os.environ['PW_TEST_DB_HOST'] = os.environ['MYSQL_HOST'] + os.environ['PW_TEST_DB_PORT'] = os.environ['MYSQL_3306_TCP'] + + # # Core settings # https://docs.djangoproject.com/en/1.11/ref/settings/#core-settings diff --git a/tox.ini b/tox.ini index bfca0538..05db1bcd 100644 --- a/tox.ini +++ b/tox.ini @@ -1,6 +1,7 @@ [tox] minversion = 2.0 envlist = pep8,docs,py{27,34}-django111,py{35,36}-django{111,20,21,22} +skip_missing_interpreters = True skipsdist = True [testenv] @@ -17,6 +18,9 @@ deps = django22: django>=2.2,<2.3 django22: djangorestframework>=3.9.2 django22: django-filter>=2.1,<3.0 +docker = + postgres: postgres:9.6 + mysql: mysql:8.0 setenv = DJANGO_SETTINGS_MODULE = patchwork.settings.dev PYTHONDONTWRITEBYTECODE = 1 @@ -24,15 +28,25 @@ setenv = py27: PYTHONWARNINGS = once py{34,36}:PYTHONWARNINGS = once,ignore::ImportWarning:backports py35:PYTHONWARNINGS = once,ignore::ResourceWarning:unittest.suite,ignore::ImportWarning:backports + postgres: PW_TEST_DB_TYPE = postgres + postgres: PW_TEST_DB_USER = postgres + postgres: PW_TEST_DB_PASS = password + mysql: PW_TEST_DB_USER = root + mysql: PW_TEST_DB_PASS = password passenv = http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY PW_TEST_DB_TYPE PW_TEST_DB_USER PW_TEST_DB_PASS PW_TEST_DB_HOST PW_TEST_DB_PORT +dockerenv = + postgres: POSTGRES_PASSWORD=password + mysql: MYSQL_ROOT_PASSWORD=password commands = - python {toxinidir}/manage.py test --noinput '{posargs:patchwork}' + python {toxinidir}/manage.py test --noinput {posargs:patchwork} [testenv:bashate] deps = bashate +docker = +dockerenv = whitelist_externals = bash commands = bash -c "find {toxinidir} \ @@ -43,6 +57,8 @@ commands = [testenv:pep8] basepython = python2.7 deps = flake8 +docker = +dockerenv = commands = flake8 {posargs} patchwork [flake8] @@ -57,6 +73,8 @@ exclude = ./patchwork/migrations [testenv:docs] deps = -r{toxinidir}/docs/requirements.txt +docker = +dockerenv = commands = sphinx-build -E -W -b dirhtml -d docs/_build/doctrees docs docs/_build/html @@ -67,9 +85,6 @@ deps = -r{toxinidir}/requirements-prod.txt commands = pylint patchwork --rcfile=pylint.rc -[testenv:venv] -commands = {posargs} - [testenv:coverage] basepython = python2.7 deps =