From patchwork Mon Oct 16 21:59:02 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Packham X-Patchwork-Id: 826532 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="hZadxpnw"; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yGC0J3qJFz9sRq for ; Tue, 17 Oct 2017 08:59:28 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id E098D2D437; Mon, 16 Oct 2017 21:59:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7vNgytMGyJ57; Mon, 16 Oct 2017 21:59:23 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 7A5192F33E; Mon, 16 Oct 2017 21:59:23 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id C662A1CEC57 for ; Mon, 16 Oct 2017 21:59:21 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id C0B8F8479A for ; Mon, 16 Oct 2017 21:59:21 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HhVbS0yd3Cxp for ; Mon, 16 Oct 2017 21:59:20 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pf0-f196.google.com (mail-pf0-f196.google.com [209.85.192.196]) by fraxinus.osuosl.org (Postfix) with ESMTPS id BC41384762 for ; Mon, 16 Oct 2017 21:59:20 +0000 (UTC) Received: by mail-pf0-f196.google.com with SMTP id x7so16813749pfa.1 for ; Mon, 16 Oct 2017 14:59:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=rw4wpP91TL8dYCeoG5me2MHzw5u15DqxUv0/81pXmSk=; b=hZadxpnw9m7QW51Axl1qOjcblNtc5WVwhGk6tWdVHdjkeVDRAPyJyBYkq8H8aoMV7c xovU0tYGKG0TEFoAApCmTH5g7TS2yQmIl5Y+F7ruibTFIJ+hphu5lkyA3lt3T3RF5vGv 7857IuEZo19iBYJmzZ0Gk35pmsHdBzqOjcilWfZzWRfW2xE3d+9LEXKyXKuH5lLigGEE gJOtB2qjBmFUU+kJqJOJlc6kliFPJkfcsTTxY5A6hLUmTiWzGWVDVt/+HTBxj4d5Z2CE EkVqi0Rq8mIu8EgrPq+pcyV53un1rIlzbzd8nZJKgRCTkaPPQmbPywJ663LV3VxuQuuJ Wpug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=rw4wpP91TL8dYCeoG5me2MHzw5u15DqxUv0/81pXmSk=; b=rwE0dOmJik/QV9a6PM8UDLhWrkF+3isfKt1KUFtvc6WYsqIY512qW2WXeyV/0q2R7/ u/7hjwVIuCVDBIW9BStvSHm9jB8+Qxxl9o1cNACqE2K/EAT8GA/KRWRL+JsNmxjxtVin uYWOzrEbYnWwFIRWrjVdcDXcFPVHEmbsssr/7eSgJG3ip1vzknEUJ4MD19ojFYZq25e4 /HjT6vdvwI/OIE861/ZHhCPjttw83jxmKoufau8DnLQRfDIwStuD3TEt+QeEnpbs+z2U fomjRKULqoERgcgBRgiYZKnXAInqoFSHkK8woy1jZeDRd4KAl8NVTSJ2gK1xWFq027JT xyHw== X-Gm-Message-State: AMCzsaUobif3NSzYEFfieSOIYcl1uv1I4JlrXBmfK8CmtMaoXI8dEnsE N1NcbTzitlI5AhuHTGSnQIaHENJi X-Google-Smtp-Source: ABhQp+SPQb8Bnuz197Iy61ZcoQO1nzarVe6QdF+8vumDg2URsJ84nDM1HJcIm+vxvoCIZSBx8/3qYA== X-Received: by 10.84.132.42 with SMTP id 39mr3372539ple.382.1508191160226; Mon, 16 Oct 2017 14:59:20 -0700 (PDT) Received: from chrisp-dl.ws.atlnz.lc ([2001:df5:b000:22:3a2c:4aff:fe70:2b02]) by smtp.gmail.com with ESMTPSA id f10sm14765118pgr.8.2017.10.16.14.59.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 16 Oct 2017 14:59:19 -0700 (PDT) From: Chris Packham To: buildroot@buildroot.org Date: Tue, 17 Oct 2017 10:59:02 +1300 Message-Id: <20171016215904.3380-2-judge.packham@gmail.com> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171016215904.3380-1-judge.packham@gmail.com> References: <20171016215904.3380-1-judge.packham@gmail.com> Cc: Andrey Smirnov , Thomas Petazzoni , "Yann E . MORIN" Subject: [Buildroot] [PATCH 1/3] zstd: install to staging directory X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Install libzstd to the staging directory. Signed-off-by: Chris Packham --- package/zstd/zstd.mk | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/package/zstd/zstd.mk b/package/zstd/zstd.mk index 089d654749..f149ad2f91 100644 --- a/package/zstd/zstd.mk +++ b/package/zstd/zstd.mk @@ -6,6 +6,7 @@ ZSTD_VERSION = v1.3.2 ZSTD_SITE = $(call github,facebook,zstd,$(ZSTD_VERSION)) +ZSTD_INSTALL_STAGING = YES ZSTD_LICENSE = BSD-3-Clause, GPL-2.0 ZSTD_LICENSE_FILES = LICENSE COPYING @@ -46,4 +47,9 @@ define ZSTD_INSTALL_TARGET_CMDS DESTDIR=$(TARGET_DIR) PREFIX=/usr -C $(@D)/programs install endef +define ZSTD_INSTALL_STAGING_CMDS + $(TARGET_MAKE_ENV) $(TARGET_CONFIGURE_OPTS) $(MAKE) $(ZSTD_OPTS) \ + DESTDIR=$(STAGING_DIR) PREFIX=/usr -C $(@D)/lib install +endef + $(eval $(generic-package)) From patchwork Mon Oct 16 21:59:03 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Packham X-Patchwork-Id: 826533 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="H6D6sQlH"; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yGC0L4SL9z9sRq for ; Tue, 17 Oct 2017 08:59:30 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 9D2302FA9E; Mon, 16 Oct 2017 21:59:28 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YMX5rND+iGuU; Mon, 16 Oct 2017 21:59:26 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 87D2F2F846; Mon, 16 Oct 2017 21:59:26 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 6ECD21CEC57 for ; Mon, 16 Oct 2017 21:59:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 687DC8711F for ; Mon, 16 Oct 2017 21:59:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QcApKb90NNn7 for ; Mon, 16 Oct 2017 21:59:24 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pg0-f68.google.com (mail-pg0-f68.google.com [74.125.83.68]) by hemlock.osuosl.org (Postfix) with ESMTPS id F11418734B for ; Mon, 16 Oct 2017 21:59:23 +0000 (UTC) Received: by mail-pg0-f68.google.com with SMTP id p9so3298625pgc.8 for ; Mon, 16 Oct 2017 14:59:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=p1vU4jcLyd1RAkK5A7XLuhC8R0KQzkME90xZCpPOpsQ=; b=H6D6sQlHfp3ae+o0eOQmzCi0chrATWDwZU06bptZ01Vs1shl2/ovFFDHLfcA4efvdN O6wfAkKc0dZYJ0QdAJMkLlHEGrVF4jSEn6Jcmefoan2ybUuzkzDV2N6rNPvRJ1LlYaRK ZrXR41m8qFBk0DBq3nTmNzBzZ+s4+TRs/+YUUH+Tpcs4ZHqWOcvAA3ix8FsrRzCRgq/g KHqQNiCK3DdNvbs71SvIy1VWUZAUqQsG4Hh7dOgYs/tysUT6ai40G2mOzwai2VaoPkUm 3e3L+/tnYeVnETtt11aksaf53PxiWViXF5OJ1DjC8nKnpqU8f0ilZsadZaGz4rObakrH eKTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=p1vU4jcLyd1RAkK5A7XLuhC8R0KQzkME90xZCpPOpsQ=; b=VUnqBbfxovqvjhxzpiDEDQzF8BRAA8HgNILZ1OjWoomHJW0Y+aZzQCSQoi2d1Qql9G foznYrUbU/eR9DC1kyuSxDUMyaQ/BZtbwzYhhlb81FTna7W5EvoeSoZVc1saTmCwSgBD hvbo4NPxVeTNrRCYsr6MaYLqiC1ojk5EjSSuRpoFuJEgirvkgq3u7WIWrkYRlGcVMO1t Lfrea0L/DWRdEsFA3724aYUtOAIbj1QmaLq5iBIGOxa/75ch0JjGupkzCkhEAXAVfb6b hsqylZIytptF/zf05Bp/mSfj/9ZhS+S5jGZY81goy66Co5KhwbttcZBz6s10T+3GJiHz fs7Q== X-Gm-Message-State: AMCzsaWxBPXJKwSx3iKx+Fp0UPLZ2iA+Fqbi5yOSviIjr8oi0CCnmt6K 8Ig+wZoIU0DcS+M6a6GqCzUrfgii X-Google-Smtp-Source: AOwi7QALG1yq+roJb2TaB2PT+k2u+zoy5vdRgLvCZo03tHeyZZXZexglu42jWIJv433bXZBsqccHAg== X-Received: by 10.159.218.69 with SMTP id x5mr9839765plv.4.1508191163407; Mon, 16 Oct 2017 14:59:23 -0700 (PDT) Received: from chrisp-dl.ws.atlnz.lc ([2001:df5:b000:22:3a2c:4aff:fe70:2b02]) by smtp.gmail.com with ESMTPSA id f10sm14765118pgr.8.2017.10.16.14.59.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 16 Oct 2017 14:59:22 -0700 (PDT) From: Chris Packham To: buildroot@buildroot.org Date: Tue, 17 Oct 2017 10:59:03 +1300 Message-Id: <20171016215904.3380-3-judge.packham@gmail.com> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171016215904.3380-1-judge.packham@gmail.com> References: <20171016215904.3380-1-judge.packham@gmail.com> Cc: Andrey Smirnov , Thomas Petazzoni , "Yann E . MORIN" Subject: [Buildroot] [PATCH 2/3] zstd: build for host X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Signed-off-by: Chris Packham --- package/zstd/zstd.mk | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/package/zstd/zstd.mk b/package/zstd/zstd.mk index f149ad2f91..01fec7bb76 100644 --- a/package/zstd/zstd.mk +++ b/package/zstd/zstd.mk @@ -37,6 +37,26 @@ else ZSTD_OPTS += HAVE_LZ4=0 endif +HOST_ZSTD_DEPENDENCIES = host-zlib host-lz4 host-xz + +HOST_ZSTD_OPTS = \ + HAVE_THREAD=1 \ + HAVE_ZLIB=1 \ + HAVE_LZMA=1 \ + HAVE_LZ4=1 + +define HOST_ZSTD_BUILD_CMDS + $(HOST_MAKE_ENV) $(HOST_CONFIGURE_OPTS) $(MAKE) $(HOST_ZSTD_OPTS) \ + -C $(@D) zstd +endef + +define HOST_ZSTD_INSTALL_CMDS + $(HOST_MAKE_ENV) $(HOST_CONFIGURE_OPTS) $(MAKE) $(HOST_ZSTD_OPTS) \ + DESTDIR=$(HOST_DIR) PREFIX=/usr -C $(@D)/programs install + $(HOST_MAKE_ENV) $(HOST_CONFIGURE_OPTS) $(MAKE) $(HOST_ZSTD_OPTS) \ + DESTDIR=$(HOST_DIR) PREFIX=/usr -C $(@D)/lib install +endef + define ZSTD_BUILD_CMDS $(TARGET_MAKE_ENV) $(TARGET_CONFIGURE_OPTS) $(MAKE) $(ZSTD_OPTS) \ -C $(@D) zstd @@ -53,3 +73,4 @@ define ZSTD_INSTALL_STAGING_CMDS endef $(eval $(generic-package)) +$(eval $(host-generic-package)) From patchwork Mon Oct 16 21:59:04 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Packham X-Patchwork-Id: 826534 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="ITJQe/u2"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yGC0c0xr8z9sRW for ; Tue, 17 Oct 2017 08:59:44 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 596F98484F; Mon, 16 Oct 2017 21:59:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sT3J2-_O7Xtd; Mon, 16 Oct 2017 21:59:36 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 9C6F284750; Mon, 16 Oct 2017 21:59:31 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id B46E11CEC57 for ; Mon, 16 Oct 2017 21:59:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id A89CD84750 for ; Mon, 16 Oct 2017 21:59:29 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XqymQ2JpM5b4 for ; Mon, 16 Oct 2017 21:59:27 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pg0-f65.google.com (mail-pg0-f65.google.com [74.125.83.65]) by fraxinus.osuosl.org (Postfix) with ESMTPS id D157D84762 for ; Mon, 16 Oct 2017 21:59:27 +0000 (UTC) Received: by mail-pg0-f65.google.com with SMTP id s75so7049091pgs.0 for ; Mon, 16 Oct 2017 14:59:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=90F3gx59FNraCtss/ZWJztajEuSQaoPYeQQT2g/IK2E=; b=ITJQe/u2Xl4nGaxqJKYE+/3+60h3ArTVRJfh1AJYz80c2xTjlX/9Dbtqu9f2nouVBb yNPwAF9inIHfeIcUVn1uTUWrHC2JyG/ijX7hGJ6wVVHGtvFQcXYEnrgR/cfRUmwbEGfw FuOkL/p3Fo1s3ORCTP4xBn9hyIUewqxsutJAwVFVO9U8wN4ZxFZIa2E0vS+aK4OCZGDk UQL1a727n0AX6wEmrdFhFFiNmn2MTAiWkZ8wiS0PWQ+o4lp9WzsC9udVQEsKO/Eg+sRk yCsiramjAhwRB9IU5YAQPyJjeITm2A4WIWgsa2RS94k8Cgho0eLaW4/Zdfixo4Z660Uc jeYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=90F3gx59FNraCtss/ZWJztajEuSQaoPYeQQT2g/IK2E=; b=HckHHS6JRDGLCSbHFECyetg85zVwV23bjVg0ycw3t59nv5+G0Xt7/78pDlD3yQ5/oh 4sONqSZLwpHn8XjcqYFZ95Zp4e2PTfIsPl5KYu2zhRror2Oig55xFcPYd4F3Lys7L4lW /HZtisApppoAb3pYLA+40GxQsvua1MEIQ1C8+Hbmhi3SrcjTJ7+OLhwyKbzHnd465lj5 cmE8LbTT1JPesU2Ld8EHfSUnXybGcdco5z+oz930SmWX5dpE1BOwRmiU5wqVUDQFPybQ FFBk9HKprWy8rDRhhcws7phbHKCd8FhQj0uKvomrSnWZZgd2dY795Plb2oqdZwKwbDCx 6Kiw== X-Gm-Message-State: AMCzsaUq0/JjSsvrLAghB17uzCAeceKIUBcZ7CHaZ2olsl6z1zYJmOkW lVqrjhvaM0tyCdcLpVPWKrDcN89y X-Google-Smtp-Source: AOwi7QBJAZpdBUBDWaXBVL9V8Pie9rNq3Sosc/XLC+QbeOZZ7yUVHSxlFGidy/YwWIa56PSTj7u60w== X-Received: by 10.84.236.12 with SMTP id q12mr10176117plk.204.1508191167000; Mon, 16 Oct 2017 14:59:27 -0700 (PDT) Received: from chrisp-dl.ws.atlnz.lc ([2001:df5:b000:22:3a2c:4aff:fe70:2b02]) by smtp.gmail.com with ESMTPSA id f10sm14765118pgr.8.2017.10.16.14.59.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 16 Oct 2017 14:59:26 -0700 (PDT) From: Chris Packham To: buildroot@buildroot.org Date: Tue, 17 Oct 2017 10:59:04 +1300 Message-Id: <20171016215904.3380-4-judge.packham@gmail.com> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171016215904.3380-1-judge.packham@gmail.com> References: <20171016215904.3380-1-judge.packham@gmail.com> Cc: Andrey Smirnov , Thomas Petazzoni , "Yann E . MORIN" Subject: [Buildroot] [PATCH 3/3] squashfs: add support for zstd X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Import the upstream patch to add support for using zstd to compress squashfs images. Signed-off-by: Chris Packham --- fs/squashfs/Config.in | 3 + fs/squashfs/squashfs.mk | 2 + .../0006-squashfs-tools-Add-zstd-support.patch | 420 +++++++++++++++++++++ package/squashfs/Config.in | 5 + package/squashfs/squashfs.mk | 12 +- 5 files changed, 440 insertions(+), 2 deletions(-) create mode 100644 package/squashfs/0006-squashfs-tools-Add-zstd-support.patch diff --git a/fs/squashfs/Config.in b/fs/squashfs/Config.in index ca9ddb2218..dde2097cb7 100644 --- a/fs/squashfs/Config.in +++ b/fs/squashfs/Config.in @@ -27,5 +27,8 @@ config BR2_TARGET_ROOTFS_SQUASHFS4_LZO config BR2_TARGET_ROOTFS_SQUASHFS4_XZ bool "xz" +config BR2_TARGET_ROOTFS_SQUASHFS4_ZSTD + bool "zstd" + endchoice endif diff --git a/fs/squashfs/squashfs.mk b/fs/squashfs/squashfs.mk index 7de7f51af1..5733fbef1e 100644 --- a/fs/squashfs/squashfs.mk +++ b/fs/squashfs/squashfs.mk @@ -16,6 +16,8 @@ else ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_LZMA),y) ROOTFS_SQUASHFS_ARGS += -comp lzma else ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_XZ),y) ROOTFS_SQUASHFS_ARGS += -comp xz +else ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_ZSTD),y) +ROOTFS_SQUASHFS_ARGS += -comp zstd else ROOTFS_SQUASHFS_ARGS += -comp gzip endif diff --git a/package/squashfs/0006-squashfs-tools-Add-zstd-support.patch b/package/squashfs/0006-squashfs-tools-Add-zstd-support.patch new file mode 100644 index 0000000000..ca638f2644 --- /dev/null +++ b/package/squashfs/0006-squashfs-tools-Add-zstd-support.patch @@ -0,0 +1,420 @@ +From 57a3cf95b276946559f9e044c7352c11303bb9c1 Mon Sep 17 00:00:00 2001 +From: Sean Purcell +Date: Thu, 3 Aug 2017 17:47:03 -0700 +Subject: [PATCH v6] squashfs-tools: Add zstd support + +This patch adds zstd support to squashfs-tools. It works with zstd +versions >= 1.0.0. It was originally written by Sean Purcell. + +Signed-off-by: Sean Purcell +Signed-off-by: Nick Terrell +--- +v4 -> v5: +- Fix patch documentation to reflect that Sean Purcell is the author +- Don't strip trailing whitespace of unreleated code +- Make zstd_display_options() static + +v5 -> v6: +- Fix build instructions in Makefile + + squashfs-tools/Makefile | 20 ++++ + squashfs-tools/compressor.c | 8 ++ + squashfs-tools/squashfs_fs.h | 1 + + squashfs-tools/zstd_wrapper.c | 254 ++++++++++++++++++++++++++++++++++++++++++ + squashfs-tools/zstd_wrapper.h | 48 ++++++++ + 5 files changed, 331 insertions(+) + create mode 100644 squashfs-tools/zstd_wrapper.c + create mode 100644 squashfs-tools/zstd_wrapper.h + +diff --git a/squashfs-tools/Makefile b/squashfs-tools/Makefile +index 52d2582..22fc559 100644 +--- a/squashfs-tools/Makefile ++++ b/squashfs-tools/Makefile +@@ -75,6 +75,18 @@ GZIP_SUPPORT = 1 + #LZMA_SUPPORT = 1 + #LZMA_DIR = ../../../../LZMA/lzma465 + ++ ++########### Building ZSTD support ############ ++# ++# The ZSTD library is supported ++# ZSTD homepage: http://zstd.net ++# ZSTD source repository: https://github.com/facebook/zstd ++# ++# To build using the ZSTD library - install the library and uncomment the ++# ZSTD_SUPPORT line below. ++# ++#ZSTD_SUPPORT = 1 ++ + ######## Specifying default compression ######## + # + # The next line specifies which compression algorithm is used by default +@@ -177,6 +189,14 @@ LIBS += -llz4 + COMPRESSORS += lz4 + endif + ++ifeq ($(ZSTD_SUPPORT),1) ++CFLAGS += -DZSTD_SUPPORT ++MKSQUASHFS_OBJS += zstd_wrapper.o ++UNSQUASHFS_OBJS += zstd_wrapper.o ++LIBS += -lzstd ++COMPRESSORS += zstd ++endif ++ + ifeq ($(XATTR_SUPPORT),1) + ifeq ($(XATTR_DEFAULT),1) + CFLAGS += -DXATTR_SUPPORT -DXATTR_DEFAULT +diff --git a/squashfs-tools/compressor.c b/squashfs-tools/compressor.c +index 525e316..02b5e90 100644 +--- a/squashfs-tools/compressor.c ++++ b/squashfs-tools/compressor.c +@@ -65,6 +65,13 @@ static struct compressor xz_comp_ops = { + extern struct compressor xz_comp_ops; + #endif + ++#ifndef ZSTD_SUPPORT ++static struct compressor zstd_comp_ops = { ++ ZSTD_COMPRESSION, "zstd" ++}; ++#else ++extern struct compressor zstd_comp_ops; ++#endif + + static struct compressor unknown_comp_ops = { + 0, "unknown" +@@ -77,6 +84,7 @@ struct compressor *compressor[] = { + &lzo_comp_ops, + &lz4_comp_ops, + &xz_comp_ops, ++ &zstd_comp_ops, + &unknown_comp_ops + }; + +diff --git a/squashfs-tools/squashfs_fs.h b/squashfs-tools/squashfs_fs.h +index 791fe12..afca918 100644 +--- a/squashfs-tools/squashfs_fs.h ++++ b/squashfs-tools/squashfs_fs.h +@@ -277,6 +277,7 @@ typedef long long squashfs_inode; + #define LZO_COMPRESSION 3 + #define XZ_COMPRESSION 4 + #define LZ4_COMPRESSION 5 ++#define ZSTD_COMPRESSION 6 + + struct squashfs_super_block { + unsigned int s_magic; +diff --git a/squashfs-tools/zstd_wrapper.c b/squashfs-tools/zstd_wrapper.c +new file mode 100644 +index 0000000..dcab75a +--- /dev/null ++++ b/squashfs-tools/zstd_wrapper.c +@@ -0,0 +1,254 @@ ++/* ++ * Copyright (c) 2017 ++ * Phillip Lougher ++ * ++ * This program is free software; you can redistribute it and/or ++ * modify it under the terms of the GNU General Public License ++ * as published by the Free Software Foundation; either version 2, ++ * or (at your option) any later version. ++ * ++ * This program is distributed in the hope that it will be useful, ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++ * GNU General Public License for more details. ++ * ++ * zstd_wrapper.c ++ * ++ * Support for ZSTD compression http://zstd.net ++ */ ++ ++#include ++#include ++#include ++#include ++#include ++ ++#include "squashfs_fs.h" ++#include "zstd_wrapper.h" ++#include "compressor.h" ++ ++static int compression_level = ZSTD_DEFAULT_COMPRESSION_LEVEL; ++ ++/* ++ * This function is called by the options parsing code in mksquashfs.c ++ * to parse any -X compressor option. ++ * ++ * This function returns: ++ * >=0 (number of additional args parsed) on success ++ * -1 if the option was unrecognised, or ++ * -2 if the option was recognised, but otherwise bad in ++ * some way (e.g. invalid parameter) ++ * ++ * Note: this function sets internal compressor state, but does not ++ * pass back the results of the parsing other than success/failure. ++ * The zstd_dump_options() function is called later to get the options in ++ * a format suitable for writing to the filesystem. ++ */ ++static int zstd_options(char *argv[], int argc) ++{ ++ if (strcmp(argv[0], "-Xcompression-level") == 0) { ++ if (argc < 2) { ++ fprintf(stderr, "zstd: -Xcompression-level missing " ++ "compression level\n"); ++ fprintf(stderr, "zstd: -Xcompression-level it should " ++ "be 1 <= n <= %d\n", ZSTD_maxCLevel()); ++ goto failed; ++ } ++ ++ compression_level = atoi(argv[1]); ++ if (compression_level < 1 || ++ compression_level > ZSTD_maxCLevel()) { ++ fprintf(stderr, "zstd: -Xcompression-level invalid, it " ++ "should be 1 <= n <= %d\n", ZSTD_maxCLevel()); ++ goto failed; ++ } ++ ++ return 1; ++ } ++ ++ return -1; ++failed: ++ return -2; ++} ++ ++/* ++ * This function is called by mksquashfs to dump the parsed ++ * compressor options in a format suitable for writing to the ++ * compressor options field in the filesystem (stored immediately ++ * after the superblock). ++ * ++ * This function returns a pointer to the compression options structure ++ * to be stored (and the size), or NULL if there are no compression ++ * options. ++ */ ++static void *zstd_dump_options(int block_size, int *size) ++{ ++ static struct zstd_comp_opts comp_opts; ++ ++ /* don't return anything if the options are all default */ ++ if (compression_level == ZSTD_DEFAULT_COMPRESSION_LEVEL) ++ return NULL; ++ ++ comp_opts.compression_level = compression_level; ++ ++ SQUASHFS_INSWAP_COMP_OPTS(&comp_opts); ++ ++ *size = sizeof(comp_opts); ++ return &comp_opts; ++} ++ ++/* ++ * This function is a helper specifically for the append mode of ++ * mksquashfs. Its purpose is to set the internal compressor state ++ * to the stored compressor options in the passed compressor options ++ * structure. ++ * ++ * In effect this function sets up the compressor options ++ * to the same state they were when the filesystem was originally ++ * generated, this is to ensure on appending, the compressor uses ++ * the same compression options that were used to generate the ++ * original filesystem. ++ * ++ * Note, even if there are no compressor options, this function is still ++ * called with an empty compressor structure (size == 0), to explicitly ++ * set the default options, this is to ensure any user supplied ++ * -X options on the appending mksquashfs command line are over-ridden. ++ * ++ * This function returns 0 on sucessful extraction of options, and -1 on error. ++ */ ++static int zstd_extract_options(int block_size, void *buffer, int size) ++{ ++ struct zstd_comp_opts *comp_opts = buffer; ++ ++ if (size == 0) { ++ /* Set default values */ ++ compression_level = ZSTD_DEFAULT_COMPRESSION_LEVEL; ++ return 0; ++ } ++ ++ /* we expect a comp_opts structure of sufficient size to be present */ ++ if (size < sizeof(*comp_opts)) ++ goto failed; ++ ++ SQUASHFS_INSWAP_COMP_OPTS(comp_opts); ++ ++ if (comp_opts->compression_level < 1 || ++ comp_opts->compression_level > ZSTD_maxCLevel()) { ++ fprintf(stderr, "zstd: bad compression level in compression " ++ "options structure\n"); ++ goto failed; ++ } ++ ++ compression_level = comp_opts->compression_level; ++ ++ return 0; ++ ++failed: ++ fprintf(stderr, "zstd: error reading stored compressor options from " ++ "filesystem!\n"); ++ ++ return -1; ++} ++ ++static void zstd_display_options(void *buffer, int size) ++{ ++ struct zstd_comp_opts *comp_opts = buffer; ++ ++ /* we expect a comp_opts structure of sufficient size to be present */ ++ if (size < sizeof(*comp_opts)) ++ goto failed; ++ ++ SQUASHFS_INSWAP_COMP_OPTS(comp_opts); ++ ++ if (comp_opts->compression_level < 1 || ++ comp_opts->compression_level > ZSTD_maxCLevel()) { ++ fprintf(stderr, "zstd: bad compression level in compression " ++ "options structure\n"); ++ goto failed; ++ } ++ ++ printf("\tcompression-level %d\n", comp_opts->compression_level); ++ ++ return; ++ ++failed: ++ fprintf(stderr, "zstd: error reading stored compressor options from " ++ "filesystem!\n"); ++} ++ ++/* ++ * This function is called by mksquashfs to initialise the ++ * compressor, before compress() is called. ++ * ++ * This function returns 0 on success, and -1 on error. ++ */ ++static int zstd_init(void **strm, int block_size, int datablock) ++{ ++ ZSTD_CCtx *cctx = ZSTD_createCCtx(); ++ ++ if (!cctx) { ++ fprintf(stderr, "zstd: failed to allocate compression " ++ "context!\n"); ++ return -1; ++ } ++ ++ *strm = cctx; ++ return 0; ++} ++ ++static int zstd_compress(void *strm, void *dest, void *src, int size, ++ int block_size, int *error) ++{ ++ const size_t res = ZSTD_compressCCtx((ZSTD_CCtx*)strm, dest, block_size, ++ src, size, compression_level); ++ ++ if (ZSTD_isError(res)) { ++ /* FIXME: ++ * zstd does not expose stable error codes. The error enum may ++ * change between versions. Until upstream zstd stablizes the ++ * error codes, we have no way of knowing why the error occurs. ++ * zstd shouldn't fail to compress any input unless there isn't ++ * enough output space. We assume that is the cause and return ++ * the special error code for not enough output space. ++ */ ++ return 0; ++ } ++ ++ return (int)res; ++} ++ ++static int zstd_uncompress(void *dest, void *src, int size, int outsize, ++ int *error) ++{ ++ const size_t res = ZSTD_decompress(dest, outsize, src, size); ++ ++ if (ZSTD_isError(res)) { ++ fprintf(stderr, "\t%d %d\n", outsize, size); ++ ++ *error = (int)ZSTD_getErrorCode(res); ++ return -1; ++ } ++ ++ return (int)res; ++} ++ ++static void zstd_usage(void) ++{ ++ fprintf(stderr, "\t -Xcompression-level \n"); ++ fprintf(stderr, "\t\t should be 1 .. %d (default " ++ "%d)\n", ZSTD_maxCLevel(), ZSTD_DEFAULT_COMPRESSION_LEVEL); ++} ++ ++struct compressor zstd_comp_ops = { ++ .init = zstd_init, ++ .compress = zstd_compress, ++ .uncompress = zstd_uncompress, ++ .options = zstd_options, ++ .dump_options = zstd_dump_options, ++ .extract_options = zstd_extract_options, ++ .display_options = zstd_display_options, ++ .usage = zstd_usage, ++ .id = ZSTD_COMPRESSION, ++ .name = "zstd", ++ .supported = 1 ++}; +diff --git a/squashfs-tools/zstd_wrapper.h b/squashfs-tools/zstd_wrapper.h +new file mode 100644 +index 0000000..4fbef0a +--- /dev/null ++++ b/squashfs-tools/zstd_wrapper.h +@@ -0,0 +1,48 @@ ++#ifndef ZSTD_WRAPPER_H ++#define ZSTD_WRAPPER_H ++/* ++ * Squashfs ++ * ++ * Copyright (c) 2017 ++ * Phillip Lougher ++ * ++ * This program is free software; you can redistribute it and/or ++ * modify it under the terms of the GNU General Public License ++ * as published by the Free Software Foundation; either version 2, ++ * or (at your option) any later version. ++ * ++ * This program is distributed in the hope that it will be useful, ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++ * GNU General Public License for more details. ++ * ++ * zstd_wrapper.h ++ * ++ */ ++ ++#ifndef linux ++#define __BYTE_ORDER BYTE_ORDER ++#define __BIG_ENDIAN BIG_ENDIAN ++#define __LITTLE_ENDIAN LITTLE_ENDIAN ++#else ++#include ++#endif ++ ++#if __BYTE_ORDER == __BIG_ENDIAN ++extern unsigned int inswap_le16(unsigned short); ++extern unsigned int inswap_le32(unsigned int); ++ ++#define SQUASHFS_INSWAP_COMP_OPTS(s) { \ ++ (s)->compression_level = inswap_le32((s)->compression_level); \ ++} ++#else ++#define SQUASHFS_INSWAP_COMP_OPTS(s) ++#endif ++ ++/* Default compression */ ++#define ZSTD_DEFAULT_COMPRESSION_LEVEL 15 ++ ++struct zstd_comp_opts { ++ int compression_level; ++}; ++#endif +-- +2.9.5 diff --git a/package/squashfs/Config.in b/package/squashfs/Config.in index 70c0fc9808..e793e8c977 100644 --- a/package/squashfs/Config.in +++ b/package/squashfs/Config.in @@ -41,6 +41,11 @@ config BR2_PACKAGE_SQUASHFS_XZ help Support XZ compression algorithm +config BR2_PACKAGE_SQUASHFS_ZSTD + bool "zstd support" + select BR2_PACKAGE_ZSTD + help + Support ZSTD compression algorithm endif comment "squashfs needs a toolchain w/ threads" diff --git a/package/squashfs/squashfs.mk b/package/squashfs/squashfs.mk index 9599d892b2..4ec15508e4 100644 --- a/package/squashfs/squashfs.mk +++ b/package/squashfs/squashfs.mk @@ -32,6 +32,13 @@ else SQUASHFS_MAKE_ARGS += XZ_SUPPORT=0 endif +ifeq ($(BR2_PACKAGE_SQUASHFS_ZSTD),y) +SQUASHFS_DEPENDENCIES += zstd +SQUASHFS_MAKE_ARGS += ZSTD_SUPPORT=1 COMP_DEFAULT=zstd +else +SQUASHFS_MAKE_ARGS += ZSTD_SUPPORT=0 +endif + ifeq ($(BR2_PACKAGE_SQUASHFS_LZO),y) SQUASHFS_DEPENDENCIES += lzo SQUASHFS_MAKE_ARGS += LZO_SUPPORT=1 COMP_DEFAULT=lzo @@ -46,7 +53,7 @@ else SQUASHFS_MAKE_ARGS += GZIP_SUPPORT=0 endif -HOST_SQUASHFS_DEPENDENCIES = host-zlib host-lz4 host-lzo host-xz +HOST_SQUASHFS_DEPENDENCIES = host-zlib host-lz4 host-lzo host-xz host-zstd HOST_SQUASHFS_MAKE_ARGS = \ XATTR_SUPPORT=1 \ @@ -54,7 +61,8 @@ HOST_SQUASHFS_MAKE_ARGS = \ GZIP_SUPPORT=1 \ LZ4_SUPPORT=1 \ LZO_SUPPORT=1 \ - LZMA_XZ_SUPPORT=1 + LZMA_XZ_SUPPORT=1 \ + ZSTD_SUPPORT=1 define SQUASHFS_BUILD_CMDS $(TARGET_MAKE_ENV) $(MAKE) \