[1/1] package/rust-coreutils:new package

The Rust version aims to be a drop-in replacement for GNU coreutils,
designed to work across a wide range of platforms. Adding it to
Buildroot can help make it available on embedded devices.

This is the initial version of the package, supporting only the
multicall version (single binary). It allows starting to work with the
utilities while avoiding complex configurations and conflicts with
existing tools.

Signed-off-by: Alexander Shirokov <shirokovalexs@gmail.com>
---
 package/Config.in                          |  1 +
 package/rust-coreutils/Config.in           | 12 ++++++++++++
 package/rust-coreutils/rust-coreutils.hash |  3 +++
 package/rust-coreutils/rust-coreutils.mk   | 13 +++++++++++++
 4 files changed, 29 insertions(+)
 create mode 100644 package/rust-coreutils/Config.in
 create mode 100644 package/rust-coreutils/rust-coreutils.hash
 create mode 100644 package/rust-coreutils/rust-coreutils.mk

Hello Alexander,

Le 08/12/2024 à 14:40, Alexander Shirokov a écrit :
> The Rust version aims to be a drop-in replacement for GNU coreutils,
> designed to work across a wide range of platforms. Adding it to
> Buildroot can help make it available on embedded devices.
> 
> This is the initial version of the package, supporting only the
> multicall version (single binary). It allows starting to work with the
> utilities while avoiding complex configurations and conflicts with
> existing tools.

There was a previous attempt last year to introduce uutils-coreutils package but
Thomas reported some issues on ARM targets [1]. Can you check if they were fixed
since then?

Unlikely Sebastian (my former intern) was not able to resend an updated version.

Note: We try to keep the upstream project name to name the Buildroot package. In
this case it should be uutils-coreutils not rust-coreutils.

Also we should take care when several "coreutils" provider is selected to avoid
any binaries override.

Can you adopt the patch [1] and address the remaining issues?
Also having a runtime test would be great.

Thanks!

[1] https://lore.kernel.org/buildroot/20230930215940.7644e93c@windsurf/

Best regards,
Romain


> 
> Signed-off-by: Alexander Shirokov <shirokovalexs@gmail.com>
> ---
>  package/Config.in                          |  1 +
>  package/rust-coreutils/Config.in           | 12 ++++++++++++
>  package/rust-coreutils/rust-coreutils.hash |  3 +++
>  package/rust-coreutils/rust-coreutils.mk   | 13 +++++++++++++
>  4 files changed, 29 insertions(+)
>  create mode 100644 package/rust-coreutils/Config.in
>  create mode 100644 package/rust-coreutils/rust-coreutils.hash
>  create mode 100644 package/rust-coreutils/rust-coreutils.mk
> 
> diff --git a/package/Config.in b/package/Config.in
> index 60cdb179a3..48b1d96b4e 100644
> --- a/package/Config.in
> +++ b/package/Config.in
> @@ -2874,6 +2874,7 @@ menu "System tools"
>  	source "package/rauc-hawkbit-updater/Config.in"
>  	source "package/rsyslog/Config.in"
>  	source "package/runc/Config.in"
> +	source "package/rust-coreutils/Config.in"
>  	source "package/s390-tools/Config.in"
>  	source "package/s6/Config.in"
>  	source "package/s6-linux-init/Config.in"
> diff --git a/package/rust-coreutils/Config.in b/package/rust-coreutils/Config.in
> new file mode 100644
> index 0000000000..0371805925
> --- /dev/null
> +++ b/package/rust-coreutils/Config.in
> @@ -0,0 +1,12 @@
> +config BR2_PACKAGE_RUST_COREUTILS
> +	bool "rust-coreutils"
> +	depends on BR2_PACKAGE_HOST_RUSTC_TARGET_ARCH_SUPPORTS
> +	select BR2_PACKAGE_HOST_RUSTC
> +	help
> +	  Rust coreutils is a cross-platform reimplementation of the GNU
> +	  coreutils, aiming to be a drop-in replacement.
> +
> +	  Select this option to build a multicall binary, named
> +	  'coreutils'.
> +
> +	  https://github.com/uutils/coreutils
> diff --git a/package/rust-coreutils/rust-coreutils.hash b/package/rust-coreutils/rust-coreutils.hash
> new file mode 100644
> index 0000000000..bb8e4ac6ca
> --- /dev/null
> +++ b/package/rust-coreutils/rust-coreutils.hash
> @@ -0,0 +1,3 @@
> +# Locally calculated
> +sha256  28e7c79e608989af331fface204fd922eec2a22c4aa2c43a91dced884b6de8d2  rust-coreutils-0.0.28-cargo2.tar.gz
> +sha256  29953405eaa3dcc41c37d1621d55b6a47eee93e05613e439e73295029740b10c  LICENSE
> diff --git a/package/rust-coreutils/rust-coreutils.mk b/package/rust-coreutils/rust-coreutils.mk
> new file mode 100644
> index 0000000000..ca41f7101a
> --- /dev/null
> +++ b/package/rust-coreutils/rust-coreutils.mk
> @@ -0,0 +1,13 @@
> +################################################################################
> +#
> +# rust-coreutils
> +#
> +################################################################################
> +
> +RUST_COREUTILS_VERSION = 0.0.28
> +RUST_COREUTILS_SITE = $(call github,uutils,coreutils,$(RUST_COREUTILS_VERSION))
> +RUST_COREUTILS_LICENSE = MIT
> +RUST_COREUTILS_LICENSE_FILES = LICENSE
> +RUST_COREUTILS_CPE_ID_VALID = YES
> +
> +$(eval $(cargo-package))

Hello Romain,

Thank you for your answer

> There was a previous attempt last year to introduce uutils-coreutils package
> but Thomas reported some issues on ARM targets [1]. Can you check if they were
> fixed since then?

Sure, I'll take a look at the previous attempt and try to build for ARM targets.
Any specific recommendations on which targets to check?

A few more words about the ARM build: I used 'test-pkg' for a quick test of my
changes

./utils/test-pkg  -p rust-coreutils
                    bootlin-armv5-uclibc [1/6]: SKIPPED
                     bootlin-armv7-glibc [2/6]: OK
                   bootlin-armv7m-uclibc [3/6]: SKIPPED
                     bootlin-x86-64-musl [4/6]: OK
                      br-arm-full-static [5/6]: SKIPPED
                             arm-aarch64 [6/6]: OK
6 builds, 3 skipped, 0 build failed, 0 legal-info failed, 0 show-info failed

armv7 and aarch64 seem fine. armv5 was skipped, but I can try it as well

> Note: We try to keep the upstream project name to name the Buildroot package

I also had some doubts about the naming :) I'll fix. Thank you.

> Also we should take care when several "coreutils" provider is selected to
> avoid any binaries override.

Agree. But my initial idea here was to add uutils-coreutils in several steps.

The first step is to add a single binary (possibly with a rename, for example,
uu_coreutils). This will allow us to deliver, debug, and compare
uutils-coreutils with other implementations with no or minimal conflicts. This
step is mostly for development, collecting statistics, and improving
uutils-coreutils.

The second step is adding the ability to replace coreutils with
uutils-coreutils. This is a good place to add providers, flags for switching
between single/multi binaries, and so on.

Anyway, it's just my initial idea. If a better one exists, we can discuss it and
consider another solution.

> Can you adopt the patch [1] and address the remaining issues?

Sure. I'll take a look at [1] and the remaining issues.

> Also having a runtime test would be great.

Romain, sorry, I'm not sure I fully understand you. Could you please clarify
what you mean by "having a runtime test"?

Are you referring to running uutils-coreutils on target devices? Or adding the
ability to include unit tests in an image to run on target devices? Or something
else?

Thank you

Best regards,
Alexander

Hello Alexander, All,

Le 08/02/2025 à 15:01, Alexander Shirokov a écrit :
> Hello Romain,
> 
> Thank you for your answer
> 
>> There was a previous attempt last year to introduce uutils-coreutils package
>> but Thomas reported some issues on ARM targets [1]. Can you check if they were
>> fixed since then?
> 
> Sure, I'll take a look at the previous attempt and try to build for ARM targets.
> Any specific recommendations on which targets to check?
> 
> A few more words about the ARM build: I used 'test-pkg' for a quick test of my
> changes
> 
> ./utils/test-pkg  -p rust-coreutils
>                     bootlin-armv5-uclibc [1/6]: SKIPPED
>                      bootlin-armv7-glibc [2/6]: OK
>                    bootlin-armv7m-uclibc [3/6]: SKIPPED
>                      bootlin-x86-64-musl [4/6]: OK
>                       br-arm-full-static [5/6]: SKIPPED
>                              arm-aarch64 [6/6]: OK
> 6 builds, 3 skipped, 0 build failed, 0 legal-info failed, 0 show-info failed
> 
> armv7 and aarch64 seem fine. armv5 was skipped, but I can try it as well

When I saw your contribution, I remembered the build issue reported by Thomas
some time ago.
I don't remember the details but it would be great to check if the issue can be
reproduced or not.

> 
>> Note: We try to keep the upstream project name to name the Buildroot package
> 
> I also had some doubts about the naming :) I'll fix. Thank you.
> 
>> Also we should take care when several "coreutils" provider is selected to
>> avoid any binaries override.
> 
> Agree. But my initial idea here was to add uutils-coreutils in several steps.
> 
> The first step is to add a single binary (possibly with a rename, for example,
> uu_coreutils). This will allow us to deliver, debug, and compare
> uutils-coreutils with other implementations with no or minimal conflicts. This
> step is mostly for development, collecting statistics, and improving
> uutils-coreutils.
> 
> The second step is adding the ability to replace coreutils with
> uutils-coreutils. This is a good place to add providers, flags for switching
> between single/multi binaries, and so on.
> 
> Anyway, it's just my initial idea. If a better one exists, we can discuss it and
> consider another solution.

This deserve an explanation in the commit log I guess. But users selecting this
package will expect that the rust variant is used by default.

See similar handling between busybox and coreutils.

> 
>> Can you adopt the patch [1] and address the remaining issues?
> 
> Sure. I'll take a look at [1] and the remaining issues.

Thanks!

> 
>> Also having a runtime test would be great.
> 
> Romain, sorry, I'm not sure I fully understand you. Could you please clarify
> what you mean by "having a runtime test"?
> 
> Are you referring to running uutils-coreutils on target devices? Or adding the
> ability to include unit tests in an image to run on target devices? Or something
> else?

Buildroot uses a testsuite to perform some runtime testing using Qemu.

See:
https://buildroot.org/downloads/manual/manual.html#_using_the_runtime_tests_framework

Here an example of the testsuite result:
https://gitlab.com/buildroot.org/buildroot/-/pipelines/1652320183/failures

(tests.package.test_rust.TestRustVendoring)
https://gitlab.com/buildroot.org/buildroot/-/jobs/9019173822

It's not mandatory but highly recommended.

Best regards,
Romain


> 
> Thank you
> 
> Best regards,
> Alexander

Hello Romain, All,

Sorry for the delayed response. Here’s an update from my side.

I tested the old patches with the ARM build and was able to reproduce the error
on the original 0.0.17 version, as well as on newer 0.0.28 and 0.0.30 versions.
The issue comes from the "stdbuf" crate, which is enabled by the

  UUTILS_COREUTILS_CARGO_BUILD_OPTS = --features unix 

flag. 

This is a known issue and has been documented on the uutils side [2].

I’d like to recheck some aspects of the uutils side, especially since it has its
own build.rs script, which could be a source of portability issues.

As a temporary solution we can disable the "--features unix" and use only the
default core utilities. On one hand, this will allow us to build and deliver a
wide range of utilities. On the other hand, it will provide a good way to
reproduce and fix the issue.

Links [3] and [4] provide detailed information about which utilities are
included in the "core" and which ones are provided by the "unix" feature.

In the next few days, I plan to work on and share:

- Updating the original patch [1] to use the latest uutils-coreutils version
- Fixing the build by disabling "--features unix" by default and allowing it
  over configuration flag
- Adding runtime tests
- Sharing the changes as package/uutils-coreutils v4

Any comments and suggestions are very welcome.

Thank you!

Links:

[1] - https://patchwork.ozlabs.org/project/buildroot/patch/20230317151802.1242858-1-sebastian.weyer@smile.fr/ 
[2] - https://github.com/uutils/coreutils/blob/aea23408fd839e8f3316d5e5839bec439841af5d/Cargo.toml#L191
[3] - https://github.com/uutils/coreutils/blob/aea23408fd839e8f3316d5e5839bec439841af5d/Cargo.toml#L62
[4] - https://github.com/uutils/coreutils/blob/aea23408fd839e8f3316d5e5839bec439841af5d/Cargo.toml#L154

The new version with fixes and improvements is
here:https://patchwork.ozlabs.org/project/buildroot/patch/20250402165913.39797-2-shirokovalexs@gmail.com/