RFC: kernel: Remove CFQ activate BFQ

The CFQ IO scheduler is deleted from the Linux kernel
since commit f382fb0bcef4c37dc049e9f6963e3baf204d815c
"block: remove legacy IO schedulers".

The new multiqueue block layer has an IO scheduler named
BFQ (Budget Fair Queue) that is suitable for slow
single-queue block devices such as (S)ATA harddrives,
flash memories, MMC/SD cards and USB dongles, so activate
that instead.

To actually use BFQ by default distributions such as
Fedora uses this (/lib/udev/rules.d/60-block-scheduler.rules):

ACTION=="add", SUBSYSTEM=="block", \
  KERNEL=="mmcblk*[0-9]|msblk*[0-9]|mspblk*[0-9]|sd*[!0-9]|sr*", \
  ATTR{queue/scheduler}="bfq"

We are not using udev so we need a similar rule in a
similar patch for procd, or we can patch the kernel to
use BFQ by default if we do not expect people to use
OpenWRT with enterprise storage arrays.

Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
---
This patch is more of an RFC, I am in contact with the
author of BFQ if we need to discuss its applicability for
OpenWRT targets. I can make a kernel patch to make BFQ
the default IO scheduler as well.
---
 target/linux/generic/config-5.10 | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

Shoud it be provided as an option to use bfq scheduler? Rrouters do not 
involve many I/O operations so I guess mq-deadline is enough.
Reference: 
https://github.com/NoTengoBattery/openwrt-mt7621/commit/a2385cd792542da38c90a0718e9a7952f848ecba

> The CFQ IO scheduler is deleted from the Linux kernel
> since commit f382fb0bcef4c37dc049e9f6963e3baf204d815c
> "block: remove legacy IO schedulers".
> 
> The new multiqueue block layer has an IO scheduler named
> BFQ (Budget Fair Queue) that is suitable for slow
> single-queue block devices such as (S)ATA harddrives,
> flash memories, MMC/SD cards and USB dongles, so activate
> that instead.
> 
> To actually use BFQ by default distributions such as
> Fedora uses this (/lib/udev/rules.d/60-block-scheduler.rules):
> 
> ACTION=="add", SUBSYSTEM=="block", \
>    KERNEL=="mmcblk*[0-9]|msblk*[0-9]|mspblk*[0-9]|sd*[!0-9]|sr*", \
>    ATTR{queue/scheduler}="bfq"
> 
> We are not using udev so we need a similar rule in a
> similar patch for procd, or we can patch the kernel to
> use BFQ by default if we do not expect people to use
> OpenWRT with enterprise storage arrays.
> 
> Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
> ---
> This patch is more of an RFC, I am in contact with the
> author of BFQ if we need to discuss its applicability for
> OpenWRT targets. I can make a kernel patch to make BFQ
> the default IO scheduler as well.
> ---
>   target/linux/generic/config-5.10 | 3 +--
>   1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/target/linux/generic/config-5.10 b/target/linux/generic/config-5.10
> index 120d1d8c3f4a..fda3ef301dde 100644
> --- a/target/linux/generic/config-5.10
> +++ b/target/linux/generic/config-5.10
> @@ -2601,8 +2601,7 @@ CONFIG_INPUT_MISC=y
>   # CONFIG_INV_MPU6050_SPI is not set
>   # CONFIG_IOMMU_SUPPORT is not set
>   # CONFIG_IONIC is not set
> -# CONFIG_IOSCHED_BFQ is not set
> -# CONFIG_IOSCHED_CFQ is not set
> +CONFIG_IOSCHED_BFQ=y
>   CONFIG_IOSCHED_DEADLINE=y
>   CONFIG_IOSCHED_NOOP=y
>   CONFIG_IO_STRICT_DEVMEM=y
>

On Wed, Apr 14, 2021 at 4:46 AM Linus Walleij <linus.walleij@linaro.org> wrote:
>
> The CFQ IO scheduler is deleted from the Linux kernel
> since commit f382fb0bcef4c37dc049e9f6963e3baf204d815c
> "block: remove legacy IO schedulers".
>
> The new multiqueue block layer has an IO scheduler named
> BFQ (Budget Fair Queue) that is suitable for slow
> single-queue block devices such as (S)ATA harddrives,
> flash memories, MMC/SD cards and USB dongles, so activate
> that instead.
This should be tested first.

Most OpenWrt routers use squashfs with a jffs2 overlay. Last I tested,
there was no difference in speed. There would be when using USB or
SATA connected drives.
>
> To actually use BFQ by default distributions such as
> Fedora uses this (/lib/udev/rules.d/60-block-scheduler.rules):
>
> ACTION=="add", SUBSYSTEM=="block", \
>   KERNEL=="mmcblk*[0-9]|msblk*[0-9]|mspblk*[0-9]|sd*[!0-9]|sr*", \
>   ATTR{queue/scheduler}="bfq"
>
> We are not using udev so we need a similar rule in a
> similar patch for procd, or we can patch the kernel to
> use BFQ by default if we do not expect people to use
> OpenWRT with enterprise storage arrays.
>
> Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
> ---
> This patch is more of an RFC, I am in contact with the
> author of BFQ if we need to discuss its applicability for
> OpenWRT targets. I can make a kernel patch to make BFQ
> the default IO scheduler as well.
Last time I posted a patch messing with IOSCHED it got rejected.
Specifically because it removed all io schedulers except noop, which
seems to perform equal to CFQ last I tested.
> ---
>  target/linux/generic/config-5.10 | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/target/linux/generic/config-5.10 b/target/linux/generic/config-5.10
> index 120d1d8c3f4a..fda3ef301dde 100644
> --- a/target/linux/generic/config-5.10
> +++ b/target/linux/generic/config-5.10
> @@ -2601,8 +2601,7 @@ CONFIG_INPUT_MISC=y
>  # CONFIG_INV_MPU6050_SPI is not set
>  # CONFIG_IOMMU_SUPPORT is not set
>  # CONFIG_IONIC is not set
> -# CONFIG_IOSCHED_BFQ is not set
> -# CONFIG_IOSCHED_CFQ is not set
> +CONFIG_IOSCHED_BFQ=y
>  CONFIG_IOSCHED_DEADLINE=y
>  CONFIG_IOSCHED_NOOP=y
>  CONFIG_IO_STRICT_DEVMEM=y
> --
> 2.29.2
>
>
> _______________________________________________
> openwrt-devel mailing list
> openwrt-devel@lists.openwrt.org
> https://lists.openwrt.org/mailman/listinfo/openwrt-devel

On Thu, Apr 15, 2021 at 12:45 AM Rosen Penev <rosenp@gmail.com> wrote:
> On Wed, Apr 14, 2021 at 4:46 AM Linus Walleij <linus.walleij@linaro.org> wrote:

> > The new multiqueue block layer has an IO scheduler named
> > BFQ (Budget Fair Queue) that is suitable for slow
> > single-queue block devices such as (S)ATA harddrives,
> > flash memories, MMC/SD cards and USB dongles, so activate
> > that instead.
>
> This should be tested first.

Yeah I have tested it a lot. Running it on a host of Arm systems
as we worked with the author, also stress tested with iozone
for example. It's quite stable now and Fedora trust it enough
to enable it by default.

> Most OpenWrt routers use squashfs with a jffs2 overlay. Last I tested,
> there was no difference in speed. There would be when using USB or
> SATA connected drives.

BFQ isn't really about speed, but latency.

OpenWRT is a bit ambiguous about separating router and
NAS usecases, for routers I agree, no big deal, but for NAS
devices, BFQ will detect interactive loads. This happens
for example when a user is streaming a movie from a
drive, then other IO will be deprioritized and the interactive
video stream will be prioritized and less prone to skip.

I asked Paolo at some point whether interaction over
the network (Samba, NFS) will be detected if interactive
and yes, it should do that. It is really all about the
behaviour of the process, such as smbd/nfsd.

Another factor is that BFQ helps to not bring down a system
which starts thrashing to swap. I blogged about it here:
https://people.kernel.org/linusw/bfq-saved-me-from-thrashing
Essentially it will priotitize the interactive shell over the thrashing
processes. This is maybe more esoteric but sometimes
a life-saver.

> Last time I posted a patch messing with IOSCHED it got rejected.
> Specifically because it removed all io schedulers except noop, which
> seems to perform equal to CFQ last I tested.

It seems they are all in a module package now which is
neat, I can probably augment the patch to just delete CFQ
from the v5.10 config I guess, MQ_DEADLINE will be the
default anyway.

Yours,
Linus Walleij

On Wed, Apr 14, 2021 at 2:01 PM Fusion <qydwhotmail@gmail.com> wrote:

> Shoud it be provided as an option to use bfq scheduler? Rrouters do not
> involve many I/O operations so I guess mq-deadline is enough.
> Reference:
> https://github.com/NoTengoBattery/openwrt-mt7621/commit/a2385cd792542da38c90a0718e9a7952f848ecba

Aha I didn't know there was a package for it, nice. Then I would only
need to patch some rules into that package which is more of a local
change.

Thanks!
Linus Walleij