mbox series

[v3,0/6] media: Introduce Allwinner H3 deinterlace driver

Message ID 20191016192807.1278987-1-jernej.skrabec@siol.net
Headers show
Series media: Introduce Allwinner H3 deinterlace driver | expand

Message

Jernej Škrabec Oct. 16, 2019, 7:28 p.m. UTC 
Starting with H3, Allwinner began to include standalone deinterlace
core in multimedia oriented SoCs. This patch series introduces support
for it. Note that new SoCs, like H6, have radically different (updated)
deinterlace core, which will need a new driver.

v4l2-compliance report:
v4l2-compliance SHA: dece02f862f38d8f866230ca9f1015cb93ddfac4, 32 bits

Compliance test for sun8i-di device /dev/video0:

Driver Info:
        Driver name      : sun8i-di
        Card type        : sun8i-di
        Bus info         : platform:sun8i-di
        Driver version   : 5.3.0
        Capabilities     : 0x84208000
                Video Memory-to-Memory
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04208000
                Video Memory-to-Memory
                Streaming
                Extended Pix Format

Required ioctls:
        test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
        test second /dev/video0 open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK
        test for unlimited opens: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
        test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
        test VIDIOC_QUERYCTRL: OK (Not Supported)
        test VIDIOC_G/S_CTRL: OK (Not Supported)
        test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
        test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
        test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
        Standard Controls: 0 Private Controls: 0

Format ioctls:
        test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
        test VIDIOC_G/S_PARM: OK (Not Supported)
        test VIDIOC_G_FBUF: OK (Not Supported)
        test VIDIOC_G_FMT: OK
        test VIDIOC_TRY_FMT: OK
        test VIDIOC_S_FMT: OK
        test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
        test Cropping: OK (Not Supported)
        test Composing: OK (Not Supported)
        test Scaling: OK

Codec ioctls:
        test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
        test VIDIOC_G_ENC_INDEX: OK (Not Supported)
        test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
        test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
        test VIDIOC_EXPBUF: OK
        test Requests: OK (Not Supported)

Total for sun8i-di device /dev/video0: 44, Succeeded: 44, Failed: 0, Warnings: 0

Please take a look.

Best regards,
Jernej

Changes from v2:
- added acked-by and review-by tags
- fixed schema path in H3 deinterlace binding
- moved busy check after format args check

Changes from v1:
- updated Maxime's e-mail in DT binding
- removed "items" for single item in DT binding
- implemented power management
- replaced regmap with direct io access
- set exclusive clock rate
- renamed DEINTERLACE_FRM_CTRL_COEF_CTRL to DEINTERLACE_FRM_CTRL_COEF_ACCESS

Jernej Skrabec (6):
  dt-bindings: bus: sunxi: Add H3 MBUS compatible
  clk: sunxi-ng: h3: Export MBUS clock
  ARM: dts: sunxi: h3/h5: Add MBUS controller node
  dt-bindings: media: Add Allwinner H3 Deinterlace binding
  media: sun4i: Add H3 deinterlace driver
  dts: arm: sun8i: h3: Enable deinterlace unit

 .../bindings/arm/sunxi/sunxi-mbus.txt         |    1 +
 .../media/allwinner,sun8i-h3-deinterlace.yaml |   75 ++
 MAINTAINERS                                   |    7 +
 arch/arm/boot/dts/sun8i-h3.dtsi               |   13 +
 arch/arm/boot/dts/sunxi-h3-h5.dtsi            |    9 +
 drivers/clk/sunxi-ng/ccu-sun8i-h3.h           |    4 -
 drivers/media/platform/sunxi/Kconfig          |    1 +
 drivers/media/platform/sunxi/Makefile         |    1 +
 drivers/media/platform/sunxi/sun8i-di/Kconfig |   11 +
 .../media/platform/sunxi/sun8i-di/Makefile    |    2 +
 .../media/platform/sunxi/sun8i-di/sun8i-di.c  | 1020 +++++++++++++++++
 .../media/platform/sunxi/sun8i-di/sun8i-di.h  |  237 ++++
 include/dt-bindings/clock/sun8i-h3-ccu.h      |    2 +-
 13 files changed, 1378 insertions(+), 5 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/media/allwinner,sun8i-h3-deinterlace.yaml
 create mode 100644 drivers/media/platform/sunxi/sun8i-di/Kconfig
 create mode 100644 drivers/media/platform/sunxi/sun8i-di/Makefile
 create mode 100644 drivers/media/platform/sunxi/sun8i-di/sun8i-di.c
 create mode 100644 drivers/media/platform/sunxi/sun8i-di/sun8i-di.h

Comments

Hans Verkuil Oct. 17, 2019, 7:51 a.m. UTC | #1 
On 10/16/19 9:28 PM, Jernej Skrabec wrote:
> Allwinner H3 SoC contains deinterlace unit, which has several modes of
> operation - bypass, weave, bob and mixed (advanced) mode. I don't know
> how mixed mode works, but according to Allwinner it gives best results,
> so they use it exclusively. Currently this mode is also hardcoded here.
> 
> For each interleaved frame queued, this driver produces 2 deinterlaced
> frames. Deinterlaced frames are based on 2 consequtive output buffers,
> except for the first 2, where same output buffer is given to peripheral
> as current and previous.
> 
> There is no documentation for this core, so register layout and fixed
> values were taken from BSP driver.
> 
> I'm not sure if maximum size of the image unit is capable to process is
> governed by size of "flag" buffers, frequency or it really is some HW
> limitation. Currently driver can process full HD image in ~15ms (7.5ms
> for each capture buffer), which allows to process 1920x1080@60i video
> smoothly in real time.
> 
> Signed-off-by: Jernej Skrabec <jernej.skrabec@siol.net>
> ---
>  MAINTAINERS                                   |    7 +
>  drivers/media/platform/sunxi/Kconfig          |    1 +
>  drivers/media/platform/sunxi/Makefile         |    1 +
>  drivers/media/platform/sunxi/sun8i-di/Kconfig |   11 +
>  .../media/platform/sunxi/sun8i-di/Makefile    |    2 +
>  .../media/platform/sunxi/sun8i-di/sun8i-di.c  | 1020 +++++++++++++++++
>  .../media/platform/sunxi/sun8i-di/sun8i-di.h  |  237 ++++
>  7 files changed, 1279 insertions(+)
>  create mode 100644 drivers/media/platform/sunxi/sun8i-di/Kconfig
>  create mode 100644 drivers/media/platform/sunxi/sun8i-di/Makefile
>  create mode 100644 drivers/media/platform/sunxi/sun8i-di/sun8i-di.c
>  create mode 100644 drivers/media/platform/sunxi/sun8i-di/sun8i-di.h
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index c7b48525822a..c375455125fb 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -4646,6 +4646,13 @@ M:	"Maciej W. Rozycki" <macro@linux-mips.org>
>  S:	Maintained
>  F:	drivers/net/fddi/defxx.*
>  
> +DEINTERLACE DRIVERS FOR ALLWINNER H3
> +M:	Jernej Skrabec <jernej.skrabec@siol.net>
> +L:	linux-media@vger.kernel.org
> +T:	git git://linuxtv.org/media_tree.git
> +S:	Maintained
> +F:	drivers/media/platform/sunxi/sun8i-di/
> +
>  DELL SMBIOS DRIVER
>  M:	Pali Rohár <pali.rohar@gmail.com>
>  M:	Mario Limonciello <mario.limonciello@dell.com>
> diff --git a/drivers/media/platform/sunxi/Kconfig b/drivers/media/platform/sunxi/Kconfig
> index 71808e93ac2e..d7a5621bf327 100644
> --- a/drivers/media/platform/sunxi/Kconfig
> +++ b/drivers/media/platform/sunxi/Kconfig
> @@ -1,2 +1,3 @@
>  source "drivers/media/platform/sunxi/sun4i-csi/Kconfig"
>  source "drivers/media/platform/sunxi/sun6i-csi/Kconfig"
> +source "drivers/media/platform/sunxi/sun8i-di/Kconfig"

This is a m2m driver, so this belongs in drivers/media/platform/Kconfig in the
Memory-to-memory section.

> diff --git a/drivers/media/platform/sunxi/Makefile b/drivers/media/platform/sunxi/Makefile
> index a05127529006..3878cb4efdc2 100644
> --- a/drivers/media/platform/sunxi/Makefile
> +++ b/drivers/media/platform/sunxi/Makefile
> @@ -1,2 +1,3 @@
>  obj-y		+= sun4i-csi/
>  obj-y		+= sun6i-csi/
> +obj-y		+= sun8i-di/
> diff --git a/drivers/media/platform/sunxi/sun8i-di/Kconfig b/drivers/media/platform/sunxi/sun8i-di/Kconfig
> new file mode 100644
> index 000000000000..dbd77a61e3b3
> --- /dev/null
> +++ b/drivers/media/platform/sunxi/sun8i-di/Kconfig
> @@ -0,0 +1,11 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +config VIDEO_SUN8I_DEINTERLACE
> +	tristate "Allwinner Deinterlace driver"
> +	depends on VIDEO_DEV && VIDEO_V4L2
> +	depends on HAS_DMA
> +	depends on OF
> +	depends on PM
> +	select VIDEOBUF2_DMA_CONTIG
> +	select V4L2_MEM2MEM_DEV
> +	help
> +	   Support for the Allwinner Deinterlace unit found on some SoCs.

Shouldn't this depend on ARCH_SUNXI || COMPILE_TEST?
And also on COMMON_CLK?

Regards,

	Hans
Maxime Ripard Oct. 17, 2019, 9:28 a.m. UTC | #2 
Hi,

I have a small comment that can definitely be addressed in a subsequent patch

On Wed, Oct 16, 2019 at 09:28:06PM +0200, Jernej Skrabec wrote:
> +	dev->bus_clk = devm_clk_get(dev->dev, "bus");
> +	if (IS_ERR(dev->bus_clk)) {
> +		dev_err(dev->dev, "Failed to get bus clock\n");
> +
> +		return PTR_ERR(dev->bus_clk);
> +	}
> +
> +	dev->mod_clk = devm_clk_get(dev->dev, "mod");
> +	if (IS_ERR(dev->mod_clk)) {
> +		dev_err(dev->dev, "Failed to get mod clock\n");
> +
> +		return PTR_ERR(dev->mod_clk);
> +	}
> +
> +	dev->ram_clk = devm_clk_get(dev->dev, "ram");
> +	if (IS_ERR(dev->ram_clk)) {
> +		dev_err(dev->dev, "Failed to get ram clock\n");
> +
> +		return PTR_ERR(dev->ram_clk);
> +	}
> +
> +	dev->rstc = devm_reset_control_get(dev->dev, NULL);
> +	if (IS_ERR(dev->rstc)) {
> +		dev_err(dev->dev, "Failed to get reset control\n");
> +
> +		return PTR_ERR(dev->rstc);
> +	}
> +
> +	clk_set_rate_exclusive(dev->mod_clk, 300000000);

clk_set_rate_exclusive puts a pretty big constraint on the clock tree,
and we shouldn't really enforce it if the device is unused.

I guess we should move it to the runtime_pm resume hook (with the
put_exclusive call in suspend).

Otherwise, that patch is
Acked-by: Maxime Ripard <mripard@kernel.org>

Maxime
Jernej Škrabec Oct. 17, 2019, 4:50 p.m. UTC | #3 
Dne četrtek, 17. oktober 2019 ob 09:51:28 CEST je Hans Verkuil napisal(a):
> On 10/16/19 9:28 PM, Jernej Skrabec wrote:
> > Allwinner H3 SoC contains deinterlace unit, which has several modes of
> > operation - bypass, weave, bob and mixed (advanced) mode. I don't know
> > how mixed mode works, but according to Allwinner it gives best results,
> > so they use it exclusively. Currently this mode is also hardcoded here.
> > 
> > For each interleaved frame queued, this driver produces 2 deinterlaced
> > frames. Deinterlaced frames are based on 2 consequtive output buffers,
> > except for the first 2, where same output buffer is given to peripheral
> > as current and previous.
> > 
> > There is no documentation for this core, so register layout and fixed
> > values were taken from BSP driver.
> > 
> > I'm not sure if maximum size of the image unit is capable to process is
> > governed by size of "flag" buffers, frequency or it really is some HW
> > limitation. Currently driver can process full HD image in ~15ms (7.5ms
> > for each capture buffer), which allows to process 1920x1080@60i video
> > smoothly in real time.
> > 
> > Signed-off-by: Jernej Skrabec <jernej.skrabec@siol.net>
> > ---
> > 
> >  MAINTAINERS                                   |    7 +
> >  drivers/media/platform/sunxi/Kconfig          |    1 +
> >  drivers/media/platform/sunxi/Makefile         |    1 +
> >  drivers/media/platform/sunxi/sun8i-di/Kconfig |   11 +
> >  .../media/platform/sunxi/sun8i-di/Makefile    |    2 +
> >  .../media/platform/sunxi/sun8i-di/sun8i-di.c  | 1020 +++++++++++++++++
> >  .../media/platform/sunxi/sun8i-di/sun8i-di.h  |  237 ++++
> >  7 files changed, 1279 insertions(+)
> >  create mode 100644 drivers/media/platform/sunxi/sun8i-di/Kconfig
> >  create mode 100644 drivers/media/platform/sunxi/sun8i-di/Makefile
> >  create mode 100644 drivers/media/platform/sunxi/sun8i-di/sun8i-di.c
> >  create mode 100644 drivers/media/platform/sunxi/sun8i-di/sun8i-di.h
> > 
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index c7b48525822a..c375455125fb 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -4646,6 +4646,13 @@ M:	"Maciej W. Rozycki" <macro@linux-mips.org>
> > 
> >  S:	Maintained
> >  F:	drivers/net/fddi/defxx.*
> > 
> > +DEINTERLACE DRIVERS FOR ALLWINNER H3
> > +M:	Jernej Skrabec <jernej.skrabec@siol.net>
> > +L:	linux-media@vger.kernel.org
> > +T:	git git://linuxtv.org/media_tree.git
> > +S:	Maintained
> > +F:	drivers/media/platform/sunxi/sun8i-di/
> > +
> > 
> >  DELL SMBIOS DRIVER
> >  M:	Pali Rohár <pali.rohar@gmail.com>
> >  M:	Mario Limonciello <mario.limonciello@dell.com>
> > 
> > diff --git a/drivers/media/platform/sunxi/Kconfig
> > b/drivers/media/platform/sunxi/Kconfig index 71808e93ac2e..d7a5621bf327
> > 100644
> > --- a/drivers/media/platform/sunxi/Kconfig
> > +++ b/drivers/media/platform/sunxi/Kconfig
> > @@ -1,2 +1,3 @@
> > 
> >  source "drivers/media/platform/sunxi/sun4i-csi/Kconfig"
> >  source "drivers/media/platform/sunxi/sun6i-csi/Kconfig"
> > 
> > +source "drivers/media/platform/sunxi/sun8i-di/Kconfig"
> 
> This is a m2m driver, so this belongs in drivers/media/platform/Kconfig in
> the Memory-to-memory section.
> 
> > diff --git a/drivers/media/platform/sunxi/Makefile
> > b/drivers/media/platform/sunxi/Makefile index a05127529006..3878cb4efdc2
> > 100644
> > --- a/drivers/media/platform/sunxi/Makefile
> > +++ b/drivers/media/platform/sunxi/Makefile
> > @@ -1,2 +1,3 @@
> > 
> >  obj-y		+= sun4i-csi/
> >  obj-y		+= sun6i-csi/
> > 
> > +obj-y		+= sun8i-di/
> > diff --git a/drivers/media/platform/sunxi/sun8i-di/Kconfig
> > b/drivers/media/platform/sunxi/sun8i-di/Kconfig new file mode 100644
> > index 000000000000..dbd77a61e3b3
> > --- /dev/null
> > +++ b/drivers/media/platform/sunxi/sun8i-di/Kconfig
> > @@ -0,0 +1,11 @@
> > +# SPDX-License-Identifier: GPL-2.0-only
> > +config VIDEO_SUN8I_DEINTERLACE
> > +	tristate "Allwinner Deinterlace driver"
> > +	depends on VIDEO_DEV && VIDEO_V4L2
> > +	depends on HAS_DMA
> > +	depends on OF
> > +	depends on PM
> > +	select VIDEOBUF2_DMA_CONTIG
> > +	select V4L2_MEM2MEM_DEV
> > +	help
> > +	   Support for the Allwinner Deinterlace unit found on some SoCs.
> 
> Shouldn't this depend on ARCH_SUNXI || COMPILE_TEST?
> And also on COMMON_CLK?

Yes to both. Also I don't see a reason why it would depend on HAS_DMA, so I 
will remove that.

Best regards,
Jernej

> 
> Regards,
> 
> 	Hans
Jernej Škrabec Oct. 17, 2019, 4:54 p.m. UTC | #4 
Dne četrtek, 17. oktober 2019 ob 11:28:00 CEST je Maxime Ripard napisal(a):
> Hi,
> 
> I have a small comment that can definitely be addressed in a subsequent
> patch
> On Wed, Oct 16, 2019 at 09:28:06PM +0200, Jernej Skrabec wrote:
> > +	dev->bus_clk = devm_clk_get(dev->dev, "bus");
> > +	if (IS_ERR(dev->bus_clk)) {
> > +		dev_err(dev->dev, "Failed to get bus clock\n");
> > +
> > +		return PTR_ERR(dev->bus_clk);
> > +	}
> > +
> > +	dev->mod_clk = devm_clk_get(dev->dev, "mod");
> > +	if (IS_ERR(dev->mod_clk)) {
> > +		dev_err(dev->dev, "Failed to get mod clock\n");
> > +
> > +		return PTR_ERR(dev->mod_clk);
> > +	}
> > +
> > +	dev->ram_clk = devm_clk_get(dev->dev, "ram");
> > +	if (IS_ERR(dev->ram_clk)) {
> > +		dev_err(dev->dev, "Failed to get ram clock\n");
> > +
> > +		return PTR_ERR(dev->ram_clk);
> > +	}
> > +
> > +	dev->rstc = devm_reset_control_get(dev->dev, NULL);
> > +	if (IS_ERR(dev->rstc)) {
> > +		dev_err(dev->dev, "Failed to get reset control\n");
> > +
> > +		return PTR_ERR(dev->rstc);
> > +	}
> > +
> > +	clk_set_rate_exclusive(dev->mod_clk, 300000000);
> 
> clk_set_rate_exclusive puts a pretty big constraint on the clock tree,
> and we shouldn't really enforce it if the device is unused.

That is true in general, but as I said before, that is not really an issue for 
H3. Deinterlace clock default parent is peripheral clock, which is fixed to 600 
MHz and doesn't change.

> 
> I guess we should move it to the runtime_pm resume hook (with the
> put_exclusive call in suspend).

Ok, I'll move it in case that this deinterlace core is used on other SoCs with 
non-fixed parent clock.

> 
> Otherwise, that patch is
> Acked-by: Maxime Ripard <mripard@kernel.org>

Thanks!

Best regards,
Jernej