Discussion:
i.MX6 status for IPU/VPU/GPU
Jean-Michel Hautbois
2014-07-28 16:24:45 UTC
Permalink
Hi there !

We have a custom board, based on i.MX6 SoC.
We are currently using Freescale's release of Linux, but this is a
3.10.17 kernel, and several drivers are lacking (adv7611 for instance)
or badly written (all the MXC part).
As we want to have nice things :) we would like to use a mainline
kernel, or at least a tree which can be mainlined.

It seems (#v4l told me so) that some people (Steeve :) ?) are working
on a rewriting of the IPU and all DRM part for i.MX6.
What is the current status (compared to Freescale's release maybe) ?
And what can we expect in a near future ? Maybe, how can we help too ?

Thanks,
JM
Robert Schwebel
2014-07-28 18:59:49 UTC
Permalink
Hi,
Post by Jean-Michel Hautbois
We have a custom board, based on i.MX6 SoC.
We are currently using Freescale's release of Linux, but this is a
3.10.17 kernel, and several drivers are lacking (adv7611 for instance)
or badly written (all the MXC part).
As we want to have nice things :) we would like to use a mainline
kernel, or at least a tree which can be mainlined.
It seems (#v4l told me so) that some people (Steeve :) ?) are working
on a rewriting of the IPU and all DRM part for i.MX6.
What is the current status (compared to Freescale's release maybe) ?
And what can we expect in a near future? Maybe, how can we help too ?
Pengutronix is continuously working on mainlining more parts of the
i.MX6 video and graphics subsystem, including the components you have
mentioned. We are posting patches here when they are ready for mainline
review.

Regards,
Robert (for commercial help, please contact me by email)
--
Pengutronix e.K. | |
Industrial Linux Solutions | http://www.pengutronix.de/ |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
Steve Longerbeam
2014-07-28 21:15:58 UTC
Permalink
This post might be inappropriate. Click to display it.
Tim Harvey
2014-08-04 06:14:22 UTC
Permalink
Post by Steve Longerbeam
Post by Robert Schwebel
Hi,
Post by Jean-Michel Hautbois
We have a custom board, based on i.MX6 SoC.
We are currently using Freescale's release of Linux, but this is a
3.10.17 kernel, and several drivers are lacking (adv7611 for instance)
or badly written (all the MXC part).
As we want to have nice things :) we would like to use a mainline
kernel, or at least a tree which can be mainlined.
It seems (#v4l told me so) that some people (Steeve :) ?) are working
on a rewriting of the IPU and all DRM part for i.MX6.
What is the current status (compared to Freescale's release maybe) ?
And what can we expect in a near future? Maybe, how can we help too ?
Hi Jean-Michel,
I did post a v4l2 video capture driver for i.MX6 to linux-media.
The main complaint from Philip at Pengutronix is that it does not
support the media device framework.
Philipp,

It is unfortunate that the lack of the media device framework is
holding back acceptance of Steve's patches. Is this something that can
be added later? Does your patchset which you posted for reference
resolve this issue and perhaps is something that everyone could agree
on for a starting point?

Regards,

Tim
Post by Steve Longerbeam
The customer I am currently working for has no real interest in the
media controller API, and the driver I posted has all the features they
require, so any work I do to add that support to the driver would have
to be in my spare time, and I don't have much. If our customer were to
request and fund media control support, that would be ideal, but as it is
I can only spend limited time on it. So if you are interested in helping
out in the media device effort I can send what I have so far.
I have not provided any patches to i.MX6 DRM/KMS drivers. We have
developed new features (overlay plane global/local alpha, hardware gamma
correction, color-keying, and others) for for that component but haven't
posted them yet.
Steve
Post by Robert Schwebel
Pengutronix is continuously working on mainlining more parts of the
i.MX6 video and graphics subsystem, including the components you have
mentioned. We are posting patches here when they are ready for mainline
review.
Regards,
Robert (for commercial help, please contact me by email)
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Philipp Zabel
2014-08-04 11:54:17 UTC
Permalink
Hi Tim,
Post by Tim Harvey
Philipp,
It is unfortunate that the lack of the media device framework is
holding back acceptance of Steve's patches. Is this something that can
be added later? Does your patchset which you posted for reference
resolve this issue and perhaps is something that everyone could agree
on for a starting point?
We should take this step by step. First I'd like to get Steve's ipu-v3
series in, those don't have any major issues and are a prerequisite for
the media patches anyway.

The capture patches had a few more issues than just missing media device
support. But this is indeed the biggest one, especially where it
involves a userspace interface that we don't want to have to support in
the future.
My RFC series wasn't without problems either. I'll work on the IPU this
week and then post another RFC.

regards
Philipp
Jean-Michel Hautbois
2014-08-05 08:07:11 UTC
Permalink
Post by Philipp Zabel
Hi Tim,
Post by Tim Harvey
Philipp,
It is unfortunate that the lack of the media device framework is
holding back acceptance of Steve's patches. Is this something that can
be added later? Does your patchset which you posted for reference
resolve this issue and perhaps is something that everyone could agree
on for a starting point?
We should take this step by step. First I'd like to get Steve's ipu-v3
series in, those don't have any major issues and are a prerequisite for
the media patches anyway.
The capture patches had a few more issues than just missing media device
support. But this is indeed the biggest one, especially where it
involves a userspace interface that we don't want to have to support in
the future.
My RFC series wasn't without problems either. I'll work on the IPU this
week and then post another RFC.
Are the capture patches using v4l2_async_notifier_register() ?
I can help integrating/testing all these patches, if you want...

JM
Jean-Michel Hautbois
2014-08-27 07:13:10 UTC
Permalink
Hi Phillip,
Post by Philipp Zabel
We should take this step by step. First I'd like to get Steve's ipu-v3
series in, those don't have any major issues and are a prerequisite for
the media patches anyway.
The capture patches had a few more issues than just missing media device
support. But this is indeed the biggest one, especially where it
involves a userspace interface that we don't want to have to support in
the future.
My RFC series wasn't without problems either. I'll work on the IPU this
week and then post another RFC.
Any news about this ? I saw your patchset from june 12th.
What is the current status of this RFC and is there a way to help
integrating/testing it ? Do you have a public git repository I can
fetch and merge in order to test ?


Thanks,
JM
Steve Longerbeam
2014-08-27 14:23:29 UTC
Permalink
Post by Jean-Michel Hautbois
Hi Phillip,
Post by Philipp Zabel
We should take this step by step. First I'd like to get Steve's ipu-v3
series in, those don't have any major issues and are a prerequisite for
the media patches anyway.
The capture patches had a few more issues than just missing media device
support. But this is indeed the biggest one, especially where it
involves a userspace interface that we don't want to have to support in
the future.
My RFC series wasn't without problems either. I'll work on the IPU this
week and then post another RFC.
Any news about this ? I saw your patchset from june 12th.
What is the current status of this RFC and is there a way to help
integrating/testing it ? Do you have a public git repository I can
fetch and merge in order to test ?
Hi Jean-Michel, Phillip,

I've done some work on Philipp's June 12 patchset, converting
the CSI driver to a CSI subdev entity, and fixing some issues here
and there. This June 12 patchset doesn't appear to be a fully working
driver, Phillip correct me if I am wrong. I can post this work as it
exists, it is incomplete but compiles.

I've also worked out what I think is a workable video pipeline graph for i.MX,
suitable for defining the entities, pads, and links. Unfortunately I haven't
been able to spend as much time as I'd like on it.

The complete driver I posted to the list does have some minor issues
mostly suggested by Hans Verkuil (switch to new selection API instead
of cropping API for example). It is a full featured driver but it does not
implement the media device framework, i.e. user does not have direct
control of the video pipeline, rather the driver chooses the pipeline based
on the traditional inputs from user (video format and controls).

If there is interest I can submit another version of the traditional driver
to resolve the issues. But media device is a major rework, so I don't
know whether it would make sense to start from the traditional driver
and then implement media device on top later, since media device
is almost a complete rewrite.

Steve
Jean-Michel Hautbois
2014-09-09 07:49:56 UTC
Permalink
Post by Steve Longerbeam
Hi Jean-Michel, Phillip,
Hi Steve,
Post by Steve Longerbeam
I've done some work on Philipp's June 12 patchset, converting
the CSI driver to a CSI subdev entity, and fixing some issues here
and there. This June 12 patchset doesn't appear to be a fully working
driver, Phillip correct me if I am wrong. I can post this work as it
exists, it is incomplete but compiles.
Dos it compile against a 3.17-rc3 kernel :) ?
Post by Steve Longerbeam
I've also worked out what I think is a workable video pipeline graph for i.MX,
suitable for defining the entities, pads, and links. Unfortunately I haven't
been able to spend as much time as I'd like on it.
This is very interesting, do you have written this somewhere ?
Post by Steve Longerbeam
The complete driver I posted to the list does have some minor issues
mostly suggested by Hans Verkuil (switch to new selection API instead
of cropping API for example). It is a full featured driver but it does not
implement the media device framework, i.e. user does not have direct
control of the video pipeline, rather the driver chooses the pipeline based
on the traditional inputs from user (video format and controls).
If there is interest I can submit another version of the traditional driver
to resolve the issues. But media device is a major rework, so I don't
know whether it would make sense to start from the traditional driver
and then implement media device on top later, since media device
is almost a complete rewrite.
I, at least, am interested by this driver, even in its "traditionnal"
form :). If you don't want to submit it directly because this is not
using media controller, this is ok, you can provide me a git repo in
order to get it, or send a patchset.

JM
Hans Verkuil
2014-09-09 07:52:08 UTC
Permalink
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
Hi Jean-Michel, Phillip,
Hi Steve,
Post by Steve Longerbeam
I've done some work on Philipp's June 12 patchset, converting
the CSI driver to a CSI subdev entity, and fixing some issues here
and there. This June 12 patchset doesn't appear to be a fully working
driver, Phillip correct me if I am wrong. I can post this work as it
exists, it is incomplete but compiles.
Dos it compile against a 3.17-rc3 kernel :) ?
Post by Steve Longerbeam
I've also worked out what I think is a workable video pipeline graph for i.MX,
suitable for defining the entities, pads, and links. Unfortunately I haven't
been able to spend as much time as I'd like on it.
This is very interesting, do you have written this somewhere ?
Post by Steve Longerbeam
The complete driver I posted to the list does have some minor issues
mostly suggested by Hans Verkuil (switch to new selection API instead
of cropping API for example). It is a full featured driver but it does not
implement the media device framework, i.e. user does not have direct
control of the video pipeline, rather the driver chooses the pipeline based
on the traditional inputs from user (video format and controls).
If there is interest I can submit another version of the traditional driver
to resolve the issues. But media device is a major rework, so I don't
know whether it would make sense to start from the traditional driver
and then implement media device on top later, since media device
is almost a complete rewrite.
I, at least, am interested by this driver, even in its "traditionnal"
form :). If you don't want to submit it directly because this is not
using media controller, this is ok, you can provide me a git repo in
order to get it, or send a patchset.
Is it possible to create a staging driver? Even if there are bits missing,
having the code in the kernel as a staging driver would help a lot.

Regards,

Hans
Steve Longerbeam
2014-09-09 16:06:06 UTC
Permalink
Post by Hans Verkuil
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
Hi Jean-Michel, Phillip,
Hi Steve,
Post by Steve Longerbeam
I've done some work on Philipp's June 12 patchset, converting
the CSI driver to a CSI subdev entity, and fixing some issues here
and there. This June 12 patchset doesn't appear to be a fully working
driver, Phillip correct me if I am wrong. I can post this work as it
exists, it is incomplete but compiles.
Dos it compile against a 3.17-rc3 kernel :) ?
Post by Steve Longerbeam
I've also worked out what I think is a workable video pipeline graph for i.MX,
suitable for defining the entities, pads, and links. Unfortunately I haven't
been able to spend as much time as I'd like on it.
This is very interesting, do you have written this somewhere ?
Post by Steve Longerbeam
The complete driver I posted to the list does have some minor issues
mostly suggested by Hans Verkuil (switch to new selection API instead
of cropping API for example). It is a full featured driver but it does not
implement the media device framework, i.e. user does not have direct
control of the video pipeline, rather the driver chooses the pipeline based
on the traditional inputs from user (video format and controls).
If there is interest I can submit another version of the traditional driver
to resolve the issues. But media device is a major rework, so I don't
know whether it would make sense to start from the traditional driver
and then implement media device on top later, since media device
is almost a complete rewrite.
I, at least, am interested by this driver, even in its "traditionnal"
form :). If you don't want to submit it directly because this is not
using media controller, this is ok, you can provide me a git repo in
order to get it, or send a patchset.
Is it possible to create a staging driver? Even if there are bits missing,
having the code in the kernel as a staging driver would help a lot.
Hi Hans, that's a good idea. I can post it as a staging driver.

The capture driver does require more support in the i.MX IPU driver which is
not yet merged or proposed. Phillip has forwarded most of them to drm-next,
but there are a few more required. I need to post those patches to drm-next
as a first step.

I can start working on converting the driver to staging and addressing
the earlier issues, but posting it to media-tree will need to wait until the IPU
patches are merged, unless I include the IPU patches along with the capture
driver patchset.

Steve
Steve Longerbeam
2014-09-09 16:12:21 UTC
Permalink
Hi Jean-Michel,
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
Hi Jean-Michel, Phillip,
Hi Steve,
Post by Steve Longerbeam
I've done some work on Philipp's June 12 patchset, converting
the CSI driver to a CSI subdev entity, and fixing some issues here
and there. This June 12 patchset doesn't appear to be a fully working
driver, Phillip correct me if I am wrong. I can post this work as it
exists, it is incomplete but compiles.
Dos it compile against a 3.17-rc3 kernel :) ?
No, not anymore, the original posted driver was against 3.16 IIRC.
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
I've also worked out what I think is a workable video pipeline graph for i.MX,
suitable for defining the entities, pads, and links. Unfortunately I haven't
been able to spend as much time as I'd like on it.
This is very interesting, do you have written this somewhere ?
Yes, I'll try to find some time to create a pdf image.
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
The complete driver I posted to the list does have some minor issues
mostly suggested by Hans Verkuil (switch to new selection API instead
of cropping API for example). It is a full featured driver but it does not
implement the media device framework, i.e. user does not have direct
control of the video pipeline, rather the driver chooses the pipeline based
on the traditional inputs from user (video format and controls).
If there is interest I can submit another version of the traditional driver
to resolve the issues. But media device is a major rework, so I don't
know whether it would make sense to start from the traditional driver
and then implement media device on top later, since media device
is almost a complete rewrite.
I, at least, am interested by this driver, even in its "traditionnal"
form :). If you don't want to submit it directly because this is not
using media controller, this is ok, you can provide me a git repo in
order to get it, or send a patchset.
I think I'll follow Hans' proposal and submit it again to media-tree as
a staging driver.

Steve
Philipp Zabel
2014-09-09 17:40:21 UTC
Permalink
Hi Steve,
Post by Steve Longerbeam
Hi Jean-Michel,
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
Hi Jean-Michel, Phillip,
Hi Steve,
Post by Steve Longerbeam
I've done some work on Philipp's June 12 patchset, converting
the CSI driver to a CSI subdev entity, and fixing some issues here
and there. This June 12 patchset doesn't appear to be a fully working
driver, Phillip correct me if I am wrong. I can post this work as it
exists, it is incomplete but compiles.
Dos it compile against a 3.17-rc3 kernel :) ?
No, not anymore, the original posted driver was against 3.16 IIRC.
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
I've also worked out what I think is a workable video pipeline graph for i.MX,
suitable for defining the entities, pads, and links. Unfortunately I haven't
been able to spend as much time as I'd like on it.
This is very interesting, do you have written this somewhere ?
Yes, I'll try to find some time to create a pdf image.
I'd be very interested in this, too. I have in the meantime started to
implement everything that has a source or destination selector in the
Frame Synchronization Unit (FSU) as media entity. I wonder which of
these parts should reasonably be unified into a single entity:

CSI0
CSI1
SMFC0
SMFC1
SMFC2
SMFC3
IC preprocessor (input to VF and ENC, if I understood correctly)
IC viewfinder task (scaling, csc)
IC encoding task
IC post processing task
IRT viewfinder task (rotation)
IRT encoding task
IRT post processing task
VDIC (deinterlacing, combining)
(and probably some entry for DP/DC/DMFC for the direct
viewfinder path)

I suppose the SMFC channels need to be separate because they can belong
to different pipelines (and each entity can only belong to one).
The three IC task entities could probably be combined with their
corresponding IRT task entity somehow, but that would be at the cost of
not being able to tell the kernel whether to rotate before or after
scaling, which might be useful when handling chroma subsampled formats.

I have put my current state up here:

git://git.pengutronix.de/git/pza/linux.git test/nitrogen6x-ipu-media

So far I've captured video through the SMFC on a Nitrogen6X board with
OV5652 parallel camera with this.
Post by Steve Longerbeam
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
The complete driver I posted to the list does have some minor issues
mostly suggested by Hans Verkuil (switch to new selection API instead
of cropping API for example). It is a full featured driver but it does not
implement the media device framework, i.e. user does not have direct
control of the video pipeline, rather the driver chooses the pipeline based
on the traditional inputs from user (video format and controls).
If there is interest I can submit another version of the traditional driver
to resolve the issues. But media device is a major rework, so I don't
know whether it would make sense to start from the traditional driver
and then implement media device on top later, since media device
is almost a complete rewrite.
I, at least, am interested by this driver, even in its "traditionnal"
form :). If you don't want to submit it directly because this is not
using media controller, this is ok, you can provide me a git repo in
order to get it, or send a patchset.
I think I'll follow Hans' proposal and submit it again to media-tree as
a staging driver.
I'm not too fond of adding a staging driver that we know will have to be
replaced. Maybe we could work together to get a media entity based
version up to speed?

regards
Philipp
Steve Longerbeam
2014-09-11 01:17:03 UTC
Permalink
This post might be inappropriate. Click to display it.
Philipp Zabel
2014-09-11 13:26:59 UTC
Permalink
Hi Steve,

Am Mittwoch, den 10.09.2014, 18:17 -0700 schrieb Steve Longerbeam:
[...]
[...]
Post by Steve Longerbeam
Post by Philipp Zabel
I have in the meantime started to
implement everything that has a source or destination selector in the
Frame Synchronization Unit (FSU) as media entity. I wonder which of
[...]
Post by Steve Longerbeam
Post by Philipp Zabel
SMFC0
SMFC1
SMFC2
SMFC3
I don't really see the need for an SMFC entity. The SMFC control can
be integrated into the CSI subdev.
Granted, this is currently is a theoretical question, but could we
handle a single MIPI link that carries two or more virtual channels with
different MIPI IDs this way?
Post by Steve Longerbeam
Post by Philipp Zabel
IC preprocessor (input to VF and ENC, if I understood correctly)
IC viewfinder task (scaling, csc)
IC encoding task
IC post processing task
I see either three different IC subdev entities (IC prpenc, IC prpvf,
IC pp), or a single IC entity with three sink pads for each IC task.
The former could work, the latter won't allow to have pre and post
processing on separate pipelines.
Post by Steve Longerbeam
Post by Philipp Zabel
IRT viewfinder task (rotation)
IRT encoding task
IRT post processing task
well, the IRT is really just a submodule enable bit, I see no need
for an IRT subdev, in fact IRT has already been folded into ipu-ic.c
as a simple submodule enable/disable. Rotation support can be
implemented as part of the IC entities.
My current understanding is that the IRT is strictly a mem2mem device
using its own DMA channels, which can be channel-linked to the IC (and
other blocks) in various ways.
Post by Steve Longerbeam
Post by Philipp Zabel
VDIC (deinterlacing, combining)
I am thinking VDIC support can be part of the IC prpvf entity (well,
combining is not really on my radar, I haven't given that much thought).
Post by Philipp Zabel
(and probably some entry for DP/DC/DMFC for the direct
viewfinder path)
Ugh, I've been ignoring that path as well. Freescale's BSP releases
and sample code from their SDK's have no example code for the
direct-to-DP/DC/DMFC camera viewfinder path, so given the quality
of the imx TRM, this could be a challenge to implement. Have you
gotten this path to work?
Not yet, no.
Post by Steve Longerbeam
Post by Philipp Zabel
I suppose the SMFC channels need to be separate because they can belong
to different pipelines (and each entity can only belong to one).
I see the chosen SMFC channel as an internal decision by the
CSI subdev.
Can we handle multiple outputs from a single CSI this way?
Post by Steve Longerbeam
Post by Philipp Zabel
The three IC task entities could probably be combined with their
corresponding IRT task entity somehow, but that would be at the cost of
not being able to tell the kernel whether to rotate before or after
scaling, which might be useful when handling chroma subsampled formats.
I'm fairly sure IC rotation must always occur _after_ scaling. I.e.
raw frames are first passed through IC prpenc/prpvf/pp for scaling/CSC,
then EOF completion of that task is hardware linked to IRT.
There could be good reasons to do the rotation on the input side, for
example when upscaling or when the output is 4:2:2 subsampled. At least
the FSU registers suggest that channel linking the rotator before the IC
is possible. This probably won't be useful for the capture path in most
cases, but it might be for rotated playback.
Post by Steve Longerbeam
Post by Philipp Zabel
git://git.pengutronix.de/git/pza/linux.git test/nitrogen6x-ipu-media
So far I've captured video through the SMFC on a Nitrogen6X board with
OV5652 parallel camera with this.
Thanks Phillip, I'll take a look! Sounds like a good place to start.
I assume this is with the video mux entity and CSI driver? I.e. no
IC entity support yet for scaling, CSC, or rotation.
Yes, exactly.

regards
Philipp
Jean-Michel Hautbois
2014-09-15 14:13:39 UTC
Permalink
Post by Jean-Michel Hautbois
Hi Steve,
[...]
[...]
Post by Steve Longerbeam
Post by Philipp Zabel
I have in the meantime started to
implement everything that has a source or destination selector in the
Frame Synchronization Unit (FSU) as media entity. I wonder which of
[...]
Post by Steve Longerbeam
Post by Philipp Zabel
SMFC0
SMFC1
SMFC2
SMFC3
I don't really see the need for an SMFC entity. The SMFC control can
be integrated into the CSI subdev.
Granted, this is currently is a theoretical question, but could we
handle a single MIPI link that carries two or more virtual channels with
different MIPI IDs this way?
Post by Steve Longerbeam
Post by Philipp Zabel
IC preprocessor (input to VF and ENC, if I understood correctly)
IC viewfinder task (scaling, csc)
IC encoding task
IC post processing task
I see either three different IC subdev entities (IC prpenc, IC prpvf,
IC pp), or a single IC entity with three sink pads for each IC task.
The former could work, the latter won't allow to have pre and post
processing on separate pipelines.
Post by Steve Longerbeam
Post by Philipp Zabel
IRT viewfinder task (rotation)
IRT encoding task
IRT post processing task
well, the IRT is really just a submodule enable bit, I see no need
for an IRT subdev, in fact IRT has already been folded into ipu-ic.c
as a simple submodule enable/disable. Rotation support can be
implemented as part of the IC entities.
My current understanding is that the IRT is strictly a mem2mem device
using its own DMA channels, which can be channel-linked to the IC (and
other blocks) in various ways.
Post by Steve Longerbeam
Post by Philipp Zabel
VDIC (deinterlacing, combining)
I am thinking VDIC support can be part of the IC prpvf entity (well,
combining is not really on my radar, I haven't given that much thought).
Post by Philipp Zabel
(and probably some entry for DP/DC/DMFC for the direct
viewfinder path)
Ugh, I've been ignoring that path as well. Freescale's BSP releases
and sample code from their SDK's have no example code for the
direct-to-DP/DC/DMFC camera viewfinder path, so given the quality
of the imx TRM, this could be a challenge to implement. Have you
gotten this path to work?
Not yet, no.
Post by Steve Longerbeam
Post by Philipp Zabel
I suppose the SMFC channels need to be separate because they can belong
to different pipelines (and each entity can only belong to one).
I see the chosen SMFC channel as an internal decision by the
CSI subdev.
Can we handle multiple outputs from a single CSI this way?
Post by Steve Longerbeam
Post by Philipp Zabel
The three IC task entities could probably be combined with their
corresponding IRT task entity somehow, but that would be at the cost of
not being able to tell the kernel whether to rotate before or after
scaling, which might be useful when handling chroma subsampled formats.
I'm fairly sure IC rotation must always occur _after_ scaling. I.e.
raw frames are first passed through IC prpenc/prpvf/pp for scaling/CSC,
then EOF completion of that task is hardware linked to IRT.
There could be good reasons to do the rotation on the input side, for
example when upscaling or when the output is 4:2:2 subsampled. At least
the FSU registers suggest that channel linking the rotator before the IC
is possible. This probably won't be useful for the capture path in most
cases, but it might be for rotated playback.
Post by Steve Longerbeam
Post by Philipp Zabel
git://git.pengutronix.de/git/pza/linux.git test/nitrogen6x-ipu-media
So far I've captured video through the SMFC on a Nitrogen6X board with
OV5652 parallel camera with this.
Thanks Phillip, I'll take a look! Sounds like a good place to start.
I assume this is with the video mux entity and CSI driver? I.e. no
IC entity support yet for scaling, CSC, or rotation.
Yes, exactly.
I have tried both of your branches (Steve and Philip) and they both
are interesting.
I easily added adv76xx devices to Steve's work, but there is no media
controller support, as you previously said.
I cannot get adv7611 working on Philip's branch. I tried to do the
same as your "add OV5642 parallel camera" commit, but I don't see a
link between csi and adv even though I asked for it in DT (I removed
not useful stuff in the following paste) :

&i2c2 {
hdmiin1: ***@4c {
port {
hdmi0_out: ***@1 {
reg = <1>;
remote_endpoint = <&csi0_from_adv7611>;
};
};
};

&csi0 {
csi0_from_adv7611: endpoint {
remote_endpoint = <&hdmi0_out>;
};
};

Is there something specific that needs to be done in order to get the
link on boot ?

Even if this is still WIP, this is great, it helps me a lot :) !
Thanks,
JM
Steve Longerbeam
2014-09-09 16:28:49 UTC
Permalink
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
The complete driver I posted to the list does have some minor issues
mostly suggested by Hans Verkuil (switch to new selection API instead
of cropping API for example). It is a full featured driver but it does not
implement the media device framework, i.e. user does not have direct
control of the video pipeline, rather the driver chooses the pipeline based
on the traditional inputs from user (video format and controls).
If there is interest I can submit another version of the traditional driver
to resolve the issues. But media device is a major rework, so I don't
know whether it would make sense to start from the traditional driver
and then implement media device on top later, since media device
is almost a complete rewrite.
I, at least, am interested by this driver, even in its "traditionnal"
form :). If you don't want to submit it directly because this is not
using media controller, this is ok, you can provide me a git repo in
order to get it, or send a patchset.
Hi Jean-Michel, I forgot to mention I will be working on the staging
capture driver in a copy of the media-tree on github at:

***@github.com:slongerbeam/mediatree.git

Steve
Jean-Michel Hautbois
2014-09-10 16:08:15 UTC
Permalink
Post by Steve Longerbeam
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
The complete driver I posted to the list does have some minor issues
mostly suggested by Hans Verkuil (switch to new selection API instead
of cropping API for example). It is a full featured driver but it does not
implement the media device framework, i.e. user does not have direct
control of the video pipeline, rather the driver chooses the pipeline based
on the traditional inputs from user (video format and controls).
If there is interest I can submit another version of the traditional driver
to resolve the issues. But media device is a major rework, so I don't
know whether it would make sense to start from the traditional driver
and then implement media device on top later, since media device
is almost a complete rewrite.
I, at least, am interested by this driver, even in its "traditionnal"
form :). If you don't want to submit it directly because this is not
using media controller, this is ok, you can provide me a git repo in
order to get it, or send a patchset.
Hi Jean-Michel, I forgot to mention I will be working on the staging
I took your mx6-camera-staging branch and merger it, but compile fails :
drivers/staging/media/imx6/capture/mx6-vdic.c:815:2: error: implicit
declaration of function 'ipu_mbus_code_to_fourcc'

Maybe isn't it ready yet ? I always want to go faster than music... :)

Thanks,
JM
Steve Longerbeam
2014-09-10 16:25:58 UTC
Permalink
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
The complete driver I posted to the list does have some minor issues
mostly suggested by Hans Verkuil (switch to new selection API instead
of cropping API for example). It is a full featured driver but it does not
implement the media device framework, i.e. user does not have direct
control of the video pipeline, rather the driver chooses the pipeline based
on the traditional inputs from user (video format and controls).
If there is interest I can submit another version of the traditional driver
to resolve the issues. But media device is a major rework, so I don't
know whether it would make sense to start from the traditional driver
and then implement media device on top later, since media device
is almost a complete rewrite.
I, at least, am interested by this driver, even in its "traditionnal"
form :). If you don't want to submit it directly because this is not
using media controller, this is ok, you can provide me a git repo in
order to get it, or send a patchset.
Hi Jean-Michel, I forgot to mention I will be working on the staging
drivers/staging/media/imx6/capture/mx6-vdic.c:815:2: error: implicit
declaration of function 'ipu_mbus_code_to_fourcc'
Maybe isn't it ready yet ? I always want to go faster than music... :)
Hi JM, yes I'm still working on it, I'll let you know when I think it's
in a good-enough state.

Steve
Steve Longerbeam
2014-09-11 00:37:45 UTC
Permalink
Post by Steve Longerbeam
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
The complete driver I posted to the list does have some minor issues
mostly suggested by Hans Verkuil (switch to new selection API instead
of cropping API for example). It is a full featured driver but it does not
implement the media device framework, i.e. user does not have direct
control of the video pipeline, rather the driver chooses the pipeline based
on the traditional inputs from user (video format and controls).
If there is interest I can submit another version of the traditional driver
to resolve the issues. But media device is a major rework, so I don't
know whether it would make sense to start from the traditional driver
and then implement media device on top later, since media device
is almost a complete rewrite.
I, at least, am interested by this driver, even in its "traditionnal"
form :). If you don't want to submit it directly because this is not
using media controller, this is ok, you can provide me a git repo in
order to get it, or send a patchset.
Hi Jean-Michel, I forgot to mention I will be working on the staging
drivers/staging/media/imx6/capture/mx6-vdic.c:815:2: error: implicit
declaration of function 'ipu_mbus_code_to_fourcc'
Maybe isn't it ready yet ? I always want to go faster than music... :)
Hi JM, yes I'm still working on it, I'll let you know when I think it's
in a good-enough state.
Hi JM, staging capture driver now compiles at
***@github.com:slongerbeam/mediatree.git, mx6-camera-staging
branch.

It is lightly tested on sabrelite quad (with parallel ov5642), sabreauto quad
(with adv7180 and NTSC/PAL sources), and sabresd quad (with MIPI CSI-2
ov5640).

I have not yet made most of the suggested changes from my version 1 posting
a couple months ago, so for the most part it is in the same state. Still in the process
of making those suggested changes.

Steve
Jean-Michel Hautbois
2014-10-02 14:50:39 UTC
Permalink
Hi Steve,
Post by Steve Longerbeam
The complete driver I posted to the list does have some minor issues
mostly suggested by Hans Verkuil (switch to new selection API instead
of cropping API for example). It is a full featured driver but it does not
implement the media device framework, i.e. user does not have direct
control of the video pipeline, rather the driver chooses the pipeline based
on the traditional inputs from user (video format and controls).
Here is my first step toward MC support from your work :
https://github.com/Vodalys/linux-2.6-imx/commit/8f0318f53c48a9638a1963b395bc79fbd7ba4c07

This is a WIP, so some parts of code are commented out awaiting a
nicer solution.
I also keep using your eplist array for the moment, and open will
obviously fail when trying to power sensor.
But what I wanted was a complete MC support with parsing links from DT
and I used Laurent's work intensively :).
Post by Steve Longerbeam
I've also worked out what I think is a workable video pipeline graph for i.MX,
suitable for defining the entities, pads, and links. Unfortunately I haven't
been able to spend as much time as I'd like on it.
Did you find some time to write the pdf you mentioned ?

Thanks for your work again,
JM
Carlos Sanmartín Bustos
2014-10-03 10:25:35 UTC
Permalink
Sorry for the resend, I forget send in plain text.
Hi all,

I'm interested in this driver with MC support too. I join the
conversation and if I have time can try to develop some functionality.

Only one question:

2014-10-02 16:50 GMT+02:00 Jean-Michel Hautbois
Post by Jean-Michel Hautbois
Hi Steve,
Post by Steve Longerbeam
The complete driver I posted to the list does have some minor issues
mostly suggested by Hans Verkuil (switch to new selection API instead
of cropping API for example). It is a full featured driver but it does not
implement the media device framework, i.e. user does not have direct
control of the video pipeline, rather the driver chooses the pipeline based
on the traditional inputs from user (video format and controls).
https://github.com/Vodalys/linux-2.6-imx/commit/8f0318f53c48a9638a1963b395bc79fbd7ba4c07
This is a WIP, so some parts of code are commented out awaiting a
nicer solution.
I also keep using your eplist array for the moment, and open will
obviously fail when trying to power sensor.
But what I wanted was a complete MC support with parsing links from DT
and I used Laurent's work intensively :).
You are forking the Freescale linux-2.6-imx repository if I understood
well. Why not fork the linux-media repository? It's closer to mainline
kernel I think it's better.

Regards,
Carlos

2014-10-02 16:50 GMT+02:00 Jean-Michel Hautbois
Post by Jean-Michel Hautbois
Hi Steve,
Post by Steve Longerbeam
The complete driver I posted to the list does have some minor issues
mostly suggested by Hans Verkuil (switch to new selection API instead
of cropping API for example). It is a full featured driver but it does not
implement the media device framework, i.e. user does not have direct
control of the video pipeline, rather the driver chooses the pipeline based
on the traditional inputs from user (video format and controls).
https://github.com/Vodalys/linux-2.6-imx/commit/8f0318f53c48a9638a1963b395bc79fbd7ba4c07
This is a WIP, so some parts of code are commented out awaiting a
nicer solution.
I also keep using your eplist array for the moment, and open will
obviously fail when trying to power sensor.
But what I wanted was a complete MC support with parsing links from DT
and I used Laurent's work intensively :).
Post by Steve Longerbeam
I've also worked out what I think is a workable video pipeline graph for i.MX,
suitable for defining the entities, pads, and links. Unfortunately I haven't
been able to spend as much time as I'd like on it.
Did you find some time to write the pdf you mentioned ?
Thanks for your work again,
JM
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Jean-Michel Hautbois
2014-10-03 10:27:29 UTC
Permalink
Post by Carlos Sanmartín Bustos
Hi all,
I'm interested in this driver with MC support too. I join the convers=
ation
Post by Carlos Sanmartín Bustos
and if I have time can try to develop some functionality.
2014-10-02 16:50 GMT+02:00 Jean-Michel Hautbois
Post by Jean-Michel Hautbois
Hi Steve,
2014-08-27 16:23 GMT+02:00 Steve Longerbeam
The complete driver I posted to the list does have some minor is=
sues
Post by Carlos Sanmartín Bustos
Post by Jean-Michel Hautbois
mostly suggested by Hans Verkuil (switch to new selection API in=
stead
Post by Carlos Sanmartín Bustos
Post by Jean-Michel Hautbois
of cropping API for example). It is a full featured driver but i=
t does
Post by Carlos Sanmartín Bustos
Post by Jean-Michel Hautbois
not
implement the media device framework, i.e. user does not have di=
rect
Post by Carlos Sanmartín Bustos
Post by Jean-Michel Hautbois
control of the video pipeline, rather the driver chooses the pip=
eline
Post by Carlos Sanmartín Bustos
Post by Jean-Michel Hautbois
based
on the traditional inputs from user (video format and controls).
https://github.com/Vodalys/linux-2.6-imx/commit/8f0318f53c48a9638a19=
63b395bc79fbd7ba4c07
Post by Carlos Sanmartín Bustos
Post by Jean-Michel Hautbois
This is a WIP, so some parts of code are commented out awaiting a
nicer solution.
I also keep using your eplist array for the moment, and open will
obviously fail when trying to power sensor.
But what I wanted was a complete MC support with parsing links from =
DT
Post by Carlos Sanmartín Bustos
Post by Jean-Michel Hautbois
and I used Laurent's work intensively :).
You are forking the Freescale linux-2.6-imx repository if I understoo=
d well.
Post by Carlos Sanmartín Bustos
Why not fork the linux-media repository? It's closer to mainline kern=
el I
Post by Carlos Sanmartín Bustos
think it's better.
Well, this is kind of a mess in this github :).
But the branch indicated is a clone of vanilla, not on linux-2.6-imx
repository. I should have done a new repository, sorry for the
confusion.

JM
Steve Longerbeam
2014-10-06 01:03:03 UTC
Permalink
Post by Jean-Michel Hautbois
Hi Steve,
Post by Steve Longerbeam
The complete driver I posted to the list does have some minor issues
mostly suggested by Hans Verkuil (switch to new selection API instead
of cropping API for example). It is a full featured driver but it does not
implement the media device framework, i.e. user does not have direct
control of the video pipeline, rather the driver chooses the pipeline based
on the traditional inputs from user (video format and controls).
https://github.com/Vodalys/linux-2.6-imx/commit/8f0318f53c48a9638a1963b395bc79fbd7ba4c07
This is a WIP, so some parts of code are commented out awaiting a
nicer solution.
I also keep using your eplist array for the moment, and open will
obviously fail when trying to power sensor.
But what I wanted was a complete MC support with parsing links from DT
and I used Laurent's work intensively :).
Hi Jean-Michel,

Ok thanks for the work, I will try to find time to study it.
Post by Jean-Michel Hautbois
Post by Steve Longerbeam
I've also worked out what I think is a workable video pipeline graph for i.MX,
suitable for defining the entities, pads, and links. Unfortunately I haven't
been able to spend as much time as I'd like on it.
Did you find some time to write the pdf you mentioned ?
Finally did. I will send it directly to you, as I'm sure it will get
stripped here.

Steve

Loading...