The video4linux subsystem of the kernel which deals with video capture, video output and hardware video codecs has a very large API with many ioctls, settings, options and capabilities. And most hardware will only use a fraction of that. This makes it hard to test whether your driver implements everything it should and it makes it hard to test if your application supports all hardware variants.
Providing tools that allow you gain confidence about the quality of the code you are writing, whether it is a driver or an application, would be very helpful indeed. As co-maintainer of the subsystem and as part of my job trying to convince the industry to switch to the V4L2 API instead of (Oh no! Not again!) rolling your own API I thought this was a worthy cause to spend time on.
I started writing a utility called
v4l2-compliance to test drivers over 6 years ago, but for a long time it only tested a fraction of the V4L2 API. The test coverage slowly increased over the years but it wasn’t until February this year that it became a really powerful tool when support for testing video streaming was added. Today it has test coverage of around 90% of the API and new V4L2 drivers must pass the
v4l2-compliance tests before they are allowed in the kernel.
One important missing piece in the compliance utility is testing for the various cropping, composing and scaling combinations. The main reason being that it wasn’t always clear in the API what the interaction should be between the various actions. E.g. changing a crop rectangle might require a change to the compose rectangle as well. So should that be allowed or should an error be returned instead? (Answer: yes, that’s allowed). I hope to add support for testing this some time this year.
It would be nice if this could be easily tested with an application and a driver that supports all the various combinations. But no such driver exists, and that brings me to the second part of this post: how do you test an application against the bewildering array of hardware? All too often application developers only test their application against the hardware they own, and so it is likely it will fail miserably when using it with hardware that implements a different subset of the V4L2 API.
The answer to this question is that a virtual V4L2 driver is needed that implements as much of the V4L2 API as is possible and that can be configured in various ways to accurately model real hardware. Today there is a virtual video driver in the kernel called
vivi, but unfortunately that driver doesn’t act at all as real hardware does. And it only supports simple video capture which is just a small subset of the whole API.
In order to resolve this situation I wrote a new driver called vivid: Virtual Test Driver. This driver covers most of the V4L2 API and is ideal for testing your application. Writing this driver was very useful since it forced me to think about some of the dark and dusty corners of the V4L2 API, and some of those corners needed a big broom to clean up. I found a variety of bugs in the V4L2 core and the API documentation just because this driver exercised parts of the API that are rarely if ever used.
I also realized that a driver like this is ideal to emulate hardware that is not yet available and can be used to prototype your upcoming product in the absence of the actual hardware. It’s a logical consequence of the requirement that in order for the virtual video driver to be really useful it has to accurately model hardware.
It also had an immediate beneficial effect on the two ‘golden reference’ utilities that control V4L2 drivers: the command line
v4l2-ctl utility and the GUI equivalent
qv4l2. After all, in order to test whether the
vivid driver works you need applications to test the driver. As a result both utilities improved as more features were added to the driver, which all needed to be tested by those applications. So the driver has already fulfilled its promised to help test and improve applications.
All utilities mentioned in this article are part of the v4l-utils git repository.
If you would like to know more about V4L2 driver and application testing, then attend my presentation on this topic during the upcoming LinuxCon North America in Chicago!
Tags: Linux Kernel, LinuxCon North America 2014, Testing, video4linux
During this year’s Embedded Linux Conference in San Jose I will be presenting An Introduction to the Video4Linux Framework. As a long-term kernel contributor (since 2003), author of over 2000 kernel patches and co-maintainer of the video4linux kernel subsystem I have been working on improving that subsystem so it can handle the highly complex video capture and codec pipelines that are available on a modern SoC.
As part of that work a lot of attention was given (and still is!) to the supporting framework provided by the video4linux subsystem to simplify the driver code. Video is complex and driver developers should be able to concentrate on configuring the hardware without the V4L API getting in the way.
This presentation will demonstrate how to make a fairly simple video capture driver using the latest framework features. Using that framework greatly reduces the amount of work required to write a driver compared to what would have been needed just a few years ago.
You are all welcome to attend this presentation! And for those who are interested: on Friday there will be a video4linux mini-summit, see the announcement here.
Finally, if you have any questions about the video4linux kernel subsystem, please don’t hesitate to contact me at any time. I am always happy to answer them as best as I can.
Tags: Embedded Linux Conference, Linux Kernel, video4linux, video4linux framework
Now that OpenDaylight has arrived, it’s time to explain why I made the Open Source choices eventually embraced by its Founders and the community at large. One doesn’t often see such leaders as Cisco, IBM, Intel, HP, Juniper, RedHat, VMWare, NEC, Microsoft and others agree, share and collaborate on such key technologies, let alone the latter engaging in a Linux Foundation based community (some thought hell will freeze over before that would ever happen, though it got pretty cold at times last Spring).
For those of you not familiar with OpenDaylight (see “Meet Me On The Equinox”, not a homage to Death Cab for Cutie or my Transylvanian homeland), IBM and Cisco have actually started this with an amazing set of partners, nearly that ephemeral Equinox this year (~11am, March 20th) though we couldn’t quite brag about it until all our partners saw the daylight, which by now, we’re hoping everyone does. It was hard not to talk about all this as we saw those half baked, speculative stories before the Equinox – amazing how information flew, distorted as it were, but it did; I wish source code would be that “rapid”, we’d all be so much better for it…
The Open Source model for OpenDaylight is simple, it has only two parts: the community is hosted in the Linux Foundation and the license is Eclipse. The details are neatly captured in a white paper we wrote and published in the Linux Foundation. Dan Frye, my friend and fellow counterpart at IBM and I came up with the main points after two short meetings. It would have been one, but when you work for such giants as our parent companies and soon to be OpenDaylight partners, one has to spend a little more time getting everyone to see the daylight. It boils down to two things, which I am convinced are the quintessential elements of any successful open source project.
1) Community. Why? Because it trumps everything: code, money and everything else. A poor community with great code equals failure (plenty of examples of that). A great community with poor (or any) code equals success (plenty of examples of that too). Why? Because open source equals collaboration, of the highest kind: I share with you, and you with me, whatever I have, I contribute my time, my energy, my intellectual property, my reputation, etc.. And ultimately it becomes “ours”. And the next generation’s. Open Source is not a technology; it’s a development model. With more than 10 million open source developers world wide, it happens to be based on collaboration on a scale and diversity that humanity has never experienced before. Just think about what made this possible and the role some of the OpenDaylight partners have already played in it since the dawn of the Internet. Dan Frye and I agreed that the Linux Kernel community is the best in the world and so we picked the closest thing to it to model and support ours, the Linux Foundation.
2) Fragmentation, or anti-fragmentation, actually. Why? The biggest challenge of any open source project is how to avoid fragmentation (the opposite of collaboration). Just ask Andy Rubin and the Android guys what they fear the most. Just ask any open source project’s contributors, copyright holders, or high priests, how much they appreciate an open source parasite that won’t give back. Though we would have liked to go deeper, we settled on Eclipse, largely because of the actual language and technology we dealt with in the OpenDaylight Controller: most, if not all the initial code is Java, and though some are worried about that, I’m sure Jim Gosling is proud (btw, I’m not sure the Controller has to stay that way, I actually agree with Amin Vahdat), but we had to start somewhere. Plus having a more friendly language NB (northbound, as in the applications run on top of the Controller) is such a cool thing, we think that the #1 open source (Eclipse) and the #1 commercial (Microsoft) IDE’s are going to be very good to it, so why not? There are more reasons that pointed in the Eclipse direction, and other reasons for such wonderful alternatives (as APL or MPL, perhaps the subject of another post, some day). But when it comes to understanding the virtues of them all, no one understands them better than the amazing founders of these license models, most of them from IBM, of course (I wish they did that when I was there).
What happened between the Equinox and Solstice is a fascinating saga within the OpenDaylight community which I think played its course in the spirit of total and complete openness, inclusion, diversity, respect of the individual and the community, and most of all, that code rules – we do believe in running code and community consensus. I tip my hat to all my fellow colleagues that learned these two things along the way, the enormous talent at the Eclipse and Linux Foundation that helped us launch, and even the analysts who tried (and did incredibly well at times) to speculate the secret reasons why these partners came up with the model we did: there is no secret at all, my friends, we’re simply creating a community that is truly open, diverse, inclusive, and never fragmented. Just like a big, happy family. Welcome to OpenDaylight, we hope you’ll stay!
Tags: Android, apache, Cisco, community, controller, Eclipse, Fragmentation, IDE, java, Linux Foundation, Linux Kernel, ONE, onePK, open source, open source model, opendaylight, SDN