Tuesday, November 8, 2011

An HTML5-Windows Azure Dashboard, Part 2: HTML5 Video

As I continue to experiment with HTML5-cloud dashboards, I wanted to move beyond charts and also include media content, especially HTML5 video. More and more, modern interfaces need movement and video is one source of movement. Of course, it should only be there if it’s providing some value and you definitely don’t want to overdo it. In particular you need to think about the impact of video on tablets and phones where download speed and device power may be constrained.

In my second dashboard prototype, I created another fictional company, Contoso Health. In this medical scenario there are plenty of places for video: patient monitoring, MRIs, endoscopy, even reference materials. It is in need of some optimization for tablets and especially phones but it’s off to a good start. It demos well on computers and tablets and works in the latest editions of Chrome, FireFox, IE, and Safari. While there is plenty of work still to be done to make this a real functional dashboard—the content is somewhat canned in this first incarnation-- it does convey a compelling vision for where this can go.

HTML5 Video
This was my first opportunity to play with HTML5 video.

The good news about HTML5 video is we can do video and audio in modern browsers without plug-ins, and thanks to hardware acceleration we have more horsepower on the web client side than ever before. Even the media controls are built into the browsers. Below you can see the browser controls for IE9, Chrome, and FireFox (which only appear when you hover over or touch the video). In addition, Safari Mobile on the iPad will let you take a video full-screen.

The bad news is, video can be complicated at times and there is no single best format: the browser makers have not been able to come to agreement on a common video format they will support. That means you’ll be encoding your video to several formats if you want broad browser coverage.

A good place to see what HTML5 video looks like is at the HTML5 video online tutorial at w3schools.com. Within a <video> tag, you specify one or more video sources. If the browser doesn’t support any of your choices, the content within the <video> tag is displayed, most commonly an apologetic message or fallback content such as an image.

<video width="320" height="240" controls="controls">
  <source src="movie.mp4" type="video/mp4" />
  <source src="movie.ogg" type="video/ogg" />
  Your browser does not support the video tag.
Now about video formats. Microsoft is championing H.264, which several other browsers also support, and FireFox prefers OGG-Theora. From my experimentation, if you provide your video content in these two formats you’ll be largely in good shape across popular browsers and devices. If you find you need more coverage and want to add a third format, you might look into WebM.

My starting point was stock footage MPEGs, and I needed to create H.264 and OGG video content from that. My first thought was to use Expression Encoder but it turns out the version available to me doesn’t have the codecs needed. I ended up using AVS Video Converter for conversion to H.264 video and FoxTab Video Converter for conversion to OGG, both low-priced tools.
My video tags ended up looking like this:
<video id="video1" controls="controls" loop autoplay>
  <source src="heartscan.mpg" type="video/mp4" />
  <source src="heartscan.ogg" type="video/ogg" />
  Your browser does not support the video tag.
In addition to having good format coverage, the correct MIME type (Content-Type header) needs to be served up for each video. Since we’re accessing the video blobs from Windows Azure blob storage, we put them in a container that allows web access and set each blob’s ContentType metadata field. I received the best results with “video/mp4” for H.264 and “video/ogg” for OGG. I didn’t use WebM, but if you’re planning to I believe “video/webm” is the right content type to specify.

Now that I’ve made some progress envisioning the dashboard user experience it’s time to start making it functional. Stay tuned!

1 comment:

jr said...

One of the cool things I like about HTML5 video is the timeupdate event. This makes it easy to track the progress of the video and perform actions at certain points within the video. For example, if you had a promo video for several products you could display specific text for each product at the same time that the product appears in the video. I realize that Flash (and others) can do this also, but I think it is easier in HTML5.