An HTML5-Windows Azure Dashboard, Part 2: HTML5 Video

As I continue to experiment with HTML5-cloud dashboards, I wanted to move beyond charts and also include media content, especially HTML5 video. More and more, modern interfaces need movement and video is one source of movement. Of course, it should only be there if it’s providing some value and you definitely don’t want to overdo it. In particular you need to think about the impact of video on tablets and phones where download speed and device power may be constrained.

In my second dashboard prototype, I created another fictional company, Contoso Health. In this medical scenario there are plenty of places for video: patient monitoring, MRIs, endoscopy, even reference materials. It is in need of some optimization for tablets and especially phones but it’s off to a good start. It demos well on computers and tablets and works in the latest editions of Chrome, FireFox, IE, and Safari. While there is plenty of work still to be done to make this a real functional dashboard—the content is somewhat canned in this first incarnation-- it does convey a compelling vision for where this can go.

HTML5 Video
This was my first opportunity to play with HTML5 video.

The good news about HTML5 video is we can do video and audio in modern browsers without plug-ins, and thanks to hardware acceleration we have more horsepower on the web client side than ever before. Even the media controls are built into the browsers. Below you can see the browser controls for IE9, Chrome, and FireFox (which only appear when you hover over or touch the video). In addition, Safari Mobile on the iPad will let you take a video full-screen.

The bad news is, video can be complicated at times and there is no single best format: the browser makers have not been able to come to agreement on a common video format they will support. That means you’ll be encoding your video to several formats if you want broad browser coverage.

A good place to see what HTML5 video looks like is at the HTML5 video online tutorial at w3schools.com. Within a <video> tag, you specify one or more video sources. If the browser doesn’t support any of your choices, the content within the <video> tag is displayed, most commonly an apologetic message or fallback content such as an image.

<video width="320" height="240" controls="controls">
  <source src="movie.mp4" type="video/mp4" />
  <source src="movie.ogg" type="video/ogg" />
  Your browser does not support the video tag.
</video>

Now about video formats. Microsoft is championing H.264, which several other browsers also support, and FireFox prefers OGG-Theora. From my experimentation, if you provide your video content in these two formats you’ll be largely in good shape across popular browsers and devices. If you find you need more coverage and want to add a third format, you might look into WebM.

My starting point was stock footage MPEGs, and I needed to create H.264 and OGG video content from that. My first thought was to use Expression Encoder but it turns out the version available to me doesn’t have the codecs needed. I ended up using AVS Video Converter for conversion to H.264 video and FoxTab Video Converter for conversion to OGG, both low-priced tools.

My video tags ended up looking like this:

<video id="video1" controls="controls" loop autoplay>
  <source src="heartscan.mpg" type="video/mp4" />
  <source src="heartscan.ogg" type="video/ogg" />
  Your browser does not support the video tag.
</video>

In addition to having good format coverage, the correct MIME type (Content-Type header) needs to be served up for each video. Since we’re accessing the video blobs from Windows Azure blob storage, we put them in a container that allows web access and set each blob’s ContentType metadata field. I received the best results with “video/mp4” for H.264 and “video/ogg” for OGG. I didn’t use WebM, but if you’re planning to I believe “video/webm” is the right content type to specify.

Now that I’ve made some progress envisioning the dashboard user experience it’s time to start making it functional. Stay tuned!

An HTML5-Windows Azure Dashboard, Part 1: Design Decisions

Business dashboards are big, and my aim is to create a stunning one that combines my twin passions for HTML5 and Windows Azure. In this series I’ll be sharing my progress on iterative development of an HTML5 dashboard hosted in Microsoft’s Windows Azure cloud that works on computers, tablets, and phones. Here in Part 1 I’ll share initial thoughts on design and a glimpse of our first prototype, themed for a fictional company named Fabrikam Imports. In subsequent posts I’ll have additional companies set up with a variety of themes and content.

Goals

Here are my goals for the dashboard, which will be realized in steps:

Compelling. I want to provide a compelling user experience with animation and transitions. It will take some refinement to find the balance between an interesting level of movement and being overly flashy or distracting.

Branded. Since the dashboard will most often be used by businesses, it will need to carry their branding. This can be done overtly with a corporate logo and reinforced with a background , such as the bamboo image we’re using for Fabrikam Imports.

Broad Reach. The dashboard needs to be accessible across computers, tablets, and phones and work well whether accessed by mouse or touch. This means fluid layout that accommodates different form factors and orientations, and interactive elements large enough to support touch.

Useful. The dashboard is more than eye candy: it also needs to be a valuable source of aggregated business information. Which means it had better be able to integrate to many kinds of data sources.

Content Versatility. Charts aren’t the only content to be shown. I’d ultimately like to also support tables, maps, media, and other content. In some cases we’ll provide more than one view of the same content.

Data Source Versatility. Supporting the most popular formats and access methods should make it easy to integrate dashboards to many kinds of data sources without a lot of rocket science. I initially plan to support data in XML and JSON format data that can be retrieved from cloud blob storage, web services, or feeds.

Explorable. While I’m not trying to compete with BI offerings, I would like to have more than one dimension to the data. When viewing a chart of data, you should be able to select a value of interest and drill down a level if the data allows that.

Lightweight and Deluxe Editions. For my dashboard, I’m envisioning a lightweight edition and a deluxe edition. I’ll explain the differences later in this post.

Fast Setup. The lightweight version should allow you to set up a decent dashboard in just an hour or two if you have your data available in XML or JSON format. Setup time for the deluxe implementation will depend on how ambitious you are and will typically involve some custom integration work.

The Front End

If you’ve been following this blog, you’ve noticed that many of my recent experiments in HTML5 have been around controls such as a counter, bar chart, and ticker. That’s all been driving toward creation of a dashboard. But creating all the controls I need on my own (and taking them to commercial grade) would take a long time, so the best approach will be a flexible dashboard that can adapt to include controls from a variety of sources. With this approach, I can combine best of breed controls with my own. For charting, I’m currently trying out JSCharts. So far I’ve found JSCharts to be easy to use and it displays very efficiently. I am encountering some display issues however after resizing so the jury is still out on whether I’ll be using it in the long term. Either way, I eventually want to support multiple chart packages.

The current layout has a corporate logo and a ticker at top (which can be paused/resumed with a click or touch). Below this I have three views in mind: list, tiles, and zoom.

List, Tiles, and Zoom Views

List View. In list view, you’ll see a list of content that’s available. You can check an item to control whether it is shown or hidden in other views, and you can select an item to go directly to it in zoom view.

Tiles View. In tiles view, you’ll see all of the content (except items you’re chosen to hide) shown in fluid layout to fit your device. Content buttons allow you to take actions on the data, such as zooming in for a detail view.

Zoom View. In zoom view you’re focused on one item of content and it fills the full window. Here you might be able to see the content in more than one way, such as a table and chart side-by-side. It’s also from here you may be able to explore some data sets by drilling down.

Implementation technologies for the front end are HTML5, CSS3, JavaScript, and JQuery. Content controls will be a mix of my HTML5 controls and public/commercial controls.

The Back End

The back end runs in the Windows Azure cloud. I mentioned earlier I have two editions in mind, a lightweight edition and a deluxe edition.

Lightweight Edition

The lightweight edition is meant to be fast and simple to set up and incredibly cheap to operate. It runs completely out of Windows Azure blob storage and that’s also where the data is kept. This will generally cost just a few bucks a month to operate. Accessing the dashboard URL in a browser transfers the front end code down to run in the web client. Ajax calls issued by the web client pick up data files from blob storage (which are in XML or JSON format) and the dashboard content is populated. To update the data in the dashboard, you push out new data in the form of XML or JSON files to blob storage. This can be done with small ‘update agent’ programs that integrate to your internal systems or manually. This model does have some limitations, namely authentication. The best you can do to secure this is to generate shared access signatures (secretive URLs) for the blob container. On the plus side, this model is fast and cheap, and it's fantastic for throwing together great looking demos at a moment's notice.

Lightweight Dashboard Model

Here’s how access works in the lightweight model:

1.     User visits their dashboard URL, an HTML file hosted in Windows Azure Blob storage.

2.     The dashboard client HTML5/CSS/JavaScript starts running in the user’s browser, which could be on a PC, tablet, or phone.

3.     The web client issues asynchronous Ajax queries to the dashboard web services to retrieve content. The content is in the form of XML or JSON data files stored in blob storage. As the data is returned back to the web client it is rendered as content panels in the dashboard.

Deluxe Edition

For deluxe implementations, you will want domain identity or web identity authentication of your users and deeper integration to your systems. For that we will run on Windows Azure Compute and integrate more formally with internal, partner, or online data sources which might include your CRM, data warehouse, and other sources of business intelligence accessed via feeds or web services. The data can be pulled on-demand from its source systems in real time, or the data can be periodically pushed up to blob storage as in the lightweight model. For this level of dashboard you will want to combine the dashboard core with a consulting engagement to set up the authentication and integration.

Deluxe Dashboard Model

Here’s how access works in the deluxe model:

1.     User visits their dashboard URL, an ASP.NET MVC3 web server hosted in Windows Azure Compute.

2.     User is sent to the designated identity provider to sign in. This could be ADFS, a web identity, or the Windows Azure Access Control Service.

3.     Upon successful sign-on, the user’s browser is redirected back to the dashboard web server and the dashboard client HTMl5/CSS/JavaScript starts running in the user’s browser, which might be on a PC, tablet, or phone.

4.     The web client issues asynchronous Ajax queries to the dashboard web services to retrieve content. That content can come from querying enterprise systems, feeds, or web services directly or by returning recent data snapshots out of blob storage. With either approach, XML or JSON data is returned back to the web client and rendered as content panels in the dashboard.

The first prototype implements some of the above design goals in the lightweight model, all hosted out of Windows Azure blob storage. More to come!

Next Part 2: HTML5 Video