HTML5 Video Now!

Jason Ronallo
Associate Head, Digitial Library Initiatives
North Carolina State University Libraries
@ronallo

Hi. I'm Jason, and I'm going to talk to you about HTML5 Video.

Can you use HTML5 Video now?

Here's Why

Increasingly people are visiting your site on devices which do not support Flash or do not have Flash installed by default. And your video has still got to work. HTML5 Video can fix that.

Simplest Example of HTML5 Video That Could Possibly Work

<video 
  src="videos/getting_a_book.mp4" 
  controls>
</video>

Among the semantic elements HTML5 adds are media elements like the audio element or video element used here. It can have a source attribute pointing to a video source. And then the controls attribute uses the browser's native video controls.

Example Right Here In The Slide Deck

The Web Has Video In It Now

Video is now native to the Web which means all sorts of exciting possibilities. I'm just scratching the surface here today.

Simple Fallback

<video src="videos/getting_a_book.mp4" controls>
  <div>Your browser is crufty and cannot play this 
    video. You might try to 
    <a href="videos/getting_a_book.mp4">download it</a>. 
    You'd be better off getting a modern browser, 
    though, so you can experience video in your Web.</div>
</video>

But what about older browsers that do not know about the HTML5 video element?

Here's one way we can provide a fallback. It just shows a paragraph with some text suggesting download of the video.

And here's the first thing you really need to know about the video tag. If you include a video source on the page then anyone can easily download the video file. In many cases this is just fine and HTML5 Video will be great. In other cases you may need to be more restrictive than HTML5 allows.

Firefox asks:
"What's an MP4?!"

Firefox can't open MP4s

Firefox knows about HTML5 Video, but if you view this video in Firefox, you might see this ugly image. It says "Video format or MIME type not supported."

Can't we all just get along?

Browser support for video formats chart

Problem is that browser vendors could not agree on a single format and codec that every browser would be required to support. Some wanted Ogg Theora and others wanted MP4 with H.264. Here's a simplified chart taken from Wikipedia.

So if you want your video to play across as many browsers as possible then you'll have to encode your video more than once.

This has been a real drag on adoption of HTML5 Video. Unless you're Netflix you'd rather not have to encode your video so many times.

The situation is getting better though. If you want to cover just about all modern browsers you only really need to convert to MP4 and WebM.

Multiple Sources

<video controls>
  <source src="videos/getting_a_book.mp4"/>
  <source src="videos/getting_a_book.webm"/>
  <div>Your browser is crufty...</div>
</video>

So here we remove the src attribute of the video element and instead add source elements within the video element. Browsers can then pick which source they want to play. The browser sees if it can play each video source in the order it is in in the DOM. This makes more browsers happy.

Poster Image

<video 
   poster="videos/getting_a_book.png" 
   controls>
  <source 
     src="videos/getting_a_book.mp4"/>
  <source 
     src="videos/getting_a_book.webm"/>
</video>

Before our video just showed up on the page as a black block before it started playing. This example looks better because it includes a poster image. This is an image that displays to the user before the video is played, even without downloading any of the video. This is just done by including a poster attribute that points to an image that is representative of the video. This gives the user some idea what they're going to see before they click on it. That's nice to visitors.

Save the Time of the Client

<video 
   poster="videos/getting_a_book.png" 
   controls preload="metadata">
  <source 
    src="videos/getting_a_book.mp4" 
    type="video/mp4"/>
  <source 
    src="videos/getting_a_book.webm" 
    type="video/webm"/>
</video>

You can see the type attribute I've added to the sources. These let the client determine which, if any, source of video to download without having to download part of the video. Firefox knows that on most platforms it can't play an MP4 so it doesn't even try to download the video with the video/mp4 MIME type. It then goes to the next source which is WebM and which it can play.

More Exact MIME type

<video 
  poster="videos/getting_a_book.png" 
  controls>
  <source 
    src="videos/getting_a_book.mp4" 
    type='video/mp4; codecs="avc1.4D401E,mp4a.40.2"'/>
  <source 
    src="videos/getting_a_book.webm" 
    type='video/webm; codecs="vp8,vorbis"'/>
</video>

If you get even more exact with MIME type parameters, then the browser has an even better idea whether it can play that format or not. We'll see how else we can play around with mimetypes in a bit.

Attributes

autoplay
controls
height
loop
muted
preload
poster
width

Here are all of the attributes which the video element uses. Autoplay is generally a bad idea and won't work on most mobile devices to save bandwidth. We'll see more about controls shortly. Loop may totally annoy your visitors. Muted allows you to have the video start muted.

JavaScript API

One of the coolest thing about having video be part of the web is that there's now a standard way to work with video with JavaScript.

slide conventions

var video = $('video')[0];

var video = document.getElementsByTagName('video')[0];

So I'm going to show a bit of JavaScript. One convention that I use is that any time I refer to the variable video I've gotten the first and only video element on the page.

Only One Video Per
Video Play Page!!!

No Playlist Pages, Please!

For the most part the advice is to have a dedicated video play page. Only ever play one video on any page and every video should have only one play page. So every video has one and only one URL where it can be viewed. This helps with search engine indexing. I also believe it is probably an accessibility issue. While we'll see how it is totally possible to do video play pages. Just don't.

Properties

buffered
currentTime
duration
ended
paused

played
readyState
seekable
videoHeight
videoWidth
volume

Video has all sorts of properties that you can read and some you can set using JavaScript. So if I start playing this video you can see these values begin to change. You can then check these values in your JavaScript to manipulate them.

Duration

var duration = video.duration;
$('#video-duration').html(duration);

Display the duration on the video thumbnail

I've used the duration property so that we can cache the duration in the database when a new video resource is created in my apps. I can then display it later overlayed on video thumbnails. videoHeight and videoWidth are another useful one so that you know the intrinsic dimensions of the video.

Methods

There are also a lot of JavaScript methods for interrogating your video.

canPlayType MIME type

video.canPlayType('video/mp4');

We can start with just a simple example to show how the browser makes the determination whether it can play a particular MIME type.

canPlayType MIME type with codec

video.canPlayType('video/mp4;codecs="avc1.4D401E, mp4a.40.2"');

And if we're more specific in codec choice, then the browser can have more certainty that it can play that type.

Custom Controls

$('#no-controls-controls .play').on('click', function(){
  $('video#no-controls')[0].play();
});

Here's just how you would attach an event handler to this play button to use the play method to play the video.

Switcheroo

video.src = "http://archive.org/.../TheAdventuresOfJr.TheRaindrop.ogv";
video.play();

You can also change the source of the video like this. Let a new era of Rick Rolling begin!

Events

There are all sorts of events that you can listen for to then manipulate the video.

timeupdate & currentTime

$('#current_time_video').on( 'timeupdate', function(){
  var current_time = $('#current_time_video')[0].currentTime; 
  $('#current_time').html(current_time);
});

Here we listen to the current video for the timeupdate event. Then we ask the video for the current time and display it on the page. So this event triggers several times a second. There's a lot more you could do with this to sync up video with other resources.

Video Analytics

$('video').on('play', function(){
  // send a remote tracking beacon on play
});
$('video').on('pause', function(){
  // track the pause event
});
$('video').on('seeked', function(){
  // Track the seeked event. This gets fired when the 
  // currentTime is set. For instance it gets
  // triggered with a click on the time rail control.
});
$('video').on('ended', function(){
  // triggered when video ends
});

You can use video events to to video analytics. How many times does the video start playing or has ended?

Video Engagement Analytics

You can also do video engagement analytics by looking at the played property. I've got a blog post you can read about how to do this.

Styled with CSS

One of the cool things about having video on the Web is that it can be styled with CSS. Anything you can do with CSS you can do with the video element. Here we can add an opacity to the video element itself which shows up nicely on our gradient background.

Polyfills

http://praegnanz.de/html5video/

While you could create your own controls and style them however you want, you probably don't want to. You'll probably want to use a polyfill. You especially want this so that you can do a Flash fallback. Talk about "Unified Look/API" across HTML5 and Flash players.

tracks

<video>
  ...
  <track label="English subtitles" 
    kind="subtitles" 
    srclang="en" 
    src="videos/getting_a_book.vtt" 
    default="true" 
    id="subtitles"></track>
</video>

To make your media accessible you can include subtitle, description, and captions tracks.

WebVTT

WEBVTT

1
00:00:03.000 --> 00:00:08.616
Once you've used the library's online 
catalog to identify a book you want, 
you have to go get the book.

2
00:00:08.616 --> 00:00:13.308
On the bottom of a book's catalog 
page, there's information about the 
location of the book.

The format of these track files is WebVTT--a new format for cues. It is just a simple text file really and there are increasingly tools to help with creation of them. I've got a forthcoming blog post about some other uses you can make of your WebVTT files.

Captions

So here's our example video with captions. Some browsers are starting to show captions natively, while others still rely on a polyfill.

Server Setup

Apache:

AddType video/mp4 .mp4
AddType video/webm .webm
AddType video/ogg .ogv

You can just use a regular Web server. Nothing special about it. Apache and nginx will work just fine. You have to make sure your server returns the correct MIME types. Read my blog post on other things you'll want to check on your server.

Progressive Download and Range Requests

$ curl -I http://siskel.lib.ncsu.edu/RIS/getting_a_book/getting_a_book.mp4
HTTP/1.1 200 OK
Content-Length: 8343631
Content-Type: video/mp4
Last-Modified: Thu, 20 Dec 2012 19:40:10 GMT
Accept-Ranges: bytes
ETag: "f79b80d2e9decd1:89fd"
Server: Microsoft-IIS/6.0
Access-Control-Allow-Origin: *
X-Powered-By: ASP.NET
Date: Sat, 22 Dec 2012 22:04:23 GMT

HTML5 video can start playing before the whole video is downloaded. How does this work? Browsers can request just a range of bytes and the server can return just those bytes. That combined with optimizing your video to include the metadata at the beginning of the file, and it just takes downloading the first chunk before the video can start to play. You can check right now whether your server will allow for progressive download by checking for the Accept-Ranges header. The value should be bytes.

SEO

Single video play page per video
Video sitemap
Microdata (or RDFa Lite if that your thing) + Schema.org

Future of Media on the Web

DRM

encrypted media extensions screenshot

DRM may be coming. Google, Microsoft, and Netflix have put in a proposal.

Streaming

Since HTML5 Video doesn't yet do real streaming, expect there to be some work on that in the coming couple years.

Popcornjs

Media Fragments

Media Fragments are a specification for addressing a particular fragment of time-based media. So once you have a URI for a fragment you can then do things like annotate it. I think this is going to be very useful for researchers once the tooling is more mature.

Web Audio API

<audio controls preload="auto" src="audio/laser.ogg"></audio>

WebAudio

Audio is cool and all, but the web audio API is Cooler. While the HTML5 Audio element can only play through once. The Web Audio JavaScript API let's you play sounds over top of each other with real time mixing. You can also add all sorts of cool effects. This is a simple demonstration of low pass and high pass filters that reuse the same laser sound. This will be good for games. I have no idea what the library application is, but it is soooo cool. Try using the Z, X, and C keys to laser something!

WebRTC

Peer to peer real time communications is now in Chrome and Firefox.

Project Links

Kinney A Rails Engine for oral history video clips projects. https://github.com/jronallo/kinney
webvtt A WebVTT parser library written in Ruby. https://github.com/jronallo/webvtt
Blog http://ronallo.com/blog/categories/video/

Jason Ronallo

@ronallo
http://ronallo.com

So now I hope the video of this presentation will be made accessible with HTML5 Video.

Slide Deck and Speaker Notes

http://html5-video-presentation.herokuapp.com/

The HTML5 Video Now! slide deck and speaker notes by Jason Ronallo is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
Permissions beyond the scope of this license may be available at http://ronallo.com.