Adding captions and subtitles

1. A basic HTML5 video
2. Adding subtitles
3. WebVTT syntax basics
4. Building a custom menu with JavaScript
5. Captions versus subtitles
6. Cue settings
7. Styling text cues
8. Tags available to cues

A solid basis to start from

• Multiple source tags for cross browser support.
• Include default controls.
• Include a fallback direct link to the video, for non-supporting browsers.

HTML5 video example

		  <video controls preload="metadata">
		      <source src="../video/sintel-short.mp4" type="video/mp4">
		      <source src="../video/sintel-short.webm" type="video/webm">

		      <p>It appears that your browser doesn't support HTML5 video.
		      Here's a <a href="../video/sintel-short.mp4">direct link to
		      the video instead</a>.</p>
		  </video>
		

Code time: tutorial start

<track> elements

		  <track label="English" kind="subtitles" srclang="en"
		    src="vtt/sintel-subtitles-en.vtt" default>
          <track label="Deutsch" kind="subtitles" srclang="de"
            src="vtt/sintel-subtitles-de.vtt">
          <track label="Español" kind="subtitles" srclang="es"
            src="vtt/sintel-subtitles-es.vtt">
		

<track> attributes

• default: Sets the default text track to display (only one element per video can have this), which can be overridden by JavaScript or user preferences.
• kind: Indicates what kind of text track each track is. Here we only have subtitles, but you can also have captions, descriptions and others.
• label: A user-readable title for the text track, which is displayed by the browser when listing available text tracks.
• src: The URL of the text track file.
• srclang: A language code indicating to the browser what language the text tracks are written in.

Code time: tutorial step 2

Cross browser UX not very good for track

• IE11: provides menu for selecting required language. Defaults to "default" track, otherwise shows none by default.
• Firefox: loads first text track it has "default", otherwise it shows nothing. Doesn't provide a menu to switch tracks.
• Chrome and Opera: Load the captions set that matches the browser language, then the "default", then nothing. Doesn't provide a menu to switch tracks.
• Safari: does a good job, similar to IE.

You should therefore...

Implement a custom UI for <track> elements using JavaScript!

WebVTT looks like so

		  WEBVTT
		   
          0
          00:00:00.000 --> 00:00:12.000
          [Test]
           
          1
          00:00:18.700 --> 00:00:21.500
          This blade has a dark past.
		

WebVTT syntax explanation

• File must start with WEBVTT.
• Next include separate text track blocks, each starting with a number. The numbers must go up in order.
• Second line of each block is a timestamp range, specifying what time the text track should shown, and disappear again.
• Time format is hh:mm:ss:msmsms, allowing for very precise times.
• The third line onwards (you can include multiple lines) is the text you want to display.

You can use whatever UI you want

We are using a select menu:

		  <form>
		    <select name="select">
		   
		    </select>
	      </form>
		

Basic WebVTT API information

• All the text tracks can be acced through video.textTracks.
• Each text track has properties to access the features you set earlier in its attributes.
• For example textTrack.label, textTrack.kind, or textTrack.language.
• Show or hide a text track using textTrack.mode = showing or hidden.

Code time: tutorial step 3

Set variables then hide all tracks

		  var video = document.querySelector('video');
          var select = document.querySelector('select');
          function hideTracks() {
            for (var i = 0; i < video.textTracks.length; i++) {
              video.textTracks[i].mode = 'hidden';
            }
          }
           
          hideTracks();
		

Build the select menu

		  var tracksOff = document.createElement('option');
          tracksOff.setAttribute('value','off');
          tracksOff.textContent = 'Tracks off';
          select.appendChild(tracksOff);
		

Build the select menu

          for (var i = 0; i < video.textTracks.length; i++) {
            var curTrack = video.textTracks[i];
            var addTrackOpt = document.createElement('option');
            addTrackOpt.setAttribute('value',curTrack.kind + '-' + curTrack.language);
            addTrackOpt.textContent = curTrack.label + ' ' + curTrack.kind;
            select.appendChild(addTrackOpt);
          }
		

Add an event listener

This runs the trackChange() function whenever the a new select value is selected

          select.addEventListener('change',function() {
            trackChange(select.value);
          });
		

The trackChange() function

          function trackChange(value) {
            if(value === 'off') {
              hideTracks();
            } else {
              hideTracks();
              var splitValue = value.split('-');
    
              for (var i = 0; i < video.textTracks.length; i++) {
                if(video.textTracks[i].kind === splitValue[0]) {
                  if(video.textTracks[i].language === splitValue[1]) {
                    video.textTracks[i].mode = 'showing';
                  }
                }
              }
            }
          }
		

How trackChange() works

• If 'off' value selected, hide all tracks.
• If another value selected, hide all tracks, then...
• Split the select value into track kind and language.
• Loop through the video text tracks to find the one that has that kind and language.
• Show the text track that matches.

Subtitles versus captions

• Subtitles are for people who can hear the audio dialog, but can't understand the language.
• They only include the words that are being spoken and are not positioned.
• Captions are for people who are deaf or hard of hearing.
• They include information on who is speaking, with positioning.
• They also include information to describe music ambient sound effects.

Caption <track> example

          <track label="English" kind="captions" srclang="en"
            src="vtt/sintel-captions-en.vtt">        
		

Code time: tutorial step 4

WebVTT gets more complex

          1
          00:00:18.700 --> 00:00:21.500 line:20% align:end
          <c.man><b>Man</b>: This blade has a dark past.  </c>
		

Cue settings

• vertical: Specifies vertical text, not horizontal.
• line: Specifies the line the text appears on.
• position: Specifies the position the text will appear at along the line.
• size: Specifies the size (width) of the text.
• align: Specifies the text alignment.

Styling captions

Caption text cues have some special tags marking them up, stylable via CSS Extensions.

          <c.man><b>Man</b>: This blade has a dark past.  </c>
		

Styling captions

The ::cue pseudo-element is the key to targetting text track cues for styling.

          video::cue { whitespace: pre; }
          video::cue(.man) { color:yellow }
		

What properties can be applied to text cues?

• color
• opacity
• visibility
• text-decoration
• text-shadow
• background shorthand properties
• outline shorthand properties
• font shorthand properties, and line-height
• whitespace

Available text cue tags

• Timestamp, e.g. moon <00:17.500>hits your eye
• Class tag, e.g. <c.classname>text</c>
• Italics tag, e.g. <i>text</i>
• Bold tag, e.g. <b>text</b>
• Underline tag, e.g. <u>text</u>
• Ruby/Ruby text tags, e.g. <ruby>WWW<rt>World Wide Web</rt>oui<rt>yes</rt></ruby>
• Voice tag, e.g. <v bob>text</v>

Adding a default selection

if(curTrack.language === 'en' && curTrack.kind === 'subtitles') {
  addTrackOpt.setAttribute('selected','selected');
    trackChange(select.value);
  }

Finished!

Further resources

• Adding captions and subtitles to HTML5 video (MDN)
• Getting Started With the Track Element (HTML5 Rocks)
• Working with HTML5 Video: Adding captions and subtitles (Content kit homepage)

Getting involved with the MDN community

• Visit MDN — sign up to the site, learn more, fix things, write docs!
• Find MDN on Mozilla IRC — join the #mdn channel and ask questions
• You can also contact us on the dev-mdc mailing list — we'd love to hear from you.

Credits

• "Matrix" cover pic by David Asch