KISVid T - Using Tracks with Videos

This program illustrates using three non-standard ways of supplying subtitle tracks for videos in HTML5. The cues for tracks are normally stored in WebVTT file with a .vtt file type. These files are simply text files with a specified format. The correct mime type for these files is text/vtt. Unfortunately, many servers do not supply the correct file type. While most browsers ignore this problem, Internet Explorer 11 refuses to use a .vtt file with the wrong mime type. This means that many users will be unable to see valid subtitle track cues. This leads to the question, "Are there any workarounds for this problem?"

This tutorial looks at three workarounds. So in addition the standard way assumed by HTML5, we will look at creating cues with

These techniques will work because the HTML5 standard provides a means of creating tracks and adding cues to them.

This tutorial also demonstrates how multiple tracks and videos can be implement in HTML5 using Processing.js. It suggests some ways to solve problems when trying to use tracks. If you only want to add a single track to a video, you are familiar with WebVTT file format, you don't want any sophisticated styling, and your server supplies the correct mime type for .vtt files, you may not need this tutorial. It intended for developers who are having problems.

The coding is done in Processing.js. Much of it is very close to the coding that would be used in Javascript. Most programmers preferring to Javascript should be able to convert it with little difficulty. Javascript coders would probably prefer to use different buttons. The coding handling the videos is similar to that uses in the KISVid3 adapted to the array structure used in KISVid5 tutorials.

Just a quick word about the Spanish and German tracks. Unfortunately I am not proficient with either of these languages. The Spanish and German tracks for "developerStories" simply count in that language. The German track for "Kitten" was translated by a on-line translator. I have no idea how good the translation is.

Credits: The jwplayersFeatures video and tracks are designed to test implementations of several track properties. The video is complements of the Blender Open Movie Project. The track is a modified and expanded version of track created by http://www.jwplayer.com/. The developerStories video was developed by Google as an early demo of tracks but has been used in other web pages that discuss tracks (e.g. http://www.html5rocks.com/en/tutorials/track/basics/?redirect_from_locale=it).

The GUI sketch

Much of the sketch is similar to the other sketches in the tutorial series. One of the new features is the "Track source" line below the "Source file" line that gives the source of the current track. The three lines of blue buttons are as in some of the other tutorials. Clicking an up or down button changes the setting by 10%. At the very bottom on the left is a comment about the status of the video which unfortunately has not been 100% accurate.

On the left near the bottom are three radio button choices for the video. Above them are 1 to 3 (depending on the video) radio button choices for the track language. The number before the language encodes information about the status of the track. Numbers greater than 100 indicate the .vtt file for the track has not been processed by the browser yet. Negative numbers also mean that the track has not been created yet. "-2" indicates that the source will be created from code. "-3" indicates the track will be created by reading cues from the .html file. Finally, numbers from 0 to 99 indicate the track as been processed and give the number of the track in the array of tracks in the video object.

Lets consider the three radio buttons in the lower right. When active, the "Hide subtitles" button will conceal the subtitles.

The "Show information" button will add a considerable amount of information about the current track just below the video frame. On the right there is a listing of the current cues together with their optional identifiers, start time and end time. On the left is a significant amount of information about the status of the track and different structures as specified the standards. In some cases, this information can be helpful during debugging.

Finally the "Read VTT files" button. When this button is active, any further reads of a track will be done by using XMLHttpRequest even if the browser is capable of handling the track in the normal way. It is important to note that at least on my server at the present time. IE 11 always uses this method even if the button has not been selected. Also note that tracks that have already been read are not reread when the button is selected and tracks that are generated by code or from HTML cannot be processed by this method.

The state of tracks in HTML5

The state of tracks has improved significantly in the last couple of years. It appears that the concept of tracks was added to HTML5 before any much thought was given to how it would be implemented. But now all the major browsers support tracks - at least parts of it. For example, the developerStories video in http://www.html5rocks.com/en/tutorials/track/basics/?redirect_from_locale=it has a English track that works in the major browsers except IE11. (That site does not include a the .mp4 version of the video which is the only video type supported by IE11.) However, (at least of February, 2015) if you try to use tracks on your own, you are very apt to find that there are significant problems.

We have already mentioned that Internet Explorer 11 requires the correct mime type for vtt files. Unfortunately my server (as of February 2015) does not supply the correct file type for vtt files. If your server does not supply the correct mime type, there are some possible workarounds. This tutorial suggest three workarounds that I haven't seen elsewhere.

First a comment about using browsers. Only Firefox allows running Processing.js code without using a server. Hence the author has done most of the development with it. Unfortunately, having a sketch that works properly in Firefox, doesn't mean that the code will work properly in other browsers when it is moved to a server. In addition, Firefox has not implemented some features available in some other browsers. The previously mentioned Internet Explores requirement of the correct mime type is only one such problem. Clicking the "+/-" link below will show some comparisons about features impliment by various browsers.

[+/-] Track sources used in this sketch and styling comparisons

The author has a PC and hence has been unable to doing testing with the latest versions of Safari. It is his understanding that it handles tracks. The sketch has been tested in Firefox 35, Opera 26, Chrome 40, and Internet Explorer 11 running on an Windows 7 computer.

This tutorial is very long and readers often may be interested in a certain part(s). To simplify finding a particular part, readers can just click the [+/-] links to expand the part they are interested in.

[+/-] Basic concepts of tracks


[+/-]API and DOM interfaces for tracks

[+/-]API interfaces for cues

[+/-] WebVTT file format

[+/-] The <track ... > tag.

[+/-] Using programming as an alternative for creating tracks

We will now turn our attention to the 3 workarounds. The KISVidT code for the three methods is stored in separate files. Users will normally only use one of the methods in any particular application.

[+/-] Workaround 1: Using code to create a track

[+/-] Workaround 2: Reading track information from the HTML file

[+/-] Workaround 3: Reading track information from the VTT file

[+/-] Modifying a track

[+/-] Putting it all together - the main file

< Previous?????     Next????? >

The following links load the indicated file. You can use your "Reload" or "Refresh" button to return to this page.

The KISVidT.html file The original .html file for KISVidT
The codeNewTrack.pde file Workaround 1
The readHTMLFile.pde file Workaround 2
The readVTTFile.pde file Workaround 3
The KISVidT.pde file The main file
The CircleButton.pde file The coding for the buttons
The replaceAll.pde file The general purpose replaceAll method
The getAllProperties.pde A method that lists properties of an HTML structure

Updated 2/18/15