Jtdylktuy

Ever since I figured I can play .MP4 videos in my Firefox in Ubuntu 18.04 (e.g. my Firefox 43 still can't do it, my Firefox 60 can), I've been very happy with the concept, especially if I have to upload a video somewhere for sharing; I just need to set up Mime Types on the server, that is throw in a .htaccess file on an Apache server with:

AddType video/mp4 .mp4

... then I simply open a http(s)://...mp4 link in the browser, and its done.

However, in many cases I'd also like to add subtitles - and for one, I wouldn't want to "burn them in" the video as pixels (as in "hard subtitles"), I'd like them to remain as separate text information. I know the .mkv container supports this, but its not supported in Firefox (1422891 - [Rethink] Support mkv|matroska|video/x-matroska in Firefox).

So, I just found that the .MP4 as container can actually include separate subtitles:

• [all variants] MKV to MP4 Conversion


• https://forum.videohelp.com/threads/391199-How-to-add-own-subtitles-%28resp-pure-text%29-into-existing-MP4-video

Also, https://en.wikipedia.org/wiki/Comparison_of_video_container_formats notes for MP4 "Captions / subtitles":

ttxt, VobSubs with private objectTypeIndication not usable with the sceneDescription (started by Nero), BIFS

VOBSUB is a .bmp/pixel format so it doesn't interest me, but ttxt is interesting.

However, I would like to embed subtitles in an .MP4 file once, and then have the subtitled file viewable (with subtitles) both on the web (hopefully just by uploading on a server that serves correct MIME type, without any javascript), - and on a desktop player.

What procedure could I use for this, and is there an example of it, so I know what I can approximately expect?

https://apple.stackexchange.com/questions/116927/auto-add-subtitles-into-the-mp4-videos :

mp4 does support ttxt and VobSub but not srt or ass.

https://gpac.wp.imt.fr/mp4box/ttxt-format-documentation/

The 3GPP consortium has defined a standard for text streaming, independent of any scene description such as SMIL, SVG or BIFS: 3GPP Timed Text ... also known as MPEG-4 Streaming Text

In order to help authoring text streams, an XML format has been developed in GPAC, called TTXT for timed-text – the extension used being.ttxt.

ffmpeg support for tx3g

The FFmpeg codec mov_text and tx3g are the same thing: MPEG-4 Part 17 (Timed Text).

The info on this is a bit scattered, so it was a bit difficult to come up with an example; of course, I'd still love to hear if there's a better way of doing this. But here is what I did on Ubuntu 18.04 - first, a temporary directory:

cd /tmp

mkdir videotest

cd videotest

Let's start with getting an input video - the Big Buck Bunny trailer:

wget http://download.blender.org/peach/trailer/trailer_480p.mov 

 -O big_buck_bunny_trailer_480p.mov

This is in .mov format, but that's no problem, we can convert it as per Encoding video for the web gist:

ffmpeg -i big_buck_bunny_trailer_480p.mov 

 -vcodec libx264 -pix_fmt yuv420p -acodec aac 

 -strict -2 -profile:v baseline -level 3 

 -vf "pad=ceil(iw/2)*2:ceil(ih/2)*2" 

big_buck_bunny_trailer_480p.mp4

(Note that without the -vf ... option, we'll get a "get width not divisible by 2 (853x480)" error by ffmpeg.

Then, let's get the subtitles for this trailer - I found them here:

wget https://raw.githubusercontent.com/moust/MediaPlayer/master/demo/subtitles.srt 

 -O big_buck_bunny_trailer_480p.srt

At this point, we can quickly try vlc big_buck_bunny_trailer_480p.mp4 - this should play the video, and automatically load the .srt subtitles with the matching filename.

So far, I've used these software versions on Ubuntu 18.04:

$ vlc --version

VLC media player 3.0.4 Vetinari (revision 3.0.4-0-gf615db6332)

$ ffmpeg -version

ffmpeg version 3.4.4-0ubuntu0.18.04.1 Copyright (c) 2000-2018 the FFmpeg developers

Now, we need to get MP4Box, which we can do with:

sudo apt install gpac

Note that https://gpac.wp.imt.fr/downloads/gpac-nightly-builds/ currently have latest version of gpac_0.7.2-DEV-rev985; but the Ubuntu 18.04 version is:

$ apt-show-versions -r ^gpac

gpac:amd64/bionic 0.5.2-426-gc5ad4e4+dfsg5-3 uptodate

$ MP4Box -version

MP4Box - GPAC version 0.5.2-DEV-revVersion: 0.5.2-426-gc5ad4e4+dfsg5-3

Now we can first convert .srt to .ttxt:

MP4Box -ttxt big_buck_bunny_trailer_480p.srt 

 -out big_buck_bunny_trailer_480p.ttxt

... and then ember .ttxt in the .mp4:

MP4Box -add 

 big_buck_bunny_trailer_480p.ttxt:lang=eng":name=English":layout=0x60x0x-1:group=2 

 big_buck_bunny_trailer_480p.mp4 

-out subtitled_video.mp4

This subtitled_video.mp4 will show subtitles in VLC 3.0.4, but I have to explicitly right-click and choose Subtitle/Sub Track/Track 1 - [English] in VLC. Opening subtitled_video.mp4 in Firefox 60 will show no subtitles, nor a button/context menu to enable them.

At this point, let's note that Firefox does have some UI for subtitles:

• 515898 - implement UI for subtitles on html5 video/audio


• 887934 - [webvtt] Update controls to include options for closed captioning

... however, it seems to be only for subtitles in WebVTT / wvtt / .vtt format:

• 
  https://hacks.mozilla.org/2014/07/adding-captions-and-subtitles-to-html5-video/ (also available on MDN)

... and that, only via HTML5/JS using the <video> tag (so we cannot just embed subtitles in a video, and open a direct http* link to the video file, and expect subtitles to show). Furthermore, as per:

• https://stackoverflow.com/questions/27223702/is-it-possible-for-html5-video-to-play-subtitles-embedded-in-mp4-file-as-a-track

... even with <video> tag, currently subtitles embedded in video are not supported (a separate .vtt subtitles file is expected) - except, for mp4, by the MP4Box.js JS library.

Note that for this MP4Box.js library:

• 
  http://download.tsi.telecom-paristech.fr/gpac/mp4box.js/ is the old demo page


• 
  https://gpac.github.io/mp4box.js/test/index.html is the new demo page

Also, the Git repository has recent activity, however, on https://github.com/gpac/mp4box.js/releases the last one with individual files is v0.3.15 (Mar 16, 2016), and the last .zip is v0.3.16 (Jul 12, 2016); and building the full JS library from source is a bit involved (you have to use grunt).

Also, the MP4Box.js is intended for a lot more involved tasks with MP4 than just subtitles, and the demo page is quite complex, aiming to demonstrate all of that. So, it took me a while to reduce the demo page, and come up with an HTML that will just open and play a local .mp4 file, here it is:

• https://sourceforge.net/p/sdaaubckp/code/HEAD/tree/single-scripts/test-MP4Box.js-v0.3.15.html

Note that it uses the v0.3.15 of the MP4Box.js library (it was developed from the old demo page); especially since newer versions of the library (and the new demo page) have a slightly different API, meaning that the old JS code will break when used with a newer library.

(also note that this test page otherwise breaks at half download, at mp4box.releaseUsedSamples, with x.samples undefined message; the page attempts to work around this by skipping this statement, which allows the download to complete to about 1 second short of end in Firefox 60 - meaning most of the video will be playable, but you won't be able to loop it)

With this in mind, if we place test-MP4Box.js-v0.3.15.html in the same directory as subtitled_video.mp4 (in which .ttxt subtitles were embedded) and open the .html page in Firefox, we'll get this shown in JavaScript Console:

[0:00:00.552] [MSE] MIME type 'video/mp4; codecs="tx3g"' not supported for creation of a SourceBuffer for track id 3    mp4box.all.js:33:6

TypeError: Argument 1 of HTMLMediaElement.addTextTrack '' is not a valid value for enumeration TextTrackKind.     test-MP4Box.js-v0.3.15.html:412:20

So, tx3g/txtt embedded subtitles in .mp4 are viewable in VLC 3.0.4, but are not supported by MP4Box.js - the only embedded subtitles supported in MP4Box.js are WebVTT.

Thankfully, we can use MP4Box to embed the .srt we already have, as WebVTT:

MP4Box -add big_buck_bunny_trailer_480p.srt:fmt=VTT 

 big_buck_bunny_trailer_480p.mp4 

-out subtitled_video.mp4

... and if we now open the test-MP4Box.js-v0.3.15.html in Firefox 60, we'll see the subtitles and a CC button:

However, if we open this subtitled_video.mp4 with embedded WebVTT subtitles in VLC 3.0.4, we'll notice that sadly the subtitles can not be reproduced.

Though, support for WebVTT in .mp4 has been added in this commit mux: mp4: add support for webvtt · videolan/vlc@86f5f48 · GitHub from Jan 22, 2018. To get this, I had to install the latest VLC nightly as a snap via:

sudo snap install --channel=edge vlc

... which gave me:

$ snap run vlc --version

VLC media player 4.0.0-dev Otto Chriek (revision 4.0.0-dev-5939-gee31d91)

... and this VLC 4.0.0 can, finally, play the subtitles in subtitled_video.mp4 with embedded WebVTT subtitles. ( Though, note difficulties in https://askubuntu.com/questions/1106832/running-a-snap-installed-side-by-side-parallel-to-a-debian-package )

Since the above is a bit complicated, I opted for another approach, written in this test script (using the same files in this post; script should also work locally, via file:// protocol):

• https://sourceforge.net/p/sdaaubckp/code/HEAD/tree/single-scripts/test-srt-webvtt.js.html

Basically, that one is just a standard HTML5 video player, with a subtitle track containing a href to the .srt file, which as mentioned before, the HTML5 video engine does not support. However, at page load, the .srt filename is read via the src attribute of the subtitle <track>, the file is read via XHR, and then passed off to https://github.com/imshaikot/srt-webvtt which generates WebVTT from the .srt on the fly, and applies it to the subtitle track, so it can work.

So, while not as easy as just dumping a video with embedded subtitles on a server - all I have to do here is write in the filenames of the .mp4 and .srt inside the HTML <video> player code, and I don't need to worry about conversion any further, as its done on the fly.