Video streaming

Video streaming is awkward: it seems to be relatively common, I know at least a few people who worked on software for that (always wondered how come that there's no existing tools), and it's still tricky to set. Although the situation is similar with many other things: somehow there's not much of usable software, despite all the programming that gets done; even XKCD #949 is still relevant.

I needed to read RTSP streams from IP cameras, making them available to web browsers via HTML5 Video. The container formats commonly supported by web browsers are:

Usually H.264 video and AAC audio; non-free/patent encumbered; by Apple.
Usually VP9 video and Vorbis or Opus audio; free; by On2, Xiph, Matroska, Google.
Usually Theora video and Vorbis or Opus audio; free; by Xiph.

I'd ignore the first one completely if it wasn't the most widely supported by web browsers (particularly Safari and IE/Edge don't support anything else). As of software, there are:

Supports FLV/MP4, requires to rebuild nginx, making maintenance rather painful (manual rebuilds on each update, it's not in Debian or CentOS repositories). But has plenty of features.
No config reload without restart, apparently to be discontinued soon. Supports plenty of formats, but doesn't have many features as a server.
By Xiph, supports Ogg and WebM, MP4 is not supported because it's non-free (though sort of works for a single client).
proprietary, half-baked, and/or abandoned software
There's a few of those, and they are used in production, but that'd be a pain to work with.

After the initial investigation, I've set an Icecast-based system. To make Icecast only create streams on demand, using RTSP streams as source, one has to use hacks: either turn that RTSP stream into HTTP one somehow, and use it as an on-demand relay, or set pseudo-authentication (listener_add and listener_remove, to track when a source stream is needed), running and killing ffmpeg instances when needed. I did the latter, with another program updating and reloading icecast.xml, and it worked – though duplicate users should be allowed for that, or it wouldn't be reliable. To support IE/Edge and Safari, there is ogv.js, but it's quite laggy to use JS for that. A custom libav-based MPEG-4 streaming server worked for Edge only occasionally, probably had something to do with installed codecs or drivers (can be tested with nc and ffmpeg rather easily); worked for FF with particular ffmpeg settings (not clear which, probably has something to do with framerate), but worked in VLC and ffplay. Apparently only HLS (with hls.js) is reliable for playback in Edge (I've made the same program that I use for Icecast pseudo-authentication to serve HLS files, so that it can easily keep track of all the clients and streams). Oh, and Chromium doesn't support HTTP basic authentication for embedded videos. There's actually more to it, since many things around WWW are broken, but that's a rough outline.

I guess that things would have been a bit nicer if we weren't using protocols designed for hypertext transfer to transfer video streams, along with the programs that are supposed to render hypertext documents, but are used to watch video; they may be not the best fit. Although it works, mostly, with some hacks.

Later it turned out that the computational costs of real-time transcoding were too high for the used hardware, source streams were using H.264, and HLS is supported in all major web browsers, so I had to disable the ogg/Icecast/transcoding bits, only using HLS. That's how some of the bad setups happen.