Video Container/Video codec/Audio Codec Preferences?

[snoopy]

I'm wondering about moving away from the .avi container, and have pretty much decided to move away from mp3 as an audio codec.

Here's why: AVI is an old container that's on it's way out
mp3 doesn't support more than 2 audio channels

So,

What do you recommend? I'm looking for something with decent compression, and intend to be saving 720p resolution video, and potentially surround audio with minimal lossiness.

[Jeff250]

Your two choices these days are H.264 video + AAC audio wrapped in an MP4 container, or VP8 video + Vorbis audio wrapped in a WebM (mkv) container. Both have open source encoders and decoders and are supported by ffmpeg with the right libraries. H.264/AAC are patent encumbered, but they are supported on more devices. H.264 is arguably slightly better than VP8, but Vorbis is arguably slightly better than AAC. Pick your poison.

[Krom]

Matroska (.MKV) is pretty much the best (from both a functional and licensing standpoint) video container out there but it has limited support outside of PCs. It is relatively easy to use and has less overhead than AVI, basic muxer and utilities can be found here: http://www.bunkus.org/videotools/mkvtoolnix/ Supports: practically everything, you can even embed soft subtitles and specific fonts for them to use into one.

MP4 is more supported in media streaming devices / playstations / xboxes, but licensing could be a pain and it doesn't support quite as many goodies as Matroska (but is still a massive improvement over avi).

For the audio codecs, for movies there are generally two camps I see, the first is obtaining the original Dolby Digital (AC3) / DTS tracks directly like from a DVD source and just muxing them directly into your video container. Or to convert it to another codec like AAC or the lossless FLAC which I believe can support up to 8 channels without issue. The only problem is FLAC and AAC can't be bit streamed to supporting amplifiers like [DD] or DTS can be. So FLAC and AAC basically only work if a PC is playing the file and is hooked up properly to a sound system with enough channels, although there should be more media streamer / ps3 / xbox support for AAC.

As for the video codec, just about everything is h.264 these days, it is pretty much the most efficient codec around right now and is widely supported even in set top boxes and streamers. Xvid is still around for older / slower machines, but is definitely not a serious contestant for 720p / 1080p video. Another option is x264, but I don't think it is as well supported outside of the PC space as h.264 is.

Also you can put h.264 + AAC into a mkv file, or you could put h.264 + FLAC into it, or you could put h.264 + MP3 + AAC + FLAC + Vorbis into one, you can stuff it with multiple streams and pick which one you want to use at playback. The container even supports multiple video streams if you ever wanted to do that.

[Jeff250]

x264 is just a really good H.264 encoder. It's not a competing codec.

Why no VP8/Vorbis love?

I don't like FLAC. To me, the benefits don't seem worth the price in space, but it's worth considering.

[Jeff250]

Here's the problem as I see it. If you want to use a current generation video codec, going anywhere outside of the h.264/AAC/MP4 stack is precluding being able to play it on most current generation consumer devices. Even using mkv at all will mean that it won't be playable on a PS3/Xbox/etc. If you want to use mkv to avoid licensing issues, it won't work if you still stick with h.264 for video encoding. If you're going to use mkv, why not use Vorbis--a superior audio codec--with VP8? This way you avoid all licensing issues, and there's at least a chance that hardware will start supporting WebM.

[Krom]

He is going to be playing it back on a PC: http://www.descentbb.com/viewtopic.php?p=62104#62104

FLAC is a pretty excessive option for movies I agree, but for CD music it is the best because its lossless and can be quickly/easily converted to portable media player friendly formats if you want to take it on the road. (My 4 year old PC can convert FLAC to MP3 at 70x playback speed.)

[Jeff250]

re: .com thread, that's probably the best of both worlds for strict PC use. Plus, I'd think it wouldn't be too hard to remux the streams back into an *.mp4 container if the need ever arose.

[snoopy]

Thanks for the feedback.

I don't have the AV box yet, so I'll have to make sure I get appropriate cables when I do.

[snoopy]

The audio coming out of my capture card is AC3, so I've set the audio to copy.

After some experimentation: using mplayer to create an EDL cutlist, mencoder to encode (h264 video, copy audio), and mkvmerge to convert the container from avi to mkv.

I couldn't figure out a good way to cut the video after encoding, and ffmpeg doesn't like edl files, so I think I have a workable solution.

[snoopy]

Final update (I think):

Mencoder throws some sort of \"stuff\" into the AC3 stream to repair the time glitches produced by a weak signal and/or just in general over broadcast TV. Mkvmerge doesn't like the stuff, dumps it, and in the process introduces sync issues.

My solution: encode to faac.

[snoopy]

After much experimentation.....


some interesting things to note:

mplayer has a bug/problem related to EDL's and skipping sections of the video- it over-fills the audio buffer if you're trying to any sort of encode to it... a \"copy\" codec is the only one that seems to work.

neither ffmpeg or mplayer seem to particularly like re-encoding 5.1 audio from AC3. I got ffmpeg to do it, re-encoding to AAC, but it jumbled the channels and appears to have dropped one of them in the process. (Okay, it doesn't really jumble the channels, it just doesn't and can't un-jumble them because channels 0-5 correspond to different sequences of speakers for different codecs.)

mplayer has an audio filter to re-encode the AAC audio back to AC3 as it plays- Krom you might want to check it out. It appears to work in that it kicks out 5.1 channels to my receiver. I don't know about the channel mapping... I encoded AC3-AAC, and then played with on the fly conversion back to AC3 and the channels ended up jumbled, so I'm not sure if I was just doing something wrong in the playback, or encode, or both, but whatever. It said it was guessing about the channels mappings. I'm going to experiment with it a little bit more (but not too much) because-


I wound up with an mkv container, VP8, and a (direct copy from the source) AC3 format. I would like to patent-unencumbered.... I figure I get a bit of a pass since I'm not actually doing the AC3 encoding, I'm just copying it through.

If I manage to figure out the channels mappings, I'll let you guys know. Krom, mplayer isn't great on the GUI end of things, but it makes a great player, IMO, and you might be able to get rid of your hard-wired channels. Here's my command line:

mplayer -fs -zoom -quiet -nosub -vo xv -afm hwac3 -af lavcac3enc=1 -channels 6 \"$input\"

[Krom]

I'm using a custom build of MPC-HC for and a custom build of ffdshow for most video codecs (h.264 is done on the GPU by coreavc). I also use the haali media splitter for mkv support and disable most of the internal filters in MPC-HC because they are often old and broken or missing functionality. For AC3 or DTS audio I just have the system pass it straight to the SPDIF without decoding it, the receiver at the other end does the decoding.

One thing to keep in mind is that AC3 and AAC are both lossy formats, the more times you re-encode from one lossy format to another the worse the result ends up sounding. So getting that direct stream copy of the AC3 to pass straight through the SPDIF and work is generally the best option.

If I have something that is encoded in 5.1 AAC or some other format like FLAC, I can either wire my amp up to its 6ch analog mode which is a bit of a pain and disables all its DSPs. Or what I usually do is set Windows to on the fly encode ALL audio on the entire system to AC3 which is an option in my audio driver. It is pure software encoding and eats about 5-15% CPU time even when NOTHING is playing, but it is plenty effective for a quick fix to get all 6 channels out of a video if I really want them. Although most of the time I'm using headphones anyway so I just let ffdshow downmix AAC/FLAC to stereo which fits in the PCM stream. If it is AC3 or DTS then the amp downmixes it for the headphones instead of ffdshow and I still get the best quality.

[snoopy]

Good to know.

I think I finally (really) finished.

I tried an AC3 direct stream copy, and it didn't work for me. As part of cutting the video, mencoder messes with the AC3 stream which results in invalid bits that both the mkv and mp4 containers choke on.

The ffmpeg faac plugin has stream issues, so here's what I ended up doing, that (actually) works, keeping the audio and video synced and stuff:

Original mpeg from TV capture card (malformed mpeg stream, wierd timestamps/indexing) --> Mencoder copy/copy, with cutting, to avi container --> ffmpeg raw PCM audio dump (sync to video stream) --> faac audio encode --> ffmpeg 2-pass video encode to VP8, no audio processing --> mkvmerge ffmpeg-produced video and faac-produced audio.

I end up with a file that's got 5.1 audio correctly mapped and high-quality video. I think the only thing I'd consider changing would be using flac instead of faac, but I just not got faac working so I don't want to hose it up, and need a break from screwing with it.

I actually experimented with dumping the video and encoding it directly with vpxenc.... but at a intermediate file cost of ~ 1GB/min @ 1920x1080 I just don't have the space to manage it, and I can't pipe because I want to do 2-pass encoding.

Krom: Mplayer does the same thing that you're saying that your audio driver can do- so, same difference.

[Jeff250]

Interesting, so using faac independently of ffmpeg is what solved your channel mismatching?

Is there a reason why you aren't using ffmpeg for the muxing? It seems like you could combine the last two steps by doing something like (off the top of my head)...

Code: Select all

ffmpeg -i foo.aac -i bar.mpg -acodec copy -vcodec libvpx -f matroska ... bletch.mkv

Did you have problems with ffmpeg's muxing?

[snoopy]

Jeff,

I think you're right. At this point, meh. No particular problems with ffmpeg muxing. Like I said, if I dump the raw stream that came out of my capture card into ffmpeg the A/V sync is all wacky, but for some reason mplayer isn't tricked.

[snoopy]

After having encoded a few things:

I have to say that I'm very happy with my setup. The picture quality is great, the surround sound works as desired, and I have no patent encumberedness.

[SirWinner]

h.264... I'll pass... currently working on a secret project that uses a much much better Codec for Video Compression that will soon be available as part of our software package.

AVI files can be HUGE in comparison to even somewhat recent compression methods for saving Video Files.

[Jeff250]

AVI files can be HUGE in comparison to even somewhat recent compression methods for saving Video Files.

AVI is just a container format. In theory, you can pair up any video and audio codec in them, including h.264. It doesn't make much sense to talk about the compression of AVI.

h.264... I'll pass... currently working on a secret project that uses a much much better Codec for Video Compression

I'm rather skeptical.

[Grendel]

Depends on how it's better -- smaller or better IQ ? Both is possible, just not at the same time. (so far )

[Krom]

AVI files can be HUGE in comparison to even somewhat recent compression methods for saving Video Files.

AVI is just a container format. In theory, you can pair up any video and audio codec in them, including h.264. It doesn't make much sense to talk about the compression of AVI.

Correct; AVI is just a container but it does have a good bit more overhead than Matroska or MP4. I once took a 200 MB video out of an AVI and put it in a Matroska (MKV) via direct stream copy with no re-compression and it lost 5 MB of weight in the process. Nothing major, but did show the better efficiency of newer containers.

Of course getting h.264 into an AVI is a major hack job, it can't handle some of the newer codecs out there, and things like variable frame rate/etc flat out don't work in AVI.

[snoopy]

The overhead/etc. issues with AVI is why I moved away from it. Mencoder can stuff thing into and AVI container that don't belong there... to the point that you can create avi files with mencoder that only it can read.

I bet there are ways out there to improve compression at equal lossiness over what h.264 does. I saw a program that rendered something like 10 minutes of high quality video from a several kilobyte datafile, but that was a bit of a novelty- I took at few basic shapes, with no textures, and replicated them lots on interesting ways to make it look like a city-scape scene.

I'd also be willing to be that compression time will continue to go up and up as the algorithms for the compression get more and more sophisticated. Also, the power needed to render them will increase.

[Jeff250]

Sorry to bump this up snoopy, but do you know if your ffmpeg was built with libfaac or with ffmpeg's own (still buggy) aac encoder? I recently got bit by this, so I'm wondering if this might have been your problem too.

If it's compiled with libfaac, you should see at least:

Code: Select all

--enable-nonfree --enable-libfaac

in the configuration parameters when you run ffmpeg without any arguments.

And were you using

Code: Select all

-acodec libfaac

?

[snoopy]

Code: Select all

ffmpeg -codecs | grep faac
FFmpeg version git-2611e52, Copyright (c) 2000-2011 the FFmpeg developers
  built on Feb  6 2011 10:03:23 with gcc 4.5.2 20110127 (prerelease)
  configuration: --prefix=/usr --enable-gpl --enable-libmp3lame --enable-libvorbis --enable-libfaac --enable-libxvid --enable-libx264 --enable-libvpx --enable-libtheora --enable-postproc --enable-shared --enable-x11grab --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libschroedinger --enable-libopenjpeg --enable-version3 --enable-nonfree --enable-runtime-cpudetect --disable-debug
  libavutil    50. 36. 0 / 50. 36. 0
  libavcore     0. 16. 1 /  0. 16. 1
  libavcodec   52.108. 0 / 52.108. 0
  libavformat  52. 94. 0 / 52. 94. 0
  libavdevice  52.  2. 3 / 52.  2. 3
  libavfilter   1. 74. 0 /  1. 74. 0
  libswscale    0. 12. 0 /  0. 12. 0
  libpostproc  51.  2. 0 / 51.  2. 0
  EA    libfaac         libfaac AAC (Advanced Audio Codec)

Note: the channel mapping doesn't work natively in ffmpeg, even with libfaac. If you encode directly using faac, you can specify which channels correspond to center and LFE, and it defaults to 3,4. When I dump the audio from AC3 to PCM using ffmpeg, it puts those channels there. If I try to convert directly, within ffmpeg from ac3 to aac, the channels get all mangled. I downloaded an AC3 channel test video from somewhere to test this out.

A recent patch to ffmpeg/libvpx broke 2-pass functionality. (I think maybe it was ffmpeg, because x264 seems to also be having issues, so I'm actually in the midst of revisiting my compression scheme as I type this.)