Sunday, May 17, 2020

FL Studio and FFmpeg Libraries

Did you know that under the hood, FL Studio uses FFmpeg for some of their operations? For example, the Fruity Video Player plugin uses FFmpeg to load wide array of video codecs. And ZGameEditor Visualizer export function uses FFmpeg libraries for its video encoding.

Is this good or bad? Both. It's good because FFmpeg is the mother of codecs and formats, so it can decode lots of audio and video formats. But it's bad because FL Studio's bundled FFmpeg libraries are LGPL which lack some video encoder like x264. This causes problem where ZGameEditor Visualizer plugin lossy encoding option exports to H263 when using .mkv.

Note: ZGameEditor Visualizer seems always export to PNG in MP4 if you choose MP4 regardless of the "Uncompressed video" option.

So is it possible to change the old H263 to H264? Yes. LGPL software must be dynamically linked so it's user-replaceable, and fortunate for us, FL Studio dynamically links to FFmpeg. So how to replace the libraries? Just follow the steps below. Note that I assume 64-bit version of Windows and you're using 64-bit version of FL Studio executable.
  1. Make sure FL Studio is not running.
  2. Download 64-bit Windows FFmpeg shared binaries (release essential or release full). If you already have it downloaded before, you can use that. Just make sure it's latest version.
  3. Navigate to "%ProgramFiles(x86)%\Image-Line\Shared\ffmpeg\x64" and replace all the DLLs (except ILVideo_x64.dll) with the DLLs from the downloaded zip. Just the DLLs, not the exes too. It's good idea to backup all the DLL files there, just in case.

UPDATE: If you come here before, you'll notice I changed the link. This is because Zeranoe no longer provides FFmpeg binaries.

 Now start the FL Studio again, export to .mkv and it will use H264 (x264).

So what about other video codecs? Unfortunately FL Studio specialize MP4 extension and forces PNG in MP4 when used. But from my experiment, here are list of possible extension to use and the resulting video codec:

  • .mkv - H264 video codec (high profile) and Vorbis audio codec
  • .webm - VP9 video codec and Opus audio codec
  • .ogv - Theora video codec and Vorbis audio codec
  • .nut - MPEG4 video codec and AAC audio codec
Any other benefits of replacing the DLLs? Not sure if this another feature is caused by replacing DLL, but you can basically load any audio as long as the extension is one of FL Studio can recognize. Sadly that means you have to change the extension to .wav, .flac, .wv, .ogg, or .mp3 first before loading it to FL Studio.

I just hope the ZGameEditor Visualizer export function has more option of controlling the video output when using custom presets. I'd love that.

Thursday, May 14, 2020

Container vs. Codec

Often people misinterpret container as video format. Here's one simple scenario.

Say, there's A who have very old phone, released in 2007, and B who have smartphone released in 2019.
A: Can you send me that MP4 video in your smartphone
B: Sure.
A: Why it doesn't play in my phone? The phone says it supports MP4 video format but it can't play your video. Your video is bad and so do you.

So let's make this clear. First of all there are no such thing as "MP4 video format". Keep your questions for later. In the world of multimedia files such as video and audio, you have to know 3 things:
  1. Container file
  2. Audio codec
  3. Video codec
Container is a file that stores information how the video and the audio are stored inside the file. Sometimes also subtitles and additional files. Container also has information about what decoder should be used to decode the video and the audio and also contains when those video and audio should be decoded and presented to user. Here are some popular container file:
  1. MP4. Yes MP4 is a container, not a video format. Now you know!
  2. WebM.
  3. Matroska. Also known as MKV. The mother of container as it supports almost every codec in existence.
So, if MP4 is not video format, what are actually video formats? That's what called video codecs. It represents the data about how the video are encoded, basically video codec is what important about compatibility. Even if, say I use MP4 which was supported in A's phone above, but I use more recent codec, then A's phone won't able to play the video, despite being MP4.

So what are kinds of video codecs?
  1. MPEG4, XviD/DivX family goes here. A very old codec.
  2. H264, most popular codecs since smartphone existed.
  3. H265, recent codec which provides smaller size and better quality.
  4. VP9, royalty-free codec by Google which compete with H264.
  5. AV1, royalty-free codec various vendors which compete with H265.
PS: "royalty-free" term doesn't mean anything from user perspective. It only matters from developer perspective.

For scenario above, if B's video contains H265 codec, then A's phone won't able to play it, even if the video itself is inside MP4 container file. Now everything makes sense, right?

Then there's also audio codec. Watching silent video is not very fun right? Then here comes the audio codec. The definition is same as video codec above, but for audio instead. There's only one difference, most audio codecs can be extracted out of their container, being standalone file. That's not possible for video codec.

So, list of audio codec please? Okay.
  1. AAC. Its standalone file extension is .aac (actually MPEG ADTS). Can be placed inside MP4 container.
  2. Opus. Must be placed in Ogg or WebM container.
  3. Vorbis. Must be placed in Ogg container.
  4. FLAC. Its standalone file extension is .flac. Can be placed inside Ogg container.
  5. MP3. Its standalone file extension is .mp3. Can be placed inside MP4 container.
Whoa, hang on, so Ogg is not an audio format? Yes. Ogg is also a container. It can contain Theora video codec.

So the conclusion is, if the video file is in extension that you know, that doesn't guarantee your device can play it. Like, you feel your device is superior because it can decode AV1 in MP4 until FL Studio's ZGameEditor Visualizer lossless video export function writes PNG image inside MP4.