transitive-bullshit/ffmpeg-gl-transition

ffmpeg-gl-transition

FFmpeg filter for applying GLSL transitions between video streams (gl-transitions).

(example crosswarp transition)

Note

If you want an easier solution, I recommend checking out ffmpeg-concat, an npm module and CLI that allows you to concat a list of videos together using a standard build of ffmpeg along with the same sexy OpenGL transitions.

Intro

FFmpeg is the defacto standard in command-line video editing, but it is really difficult to concatenate videos together using non-trivial transitions. Here are some convoluted examples of a simple cross-fade between two videos. FFmpeg filter graphs are extremely powerful, but for implementing transitions, they are just too complicated and error-prone.

GL Transitions, on the other hand, is a great open source initiative spearheaded by Gaëtan Renaudeau that is aimed at using GLSL to establish a universal collection of transitions. Its extremely simple spec makes it really easy to customize existing transitions or write your own as opposed to struggling with complex ffmpeg filter graphs.

This library is an ffmpeg extension that makes it easy to use gl-transitions in ffmpeg filter graphs.

Building

Since this library exports a native ffmpeg filter, you are required to build ffmpeg from source. Don't worry, though -- it's surprisingly straightforward.

Dependencies

First, you need to install a few dependencies. Mac OS is very straightforward. On Linux and Windows, there are two options, either using EGL or not using EGL. The main advantage of using EGL is that it is easier to run in headless environments.

Mac OS

GLEW + glfw3

brew install glew glfw

Mac OS users should follow instructions for not using EGL.

Linux with EGL

We default to EGL rather than GLX on Linux to make it easier to run headless, so xvfb is no longer needed.

glvnd1.0 building from source

mesaGL>=1.7 mesaGLU>=1.7

yum install mesa-libGLU mesa-libGLU-devel

GLEW >=2.0 building from source

Linux without EGL

If you don't want to use EGL, just comment out this line in vf_gltransition.c

#ifndef __APPLE__
# define GL_TRANSITION_USING_EGL // remove this line if you don't want to use EGL
#endif

GLEW

yum install glew glew-devel

glfw building from source

On headless environments without EGL, you'll also need to install xvfb.

pkg install xorg-vfbserver (FreeBSD)
apt install xvfb (Ubuntu)

Xvfb :1 -screen 0 1280x1024x16
export DISPLAY=:99

Building ffmpeg

git clone http://source.ffmpeg.org/git/ffmpeg.git ffmpeg
cd ffmpeg

cp ~/ffmpeg-gl-transition/vf_gltransition.c libavfilter/
git apply ~/ffmpeg-gl-transition/ffmpeg.diff

Non-EGL:

./configure --enable-libx264 --enable-gpl --enable-opengl \
            --enable-filter=gltransition --extra-libs='-lGLEW -lglfw'
make

EGL:

./configure ... --extra-libs='-lGLEW -lEGL'
make

Notes:

• See the official ffmpeg compilation guide for help building ffmpeg on your platform. I've thoroughly tested this filter on macOS Sierra (macOS compilation guide).
• Depending on your platform, there may be slight variations in how GLEW and glfw are named (with regard to --extra-libs, above), e.g. -lglew or -lglfw3 - check pkg-config.
• The above example builds a minimal ffmpeg binary with libx264, but there's nothing codec-specific about the filter itself, so feel free to add or remove any of ffmpeg's bells and whistles.

Here's an example of a more full-featured build configuration:

./configure --prefix=/usr/local --enable-gpl --enable-nonfree --enable-libass \
  --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libtheora \
  --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 \
  --enable-libopus --enable-libxvid \
  --enable-opengl --enable-filter=gltransition --extra-libs='-lGLEW -lglfw'

You can verify that the gltransition filter is available via:

./ffmpeg -v 0 -filters | grep gltransition

Usage

Default Options:

./ffmpeg -i media/0.mp4 -i media/1.mp4 -filter_complex gltransition -y out.mp4

Custom Options:

./ffmpeg -i media/0.mp4 -i media/1.mp4 -filter_complex "gltransition=duration=4:offset=1.5:source=crosswarp.glsl" -y out.mp4

Params:

• duration (optional float; default=1) length in seconds for the transition to last. Any frames outputted after this point will pass through the second video stream untouched.
• offset (optional float; default=0) length in seconds to wait before beginning the transition. Any frames outputted before this point will pass through the first video stream untouched.
• source (optional string; defaults to a basic crossfade transition) path to the gl-transition source file. This text file must be a valid gl-transition filter, exposing a transition function. See here for a list of glsl source transitions or the gallery for a visual list of examples.

Note that both duration and offset are relative to the start of this filter invocation, not global time values.

Examples

See concat.sh for a more complex example of concatenating three mp4s together with unique transitions between them.

For any non-trivial concatenation, you'll likely want to make a filter chain comprised of split, trim + setpts, and concat (with the v for video option) filters in addition to the gltransition filter itself. If you want to concat audio streams in the same pass, you'll need to additionally make use of the asplit, atrim + asetpts, and concat (with the a for audio option) filters.

There is no limit to the number of video streams you can concat together in one filter graph, but beyond a couple of streams, you'll likely want to write a wrapper script as the required stream preprocessing gets unwieldly very fast. See here for a more understandable example of concatenating two, 5-second videos together with a 1s fade inbetween. See here for a more complex example including audio stream concatenation.

Todo

• simplify filter graph required to achieve multi-file concat in concat.sh
• support default values for gl-transition uniforms
  • this is the reason a lot of gl-transitions currently appear to not function properly
• remove restriction that both inputs be the same size
• support general gl-transition uniforms
• add gl-transition logic for aspect ratios and resize mode
• transpile webgl glsl to opengl glsl via angle

Related

• ffmpeg-concat - Concats a list of videos together using ffmpeg with sexy OpenGL transitions. This module and CLI are easier to use than the lower-level custom filter provided by this library.
• Excellent example ffmpeg filter for applying a GLSL shader to each frame of a video stream. Related blog post and follow-up post.
• gl-transitions and original github issue.
• Similar project that attempts to use frei0r and MLT instead of extending ffmpeg directly.
• FFmpeg filter guide.
• awesome-ffmpeg - A curated list of awesome ffmpeg resources with a focus on JavaScript.

License

MIT © Travis Fischer

Support my open source work by following me on twitter