A clean Hugo theme for a happy guy

xstack - A Filter Worth Knowing

October 7, 2019

If you burrow into the documentation of ffmpeg, the excellent audio/visual toolkit, you will find, deep inside the section on Video Filters, the description of a filter called xstack.

This filter allows you to create a mosaic or matrix of video feeds, like a wall of videos:

The above figure shows how nine videos might look in such a matrix. The example feeds shown above are just a color source input, each one including a drawtext element that identifies the feed. The scripts in the code described in this techbits article created this exact image.

I’ve written up a How-To on the ffmpeg wiki on how to create and use the xstack filter. There, you can learn about the how the xstack layout works. The scripts in this collection create the required layouts automatically. (See the creatematrix.pl script for details.)

Let’s delve into the code used to create the examples. Grab the tarball here (sha256 checksum) and follow along. Obviously, you’ll need to install ffmpeg first. Follow the package installation instructions for your system to install ffmpeg.

You will also need:

• printf(1), the command line print formatting utility,
• perl(1) - a recent version of the perl scripting program.

Find a suitable working directory, and extract the tarball using your system’s tar command.

README.txt

Let’s start with README.txt, where you can find the list of files and what they do:

colors.txt - a file containing the color definitions found
             in http://ffmpeg.org/ffmpeg-all.html#Color

getcolor.pl - a perl script to extract and print a random
              color.  Used in make1mkv.sh, play1mkv.sh
 
make1mkv.sh - this script creates a 30 second .mkv video file
              and places it into ./videos.

make1driver.sh - this script calls make1mkv.sh repeatedly
               to create and save .mkv files to ./videos

play1mkv.sh - this script creates a .mkv video file
                  and pipes it to ffplay to play for 10 seconds.
                  the file is not saved.

play1driver.sh - this script calls makeplay1mkv.sh repeatedly
                   to play a .mkv video. No files are saved.

creatematrix.pl - this perl script  creates a shell script that
                  displays a matrix of files in ./videos.
                  the size of the matrix and choice of
                  row or column major order can be specified
                  as options.

The scripts that create and use videos will default to a directory named ./videos in your current directory which you will need to create (mkdir videos).

(Note: For understanding row major order and column major order, see this Wikipedia entry.)

Let’s start with make1mkv.sh. This script will create one mkv video file in the videos directory.

sh make1mkv.sh 01

To play this file, use the ffplay command as follows:

ffplay -i videos/01.mkv -t '10' -autoexit

This will play the video you just created for 10 seconds and then exit.

You could create multiple files using the make1mkv.sh1 script with different parameters by hand, but there is a driver script, make1driver.sh that will do this for you.

/bin/sh make1driver.sh

Two notes here - First, the make1driver.sh script uses printf(1), a command line print formatting program. Check your system documentation to make sure this program exists on your system. You may have to adjust the command syntax in some cases.

Second, by default, the script creates 36 files in the videos directory. The files are small, just 20-25K bytes each, and, if printf(1) is working, they will be named 01.mkv, 02.mkv, 03.mkv, etc.

If all that worked, you will see 36 files in ./videos. Now you can start creating your video matrix.

The creatematrix.pl perl script is the script used to create the video maxtrix. Use the “-h” option to get a usage message about the program:

perl creatematrix.pl -h

Usage: creatematrix.pl  [2|3|4|5|6] [row|col]
       where num is the size of the created matrix
       and row or col specifies major order (default is row)
       Sizes are as follows:
         2 = vga   (640x480)  2 videos (2x2)
         3 = qvga  (320x240)  9 videos (3x3 default)
         4 = qvga  (320x240) 16 videos (4x4)
         5 = qqvga (160x120) 25 videos (5x5)
         6 = qqvga (160x120) 36 videos (5x5)
  Files are looked for in the ./videos directory.

The size definitions of vga, qvga, qqvga are documented on the ffmpeg website.

With no options, the script will output a bourne shell script that creates a 3x3 video matrix in row major order:

#!/bin/sh

# Start of Program

# matrix_size = []
# order_parm  = []
# matrixvar = [3]
# ordervar  = [row]
# maxcount  = [9]
# path      = [./videos]
# There are 36 files in ./videos
# We will only use [9] files.

ffmpeg \
   -i ./videos/01.mkv \
   -i ./videos/02.mkv \
   -i ./videos/03.mkv \
   -i ./videos/04.mkv \
   -i ./videos/05.mkv \
   -i ./videos/06.mkv \
   -i ./videos/07.mkv \
   -i ./videos/08.mkv \
   -i ./videos/09.mkv \
  -filter_complex " \
      [0:v] setpts=PTS-STARTPTS, scale=qvga [a0]; \
      [1:v] setpts=PTS-STARTPTS, scale=qvga [a1]; \
      [2:v] setpts=PTS-STARTPTS, scale=qvga [a2]; \
      [3:v] setpts=PTS-STARTPTS, scale=qvga [a3]; \
      [4:v] setpts=PTS-STARTPTS, scale=qvga [a4]; \
      [5:v] setpts=PTS-STARTPTS, scale=qvga [a5]; \
      [6:v] setpts=PTS-STARTPTS, scale=qvga [a6]; \
      [7:v] setpts=PTS-STARTPTS, scale=qvga [a7]; \
      [8:v] setpts=PTS-STARTPTS, scale=qvga [a8]; \
      [a0][a1][a2][a3][a4][a5][a6][a7][a8]xstack=inputs=9:layout=0_0|w0_0|w0+w1_0|0_h0|w0_h0|w0+w1_h0|0_h0+h1|w0_h0+h1|w0+w1_h0+h1[out] \
      " \
    -map "[out]" \
    -c:v libx264 -t '30' -f matroska -  | ffplay -autoexit -left 10 -top 10  - 


# End of Program

You can copy and paste the output into a terminal command line session, or redirect to a temporary file and run that with the bourne shell (/bin/sh):

perl creatematrix.pl > t.sh

/bin/sh t.sh

Ffmpeg will display a video matrix similar to Figure 1 above that will display for 30 seconds by default and then autoexit.

You can tweak the settings in the shell script, or to make scripts with your preferred values, edit the creatematrix.pl script. See below for additional discussion of the creatematrix.pl script.

With suitable parameters, you can create a 4x4, 5x5, or 6x6 matrix in either row or column major order:

perl creatematrix.pl 4 col > t.sh

/bin/sh t.sh

perl creatematrix.pl 6 row > t.sh

/bin/sh t.sh

To display larger sizes, you will need to edit the make1driver.sh script to create more files. And you need to edit the creatematrix.pl script to allow larger sizes, both left as an exercise for the reader. (Hint- in creatematrix.pl, look for where the variable $matrixvar is constrained.)

How high can you go? See if you can beat this “Matrix Madness” display:

Enjoy!