Video compositing with Natron

Although traditional video editing programs allow you to trim footage and arrange everything into a pleasing whole, compositing applications such as Natron can help you create effects in postproduction on clips. Multiple effects can easily be connected and selected in a series through the user interface.

With a compositing application, using green-screen or news casting effects is little more than a few clicks away.

Considerable Control

Development of the free and open source Natron program began in mid-2013 at the French institute for scientific research (INRIA), which provided the necessary start-up funding [2]. In December, the project director, Alexandre Gauthier, won the INRIA "Boost Your Code" competition, which allowed him to work on Natron full time for 12 months.

Since June 2014, the development team has periodically released beta versions, which offer an amazing range of functions and by and large ran stably during tests (see the "Crash Risk" box). At the time of writing, the most current version was 1.0.0 – Release Candidate 2, which is the foundation for the fully stable version.

Crash Risk

In our tests on Ubuntu 4.14 in the 64-bit version, Natron 1.0.0 crashed occasionally. Crashes occurred especially when importing videos and updating the preview. Apparently, the culprit was an external FFmpeg library, which also took down the compositing program. In particular, FFmpeg would not read HD videos in the popular AVCHD format. Every attempt resulted in a failure with a black preview, which was then followed by the crash. However, when using openSUSE 13.1, this error did not occur and Natron ran stably.

Natron stores any intermediate action and then tries to recover from a crash to the previous state. This worked on our Ubuntu test machine quite reliably. However, if you try out Natron, you should still periodically save your project.

Natron is not intended for amateur filmmakers who want to quickly produce showy results. This fact is made clear through its unusual interface, which is heavily modeled after the professional program Nuke [3]. Instead of having a timeline, the videos and effects are presented in small boxes that the user interconnects. This concept takes some getting used to but makes building more difficult scenes easier than using conventional video editing programs.

Like the Pros

Because of Natron's desire to appeal professional filmmakers, the application implements a wide variety of features. Natron stores colors internally as 32-bit floating-point numbers, which enables precise color correction and color manipulation. Color management is handled by the OpenColorIO library.

OpenImageIO and FFmpeg import image and video files in various formats. Natron also is equipped with multiple effects. Besides the obligatory color correction and green-screen effects, a simple rotoscoping tool [4] is also included. Further video effects can be incorporated via the plugin interface OpenFX. Natron supports not only open source but also commercial OpenFX plugins, such as Furnace from The Foundry [5].

Users can see the result of changed settings without delay in the preview. This feature is made possible through particularly efficient processing carried out under the hood in which the program uses all of the available processor cores. At the same time,

Natron requires very few hardware resources: According to the developer, not more than 3GB of memory is required and the graphics card only needs to be supported by Standard OpenGL 1.5 with some extensions (i.e., GL_ARB_texture_non_power_of_two , GL_ARB_shader_objects , GL_ARB_vertex_buffer_object , and GL_ARB_pixel_buffer_object ).

Commercial programs in contrast are much more resource hungry. The rendering of produced scenes can also be run through a command-line program, which can be launched on a dedicated server or server farm.

Clips

To install Natron, click Download on the right-hand side of the homepage [1]. On the subsequent page, below the penguin icon, you will find a first link for Natron for 32-bit systems. The second link is for 64-bit Linux systems.

In both cases, you will get a tar archive, which you can unzip to your hard drive (note that this file is not packed with Gzip). Click on the unzipped program to run it, and an install wizard should appear and ask you some questions.

After clicking Next , enter the desired installation directory for Natron. If you are unsure of how to do this, you can also accept the defaults. In the Next step, select the components to install (Figure 1). You will at least need the Natron program itself, the Natron/OFX Core Libraries , and the OFX IO Plugins .

Figure 1: The installation wizard installs the effects you want.

The remaining options provide additional plugins and packages with effects. If you are using Natron for the first time, then you should use the default settings. By doing this, you will not be stuck with an unmanageably large amount of effects.

After clicking Next , the install wizard will display the software licenses of all the components used by Natron. Select I accept the license agreement , click Next , and let Natron Install . Within some desktop environments, such as Unity in Ubuntu, the install wizard will automatically create an entry in the Start menu. If this is not the case for you, can always run the script Natron in the installation directory from now on.

Area Management

After starting, the main window (shown in Figure 2) will appear. The window is divided into three areas by default (preview is in the upper left). The gray shows the finished scenes that will be visible later. This feature will be useful when you enlarge your video clips or create a scene out of multiple video clips.

Figure 2: The main window of Natron looks tidy only right after the start. As with all video editing programs, your monitor can never be big enough.

Imagery outside of the gray rectangle will not be visible later. You can put effects together in the bottom left. Additionally, the settings of the effects are listed on the right side of the Natron interface. At this point, the settings of the project should be left at their default values.

You can adjust the three areas of the main window to suit your needs. You can adjust their size by grabbing and dragging the dividing lines with the mouse within these windows. Because of the gray tones, the dividing lines can be difficult to see. However, when the mouse pointer crosses a line, the pointer turns into a double arrow and that helps.

Take a look at the lower left pane; you should see two tabs. You should be on the NodeGraph tab. Natron's drawing surface is displayed here. Every effect and every video snippet appears as a small box (which Natron calls nodes ) on the display.

When you start out, you will see a rectangle with the words Viewer1 . This rectangle represents the preview monitor. This node has a connector pointing upwards. Because it is not currently connected to anything else, you will not see an image in the preview.

Knotting Techniques

To change this, click the workspace with the right mouse button and select the point Image | Readers | ReadFFmpeg . Then, select a video from your hard drive. Natron cannot currently import videos directly from a camera. You must therefore first copy videos to your hard disk and, if applicable, convert to another format.

You have now added a new node that opens and displays a video from disk (Figure 3). The node appears within the drawing surface as a box. Note that sometimes it conceals the Viewer1 node. Click and drag it to show the viewer node.

Figure 3: The node type ReadFFmpeg loads a video using FFmpeg.

The new node, labeled ReadFFmpeg1 , is highlighted in yellow, which means it's selected. The settings for this can be found on the right side of the window in the yellow marked area. Do not be daunted by the plethora of settings; most can be left at their defaults. In the case of new nodes, you can find ReadFFmpeg1 under File once the node is imported. You can also always select a different file and change the result later.

Entangled

On the NodeGraph tab, you can move each node using drag and drop. Position the ReadFFmpeg1 node near Viewer1 . The node ReadFFmpeg1 has a connector hanging out of its lower end. You want to connect both nodes. Here, you have two options. You can click ReadFFmpeg1 's connector with your mouse and then drag it onto Viewer1 while holding down the mouse button.

Because this action is somewhat fiddly, you can alternatively just pull the ReadFFmpeg1 node in the direction of Viewer1 . Eventually, a green connecting line will appear between the two. If you then release the mouse button, the two nodes will be connected automatically. In any case, the result should look like Figure 4. The node ReadFFmpeg1 now reads the video from the hard disk and pushes it through its output into the input of node Viewer1 . This displays the video in the preview.

Figure 4: The viewer displays the video output from the ReadFFmpeg1 node.

You can control the preview by using the buttons under the preview screen. With the triangle pointing to the right, you start playback. Between the preview and the NodeGraph register, you will find a small timeline. Natron does not work just with time designations; it also counts the images (frames) from the start.

The number above the orange triangle shows the number of the currently displayed frame. As soon as you touch the mouse to the timeline, a slightly darkened orange triangle will appear. If you click this, the preview will jump to the corresponding image. You can also hold down the left mouse button and then quickly go through the motions with your mouse through the video. This called scrubbing .

Effective

For each effect, you must create a node that connects to the other nodes in some way. For example, if you are trying to build in a fade-to-black effect, the scene should include a black background and then slowly cut into it from the video. To do this, first click with the left mouse button on a free area of the drawing screen. This step ensures that no node is selected. Otherwise, Natron would just automatically connect the newly created node to whatever is selected. Now, click on the right mouse button and select Merge | Merge . As the name suggests, this effect can merge transferred material through inputs A and B and blend both.

The new merge node will appear in the preview. Next, you must cut the connection between ReadFFmpeg1 and the Viewer1 by clicking on the black line. Next, connect the output from Merge1 to the 1 input of Viewer1 . It is sufficient to drag the output with the mouse onto Viewer1 the node.

Using the same principal, you can now connect the input B with the output of ReadFFmpeg1 . Now the only part missing is the black background for input A . Another node appears on the workspace. Ensure that no nodes are selected and then click with the right mouse button and select Image | Constant . Connect the output of the node Constant1 to the input from A of the effects Merge1 . The result should look like what is shown in Figure 5.

Figure 5: Merge1 mixes two clips, one after another. The setting of the effects will determine how this happens.

The Constant1 node delivers a black picture to the effect Merge1 . At this point, this leads on to the Viewer1 , which is why the preview now appears all black. If you would like to select a different color, click on the on the color wheel symbol within the settings of the node Constant1 .

This opens the color palette from Figure 6. As soon as another color is selected, you will immediately see a matching result in the preview. For this example, I'll stick with black (all of the settings of the fields r , g , and, b must be 0).

Figure 6: Natron directly displays any changes. Here, the newly selected color appears in the preview window.

Illusions

Now, you can apply settings to the effects Merge1 . If you do not find the appropriate area on the right side of the window right away, click on the effect in the NodeGraph tab. Natron then draws a yellow frame around its settings. In this example, only the regulator Mix is of importance. At the moment, it is set to 1 , whereby the node outputs the picture from the A input.

If you drag the slider to the left, Merge1 mixes the video image with its B input. If the value is 0 , then only the video will be visible. The settings that are carried out here are for the entirety of the scene and there is consequently no gentle fade.

To change this, go to the preview and make sure the orange-colored triangle is on the 0 in the timeline. Check that the Mix regulator is set to 1.00 in the merge node's settings. Merge1 should now only display a black picture. To the right of the Mix regulator is a symbol with a wavy line. Click on this and select Set Key from the menu. In doing this, the effect Merge1 will recognize that the first picture of the regulator Mix is set to 1 .

Now place the triangle on frame 100 in the preview. This is the point at which you want to be able to see the video. To set this up, pull the Mix regulator to 0 . By doing this, Merge1 will now have recognized two settings: On picture 0, Mix is set to 1 , whereas the Mix regulator on picture 100 is set to 0 .

All of these pictures that change settings are called keyframes , or keys for short. Merge1 automatically calculates the appropriate intermediate position for all the frames in between.

If you rewind the preview all the way to the beginning and then start playback, the video will slowly fade in from black. You can quickly jump between the two keyframes with the symbols below the preview that show a white triangle with a key. You can set as many additional keyframes as you wish as long as you select the appropriate frame and then shift the regulator.

To delete a keyframe, select it in the preview. Once this has been done, the small orange triangle will turn blue. Click on the right, next to the Mix regulator, click once more on the wavy line symbol, and select Remove Key . Following the same principal, you can also add all other keyframe settings, behind which a wavy line symbol exists.

Surfers

Currently, Merge1 blends the video with uniform speed. If you want to make the overlay quick in the first 25 frames and then you want it to go slower, click on the wavy line symbol and select Show in curve editor . Natron should now switch to the Curve Editor in the lower left pane.

The diagram here visualizes how the regulator changes the Mix over time. Time is plotted from left to right. The scale with the frame numbers can be found on the bottom edge. The vertical red line marks the beginning of the video. The orange line marks the position that is currently displayed in the preview.

To the right, next to frame 100, is a blue point as well as the second keyframe. The sloping line graphically displays how the Mix regulator changes: On the first keyframe, Mix is set to 1 , on the right-hand one, it's set to 0 , and the line falls uniformly henceforth. Natron selects the color of the line randomly, and it happens to be blue in Figure 7.

Figure 7: You can determine the relative effect in the Curve Editor.

Now, you can not only move the keyframes by dragging and dropping them within the Curve Editor , you can also change the shape of the line. To do this, click on any of the two keyframes. Two white squares should appear the left and right of the selected key. If you drag either of them with your mouse, you can change the slope of the curve (Figure 8).

Figure 8: You can change the slope of the line and thus the control of the effect by using the white dots.

Currently, this process is a bit fiddly because the Mix regulator only allows for values between 0 and 1 despite the fact that Natron would definitely like to display values up to 50. Nonetheless, you may zoom in the pane with the mouse wheel. Natron then travels to the point you aim at with the mouse pointer. Unfortunately, there is no scroll bar in Natron 1.0.0 yet, so you will have to zoom in and out to change positions.

The blue line is straight by default. If you only drag one of the white points, you will only change your control. On the other hand, if you select and change multiple white points, the line will depict a curve. You can configure the transition to start fast and then slow down (or vice versa).

You can also display and change the curve for all other effects that use keyframes. Note that using the right mouse button, you can select different interpolation techniques in the Curve Editor under Interpolation .

Packaging

You must be able to render the final scene out of Natron. You can use nodes for this process, too. Switch over to the NodeGraph tab, click on the drawing surface with the right mouse button, and select Image | Writers | WriteFFmpeg . In the window that appears, select the file format at the lower left; for example, select mp4 for a film in MPEG4 file format. Note that this only pertains to the file format and not the compression format.

Assign a file name to the right next to Sequence and click on Save . This creates the new node WriteFFmpeg1 , which you can connect to the output of the Merge1 node. Outputs can be attached to multiple nodes. These will then preserve the same data.

If you click on Render in the settings on the right-hand side of the control, the WriteFFmpeg1 node will write the obtained video first only in a file. Beforehand, you should correctly set the Format . Natron does not currently offer pre-settings, so users must be well versed in the desired compression procedure.

For MPEG4 video, select MP4 , expand the Advanced tab, and select Codec for MPEG-4 part 2 (Figure 9). Once all of the settings are correct, click on Render . If you want to save only a single image on your hard disk, you can use the Image | Write node, load the reversed image, and send it to its output. New nodes can be generated by means other than just using the right mouse button. All nodes can also be found behind the honeycomb-shaped icons on the left side of the window. To delete a new node, select it and then simply press the delete key. See the "What's in a Name?" box for information on naming nodes.

Figure 9: With these settings, the WriteFFmpeg1 node produces a video in MP4 format.

What's in a Name?

Currently, the names of the nodes are somewhat nondescript. To rename ReadFFmpeg1 to the somewhat better Big Buck Bunny Video , select the node (this should be highlighted in yellow), and click on the right side of the settings on ReadFFmpeg1 (to the right, next to the red button). Now, you can overwrite the previous name with Big Buck Bunny Video . Natron centers the NodeGraph regulator on the lower left side of the corresponding node with just one click on the red button.

Custom Work

The WriteFFmpeg node automatically overrides the delivered image dimensions. If you want a different resolution, set the video format under the Project Settings drop-down menu to Output Format in the format you want Natron to later produce. This can be found on the right-hand side of the window. If you want to process it with another video editing program, such as Kdenlive, you should choose the video format in which your video camera has recorded.

If this format is not included, click on New Format , enter the desired pixel width into the w field, enter the desired pixel height into the h field, and then enter the pixel axis ration into pixel aspect . Natron does not adapt the video footage to the new resolution automatically. To enlarge or shrink a video, use the Transform | Transform node.

Outlook

The next version of Natron should add a tracker node. In addition to this, users can mark a point in a video. Natron then follows this point automatically through the entire scene. Other effects can be controlled in turn with the resulting path. Moreover, Natron should be able to be controlled by Python script (remote) controls. In conclusion, users should be able to quickly retrieve predefined settings via presets.

Those wishing to help in the development can obtain the complete source code from Natron account on GitHub [6]. The code itself is under a v2 Mozilla Public License. In the future, the developers want to also provide complete packages for various Linux distributions. The developers are currently looking for package maintainers.

Bottom Line

Natron is aimed at professional users, but it also provides ambitious amateur filmmakers with good set of tools. The node concept takes a bit of getting used to and requires some time for training. However, it pays off in the long run, because Natron can also create more complex effects.

In its current version, Natron does not yet offer the same rich set of features that its commercial competitors do. However, the developers are on the right track and version 1.0.0 is definitely worth a look.