Today, I merged a truly amazing feature that Benjamin M. Schwartz has been working on in the past few weeks in PiTiVi: alignment of clips based on waveform analysis. From commit aa68410…’s description:
The auto-aligner shifts clips into the timeline so that their contents are synchronized. It works by analyzing the audio track’s volume as it changes over time, under the assumption that when there are multiple recordings of a single event, they will be loud, or quiet, at the same time.
I wish I had that, back when I edited my LGM talk a few months ago. Let’s say you have a bunch of video footage and sound recordings of the same event, like this:
What you do is:
- Put the “clip that starts first” at the top of the timeline*
- Select the relevant clips
- Press the “Align” button
- Wait while our highly accurate shark-mounted lasers position your clips
So you go from this:
To this:
…with the click of a button. Very handy for multicam shots, conference recordings and multi-mic setups. Next up on the list: making it faster.
Please test this feature and see how it works out for you! In the testing I’ve done so far, it has been pretty accurate with all sorts of media files. If you can find cases where it isn’t able to sync up your audio correctly, please provide some sample files for us to play with.
Update: Benjamin wrote a nice post about his work on this feature (for the more technically-inclined among you) and his experience with the PiTiVi community. Technical discussions should probably happen as comments on his blog post.
*: Currently, the autoalign feature always considers the first selected clip at the top to be the “primary” clip; this means that if you’re flush at the beginning of the timeline and your topmost clip is not the one that starts the earliest, it won’t work properly. Patches or suggestions to make this more intuitive and fool-proof are welcome.
You can check out my main website, find me on Twitter or Mastodon or subscribe to my YouTube channel.
Comments
6 responses to “Automated multicamera clip syncing”
To find the beginning. You could simply try autocorrelation between the different audio tracks.
Do an auto correlation of all audio tracks an store the time that the tracks start to correlate.
tracks[1…n]
select track[1] as start track
for i = 2; i 0
% the tracks started to correlate some time into the track[i] this mean that track[i] have some thing recorded before it is the same as start track. Select track[i] as start track and do next iteration
if time < 0
% opposite of the case above. Keep start track as is.
if "no correlation at all"
% do nothing. The tracks look nothing alike to keep the current start track is a guess as good as any.
end for
% To play it safe you should run the loop until start track dont change anymore. For all iterations except the first you should always be able to exclude a few tracks.
Spontaneously I get a feeleing that if you correlate all tracks with each other an store the result in a matrix you should be able to get the same result but I need to think a bit more about that.
cheers
Anders
anders: I made a more in-depth post about the algorithm at my website. If you’re interested you can also read the source!
http://git.pitivi.org/?p=pitivi.git;a=blob;f=pitivi/timeline/alignalgs.py;hb=HEAD#l70
http://git.pitivi.org/?p=pitivi.git;a=blob;f=pitivi/timeline/align.py;hb=HEAD#l358
As for the need to identify which clip starts first, it is only because PiTiVi does not allow clips to be moved before time zero. The Auto-Aligner resolves this by shifting clips into alignment, and cutting off the beginning if they shift to a negative start time, which could be confusing to a user. Apart from the time-zero limit, there’s no difficulty caused by correlation peaks at negative shifts.
I always liked PiTiVi, and I felt bad when its development slid a bit, but it appears things are picking up again and improving in a lot of areas. OpenShot is another very good editor that rose to prominence while PiTiVi stagnated a bit, but it seems PiTiVi is stable and the project’s vision is pretty solid.
I haven’t used Premiere for a while since I got PiTiVi- that might have more to say about the depth of my video work than it does about PiTiVi’s sufficiency, however.
I see not that a little bit of my post was probably lost due to illegal characters.
Im on vacation right now, i dont think i have time to look on the code.
Is this is any of the PPA builds yet?
@Josh: nope, not yet, the official PPA is only for released versions.