Video Tagging

Rapid compilations on a theme

Tagging allowed me to throw this compilation together very rapidly.

Selma and Eva from Franz on Vimeo.

Supercuts

After video capturing an event, I try to process and distribute it as fast as possible. However, sometimes there may occasion to create a compilation of a particular person or theme way into the future.

Ideally I want to be able to find all clips where the given subject is visible. Something like automatic face detection in photos except for video.

To date, I have not come across an elegant solution for automatic video analysis and it remains a fairly complex problem1.

Even the masterful cassetteboy duo are having to rely on manual processes to create their videos.

CassetteBoy

Cassetteboy, the artists behind youtube mashups like this and this, reveal their exhaustive and manual workflow in the following video. They utilise a database for searching subtitles

Some of it’s very boring stuff that could potentially be done by a machine … it would free us up to do the fun stuff. That would be ideal. – Cassette Boy2

Political supercuts utilise this method for isolating words and phrases.

This technique is useful for digital archivists at the BBC and beyond. 3

But this does not assist with finding videos without a spoken track or subtitles like most home videos.

Even if a solution existed4, our highly evolved visual recognition subsystems can scan through videos and tag people quicker than my cpu so that is what I am doing for now.

My primitive workflow uses Finder tags in order to add meta to video clips to allow for easily gathering media with shared content.

The process

When processing an event, I trim the fat by splitting clips in quicktime and trashing uninteresting ranges, then tagging what remains.

I create a smart search to list all media files without finder tags and process these at regular intervals and on import.

Portability

Finder tags means I am not locked into a proprietary tagging system like those provided with video editors.

Arguably I am still locked into the Apple ecosystem but that depends on how portable these extended attributes are. I could perhaps have the tags written to the filename if I decided this no longer a good solution. Brent Terpstra has created a utility for writing tags to the filesystem for backup purposes - tagbak

Spotlight tags can also be imported into Final cut 5

Gotchas

Tags are not preserved by backup utilities like backblaze see how to lose your tags

  1. Digital Video Archives: Managing Through Metadata

  2. The Foundry Session: Creativity in an age of AUTOMATION

  3. 2016-05-05-3-01 IPTC Video Metadata Draft 2 June 2015

  4. final cut pro can detect people and tag videos with one of the following: One Person, Two Persons, Group, Close Up Shot, Medium Shot, and Wide Shot. The analysis is quick and reduces the problem domain to a smaller set of frames on which face detection could be run. Final Cut Pro X: Video and still-image analysis options

  5. apple support - importing video