Getting the resources together to make a film is tough. Concessions are made daily in order to get the movie in the can (or on the hard drive, because let’s face it, hardly anyone is canning exposed film these days). But there’s one area where you should not compromise, and that is with the safety of your footage.
Whether it’s a student film, a tv pilot, a doc, or a narrative feature film, all too often there is not enough thought put into your project’s life after you say, “That’s a wrap!” on the day’s shooting. And we all know a filmmaker who’s lost precious footage due to poor backup practices and bad luck.
The life and safety of your footage is something that you need to be thinking about no matter the size of your production. There are a myriad of options for the storage of your footage, so here we will take a look at best practices in media management and the life of your footage after it’s “in the can” and explain all of the options out there.
- Media Managers: To Hire or Not to Hire
- The Birthplace of Your Media
- The Footage Transfer
- Software Options for Offloading Your Video
- Watching the Footage with Your Own Two Eyes
- Always Have Two Copies (Three is Much Better)
- Quick Dos and Don’ts of On Set Media Management
- Where Do You Put All That Footage?
- Communication Between Post and Production is Pivotal
- Hard Drive Options
- Each Project Has Its Own Challenges. Choose Your Own Adventure
Media Managers: To Hire or Not To Hire
The person who is in charge of your shot footage at the end of the day is going to have an awesome responsibility. It’s arguably the most important one on set, because if it isn’t done correctly, then all your hard won footage can be gone in the electronic blink of a crashed hard drive.
If you can afford it, a Media Manager who focuses on this is well worth their weight in gold. But this isn’t always possible.
Often times on smaller shoots, the job of managing media and offloading shot cards/drives may fall to the Assistant Camera person or the producer/director. To understand your best options with media management, let’s take a look at the life of media on a variety of different sets.
The Birthplace Of Your Media
Each day, you will shoot scenes for your project. You have a set number of CF, SD, XQD cards, or SSD drives that you load into your cameras to capture this footage. But the number of cards/drives you have to capture media is finite, and most productions can’t afford to simply use a brand new card/drive and then keep it on a shelf for the duration of post production.
So for longer shoots and multi-day shoots, you often have to re-use these media cards/drives. That means you have to delete your footage from the cards/drives to make room for new media you’re shooting for your next scene. So scene 47 with that steadicam shot at sunset during the winter solstice – once it’s been offloaded safely, the card/drive it’s on is going to be erased and written over to capture more media. Sometimes even on that very same day. Frightening, yes?
This is why the position of Media Manager exists. Because your media manager is the person who will look you in the eye and say, “I have safely gotten this footage from the card/drive. You can format it and re-use it for the next scene.”
If you are producing something on a tight budget, and you don’t have someone to hand this responsibility, then the buck stops with you and you will have to add it to the juggling act you’re doing on a daily basis to make the project happen.
The Footage Transfer
When a card/drive is full, the Media Manager or person taking on media management responsibilities must safely transfer the footage over to your storage solution.
This is not a simple drag and drop operation in Finder. When moving over footage, you should always use a software program that runs checksums. This is a process that compares your newly copied file with the original file it is copied from. It verifies that all the files have transferred successfully, and is able to point out if there are any errors in the transfer in pdf reports it generates.
But wait, what’s that you say? You don’t need to buy any fancy program. You can just run a simple ‘command+i’ on your folders to check that the GBs are the same between your card/drive and the copy you made.
Not so soon! While that works in many cases, it can also fail for various reasons. The programs check every one and every zero and provides a great degree of accuracy and piece of mind for your valuable footage (especially if you don’t have time to watch every frame of the footage yourself to make sure it has copied over).
If you are running on a slim budget, a good transfer program is a huge time saver. You can set a card and let it run while you produce your project, and then come back to it to find the reports and checksums have run and you can use them to help confirm your copy has worked at a glance. So, what should you use? Read on…
Software Options for Offloading Your Video
ShotPut Pro has become an industry standard for software of this kind, and it can offload multiple streams of the same source footage to different hard drives simultaneously, and is relatively inexpensive.
It has changed from a one time purchase price to a yearly account, but the program is good enough to be worth paying the $150 upfront. And there are presets you can use to help ensure the organization of the footage for the edit.
There’s another program with a simple and fast interface called Hedge that gives you almost everything ShotPut Pro has at the one time price of $49.
Our very own Daniel Gurzi, Director of Business Development here at KitSplit, was recently Media Manager on a commercial directed by Spike Lee and he came back to the office singing Hedge’s praises:
“So I was transfering 2x XQD cards (fs7), 2x micro sd cards (GoPros), a cf card (audio) and a usb3 file transfer from a digital bolex all at the same time. And it was routing to 3 different hard drives all at the same time and then I’d have like an hour and a half to wait for it to transfer. And I’d get a text message when it was done. It’s brilliant.”
Watching the Footage with Your Own Two Eyes
Checksums and verification reports in a program like ShotPut Pro go a long way to make one feel safe that files have been transferred. But the next thing whomever is managing your media will do is watch the footage to make sure it’s all there.
A Media Manager will have time to really scan through and watch whole sections. If you’re managing your own media, you likely won’t have time to watch all the footage yourself. But in that case, you should find the time to at least open each clip and make sure it plays.
This is also why transfer software is important – in lieu of being able to spend time on set checking your dailies, you get that extra check from the software, and then you run each clip and confirm each video clip plays. That may be all you have time for, but it is better than doing a simple ‘command+i’ check!
Always Have Two Copies (Three is Much Better)
Redundancy is important. No matter the size of your project, you have to have at least 2 places to keep your media. There are no exceptions here.
If you try to skate by having only one hard drive, and something happens to that drive (somebody has butterfingers and drops it or there’s a power surge, or it just fails for no reason which is a thing—who knows what!) then your movie is gone. I know this seems like common sense, but I couldn’t call this The Ultimate Guide if I left that out.
And three copies is highly recommended—two backups and one main copy. Why? Because Murphy’s Law applies to hard drives too. Ideally, you have three backups, in at least 2 separate locations (so that if one hard drive is hit by a fire or flood or theft, the other one will still be safe).
Pixar nearly lost all of Toy Story 2 when their main system and their backup were lost. They didn’t fail at exactly the same time: the backup failed at an unknown point in time, and they only discovered the issue when their main system failed. It happens!
After verifying that you have (at least) two independent copies of the footage, your Media Manager will then be able to hand the card/drive back to the AC and tell them without reservation that the cards are ready to be formatted and used again to capture more footage.
Here are a few other simple rules for the on-location life of your footage:
Quick Dos and Don’ts of On Set Media Management
- No Drinks Near The Drives. Ever. If someone puts a drink on the table you’re setting media up on, kindly ask them to remove it. Set up a sign that reads “Media Only – No Drinks”. Otherwise you’re asking for trouble.
- Keep Your Other Programs Off When You’re Transferring Footage. You can keep from checking facebook for a few minutes. Your processor will work faster if you don’t have anything else open, and more importantly, nothing else will freeze your computer resulting in a pinwheel of death situation that might corrupt your footage. You don’t want to lose footage because you wanted to binge watch Black Mirror while you were copying media. Do that on your phone!
- Mark Your Power Cord. Wherever it is you pull power from on your location or set, follow it back to the source and take a little bit of gaff tape and label it “Media – DO NOT UNPLUG”. Sometimes your DP or gaffer might need to repatch power. If they do this to your powered drive in the middle of a transfer it could conceivably corrupt the footage on your media. So do your best to make it fool-proof and label your power.
- Don’t Transfer Footage In The Car. Ever. I know, you think you can play catchup on your way to the next location so you get that first card done and you can get some actual sleep after you wrap instead of spending 2 hours managing media. But you’re one speed bump away from losing footage. Just don’t do it.
Where Do You Put All That Footage?
With file sizes increasing astronomically (4K, 6K, 8K etc…), TBs of footage after a few days’ shooting is becoming the norm for many. So what options do you have for the storage of these veritable mountains of ones and zeros?
Communication Between Post and Production is Pivotal
In order to decide about hard drive space, as well as card/drive media for daily use, the Camera Department, Production Department, and the Post Production team need to have a conversation prior to the shoot beginning. Loop in your Media Manager as well so that everyone can understand what codec and camera settings they will be shooting in and what amount of footage the day’s shoot is likely to generate.
This will affect how many cards/drives you need and what hard drive solution you choose. I was on a pickup up shoot for a pilot on a major network last month, and on the first day, the DP told the AC that they would be shooting 2K. The previous shooting had been in 1080, but a conversation with the Post Supervisor had been had and they’d decided to do this switch for the pick up days to maximize reframing options and the color space for a tight turn around in the edit.
So the DP and the Post Supe were on the same page, but neither department spoke to the Production Manager, and they had only rented enough CFast cards in anticipation of shooting 1080 like they had on the previous shoot days. It became quickly apparent that shooting 2K was not going to be possible for the remainder of the shoot. Then production spent valuable shooting time dealing with a media issue that could have been worked out in a few emails before production started.
Moral of the story is, always make sure that all departments are on the same page with media storage. It is pivotal.
If you yourself are the Production Department, the Camera Department, and the Post Department, then do your research! Every camera out there will have specs on what setting produces what amount of footage. Then do the math. Try to figure out what your average amount of daily footage will be, and then purchase drives to fit that amount.
Hard Drive Options
There are two main questions you’ll have to ask yourself when choosing a hard drive. How much money do you have in your post budget and how much footage are you going to have? This will largely determine what you can afford.
When a film is out on location, it is customary to have “shuttle drives” that your Media Manager will use in the field. These drives will take the day’s or week’s footage and then they will be shuttled physically to Post Production so that the Assistant Editors can offload them onto their main editing system.
Usually a shuttle drive is a hard drive that is more ‘rugged’ than the one you’ll be using on your edit. How large a hard drive you get depends on how often you are planning to shuttle it to your post department. But one thing you’ll want to do is get a hard drive with as high an RPM as possible.
What is RPM?
RPM stands for Revolutions Per Minute and the higher your RPM, the higher the speed of your hard drive. If you want to avoid long copy over times (and paying your Media Manager expensive overtime) then pay a little extra to get a higher RPM. For footage transfers, a 7,200 RPM hard drive is fast enough for most situations.
Now I’m willing to bet there are a few of you out there already looking at your budget and saying to yourselves that there is no line item there for the luxury of a Shuttle Drive. You’re not alone…
If Your Shuttle Drive Is Also Your Edit Drive
Let’s face it, this is not ideal. You don’t want to be lugging your entire movie around with you to every location. It’s not ideal or recommended to keep your entire movie with you on location because the more time your hard drive spends on the move, from location to location, the more chances there are for something to go wrong.
But on smaller budgeted shoots, it is often a necessity. So if you are planning on editing your film on the same drive that your Media Manager is using on location, let’s talk about your best options.
The Budget Travel Pick
If you’re going to be on the move a lot, our Travel Pick would be an external hard drive from Glyph Rugged. Glyph is a company I’ve used over the years and rarely had issue with. Depending on the type of camera you are using and the frequency that you’ll be offloading these drives to the Post Department will depend on what size you might use. There are 2TB and 4TB versions that can work great as shuttle drives.
And this series of Glyphs are 7,200 RPM vs the Lacie Rugged that tend to be 5,400 RPM. And if you’re editing off of your shuttle drive too, and using proxy files, the USB C connection should be able to handle it (as long as the processor on your computer can).
Don’t Forget The Proxies!
Chances are, you won’t be editing straight from the raw files you’ve shot. Editing at full resolution is a luxury that many productions don’t have because it requires intense processing power from your computer. So you will be making proxy files that are of a lesser quality so that the editing programs and your computer can allow you to edit in real time.
These proxy files do take up space. And you need to anticipate that when you’re purchasing hard drives. You want to make sure that you have enough space not just for your daily intake of Raw footage, but that you leave room for the Proxies.
And so I reiterate, have discussions with your Post Production team and your Camera Department. Take their recommendations seriously. And if you are editing your own footage and are the de facto Post Dept yourself, then do your research. There are camera tests you have to run, Creative Cow forums you have to stalk to determine what sort of space your proxies will take up if you shot, for example, 4K RAW and are looking to edit in Prores 422 HQ.
RAID Hard Drives or… What is RAID?
Raid Storage is one way that you may want to store your footage and edit off of. There are several types of Raid Arrays, and it is a complicated choice to decide what fits your production. But if your film has enough budget and you are not simply editing off of your shuttle drives, then your Editor will likely consider a Raid set up.
What is RAID, you ask? A RAID Drive is a way of putting many hard drives together so that they respond on your computer as one drive with increased speed and capacity. So you essentially have 2 or more hard drives bundled together. RAIDs are awesome: depending on the type, they can give you extra transfer speeds, allow you to combine multiple drives and/or help protect against drive failure. However, if you’re using RAID’s for backups, keep in mind a RAID only protects against drive failure, which is only one of many ways that you can lose data. A RAID is still one device, so it (and two backups) can fail at once.
Here are the various RAID types along with the specific advantages of each:
RAID 0 – The Fast Choice
Raid 0 splits up information from your files across all of the drives in the Raid Array in a process called striping. Having data striped across several different physical drives means your computer’s processor can access them all simultaneously.
This increases the read/write times of the drives since you have 2 or more drives working at the same time on the same task (and when playing back 4K video, a fast read time is one thing you’ll need to view footage without hiccups).
RAID 1 – Making Backups Simple
Raid 1 is a simple mirroring of information between hard drives in your Raid. Any change you make to one hard drive will be reflected on the other. This makes for redundancy and is essentially a working hard drive paired in one housing with its own mirrored backup.
So if you have an 8TB Raid 1 Array, it will actually be 4TB for your edit drive and then 4TB for the attached mirrored copy.
There are pros and cons here. Raid 1 does not increase the read/write times of your hard drive array like Raid 0 does. It is simply a backup copy.
Also, it is safe to have a mirrored backup of your edit and all the footage in a Raid 1, but many editors would prefer to keep their backup on another hard drive so there is physically another copy in the event of butterfingers dropping the drive or some sort of power surge catastrophe.
RAID 5 – Safe and Speedy!
Raid 5 also uses striping, but to a safer degree than Raid 0. You need at least 4 drives in the array for Raid 5 because it uses striping to spread out information so you can access it faster just like Raid 0. But the important difference here is that it also spreads out a backup.
So if any one of your hard drives were to fail, two of the others would have the data backed up and could reconstruct it. This obviously means that some of the storage is earmarked for backup, but it is a combination of striping for speed and backing up for redundancy that makes this system a solid and extremely popular choice for video editors today. However, it costs more. Also, note that it can be somewhat volatile. Because of how RAID 5 works, double failures (where two disks have errors simultaneously) are easy to miss until it’s too late and your data is gone. If you are doing RAID 5 make sure to have another copy on another drive.
Raid 5 – Travel Pick
One Raid system that comes highly recommended is the G-Tech Shuttle with thunderbolt 3 connection. G-Tech is a company that has been in the game for a long time, and have proven reliable.
There are a variety of TB sizes you can get with this option, so figure out how much space you’re going to need for the Raw footage and the Proxies that you’ll be editing from. You can get these drives from 16TB to 48TB.
The drives inside will be at least 7,200 RPM and you can set them up for 10,000 RPM as well depending on your playback needs.
And this G-Tech Shuttle is built to do just that – shuttle around. It is sturdy enough to bring you to and from your locations if budget restraints mean you can only afford this drive and not a few of the rugged Glyphs to card around your raw footage. So take that into consideration when thinking about the option that best suits your production.
One downside of a Raid 5 system such as this is that if a failure happens on one of the drives, it can take a good bit of time (a day even) for the other drives to rebuild the footage contained in the failed drive. This is time that your editor can’t be editing. But it’s better than losing the footage!
Budget Backup For Raid Systems
If you do decide to put your budget into a high quality Raid 0 or 5 for speed, you might consider having an inexpensive hard drive with a low RPM to serve as a physical backup. This won’t be one you can edit on, but if you had a failure on your Raid system, you could rebuild it by transferring the footage back to it from this other backup.
A series of economical 8TB drives from Western Digital work well for this purpose. If something were to happen to your Raid, once you replace whatever the faulty drive is, you can copy over the footage from this cheap backup to your new Raid Array.
It is a fairly inexpensive way to have piece of mind that you have everything backed up. It does take time to reconstruct a drive, and it’s never something you hope you’ll have to do, but this can be an affordable way to prepare for it.
NAS Storage Possibilities
For those of you with a larger budget that are editing in either a post production facility or office, you may consider NAS storage. Network Attached Storage is a system whereby you set up your hard drive to be accessible by multiple computers at the same time over a shared network.
If you have multiple editors working at once, or have tight deadlines and need your assistant editor to be working and organizing at the same time the editor is cutting away, then NAS storage might be what you’re looking for.
And if you plan on editing with 4K footage, many Nas Systems can work with it using the incredibly fast 10 GbE connections. This is another expense that not all productions will be able to afford, but it is nice when you can.
Our Luxury Pick – NAS Storage
Companies like Synology and QNAP are at the forefront of this kind of storage solution. The QNAP TVS-1282T3 12-Bay NAS Enclosure is a behemoth of storage potential that can give you a solid video editing connection over 10GbE and thunderbolt. Its 12 bays can be filled with hard drive sizes of your choosing to accommodate for as many TBs of footage as your production needs in whatever Raid configuration you choose.
Synology’s DiskStation DS 1817 Nas system is incredibly customizable. It has 8 bays which can be built out with drives of various size to give you up to 80 TBs (and it can be expanded with 10 more bays for up to 180TBs of storage!).
The DS 1817 also sports two 10 Gbe ports, and that speed will allow you to edit 4K files as long as your computer is fast enough to keep up. And we’ve heard that Synology is easy to use without an IT department, and that Synology has responsive and helpful customer support system when needed.
One other huge advantage other than speed with Synology Nas Systems is that they can work seamlessly with cloud based storage company Backblaze to offer your production an unlimited and cost effective way to backup your footage.
Our Cloud Based Pick – Unlimited Space!
Unlimited!? Yes, that’s right! Unlimited storage can be had if you look to the clouds. Of course, speed-wise you can’t edit from cloud footage (yet!) but it can be great for backups. There are several companies out there that offer a service to provide productions unlimited capacity for their video footage.
If you want to set up your Nas system as a fast Raid 1 or 5, but also want to have a complete backup of it accessible, you could sync it up to cloud based storage for additional piece of mind.
There are a few options out there. The cheapest are Amazon Glacier and Backblaze. Blackbaze is our pick, as a company that has been at the forefront of online data backup for 10 years. They offer B2 Cloud Storage where you can upload terabyte after terabyte of 4K footage and it won’t even blink. It offers storage at $0.005 per GB to upload and then $0.01 to download it again (in the event that you needed to rebuild a damaged Raid, for instance).
And Backblaze’s software is quite simple to operate, and can be integrated into Synology Nas Systems with a few clicks. So your DS 1817 80 TB system can be set up in whatever Raid configuration you want and everything on it can be backed up onto your B2 Storage Cloud for safe keeping. Keep in mind that, depending on your internet connection, cloud backup can take time both for uploading and downloading (you can use these formulas from our friends over at frame.io to calculate the download time). If you can’t download your files quickly either because of their file size or your internet speed, only use cloud backups for long-term archiving.
Each Project Has Its Own Challenges. Choose your own adventure
Every film you make is going to have a different solution regarding Post Production and backing up. There is no one answer, and every option has tradeoffs.
But understanding the limitations of budget and personnel can help you evaluate your own situation to determine what hard drive solution is right for you as you manage your media through the life of your project and beyond. But always consider the value of the files, and handle them with care and advance planning!
Did you find what you were looking for? Make sure to subscribe to our newsletter to get resources and articles like this one delivered to your inbox every week!
