Return to Level1Techs.com

Data storage for video assets - best strategy?

Hey Folks,

Been searching for recent case studies on video asset storage strategies. We are currently relying on Thunderbolt 3 to Promise Pegasus R32 units and adding arrays / boxes as needed, but is there a better solution?

@wendell did a piece on using our Canadian friends at 45 Drives - in this scenario, are you able to edit the video that resides on the Q30 or do you have to migrate it to the local machines, do your editing, then transferring back to the Q30?

We work with RED R3D which can get pretty demanding when it comes to throughput.

Curious what people are doing out there? TIA

Hey,

This may be a me, work thing, so: Are you a post house, agency, prod house, touring, or live events?

Your needs really depend on the number of editors and what kind of assets were talking about as well as your archiving needs.

So I will start by plugging our solution EVO Signature Series | Storage | Ross Video

We OEM it from SNS but its built and tuned for the type of news and content heavy productions our clients need. This is NOT a dyi product, its a professional one with professional 24/h support and known validated performance with both different codecs and editing solutions.

One of the common mistakes is that people think storage is storage and opt for IT based solutions over video options. While the core components might be the same, its all in the software and tuning of the systems. There is a reason professional video solutions cost so much, and why a consumer or even IT product might not be able to meet your needs.

//edit: also Ross is a Canadian company too if it matters to you.

Ottawa - cool! I’m just down the road in Kingston. Being Canadian is great but def not a requirement.

Hadn’t heard of the EVO Signature series - I’ve got some reading to do. Any pricing information? Quick glance at the website didn’t reveal any costs. Just two of us editing.

Yeah we dont do pricing on our site because most of our sales go through partners.

With Evo you would edit direct on the system, you would probably want 10g cards in your editing workstations and get a multi 10g interface for disk shelf. Ideally you would get more than just 2 so you can add editors in the future should you expend. Or not if you know 2 people will be it for the foreseeable future (5-10y) which is the normal lifecycle of a product like this.

So to start out with I would probably look at an 8 bay 32tb system which would be in the neighbourhood of 24,000 Canadian rupees. The challenge with Red is that its highly configurable so knowing the final datarate can influence the requirements but assuming your using 3:1 Redcode thats about 2Gbps per stream, and an 8 bay can do around 6gb so 2-3 streams depending on overhead.

You may want to visit the Linus Tech Tips YT channel and have a browse through their back-catalogue. They’ve frequently posted videos on how they optimised their workflow using high speed hardware/networking. For an appetizer, they edit the 8k and 12K footage directly from the company servers after ingesting it via their fibre-based network bone. They’re also upfront when they fail, so others don’t make the same mistakes :wink:

The number of systems he has built that did not work should tell you what you need to know about why professional systems cost what they do :stuck_out_tongue:

Also, you should see the issues they have with the Black Magic 12k cameras.

I’m no sadist! :slight_smile:

I’m only aware of one actual failed attempt (New Whonnock server, Wendell participated in the troubleshooting) all others basically worked but quickly couldn’t keep up with the requested work load and had to be replaced. Remember, Linus is “cheap”, he wants it for rock-bottom price so he has to do stuff himself. It also creates content, which is the primary goal of his channel. So, his server room is not just a tool to use, but also a content generator. And frankly, I love seeing him do server stuff, so he has my blessing :wink:

I’ll be honest - I don’t watch much of Linus. I did see his piece on the BM 12K and how he whined about the problems with the files in … wait for it… Premiere. I mean, really Linus? Use Davinci Resolve my friend. That’s why I’m here - @wendell is awesome.

1 Like

I enjoy watching too, the only issue I take is that people who are entering professional markets like content production, and live streaming dont always understand the nuance that you do.

Linus is cheap, he needs to push lots of content, and your 100% right. He also does not try to lead people down the wrong path, BUT its often not until his follow up video’s where we learn what did not work. So its easy to watch the first BMD 12k video and think man, these cameras are soo good he pre-ordered 10!

And BMD is often a good value for lots of people, but its decidedly pro-sumer gear. Its for hobbiest content creators or people starting a new profession.

Its generally bad money because the low up front cost is accomplished by 0 support (literally they wont fix anything, if your in the warranty period they just send a replacement) unfinished features, or in some cases bugs that will never be fixed.

Their biz model is to mass produce low cost high volume specialized equipment with a goal of making it last at least 1 year.

If you look at all his storage solutions its the same sort of thing. He keeps trying to roll his own (fine) but he makes all sorts of assumptions and does not put the time and research into it before buying (or asking for) all the stuff throwing it together and finding out it wont work.

First he pushed Unraid, great its fine for bulk nearline/warm storage but the level of redundancy is not sufficient for most commercial applications. Then once he trusted Anthony and Jake he migrated to a custom built ZFS setup which has been better but could not scale.

Then they tried that NVME setup and to be fair, it should have worked and that is on Intel. But because the rolled their own and there was no certification or validation so they got burned. I honestly felt bad about that one because its not really his fault.

Now they are on a less custom more validated setup which is working well for now but it will still suffer from an inability to scale once they start running out of space.

So here we are, @Bumperdoo is clearly a professional who is using a high end camera and editing and needs storage.

They see an interesting video on 45 drives and ask if this would be good for content creation because the youtube channel that made the video is using it for storage.

There is nothing wrong with 45 drives, my point is that not all storage is equal. Even if the core parts are the same. Companies like ours and like Studio Network Solutions have spent decades validating and testing to be able to offer guaranteed performance. We have actual support structures that are designed around the specifics of our industry so we understand what it takes to hit an on-air date.

Most of our clients dont get to “fix it in Post” because everything they do is live to air/web so we dont get a chance to fix things.

That is what people pay for when they buy our solutions, the experience, the reliability, the support, and the comfort of mind that when you go live all your mission critical equipment is ready to make your production error free.

1 Like

I think what my research says so far is this - continue down the TB3 path for now… IP is interesting but tough to match TB3 for sheer throughput. Promise has a long history in the business of making really strong gear… they’ve supported me many times over the past decade.

I am going to toy with adding an Optane drive as a Level 2 cache for the TB RAID arrays - could be interesting… man, Optane is pricey spicey sauce … should be interesting. Will hopefully have some initial results in a week. I know no one will sleep till I post them? :slight_smile:

1 Like

With just 2 of you that is probably a good call. I only brought up our solution because I know how heavy a REDCODE can be.

And i was not sure if you were looking for a longterm solution with the ability to scale with your productions.

Plenty of options out there, but IMO spend the extra $$ to get things that are designed for video editing. IT/Prosumer products just dont tend to cut it.

1 Like