Need a better solution to my Home Theatre PC

Networking Networking Hardware
1

I am hoping that the users here might assist me in finding a better solution to the storage of my movies for my Home Theater system?
Here is what I have at present:
I run a Windows 7 environment using a 2-PC network utilizing Windows Work Groups, the network uses CAT 5e cable running at a speed of 1-GB.
The system hardware configuration in my desktop computer is running Windows 7 professional with an FX 8350 CPU a Samsung EVO SSD and a 2-TB Red hard drive.
The HTPC is running a light version of Windows 7 with a low powered Intel processor, a small SSD and a 2-TB Green hard drive.

How I do things now, all of the movies come into my workstation where they are cracked or decoded then a the files are sent to the HTPC in groups and are listed there for viewing. If I decide to keep the movie it is moved to a sub-directory
within the HTPC such as (Action, Drama, Sci Fi etc.)
In addition I maintain a copy of these movies on my work station for back up purposes.

The issue that has developed is that the 2-databases are not balanced as sometime I might have a movie in action on the HTPC and in Sci-Fi on the workstation. This is due to the time frame being used to update the workstation, often days later. I have no other back ups other than the two separate hard drives which are close to being Identical. I have used a program for Synchronization in the past but this has caused many issues which need many hours of work to try and get the two databases close to the same. The size of my current movie files is 922-GB's.

What I want or rather need is a better solution, a 2-Bay NAS with RAID 1 might be better and some way to integrate
a hard drive back up solution that is automated.

I will gratefully receive suggestions and comments, thank you!

2

A FreeNAS box maybe, FreeNAS with ZFS is probably the best solution, and you can run a PLEX server as a Plugin.
http://www.freenas.org

3

Rather than storing your movies in genre folders it might be better to store them in a single folder and use something like kodi or plex to handle organisation. That will make it easy to sync. Speaking of, bittorrent sync and it's equivalents are an easy way to keep files synced over a network.

Personally for a media library I use snapraid for redundancy. The advantages are that you can add or remove disks to the array without having to start over, you can start with already filled disks, and if too many disks fail that it can't recover from you only lose the data which was on those failed disks. The disadvantage is that it is a type of snapshot redundancy, rather than real time. This means you have to run the sync manually (or you can automate it) and it will only be able to recover up to the last sync. This is fine for static files (like a media library) but you wouldn't use it if you're modifying files often.

4

Thank you thomas9151 & Dexter_Kane for you comments and suggestions, it is much appreciated.

I in the past, as in several years ago used XBMC on my HTPC but moved away from it do to the simplicity of a file structure to find the category of movies that I wanted to watch. Based on the comments I think that tackling my issues will require a two step process. The first is getting my movie collection cleaned up and synchronized, as a result I will give Dexter_Kane solution a try.

The second part is going the way of a NAS box or even a Drobo solution but not being a computer Knob I will need a bit of hand holding in trying to get this right. From what I have read building a Linux based NAS box running ZFS would be the best solution however I am not sure of how this would work in my set up?
With a NAS set up my HTPC would access the movies on the NAS through the network so that Kodi could play them, is that correct. If I had 3 drives in the box could I have 2 or them in a RAID 1 configuration then the third drive available for direct backup of the movies? This would be faster than doing that back up through the Gigabyte NIC?

Thoughts please?

5

Ya, FreeNAS+Plex

But how much did you want to spend?

6

yeah, Kodi can access files over a network, I have my HTPC set up this way.

Yeah that would work. But given the use case it's probably not necessary. For a media library you don't really need the availability of a RAID 1 setup so just having a storage disk and a backup disk is really all you need (Especially considering that you have the DVDs/Blu-rays or whatever anyway)

On my NAS I run linux with snapraid for redundancy and AUFS for disk pooling. For a media library I really can't recommend this setup enough. It has the minimum amount of risk of data loss, it's highly scalable (you can add new disks at will without rebuilding the array) and using AUFS there's no performance loss with disk pooling. Here's a comparison which may help you in choosing your storage method for your NAS http://www.snapraid.it/compare

ZFS is also good, and I'd recommend it over RAID because of it's protection against bit rot and data corruption. But you will have to plan it out as you can't add new disks to an existing array (You can however join a new array to an existing one).

BTRFS is also an option, it's similar to RAID1 but with protection against data corruption.

7

As a person retired and living on a pension I have to watch out for my budget, however I think around $500.00 US would be my Ouch point.

8

Also if you manually download your media you might want to check out sonarr or couch potato. They will allow for almost fully automated media acquisition.

9

Hello Dexter, thank you for the response again.
At the present time I am working on getting my two hard disks identical. I have installed kodi-15.2-Isengard on both my desktop and HTPC. In addition I have as you mentioned placed all of the movies into one sub-directory removing them from the file structure I used previously. I intend today to try Syncing the two drives to see where missing or misplaced movies are and correct that. Once done I will be ready for the next step whatever that might be.
For me I want to keep things as simple as possible so following what you have done and that works well for you is my preferred option.

Regarding RAID and its variants I have some knowledge here as in the distant past I ran RAID level 5 in a SCSI environment with a high end RAID controller. Things have improved much over time with software RAID.
I have I feel two solutions one is a pre-built Synology solution or build myself a NAS box from scratch.
As mentioned I want to keep this simple however the building option has the greatest future protection built into it.
As you are very happy with your solution I would like to follow a build path similar to you build.
Can you please give me a detailed configuration of what your NAS box is like? Also I would like to know if you are using EEC memory in your environment?

Thank you again for your assistance.

10

Thank you for the recommendations.

11

Welp, you could pick up an 8320e + asrock 970M board for like 200 bucks and then just run freeNAS with a plex plug in, and your network storage could just encode things itself since it would have more performance and all, generally you want ECC for freeNAS, but for a media server it should be fine

12

some FreeNAS info,
FreeNAS is based on FreeBSD, and RAID with ZFS, AKA RAId-z, works like this,
you add disks in a array called a [vdev]
and when you add more disks, its to a new array AKA [vdev]
and the [vdev]'s gets striped together.

     Mirror			 	                 Mirror
   disk1 disk2                       	              disk3 disk4
     [vdev1] --------------------stripe---------------- [vdev2]
        |						   |
        |						   |
        |						   |
        ----------------------------------------------------
	    		            |
                                    |
		   	        the pool
			            |
			            |
		             Folders\datasets

but your [vdev]'s have to be identical in number of disks, size, and RAId-z level, so it just takes som planning ahead, like don't use 500GB HDD's, and i would recommend RAId-z2 if your data, is irreplaceable, because when a disk fails and you go to replace it, if you remove the wrong one you could lose all your data,

RAId-z levels
RAId-z Mirrored add 2 disks at a time, and you can lose 1 disk per [vdev]

RAId-z1 add 3,5,9 disks at a time, and you can lose 1 disk per [vdev]

RAId-z2 add 4,6,10 disks at a time, and you can lose 2 disks per [vdev]

RAId-z3 add 5,7,11 disks at a time, and you can lose 3 disks per [vdev] not recommended

13

This is pretty awesome, expensive, but awesome.

-17x SATA3 (6Gbps) ports
-Intel Atom processor C2750 20W 8-Core
-Supports up to 64GB DDR3 ECC
-Dual Gigabit LAN
-IPMI

Alternatively


14

My set up is a little complicated, I have two servers with storage spread across both of them but shared from a single network share.

This is the most recent one

https://forum.teksyndicate.com/t/build-log-silverstone-ds380-nas-build/69766?u=dexter_kane

It's a little overkill and does use ecc ram but only because I was unsure about compatibility with non ecc, not because I need it.

I this machine I have two data disk and one parity disk, but eventually as I need more storage I will have six data disks and two parity disks. This is one of the reasons I like snapraid so much is the you can add or remove disks arbitrarily rather than need to add multiple disks at a time or rebuild the array. The other big factor for me was that if too many disks fail I will only lose the data on that disk, in raid and zfs you lose everything. I can also restore deleted files so it's sort of like a cross between redundancy and backup.

I use AUFS for disk pooling, in this set up the disks are all independent so they can be added or removed from the pool and files on the disks can be accessed directly as well as using the pool. This is how I'm able to spread my storage across two machines.

15

Hello Dexter_Kane
Yes your NAS server/servers are a lot more than what I am looking at.
I am looking at this project from a more basic utilization point of view, what I mean by that is that the only requirement that I have is a repository for my 1TB of movies that I have saved on a 2TB hard disks. Based on our discussion you had mentioned that as the database of movies are stagnant other than as read files, thus duplication should not be necessary. However as I will not be having an identical copy of the movies on a second drive I will need a backup process for safeties’ sake.
Regarding the building of a NAS system I want to keep the size of the case as small as possible. I have several cases available to me here in SE Asia and they allow for a minimum of 4- 3 ½” hard drives. Based on your comments I should need only two drives in the system one for data the other for back up or ??? As my requirements are low I could get away with a micro or mini AT board with a medium powered CPU. My thinking regarding Ram for the system is 8-Gbs.. Regarding a boot drive many people have mentioned using a memory stick for the Linux operating system but failing that a small SSD will do the trick.
I have some parts around the house that I can use however for the most part the parts would be new. What Linux operating system would you recommend? As you had suggested Snapraid and AUFS, I would like to go with both of these products.
This is where I am at this point in time, the synchronization of the two drives is now complete and both of the 2-Tb hard drives are identical in regard to the movies that they contain.
I have a question regarding Snapraid as an application, as the hard drives are not hardware linked by hardware Raid can one use different brands and sizes of hard disks within Snapraid and run a RAID level approximating RAID 5 ?
Final comment, please give me feedback on what I have mentioned regarding my hardware requirements. You do not have to be specific as to motherboard of CPU as I will work that out but rather what you would consider a decent NAS system based on my requirements.

16

Yes, you can use different brands and sizes of disks, even different file systems. The only requirement is that the parity disk is as large or larger than the largest data disk. Snapraid is similar to RAID 5 and 6, it supports up to six (I think) parity disks. At a minimum you would need three disks (like a RAID 5) two for data and one for parity.

The hardware requirements for something like this are pretty low, you don't need a great CPU (intel atom or celeron or equivalent AMD chip) and even 4GB or RAM would be heaps. You can run linux off a USB stick but an SSD will be faster, it's up to you if you want to spend the money on faster boot times or not. On my server I use ubuntu server, I know that AUFS doesn't work (or atleast is hard to get working) on fedora based distros so I'd stick to something debian based.

If you want to use snapraid you will need three disks at a minimum, but if you think that you will only need 2TB of storage then just having two 2TB disks and backing one up to the other will work fine too. For a media library I'd use backups rather than a RAID 1 configuration as you don't really need the availability of RAID and having a backup means you're protected against more that just disk failure (accidental deletes, data corruption etc.) For a two disk setup you could also have a look at BTRFS on linux, it's similar to RAID 1 except it has integrity checks and supports snapshot backups similar to shadow copy on windows. There's a NAS OS called rockstor which uses BTRFS that might be worth having a look at for a two disk NAS build.

Yeah, my set up is a little bit silly to be honest, but it does demonstrate the scalability of using snapraid and AUFS, which for me is the main reason I use it over something else.

17

I want to thank everyone for their suggestions and their thoughts regarding this project it is very much appreciated.

1 Like