UNRAID NAS Project

 

 sam9s NAS project powered by UNRAID!!

Hello, friends welcome to Sam's NAS Project Thread.

So to begin... The entire NAS experience would be divided into three big parts.

1. Introduction and Comparison
2. Quintessential Features of NAS
3. Online Access and Security.

@@@@@@@@@@ - PART 1 :: Introduction and Comparison- @@@@@@@@@@​

As you must be knowing (at least the regulars here) that I have been working on my NAS project for a pretty long time now, and though it was up and running a week back, tweaks and upgrades made me stop posting my thread, as I thought it would be better to post it when the entire array is running with everything.

So finally after more than a month of Research and Development, I have my NAS ready, with all the "actual" features of NAS and not like that were been projected by some..... in the other thread...

So here we go ...... Its gonna be a long long long seriously long post/thread  so bear with me, but people who really are interested in building up a DIY NAS with a very decent affordable price, kindly take the pain to read everything...

**********THE HARDWARE**********

NAS needs even lesser powered hardware than an HTPC and I have been screaming about the same where ever I can ... . You DO NOT need i3sss, or i5sss not even the C2Qsss, A simple early generation C2D is more than sufficient.

This is how I chose my hardware.

1. Processor:: As I said you dont need a beefy processor for a NAS. Anything post-Pentium 4 would do. I chose the oldest of a Pentium Dual Core processor E2140

2. Motherboard:: You need to be a bit specific in choosing the mobo as you do not need quite a few things that are there in current gen boards. What you need to be very sure of is...

A. It has at least 4 SATA ports, more the better but we can start with 4, and dont worry I am gonna answer the doubt "What if I want to add more HDD"

B. A Non-Gigabyte board if you are using the OS which I am using. The reason I will explain in detail later.

C. AHCI complaint, which usually all the Core 2 Due boards are, but still make sure it is there. (needed for Hot Swapping plus drive write speeds improve a lot)

D. At least one PCIe x4 slot.

E. Can boot from USB

All these are very trifle requirements, but nevertheless, they are needed and hence I am mentioning them. I went for MSI P43 NEO

3. RAM:: Any generic ram. 2GB recommended, 667/800 Mhz. Transcend is the VFM here. I went for Transcend 2GB 800Mhz

4. GPU:: No need for a GPU as the onboard GPU is more than sufficient. We won't need a display as such. The only time you might want to have a display is while doing the initial setup of the OS and BIOS or in a rare case web gui/Telnet not responding due to some network failure.

5. Cabinet:: This is the part where it would depend entirely on how high you wanna go. You can start with a basic PC case and go up to a dedicated 10-15-20 drive bay NAS case. I would suggest starting with a mid-level PC case that can let you add 4-5 internal SATA HDDs and also have 4-5 5.25" drive bays, wherein you can fit external HDD bays that can support 4 more HDDs. I went for Cooler Master Elite 334 PC case, that can support 5 internal HDDs and 4 external 5.25" bays which I can use to add another set of 5 HDDs. (using external Hard drive bays.) So in total in this case I can have 10 HDDs. Plus the case has 2 fans for cooling and the option to add one more. The HDDs cage has a screw-less design and overall case quality is very decent.

Later as and when you increase your array, you can go for dedicated NAS-specific cases.

6. A Gigabit LAN:: If your mobo supports it well and good else goes for an external one. I went for a D-Link DGE-528T Gigabit LAN card.

7. Hard Drives:: Again absolutely no need to go for 6Gbps HDD or SSDs. A normal 3Gbps SATA II 7200rpm/5400rpm drives are more than sufficient to provide the bandwidth for even BD ISOs. It's the network throughput that needs to be good. I went for WD Green 5200 RPM (basically the cheapest one)

8. PSU:: Another major component that needs to be quality, as your NAS would be running 24x7 in all probability and since HDDs are the only major thing that would be running you dont need a high-wattage PSU. To give you an idea, A 500W PSU would be sufficient to run the above hardware with 15, 7200 rpm drives.
I opted for Cooler Master Elite Power 460W


******** That is about it where the basic hardware ends as far as setting up a basic 5-drive NAS is concerned ********

If you want to upgrade and have further features like external Hot Swapping, additional hardware can be added to the above existing one. I will discuss that as well.


**********THE PRICE**********

Now let's just calculate the price one needs to spend to get a basic NAS setup, taking the above hardware as a reference.

1. E2140 Proc + P43 Neo ...... I managed to get it for 3000 (USED)
2. RAM 2 GB 800Mhz ............ 500 (NEW)
3. Cabinet CM Elite 334 ........ 2500 (NEW)
4. Gb LAN ...........................800 (NEW)
5. PSU ................................2900 (NEW)
Total =============== 9700

Only of course that's without HDD.

Let's add HDD. Assuming you are using the free version of the NAS OS that I am using, you can use 3 HDDs at the max. The Plus version lets you add 4 more hard drives....... but that's gonna set you back another ~3500 for the OS (you cannot have the pirate version for this  ....sorry!!).

Anyway, let's add the hard drives and calculate. With the free version 3x2TB HDD will give 6TB NAS that equates to 3800x3=11400 approx. Add this to my above price and we have a 6TB ready NAS for 21100.ONLY, as against to 30-40K NAS that was been suggested in the other model thread for NAS. And if people have read that thread 20K was the max I quoted for a basic nas with 6TB support.

This would give most of the resource-hungry people out there a decent start in the world of NAS. If you want more, of course, you have to spend more

******** Let's see what else we can add to the above-existing Hardware to make it a more professional NAS ********


1. Sata Add-on Card:: P43 has 6 SATA ports which would be pretty sufficient for most people to start off with, however, I am sure with time you would like to add more Hard Drives to your Array and that's where the PCIe SATA Add on cards come into the picture. SATA Add on cards let you add more SATA drives from 2 to 8 drives. Something like Add-on Card OC-SASLP-MV8. A bit expensive but worth coz you would have 8 more sata ports at your disposal if at all you feel the need to. I personally have not gone for this as 6 ports have given me 5 TB of space (2+1+1+1+2) (parity)). One 2TB is for Parity which is not used for data.

NOTE ::: I am using the PLUS version

2. Hot Swap Drive Bay:: Hot Swap bays as the name suggests give you the ability to add/remove/exchange drives on the fly right from outside of your case. However, your OS and BIOS both should be able to support it. My OS does and AFA Bios is concerned that's where AHCI comes that I mentioned above under the Mobo category. To support hot swap your Bios/Mobo should have AHCI support. Of course, AHCI is not just for this. It also improves the drive read/write performance. I bought just one Hot Swap bay as I just wanted to have a way to copy data to my Array via internal sata drive instead of LAN (because of the speed of course.)

Vantec 4 Easy SWAP Demo - YouTube

3. NAS Case ::: When you add the sata port addon card you also would like to have a dedicated NAS Case, that can give you more space and ease of expandability. Though I personally feel for a Home NAS, a decent PC case with an external HDD bay is more than enough you need. However, for people who do wanna have something extra, you can choose one of
THESE. You can also go for something like Antec 1200, it's a huge case and can accommodate 4x5HDD cages giving you a capacity of whooping 20 Hard Drives.

**********THE OPERATING SYSTEM**********

Now comes the most important question......Which Operation System????

Two broad categories

1. Windows
2. Non-Windows.

~~~~~WINDOWS~~~~~

Windows has its offering called WHS (Windows Home Server) in the NAS/ Media Server segment, but frankly and personally I did not want to go for Windows. Two main reasons,

1) Huge, just like windows 7, so in compassion difficult to maintain. A separate complete HDD or partition for it. etc etc

2) It uses mirroring for data security, so this means you have to have double the space of your data to make it secure. One disk protection for one is not worth it unless you have mission-critical data in which case some people go for even triple replication.

There are other very technical reasons as well, which I would be comparing in a moment as those reasons were also the reasons I did not opt for another non-windows OS.

Also, WHS only supports RAID 1 and 0.

~~~~~NON WINDOWS~~~~~

It was a fight in here, with loads of options to choose from. Few are completely free and few offer only the basic version free. Let's see what were the options to choose from. We will try to compare them extensively as well to get a better picture in comparison.

Let's first list what were the main criteria that I needed to have in a Non-Windows OS for my NAS. (these points are merely from a layman's perspective) as when I researched more, there were more technical reasons I found that made better sense to choose what I finally went for.

1. First and foremost being a Linux Layman, it should be as easy as possible to configure.

2. Easy expandability, it should not be difficult to expand my storage whenever I want.

3. Easy swap-ability.

4. Secure

Following were my options ...

1. FreeNAS
2. FlexRAID
3. Free BSD
4. Openfiler
5. Unraid

My initial choices were FreeNAS and FlexiRAID, but after extensive research, I went for UNRAID before I go further let me tell you that I was/am so impressed with UNRAIDs performance that I finally bought the Plus version. (Details in a moment).

Let's have a basic comparison between the Operating Systems

==========FREE NAS==========

The most common and the most known solution for a Home NAS is a Free NAS especially when it's free., but when I did an extensive googling, I found many aspects do not go in favour of my requirements listed above.

PROsss

1. Free NAS was pretty close to Unraid when it came to ease of installation.
2. Very low resource hungry even lesser than Unraid in this aspect
3. Uses ZFS filesystem which is much faster than ReiserFS that Unraid uses.
3. More protocols to share than unraid
4. Since its Opensourse is Free so no limitation on any of its features.

However, there are more major cons than pros, especially with the kind of Home Media Server I wanted (in fact most of us would want)

1. Configuration was complex. I tried FreeNAS on my VM machine (under Free BSD) followed a couple of tuts, but was totally unimpressed by the interface and the configuration.

2. FreeNAS is inflexible, and not very forgiving if you have two simultaneous disk failures, as it uses a more traditional RAID setup (RAID5), which uses stripping. This means if you lose 2 disks you lose the entire Array. This is not the case with Unraid.

3. Difficult, to expand.....As I said it uses striping of data across drives, expanding is not as easy as just popping any size drive into the array and start using it. Yes, you can add hard drives, but what you need to understand here is, ..... once the array is constructed (under Raid 5) and running you can not add further hard drives to it. If you want to add more disks, you need to create a new array. This is not the case with Unraid.

==========Flex RAID==========

FlexRaid is another freeware NAS OS based on Linux. It's very much like being unraid when it comes to how the data is stored. FlexRAID also does not strip data across drives unlike FreeNAS, which has its own advantages and disadvantages as well. I was about to go for FlexRAID when a few of the features put me down. e.g.

1. Though it works on parity like UNraid, it's actually a snapshot-based system. Meaning that the way it takes a backup is by taking a snapshot of the parity, so unless the parity of the files does not change you are ok. For this to happen properly you have to refresh the parity every now and then (isn't automated and hence called Snap Shot). Media files do not change frequently to an extent I agree, but I still would like to have a more robust parity backup system like Unraid.

2. Second major disadvantage for me was, FlexiRAID is not an OS in itself, it is more like software that is installed on the host, so you need something like WHS or W7 to run FlexiRAID. This might be useful for those who do not want to venture into the Linux category, AT ALL, (for doing other server-based tasks) but still want a Linux-based OS for their NAS.

3. Does not have a WebGUI, needs command line management Advantages.

Major Advantage over other OS

4. Multithreaded.

5. Parity may be spread across multiple drives

6. Works on windows so easy to add features and other stuff for your server than Linux-based Unraid.

OpenFiler and FreeBSD--

Did not venture much into these two as I already had started to lean towards Unraid ..... but if people are interested google is your mate... 


==========UNRAID==========

Coming to what I finally chose. UNRAID, developed by Lime-Technology. gives everything that a home Linux-based NAS OS should give, bearing a few exceptions that still make it worthwhile.

Before I go on about why I chose UNRAID, let me introduce you to the UNRAID mega thread on AV Science Forum. If you do decide to venture out into this product then read as much as possible, I will also provide complete instructions on how you can install UNRAID............ plus links to various forum threads and articles that would be beneficial to you.

First, let's be clear few things UNRAID is not popular for, or shall I say let's point out the cons, and negatives of UNRAID first, which might provide a better insight into the comparison

1. I Unraid doesn't actually work on RAID 1,5, so not stripping here. the performance lacks for sure, the Read/Write speed is something that Unraid would not be proud of. Read/Write performance is limited to a MAXIMUM of single disk performance. It will be less because of parity overhead, depending on the system CPU and memory available. (Still enough to have BD ISO stream over a gigabit LAN). UNRAID is more like RAID 4 (but without the stripping part)

2. Unraid requires/uses ReiserFS which would not allow a disk with FAT/NTFS to be used straight away, it needs to be formatted per Unraid. Doing it the usual way is a long too long process, some scripts would do a "Preclear" on the drives and prepare the HDD for Unraid. Though the preclear process is again too lengthy (15 Hrs for 1TB drive), the good thing is....... it works in the background and you still have your Server for use in the foreground (unless you are preclearing your first drive, which is but obvious)

3. One of the major deal breakers for Unraid can be the fact that if at all you want to make your NAS Server do anything else apart from the basic Out-of-the-box functionality, like torrents, RSS Feed, Media Streamer, VPN...etc etc, you need to climb the somewhat steep learning curve of Linux, and its commands and how it works...... Sure you can just follow the steps and tuts, but not all tuts cover absolutely every troubleshooting steps
one can do...... in case things don't work or go wrong. So you got to have some level of knowledge of Linux OR have the tenacity, and the persistence to read, learn, implement and then re-read, and re-learn if that implementation does not work. If you feel that you are up to this, you gonna enjoy UNRAID like anything........as once configured it is a pleasure to work with.

**********What Unraid is Worth for**********

Let's come to the main point, what Unraid is worth. To start off with......I downloaded the basic version of Unraid which is free, test and see what it actually offers.

The major limitations of the basic version are...

1. Support for only 3 drives
2. User Share security is not present.

Apart from this it almost has everything that the pro version has. Detail comparison can be read HERE. Ok so off I went and downloaded the Basic version and followed the installation tutorial. (I will share that later) At that time I had only 2 disks, one 2TB and one 1TB. The entire installation takes place on a flash drive and the OS runs off the flash drive itself. I now am amazed that an entire OS that does so many things just runs off from a USB stick...

not take more than 512MB of space. The entire basic installation and configuration of Unraid took not more than 30 minutes. In fact, installing on the flash drive is a 5 minutes job, you then boot from the USB and are presented with the Unraid Linux command prompt.

The rest of the few configurations are done on the command prompt itself. Before you can use your disks in an Unraid Array, you need to do a preclear on the disks. Preclear is a utility to "burn in" a new disk and is done via preclear_disk.sh script. The advantage of the script is that you can run the process in the background,...... and if it's not the first disk on your array, you can keep using your array while the preclear is going on in the background.

Anyway coming at the advantages of UNRAID which makes it worth...

1. Very Easy install

2. Uses parity protection (think Raid-4 minus stripping), instead of the conventional

RAID 1 or RAID 5 for data protection. Unraid does not follow the RAID route exactly, it provides aggregate parity for individual disk fault tolerance and parity is calculated for all disks. This approach provides two major advantages...

A. First you dont need double the amount of disk space since it's not mirroring as in RAID 1. With this approach, 15 Disks can be secured using one parity disk (it should be equal to or more than the height capacity disk in your array)

B. If you lose any one disk in your array, the disk can be rebuilt using the parity drive, however in case of a two-disk failure, you still won't lose ALL of your data (like in a typical RAID-5 setup), you will only lose the data on the 2 disks. All other disks will be fine. With a conventional setup like Raid 5 if you lose two disks you lose the entire array.

(Reason being that the data is spanned across all drives, stripping is there. with Unraid Single parity and no striping allows for less chance of total data loss when two hard drives fail.)

3. Expandability ::: Expandability is the easiest and the strongest part of Unraid. Since the data is not spanned and stripped, every drive is individual on its own, not like one big hard drive of an Array as the conventional raid works. You just pick the HDD any size and throw it in the Unraid Array and it expands, Unlike Free RAID (Which works on conventional Raid 0, 1, 5 and JBOD) where more planning is involved when it comes to upgrading strategy.

4. Better Power management :::: Unraid aims to maximise useable HDD space, and as it does not strip files across multiple drives, they can individually spin down, down when not in use! or can be spun down as groups. Automated spin-down can also be set for the drives. A very nifty feature if you ask me. Some good threads to ponder on the comparison of Unraid with other OSs

unRAID or FreeNAS

http://www.avsforum.com/avs-vb/showthread.php?p=18784460

http://www.avsforum.com/avs-vb/showthread.php?t=1287508


**********Unraid Interface**********

Everything you need to monitor is done through the WebGUI of UNraid. Make sure to install, Unmenu (instructions in the tut), which is another plug-in for a better Web interface and experience. For the most basic functionality you only need to go to the command prompt a couple of times, like while preclearing the HDD, installing Unmenue, etc. Once all this is done and you have your drives in the Array, you dont need the command line. You manage your drives, shares, and permissions via Web Interface, plus all monitoring like disk usage, disk spin-up, and bandwidth, is also done through a web interface.

As I mentioned earlier, experiment only when you want extra features and want more out of your Unraid server, because you would have to venture to a steep earning curve of Linux on the command line. Even though it is no rocket science. Step-by-step tutorials are provided all over Unraid forums and we have one of the best supporting Unraid forums on the net. Now let's look at the web interface of Unraid. We access the web interface by just putting the ip of your Unraid server. Below is how the web interface looks for Unraid 5 Beta 11. If you plan to start with Unraid 4.7, the interface is different.

You can see how neatly it has been laid out. I have 4 disks (2+1+1+1) and no parity yet, which I am planning to get this weekend (another 2TB drive - Parity should be equal to or greater than your largest drive). The array is online right now, and we have spin up & down buttons that would immediately do the same if intended (individual disks can also be spun up/down which I will show in another snapshot). You can see a white blob in front of each drive except disk1. They are not white actually but are blinking white and green (were white at the moment I took the snapshot), meaning that disks 1,2 and 3 are not being used and are not spinning. Disk 1 is.

When you have to add a new drive to the disk, you need to shut down the server, plug the drive in the SATA port and boot it again. If you have set your Array to auto restart then you will see the array online when you boot back. 

You can see that I have disk5 and cache as well. Since I have the Plus version of Unraid I can add a maximum of 7 drives including parity or Cache. Depending on the way I manage. I can have 5 data disks, one parity and one for cache OR 6 data disk, one for parity and omit cache .......and likewise.

NOTE :::: Before you add the drive from the drop-down to the array make sure it is been precleared as directed in the tutorial. Once that is done. You just add the drive in the array from the drop-down menu. A format option appears and the drive is shown unformatted under device status. The format takes only a minute at the max if the drive is already precleared. When the format completes just start the array and you have expanded your Array ......... as simple as that.

The rest of the options on this page are self-explanatory.

Let's look at other tabs.

SHARES .......

The shares tab is the place where you create shares. The concept of shared in Unraid is very very different from the usual folders. It would be a bit out of scope and lengthy to explain the same here, so I am providing you with the link that explains it very nicely, and I do request people to read it carefully as that would affect the way you copy and access the data on to your NAS.


**********Unraid Unmenu Interface**********

Unmenu is the intrinsic part of Unraid and simply cannot/should not be avoided. It is an enhanced Web management page for unRAID that provides several user-requested features and is relatively easy to extend. Installing it now gives you access to screen and email notifications, both of which are useful for preclearing hard drives.

@@@@@@@@@@ - End Of PART 1- @@@@@@@@@@​

This would end  PART 1 of  Sam's NAS project powered by Unraid. Next in part 2, I will discuss, how I have configured the quintessential features of the NAS, like.

1. Hot Swapping.
2. ftp access to your server
3. VPN access
3. Configuring telnet
4. Using Putty


Part 3 will discuss one of the most important aspects of your NAS, Online access and security, which we would further divide into two parts


First:: NAS access across the internet
Second:: How to secure your NAS when it's accessed Online

Hope you have enjoyed and learnt a few things from this venture I have been working on for a month now, after my HTPC project. Thanks for having the patience and interest to read this through (I hope you did ). I shall try to extend my experience as much as I can if people do decide to DIY their own NAS based on UNRAID.

Stay put for the next parts ......

Regards
Sammy