RAID 1 set-up, USB vs. NAS, and other considerations

[Uncwilly]

This is not exactly a GIMPS related hardware question, if the mods want to move it to the lounge or where ever, fine.

Having had a failure of a device and also having a lot of media files, etc. I think that it is time for me to invest in a proper back-up and general storage device. Here is the situation:

I plan on buying a new laptop (even if I can get the old one up and running, it is a 2008 model) and am looking at getting a SDD vs. HDD. My old machine had 500GB on-board. Much of that was taken up with music (not even close to 1/2 of my CD collection was on there), photos, audio projects, and video projects (again, I could easily fill 2 or 3 TB with projects that I am involved with). I don't think that I need to invest in a SDD that can handle 2 TB. I should off-load most of the files to a device that I don't tote around.
Also, I would like the device to handle back-ups of the data from the laptop.
And after the current failure mode, I am thinking of a straight RAID 1 in a single device.

So, I am thinking about USB (or FW) solutions or maybe a NAS device (would be nice to have my files available to my media players in the other rooms. I have seen some devices that have a MyCloud (or similar) function, where the user can access their files on their home network remotely.

I would like to have a single device that handles the situation, not a group that relies on external action. Looking for a good blend of size, cost, and easy of use. Would like a relatively small form factor and not to noisy. (If a mini PC is the best set-up, I might take that option).

Also, any preferred back-up softwares?

[kladner]

Also, any preferred back-up softwares?

Check out Cobian. It is free and flexible.

[retina]

RAID is not a backup solution. Repeat that to yourself 100 times.

If you overwrite/delete something from a working RAID system then those changes will be replicated across all participating volumes immediately.

RAID is an availability solution. Nothing else.

[kladner]

Originally Posted by retina

RAID is not a backup solution. Repeat that to yourself 100 times.

If you overwrite/delete something from a working RAID system then those changes will be replicated across all participating volumes immediately.

RAID is an availability solution. Nothing else.

The author of Cobian has a mild rant in an FAQ against the idea that backups should mirror the source drives. Cut short, "It is stupid to do this."

I have a NAS which is mirrored internally, but I still back it up to USB 3 drives.

[Mark Rose]

For an easy backup solution with unlimited storage space, check out BackBlaze. Amazon also has an unlimited backup service.

[Uncwilly]

Originally Posted by retina

RAID is not a backup solution. Repeat that to yourself 100 times.

Originally Posted by Uncwilly

Also, I would like the device to handle back-ups of the data from the laptop.

Notice, the word "also". Actually the device acting as the backup for the laptop's drive is it's main function that I am looking for. I am not really worried about restoring previous versions of edited files. More worried about recovering files if the laptop's drive dies.

What do you use for your backups? Are your backups on RAID systems or on single drives? Do you use a NAS as a file server? Do you access the files on such a server when away from your lair? If so, what solution do you use?

[Chuck]

I have a QNAP 8-bay device populated with 6 2Tb discs acting as a NAS server. I use it for storing photos and MP3 files created from my collection of 11,000 CDs.

The server is set up as a RAID 6 array (double distributed parity) giving it a capacity of about 8 Tb.

I have my photos and CDs divided into directories on the NAS. The data, once written, is never going to change. I make optical disk backups of each directory as it is completed (using 25 or 50 Gb writeable blu-ray disks).

[henryzz]

One option would be a hybrid sshd which has the performance of an ssd for common tasks such as booting but has storage capacity cheap.
The issue might be finding a 2TB version of this for a laptop. The best I have seen is 1TB with 32 GB of NAND.

[retina]

Originally Posted by Uncwilly

What do you use for your backups? Are your backups on RAID systems or on single drives? Do you use a NAS as a file server? Do you access the files on such a server when away from your lair? If so, what solution do you use?

My minions handle all the details, but essentially the backups are to dedicated remote storage drive clusters. We use a versioned file system so all previous copies of any file are available. If ever there is some ransomware/malware/whateverware that might get hold then all it could do is make another new version of the files with its evil keys. That would be annoying but old data is not overwritten so recovery is relatively easy.

Backups are automated daily. The storage drives are rotated weekly from a pool. They are not all 24/7 online or instant except for the current sets that are in rotation. We don't synchronise between each of the pools because it is too cumbersome and it is not necessary. We can always find the most recent version of any file without too much hassle if/when it is ever needed, albeit with a bit of manual intervention.

When I am away from my lair I use a VPN to access any data that I can't take with me locally.

[bgbeuning]

The simplest backup is an external USB disk drive. You can buy empty
cases and put your disk (6 TB) of choice inside. Be sure to get USB 3.0
for a huge speed boost.

SSD drives are much more reliable than HDD. SSD have the limit on number
of writes issue, but you can use a S.M.A.R.T. disk monitoring tool to predict
when an SSD will fail from too many writes. There is a reason why military
laptops use SSD.

Do you really need all your media on your laptop? Look into DLNA/UPNP
servers (like Windows media player) that can provide access to your media
files to any device on your network. You put all your music and pictures
on some PC with plenty of disk space and then access them remotely.
You can also now buy devices whose only purpose is to access media like this.
(Never tried accessing my media server from outside my house network,
but it is probably possible.)
Then your laptop only needs the files you are changing. My Android tablets
use the bubbleupnp app for in house media access.

My NAS is a Linux box with eight 3 TB WD red disks in a ZFS volume.
ZFS is like RAID 5 but they address the RAID 5 problems. I get write
speeds of 400 MB/s. Also it takes steps to find bad blocks in unused space.
It can do file versioning like "the evil one" explained.
Linux can speak SMB (Windows folder shares), NFS (Unix shares), or iSCSI.

The Linux program minidlna is a simple way to make your NAS be a media server.
There are lots of other more sophisticated media solutions for Linux.

Foot note:

When building a RAID volume it is important to use enterprise disks instead of
desktop disks.
1. TLER (Time Limited Error Recovery) When a desktop disk encounters a bad
block, it will keep trying to read it forever. This is a good plan for a solo disk.
But in a RAID volume two things happen. (a) After some amount a time the RAID
controller will give up waiting, and say the disk is dead. If you have spare disks
configured, the RAID controller will start rebuilding the volume using a spare.
(b) Also in a RAID volume, the controller can usually get the data from other disks.
2. Vibrations - a disk is a mechanical spinning device. When the disk head moves
it causes vibrations. When eight disk heads are all moving at the same time, it
causes lots of vibrations. Enterprise disks are built to deal with this.

[Mark Rose]

You can also use NAS disks. I'm quite happy with Hitachi 4 TB drives.