Getting right hardware for first RAID setup

No we cannot expand a volume by adding drives. Drives generally follow a curve, and as you noticed largest are often the most expensive past the difference in capacity. 8-12TB seems to be the optimal price points currently. Your problem might be too much data for your budget, as you have barely any budgets for backups, as one location for data, even with RAID, is risky.

For enclosures, it is probably best to keep all disks in the same enclosure, as you add electrical points of contact. More cables, more enclsoures, more to go wrong, not less. Note: there is no power /surge protection in Thunderbay's, so you want at least a power conditioner or a UPS that maintains constant power.

RAID 5 is faster than RAID 10. RAID 10 is easier to recover data from in a disaster, but there is a 50% loss of capacity with RAID 10.

The scare stories about large RAID 5 volumes are plain wrong. As a test, we ran a RAID 5 with continual heavy access for over 2 years, without a single corrupted (bit rot) sector. (Same as with Mac Memory, stories about ECC RAM, etc do not apply on Macs') According to the scare stories on RAID 5, we should have averaged one every other month.

Rebuild times are identical with SoftRAID, it depends on the disk, not the RAID level. Also SoftRAID does not degrade performance during rebuilds, unlike hardware RAID.

When you purchase OWC populated enclosures the drives have 3 year warranties, and have already been "certified". Its a bonus feature.

Also, all enclosures you buy, even with SoftRAID XT or Lite XT, will upgrade free to SoftRAID 6.

Hope this helps.

@softraid-support Thanks for the fast reply, this has certainly given me some extra factors to consider and altered the equation a little.  You're right about the data capacity being a budgetary issue as I'm essentially having to double my already quite large current capacity to get the RAID going.  Backups are something I've struggled with.  Several years ago I had a matching set of drives in a safe which I'd use to manually clone my drives but that became less practical as capacities and costs soared.  I have my most important personal files saved both in the cloud and backed up to the separate 8TB Time Machine drive but this doesn't allow me to backup my media library.  I've tried CrashPlan for offsite backups before too but the software was unreliable and with 18 Mb/s upload on my internet it just couldn't keep up with the amount of data I have. My building is planning to have 1Gb/s download and upload fibre installed hopefully within the next year so that would significantly boost the potential to use something like BackBlaze.  I wonder if there would be a way or if it's desirable to repurpose my existing bay and drives into some sort of backup solution of whether it being USB 3 would be a bottleneck.

Are RAIDs more vulnerable to power outages? They're relatively rare here but do happen.  My drive bay, Time Machine drive and the Mac mini are connected to a surge-protected smart outlet from TP-Link but I don't know if that is enough.  I've looked at UPS before but they seem expensive and offer only a short period of power so I don't know if that would be enough unless they can trigger an auto shutdown.  On a related note would a macOS crash at any point be catastrophic to the SoftRAID? I'll avoid splitting across two devices if you say it increases the risk of issues, it was only because SoftRAID recommended it on the "Go faster" page that I was considering it as an option.

That's interesting what you say about RAID 5 and very helpful thanks because most comments I see tend to say it performs worse and struggles with rebuilds and I mainly see scare stories about second drives failing during rebuilds or UREs.  From what you're saying it sounds like RAID 5 would actually be advantageous in my scenario, especially with giving me more available storage.  Is this down to how SoftRAID works as opposed to hardware RAID?  I've also seen some comments on the SoftRAID site about RAID 6 coming and a RAID 5 being convertible to that.  Is that basically the same as RAID 5 but with an extra drive for protection?

Unfortunately there's extremely limited availability of the ThunderBays in the UK so the pre-populated ones tend to come at a really cost-prohibitive premium over buying the drives separately.  I did read about the certifying feature though and if I've understood correctly that should be the first thing I do when I get the hardware?

Taking into account your info and the pricing/availability it seems I have a few options.  The cheapest would be a ThunderBay 4 and 4x12TB WD Ultrastars which would match my current 36TB usable in RAID 5 but leave me with limited potential for future capacity increase.  The alternative is to get the ThunderBay 8 and increase my overall capacity.  Is there an advantage for my use to having more but smaller drives?  For example there's little price difference between 6x12TB WD Ultrastars (60TB usable) and 8x8TB WD Gold (56TB usable). For some reason the 12TBs are cheaper than the 10TBs which is why I've not considered those. The SoftRAID checker/calculator HERE seems to suggest more of the 8TBs would be quite a bit faster but offer 8TB vs 12TB protection.  And should I not worry about the write speeds it shows for my use? I must admit both those configurations seem like they might be overkill for capacity for RAID 5 whereas when I'd been looking at RAID 10 they seemed essential to get enough. If RAID 5 doesn't rely on mirrors would 5x12TB or 7x8TB work or is it best to stick to even numbers?  Ultimately it'll come down to a balance of cost vs futureproofing as there's nothing worse than watching your free capacity dwindle!

It's great to hear that I won't have to worry about the software upgrade separately if the ThunderBay comes thanks.

Sorry for the extra questions but I think the info you gave me has helped me narrow down my options a little (and made RAID 5 a lot less scary!).  Thanks.

@guygamer

Check out owcshop.eu for direct purchasing across Europe.

Here is the Thunderbolt 8 page:

https://www.owcshop.eu/catalog/index.php/external-storage-thunderbay-c-335_434

We are also partnering with Acronis for a 1 year trial of Acronis network backup with Thunderbay's, so you can back up all your devices to other devices, like what Crash plan was. Plus Acronis is big on cloud storage/cyber protection. We will do more with Acronis over time.

At the least get a power conditioner. In the US, they are under $100, so you should be able to get for instance a APC Line conditioner for a reasonable price. the idea behind UPS's on your storage is not to keep your drives on line for a long time, but prevent sudden power offs which can damage your data.

Surge protectors do very little and those that do are useless after 6 months of use anyway. Pure marketing.

We gave a talk at macsysadmin.se several years ago. Watch this video to learn why disk certify is important:

http://docs.macsysadmin.se/2016/2016doc.html
Look for the presentation (6th down on the left)
"How can you avoid the catastrophes caused by disk drive failure" by Tim Standing

Seagate has successfully made a comeback, the Exos and Iron Wolf are pretty reliable. As are the Toshiba and UltraStar (was Hitachi). I am long time skeptical of WD personally. But I am not up to date any longer. I think drives are mostly commodity and marketing, as long as you stay away from SMR recording drives, another topic.

Good luck

@softraid-support Thanks, I followed up with the EU OWC but as the UK is outside of the EU it would be unpredictable for tax and import duties to buy it that.

The Acronis backup sounds useful.  Thanks for pointing that out.  I might try to use my old drives as a local backup.

I'm still working on identifying a suitable power conditioner / UPS but think I've found some options and need to look into them more and figure out how to wire things in my flow with the smart outlet.

The video was very interesting thanks and very informative on something that isn't often talked about.  I'll definitely do a certify on all disks before setting anything up.

I think I've settled on 8x8TB Ultrastar HC320 drives as I am able to get them at a very competitive price and they seem to be very high performing and are CMR based.  I found them even cheaper elsewhere but they only came with a one year warranty with the retailer instead of the usual 5 year manufacturer warranty, so I'm leaning towards the extra cover if I can.

I haven't been able to find the Thunderbay 8 in stock anywhere in the UK but OWC are selling Thunderbay 4 direct through Amazon where it works out the same price for two of those.  I know initially you said to avoid multiple enclosures but I've been seeing a lot about it being best to split between enclosures and attach to separate TB3 buses on the Mac Mini. Given all I can get my hands on would be the ThunderBay 4s would there be any significant drawback you were concerned about beyond having another device in the mix?  Would everything be lost if one enclosure died or shut off for some reason.  This does raise a question on the license too - I know you said with a new device it would be a free upgrade to SoftRaid 6 but what would be the upgrade situation or cost with Pro as I believe that's needed for splitting across enclosures?

A last question on future RAID 6 support - I believe I've seen on here somewhere that it's coming and that it would be possible to upgrade a RAID 5 to RAID 6.  Have I read that correctly and if so is that still the plan?  Once the RAID is built and running I'm unlikely to be in a position to rebuild it from scratch so I'm hoping that's the plan. I like the idea of the extra parity drive protection and have planned the drives on the basis of getting the best performance and capacity out of RAID 6.

Just wanted to try and clear up these points before I hit the buy button now that I'm almost ready to finalise it.  Better to ask a few more questions now than miss something and give myself a huge nightmare to deal with down the line!  Thanks!

@guygamer

The Pro license upgrade from XT is another $179 US. You can buy that from our store, from inside the SoftRAID application.

the only time a second enclosure is really an issue is if the Thunderbolt interference issue becomes a problem and disks eject. It is impossible to predict this, but the more devices, the more likely something will trigger some disk ejects. We really wish Apple/Intel would address this. Its hard to believe they seriously think users are not seeing this.

We do intend to add RAID 5 to 6 conversion. However, it is not likely to be for example 8 drives, to 8 drives RAID 6, more likely 7 drives to adding an 8th drive. (at least early on)

Good luck!

@softraid-support

Thanks again for your help, this has helped me finalise the project.  I've ordered the Ultrastar 8TB drives so they should be here within the week.  I hadn't come across the Thunderbolt interference issue you mentioned but from what I can tell it seems to be to do with the wifi antenna on the Mac mini so I can always disable that if there's a problem as I have it connected via ethernet.  I had the two Thunderbay 4 enclosures delivered today from OWC's Amazon store.  I just felt more comfortable ordering via OWC and Amazon that waiting for a sole UK reseller to get the Thunderbay 8 back in stock.  For the UPS I'm thinking an APC Back-UPS unit is going to be my best option as it will connect to the Mac mini and power it down during a prolonged outage.

Thanks for the info on the RAID 5 to 6 conversion, that's saved a big potential headache down the road.  I'll just set it up initially as a 7 drive RAID 5 and hold the final drive back until RAID 6 is added.