First RAID 5 volume in SoftRAID - speeds seem low

Speeds are slow! For an 8 drive RAID 5, you should be getting >1000 reads and probably >800 writes.

I assume this is Thunderbolt. AJA will give you more consistent results, I would use 5K video, RGBA codec and 16GB or higher file size.

@softraid-support

Oh dear this isn't looking good then!  Thanks for getting back to me.

That was the best result I got and I had to reboot to get it to improve to that.  Been copying files over from the old drives most of the day and at one point I had to reboot because it was grinding to a halt speed wise on transfers and improved some after the reboot.  Seems to go in bursts of speed.  I'd just assumed the read speed on the old USB 3.0 enclosure drives was acting as a bottleneck but obviously that's not the case on the test.  No obvious errors or anything flagging up in SoftRAID.

Yes it's Thunderbolt 3.  I've got the two Thunderbay 4 bays connected to different Thunderbolt buses on the Mac mini too as I thought that was supposed to give it a boost.  They aren't showing any problems in System Report on the two ports.  Just the 7 drives at the moment - kept the 8th back after you said about the RAID 5 to RAID 6 upgrade being only for adding a disk.

Do you have any ideas why it could be so slow or anything I could check?  I've tried to follow your instructions on different things like extensions from other forum posts so think I've checked everything I can think of.  I don't know if it helps but I took a fresh SoftRAID report right after performing the AJA test in the screenshot.  Hopefully we can get to the bottom of it as it seems to be missing the speeds you mentioned by a long way right now.

Thanks

@guygamer

Are you able to delete the volume?

What if you create a non RAID single disk volume?

then a 2 drive stripe?

then a 4 drive stripe?

then an 8 drive stripe?

and run AJA on each?

Then for comparison, a 4 drive RAID 5 and an 8 drive RAID 5?

No need to attach screen shots, just a small table with each volume type and read/write speeds. Lets see if I can detect a pattern here.

@softraid-support

That broke my heart a little to delete the volume after spending all yesterday transferring stuff over to it!  But better to get this right and figured out now. I guess I shouldn't have been so hasty to try and get it setup.

I ran all the tests you asked for.  Repeated each a few times just to make sure there were no anomalous results.  Did some of them with volumes located on one Thunderbay and then again with the volume split across both enclosures.

To rule out external factors I also tried all of the following and repeated the 8 stripe test but none had an impact on results: Thunderbays both connected to first Thunderbolt bus on Mac mini, both connected to second bus, Thunderbays daisy chained to each other, all other USB devices disconnected, HDMI cable disconnected.  I've also had the wifi adapter disabled since before I received the Thunderbays.

Results (using the AJA settings you recommended):

Seems like the performance is nowhere near where it should be on any of them and weird that going from 4 drives to 8 has no impact on the stripe and a big write hit on the RAID 5.

Hopefully something jumps out at you from the numbers!  Thanks.

@guygamer

This is a Flex 8? Or a Thunderbay 8?

@softraid-support

It's two Thunderbay 4 units.

@guygamer

thanks, it must be something about those drives, I will try to investigate, but don't think we have those particular drives in our test lab.

@softraid-support

Thanks.  I thought the Ultrastar drives sounded the best of the options I looked at too and I didn't see anything specification-wise that looked different to other drives.  The WD product code for them is 0B36404 if that helps.  The full model is HGST HUS728T8TALE6L4. Not really sure what my options are here as it doesn't seem like there's any point setting everything up and transferring my data if it's going to be stuck at those speeds.

@guygamer

I am not sure the cause, but need to investigate this.

@softraid-support Thanks, I really hope this turns out to be a resolvable issue as the only other drive options when I was putting this build together were a lot more expensive for lower spec drives. Trying to return everything would be a bit of a nightmare to deal with.  For what it's worth I installed SoftRAID on a 2019 15" MacBook Pro this evening and hooked up the Thunderbays to the Thunderbolt 3 and repeated the AJA tests on there.  Unfortunately I got the same speed results as on the Mac mini.

Also I created a non-RAID volume on each drive and ran some tests. First Thunderbay 4 - drive A hit 230MB/s R+W but B, C and D were not going above around 167MB/s. Second Thunderbay 4 drive A hit 230MB/s and B,C and D around 200MB/s. I've no idea if that's useful or significant information but at this point I'm just clutching for any data point!

@guygamer

The speed discepanrcy won't explain your results totally, but try this swap possitions. Does the slower performance stay with the slots, or the drives?

@softraid-support I just tried various combinations of disks and basically now regardless of positions they've all down to 150-160MB/s R+W.  Nothing is now fast in previous faster slots and the previous faster drives are slow everywhere, including when restored to their previous position.  It's infuriatingly trying to find a culprit when they're so inconsistent. Maybe tomorrow I'll put the new drives in the old USB3 enclosure just to see what that shows up.  And put my existing data/media drives in one of the Thunderbays to see if they experience the same low speeds.  The Thunderbolt cables shouldn't be an issue right?  I mean they're the brand new ones that came with each Thunderbay 4.  Thanks.

@guygamer

Make sure your test process is consistent. I recommend using AJA system test, with 5K video, RGBA codec and 16 or 64GB file size. then you have something reproduceable.

@softraid-support 

Yeah I've been using those same settings (5K video, RGBA codec, 16GB file size) for every test just to make sure they're consistent and that I'm comparing like for like. This afternoon I put the new Ultrastar drives into my existing enclosure which is a SilverStone/Teratrend TS432U USB 3.1 (gen 1) 4 drive bay (it has hardware RAID which I've never touched) and put my existing drives into one of the Thunderbay 4 units to test.  

Not sure how much value they have but here are the results for the Ultrastar drives in the SilverStone/Teratrend TS432U USB 3.1 (gen 1) enclosure:

They look terrible but I don't know if it's just the enclosure or old USB interface.

Whilst I was at it I did the same AJA test on my existing non-RAID drives (all Seagate Ironwolf NAS - 10TB and 12TB are helium filled) using both enclosures:

Seems odd that they don't perform better in a Thunderbay 4.

I don't know if there's anything useful amongst that data at all but couldn't think of any other combination or configuration I haven't tried already!