First RAID 5 volume...
 
Notifications
Clear all

First RAID 5 volume in SoftRAID - speeds seem low

Page 1 / 5
guygamer
(@guygamer)
Eminent Member

Hi

I finally got all the components and the disks passed certification last night so I've now created my first RAID 5 volume using SoftRAID. 7x8TB HC320 Ultrastar drives (saving the 8th as previously discussed to upgrade RAID 5 to RAID 6) in two Thunderbay 4 enclosures each connected to a different Thunderbolt 3 bus on a 2018 Mac Mini. SoftRAID 6.0.5 with pro license. No daisy chaining. Initialised and formatted as HFS+ journaled using SoftRAID.

I've been using the Black Magic and AJA disk speed test utilities I've seen mentioned and the speeds seem a little on the low side albeit there's variance on the reads.  Write speed varies from 100MB/s to 250MB/s and read speeds vary between 250MB/s and 550MB/s mostly although this one occasionally spikes to either 800MB/s or a couple of times it's exceeded 1000MB/s for maybe 30 seconds before dropping back to the lower range.  Are these speeds normal and is the variance in the read speeds normal?  Maybe it's just unrealistic expectations but I thought both readings would be consistently higher than that with the set up.

Write cache is enabled in the SoftRAID monitor settings.  I've done the Reduced Security for SecureBoot and the SoftRAID driver is up to date and loading.  So I think I've done everything I'm supposed to.

I've attached a support file from SoftRAID in case that is useful or shows something I'm not aware of.

Thanks.

 

Quote
Topic starter Posted : 30/05/2021 12:26 pm
(@softraid-support)
Member Admin

Speeds are slow! For an 8 drive RAID 5, you should be getting >1000 reads and probably >800 writes.

I assume this is Thunderbolt. AJA will give you more consistent results, I would use 5K video, RGBA codec and 16GB or higher file size.

ReplyQuote
Posted : 30/05/2021 9:33 pm
guygamer
(@guygamer)
Eminent Member

@softraid-support

Oh dear this isn't looking good then!  Thanks for getting back to me.

Screenshot 2021 05 31 at 05.40.29 aerate

That was the best result I got and I had to reboot to get it to improve to that.  Been copying files over from the old drives most of the day and at one point I had to reboot because it was grinding to a halt speed wise on transfers and improved some after the reboot.  Seems to go in bursts of speed.  I'd just assumed the read speed on the old USB 3.0 enclosure drives was acting as a bottleneck but obviously that's not the case on the test.  No obvious errors or anything flagging up in SoftRAID.

Yes it's Thunderbolt 3.  I've got the two Thunderbay 4 bays connected to different Thunderbolt buses on the Mac mini too as I thought that was supposed to give it a boost.  They aren't showing any problems in System Report on the two ports.  Just the 7 drives at the moment - kept the 8th back after you said about the RAID 5 to RAID 6 upgrade being only for adding a disk.

Do you have any ideas why it could be so slow or anything I could check?  I've tried to follow your instructions on different things like extensions from other forum posts so think I've checked everything I can think of.  I don't know if it helps but I took a fresh SoftRAID report right after performing the AJA test in the screenshot.  Hopefully we can get to the bottom of it as it seems to be missing the speeds you mentioned by a long way right now.

Thanks

ReplyQuote
Topic starter Posted : 30/05/2021 11:55 pm
(@softraid-support)
Member Admin

@guygamer

Are you able to delete the volume?

What if you create a non RAID single disk volume?

then a 2 drive stripe?

then a 4 drive stripe?

then an 8 drive stripe?

and run AJA on each?

Then for comparison, a 4 drive RAID 5 and an 8 drive RAID 5?

No need to attach screen shots, just a small table with each volume type and read/write speeds. Lets see if I can detect a pattern here.

ReplyQuote
Posted : 31/05/2021 1:10 am
guygamer
(@guygamer)
Eminent Member

@softraid-support

That broke my heart a little to delete the volume after spending all yesterday transferring stuff over to it!  But better to get this right and figured out now. I guess I shouldn't have been so hasty to try and get it setup.

I ran all the tests you asked for.  Repeated each a few times just to make sure there were no anomalous results.  Did some of them with volumes located on one Thunderbay and then again with the volume split across both enclosures.

To rule out external factors I also tried all of the following and repeated the 8 stripe test but none had an impact on results: Thunderbays both connected to first Thunderbolt bus on Mac mini, both connected to second bus, Thunderbays daisy chained to each other, all other USB devices disconnected, HDMI cable disconnected.  I've also had the wifi adapter disabled since before I received the Thunderbays.

Results (using the AJA settings you recommended):

 

Write (MB/s)

Read (MB/s)

Single non-RAID volume

231

231

2 drive stripe (same bay)

389

392

2 drive stripe (split bay)

379

380

4 drive stripe (same bay)

571

593

4 drive stripe (split bay)

568

582

8 drive stripe

619

592

4 drive RAID 5 (same bay)

426

435

4 drive RAID 5 (split bay)

398

423

8 drive RAID 5

204

402

Seems like the performance is nowhere near where it should be on any of them and weird that going from 4 drives to 8 has no impact on the stripe and a big write hit on the RAID 5.

Hopefully something jumps out at you from the numbers!  Thanks.

ReplyQuote
Topic starter Posted : 31/05/2021 10:51 am
(@softraid-support)
Member Admin

@guygamer

This is a Flex 8? Or a Thunderbay 8?

ReplyQuote
Posted : 31/05/2021 11:11 am
guygamer
(@guygamer)
Eminent Member

@softraid-support

It's two Thunderbay 4 units.

ReplyQuote
Topic starter Posted : 31/05/2021 11:13 am
(@softraid-support)
Member Admin

@guygamer

thanks, it must be something about those drives, I will try to investigate, but don't think we have those particular drives in our test lab.

ReplyQuote
Posted : 31/05/2021 1:11 pm
guygamer
(@guygamer)
Eminent Member

@softraid-support

Thanks.  I thought the Ultrastar drives sounded the best of the options I looked at too and I didn't see anything specification-wise that looked different to other drives.  The WD product code for them is 0B36404 if that helps.  The full model is HGST HUS728T8TALE6L4. Not really sure what my options are here as it doesn't seem like there's any point setting everything up and transferring my data if it's going to be stuck at those speeds.

ReplyQuote
Topic starter Posted : 31/05/2021 1:24 pm
(@softraid-support)
Member Admin

@guygamer

I am not sure the cause, but need to investigate this.

ReplyQuote
Posted : 31/05/2021 3:41 pm
guygamer
(@guygamer)
Eminent Member

@softraid-support Thanks, I really hope this turns out to be a resolvable issue as the only other drive options when I was putting this build together were a lot more expensive for lower spec drives. Trying to return everything would be a bit of a nightmare to deal with.  For what it's worth I installed SoftRAID on a 2019 15" MacBook Pro this evening and hooked up the Thunderbays to the Thunderbolt 3 and repeated the AJA tests on there.  Unfortunately I got the same speed results as on the Mac mini.

Also I created a non-RAID volume on each drive and ran some tests. First Thunderbay 4 - drive A hit 230MB/s R+W but B, C and D were not going above around 167MB/s. Second Thunderbay 4 drive A hit 230MB/s and B,C and D around 200MB/s. I've no idea if that's useful or significant information but at this point I'm just clutching for any data point!

ReplyQuote
Topic starter Posted : 31/05/2021 6:32 pm
(@softraid-support)
Member Admin

@guygamer

The speed discepanrcy won't explain your results totally, but try this swap possitions. Does the slower performance stay with the slots, or the drives?

ReplyQuote
Posted : 31/05/2021 11:22 pm
guygamer
(@guygamer)
Eminent Member

@softraid-support I just tried various combinations of disks and basically now regardless of positions they've all down to 150-160MB/s R+W.  Nothing is now fast in previous faster slots and the previous faster drives are slow everywhere, including when restored to their previous position.  It's infuriatingly trying to find a culprit when they're so inconsistent. Maybe tomorrow I'll put the new drives in the old USB3 enclosure just to see what that shows up.  And put my existing data/media drives in one of the Thunderbays to see if they experience the same low speeds.  The Thunderbolt cables shouldn't be an issue right?  I mean they're the brand new ones that came with each Thunderbay 4.  Thanks.

ReplyQuote
Topic starter Posted : 31/05/2021 11:51 pm
(@softraid-support)
Member Admin

@guygamer

Make sure your test process is consistent. I recommend using AJA system test, with 5K video, RGBA codec and 16 or 64GB file size. then you have something reproduceable.

ReplyQuote
Posted : 01/06/2021 2:48 am
guygamer
(@guygamer)
Eminent Member

@softraid-support 

Yeah I've been using those same settings (5K video, RGBA codec, 16GB file size) for every test just to make sure they're consistent and that I'm comparing like for like. This afternoon I put the new Ultrastar drives into my existing enclosure which is a SilverStone/Teratrend TS432U USB 3.1 (gen 1) 4 drive bay (it has hardware RAID which I've never touched) and put my existing drives into one of the Thunderbay 4 units to test.  

Not sure how much value they have but here are the results for the Ultrastar drives in the SilverStone/Teratrend TS432U USB 3.1 (gen 1) enclosure:

 

Write MB/s

Read MB/s

Single non-RAID volume

126

129

2 drive stripe

125

129

4 drive stripe

125

129

4 drive RAID 5

90

96

They look terrible but I don't know if it's just the enclosure or old USB interface.

Whilst I was at it I did the same AJA test on my existing non-RAID drives (all Seagate Ironwolf NAS - 10TB and 12TB are helium filled) using both enclosures:

 

Write MB/s

Read MB/s

6TB - Thunderbay 4

174

181

6TB - TS432U

196

203

8TB - Thunderbay 4

177

182

8TB - TS432U

196

197

10TB - Thunderbay 4

116

118

10TB - TS432U

225

231

12TB - Thunderbay 4

151

158

12TB - TS432U

157

159

Seems odd that they don't perform better in a Thunderbay 4.

I don't know if there's anything useful amongst that data at all but couldn't think of any other combination or configuration I haven't tried already!

ReplyQuote
Topic starter Posted : 01/06/2021 12:23 pm
Page 1 / 5
Share:
close
open