RAID5 was a bit slo...
 
Notifications
Clear all

RAID5 was a bit slow so I ran DiskWarrior, now all 4 disks i/o errors, degraded, failed

29 Posts
2 Users
3 Likes
178 Views
(@stevek)
Posts: 86
Member
Topic starter
 

I searched the forums and saw a post saying this might be related to a power draw issue but that was for SSDs, and this is a ThunderBay 4 with 4 HDDs.

I keep lots of video on a 4 HDD ThunderBay 4 running SoftRAID 7.5, RAID5. It's running on a late 2012 Mac mini that I use for Plex and Time Machine backups running Monterey via OpenCore Legacy Patcher. I've been running it this way for maybe 18 months without a hitch.

Recently that volume has been transferring video files slowly. No errors, SMART reports OK. Today I rebuilt that volume's directory after DiskWarrior reported it had an optimization index of 8 and needed a rebuild. After replacing the rebuilt directory I got SoftRAID popup after popup showing i/o errors, now showing degraded disks and failures.

I unmounted and am now shutting down the ThunderBay. Can someone give me a hand when you have a minute, please? I'm hoping this is not a "real" failure since all 4 drives going at once is pretty rare. Thank you, guys.

Screenshot 2023 10 08 at 6.54.13 PM

 

 
Posted : 08/10/2023 7:07 pm
(@softraid-support)
Posts: 7759
Member Admin
 

this is why support files are better. I have to guess, but likely the directory was damaged and pointing into space. So the disks responded with disk errors.

Clear the IO counters (errors) on the disks and volume.

then "recover failed disks" on the volume. when prompted to validate, its your choice, but I would go ahead and validate, it does not hurt. You can use your volume while it validates.

 
Posted : 09/10/2023 12:39 am
(@stevek)
Posts: 86
Member
Topic starter
 

Thank you. I can email a tech support report to you, do you mind?

 
Posted : 09/10/2023 6:31 am
(@softraid-support)
Posts: 7759
Member Admin
 

@stevek 

No we do not mind, as long as you open a case, and wait for a response, so you can attach it. You can also attach it here.

 
Posted : 09/10/2023 8:19 am
(@stevek)
Posts: 86
Member
Topic starter
 

@softraid-support I opened a case: 02084913. Here's the file. The ThunderBay in question was not powered on or mounted; if it needs to be I can connect and generate another report. Thank you. 

This post was modified 2 months ago by SoftRAID Support
 
Posted : 09/10/2023 8:35 am
(@softraid-support)
Posts: 7759
Member Admin
 

@stevek Yes, I need the disks connected at the time you saved the support file. Volume does not have to be mounted, but the disks need to be connected.

 
Posted : 09/10/2023 3:41 pm
(@stevek)
Posts: 86
Member
Topic starter
 

@softraid-support Here you go.

 
Posted : 09/10/2023 3:56 pm
(@softraid-support)
Posts: 7759
Member Admin
 

@stevek 

the main RAID volume is unmounted. Does it mount now? I want to be sure before giving you steps.

 
Posted : 09/10/2023 4:32 pm
(@stevek)
Posts: 86
Member
Topic starter
 

@softraid-support Yes it can still mount. It just throws up a lot of warnings about errors, failing disks, etc., so I thought I should keep it unmounted (and powered off) for now, just in case.

 
Posted : 09/10/2023 8:27 pm
(@softraid-support)
Posts: 7759
Member Admin
 

@stevek 

Fixing this is easy then. It is still working.

Clear the IO counters on all the disks (errors only)
then use "recover failed disks" on the volume. Validate when prompted. You can use the volume during the validate.

 

 
Posted : 10/10/2023 2:31 am
(@stevek)
Posts: 86
Member
Topic starter
 

@softraid-support That was easy, thanks.

I waited 10 minutes or so to let it finish all activity and I checked to make sure it wasn't indexing then I copied some files and ran some disk speed tests; it's still running much slower than usual. I have no idea what's going on. Two disks have something like ~50,000 on them and the other two have ~9,000. But they are good 4TB HGST and Hitachi disks. SoftRAID RAID5 is usually much faster than this.

 

Screenshot 2023 10 10 at 7.12.32 AM
DiskSpeedTest
 
Posted : 10/10/2023 7:30 am
(@softraid-support)
Posts: 7759
Member Admin
 

@stevek 

Use AJA for your tests. Its more consistent and reproducible across systems.

https://www.aja.com/products/io-4k-plus#support
Click Software and download AJA

Install the bare essentials, you don't need the rest.
Run AJA, set resolution to any 5K video, Codec to 16bitRGBA, and filesize to 16GB or 64GB
In settings enable Dual DMA engine
See what results you get.

 
Posted : 10/10/2023 2:11 pm
SteveK reacted
(@stevek)
Posts: 86
Member
Topic starter
 

@softraid-support Thanks. Here's what I got. No Time Machine, not indexing, Plex not running, no other apps running except some menu bar stuff like Default Folder, Lulu, SwitchRes, etc.

Screenshot 2023 10 11 at 10.10.02 AM
Screenshot 2023 10 11 at 11.30.19 AM

 

 
Posted : 11/10/2023 11:50 am
(@softraid-support)
Posts: 7759
Member Admin
 

@stevek 

Terrible!

This looks like the performance you would get with Encrypted APFS on HDD's

Verify each disk one by one for a few minutes. Have Activity MOnitor open, and select the disks tab. On the bottom it should show the throughput on the disk, which should be 150-200MB/s, depending on the drives. Cancel the verify once you establish the speed.
Then validate the volume, same thing. What are you seeing?

 
Posted : 11/10/2023 9:31 pm
SteveK reacted
(@stevek)
Posts: 86
Member
Topic starter
 

@softraid-support I thought so too! They’re HFS, non-encrypted. I’ll try your suggestion as soon as I can and report back. Thanks again. 

 
Posted : 12/10/2023 6:34 am
Page 1 / 2
Share:
close
open