Issues and question...
 
Notifications
Clear all

Issues and questions with transferring array to Mac Mini

11 Posts
2 Users
0 Reactions
1,290 Views
(@jalari)
Posts: 6
Active Member
Topic starter
 

Hello! First, I am sorry for this massive book, but this has been a long struggle over many, many months in the making, so I have a lot to explain here before I get to my questions.

So I have a pretty unique setup where I have 4 x OWC Express 4M2 4-Slot M.2 NVMe SSD Enclosure's, each with 4 x 4TB NVMe drives (a generic brand of drives, all from the same maker) for a total of 16 drives in a RAID 5 config. I've had to purchase the drives over time, so unfortunately, despite all being purchased the same way on Amazon, I would sometimes get a slightly newer model than the others, so that could partly be related to my issues, but not sure. I have them wired up to a Windows 11 computer (I am mostly a windows user) using an "ASUS ThunderboltEX 4" card that works with the Asus board that computer has, and they are daisy chained to each other and the connected to the 40GB/s port (TB4 port) on the card. The primary use of this array is for long term storage of all my Blu-Ray movie and TV shows that I have been ripping myself so they can be played over the network on my media playing computers.

Since setting it up, it has been working for the most part excluding some initial issues I had and eventually resolved. (Had a few bad drives) So before I transferred my data over after my 2nd attempt to set this all up, I did run the certify on every drive to make sure everything was 100% this time since my first time doing all of this, I had bad drives causing me all sorts of headaches. While eventually every drive was certified, I did find that I had to certify only a couple at a time and wait for them to finish. If I tried to do too many at once, they would randomly fail the certification. This issue could be related to all of my issues below, just not sure.

Once the data was transferred over, I've had random issues with data being corrupted. At first I thought it was just a problem here and there, but when I started to run into files that were inaccessible, I ran a video health check on all of my Anime (since a large portion of my collection is Anime). The video health check software basically attempts to decode the videos and see if it has any trouble doing that. After it was done, I found that 88+ episodes seems to be corrupted beyond recovery. It's a small amount considering I have over 25k video files related to anime alone, so I was not super concerned at the time. However I later started to notice that even way more files seem to have data corruption at the very start of the video for some reason. Only happens at the start of the videos and only lasts a few seconds, but it's never in the middle or end which is strange. I am seeing this as I watch all the anime to verify they work and the subtitles play right (while also watching show's I've never seen). I am also doing this since I generally watch Japanese dubbed and need to edit the files so the English subs are the default track. None of these files with this issue were logged by the health check since they are playing, but clearly something is broken somewhere.

So that's the basic detail of what I've been dealing with, but what brings me here today is I've recently received an error from SoftRaid 8 saying that my data is out of sync and I need to rebuild. First I need to mention that since setting all of this up, the Read/Write speeds were WAY lower than I expected considering the NVMe drives can get around 2000~3000MB/s on their own, but in the array, the transfer speeds to the computer were around 60~80MB/s. Recent updates to the windows version of SoftRaid did seem to improve on that and now I get around 125MB/s if I'm lucky, but clearly nowhere near what it should be, but I suspect its because I have so many drives in this array, their mixed models, and I think I also used the wrong setting when I first set the array up (selecting it to be Optimized for Digital Video instead of workstation or whatever the other option was). This largest factor may be because the enclosures are TB2 and probably only use a single PCIe lane, so I suspect those are the main reasons.

Now with all that backstory out of the way (sorry lol), lets get down to my issue. So with the error coming up saying I need to rebuild and the corruption issues, I told the software to try the rebuild. It starts it and everything seems like its working, but the SSD activity from each drive almost always stays at 0%, and the estimated time to complete the rebuilt gets larger and larger, settling at around 20 days or longer which makes no sense to me. The first time I tried, it said around 40 days if I recall right after it had been running for a few days. I know its a 64TB array (54.6TB really after formatting and the RAID 5 config), but that still seems like way too long to me, especially since the SSD activity almost never does anything except for the occasional spike in activity here and there. Despite all that, I let it run hoping everything gets resolved, but sadly things go down hill from there.

If I let it run and eventually I would find the computer has restarted and the rebuild is no longer running. At first I assumed it was windows update running and making the computer restart after the updates were done, but I finally got to witness this happen in real time after trying again and it seems that windows is just crashing. Not sure if its a blue-screen or what since the computer this is running on does not have a monitor hooked up (Using remote desktop to manage it) and the event logs have no details about why it suddenly restarted. I just know that every time I try to do the rebuild, at some random point it will crash windows and ends up rebooting.

Because of all of these issues, I've sort of reached my limit and feel like I need to take it to the next and extreme step. I just ordered a 2023 model Mac Mini M2 with TB4 ports. I just don't trust the windows software anymore what with all of these issues and the issues I had initially, so thought it may be worth it to try using a the latest Mac Mini for this, plus I can use it for other things with my work so it will be a win=win I hope.

The Mac should arrive today and I will start the process to move things to it, but this is where my questions finally come into play. I know I need to transfer my Softraid license to the mac, but will this also require me to format the array and start it fresh using the Mac partitioning and all that? I only ask because I spent weeks/months ripping and organizing all my media files, and I really do not want to do all of that again, but I will if I have to. I could try to copy the data off onto several 16Tb external drives I got because of all of this, but I am not sure there is any point to me copying the data if a lot of it is corrupted. I haven't even checked all of my movie and TV show rips to see if they have the same issues. The Anime was the first media added to the array and my most important media, so its what I checked. I am a bit torn on the best course of action once I get the mac. My hope is that I can plug it into the mac, and try to run the rebuild on that instead and hope for better results, but considering its a windows NTFS partition, will that even be possible or do I need to start from scratch?

Any suggestions on what I should do once I get the Mac? I don't have much experience with Mac's except for an older Macbook air I recently purchased for use in beta testing with my job, but it only had a TB2 port and hooking the drives up to it would of needed several adapters from what I can tell, plus its not super powerful so not the best option for all of this. I've tried to attach the OWC diagnostics data just in case, but the website wont let me attach that file type it seems. Not sure the proper procedure for that so let me know if I just need to include a certain file, if at all. Thanks again for any advise anyone can give, and again, apologies for the long book, lol.

 
Posted : 23/08/2024 7:15 am
(@softraid-support)
Posts: 9200
Member Admin
 

Simply put, your Mac will just mount the volume automatically.

You can deactivate your license on windows, or you can also deactivate it here:
https://software.owc.com/license/

 

the fact you could not certify all the disks at the same time means the hardware you were using was probably not the best Thunderbolt support.

One warning, 4 devices over Thunderbolt may not be 100% stable. What could happen are random times when the disk eject. (any interruption of signal and disks power down, a known Thunderbolt issue)

to start with, make sure all cables are tightly connected and they never move! Lets see how it goes.

BTW: if your file system is ExFat, backup/delete the volume and create a new one with either APFS or HFS.

Mac's can handle NTFS, but you are relying on a third party driver.

If you already have HFS as your file system, nothing to do there.

 
Posted : 23/08/2024 1:58 pm
(@jalari)
Posts: 6
Active Member
Topic starter
 

@softraid-support 

Thanks for the reply. I received my mac mini and got things setup, transferred the license and all that. Thankfully, as you suggested, the Mac is reading the array okay, althought I got a large amount of S.M.A.R.T warnings for several drives, so that is not a great sign. It did instantly start the rebuild process and appears to be working 100% better than it did on my windows PC so far. I am going to let it finish the rebuild and see what happens. Its my hope that most of the smart issues were related to the windows PC, but we will see. As you said, having this many drives connected is not the best idea. For now I am going to let it rebuild while I learn how to use this mac. Need to figure out how to share the folders over the network like I was with my windows pc. Big learning curve ahead of me, ha ha. I'll update this post once I have more ingo.

 

 
Posted : 23/08/2024 2:14 pm
(@jalari)
Posts: 6
Active Member
Topic starter
 

Well, that did not last very long. It shortly failed the rebuild after that, so I decided to restart as I suspected that plugging them in and trying to use them caused some issues. That does seem to be the case as the warnings and errors are mostly gone now. My only issue right now is in order to rebuild, it says it needs to be mounted, but when I try to mount, it tries for a while and eventually gives up but does act like its trying to rebuild still. The odd thing is that it already seems to be mounted, at least I have the icon on the desktop and can see/access all the files, but softraid thinks it's not mounted and eventually fails the rebuild. Do I need to eject the drive first from the desktop, and then mount it? Not sure what ejecting the drive will do so I want to see what your thoughts are before I try that.

 
Posted : 23/08/2024 3:04 pm
(@softraid-support)
Posts: 9200
Member Admin
 

@jalari 

Save a SoftRAID tech support file (Utilities menu) and attach it here. I can check with that.

 
Posted : 23/08/2024 3:36 pm
(@jalari)
Posts: 6
Active Member
Topic starter
 

I tried to attach it before but it did not seem to let me. I guess I did it wrong. Here is the support file from the mac. Not going to upload the one from the PC as I noticed it seems to put my license code in the report.

 

 
Posted : 23/08/2024 3:40 pm
(@softraid-support)
Posts: 9200
Member Admin
 

@jalari 

Windows support files cannot be posted here.

You have a serious issue. What file system is on this volume? You may need to run a data recovery application to get the data off.

Its possible the disk errors are from the prior installation, but generally they do indicate failed disks. Lets first worry about the data.

 
Posted : 23/08/2024 8:22 pm
(@jalari)
Posts: 6
Active Member
Topic starter
 

@softraid-support 

 

As I mentioned in the original post, this array was moved from my windows installation to my new Mac Mini I just got just for all of this. I did fear that i may need to wipe the array and start it fresh from the Mac, but that will be pretty tricky since I would need to copy most of the data off first to try and keep what I can without the need to re-rip all of my movies and TV shows again. It was a large amount of work and I know I am going to need to do some of it anyway to replace the corrupted files, but I am trying to avoid as much of that as I can. It sounds like I may have no choice but to go down that route. Trying to use a NTFS drive setup on a Mac just is not a great idea I guess. It would of been nice to keep it within windows, but the software not really being made for windows is a bit of an issue. I originally wanted Raid 6, but that was not supported on windows. Plus other issues like not being able to change the storage type from Digital video to workstation on the windows platform was a bit annoying too. Anyway, it sounds like I will have no choice but to start over. I'll try and backup what I can from it, but sounds like I will need to re-rip everything anyway. Thanks and if I have anymore questions, I will let you know.

 
Posted : 24/08/2024 8:39 am
(@softraid-support)
Posts: 9200
Member Admin
 

@jalari 

Note: your problem on Windows was clearly hardware. reliable hardware does not corrupt data.

The volume data should be recoverable, but if there is actually data corruption already, it will be in the recovery.

There are two "Data recovery" applications we have found reliable for recovering data from volumes. These do not "repair" your volume, they scan the volume, and enable you to copy the data to another device. R-Studio ( https://www.r-studio.com), and Disk Drill ( https://www.cleverfiles.com) are the two reliable recovery apps we have tested.

The problem you have is corruption is hard to locate, until you test the files.

You could scan the volume for free, and see if they find the "structure", before you even commit. I would try R-Studio on this recovery.

 
Posted : 24/08/2024 12:10 pm
(@jalari)
Posts: 6
Active Member
Topic starter
 

Well, so far so good. I was able to copy off nearly everything from the array. There was a short list of files I could not access, but I can easily replace those. After finishing that, I erased the array and set it all up on the mac with the formatting it uses. Been copying things back onto the array and I'm about 30tb in and so far there has been zero errors, unless how it was acting before. It's even WAY faster than it was with the windows system for my read/write speeds. So far I am very happy to of made this change. I had a strong feeling that most of my issues were with the windows version of softraid. While the hardware was suspect, I just could not imagine I had that many bad drives. I guess I will see how the rest of the copying goes and see if any other errors pop up, but so far it seems to be 100% rock solid and so much better than it was. It sort of bugs me that I needed to go to this extreme, but I can't argue with the results.

Now I just need to see if there is some way for my to use my MegaRAID PCIe card that is setup with my old 32tb HDD array on the mac so its all on a single system. The mac has no internal PCIE slots I can use but there are ways for me to use a thunderbolt to pcie riser for something like this, but those mostly seem to be geared towards GPU's. I guess I will give it a shot and hope it works too.

 
Posted : 30/08/2024 2:56 pm
(@softraid-support)
Posts: 9200
Member Admin
 

@jalari 

We sell a couple solutions, such as the Helios that have a PCI card available. If it is a full size card, you may need the "node titan", which handles the long PCI cards.

 
Posted : 01/09/2024 5:28 pm
Share:
close
open