Rebuild unnecessari...
 
Notifications
Clear all

Rebuild unnecessarily inefficient

 SLeM
(@slem)
Eminent Member

I created a RAID 5 across 3 drives, 500 GB total, HFS+. I did not write any files to the drive. I removed a drive from the RAID, and added it back in, triggering a rebuild.

The rebuild time for this newly-minted filesystem was given as over 1 hour. For no more than five files (.DS_store and some fseventd files.)

If you can build the initial empty filesystem in a small number of seconds, it would seem that you should be able to do much better than 1/2 TB per hour in rebuilding an empty filesystem.

Quote
Topic starter Posted : 07/01/2021 11:32 pm
Topic Tags
(@softraid-support)
Member Admin

RAID is not a file system event, it is a disk level event. We do have a table that enables "fast mirror rebuilds", where only changes sections of the disk are written to.

But a full rebuild has to be performed first.

When you removed disk, you wiped that drive from the volume. SoftRAID needs to rebuild the entire volume space. If this were "file sync" you would be correct. But this happens at the disk level.

File level "RAID" could not be in real time.

Hope this explains.

ReplyQuote
Posted : 08/01/2021 1:09 am
 SLeM
(@slem)
Eminent Member

No, it does not explain. You are able to initialize a filesystem quickly, which leads us to one of two possibilities:

  1. the parity system just happens to be such that all-zero on two parts leads to all-zero on the third part; or
  2. That you have headers of used blocks that you initialize to all false so that you do not need to build the parity on those sections.

Does #1 hold up? No, it does not: you do not spend the time to rewrite zeros to all of the blocks, so the blocks contain whatever they happen to contain.

Therefore you must have some kind of headers indicating which blocks are used.

And that means that when you go to rebuild, you can consult those headers to know which blocks to not bother rebuilding parity for.

When I say "used blocks", that could be just at the initial allocation stage, that does not need to track filesystem changes: it could reasonably just need to track whether the section has been written to at least once since the filesystem was initialized.

ReplyQuote
Topic starter Posted : 08/01/2021 10:40 am
(@softraid-support)
Member Admin

I know what you are saying, but for protection of data, we chose a maximum safety mechanism for Mirror volumes. It is well thought out, secure and accounts for all kinds of corner cases.

ReplyQuote
Posted : 08/01/2021 5:55 pm
Share:
close
open