ReadyNAS RAID Levels

ReadyNAS comes in a wide variety of RAID levels, none of them seemingly matching the standard. There are (in different ReadyNAS devices)

  • Flex-RAID
  • X-RAID
  • X-RAID2

Flex-RAID is the most simple of them all. It is a stand-in for manual configuration. Once you choose Flex-RAID, the system asks you to choose between any of the standard RAID levels, and if you go for multiple arrays, then how many disks are allocated to each of the arrays.

X-RAID and X-RAID2 both hide RAID settings from user. Both will automatically expand array if more disks are added, or if enough disks are replaced with larger ones and the array can be expanded maintaining redundancy. Internally,

  1. on one disk, it is just a simple partition,
  2. as the second disk is added, that simple partition is converted to a RAID1,
  3. at the addition of the third disk,  RAID1 is converted to a RAID5,
  4. as more disks are added, RAID5 is reshaped to accommodate additional disks.

The exact difference between X-RAID and X-RAID2 is sort of moot. For all intents and purposes, once the RAID crashed, they are the same.

Home NAS Recovery works with any of these configurations except

  1. where disks of different sizes are involved
  2. where multiple RAIDs are involved; Home NAS Recovery requires RAIDs fed to it one by one. If you have multiple RAIDs and you do not remember which disks form which RAID, your case is likely to end up in a data recovery lab anyway.

Talk about taking risks

This thread is about unnecessary risks.

It starts with some sort of intermittent hardware failure, probably not-quite-dead disk or something.

ReadyNAS Duo [X-RAID]. I noticed is was offline in my LAN and I couldn’t access …. The device seemed to have frozen and it wouldn’t shut down normally using the power button, I had to pull the cable. When restarting … it didn’t boot up properly and become usable again

When asked if he has a valid backup, he goes on to say

For the relatively brief periods of time I can get it up and running I can browse the shares on the network as normal. i was kind of hoping to address the cause of it falling over without having to move the data off the disks.

Well, this is bold. I daresay it is sorta kinda bordering on excessively bold. In any anomaly like this, there are only two alternative courses of action

  1. back up data from the NAS, with the disks in the NAS, or
  2. get the disks out and clone them to blank new disks.

depending on the exact situation, one or the other may be better, but backup goes first in any case.

ReadyNAS BTRFS

There are few programs to help you with ReadyNASes of late. This is because of BTRFS filesystem used by NETGEAR.

This happens to be an example case

Readynas Ultra 4 Plus – I think the Data is okay, but the Readynas is not functioning. After attempting all official methods to revive a Netgear RN-102ND NAS, I’ve removed the single WD 3TB Green drive to recover its data. Slaving it to a Windows PC, I open a popular program around here called Sysinternals Linux Reader for recovery. However…a dialogue appears stating “can’t open disk: Btrfs Volume 1 (0e34c953:data, raid) Check the disk and try again”.

Sysinternals Linux Reader does not know BTRFS. Furthermore, readers are not very good in reading damaged filesystem. By the time all official methods were attempted with no effect, you most likely need recovery software, not readers. You know, we have one, BTRFS and all.

Synology SHR

Recovery of a crashed volume on a Synology (here), can be complicated if SHR is involved,

I have a DS1010+ with all five drives composing a single SHR (Synology Hybrid Raid) volume.

I had a fan go bad … which seems to have corrupted the volume. Currently the volume reads as “Crashed” and two of the five drives read as “System Partition Failure”. … S.M.A.R.T. Test … resulted in “Normal” readings.

… has anyone has had success recovering data from a crashed volume that used SHR vs. a more universal RAID format. Or could anyone suggest a data recovery process or software that might work with SHR.

As far as the disks are installed all at the same time, and all the disks are identical, Synology Hybrid RAID is the same as normal RAID5. If one more disk of the same size is added, the array is expanded but still conforms to a RAID5 spec. If the disks are replaced with larger ones to expand capacity, the SHR capabilities come into play and the array does not conform to a RAID5 spec any longer.

One way to deal with that is to call Synology support and have them try and fix the issue. The other option is to use our Home NAS Recovery, but it will only work with SHR if all disks are of the same size and all were installed at the same time.

How long to rebuild?

There is a story of routine drive replacement going belly up

[I] have a Netgear ReadyNAS NV+ (RND4410)… [containing] four 1TB-disks…I changed all disks to 2Tb-disks. As it’s a X-Raid NAS I just pulled the disk from slot 1 and mounted the first new 2tb disk…[NAS] started the sync… 3 days later it still said syncing and I could still not reach the web-interface.

I pulled the plug and rebooted the NAS.

It did a check on the NAS volume and then it said booting with the LED on slot 1 still blinking.

[30 hours later it was] stuck on Booting and can not reach it in the web-interface.

despite the NAS owner specifically states that

No smart errors detected before start.

The case still looks very much like a failure of the second disk during rebuild. The most likely reason of a NAS not being able to complete the rebuild while blinking its disk LEDs is a read failure on one of the remaining drive. The X-RAID mode in ReadyNAS is essentially a RAID5, so once the disk fails, the NAS is supposed to remain online and rebuild the content of the failed disk once said disk is replaced. Observed result is that the NAS does not even remain online. Leaving aside the possibility of the new disk being a dud, another likely reason is that the second disk failed to read a block and the NAS is now locked up, endlessly retrying the read.

What could be done in this situation?

  1. Pull the new disk out and try to start up the NAS. If the NAS starts, shut it down again and try with another blank new disk. This rules out the possibility of using a dud as replacement.
  2. If that does not work, try and put the original disk back. This further rules out any  incompatibility of the new disks with the NAS. The entire batch of new disks may be DOA or incompatible. These things happen, albeit rarely.
  3. If putting the original disk back does not help, yet another attempt should be made without the disk at all (with both original and replacement out). If the NAS comes online, first thing should be made is to back up its content, most starting with the most important.

If none of this works, we have Home NAS Recovery for you. You should give it all the disks which were in the NAS all the time. That is, you’d better leave out the original disk which was removed and any replacement disks. The only set of disks guaranteed to be in sync with each other are the disks which were never removed from the NAS.

 

iSCSI

Sometimes, there are problems with iSCSI LUNs hosted on the NAS. iSCSI is a protocol of presenting some storage over the network (over the Internet for i) as a hard drive (a SCSI hard drive). i + SCSI = iSCSI. With NASes, some part of the NAS disk space is configured to be accessible over iSCSI. Each distinct piece of disk space presented over iSCSI is called LUN. The PCs connected to the NAS sees an iSCSI LUN same as its own physical drive. This is all you need to know about iSCSI, unless you want to dig deeper. If you want more, start with Wikipedia article.

There are two types of problems with iSCSI.

First type is illustrated on the QNAP forum – iSCSI LUN turned to RAW,

[I] have had iSCSI on my TS-212… Is see the iSCSI drive, but when I try to access the drive Windows says that drive needs to be formated before it will be used. When I checked the file system it says that disk is RAW instead of NFTS…

This case does not look like it requires any recovery on the NAS side. The faulty “disk” should be treated in the same was as  one would treat a local drive. A plethora of software is available for you to choose from to work with RAW filesystem in Windows.

Second type of the problem is when the “disk”, that is, iSCSI LUN, is no longer visible, or cannot be “attached” to the PC. Depending on the exact setup of the NAS, recovery may be quite a complex process, beyond the scope of Home NAS Recovery. If you have this type of problem, you probably should be looking for a data recovery service.

Dangers of filesystem check

This case highlights how a filesystem check can go wrong

This morning, I was copying files from one of the folders on the N4100pro to another folder on the same nas and hit an file system error.

Therefore, i did a file system check on NAS Admin page. The check took almost 12 hours to complete and when the nas restarted, my raid is gone.

The filesystem consistency check (and fix), does not normally makes things worse than they were. However, if the underlying RAID is somehow broken, or if the filesystem damage is real bad, the filesystem checker will trash the entire volume instead of fixing it.

These cases are usually not repairable. Filesystem check modifies the only existing copy of inodes (the key element of the filesystem) without leaving any copy of the original state. With the inodes broken, proper recovery is not really possible. You can get some bits here and there, but file names are gone.

QNAP and firmware problems

This topic on the QNAP forum provides multiple cases of the NAS failing after either a reboot, or a firmware upgrade. Some users hint to the firmware problem of some kind, but generally the descriptions are as follows

TS-409 lost configuration after reboot

I have a TS-409 (2.1.0 Build 0904T firmware) with 2x WD10EACS drives in RAID1. Following a reboot the device has dropped is showing as an unconfigured device. The drives are showing as ‘Invalid’…

I think that [updating firmware] may have caused this.

I have encountered similar problem with TS-409 QNAP version 2.1.0 (0624T) with 4 1 TB Seagate drives in RAID 5 configuration. All my settings are lost and the message says the server is not configured yet.

And then there is another topic on a similar, if not the same, issue

Suddenly TS-219P reports “Hard Drive Missing”, “Install and format at least one hard drive before using the NAS…”

Yesterday we installed an upgrade of the system software and added a user…

We don’t have any Linux computers, only Windows.

First of all, the basic sysadmin rule is if it is not broken, do not fix it. Once the NAS unit is assembled, configured, and filled with data, there is little, if any, need for firmware upgrades. Firmware upgrade in a NAS often involves rewrite of the entire operating system inside and migration of old settings to their corresponding last version equivalents. This does not always go smooth, so they warn you every time you’d better have backup before updating firmware.

If your unit has automatic firmware update, turn it off once you complete the initial setup.

However, once things start crashing around, we can still recover data with our Home NAS Recovery. Firmware update does not usually go around trashing actual file content on the disks. The configuration is lost, but this is something our software can handle.

 

 

QNAP and broken disks

This case highlights one quirk in QNAPs,

TS-212 disk2 in raid 1 replaced after failure won’t rebuild. ISSUE is that the second drive in raid 1 array failed and in degraded mode. Second drive is in an unmounted state and will not mount on it’s own or format.

original drives where:

disk1: Seagate ST3000DM001-1CH1CC24 – 2794.52GB

disk2: Seagate ST3000DM001-1CH1CC24 – 2794.52GB

disk2 failed so swapped disk2 for: WDC WD3000F9YZ-09N2001.0 – 2794.52GB

I tried the following commands [whatever]. always gets stuck saying

disk 2 is busy so it can’t add it to the raid 1 array.

Although the model is identified as TS-212, the problem affects many if not all QNAP models. If there is a disk  with bad sectors, the unit will sort of stall and will be not able to read any of the disks in it, or will read it very slowly, so that waiting for even simple operation to complete may be not worth it. Disk models, Seagate whatever or WD, are a red herring – QNAP may behave the same with a disk of any vendor if the disk has bad blocks on it.

The solution in this case is to shut down QNAP, remove disks one-by-one and clone them to a new disks, then re-insert disks into the QNAP and power it back on. Once there are no bad sectors on the disks, the recoveries/rebuilds usually proceed on their merry way.

I do not know the exact reason why this happens, but it may well be something associated with either the QNAP chipset or some quirk of the firmware.

Recuva and remote undelete

This conversation happens on Piriform’s forum,

I have deleted a file on a network Buffalo drive. I am using the free version of Recuva. I can navigate ok to the folder but the OK Button stays greyed out.

 

I believe recuva works only on local physical drives

 

Does the business edition of recova only work on local drives also?

Any version of any commercially available data recovery software requires the drive to be removed from the NAS. The definition of local physical drive may vary slightly, but certainly a drive still in a working NAS isn’t one of them.