Hey community, reasonably new to the application, i have searched the forum for a similar issue. been able to run a backup of some 300gb fine to a local USB drive. now i am trying the same process for a much larger 14.2TB. This has been running for some weeks… Now towards the end i get a “Failed to download the configuration file from the storage: stat /EXTERNAL_DISK_NAME/config: input/output error” 
Is this disk corruption?
Cheers
Is the disk still mounted? can you copy those files to say /dev/null? e.g. cp /EXTERNAL_DISK_NAME/config /dev/null – does it work?
What filesystem is on the drive? Is there free space?
I tried to chown back to the primary user now non of the folders are visible per the above screen shot. ie cannot cd to the storage, presented with the same i/o error and does not read anything from the storage ie disk utilisation of SMART data.
The file system is XFS and its using a raid 5 external drive with total 20TB of which 11.8tb was used.
i had already tried the chown before receiving the cp suggestion, cp suggestion wont work now as it returns ‘no such file or directory’.
I have recently had rolling power outages, and the drive may have lost power as some point abruptly. I say may as that UPS does not always last the full cycle and i was unable to shut down manually the external destination drive.
Any further thoughts before i try a format of the drive and start the multi week backup again.?
Bingo.
- Run raid repair/scrub
- Run filesystem repair (fsck_xfs)
- Repair duplicacy datastore (by running check, deleting corrupted chunks, and then running a full backup with a new temporary id to attempt to re-create bad chunks; followed by another check of original backup and pruning snapshots that still have missing chunks)
Going forward – at least fix your UPS situation and auto-shutdown on power loss. Ideally yet – don’t use external RAID DAS contraption at all; your case is a perfect illustration of why. This was never a recommended way to go about raid – but today there are much superior approaches with filesystems that combine raid functionality – such as zfs and BTRFS. Still, maintaining uninterrupted power is a must.
Alternatively, to improve safety, if hardware change is not feasible right away, you may want to consider dismantling your raid array and converting the enclosure to a simple bridge hosting independent drives and use higher level solutions to create redundancy and parallelism – minio comes to mind. Duplicacy supports S3.
I have very much been on the fence with a firm decision between RAID5 and running independent drives like you say. I have invested already to purchase 3x10tb drives in a 5 drive Orico 3559RU3. I was stung by a raid failure of Drobo a few years back, which lead to large data recovery bill. This lead to this solution NAS and a offline backup. I have tried Duplicati (this failed to complete such a large backup), I have tried Rsync with a user script this was un-encrypted and messy file structures (with a better formed user script it could do what was needed i believe, but its a bit over my head, need more simplicity).
I see the potential for MinIO, i have considered kubernetes etc for distributed machines for a type of global state, but expense and my need does not require such advanced methods at this time.
Now attempting Duplicacy. I am looking at using MinIO, researching now. As i was pretty badly burned by Drobo rubbish i have lost trust in RAID, The decision to continue using RAID 5 very much hinges on 1.at least there is redundancy for drive failure 2. but there is the complexity threat that raid fails and loose everything.
If MinIO does as you say a higher level redundancy and can use the drives as independent drives in an efficient way then that is a huge help to swing away from RAID.
Anyway enough rambling, i’ll get to work.
… provided you have some other sort of redundancy — because disks developing bad sectors is just a fact of life. Raid5 is fine – but for it to work properly there should be some guarantees about power: either the hardware raid controller must have battery backup (not sure if Orico does) or power cannot be interrupted to neither host nor box until it’s gracefully shut down – otherwise data will be corrupted (the keyword is Write Hole).
Drobo uses some proprietary contraption. It’s not a good thing. It’s best to stick to industry standard time proven mechanisms.
That’s what backups are for – any contraption can and likely will fail.
Oh yes. It’s total garbage. Not sure why does it pop up first in search results. It does not even have a stable version for years – who thought EoLing v1 while not having stable v2 was a good idea? And yes, it does not work relibaly at all.
That would be either inadequate or too much work to construct your own backup solution on top of rsync as transport.
Sure. Minio can spread data store over multiple drives with specified redundancy. Hopefully you can turn off raid controller on your contraption so it presents separate disks to the system. You still need to ensure uninterrupted power – but at least the write hole will be less disastrous.
Raid is not bad in itself. It’s reliance on a Raid implementation on an external DAS appliance that is problematic. You may consider instead getting an off the shelf NAS – like Synology, or putting together your own hardware for True Nas, thus getting a purpose-built software, checksumming file system (btrfs or zfs) and very reliable time tested storage appliance as a result.
Above all, my main concern here would be to avoid falling for the sunk cost fallacy: if the contraption does not work for you – sell it on eBay. Don’t invest more of your time designing and maintaining an ill-fitting solution just because you already happen to have that hardware.
Thanks for all the input. I have seen MinIO requires multiples of 4 drives. I have formatted the raid 5 drive and zeroes all sectors and will try again bith my backup. due to needing an additional 10tb drive for MinIO and server resources to run the function of MinIO. I will try again with my raid5 and evolve to MinIO at later date. I have broken my backup into smaller tasks that will allow to shutdown clean in outage so, hopefully it wont happen again. cheers for the indepth feedback
This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.