SOLVED Please Help! Pool Status Offline

[bent98]

If your pool shows as ONLINE then the Import command won't have any effect. Have you rebooted the system since then, or used the TrueNAS webUI to export and re-import the pool?

No, I was waiting for you to respond

 

[bent98]

If your pool shows as ONLINE then the Import command won't have any effect. Have you rebooted the system since then, or used the TrueNAS webUI to export and re-import the pool?

So I just rebooted and it still has pool online and I can see the shares.

 

[bent98]

If your pool shows as ONLINE then the Import command won't have any effect. Have you rebooted the system since then, or used the TrueNAS webUI to export and re-import the pool?

I am running a scrub as per your instructions. It will take some time for it to complete.

Apparently there were two errors there from the last scub on the 12/15/23. Again , all smart long tests were good free of errors.
After the scrub runs, what should I do? Also, should I check to see if the pool flags can be upgraded via the Storage tools? Wondering if the reason why the pool went offline was because of an incompatibility of pool flags and software updates?

 

[joeschmuck]

I recommend that you backup anything very important to you.

I believe that since you have rebooted, there is nothing more to do. What version of TrueNAS are you on right now? If you are happy it's running, then I'd let it be. It is possible the upgrade cause some data corruption but proving that would be difficult unless it happens again. If you are not in 13.0-U6.1 and desire to go that way, make another TrueNAS configuration backup, then download the update from the TrueNAS website, do not use the GUI download and install method. You can then update using big file and hopefully it all works well.

But!!!! Wait for @HoneyBadger to provide follow-on advice.

@HoneyBadger has given some great advice here, I learned a bit myself. And it looks like you did get a Christmas Miracle.

 

[bent98]

I recommend that you backup anything very important to you.

I believe that since you have rebooted, there is nothing more to do. What version of TrueNAS are you on right now? If you are happy it's running, then I'd let it be. It is possible the upgrade cause some data corruption but proving that would be difficult unless it happens again. If you are not in 13.0-U6.1 and desire to go that way, make another TrueNAS configuration backup, then download the update from the TrueNAS website, do not use the GUI download and install method. You can then update using big file and hopefully it all works well.

But!!!! Wait for @HoneyBadger to provide follow-on advice.

@HoneyBadger has given some great advice here, I learned a bit myself. And it looks like you did get a Christmas Miracle.

Yes I am on u6.1 I am grateful for everyone’s help. Fingers crossed it holds up. I did make a config back up of the u6.1 just now. I’d like to look into getting a back up for my back up and will also look into a cloud service my most important data.

I’m thinking about buying 4x22gb WD red pro drives . What do you recommend for an enclosure . I’d like something that has 2.5 /10g support

 

[HoneyBadger]

At this point, since you've rebooted and have your pool back, let the scrub complete, and prioritize making additional backups of any data you consider critical.

For an "enclosure" do you mean a brand new case for an additional system, or a way to attach the 4x22T drives to your existing system?

 

[bent98]

At this point, since you've rebooted and have your pool back, let the scrub complete, and prioritize making additional backups of any data you consider critical.

For an "enclosure" do you mean a brand new case for an additional system, or a way to attach the 4x22T drives to your existing system?

A new system

 

[joeschmuck]

Wow, four 22TB drives, in what configuration? The resilvering times will be astronomical. Large drives can be more bang for your buck, however there are some downsides.

Hey, I'm glad your system is up and running. Well time to roll on out of here and sleep.

 

[bent98]

Wow, four 22TB drives, in what configuration? The resilvering times will be astronomical. Large drives can be more bang for your buck, however there are some downsides.

Hey, I'm glad your system is up and running. Well time to roll on out of here and sleep.

I have a ton of UHD movies backedup. What do you recommend? Would D-RAID be an option for large drives?

 

[chuck32]

I'm really glad you can access your data again, was rooting for you!

Great forums here!

Personally I'd think about mirrored vdevs for that large drives. Resilvering is then a simple copy instead of recalculating parity.

 

[Constantin]

At the same time, how often do drives go bad? Assuming you’ve pre-qualified cold spares with badblocks or whatever flavor of disk-torturing program you prefer, a long resilver is not the end of the world.

For me, the bigger issue is that 10TB He10s no longer seem to be made, and I really like the lower power consumption and likely longer life of He drives vs. air-filled ones.

 

[bent98]

At the same time, how often do drives go bad? Assuming you’ve pre-qualified cold spares with badblocks or whatever flavor of disk-torturing program you prefer, a long resilver is not the end of the world.

For me, the bigger issue is that 10TB He10s no longer seem to be made, and I really like the lower power consumption and likely longer life of He drives vs. air-filled ones.

I m just a noob at this but I remember reading that if your drives fails and the recovering process takes a long time there is a higher chance the others can fail as they usually go in the same timeframe, Draid seems to be a more distributed process of recovering hence less change for failure during the process that’s why I’m asking if that’s the way to go on large drives . It seems they added Draid in the latest truenas scale release (TrueNAS SCALE 23.10).

"

Sequential Resilver​

This is a new resilver type used by dRAID where blocks are read and restored sequentially. This greatly increases resilver operation speed as it does not require block tree traversal.

Sequential resilver is made possible by the use of fixed stripe width in dRAID. During a sequential resilver data is read from all pool members.

Checksums are not validated during a sequential resilver. However, a scrub begins after the sequential resilver finishes and verifies data checksums."

ZFS dRAID Primer

Background and general information about the dRAID storage solution included in OpenZFS version 2.1.0.

www.truenas.com

 

[joeschmuck]

I'm not really familiar with dRAID as I only recently learned of its existence a few days ago and briefly looked into it. My take away was it was designed for greater than 100 drive systems. I don't know of many people here that have that many drives. I know of one person who comes close with just over 60 drives. My point, that is not the majority of Home TrueNAS users and your system is not even close to 100 drives. So long as you are fine with long resilver times and you try to stay on top of all the testing and results, I think you will be fine. A real typical issue that we see here is people building a NAS and it just seems to work great, then they forget about the NAS, routine maintenance, everything. Then years later it fails and then you see frantic postings for help. It is happening a lot, I think due to how long TrueNAS has been available and hard drive failures happening well beyond the warranty date.

By the way, what happened to you, I don't think was a drive failure issue at all, likely the upgrade and the rebooting process, something in there caused the problem, but that is just what I think, can't prove. I woke up to a checksum failure myself this morning but I know I caused it because I switch between CORE and SCALE frequently, twice yesterday, and I do have three very old drives in my pool but I keep an eye on them.

Wow, 22TB hard drives, that is crazy how much can be packed into a small drive these days.

As mentioned above, burn in your drives, which means to test the hell out of them. I'm not certain what the best test is for four 22TB drives but I suspect it will be the solnet-arrary-test-v3 script and run it in burn-in mode for a few weeks. 22TB drives will take a long time to test it properly/completely.

Good luck on your next adventure.

 

[Constantin]

Plus, if you buy your drives used like I did, they likely feature a significant amount of entropy re: age, use pattern, batch number, etc. Assuming the drives are ventilated well, a rebuild is not that different than a scrub - a long, sustained I/O. In my old Mini XL case, those drive temps would go from 30 to 45+ degrees Celsius during a scrub.

The biggest plus for very large capacity drives is lower enclosure and electrical costs, less noise, etc. The best bang for the buck re $/TB is usually found at much lower capacities, however. The cumulative failure rate probability being what it is, a pool with more drives will be more likely to have a drive failure than one with fewer drives, all other factors being equal.

I’d take a close look at whether buying a large 847 case or SM equivalent from eBay combined with lower capacity but more drives would not result in a more performant pool (with lots of VDEVs) at lower cost than whatever the OP is contemplating. Yes, more power will be needed but a lot of money could be saved with lower cost disk drives and importantly, the SM PSUs are usually Ti (or can be upgraded to same) and offer redundancy.

A large case gives ample room for future expansion, ditto gradual upgrades of those drives would also allow for further expansion in the future.

 

[Davvo]

@chuck32 I'm not sure I would go 2-way mirrors with 22TB drives.

@bent98 I'd suggest opening a new thread providing us with the relevant information (ie storage space needed, typical file size/type, etc...) since it's a separate issue and this thread is already a bit cluttered; if you do so, remember to mark this as solved.

 

[bent98]

@chuck32 I'm not sure I would go 2-way mirrors with 22TB drives.

@bent98 I'd suggest opening a new thread providing us with the relevant information (ie storage space needed, typical file size/type, etc...) since it's a separate issue and this thread is already a bit cluttered; if you do so, remember to mark this as solved.

Will do. I have to build another Box so that I can back up my back up. Before I closed this out as solved. I just finished the Scrub. How do I investigate those 2 errors? Do I need to be concerned?

 

[Davvo]

Do I need to be concerned?

I don't think so.

 

[bent98]

Code:

root@freenas:~ # zpool status -v BIGASSRAID
  pool: BIGASSRAID
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 12:50:50 with 2 errors on Tue Dec 26 10:42:52 2023
config:

        NAME                                            STATE     READ WRITE CKSUM
        BIGASSRAID                                      ONLINE       0     0 0
          raidz2-0                                      ONLINE       0     0 0
            gptid/1cc4ce99-7c53-11e7-85fb-e0071bff267e  ONLINE       0     0 0
            gptid/1d45eb56-7c53-11e7-85fb-e0071bff267e  ONLINE       0     0 0
            gptid/1dce4f37-7c53-11e7-85fb-e0071bff267e  ONLINE       0     0 0
            gptid/1e4d4aee-7c53-11e7-85fb-e0071bff267e  ONLINE       0     0 0
            gptid/1edd586a-7c53-11e7-85fb-e0071bff267e  ONLINE       0     0 0
            gptid/1f7159ec-7c53-11e7-85fb-e0071bff267e  ONLINE       0     0 0
            gptid/1fed78e0-7c53-11e7-85fb-e0071bff267e  ONLINE       0     0 0

 

[joeschmuck]

scrub repaired 0B in 12:50:50 with 2 errors

All is good again, nothing more to do.

 

[bent98]

All is good again, nothing more to do.

Thanks to all. I really appreciate it!! Happy Holidays.

I will marked solved and open up new thread for hardware recommendations for new server to back this one up.