is this setup possible?

[EvanNewton]

so, I am looking at making a setup with a Dell PowerEdge FX2S I might be able to acquire with 2 Dell PowerEdge FC630 Blade Servers. Id use a couple 12Gbs HBAs in the blade chassis chassis to connect to a supermicro CSE-826SE1C-R1K02JBOD. I'm sure this would work with a single server, but can truenas scale installed on each of the blade serve be able to access the same zfs pool created with the disks in the jbod? or should i look for a raid enclosure instead of a jbod enclosure?

 

[Ericloewe]

but can truenas scale installed on each of the blade serve be able to access the same zfs pool created with the disks in the jbod?

No. At least not without substantial hacking and with a different JBOD backplane (E1 -> Single expander, meaning single-port SAS only, to have two hosts, which is not a trivial thing, you'd need an E2 backplane with two expanders). That's the sort of thing iX charges good money for, and theirs is more integrated than two hosts that happen to share a chassis.

or should i look for a raid enclosure instead of a jbod enclosure?

That would not really solve your problem either.

So, what's the use case? Let's work from there and figure things out.

 

[sretalla]

The SCALE approach is "scale out", so what you're talking about there doesn't match with glusterfs and replicating the storage between SCALE storage cluster nodes.

Each node has its own storage and replicates with the other(s) and gluster takes care of the availability at the client level ensuring connection remains if one or more nodes go offline.

 

[EvanNewton]

So, what's the use case? Let's work from there and figure things out.

So basic issue. We are a Small police service with a small budget, and big aspirations... we have in car cameras that download automatically to one server with about 10TB of storage, we have interview rooms that upload to another server with its own 10TB of storage, we might be getting body cameras that will need more storage, and our file server with 20TB. I want to combine everything into one storage device and virtualize the servers. (I have 2 ESXi hosts already running some servers with vCenter)

So I was thinking of redundant storage controllers with some kind of DAS storage enclosures so i can expand by adding another enclosure when needed. (or adding another for replication) starting with about 100TB of raw storage.

I had received a quote from TrueNAS for an enterprise setup but it was around $40K, not unreasonable as I have received a quote from purestorage for $140k for 1/5 the capacity, but still outside what I would be able to convince them to let me spend.

That would not really solve your problem either.

so something like the SuperChassis 826BE2C-R609JBOD or 216BE2C-R609JBOD which both have dual expander backplanes could work? even if fail over isn't automatic and I have to do some manual work to get the host switched over isn't a massive problem.

The SCALE approach is "scale out", so what you're talking about there doesn't match with glusterfs and replicating the storage between SCALE storage cluster nodes.

Each node has its own storage and replicates with the other(s) and gluster takes care of the availability at the client level ensuring connection remains if one or more nodes go offline.

Thanks for the info!

 

[Ericloewe]

so something like the SuperChassis 826BE2C-R609JBOD or 216BE2C-R609JBOD which both have dual expander backplanes could work? even if fail over isn't automatic and I have to do some manual work to get the host switched over isn't a massive problem.

At that point, you might as well swap over the cables as well and save money and considerable headaches. I'm afraid the second server in this scenario is adding very little, unless you add a second set of disks.

 

[HoneyBadger]

So basic issue. We are a Small police service with a small budget, and big aspirations... we have in car cameras that download automatically to one server with about 10TB of storage, we have interview rooms that upload to another server with its own 10TB of storage, we might be getting body cameras that will need more storage, and our file server with 20TB. I want to combine everything into one storage device and virtualize the servers. (I have 2 ESXi hosts already running some servers with vCenter)

Unless you need something very specifically Windows-y out of your file sharing services, you can share SMB or NFS directly from the TrueNAS server. Would eliminate the need to have a virtual file server on top of ESXi/vSphere, and give you better overall efficiency (as you won't be writing files to a server that then writes blocks to TrueNAS)

If you're thinking of running some VMs directly off the TrueNAS server that's a whole difference ball of wax as well requiring a significantly different pool structure than bulk video/file archival. Can be designed for of course.

So I was thinking of redundant storage controllers with some kind of DAS storage enclosures so i can expand by adding another enclosure when needed. (or adding another for replication) starting with about 100TB of raw storage.

Redundant controllers are about uptime assurances; given the stated workload you'd probably get more value/assurance over having a replicated data copy, although the volume makes it challenging here. If you were stating that this would run a bunch of production database VMs or other things where you derived income from uptime then I'd lean more on having a redundant controller solution.

You can easily hang multiple DAE's off of a rackmount - depending on the intended use I'd lean towards a 2U server as your head unit (with room for 2.5" drives for cache, meta, or a small/fast standalone pool) and just daisy-chain SAS cables as I went.

I had received a quote from TrueNAS for an enterprise setup but it was around $40K, not unreasonable as I have received a quote from purestorage for $140k for 1/5 the capacity, but still outside what I would be able to convince them to let me spend.

Pure only sells all-flash so I'm unsurprised about the cost here. And while your budget might not allow for everything you want make sure the higher-ups know about the compromises that will be necessary if they make you do this on a Popsicle-stick-and-duct-tape kind of priceline.

so something like the SuperChassis 826BE2C-R609JBOD or 216BE2C-R609JBOD which both have dual expander backplanes could work? even if fail over isn't automatic and I have to do some manual work to get the host switched over isn't a massive problem.

In a "here be dragons" kind of sense you can do this with two separate servers by keeping the config synced, and just having the "standby" server not import the pool unless you power it on/manually import the pool/switch the DNS record or IPs.

Having a spare chunk of server hardware and routine backups of the config file is probably sufficient for most use cases, but the need for true HA with active/passive controllers is driven by the answer to the question of "How long can you afford to be down for?"