I have several incoming streams of data. There are up to 4 streams coming in @ 20 Megabits per second and up to 6 streams coming in @ 40 Megabits per second.
Not only do I need to be able to write all 10 streams for about 4-5 hours at a time but I need to be able to read 2-4 different streams while writing.
The real problem I'm running into is that the software I'm migrating to is extremely limited. It can only write to one directory. It can be direct attached or shared or w/e but it can only be one directory. Its predecessor had semi intelligent drive selection (I could give it a pool of drives to pick from and it would choose the one with the most free space). I looked into software that would move the files once they finished writing so that they can be read off a different drive to reduce seeking and preserve write space, but I am NOT excited about that, nor do I trust its functionality.
I will be using 1.5TB 7200 RPM SATA Seagate drives (Model ST31500341AS).
I am willing to get a few OCZ Vertex 4 drives for some sort of Hybrid Storage Pool or possibly to mirror for ZFS Log drives & a cache. I don't know enough about my options to search for information.
I would like to keep this solution virtual in an ESXi box, but I am willing to move it to something standalone depending on what I learn and other factors. If I move it to something dedicated then I would like to use an ATOM 525 or something very energy efficient (the problem I'm reading about there is I should have a lot more RAM then 4 GB if I want to use ZFS). The two controllers I'm considering are LSI "9201-16i" or "9280-24i4e". My heart isn't set on them, especially if I can find a card that I could give a FreeNAS guest direct access to the card.
Finally I'm not sure if I want to use FreeBSD or FreeNAS. This is one aspect of my storage project. I may choose to run a database at some point. If I use a standalone storage server I may wish to use FreeBSD so I can run the database from the storage server.
Anyway that's what my project is looking like right now. I'm very new at this and trying to read as much as I can, but I'm not the best reader and have trouble understanding a lot of the technical stuff I am reading, so any insight is helpful even if it's a link to some plain English reading materials.
Not only do I need to be able to write all 10 streams for about 4-5 hours at a time but I need to be able to read 2-4 different streams while writing.
The real problem I'm running into is that the software I'm migrating to is extremely limited. It can only write to one directory. It can be direct attached or shared or w/e but it can only be one directory. Its predecessor had semi intelligent drive selection (I could give it a pool of drives to pick from and it would choose the one with the most free space). I looked into software that would move the files once they finished writing so that they can be read off a different drive to reduce seeking and preserve write space, but I am NOT excited about that, nor do I trust its functionality.
I will be using 1.5TB 7200 RPM SATA Seagate drives (Model ST31500341AS).
I am willing to get a few OCZ Vertex 4 drives for some sort of Hybrid Storage Pool or possibly to mirror for ZFS Log drives & a cache. I don't know enough about my options to search for information.
I would like to keep this solution virtual in an ESXi box, but I am willing to move it to something standalone depending on what I learn and other factors. If I move it to something dedicated then I would like to use an ATOM 525 or something very energy efficient (the problem I'm reading about there is I should have a lot more RAM then 4 GB if I want to use ZFS). The two controllers I'm considering are LSI "9201-16i" or "9280-24i4e". My heart isn't set on them, especially if I can find a card that I could give a FreeNAS guest direct access to the card.
Finally I'm not sure if I want to use FreeBSD or FreeNAS. This is one aspect of my storage project. I may choose to run a database at some point. If I use a standalone storage server I may wish to use FreeBSD so I can run the database from the storage server.
Anyway that's what my project is looking like right now. I'm very new at this and trying to read as much as I can, but I'm not the best reader and have trouble understanding a lot of the technical stuff I am reading, so any insight is helpful even if it's a link to some plain English reading materials.
