Slow SAMBA When Moving large amount of files locally...

[Chris Moore]

i seem to recall a previous issue I had with encryption

Sorry, you didn't say anything about encryption before now.
That changes everything. Full stop.

Is the pool we have been discussing encrypted, or not?

 

[wblock]

FreeNAS 9.1.1 is very old. It has old versions of FreeBSD and Samba. Both have had bug fixes and speed improvements over the years.

I would back that system up to external media immediately. All of your data, plus the FreeNAS configuration files plus encryption or SSH keys.

If you can't really use the external backups you have now, consider how would affect recovery if you really needed them.

Like always, the answers to a lot of questions come down to this: what is your data worth? What would it cost to recreate?

If it were me, I'd use new hardware, install FreeNAS 11 on it, then restore the backups to that.

 

[Dreded]

Sorry, you didn't say anything about encryption before now.
That changes everything. Full stop.

Is the pool we have been discussing encrypted, or not?

not

as I said I found my current problem.. it was Dos File Attributes now I just need to figure out how to make all the ~Filename files that word and excel create and hide using the +H attribute not appear

 

[Dreded]

FreeNAS 9.1.1 is very old. It has old versions of FreeBSD and Samba. Both have had bug fixes and speed improvements over the years.

I would back that system up to external media immediately. All of your data, plus the FreeNAS configuration files plus encryption or SSH keys.

If you can't really use the external backups you have now, consider how would affect recovery if you really needed them.

Like always, the answers to a lot of questions come down to this: what is your data worth? What would it cost to recreate?

If it were me, I'd use new hardware, install FreeNAS 11 on it, then restore the backups to that.

i am very confident my current backups will work just fine.. as i stated several by now I am making a new system within the month I don't want to have the potential down time due to upgrading this system and having to restore from backup when I will have a new system very shortly that will be all up to date

 

[Chris Moore]

it was Dos File Attributes now I just need to figure out how to make all the ~Filename files that word and excel create and hide using the +H attribute not appear

I don't think this is an issue. Those files are deleted when the program exits and the user should be knowledgeable enough to know.

 

[Dreded]

I don't think this is an issue. Those files are deleted when the program exits and the user should be knowledgeable enough to know.

Well unfortunately its the big boss man that complains of these.. he has a tendency to open like 10-15 files at one time inside the same folder and then he tends to open the ~ files by mistake and cause all sorts of havoc for himself.. so previously we(I) thought this was just causing us a minor performance hit(enabling the dos attributes) in the morning as we opened the file server for the first time after being idle all night... now with me moving around a ton of files organizing everything it came to light that this issue was actually what was causing all the "Slowness" id hear about in passing comments(I dont use the fileserver much myself directly)

So in the past the not having the ~$ files appear was worth a minor performance issue but now that I know its not a minor issue I am wondering if there is some sort of other fix.. can I tell word/excel to store these files someplace else sort of a thing.

also those files dont always delete when word/excel closes if you keep data on the clipboard it keeps them there

 

[toadman]

ok found my issue..

it was the CIFS "Support DOS File Attributes" which I had known previously to be a small issue however it was decided that the small occasional performance hit was preferable over seeing all of the Hidden MS Word/Excel files(the ~filename files) that excel and word create each time you open a file and sometimes leave behind even after the file is close(these ones are the real issue)

Whew. Glad you solved it! And glad you didn't replace all your hardware. :)

 

[Dreded]

Whew. Glad you solved it! And glad you didn't replace all your hardware. :)

You misunderstood.. replacing hardware anyhow this system will be relegated to a backup... I now need to decide what to do for a new system... but that will be a new post :) we are moving offices so it became obvious to me the current issues as I organized data to get ready for the copying to new system... in my office I have the opposite problem of most offices none of my users ever complain about anything so I never know of issues until they become catastrophic or I go to do something myself that I don't normally do... I am glad I found the problem too =) it's funny how many of the office staff commented on how much quicker the file server seemed today I just kept thinking "why didn't you say anything months ago.. "

 

[Chris Moore]

I now need to decide what to do for a new system...

What country / state are you in? It helps when suggesting hardware options.

 

[Dreded]

What country / state are you in? It helps when suggesting hardware options.

Canada/us (I live near the Washington border) I live in canada

 

[Chris Moore]

Canada/us (I live near the Washington border) I love in canada

So you could receive goods shipped to you at a US address, to save on shipping charges?

 

[Dreded]

So you could receive goods shipped to you at a US address, to save on shipping charges?

that is correct, I do find these days that its so close in price that its not worth the effort of doing U.S. shipping(unless ebay etc) as my time has a value... it takes me 30min to drive into states and back(since i used to own a decently sized computer store I get pretty good prices from the wholesaler even though I don't order much anymore they like me enough to allow me to buy 1 or 2 systems here and there at my old prices(used to do $25,000 - $50,000 a month through them))

Just so you are aware I am definitely not trying to build a budget system.. as long as I can justify why I need to spend the money I can spend as much as I want within say... $30,000.

When I said I needed a new NAS when we move offices I was told to write up a proposal... then the next question was "Just so I have an idea are we thinking $5,000 or $50,000?" I said most likely under $5,000 but I need to do a bit of research to be sure.

As I said as long as I can make a intelligent argument for it and how it will benifit us over spending less the budget might as well be unlimited.

I know we will be hiring 2 new CAD people and 2 new office workers(think secretary for requirements so MS Office) and a new engineer in the next 6 months.. we currently have 2 CAD people, 1 engineer and 9 Office people.

The hardware will also be replaced every 5 yrs just to avoid things getting old(which is whats happening now) so I only need something to fit our needs for 5 yrs and I do not see the office staff growing anymore after this year in that time.

 

[Chris Moore]

Just so you are aware I am definitely not trying to build a budget system.. as long as I can justify why I need to spend the money I can spend as much as I want within say... $30,000.

Have you looked at the FreeNAS certified systems that iXsystems sells? They also sell the TrueNAS systems that might actually be a better fit for the situation.
How much storage do you anticipate needing in the next 5 years?

 

[Chris Moore]

Also, iXsystems has HA systems with dual controller. What is the importance of always being available?

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[Dreded]

Also, iXsystems has HA systems with dual controller. What is the importance of always being available?

Sent from my SAMSUNG-SGH-I537 using Tapatalk

Meant to answer these in my post above before you asked them but apparently forgot.. I'll blame that on 3am ;)

That's the thing our storage requirements are rather meager.. currently I'm storing 2tb including data I'd consider throw away (user personal data such as music etc makes up half of this..that we only store cause we can, if it came to a situation where I needed the space im not paying to backup the music collection for the secretary:))

I don't see that data doubling (our data doesn't grow exponentially) but since data is cheap let's say 6tb (I'm not a monster after all).

As far as required uptime... that all depends if we go to a work off of storage situation or not... currently the cad guys like storing data they are currently working on that week on their local computers (nice fast ssd scratch) then moving it to storage when ready.. we don't consider a hour of lost time as terribly important so this gets shadow copied and backed up once per hour to the server (user backup) all email is cloud based.. and I really see ms office going that way soon too...

So our uptime needs right now is basically as long as it can be restored over a weekend we are good... and seeing as how my restore process right now is 20MB/s this can be accomplished... if our data doubles with our future system this can also be accomplished.. I could make the whole process at least 3x faster by also backing up to a usb 3.0 external off site (10min drive from new office) and driving to grab it if needed to restore from


Tldr; only 6tb and right now uptime isn't terribly important but that depends if we start working off the server directly and that depends on server speed compared to a local ssd vs cost to make server speed much faster(than current server speed not vs an ssd, I would say this point is at the 250mb level / practicality)

 

[Chris Moore]

only 6tb and right now uptime isn't terribly important but that depends if we start working off the server directly and that depends on server speed compared to a local ssd vs cost to make server speed much faster(than current server speed not vs an ssd, I would say this point is at the 250mb level / practicality)

What about the 10GB network plan? Is that just a possibility or, is that in the works? If you get a new storage server that is fast and a 10GB infrastructure to support communication between it and the desktop, it could be as fast as accessing a local SSD. It will drive the server cost up a little, but should be doable well within the $30k budget, even counting the cost of a small 10GB switch.

PS. Will you want a rack mounted unit or a tower?

 

[Dreded]

What about the 10GB network plan? Is that just a possibility or, is that in the works? If you get a new storage server that is fast and a 10GB infrastructure to support communication between it and the desktop, it could be as fast as accessing a local SSD. It will drive the server cost up a little, but should be doable well within the $30k budget, even counting the cost of a small 10GB switch.

PS. Will you want a rack mounted unit or a tower?

Probably rack mounted... I need to wait a little longer to see what is getting left behind when the current people in our new building move out... I don't have a direct need for it to be rack mounted but I do have a specific server room so prettyness is not a goal(some would argue racks are pretty too) my current stuff is not rack mounted although I have the adapter plates for the gigabit switch still

As far as going to 10gb it's just a possibility as it all needs to be in the proposal and a cost vs benefit argument needs to be made.. in reality I think 10gb from the nas to the switch will make sense but 10gb from switch to desks (given the cad guys are all over the place in the office as they get their own offices and are not sent to cubicle doom) will not make sense... if there is a second 10gb port available my desk will be a mear 10' away ;) (I somehow lucked out and am also not in cubicle doom)

 

[Chris Moore]

As far as going to 10gb it's just a possibility as it all needs to be in the proposal and a cost vs benefit argument needs to be made.. in reality I think 10gb from the nas to the switch will make sense but 10gb from switch to desks (given the cad guys are all over the place in the office as they get their own offices and are not sent to cubicle doom) will not make sense... if there is a second 10gb port available my desk will be a mear 10' away

If you can wrangle a little money for a new switch, this one has four SFP+ 10GB network ports. You could connect one to the server and another to your workstation and have a couple more for other equipment that needs fast access. The rest of the ports would all have 1GB connectivity to the server by way of the 10GB port.
https://www.newegg.com/Product/Product.aspx?Item=N82E16833623043
It is a decent option and leaves plenty of room to grow.

 

[Dreded]

That is an item that getting from the states will make sense(3x cost in canada) appreciate the suggestion.. looks along the lines of what we will need/want

 

[Dreded]

you think it makes sense to go with 10Gb only to the switch? am I correct in my understanding that this would allow me to saturate 9-10 connections(assuming the NAS can provide that much data) at 1Gb speeds? and if so is network bridging still a viable tech? (i think thats the right name) like would running 2x1Gb to a single computer allow me to get 2Gb speeds?

currently we are not terribly unhappy with the performance we get with Gb speeds and given that the server only has a single 1Gb connection to the switch I can currently give 9-10 people 100Mb speeds(yes?) so having the NAS connect to the switch at 10Gb effectively gives us 10x the performance(assuming everyone is trying to pull data all at once) than we have now?

also in tjhe new office we will all be working off the same High quality managed switch currently we are in many different buildings and as this system has slowly been upgraded from a 2 man home operation most of our hardware is consumer level $100 routers right now so I have to assume switching to proper hardware is going to give us a huge bump in latency etc