Moving from one folder to another takes too much time

[Plato]

Hi,

I built a NAS server for a week and created a jail with Sickrage running on it.

It has 9 x 6TB disks in a RAIDZ1 pool with 32 GB RAM on C2750D4I.

Deduplication for zpool is off and compression is on ( lz4 )

I have some directories for download and another for Sickrage TV folder.

I setup Sickrage to move files to its' new location in TV folder ( not link )...

So I copied all my files from my old cloud disks ( WD Live / WD World ) into NAS's download/TV folder..

Added shows in Sickrage and then choose Manual Post Processing with Move option...

Well...It moves....But it doesn't really... It really copies from /download/TV folder to /TV/<Show Name> folder and then it deletes from its' original location..

When I do the same thing on a USB disk with SickRage, it moves immediately ( because, really, all it does is changes its' folder name and rename the file )..

What could be the problem? Why is this takes this much time?

 

[depasseg]

It has 9 x 6TB disks in a RAIDZ1 pool

1st, just so you are aware, this is a very risky config due to the fact that if 1 drive fails and you replace it, you have an increased chance of losing a 2nd drive while the pool is being rebuilt, which would cause a total loss of data. It would be better to use RAID Z2 or have backups.

As for your question, it really sounds like a sickrage configuration. The only thing I can think of if your jail and dataset configs aren't quite right. If /download and /TV are on different datasets, then the files will actually be copied. Normally if you move a file between 2 directories, just the pointer is changed, and the data isn't rewritten. That's what allows the move to happen quickly.

 

[Plato]

Yes, I am aware of the risks, actually at first I thought it may be better to build a RAIDZ2, but then the data I store in this system is not that important ( I'll use this as storage for TV shows / movies I'll watch ).. I'll also configure it with Crashplan for important data I store in this NAS. But I don't think it'll amount to anything, maybe a few gigs of family photos etc..

Well.. I know how move really works, but I cannot find anything wrong with Sickrage configuration and it's really weird. It seems it's copying data from first location to second and when there are no errors deletes the first one. I just cannot find where the option to NOT do that. There is always hard link, but I don't want my files to stay inside download folder and make a total mess.

 

[depasseg]

I think the move is happening because of your jail storage configuration. What are the exact paths within the jail? And what are the jail storage configuration settings?

 

[Plato]

as I wrote in the post, downloads are in /media/downloads/TV folder. Sickrage TV folder is /media/TV folder..

/media is my zpool so they're in the same zpool.. That's why I cannot understand the reason for this problem.

/mnt/media is redirected to /media in jail ( I didn't plan to use /media folder in the jail so I used that one )

 

[depasseg]

Have you tried manually copying a large file from one sub folder to the other from within the jail? I'm curious if the delay is related to freenas or sickrage.

Again, my only guess is that you are moving from storage inside the jail to storage outside the jail, which would actually copy all the data, not just update the pointer.

 

[Plato]

Yup, it takes nanoseconds to move a 4 GB file from the download folder to actual TV folder.. And I also did this in jail. When you think that all the files ( downloads and actual location ) are in the same zpool there shouldn't be any problem.

It seems the problem is SickRage. I should check the configuration options to see what's really wrong about it.

 

[Plato]

Checked it everywhere but couldn't find any problem in config files. Anyone have any idea why this could happen?

 

[INCSlayer]

are the folders on different datasets?
because i have had slow performance when moving things from one dataset to another even though they are in the same pool

 

[Plato]

I only created one huge dataset in that zpool. So that shouldn't be a problem.

 

[depasseg]

I only created one huge dataset in that zpool. So that shouldn't be a problem.

The only other similar problem could be moving files in and out of the Jail. For instance, in side the jail, if you move something from/tmp to /media, it will copy all the data, because /tmp is inside the jail, and you have /media mounted as data from outside the jail. At least that is my understanding.

 

[Plato]

Hmm.. My jail is on another zpool so it may be the reason. The jail is on a separate SSD disk but why should it move it to /tmp in SSD zpool and then back to /media in media zpool. This is really weird.

 

[depasseg]

why should it move it to /tmp in SSD zpool and then back to /media

I just picked /tmp as an example, but I would look at your sickrage config closely.

 

[Plato]

Hmm.. Just realized something but I'm not sure if it's the problem or not...

The cache folder looks like this:
Cache Folder: /usr/local/sickrage/cache

So it may be possible that sickrage moves to cache folder, renames it and then moves it to final position...

I'll check it with debug parameter to see what really is going on. I'm not sure if it'll be logged but it's worth a try.

 

[depasseg]

Just try changing the cache location to /media/cache and re run to see if performance improves.

alternatively, try copying a file from /usr/local/sickrage/cache to /media/tv

 

[Plato]

BTW, something weird happened.. After enabling the debug and monitoring log file with tail, all files now move to new location with no delay... Weird..

So, the problem seems to be solved but I have no idea what was it in the first place.