OwnCloud PBI performance

[loiosh]

The PBI of owncloud isn't configured to be terribly performant. The two biggest points I see are that it's using sqlite as opposed to a more robust rdb, and does not have APC cache enabled.

I'm experiencing horrid lag when trying to add files. (4-10 seconds per file, and I store all my development git repositories here, so there are almost 100k files in the directory I was hoping to have owncloud keep synchronized. It's been synchronizing for 2 days thus far, and has gotten 1/3 of the way. for reference, the entire dataset is 3G. In 2 days, it's gotten through about 800M of it.)

I personally consider the current state of performance of the owncloud pbi a bug, because I'd imagine it to be a reasonable expectation for a pbi to be able to provide reasonably good performance of the given application. I sincerely appreciate the convenience, don't get me wrong.


In doing some general reading, it seems that adding APC to apache, so that php doesn't always have to be compiled on the fly, along with using a real DB might have a drastic positive impact on performance. Is it likely that this will be added? or should I go the route of manually creating my own jail, and try to work through installing owncloud manually in an attempt to gain enough performance that owncloud is relatively usable?

 

[cyberjock]

There is a ticket on this, and it's basically "not gonna happen". The PBI is meant as a small push-button install and is not meant for high-end massive user or file base. If the PBI won't do for you there are quite a few guides that show you how to setup Owncloud in a more performant way (albeit at a MUCH increase amount of system resources).

 

[loiosh]

While I totally understand that reasoning, I disagree with the assertion that my use case is atypical. I have *2* users, and the shared data between all users is 5G.

I still believe that the current setup isn't really usable, but I absolutely understand where you're coming from.

 

[cyberjock]

Well, as I responded to the other ticket, I'll tell you how things are:

The current implementation, while slow on an extremely large scale, is typically considered "acceptable" for small home users. It's not CPU intensive, it's not a memory hog, and it "gets the job done".

Migrating to higher performing options (such as MySQL and things like Apache) increases the CPU usage and memory usage by a metric a**ton. One friend setup Owncloud in a jail and he had 1GB of RAM devoted to the Owncloud jail with zero users and no files.

I personally used the PBI for about 5 months and while a bit slow was acceptable. The initial sync took overnight for me, but after that updating 50-100 files all at once was not a challenge.

But again, I'll go back to what I said before. The PBI is only meant for small home users. Expecting Owncloud to need 1GB of RAM with no users and no files is just out of the question for most home users. We have plenty of users with 8-16GB of RAM, and devoting 1GB to Owncloud before doing anything with it is just out of the question.

To be honest (and blunt) your case sounds more like a professional or business application, and that is not the expected user group for the PBI. You are more than welcome to be upset about the implementation, but the PBI was not really meant for your workload. I have 15GB of Owncloud data in 40k files and I stopped using the PBI because I wanted more performance. I had other mitigating reasons why my FreeNAS box was slow and I felt that I could be better served handling the jail myself.

Many of the PBIs are not maintained by iX and the "developer" of the PBI is literally a volunteer. Some of them are just forum users that like to do PBIs for the benefit of the community. That being said, you are more than welcome to develop a "massive Owncloud PBI" and iXsystems would probably be willing to post it for the community.

There are a few guides (once of which I used and commented in) and it works great for me. I don't have the performance drawbacks and as a single user my system has more than enough RAM for a large install. I'd *strongly* recommend you give one of those guides a whirl. They are dumbed down alot and even my mom could use them. ;)

 

[loiosh]

I guess I'm coming off a bit abrasive. That's really not my intent. I'm simply trying to help. I'm trying to illustrate what could arguably be a flaw in the current offering, so as to work towards a solution. I'm absolutely willing to contribute towards it. It's not terribly easy to find the process by which to help improve the existing offering.

I don't mean to sound disrespectful at all. I totally appreciate the existing work. I'd like to help make it better. I agree that the pbi should be respectful to system resources. I also assert that the community at large would benefit from a single image that has the tooling to configure itself a little bit more flexibly. I'd happily help to extend the current pbi to be configurable in such a way that you could use the same image in a small/medium environment, by telling it at install time if you want it configured to consume minimal resources, or to configure itself for speed. It seems less efficient to have two completely different installer releases for the same product when one could, with relative ease, be asked to configure itself a little more flexibly.

I initially looked on the big bad interwebs for the pbi creation process, and found this page http://doc.freenas.org/index.php/Creating_your_own_PBIs ...
however the links are out of date...

I looked here:
http://doc.freenas.org/index.php/Installing_a_PC-BSD_PBI

but that says that it doesn't work in 9.2.0. it's unclear to me if that is meant to express <= 9.2.0, >= 9.2.0, or ==9.2.0.

So I looked for the owncloud pbi source/project...
It's not listed here http://doc.freenas.org/index.php/Available_FreeNAS®_PBIs

...or here http://forums.freenas.org/index.php?threads/owncloud-plugin-now-available-in-plugins-menu.15046

it's not really obvious if I can easily clone or contribute to it here:
https://bugs.freenas.org/projects/f...lugins/revisions/master/show/plugins/owncloud

There's not really a good contributing doc either:
http://doc.freenas.org/index.php/Section_4:_Contributing_to_FreeNAS

That page is the first link on google for 'how to contribute to freenas'. However it is also devoid of content like contribution process, or the location of the repo.

that's what brought me to the forum wherein it said to report things like this.. So I did.

Now, After searching the forums, I found this:
https://github.com/josh4trunks/FreeNAS-Plugins

and when re-compiling my steps, to write this note, I additionally discovered this:
https://github.com/freenas/freenas-plugins
Which really sucks, because it's a link right off of the main site: http://www.freenas.org/our-community/

So, I initially _did_ look to just contribute these changes to the existing stuff. It's just the documentation isn't conducive to my brain's way of attempting to discover information. Again, I reiterate, I'm not trying to complain. I'm trying to illuminate a stumbling block and help fix things.

it'd be nicer if the mentioned pages / posts were a little more informative.
'Hey, We have a new plugin! Check it out!'

... Where's the link to the source, or documentation, or release notes, or anything to clue someone in as to how the team wants contributions

I thought this was the preferred means of conveying broken things, as I had a non-intuitive experience finding the code or contribution paradigm.

Now that I've found the code, I'll see if I can't get something positive going, and then try to get it contributed... rather than just sit and irritate people :)

 

[cyberjock]

Yeah. I'd love to have that kind of flexibility. The problem is that by simply switching from sqlite to mysql you just added 1GB of RAM to your requirements. That's all! Nothing special needed to see your needs blow way out of proportion. Unfortunately, asking if you want a light or heavy system isn't really an option with the PBI framework (which FreeNAS didn't create but simply supports). The solution if you want that kind of flexibility is to either make it yourself or make a PBI for the world that can say "Owncloud(heavy user/resources required)" over the standard build.

To be technical though, there really isn't an "installer" for Owncloud.

The documentation has taken a nasty turn. As you may or may not know 9.2.1.x has been a nightmare. Release cycles that were less than a week were observed and the documentation hasn't been able to keep up. Samba4 was much more of a problem than was expected. ;)

And as someone that looked at trying to write a PBI but hasn't actually gotten a working one I will say that it is not something you "just pick up". It takes serious determination, a lot of research, and a boatload of trial and error to get it right. I gave up after almost a week of effort and never looked back. iX is well aware of the major obstacles to PBIs and are considering various options to make it simpler. Something like a "here's how you build a PBI for program X" and includes very detailed step by step instructions is the final answer. I'm not sure. I just know that PBI creation isn't easy, sucks to do as an unpaid volunteer, and when you consider the lack of customizability of just doing a full fledged jail for your own unique need there's no argument that PBIs are limiting. Updating PBIs is hardly trivial and I'd say more than 75% of users that use the Plex PBI have abandoned it and moved to a full jail they built and update themselves. It just makes more sense.

Me personally, I avoid PBIs like the plague. I also regularly recommend people not use the PBIs because upgrading and maintaining them is not trivial in the slightest. Either I'm interested enough to get it working in a full jail or its just not that important to me. ;)