Trying to use WebDAV on TrueNAS Scale as Duplicati backup destination

[NugentS]

And I am getting "Failed to connect: The remote server returned an error: (401) Unauthorized"

Process:
1. Created a new top level dataset on a scratch pool called test-webdav
2. Configured WebDAV (yes I know its not secure yet)

3. Added a webdav share

Its R/W and the permissions got changed in what look like a correct manner
4. Using Firefox I browse to http://IPAddress/testwebdav and get a username / password prompt. Type in webdav/"Password" and I get an empty Index of /testwebdav
5. Setup a test backup with a few files using Duplicati

But when I press Test connection I get the

I have tried various / and \ beginning and end of the Path on Server and even tried the /mnt/ScratchSSD/test-webdav (which I am pretty certain is incorrect)

Anyone tried and got working anything similar?

Permissions on the test-webdav dataset are:

webdav is owner and group owner.

 

[morganL]

its a little bit confusing with webdav having two roles and one of them being modify only... can we remove the conflict and then see what happens.

When you access from your browser can you create and then modify files?

 

[NugentS]

As requested

Honestly I am not sure what I should be able to do from the browser. All I get is

Am I meant to have more? Never tried to use WebDav before although your comment makes me think I may not have write access. I did restart webdav

 

[morganL]

Looks like Webdav needs a client for writing files .. there were browser extensions, but doesn't seem to be very active.

 

[NugentS]

I was only using the browser to check that something was available on the port - its not how I want to use the service. I have successfuly used WebDAV into a Synology from Duplicati before now (although currently not doing so)

 

[NugentS]

Got back to this today
I have tried every option in Path on Server along with the correct username / password to get access to WebDAV from duplicati. Nothing works

What is worse is I have (that I can find) no logs that tell me what is going wrong. The only thing I can find is in httpd-access.log with the useful (not)
192.168.38.32 - - [16/Jan/2023:06:21:05 +0000] "PROPFIND /testwd/ HTTP/1.1" 401 381
192.168.38.32 - - [16/Jan/2023:06:21:09 +0000] "PROPFIND /testwd/ HTTP/1.1" 401 381
192.168.38.32 - - [16/Jan/2023:06:21:14 +0000] "PROPFIND /testwd/ HTTP/1.1" 401 381
192.168.38.32 - - [16/Jan/2023:06:21:56 +0000] "PROPFIND /testwd/ HTTP/1.1" 401 381
192.168.38.32 - - [16/Jan/2023:06:22:01 +0000] "PROPFIND /testwd/ HTTP/1.1" 401 381
192.168.38.32 - - [16/Jan/2023:06:22:06 +0000] "PROPFIND /testwd/ HTTP/1.1" 401 381
192.168.38.32 - - [16/Jan/2023:06:22:26 +0000] "PROPFIND /testwd/ HTTP/1.1" 401 381

Does anyone actually use WebDAV?

 

[anodos]

Works for me.
https://192.168.0.154:8081/davshare
Is how I mounted (via SSL). At least in your screenshot it doesn't appear you're setting the correct port.


Alternative possibilities:
1) you didn't set the password correctly in webdav service screen
2) you have permissions on /mnt/ScratchSSD that prevent access to the path

 

[NugentS]

I am using HTTP rather than HTTPS - although I have tried both.
I have confused things my moving the webdav test share to another TN Scale box (test server) - so some things have changed since I posted the screen shots. I will repost in the next message
Password has been typed in more times than I care to remember, including copy and paste from a notepad document
The share has permissions set to webdav, set in the process of setting the share up
What additional permissions should I need - I let TN set all the permissions.

 

[NugentS]

Dataset Permissions

Root dataset permissions /mnt/TestPool

Duplicati setup

Not ethat I think I have tried just about every variety of options in the path on server field including full path, partial path, leading slashes, trailing slashes

When I got to http://192.168.38.36:8080 I get a username / passwprd login prompt. I enter the expected details and this is what I get.

I then tried winscp to gain access. After editing the raw settings to add RemoteDirectory=/testwd it works or typing http://192.168.38.36:8080/testwd/ at the browser and I can see a file I uploaded.

It looks as though duplicati isn't passing the /testwd/ through, or is doing so in some manner than TN doesn't expect

 

[anodos]

View attachment 62510
View attachment 62511
Dataset Permissions
View attachment 62512
Root dataset permissions /mnt/TestPool
View attachment 62513
Duplicati setup
View attachment 62514
Not ethat I think I have tried just about every variety of options in the path on server field including full path, partial path, leading slashes, trailing slashes

View attachment 62515
When I got to http://192.168.38.36:8080 I get a username / passwprd login prompt. I enter the expected details and this is what I get.

I then tried winscp to gain access. After editing the raw settings to add RemoteDirectory=/testwd it works or typing http://192.168.38.36:8080/testwd/ at the browser and I can see a file I uploaded.

It looks as though duplicati isn't passing the /testwd/ through, or is doing so in some manner than TN doesn't expect

You could probably take a pcap of the duplicati traffic using tcpdump and look at it in wireshark (there is probably a protocol dissector for webdav there).

 

[NugentS]

That may take me a bit to do. I will try and figure out how. Its probably easier if I take the dump at the WebDAV server end rather than the client - which will have a lot more trees in the wood than the server

 

[NugentS]

OK - that worked.
It was digest authentication vs basic. About the only thing I didn't change. It doesn't work if I force digest complaining "Failed to connect: The request requires buffering data to succeed." whatever that means - but basic auth works for this purpose, over a VPN.

My next issue is that on a LAN connection, 10Gb its running at 1.95KB/s with plenty of space (destination pool is 1% used)

 

[anodos]

OK - that worked.
It was digest authentication vs basic. About the only thing I didn't change. It doesn't work if I force digest complaining "Failed to connect: The request requires buffering data to succeed." whatever that means - but basic auth works for this purpose, over a VPN.

My next issue is that on a LAN connection, 10Gb its running at 1.95KB/s with plenty of space (destination pool is 1% used)

Perhaps their client doesn't properly support digest authentication. If this is the case, I'd say this doesn't bode well for your general foray into using webdav with this client application (e.g. not very well-maintained). Maybe you can configure it to use SMB over your VPN?

 

[NugentS]

I would love to use SMB. Unfortunately that's not an option.
I have figured out (sort of) the next issue is that uploading files fails - and I can't see why. It seems to a common theme for duplicati - so I am going off the product.

Can anyone suggest an alternative.
I need to backup SMB shares, ideally to another SMB destination, but with the ability to encrypt the destination. The source is unencrypted and needs to remain so but the destination should be encrypted by the backup app, and not by truenas

I also need to do incremental / differential backups with some form of retention. Snapshots are not the answer I am afraid, although they are often a good answer. A GUI is also kinda nice - read vital, so I can tell whats its doing, when and how and make adjustments

What am I trying to do:
I have a NAS, a friend has a NAS. We have an wireguard VPN running between the two.
He wants to provide me with some storage for backups and I will do the same in reverse. However we want to do this properly so that he doesn't have access to my data in a readable form and again vice versa. He also doesn't want to expose credentials to me and again vice versa other than specific ones for the dataset involved., and certainly no root / admin type credentials. We only need to backup SMB data - no zvols etc. Looking for an app to do backups, can run under portainer or as a TN Scale app or even a VM. Ideally would run on the NAS itself in some manner

 

[morganL]

I would love to use SMB. Unfortunately that's not an option.
I have figured out (sort of) the next issue is that uploading files fails - and I can't see why. It seems to a common theme for duplicati - so I am going off the product.

Can anyone suggest an alternative.
I need to backup SMB shares, ideally to another SMB destination, but with the ability to encrypt the destination. The source is unencrypted and needs to remain so but the destination should be encrypted by the backup app, and not by truenas

I also need to do incremental / differential backups with some form of retention. Snapshots are not the answer I am afraid, although they are often a good answer. A GUI is also kinda nice - read vital, so I can tell whats its doing, when and how and make adjustments

What am I trying to do:
I have a NAS, a friend has a NAS. We have an wireguard VPN running between the two.
He wants to provide me with some storage for backups and I will do the same in reverse. However we want to do this properly so that he doesn't have access to my data in a readable form and again vice versa. He also doesn't want to expose credentials to me and again vice versa other than specific ones for the dataset involved., and certainly no root / admin type credentials. We only need to backup SMB data - no zvols etc. Looking for an app to do backups, can run under portainer or as a TN Scale app or even a VM. Ideally would run on the NAS itself in some manner

It would be easier if both NAS were ZFS. I assume not.

 

[NugentS]

They are in fact both TrueNAS Scale

 

[NugentS]

Someone is now going to point out that I am missing something that makes this real easy
I don't think snapshot replication works as I would need root access on the other NAS to set up the SSH pairing. It does mention admin rights required

 

[NugentS]

And what I had missed.
Configured the S3 service - which for whatever reason Duplicati doesn't work with
Used Cloud Sync Tasks - Amazon AWS - but (and this is what I had missed) Amazon S3 doesn't mean it has to be Amazon. It can be a local S3 instance from (just as a random example) TN's Minio service.

I can encrypt the end result with no issues, without having to encrypt the source.

What I cannot do is lifecycle / retention. But this is at least a step forward