This special remote type stores file contents in a WebDAV server.
configuration
The environment variables WEBDAV_USERNAME and WEBDAV_PASSWORD are used
to supply login credentials. You need to set these only when running git
annex initremote (or enableremote), as they will be cached in a file
only you can read inside the local git repository.
A number of parameters can be passed to git annex initremote to configure
the webdav remote.
encryption- One of "none", "hybrid", "shared", or "pubkey". See encryption.keyid- Specifies the gpg key to use for encryption.embedcreds- Optional. Set to "yes" embed the login credentials inside the git repository, which allows other clones to also access them. This is the default when gpg encryption is enabled; the credentials are stored encrypted and only those with the repository's keys can access them.It is not the default when using shared encryption, or no encryption. Think carefully about who can access your repository before using embedcreds without gpg encryption.
url- Required. The URL to the WebDAV directory where files will be stored. This can be a subdirectory of a larger WebDAV repository, and will be created as needed. Use of a https URL is strongly encouraged, since HTTP basic authentication is used.exporttree- Set to "yes" to make this special remote usable by git-annex-export. It will not be usable as a general-purpose special remote.chunk- Enables chunking when storing large files.chunksize- Deprecated version of chunk parameter above.
Do not use for new remotes. It is not safe to change the chunksize setting of an existing remote.
Setup example:
# WEBDAV_USERNAME=id@joeyh.name WEBDAV_PASSWORD=xxxxxxx git annex initremote box.com type=webdav url=https://dav.box.com/dav/git-annex chunk=10mb keyid=id@joeyh.name
Unfortunately, trying to set up the following webdav servers fail. Some of the terminal output of git-annex --debug initremote ... is below, but it may not really helpful. Can I help any further, can a webdav remote be set up in a different way? (The box.com webdav initremote worked fine.) Cheers
With livedrive.com:
With sd2dav.1und1.de:
I tried signing up for livedrive, but I cannot log into it with WebDav at all. Do they require a Pro account to use WebDav?
When it's "testing webdav server", it tries to make a collection (a subdirectory), and uploads a file to it, and sets the file's properties, and deletes the file. One of these actions must be failing, perhaps because the webdav server implementation does not support it. Or perhaps because the webdav client library is doing something wrong. I've instrumented the test, so it'll say which one.
I've identified the problem keeping it working with livedrive. Once a patch I've written is applied to the Haskell DAV library, I'll be able to update git-annex to support it.
I don't know about sd2dav.1und1.de. The error looks like it doesn't support WebDAV file locking.
Hi Joey,
you are right, the 1-und-1.de implementation of WebDAV does not support locking.
Do you think there is a way to make that service work with git-annex anyhow -- say, by allowing the user to disable file-locking? I've worked around the problem with 1-und-1.de by using fuse+davfs2 to mount the storage in my file system and access it through the directory backend, but FUSE is dead slow, unfortunately, and this solution really sucks.
It seems it must advertise it supports the LOCK and UNLOCK http actions, but fails when they're used.
The DAV library I am using always locks if it seems the server supports it. So this will need changes to that library. I've filed a bug requesting the changes. http://bugs.debian.org/698379
Trying to use webdav leads in:
How can I disable the SSL certificate check?
You can change the webdav password by running:
WEBDAV_USERNAME=whatever WEBDAV_PASSWORD=newpassword git annex enableremote $remotename
I tried it, but I do have som troubles. Changing password seemed to have worked. At least I do think it did say OK. Any further action generates the following response:
$ git annex fsck --from box.com3 Literatur/Science/Papers/Modén/2006.pdf
fsck Literatur/Science/Papers/Modén/2006.pdf (gpg) (checking box.com3...) (StatusCodeException (Status {statusCode = 401, statusMessage = "Unauthorized"}) [("Server","nginx"),("Date","Sat, 17 May 2014 19:04:51 GMT"),("Content-Type","application/xml; charset=utf-8"),("Content-Length","235"),("Connection","keep-alive"),("Vary","Host"),("WWW-Authenticate","Basic realm=\"dav.box.com\""),("X-Response-Body-Start","<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<d:error xmlns:d=\"DAV:\" xmlns:s=\"http://sabredav.org/ns\">\n <s:exception>Sabre_DAV_Exception_NotAuthenticated</s:exception>\n <s:message>Username or password does not match</s:message>\n</d:error>\n")] (CJ {expose = []})) failed git-annex: fsck: 1 failed
It seems it is not available to use the correct password. I tried to use initremote and create a new remote, and this works. In the end no data is lost, as three copies out of my four are still there. I will keep my old remote around for some time if there is any sort of errorfinding that can be done.
When trying to enable box.com for jabber sharing, I get:
How can I debug this situation?
I have a similar problem to Maarten's, with some potential differences: * The WebDAV server is actually an ownCloud 7 instance; * The WebDAV server's SSL cert is issued by CAcert (whose root keys are otherwise installed on my system); * The cetificate lists the WebDAV VHost's name as an Subject Alt Name rather than its Common Name.
failed git-annex: initremote: 1 failed
1und1 (a german ISP) will give you up to 1 TB of space hooked to your DSL account. The setup is a bit weird, so here a short way through that worked for me.
FIrst you need to create a service account to connect to your online storage. To create this account you need to go to the legacy control center. https://login.1und1.de/xml/config/ConfigMain;jsessionid=expired.TCpfix90a?__reuse=123 Go to "Online Speicher" and activate it. Next go to "Zugänge > Dienstepasswort" and ensure that one is set up. It seems that you have to wait some time until the password is useable.
The host you need to use is: https://sd2dav.1und1.de/ instead of the one that is mentioned in the manual.
Now you can set the remote up:
When I try to use a WebDAV server as remote storage, "git annex initremote" stops with the error message
This looks like I need to explicitly add WebDAV support, but then this page says that it is built in.
I installed git-annex via Guix, so I checked the Guix package definition where I found the configuration flag "-WebDAV". I didn't find any documentation on this flag, nor configuration flags in general, so I don't know if this asks for including or excluding WebDAV support (or maybe even something completely different). So I compiled git-annex under Guix both with and without this flag. In both cases, the build log includes the lines
There is no further explanation, so I have no idea why WebDAV was excluded. Any suggestions? I am completely unfamiliar with Haskell and its build system, so I don't even know how to explore this any further.
Yes, that flag disables building with webdav. And if the build environment is limited in some way that the DAV dependency is not available, that flag can be turned on automatically by the dependency resolver.
Probably guix needs to package the DAV library or add it to the dependencies of the package such that it's available to build with.
Actually, I've gone ahead and removed the build flag in the next git-annex version, so it will always be required to build.
That someone thanks you for working on the patches (and is sorry about the duplicate effort)
Well, better two packagers than zero
If I figured out what happened, it was my question here that prompted Joey to update git-annex to make WebDAV a requirement, which in turn prompted you to package the new dependencies in order to update the package in Guix. Complaint-driven packaging
It wasn't a waste of time either, because at that occasion I found out that the Guix infrastructure for Haskell code is very nice. I hadn't done much more than import the packages from Hackage.
Hi Joey et al.,
Thanks yet again for a fantastic tool.
We are using it to distribute scientific data, via a webdav share backed by a customised owncloud deployment.
As this repository will be public, we want to have two special remotes: data-rw and data. The former is used by staff to upload data, and uses a share URL & password with list-read-write-upload-delete permission which works fine.
We also have a share link that only allows listing and reading content. If I try to initremote with the same settings as for the rw remote (but changing the username and password), then we get a failure during remote testing, as git-annex is unable to create a test file. I can get it to work if I temporary enable RW access for that share URL, but then an external user's
git annex enableremotewould fail when they go to initialise it later after I've reset the share link to RO.I've tried with
git annex initremote ... readonly=true, but that complains that readonly isn't a valid parameter for webdav. I also see in the source code that readonly is hard-coded to false for the webdav remote (I think -- my Haskell is very rusty).Is there any chance that we can enable setting readonly=true for webdav, and disabling the write tests for a webdav remote, effectively turning it into a RO special remote a la http/ftp/etc?
I've tried using httpalso or web, but that's incompatible with the chunking we need to enable for Webdav.
Any other suggestions?
Thanks again and in advance,
KDM
httpalso now supports chunking. So I think there's no need to add readonly support to webdav, probably. But if you disagree, I do think it would be possible to add. Just probably not useful.. after all webdav minus writing is little different than http. If you disagree, open a todo.