Please describe the problem.
I have 2 indirect mode repos, both on network filesystems, that I have only used for adding
data on one end, then syncing via git annex sync and git annex get. The problem
is that.nfs copies are being made for each git annex object data file, e.g:
./.git/annex/objects/34/2x/SHA256E-s4112535690--c5f0e5a8af7bf17dd4a8ca192c8ddfb01fe6ec10908c80cffa5ac64c00e28443.vtk.gz/.nfs0000000006d0018600002147
Reading up on .nfs files, they are generated when "an open file is removed but is still being accessed".
What steps will reproduce the problem?
Clone a git annex repo on a network file system, run
git annex sync ,
git annex drop ,
git annex get
What version of git-annex are you using? On what operating system?
- git-annex version: 5.20140818-g10bf03a
- 2.6.34.9-69.fc13.x86_64 fedora 13
- 2.6.32-279.22.1.el6.x86_64 centOS
Please provide any additional information below.
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
# End of transcript or log.
closing because annex.pidlock exists, and git-annex init checks if FCNTL locking works, and if not, enables it. So if the NFS server does not support FCNTL locking at all, it will use pidlock automatically. And if the NFS server does support locking, there does not seem to be a good way to probe that it's NFS and enable annex.pidlock. --Joey
git-annex uses posix file locking to avoid scenarios that could otherwise result in data loss when dropping a file.
This indeed involves keeping a file open and locked while removing it.
Since NFS's posix file locking support ranges from poor to completely broken, it's not wise to use git-annex over NFS at all. It's much nicer to use it locally, with a git remote accessing the file server over eg, ssh.
I can't see any changes to git-annex that could improve its handing of NFS, unless there's a good way to detect a NFS filesystem and make
git annex initrefuse to set up a repository there.I've just added annex.pidlock to git-annex. Configure that to true, and it will use one top-level pid lock file, rather than these posix locks. Might work better on NFS.
annex.pidlock is enabled automatically if probing detects that posix locks cannot be made at all. However, if the NFS lets posix locks be made, just with the poor behavior described here, the probing can't detect that and so you'd need to enable it manually.