As I'm not sure whether this is my fault or a bug of git-annex, I'll post it in forum first.
I have a normal v6 git-annex setup, with a desktop, laptop and usb stick.
All repositories are in sync, also the usb drive.
There are some unlocked files, which are fine.
But there are also some locked files, which appear to be broken.
Checking the target, there is a big difference between the paths. On the computers, it's .git/annex/objects/0W/vj/SHA256E-s10230770--dcbf7418ff3c9698e9d98418fd07a52afec172aeef80aee2a84dcfdaeef859ed/SHA256E-s10230770--dcbf7418ff3c9698e9d98418fd07a52afec172aeef80aee2a84dcfdaeef859ed.
Also the usb drive links to this location, but the file does not exist.
Searching with the firs part of the hash, I got some interesting result: the file was stored in .git/annex/objects/1cc/840/SHA256E-s10230770--dcbf7418ff3c9698e9d98418fd07a52afec172aeef80aee2a84dcfdaeef859ed.
The difference is that the two first directories have three instead of two letters and only one layer.
The hash remains the same, and the file is valid.
$ cd desktop
$ git annex sync --content
(success)
$ readlink -f locked-file
.git/annex/objects/0W/vj/SHA256E-s10230770--dcbf7418ff3c9698e9d98418fd07a52afec172aeef80aee2a84dcfdaeef859ed/SHA256E-s10230770--dcbf7418ff3c9698e9d98418fd07a52afec172aeef80aee2a84dcfdaeef859ed
$ cd usb-drive
$ readlink -f locked-file
(no output → broken)
$ find .git/annex/objects/ -name *SHA256E-s10230770*
.git/annex/objects/1cc/840/SHA256E-s10230770--dcbf7418ff3c9698e9d98418fd07a52afec172aeef80aee2a84dcfdaeef859ed
(file is at different place)
I did some troubleshooting and unspecific fixing.
$ git annex whereis locked-file
whereis locked-file (3 copies)
d0a64b4b-054e-4c42-a64f-58a08591abab -- laptop
e3013e6b-a06a-46a5-a67a-ce9ec0520d21 -- desktop
ebd16ddb-548c-4078-b35e-087132523924 -- usb-drive [here]
ok
$ git annex fsck
(everything fine, also unlocked-file was checked)
$ git annex fix
(no error, but no solution)
$ git annex unlock locked-file
(file is there and editable, great!)
$ git annex lock locked-file
(symlink is there and points… right, to the wrong location)
$ git annex drop locked-file && git annex get locked-file
(no change visible in work directory)
Specially on this drive disk space is important, so it would be for me a bad solution to keep them unlocked.
As I mentioned earlier, this could be also my fault, as I have done some not-often used methods
- I rebased the history and checked manually (without
git annex sync) themasterandsynced/masterbranch to the same commit on all repositories, no errors reported - I used a v5 repository on laptop before syncing with the drive, then upgraded. I don't think it is the problem here as it works on desktop after the sync as wanted
What OS?
What filesystem is the drive formatted with?
What version of git-annex?
OS is on both computers the same, on ext4. The usb drive has 8GB and is NTFS formatted. About NTFS, I wasn't sure, so I left the partition as it was.
I built git-annex with
stackthree times on top of the latest tag, the version number differed in minor afaik (but can't remember the old ones). I'm aware that I'm using an unstable version (forgot to mention it, sry about that), but it was on the two latest builds. I also don't know where the information is what the stable release is…I suppose it must have something to do with NTFS, but I can't reproduce it on NTFS.. Normally git-annex will refuse to use symlinks on NTFS entirely.
Can you please paste in the whole .git/config file of the repository?
Gotcha!
core.symlinksistrue! I set it tofalseand runningunlockworks fine, but afterlockthe file contents gets replaced with the two digits directories (not the three digits directories which are the place where the object is saved). Also, I can't unlock the file afterwards anymore.However, I would be fine if symlinks work fine on other file system (as mentioned, would be great for saving space and have the access to the files) or other method.
Ok, I reproduced the problem. Normally core.symlinks will be false by on NTFS. You have to manually set it to true to experience this problem AFAICS.
I was able to lock a file (resulting in a broken symlink) and then successfully unlock it and the content was back in place.
git-annex always uses the lower case hash directory names when on a crippled filesystem, since that's more portable and avoids lots of potential problems. By configuring core.symlinks=true, you make git-annex support locking files using symlinks, but these symlinks can't point to the actual content location.
I think it makes sense for git-annex to use the the mixed case hash directory names when core.symlinks=true even if the filesystem is crippled. There's some foot-shooting potential, since some crippled fileystems don't support the mixed-case hash directories. But, you have to manually set up this configuration. I've made a change along these lines.
Thanks for clearing it up, so I guess it was me. Otherwise, I have some questions:
annex.crippledfilesystemtofalseor is that dangerous on NTFS?I'm also fine with redirecting me to a FAQ, thanks again for your time and efforts.
You might be able to unset annex.crippledfilesystem. Or it might cause some stuff to break. Especially if you access this drive from Windows.
AFAIK symlinks don't work in Windows; at least git-annex does not know how to create them on Windows. (Windows has symlink-ish things and git might be able to use them.)
The link target is committed to the master branch, so you need to be able to check out that same branch in other clones and have it point at the content. So there can only be one link target used, and by default that's the mixed case one. See tuning for how to change that.