Dear all, In the first place I will explain what I want to achieve and then the way I figured out to do it. I have a lot of scientific simulations which I need to be traceable (cross check experiment version with code version, etc). I run them mostly on my ubuntu box (14.10, ext4) and store them on an indirect repo. From time to time, I find useful to work on them on my laptop (OSX 10.10, low disk space, hfs+). My idea was to full sync the main repo with a usb drive, and then at home get only the files I need to work from the USB drive. I faced a huge interoperability problem regarding the filesystem format. I tried using ext4, hfs+, fat32 and exfat. Regarding performance and stability, exfat is the only usable option between linux and osx (using hfs+ got the repo corrupted many times while writing from linux). However that forces me to use either a direct mode repo on the USB drive or a bare one.
Scenario 1: Direct mode on USB drive While this worked in another case, in this case when setting up the repo with the assistant it deleted all symlinks from the main repo. I wasn't able to correctly do this and finally gave up. Can anyone help me setting up this scenario? It will be like the following:
- ubuntu box (ext4): main repo, indirect mode, all files
- usb drive (exfat): sync repo, direct mode, all files (having the files available is a plus)
- osx laptop (hpfs+): partial repo, indirect mode, some files
Scenario 2: Bare repo on USB drive In this case the bare repo gets synced, with all data (2.6Gb). However when I clone it on the laptop, I get the message that files are not available, it says that they are only on main repo. This puzzles me because files seem to be on the bare repo. What's wrong in this case? How can I debug this?
You should be able to use a direct mode repo on a USB drive. Of course, if you delete files from there, the deletion will propigate to other clones of that repo. It could be that there's some bug with direct mode repos that causes file deletions to be incorrectly committed; if so you could a) revert the bad commits and b) file a bug report with details to reproduce the problem.
As to a bare repo on a USB drive, it will work too. It sounds to me like you forgot to run
git annex syncafter copying the files to that repo. And/or forgot to rungit annex syncbefore asking git-annex to get the files from the drive.That is to say, the fulll process would be:
Plug drive into first computer, and run in that computer's repo:
And then move drive to other computer, and run in that computer's repo:
I fnally made it work. I tried git-annex sync wihtout luck. What tricked me was the size of the bare repo, it made me thought that it had the files. However after I've made a git annex get on the bare repo it got all the files and after that everthing worked as expected. So, if I don't get you wrong, it is safer to use a bare repo on the USB drive? Thanks for the help. Best regards, Juan
Well, I'm glad you solved it.
I'd recommend a bare repo on a USB drive because it's the simplest thing, but a non-bare repo will also work ok.
After struggling a lot, I tried to do the same but with an fat32 drive. Seems that some of the problems I'm experiencing are due to some (linux) filesystem functions working different with fat and exfat. Just to get this right, using repos on USB drive formatted as fat32 works perfectly under linux and mac. When I go to the exfat drive, linux fails, and osx keeps working. Are you able to test it? Steps to reproduce it: 1)make a test repo on linux: cd /tmp; mkdir test_repo; cd test repo; git init; git annex init "test repo on linux box" 2) go to exfat drive and clone it: git clone /tmp/rest_repo
Please make sure you have the correct access rights and the repository exists.
I had never heard of exFAT before.
I guess you were mounting it in linux using the FUSE implementation of exFAT. It's not very surprising when FUSE filesystems leave off support for some syscalls. In this case, chmod.
Since even git clone fails, git-annex obviously cannot work.
A bare repository might be the only option if you're stuck using exFAT for some reason.