Recent comments posted to this site:

shouldn't this tip be merged into the scalability page directly?
Comment by anarcat Mon Apr 24 14:10:29 2017

For the "archivist use case" of annex, this might lead to tens or hundreds of MBs of disk occupied by symlinks which actually don't add up to more than a few MBs.

$ pwd
/home/sorting_annex/mnt/keyfile
$ du -shc *-*
...
33M     fd0dc9d3-ad62-429e-ba1b-acc26a453ca4
33M     fd2fc989-bea7-4ffb-bbc8-2e34cd0e5be5
33M     fd79bbd4-d41e-4ea8-acc8-86437c5eed7c
33M     ffbd042e-f6d9-4450-9a57-8ed1086f587c
2.7G    total
$

Just a bit :P (yes, that is 2.7G of symlinks so far)

Comment by CandyAngel Mon Apr 24 13:55:02 2017

After posting this bug, I found a git annex tip on running a standalone build on the Synology NAS, so that may provide another workaround for me. Particularly since the Synology DS216+ is actually a x86-64 based system:

ewen@nas01:/volume1$ uname -a
Linux nas01 3.10.102 #15047 SMP Thu Feb 23 02:23:28 CST 2017 x86_64 GNU/Linux synology_braswell_216+II
ewen@nas01:/volume1$ 

but it is probably still worth figuring out the cause of the transfer lock issue on SMB for other SMB use cases.

Ewen

Comment by ewen Sun Apr 23 07:52:19 2017

The fix a couple of months ago appears to allow git annex init to complete on a SMB file share, which is great. But FTR there are currently issues with file transfers (link is to bug report about failing to get transfer lock), so use with a NAS is still "work in progress".

Ewen

Comment by ewen Sun Apr 23 07:36:08 2017

Hi, due to my requirement I need to revert vm image every time before running it via "git reset --hard" which is really fast on the other hand "git annex unlock" takes really long, I run git-annex on Centos 6 and git-annex version git-annex-3.20120522-2.1.el6.x86_64, if I update git-annex version can it help to fasten "unlock"?

thanks a lot

Comment by Murat Tue Apr 18 16:26:06 2017

Another use case is when you use annex for backup. For example I keep most of my files (dotfiles, scripts, music, books, etc) in a "home" git annex repo. Part of them is stored in google drive, part in a pendrive, part in my home computer, part in my work computer. Every once in a while I sync everything into my home computer and into an additional external hard drive remote that I keep as a backup. Sadly, my archived git projects can't be managed like that. Nevertheless, it's not a big deal since they are already in the cloud (gitlab or github) and in my local filesystem. Besides, since they are archived, I can just create tarballs and add them to the annex (and in case annex allowed to store .git directories, I'd not be really comfortable with the huge amount of symlinks that would produce).

That said, it's kind of a bathos that the illusion of a distributed, decentralized, redundant, versioned, annotated, etc. filesystem over git is broken by git itself.

Comment by memeplex Fri Apr 14 22:35:49 2017

It's strange. I made a change to the file on Horus, it gets propagated to Marduk, but Marduk does not know about. whereis still shows the file is only at Horus. At Asaru the symlink is deleted (so the change info was propagated), but instead there is a symlink on invalid content (like in indirect mode)

The connections are all over SSH: Horus -> Marduk <- Asaru

Sorry, I'm utterly confused...

Comment by Horus Wed Apr 12 21:21:08 2017

This sounds a lot like what i was trying to do in [[todo/dumb, unsafe, human-readable_backend]], except done properly. :)

I was wondering about that asymmetry recentrly, and it would seem like a good idea to fix this. the --to remote flag could especially be useful for directory, rsync or even webdav remotes. i am not sure how this could be implemented, but i would certainly use this.

having addurl work would be an awesome bonus, especially with webdav, where the mapping can be easily guessed (like S3). could be trickier with rsync/directory because then the user would need to tell git-annex what the base url is somewhere, but would fix a ton of use cases i had for this, like original filename on s3, ?Facilitate public pretty S3 URLs, hide missing files and, my favorite, syncing music to my android device. it would also automate, extend and simplify the publishing your files to the public use case.

so thanks for this effort, i think it's a great idea and i'm excited to test this! -- anarcat

Comment by anarcat Sat Apr 8 20:21:41 2017

How do you check if ssh has established a cached ssh connection?

ssh -O check -- somewhat of an additional overhead, but possible

GIT_SSH_COMMAND is used for every call to ssh in git-annex.

so then theoretically we could implement "may be ..." strategy on our end in our sshrun.

Comment by yarikoptic Sat Apr 8 03:16:48 2017

I wonder if you forgot to run git update-server-info?

That was it! Thanks!

I actually didn't forget to run this, but I did not realize that it needed to be run on the server, so I was running it on my local annex instead. Now that I've run it everything works!

Comment by lee Fri Apr 7 21:10:34 2017