This special remote type rsyncs file contents to somewhere else.
Setup example:
# git annex initremote myrsync type=rsync rsyncurl=rsync://rsync.example.com/myrsync encryption=shared
# git annex describe myrsync "rsync server"
Or for using rsync over SSH
# git annex initremote myrsync type=rsync rsyncurl=ssh.example.com:/myrsync encryption=shared
# git annex describe myrsync "rsync server"
configuration
These parameters can be passed to git annex initremote to configure rsync:
rsyncurl- Required. This is the url orhostname:/directoryto pass to rsync to tell it where to store content.encryption- One of "none", "hybrid", "shared", or "pubkey". See encryption.keyid- Specifies the gpg key to use for encryption.exporttree- Set to "yes" to make this special remote usable by git-annex-export. It will not be usable as a general-purpose special remote.annexobjects- When set to "yes" along with "exporttree=yes", this allows storing other objects in the remote along with the exported tree. They will be stored under .git/annex/objects/ in the remote.shellescape- Optional. This has no effect when using rsync 3.2.4 or newer. Set to "no" to avoid shell escaping normally done when using older versions of rsync over ssh. That escaping is needed with typical setups, but not with some hosting providers that do not expose rsynced filenames to the shell. You'll know you need this option ifgit annex getfrom the special remote fails with an error message containing a single quote (') character. If that happens, you can run enableremote setting shellescape=no.chunk- Enables chunking when storing large files.
This is typically not a win for rsync, so no need to enable it. But, it makes this interoperate with the directory special remote.
The remote.name.annex-rsync-options git configuration setting can be used
to pass parameters to rsync. To pass parameters to rsync only when it's
downloading and uploading, use remote.name.annex-rsync-download-options
and remote.name.annex-rsync-upload-options
annex-rsync-transport
You can use the remote.name.annex-rsync-transport git configuration
setting to choose whether we run rsync over ssh or rsh. This setting
is also used to specify parameters that git annex will pass to ssh/rsh.
ssh is the default transport; if you'd like to run rsync over rsh:
git config remote.name.annex-rsync-transport rsh
To pass parameters to ssh/rsh, include the parameters after "rsh" or
"ssh". For example, to configure ssh to use the private key at
/path/to/private/key:
git config remote.name.annex-rsync-transport "ssh -i /path/to/private/key"
Hi, I would like to see a example of setting up / using e.g. rsync.net as a repo.
Please also provide a highlevel description of how the rsync repo fits in with git-annex.
Q1. can you adds files to the remote rsync repo, and will they be detected and synced back ? Q2. is all the git history rsync'd to remote ? how do i recover if i loose all data except the remote rsync repo ?
Your comment "No, special remotes do not contain a copy of the git repository"
so a special remote is
"A. just a remote filesystem, that contains the objects with sha1 names ? "
"D. the data on the cloud/rsync storage can not be used directly it has to moved into a git-annex capable storage location. "
Would it not be better to mount the remote storage over ssh(sshfs) and then use full git-annex on mounted directory ?
how would i use rsync.net as a full repository ? (annex files and git repo)
It support's rsync, and I discovered now git as well.
I had a look at the webapp and can't figure out how to set it up, so I can have multiple remote's sync to rsync.net for annex files and git sync.
Link: http://www.rsync.net/resources/howto/github_integration.html
rsync has a --daemon mode with a simple challenge-response authentication but no encryption. This offers a nice lightweight alternative to ssh, especially when we store/transfer encrypted content anyways. Is this already supported in git-annex, if yes how to set it up?
If I get it correctly, it is mainly useable as a backup, which will accumulate contents of the objects managed by git-annex over time.
It would be great to have a use case illustrating its use in concrete matters. Thanks in advance.
There are many use cases for a rsync special remote. You could use it as a backup. You could use it to archive files offline in a drive with encryption enabled so if the drive is stolen your data is not. You could
git annex move --to rsyncremotelarge files when your local drive is getting full, and thengit annex movethe files back when free space is again available. You could have one repository copy files to a rsync remote, and thengit annex getthem on another repository, to transfer the files between computers that do not communicate directly. The git-annex assistant makes it easy to set up rsync remotes using this last scenario, which is referred to as a transfer repository, and arranges to drop files from the transfer repository once they have been transferred to all known clients.None of these use cases are particular to rsync remotes. Most special remotes can all be used in these and other ways. It largely doesn't matter for your use what underlying transport the special remote uses.
@davidriod if you're using a rsyncurl that uses ssh, then yes, the transmission goes over a secure connection. If the rsyncurl uses the rsync protocol, there would be no encryption.
Of course, encryption=none does not keep the data encrypted at rest, so the admin of the rsync server can see it etc.
@davidriod you can do things like this with special remotes, as long as the special remotes are not encrypted.
I don't really recommend it. With such a shared special remote R and two disconnected git repos -- call them A and B, some confusing situations can occur. For example, the only copies of some files may be on special remote R and git repo B. A knows about the copy in R, so git-annex is satisfied there is one copy of the file. But now, B can drop the content from R, which is allowed as the content is in B. A is then left unable to recover the content of the files at all, since they have been removed from R.
Better to connect the two repositories A and B, even if you do work in two separate branches. Then if a file ends up located only on B, A will be able to say where it is, and could even get it from B (if B was set up as a remote).
turns out that if you get into the habit of writing rsync remotes like rclone remotes, git annex breaks
I had a SSH config entry with the name
annexand set the rsync URL toannex:. The works for moving content in, but fsck and get will fail and undo the content tracking.