Maybe this issue was the same, or at least related to, this one: https://git-annex.branchable.com/bugs/git_annex_checkpresentkey_removes_git_credentials/ ?

I'm trying to use git-annex to manage music on my phone (surely a common-ish use case?), so I have a git-annex checkout under /sdcard/Music/. It sort of works? I did a clone under there (I think, it was a few weeks ago), it seemed to check out my files okay and set e.g. core.symlinks = false and annex.sshcaching = true. However, I keep getting warnings that I should run git annex restage or that some git hook didn't run because it wasn't executable. I'm also now using git annex adjust --hide-missing --unlock since I think locked files just don't work on exfat(?). Also, there's various characters that aren't supported that caused a lot of confusion when I was first setting it up... I think the Nix-on-Droid app-specific directory has a better filesystem, but I don't think my music player would be able to access that.

Do folks have tips on using git-annex on Android? My suspicion is that because of the filesystem and maybe other things, there's a fair amount of details that don't apply to git-annex usage on a normal Linux machine, and I don't really understand Android, exfat, or git well enough to confidently compensate -- more of a howto or rundown of features/config that's likely to come up on Android would be really helpful for this page.

Forwarding to port 22 shouldn't require root, e.g. the mentioned fowl tunnel could also be used to tunnel a local port to a remote ssh server on port 22. You just cannot listen on a local privileged port, but that shouldn't be a problem.

There are a bunch of "tunnelers" like serveo, e.g. ngrok and zrok, but the disadvantage of that is that it still requires a running ssh server.

My imagined use-case would be something like two phones or laptops behind NAT without tor or a ssh daemon. I think with magic-wormhole's dilation feature it would be possible to make it so that you could run git annex remotedaemon or git annex assistant on one or both devices (after pairing) and have them communicate without any further setup required.

Since magic-wormhole is already used for pairing it wouldn't even be a new dependency.

Maybe this is already implementable from outside git-annex as a custom git-remote though, I'd have to take a look at what git-remote-tor-annex is really doing...

It would be useful to have a git annex sync --ff-only option. I have an alias for git pull --ff-only that I use most of the time, and it seems like a git annex counterpart would be reasonable. If only one of my local repo and the remote repo have changed, I'm happy to resolve things automatically. If both have changed, then I'm going to want to think about what to do -- maybe rebase locally, maybe something else. Of course, I can manually check before running git annex sync or use git pull --ff-only myself, but especially with several remotes, that could take some effort, and this is what we have computers for.

I guess there's a question of what to do when some remotes can be fast-forwarded to and others would need a merge. I think think my ideal behavior is that if some updates can't be done without merge commits, it doesn't update any branches. But it'd also be fine to do as many updates as it can without any merges. Or do some prefix of the fast-forward updates, and then error out when it gets to the first merge. Whichever of these apply, of course it should display something if it can't handle things with fast-forwards exclusively.

Don't know your use case excatly, but just today I stumbled upon Serveo.net, which facilitates no-login, no-install reverse SSH tunnels, with with you can effectively break out of NAT'ed networks without root privileges. Together with autossh for persistence, maybe that can help you? Although I guess for forwarding to a privileged port 22 you would still need root privileges, meh...

For future google searchers:

When interfacing with Ceph storage via the S3 backend, I get errors like the following on larger files

XmlException {xmlErrorMessage = "Missing ETag"}

Like above, these 'errors' are actually successes with a non-compliant S3 backend that is missing either the Location or Etag file.

I confirm that setting partsize > chunk works around this issue, in my case chunk=4GiB partsize=5GiB.

Best, Kevin

If you're importing and exporting to the same remote, what happens when there's a conflict? Eg, whatever else is writing files to the remote writes to a file, but you locally modify the same file, and export a tree, without importing the new version first. That would overwrite the modified file on the remote, losing data.

The man page for git-annex-export states that this shouldn't be a problem, or at least shouldn't lead to data loss (might still require manual intervention):

When a file on a special remote has been modified by software other than git-annex, exporting to it will not overwrite the modified file, and the export will not succeed. You can resolve this conflict by using git annex import.

Maybe this was improved after this forum post happened?

There is probably some potential for issues when exporting and writing with another program at the same time, although that might be mitigated with webdav locks, for the webdav case. Also, you state that this is already a problem with the remotes that support export and import now.

Is this concern outdated? I would love to be able to import and export to webdav, so that I could use a Nextcloud as a "frontend" to a git-annex repository, getting the "best of both worlds" so to speak.

Thanks for the hints. Seeing that https://git-annex.branchable.com/bugs/enableremote_fails_with___34__wrong_reason__34___stated/ is marked as fixed prompted me to check again and it turns out that I reported the wrong git-annex version above: while that is the local version, the remote system has 10.20230626-g8594d49 (the latest one available from conda-forge, unfortunately that is rather old now), and it makes sense that the relevant version in this case is the one running on the remote system.

So I guess this is fixed already, my git-annex is just too old (although I haven't double checked, I don't see a convenient way right now to get a more recent git-annex version onto that remote system -- HPC, no root).

I just want to say that this would be nice to have for https://codeberg.org/matrss/forgejo-aneksajo as well, since only being able to use ssh is a pain point in some places (e.g. outgoing ssh connections are forbidden on the HPC systems at FZ Jülich, which means an intermediary is necessary to copy data from the HPC systems to a forgejo-aneksajo instance, currently). Being able to read and write via https (possibly reusing the existing access tokens from forgejo? That would be on me to see if it is doable) would alleviate this problem entirely.

possibly related:

• https://git-annex.branchable.com/forum/When--git-dir_is_not_in--work-tree/
• https://git-annex.branchable.com/bugs/enableremote_fails_with34wrong_reason34stated/

As a sidenote, I have been having nothing but pain and suffering with sharing git repos between users. The only viable way for me was to always access a git repo with the same user, otherwise it's been permission hell.