Recent comments posted to this site:

Hi, I've been bit by this bug and cannot depend on the website to update its TLS. Is there a way for git-annex to "ignore" such a restriction (apart from using an older version) ? Thanks,

Comment by AlexPraga Thu Nov 7 21:46:32 2024

Note that, like git-annex unregisterurl, using this option unregisters an url from a special remote, but it does not mark the content as not present in that special remote.

But then how to mark "not present" in that remote? After git-annex-reregisterurl I still have [datalad] remote listed although url now, correctly, associated with web. I have tried drop but that didn't work

❯ git annex drop --from datalad images/neurodesk/neurodesk-afni--21.2.00.simg
drop images/neurodesk/neurodesk-afni--21.2.00.simg (from datalad...) (locking smaug...) 
  Removal of content from urls is not possible
failed
drop: 1 failed

Could there may be option --fully or --mark-not-present option to make it 1 invocation operation?

Comment by yarikoptic Tue Nov 5 20:59:15 2024

I think this only happens with some rclone remote backends (like pcloud). The pcloud backend definitely keeps partially uploaded files, under the name of the full file. The backend attempts to do the right thing and uses the nopartial option of the pcloud API, but this does not work as it should [1].

I believe the latest rclone updates in 1.68.x should fix this issue, because they handle partial uploads in rclone itself [2].

Re: rclone gitannex: I only updated one client to use this, but I've also been careful to never interrupt uploads, so I can't tell. But I don't see how it behaves differently in this regard.

[1] https://forum.rclone.org/t/pcloud-keeps-partial-uploads/46026 [2] See changelog, the OpenWriterAt feature implies PartialUploads: https://rclone.org/changelog/#v1-68-0-2024-09-08

Comment by mike Tue Nov 5 15:43:09 2024

With the latest git-annex-standalone I am now getting a different error: initially pushing to an rclone special remote seems to work, but cloning and other operations like git remote show {remote} fail:

$ git annex initremote icg1220-remote-dir type=rclone encryption=none rcloneremotename=':sftp,host=icg1220'
initremote icg1220-remote-dir 2024/11/04 15:48:19 NOTICE: Can't save config "shell_type" for on the fly backend ":sftp"
ok
(recording state in git...)
$ git annex enableremote icg1220-remote-dir --with-url
enableremote icg1220-remote-dir 2024/11/04 15:48:27 NOTICE: Can't save config "shell_type" for on the fly backend ":sftp"
ok
(recording state in git...)
$ git annex push
copy qwer.txt 2024/11/04 15:48:32 NOTICE: Can't save config "shell_type" for on the fly backend ":sftp"
(to icg1220-remote-dir...) 
2024/11/04 15:48:33 NOTICE: Can't save config "md5sum_command" for on the fly backend ":sftp"
2024/11/04 15:48:33 NOTICE: Can't save config "sha1sum_command" for on the fly backend ":sftp"
ok
(recording state in git...)
push icg1220-remote-dir 
Full remote url: annex::85e65a54-6bff-49a7-9e62-db59d8912ceb?encryption=none&type=rclone
2024/11/04 15:48:35 NOTICE: Can't save config "shell_type" for on the fly backend ":sftp"
2024/11/04 15:48:35 NOTICE: Can't save config "md5sum_command" for on the fly backend ":sftp"
2024/11/04 15:48:35 NOTICE: Can't save config "sha1sum_command" for on the fly backend ":sftp"
To annex::
 * [new branch]      main -> synced/main
 * [new branch]      git-annex -> synced/git-annex
Full remote url: annex::85e65a54-6bff-49a7-9e62-db59d8912ceb?encryption=none&type=rclone
2024/11/04 15:48:39 NOTICE: Can't save config "shell_type" for on the fly backend ":sftp"
git-annex: /tmp: createDirectoryFrom: not located in .git: user error
ok
$ git remote show icg1220-remote-dir
Full remote url: annex::85e65a54-6bff-49a7-9e62-db59d8912ceb?encryption=none&type=rclone
2024/11/04 15:48:54 NOTICE: Can't save config "shell_type" for on the fly backend ":sftp"
git-annex: /tmp: createDirectoryFrom: not located in .git: user error
[ble: exit 128]

This is what's created on the remote:

$ ls -la git-annex-rclone/
insgesamt 24
drwxr-xr-x  2 icg149 icg1 4096 Nov  4 15:48 .
drwx------ 81 icg149 icg1 4096 Nov  4 15:48 ..
-rw-r--r--  1 icg149 icg1 3079 Nov  4 15:48 GITBUNDLE-s3079--85e65a54-6bff-49a7-9e62-db59d8912ceb-d34f9bc87f73892ce9cb135ebe0d0c9c98e01b8b7845f6ef15a6dae4bf45a8dc
-rw-r--r--  1 icg149 icg1  119 Nov  4 15:48 GITMANIFEST--85e65a54-6bff-49a7-9e62-db59d8912ceb
-rw-r--r--  1 icg149 icg1  119 Nov  4 15:48 GITMANIFEST--85e65a54-6bff-49a7-9e62-db59d8912ceb.bak
-rw-r--r--  1 icg149 icg1    5 Nov  4 15:47 MD5E-s5--2b00042f7481c7b056c4b410d28f33cf.txt
Comment by matrss Mon Nov 4 14:52:55 2024

Hi,

Seem like I've managed to fix it by openning .git/HEAD file with VIM, and then changing:

ref: refs/heads/views/main(tag=_)

to this:

ref: refs/heads/main

Saved the file, then ran git stash --all. Which returned the repo to the original state.

Also before git stash --all, I ran git restore . git annex fsck, and git add -A but those didn't change anything.

Comment by psxvoid Mon Nov 4 13:54:23 2024

FWIW, I keep running into this. Re

But: If this change were made, it would risk breaking existing working setups, that happen to have a push url that points to a different repository.

pushurl could take precedence, as overwrite the pushInsteadOf mapped value (did not check what git's behavior in presence of both pushurl and pushInsteadOf).

Comment by yarikoptic Sun Nov 3 14:48:03 2024

Hi,

One option is to use bindfs --resolve-symlinks FUSE to resolve simlinks, and then share the "un-symlinked" directory to whatever service you want, including NFS.

Personally, I'm using bindfs --resolve-symlinks with nextcloud instance - works great.

Comment by psxvoid Sun Nov 3 13:44:44 2024
FWIW, since I failed to install git-annex on boox, thought to attempt importtree/exporttree webdav special remote so was also displeased to see that importtree is not supported :-/
Comment by yarikoptic Fri Nov 1 02:29:23 2024

Reproducible here. Also this does break getting files from encrypted special remotes, not just the test suite.

The file is mode 444, which is a permission git-annex sets. Then git-annex opens it for write. Oops..

Bisected to 835283b86240ef4b68c44f4332ac1b644e08e49f.

Fixed.

Comment by joey Mon Oct 28 15:29:56 2024

git-annex builds with stack have used tls-2.0.x since August. I think many other builds are still using older tls from before this change, eg Debian is still on tls-1.8.0.

So it's possible that more outdated servers will be causing problems as things continue to upgrade. It seems worth leaving this bug open for now.

Also it seems pretty clear that TLS 1.2 without EMS is insecure, but I don't know if the insecurity is of a kind that is likely to affect git-annex users. Bearing in mind that git-annex can be used to upload sensitive information to HTTP servers, caution is warrented.

Comment by joey Tue Oct 22 17:39:17 2024