Hm, IMHO such a “dry-run” flag would be most beneficial for better understanding the preferred/required content settings and try their effect while working on finding the right expression. Therefore, the “drop”, “get” and “sync --content” commands would benefit from such a flag, especially in combination with flags such as “--auto”. Maybe it's just me, but I am not able to utilize the “find” command in a way that it respects preferred content and numcopies and further settings in the same way as the three commands above would do, and testing preferred content expressions is therefore a blind flight.

Another use case for a user-defined string field in a key is, for URL or WORM keys, to include additional information unique to the object. E.g. for an s3:// URI (handled by an external special remote) could embed the ETag in the key. Or for a dx:// URI to include the DNAnexus file ID .

From key format, it seems like adding -uXXXXXX- (with XXXXXX a user-specified string) would not break compatibility?

What is the intended use case for CHECKURL-MULTI? Are there examples of external special remote implementations that use this response?

I am stupid talking about executable files hardlinking. I think I just chmod-ed already hardlinking files, that's how I got it. No surprise.

I am ok with this quirk (executable files are not thinned), but just curious: what exactly influenced such design decision?

When two files have the same content, annex.thin will only make one of them be a hard link to the annex object. The other file will have its own redundant copy of the content. This is the only way to prevent an edit to one file immediatly changing the other file, which would be very surprising behavior.

Ah, that explains my biggest confusion. Thanks!

When the file in git has the executable bit set, annex.thin is not honored for that file either. That's a lot simpler than juggling permissions around.

Ok, I will take a note. But was it this way from the very beginning, or is this a later-added idea? From my observations, this is not true. I do have a large annex repo, I have done a global chmod -R u=rwx,g=rx,o=rx *, all my files have exec bit set and all of them are hardlinked (except for the files which are duplicates of other hardlinked files). I don't remember for sure, it might be that I've chmod-ed files in .git/annex/objects as well

When two files have the same content, annex.thin will only make one of them be a hard link to the annex object. The other file will have its own redundant copy of the content. This is the only way to prevent an edit to one file immediatly changing the other file, which would be very surprising behavior.

When the file in git has the executable bit set, annex.thin is not honored for that file either. That's a lot simpler than juggling permissions around.

Does that explain everything you're seeing, or is there still a bug buried in that transcript? I have not had time to read the whole thing.

I implemented git annex adjust --hide-missing. It can be run after any change to content availability to update the adjusted branch, hiding or unhiding files.

My implementation is not as fast as it could be; each update is a linear scan of the whole original branch and rebuild of the adjusted branch. But it all works so we'll defer speeding this up to later bug reports about it being too slow.

Good news everyone! git annex adjust --hide-missing sets up a branch where only files with content available are included. And once in such a branch, git annex sync updates it as necessary to reflect changes in content availability.

Good news everyone! git annex adjust --hide-missing sets up a branch where only files with content available are included. And once in such a branch, git annex sync updates it as necessary to reflect changes in content availability.

I had a similar problem yesterday with the latest x86-64 standalone build, git-annex ver. 6.20180927-g897e5ba57

Short story

I solved this adding this to my .bashrc

export LC_ALL=C

Non so short story

I need the standalone version for I'm using an hosting service with ssh connection; it's just a "classic" LAMP hosting space, not a full VM, so I cannot install any package on it

In 2016 I was able to install the standalone git-annex version 6.20161231-gc8eeb17da, along with gitolite for access control, and all was working well until recently on one of my machines I upgraded to ver. 6.20180913-1 amd64: trying to get some content always failed with

  [2018-10-17 17:56:52.199123421] P2P > ERROR auth failed

  fd:19: hClose: resource vanished (Broken pipe)
  failed

After a brief search I found get over ssh fails with fd:19: hClose: resource vanished - comment 1 and decidet to upgrade git-annex server side

After upgrade I got the error reported in this bug report subject and after a quick research I found a similar report on askubuntu

The first thing I tried was

LC_ALL=C git annex version

and I the error "disappeared": bingo!

I realized that LC_ALL was unset in my gitolite user profile, so I added it to my .bashrc and now all is working (almost) fine.

Why almost?

Well: it's surely not related to git-annex but to my hosting server.

Every time I get some file from the remote on that hosting space I get an error

 ERROR: ld.so: object '/lib/security/hosting-securize.so' from /etc/ld.so.preload cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.

but fortunately it is ignored and the files are downloaded as expected

I still had no time to investigate why I get a ELFCLASS32 (I'm sure I installed the amd64 standalone)... but since it works I leave it as a background wishlist

The end

That's all, I hope this will help others with similar issues

Bye! Giovanni Biscuolo