Recent comments posted to this site:

I don't understand what you mean with the find command. Are you talking about the "unknown" values in the json?

Oh, I suppose you mean particularly that the bytesize is unknown.

Well, this would make find output differ depending on whether the key is present or not, in cases where it would otherwise be the same. And it would change machine-consumable output in a way that for all I know would break stuff.

So, changing that seems like a bad idea.

Comment by joey Mon Nov 20 18:18:07 2017

Shoudn't the same apply to "find" output?

$> git annex find --not --in localhost --json Brainhack2012__Share_your_tools__But_fear_the_wombat_.mp4
{"bytesize":"unknown","mtime":"unknown","keyname":"quvi:https://www.youtube.com/watch,63v,618t6znEOEDVo","backend":"URL","key":"URL--quvi:https://www.youtube.com/watch,63v,618t6znEOEDVo","humansize":"unknown","hashdirmixed":"kq/PM/","file":"Brainhack2012__Share_your_tools__But_fear_the_wombat_.mp4","hashdirlower":"8ba/bf9/"}

$> sudo dpkg -i /tmp/git-annex-standalone_6.20171114+gitg5e6c3ba30-1\~ndall+1_amd64.deb 
[sudo] password for yoh: 
(Reading database ... 706557 files and directories currently installed.)
Preparing to unpack .../git-annex-standalone_6.20171114+gitg5e6c3ba30-1~ndall+1_amd64.deb ...
Unpacking git-annex-standalone (6.20171114+gitg5e6c3ba30-1~ndall+1) over (6.20171018+gitgbb20b1ed3-1~ndall+1) ...
Setting up git-annex-standalone (6.20171114+gitg5e6c3ba30-1~ndall+1) ...
...

$> git annex find --not --in localhost --json Brainhack2012__Share_your_tools__But_fear_the_wombat_.mp4
{"bytesize":"unknown","mtime":"unknown","keyname":"quvi:https://www.youtube.com/watch,63v,618t6znEOEDVo","backend":"URL","key":"URL--quvi:https://www.youtube.com/watch,63v,618t6znEOEDVo","humansize":"unknown","hashdirmixed":"kq/PM/","file":"Brainhack2012__Share_your_tools__But_fear_the_wombat_.mp4","hashdirlower":"8ba/bf9/"}

$> ls -lL Brainhack2012__Share_your_tools__But_fear_the_wombat_.mp4
-r-------- 1 yoh yoh 64354577 Mar 22  2014 Brainhack2012__Share_your_tools__But_fear_the_wombat_.mp4
Comment by yarikoptic Thu Nov 16 19:11:51 2017

I suppose that, since some remotes don't have progress display implemented, in paricular some external special remotes, there's no point in worrying about interface consistency. So, may as well display progress when we can.

Comment by joey Tue Nov 14 20:17:10 2017

I'm nearly 100% certain I did a drop --force by mistake. I'm pulling the files from backups.

thank you!

Comment by astrophoenix Tue Nov 14 19:45:41 2017

Messages.Progress.metered is what looks at the keySize, and when it's not known, displays no meter. So it would need an additional Maybe FilePath that's the file being uploaded, to look at when the keySize is not known.

That does not seem too hard a change to make; I'm not convinced the extra complexity is worth it, since this would only add progress for uploads, and not for downloads.

There is something to be said for consistency; and if some transfers of a key have progress and others do not, it seems the user might get confused, while if nothing does, the user can conclude that git-annex is not able to provide progress for a key that does not contain a size, and if they don't like that, avoid things that generate such keys.

Comment by joey Tue Nov 14 19:29:21 2017

It indeed would be possible for copy --to to check the actual file size when the key does not have a known size, and use that for progress. I don't know how hard it would be.

Note that, even if that were done, there's no guarantee that a given remote will update progress information, and if it doesn't, --json-progress won't result in any. So your code certianly needs to handle that case.

Comment by joey Tue Nov 14 19:19:30 2017

There are two possible things that could have happened.

  1. Maybe something somehow wrong got committed to the git repository for the missing files.

    If so, you can always use git to check out the older version before that commit, and will then be able to see the files; git-annex will know where the content of them is, etc. You can also use git revert to revert such a bad commit.

  2. Maybe you accidentially dropped the content of the files with --force, and if so, you may have lost the content.

    If you run git annex log on one of the files, and look at the lines starting with "-", you can see when the file content was removed from repositories. But there's no way to get it back unless you have another copy somewhere.

Comment by joey Tue Nov 14 17:16:54 2017

The git bug is fixed in git's pu branch, or as Peff says, at least "papered over", and I've tested the fix with git-annex and it seems to avoid the problem. So as long as 5eb0e541132208a9027cc090943276fa52f29c71 gets into a git release, this bug can be closed without any changes to git-annex.

Comment by joey Mon Nov 13 17:09:31 2017

oh, it might have been something dumb i did:

at some point I wanted to delete a few files, so trying to be cute I did this:

m=OneMovie.m4v git annex drop --force $m; git rm $m; git ci -m "rm $m"

I noted that didn't work, but didn't realize it might've taken out most/all the files in the folder

Comment by astrophoenix Sun Nov 12 21:01:28 2017

I just discovered that cloning over ssh an gcrypt encrypted repository and enabling the remote afterwards is somehow messing up the git config:

git clone grypt::ssh://user@ip.com:/mnt/encrypted_backup cd encrypted_backup git annex enableremote encrypted_backup gitrepo=/.../encrypted_backup

leads to following in the .git/config of the just cloned repository:

... [remote "origin"] url = grypt::ssh://user@ip.com:/mnt/encrypted_backup gcrypt-id = :id:12312312 fetch = +refs/heads/:refs/remotes/origin/

[remote "encrypted_backup"] url = grypt::ssh://user@ip.com:/mnt/encrypted_backup fetch = +refs/heads/:refs/remotes/server/ gcrypt-participants = keyid gcrypt-signingkey = keyid gcrypt-publish-participants = true gcrypt-id = :id:adsasd annex-gcrypt = shell annex-uuid = 312312312 ...

Note, that for the remote "origin" some config like the signingkey is missing compared to the remote "encrypted_backup"

Then, running git annex sync --content

leads to a error saying

"gcrypt: Failed to decrypt manifest!"

during the push process. After that I am not able to sync the repository anymore, even with the original repostitory, which initiated the remote. The encrypted_backup is then somehow messed up.

Removing the "origin" remote via git remote remove origin

solves the problem for me. But that command has to be launched right before the first sync, pull or push command! Otherwise the sync process cannot be done anymore.

Comment by spam Sat Nov 11 19:51:56 2017