Please describe the problem.

I am looking into decrypting files in special remotes without git-annex also with comments in disaster_recovery_with_an_encrypted_special_remote. User is trying to figure out filename encryption scheme on special remotes for sharedpubkey encryption scheme.

sharedpubkey is using a different filename encryption method than shared on special remotes but encryption page implies they should be using the same method: “regular public key encryption with shared filename encryption (encryption=sharedpubkey)”.

What steps will reproduce the problem?

andrew@bumblebee /tmp$ mkdir repo1
andrew@bumblebee /tmp$ cd repo1/
andrew@bumblebee /tmp/repo1$ git init
Initialized empty Git repository in /private/tmp/repo1/.git/
andrew@bumblebee /tmp/repo1$ git annex init
init  ok
(recording state in git...)
andrew@bumblebee /tmp/repo1$ echo "hello a" > a.txt
andrew@bumblebee /tmp/repo1$ git annex add a.txt 
add a.txt ok
(recording state in git...)
andrew@bumblebee /tmp/repo1$ mkdir /tmp/remote1
andrew@bumblebee /tmp/repo1$ git annex initremote remote1 type=directory directory=/tmp/remote1 encryption=sharedpubkey keyid=0C5C0618
initremote remote1 (encryption setup) (to gpg keys: 222365310C5C0618) ok
(recording state in git...)
andrew@bumblebee /tmp/repo1$ git annex copy a.txt --to=remote1
copy a.txt (to remote1...) 
ok                              
(recording state in git...)
andrew@bumblebee /tmp/repo1$ mkdir /tmp/remote2
andrew@bumblebee /tmp/repo1$ git annex initremote remote2 type=directory directory=/tmp/remote2 encryption=shared
initremote remote2 (encryption setup) (encryption key stored in git repository) ok
(recording state in git...)
andrew@bumblebee /tmp/repo1$ git annex copy a.txt --to=remote2
copy a.txt (to remote2...) 
ok                              
(recording state in git...)
andrew@bumblebee /tmp/repo1$ ls -la /tmp/remote1/
432/ tmp/ 
andrew@bumblebee /tmp/repo1$ ls -la /tmp/remote1/432/d7f/GPGHMACSHA1--6e830539e4aac12d435b2abc1f693e0dfacf5e89/GPGHMACSHA1--6e830539e4aac12d435b2abc1f693e0dfacf5e89 
-r--r--r--  1 andrew  wheel  340 Jul 19 13:40 /tmp/remote1/432/d7f/GPGHMACSHA1--6e830539e4aac12d435b2abc1f693e0dfacf5e89/GPGHMACSHA1--6e830539e4aac12d435b2abc1f693e0dfacf5e89
andrew@bumblebee /tmp/repo1$ ls -la /tmp/remote2/
411/ tmp/ 
andrew@bumblebee /tmp/repo1$ ls -la /tmp/remote2/411/ad6/GPGHMACSHA1--0332a66cded87db8ec8427280c171133c958754f/GPGHMACSHA1--0332a66cded87db8ec8427280c171133c958754f 
-r--r--r--  1 andrew  wheel  78 Jul 19 13:40 /tmp/remote2/411/ad6/GPGHMACSHA1--0332a66cded87db8ec8427280c171133c958754f/GPGHMACSHA1--0332a66cded87db8ec8427280c171133c958754f
andrew@bumblebee /tmp/repo1$ ls -la a.txt 
lrwxr-xr-x  1 andrew  wheel  186 Jul 19 13:37 a.txt -> .git/annex/objects/8k/z6/SHA256E-s8--448a19594af45f888493a4cb8b6e12ed42fc773dab6db35a299370ebfc805270.txt/SHA256E-s8--448a19594af45f888493a4cb8b6e12ed42fc773dab6db35a299370ebfc805270.txt
andrew@bumblebee /tmp/repo1$ echo -n "SHA256E-s8--448a19594af45f888493a4cb8b6e12ed42fc773dab6db35a299370ebfc805270.txt" | openssl dgst -SHA1 -hmac "$(git show git-annex:remote.log | grep 'name='"remote2 " | sed -E 's/.*cipher=([^ ]+) .*/\1/' | base64 -D | tr -d '\n' | head  -c 256)"
0332a66cded87db8ec8427280c171133c958754f
andrew@bumblebee /tmp/repo1$ echo -n "SHA256E-s8--448a19594af45f888493a4cb8b6e12ed42fc773dab6db35a299370ebfc805270.txt" | openssl dgst -SHA1 -hmac "$(git show git-annex:remote.log | grep 'name='"remote1 " | sed -E 's/.*cipher=([^ ]+) .*/\1/' | base64 -D | tr -d '\n' | head  -c 256)"
bb3b1f4b27164c2edc81da32c43c64e8a4be75d8
andrew@bumblebee /tmp/repo1$ 

What version of git-annex are you using? On what operating system?

git-annex version: 6.20180626-gdf91a5cff
build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV FsEvents ConcurrentOutput TorrentParser MagicMime Feeds Testsuite
dependency versions: aws-0.17.1 bloomfilter-2.0.1.0 cryptonite-0.23 DAV-1.3.1 feed-0.3.12.0 ghc-8.0.2 http-client-0.5.7.0 persistent-sqlite-2.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
operating system: darwin x86_64
supported repository versions: 3 5 6
upgrade supported from repository versions: 0 1 2 3 4 5
local repository version: 5

You can see that I am able to generate correctly the special remote key used by remote2 the shared remote, 0332a66cded87db8ec8427280c171133c958754f. But using the same method for remote1 the sharedpubkey remote does not yield a meaningful key bb3b1f4b27164c2edc81da32c43c64e8a4be75d8

Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)

Yes. —andrew

done