forum/Issues with IPFS special remotegit-annexhttp://git-annex.branchable.com/forum/Issues_with_IPFS_special_remote/git-annexikiwiki2021-06-21T17:21:59Zcomment 1http://git-annex.branchable.com/forum/Issues_with_IPFS_special_remote/comment_1_74c586e5af079f0d6c42975ac294e766/Lukey2021-06-18T14:50:45Z2021-06-18T14:50:44Z
You have to setup a way so the two remotes can push/pull the git data itself too. You could add each other as git remotes or setup a git server somewhere that both repos talk to.
comment 2http://git-annex.branchable.com/forum/Issues_with_IPFS_special_remote/comment_2_283aafce5dc1735310daa8329a432002/kinshukkashyap.me2021-06-18T15:11:23Z2021-06-18T15:11:23Z
<p>Yes I have a repository on Github. Starting with an empty repo, I clone it on both machines, then run all the git-annex commands (first enabling the remote, then adding the file, then copy to remote, then sync).
Is this what you meant?</p>
setting up synchttp://git-annex.branchable.com/forum/Issues_with_IPFS_special_remote/comment_3_fa017d01e9eca59074adf2224e20994b/Ilya_Shlyakhter2021-06-18T18:54:59Z2021-06-18T18:54:59Z
Are you running <code>git-annex-sync</code> with the <code>--content</code> flag?
comment 4http://git-annex.branchable.com/forum/Issues_with_IPFS_special_remote/comment_4_e49c62ca2e0f0d455469d5bc4483aff3/kinshukkashyap.me2021-06-19T10:22:58Z2021-06-19T10:22:58Z
No, I'm not using the <code>--content</code> flag. I thought that was for syncing content as well, rather than just the locations of files?
comment 5http://git-annex.branchable.com/forum/Issues_with_IPFS_special_remote/comment_5_2147808f9b15be995988e2c71bc0b271/Lukey2021-06-19T12:14:33Z2021-06-19T12:14:33Z
<p>Hmm, it should work as you described it. Did you run <code>git annex sync</code> on the 2nd repo <em>after</em> running <code>git annex sync</code> on the 1st repo (where you copied the files to the ipfs remote)? Can you post the output of the following actions, in that order:</p>
<pre><code>repo1# git annex copy somefile --to=ipfs
repo1# git annex sync
repo2# git annex sync
</code></pre>
comment 6http://git-annex.branchable.com/forum/Issues_with_IPFS_special_remote/comment_6_3a34db5281419578201f5f74cf97a830/Ilya_Shlyakhter2021-06-19T18:31:59Z2021-06-19T18:31:59Z
<blockquote><p>No, I'm not using the --content flag. I thought that was for syncing content as well, rather than just the locations of files?</p></blockquote>
<p>You're right, sorry, I misread your post. For syncing just the location info <code>--content</code> is not needed.</p>
comment 7http://git-annex.branchable.com/forum/Issues_with_IPFS_special_remote/comment_7_765fef6f0b5da3e5c87c029fea2362a1/kinshukkashyap.me2021-06-20T06:54:37Z2021-06-20T06:54:37Z
<p>Here's a detailed sequence of what I'm doing:
I made an empty repository on github. Then, on machine #1:</p>
<pre><code>$ git init
Reinitialized existing Git repository in /home/kinshuk/code/annex-again/.git/
$ git annex init origin
init origin (scanning for unlocked files...)
Remote origin not usable by git-annex; setting annex-ignore
ok
(recording state in git...)
$ git annex initremote ipfs type=external externaltype=ipfs encryption=none
initremote ipfs ok
(recording state in git...)
$ vim test1.txt
$ git annex add test1.txt
add test1.txt
ok
(recording state in git...)
$ git annex copy --to ipfs
copy test1.txt (to ipfs...)
ok
(recording state in git...)
$ git annex whereis
whereis test1.txt (2 copies)
27123293-bec2-49a6-ac67-8c87fe72ffd6 -- origin [here]
e7582073-415a-48d2-9dec-29c58311fd21 -- [ipfs]
ipfs: ipfs:QmUk36hewmeivRKf1LKYvn2sG1ALqnt2NtAxaJAHJrrSH8
ok
$ git annex sync
commit
[master (root-commit) d2860cd] git-annex in origin
1 file changed, 1 insertion(+)
create mode 120000 test1.txt
ok
pull origin
ok
push origin
Enumerating objects: 26, done.
Counting objects: 100% (26/26), done.
Delta compression using up to 4 threads
Compressing objects: 100% (20/20), done.
Writing objects: 100% (26/26), 2.33 KiB | 1.16 MiB/s, done.
Total 26 (delta 3), reused 0 (delta 0)
remote: Resolving deltas: 100% (3/3), done.
To https://github.com/plsrgb/annex-again
* [new branch] git-annex -> synced/git-annex
* [new branch] master -> synced/master
ok
$ git annex log
+ Sun, 20 Jun 2021 12:02:21 IST test1.txt | e7582073-415a-48d2-9dec-29c58311fd21 -- [ipfs]
+ Sun, 20 Jun 2021 12:02:08 IST test1.txt | 27123293-bec2-49a6-ac67-8c87fe72ffd6 -- origin
</code></pre>
<p>Then I clone the repository from github on machine #2:</p>
<pre><code>$ git annex sync
commit
On branch synced/git-annex
Your branch is up to date with 'origin/synced/git-annex'.
nothing to commit, working tree clean
ok
pull origin
ok
push origin
Enumerating objects: 9, done.
Counting objects: 100% (9/9), done.
Delta compression using up to 12 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 750 bytes | 750.00 KiB/s, done.
Total 6 (delta 2), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To https://github.com/plsrgb/annex-again
c95a7b3..d9cc5d1 git-annex -> synced/git-annex
* [new branch] synced/git-annex -> synced/synced/git-annex
ok
To https://github.com/plsrgb/annex-again ! [rejected] synced/git-annex -> synced/git-annex (non-fast-forward)error: failed to push some refs to 'https://github.com/plsrgb/annex-again'hint: Updates were rejected because the tip of your current branch is behindhint: its remote counterpart. Integrate the remote changes (e.g.hint: 'git pull ...') before pushing again.hint: See the 'Note about fast-forwards' in 'git push --help' for details.
</code></pre>
<p>To resolve this, I run git pull, then run sync again:</p>
<pre><code>$ git pull
Updating c95a7b3..d9cc5d1
Fast-forward
remote.log | 2 +-
uuid.log | 1 +
2 files changed, 2 insertions(+), 1 deletion(-)
$ git annex sync
commit
On branch synced/git-annex
Your branch is up to date with 'origin/synced/git-annex'.
nothing to commit, working tree clean
ok
pull origin
ok
push origin
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/plsrgb/annex-again
c95a7b3..d9cc5d1 synced/git-annex -> synced/synced/git-annex
ok
</code></pre>
comment 8http://git-annex.branchable.com/forum/Issues_with_IPFS_special_remote/comment_8_edd64d315651d5c7d5e1f802b68b8b68/Lukey2021-06-20T07:15:04Z2021-06-20T07:15:04Z
Ah that explains it. You have to switch to the <code>master</code> branch on the 2nd repository with <code>git checkout master</code>.
comment 9http://git-annex.branchable.com/forum/Issues_with_IPFS_special_remote/comment_9_07b1f3c1e744e910f1738925d79e3d1a/kinshukkashyap.me2021-06-20T07:44:34Z2021-06-20T07:44:34Z
<p>That works! That completely slipped my mind.</p>
<p>Thanks a lot!</p>
comment 10http://git-annex.branchable.com/forum/Issues_with_IPFS_special_remote/comment_10_d13e439c3c242606d8b4377831e38a2b/joey2021-06-21T16:17:57Z2021-06-21T15:38:06Z
<p>Hmm, github has found some way to break usual git operation here,
by making the git-annex branch be the one that git checks out
on clone.</p>
<p>I was able to reproduce that with the bare repository on github,
the list of branches in their UI started with synced/git-annex,
and cloning that repo resulted in that branch being checked out.</p>
<p>Compare with a bare repository on a regular ssh server, git clone
picks master as the head branch, because the bare git repo has HEAD
set to master. (That happens at init time, the init
branch name is configurable now.)</p>
<p>Seems that github is assuming that the first branch pushed is the head
branch. It may be that's the best they can do, since the git push protocol
does not specify what the HEAD is, and they don't want to default to a name
like "master".</p>
<p>If git-annex sync pushed master before git-annex and synced/master, it
would avoid this problem. But pushing master to a non-bare remote often
fails due to non-fast-forward, and so the stderr of the master push is
discarded to avoid displaying an error message. Which also
prevents seeing the progress display for that push. So, it pushes
synced/master before master in order for the data transfer and progress
display to be done there. It is possible to at least push git-annex
after synced/master, but that will leave github defaulting to synced/master
in this case, which remains weird behavior. Oh well, it's a behavior
apparently specific to github, and I can't see a good way for git-annex
to avoid it given its other constraints.</p>
comment 11http://git-annex.branchable.com/forum/Issues_with_IPFS_special_remote/comment_11_e53b0158471605950896cafac7aac4da/joey2021-06-21T17:21:59Z2021-06-21T16:25:13Z
Changed the sync order as I discussed.