Idly syncing back and forth creates commits

Say I have a git annex repository with one regular remote. The status of both remotes is clean.

If I run git annex sync in the main remote, it will not create any commits.

However, running sync alternatively in both remotes will keep creating regular and merge commits.

I wonder if it would be possible to avoid this behavior.

RSS Atom

comment 1

This is expected if you have any special remotes with remote..annex-tracking-branch configured.

Comment by Lukey — Thu Sep 15 15:55:31 2022

Remove comment

comment 2

I did not configure any special remotes. The only ones that are shown by git annex info are web and bittorrent, but I believe that this is the default. Are these the culprit?

Comment by eph — Fri Sep 16 09:30:43 2022

Remove comment

comment 3

The built in web and bittorrent special remotes are not at fault.

I don't know what is causing this, but am interested in finding out. I have not seen this behavior in a long time. Back when git-annex had direct mode, it did sometimes behave this way, IIRC, but that was a long time ago. What version of git-annex are you using?

It would be helpful if you could show an example of this happening, with details about what the commits look like and what the output is of git-annex sync when they are created.

When I try this, it doesn't happen:

joey@darkstar:~/tmp/bench/bar>cd ../foo
joey@darkstar:~/tmp/bench/foo>git-annex sync
commit
On branch master
nothing to commit, working tree clean
ok
pull bar
ok
joey@darkstar:~/tmp/bench/foo>cd ../bar
joey@darkstar:~/tmp/bench/bar>git-annex sync
commit
On branch master
nothing to commit, working tree clean
ok
pull foo
ok
joey@darkstar:~/tmp/bench/bar>cd ../foo
joey@darkstar:~/tmp/bench/foo>git-annex sync
commit
On branch master
nothing to commit, working tree clean
ok
pull bar
ok
joey@darkstar:~/tmp/bench/foo>cd ../bar
joey@darkstar:~/tmp/bench/bar>git-annex sync
commit
On branch master
nothing to commit, working tree clean
ok
pull foo
ok

After making a change, it quickly settles down:

joey@darkstar:~/tmp/bench/bar>date > new
joey@darkstar:~/tmp/bench/bar>git-annex add
add new 
ok                                
(recording state in git...)
gijoey@darkstar:~/tmp/bench/bar>git-annex sync
commit 
[master f9e0940] git-annex in bar
 1 file changed, 1 insertion(+)
 create mode 120000 new
ok
pull foo 
ok
push foo 
Enumerating objects: 16, done.
Counting objects: 100% (16/16), done.
Delta compression using up to 4 threads
Compressing objects: 100% (8/8), done.
Writing objects: 100% (11/11), 969 bytes | 484.00 KiB/s, done.
Total 11 (delta 3), reused 0 (delta 0), pack-reused 0
To ../foo
   eeaea21..4127372  git-annex -> synced/git-annex
   e8cb8fe..f9e0940  master -> synced/master
ok
joey@darkstar:~/tmp/bench/bar>cd ../foo
joey@darkstar:~/tmp/bench/foo>git-annex sync
commit 
On branch master
nothing to commit, working tree clean
ok
merge synced/master 
Updating e8cb8fe..f9e0940
Fast-forward
 new | 1 +
 1 file changed, 1 insertion(+)
 create mode 120000 new
ok
pull bar 
From ../bar
   a5cd9dd..4127372  git-annex     -> bar/git-annex
   e8cb8fe..f9e0940  master        -> bar/master
   e8cb8fe..f9e0940  synced/master -> bar/synced/master
ok
joey@darkstar:~/tmp/bench/foo>cd ../bar
joey@darkstar:~/tmp/bench/bar>git-annex sync
commit 
On branch master
nothing to commit, working tree clean
ok
pull foo 
From ../foo
   e8cb8fe..f9e0940  master     -> foo/master
ok

Making different changes in the two repositories at the same time similarly settles down quickly, after one merge commit.

Comment by joey — Fri Sep 16 16:40:02 2022

Remove comment

comment 4

Hi joey,

thanks for looking into this. I am using git-annex from Debian stable. No special configuration. The following complete log demonstrates what I mean. I only edited my real name and email address out from the git log.

cwg@drac:/tmp% git annex version
git-annex version: 8.20210223
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: linux x86_64
supported repository versions: 8
upgrade supported from repository versions: 0 1 2 3 4 5 6 7
cwg@drac:/tmp% git init a
Initialized empty Git repository in /tmp/a/.git/
cwg@drac:/tmp% cd a
cwg@drac:/tmp/a% git annex init
init  (scanning for unlocked files...)
ok
(recording state in git...)
cwg@drac:/tmp/a% seq 10 >foo
cwg@drac:/tmp/a% git annex add foo
add foo
ok
(recording state in git...)
cwg@drac:/tmp/a% git commit -am 'add foo'
[main (root-commit) f551362] add foo
 1 file changed, 1 insertion(+)
 create mode 120000 foo
cwg@drac:/tmp/a% cd ..
cwg@drac:/tmp% git clone a b
Cloning into 'b'...
done.
cwg@drac:/tmp% cd b
cwg@drac:/tmp/b% git annex init
init  (merging origin/git-annex into git-annex...)
(recording state in git...)
(scanning for unlocked files...)
ok
(recording state in git...)
cwg@drac:/tmp/b% seq 20 >bar
cwg@drac:/tmp/b% git annex add bar
add bar
ok
(recording state in git...)
cwg@drac:/tmp/b% git commit -am 'add bar'
[main d064d80] add bar
 1 file changed, 1 insertion(+)
 create mode 120000 bar
cwg@drac:/tmp/b% git annex copy --to origin bar
copy bar (to origin...)
ok
(recording state in git...)
cwg@drac:/tmp/b% git annex sync
commit
On branch main
Your branch is ahead of 'origin/main' by 1 commit.
  (use "git push" to publish your local commits)

nothing to commit, working tree clean
ok
pull origin
remote: Enumerating objects: 6, done.
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 5 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (5/5), 496 bytes | 496.00 KiB/s, done.
From /tmp/a
   728f1f6..d8ad263  git-annex  -> origin/git-annex
ok
(merging origin/git-annex into git-annex...)
(recording state in git...)
push origin
Enumerating objects: 32, done.
Counting objects: 100% (31/31), done.
Delta compression using up to 4 threads
Compressing objects: 100% (20/20), done.
Writing objects: 100% (24/24), 2.21 KiB | 1.10 MiB/s, done.
Total 24 (delta 6), reused 0 (delta 0), pack-reused 0
To /tmp/a
 * [new branch]      git-annex -> synced/git-annex
 * [new branch]      main -> synced/main
ok
cwg@drac:/tmp/b% cd ../a
cwg@drac:/tmp/a% git annex sync
commit
On branch main
nothing to commit, working tree clean
ok
merge synced/main
Merge made by the 'recursive' strategy.
 bar | 1 +
 1 file changed, 1 insertion(+)
 create mode 120000 bar
ok
cwg@drac:/tmp/a% cd -
/tmp/b
cwg@drac:/tmp/b% git annex sync
commit
On branch main
Your branch is ahead of 'origin/main' by 1 commit.
  (use "git push" to publish your local commits)

nothing to commit, working tree clean
ok
pull origin
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (1/1), 223 bytes | 223.00 KiB/s, done.
From /tmp/a
   f551362..df99aad  main        -> origin/main
   d064d80..df99aad  synced/main -> origin/synced/main

Already up to date!
Merge made by the 'recursive' strategy.

Already up to date.
ok
push origin
Enumerating objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), 248 bytes | 248.00 KiB/s, done.
Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
To /tmp/a
   df99aad..9baa35b  main -> synced/main
ok
cwg@drac:/tmp/b% cd -
/tmp/a
cwg@drac:/tmp/a% git annex sync
commit
On branch main
nothing to commit, working tree clean
ok
merge synced/main
Already up to date!
Merge made by the 'recursive' strategy.
ok
cwg@drac:/tmp/a% cd -
/tmp/b
cwg@drac:/tmp/b% git annex sync
commit
On branch main
Your branch is ahead of 'origin/main' by 1 commit.
  (use "git push" to publish your local commits)

nothing to commit, working tree clean
ok
pull origin
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (1/1), 219 bytes | 219.00 KiB/s, done.
From /tmp/a
   df99aad..c89cab6  main        -> origin/main
   9baa35b..c89cab6  synced/main -> origin/synced/main

Already up to date!
Merge made by the 'recursive' strategy.

Already up to date.
ok
push origin
Enumerating objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), 249 bytes | 249.00 KiB/s, done.
Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
To /tmp/a
   c89cab6..fe3e4ec  main -> synced/main
ok
cwg@drac:/tmp/b% cd -
/tmp/a
cwg@drac:/tmp/a% git annex sync
commit
On branch main
nothing to commit, working tree clean
ok
merge synced/main
Already up to date!
Merge made by the 'recursive' strategy.
ok
cwg@drac:/tmp/a% cd -
/tmp/b
cwg@drac:/tmp/b% git annex sync
commit
On branch main
Your branch is ahead of 'origin/main' by 1 commit.
  (use "git push" to publish your local commits)

nothing to commit, working tree clean
ok
pull origin
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (1/1), 221 bytes | 221.00 KiB/s, done.
From /tmp/a
   c89cab6..4a018a6  main        -> origin/main
   fe3e4ec..4a018a6  synced/main -> origin/synced/main

Already up to date!
Merge made by the 'recursive' strategy.

Already up to date.
ok
push origin
Enumerating objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), 247 bytes | 247.00 KiB/s, done.
Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
To /tmp/a
   4a018a6..8a29705  main -> synced/main
ok
cwg@drac:/tmp/b% git log --graph
*   commit 8a29705a4d1ccb3d2e08284198ac27d974144cee (HEAD -> main, origin/synced/main, synced/main)
|\  Merge: fe3e4ec 4a018a6
| | Author: cwg@drac
| | Date:   Sat Sep 17 23:07:58 2022 +0200
| |
| |     Merge remote-tracking branch 'refs/remotes/origin/main'
| |
| *   commit 4a018a641d75f0d385399c97bc8ec220b25de252 (origin/main, origin/HEAD)
| |\  Merge: c89cab6 fe3e4ec
| |/  Author: cwg@drac
|/|   Date:   Sat Sep 17 23:07:54 2022 +0200
| |
| |       Merge branch 'refs/heads/synced/main'
| |
* | commit fe3e4ec39d818b97a666ebeffeb68f026145493e
|\| Merge: 9baa35b c89cab6
| | Author: cwg@drac
| | Date:   Sat Sep 17 23:07:47 2022 +0200
| |
| |     Merge remote-tracking branch 'refs/remotes/origin/main'
| |
| *   commit c89cab61a76e4488f1c979ba1d7ba0f9c9756e68
| |\  Merge: df99aad 9baa35b
| |/  Author: cwg@drac
|/|   Date:   Sat Sep 17 23:07:40 2022 +0200
| |
| |       Merge branch 'refs/heads/synced/main'
| |
* | commit 9baa35b4a28cd0f4002a90ea4670a21896b1e5a0
|\| Merge: d064d80 df99aad
| | Author: cwg@drac
| | Date:   Sat Sep 17 23:07:32 2022 +0200
| |
| |     Merge remote-tracking branch 'refs/remotes/origin/main'
| |
| *   commit df99aad6ba5b4ce375834f909519561b4feaaac1
| |\  Merge: f551362 d064d80
| |/  Author: cwg@drac
|/|   Date:   Sat Sep 17 23:07:16 2022 +0200
| |
| |       Merge branch 'refs/heads/synced/main'
| |
* | commit d064d809e24868543c6609757750a6dc9cd58274
|/  Author: cwg@drac
|   Date:   Sat Sep 17 23:06:15 2022 +0200
|
|       add bar
|
* commit f55136210ce7844a21a415ba0154d136fd15508f
  Author: cwg@drac
  Date:   Sat Sep 17 23:05:09 2022 +0200

      add foo

Comment by eph — Sat Sep 17 21:19:13 2022

Remove comment

comment 5

I can't reproduce that behavior with either the current git-annex head, or with git-annex 8.20210223, running the same sequence of commands.

I think that you have git configured not to do fast-forward merges. If you compare your transcipt with mine (below), your has "Merge made by the 'recursive' strategy" where mine has a Fast-forward.

The fast-forwards cause there to be no remaining differences between the repos, while making a merge commit means there is always a remaining change. Presumably, git would behave the same if you were manually doing git pulls and pushes, rather than using git-annex sync.

Probably you have merge.ff set to false, or some similar configuration.

$ cat ~/sh
#!/bin/sh
git annex version
chmod +w -R a b
rm -rf a b
git init a
cd a
git annex init
seq 10 > foo
git annex add foo
git commit -am 'add foo'
cd ..
git clone a b
cd b
git annex init
seq 20 > bar
git annex add bar
git commit -am 'add bar'
git annex copy --to origin bar
git annex sync
cd ../a
git annex sync
cd ../b
git annex sync
cd ../a
git annex sync
cd ../b
git annex sync
cd ../a
git annex sync
cd ../b
git annex sync
git log --graph
$ sh ~/sh
git-annex version: 8.20210128-g5e5829a8d
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: linux x86_64
supported repository versions: 8
upgrade supported from repository versions: 0 1 2 3 4 5 6 7
Initialized empty Git repository in /home/joey/tmp/git-annex.linux/bench/a/.git/
init  (scanning for unlocked files...)
ok
(recording state in git...)
add foo 
ok                                
(recording state in git...)
[master (root-commit) 9a73c26] add foo
 1 file changed, 1 insertion(+)
 create mode 120000 foo
Cloning into 'b'...
done.
init  (merging origin/git-annex into git-annex...)
(recording state in git...)
(scanning for unlocked files...)
ok
(recording state in git...)
add bar 
ok                                
(recording state in git...)
[master d7d507b] add bar
 1 file changed, 1 insertion(+)
 create mode 120000 bar
copy bar (to origin...) 
ok                                
(recording state in git...)
commit 
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
  (use "git push" to publish your local commits)

nothing to commit, working tree clean
ok
pull origin 
remote: Enumerating objects: 6, done.
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 5 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (5/5), 487 bytes | 81.00 KiB/s, done.
From /home/joey/tmp/git-annex.linux/bench/a
   4153d9e..c70f135  git-annex  -> origin/git-annex
ok
(merging origin/git-annex into git-annex...)
(recording state in git...)
push origin 
Enumerating objects: 32, done.
Counting objects: 100% (31/31), done.
Delta compression using up to 4 threads
Compressing objects: 100% (20/20), done.
Writing objects: 100% (24/24), 2.23 KiB | 2.23 MiB/s, done.
Total 24 (delta 5), reused 0 (delta 0), pack-reused 0
To /home/joey/tmp/git-annex.linux/bench/a
 * [new branch]      git-annex -> synced/git-annex
 * [new branch]      master -> synced/master
ok
commit 
On branch master
nothing to commit, working tree clean
ok
merge synced/master 
Updating 9a73c26..d7d507b
Fast-forward
 bar | 1 +
 1 file changed, 1 insertion(+)
 create mode 120000 bar
ok
commit 
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
  (use "git push" to publish your local commits)

nothing to commit, working tree clean
ok
pull origin 
From /home/joey/tmp/git-annex.linux/bench/a
   9a73c26..d7d507b  master     -> origin/master
ok
commit 
On branch master
nothing to commit, working tree clean
ok
commit 
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
ok
pull origin 
ok
commit 
On branch master
nothing to commit, working tree clean
ok
commit 
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
ok
pull origin 
ok
* commit d7d507b48282c7496063b7fda83eb3b3add12bb9 (HEAD -> master, origin/synced/master, origin/master, origin/HEAD, synced/master)
| Author: Joey Hess <joeyh@joeyh.name>
| Date:   Tue Sep 20 12:08:37 2022 -0400
| 
|     add bar
| 
* commit 9a73c269c77ca51c72a9730364907e32fa7a4b1d
  Author: Joey Hess <joeyh@joeyh.name>
  Date:   Tue Sep 20 12:08:36 2022 -0400

      add foo

Comment by joey — Tue Sep 20 15:53:54 2022

Remove comment

Add a comment