Since upgrading to git-annex 7.20190912, when doing git status I keep getting the message Refresh index:, and there is a delay until it turns into something like Refresh index: 100% (601422/601422), done.. I don't recall this happening with earlier git-annex versions. Have others seen this? (Not a "bug" in terms of correctness, but posting here as it affects usability and speed.)
This was fixed in faf84aa5c2b963a41e56d02d6bbd2fbda7da9ceb, as far as handling of unlocking of many files goes.
For other cases where git needs to smudge a lot of files, in v9 and above repos,
git-annex filter-processis used, which can speed it up significantly.While it seems that the bug submitter never followed up with what they were doing that caused the problem, this seems likely enough fixed to call it done. --Joey
When using v7 repositories, git will do this when there are unlocked files that have had their content changed.
git only displays that message when it thinks its going to have to do a lot of work (re-smudging a lot of files), so you'd mostly see this happen in a larger repository that has had a lot of (unlocked) files change in some way.
I noticed that
git annex dropof an unlocked file causes the nextgit statusto re-smudge the file. That's surprising because git-annex internally updates the index using git update-index, so git should not see any need to revisit it.So my guess is you were getting or dropping a lot of unlocked files when you saw that.
Dropping a single unlocked file and then running git status with
GIT_TRACE=1replicates the additional smudge every time.The index before and after git status smudging have identical content, so git-annex seems to have updated it correctly already. It seems that the mtime of the index file is causing git to do that one additional smudge.
Eg, the file git-annex dropped had a mtime of 13:02:43.716964483. The ~/index file that git-annex generated has a mtime of 13:02:43.752964130 which is newer, but only by a fraction of a second. So, git probably assumes the mtimes are sufficiently close that it can't trust that the index file really reflects the current content of the work tree file. And so it re-smudges the work tree file unncessarily.
I have not been able to find a number of files to drop that replicates the bug report. When a lot of files are dropped, it starts taking sufficiently long to update the index file that it ends up with a newer timestamp, which avoids the unnecessary additional smudging. The worse case I have found here is dropping 9 files causes all 9 to get re-smudged, but that's not enough to get git to use the progress display.
Anyway, if git-annex could preserve the mtime of an unlocked file when writing its pointer file or when populating it with content, that would avoid the unnecessary smudging. (Which seems better than adding a delay when updating the index file, or setting the index's mtime ahead of the current time..)
That's easier done for pointer files than populated files, because with annex.thin the content is a hard link and it would probably not be good to change its mtime.
For now, I didn't do it extensively, but only in depopulatePointerFile. That was enough to eliminate the unnecessary smudging after drop that I was seeing.
There is not enough information in this bug report for me to know if the problem I fixed was the problem that was reported.
I'm going to assume so, if there is no followup soon.
Here's the one. Now that a dozen changed files are committed, it doesn't happen. As long as they were modified, it happened every time. (On the other repo with 100,000+ files, it takes several minutes to complete this check. Absolutely kills my git aware shell prompt.)