Please add the feature/switch --json-progress to git annex fsck --json.
For a basic version, it would be nice to show the percentage using the number of processed chunks / total number of chunks for a given file while calculating checksum.
Let's not forget that a file on a remote can be fsck-ed when using --from <remote>. In this case if the file is not present locally, the file must be downloaded first (--json-progress is already supported for get). Usually this takes much-much longer as the checksum calculation. When checking a large file (~1Gb) on a not too fast remote, it's quite a time for the first "response" that something is happening when using the --json switch. To be exact, the first and only response is that the check has finished (happy way). It would be nice to see the json progress of the download+checksum calculation.
Test environment:
Docker Community 20.10.5
Ubuntu 20.04.2 LTS (Focal Fossa)
git-annex version: 8.20210224-gf951847c6 (kitenet: SHA256E-s51232145--1e57d2e334cffdb3c3874785cfba4f6d5fc521808d0b1deef4fb68160dd88d48.tar.gz)
done via incremental checksum while downloading, for nearly all remotes. --Joey
Is this possible? Or makes no sense? Do you plan implementing this?
Thank you, Bence
It seems problimatic to have json progress for both downloading the file and for checksumming it. I don't know how that json would look.
In the meantime, though, git-annex has changed to checksum files while they are being downloaded from most remotes, and when it can do that, fsck --from avoids the separate checksumming step.
While some special remotes do not support that, it's a very limited list: only bittorrent on linux, and adb, external, gcrypt, hook, and rsync on other OSs.
This makes me think that complicating the json output to display two different kinds of progress is probably not worth the complication.
The --json-progress flag should produce progress. I do not understand why it would be a problem to intermix types of 'action' in the output. If file A is uploaded and file B is downloaded and checksummed, that's (at least) three different progress objects to produce. This is exactly the type of information I want to see on the screen instead of wondering if it's stuck. I would also apply this pattern to sync and friends.
Upload A 35%
Upload A 100%
Download B 100%
Checksum B 50%
Checksum B 100%
Go ahead and make per-file json error results available and describe in the documentation.