I'm running three repositories (laptop, usb connected to laptop, local server) that are connected via ssh access. Running the webapp on both laptop and server, I'm quite frequently seeing in both webapps:
warning
Pusher crashed: fd:56: hPutStr: illegal operation (handle is closed)
Pushing the "Restart thread" button sometimes dismisses the warning, sometimes it immediatly comes back even when trying several times. A similar warning sometimes shows up for a Netwatcher thread.
Not sure but when that crash occurs tranfers between the repos seem to get stalled.
Server: git-annex 7.20190129-2~bpo9+1 on debian stretch Laptop: git-annex 7.20190129-3 on debian testing
Log snippet:
[2019-09-07 14:10:30.330041206] Committer: Adding jmg_0578.jpg5895 jmg_0578.jpg
add incoming/5d/jmg_0578.jpg [2019-09-07 14:10:30.358484399] Committer: Committing changes to git
(recording state in git...)
[2019-09-07 14:13:01.986780732] Pusher: Syncing with usbimg, cube_media_srv_img
Pusher crashed: fd:56: hPutStr: illegal operation (handle is closed)
[2019-09-07 14:13:02.009823288] Pusher: warning Pusher crashed: fd:56: hPutStr: illegal operation (handle is closed)
[2019-09-07 14:13:49.112731437] RemoteControl: Syncing with cube_media_srv_img
(merging synced/git-annex into git-annex...)
fd:56: hPutStr: illegal operation (handle is closed)
fd:56: hPutStr: illegal operation (handle is closed)
From ssh://git-annex-cube-flixh_22_.2Fmedia.2Fsrv.2Fimg/media/srv/img
3c9104b770..cea5c2f058 annex/direct/master -> cube_media_srv_img/annex/direct/master
71006e08a7..536584bdfa git-annex -> cube_media_srv_img/git-annex
fd:56: hPutStr: illegal operation (handle is closed)
[2019-09-07 14:15:58.002820518] Committer: Adding jmg_0592...ftool_tmp
[2019-09-07 14:15:59.526375687] Committer: Adding jmg_0592...ftool_tmp jmg_0592.cr2
add incoming/5d/jmg_0592.cr2 [2019-09-07 14:15:59.599057025] Committer: Committing changes to git
(recording state in git...)
[2019-09-07 14:15:59.670107852] Pusher: Syncing with usbimg, cube_media_srv_img
Pusher crashed: fd:56: hPutStr: illegal operation (handle is closed)
[2019-09-07 14:15:59.683904185] Pusher: warning Pusher crashed: fd:56: hPutStr: illegal operation (handle is closed)
[2019-09-07 14:18:09.282924907] Committer: Adding CI1026+1375.JPG5895
I'm having the same issue on my machines. Is there anything I can do to help debugging the issue?
The error occurs multiple times per day and git-annex stops working after that (or at least pushing). Unfortunately, it happens so frequently, that I'm not always aware of.
git-annex version
log messages ``` [2020-04-22 23:46:32.097215608] NetWatcherFallback: Syncing with annexhost Everything up-to-date [2020-04-23 00:46:32.359155185] NetWatcherFallback: Syncing with annexhost Everything up-to-date [2020-04-23 00:50:54.17444612] Committer: Adding My_File.org add My_File.org ok [2020-04-23 00:50:54.178821719] Committer: Committing changes to git (recording state in git...) [2020-04-23 00:50:54.203132132] Pusher: Syncing with annexhost (recording state in git...)
M100% 5.67 KiB 119 KiB/s 0sTo ssh://annexhost.mydomain.test:/home/jascha.geerds/Org/Privat 72f666e..6c6173d git-annex -> synced/git-annex ee59f26..dbd86f6 master -> synced/master ^M ^M[2020-04-23 00:50:54.396222895] Transferrer: Uploaded My_File.org [2020-04-23 00:50:56.593376023] Pusher: Syncing with annexhost Pusher crashed: fd:49: hPutStr: illegal operation (handle is closed) [2020-04-23 00:50:56.596448524] Pusher: warning Pusher crashed: fd:49: hPutStr: illegal operation (handle is closed) [2020-04-23 01:44:22.764148489] NetWatcher: Syncing with annexhost mux_client_request_session: read from master failed: Broken pipe Von ssh://annexhost.mydomain.test:/home/jascha.geerds/Org/Privat 6c6173d..eea706b git-annex -> annexhost/git-annex fd:49: hPutStr: illegal operation (handle is closed) git-annex: fd:49: hPutStr: illegal operation (handle is closed) [2020-04-23 16:37:01.848110929] NetWatcherFallback: Syncing with annexhost NetWatcherFallback crashed: fd:49: hPutStr: illegal operation (handle is closed) [2020-04-23 16:37:01.903992912] NetWatcherFallback: warning NetWatcherFallback crashed: fd:49: hPutStr: illegal operation (handle is closed) [2020-04-23 17:00:15.467028948] Committer: Adding My_File.org add My_File.org [2020-04-23 17:00:15.478417574] Committer: Committing changes to git (recording state in git...) [2020-04-23 17:37:12.269224569] Committer: Committing changes to git (recording state in git...) [2020-04-23 17:37:13.377765106] Committer: Committing changes to git ```
I am still suffering from the same problem, git-annex versions are: laptop git-annex version: 8.20210223 server git-annex version: 8.20200330
Also as soon as there is a bit more something to be synced the repositories frequently need to be repaired, which on the server takes days for this 130GB repo.
Anything I can do to help debug this?