Please describe the problem.
Syncing a 20GB video file causes this error. I have no problems with 8GB files.
What steps will reproduce the problem?
See additional info
What version of git-annex are you using? On what operating system?
git-annex version: 5.20140920-gb0c4300 build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV DNS Feeds Quvi TDFA CryptoHash key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL remote types: git gcrypt S3 bup directory rsync web webdav tahoe glacier ddar hook external local repository version: 5 supported repository version: 5 upgrade supported from repository versions: 2 3 4
git version 1.9.4.msysgit.2
Windows 7 64bit
Please provide any additional information below.
Z:\>git clone L:\repositories\bigFilesTest.git-annex
Cloning into 'bigFilesTest.git-annex'...
done.
Z:\>cd bigFilesTest.git-annex
Z:\bigFilesTest.git-annex>git annex init "cloned"
init cloned
Detected a filesystem without fifo support.
Disabling ssh connection caching.
Detected a crippled filesystem.
Enabling direct mode.
ok
(Recording state in git...)
Z:\bigFilesTest.git-annex>git annex add test20GBVideo.mkv
add test20GBVideo.mkv ok
(Recording state in git...)
Z:\bigFilesTest.git-annex>git annex sync --debug
[2014-10-18 15:39:02 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","show-ref","git-annex"]
[2014-10-18 15:39:02 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","show-ref","--hash","refs/heads/git-annex"]
[2014-10-18 15:39:02 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","log","refs/heads/git-annex..54de336a3423f7f8f72f897effd29f952534c24e","-n1","--pretty=%H"]
[2014-10-18 15:39:02 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","log","refs/heads/git-annex..53cfcf38b40247b3992b6007336b2c915a945ad4","-n1","--pretty=%H"]
[2014-10-18 15:39:02 Mitteleuropäische Sommerzeit] chat: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","cat-file","--batch"]
[2014-10-18 15:39:02 Mitteleuropäische Sommerzeit] read: git ["config","--null","--list"]
commit [2014-10-18 15:39:02 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","ls-files","--stage","-z","--others","--exclude-standard","--","Z:\\bigFilesTest.git-annex"]
[2014-10-18 15:39:02 Mitteleuropäische Sommerzeit] chat: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","cat-file","--batch"]
(Recording state in git...)
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] call: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","add","-f","test20GBVideo.mkv"]
fatal: Cannot handle files this big
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","show-ref","--head"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","diff-index","-z","--raw","--no-renames","-l0","--cached","HEAD"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","symbolic-ref","HEAD"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","show-ref","--hash","refs/heads/annex/direct/master"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","write-tree"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","rev-parse","b12e8477242df97be13c1395db143f860ce8e895:"]
ok
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","symbolic-ref","HEAD"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","show-ref","refs/heads/annex/direct/master"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] call: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","show-ref","--verify","-q","refs/heads/synced/master"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","log","refs/heads/annex/direct/master..refs/heads/synced/master","-n1","--pretty=%H"]
pull origin
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] call: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","fetch","origin"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] call: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","show-ref","--verify","-q","refs/remotes/origin/annex/direct/master"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] call: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","show-ref","--verify","-q","refs/remotes/origin/synced/master"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","log","refs/heads/synced/master..refs/remotes/origin/synced/master","-n1","--pretty=%H"]
ok
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","show-ref","git-annex"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","show-ref","--hash","refs/heads/git-annex"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","log","refs/heads/git-annex..54de336a3423f7f8f72f897effd29f952534c24e","-n1","--pretty=%H"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","log","refs/heads/git-annex..53cfcf38b40247b3992b6007336b2c915a945ad4","-n1","--pretty=%H"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] call: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","branch","-f","synced/master"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] call: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","branch","-f","master"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] call: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","show-ref","--verify","-q","refs/remotes/origin/synced/master"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","log","refs/remotes/origin/synced/master..refs/heads/synced/master","-n1","--pretty=%H"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] call: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","show-ref","--verify","-q","refs/remotes/origin/git-annex"]
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","log","refs/remotes/origin/git-annex..git-annex","-n1","--pretty=%H"]
push origin
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] call: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","push","origin","+git-annex:synced/git-annex","annex/direct/master:synced/master"]
Everything up-to-date
[2014-10-18 15:39:03 Mitteleuropäische Sommerzeit] read: git ["--git-dir=Z:\\bigFilesTest.git-annex\\.git","--work-tree=Z:\\bigFilesTest.git-annex","-c","core.bare=false","push","origin","master"]
ok
I recently fixed a bug that would likewise cause git-annex to fail on windows with a large file, although I didn't see it failing with this message.
There's a good chance that git-annex version 5.20150205 might fix this problem.
I was able to successfully use git-annex with a 6 gb file, after the fixes.
Please check, and followup. I'm closing this bug provisionally in the meantime.
I have a similar problem on Windows 7 64bit trying to add files that are around 5GB in size. I tried repository version 5 and 6 with same results.
git-annex version: 6.20160229-g37a89cc build flags: Assistant Webapp Pairing Testsuite S3(multipartupload) WebDAV ConcurrentOutput TorrentParser Feeds Quvi key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external local repository version: 5 supported repository versions: 5 6 upgrade supported from repository versions: 2 3 4 5
git (not git-annex) will throw this error if a file size is greater than
size_t.This bug report seemed to originally concern git add being run on such a file, but I can't see how git-annex would do that, it doesn't add large files to git.
I think that in the case of git-annex status, when it runs git status, that looks at work tree files, and so falls over if they're large, even if what's checked into git is a nice small git-annex symlink. This would also probably affect other places where git looks at worktree files, perhaps git diff (in v6 repo mode).
Reopening bug report.