Please describe the problem.

When trying to add a large file (compared to the amount of RAM on the machine) to a v6 repository using git add large_file.iso, I get fatal: Out of memory, realloc failed. Adding the file with git annex add large_file.iso works fine.

What steps will reproduce the problem?

On a machine with 4 GiB of RAM, I'm trying to add a 2.8 GiB file called large_file.iso to a freshly created v6 repository:

$ mkdir annex $ cd annex $ git init $ git annex init --version=6 $ cp /tmp/large_file.iso . $ git add large_file.iso fatal: Out of memory, realloc failed

What version of git-annex are you using? On what operating system?

  • git-annex version: 6.20170101
  • git version 2.11.0
  • Debian GNU/Linux 9.0 (stretch)

I also tried git-annex 6.20170321, with no success.

Please provide any additional information below.

$ GIT_TRACE=1 git add large_file.iso 06:48:23.048295 git.c:371 trace: built-in: git 'add' 'large_file.iso' fatal: Out of memory, realloc failed 06:48:23.048725 run-command.c:350 trace: run_command: 'git-annex smudge --clean '\''large_file.iso'\'''

So the problem seems to be with Git trying to allocate memory the size of the file, rather that git-annex itself:

$ ltrace git add large_file.iso 2>&1 | grep realloc | tail -n3 realloc(0, 2902458369) = 0 realloc(0, 2902458369) = 0 fputc('\n', 0x7ff2c98ce520Out of memory, realloc failed

$ git-annex smudge --clean large_file.iso < large_file.iso /annex/objects/SHA256E-s2902458368--f8ef73594445624eabc1ea4a567ebedb8841fd248347c1aa01538a8de705c5c1.iso

I have no idea why it needs to do that, though.

Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)

I've been using git-annex v5 repositories without any issues, and with smaller files, repository v6 works great!

dup; done --Joey