Please describe the problem.

When trying to add a large file (compared to the amount of RAM on the machine) to a v6 repository using git add large_file.iso, I get fatal: Out of memory, realloc failed. Adding the file with git annex add large_file.iso works fine.

What steps will reproduce the problem?

On a machine with 4 GiB of RAM, I'm trying to add a 2.8 GiB file called large_file.iso to a freshly created v6 repository:

$ mkdir annex
$ cd annex
$ git init
$ git annex init --version=6
$ cp /tmp/large_file.iso .
$ git add large_file.iso
fatal: Out of memory, realloc failed

What version of git-annex are you using? On what operating system?

  • git-annex version: 6.20170101
  • git version 2.11.0
  • Debian GNU/Linux 9.0 (stretch)

I also tried git-annex 6.20170321, with no success.

Please provide any additional information below.

$ GIT_TRACE=1 git add large_file.iso 
06:48:23.048295 git.c:371               trace: built-in: git 'add' 'large_file.iso'
fatal: Out of memory, realloc failed
06:48:23.048725 run-command.c:350       trace: run_command: 'git-annex smudge --clean '\''large_file.iso'\'''

So the problem seems to be with Git trying to allocate memory the size of the file, rather that git-annex itself:

$ ltrace git add large_file.iso 2>&1 | grep realloc | tail -n3
realloc(0, 2902458369)                           = 0
realloc(0, 2902458369)                           = 0
fputc('\n', 0x7ff2c98ce520Out of memory, realloc failed
$ git-annex smudge --clean large_file.iso < large_file.iso 
/annex/objects/SHA256E-s2902458368--f8ef73594445624eabc1ea4a567ebedb8841fd248347c1aa01538a8de705c5c1.iso

I have no idea why it needs to do that, though.

Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)

I've been using git-annex v5 repositories without any issues, and with smaller files, repository v6 works great!