Please describe the problem.

If I run git annex fsck with the parallel jobs switch (e.g. -J2) it fails when it encounters files with non-ASCII filenames. It works fine without this switch.

What steps will reproduce the problem?

Make a git annex repository with non-ascii filenames, add the files, and then run fsck with the parallel switch. A script to do this is included below.

What version of git-annex are you using? On what operating system?

Arch Linux (64bit)

git-annex version: 6.20160114-g297a744 build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3(multipartupload) WebDAV Inotify DBus DesktopNotify XMPP ConcurrentOutput DNS Feeds Quvi TDFA TorrentParser key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external local repository version: 5 supported repository versions: 5 6 upgrade supported from repository versions: 0 1 2 4 5

Please provide any additional information below.

#!/bin/bash
set -o errexit -o nounset

mkdir testing-repo
cd testing-repo
git init
git annex init testing-repo

make_fake_file() {
  local filename="$1"
  mkdir -p "$(dirname "$filename")"
  echo "hello world" > "$filename"
  git annex add "$filename"
}

export -f make_fake_file
parallel -j1 'make_fake_file {}' <<EOF
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/folder.jpg
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/01 - 遥か彼方 - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/02 - 未来の破片 - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/03 - アンダースタンド - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/04 - 君という花 - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/05 - リライト - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/06 - 君の街まで - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/07 - ループ & ループ - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/08 - ブラックアウト - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/09 - ブルートレイン - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/10 - 或る街の群青 - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/11 - アフターダーク - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/12 - 転がる岩、君に朝が降る - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/13 - ムスタング - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/14 - 藤沢ルーザー - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/15 - 新世紀のラブソング - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/16 - ソラニン - ASIAN KUNG-FU GENERATION.flac
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/17 - マーチングバンド - ASIAN KUNG-FU GENERATION.flac
EOF

git commit -m 'Add testing files.'
git annex fsck -J2

On my system this produces:

fsck ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/01 - git-annex: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character)
FAIL 1

(The FAIL 1 output is just my terminal printing that the exit code was 1)

Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)

Plenty. In fact I've been using it for a long time - I just only recently tried to use -J2 to speed up the fsck'ing :)

I've worked around this by detecting the non-unicode locale and avoiding the fancy concurrent output which needs it. So -J will work, just not with concurrent progress. I think this is the best that can be done reasonably, so done. --Joey