There's something rotten in POSIX fctnl locking. It's not composable, or thread-safe.
The most obvious problem with it is that if you have 2 threads, and they both try to take an exclusive lock of the same file (each opening it separately) ... They'll both succeed. Unlike 2 separate processes, where only one can take the lock.
Then the really crazy bit: If a process has a lock file open and fcntl locked, and then the same process opens the lock file again, for any reason, closing the new FD will release the lock that was set using the other FD.
So, that's a massive gotcha if you're writing complex multithreaded code. Or generally for composition of code. Of course, C programmers deal with this kind of thing all the time, but in the clean world of Haskell, this is a glaring problem. We don't expect to need to worry about this kind of unrelated side effect that breaks composition and thread safety.
After noticing this problem affected git-anenx in at least one place, I have to assume there could be more. And I don't want to need to worry about this problem forever. So, I have been working today on a clean fix that I can cleanly switch all my lock-related code to use.
One reasonable approach would be to avoid fcntl locking, and use flock. But, flock works even less well on NFS than fcntl, and git-annex relies on some fcntl locking features. On Linux, there's an "open file description locks" feature that fixes POSIX fnctl locking to not have this horrible wart, but that's not portable.
Instead, my approach is to keep track of which files the process has locked. If it tries to do something with a lockfile that it already has locked, it avoids opening the same file again, instead implements its own in-process locking behavior. I use STM to do that in a thread-safe manner.
I should probably break out git-annex's lock file handling code as a
library. Eventually.. This was about as much fun as a root canal, and I'm
having a real one tomorrow. 
git-annex is now included in Stackage!
Daniel Kahn Gillmor is doing some work on reproducible builds of git-annex.