This is where announcements of new releases, features, and other news is posted. git-annex users are recommended to subscribe to this page's RSS feed. Also, see security for security announcements.

RSS Atom Add a new post titled:

git-annex 8.20200908 released with these changes

  • Added httpalso special remote, which is useful for accessing content stored on other remotes that is published by http.
  • The external special remote protocol got an ASYNC extension. This can be used by an external special remote to let a single process perform concurrent actions, rather than multiple processes being started, when that is more efficient.
  • Retry transfers to exporttree=yes remotes same as for other remotes.
  • import: Retry downloads that fail, same as is done for downloads generally.
  • Limit retrying of failed transfers when forward progress is being made to 5, to avoid some unusual edge cases where too much retrying could result in far more data transfer than makes sense.
  • Exposed annex.forward-retry git config, to configure the forward retry behavior that git-annex has had for a long time.
  • sync, assistant, merge: When merge.directoryRenames is not set, default it it to "false", which works better with automatic merge conflict resolution than git's ususual default of "conflict". (This is not done when automatic merge conflict resolution is disabled.)
  • resolvemerge: Improve cleanup of cruft left in the working tree by a conflicted merge.
  • Support git remotes where .git is a file, not a directory, eg when --separate-git-dir was used.
  • Fixed several cases where files were created without file mode bits that the umask would usually set. This included exports to the directory special remote, torrent files used by the bittorrent special remote, hooks written by git-annex init, and some log files in .git/annex/
  • Fix reversion in 7.20190322 that made addurl --file not be honored when youtube-dl was used to download media.
  • Fix reversion in 8.20200617 that made annex.pidlock being enabled result in some commands stalling, particularly those needing to autoinit.
  • Display warning when external special remote does not start up properly, or is not usable.
  • Display a message when git-annex has to wait for a pid lock file held by another process.
  • test: Stop gpg-agent daemons that are started for the test framework's gpg key.
  • Removed the S3 and WebDAV build flags so these special remotes are always supported.
  • stack.yaml: Updated to lts-16.10.
Posted Tue Sep 8 18:21:37 2020

git-annex 8.20200810 released with these changes

  • Added support for external backend programs. So if you want a hash that git-annex doesn't support, or something stranger, you can write a small program to implement it.
  • Fix a bug in find --branch in the previous version.
  • importfeed: Fix reversion that caused some '.' in filenames to be replaced with '_'
  • Fix a lock file descriptor leak that could occur when running commands like git-annex add with -J. Bug was introduced as part of a different FD leak fix in version 6.20160318.
  • Fix a hang when using git-annex with an old openssh 7.2p2, which had some weird inheriting of ssh FDs by sshd. Bug was introduced in git-annex version 7.20200202.7.
  • move, copy --to: Sped up seeking files by 2x.
  • drop: Sped up seeking files to drop by 2x, and also some performance improvements to checking numcopies.
  • Deal with unusual IFS settings in the shell scripts for linux standalone and OSX app. Thanks, Yaroslav Halchenko
  • Avoid complaining that a file with "is beyond a symbolic link" when the filepath is absolute and the symlink in question is not actually inside the git repository.
  • Slightly sped up the linux standalone bundle.
  • Support building with dlist-1.0
Posted Mon Aug 10 23:23:08 2020

git-annex 8.20200720.1 released with these changes

  • Fix a bug in find --batch in the previous version.
Posted Mon Jul 20 23:58:46 2020

git-annex 8.20200720 released with these changes

  • import: Added --no-content option, which avoids downloading files from a special remote. Currently only supported by the directory special remote.
  • Honor annex.largefiles when importing a tree from a special remote. (Except for when --no-content is used.)
  • Fix a deadlock that could occur after git-annex got an unlocked file, causing the command to hang indefinitely. Known to happen on vfat filesystems, possibly others.
  • Build with the http-client-restricted and git-lfs libraries when available, otherwise use the vendored copy as before.
  • testremote: Fix over-allocation of resources and bad caching, including starting up a large number of external special remote processes. (Regression introduced in version 8.20200501)
  • test: Fix some test cases that assumed git's default branch name.
  • importfeed: Added some additional --template variables: itempubyear, itempubmonth, itempubday, itempubhour, itempubminute, itempubsecond.
  • Made several special remotes support locking content on them, which allows dropping from other special remotes in some situations where it was not possible before. Supported special remotes: S3 (with versioning=yes), git-lfs, tahoe
  • Fix reversion that broke passing annex. and remote..annex-* git configs with -c. (Since version 8.20200330.)
  • Bring back git-annex branch read cache. This speeds up some operations, eg git-annex sync --content --all gets 20% faster.
  • Fix a recently introduced bug that could cause a "fork: resource exhausted" after getting several thousand files.
  • Sped up the --all option by 2x to 16x by using git cat-file --buffer. Thanks to Lukey for finding this optimisation.
  • Sped up seeking for annexed files to operate on by a factor of nearly 2x.
  • Sped up sync --content by 2x and other commands like fsck --fast and whereis by around 50%, by using git cat-file --buffer.
  • importfeed: Made checking known urls step around 15% faster.
  • fsck: Detect if WORM keys contain a carriage return, and recommend upgrading the key. (git-annex could have maybe created such keys back in 2013).
  • When on an adjust --hide-missing branch, fix handling of files that have been deleted but the deletion is not yet staged.
Posted Mon Jul 20 18:41:53 2020

git-annex 8.20200617 released with these changes

  • Added annex.skipunknown git config, that can be set to false to change the behavior of commands like git annex get foo*, to not skip over files/dirs that are not checked into git and are explicitly listed in the command line.
  • annex.skipunknown is planned to change to default to false in a git-annex release in early 2022. If you prefer the current behavior, you can explicitly set it to true.
  • Try to enable special remotes configured with autoenable=yes when git-annex auto-initialization happens in a new clone of an existing repo. Previously, git-annex init had to be explicitly run to enable them. Special remotes cannot display anything when autoenabled this way, to avoid interfering with the output of git-annex query commands.
  • export: Added options for json output.
  • import: Added --json-progress.
  • addurl: Make --preserve-filename also apply when eg a torrent contains multiple files.
  • Fix a crash or potentially not all files being exported when sync -J --content is used with an export remote.
  • export: Let concurrent transfers be done with -J or annex.jobs.
  • move --to, copy --to, mirror --to: When concurrency is enabled, run cleanup actions in separate job pool from uploads.
  • init: If lock probing stalls for a long time (eg a broken NFS server), display a message to let the user know what's taking so long.
  • init: When annex.pidlock is set, skip lock probing.
  • Fix file descriptor leak when importing from a directory special remote that is configured with exporttree=yes.
  • Note that external special remote programs should not block SIGINT or SIGTERM.
  • Avoid creating the keys database during init when there are no unlocked files, to prevent init failing when sqlite does not work in the filesystem.
  • import: Avoid using some strange names for temporary keys, which broke importing from a directory special remote onto a vfat filesystem.
  • S3: The REDUCED_REDUNDANCY storage class is no longer cheaper so stop documenting it, and stop offering it as a choice in the assistant.
  • Improve display of problems auto-initializing or upgrading local git remotes.
  • When a local git remote cannot be initialized because it has no git-annex branch or a .noannex file, avoid displaying a message about it.
  • checkpresentkey: When no remote is specified, try all remotes, not only ones that the location log says contain the key. This is what the documentation has always said it did.
  • Fix regression in external special remote handling: GETCONFIG did not return a value that was set with SETCONFIG immediately before. (Regression introduced in version 7.20200202.7)
  • Fix bug that made initremote of extrnal special remotes with embedcreds=yes or gpg encryption not store the creds in the git-annex branch. git-annex-remote-googledrive one was special remote affected by this bug. (Regression introduced in version 7.20200202.7)
  • Fix bug that made creds not be stored in git when a special remote was initialized with gpg encryption, but without an explicit embedcreds=yes. (Regression introduced in version 7.20200202.7)
  • Fix a annex.pidlock issue that made eg git-annex get of an unlocked file hang until the annex.pidlocktimeout and then fail.
Posted Wed Jun 17 20:00:34 2020

Similar to the yearly git user survey, there is a 2018 git-annex user survey.

If you use git-annex, please take a few minutes to answer the questions!

Since git-annex is seeing increasing use to manage scientific data, the survey also includes some extra questions about that at the end.

Posted Sat Dec 1 15:46:38 2018

git-annex 6.20180626 is an important security fix release.

See the advisory for details about the security holes fixed in this release.

After upgrading git-annex, you should restart any git-annex assistant processes.

Several changes to git-annex's behavior had to be made as part of the security fixes:

  • A security fix has changed git-annex to refuse to download content from some special remotes when the content cannot be verified with a hash check. In particular URL and WORM keys stored on such remotes won't be downloaded. See the documentation of the annex.security.allow-unverified-downloads configuration for how to deal with this if it affects your files.

  • A security fix has changed git-annex to only support http, https, and ftp URL schemes by default. You can enable other URL schemes, at your own risk, using annex.security.allowed-url-schemes.

  • A related security fix prevents git-annex from connecting to http servers (and proxies) on localhost or private networks. This can be overridden, at your own risk, using annex.security.allowed-ip-addresses.

  • Setting annex.web-options no longer is enough to make curl be used, and youtube-dl is also no longer used by default. See the documentation of annex.security.allowed-http-addresses for details and how to enable them.

  • The annex.web-download-command configuration has been removed, use annex.web-options instead.

Posted Tue Jun 26 16:00:19 2018

Similar to the yearly git user survey, there is a 2015 git-annex user survey.

If you use git-annex, please take a few minutes to answer the questions!

Posted Mon Oct 19 20:05:12 2015

Similar to the yearly git user survey, I am doing a 2013 git-annex user survey.

If you use git-annex, please take a few minutes to answer my questions!

Posted Fri Nov 22 17:17:21 2013

Thanks to my recent successful git-annex crowdfunding campaign, $1000 has been set aside to award others for their contributions to git-annex by the end of 2013. This is not a lot of money, but I hope that the reward and recognition will encourage some more people to work on git-annex.

You don't need to know Haskell! You could contribute some interesting and useful special remote hooks, or write better documentation.

On the other hand, if you have learned Haskell, this might be your first chance to make some money with it! Some ideas for things to work on that would be particularly award-worthy:

  • Port git-annex to IOS, using ghc-ios. I don't plan to ever do this myself, but you could make it happen.
  • Get the git-annex assistant working in Windows, building on the existing port to Windows. Otherwise, this is on my roadmap for February.
  • Solve existing bugs or todo items. There are more than I can keep up with.

The amount of the awards will vary depending on the size of the contribution. Since this is getting started later than is ideal, I reserve the right to extend it past the end of the year if I don't get enough participants.

If you'd like to participate in this program, just email me at id@joeyh.name. --Joey

Posted Thu Oct 10 22:02:33 2013

git-annex will be briefly presented at FOSDEM, on Sunday February 4th at 15:40. Details.

Thanks to Richard Hartmann for making this presentation.

Posted Sat Feb 4 18:30:28 2012

Christophe-Marie Duquesne has just announced Sharebox, a FUSE filesystem relying on git-annex:

What are your goals?  
Seamless synchronization "à la dropbox".  
Ability to use with big binary files such as mp3/movies.  
Entirely decentralized.  
Don't use unnecessary space  
Keep it simple: avoid special VCS commands and keep a filesystem  
interface as much as possible.

While still alpha, this is promising. --Joey

Posted Thu Mar 31 18:06:14 2011

Linux Weekly News has a nice article on git-annex in it this week.

Posted Thu Dec 9 15:35:07 2010