This is where announcements of new releases, features, and other news is posted. git-annex users are recommended to subscribe to this page's RSS feed. Also, see security for security announcements.

RSS Atom Add a new post titled:

git-annex 8.20201116 released with these changes

  • move: Fix a regression in the last release that made move --to not honor numcopies settings.
  • sync --content: Fix a bug where a file that was not preferred content could be transferred to a remote. This happened when the file got deleted after the sync started running.
  • Warn when adding a annex symlink or pointer file that uses a key that is not known to the repository, to prevent confusion if the user has copied it from some other repository.
  • examinekey: Added two new format variables: objectpath and objectpointer
  • examinekey: Added --migrate-to-backend
Posted Mon Nov 16 13:39:16 2020

git-annex 8.20201103 released with these changes

  • move: Improve resuming a move that was interrupted after the object was transferred, in cases where numcopies checks prevented the resumed move from dropping the object from the source repository.
  • When a special remote has chunking enabled, but no chunk sizes are recorded (or the recorded ones are not found), speculatively try chunks using the configured chunk size.
  • Fixed some problems that prevented this command from working: git submodule foreach git annex init
  • Improve shutdown process for external special remotes and external backends. Make sure to relay any remaining stderr from the process, and potentially avoid the process getting a SIGPIPE if it writes to stderr too late.
  • Fix a bug that prevented linux standalone bundle from working on a fresh install.
  • Windows build changed to one done by the datalad-extensions project using Github actions.
  • Windows build now includes libmagic, so mimetype and mimeencoding will work. Thanks to John Thorvald Wodder II and Yaroslav Halchenko for their work on this.
  • view: Avoid using ':' from metadata when generating a view, because it's a special character on Windows ("c:")
  • Fix a memory leak introduced in the last release.
  • add, import: Fix a reversion in 7.20191009 that broke handling of --largerthan and --smallerthan.
  • view: Fix a reversion in 8.20200522 that broke entering or changing views.
  • Fix build on Windows with network-3.
  • testremote: Display exceptions when tests fail, to aid debugging.
Posted Tue Nov 3 15:54:01 2020

git-annex 8.20201007 released with these changes

  • --json output now includes a new field "input" which is the input value (filename, url, etc) that caused a json object to be output.
  • --batch combined with -J now runs batch requests concurrently for many commands. Before, the combination was accepted, but did not enable concurrency. Since the output of batch requests can be in any order, --json with the new "input" field is recommended to be used, to determine which batch request each response corresponds to.
  • aws-0.22 improved its support for setting etags, which improves support for versioned S3 buckets.
  • Serialize use of C magic library, which is not thread safe. This fixes failures uploading to S3 when using -J.
  • add, addurl, importfeed, import: Added --no-check-gitignore option for finer grained control than using --force.
  • import: Check gitignores when importing trees from special remotes.
  • addunused: Don't check .gitignores when adding files.
  • Improve the "Try making some of these repositories available" message, with some hints for the user for what to do.
  • Improve --debug output to show pid of processes that are started and stopped.
  • sync --all: Sped up seeking to around twice as fast, by avoiding a pass over the worktree files when preferred content expressions of the local repo and remotes don't use include=/exclude=.
  • Sped up seeking for files to operate on, when using options like --copies or --in, by around 20%
  • import --no-content: Check annex.largefiles, and import small files into git, the same as is done when importing with content. If the largefiles expression needs the file content available (due to mimetype or mimeencoding being used), the import will fail.
  • sync: When run without --content, import without copying from importtree=yes directory special remotes. (Other special remotes may support this later as well.)
  • addurl: Avoid a redundant git ignores check for speed.
  • upgrade: Avoid an upgrade failure of a bare repo in unusual circumstances.
  • httpalso: Support being used with special remotes that do not have encryption= in their config.
  • Parse youtube-dl progress output, which lets progress be displayed when doing concurrent downloads.
  • Fix build with Benchmark build flag.
  • Enable building with git-annex benchmark by default, only turning it off when the criterion library is not installed.
  • runshell: Fix a edge case where rm errors were sent to stdout, which could confuse things parsing git-annex output.
  • runshell: Update files atomically when preparing to run git-annex.
  • Fix a build failure on Windows.
Posted Wed Oct 7 18:19:12 2020

git-annex 8.20200908 released with these changes

  • Added httpalso special remote, which is useful for accessing content stored on other remotes that is published by http.
  • The external special remote protocol got an ASYNC extension. This can be used by an external special remote to let a single process perform concurrent actions, rather than multiple processes being started, when that is more efficient.
  • Retry transfers to exporttree=yes remotes same as for other remotes.
  • import: Retry downloads that fail, same as is done for downloads generally.
  • Limit retrying of failed transfers when forward progress is being made to 5, to avoid some unusual edge cases where too much retrying could result in far more data transfer than makes sense.
  • Exposed annex.forward-retry git config, to configure the forward retry behavior that git-annex has had for a long time.
  • sync, assistant, merge: When merge.directoryRenames is not set, default it it to "false", which works better with automatic merge conflict resolution than git's ususual default of "conflict". (This is not done when automatic merge conflict resolution is disabled.)
  • resolvemerge: Improve cleanup of cruft left in the working tree by a conflicted merge.
  • Support git remotes where .git is a file, not a directory, eg when --separate-git-dir was used.
  • Fixed several cases where files were created without file mode bits that the umask would usually set. This included exports to the directory special remote, torrent files used by the bittorrent special remote, hooks written by git-annex init, and some log files in .git/annex/
  • Fix reversion in 7.20190322 that made addurl --file not be honored when youtube-dl was used to download media.
  • Fix reversion in 8.20200617 that made annex.pidlock being enabled result in some commands stalling, particularly those needing to autoinit.
  • Display warning when external special remote does not start up properly, or is not usable.
  • Display a message when git-annex has to wait for a pid lock file held by another process.
  • test: Stop gpg-agent daemons that are started for the test framework's gpg key.
  • Removed the S3 and WebDAV build flags so these special remotes are always supported.
  • stack.yaml: Updated to lts-16.10.
Posted Tue Sep 8 18:21:37 2020

Similar to the yearly git user survey, there is a 2018 git-annex user survey.

If you use git-annex, please take a few minutes to answer the questions!

Since git-annex is seeing increasing use to manage scientific data, the survey also includes some extra questions about that at the end.

Posted Sat Dec 1 15:46:38 2018

git-annex 6.20180626 is an important security fix release.

See the advisory for details about the security holes fixed in this release.

After upgrading git-annex, you should restart any git-annex assistant processes.

Several changes to git-annex's behavior had to be made as part of the security fixes:

  • A security fix has changed git-annex to refuse to download content from some special remotes when the content cannot be verified with a hash check. In particular URL and WORM keys stored on such remotes won't be downloaded. See the documentation of the annex.security.allow-unverified-downloads configuration for how to deal with this if it affects your files.

  • A security fix has changed git-annex to only support http, https, and ftp URL schemes by default. You can enable other URL schemes, at your own risk, using annex.security.allowed-url-schemes.

  • A related security fix prevents git-annex from connecting to http servers (and proxies) on localhost or private networks. This can be overridden, at your own risk, using annex.security.allowed-ip-addresses.

  • Setting annex.web-options no longer is enough to make curl be used, and youtube-dl is also no longer used by default. See the documentation of annex.security.allowed-http-addresses for details and how to enable them.

  • The annex.web-download-command configuration has been removed, use annex.web-options instead.

Posted Tue Jun 26 16:00:19 2018

Similar to the yearly git user survey, there is a 2015 git-annex user survey.

If you use git-annex, please take a few minutes to answer the questions!

Posted Mon Oct 19 20:05:12 2015

Similar to the yearly git user survey, I am doing a 2013 git-annex user survey.

If you use git-annex, please take a few minutes to answer my questions!

Posted Fri Nov 22 17:17:21 2013

Thanks to my recent successful git-annex crowdfunding campaign, $1000 has been set aside to award others for their contributions to git-annex by the end of 2013. This is not a lot of money, but I hope that the reward and recognition will encourage some more people to work on git-annex.

You don't need to know Haskell! You could contribute some interesting and useful special remote hooks, or write better documentation.

On the other hand, if you have learned Haskell, this might be your first chance to make some money with it! Some ideas for things to work on that would be particularly award-worthy:

  • Port git-annex to IOS, using ghc-ios. I don't plan to ever do this myself, but you could make it happen.
  • Get the git-annex assistant working in Windows, building on the existing port to Windows. Otherwise, this is on my roadmap for February.
  • Solve existing bugs or todo items. There are more than I can keep up with.

The amount of the awards will vary depending on the size of the contribution. Since this is getting started later than is ideal, I reserve the right to extend it past the end of the year if I don't get enough participants.

If you'd like to participate in this program, just email me at id@joeyh.name. --Joey

Posted Thu Oct 10 22:02:33 2013

git-annex will be briefly presented at FOSDEM, on Sunday February 4th at 15:40. Details.

Thanks to Richard Hartmann for making this presentation.

Posted Sat Feb 4 18:30:28 2012

Christophe-Marie Duquesne has just announced Sharebox, a FUSE filesystem relying on git-annex:

What are your goals?  
Seamless synchronization "à la dropbox".  
Ability to use with big binary files such as mp3/movies.  
Entirely decentralized.  
Don't use unnecessary space  
Keep it simple: avoid special VCS commands and keep a filesystem  
interface as much as possible.

While still alpha, this is promising. --Joey

Posted Thu Mar 31 18:06:14 2011

Linux Weekly News has a nice article on git-annex in it this week.

Posted Thu Dec 9 15:35:07 2010