This is where announcements of new releases, features, and other news is posted. git-annex users are recommended to subscribe to this page's RSS feed. Also, see security for security announcements.
git-annex 10.20230926 released with these changes
- Fix more breakage caused by git's fix for CVE-2022-24765, this time involving a remote (either local or ssh) that is a repository not owned by the current user.
- Fix using git remotes that are bare when git is configured with safe.bareRepository = explicit.
- Fix linker optimisation in linux standalone tarballs.
- adb: Avoid some problems with unusual characters in exporttree filenames that confuse adb shell commands.
- push: When on an adjusted branch, propagate changes to parent branch before updating export remotes.
- lookupkey: Added --ref option.
- enableremote: Avoid overwriting existing git remote when passed the uuid of a specialremote that was earlier initialized with the same name.
- Support being built with crypton rather than the no-longer maintained cryptonite.
- Removed the vendored git-lfs and the GitLfs build flag.
We are happy to announce that the call for participation for the first ever Distribits meeting is now online at https://distribits.live/
This meeting is organized by the folks behind git-annex and DataLad. We aim to bring together enthusiasts of tools and workflows in the domain of distributed data.
We are looking forward to April 2024 and to meeting people from everywhere, online and in Düsseldorf.
Registration is open now and until October 15. Please consider submitting a talk if you are doing something interesting or unusual with git-annex.
git-annex 10.20230828 released with these changes
- oldkeys: New command that lists the keys used by old versions of a file.
- Fix behavior of onlyingroup.
- info: Added --dead-repositories option.
- Significant startup speed increase by avoiding repeatedly checking if some remote git-annex branch refs need to be merged.
- Fix behavior when importing a tree from a directory remote when the directory does not exist. An empty tree was imported, rather than the import failing.
- sync, assist, push, pull: Skip more types of remotes when they are not available due to eg being on a drive that is offline. (directory, borg, bup, ddar, gcrypt, rsync)
- info: Added available to the info displayed for a remote.
- Added AVAILABILITY UNAVAILABLE and the UNAVAILABLERESPONSE extension to the external special remote protocol.
- The remote.name.annex-availability git config is no longer used.
- Avoid using curl when annex.security.allowed-ip-addresses is set but neither annex.web-options nor annex.security.allowed-url-schemes is set to a value that needs curl.
- Stop bundling curl in the OSX dmg and linux standalone image.
- diffdriver: Added --get option.
- diffdriver: Refuse to run when not in a git-annex repository.
git-annex 10.20230802 released with these changes
- satisfy: New command that gets/sends/drops content to satisfy preferred content settings. This is like to the --content part of git-annex sync.
- --explain: New option to display explanations of what git-annex takes into account when deciding what to do. Including explaining matching of preferred content expressions, annex.largefiles, and annex.addunlocked.
- reinject: Added --guesskeys option.
- diffdriver: Added --text option for easy diffing of the contents of annexed text files.
- assist: With --jobs, parallelize transferring content to/from remotes.
- Bug fix: Re-running git-annex adjust or sync when in an adjusted branch would overwrite the original branch, losing any commits that had been made to it since the adjusted branch was created.
- Bug fix: Fix behavior when git is configured with safe.bareRepository = explicit.
- importfeed bug fix: When -J was used with multiple feeds, some feeds did not get their items downloaded.
- importfeed: Add feedurl to the metadata (and allow it to be used in the --template)
- Improve resuming interrupted download when using yt-dlp.
- S3: Amazon S3 buckets created after April 2023 do not support ACLs, so public=yes cannot be used with them. Existing buckets configured with public=yes will keep working.
- S3: Allow setting publicurl=yes without public=yes, to support buckets that are configured with a Bucket Policy that allows public access.
- directory, gcrypt: Remove empty hash directories when dropping content.
- dropunused: Support --jobs
- Support "onlyingroup=" in preferred content expressions.
- Support --onlyingroup= matching option.
- Setup.hs: Stop installing man pages, desktop files, and the git-annex-shell and git-remote-tor-annex symlinks. Anything still relying on that, eg via cabal v1-install will need to change to using make install-home.
- Support building with unix-compat 0.7
- Support building with unix-2.8.0.
- stack.yaml: Update to build with ghc-9.6.2 and aws-0.24. (For windows, stack-lts-18.13.yaml has to be used instead for now.)
News for git-annex 10.20230626:
git-annex (10.20230626) upstream; urgency=medium . Many commands now quote filenames that contain unusual characters the same way that git does, to avoid exposing control characters to the terminal. The core.quotePath config can be set to false to disable this quoting.
git-annex 10.20230626 released with these changes
- Split out two new commands, git-annex pull and git-annex push. Those plus a git commit are equivalent to git-annex sync. (Note that the new commands default to syncing content, unless annex.synccontent is explicitly set to false.)
- assist: New command, which is the same as git-annex sync but with new files added and content transferred by default.
- sync: Started a transition to --content being enabled by default. When used without --content or --no-content, warn about the upcoming transition, and suggest using one of the options, or setting annex.synccontent.
- sync: Added -g as a short option for --no-content.
- Many commands now quote filenames that contain unusual characters the same way that git does, to avoid exposing control characters to the terminal.
- Support core.quotePath, which can be set to false to display utf8 characters as-is in filenames.
- Control characters in non-filename data coming from the repository or other possible untrusted sources are filtered out of the display of many commands. When the command output is intended for use in scripting, control characters are only filtered out when displaying to the terminal.
- find, findkeys, examinekey: When outputting to a terminal and --format is not used, quote control characters. Output to a pipe is unchanged. (Similar to the behavior of GNU find.)
- addurl --preserve-filename now rejects filenames that contain other control characters, besides the escape sequences it already rejected.
- init: Avoid autoenabling special remotes that have control characters in their names.
- Support core.sharedRepository=0xxx at long last.
- Support --json and --json-error-messages in many more commands (addunused, configremote, dead, describe, dropunused, enableremote, expire, fix, importfeed, init, initremote, log, merge, migrate, reinit, reinject, rekey, renameremote, rmurl, semitrust, setpresentkey, trust, unannex, undo, uninit, untrust, unused, upgrade)
- importfeed: Support -J
- importfeed: Support --json-progress
- httpalso: Support being used with special remotes that use chunking.
- Several significant speedups to importing large trees from special remotes. Imports that took over an hour now take only a few minutes.
- Cache negative lookups of global numcopies and mincopies. Speeds up eg git-annex sync --content by up to 50%.
- Speed up sync in an adjusted branch by avoiding re-adjusting the branch unnecessarily, particularly when it is adjusted with --hide-missing or --unlock-present.
- config: Added the --show-origin and --for-file options.
- config: Support annex.numcopies and annex.mincopies.
- whereused: Fix display of branch:file when run in a subdirectory.
- enableremote: Support enableremote of a git remote (that was previously set up with initremote) when additional parameters such as autoenable= are passed.
- configremote: New command, currently limited to changing autoenable= setting of a special remote.
- Honor --force option when operating on a local git remote.
- When a nonexistant file is passed to a command and --json-error-messages is enabled, output a JSON object indicating the problem. (But git ls-files --error-unmatch still displays errors about such files in some situations.)
- Bug fix: Create .git/annex/, .git/annex/fsckdb, .git/annex/sentinal, .git/annex/sentinal.cache, and .git/annex/journal/* with permissions configured by core.sharedRepository.
- Bug fix: Lock files were created with wrong modes for some combinations of core.sharedRepository and umask.
- initremote: Avoid creating a remote that is not encrypted when gpg is broken.
- log: When --raw-date is used, display only seconds from the epoch, as documented, omitting a trailing "s" that was included in the output before.
- addunused: Displays the names of the files that it adds.
- reinject: Fix support for operating on multiple pairs of files and keys.
- sync: Fix buggy handling of --no-pull and --no-push when syncing --content. With --no-pull, avoid downloading content, and with --no-push avoid uploading content. This was done before, but inconsistently.
- uninit: Avoid buffering the names of all annexed files in memory.
- Fix bug in -z handling of trailing NUL in input.
- version: Avoid error message when entire output is not read.
- Fix excessive CPU usage when parsing yt-dlp (or youtube-dl) progress output fails.
- Use --progress-template with yt-dlp to fix a failure to parse progress output when only an estimated total size is known.
- When yt-dlp is available, default to using it in preference to youtube-dl. Using youtube-dl is now deprecated, and git-annex no longer tries to parse its output to display download progress
- Improve resuming interrupted download when using yt-dlp or youtube-dl.
- assistant: Add dotfiles to git by default, unless annex.dotfiles is configured, the same as git-annex add does.
- assistant --autostop: Avoid crashing when ~/.config/git-annex/autostart lists a directory that it cannot chdir to.
- Fix display when run with -J1.
- assistant: Fix a crash when a small file is deleted immediately after being created.
- repair: Fix handling of git ref names on Windows.
- repair: Fix a crash when .git/annex/journal/ does not exist.
- Support building with optparse-applicative 0.18.1 (Thanks, Peter Simons)
git-annex 10.20230407 released with these changes
- Fix laziness bug introduced in last release that breaks use of --unlock-present and --hide-missing adjusted branches.
- Support user.useConfigOnly git config.
- registerurl, unregisterurl: Added --remote option.
- registerurl: When an url is claimed by a special remote other than the web, update location tracking for that special remote. (This was the behavior before version 6.20181011)
- Sped up sqlite inserts 2x when built with persistent 2.14.5.0
- git-annex.cabal: Prevent building with unix-compat 0.7 which removed System.PosixCompat.User.
Similar to the yearly git user survey, there is a 2018 git-annex user survey.
If you use git-annex, please take a few minutes to answer the questions!
Since git-annex is seeing increasing use to manage scientific data, the survey also includes some extra questions about that at the end.
git-annex 6.20180626 is an important security fix release.
See the advisory for details about the security holes fixed in this release.
After upgrading git-annex, you should restart any git-annex assistant processes.
Several changes to git-annex's behavior had to be made as part of the security fixes:
A security fix has changed git-annex to refuse to download content from some special remotes when the content cannot be verified with a hash check. In particular URL and WORM keys stored on such remotes won't be downloaded. See the documentation of the annex.security.allow-unverified-downloads configuration for how to deal with this if it affects your files.
A security fix has changed git-annex to only support http, https, and ftp URL schemes by default. You can enable other URL schemes, at your own risk, using annex.security.allowed-url-schemes.
A related security fix prevents git-annex from connecting to http servers (and proxies) on localhost or private networks. This can be overridden, at your own risk, using annex.security.allowed-ip-addresses.
Setting annex.web-options no longer is enough to make curl be used, and youtube-dl is also no longer used by default. See the documentation of annex.security.allowed-http-addresses for details and how to enable them.
The annex.web-download-command configuration has been removed, use annex.web-options instead.
Similar to the yearly git user survey, there is a 2015 git-annex user survey.
If you use git-annex, please take a few minutes to answer the questions!
Similar to the yearly git user survey, I am doing a 2013 git-annex user survey.
If you use git-annex, please take a few minutes to answer my questions!
Thanks to my recent successful git-annex crowdfunding campaign, $1000 has been set aside to award others for their contributions to git-annex by the end of 2013. This is not a lot of money, but I hope that the reward and recognition will encourage some more people to work on git-annex.
You don't need to know Haskell! You could contribute some interesting and useful special remote hooks, or write better documentation.
On the other hand, if you have learned Haskell, this might be your first chance to make some money with it! Some ideas for things to work on that would be particularly award-worthy:
- Port git-annex to IOS, using ghc-ios. I don't plan to ever do this myself, but you could make it happen.
- Get the git-annex assistant working in Windows, building on the existing port to Windows. Otherwise, this is on my roadmap for February.
- Solve existing bugs or todo items. There are more than I can keep up with.
The amount of the awards will vary depending on the size of the contribution. Since this is getting started later than is ideal, I reserve the right to extend it past the end of the year if I don't get enough participants.
If you'd like to participate in this program, just email me at id@joeyh.name. --Joey
git-annex will be briefly presented at FOSDEM, on Sunday February 4th at 15:40. Details.
Thanks to Richard Hartmann for making this presentation.
Christophe-Marie Duquesne has just announced Sharebox, a FUSE filesystem relying on git-annex:
What are your goals? Seamless synchronization "à la dropbox". Ability to use with big binary files such as mp3/movies. Entirely decentralized. Don't use unnecessary space Keep it simple: avoid special VCS commands and keep a filesystem interface as much as possible.
While still alpha, this is promising. --Joey
Linux Weekly News has a nice article on git-annex in it this week.