Got caught up on a big queue of messages today. Mostly I hear from people when git-annex is not working for them, or they have a question about using it. From time to time someone does mention that it's working for them.

We have 4 or so machines all synching with each other via the local network thing. I'm always amazed when it doesn't just explode :)

Due to the nature of git-annex, a lot of people can be using it without anyone knowing about it. Which is great. But these little success stories can make all the difference. It motivates me to keep pounding out the development hours, it encourages other people to try it, and it'd be a good thing to be able to point at if I tried to raise more funding now that I'm out of Kickstarter money.

I'm posting my own success story to my main blog: git annex and my mom

If you have a success story to share, why not blog about it, microblog it, or just post a comment here, or even send me a private message. Just a quick note is fine. Thanks!


Going through the bug reports and questions today, I ended up fixing three separate bugs that could break setting up a repo on a remote ssh server from the webapp.

Also developed a minimal test case for some gnucash behavior that prevents the watcher from seeing files it writes out. I understand the problem, but don't have a fix for that yet. Will have to think about it. (A year ago today, my blog featured the first release of the watcher.)

I've been real quiet cause everything's been working great. :)

One thing I have not messed with yet is the whole XMPP thing. I mostly don't share between two machines that are online at the same time, I just use git annex for backups and to have content "optionally" available. I should give XMPP a try sometime.

Comment by edheil [wordpress.com] Tue Jun 25 22:26:27 2013

Annex is working great for me since 8/2012 or something... Using it a lot and the Assistant is awesome!

If you would raise some more funding, I'd be more than happy to support you once more - really great software!

Comment by subito Wed Jun 26 07:00:15 2013
:)

Poets aren't humble people. We revel in our own narcissism. So being mentioned in my brother's blog was great for me.

I am proud of the success of Git Annex.

(My poems tend to be much better than this.)

Comment by Maggie Wed Jun 26 15:04:45 2013
I use git-annex for my picture collection. I can add pictures from my computers, or my FTP server when someone wants to send me their pictures. Then I push them to my online gallery. Git annex is great to keep all my repository synchronized while not requesting that all my pictures are present on my laptop. Thank you!
Comment by nicolas-schodet [myopenid.com] Wed Jun 26 18:10:05 2013

Excerpt from an email I received: --Joey

Another story, but without the assistant this time: I've "designed" a small f2f file sharing network with it, I'm using it with a friend to share accross several computer. This is basically a workflow with git annex and gitoline, the flexibility of git and the capacities of git annex just gave us this out of the box. Some specs are here if you are curious http://worlddomination.be/ideas/2012/idea-n-21-build-a-f2f-sharing-network-using -git-annex.html

Comment by joeyh.name Wed Jun 26 19:39:15 2013

I'm using Ubuntu, and I do a clean install about every 6 months when the new versions come out. I also have 2-3 systems I need to keep in sync. I use git annex to store files I want to keep between OS reinstalls, and to sync the various systems I need data on. I also use it to backup data to the cloud and to bup. I have 6+ repos, 235 GB of data, and 31,029 known annex keys (thanks git annex map and git annex status!); nevertheless git annex is generally pretty cooperative. I'm not using the assistant yet, but I'd like to eventually. I almost don't have to think about the stuff annex handles. It's really fun to install a fresh OS, clone the git annex repo, and have my entire annex directory tree available immediately.

Thanks Joey! You've done a great job!

Comment by andy Thu Jun 27 02:03:00 2013

Amazing tweet I found! --Joey

Git-annex now auto-syncing photos from my android phone to a Tor hidden SSH service I control (via @guardianproject's Orbot) #prismbreak

https://twitter.com/graphiclunarkid/status/347658443479449601

Comment by joeyh.name Thu Jun 27 06:31:29 2013
We use rsync and tar to backup to a server on the local network, and since we discoverd git-annex, we use this mechanism to automatically put all this data in both a local bup repo and a remote one. This gives us a daily full backup, with history available, both locally and off-site. Thanks to bup our 4T of data gets deduplicated to about 270G, only a couple of megs get transferred every night, and thanks to git-annex everything just works, like it should!
Comment by Tim Thu Jun 27 13:14:43 2013

Hi,

I have moved all my photographs to an annex running on a Ubuntu file server while doing photo editing, presenting etc. on my MacBook. The annex is great in selectively moving files between these machines for editing and presentation purposes. I started using it around christmas season last year and added a glacier "backup" recently. My statistics: 6 repositories in total, more than 300GB (dedup'd from 380GB), 60.000 local keys.

What is missing from my point of view (and I would happily join crowd funding): stable windows and android versions, managing selective transfers from the assistant (select folders to be transferred from/to a specific repository), options for running the annex as a server daemon with web GUI (authentication to GUI + fixed URLs, status reports on GUI, etc.). All these would dramatically increase the WAF.

Comment by Hendrik Thu Jun 27 14:09:47 2013

Hi Joey,

First, thanks very much for your effort put into git-annex. I store my photos in git-annex and I like it because:

  • I want to store multiple copies of the pictures and I can see where they are,
  • I can have a global view of the collection on my laptop without carrying all the files.
  • the indirect mode makes it difficult to me to do stupid things.

So there's nothing fancy, but it works for me very well. From time to time I try the assistant's new features and I'm planning to replace Dropbox eventually.

Secondly, thanks very much for the dev blog! I was looking forward every morning to see what's coming next, what problems you are facing and how you're planning to solve them. It is simply amazing what you've achieved in the last year and your posts gave me a lot of motivation too (in finishing my thesis :-))!

Thanks!

Comment by David Thu Jun 27 16:52:12 2013

Excerpt from a longer comment by Florian:

I know that my expectations in this project are a little bit high and I see how much work it was to get where we are now. Once again thank you for the excellent work done so far. I don't fear to synchronize important data with git-annex. I even use git-annex to backup and synchronize the member database for our club (wannabe hackerspace).

Comment by joeyh.name Thu Jun 27 18:11:03 2013

From a mail I received: --Joey

Git-annex makes it possible for me to have lots of data. Before I started using it I had a music library and a few massive tar files, none of it properly backed up, and occasionally there would be random corruption (I still have broken mp3s which jump and stutter in places) because I would carelessly cp them from HDD to HDD. So I'd avoid actually letting data accumulate; this was inconvenient in itself. Now git-annex does everything for me.

Following your development blog, as a novice programmer, has been really enjoyable and interesting. My only experience of professional IT has put me of but it's good to see you working through things methodically and properly.

Comment by joeyh.name Thu Jun 27 18:11:45 2013

Hi Joey,

More than ten years ago, I was looking for a software to index the content of all my CD-ROMs in order to create a kind of "table of content". Today there is git-annex!

I discovered git-annex few months ago and it really helps me to keep my file well organized. I missed the first crowd funding campaign but would be happy to contribute to the next one :D

I mainly use four different annexes for:

  • my private photos and videos,
  • my music collection,
  • my other multimedia files (movies...),
  • my other files that are too large to be commited into git directly (pdf, binaries...).

I use git-annex manually via the command line. I started to play with the assistant on my Android tablet and I plan to install it on my wife's Macbook to synchronize her data with the NAS.

A standalone build for arm to run git-annex on my NAS (QNAP 219 PII) would be perfect :-P

Thank you very much for all your great softwares (yes I also use etckeeper, some moreutils tools and more recently ikiwiki).

-Julien

Comment by jlefrique Thu Jun 27 19:59:47 2013
git-annex has been working brilliantly for me. I have been slowly moving content out of the various places I've had it stored (file servers, folders hidden away on all my machines, Dropbox, Google Drive, etc.) into a single git-annex repo. The idea has been to mimic Dropbox's single folder, but with much more flexibility: I sync my working machines, my home fileserver, my VPS (for file access on the go), Android devices, and even a couple of dumb MP3 players from git-annex. Best of all, I have one place to go for all of my files, and if they're not on the device at the moment I need them, they're a download away. I didn't find git-annex until after the first crowdfunding campaign had completed, but if there's another, I would be a happy supporter.
Comment by EvanDeaubl Fri Jun 28 18:35:55 2013

Looks like I started using git-annex in November 2010.

I'd already been tracking lots of small files in my home directory (settings, scripts, notes, todo list, timesheets, invoices, etc) with git.

So awesome to get backups, syncing and manual merge conflict resolution.

git-annex extended that to handle my larger files. git-annex's ability to be selective about what is stored where, allowed my to still be able to git clone my home directory onto small devices (I've got a debian chroot with only 6GB or so, and I installed debian on my new chromebook, that's only got 16GB of disk space (for the whole system.))

I love having the full catalog visible on my filesystem with symlinks, and I can just request a file, and git-annex will figure out where to get it from, even if I'm away from home on my laptop.

I just got a 2nd external hard drive, and I'm delighted with how easy it was to get a backup of everything on there.

I was able to quickly and safely clear up space on my desktop computer! I'm confident that my data is safe because I've got git-annex configured to make sure there's at least two copies of my videos, and 3 copies of everything else.

I've finally gone from either keeping hard drives from my past computers or keeping a full copy of the data, to having a proper archiving and backup system for all files I care about.

I've got over 12,000 files stored on my main annex, and it's perfectly responsive.

Joey does a great job, and has been very responsive on the rare occasions where I needed help.

Three cheers!

-- Jason

Comment by JasonWoof Sat Jun 29 03:48:08 2013

I'm using git-annex to consolidate all the stuff that accumulated for years on various drives I have. Music I downloaded and bought (don't wanna loose those), photos from 2003 and on, archives of school and work stuff etc. Using it for 6-7 months now, I came to trust git-annex, I very much like the command line robustness it has and I really like that all my files are accessible in a single tree. It would be cool to have some metadata storage (for file dates, tags and such) and maybe spotlight-like search (elastic-search comes to mind).. Still, without these, it has a great problem-solution fit for me!

Thanks Joey for all the great work, I really hope you can continue improving it, I would gladly join to a second kickstarter funding!

Comment by dzsino Sun Jun 30 13:10:35 2013

I've got an annex with all my DVD ISOs that's nearly 3TB alone, uses the SHA512 hash and gets split over three computers, 6 external SATA drives and 3 USB drives. No errors so far, and the issues with git-annex I've had have all been addressed very quickly (thanks Joey!) and in a non-destructive manner so I don't have to re-init a repo. I also use it to manage my music library, which allows to me to sleep at night without worrying about a corrupted file spreading through all my backups over time since git-annex has a fsck and hashing utility. I can't imagine using anything else for managing collections of files: it offloads the task of keeping track of, getting, verifying, and general management of files so I don't even need to worry about the boring stuff, I just get around to watching/listening to my media collection.

Thanks for all the work, it's changed my workflow and file management (I get more sleep too)!

Comment by GLITTAH Tue Jul 2 00:12:27 2013

Another good tweet: --Joey

My google Reader replacement: newsbeuter with the data directory stored in an autosynced git-annex repo.

https://twitter.com/JustinAzoff/status/352256199070199809

Comment by joeyh.name Wed Jul 3 15:51:48 2013

Forgot to add: I'm horrible at keeping things organized. Git-annex is a huge help with that -- it keeps track of files and the number of their copies. When I first started using it, I had old songs duplicated across 4 drives, while 2 of those drives had old tags and were obsolete, and one drive had the correct tags, but not the correct filenames. Every few years I'd have to go through several TB of media files, hand checking each of them to see what was old, what was unwanted, etc. Once my media libraries got larger than consumer-grade HDD's, I had to find a better solution. Git-annex removes that burden, and keeps things organized for me. Once I get an album or movie I tag/rename it the way I want, throw it in the annex and let my computer do the boring and tedious parts like automatically verifying and making sure there's enough copies. If I change a file, that change propagates through all my repo clones. It wasn't until after I started using git-annex that I realized how stressed out I was getting over simply maintaining a media library.

If you do another fundraiser you've got my support!

Comment by GLITTAH Wed Jul 3 19:28:42 2013
I'm using git-annex to make videos, pictures and music available on all my devices (windows, android, linux). It has saved me on trips with my 2yr daughter when she's had meltdowns; I have her favorite movie ready. I quickly exceeded the free dropbox quota and git-annex has replaced the need and gives me greater confidence and control over my data. Thank you!
Comment by Joe Thu Jul 4 12:41:35 2013

I'm a graduate student, so I write a lot. I started using git to manage my latex sources a while ago, but I didn't like gumming up my repository with a bunch of figures (large binaries). I was constantly forgetting to scp the latest versions of my figures from here to there (as I do my work on a number of different computers.) Git annex gives me a good way to version the figures and manage them with the same tool I'm using for the text.

My use case seems pretty far from typical, judging from the other comments here, so I thought I should add my two cents! I'm constantly trying to sell this approach to my fellow students.

Also, I'm going to slip in a plug for another great Haskell project here -- Pandoc. I work with people who suggest changes to my drafts, but only using Word's "Track Changes". So now I'm generating Word docs from Pandoc markdown for my collaborators to edit. When it comes time to publish, I can convert my markdown to latex and clean up the formatting.

Why are all the really cool projects written in Haskell? :)

Comment by Matthew Thu Jul 4 15:32:24 2013
Just like Git, git-annex is teh awesome. I've used it for almost a year now, and I've never had more control of all my stuff. I store all my valuable (and also not so valuable) files in it, more than a terabyte, and it's rapidly growing. Movies, pictures, recordings, software, podcasts, DVDs and CDs, you name it. It's rock solid with a really good design. Joey says it isn't a backup solution, but I strongly disagree. My data have never been safer. git-annex ensures that I have several copies in different locations, I always know where all the copies are, and the disk space is used way more efficiently now that I don't have unnecessary duplicates everywhere anymore.
Comment by sunny256 Mon Jul 15 12:37:54 2013