The git-annex Windows port is not ready for prime time. But it does exist now! --Joey

status

  • Doesn't daemonize. Maybe use http://hackage.haskell.org/package/Win32-services or perhaps easier, http://hackage.haskell.org/package/Win32-services-wrapper

  • XMPP library not yet built. (See below.)

  • View debug log is empty in windows -- all logs go to console. This messes up a few parts of UI that direct user to the debug log. Should try to get rid of the console, but only once ssh passwords (and possibly gpg) are not prompted there anymore.

  • Local pairing seems to fail, after acking on Linux box, it stalls. (Also, of course, the Windows box is unlikely to have a ssh server, so only pairing with a !Windows box will work.)

  • gcrypt is not ported to windows (and as a shell script, may need to be rewritten)

  • Incremental fsck sets the sticky bit to record when a file is fscked, and this is not done on windows, so fsck doesn't behave incrementally there.

  • Deleting a git repository from inside the webapp fails "RemoveDirectory permision denied ... file is being used by another process"

potential encoding problems

Unicode file names ignored on Windows is fixed, but some potential problems remain, since the FileSystemEncoding that git-annex relies on seems unreliable/broken on Windows.

  • When git-annex displays a filename that it's acting on, there can be mojibake on Windows. For example, "háčky.txt" displays the accented characters as instead the pairs of bytes making up the utf-8. Tried doing various things to the stdout handle to avoid this, but only ended up with encoding crashes, or worse mojibake than this.

  • md5FilePath still uses the filesystem encoding, and so may produce the wrong value on Windows. This would impact keys that contain problem characters (probably coming from the filename extension), and might cause interoperability problems when git-annex generates the hash directories of a remote, for example a rsync remote.

  • encodeW8 is used in Git.UnionMerge, and while I fixed the other calls to encodeW8, which all involved ByteStrings reading from git and so can just treat it as utf-8 on Windows (via decodeBS), in the union merge case, the ByteString has no defined encoding. It may have been written on Unix and contain keys with invalid unicode in them. On windows, the union merge code should probably check if it's valid utf-8, and if not, abort the merge.

  • If interoperating with a git-annex repository from a unix system, it's possible for a key to contain some invalid utf-8, which means its filename cannot even be represented on Windows, so who knows what will happen in that case -- probably it will fail in some way when adding the object file to the Windows repo.

  • If data from the git repo does not have a unicode encoding, it will be mangled in various places on Windows, which can lead to undefined behavior.

minor problems

  • rsync special remotes with a rsyncurl of a local directory are known buggy. (git-annex tells rsync C:foo and it thinks it means a remote host named C...)
  • webapp lets user choose to encrypt repo, and generate gpg key, before checking that gcrypt is not installed
  • Ssh connection caching does not work on Windows, so git annex get has to connect twice to the remote system over ssh per file, which is much slower than on systems supporting connection caching.
  • glacier-cli is not easily available (probably)
  • user feedback: "Git on windows installer provides openssh 4.6. git-annex installer provides openssh 6.2 . This seems to create problems regarding how known_hosts file path is setup. Setting GIT_SSH= to the git-annex openssh version fixes the problem."
    However, I don't know how to determine what that location is after it's been installed. Maybe look for ssh.exe in same directory as git-annex.exe? --Joey

stuff needing testing

  • test that adding a repo on a removable drive works; that git is synced to it and files can be transferred to it and back
  • Does stopping in progress transfers work in the webapp?

trying to build XMPP

Lots of library deps:

  1. gsasl-$LATEST.zip from http://josefsson.org/gnutls4win/ (includes gnuidn and gnutls)
  2. pkg-config from http://sourceforge.net/projects/pkgconfiglite/files/latest/download?source=files
  3. libxml2 from mingw: http://sourceforge.net/projects/mingw/files/MSYS/Extension/libxml2/libxml2-2.7.6-1/ both the -dll and the -dev
  4. Extract all the above into the Haskell platform's mingw directory. Note that pkg-config needs to be moved out of a named subdirectory.
  5. Run in DOS prompt (not cygwin!): cabal install network-protocol-xmpp

Current FAIL:

Loading package gnutls-0.1.5 ... ghc.exe: internal error: Misaligned section: 18206e5b
    (GHC version 7.6.3 for i386_unknown_mingw32)
        Please report this as a GHC bug:
    http://www.haskell.org/ghc/reportabug

https://ghc.haskell.org/trac/ghc/ticket/8830

Note: This only happens in the TH link stage. So building w/o the webapp works with XMPP.

Options:

  1. Use EvilSplicer, building first without XMPP library, but with its UI, and a second time without TH, but with the XMPP library. Partially done on the winsplicehack branch, but requires building patched versions of lots of yesod dependency chain to export modules referenced by TH splices, like had to be done on Android. Horrible pain. Ugly as hell.
  2. Make a helper program with the XMPP support in it, that does not use TH.
  3. Swich to a different XMPP client library, like http://hackage.haskell.org/package/pontarius-xmpp

What about Cygwin? It emulates POSIX fairly well under Windows (including signals, forking, fs (also things like /dev/null, /proc), unix file permissions), has all standard gnu utilities. It also emulates symlinks, but they are unfortunately incompatible with NTFS symlinks introduced in Vista due to some stupid restrictions on Windows.

If git-annex could be modified to not require symlinks to work, the it would be a pretty neat solution (and you get a real shell, not some command.com on drugs (aka cmd.exe))

Comment by Zoltán Tue May 15 00:14:08 2012

Windows support is a must. In my experience, binary file means proprietary editor, which means Windows.

Unfortunately, there's not much overlap between people who use graphical editors in Windows all day vs. people who are willing to tolerate Cygwin's setup.exe, compile a Haskell program, learn git and git-annex's 90-odd subcommands, and use a mintty terminal to manage their repository, especially now that there's a sexy GitHub app for Windows.

That aside, I think Windows-based content producers are still the audience for git-annex. First Windows support, then a GUI, then the world.

Comment by Michael Wed May 23 19:30:21 2012
Has git-annex been tested with an NTFS-formatted disk under Linux ? NTFS is supposed to be case-sensitive and to allow symlinks, and these are supposed to work with ntfs3g.
Comment by B0FH Thu Aug 2 17:45:10 2012
I successfully use git-annex on an NTFS formatted external USB drive, so yes, it is possible and works well.
Comment by Paul Thu Aug 16 19:30:38 2012

Haskell has C++ binding, so it should be possible to port it to .net/Mono with a VB GUI for Windows users. Windows has a primitive form of symlinks called shortcuts. Perhaps shortcut support in Windows could replace the use of symlinks. I've used shortcuts since XP to put my home Windows directory on another partition and never had a hitch...

If anyone is interested in working on this, hit me up. I would like to use this in my XP vbox to have access to files on my host OS...I have a student edition of Visual Studio 2005 to do an open source port...

Comment by A. D. Sicks Sun Sep 9 23:48:21 2012
It seems that NTFS (from Vista forward) has full POSIX support for symlinks. At least, Wikipedia seems to think so.. What about doing like GitHub and using MinGW for compatibility? Cygwin absolutely blows in terms of installation size and compatability with the rest of Windows.
Comment by Sean Fri Jan 11 21:44:21 2013
I was fighting my way forward until I read here that special remote with ssh+rsync and encryption doesn't work. Interestingly I got everything working so far, ssh login is keybased, gpg -k works and the remote setup also correctly cooperated with gpg... but it just didn't sync. Any ideas how complex it is to get this last missing piece moving?
Comment by Dominik Sun Jun 30 12:46:40 2013

It should be easy to fix whatever's wrong the the rsync special remote. Just a matter of debugging that.

Adding encryption support on Windows is stuck at a roadblock I don't know the way around. To drive gpg, git-annex uses the --passphrase-fd option, and sends the "passphrase" (really a big block of binary foo!) over a file descriptor of a pipe that it set up.

Windows, AFAIK, doesn't have file descriptors, or at least there is no equivilant to them that I have access to in the Haskell POSIX compatability layer for Windows. I am reluctant to fall back to using --passphrase-file on Windows, since that would be a massive security hole (as would passing the passphrase as a parameter via --passphrase=).

Comment by joeyh.name Sun Jun 30 17:58:08 2013
The tradeoff for me is a "local" security hole (where I can secure my own laptop) vs. a remotely exploitable thing... If it needs to go through a file, so be it -- it would however be good if that file would be overwritten with garbage before being deleted :-)
Comment by Dominik Wed Jul 31 10:29:51 2013
Encryption is now working on Windows.
Comment by joeyh.name Sun Aug 4 18:23:00 2013