Recent comments posted to this site:

By device, I meant the computer-like device that needs to be able to read the files. Say, an MP3 player for example.

used as a directory special remote […] you can't just plug the drive into random computer and read files

Well, that rules it out since, the device being dumb, it does need to be able to find the real files at their standard path and be able to read them as is.

Chunking is not a requirement, though, since the device's software, only supporting FAT32, would not support or expect files that cannot exist in FAT32.

Next option is to use direct mode.

Direct mode is deprecated.

Comment by chocolate.camera Thu Feb 14 12:46:17 2019
+1 for this. Maybe, add a config option to pipe a file through a particular program before sending it to a remote?
Comment by Ilya_Shlyakhter Wed Feb 13 16:48:08 2019
This can be implemented as an external special remote in Python with a combination of https://github.com/althonos/fs.sshfs and https://github.com/Lykos153/AnnexRemote . Implementing using https://www.pyfilesystem.org/ would make a remote that works with a range of filesystems ( https://www.pyfilesystem.org/page/index-of-filesystems/ ). I don't have time right now but maybe you can try.
Comment by Ilya_Shlyakhter Wed Feb 13 16:46:25 2019

I'd really welcome an sftp special remote because SFTP it is the only officially allowed way for our institution to exchange data with external collaborators and I had no luck using the gvfs approach.

Even without an sftp special remote, it would be great to have some short walkthrough how to use SFTP with git-annex (if SFTP is currently actually possible with e.g. gvfs).

Comment by Grothausmann.Roman Wed Feb 13 12:11:05 2019

RSS feeds corresponding to playlists are limited to 15 results.

Since youtube-dl can parse and show the playlist contents as JSON it would be awesome for git-annex importfeed to support youtube-dl.

Ref: https://superuser.com/a/1359132

Comment by np.gan Sun Feb 10 13:09:54 2019

I would say that FAT32 devices are perfectly valid, but only if used as a directory special remote with chunking enabled. This way you can transfer files of any size (given there is enough space of course) and keep all previous versions. The downside is that you can't just plug the drive into random computer and read files, git-annex is required, also it is important to remember that special remotes do not store metadata about files so you need your regular repository synced to a computer where you want to read special remote.

Next option is to use direct mode, this way files will be readable on any computer even without git-annex, but file size will be limited to 4Gb and only latest version of your files will be available. Note that you still can drop files from direct mode repository and these files will be replaced with placeholders, so you are not forced to copy big files to a repository with limited file size.

Actually it is possible (and I think it is a valid use case) to use both "directory special remote" and "direct mode repository" on the same drive.

If I was forced to use FAT32, I would use it this way: create directory special remote with chunking to store files, create bare repository just for metadata. This way the drive will store all versions of files, any size, even if other repositories disappear this will be a complete copy.

Comment by guzik.sergey Fri Feb 8 11:00:14 2019
you'll notice that some of those processes have been hanging out there for days. i believe the TCP sockets are long gone, at least their parent SSH process are gone. this is why i related this to the "kill git-annex assistant on logout" - but since git-annex doesn't do setsid anymore (right?) I don't understand why those process could be left there without a parent... especially since that's on the server-side, which has a (relatively) up to date git-annex version...
Comment by anarcat Thu Feb 7 21:10:19 2019

ah. i somehow missed that... i was assuming a symmetry between the process of getting and sending files, after all it's similar: there's a list of files to move around, and we iterate of them the same way.

cost doesn't apply to uploads? if so that would seem like a fair feature to add... :)

Comment by anarcat Thu Feb 7 20:59:13 2019
Question is, why isn't ghc passing the -L/path/to/libgmp option to ld (or maybe stack isn't passing it to ghc), even though the path is listed in --extra-lib-dirs when calling stack?
Comment by Ilya_Shlyakhter Thu Feb 7 20:45:11 2019

That would make it do efficient parallelization of downloads, but not of the uploads that you showed it being bottlenecked on the slowest remote.

Comment by joey Thu Feb 7 20:29:19 2019