sharebox: a FUSE filesystem for git-annexgit-annexhttp://git-annex.branchable.com/news/sharebox_a_FUSE_filesystem_for_git-annex/git-annexikiwiki2015-01-05T20:41:19Zcomment 1http://git-annex.branchable.com/news/sharebox_a_FUSE_filesystem_for_git-annex/comment_1_e238d1734238e37bb55ff952b32e06b8/Tiago2013-11-27T22:47:37Z2012-09-27T00:08:19Z
<p>This is what the assistant should aim to be..."Like Dropbox", but even better.
I would guess many people who pledged were thinking of this.</p>
I wondered if there is any even simple fuse wrapper for git-annex?http://git-annex.branchable.com/news/sharebox_a_FUSE_filesystem_for_git-annex/comment_2_341b567722797eb02bd96ffada431b0c/Yaroslav2014-12-16T16:34:06Z2014-12-16T16:34:06Z
sharebox seems aiming to achieve what assistant is aiming for (synchronization). In my usecase I wondered if there is a simple(r) FUSE wrapper for git-annex which would just 'annex get' any file which is requested (for reading).
comment 3http://git-annex.branchable.com/news/sharebox_a_FUSE_filesystem_for_git-annex/comment_3_d6f6e7181f30094339a49ab420bee380/CandyAngel2014-12-16T16:56:05Z2014-12-16T16:56:05Z
<p>@Yaroslav: I made one of these while I was messing with FUSE but found I didn't use it much.</p>
<p>If I can find it, I'll post it somewhere or if you really want it, I can just write a (much) better one!</p>
comment 4http://git-annex.branchable.com/news/sharebox_a_FUSE_filesystem_for_git-annex/comment_4_ab0d45c5058595a71656035c962c1143/Yaroslav2014-12-16T20:42:28Z2014-12-16T20:42:28Z
<p>what could I say to a "much better one" offer, besides "GO AHEAD" and "Thank you in advance"! <img src="http://git-annex.branchable.com/smileys/smile.png" alt=":)" /></p>
<p>I wonder though what joey thinks about possible utility of a basic fuse wrapper for annex, and possibly shipping it along?</p>
<p>My primary use-case would be primarily oriented for testing, e.g. if I would like to run a (sub)collection of tests (e.g. on travis) which rely on having some data from annex available, now I would need either provide some project/language specific wrapping which would check if file is available or not and then fetch it. With FUSE I thought I could just do that transparently without requiring any per-project coding/setup. Similar use-case would be analysis of some large datasets, once again, without requiring pre-fetching them in entirety and/or piece-by-piece fetching.
Another possible additional usecase/mode could also be -- expose only available files under FUSE. If easy to "trigger" it would help to provide that "lean" view I was blurbing about (https://github.com/datalad/datalad/issues/25) although it would be quite a suboptimal workaround (since if directory is heavily loaded with broken links, it would take a while for FUSE handler to first traverse the tree anyways)</p>
comment 5http://git-annex.branchable.com/news/sharebox_a_FUSE_filesystem_for_git-annex/comment_5_b92045c91d92da7db794aed2c67dde0d/CandyAngel2014-12-18T09:26:17Z2014-12-18T09:26:17Z
<p>Having the lean view would be easy to implement either as an option you pass when mounting or something you can toggle by touching a file ($MNT/.config/lean/{on,off}).</p>
<p>Regarding fetching of files, how would you like it to behave? My previous one would return EBUSY while downloading a file and ENODATA if it wasn't available and couldn't be fetched. I could, for example, make unavailable files appear as normal files (containing text regarding the download state) until they are available, then they become symlinks. What would work best for you?</p>
unavailable fileshttp://git-annex.branchable.com/news/sharebox_a_FUSE_filesystem_for_git-annex/comment_6_6f3b5d5a5781b3a570f46481dc2ebca2/Yaroslav2014-12-18T16:37:13Z2014-12-18T16:37:13Z
<p>for my use cases the best would be if FUSE simply didn't return until file becomes available. Making an option to return immediately with EBUSY/ENODATA could also be generally useful but not in my case <img src="http://git-annex.branchable.com/smileys/ohwell.png" alt=":-/" />
I wonder if any timeout would kick in in some use-cases if it takes too long?</p>
comment 7http://git-annex.branchable.com/news/sharebox_a_FUSE_filesystem_for_git-annex/comment_7_6202ae898f24b3e02bc343d0fd2ac35a/CandyAngel2014-12-19T08:40:06Z2014-12-19T08:40:06Z
<p>Okie dokie, I'll see what I can do.</p>
<p>Can you give me an idea of the annex file properties (file size, count, files per directory, directory count) etc. please?</p>
typical reposhttp://git-annex.branchable.com/news/sharebox_a_FUSE_filesystem_for_git-annex/comment_8_884a0b9571544a95fad55cb5fc5963d8/Yaroslav2014-12-19T14:05:22Z2014-12-19T14:05:22Z
<p>Thanks for doing it and asking for detail!!!
Repositories will vary quite a bit. I am currently testing how big we could actually make them (see https://github.com/datalad/datalad/issues/17) <img src="http://git-annex.branchable.com/smileys/smile4.png" alt=";)" /></p>
<p>Meanwhile here are sample few available for git clone/testing:</p>
<p>https://github.com/datalad/nih--videocast a good collection of heavyish video files
http://psydata.ovgu.de/forrest_gump/.git/ a good single dataset with probably a somewhat typical amount of data
http://data.pymvpa.org/datasets/haxby2001/.git/ relatively small dataset with typical data sizes</p>
comment 9http://git-annex.branchable.com/news/sharebox_a_FUSE_filesystem_for_git-annex/comment_9_32b5d0fb9c328fbcd8105dfa31f032d3/joey2015-01-05T20:41:19Z2015-01-05T20:38:40Z
<p>My concern about using FUSE has always been that I don't much like it
when open() hangs indefinitely, with no progress indication, and
is either downloading some large file from the network or .. just hung.</p>
<p>That doesn't strike me as a nice user interface in general,
which is why I avoided using FUSE for the assistant.</p>
<p>It might make sense in the batch use cases Yaroslav gave. If something
nice is developed, I would not be against including it in git-annex.
(Bonus if it's implemented in haskell.)</p>