forum/Get 'source' group to automatically drop with assistantgit-annexhttp://git-annex.branchable.com/forum/Get___39__source__39___group_to_automatically_drop_with_assistant/git-annexikiwiki2017-06-07T16:34:41Zcomment 1http://git-annex.branchable.com/forum/Get___39__source__39___group_to_automatically_drop_with_assistant/comment_1_ead4b5b7ec0d085d4fa87fb4e29c4d55/joey2017-05-11T16:26:13Z2017-05-11T16:06:03Z
<p>Sounds like an interesting data-gathering application, I have to say
I'm curious what it is. <img src="http://git-annex.branchable.com/smileys/smile4.png" alt=";)" /></p>
<p>If Repo1 is configured like this:</p>
<pre><code>git-annex group here source
git annex wanted here standard
</code></pre>
<p>Then it should want to drop the contents of files from Repo1 once it knows
they have reached any other repository. (Sometimes people put a repository
in a group but forget to set wanted to "standard" ...)</p>
<p>Looks like Repo1 cannot make outgoing connections to Repo2?</p>
<p>So, you need to run the assistant on Repo2 and probably on Repo1.
Then it works like this:</p>
<ol>
<li>The assistant on Repo1 will commit files as they're put in the repository.</li>
<li>The assistant on Repo2 will notice changes to Repo1, pull down the
files, see that Repo1 is configured as a source repository, and drop
the files from Repo1.</li>
<li>The assistant on Repo1 will also notice when it's sent a file to Repo2
and can drop it from Repo1 then -- but this may take a while for it to
notice and #2 will probably happen first.</li>
</ol>
<p>Note that if you get the files added and committed by some other process,
you don't really need to run the assistant on Repo1.</p>
<p>The USB drives need to be client, so that once content reaches one of them,
the content will be dropped from the transfer repositories. The way that
part should work:</p>
<ol>
<li>Content reaches Repo2.</li>
<li>The assistant in Repo3 notices the change to Repo2, pulls down
the files.</li>
<li>The assistant in Repo3 pushes the files to a connected USB drive.</li>
<li>The assistant in Repo3 drops content from itself and Repo2, since
it knows both are transfer repos and the content has reached a client
(USB).</li>
</ol>
<p>If you're having trouble getting any of this to work, I recommend
running <code>git annex sync --content</code> manually while testing it, and make sure
it does what you would expect to happen at each step.</p>
comment 2http://git-annex.branchable.com/forum/Get___39__source__39___group_to_automatically_drop_with_assistant/comment_2_835dba9de7a5eea868635dc793d4e75b/leavingchicago2017-05-11T23:23:45Z2017-05-11T23:23:45Z
<p><ASIDE>
<em>Laughing</em> I wish the data collection was interesting. Just a machine with locked down networking so that it's a tad more secure. Closest example is an EC2 instance sitting in a VPC...
</ASIDE></p>
<p>Thanks for the feedback Joey. I tried again and this approach works <strong>except</strong> where the downstream repo is direct mode - in this case the transfer repo doesn't drop. I'm not sure if it's a bug or a feature I didn't read about.</p>
<p>Happy to submit a bug report with steps if it's unexpected.</p>
comment 3http://git-annex.branchable.com/forum/Get___39__source__39___group_to_automatically_drop_with_assistant/comment_3_553542a4a2b2530863a006a1afe91637/olaf2017-05-11T23:45:36Z2017-05-11T23:45:36Z
<p>Oops. I committed the greatest sin...</p>
<pre><code>git-annex version: 6.20170321-gf3dee9d65
build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Quvi
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
local repository version: 5
supported repository versions: 3 5 6
upgrade supported from repository versions: 0 1 2 3 4 5
operating system: linux i386
</code></pre>
comment 4http://git-annex.branchable.com/forum/Get___39__source__39___group_to_automatically_drop_with_assistant/comment_4_4540a15672f9b48b0f9310c97f1a0b86/joey2017-06-07T16:34:41Z2017-06-07T16:30:29Z
<p>I would not expect the behavior with a direct mode repo to be any
different. Unless the file in direct mode is getting modified, or
perhaps git-annex gets confused and thinks it's modified. Either would
prevent dropping the content from the transfer repository to avoid losing
the unmodified content.</p>