forum/sync --content, fatal is outside repository errorsgit-annexhttp://git-annex.branchable.com/forum/sync_--content__44___fatal_is_outside_repository_errors/git-annexikiwiki2018-07-17T19:20:56Znot specific to cloud repohttp://git-annex.branchable.com/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_1_668dae37dd46047b46c9e1c23f7bcec4/public2016-11-09T21:24:59Z2016-11-09T21:24:59Z
<p>I am getting the same errors when syncing to my USB drive. So it does not appear to have anything to do with syncing to the cloud.</p>
<p>Andrew</p>
how to investigatehttp://git-annex.branchable.com/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_2_09c62e4abf4ccc0d2e030ef5e1bcdf71/andrew2016-11-16T15:37:01Z2016-11-16T15:37:01Z
<p>Any thoughts? I am unsure how to investigate where this problem is. I assume these files are in my git repo or git-annex objects but I can't seem to find them using any search commands.</p>
<p>Thanks,</p>
<p>Andrew</p>
comment 2http://git-annex.branchable.com/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_2_8f694afa77f5a835c826d29d46d44615/joey2017-02-20T17:45:16Z2016-11-16T18:52:07Z
<p>It would be helpful if you said what version of git-annex you are using.</p>
<p>And, is your git-annex repository using the new experimental v6 format? One
user reported a similar error message with a v6 git-annex repository. See
<span class="createlink"><a href="http://git-annex.branchable.com/ikiwiki.cgi?do=create&from=forum%2Fsync_--content__44___fatal_is_outside_repository_errors%2Fcomment_2_8f694afa77f5a835c826d29d46d44615&page=bugs%2Fassistant_crashes_in_TransferScanner" rel="nofollow">?</a>assistant crashes in TransferScanner</span></p>
<p>Or might your repository be using direct mode?</p>
<p>So, please paste in <code>git annex version</code> and <code>git annex info</code> output.</p>
<p>It kind of looks like it's having difficulty determining where the top of
the git repository is, or constructing a relative path to the git
repository.</p>
<p>Are there any symlinks in the path to /Users/andrew/notes ? Eg, is /Users
a symlink, or /Users/andrew a symlink, or //Users/andrew/notes itself
symlinked to elsewhere?</p>
<p>Does only <code>git annex sync --content</code> fail? What if you run, eg
<code>git annex copy --auto --to cloud</code> and <code>git annex get --auto --from cloud</code>,
does that fail similarly, or does it succeed?</p>
<p>You say it's only failing for some files. Do the filenames that it's
failing on contain any non-ascii characters?</p>
git annex copy --auto --to cloud workshttp://git-annex.branchable.com/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_4_a7f476aeacf88679f25badc78fad886a/andrew2016-11-17T17:49:27Z2016-11-17T17:49:27Z
<p>Yes, only <code>git annex sync --content</code> seems to fail. I am using v6 with a mix of unlocked and locked files. I did not know about the --auto flags for copy/get.</p>
<ul>
<li><code>git annex copy --auto --to cloud</code> works fine</li>
<li><code>git annex get --auto --from cloud</code> works fine</li>
</ul>
<p><em>Are there any symlinks in the path to /Users/andrew/notes ? Eg, is /Users a symlink, or /Users/andrew a symlink, or //Users/andrew/notes itself symlinked to elsewhere?</em></p>
<p><strong>No</strong></p>
<p><em>You say it's only failing for some files. Do the filenames that it's failing on contain any non-ascii characters?</em></p>
<p><strong>They seem normal.</strong></p>
<p><em>So, please paste in git annex version and git annex info output.</em></p>
<pre><code>git-annex version: 6.20161110-gd48f4ca
build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV FsEvents XMPP ConcurrentOutput TorrentParser MagicMime Feeds Quvi
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
local repository version: 6
supported repository versions: 3 5 6
upgrade supported from repository versions: 0 1 2 3 4 5
operating system: darwin x86_64
repository mode: indirect
trusted repositories: 0
semitrusted repositories: 10
00000000-0000-0000-0000-000000000001 -- web
00000000-0000-0000-0000-000000000002 -- bittorrent
22de57a0-c9ca-4bfe-8349-3141b3a87c8f -- Dream Objects [cloud]
334791ca-c284-4a87-a233-fc29be00d31a -- [disc_May-2-2015_a]
4c57ac0e-b8fe-4b4b-98d3-fb0a1b6b9657 -- MacBook Air [here]
6a85150d-6ea2-4ba1-92ce-8f4ef575b8e0 -- prowl MacBook Mini
896c3d52-427a-41a1-867c-d18e6740d758 -- disc_May_4_2015_1
96391b13-3981-430f-ac3b-6210e3d4e759 -- [disc_May-2-2015_b]
b4a41e90-2398-4bba-aaf5-d8f8cd78a5bc -- 2TB USB Drive [usbdrive]
e42b223d-ec04-4ad8-bdf7-8429a45d844c -- disc_May-2-2015_a
untrusted repositories: 0
transfers in progress: none
available local disk space: 2.32 gigabytes (+1 megabyte reserved)
temporary object directory size: 29.47 megabytes (clean up with git-annex unused)
local annex keys: 4104
local annex size: 10.53 gigabytes
annexed files in working tree: 6417
size of annexed files in working tree: 80.75 gigabytes
bloom filter size: 32 mebibytes (0.8% full)
backend usage:
SHA256E: 6417
</code></pre>
updates sync vs copy?http://git-annex.branchable.com/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_5_e96503f38cba755b2e6bd89b1ffab6ff/andrew2017-01-21T14:56:26Z2017-01-21T14:56:26Z
<p>Any thoughts Joey? Reminder I have a v6 repo, that spits out errors <code>git-annex: fd:24: hFlush: resource vanished (Broken pipe)</code> when using sync --content.</p>
<p>I am not sure why <code>git annex sync --content cloud</code> should spew errors, and <code>git annex copy --auto --to cloud</code> runs without issue. Anything I should do on my end to investigate this issue further?</p>
<p>Thanks,</p>
<p>Andrew</p>
RESOLVED / deleted all files then re-added as ziphttp://git-annex.branchable.com/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_6_7443fe5f7384431914c714c2b462cf5c/andrew2017-01-23T02:28:23Z2017-01-23T02:28:23Z
<p>OK. I ended up having to delete all the files from git, then re-add them zipped up.</p>
<p>I re-read the post you mentioned, this helped me resolve my issue:
<a href="http://git-annex.branchable.com/bugs/assistant_crashes_in_TransferScanner/">assistant crashes in TransferScanner</a> (from johannes)</p>
<p>Johannes dropped the file, then re-added, that fixed his issue. So I tried that first.</p>
<p>I did the following:</p>
<ul>
<li>git-annex dropped all problem files. this didn't fix issue</li>
<li>deleted all problem files (using OS-X Finder), committed the deletion with git (git add -A), issue then fixed
<ul>
<li>re-added deleted files, issue came back</li>
<li>re-added deleted files in different folder, issue came back</li>
<li>re-added deleted files all with different filenames in different path, issue came back</li>
<li>zipped up folder of problem files, then added to git-annex, no issues</li>
</ul>
</li>
</ul>
comment 7http://git-annex.branchable.com/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_7_63225cc9557886ae12b3450f68812815/joey2017-02-20T17:45:16Z2017-02-20T16:54:19Z
Can you give me some examples of the names of the "problem files"?
the problem fileshttp://git-annex.branchable.com/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_8_baefe35266021184ee78440226fe2a8d/andrew2017-02-20T18:11:03Z2017-02-20T18:11:03Z
<p>Joey,</p>
<p>the file names seem quite ordinary:</p>
<pre><code>cv/submissions/BCA_Submission/small_images/drawblocks_2015_IMG_1719.jpg
cv/submissions/BCA_Submission/small_images/drawblocks_2015_IMG_3848.jpg
cv/submissions/BCA_Submission/small_images/macropavilion_2016_IMG_0391.jpg
cv/submissions/BCA_Submission/small_images/sequencing_2016_DSC5048.jpg
cv/submissions/BCA_Submission/small_images/sequencing_2016_IMG_8231.jpg
</code></pre>
<p>Possibly this is related to an issue I had with v6 and <code>annex.largefiles</code> about 12-months ago. I had done <code>git config annex.largefiles 'largerthan=100kb and not (include=*.c or include=*.h)'</code>. But I believe this resulted in git-annex thinking locked files with no content present should be added to git instead of annex? After doing a <code>git-annex add .</code> I now had a bunch of files whose content was lost. Or perhaps I did a sync with a v5 repo, or perhaps I did a sync with a repo with a different large files settings, I can't remember. Anyhow, I managed to lose links to the file content and <code>git-annex get</code> or fsck couldn't retrieve them. I never filed a bug report because I was never able to reproduce the issue on a clean repo.</p>
<p>Anyhow, perhaps spelunking into the git log would give some answers?</p>
<p>Andrew</p>
comment 9http://git-annex.branchable.com/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_9_a9ed50cd4a1f436822ae97bfcd1ced48/andrew2018-07-17T18:58:58Z2018-07-17T18:58:58Z
Most likely related to and fixed by now fixed bug: <a href="https://git-annex.branchable.com/bugs/add_fails_with_v6_repo_when_four_levels_deep/">https://git-annex.branchable.com/bugs/add_fails_with_v6_repo_when_four_levels_deep/</a>
comment 9http://git-annex.branchable.com/forum/sync_--content__44___fatal_is_outside_repository_errors/comment_10_d4a649874ee9c9e6a4c0252700607713/joey2018-07-17T19:20:56Z2018-07-17T19:16:23Z
<p>Hmm, maybe.</p>
<p>It's very difficult to say without enough of a transcript to have any idea
at what point in the <code>git annex sync</code> it was failing. If it failed in the
"commit" part, then yes, it could be git commit running the smudge/clean filter
and the worktree path problem affecting it.</p>