Recent changes to this wiki:

annex.merge-annex-branches
Added annex.merge-annex-branches config setting which can be used to
disable automatic merge of git-annex branches.
I wonder if git-annex merge/sync/assistant should disable this
setting? Not sure yet, so have not done so. May be that users will not set
it in git config, but pass it via -c to commands that need it.
Checking the config setting adds a very small overhead, but it's
only checked once per command so should be insignificant.
This commit was supported by the NSF-funded DataLad project.
diff --git a/Annex/Branch.hs b/Annex/Branch.hs
index 070f8ff98..36cfb8b55 100644
--- a/Annex/Branch.hs
+++ b/Annex/Branch.hs
@@ -140,7 +140,13 @@ forceUpdate = updateTo =<< siblingBranches
  - Returns True if any refs were merged in, False otherwise.
  -}
 updateTo :: [(Git.Sha, Git.Branch)] -> Annex Bool
-updateTo pairs = do
+updateTo pairs = ifM (annexMergeAnnexBranches <$> Annex.getGitConfig)
+	( updateTo' pairs
+	, return False
+	)
+
+updateTo' :: [(Git.Sha, Git.Branch)] -> Annex Bool
+updateTo' pairs = do
 	-- ensure branch exists, and get its current ref
 	branchref <- getBranch
 	dirty <- journalDirty
diff --git a/CHANGELOG b/CHANGELOG
index c03db1f93..fdda10391 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -26,6 +26,8 @@ git-annex (6.20180113) UNRELEASED; urgency=medium
     log.
   * importfeed: Fix a failure when downloading with youtube-dl
     and the destination subdirectory does not exist yet.
+  * Added annex.merge-annex-branches config setting which
+    can be used to disable automatic merge of git-annex branches.
 
  -- Joey Hess <id@joeyh.name>  Wed, 24 Jan 2018 20:42:55 -0400
 
diff --git a/Types/GitConfig.hs b/Types/GitConfig.hs
index ad22dadb8..4b89a6bdc 100644
--- a/Types/GitConfig.hs
+++ b/Types/GitConfig.hs
@@ -59,6 +59,7 @@ data GitConfig = GitConfig
 	, annexBloomAccuracy :: Maybe Int
 	, annexSshCaching :: Maybe Bool
 	, annexAlwaysCommit :: Bool
+	, annexMergeAnnexBranches :: Bool
 	, annexDelayAdd :: Maybe Int
 	, annexHttpHeaders :: [String]
 	, annexHttpHeadersCommand :: Maybe String
@@ -116,6 +117,7 @@ extractGitConfig r = GitConfig
 	, annexBloomAccuracy = getmayberead (annex "bloomaccuracy")
 	, annexSshCaching = getmaybebool (annex "sshcaching")
 	, annexAlwaysCommit = getbool (annex "alwayscommit") True
+	, annexMergeAnnexBranches = getbool (annex "merge-annex-branches") True
 	, annexDelayAdd = getmayberead (annex "delayadd")
 	, annexHttpHeaders = getlist (annex "http-headers")
 	, annexHttpHeadersCommand = getmaybe (annex "http-headers-command")
diff --git a/doc/bugs/impossible_to_perform___34__read-only__34___git_annex_info_without_write_permissions/comment_3_05f7eec04634e5b4e200cf3dca1bb1b1._comment b/doc/bugs/impossible_to_perform___34__read-only__34___git_annex_info_without_write_permissions/comment_3_05f7eec04634e5b4e200cf3dca1bb1b1._comment
new file mode 100644
index 000000000..3edb6654b
--- /dev/null
+++ b/doc/bugs/impossible_to_perform___34__read-only__34___git_annex_info_without_write_permissions/comment_3_05f7eec04634e5b4e200cf3dca1bb1b1._comment
@@ -0,0 +1,33 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2018-02-22T17:27:06Z"
+ content="""
+There are situations where a git command that appears to be read-only,
+such as `git status` actually writes to the repository behind the scenes.
+It looks like git ignores write errors in at least some such cases. So
+there is precident for implicit read-only support, but I think not in cases
+where it would involve significant behavior changes, like it would for the
+git-annex branch auto-merging. In git's case the behavior change probably
+only involves repeated `git status` runs being slower than otherwise,
+or something like that.
+
+As well as populating the .git/annex/index file with information merged in from
+recently fetched git-annex branches, git-annex may need to write to other files
+in order to prepare caches needed to perform what appears to be "read-only"
+query operation, or to lock files in order to prevent someone who does have
+write access from dropping them in a situation where that will lose data.
+An example of the latter is running `git annex drop` in a repository you do have
+write access to, and it needing to exclusively lock files in origin,
+which requires write access to origin as well. Without write access,
+the drop may fail.
+
+The --read-only flag seems to be setting up a situation where git-annex handles
+some things being read-only, but then someone expects the flag to 
+make some other thing work read-only, which git-annex can't manage to support
+for whatever reason. 
+
+So I prefer a more specific name, like annex.merge-annex-branches=false.
+
+Implemented that.
+"""]]
diff --git a/doc/git-annex-config.mdwn b/doc/git-annex-config.mdwn
index bf24251d8..ed602ec95 100644
--- a/doc/git-annex-config.mdwn
+++ b/doc/git-annex-config.mdwn
@@ -34,9 +34,9 @@ These settings can be overridden on a per-repository basis using
 
 * `annex.resolvemerge`
 
-  Set to false to prevent merge conflicts being automatically resolved
-  by the git-annex assitant, git-annex sync, git-annex merge, 
-  and the git-annex post-receive hook.
+  Set to false to prevent merge conflicts in the checked out branch
+  being automatically resolved by the git-annex assitant,
+  git-annex sync, git-annex merge, and the git-annex post-receive hook.
 
 * `annex.synccontent`
 
diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn
index 684e5c60a..db8cfca61 100644
--- a/doc/git-annex.mdwn
+++ b/doc/git-annex.mdwn
@@ -974,6 +974,15 @@ Here are all the supported configuration settings.
   since it could garbage collect objects that are staged in git-annex's
   index but not yet committed.
 
+* `annex.merge-annex-branches`
+
+  By default, git-annex branches that have been pulled from remotes
+  are automatically merged into the local git-annex branch, so that
+  git-annex has the most up-to-date possible knowledge.
+
+  To avoid that merging, set this to "false". This can be useful
+  particularly when you don't have write permission to the repository.
+
 * `annex.hardlink`
 
   Set this to `true` to make file contents be hard linked between the
@@ -1054,8 +1063,9 @@ Here are all the supported configuration settings.
 
 * `annex.resolvemerge`
 
-  Set to false to prevent merge conflicts being automatically resolved
-  by the git-annex assitant, git-annex sync, git-annex merge,
+  Set to false to prevent merge conflicts in the checked out branch
+  being automatically resolved by the git-annex assitant,
+  git-annex sync, git-annex merge,
   and the git-annex post-receive hook.
 
   To configure the behavior in all clones of the repository,

wording
diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn
index f427d9409..684e5c60a 100644
--- a/doc/git-annex.mdwn
+++ b/doc/git-annex.mdwn
@@ -970,7 +970,7 @@ Here are all the supported configuration settings.
   commit the data by running `git annex merge` (or by automatic merges)
   or `git annex sync`.
 
-  Note that you beware running `git gc` if using this configuration,
+  You should beware running `git gc` when using this configuration,
   since it could garbage collect objects that are staged in git-annex's
   index but not yet committed.
 

Added a comment: --read-only flag?
diff --git a/doc/bugs/impossible_to_perform___34__read-only__34___git_annex_info_without_write_permissions/comment_2_066d2f95ffab1b36b69aa84643d67765._comment b/doc/bugs/impossible_to_perform___34__read-only__34___git_annex_info_without_write_permissions/comment_2_066d2f95ffab1b36b69aa84643d67765._comment
new file mode 100644
index 000000000..0863efa27
--- /dev/null
+++ b/doc/bugs/impossible_to_perform___34__read-only__34___git_annex_info_without_write_permissions/comment_2_066d2f95ffab1b36b69aa84643d67765._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="--read-only flag?"
+ date="2018-02-22T17:18:18Z"
+ content="""
+I hear you... so far though I was confused by the fact that what I thought would be a read-only operation was actually changing the state of the things (doing the annex merge).  Although I would have preferred just a warning like \"Cannot merge git-annex branch because of lacking permissions to do so, some information might be not up-to-date\", I wondered if then may be a generic resolution could be to add `--read-only` flag to such commands as `info` and `whereis`.  Then we (datalad) would be the one to explicitly check if there is write-permissions and if not -- issue the command in `--read-only` mode.  We might even make it a default mode of operation for some of our usecases where it would be confusing if things were changing in the background (e.g. with `ls` command)
+"""]]

response
diff --git a/doc/internals/comment_7_9c82a2878f3feb1b2a95662ed25b234b._comment b/doc/internals/comment_7_9c82a2878f3feb1b2a95662ed25b234b._comment
new file mode 100644
index 000000000..d8034bb5f
--- /dev/null
+++ b/doc/internals/comment_7_9c82a2878f3feb1b2a95662ed25b234b._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 7"""
+ date="2018-02-22T16:56:13Z"
+ content="""
+@arseny-n the misctemp directory does not normally contain anything, or
+only temp files in use by the currently running git-annex process for a
+short amount of time. The only way I know of that it can get files piled up
+in it is when you kill the git-annex process while it's using such a file.
+
+It's always safe to delete the files in misctemp as long as git-annex is
+not running. Also, the names of the files should give a pretty good clue
+about what git-annex was using the file for. For example "jlog" files are
+used for staging the journal.
+"""]]

remove duplicate comment
diff --git a/doc/internals/comment_8_b7db717de86bae61bf26599cc0f9c7cf._comment b/doc/internals/comment_8_b7db717de86bae61bf26599cc0f9c7cf._comment
deleted file mode 100644
index 2cbff5c59..000000000
--- a/doc/internals/comment_8_b7db717de86bae61bf26599cc0f9c7cf._comment
+++ /dev/null
@@ -1,14 +0,0 @@
-[[!comment format=mdwn
- username="arseny-n@6aba76e573dcdf2fd9e033fb3132944c8466125a"
- nickname="arseny-n"
- avatar="http://cdn.libravatar.org/avatar/0c138544ac6dfffb07d6a60dbe430587"
- subject=".git/annex/misctmp very large"
- date="2018-01-17T13:05:04Z"
- content="""
-Why is .git/annex/misctmp so large ? Currently I use git annex to manage pytorch models, 
-basically I have a large amount (1500 folder) of 4 Kilobytes files, some files are are bigger,
-misctmp occupies 6.2 GB, is  it ok ?
- 
-PS. Sorry, if I write this here but failed to post to the forum.
-
-"""]]

response
diff --git a/doc/forum/offsite_repo/comment_4_2e3dfd605df004330396040737a7c5ee._comment b/doc/forum/offsite_repo/comment_4_2e3dfd605df004330396040737a7c5ee._comment
new file mode 100644
index 000000000..15f8a3eff
--- /dev/null
+++ b/doc/forum/offsite_repo/comment_4_2e3dfd605df004330396040737a7c5ee._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2018-02-22T16:51:34Z"
+ content="""
+It seems likely that you have not upgraded git-annex to a version where
+fsck checks the required content yet, especially since that change has not
+yet made it into a release of git-annex. It will be in the next release, or
+you can download a daily build to get it now.
+"""]]

response
diff --git a/doc/bugs/importfeed_does_not_work_with_socks_proxy/comment_1_9797d8de55cf91288ecfc64a7e96645d._comment b/doc/bugs/importfeed_does_not_work_with_socks_proxy/comment_1_9797d8de55cf91288ecfc64a7e96645d._comment
new file mode 100644
index 000000000..3db7a070d
--- /dev/null
+++ b/doc/bugs/importfeed_does_not_work_with_socks_proxy/comment_1_9797d8de55cf91288ecfc64a7e96645d._comment
@@ -0,0 +1,23 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-02-22T16:32:33Z"
+ content="""
+It's true that importfeed calls Url.download to download the feed file, 
+without bothering to support annex.web-download-command. That could easily
+be changed.
+
+However, it also uses Url.getUrlInfo, which does not and cannot use the
+annex.web-download-command interface, and which is too complicated an
+interface to make into a hook.
+
+Indeed, annex.web-download-command was never intended to cover all
+the ways git-annex uses http, but only uses of http to download
+large file contents. And importfeed does use it for such downloads,
+but not for its other http needs.
+
+To configure use of a proxy, you would probably be best served by using
+the `http_proxy` and `https_proxy` environment variables, which are
+supported by wget, curl, and by the haskell http library that git-annex
+also uses.
+"""]]

Remove temporary code added in 6.20160619 to prime the mergedrefs log.
Repositories that are upgraded from before that version to this
one will not break, but will just not see the benefit of the mergedrefs log
speeding things up, until one new ref gets merged in.
diff --git a/Annex/Branch.hs b/Annex/Branch.hs
index c8f2f4c2f..070f8ff98 100644
--- a/Annex/Branch.hs
+++ b/Annex/Branch.hs
@@ -163,10 +163,6 @@ updateTo pairs = do
 				 - a commit needs to be done. -}
 				when dirty $
 					go branchref True [] jl
-			{- Only needed for a while, to populate the
-			 - newly added merged refs cache with already
-			 - merged refs. Can be safely removed at any time. -}
-			addMergedRefs unignoredrefs
 		else lockJournal $ go branchref dirty tomerge
 	return $ not $ null tomerge
   where
diff --git a/CHANGELOG b/CHANGELOG
index 824e4bea3..737ffc4a5 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -22,6 +22,8 @@ git-annex (6.20180113) UNRELEASED; urgency=medium
   * Added --json-error-messages option, which makes messages
     that would normally be output to standard error be included in
     the json output.
+  * Remove temporary code added in 6.20160619 to prime the mergedrefs
+    log.
 
  -- Joey Hess <id@joeyh.name>  Wed, 24 Jan 2018 20:42:55 -0400
 
diff --git a/doc/bugs/impossible_to_perform___34__read-only__34___git_annex_info_without_write_permissions/comment_1_8b60c52d8fe41718377d6d15a25cae97._comment b/doc/bugs/impossible_to_perform___34__read-only__34___git_annex_info_without_write_permissions/comment_1_8b60c52d8fe41718377d6d15a25cae97._comment
new file mode 100644
index 000000000..efef1a0e3
--- /dev/null
+++ b/doc/bugs/impossible_to_perform___34__read-only__34___git_annex_info_without_write_permissions/comment_1_8b60c52d8fe41718377d6d15a25cae97._comment
@@ -0,0 +1,20 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-02-22T16:17:35Z"
+ content="""
+The mergedrefs directory is used while building the commit to merge
+git-annex branches. So even if it was written someplace else, that commit
+would fail.
+
+I think this may be happening even when there are no
+git-annex refs to merge in, due to the transition code
+in Annex.Branch.updateTo that temporarily calls addMergedRefs
+in the "null tomerge" case. That was added in 2016, and is flagged as able
+to be safely removed. I've removed it.
+
+However, when there actually is a git-annex branch to merge, if a
+hypothetical readonly mode avoided doing so, it would necessarily see a
+different state of the git-annex branch than would be seen in non-readonly
+mode. That behavior difference could be fairly confusing potentially..
+"""]]

remove spam
diff --git a/doc/design/assistant/blog/day_288__success_stories/comment_23_87a4b839c6e7ef5ca83256ab30d06642._comment b/doc/design/assistant/blog/day_288__success_stories/comment_23_87a4b839c6e7ef5ca83256ab30d06642._comment
deleted file mode 100644
index 989c16374..000000000
--- a/doc/design/assistant/blog/day_288__success_stories/comment_23_87a4b839c6e7ef5ca83256ab30d06642._comment
+++ /dev/null
@@ -1,9 +0,0 @@
-[[!comment format=mdwn
- username="avdhusingh6497@d72e8319bb7c179778775a8b096b120fc219fbad"
- nickname="avdhusingh6497"
- avatar="http://cdn.libravatar.org/avatar/b2cea3daac3783f3472052248e6d2435"
- subject="iTunes Error 56 Apple"
- date="2018-02-22T11:48:40Z"
- content="""
-I came crossways successful ways that will help you fix the iTunes Error 56 with ease. So, here I am giving out the methods to fix the issue without any technical knowledge. https://notresponding.net/itunes-error-56/
-"""]]
diff --git a/doc/design/assistant/blog/day_288__success_stories/comment_24_6809117265d140026bb7af7dabdbdd49._comment b/doc/design/assistant/blog/day_288__success_stories/comment_24_6809117265d140026bb7af7dabdbdd49._comment
deleted file mode 100644
index 6de5812d9..000000000
--- a/doc/design/assistant/blog/day_288__success_stories/comment_24_6809117265d140026bb7af7dabdbdd49._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="avdhusingh6497@d72e8319bb7c179778775a8b096b120fc219fbad"
- nickname="avdhusingh6497"
- avatar="http://cdn.libravatar.org/avatar/b2cea3daac3783f3472052248e6d2435"
- subject="iTunes Error 56 Apple"
- date="2018-02-22T11:51:41Z"
- content="""
-I came crossways successful ways that will help you fix the iTunes Error 56 with ease. So, here I am giving out the methods to fix the issue without any technical knowledge. [iTunes Error 56 Apple](https://notresponding.net/itunes-error-56/)
-
-"""]]

Added a comment: iTunes Error 56 Apple
diff --git a/doc/design/assistant/blog/day_288__success_stories/comment_24_6809117265d140026bb7af7dabdbdd49._comment b/doc/design/assistant/blog/day_288__success_stories/comment_24_6809117265d140026bb7af7dabdbdd49._comment
new file mode 100644
index 000000000..6de5812d9
--- /dev/null
+++ b/doc/design/assistant/blog/day_288__success_stories/comment_24_6809117265d140026bb7af7dabdbdd49._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="avdhusingh6497@d72e8319bb7c179778775a8b096b120fc219fbad"
+ nickname="avdhusingh6497"
+ avatar="http://cdn.libravatar.org/avatar/b2cea3daac3783f3472052248e6d2435"
+ subject="iTunes Error 56 Apple"
+ date="2018-02-22T11:51:41Z"
+ content="""
+I came crossways successful ways that will help you fix the iTunes Error 56 with ease. So, here I am giving out the methods to fix the issue without any technical knowledge. [iTunes Error 56 Apple](https://notresponding.net/itunes-error-56/)
+
+"""]]

Added a comment: iTunes Error 56 Apple
diff --git a/doc/design/assistant/blog/day_288__success_stories/comment_23_87a4b839c6e7ef5ca83256ab30d06642._comment b/doc/design/assistant/blog/day_288__success_stories/comment_23_87a4b839c6e7ef5ca83256ab30d06642._comment
new file mode 100644
index 000000000..989c16374
--- /dev/null
+++ b/doc/design/assistant/blog/day_288__success_stories/comment_23_87a4b839c6e7ef5ca83256ab30d06642._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="avdhusingh6497@d72e8319bb7c179778775a8b096b120fc219fbad"
+ nickname="avdhusingh6497"
+ avatar="http://cdn.libravatar.org/avatar/b2cea3daac3783f3472052248e6d2435"
+ subject="iTunes Error 56 Apple"
+ date="2018-02-22T11:48:40Z"
+ content="""
+I came crossways successful ways that will help you fix the iTunes Error 56 with ease. So, here I am giving out the methods to fix the issue without any technical knowledge. https://notresponding.net/itunes-error-56/
+"""]]

Added a comment: switching to amazonka
diff --git a/doc/forum/Replace_glacier-cli__63__/comment_3_405aad8dd28ea035226f7fda470fde33._comment b/doc/forum/Replace_glacier-cli__63__/comment_3_405aad8dd28ea035226f7fda470fde33._comment
new file mode 100644
index 000000000..bcabcb13e
--- /dev/null
+++ b/doc/forum/Replace_glacier-cli__63__/comment_3_405aad8dd28ea035226f7fda470fde33._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="andrew"
+ avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
+ subject="switching to amazonka"
+ date="2018-02-21T18:15:05Z"
+ content="""
+Switching to amazonka would have the benefit of supporting SSL traffic for S3, which is a feature I would enjoy (see git-annex forum thread [No SSL traffic for S3?](http://git-annex.branchable.com/forum/No_SSL_traffic_for_S3__63__/)). I have no experience with the amazonka library though, so I can't comment on it.
+
+—Andrew
+"""]]

done
diff --git a/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output.mdwn b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output.mdwn
index 4bf193598..c634297d1 100644
--- a/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output.mdwn
+++ b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output.mdwn
@@ -16,3 +16,5 @@ or may be even an explicit "errormsg" or alike instead of just a generic "msg"
 
 
 [[!meta author=yoh]]
+
+> [[done]] --[[Joey]]
diff --git a/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_8_2e0ec2bb51a2a06184b94710616f211c._comment b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_8_2e0ec2bb51a2a06184b94710616f211c._comment
new file mode 100644
index 000000000..b66f78fef
--- /dev/null
+++ b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_8_2e0ec2bb51a2a06184b94710616f211c._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 8"""
+ date="2018-02-20T17:00:12Z"
+ content="""
+Testing, both of the cases in my previous message were actually already
+working as desired. I think it's done!
+"""]]

initial complaints about needing write permissions for info command
diff --git a/doc/bugs/impossible_to_perform___34__read-only__34___git_annex_info_without_write_permissions.mdwn b/doc/bugs/impossible_to_perform___34__read-only__34___git_annex_info_without_write_permissions.mdwn
new file mode 100644
index 000000000..6236174ce
--- /dev/null
+++ b/doc/bugs/impossible_to_perform___34__read-only__34___git_annex_info_without_write_permissions.mdwn
@@ -0,0 +1,19 @@
+
+### What version of git-annex are you using? On what operating system?
+
+6.20171001+gitg542d0649f-1~ndall+1
+
+### What steps will reproduce the problem?
+
+[[!format sh """
+$> git annex info     
+
+git-annex: .git/annex/misctmp/mergedrefs.0: createDirectory: permission denied (Permission denied)
+failed
+git-annex: info: 1 failed
+
+"""]]
+
+unless there is really a need to have this operations performed within the same repository/the same filesystem, I do not understand why generic $TMPDIR could not be used for such operations so no write access has to be demanded
+
+[[!meta author=yoh]]

Added a comment: Thanks
diff --git a/doc/forum/Replace_glacier-cli__63__/comment_2_c91e393951825cdcceac845e234d7497._comment b/doc/forum/Replace_glacier-cli__63__/comment_2_c91e393951825cdcceac845e234d7497._comment
new file mode 100644
index 000000000..ac7596a4b
--- /dev/null
+++ b/doc/forum/Replace_glacier-cli__63__/comment_2_c91e393951825cdcceac845e234d7497._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="moaxey"
+ avatar="http://cdn.libravatar.org/avatar/06abae5b84ce9bbcd39c1cbf49817d29"
+ subject="Thanks"
+ date="2018-02-19T20:57:09Z"
+ content="""
+I have it working smoothly now with glacier-cli.
+"""]]

update
diff --git a/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_7_246e35f32f77af3b2924577b1bf45001._comment b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_7_246e35f32f77af3b2924577b1bf45001._comment
new file mode 100644
index 000000000..7bbe0e00c
--- /dev/null
+++ b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_7_246e35f32f77af3b2924577b1bf45001._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 7"""
+ date="2018-02-19T19:49:11Z"
+ content="""
+Basic --json-error-messages implemented after 4 hours work.
+
+Still needing to be done:
+
+* A few commands like `git annex info` have a custom json outputter,
+  and may not output the error-messages field by default, or may not make
+  sense to support the option.
+* Flush json buffer after fatal error.
+"""]]

better doc for --json-error-messages
Word so warnings can be included, not only errors.
diff --git a/doc/git-annex-add.mdwn b/doc/git-annex-add.mdwn
index 432d91a64..ff7bc4004 100644
--- a/doc/git-annex-add.mdwn
+++ b/doc/git-annex-add.mdwn
@@ -68,7 +68,8 @@ annexed content, and other symlinks.
 
 * `--json-error-messages`
 
-  Include any error messages in the json, rather than output to stderr.
+  Messages that would normally be output to standard error are included in
+  the json instead.
 
 * `--batch`
 
diff --git a/doc/git-annex-addurl.mdwn b/doc/git-annex-addurl.mdwn
index 2f6d878fb..4748073ed 100644
--- a/doc/git-annex-addurl.mdwn
+++ b/doc/git-annex-addurl.mdwn
@@ -99,7 +99,8 @@ be used to get better filenames.
 
 * `--json-error-messages`
 
-  Include any error messages in the json, rather than output to stderr.
+  Messages that would normally be output to standard error are included in
+  the json instead.
 
 # CAVEATS
 
diff --git a/doc/git-annex-copy.mdwn b/doc/git-annex-copy.mdwn
index 22d1a1b8a..e817fd618 100644
--- a/doc/git-annex-copy.mdwn
+++ b/doc/git-annex-copy.mdwn
@@ -99,7 +99,8 @@ Copies the content of files from or to another remote.
 
 * `--json-error-messages`
 
-  Include any error messages in the json, rather than output to stderr.
+  Messages that would normally be output to standard error are included in
+  the json instead.
 
 # SEE ALSO
 
diff --git a/doc/git-annex-drop.mdwn b/doc/git-annex-drop.mdwn
index 651b5377d..0c07e150d 100644
--- a/doc/git-annex-drop.mdwn
+++ b/doc/git-annex-drop.mdwn
@@ -89,7 +89,8 @@ safe to do so.
 
 * `--json-error-messages`
 
-  Include any error messages in the json, rather than output to stderr.
+  Messages that would normally be output to standard error are included in
+  the json instead.
 
 # SEE ALSO
 
diff --git a/doc/git-annex-dropkey.mdwn b/doc/git-annex-dropkey.mdwn
index abdd86120..681640737 100644
--- a/doc/git-annex-dropkey.mdwn
+++ b/doc/git-annex-dropkey.mdwn
@@ -31,7 +31,8 @@ exist; using it can easily result in data loss.
 
 * `--json-error-messages`
 
-  Include any error messages in the json, rather than output to stderr.
+  Messages that would normally be output to standard error are included in
+  the json instead.
 
 # SEE ALSO
 
diff --git a/doc/git-annex-examinekey.mdwn b/doc/git-annex-examinekey.mdwn
index fd4ea570b..83baa5667 100644
--- a/doc/git-annex-examinekey.mdwn
+++ b/doc/git-annex-examinekey.mdwn
@@ -35,7 +35,8 @@ that can be determined purely by looking at the key.
 
 * `--json-error-messages`
 
-  Include any error messages in the json, rather than output to stderr.
+  Messages that would normally be output to standard error are included in
+  the json instead.
 
 * `--batch`
 
diff --git a/doc/git-annex-find.mdwn b/doc/git-annex-find.mdwn
index 71ada78c9..dafb0e7b3 100644
--- a/doc/git-annex-find.mdwn
+++ b/doc/git-annex-find.mdwn
@@ -56,7 +56,8 @@ finds files in the current directory and its subdirectories.
 
 * `--json-error-messages`
 
-  Include any error messages in the json, rather than output to stderr.
+  Messages that would normally be output to standard error are included in
+  the json instead.
 
 * `--batch`
 
diff --git a/doc/git-annex-fsck.mdwn b/doc/git-annex-fsck.mdwn
index 037ec2628..a9392935f 100644
--- a/doc/git-annex-fsck.mdwn
+++ b/doc/git-annex-fsck.mdwn
@@ -100,7 +100,8 @@ With parameters, only the specified files are checked.
 
 * `--json-error-messages`
 
-  Include any error messages in the json, rather than output to stderr.
+  Messages that would normally be output to standard error are included in
+  the json instead.
 
 # OPTIONS
 
diff --git a/doc/git-annex-get.mdwn b/doc/git-annex-get.mdwn
index 0fd10f1ea..2f3981165 100644
--- a/doc/git-annex-get.mdwn
+++ b/doc/git-annex-get.mdwn
@@ -108,7 +108,8 @@ or transferring them from some kind of key-value store.
 
 * `--json-error-messages`
 
-  Include any error messages in the json, rather than output to stderr.
+  Messages that would normally be output to standard error are included in
+  the json instead.
 
 # SEE ALSO
 
diff --git a/doc/git-annex-import.mdwn b/doc/git-annex-import.mdwn
index 020e57325..c1bcbd2b9 100644
--- a/doc/git-annex-import.mdwn
+++ b/doc/git-annex-import.mdwn
@@ -83,7 +83,8 @@ Several options can be used to adjust handling of duplicate files.
 
 * `--json-error-messages`
 
-  Include any error messages in the json, rather than output to stderr.
+  Messages that would normally be output to standard error are included in
+  the json instead.
 
 # CAVEATS
 
diff --git a/doc/git-annex-info.mdwn b/doc/git-annex-info.mdwn
index a2511a509..311edc4d9 100644
--- a/doc/git-annex-info.mdwn
+++ b/doc/git-annex-info.mdwn
@@ -28,7 +28,8 @@ for the repository as a whole.
 
 * `--json-error-messages`
 
-  Include any error messages in the json, rather than output to stderr.
+  Messages that would normally be output to standard error are included in
+  the json instead.
 
 * `--bytes`
 
diff --git a/doc/git-annex-lock.mdwn b/doc/git-annex-lock.mdwn
index 02495073b..c13654dbe 100644
--- a/doc/git-annex-lock.mdwn
+++ b/doc/git-annex-lock.mdwn
@@ -25,7 +25,8 @@ the files any longer, or have made modifications you want to discard.
 
 * `--json-error-messages`
 
-  Include any error messages in the json, rather than output to stderr.
+  Messages that would normally be output to standard error are included in
+  the json instead.
 
 # SEE ALSO
 
diff --git a/doc/git-annex-metadata.mdwn b/doc/git-annex-metadata.mdwn
index 509273488..7786d999b 100644
--- a/doc/git-annex-metadata.mdwn
+++ b/doc/git-annex-metadata.mdwn
@@ -114,7 +114,8 @@ automatically.
 
 * `--json-error-messages`
 
-  Include any error messages in the json, rather than output to stderr.
+  Messages that would normally be output to standard error are included in
+  the json instead.
 
 * `--batch`
 
diff --git a/doc/git-annex-mirror.mdwn b/doc/git-annex-mirror.mdwn
index 0c6e92126..d665a6e77 100644
--- a/doc/git-annex-mirror.mdwn
+++ b/doc/git-annex-mirror.mdwn
@@ -77,7 +77,8 @@ contents. Use [[git-annex-sync]](1) for that.
 
 * `--json-error-messages`
 
-  Include any error messages in the json, rather than output to stderr.
+  Messages that would normally be output to standard error are included in
+  the json instead.
 
 # SEE ALSO
 
diff --git a/doc/git-annex-move.mdwn b/doc/git-annex-move.mdwn
index 19bc2db68..ce932b198 100644
--- a/doc/git-annex-move.mdwn
+++ b/doc/git-annex-move.mdwn

(Diff truncated)
add --json-error-messages (not yet implemented)
Added --json-error-messages option, which includes error messages in the
json output, rather than outputting them to stderr.
The actual rediretion of errors is not implemented yet, this is only
the docs and option plumbing.
This commit was supported by the NSF-funded DataLad project.
diff --git a/CHANGELOG b/CHANGELOG
index 204004a60..fc204568e 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -19,6 +19,8 @@ git-annex (6.20180113) UNRELEASED; urgency=medium
     compile.
   * Fix behavior of --json-progress followed by --json, in which
     the latter option disabled the former.
+  * Added --json-error-messages option, which includes error messages
+    in the json output, rather than outputting them to stderr.
 
  -- Joey Hess <id@joeyh.name>  Wed, 24 Jan 2018 20:42:55 -0400
 
diff --git a/CmdLine/GitAnnex/Options.hs b/CmdLine/GitAnnex/Options.hs
index 51c55b056..143bb6498 100644
--- a/CmdLine/GitAnnex/Options.hs
+++ b/CmdLine/GitAnnex/Options.hs
@@ -191,7 +191,7 @@ annexedMatchingOptions = concat
 	[ nonWorkTreeMatchingOptions'
 	, fileMatchingOptions'
 	, combiningOptions
-	, [timeLimitOption]
+	, timeLimitOption
 	]
 
 -- Matching options that don't need to examine work tree files.
@@ -294,37 +294,51 @@ combiningOptions =
 	longopt o h = globalFlag (Limit.addToken o) ( long o <> help h <> hidden )
 	shortopt o h = globalFlag (Limit.addToken [o]) ( short o <> help h <> hidden )
 
-jsonOption :: GlobalOption
-jsonOption = globalFlag (Annex.setOutput (JSONOutput jsonoptions))
-	( long "json" <> short 'j'
-	<> help "enable JSON output"
-	<> hidden
-	)
+jsonOptions :: [GlobalOption]
+jsonOptions = 
+	[ globalFlag (Annex.setOutput (JSONOutput stdjsonoptions))
+		( long "json" <> short 'j'
+		<> help "enable JSON output"
+		<> hidden
+		)
+	, globalFlag (Annex.setOutput (JSONOutput jsonerrormessagesoptions))
+		( long "json-error-messages"
+		<> help "include error messages in JSON"
+		<> hidden
+		)
+	]
   where
-	jsonoptions = JSONOptions
+	stdjsonoptions = JSONOptions
 		{ jsonProgress = False
+		, jsonErrorMessages = False
 		}
+	jsonerrormessagesoptions = stdjsonoptions { jsonErrorMessages = True }
 
-jsonProgressOption :: GlobalOption
-jsonProgressOption = globalFlag (Annex.setOutput (JSONOutput jsonoptions))
-	( long "json-progress"
-	<> help "include progress in JSON output"
-	<> hidden
-	)
+jsonProgressOption :: [GlobalOption]
+jsonProgressOption = 
+	[ globalFlag (Annex.setOutput (JSONOutput jsonoptions))
+		( long "json-progress"
+		<> help "include progress in JSON output"
+		<> hidden
+		)
+	]
   where
 	jsonoptions = JSONOptions
 		{ jsonProgress = True
+		, jsonErrorMessages = False
 		}
 
 -- Note that a command that adds this option should wrap its seek
 -- action in `allowConcurrentOutput`.
-jobsOption :: GlobalOption
-jobsOption = globalSetter set $ 
-	option auto
-		( long "jobs" <> short 'J' <> metavar paramNumber
-		<> help "enable concurrent jobs"
-		<> hidden
-		)
+jobsOption :: [GlobalOption]
+jobsOption = 
+	[ globalSetter set $ 
+		option auto
+			( long "jobs" <> short 'J' <> metavar paramNumber
+			<> help "enable concurrent jobs"
+			<> hidden
+			)
+	]
   where
 	set n = do
 		Annex.changeState $ \s -> s { Annex.concurrency = Concurrent n }
@@ -332,12 +346,14 @@ jobsOption = globalSetter set $
 		when (n > c) $
 			liftIO $ setNumCapabilities n
 
-timeLimitOption :: GlobalOption
-timeLimitOption = globalSetter Limit.addTimeLimit $ strOption
-	( long "time-limit" <> short 'T' <> metavar paramTime
-	<> help "stop after the specified amount of time"
-	<> hidden
-	)
+timeLimitOption :: [GlobalOption]
+timeLimitOption = 
+	[ globalSetter Limit.addTimeLimit $ strOption
+		( long "time-limit" <> short 'T' <> metavar paramTime
+		<> help "stop after the specified amount of time"
+		<> hidden
+		)
+	]
 
 data DaemonOptions = DaemonOptions
 	{ foregroundDaemonOption :: Bool
diff --git a/Command.hs b/Command.hs
index d1d539f45..b886e4fe2 100644
--- a/Command.hs
+++ b/Command.hs
@@ -79,9 +79,9 @@ allowMessages = do
 noRepo :: (String -> Parser (IO ())) -> Command -> Command
 noRepo a c = c { cmdnorepo = Just (a (cmdparamdesc c)) }
 
-{- Adds global options to a command's. -}
-withGlobalOptions :: [GlobalOption] -> Command -> Command
-withGlobalOptions os c = c { cmdglobaloptions = cmdglobaloptions c ++ os }
+{- Adds global options to a command. -}
+withGlobalOptions :: [[GlobalOption]] -> Command -> Command
+withGlobalOptions os c = c { cmdglobaloptions = cmdglobaloptions c ++ concat os }
 
 {- For start and perform stages to indicate what step to run next. -}
 next :: a -> Annex (Maybe a)
diff --git a/Command/Add.hs b/Command/Add.hs
index 638da101e..10148ad50 100644
--- a/Command/Add.hs
+++ b/Command/Add.hs
@@ -22,9 +22,10 @@ import Annex.Version
 import Git.FilePath
 
 cmd :: Command
-cmd = notBareRepo $ withGlobalOptions (jobsOption : jsonOption : fileMatchingOptions) $
-	command "add" SectionCommon "add files to annex"
-		paramPaths (seek <$$> optParser)
+cmd = notBareRepo $ 
+	withGlobalOptions [jobsOption, jsonOptions, fileMatchingOptions] $
+		command "add" SectionCommon "add files to annex"
+			paramPaths (seek <$$> optParser)
 
 data AddOptions = AddOptions
 	{ addThese :: CmdParams
diff --git a/Command/AddUrl.hs b/Command/AddUrl.hs
index 995848ed2..dfdbf5b5a 100644
--- a/Command/AddUrl.hs
+++ b/Command/AddUrl.hs
@@ -34,7 +34,7 @@ import Utility.Path.Max
 import qualified Annex.Transfer as Transfer
 
 cmd :: Command
-cmd = notBareRepo $ withGlobalOptions [jobsOption, jsonOption, jsonProgressOption] $
+cmd = notBareRepo $ withGlobalOptions [jobsOption, jsonOptions, jsonProgressOption] $
 	command "addurl" SectionCommon "add urls to annex"
 		(paramRepeating paramUrl) (seek <$$> optParser)
 
diff --git a/Command/Copy.hs b/Command/Copy.hs
index b3b860fef..85a556a14 100644
--- a/Command/Copy.hs
+++ b/Command/Copy.hs
@@ -14,7 +14,7 @@ import Annex.Wanted
 import Annex.NumCopies
 
 cmd :: Command
-cmd = withGlobalOptions (jobsOption : jsonOption : jsonProgressOption : annexedMatchingOptions) $
+cmd = withGlobalOptions [jobsOption, jsonOptions, jsonProgressOption, annexedMatchingOptions] $
 	command "copy" SectionCommon
 		"copy content of files to/from another repository"
 		paramPaths (seek <--< optParser)
diff --git a/Command/Drop.hs b/Command/Drop.hs
index 275714a65..09385dddb 100644
--- a/Command/Drop.hs
+++ b/Command/Drop.hs
@@ -23,7 +23,7 @@ import System.Log.Logger (debugM)
 import qualified Data.Set as S
 
 cmd :: Command
-cmd = withGlobalOptions (jobsOption : jsonOption : annexedMatchingOptions) $
+cmd = withGlobalOptions [jobsOption, jsonOptions, annexedMatchingOptions] $
 	command "drop" SectionCommon
 		"remove content of files from repository"
 		paramPaths (seek <$$> optParser)
diff --git a/Command/DropKey.hs b/Command/DropKey.hs
index f3f2333dd..7acd3d0fa 100644
--- a/Command/DropKey.hs
+++ b/Command/DropKey.hs
@@ -13,7 +13,7 @@ import Logs.Location
 import Annex.Content
 

(Diff truncated)
Added a comment: I can reproduce
diff --git a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_1_a8a77df47eee0e15c25a89b68ec10e8f._comment b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_1_a8a77df47eee0e15c25a89b68ec10e8f._comment
new file mode 100644
index 000000000..05c2aeb8c
--- /dev/null
+++ b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_1_a8a77df47eee0e15c25a89b68ec10e8f._comment
@@ -0,0 +1,35 @@
+[[!comment format=mdwn
+ username="olaf"
+ avatar="http://cdn.libravatar.org/avatar/4ae498d3d6ee558d6b65caa658f72572"
+ subject="I can reproduce"
+ date="2018-02-19T01:05:06Z"
+ content="""
+### Please describe the problem.
+
+Creating or adding remotes to an _existing_ repository via the webapp results in
+> Internal Server Error
+>
+> there is no available git remote named \"XYZ\"
+
+**Creating a new repository** seems to create the repo and update the remotes (checked via `git` at command line) but does not update the repos in the webapp and results in the error:
+
+    19/Feb/2018:11:48:28 +1100 [Error#yesod-core] there is no available git remote named \"XYZ\" @(yesod-core-1.4.37.2-AqCgZCpSjdiDLzXFcWTxPQ:Yesod.Core.Class.Yesod ./Yesod/Core/Class/Yesod.hs:693:5)
+
+
+**Adding an existing repo** to the current repo in the webapp results in the errors:  (interesting as it first notes that the remote already exists and then complains that it's not available...)
+
+    fatal: remote XYZ already exists.
+    19/Feb/2018:11:52:24 +1100 [Error#yesod-core] there is no available git remote named \"XYZ\" @(yesod-core-1.4.37.2-AqCgZCpSjdiDLzXFcWTxPQ:Yesod.Core.Class.Yesod ./Yesod/Core/Class/Yesod.hs:693:5)
+
+
+### Version
+    git-annex version: 6.20180112
+    build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV FsEvents ConcurrentOutput TorrentParser MagicMime Feeds Testsuite
+    dependency versions: aws-0.18 bloomfilter-2.0.1.0 cryptonite-0.24 DAV-1.3.1 feed-1.0.0.0 ghc-8.2.2 http-client-0.5.7.1 persistent-sqlite-2.6.4 torrent-10000.1.1 uuid-1.2.6 yesod-1.4.5
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+    local repository version: 5
+    supported repository versions: 3 5 6
+    upgrade supported from repository versions: 0 1 2 3 4 5
+    operating system: darwin x86_64
+"""]]

Added a comment
diff --git a/doc/forum/offsite_repo/comment_3_4b233762116c52bd2a754f49152201b7._comment b/doc/forum/offsite_repo/comment_3_4b233762116c52bd2a754f49152201b7._comment
new file mode 100644
index 000000000..a0ae5865a
--- /dev/null
+++ b/doc/forum/offsite_repo/comment_3_4b233762116c52bd2a754f49152201b7._comment
@@ -0,0 +1,29 @@
+[[!comment format=mdwn
+ username="ericm"
+ avatar="http://cdn.libravatar.org/avatar/67ca64c5a99fb142fc2e3916333881ca"
+ subject="comment 3"
+ date="2018-02-18T13:46:00Z"
+ content="""
+Hi,
+
+I first tried messing around with required content settings in a test repo in direct mode. This is what I did to initialize a test repo:
+
+    git clone existing-repo
+    git annex init
+    git annex required . \"include=*.jpg\"
+
+Then I run fsck:
+
+    git annex fsck 
+
+And I get this output:
+
+    fsck viber/0.02.01.0e411306e0b3fc90.jpg (fixing link) ok
+    fsck viber/0.02.01.20943bf9c18e42.jpg (fixing link) ok
+    fsck viber/0.02.01.5126da68987493.jpg (fixing link) ok
+    ...
+
+I expected the fsck command to complain that the required jpg files are not present in the current repo and fail. However, that is not what I'm getting.
+
+Is this the expected behavior for a direct-mode repository? If not, is there something I missed?
+"""]]

diff --git a/doc/bugs/schedule.log_added_to_annex__63_____58____47____47____47__.mdwn b/doc/bugs/schedule.log_added_to_annex__63_____58____47____47____47__.mdwn
new file mode 100644
index 000000000..998213582
--- /dev/null
+++ b/doc/bugs/schedule.log_added_to_annex__63_____58____47____47____47__.mdwn
@@ -0,0 +1,55 @@
+### Please describe the problem.
+
+Look at this diff:
+
+```
+diff --git a/schedule.log b/schedule.log
+index 29c727f8c1..2f6d2f0858 100644
+--- a/schedule.log
++++ b/schedule.log
+@@ -1,19 +1 @@
+-0e6c5057-a323-4bec-b6d2-98bae312e7cd  timestamp=1518571009.042438126s
+-0e6c5057-a323-4bec-b6d2-98bae312e7cd fsck self 15m every day at any time timestamp=1503189066.20635041s
+-179b6e7f-0b2f-43b9-a95d-39df5b52d2fc  timestamp=1518565854.651788575s
+-179b6e7f-0b2f-43b9-a95d-39df5b52d2fc  timestamp=1518565869.769672252s
+-179b6e7f-0b2f-43b9-a95d-39df5b52d2fc fsck self 15m every day at any time timestamp=1503191937.325891749s
+-668bdeb3-a3e2-4a9f-ae1c-bae1880c62b8  timestamp=1516100332.848743619s
+-668bdeb3-a3e2-4a9f-ae1c-bae1880c62b8  timestamp=1518305729.698369069s
+-668bdeb3-a3e2-4a9f-ae1c-bae1880c62b8  timestamp=1518565670.560184121s
+-668bdeb3-a3e2-4a9f-ae1c-bae1880c62b8  timestamp=1518868059.554534192s
+-668bdeb3-a3e2-4a9f-ae1c-bae1880c62b8 fsck self 15m every day at any time timestamp=1504318394.78758178s
+-7f0bbee7-826f-4021-a774-0569ff58ed54 fsck self 10m every day at any time timestamp=1514213643.848668933s
+-7f0bbee7-826f-4021-a774-0569ff58ed54 fsck self 15m every day at any time timestamp=1504177427.472846327s
+-7f0bbee7-826f-4021-a774-0569ff58ed54 fsck self 1h every day at any time timestamp=1496609751.566017331s
+-83ffd5ab-40b3-4950-b437-87339242be57 fsck self 15m every day at any time timestamp=1503175558.988571178s
+-ba3a1e0a-9191-43b4-a324-1fee91a9bb8f  timestamp=1503184784.521086496s
+-ba3a1e0a-9191-43b4-a324-1fee91a9bb8f  timestamp=1518566966.391960267s
+-ba3a1e0a-9191-43b4-a324-1fee91a9bb8f fsck self 15m every day at any time timestamp=1503183823.810567346s
+-ba3a1e0a-9191-43b4-a324-1fee91a9bb8f fsck self 15m every day at any time timestamp=1503185213.589823679s
+-ba3a1e0a-9191-43b4-a324-1fee91a9bb8f fsck self 5m every day at any time timestamp=1514056713.473067585s
++/annex/objects/SHA256E-s1676--4e76f9f5e9c5eafe53367a6c493d7618366bcfea6ac8a859fcbaba55ed31f349.log
+```
+
+### What steps will reproduce the problem?
+
+Absolutely no idea.
+
+
+### What version of git-annex are you using? On what operating system?
+
+```
+git-annex version: 6.20170818
+operating system: linux x86_64
+```
+
+
+### Please provide any additional information below.
+
+None
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Considering my two other issues »WHICH LOSE DATA«, I can only conclude it’s too complicated, cf.
+
+* https://git-annex.branchable.com/bugs/Data_loss_when_copying_files_with_running_assistant/
+* https://git-annex.branchable.com/bugs/Infinite_loop_when_synchronizing_between_many_machines/

Added a comment
diff --git a/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_6_3d5a33476a3fa773fda1e5f7a2d8c453._comment b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_6_3d5a33476a3fa773fda1e5f7a2d8c453._comment
new file mode 100644
index 000000000..f7d965f3b
--- /dev/null
+++ b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_6_3d5a33476a3fa773fda1e5f7a2d8c453._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 6"
+ date="2018-02-16T18:42:28Z"
+ content="""
+FWIW `--json-error-messages` would be great to have. 
+
+`The consumer would need to use select() over stdin+stderr to observe the order they were interleaved.` would not work for us atm without major refactoring (if possible at all without significant pains)
+"""]]

comment
diff --git a/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_5_429e42b9c1989dade01300f08fc3c13c._comment b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_5_429e42b9c1989dade01300f08fc3c13c._comment
new file mode 100644
index 000000000..a1d9edc00
--- /dev/null
+++ b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_5_429e42b9c1989dade01300f08fc3c13c._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 5"""
+ date="2018-02-16T17:59:43Z"
+ content="""
+I think I'm tending toward adding a new --json-error-messages option, and
+making it add "error-messages:[]" to the json when there are no errors,
+thus keeping the json self-documenting.
+
+The json buffer will need to be flushed in the exception handler,
+as noted in my previous comment.
+"""]]

update
diff --git a/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_4_c2e5041a07787ae84f85b898b1aae6b7._comment b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_4_c2e5041a07787ae84f85b898b1aae6b7._comment
index f4d171b01..8d0d2faa5 100644
--- a/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_4_c2e5041a07787ae84f85b898b1aae6b7._comment
+++ b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_4_c2e5041a07787ae84f85b898b1aae6b7._comment
@@ -15,5 +15,6 @@ would always precede the json for the same file.
 
 The consumer would need to use select() over stdin+stderr to observe
 the order they were interleaved. May be too much to expect consumers
-to get that right.
+to get that right. Oh, and still would not help untangle the stream
+when run with -J.
 """]]

comment
diff --git a/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_4_c2e5041a07787ae84f85b898b1aae6b7._comment b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_4_c2e5041a07787ae84f85b898b1aae6b7._comment
new file mode 100644
index 000000000..f4d171b01
--- /dev/null
+++ b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_4_c2e5041a07787ae84f85b898b1aae6b7._comment
@@ -0,0 +1,19 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2018-02-16T17:29:56Z"
+ content="""
+Currently when there's an exception in processing a file, the error is thrown
+and then caught and output to stderr, and this prevents the accumulated
+json buffer for the file from being output, since the processing never
+finishes. So, you get some error message on stderr, and no indication
+in the json what file it belongs to.
+
+So, perhaps an easier fix would be to emit the json buffer in this case
+after the error message. Then stderr output, whether error or warning,
+would always precede the json for the same file.
+
+The consumer would need to use select() over stdin+stderr to observe
+the order they were interleaved. May be too much to expect consumers
+to get that right.
+"""]]

comment
diff --git a/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_3_2d41d67dcb6e700b1ecda3106e444a5e._comment b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_3_2d41d67dcb6e700b1ecda3106e444a5e._comment
new file mode 100644
index 000000000..d3872b23e
--- /dev/null
+++ b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_3_2d41d67dcb6e700b1ecda3106e444a5e._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2018-02-16T16:50:27Z"
+ content="""
+A problem with this idea is that error messages are relatively uncommon,
+and if they're hidden away in an extra field of an existing json message,
+it would be easy for the consumer to forget to handle that uncommon case.
+
+Note that git-annex currently doesn't document the actual json structure it
+uses, which is more or less ok because any given git-annex subcommand will
+always output the same json structure (except some note fields that
+only sometimes appear, whose data is targeted at humans).
+It's self-documenting. Using json for errors breaks that pattern.
+"""]]

add wishlist item "creating importfeed does not work with socks proxy"
diff --git a/doc/bugs/importfeed_does_not_work_with_socks_proxy.mdwn b/doc/bugs/importfeed_does_not_work_with_socks_proxy.mdwn
new file mode 100644
index 000000000..cee34367e
--- /dev/null
+++ b/doc/bugs/importfeed_does_not_work_with_socks_proxy.mdwn
@@ -0,0 +1,5 @@
+It appears that `git annex importfeed` can not be used in with socks proxies because it uses wget unconditionally (at least in the Debian build).
+
+For `addurl`, I could configure the system to use a socks proxy by setting `git config annex.web-download-command "curl --silent --preproxy socks4a://localhost:1080 %url -o %file"`; for `importfeed`, I found no option to override the command used to fetch the URL, and `wget` [lacks SOCKS support](https://savannah.gnu.org/bugs/?func=detailitem&item_id=43576).
+
+Please consider using `web-download-command` for `importfeed` too, introducing a dedicated option `web-get-command` (that would output to stdout rather than %file), or otherwise supporting operation behind a SOCKS proxy.

Added a comment
diff --git a/doc/forum/offsite_repo/comment_2_d4ecd0dc29597e3e0988e06ee9c4d245._comment b/doc/forum/offsite_repo/comment_2_d4ecd0dc29597e3e0988e06ee9c4d245._comment
new file mode 100644
index 000000000..bcbb5ae36
--- /dev/null
+++ b/doc/forum/offsite_repo/comment_2_d4ecd0dc29597e3e0988e06ee9c4d245._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="ericm"
+ avatar="http://cdn.libravatar.org/avatar/67ca64c5a99fb142fc2e3916333881ca"
+ subject="comment 2"
+ date="2018-02-11T08:59:07Z"
+ content="""
+Ok I think that will do. I'll give it a try when I get some time. Thanks!
+"""]]

Added a comment
diff --git a/doc/bugs/get_-J_from_ssh_remote_tries_to_lock_in_home_directory__63__/comment_1_362a858ce78fa0b8d7949fb39fb247d9._comment b/doc/bugs/get_-J_from_ssh_remote_tries_to_lock_in_home_directory__63__/comment_1_362a858ce78fa0b8d7949fb39fb247d9._comment
new file mode 100644
index 000000000..27f8f3fef
--- /dev/null
+++ b/doc/bugs/get_-J_from_ssh_remote_tries_to_lock_in_home_directory__63__/comment_1_362a858ce78fa0b8d7949fb39fb247d9._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 1"
+ date="2018-02-10T02:48:14Z"
+ content="""
+BTW remote location is an NFS mount ... git annex didn't show any issue working with it on that host any time recently
+"""]]

diff --git a/doc/bugs/get_-J_from_ssh_remote_tries_to_lock_in_home_directory__63__.mdwn b/doc/bugs/get_-J_from_ssh_remote_tries_to_lock_in_home_directory__63__.mdwn
new file mode 100644
index 000000000..77b2b1ccf
--- /dev/null
+++ b/doc/bugs/get_-J_from_ssh_remote_tries_to_lock_in_home_directory__63__.mdwn
@@ -0,0 +1,49 @@
+### Please describe the problem.
+
+had `datalad get -J4` lockup ... interrupted it, ran git annex directly to observe some errors reported and what looks like to be an attempt to determine if the home directory on remote end is a git repo... see below for details of such a run.  
+
+may be unrelated but also experiencing some lock ups [while "interacting" with this remote  from datalad](https://github.com/datalad/datalad/issues/2128)
+
+### What version of git-annex are you using? On what operating system?
+6.20180206+gitg638032f3a-1~ndall+1 on the local
+6.20180115-g56b56033a on remote
+
+### Please provide any additional information below.
+
+[[!format sh """
+$> 'git' '-c' 'receive.autogc=0' '-c' 'gc.auto=0' 'annex' 'get' '--debug' '-c' 'remote.rolando.annex-ssh-options=-o ControlMaster=auto -S /home/yoh/.cache/datalad/sockets/810d3ac4' '--json' '--json-progress' -J4  --from rolando --key SHA256E-s4295588--8db39d197775c7372bb1afff197ea724f158f3217ad064eef7ff8427a1502f15.tgz    
+fatal: Not a git repository: '../../../home/bids/.git'
+git-annex-shell: Not a git-annex or gcrypt repository.
+  Unable to run git-annex-shell on remote .
+{"byte-progress":4295588,"action":{"command":"get","note":"from rolando...","key":"SHA256E-s4295588--8db39d197775c7372bb1afff197ea724f158f3217ad064eef7ff8427a1502f15.tgz","file":null},"total-size":4295588,"percent-progress":"100%"}
+{"byte-progress":4295588,"action":{"command":"get","note":"from rolando...","key":"SHA256E-s4295588--8db39d197775c7372bb1afff197ea724f158f3217ad064eef7ff8427a1502f15.tgz","file":null},"total-size":4295588,"percent-progress":"100%"}
+{"command":"get","note":"checksum...","success":true,"key":"SHA256E-s4295588--8db39d197775c7372bb1afff197ea724f1  Unable to run git-annex-shell on remote .
+
+$> git annex drop 'sourcedata/sub-qa/ses-20180102'
+drop sourcedata/sub-qa/ses-20180102/anat/sub-qa_ses-20180102_scout.dicom.tgz (locking rolando...) ok
+(recording state in git...)
+
+$> 'git' '-c' 'receive.autogc=0' '-c' 'gc.auto=0' 'annex' 'get' '--debug' '-c' 'remote.rolando.annex-ssh-options=-o ControlMaster=auto -S /home/yoh/.cache/datalad/sockets/810d3ac4' '--json' '--json-progress' --from rolando --key SHA256E-s4295588--8db39d197775c7372bb1afff197ea724f158f3217ad064eef7ff8427a1502f15.tgz     
+{"byte-progress":4295588,"action":{"command":"get","note":"from rolando...","key":"SHA256E-s4295588--8db39d197775c7372bb1afff197ea724f158f3217ad064eef7ff8427a1502f15.tgz","file":null},"total-size":4295588,"percent-progress":"100%"}
+{"byte-progress":4295588,"action":{"command":"get","note":"from rolando...","key":"SHA256E-s4295588--8db39d197775c7372bb1afff197ea724f158f3217ad064eef7ff8427a1502f15.tgz","file":null},"total-size":4295588,"percent-progress":"100%"}
+{"command":"get","note":"checksum...","success":true,"key":"SHA256E-s4295588--8db39d197775c7372bb1afff197ea724f158f3217ad064eef7ff8427a1502f15.tgz","file":null}
+
+$> cat .git/config
+ ...
+[remote "rolando"]
+	url = bids@rolando.cns.dartmouth.edu:/inbox/BIDS/dbic/QA
+	fetch = +refs/heads/*:refs/remotes/rolando/*
+	annex-uuid = 6384a551-a41d-4290-b186-9258befede97
+	annex-ignore = false
+
+"""]]
+
+To get to that host I have `ProxyCommand ssh -q -A smaug.dartmouth.edu 'nc -w1 %h %p'` in my ~/.ssh/config for it (if relevant)
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+Most of the days of the week. Friday is a tricky one
+
+[[!meta author=yoh]]
+
+

diff --git a/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn b/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn
index 02d8be70e..b0ea92820 100644
--- a/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn
+++ b/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn
@@ -9,7 +9,7 @@ On a Windows 10 machine with Creators update (v1709) and Developer mode enabled,
 Additionally, if a `git annex sync` is performed, the symlinks are deleted.
 
 ### What version of git-annex are you using? On what operating system?
-v6.20180112
+v6.20180112 on Windows 10 with update 1709.
 
 ### Please provide any additional information below.
 Since the aforementioned Windows update (commonly referred to as Creators Update), the mklink command can be executed without requiring elevated privileges if Developer mode is enabled.

diff --git a/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn b/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn
index 1999477e5..02d8be70e 100644
--- a/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn
+++ b/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn
@@ -4,7 +4,9 @@ Since the Win10 Creators Update, if Developer Mode is enabled, symlinks can be c
 When symlinks are created, git-annex breaks in many wonderful ways. The symlinks are not detected as annexed files and `git annex info` shows that no files are in the annex.
 
 ### What steps will reproduce the problem?
-On a Windows 10 machine with Creators update (v1709) and Developer mode enabled, do a `git clone` of a git-annex repository.
+On a Windows 10 machine with Creators update (v1709) and Developer mode enabled, do a `git clone` of a git-annex repository. Then `git annex init` and `git annex info`.
+
+Additionally, if a `git annex sync` is performed, the symlinks are deleted.
 
 ### What version of git-annex are you using? On what operating system?
 v6.20180112

diff --git a/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn b/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn
index fd8c73256..1999477e5 100644
--- a/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn
+++ b/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn
@@ -12,8 +12,10 @@ v6.20180112
 ### Please provide any additional information below.
 Since the aforementioned Windows update (commonly referred to as Creators Update), the mklink command can be executed without requiring elevated privileges if Developer mode is enabled.
 See the following two articles for some background:
-- https://www.ghacks.net/2016/12/04/windows-10-creators-update-symlinks-without-elevation/
-- https://www.howtogeek.com/292914/what-is-developer-mode-in-windows-10/ 
+
+- [Windows 10 Creators Update: Symlinks without elevation
+ [ghacks.net]](https://www.ghacks.net/2016/12/04/windows-10-creators-update-symlinks-without-elevation/)
+- [What Is “Developer Mode” in Windows 10? [howtogeek.com]](https://www.howtogeek.com/292914/what-is-developer-mode-in-windows-10/)
 
 It would seem that git detects this and does not disable symlinks as git-annex expects it to do on Windows.
 

diff --git a/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn b/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn
new file mode 100644
index 000000000..fd8c73256
--- /dev/null
+++ b/doc/bugs/Symlink_support_on_Windows_10_Creators_Update_with_Developer_Mode.mdwn
@@ -0,0 +1,22 @@
+### Please describe the problem.
+Since the Win10 Creators Update, if Developer Mode is enabled, symlinks can be created without elevated privileges. During cloning of a repository with symlinks, git detects this and creates symbolic links, whereas in the past it would simply create text-like files with a path.
+
+When symlinks are created, git-annex breaks in many wonderful ways. The symlinks are not detected as annexed files and `git annex info` shows that no files are in the annex.
+
+### What steps will reproduce the problem?
+On a Windows 10 machine with Creators update (v1709) and Developer mode enabled, do a `git clone` of a git-annex repository.
+
+### What version of git-annex are you using? On what operating system?
+v6.20180112
+
+### Please provide any additional information below.
+Since the aforementioned Windows update (commonly referred to as Creators Update), the mklink command can be executed without requiring elevated privileges if Developer mode is enabled.
+See the following two articles for some background:
+- https://www.ghacks.net/2016/12/04/windows-10-creators-update-symlinks-without-elevation/
+- https://www.howtogeek.com/292914/what-is-developer-mode-in-windows-10/ 
+
+It would seem that git detects this and does not disable symlinks as git-annex expects it to do on Windows.
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+Yes. I use it frequently on multiple platforms.
+

fsck: Warn when required content is not present in the repository that requires it.
This commit was sponsored by Jack Hill on Patreon.
diff --git a/CHANGELOG b/CHANGELOG
index dbf55e21f..abb5ed0d8 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -6,6 +6,8 @@ git-annex (6.20180113) UNRELEASED; urgency=medium
   * datalad < 0.9.1 had a problem in its special remote protocol handling
     which is broken by EXTENSIONS. Make the debian git-annex package
     conflict with the problem version of datalad.
+  * fsck: Warn when required content is not present in the repository that
+    requires it.
 
  -- Joey Hess <id@joeyh.name>  Wed, 24 Jan 2018 20:42:55 -0400
 
diff --git a/Command/Fsck.hs b/Command/Fsck.hs
index 2db6e279d..8ca5b1fd0 100644
--- a/Command/Fsck.hs
+++ b/Command/Fsck.hs
@@ -1,6 +1,6 @@
 {- git-annex command
  -
- - Copyright 2010-2017 Joey Hess <id@joeyh.name>
+ - Copyright 2010-2018 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -23,6 +23,7 @@ import Logs.Location
 import Logs.Trust
 import Logs.Activity
 import Logs.TimeStamp
+import Logs.PreferredContent
 import Annex.NumCopies
 import Annex.UUID
 import Annex.ReplaceFile
@@ -40,6 +41,8 @@ import Types.ActionItem
 
 import Data.Time.Clock.POSIX
 import System.Posix.Types (EpochTime)
+import qualified Data.Set as S
+import qualified Data.Map as M
 
 cmd :: Command
 cmd = withGlobalOptions (jobsOption : jsonOption : annexedMatchingOptions) $
@@ -121,6 +124,7 @@ perform key file backend numcopies = do
 		-- order matters
 		[ fixLink key file
 		, verifyLocationLog key keystatus ai
+		, verifyRequiredContent key ai
 		, verifyAssociatedFiles key keystatus file
 		, verifyWorkTree key file
 		, checkKeySize key keystatus ai
@@ -151,6 +155,7 @@ performRemote key afile backend numcopies remote =
 	dispatch (Right False) = go False Nothing
 	go present localcopy = check
 		[ verifyLocationLogRemote key ai remote present
+		, verifyRequiredContent key ai
 		, withLocalCopy localcopy $ checkKeySizeRemote key remote ai
 		, withLocalCopy localcopy $ checkBackendRemote backend key remote ai
 		, checkKeyNumCopies key afile numcopies
@@ -286,6 +291,27 @@ verifyLocationLog' key ai present u updatestatus = do
 		showNote "fixing location log"
 		updatestatus s
 
+{- Verifies that all repos that are required to contain the content do,
+ - checking against the location log. -}
+verifyRequiredContent :: Key -> ActionItem -> Annex Bool
+verifyRequiredContent key ai@(ActionItemAssociatedFile afile) = do
+	presentlocs <- S.fromList <$> loggedLocations key
+	requiredlocs <- S.fromList . M.keys <$> requiredContentMap
+	missinglocs <- filterM
+		(\u -> isRequiredContent (Just u) S.empty (Just key) afile False)
+		(S.toList $ S.difference requiredlocs presentlocs)
+	if null missinglocs
+		then return True
+		else do
+			missingrequired <- Remote.prettyPrintUUIDs "missingrequired" missinglocs
+			warning $
+				"** Required content " ++
+				actionItemDesc ai key ++
+				" is missing from these repositories:\n" ++
+				missingrequired
+			return False
+verifyRequiredContent _ _ = return True
+
 {- Verifies the associated file records. -}
 verifyAssociatedFiles :: Key -> KeyStatus -> FilePath -> Annex Bool
 verifyAssociatedFiles key keystatus file = do
diff --git a/doc/forum/offsite_repo/comment_1_1f3ab9a39baa16e3e307b23fd1aabeb8._comment b/doc/forum/offsite_repo/comment_1_1f3ab9a39baa16e3e307b23fd1aabeb8._comment
new file mode 100644
index 000000000..79f1810b3
--- /dev/null
+++ b/doc/forum/offsite_repo/comment_1_1f3ab9a39baa16e3e307b23fd1aabeb8._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-02-08T17:32:00Z"
+ content="""
+What you're looking for is probably the [[required_content]] settings.
+
+However, `git annex fsck` did not perform any checks that required content
+was present. Now it does. Enjoy!
+"""]]
diff --git a/doc/git-annex-required.mdwn b/doc/git-annex-required.mdwn
index 1c6dac232..367c1b3a6 100644
--- a/doc/git-annex-required.mdwn
+++ b/doc/git-annex-required.mdwn
@@ -22,7 +22,10 @@ While [[git-annex-wanted]] is just a preference,
 [[git-annex-required]] designates content that should really not be
 removed. For example a file that is `wanted` can be removed with 
 `git annex drop`, but if that file is `required`, it would need to be
-removed with `git annex drop --force`.
+removed with `git annex drop --force`. 
+
+Also, `git-annex fsck` will warn about required contents that are not
+present.
 
 # NOTES
 
diff --git a/doc/required_content.mdwn b/doc/required_content.mdwn
index e17951d9d..d4535ff85 100644
--- a/doc/required_content.mdwn
+++ b/doc/required_content.mdwn
@@ -16,3 +16,7 @@ by simply using `git annex drop`. On the other hand, required content
 settings are enforced; `git annex drop` will refuse to drop a file if
 doing so would violate its required content settings.
 (Although even this can be overridden using `--force`).
+
+Also, `git-annex fsck` will warn about required contents that are not
+present.
+

Added a comment
diff --git a/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_2_3a7043652c692796b1a438a2c4c26828._comment b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_2_3a7043652c692796b1a438a2c4c26828._comment
new file mode 100644
index 000000000..0dbd715bf
--- /dev/null
+++ b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_2_3a7043652c692796b1a438a2c4c26828._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 2"
+ date="2018-02-08T18:06:54Z"
+ content="""
+yes -- I could have been clearer that association to the file (if present, may be error is generic and still would be worth to communicate via a simple json message, alike recently added INFO passed from custom special remotes) would be valuable!
+As for stderr output -- would lead to duplication (we just changed to output at least a \"tip\" of stderr output which might be heavy).  If it is a matter of not breaking tools which would not support this new behavior, having a config or --option-hate-to-suggest-to-addone, would be  valuable so we could disable \"double-casting\" the error messages
+"""]]

rewrap to fix man page format
diff --git a/doc/git-annex-required.mdwn b/doc/git-annex-required.mdwn
index 81bfd956e..1c6dac232 100644
--- a/doc/git-annex-required.mdwn
+++ b/doc/git-annex-required.mdwn
@@ -20,8 +20,8 @@ of the repository.
 
 While [[git-annex-wanted]] is just a preference,
 [[git-annex-required]] designates content that should really not be
-removed. For example a file that is `wanted` can be removed with `git
-annex drop`, but if that file is `required`, it would need to be
+removed. For example a file that is `wanted` can be removed with 
+`git annex drop`, but if that file is `required`, it would need to be
 removed with `git annex drop --force`.
 
 # NOTES

comment
diff --git a/doc/todo/--get_option_for_diffdriver/comment_1_6b2501b2dc0318242e444e8d4b8b314f._comment b/doc/todo/--get_option_for_diffdriver/comment_1_6b2501b2dc0318242e444e8d4b8b314f._comment
new file mode 100644
index 000000000..afaa14e98
--- /dev/null
+++ b/doc/todo/--get_option_for_diffdriver/comment_1_6b2501b2dc0318242e444e8d4b8b314f._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-02-08T17:28:21Z"
+ content="""
+This is a good idea. I wonder what to do about the objects it downloads for
+a diff; should they be left in the annex for later use/dropunused, or
+immediately deleted after the diff completes. 
+
+My inclination is to keep
+them around, for one thing when I'm diffing stuff I often run diff more
+than once, perhaps to widen the diff or because I want to take a second
+look at it, and re-downloading a bunch of big files would be painful then.
+
+By the way, what diffdriver program are you using? I've had a hard time
+finding any real-life examples of git diffdrivers.
+"""]]

comment
diff --git a/doc/forum/Replace_glacier-cli__63__/comment_1_318cd0ca9f4e997161ed22fffa5d5da3._comment b/doc/forum/Replace_glacier-cli__63__/comment_1_318cd0ca9f4e997161ed22fffa5d5da3._comment
new file mode 100644
index 000000000..f87c8e4f6
--- /dev/null
+++ b/doc/forum/Replace_glacier-cli__63__/comment_1_318cd0ca9f4e997161ed22fffa5d5da3._comment
@@ -0,0 +1,18 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-02-08T17:17:10Z"
+ content="""
+There's been an open feature request on the haskell aws
+for 5 years to support glacier <https://github.com/aristidb/aws/issues/81>
+at this point I am doubtful anything will ever happen there.
+
+The most likely approach seems to be to use the amazonka library, which
+supports S3 with a Glacier lifecycle. git-annex's S3 remote could be
+rewritten to use that.
+
+I think it should be possible to set up a S3 bucket and configure it with
+some web tool to have a Glacier lifecycle, and then use the existing
+git-annex S3 support to access it. Except for the problem documented at
+[[todo/wishlist__58___Restore_s3_files_moved_to_Glacier]].
+"""]]

response
diff --git a/doc/bugs/no_git-annex_shell_on_Windows/comment_12_9d445ddf84e15a59cdc2aa2b8dc30c44._comment b/doc/bugs/no_git-annex_shell_on_Windows/comment_12_9d445ddf84e15a59cdc2aa2b8dc30c44._comment
new file mode 100644
index 000000000..cebd4b950
--- /dev/null
+++ b/doc/bugs/no_git-annex_shell_on_Windows/comment_12_9d445ddf84e15a59cdc2aa2b8dc30c44._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 12"""
+ date="2018-02-08T17:13:41Z"
+ content="""
+That error message seems to indicate that git-annex does not realize it's
+being run with the git-annex-shell program name. When I run "git-annex -c
+configlist /path/to/repo" I get that error message; when I run
+git-annex-shell with the same parameters, I get the expected repository
+config output.
+
+I'm guessing that the windows symlink thing you tried to use confuses the
+program path name lookup that it tries to do. I'm pretty sure that a copy
+of the file would work.
+"""]]

comment
diff --git a/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_1_f02e43b68c219b2d7d65ea868fac125f._comment b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_1_f02e43b68c219b2d7d65ea868fac125f._comment
new file mode 100644
index 000000000..eeaafa9ed
--- /dev/null
+++ b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output/comment_1_f02e43b68c219b2d7d65ea868fac125f._comment
@@ -0,0 +1,23 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-02-08T16:59:16Z"
+ content="""
+Is the benefit of doing this being able to correlate the error message with
+the filename whose processing caused the error?
+
+For that benefit to be realized, the error message would have to be included
+in the same json object with the file being processed. If a separate json
+object were emitted containing only the error message it would not be clear
+what file it was related to (especially when git-annex is running
+concurrent jobs), and so that does not seem any better than
+output to stderr.
+
+Looks like implementation would just involve making outputError 
+check if there is a jsonBuffer, and add the error message to it,
+probably using an array since there could be multiple warning messages.
+
+I kind of feel like they should still go to stderr in addition to any json
+output, because any existing json consumers may rely on the current stderr
+behavior to let the user know what went wrong.
+"""]]

devblog
diff --git a/doc/devblog/day_484__special_remote_protocol_extensions.mdwn b/doc/devblog/day_484__special_remote_protocol_extensions.mdwn
new file mode 100644
index 000000000..68140c6fe
--- /dev/null
+++ b/doc/devblog/day_484__special_remote_protocol_extensions.mdwn
@@ -0,0 +1,12 @@
+The 
+[[external special remote protocol|design/external_special_remote_protocol]]
+had extensibility built into it for messages git-annex sends, but not
+for messages that the remote sends back to git-annex. To fix this
+asymmetry, I've added a new EXTENSIONS to the protocol, which can be used
+to find out about what new protocol extensions are supported.
+
+There was the possibility that adding that might break some external
+special remote that hardcoded the intial protocol messages. So, I checked
+all of them that I know of, and all were ok, except for older versions of
+datalad, which we were able to deal with. If you have your own external
+special remote implementation, now would be a good time to check it.

Added a comment
diff --git a/doc/design/exporting_trees_to_special_remotes/comment_21_062098e1f54b874467793e4487a45a9b._comment b/doc/design/exporting_trees_to_special_remotes/comment_21_062098e1f54b874467793e4487a45a9b._comment
new file mode 100644
index 000000000..dfb5cb456
--- /dev/null
+++ b/doc/design/exporting_trees_to_special_remotes/comment_21_062098e1f54b874467793e4487a45a9b._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 21"
+ date="2018-02-07T20:01:53Z"
+ content="""
+Ok, gotcha.  Shouldn't then EXTENSION entries also be somehow versioned per each one of them? or if needed a new extension would be born by appending a version to its name (e.g. as with all those imap, imap2, imap3, ... ;-))
+"""]]

comment
diff --git a/doc/design/exporting_trees_to_special_remotes/comment_19_1eda1af40f6ca82a8bacd19afaa749bc._comment b/doc/design/exporting_trees_to_special_remotes/comment_19_1eda1af40f6ca82a8bacd19afaa749bc._comment
new file mode 100644
index 000000000..c67370a07
--- /dev/null
+++ b/doc/design/exporting_trees_to_special_remotes/comment_19_1eda1af40f6ca82a8bacd19afaa749bc._comment
@@ -0,0 +1,18 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 19"""
+ date="2018-02-07T19:04:03Z"
+ content="""
+What if the remote wants to use some feature like NOTE, but can still
+manage to work when an old git-annex does not support it? Hard bumping the
+VERSION cannot support that. If the remote requires to be able to use NOTE
+and sees it cannot, it can still throw an error.
+
+There are a bunch of requests in the protocol that are optional for the
+remote to support; git-annex deals with remotes that don't support them in
+better ways than throwing up its hands because the special remote is too
+old. It's very good that the protocol allowed adding those extensions
+without bumping a version. The protocol is less extensible when it comes
+replies and other messages sent by the special remote, and I want to get
+the same extensibility for those.
+"""]]

Added EXTENSIONS to external special remote protocol.
Allows using new special remote messages when git-annex supports them,
and avoiding using them when git-annex is too old. The new INFO is one
such message.
There's also the possibility, currently unused, for the special remote's
reply to include some kind of extensions of its own.
Merging this is blocked by https://github.com/datalad/datalad/issues/2124
since it seems it will break datalad. I checked all the other special
remotes and they will be ok.
This commit was supported by the NSF-funded DataLad project.
diff --git a/CHANGELOG b/CHANGELOG
index d2886b745..6ed90dfd7 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -2,6 +2,7 @@ git-annex (6.20180113) UNRELEASED; urgency=medium
 
   * inprogress: Avoid showing failures for files not in progress. 
   * Added INFO to external special remote protocol.
+  * Added EXTENSIONS to external special remote protocol.
 
  -- Joey Hess <id@joeyh.name>  Wed, 24 Jan 2018 20:42:55 -0400
 
diff --git a/Remote/External.hs b/Remote/External.hs
index 5a1e7f210..bff74c3b1 100644
--- a/Remote/External.hs
+++ b/Remote/External.hs
@@ -505,7 +505,8 @@ withExternalState external = bracket alloc dealloc
 	
 	dealloc st = liftIO $ atomically $ modifyTVar' v (st:)
 
-{- Starts an external remote process running, and checks VERSION. -}
+{- Starts an external remote process running, and checks VERSION and
+ - exchanges EXTENSIONS. -}
 startExternal :: External -> Annex ExternalState
 startExternal external = do
 	errrelayer <- mkStderrRelayer
@@ -514,6 +515,18 @@ startExternal external = do
 		(const Nothing)
 		(checkVersion st external)
 		(const Nothing)
+	sendMessage st external (EXTENSIONS supportedExtensionList)
+	-- It responds with a EXTENSIONS_RESPONSE; that extensions list
+	-- is reserved for future expansion. UNSUPPORTED_REQUEST is also
+	-- accepted.
+	receiveMessage st external
+		(\resp -> case resp of
+			EXTENSIONS_RESPONSE _ -> Just (return ())
+			UNSUPPORTED_REQUEST -> Just (return ())
+			_ -> Nothing
+		)
+		(const Nothing)
+		(const Nothing)
 	return st
   where
 	start errrelayer g = liftIO $ do
diff --git a/Remote/External/Types.hs b/Remote/External/Types.hs
index 9e511e450..3b66027c6 100644
--- a/Remote/External/Types.hs
+++ b/Remote/External/Types.hs
@@ -1,6 +1,6 @@
 {- External special remote data types.
  -
- - Copyright 2013 Joey Hess <id@joeyh.name>
+ - Copyright 2013-2018 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -14,6 +14,7 @@ module Remote.External.Types (
 	ExternalType,
 	ExternalState(..),
 	PrepareStatus(..),
+	supportedExtensionList,
 	Proto.parseMessage,
 	Proto.Sendable(..),
 	Proto.Receivable(..),
@@ -80,6 +81,14 @@ data ExternalState = ExternalState
 
 type PID = Int
 
+-- List of extensions to the protocol.
+newtype ExtensionList = ExtensionList [String]
+	deriving (Show)
+
+-- When adding a new RemoteRequest, also add it to the list here.
+supportedExtensionList :: ExtensionList
+supportedExtensionList = ExtensionList ["INFO"]
+
 data PrepareStatus = Unprepared | Prepared | FailedPrepare ErrorMsg
 
 -- The protocol does not support keys with spaces in their names;
@@ -107,7 +116,8 @@ instance Proto.Serializable SafeKey where
 
 -- Messages that can be sent to the external remote to request it do something.
 data Request 
-	= PREPARE 
+	= EXTENSIONS ExtensionList
+	| PREPARE
 	| INITREMOTE
 	| GETCOST
 	| GETAVAILABILITY
@@ -129,11 +139,13 @@ data Request
 -- Does PREPARE need to have been sent before this request?
 needsPREPARE :: Request -> Bool
 needsPREPARE PREPARE = False
+needsPREPARE (EXTENSIONS _) = False
 needsPREPARE INITREMOTE = False
 needsPREPARE EXPORTSUPPORTED = False
 needsPREPARE _ = True
 
 instance Proto.Sendable Request where
+	formatMessage (EXTENSIONS l) = ["EXTENSIONS", Proto.serialize l]
 	formatMessage PREPARE = ["PREPARE"]
 	formatMessage INITREMOTE = ["INITREMOTE"]
 	formatMessage GETCOST = ["GETCOST"]
@@ -172,7 +184,8 @@ instance Proto.Sendable Request where
 
 -- Responses the external remote can make to requests.
 data Response
-	= PREPARE_SUCCESS
+	= EXTENSIONS_RESPONSE ExtensionList
+	| PREPARE_SUCCESS
 	| PREPARE_FAILURE ErrorMsg
 	| TRANSFER_SUCCESS Direction Key
 	| TRANSFER_FAILURE Direction Key ErrorMsg
@@ -202,6 +215,7 @@ data Response
 	deriving (Show)
 
 instance Proto.Receivable Response where
+	parseCommand "EXTENSIONS" = Proto.parse1 EXTENSIONS_RESPONSE
 	parseCommand "PREPARE-SUCCESS" = Proto.parse0 PREPARE_SUCCESS
 	parseCommand "PREPARE-FAILURE" = Proto.parse1 PREPARE_FAILURE
 	parseCommand "TRANSFER-SUCCESS" = Proto.parse2 TRANSFER_SUCCESS
@@ -366,3 +380,7 @@ instance Proto.Serializable ExportLocation where
 instance Proto.Serializable ExportDirectory where
 	serialize = fromExportDirectory
 	deserialize = Just . mkExportDirectory
+
+instance Proto.Serializable ExtensionList where
+	serialize (ExtensionList l) = unwords l
+	deserialize = Just . ExtensionList . words
diff --git a/doc/design/external_special_remote_protocol.mdwn b/doc/design/external_special_remote_protocol.mdwn
index d6808274e..c6b852ed4 100644
--- a/doc/design/external_special_remote_protocol.mdwn
+++ b/doc/design/external_special_remote_protocol.mdwn
@@ -41,9 +41,20 @@ the version of the protocol it is using.
 
 	VERSION 1
 
-Once it knows the version, git-annex will generally 
-send a message telling the special remote to start up.
-(Or it might send an INITREMOTE or EXPORTSUPPORTED,
+Recent versions of git-annex respond with a message indicating
+protocol extensions that it supports. Older versions of
+git-annex do not send this message. 
+
+	EXTENSIONS INFO
+
+The special remote can respond to that with its own EXTENSIONS message, which
+could have its own protocol extension details, but none are currently used.
+(It's also fine to reply with UNSUPPORTED-REQUEST.)
+
+	EXTENSIONS 
+
+Next, git-annex will generally send a message telling the special
+remote to start up. (Or it might send an INITREMOTE or EXPORTSUPPORTED,
 so don't hardcode this order.)
 
 	PREPARE
@@ -103,7 +114,7 @@ The following requests *must* all be supported by the special remote.
   So any one-time setup tasks should be done idempotently.
 * `PREPARE`  
   Tells the remote that it's time to prepare itself to be used.  
-  Only INITREMOTE or EXPORTSUPPORTED can come before this.
+  Only EXTENSIONS and INITREMOTE or EXPORTSUPPORTED can come before this.
 * `TRANSFER STORE|RETRIEVE Key File`  
   Requests the transfer of a key. For STORE, the File is the file to upload;
   for RETRIEVE the File is where to store the download.  
@@ -119,6 +130,10 @@ The following requests *must* all be supported by the special remote.
 The following requests can optionally be supported. If not handled,
 replying with `UNSUPPORTED-REQUEST` is acceptable.
 
+* `EXTENSIONS List`  
+  Sent to indicate protocol extensions which git-annex is capable
+  of using. The list is a space-delimited list of protocol extension
+  keywords. The remote can reply to this with its own EXTENSIONS list.
 * `GETCOST`  
   Requests the remote to return a use cost. Higher costs are more expensive.
   (See Config/Cost.hs for some standard costs.)
@@ -229,6 +244,10 @@ while it's handling a request.
   the remote didn't have the key at the point removal was requested.
 * `REMOVE-FAILURE Key ErrorMsg`  
   Indicates that the key was unable to be removed from the remote.
+* `EXTENSIONS List`  
+  Sent in response to a EXTENSIONS request, the List could be used to indicate
+  protocol extensions that the special remote uses, but there are currently
+  no such extensions, so the List is empty.
 * `COST Int`  
   Indicates the cost of the remote.
 * `AVAILABILITY GLOBAL|LOCAL`
@@ -402,13 +421,15 @@ handling a request.
 * `DEBUG message`
   Tells git-annex to display the message if --debug is enabled.  
   (git-annex does not send a reply to this message.)
+
+These messages are protocol extensions; it's only safe to send them to
+git-annex after it sent a EXTENSIONS that included the name of the message.
+
 * `INFO message`
   Tells git-annex to display the message to the user.  
   When git-annex is in --json mode, the message will be emitted immediately

(Diff truncated)
Added a comment
diff --git a/doc/design/exporting_trees_to_special_remotes/comment_19_00d28c758509939974e4583e9b1b9e12._comment b/doc/design/exporting_trees_to_special_remotes/comment_19_00d28c758509939974e4583e9b1b9e12._comment
new file mode 100644
index 000000000..5e1835eb3
--- /dev/null
+++ b/doc/design/exporting_trees_to_special_remotes/comment_19_00d28c758509939974e4583e9b1b9e12._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 19"
+ date="2018-02-07T18:31:45Z"
+ content="""
+> Changing VERSION would prevent any older versions of git-annex from working with that external special remote, since they would reject the unknown version. (The current parsing of VERSION also happens to preclude adding some fields after the number.)
+
+I still do not get it, sorry -- If there is an older git-annex, and a special remote requests some higher VERSION (thus stating that it needs some features older git-annex does not support), IMHO it would be perfectly fine to fail to use that remote since it wouldn't be usable anyways with that older git-annex (i.e. require some special features it does not provide).  If special remote does not need any feature not present in version `1`, it (like all of them ATM) could still keep requesting `VERSION 1` thus staying compatible with whatever old git-annex is out there.
+"""]]

response
diff --git a/doc/design/exporting_trees_to_special_remotes/comment_18_fe77370699b7ce0acd547fd1e045e254._comment b/doc/design/exporting_trees_to_special_remotes/comment_18_fe77370699b7ce0acd547fd1e045e254._comment
new file mode 100644
index 000000000..bebfee6ed
--- /dev/null
+++ b/doc/design/exporting_trees_to_special_remotes/comment_18_fe77370699b7ce0acd547fd1e045e254._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 18"""
+ date="2018-02-07T16:43:02Z"
+ content="""
+Changing VERSION would prevent any older versions of git-annex from working
+with that external special remote, since they would reject the unknown
+version. (The current parsing of VERSION also happens to preclude adding
+some fields after the number.)
+
+Since it seems completely possible to make the protocol be changed in a way
+that is backwards compatible both ways, while still letting new features to
+be used, I'd rather reserve changing VERSION for whatever future thing
+needs a full breaking bump.
+"""]]

diff --git a/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output.mdwn b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output.mdwn
new file mode 100644
index 000000000..4bf193598
--- /dev/null
+++ b/doc/todo/include_msg_with_possible_reason_why_command___40__e.g._add__41___failed_into_--json_output.mdwn
@@ -0,0 +1,18 @@
+ATM errors are output into stderr while json record lacks any hints on what went wrong
+
+[[!format sh """
+$> git annex add --json longfilenametogetlotsoferrorsandevenlonger4 
+  longfilenametogetlotsoferrorsandevenlonger4: setFileMode: permission denied (Operation not permitted)
+{"command":"add","success":false,"file":"longfilenametogetlotsoferrorsandevenlonger4"}
+"""]]
+
+would be nice(r) to have
+
+[[!format sh """
+$> git annex add --json longfilenametogetlotsoferrorsandevenlonger4 
+{"command":"add","success":false,"file":"longfilenametogetlotsoferrorsandevenlonger4", "msg": "setFileMode: permission denied (Operation not permitted)"}
+"""]]
+or may be even an explicit "errormsg" or alike instead of just a generic "msg"
+
+
+[[!meta author=yoh]]

Added a comment
diff --git a/doc/todo/INFO_message_for_custom_special_remotes/comment_2_69535d7d9b7266eaac6f9bb70d2660dc._comment b/doc/todo/INFO_message_for_custom_special_remotes/comment_2_69535d7d9b7266eaac6f9bb70d2660dc._comment
new file mode 100644
index 000000000..5714e7836
--- /dev/null
+++ b/doc/todo/INFO_message_for_custom_special_remotes/comment_2_69535d7d9b7266eaac6f9bb70d2660dc._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 2"
+ date="2018-02-06T20:15:35Z"
+ content="""
+Great!  I will give it a shot.  No problem checking version of git-annex available so I will make it conditional on git-annex version.
+
+As for \"todo on protocol...\" -- FWIW I left some comments [close to original discussion](http://git-annex.branchable.com/design/exporting_trees_to_special_remotes/#comment-b3876cc7b791c6b8cb12949bdbbf8268)
+"""]]

Added a comment: protocol message
diff --git a/doc/design/exporting_trees_to_special_remotes/comment_17_32a3240206c5ffff71a47dffa6950c48._comment b/doc/design/exporting_trees_to_special_remotes/comment_17_32a3240206c5ffff71a47dffa6950c48._comment
new file mode 100644
index 000000000..2afef92a8
--- /dev/null
+++ b/doc/design/exporting_trees_to_special_remotes/comment_17_32a3240206c5ffff71a47dffa6950c48._comment
@@ -0,0 +1,23 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="protocol message "
+ date="2018-02-06T20:03:27Z"
+ content="""
+joey wrote:
+
+    The protocol VERSION is picked by the special remote, it's not negotiated.
+
+`VERSION ` is provided to git-annex by the special remote to git-annex process.  There is no need to 'negotiate' anything - you could make git-annex understand either of:
+
+- higher `VERSION `, e.g. 
+
+  - `VERSION 2` which would support some new features which that special remote would need.  If parent git-annex is old/doesn't support that version - would fail and demand git annex upgrade
+  - `VERSION 6.20171124` (where `6.20171124` is an example of git-annex version) so if git-annex parent process is older than that it could provide a meaningful message that `git annex >= 6.20171124` is needed
+
+- `VERSION 1 feature1 feature2 ...` where those features could be the ones needed (e.g. `INFO_MSG` for [recent addition](http://git-annex.branchable.com/todo/INFO_message_for_custom_special_remotes/#comment-4dcfb7d4e6db9d5ba8a1bfeb782346b1)). And if parent git-annex doesn't know/support any particular feature, it could fail and inform user that a new annex with that feature support is needed.
+
+In either of those cases the custom special remotes page could outline added features/versions of git-annex supporting them, so may be even those above error messages could point to it.
+
+Overall, it is just a minor change to be done on git-annex side while allowing for clear(er) specification, and I do not see any need for actual \"negotiation\" -- features are either supported or not by the parent process.
+"""]]

reviewed all external special remotes for change feasability
diff --git a/doc/design/external_special_remote_protocol.mdwn b/doc/design/external_special_remote_protocol.mdwn
index e31ee9e9b..d6808274e 100644
--- a/doc/design/external_special_remote_protocol.mdwn
+++ b/doc/design/external_special_remote_protocol.mdwn
@@ -490,4 +490,7 @@ It works like this:
   PREPARE or INITREMOTE after VERSION, so this change would break them.
   I mean, they shouldn't.. But a quickly/badly written one might.
   Probably want to review all the linked external special remote programs
-  before doing this.
+  before doing this. Update: Reviewed them all, all are ok.
+  However, datalad's  datalad's customremotes/base.py reacts to an unknown
+  request by calling self.error and so seems it would crash if git-annex
+  sent PROTOCOLKEYWORDS..

improve
diff --git a/doc/git-annex-diffdriver.mdwn b/doc/git-annex-diffdriver.mdwn
index 72e0faca3..3df61154e 100644
--- a/doc/git-annex-diffdriver.mdwn
+++ b/doc/git-annex-diffdriver.mdwn
@@ -9,14 +9,19 @@ git annex diffdriver `-- cmd --opts --`
 # DESCRIPTION
 
 This is an external git diff driver shim. Normally, when using `git diff`
-with an external git driver, the symlinks to annexed files are not set up
-right, so the external git driver cannot read them in order to perform
+with an external diff driver, the symlinks to annexed files are not set up
+right, so the external diff driver cannot read them in order to perform
 smart diffing of their contents. This command works around the problem,
 by passing the fixed up files to the real external diff driver.
 
-To use, just configure git to use "git-annex diffdriver -- cmd params --"
-as the external diff command, where cmd is the real external diff
-command you want to use, and params are any extra parameters to pass
+To use this, you will need to have installed some git external diff driver
+command. This is not the regular diff command; it takes a git-specific
+input. See git's documentation of `GIT_EXTERNAL_DIFF` and
+gitattributes(5)'s documentation of external diff drivers.
+
+Configure git to use "git-annex diffdriver -- cmd params --"
+as the external diff driver, where cmd is the external diff
+driver you want it to run, and params are any extra parameters to pass
 to it. Note the trailing "--", which is required.
 
 For example, set `GIT_EXTERNAL_DIFF=git-annex diffdriver -- j-c-diff --`

response
diff --git a/doc/bugs/no_git-annex_shell_on_Windows/comment_11_b212c8b8cf361745f35a9f7ff6fee1f5._comment b/doc/bugs/no_git-annex_shell_on_Windows/comment_11_b212c8b8cf361745f35a9f7ff6fee1f5._comment
new file mode 100644
index 000000000..c052520ab
--- /dev/null
+++ b/doc/bugs/no_git-annex_shell_on_Windows/comment_11_b212c8b8cf361745f35a9f7ff6fee1f5._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 11"""
+ date="2018-02-06T17:04:51Z"
+ content="""
+If you copy git-annex.exe to git-annex-shell.exe, it should work.
+
+The installer doesn't do it, because doubling the installation size footprint
+for a feature that involves sshing into a Windows machine (not very common
+I suspect) didn't seem worth it.
+"""]]

Added INFO to external special remote protocol.
It's left up to the special remote to detect when git-annex is new enough
to support the message; an old git-annex will blow up.
This commit was supported by the NSF-funded DataLad project.
diff --git a/CHANGELOG b/CHANGELOG
index 2f60f5ecd..d2886b745 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,6 +1,7 @@
 git-annex (6.20180113) UNRELEASED; urgency=medium
 
   * inprogress: Avoid showing failures for files not in progress. 
+  * Added INFO to external special remote protocol.
 
  -- Joey Hess <id@joeyh.name>  Wed, 24 Jan 2018 20:42:55 -0400
 
diff --git a/Messages.hs b/Messages.hs
index 08a7bb719..d5dee72e2 100644
--- a/Messages.hs
+++ b/Messages.hs
@@ -19,6 +19,7 @@ module Messages (
 	showStoringStateAction,
 	showOutput,
 	showLongNote,
+	showInfo,
 	showEndOk,
 	showEndFail,
 	showEndResult,
@@ -123,7 +124,15 @@ showOutput = unlessM commandProgressDisabled $
 	outputMessage JSON.none "\n"
 
 showLongNote :: String -> Annex ()
-showLongNote s = outputMessage (JSON.note s) ('\n' : indent s ++ "\n")
+showLongNote s = outputMessage (JSON.note s) (formatLongNote s)
+
+formatLongNote :: String -> String
+formatLongNote s = '\n' : indent s ++ "\n"
+
+-- Used by external special remote, displayed same as showLongNote
+-- to console, but json object containing the info is emitted immediately.
+showInfo :: String -> Annex ()
+showInfo s = outputMessage' outputJSON (JSON.info s) (formatLongNote s)
 
 showEndOk :: Annex ()
 showEndOk = showEndResult True
@@ -165,11 +174,11 @@ indent = intercalate "\n" . map (\l -> "  " ++ l) . lines
 
 {- Shows a JSON chunk only when in json mode. -}
 maybeShowJSON :: JSON.JSONChunk v -> Annex ()
-maybeShowJSON v = void $ withMessageState $ outputJSON (JSON.add v)
+maybeShowJSON v = void $ withMessageState $ bufferJSON (JSON.add v)
 
 {- Shows a complete JSON value, only when in json mode. -}
 showFullJSON :: JSON.JSONChunk v -> Annex Bool
-showFullJSON v = withMessageState $ outputJSON (JSON.complete v)
+showFullJSON v = withMessageState $ bufferJSON (JSON.complete v)
 
 {- Performs an action that outputs nonstandard/customized output, and
  - in JSON mode wraps its output in JSON.start and JSON.end, so it's
diff --git a/Messages/Internal.hs b/Messages/Internal.hs
index 6ec72812a..3972503dc 100644
--- a/Messages/Internal.hs
+++ b/Messages/Internal.hs
@@ -17,16 +17,19 @@ withMessageState :: (MessageState -> Annex a) -> Annex a
 withMessageState a = Annex.getState Annex.output >>= a
 
 outputMessage :: JSONBuilder -> String -> Annex ()
-outputMessage jsonbuilder msg = withMessageState $ \s -> case outputType s of
+outputMessage = outputMessage' bufferJSON
+
+outputMessage' :: (JSONBuilder -> MessageState -> Annex Bool) -> JSONBuilder -> String -> Annex ()
+outputMessage' jsonoutputter jsonbuilder msg = withMessageState $ \s -> case outputType s of
 	NormalOutput
 		| concurrentOutputEnabled s -> concurrentMessage s False msg q
 		| otherwise -> liftIO $ flushed $ putStr msg
-	JSONOutput _ -> void $ outputJSON jsonbuilder s
+	JSONOutput _ -> void $ jsonoutputter jsonbuilder s
 	QuietOutput -> q
 
 -- Buffer changes to JSON until end is reached and then emit it.
-outputJSON :: JSONBuilder -> MessageState -> Annex Bool
-outputJSON jsonbuilder s = case outputType s of
+bufferJSON :: JSONBuilder -> MessageState -> Annex Bool
+bufferJSON jsonbuilder s = case outputType s of
 	JSONOutput _
 		| endjson -> do
 			Annex.changeState $ \st -> 
@@ -46,6 +49,15 @@ outputJSON jsonbuilder s = case outputType s of
 		Nothing -> Nothing
 		Just b -> Just (b, False)
 
+-- Immediately output JSON.
+outputJSON :: JSONBuilder -> MessageState -> Annex Bool
+outputJSON jsonbuilder s = case outputType s of
+	JSONOutput _ -> do
+		maybe noop (liftIO . flushed . emit)
+			(fst <$> jsonbuilder Nothing)
+		return True
+	_ -> return False
+
 outputError :: String -> Annex ()
 outputError msg = withMessageState $ \s ->
 	if concurrentOutputEnabled s
diff --git a/Messages/JSON.hs b/Messages/JSON.hs
index 3ad0e5708..6ca3c1383 100644
--- a/Messages/JSON.hs
+++ b/Messages/JSON.hs
@@ -15,6 +15,7 @@ module Messages.JSON (
 	start,
 	end,
 	note,
+	info,
 	add,
 	complete,
 	progress,
@@ -77,6 +78,11 @@ note :: String -> JSONBuilder
 note s (Just (o, e)) = Just (HM.insert "note" (toJSON s) o, e)
 note _ Nothing = Nothing
 
+info :: String -> JSONBuilder
+info s _ = Just (o, True)
+  where
+	Object o = object ["info" .= toJSON s]
+
 data JSONChunk v where
 	AesonObject :: Object -> JSONChunk Object
 	JSONChunk :: ToJSON v => [(String, v)] -> JSONChunk [(String, v)]
diff --git a/Remote/External.hs b/Remote/External.hs
index 4220c47d7..5a1e7f210 100644
--- a/Remote/External.hs
+++ b/Remote/External.hs
@@ -421,6 +421,7 @@ handleRequest' st external req mp responsehandler
 		mapM_ (send . VALUE) =<< getUrlsWithPrefix key prefix
 		send (VALUE "") -- end of list
 	handleRemoteRequest (DEBUG msg) = liftIO $ debugM "external" msg
+	handleRemoteRequest (INFO msg) = showInfo msg
 	handleRemoteRequest (VERSION _) =
 		sendMessage st external (ERROR "too late to send VERSION")
 
diff --git a/Remote/External/Types.hs b/Remote/External/Types.hs
index 77f3e837e..9e511e450 100644
--- a/Remote/External/Types.hs
+++ b/Remote/External/Types.hs
@@ -253,6 +253,7 @@ data RemoteRequest
 	| SETURIMISSING Key URI
 	| GETURLS Key String
 	| DEBUG String
+	| INFO String
 	deriving (Show)
 
 instance Proto.Receivable RemoteRequest where
@@ -276,6 +277,7 @@ instance Proto.Receivable RemoteRequest where
 	parseCommand "SETURIMISSING" = Proto.parse2 SETURIMISSING
 	parseCommand "GETURLS" = Proto.parse2 GETURLS
 	parseCommand "DEBUG" = Proto.parse1 DEBUG
+	parseCommand "INFO" = Proto.parse1 INFO
 	parseCommand _ = Proto.parseFail
 
 -- Responses to RemoteRequest.
diff --git a/doc/design/external_special_remote_protocol.mdwn b/doc/design/external_special_remote_protocol.mdwn
index 401c42d6c..e31ee9e9b 100644
--- a/doc/design/external_special_remote_protocol.mdwn
+++ b/doc/design/external_special_remote_protocol.mdwn
@@ -400,8 +400,15 @@ handling a request.
   (git-annex replies one or more times with VALUE for each url.
   The final VALUE has an empty value, indicating the end of the url list.)
 * `DEBUG message`
-  Tells git-annex to display the message if --debug is enabled.
+  Tells git-annex to display the message if --debug is enabled.  
   (git-annex does not send a reply to this message.)
+* `INFO message`
+  Tells git-annex to display the message to the user.  
+  When git-annex is in --json mode, the message will be emitted immediately
+  in its own json object, with an "info" field.  
+  (git-annex does not send a reply to this message.)  
+  This message was first supported by git-annex version 
+  6.20180206
 
 ## general messages
 
diff --git a/doc/todo/INFO_message_for_custom_special_remotes.mdwn b/doc/todo/INFO_message_for_custom_special_remotes.mdwn
index b41ea89bc..b2ed38258 100644
--- a/doc/todo/INFO_message_for_custom_special_remotes.mdwn
+++ b/doc/todo/INFO_message_for_custom_special_remotes.mdwn
@@ -1,3 +1,5 @@
 I wondered if it would be sensible to ask to extend [externals special remote protocol](https://git-annex.branchable.com/design/external_special_remote_protocol/) with ability for custom remotes to pass back some INFO level message (not only DEBUG or ERROR).  The reason is:  in datalad-archives special remote we usually need to `git annex get` first the key containing the archive, which might be sizeable.  Since there is ATM no way to communicate back to git-annex, so it could communicate back to the datalad which runs it, it results in no output/message to the user that possibly a heavy download is happening in the background.  So, we would need to establish our own communication from datalad-archives special remote all the way to top level datalad process to report that, or I wondered if may be we could report back to git-annex, and it in turn report back to the original process (running e.g. `annex get --json --json-progress`) so it could spit out that message wrapped into a json record within the stream, so we could process and output that to the user.
 
 [[!meta author=yoh]]
+
+> [[done]] --[[Joey]]
diff --git a/doc/todo/INFO_message_for_custom_special_remotes/comment_1_ea99a5099f78767859c05aeb5217a12d._comment b/doc/todo/INFO_message_for_custom_special_remotes/comment_1_ea99a5099f78767859c05aeb5217a12d._comment
new file mode 100644
index 000000000..c2e9f318e
--- /dev/null
+++ b/doc/todo/INFO_message_for_custom_special_remotes/comment_1_ea99a5099f78767859c05aeb5217a12d._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-02-06T16:58:04Z"
+ content="""
+I've added it. However, note that previous versions of git-annex will
+not react well to an unknown message being sent, so to use it safely you
+will need to detect a new enough version of git-annex. (I've had a todo item

(Diff truncated)
Added a comment: Did' nyone make it work?
diff --git a/doc/bugs/no_git-annex_shell_on_Windows/comment_10_72b2277c3728f02e4d158ebbd7db41b2._comment b/doc/bugs/no_git-annex_shell_on_Windows/comment_10_72b2277c3728f02e4d158ebbd7db41b2._comment
new file mode 100644
index 000000000..93a27d754
--- /dev/null
+++ b/doc/bugs/no_git-annex_shell_on_Windows/comment_10_72b2277c3728f02e4d158ebbd7db41b2._comment
@@ -0,0 +1,22 @@
+[[!comment format=mdwn
+ username="timotejs@7102fe834ef5514c095ceb7e09b525bafa7b1af4"
+ nickname="timotejs"
+ avatar="http://cdn.libravatar.org/avatar/3b2d892080bb891056a77a0be9ee8176"
+ subject="Did' nyone make it work?"
+ date="2018-02-06T08:15:06Z"
+ content="""
+I try to use SSH to connect to Win10 machine with annex repo.
+I am using the OpenSSH integrated into the Windows 10 (one you can install from Windows additional features thingy).
+
+it didn't know git-annex-shell so i created symlink
+
+    mklink \"C:\Program Files (x86)\Git\bin\git-annex-shell\" \"C:\Program Files (x86)\Git\bin\git-annex.exe\"
+
+and added ';.' to PATHEX
+
+Currently git-annex-shell is found but i get error:
+
+    Invalid argument `'configlist''
+
+when i try to sync remote on that W10 machine
+"""]]

diff --git a/doc/devblog/day_10__lazy_Sunday.mdwn b/doc/devblog/day_10__lazy_Sunday.mdwn
index aa6a70918..c8843d374 100644
--- a/doc/devblog/day_10__lazy_Sunday.mdwn
+++ b/doc/devblog/day_10__lazy_Sunday.mdwn
@@ -6,7 +6,7 @@ Now there's an easy way to get an overview of how close your repository
 is to meeting the configured numcopies settings (or when it exceeds them).
 
 <pre>
-# time git annex status . 
+# time git annex info . 
 [...]
 numcopies stats: 
 	numcopies +0: 6686
@@ -19,5 +19,5 @@ numcopies stats:
 	numcopies +4: 372
 </pre>
 
-This does make `git annex status` slow when run on a large directory tree,
+This does make `git annex info` slow when run on a large directory tree,
 so --fast disables that.

followup
diff --git a/doc/todo/lockdown_hooks.mdwn b/doc/todo/lockdown_hooks.mdwn
index e6777e912..c190ccd53 100644
--- a/doc/todo/lockdown_hooks.mdwn
+++ b/doc/todo/lockdown_hooks.mdwn
@@ -24,6 +24,13 @@ write bit, does not need to lockdown the files within it.
 It would be up to the command to decide how to handle the 
 core.sharedRepository configuration.
 
+These could be set in the global gitconfig file. The IncludeIf directive
+can be used to make them be used only for repositories located within a given
+mount point.
+
+git-annex test disables use of global gitconfig settings. There would need
+to be a way to let it use these.
+
 Perfomance:
 
 Hook would be called twice per store/drop of an annexed object, 
diff --git a/doc/todo/lockdown_hooks/comment_2_575c33970014662c664d71e573e718e7._comment b/doc/todo/lockdown_hooks/comment_2_575c33970014662c664d71e573e718e7._comment
new file mode 100644
index 000000000..f26180182
--- /dev/null
+++ b/doc/todo/lockdown_hooks/comment_2_575c33970014662c664d71e573e718e7._comment
@@ -0,0 +1,28 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2018-02-05T17:04:36Z"
+ content="""
+Seems likely that there are a couple of different ways to use
+ACLs to remove write access. In the simple case, any existing ACL can be
+overwritten. In other cases, some other existing ACLs will need to be
+preserved and only a single part changed. In some cases, the ACL for a user
+should be changed, in others the ACL for a group.
+
+And there are several different varieties of ACLs (POSIX, NFS, Windows). 
+And there's the immutable bit, which might be wanted in some specific
+circumstances but certianly not by most people.
+
+So it makes sense to me to not embed specific knowledge of this into git-annex.
+
+This feels to me like something that the system administrator is going to
+want to set up. It would mostly be limited to repositories inside a given
+mount point that needs the unusual lockdown method due to using NFS or
+whatever. The global gitconfig can be set up to switch on the config only
+for those repositories, and the system administrator can set up hooks
+for the particular use case.
+
+I don't see why something like datalad would need to worry about this
+detail, any more than they worry about the PATH to system programs or other
+such things that the administrator sets up.
+"""]]

initial report
diff --git a/doc/todo/INFO_message_for_custom_special_remotes.mdwn b/doc/todo/INFO_message_for_custom_special_remotes.mdwn
new file mode 100644
index 000000000..b41ea89bc
--- /dev/null
+++ b/doc/todo/INFO_message_for_custom_special_remotes.mdwn
@@ -0,0 +1,3 @@
+I wondered if it would be sensible to ask to extend [externals special remote protocol](https://git-annex.branchable.com/design/external_special_remote_protocol/) with ability for custom remotes to pass back some INFO level message (not only DEBUG or ERROR).  The reason is:  in datalad-archives special remote we usually need to `git annex get` first the key containing the archive, which might be sizeable.  Since there is ATM no way to communicate back to git-annex, so it could communicate back to the datalad which runs it, it results in no output/message to the user that possibly a heavy download is happening in the background.  So, we would need to establish our own communication from datalad-archives special remote all the way to top level datalad process to report that, or I wondered if may be we could report back to git-annex, and it in turn report back to the original process (running e.g. `annex get --json --json-progress`) so it could spit out that message wrapped into a json record within the stream, so we could process and output that to the user.
+
+[[!meta author=yoh]]

Replacing glacier-cli
diff --git a/doc/forum/Replace_glacier-cli__63__.mdwn b/doc/forum/Replace_glacier-cli__63__.mdwn
new file mode 100644
index 000000000..3557cc252
--- /dev/null
+++ b/doc/forum/Replace_glacier-cli__63__.mdwn
@@ -0,0 +1,13 @@
+I was looking for a good way to archive files to Glacier, and git-annex seems like an excellent approach.
+
+After struggling for a while with the glacier-cli project, I just don't feel that it is well maintained.
+
+ - I have to manually symlink it in place of its dependency's glacier binary.
+ - There are patches to boto that are required for it to work, which are have never been merged. And it seems from the errors I am getting that similar edits are now required in many more places.
+ - It is locked into supporting only an older version of boto.
+
+That just seems like not a great place to be starting out with an archiving project.
+
+I was wondering if you thought it might be feasible to get rid of that dependency with something else, perhaps this haskell aws project that seems to be actively maintained?
+
+https://github.com/aristidb/aws

Added a comment: Simultaneous transfers
diff --git a/doc/todo/Slow_transfer_for_a_lot_of_small_files/comment_3_dac38d9e48041b59dd1a53180e456ec9._comment b/doc/todo/Slow_transfer_for_a_lot_of_small_files/comment_3_dac38d9e48041b59dd1a53180e456ec9._comment
new file mode 100644
index 000000000..1b10f9e32
--- /dev/null
+++ b/doc/todo/Slow_transfer_for_a_lot_of_small_files/comment_3_dac38d9e48041b59dd1a53180e456ec9._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="lhunath@3b4ff15f4600f3276d1776a490b734fca0f5c245"
+ nickname="lhunath"
+ subject="Simultaneous transfers"
+ date="2018-02-02T17:37:27Z"
+ content="""
+I highly recommend ensuring that:
+1. Each remote can configure a number of maximum simultaneous transfers, where each type of remote comes with a sensible default number.
+2. Transfers to multiple individual remotes happen in parallel regardless of their simultaneous transfers setting.
+
+Judging from the fact that simultaneous transfers happen just fine when you hit the > icon in the webapp, I would assume that most of the underbelly for this is already present.
+"""]]

Added a comment: Still getting this error
diff --git a/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_10_5638e0090598425d67c073b2f04e56d5._comment b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_10_5638e0090598425d67c073b2f04e56d5._comment
new file mode 100644
index 000000000..f9b689974
--- /dev/null
+++ b/doc/bugs/__34__git-annex__58___direct__58___1_failed__34___on_Windows/comment_10_5638e0090598425d67c073b2f04e56d5._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="davclark"
+ avatar="http://cdn.libravatar.org/avatar/375c43ce3d79c0955a645e522cdab7af"
+ subject="Still getting  this error"
+ date="2018-02-02T16:45:50Z"
+ content="""
+I enabled long filename support (running Windows 10 1709 build 16299.192), did a reboot and I'm still getting this error:
+
+    git-annex: ..\..\.git\annex\objects\ada\ebe\SHA256E-s418503869--4ea1d3209d3199fed9b0e75e97cf299f59f37de2f204da2c3192ce04f69677ae.csv\SHA256E-s418503869--4ea1d3209d3199fed9b0e75e97cf299f59f37de2f204da2c3192ce04f69677ae.csv.cache: openFile: does not exist (No such file or directory)
+    failed
+    git-annex: add: 1 failed
+
+That filename is only 216 characters too... is there any way to diagnose *why* the file creation failed?
+"""]]

Added a comment: any chance to avoid necessity to config the hook(s)?
diff --git a/doc/todo/lockdown_hooks/comment_1_649dbc3c951bbabc550e12e87e45b319._comment b/doc/todo/lockdown_hooks/comment_1_649dbc3c951bbabc550e12e87e45b319._comment
new file mode 100644
index 000000000..8c35613c4
--- /dev/null
+++ b/doc/todo/lockdown_hooks/comment_1_649dbc3c951bbabc550e12e87e45b319._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="any chance to avoid necessity to config the hook(s)?"
+ date="2018-02-01T18:54:18Z"
+ content="""
+Thank you Joey for looking into this issue!  I am though a bit worried that necessity to configure using some kind of a hook would be ... suboptimal for a number of reasons.  Before laying out my argumentation, let me first ask:  why alternative \"lockdown\" mechanisms could not be sensed/configured per each repository during `init` and implemented within git-annex?
+
+As you have noted ```git-annex init's probe of the write bit handling fails...``` so git-annex already checks for a possible way to establish the \"lockdown\" for a given repository location. It just tries one possible mechanism ATM. But it could as well try multiple ways to achieve it, starting from current \"POSIX\", and then trying \"ACL\" if appropriate (i.e. tools found). Then if non-POSIX handling is needed, would simple add yet another configuration to .git/config of that repository, and consult to it to switch to corresponding lockdown implementation **within git-annex**.  This would be much more user friendly, and it would allow 3rd party tools using git-annex (such as datalad) to not worry about necessity to configure some additional hooks for a particular location, etc.
+"""]]

add todo
diff --git a/doc/todo/lockdown_hooks.mdwn b/doc/todo/lockdown_hooks.mdwn
new file mode 100644
index 000000000..e6777e912
--- /dev/null
+++ b/doc/todo/lockdown_hooks.mdwn
@@ -0,0 +1,39 @@
+Add git hooks that are used to [[internals/lockdown]] annexed objects.
+--[[Joey]]
+
+Use cases include:
+
+* Setting immutable bit on systems where git-annex is given the ability to
+  do so, to fully guard against accidental deletion in all circumstances.
+
+* For systems that ignore the write bit, but have some other way to prevent
+  write to a file (eg, ACLs or something).
+
+  Note that in such a case, `git-annex init`'s probe of the write bit
+  handling fails; as long as the hook is configured globally, it should
+  run the hook instead, and if it works, can avoid direct mode.
+
+Design:
+
+Configs: annex.lockdown-command, annex.unlockdown-command
+In these, "%path" is replaced with the file/directory to act on.
+
+Locking down a directory only needs to do the equivilant of removing its
+write bit, does not need to lockdown the files within it.
+
+It would be up to the command to decide how to handle the 
+core.sharedRepository configuration.
+
+Perfomance:
+
+Hook would be called twice per store/drop of an annexed object, 
+once for the file and once for the parent directory.
+
+On windows, called four times per lock of an annexed object, to first thaw
+it and then freeze it. This could be reduced to 2, I think. 
+On posix, the file is locked without being thawed, 
+as only read access is needed.
+
+Probably running a shell script is not too much overhead in many cases,
+if it was too slow, there could be a variant that is run once and 
+fed the names of files to operate on via stdin.

diff --git a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant.mdwn b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant.mdwn
index 57a3869e3..dd8b2a997 100644
--- a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant.mdwn
+++ b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant.mdwn
@@ -7,6 +7,11 @@ Create a repository, try to add a remote. An external disk in this case.
 ### What version of git-annex are you using? On what operating system?
 Fedora 27
 
+    $ cat /etc/fedora-release
+    Fedora release 27 (Twenty Seven)
+    $ uname -a
+    Linux greger 4.14.14-300.fc27.x86_64 #1 SMP Fri Jan 19 13:19:54 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
+
     $ git annex version
     git-annex version: 6.20180130-g39d97867a
     build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser Feeds Testsuite

Fixed some formatting
diff --git a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant.mdwn b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant.mdwn
index 6ca839b2f..57a3869e3 100644
--- a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant.mdwn
+++ b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant.mdwn
@@ -95,6 +95,7 @@ Debug log:
     31/Jan/2018:14:47:52 +0100 [Error#yesod-core] there is no available git remote named "archive" @(yesod-core-1.4.37-GCI7RasEcSMIU2vku0fHJ1:Yesod.Core.Class.Yesod ./Yesod/Core/Class/Yesod.hs:693:5)
 
 The interesting part here is that if I try to run the failing git commands in the repository it works:
+
     $ git --git-dir=../../../../run/media/dxtr/archive/annex --literal-pathspecs show-ref git-annex
     26bcec3f33162a9c1c5b7a39c4828eedfdeae41b refs/heads/git-annex
 

diff --git a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant.mdwn b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant.mdwn
new file mode 100644
index 000000000..6ca839b2f
--- /dev/null
+++ b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant.mdwn
@@ -0,0 +1,104 @@
+### Please describe the problem.
+No matter what I do I can't 
+
+### What steps will reproduce the problem?
+Create a repository, try to add a remote. An external disk in this case.
+
+### What version of git-annex are you using? On what operating system?
+Fedora 27
+
+    $ git annex version
+    git-annex version: 6.20180130-g39d97867a
+    build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser Feeds Testsuite
+    dependency versions: aws-0.17.1 bloomfilter-2.0.1.0 cryptonite-0.23 DAV-1.3.1 feed-0.3.12.0 ghc-8.0.2 http-client-0.5.7.0 persistent-sqlite-2.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+    local repository version: 5
+    supported repository versions: 3 5 6
+    upgrade supported from repository versions: 0 1 2 3 4 5
+    operating system: linux x86_64
+
+
+### Please provide any additional information below.
+
+Debug log:
+
+    [2018-01-31 14:46:15.670775214] main: starting assistant version 6.20180130-g39d97867a
+    [2018-01-31 14:46:15.776042349] Cronner: You should enable consistency checking to protect your data. 
+    (scanning...) [2018-01-31 14:46:15.988042623] Watcher: Performing startup scan
+    (started...) 
+    [2018-01-31 14:47:48.236476657] read: gpg ["--quiet","--trust-model","always","--with-colons","--list-secret-keys","--fixed-list-mode"]
+    [2018-01-31 14:47:48.262055169] process done ExitSuccess
+    [2018-01-31 14:47:51.695935824] read: git ["init","--quiet","--bare","/run/media/dxtr/archive/annex"]
+    [2018-01-31 14:47:51.700278024] process done ExitSuccess
+    [2018-01-31 14:47:51.700424216] read: git ["config","--null","--list"]
+    [2018-01-31 14:47:51.707427206] process done ExitSuccess
+    [2018-01-31 14:47:51.70868726] call: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","config","core.fsyncobjectfiles","true"]
+    [2018-01-31 14:47:51.725766269] process done ExitSuccess
+    [2018-01-31 14:47:51.728215647] read: git ["config","--null","--list"]
+    [2018-01-31 14:47:51.740757992] process done ExitSuccess
+    [2018-01-31 14:47:51.741434499] read: git ["config","--null","--list"]
+    [2018-01-31 14:47:51.753548799] process done ExitSuccess
+    [2018-01-31 14:47:51.755149461] read: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","show-ref","git-annex"]
+    [2018-01-31 14:47:51.771911199] process done ExitFailure 1
+    [2018-01-31 14:47:51.772860937] read: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
+    [2018-01-31 14:47:51.789876829] process done ExitFailure 1
+    [2018-01-31 14:47:51.790600261] call: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","show-ref","--verify","-q","origin/git-annex"]
+    [2018-01-31 14:47:51.79719711] process done ExitFailure 1
+    [2018-01-31 14:47:51.799141741] read: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","write-tree"]
+    [2018-01-31 14:47:51.807820543] process done ExitSuccess
+    [2018-01-31 14:47:51.809126461] chat: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","commit-tree","4b825dc642cb6eb9a060e54bf8d69288fbee4904","--no-gpg-sign"]
+    [2018-01-31 14:47:51.836742843] process done ExitSuccess
+    [2018-01-31 14:47:51.837688371] call: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","update-ref","refs/heads/git-annex","c1bd62a595ce9cb811ad353ed0d9eaf8cfcb0b30"]
+    [2018-01-31 14:47:51.849598566] process done ExitSuccess
+    [2018-01-31 14:47:51.850762057] call: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","config","annex.uuid","da4c4bf3-5b3d-47f7-98d6-040cf8094360"]
+    [2018-01-31 14:47:51.855957091] process done ExitSuccess
+    [2018-01-31 14:47:51.85617243] read: git ["config","--null","--list"]
+    [2018-01-31 14:47:51.858450427] process done ExitSuccess
+    [2018-01-31 14:47:51.860771677] read: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","show-ref","git-annex"]
+    [2018-01-31 14:47:51.879636685] process done ExitSuccess
+    [2018-01-31 14:47:51.880463945] read: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
+    [2018-01-31 14:47:51.897305118] process done ExitSuccess
+    [2018-01-31 14:47:51.899214969] read: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","log","refs/heads/git-annex..c1bd62a595ce9cb811ad353ed0d9eaf8cfcb0b30","--pretty=%H","-n1"]
+    [2018-01-31 14:47:51.9105804] process done ExitSuccess
+    [2018-01-31 14:47:51.912107527] chat: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","cat-file","--batch"]
+    [2018-01-31 14:47:51.916244492] chat: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
+    [2018-01-31 14:47:51.926062822] call: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","config","annex.version","5"]
+    [2018-01-31 14:47:51.943412825] process done ExitSuccess
+    [2018-01-31 14:47:51.943804264] read: git ["config","--null","--list"]
+    [2018-01-31 14:47:51.949022318] process done ExitSuccess
+    [2018-01-31 14:47:51.952482534] chat: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","hash-object","-w","--stdin-paths","--no-filters"]
+    [2018-01-31 14:47:51.954175781] feed: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","update-index","-z","--index-info"]
+    [2018-01-31 14:47:51.968113275] process done ExitSuccess
+    [2018-01-31 14:47:51.969050491] read: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
+    [2018-01-31 14:47:51.979633335] process done ExitSuccess
+    (recording state in git...)
+    [2018-01-31 14:47:51.980277288] read: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","write-tree"]
+    [2018-01-31 14:47:51.989727059] process done ExitSuccess
+    [2018-01-31 14:47:51.98998106] chat: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","commit-tree","06548fa56acf18f9c1d74a29e248c4650df15e7e","--no-gpg-sign","-p","refs/heads/git-annex"]
+    [2018-01-31 14:47:52.011569157] process done ExitSuccess
+    [2018-01-31 14:47:52.012198525] call: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","update-ref","refs/heads/git-annex","26bcec3f33162a9c1c5b7a39c4828eedfdeae41b"]
+    [2018-01-31 14:47:52.030929599] process done ExitSuccess
+    [2018-01-31 14:47:52.037715315] process done ExitSuccess
+    [2018-01-31 14:47:52.038491764] process done ExitSuccess
+    [2018-01-31 14:47:52.039098682] process done ExitSuccess
+    [2018-01-31 14:47:52.039790346] read: uname ["-n"]
+    [2018-01-31 14:47:52.043271433] process done ExitSuccess
+    [2018-01-31 14:47:52.043523608] read: git ["config","--null","--list"]
+    [2018-01-31 14:47:52.057690629] process done ExitSuccess
+    [2018-01-31 14:47:52.058740927] call: git ["--git-dir=../../../../run/media/dxtr/archive/annex","--literal-pathspecs","remote","add","greger","../../../../../home/dxtr/temp/test"]
+    [2018-01-31 14:47:52.072184794] process done ExitSuccess
+    [2018-01-31 14:47:52.072509479] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","core.bare=false","remote","add","archive","/run/media/dxtr/archive/annex"]
+    [2018-01-31 14:47:52.081339812] process done ExitSuccess
+    [2018-01-31 14:47:52.082154367] read: git ["config","--null","--list"]
+    [2018-01-31 14:47:52.093569488] process done ExitSuccess
+    31/Jan/2018:14:47:52 +0100 [Error#yesod-core] there is no available git remote named "archive" @(yesod-core-1.4.37-GCI7RasEcSMIU2vku0fHJ1:Yesod.Core.Class.Yesod ./Yesod/Core/Class/Yesod.hs:693:5)
+
+The interesting part here is that if I try to run the failing git commands in the repository it works:
+    $ git --git-dir=../../../../run/media/dxtr/archive/annex --literal-pathspecs show-ref git-annex
+    26bcec3f33162a9c1c5b7a39c4828eedfdeae41b refs/heads/git-annex
+
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+I have been using it for years. Just not the webapp :)
+

diff --git a/doc/forum/offsite_repo.mdwn b/doc/forum/offsite_repo.mdwn
new file mode 100644
index 000000000..0b950a95e
--- /dev/null
+++ b/doc/forum/offsite_repo.mdwn
@@ -0,0 +1,8 @@
+Hi there,
+
+Let's say I have a Pictures repo in my laptop. I want to have a local copy in an external backup drive and an offsite copy in google-drive. I set numcopies to 3. Is it possible to require that files in a repository have a copy in an offsite repo e.g. a google-drive special remote? So that when I run e.g. git annex fsck, I will be reminded that some local files are not yet backed up in an offsite repo. Probably via .git/config, I can declare a remote repo as an "offsite" repo. Something like: 
+
+    remote.repo_name.offsite = true
+
+
+Eric

initial whishlist
diff --git a/doc/todo/--get_option_for_diffdriver.mdwn b/doc/todo/--get_option_for_diffdriver.mdwn
new file mode 100644
index 000000000..d6ad9a2b9
--- /dev/null
+++ b/doc/todo/--get_option_for_diffdriver.mdwn
@@ -0,0 +1 @@
+since there is no generic 'fuse' mode, I would like to request to have `--get` (or `--auto-get`) option for diffdriver.  I am trying to compare files across two branches on a repo I just cloned.  I cannot download all the files  and downloading differing keys across branches for the same file is a bit painful.  So I felt that it would be super nice if git annex could auto get those files from somewhere (well -- original clone)

Added a comment
diff --git a/doc/forum/Filesystem_recommendation___40__ext4__44___btrfs__44___xfs__44___you_name_it__41____63__/comment_1_fcf7f132f771de28553bd20f4fb014cb._comment b/doc/forum/Filesystem_recommendation___40__ext4__44___btrfs__44___xfs__44___you_name_it__41____63__/comment_1_fcf7f132f771de28553bd20f4fb014cb._comment
new file mode 100644
index 000000000..a671586f9
--- /dev/null
+++ b/doc/forum/Filesystem_recommendation___40__ext4__44___btrfs__44___xfs__44___you_name_it__41____63__/comment_1_fcf7f132f771de28553bd20f4fb014cb._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 1"
+ date="2018-01-24T03:10:53Z"
+ content="""
+Just ran into this question.  Might still be of interest: for our datalad.org project I did some benchmarking of various filesystems a few years back. Here you can find the final report: [http://old.datalad.org/test_fs_analysis.html](http://old.datalad.org/test_fs_analysis.html) -- I settled on a bunch of software raid5s with BTRFS on top of them.  So far (for years), quite good.  snapshots and [btrbk](http://digint.ch/btrbk/) are also a blessing.  I use the same btrfs also as docker backend.
+"""]]

Added a comment
diff --git a/doc/todo/support_.netrc_for_fsck_--from_web/comment_2_01b513a530e8887567aca0e187ad0f7e._comment b/doc/todo/support_.netrc_for_fsck_--from_web/comment_2_01b513a530e8887567aca0e187ad0f7e._comment
new file mode 100644
index 000000000..6542c5958
--- /dev/null
+++ b/doc/todo/support_.netrc_for_fsck_--from_web/comment_2_01b513a530e8887567aca0e187ad0f7e._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="mbekkema97@66b135681014f005a3a14c4011d148fcb6655f81"
+ nickname="mbekkema97"
+ avatar="http://cdn.libravatar.org/avatar/a5037b8a5914bd9f803af7b7e881d632"
+ subject="comment 2"
+ date="2018-01-19T03:46:29Z"
+ content="""
+That's strange, I was observing the process list and didn't see any instances of wget or curl. Could it be that `fsck` uses http-conduit to ping the URL before proceeding with the full check?
+"""]]

Added a comment: Remotes when using multiple repos
diff --git a/doc/walkthrough/adding_a_remote/comment_5_bcbe480aa710ce693ee03d86884f1820._comment b/doc/walkthrough/adding_a_remote/comment_5_bcbe480aa710ce693ee03d86884f1820._comment
new file mode 100644
index 000000000..bb8271a84
--- /dev/null
+++ b/doc/walkthrough/adding_a_remote/comment_5_bcbe480aa710ce693ee03d86884f1820._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="elmimmo"
+ avatar="http://cdn.libravatar.org/avatar/4f00dfc3ad590ef7492788b854ceba78"
+ subject="Remotes when using multiple repos"
+ date="2018-01-18T07:02:18Z"
+ content="""
+I see that you manually add the first repo as a remote of the external drive, and the external drive as a remote of the original.
+
+What about when you add a second external drive? Should you then be adding the first two repos as remotes of the new one, and then go to the first two ones and add the new drive as a remote of them too? Doesn't this permutation scale out of control when you add a new drive to, say, a git annex made of 10 disks, 20 disks, etc?
+"""]]

removed
diff --git a/doc/tips/downloading_podcasts/comment_28_a708fe3c28c2f5fdabddc00e0e29e5d6._comment b/doc/tips/downloading_podcasts/comment_28_a708fe3c28c2f5fdabddc00e0e29e5d6._comment
deleted file mode 100644
index d58cb7165..000000000
--- a/doc/tips/downloading_podcasts/comment_28_a708fe3c28c2f5fdabddc00e0e29e5d6._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="git-annex@6f13b739194f758abc0b86556b7ce966c1bf3c00"
- avatar="http://cdn.libravatar.org/avatar/198790d74209efe4896fd4cfc37ec2a6"
- subject="vimeo"
- date="2018-01-17T18:22:42Z"
- content="""
-What about vimeo? importfeed says \"warning: bad feed content; no enclosures to download\". I wonder whats wrong with e.g.
-
-    git annex importfeed -F https://vimeo.com/logiingimars/videos/rss
-"""]]

Added a comment: vimeo
diff --git a/doc/tips/downloading_podcasts/comment_28_a708fe3c28c2f5fdabddc00e0e29e5d6._comment b/doc/tips/downloading_podcasts/comment_28_a708fe3c28c2f5fdabddc00e0e29e5d6._comment
new file mode 100644
index 000000000..d58cb7165
--- /dev/null
+++ b/doc/tips/downloading_podcasts/comment_28_a708fe3c28c2f5fdabddc00e0e29e5d6._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="git-annex@6f13b739194f758abc0b86556b7ce966c1bf3c00"
+ avatar="http://cdn.libravatar.org/avatar/198790d74209efe4896fd4cfc37ec2a6"
+ subject="vimeo"
+ date="2018-01-17T18:22:42Z"
+ content="""
+What about vimeo? importfeed says \"warning: bad feed content; no enclosures to download\". I wonder whats wrong with e.g.
+
+    git annex importfeed -F https://vimeo.com/logiingimars/videos/rss
+"""]]

removed
diff --git a/doc/internals/comment_6_77da73ac25410f4f2e59e2f246dc6221._comment b/doc/internals/comment_6_77da73ac25410f4f2e59e2f246dc6221._comment
deleted file mode 100644
index 6a57deead..000000000
--- a/doc/internals/comment_6_77da73ac25410f4f2e59e2f246dc6221._comment
+++ /dev/null
@@ -1,14 +0,0 @@
-[[!comment format=mdwn
- username="arseny-n@6aba76e573dcdf2fd9e033fb3132944c8466125a"
- nickname="arseny-n"
- avatar="http://cdn.libravatar.org/avatar/0c138544ac6dfffb07d6a60dbe430587"
- subject=".git/annex/misctmp very large"
- date="2018-01-17T13:04:40Z"
- content="""
-Why is .git/annex/misctmp so large ? Currently I use git annex to manage pytorch models, 
-basically I have a large amount (1500 folder) of 4 Kilbytes files, some files are are bigger,
-misctmp occupies 6.2 GB, is  it ok ?
- 
-PS. Sorry, if I write this here but failed to post to the forum.
-
-"""]]

Added a comment: .git/annex/misctmp very large
diff --git a/doc/internals/comment_8_b7db717de86bae61bf26599cc0f9c7cf._comment b/doc/internals/comment_8_b7db717de86bae61bf26599cc0f9c7cf._comment
new file mode 100644
index 000000000..2cbff5c59
--- /dev/null
+++ b/doc/internals/comment_8_b7db717de86bae61bf26599cc0f9c7cf._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="arseny-n@6aba76e573dcdf2fd9e033fb3132944c8466125a"
+ nickname="arseny-n"
+ avatar="http://cdn.libravatar.org/avatar/0c138544ac6dfffb07d6a60dbe430587"
+ subject=".git/annex/misctmp very large"
+ date="2018-01-17T13:05:04Z"
+ content="""
+Why is .git/annex/misctmp so large ? Currently I use git annex to manage pytorch models, 
+basically I have a large amount (1500 folder) of 4 Kilobytes files, some files are are bigger,
+misctmp occupies 6.2 GB, is  it ok ?
+ 
+PS. Sorry, if I write this here but failed to post to the forum.
+
+"""]]

Added a comment: .git/annex/misctmp very large
diff --git a/doc/internals/comment_7_7e40f744f9ac7f0403df9d1a2162a516._comment b/doc/internals/comment_7_7e40f744f9ac7f0403df9d1a2162a516._comment
new file mode 100644
index 000000000..3f70188fc
--- /dev/null
+++ b/doc/internals/comment_7_7e40f744f9ac7f0403df9d1a2162a516._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="arseny-n@6aba76e573dcdf2fd9e033fb3132944c8466125a"
+ nickname="arseny-n"
+ avatar="http://cdn.libravatar.org/avatar/0c138544ac6dfffb07d6a60dbe430587"
+ subject=".git/annex/misctmp very large"
+ date="2018-01-17T13:04:52Z"
+ content="""
+Why is .git/annex/misctmp so large ? Currently I use git annex to manage pytorch models, 
+basically I have a large amount (1500 folder) of 4 Kilobytes files, some files are are bigger,
+misctmp occupies 6.2 GB, is  it ok ?
+ 
+PS. Sorry, if I write this here but failed to post to the forum.
+
+"""]]

Added a comment: .git/annex/misctmp very large
diff --git a/doc/internals/comment_6_77da73ac25410f4f2e59e2f246dc6221._comment b/doc/internals/comment_6_77da73ac25410f4f2e59e2f246dc6221._comment
new file mode 100644
index 000000000..6a57deead
--- /dev/null
+++ b/doc/internals/comment_6_77da73ac25410f4f2e59e2f246dc6221._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="arseny-n@6aba76e573dcdf2fd9e033fb3132944c8466125a"
+ nickname="arseny-n"
+ avatar="http://cdn.libravatar.org/avatar/0c138544ac6dfffb07d6a60dbe430587"
+ subject=".git/annex/misctmp very large"
+ date="2018-01-17T13:04:40Z"
+ content="""
+Why is .git/annex/misctmp so large ? Currently I use git annex to manage pytorch models, 
+basically I have a large amount (1500 folder) of 4 Kilbytes files, some files are are bigger,
+misctmp occupies 6.2 GB, is  it ok ?
+ 
+PS. Sorry, if I write this here but failed to post to the forum.
+
+"""]]

Added a comment
diff --git a/doc/forum/git-annex_sync_fails_in_repo1_but_not_in_repo2/comment_2_1dfeb722edd54c68b4220d00c0a175e5._comment b/doc/forum/git-annex_sync_fails_in_repo1_but_not_in_repo2/comment_2_1dfeb722edd54c68b4220d00c0a175e5._comment
new file mode 100644
index 000000000..b40f22394
--- /dev/null
+++ b/doc/forum/git-annex_sync_fails_in_repo1_but_not_in_repo2/comment_2_1dfeb722edd54c68b4220d00c0a175e5._comment
@@ -0,0 +1,35 @@
+[[!comment format=mdwn
+ username="felix.hagemann@b76e9ea0928cf33dacffc37ec3dbecf33171a8a5"
+ nickname="felix.hagemann"
+ avatar="http://cdn.libravatar.org/avatar/1f7e89860de517a494f35ebfb385288e"
+ subject="comment 2"
+ date="2018-01-16T19:26:03Z"
+ content="""
+Deleting `.git/MERGE_HEAD` sort of fixed the problem:
+
+        client$ git-annex sync
+        commit  ok
+        pull repo2 
+
+        Already up to date!
+        Automatic merge went well; stopped before committing as requested
+
+        error: duplicate parent be6278073504cfd19400c100b1871ba4324f55db ignored
+        ok
+        push repo2 
+        Counting objects: 1, done.
+        Writing objects: 100% (1/1), 209 bytes | 209.00 KiB/s, done.
+        Total 1 (delta 0), reused 0 (delta 0)
+        To repo2:/media/srv/img
+           be62780735..020ce83c1f  annex/direct/master -> synced/master
+        ok
+        client$ git-annex sync
+        commit  ok
+        pull repo2 
+        ok
+        client$
+
+Thanks!
+
+I have no idea how the repo got in that state. Syncing started to fail some time early December and for a few months I've been only using `git-annex assistant` and `git-annex webapp`.
+"""]]

response
diff --git a/doc/forum/Why_does_git_annex_does_not_want_that_file__63__/comment_1_44cd6ca1f7215395e19ce654513197a9._comment b/doc/forum/Why_does_git_annex_does_not_want_that_file__63__/comment_1_44cd6ca1f7215395e19ce654513197a9._comment
new file mode 100644
index 000000000..709bc80fd
--- /dev/null
+++ b/doc/forum/Why_does_git_annex_does_not_want_that_file__63__/comment_1_44cd6ca1f7215395e19ce654513197a9._comment
@@ -0,0 +1,38 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-01-15T18:20:20Z"
+ content="""
+The problem is that you have the repository in two groups.
+
+The way that `standard` works is that the repository has to be in
+exactly one group and then it will use the preferred content for that
+group. It does not know what to do when there are two groups being
+combined.
+
+Solution for you is to run: 
+
+	git annex wanted . anything
+
+Since backup repositories have a copy of every file, matching anything is
+what you want it to do. Since it's also in the archive group, other archive
+repositories will treat it as an archive too, so once a file reaches your
+combo backup and archive repository, it will be removed from other
+archives.
+
+Of course, you could also OR the preferred content expression for backup
+with the preferred content expression for archive, but that would have the
+same result since "anything OR archive" matches anything.
+
+But we can't just OR the two preferred content expressions for two groups
+in general. Consider the client and archive preferred content expressions:
+
+	(include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1
+
+	(not (copies=archive:1 or copies=smallarchive:1)) or approxlackingcopies=1
+
+These are contradictory when a repository is in both the client and archive
+groups; the client doesn't want anything that's in an archive, but if the
+repository is both a client and an archive, it doesn't want any content
+that it has!
+"""]]

fix outdated docs
When a repo is in multiple standard groups,
Logs.PreferredContent.makeMatcher uses unknownMatcher,
so will not get everything, but instead defaults to wanting whatever is
present.
diff --git a/doc/git-annex-preferred-content.mdwn b/doc/git-annex-preferred-content.mdwn
index 92899c82a..54f0d4666 100644
--- a/doc/git-annex-preferred-content.mdwn
+++ b/doc/git-annex-preferred-content.mdwn
@@ -164,8 +164,6 @@ elsewhere to allow removing it).
   When a repository is in exactly one such group, you can use the "standard"
   keyword in its preferred content expression, to match whatever content
   the group's expression matches.
-  (If a repository is put into multiple standard
-  groups, "standard" will match anything.. so don't do that!)
 
   Most often, the whole preferred content expression is simply "standard".
   But, you can do more complicated things, for example:

comment
diff --git a/doc/forum/git-annex_sync_fails_in_repo1_but_not_in_repo2/comment_1_5cca2fbce3db1c691f43e43f9343f9bd._comment b/doc/forum/git-annex_sync_fails_in_repo1_but_not_in_repo2/comment_1_5cca2fbce3db1c691f43e43f9343f9bd._comment
new file mode 100644
index 000000000..8f2bd5712
--- /dev/null
+++ b/doc/forum/git-annex_sync_fails_in_repo1_but_not_in_repo2/comment_1_5cca2fbce3db1c691f43e43f9343f9bd._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-01-15T18:19:02Z"
+ content="""
+It sounds like deleting `.git/MERGE_HEAD` might fix the problem in the repo
+with the problem.
+
+I don't know how you could have gotten the repository into that state..
+"""]]

response
diff --git a/doc/forum/Improving_the_git-annex_forum/comment_1_63c52c977689a5da34fec65391c25fd6._comment b/doc/forum/Improving_the_git-annex_forum/comment_1_63c52c977689a5da34fec65391c25fd6._comment
new file mode 100644
index 000000000..0231987f6
--- /dev/null
+++ b/doc/forum/Improving_the_git-annex_forum/comment_1_63c52c977689a5da34fec65391c25fd6._comment
@@ -0,0 +1,34 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-01-15T17:33:12Z"
+ content="""
+Thanks, Florian for raising this topic.
+
+Granting the missing features you list (and perhaps adding that people seem
+to sometimes find it annoying to log in successfully here), I wonder how
+much those features are limiting use of this forum, if at all. I suppose
+one way to find out would be to move to another forum and see if traffic
+increases (or SNR or engagement improves).
+
+For myself, having the forum integrated in the git-annex repo is quite
+helpful to me, since I automatically track it along with all the other
+changes to the website, and am sure to see every single post here,
+and it meets my offline/low bandwidth needs. Also I sometimes promote
+forum posts to bug reports or make a commit following up to a forum
+post with a requested feature or fix.
+
+I suspect that the git-annex community is kind of split between people who
+are more comfortable in web forums, and people who are used to mailing
+lists, who are currently being left entirely out in the cold (and probably
+using the irc channel more). It could be that mailman3 with its mix of
+forum and mailing list could better unify the two groups. Or perhaps
+there's no point in trying to do that, and a simple mailing list would be a
+bigger win than changing the forum?
+
+I'm ok with however git-annex users choose to get together and
+talk about it, including ways that I can't/won't use myself (including
+non-free stuff like Slack..). To that end, if people want to get a
+git-annex community going somewhere, be it another forum, or reddit, or
+whatever, I'm fine with the website directing users to it.
+"""]]

remove comments that don't belong under this page
Posted long enough ago that they should have seen the responses.
diff --git a/doc/contact/comment_1_12d60f767d90bea94974e1ff6b206d31._comment b/doc/contact/comment_1_12d60f767d90bea94974e1ff6b206d31._comment
deleted file mode 100644
index f72f3fbe5..000000000
--- a/doc/contact/comment_1_12d60f767d90bea94974e1ff6b206d31._comment
+++ /dev/null
@@ -1,8 +0,0 @@
-[[!comment format=mdwn
- username="https://www.google.com/accounts/o8/id?id=AItOawlZQcpoRmyZBlsFQ7TMMYlZZdmBG1TxMzA"
- nickname="Felipe"
- subject="translation"
- date="2013-09-10T20:19:53Z"
- content="""
-I would like to translate this content of site, the tutorials, pages, etc, to portuguese. I think that can help many users in my country. How can I build a new pages here with git-annex content in portuguese?  
-"""]]
diff --git a/doc/contact/comment_2_95b6d868b913418de50ba121d71d2390._comment b/doc/contact/comment_2_95b6d868b913418de50ba121d71d2390._comment
deleted file mode 100644
index 6f451d9ae..000000000
--- a/doc/contact/comment_2_95b6d868b913418de50ba121d71d2390._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="http://joeyh.name/"
- ip="4.154.4.51"
- subject="comment 2"
- date="2013-09-12T21:08:16Z"
- content="""
-I welcome translations in theory, but I do not have any infrastructure in place to support it, so that would have to be the first step.
-
-(Also, this page is not a general-purpose discussion forum.)
-"""]]
diff --git a/doc/contact/comment_3_2cf43bd406673294e6cdbd785c4a0d0c._comment b/doc/contact/comment_3_2cf43bd406673294e6cdbd785c4a0d0c._comment
deleted file mode 100644
index 32fbc7f65..000000000
--- a/doc/contact/comment_3_2cf43bd406673294e6cdbd785c4a0d0c._comment
+++ /dev/null
@@ -1,16 +0,0 @@
-[[!comment format=mdwn
- username="https://www.google.com/accounts/o8/id?id=AItOawlkA6XinbeOdnEDxEGQUWyjqPGh0kdMXr4"
- nickname="Blake"
- subject="Starting git-annex assistant"
- date="2013-10-02T23:57:48Z"
- content="""
-Hi Joey,
-
-I am one of your original funders for git-annex. I think you have done a great job and I am proud to say I helped fund your endeavor.
-
-How do I start the git-annex assistant webapp via command line? I installed the program on my Fedora 19 computer via YUM and it does not come packaged with a desktop icon as you show in your demonstration video. I try running `$ git annex assistant` but I cannot access `http://127.0.0.1:34795/` as you do in the video. Also, the quickstart page (http://git-annex.branchable.com/assistant/quickstart/) seems to have outdated information as it states you can start the webapp via `$ git annex webapp`, however when I try to run this I get the following error: `git-annex: unknown command webapp`.
-
-Regards,
-Blake
-
-"""]]
diff --git a/doc/contact/comment_4_586a506e27379d74fbc0f4b654e89c7d._comment b/doc/contact/comment_4_586a506e27379d74fbc0f4b654e89c7d._comment
deleted file mode 100644
index 1cf9c3acc..000000000
--- a/doc/contact/comment_4_586a506e27379d74fbc0f4b654e89c7d._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="http://joeyh.name/"
- ip="108.236.230.124"
- subject="comment 4"
- date="2013-10-03T00:06:11Z"
- content="""
-Your fedora build apparently does not include the webapp, since `git annex webapp` is the right command. Install the [[install/Linux_standalone]] build instead.
-
-(Again this page is not a discussion forum. Next person to post here will waste my time since I will have to go configure the site to block comments here..)
-"""]]

close
diff --git a/doc/bugs/Adding_torrent_via_addurl_fails.mdwn b/doc/bugs/Adding_torrent_via_addurl_fails.mdwn
index c003775ba..058abaefb 100644
--- a/doc/bugs/Adding_torrent_via_addurl_fails.mdwn
+++ b/doc/bugs/Adding_torrent_via_addurl_fails.mdwn
@@ -54,3 +54,4 @@ git-annex: failed to parse torrent: Name not found in dictionary: announce
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
 
+> This was fixed in haskell-torrent version 10000.1.0. [[done]]

response
diff --git a/doc/todo/support_ssh__58____47____47___or_sftp__58____47____47___urls_via___34__built-in__34___ssh_support/comment_1_7932fd28b14898a0b4846856a27405b8._comment b/doc/todo/support_ssh__58____47____47___or_sftp__58____47____47___urls_via___34__built-in__34___ssh_support/comment_1_7932fd28b14898a0b4846856a27405b8._comment
new file mode 100644
index 000000000..13887bb87
--- /dev/null
+++ b/doc/todo/support_ssh__58____47____47___or_sftp__58____47____47___urls_via___34__built-in__34___ssh_support/comment_1_7932fd28b14898a0b4846856a27405b8._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-01-15T17:28:16Z"
+ content="""
+I don't think that all the machinery *is* there, because
+for urls there needs to be a way to check the file size,
+and it's not clear how to do that with a sftp or ssh url.
+"""]]

response
diff --git a/doc/todo/support_.netrc_for_fsck_--from_web/comment_1_27fc7996be9cbff994fb19b0e213f28b._comment b/doc/todo/support_.netrc_for_fsck_--from_web/comment_1_27fc7996be9cbff994fb19b0e213f28b._comment
new file mode 100644
index 000000000..7dfb25da9
--- /dev/null
+++ b/doc/todo/support_.netrc_for_fsck_--from_web/comment_1_27fc7996be9cbff994fb19b0e213f28b._comment
@@ -0,0 +1,29 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-01-15T17:12:43Z"
+ content="""
+This is complicated because git-annex uses wget, curl, and http-conduit at
+different times. wget always uses .netrc, curl only uses it with a --netrc
+option (which git-annex does not currently provide although the user can),
+and http-conduit does not support .netrc.
+
+The choices of which git-annex uses when are constrained by the limitations
+of all three, and were chosen to make it work as well as possible. Also,
+curl and wget are not available in all installations of git-annex.
+
+As far as I can see, `git annex fsck --from web` makes the same wget/curl
+choice as `git annex get --from web` will make, typically wget, but
+curl if wget is not available or if run with --quiet.
+
+However, `git annex fsck --fast --from web` uses http-conduit by default.
+It could be changed to default to using curl (wget is not an option there).
+
+But, I'm kind of inclined toward moving away from using wget/curl at all,
+and toward http-conduit for all http urls. Thus avoiding the
+inconsistencies and various annoying behaviors of wget/curl. So, making a
+change that requires .netrc be supported going forward needs to be
+considered in that light. One possibility would be to use
+<https://hackage.haskell.org/package/netrc> to make netrc work with
+http-conduit.
+"""]]

response
diff --git a/doc/forum/How_to_set_backend_based_on_file_size_in_.gitattributes/comment_1_c8ac6b924d4446d4a1a12fb34532fd69._comment b/doc/forum/How_to_set_backend_based_on_file_size_in_.gitattributes/comment_1_c8ac6b924d4446d4a1a12fb34532fd69._comment
new file mode 100644
index 000000000..878d589ad
--- /dev/null
+++ b/doc/forum/How_to_set_backend_based_on_file_size_in_.gitattributes/comment_1_c8ac6b924d4446d4a1a12fb34532fd69._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-01-15T16:15:39Z"
+ content="""
+annex.backend only matches on file name, so can't be used for this. I think
+your current two pass approach is the only way to do it currently.
+"""]]

diff --git a/doc/forum/How_to_set_backend_based_on_file_size_in_.gitattributes.mdwn b/doc/forum/How_to_set_backend_based_on_file_size_in_.gitattributes.mdwn
new file mode 100644
index 000000000..75a4ca7c0
--- /dev/null
+++ b/doc/forum/How_to_set_backend_based_on_file_size_in_.gitattributes.mdwn
@@ -0,0 +1,16 @@
+Hi there,
+
+In my repos, I want files larger than 500MB to use the WORM backend. So when I'm adding large files, I do it in 2 passes. I first run this:
+    
+    git annex add --largerthan=500MB --backend=WORM .
+
+to add only files larger than 500MB. The remaining files that are smaller than 500MB I then add using default backend:
+
+    git annex add
+
+Is it possible to set annex.backend in .gitattributes so that adding files larger than 500MB automatically use the WORM backend? Can I use expressions such as largerthan or smallerthan? From the example [here](https://git-annex.branchable.com/backends/), it seems that it can only be set based on file type. 
+
+
+Regards,
+
+Eric

diff --git a/doc/forum/git-annex_sync_fails_in_repo1_but_not_in_repo2.mdwn b/doc/forum/git-annex_sync_fails_in_repo1_but_not_in_repo2.mdwn
new file mode 100644
index 000000000..222befd16
--- /dev/null
+++ b/doc/forum/git-annex_sync_fails_in_repo1_but_not_in_repo2.mdwn
@@ -0,0 +1,45 @@
+Hello,
+
+I am using two repos in direct mode that are connected via ssh.
+Syncing fails in one direction but works in the other, I don't understand how to fix that.
+
+## repo2 -> repo1 works:
+
+    server$ git-annex sync
+    commit  ok
+    pull repo1 
+    From client:~/media/img/db
+     + be62780...2d74673 master     -> client/master  (forced update)
+     + be62780...2d74673 synced/master -> client/synced/master  (forced update)
+    ok
+    push repo1 
+    Total 0 (delta 0), reused 0 (delta 0)
+    To client:~/media/img/db
+       2d74673..be62780  annex/direct/master -> synced/master
+    ok
+    server$ git-annex status
+    ? .t/repo
+    ? .t/tmprepo0
+    server$
+
+## repo1 -> repo2 fails:
+
+    client$ git-annex sync
+    commit  ok
+    pull repo2 
+    
+    fatal: You have not concluded your merge (MERGE_HEAD exists).
+    Please, commit your changes before you merge.
+    
+    fatal: You have not concluded your merge (MERGE_HEAD exists).
+    Please, commit your changes before you merge.
+    failed
+    git-annex: sync: 1 failed
+    client$ git-annex status
+    client$
+
+Hopefully somebody knows how to fix that.
+
+Regards,
+
+Felix

diff --git a/doc/forum/Improving_the_git-annex_forum.mdwn b/doc/forum/Improving_the_git-annex_forum.mdwn
index 61a36f397..30d7a2973 100644
--- a/doc/forum/Improving_the_git-annex_forum.mdwn
+++ b/doc/forum/Improving_the_git-annex_forum.mdwn
@@ -7,7 +7,7 @@ git-annex has a great documentation, consisting of the wiki articles and the con
 - Bump threads on new replies (that may be a matter of taste)
 - Reply via mail
 - Ability to pin posts (at least I've never seen it)
-- Weird markdown formatting, e.g. marking code by indenting by 4 characters, which is hard to achieve for many lines at once inside a browsers text field (Joey already told me <pre> does job, too)
+- Weird markdown formatting, e.g. marking code by indenting by 4 characters, which is hard to achieve for many lines at once inside a browsers text field (Joey already told me pre does job, too)
 - no private messages
 and more...
 

diff --git a/doc/forum/Improving_the_git-annex_forum.mdwn b/doc/forum/Improving_the_git-annex_forum.mdwn
new file mode 100644
index 000000000..61a36f397
--- /dev/null
+++ b/doc/forum/Improving_the_git-annex_forum.mdwn
@@ -0,0 +1,23 @@
+Dear git-annex community,
+
+git-annex has a great documentation, consisting of the wiki articles and the converted man page. On the other hand, in my opinion there is room for improvement when it comes to the forum. It's lacking quite some features you expect from a modern forum, e.g.
+
+- Slick UI
+- User friendly way to (un)subscribe to threads
+- Bump threads on new replies (that may be a matter of taste)
+- Reply via mail
+- Ability to pin posts (at least I've never seen it)
+- Weird markdown formatting, e.g. marking code by indenting by 4 characters, which is hard to achieve for many lines at once inside a browsers text field (Joey already told me <pre> does job, too)
+- no private messages
+and more...
+
+Therefore I think, that a better forum would significantly improve git-annex's user experience!
+
+I'm hosting a Discourse forum for some time now and think it's quite a decent, user- and admin friendly software. I would be happy to host a forum for git-annex on my own server, e.g. on a subdomain of git-annex.branchable.com.
+
+I had exchanged some mails with Joey about that, he asked me to take this discussion to the forum...
+
+What are your opinions on that matter?
+
+Best,
+Florian

diff --git a/doc/todo/support_.netrc_for_fsck_--from_web.mdwn b/doc/todo/support_.netrc_for_fsck_--from_web.mdwn
new file mode 100644
index 000000000..3a8d4672b
--- /dev/null
+++ b/doc/todo/support_.netrc_for_fsck_--from_web.mdwn
@@ -0,0 +1,10 @@
+On my computer, `git annex get filename --from web` uses wget which does support
+.netrc, but for some reason `git annex fsck filename --from web` does not.
+Instead, I get a message that the content is missing and git annex is "fixing
+location log" (slightly unrelated question, but is there any way to stop it from
+doing this? Just because a URL isn't accessible now, doesn't mean it's content
+is gone permanently).
+
+I do not want to encode my credentials into the URLs (eg.
+username:password@example.com) because my password changes frequently and I would
+have to update all of the URLs.

add news item for git-annex 6.20180112
diff --git a/doc/news/version_6.20171026.mdwn b/doc/news/version_6.20171026.mdwn
deleted file mode 100644
index 83f3d993e..000000000
--- a/doc/news/version_6.20171026.mdwn
+++ /dev/null
@@ -1,14 +0,0 @@
-git-annex 6.20171026 released with [[!toggle text="these changes"]]
-[[!toggleable text="""
-   * Windows: Fix reversion that caused the path used to link
-     to annexed content to include the drive letter and full path, rather
-     than being relative. (`git annex fix` will fix up after this problem).
-   * Windows build fixed, and changed to use stack for more reliable build
-     environment.
-   * Windows: Remove wget from bundle; it needs libraries that are not
-     included, and git for windows includes curl which git-annex will use
-     instead.
-   * Add day to metadata when annex.genmetadata is enabled.
-     Thanks, Sean T Parsons
-   * stack.yaml: Added nix packages section.
-     Thanks, Sean T Parsons"""]]
\ No newline at end of file
diff --git a/doc/news/version_6.20180112.mdwn b/doc/news/version_6.20180112.mdwn
new file mode 100644
index 000000000..d4812400d
--- /dev/null
+++ b/doc/news/version_6.20180112.mdwn
@@ -0,0 +1,23 @@
+git-annex 6.20180112 released with [[!toggle text="these changes"]]
+[[!toggleable text="""
+   * Added inprogress command for accessing files as they are being
+     downloaded.
+   * Fix bug introduced in version 6.20171018 that caused some commands
+     to print out "ok" twice after processing a file.
+   * addurl: When the file youtube-dl will download is already an annexed
+     file, don't download it again and fail to overwrite it, instead just do
+     nothing, like it used to when quvi was used.
+   * addurl: Fix encoding of filename queried from youtube-dl when in
+     --fast mode.
+   * Fix several places where files in .git/annex/ were written with modes
+     that did not take the core.sharedRepository config into account.
+   * Improve startup time for commands that do not operate on remotes,
+     and for tab completion, by not unnessessarily statting paths to
+     remotes, which used to cause eg, spin-up of removable drives.
+   * Added remote.&lt;name&gt;.annex-checkuuid config, which can be set to false
+     to disable the default checking of the uuid of remotes that point to
+     directories. This can be useful to avoid unncessary drive spin-ups and
+     automounting.
+   * git-annex.cabal: Add back custom-setup stanza, so cabal new-build works.
+   * git-annex.cabal: Removed the testsuite build flag; test suite is always
+     included."""]]
\ No newline at end of file