diff --git a/doc/forum/Joey__44___when_would_you_guess_v6_to_be_ready__63__/comment_1_08d3fd14a57b5d4dfb2cac411aba757f._comment b/doc/forum/Joey__44___when_would_you_guess_v6_to_be_ready__63__/comment_1_08d3fd14a57b5d4dfb2cac411aba757f._comment
new file mode 100644
index 000000000..84d0ca100
--- /dev/null
+++ b/doc/forum/Joey__44___when_would_you_guess_v6_to_be_ready__63__/comment_1_08d3fd14a57b5d4dfb2cac411aba757f._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="spwhitton"
+ avatar="http://cdn.libravatar.org/avatar/9c3f08f80e67733fd506c353239569eb"
+ subject="current status of v6"
+ date="2018-04-21T19:47:10Z"
+ content="""
+I too would be grateful to know Joey's view on the state of v6.  I would like to replace my direct mode repos.
+
+I'm not sure that the criteria for v6 being suitable for production is that Joey is willing to set it as the default -- there might be reasons other than stability/bugginess not to set it as the default.
+"""]]

diff --git a/doc/install/Windows/comment_4_c81a36e387ca11050cf8cb977fff98ff._comment b/doc/install/Windows/comment_4_c81a36e387ca11050cf8cb977fff98ff._comment
new file mode 100644
index 000000000..d2d866fd1
--- /dev/null
+++ b/doc/install/Windows/comment_4_c81a36e387ca11050cf8cb977fff98ff._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="grawity@2ea26be48562f66fcb9b66307da72b1e2e37453f"
+ nickname="grawity"
+ avatar="http://cdn.libravatar.org/avatar/7003e967f47003bae82966aa373de8ef"
+ subject="comment 4"
+ date="2018-04-21T09:52:31Z"
+ content="""
+Is 64-bit still a problem now that git-annex has switched over from rsync to its own transport?
+"""]]

diff --git a/doc/forum/released_finder_integration_for_macOS_called_git-annex-turtle/comment_6_8b3555fe33da55c01b199efd492951b0._comment b/doc/forum/released_finder_integration_for_macOS_called_git-annex-turtle/comment_6_8b3555fe33da55c01b199efd492951b0._comment
new file mode 100644
index 000000000..f88587d35
--- /dev/null
+++ b/doc/forum/released_finder_integration_for_macOS_called_git-annex-turtle/comment_6_8b3555fe33da55c01b199efd492951b0._comment
@@ -0,0 +1,37 @@
+[[!comment format=mdwn
+ username="andrew"
+ avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
+ subject="integrating git-annex-turtle into git-annex"
+ date="2018-04-20T18:24:21Z"
+ content="""
+Ooops…, I missed this comment.
+
+I am not exactly sure what the best approach will be, but I have some thoughts. Yes, I can make an example of a folder structure I think could work well. I'll try to explain all the positives and negatives of various approaches below.
+
+The standard for application installation on the mac is to deliver a dmg that when double-clicked opens a small Finder window that contains the application (.app bundle), a link to /Applications and [a background image with an arrow](https://asmaloney.com/2013/07/howto/packaging-a-mac-os-x-application-using-a-dmg/) hinting that users should drag the .app file onto to the /Applications link (which causes Finder to initiate a copy).
+
+Currently, after installing git-annex from the dmg, when you double-click the git-annex.app folder git-annex-webapp is launched.
+
+1) One approach, to bundling git-annex-turtle would be to place git-annex-turtle inside the git-annex.app folder. That would mean, however, users need to ctrl-click then click `Show Package Contents` then browse the file structure until they find git-annex-turtle, then double-click it to launch it. That would be a bit confusing, and not very Mac-y. This could be slightly mitigated by adding an installer for git-annex that asks users if they want to auto-launch git-annex-turtle on login-in, but then we still have the issue of them not being able to re-launch it if they ever quit it. Also, installers are not very popular on the Mac.
+
+2) Another approach would be for the dmg file to contain a folder that contains two items git-annex.app and git-annex-turtle.app. Users would then drag the entire folder to their /Applications folder. Then git-annex-webapp would be launchable (from Finder) by double-clicking on /Applications/git-annex/git-annex.app, and git-annex-turtle would be launch-able by double-clicking on /Applications/git-annex/git-annex-turtle.app. The biggest drawback with this approach is that it would change all of the absolute paths to the binaries inside git-annex.app.
+
+For example, I currently have the following line in my bash_profile:
+
+    export PATH=/Applications/git-annex.app/Contents/MacOS:$PATH
+
+This would need to change to
+
+    export PATH=/Applications/git-annex/git-annex.app/Contents/MacOS:$PATH
+
+I think we probably don't want to make all users have to update all their paths to git-annex binaries.
+
+This approach, of having a folder with multiple apps is quite rare. Users, really expect their /Applications folder to only contain .app files with nice icons that they can double-click to launch things. I could only find a handful of examples of apps that are doing this (all cross-platform apps): Blender, Supercollider, Unity, Adobe (sometimes), Autodesk.
+
+3) Another approach would be to just include both the existing git-annex.app and git-annex-turtle.app (as they are) in the git-annex.dmg file. Then I could create a nice background image that suggests users should drag both git-annex.app and git-annex-turtle.app to their /Applications folder. Then users would have /Applications/git-annex.app and /Applications/git-annex-turtle.app installed. The main drawback here is that users need to know to drag both files to their /Applications folder, and they are slightly more likely to not always upgrade them, and there is the potential for them to just drag git-annex-turtle to /Applications and not git-annex, and wonder why one doesn't work. Or uninstall git-annex at some point, and then again, wonder why git-annex-turtle no longer works.
+
+Currently I am leaning towards this last option. But, I think 2) could work as well, although it would be unfortunate to break people's paths.
+
+
+Andrew
+"""]]

diff --git a/doc/install/OSX/comment_19_040d388c94ee0e537990f0cd782d8554._comment b/doc/install/OSX/comment_19_040d388c94ee0e537990f0cd782d8554._comment
new file mode 100644
index 000000000..3525b1ff9
--- /dev/null
+++ b/doc/install/OSX/comment_19_040d388c94ee0e537990f0cd782d8554._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="andrew"
+ avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
+ subject="git-annex not a cask anymore"
+ date="2018-04-20T13:21:27Z"
+ content="""
+@andrew.grangaard, I just wanted to note that git-annex is [no longer available as a cask](https://github.com/caskroom/homebrew-cask/pull/22706). It is just available as a [formula](http://formulae.brew.sh/formula/git-annex), ie `brew install git-annex` see [Homebrew](http://git-annex.branchable.com/install/OSX/Homebrew/).
+"""]]

diff --git a/doc/devblog/youtube-dl/comment_1_ff062ea66ea50d2f31a92c25017bb135._comment b/doc/devblog/youtube-dl/comment_1_ff062ea66ea50d2f31a92c25017bb135._comment
new file mode 100644
index 000000000..172eee1b2
--- /dev/null
+++ b/doc/devblog/youtube-dl/comment_1_ff062ea66ea50d2f31a92c25017bb135._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="svw"
+ avatar="http://cdn.libravatar.org/avatar/539b90cb79086a2b1d7d3fbe48fd4184"
+ subject="Passing flags to youtube-dl"
+ date="2018-04-20T05:57:39Z"
+ content="""
+Joey - is there some way to pass flags to youtube-dl?
+
+For example the `--extract-audio` flag allows the extraction of just the audio stream.
+"""]]

diff --git a/doc/bugs/Remote_webapp_on_debian_stretch_not_responding/comment_1_e2a0d0f458d263eae15640f36d59f071._comment b/doc/bugs/Remote_webapp_on_debian_stretch_not_responding/comment_1_e2a0d0f458d263eae15640f36d59f071._comment
new file mode 100644
index 000000000..8ba6d9946
--- /dev/null
+++ b/doc/bugs/Remote_webapp_on_debian_stretch_not_responding/comment_1_e2a0d0f458d263eae15640f36d59f071._comment
@@ -0,0 +1,59 @@
+[[!comment format=mdwn
+ username="moaxey"
+ avatar="http://cdn.libravatar.org/avatar/06abae5b84ce9bbcd39c1cbf49817d29"
+ subject="I also found this log"
+ date="2018-04-17T22:52:15Z"
+ content="""
+```
+   Here's everything that has been logged by the git-annex assistant, as
+   well as by programs it has run.
+[2018-04-17 21:55:06.712567732] main: starting assistant version 6.20170101
+[2018-04-17 21:55:06.754681296] Cronner: You should enable consistency checking
+to protect your data.
+
+  No known network monitor available through dbus; falling back to polling
+
+  No known volume monitor available through dbus; falling back to mtab polling
+(scanning...) [2018-04-17 21:55:06.920804804] Watcher: Performing startup scan
+(started...)
+WebApp crashed: unable to bind to local socket
+CallStack (from HasCallStack):
+  error, called at ./Utility/WebApp.hs:124:19 in main:Utility.WebApp
+[2018-04-17 21:55:07.19622235] WebApp: warning WebApp crashed: unable to bind to
+ local socket
+CallStack (from HasCallStack):
+  error, called at ./Utility/WebApp.hs:124:19 in main:Utility.WebApp
+[2018-04-17 21:57:06.821805552] main: starting assistant version 6.20170101
+[2018-04-17 21:57:06.858432032] Cronner: You should enable consistency checking
+to protect your data.
+
+  No known network monitor available through dbus; falling back to polling
+(scanning...) [2018-04-17 21:57:07.025115165] Watcher: Performing startup scan
+(started...)
+
+  No known volume monitor available through dbus; falling back to mtab polling
+WebApp crashed: unable to bind to local socket
+CallStack (from HasCallStack):
+  error, called at ./Utility/WebApp.hs:124:19 in main:Utility.WebApp
+[2018-04-17 21:57:07.243030677] WebApp: warning WebApp crashed: unable to bind t
+o local socket
+CallStack (from HasCallStack):
+  error, called at ./Utility/WebApp.hs:124:19 in main:Utility.WebApp
+[2018-04-17 21:59:07.137839178] main: starting assistant version 6.20170101
+[2018-04-17 21:59:07.177856054] Cronner: You should enable consistency checking
+to protect your data.
+
+  No known network monitor available through dbus; falling back to polling
+
+  No known volume monitor available through dbus; falling back to mtab polling
+
+(scanning...) [2018-04-17 22:24:10.365052622] Watcher: Performing startup scan
+(started...)
+(merging synced/git-annex into git-annex...)
+(recording state in git...)
+
+     Performed startup scan
+
+```
+
+"""]]

diff --git a/doc/bugs/Remote_webapp_on_debian_stretch_not_responding.mdwn b/doc/bugs/Remote_webapp_on_debian_stretch_not_responding.mdwn
new file mode 100644
index 000000000..9109ae346
--- /dev/null
+++ b/doc/bugs/Remote_webapp_on_debian_stretch_not_responding.mdwn
@@ -0,0 +1,16 @@
+In a fresh repository installation on debian stretch (arm) (git-annex version: 6.20170101)
+the remote webapp will start and present the dashboard page,
+however any request made by clicking any link does not receive a response.
+
+I have a workaround: to ssh into my server and access the webapp locally using lynx browser, and that does seem to work well.
+
+It presents me with a warning:
+
+warning
+
+   WebApp crashed: xdg-open: createProcess: runInteractiveProcess: exec:
+   does not exist (No such file or directory)
+
+I'm not getting any errors, just the requests not returning responses.
+
+I'm pretty new to this software.

diff --git a/doc/install/Android/comment_14_19909ddf4e7e6da0e814e165c5c9b44a._comment b/doc/install/Android/comment_14_19909ddf4e7e6da0e814e165c5c9b44a._comment
new file mode 100644
index 000000000..b499cdddf
--- /dev/null
+++ b/doc/install/Android/comment_14_19909ddf4e7e6da0e814e165c5c9b44a._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="lykos@d125a37d89b1cfac20829f12911656c40cb70018"
+ nickname="lykos"
+ avatar="http://cdn.libravatar.org/avatar/085df7b04d3408ba23c19f9c49be9ea2"
+ subject="comment 14"
+ date="2018-04-17T08:38:32Z"
+ content="""
+There is now an [adb special remote](https://git-annex.branchable.com/todo/adb_special_remote).
+"""]]

diff --git a/doc/bugs/fatal__58___git-write-tree__58___error_building_trees/comment_4_f3af2244be7bf7461882b0e9e527922a._comment b/doc/bugs/fatal__58___git-write-tree__58___error_building_trees/comment_4_f3af2244be7bf7461882b0e9e527922a._comment
new file mode 100644
index 000000000..f14e42d1b
--- /dev/null
+++ b/doc/bugs/fatal__58___git-write-tree__58___error_building_trees/comment_4_f3af2244be7bf7461882b0e9e527922a._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 4"
+ date="2018-04-17T03:52:55Z"
+ content="""
+thank you Joey.  It seems to be healed now (after ` rm ../.git/annex/index*`, since there also was .lck file, just killed them together, annex fsck didn't raise any alarm)
+"""]]

diff --git a/doc/bugs/android__58___cannot_link_executable/comment_7_1d37b251f1f9bf31647eb13322b92cce._comment b/doc/bugs/android__58___cannot_link_executable/comment_7_1d37b251f1f9bf31647eb13322b92cce._comment
new file mode 100644
index 000000000..51d6a1f12
--- /dev/null
+++ b/doc/bugs/android__58___cannot_link_executable/comment_7_1d37b251f1f9bf31647eb13322b92cce._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 7"""
+ date="2018-04-16T23:49:20Z"
+ content="""
+Even with gcc and ld wrapped to pass `-pie` and strip `-no-pie`,
+ghc is not producing PIE binaries. Verified with 
+"readelf -s git-annex | grep main", and the address for
+main is a full address not an offset.
+
+The same android gcc toolchain does produce 
+PIE executables by default when building C code.
+
+Unclear why..
+"""]]

diff --git a/doc/bugs/fatal__58___git-write-tree__58___error_building_trees/comment_3_6bd0caa81cb87d30355cc8ed9fc6d958._comment b/doc/bugs/fatal__58___git-write-tree__58___error_building_trees/comment_3_6bd0caa81cb87d30355cc8ed9fc6d958._comment
new file mode 100644
index 000000000..389b9d79f
--- /dev/null
+++ b/doc/bugs/fatal__58___git-write-tree__58___error_building_trees/comment_3_6bd0caa81cb87d30355cc8ed9fc6d958._comment
@@ -0,0 +1,20 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2018-04-16T20:24:33Z"
+ content="""
+@yarikoptic, yes, the filename there is a git-annex branch filename.
+So, .git/annex/index is where the problem lies in your case.
+
+(Probably in the original bug reporter's case too; they trimmed out the
+important part of the error message it seems. git fsck won't detect a
+problem with .git/annex/index.)
+
+The best thing to do in this case is to delete .git/annex/index,
+since git-annex can always recover from that by creating a new one that
+does not point to git objects that have mysteriously vanished from your
+disk.
+
+Since information that was being written to the git-annex branch has
+apparently been lost, you'll also want to `git annex fsck --fast`
+"""]]

diff --git a/doc/bugs/android__58___cannot_link_executable/comment_6_059393465b26596c0af8cfaaef46c11d._comment b/doc/bugs/android__58___cannot_link_executable/comment_6_059393465b26596c0af8cfaaef46c11d._comment
new file mode 100644
index 000000000..8563f0d70
--- /dev/null
+++ b/doc/bugs/android__58___cannot_link_executable/comment_6_059393465b26596c0af8cfaaef46c11d._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 6"""
+ date="2018-04-16T20:36:19Z"
+ content="""
+Apparently this might have to do with ghc passing -no-pie to GCC,
+and they added a -fPIE flag to request a PIE.
+<https://ghc.haskell.org/trac/ghc/ticket/13702>
+
+Since the git-annex for android autobuilder uses a hacked up ghc
+cross compiler for android, which is quite out of date, that would need to
+be updated, as well as probably refeshing all the cabal library patches and
+hacks for android.
+"""]]

diff --git a/CHANGELOG b/CHANGELOG
index b9701f354..126a80dc6 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -7,6 +7,8 @@ git-annex (6.20180410) UNRELEASED; urgency=medium
     Note that it's still allowed to move the content of a file
     from one repository to another when numcopies is not satisfied, as long
     as the move does not result in there being fewer copies.
+  * Fix mangling of --json output of utf-8 characters when not
+    running in a utf-8 locale.
 
  -- Joey Hess <id@joeyh.name>  Mon, 09 Apr 2018 14:03:28 -0400
 
diff --git a/Command/Info.hs b/Command/Info.hs
index 6ec6a1f71..cc9c1b5fe 100644
--- a/Command/Info.hs
+++ b/Command/Info.hs
@@ -11,9 +11,8 @@ module Command.Info where
 
 import "mtl" Control.Monad.State.Strict
 import qualified Data.Map.Strict as M
-import qualified Data.Text as T
+import qualified Data.Vector as V
 import Data.Ord
-import Data.Aeson hiding (json)
 
 import Command
 import qualified Git
@@ -34,6 +33,7 @@ import Config
 import Git.Config (boolConfig)
 import qualified Git.LsTree as LsTree
 import Utility.Percentage
+import Utility.Aeson hiding (json)
 import Types.Transfer
 import Logs.Transfer
 import Types.Key
@@ -283,7 +283,7 @@ simpleStat desc getval = stat desc $ json id getval
 nostat :: Stat
 nostat = return Nothing
 
-json :: ToJSON j => (j -> String) -> StatState j -> String -> StatState String
+json :: ToJSON' j => (j -> String) -> StatState j -> String -> StatState String
 json fmt a desc = do
 	j <- a
 	lift $ maybeShowJSON $ JSONChunk [(desc, j)]
@@ -422,7 +422,7 @@ transfer_list :: Stat
 transfer_list = stat desc $ nojson $ lift $ do
 	uuidmap <- Remote.remoteMap id
 	ts <- getTransfers
-	maybeShowJSON $ JSONChunk [(desc, map (uncurry jsonify) ts)]
+	maybeShowJSON $ JSONChunk [(desc, V.fromList $ map (uncurry jsonify) ts)]
 	return $ if null ts
 		then "none"
 		else multiLine $
@@ -438,11 +438,11 @@ transfer_list = stat desc $ nojson $ lift $ do
 		, maybe (fromUUID $ transferUUID t) Remote.name $
 			M.lookup (transferUUID t) uuidmap
 		]
-	jsonify t i = object $ map (\(k, v) -> (T.pack k, v)) $
-		[ ("transfer", toJSON (formatDirection (transferDirection t)))
-		, ("key", toJSON (key2file (transferKey t)))
-		, ("file", toJSON afile)
-		, ("remote", toJSON (fromUUID (transferUUID t)))
+	jsonify t i = object $ map (\(k, v) -> (packString k, v)) $
+		[ ("transfer", toJSON' (formatDirection (transferDirection t)))
+		, ("key", toJSON' (transferKey t))
+		, ("file", toJSON' afile)
+		, ("remote", toJSON' (fromUUID (transferUUID t)))
 		]
 	  where
 		AssociatedFile afile = associatedFile i
@@ -476,10 +476,13 @@ numcopies_stats :: Stat
 numcopies_stats = stat "numcopies stats" $ json fmt $
 	calc <$> (maybe M.empty numCopiesVarianceMap <$> cachedNumCopiesStats)
   where
-	calc = map (\(variance, count) -> (show variance, count)) 
+	calc = V.fromList
+		. map (\(variance, count) -> (show variance, count)) 
 		. sortBy (flip (comparing fst))
 		. M.toList
-	fmt = multiLine . map (\(variance, count) -> "numcopies " ++ variance ++ ": " ++ show count)
+	fmt = multiLine 
+		. map (\(variance, count) -> "numcopies " ++ variance ++ ": " ++ show count)
+		. V.toList
 
 reposizes_stats :: Stat
 reposizes_stats = stat desc $ nojson $ do
diff --git a/Command/MetaData.hs b/Command/MetaData.hs
index ef3f1da9a..282b7fda0 100644
--- a/Command/MetaData.hs
+++ b/Command/MetaData.hs
@@ -14,12 +14,12 @@ import Logs.MetaData
 import Annex.WorkTree
 import Messages.JSON (JSONActionItem(..))
 import Types.Messages
+import Utility.Aeson
 
 import qualified Data.Set as S
 import qualified Data.Map as M
 import qualified Data.Text as T
 import qualified Data.ByteString.Lazy.UTF8 as BU
-import Data.Aeson
 import Control.Concurrent
 
 cmd :: Command
@@ -115,7 +115,7 @@ perform c o k = case getSet o of
 cleanup :: Key -> CommandCleanup
 cleanup k = do
 	m <- getCurrentMetaData k
-	let Object o = toJSON (MetaDataFields m)
+	let Object o = toJSON' (MetaDataFields m)
 	maybeShowJSON $ AesonObject o
 	showLongNote $ unlines $ concatMap showmeta $
 		map unwrapmeta (fromMetaData m)
@@ -129,8 +129,8 @@ cleanup k = do
 newtype MetaDataFields = MetaDataFields MetaData
 	deriving (Show)
 
-instance ToJSON MetaDataFields where
-	toJSON (MetaDataFields m) = object [ (fieldsField, toJSON m) ]
+instance ToJSON' MetaDataFields where
+	toJSON' (MetaDataFields m) = object [ (fieldsField, toJSON' m) ]
 
 instance FromJSON MetaDataFields where
 	parseJSON (Object v) = do
diff --git a/Command/Whereis.hs b/Command/Whereis.hs
index fddb2b5c6..b14e231c1 100644
--- a/Command/Whereis.hs
+++ b/Command/Whereis.hs
@@ -15,6 +15,7 @@ import Remote.Web (getWebUrls)
 import Annex.UUID
 
 import qualified Data.Map as M
+import qualified Data.Vector as V
 
 cmd :: Command
 cmd = noCommit $ withGlobalOptions [jsonOptions, annexedMatchingOptions] $
@@ -77,7 +78,7 @@ perform remotemap key = do
 	untrustedheader = "The following untrusted locations may also have copies:\n"
 	ppwhereis h ls urls = do
 		descm <- uuidDescriptions
-		let urlvals = map (\(u, us) -> (u, Just us)) $
+		let urlvals = map (\(u, us) -> (u, Just (V.fromList us))) $
 			filter (\(u,_) -> u `elem` ls) urls
 		prettyPrintUUIDsWith (Just "urls") h descm (const Nothing) urlvals
 
diff --git a/Key.hs b/Key.hs
index 44e9acea4..ade012a4b 100644
--- a/Key.hs
+++ b/Key.hs
@@ -22,7 +22,6 @@ module Key (
 	prop_isomorphic_key_decode
 ) where
 
-import Data.Aeson
 import Data.Char
 import qualified Data.Text as T
 
@@ -30,6 +29,7 @@ import Common
 import Types.Key
 import Utility.QuickCheck
 import Utility.Bloom
+import Utility.Aeson
 import qualified Utility.SimpleProtocol as Proto
 
 stubKey :: Key
@@ -155,8 +155,8 @@ instance Hashable Key where
 	hashIO32 = hashIO32 . key2file
 	hashIO64 = hashIO64 . key2file
 
-instance ToJSON Key where
-	toJSON = toJSON . key2file
+instance ToJSON' Key where
+	toJSON' = toJSON' . key2file
 
 instance FromJSON Key where
 	parseJSON (String t) = maybe mempty pure $ file2key $ T.unpack t
diff --git a/Messages/JSON.hs b/Messages/JSON.hs
index 897eb8cbf..e63a263e3 100644
--- a/Messages/JSON.hs
+++ b/Messages/JSON.hs
@@ -26,12 +26,10 @@ module Messages.JSON (
 	JSONActionItem(..),
 ) where
 
-import Data.Aeson
 import Control.Applicative
 import qualified Data.Map as M
-import qualified Data.Text as T
 import qualified Data.Vector as V
-import qualified Data.ByteString.Lazy as B
+import qualified Data.ByteString.Lazy as L
 import qualified Data.HashMap.Strict as HM
 import System.IO
 import System.IO.Unsafe (unsafePerformIO)
@@ -44,6 +42,7 @@ import Types.Messages
 import Key
 import Utility.Metered
 import Utility.Percentage
+import Utility.Aeson

(Diff truncated)

diff --git a/doc/install/Android/comment_13_169025cf0b415d5a45dd89856e53cca9._comment b/doc/install/Android/comment_13_169025cf0b415d5a45dd89856e53cca9._comment
new file mode 100644
index 000000000..0564f974c
--- /dev/null
+++ b/doc/install/Android/comment_13_169025cf0b415d5a45dd89856e53cca9._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="hobbes@b2cacef69071743c3a831e60511062f7e014e52f"
+ nickname="hobbes"
+ avatar="http://cdn.libravatar.org/avatar/44b70169c4d862b3619812c360aa8f1e"
+ subject="Modern Android?"
+ date="2018-04-16T19:28:09Z"
+ content="""
+Hey there!
+
+It's been a while since the android app has worked. Are there any plans to get it working on modern Android versions? I assume it's pretty low demand, but it would make git-annex just so much more useful for me.
+
+To clarify, I am experiencing this issue, as will everybody using Android >=6:
+
+https://git-annex.branchable.com/bugs/android__58___cannot_link_executable/
+https://developer.android.com/about/versions/marshmallow/android-6.0-changes.html#behavior-runtime
+
+I'm not certain, but I think it might just be a matter of compiling git-annex with this flag and releasing a new APK?
+
+https://ghc.haskell.org/trac/ghc/ticket/13702
+
+I'd be delighted to test if given an APK! Thanks again for the lovely git magic :).
+
+
+
+
+"""]]

diff --git a/doc/bugs/json_should_be_utf8_regardless_of_locale.mdwn b/doc/bugs/json_should_be_utf8_regardless_of_locale.mdwn
index c0bc8fa79..66d0de224 100644
--- a/doc/bugs/json_should_be_utf8_regardless_of_locale.mdwn
+++ b/doc/bugs/json_should_be_utf8_regardless_of_locale.mdwn
@@ -1,3 +1,7 @@
 json is defined as always utf-8. However, when LANG=C,
 git-annex --json currently outputs "file":"���������"
 instead of "file":"äöü東" for that utf-8 filename. --[[Joey]]
+
+(Note that git-annex can operate on non-utf8 filenames; it's not defined
+what the json contains then, which might or might not be considered a bug
+but this is not about that.)

diff --git a/doc/bugs/json_should_be_utf8_regardless_of_locale.mdwn b/doc/bugs/json_should_be_utf8_regardless_of_locale.mdwn
new file mode 100644
index 000000000..c0bc8fa79
--- /dev/null
+++ b/doc/bugs/json_should_be_utf8_regardless_of_locale.mdwn
@@ -0,0 +1,3 @@
+json is defined as always utf-8. However, when LANG=C,
+git-annex --json currently outputs "file":"���������"
+instead of "file":"äöü東" for that utf-8 filename. --[[Joey]]

diff --git a/doc/bugs/fatal__58___git-write-tree__58___error_building_trees/comment_2_4412fe19280551109bed2fde7a921079._comment b/doc/bugs/fatal__58___git-write-tree__58___error_building_trees/comment_2_4412fe19280551109bed2fde7a921079._comment
new file mode 100644
index 000000000..ba0aec5c8
--- /dev/null
+++ b/doc/bugs/fatal__58___git-write-tree__58___error_building_trees/comment_2_4412fe19280551109bed2fde7a921079._comment
@@ -0,0 +1,47 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="old issue bites back"
+ date="2018-04-16T05:53:00Z"
+ content="""
+Sorry to bring the old issue up but we just ran into the same situation, and I wonder if any idea on how to mediate -- unlike other cases where it happens due to corrupt index, here I guess it goes about \"git-annex\" branch (or its some kind of local index cache?)
+
+[[!format sh \"\"\"
+nastase@head1:~/attention/raw_bids/stimuli$ git add README.md
+nastase@head1:~/attention/raw_bids/stimuli$ git commit -m 'README for stimuli' README.md 
+(recording state in git...)
+error: invalid object 100644 fd601b9139518bd6737ad20e55198dd1854fa0f9 for '001/92f/MD5E-s1413--241f0946a6aecfe12a7c031cb2339f9d.pklz.log'
+fatal: git-write-tree: error building trees
+git-annex: failed to read sha from git write-tree
+CallStack (from HasCallStack):
+  error, called at ./Git/Sha.hs:18:15 in main:Git.Sha
+
+nastase@head1:~/attention/raw_bids/stimuli$ git annex fsck
+curl: (22) The requested URL returned error: 404 Not Found
+
+  Remote openfmri not usable by git-annex; setting annex-ignore
+fsck bird_eating_1.mp4 (checksum...) ok
+...
+fsck ungulate_swimming_2.mp4 (checksum...) ok
+(recording state in git...)
+error: invalid object 100644 fd601b9139518bd6737ad20e55198dd1854fa0f9 for '001/92f/MD5E-s1413--241f0946a6aecfe12a7c031cb2339f9d.pklz.log'
+fatal: git-write-tree: error building trees
+git-annex: failed to read sha from git write-tree
+CallStack (from HasCallStack):
+  error, called at ./Git/Sha.hs:18:15 in main:Git.Sha
+
+nastase@head1:~/attention/raw_bids/stimuli$ git --version
+git version 2.13.0.rc1.294.g07d810a77f
+
+nastase@head1:~/attention/raw_bids/stimuli$ git annex version
+git-annex version: 6.20180329+gitga5fe62bed-1~ndall+1
+
+\"\"\"]]
+
+NB The issue came earlier and I have just upgraded git from elderly 2.7.0 and git-annex-standalone from a few months back.
+
+Any advice would be welcome
+
+[[!meta author=yoh]]
+
+"""]]

diff --git a/doc/bugs/Assertion___96__cnt___60_____40__sizeof___40____95__nl__95__value__95__type__95__LC__95__TIME__41_____47___sizeof___40____95__nl__95__value__95__type__95__LC__95__TIME__91__0__93____41____41____39___failed..mdwn b/doc/bugs/Assertion___96__cnt___60_____40__sizeof___40____95__nl__95__value__95__type__95__LC__95__TIME__41_____47___sizeof___40____95__nl__95__value__95__type__95__LC__95__TIME__91__0__93____41____41____39___failed..mdwn
index b7da78d94..fd9f32e08 100644
--- a/doc/bugs/Assertion___96__cnt___60_____40__sizeof___40____95__nl__95__value__95__type__95__LC__95__TIME__41_____47___sizeof___40____95__nl__95__value__95__type__95__LC__95__TIME__91__0__93____41____41____39___failed..mdwn
+++ b/doc/bugs/Assertion___96__cnt___60_____40__sizeof___40____95__nl__95__value__95__type__95__LC__95__TIME__41_____47___sizeof___40____95__nl__95__value__95__type__95__LC__95__TIME__91__0__93____41____41____39___failed..mdwn
@@ -1,8 +1,7 @@
 ### Please describe the problem.
 
-Synology NAS, 32-bit prebuilt version
-
 git annex version
+
 sh: loadlocale.c:129: _nl_intern_locale_data: Assertion `cnt < (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))' failed.
 Aborted
 
@@ -12,8 +11,23 @@ git annex version
 
 ### What version of git-annex are you using? On what operating system?
 
+[[!format sh """
+
+Synology NAS, 32-bit prebuilt version
+
 wget https://downloads.kitenet.net/git-annex/linux/current/git-annex-standalone-i386.tar.gz
 --2018-04-08 04:26:29--  https://downloads.kitenet.net/git-annex/linux/current/git-annex-standalone-i386.tar.gz
+Resolving downloads.kitenet.net... 66.228.36.95, 2600:3c03::f03c:91ff:fe73:b0d2
+Connecting to downloads.kitenet.net|66.228.36.95|:443... connected.
+HTTP request sent, awaiting response... 200 OK
+Length: 60787901 (58M) [application/x-gzip]
+Saving to: 'git-annex-standalone-i386.tar.gz'
+
+100%[================================================================================================================================================================================================================================================>] 60,787,901  1.52MB/s   in 39s    
+
+2018-04-08 04:27:09 (1.49 MB/s) - 'git-annex-standalone-i386.tar.gz' saved [60787901/60787901]
+
+"""]]
 
 ### Please provide any additional information below.
 

diff --git a/doc/bugs/Assertion___96__cnt___60_____40__sizeof___40____95__nl__95__value__95__type__95__LC__95__TIME__41_____47___sizeof___40____95__nl__95__value__95__type__95__LC__95__TIME__91__0__93____41____41____39___failed..mdwn b/doc/bugs/Assertion___96__cnt___60_____40__sizeof___40____95__nl__95__value__95__type__95__LC__95__TIME__41_____47___sizeof___40____95__nl__95__value__95__type__95__LC__95__TIME__91__0__93____41____41____39___failed..mdwn
new file mode 100644
index 000000000..b7da78d94
--- /dev/null
+++ b/doc/bugs/Assertion___96__cnt___60_____40__sizeof___40____95__nl__95__value__95__type__95__LC__95__TIME__41_____47___sizeof___40____95__nl__95__value__95__type__95__LC__95__TIME__91__0__93____41____41____39___failed..mdwn
@@ -0,0 +1,30 @@
+### Please describe the problem.
+
+Synology NAS, 32-bit prebuilt version
+
+git annex version
+sh: loadlocale.c:129: _nl_intern_locale_data: Assertion `cnt < (sizeof (_nl_value_type_LC_TIME) / sizeof (_nl_value_type_LC_TIME[0]))' failed.
+Aborted
+
+### What steps will reproduce the problem?
+
+git annex version
+
+### What version of git-annex are you using? On what operating system?
+
+wget https://downloads.kitenet.net/git-annex/linux/current/git-annex-standalone-i386.tar.gz
+--2018-04-08 04:26:29--  https://downloads.kitenet.net/git-annex/linux/current/git-annex-standalone-i386.tar.gz
+
+### Please provide any additional information below.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+

diff --git a/doc/devblog/day_496__move_numcopies_safety_revisited.mdwn b/doc/devblog/day_496__move_numcopies_safety_revisited.mdwn
new file mode 100644
index 000000000..9d96e0cdd
--- /dev/null
+++ b/doc/devblog/day_496__move_numcopies_safety_revisited.mdwn
@@ -0,0 +1,12 @@
+After talking it over in [[bugs/move_violates_numcopies]], we found a nicer
+compromise for `git annex move`. Rather than strictly enforcing numcopies,
+it avoids making any bad situations worse. For example, when there's
+only one copy of a file, it can be moved even if numcopies is higher.
+But, when numcopies is 2 and the source and destination repos have a copy,
+move will not drop from the source repo, since that would make it worse.
+
+Implemented that today. While doing so I got bit by the inverted Ord
+instance for TrustLevel, so spent a while cleaning that up.
+
+Today's work was sponsored by Jake Vosloo on
+[Patreon](https://patreon.com/joeyh/).

diff --git a/CHANGELOG b/CHANGELOG
index 2440cb247..b9701f354 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,8 +1,12 @@
 git-annex (6.20180410) UNRELEASED; urgency=medium
 
-  * move: Added --safe option, which makes move honor numcopies settings.
-    Also --unsafe enables the default behavior, anticipating that the
-    default may one day change.
+  * move: Now takes numcopies configuration, and required content
+    configuration into account, and refuses to reduce the current
+    number of copies of a file, or remove content that a repository
+    requires. --force can override these checks.
+    Note that it's still allowed to move the content of a file
+    from one repository to another when numcopies is not satisfied, as long
+    as the move does not result in there being fewer copies.
 
  -- Joey Hess <id@joeyh.name>  Mon, 09 Apr 2018 14:03:28 -0400
 
diff --git a/Command/Move.hs b/Command/Move.hs
index df97b7969..43df414d2 100644
--- a/Command/Move.hs
+++ b/Command/Move.hs
@@ -15,6 +15,7 @@ import qualified Remote
 import Annex.UUID
 import Annex.Transfer
 import Logs.Presence
+import Logs.Trust
 import Annex.NumCopies
 
 import System.Log.Logger (debugM)
@@ -28,7 +29,7 @@ cmd = withGlobalOptions [jobsOption, jsonOptions, jsonProgressOption, annexedMat
 data MoveOptions = MoveOptions
 	{ moveFiles :: CmdParams
 	, fromToOptions :: FromToHereOptions
-	, removeWhenOptions :: RemoveWhen
+	, removeWhen :: RemoveWhen
 	, keyOptions :: Maybe KeyOptions
 	, batchOption :: BatchMode
 	}
@@ -37,7 +38,7 @@ optParser :: CmdParamsDesc -> Parser MoveOptions
 optParser desc = MoveOptions
 	<$> cmdParams desc
 	<*> parseFromToHereOptions
-	<*> parseRemoveWhenOptions
+	<*> pure RemoveSafe
 	<*> optional (parseKeyOptions <|> parseFailedTransfersOption)
 	<*> parseBatchOption
 
@@ -45,34 +46,20 @@ instance DeferredParseClass MoveOptions where
 	finishParse v = MoveOptions
 		<$> pure (moveFiles v)
 		<*> finishParse (fromToOptions v)
-		<*> pure (removeWhenOptions v)
+		<*> pure (removeWhen v)
 		<*> pure (keyOptions v)
 		<*> pure (batchOption v)
 
-data RemoveWhen = RemoveSafe | RemoveUnsafe | RemoveNever
+data RemoveWhen = RemoveSafe | RemoveNever
 	deriving (Show, Eq)
 
-parseRemoveWhenOptions :: Parser RemoveWhen
-parseRemoveWhenOptions = 
-	flag' RemoveSafe
-		( long "safe"
-		<> short 's'
-		<> help "preserve numcopies"
-		)
-	<|> flag' RemoveUnsafe
-		(long "unsafe"
-		<> short 'u'
-		<> help "do not preserve numcopies (default)"
-		)
-	<|> pure RemoveUnsafe
-
 seek :: MoveOptions -> CommandSeek
 seek o = allowConcurrentOutput $ do
-	let go = whenAnnexed $ start (fromToOptions o) (removeWhenOptions o)
+	let go = whenAnnexed $ start (fromToOptions o) (removeWhen o)
 	case batchOption o of
 		Batch -> batchInput Right (batchCommandAction . go)
 		NoBatch -> withKeyOptions (keyOptions o) False
-			(startKey (fromToOptions o) (removeWhenOptions o))
+			(startKey (fromToOptions o) (removeWhen o))
 			(withFilesInGit go)
 			=<< workTreeItems (moveFiles o)
 
@@ -141,60 +128,58 @@ toPerform dest removewhen key afile fastcheck isthere =
 				upload (Remote.uuid dest) key afile stdRetry $
 					Remote.storeKey dest key afile
 			if ok
-				then finish $
+				then finish False $
 					Remote.logStatus dest key InfoPresent
 				else do
 					when fastcheck $
 						warning "This could have failed because --fast is enabled."
 					stop
-		Right True -> finish $
+		Right True -> finish True $
 			unlessM (expectedPresent dest key) $
 				Remote.logStatus dest key InfoPresent
   where
-	finish :: Annex () -> CommandPerform
-	finish setpresentremote = case removewhen of
+	finish deststartedwithcopy setpresentremote = case removewhen of
 		RemoveNever -> do
 			setpresentremote
 			next $ return True
-		_ -> lockContentForRemoval key $ \contentlock -> do
-			numcopies <- case removewhen of
-				RemoveUnsafe -> pure (NumCopies 1)
-				_ -> getAssociatedFileNumCopies afile
-			u <- getUUID
-			let drophere proof = do
-				liftIO $ debugM "drop" $ unwords
-					[ "Dropping from here"
-					, "proof:"
-					, show proof
-					]
-				-- Drop content before updating location logs,
-				-- in case disk space is very low this frees
-				-- up space before writing data to disk.
-				removeAnnex contentlock
-				next $ do
-					setpresentremote
-					Command.Drop.cleanupLocal key
-			let faileddrophere = do
-				warning "Not enough copies exist to drop from here (use --unsafe to avoid this check)"
-				next $ do
-					setpresentremote
-					return True
-			(tocheck, verified) <- verifiableCopies key [u]
-			verifyEnoughCopiesToDrop "" key (Just contentlock)
-				 numcopies [] verified
-				 (UnVerifiedRemote dest : tocheck)
-				 drophere faileddrophere
+		RemoveSafe -> lockContentForRemoval key $ \contentlock -> do
+			srcuuid <- getUUID
+			let destuuid = Remote.uuid dest
+			willDropMakeItWorse srcuuid destuuid deststartedwithcopy key afile >>= \case
+				DropAllowed -> drophere setpresentremote contentlock "moved"
+				DropCheckNumCopies -> do
+					numcopies <- getAssociatedFileNumCopies afile
+					(tocheck, verified) <- verifiableCopies key [srcuuid]
+					verifyEnoughCopiesToDrop "" key (Just contentlock)
+						 numcopies [srcuuid] verified
+						 (UnVerifiedRemote dest : tocheck)
+						 (drophere setpresentremote contentlock . showproof)
+						 (faileddrophere setpresentremote)
+				DropWorse -> faileddrophere setpresentremote
+	showproof proof = "proof: " ++ show proof
+	drophere setpresentremote contentlock reason = do
+		liftIO $ debugM "move" $ unwords
+			[ "Dropping from here"
+			, "(" ++ reason ++ ")"
+			]
+		-- Drop content before updating location logs,
+		-- in case disk space is very low this frees
+		-- up space before writing data to disk.
+		removeAnnex contentlock
+		next $ do
+			() <- setpresentremote
+			Command.Drop.cleanupLocal key
+	faileddrophere setpresentremote = do
+		showLongNote "(Use --force to override this check, or adjust numcopies.)"
+		showLongNote "Content not dropped from here."
+		next $ do
+			() <- setpresentremote
+			return False
 
-{- Moves (or copies) the content of an annexed file from a remote
- - to the current repository.
- -
- - If the current repository already has the content, it is still removed
- - from the remote.
- -}
 fromStart :: RemoveWhen -> AssociatedFile -> Key -> ActionItem -> Remote -> CommandStart
-fromStart removewhen afile key ai src
-	| removewhen == RemoveNever = stopUnless (not <$> inAnnex key) go
-	| otherwise = go
+fromStart removewhen afile key ai src = case removewhen of
+	RemoveNever -> stopUnless (not <$> inAnnex key) go
+	RemoveSafe -> go
   where
 	go = stopUnless (fromOk src key) $ do
 		showMoveAction removewhen key ai
@@ -218,49 +203,50 @@ fromPerform :: Remote -> RemoveWhen -> Key -> AssociatedFile -> CommandPerform
 fromPerform src removewhen key afile = do
 	showAction $ "from " ++ Remote.name src
 	ifM (inAnnex key)
-		( dispatch removewhen True
-		, dispatch removewhen =<< go
+		( dispatch removewhen True True
+		, dispatch removewhen False =<< go
 		)
   where
 	go = notifyTransfer Download afile $ 

(Diff truncated)

diff --git a/doc/tips/deleting_unwanted_files.mdwn b/doc/tips/deleting_unwanted_files.mdwn
index b326b537e..8c09b42aa 100644
--- a/doc/tips/deleting_unwanted_files.mdwn
+++ b/doc/tips/deleting_unwanted_files.mdwn
@@ -8,7 +8,7 @@ In this method, you just remove annexed files whenever you want, and commit the
 
 In an indirect mode repo, you can do this the same way you would in a regular git repository. For example, `git rm foo; git commit -m "removed foo"`. This leaves the contents of the files still in the annex, not really deleted yet.
 
-If you have a direct mode repo, you can't run `git rm` on it. Instead, you can just delete files using `rm` or your file manager, and then run `git annex sync` to commit the deletion. That will delete the file's content from your disk. Even if it's the only copy of the file!
+If you have a direct mode repo, you can't run `git rm` in it. Instead, you can just delete files using `rm` or your file manager, and then run `git annex sync` to commit the deletion. That will delete the file's content from your disk. Even if it's the only copy of the file!
 
 Either way, deleting files can leave some garbage lying around in either the local repository, or other repositories that contained a copy of the content of the file you deleted. Eventually you'll want to free up some disk space used by one of these repositories, and then it's time to take out the garbage.
 

diff --git a/doc/tips/deleting_unwanted_files.mdwn b/doc/tips/deleting_unwanted_files.mdwn
index 8c09b42aa..b326b537e 100644
--- a/doc/tips/deleting_unwanted_files.mdwn
+++ b/doc/tips/deleting_unwanted_files.mdwn
@@ -8,7 +8,7 @@ In this method, you just remove annexed files whenever you want, and commit the
 
 In an indirect mode repo, you can do this the same way you would in a regular git repository. For example, `git rm foo; git commit -m "removed foo"`. This leaves the contents of the files still in the annex, not really deleted yet.
 
-If you have a direct mode repo, you can't run `git rm` in it. Instead, you can just delete files using `rm` or your file manager, and then run `git annex sync` to commit the deletion. That will delete the file's content from your disk. Even if it's the only copy of the file!
+If you have a direct mode repo, you can't run `git rm` on it. Instead, you can just delete files using `rm` or your file manager, and then run `git annex sync` to commit the deletion. That will delete the file's content from your disk. Even if it's the only copy of the file!
 
 Either way, deleting files can leave some garbage lying around in either the local repository, or other repositories that contained a copy of the content of the file you deleted. Eventually you'll want to free up some disk space used by one of these repositories, and then it's time to take out the garbage.
 

diff --git a/doc/bugs/move_violates_numcopies/comment_6_2ffb920ae8e5ebfacf9230a1a7c0c7c3._comment b/doc/bugs/move_violates_numcopies/comment_6_2ffb920ae8e5ebfacf9230a1a7c0c7c3._comment
new file mode 100644
index 000000000..6bee1db8d
--- /dev/null
+++ b/doc/bugs/move_violates_numcopies/comment_6_2ffb920ae8e5ebfacf9230a1a7c0c7c3._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="richih@50508f31e0ee95720acd0120e16d6bdcad9d104b"
+ nickname="richih"
+ avatar="http://cdn.libravatar.org/avatar/499771047201f3eb29a462897b50a5f3"
+ subject="comment 6"
+ date="2018-04-11T07:34:43Z"
+ content="""
+\"not making it worse\" sounds like acceptable middle ground to me. If you are not erroring out, at least printing a warning would be good. OTOH, this easily gets hidden in a flood of message if you're moving more than just a few files.
+
+Still, principle of least surprise would point towards not going against basic safety measures in any case.
+"""]]

diff --git a/doc/bugs/move_violates_numcopies/comment_5_4ef4c6180c71326faa814bb096dab500._comment b/doc/bugs/move_violates_numcopies/comment_5_4ef4c6180c71326faa814bb096dab500._comment
new file mode 100644
index 000000000..c60f55458
--- /dev/null
+++ b/doc/bugs/move_violates_numcopies/comment_5_4ef4c6180c71326faa814bb096dab500._comment
@@ -0,0 +1,32 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 5"""
+ date="2018-04-10T22:39:30Z"
+ content="""
+Another way to approach this is: `move` should not make
+a situation worse, but is not required to make it better.
+
+That would allow moving a file from A to B when numcopies wants 2 copies
+but only one copy exists, because the file being on B is no worse than it
+being on A.
+
+But, if B already has a copy of the file, move would error rather than the
+current behavior of removing from A, when numcopies wants two copies.
+
+And, if B is untrusted (and A is not), or A has the file as required
+content, moving to B would also error, as in both situations it makes
+things worse.
+
+This seems better than the ideas above, because it keeps move a somewhat
+lowlevel operation, like it always has been, but no longer an unsafe one.
+It matches many of my uses of move, when perhaps I want more copies than I
+have, but can't currently spare the space (or am moving the file to a repo
+that will later let it get replicated elsewhere).
+
+It also means that after `git annex move --from`, the local repository will 
+always have the file present, rather than move sometimes failing before
+getting it due to numcopies. (And the converse with `--to`.)
+
+I think this is a small enough change from the current behavior of move
+that it can get away with not having a transition plan.
+"""]]

diff --git a/doc/bugs/move_violates_numcopies/comment_4_370ab3e7adbe6f1888eb5ebe542d18ad._comment b/doc/bugs/move_violates_numcopies/comment_4_370ab3e7adbe6f1888eb5ebe542d18ad._comment
new file mode 100644
index 000000000..68fda3b6a
--- /dev/null
+++ b/doc/bugs/move_violates_numcopies/comment_4_370ab3e7adbe6f1888eb5ebe542d18ad._comment
@@ -0,0 +1,21 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2018-04-10T17:04:14Z"
+ content="""
+Candyangel, thanks, failing is indeed the better thing to do
+than copying when it's not safe to move. That makes sense.
+
+Also, `git annex move` should honor required contents, and refuse to move
+content away from a repository that requires it.
+
+It would be ok to use --force instead of --unsafe, but it doesn't allow for
+a staged transition from --unsafe by default to --safe by default. But, if
+we decide a stanged transition is not needed, I would be inclined to use
+the --force.
+
+Move failing in these situations seems less likely to badly break existing
+workflows than move leaving both copies would; the caller will see
+that git-annex errored, rather than it silently leaving extra copies.
+So perhaps a staged transition could be skipped.
+"""]]

diff --git a/doc/bugs/move_violates_numcopies/comment_3_491d81f636a4da1deefd2d82063e2f05._comment b/doc/bugs/move_violates_numcopies/comment_3_491d81f636a4da1deefd2d82063e2f05._comment
new file mode 100644
index 000000000..34abb9ff8
--- /dev/null
+++ b/doc/bugs/move_violates_numcopies/comment_3_491d81f636a4da1deefd2d82063e2f05._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="richih@50508f31e0ee95720acd0120e16d6bdcad9d104b"
+ nickname="richih"
+ avatar="http://cdn.libravatar.org/avatar/499771047201f3eb29a462897b50a5f3"
+ subject="comment 3"
+ date="2018-04-10T15:49:35Z"
+ content="""
+I agree with anarcat.
+
+The core use case of git-annex is to maintain sets of known-good data. This function mainly relies on a directory structure, checksums, a defined minimum of copies, and tracking where they are. I would never have assumed that I would be able to get git-annex to go below the mincopies, at least not unless I was deep into the innards of git-annex and/or abusing --force or the like.
+
+-- RichiH
+"""]]

diff --git a/doc/bugs/move_violates_numcopies/comment_2_57299719a6e8f3615819da4fbc460da5._comment b/doc/bugs/move_violates_numcopies/comment_2_57299719a6e8f3615819da4fbc460da5._comment
new file mode 100644
index 000000000..178164c81
--- /dev/null
+++ b/doc/bugs/move_violates_numcopies/comment_2_57299719a6e8f3615819da4fbc460da5._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="anarcat"
+ avatar="http://cdn.libravatar.org/avatar/4ad594c1e13211c1ad9edb81ce5110b7"
+ subject="force?"
+ date="2018-04-10T14:20:08Z"
+ content="""
+This definitely seems strange to me. i've been using git-annex for a long time, and I have never known `move` was unsafe by default. so I think that `--safe` should definitely be made defautl.
+
+but beyond that, I wouldn't use `--unsafe` as a new flag: we already have `--force` for `--drop` for example, shouldn't we reuse that here as well?
+"""]]

diff --git a/doc/bugs/move_violates_numcopies/comment_1_36e22ecc569f097e4cc58c20d0fa4876._comment b/doc/bugs/move_violates_numcopies/comment_1_36e22ecc569f097e4cc58c20d0fa4876._comment
new file mode 100644
index 000000000..d59c6ead8
--- /dev/null
+++ b/doc/bugs/move_violates_numcopies/comment_1_36e22ecc569f097e4cc58c20d0fa4876._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="CandyAngel"
+ avatar="http://cdn.libravatar.org/avatar/15c0aade8bec5bf004f939dd73cf9ed8"
+ subject="comment 1"
+ date="2018-04-10T10:39:23Z"
+ content="""
+If git-annex knows it isn't going to do what I've instructed (move the file), I think it should just fail the command immediately, rather than copying but not dropping.
+
+You could make *--safe* fallback to copy (for those that want that behaviour) and *--unsafe* has the risky behaviour, but *move* should either do just that, move the file, or do nothing.
+"""]]

diff --git a/doc/devblog/day_495__move_numcopies_safety.mdwn b/doc/devblog/day_495__move_numcopies_safety.mdwn
new file mode 100644
index 000000000..016124491
--- /dev/null
+++ b/doc/devblog/day_495__move_numcopies_safety.mdwn
@@ -0,0 +1,20 @@
+New version released today with adb special remote, http connection
+caching, improved progress displays, annex.retry, and other changes.
+
+I've been rethinking `git annex move` in the context of numcopies
+checking. Thanks to a user posting
+[[forum/git-annex_move_does_not_appear_to_respect_numcopies]].
+Of course, move is known not to do that, but it's useful to get a
+perspective that this is susprising behavior and not wanted by that user,
+and poorly documented besides.
+
+So, I added `git annex move --safe` which does honor numcopies, so 
+it only does a copy when there are not enough copies to move.
+
+I'm leaning toward making that the default behavior, and
+needing `git annex move --unsafe` to get the current behavior of moving
+without a net. Of course, lots of us probably use move and like the current
+behavior, and such a change can break workflows and scripts.
+There might be a transition period where move warns when run without --safe
+or --unsafe. Feedback welcomed on the bug report 
+[[bugs/move_violates_numcopies]].

diff --git a/Annex/NumCopies.hs b/Annex/NumCopies.hs
index 9fea49db6..60d918cae 100644
--- a/Annex/NumCopies.hs
+++ b/Annex/NumCopies.hs
@@ -11,6 +11,7 @@ module Annex.NumCopies (
 	module Types.NumCopies,
 	module Logs.NumCopies,
 	getFileNumCopies,
+	getAssociatedFileNumCopies,
 	getGlobalFileNumCopies,
 	getNumCopies,
 	deprecatedNumCopies,
@@ -69,6 +70,10 @@ getFileNumCopies f = fromSources
 	, deprecatedNumCopies
 	]
 
+getAssociatedFileNumCopies :: AssociatedFile -> Annex NumCopies
+getAssociatedFileNumCopies (AssociatedFile afile) =
+	maybe getNumCopies getFileNumCopies afile
+
 {- This is the globally visible numcopies value for a file. So it does
  - not include local configuration in the git config or command line
  - options. -}
diff --git a/CHANGELOG b/CHANGELOG
index 4151c6b78..2440cb247 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,3 +1,11 @@
+git-annex (6.20180410) UNRELEASED; urgency=medium
+
+  * move: Added --safe option, which makes move honor numcopies settings.
+    Also --unsafe enables the default behavior, anticipating that the
+    default may one day change.
+
+ -- Joey Hess <id@joeyh.name>  Mon, 09 Apr 2018 14:03:28 -0400
+
 git-annex (6.20180409) upstream; urgency=medium
 
   * Added adb special remote which allows exporting files to Android devices.
diff --git a/Command/Copy.hs b/Command/Copy.hs
index 3b5080c3b..daf2e66bc 100644
--- a/Command/Copy.hs
+++ b/Command/Copy.hs
@@ -50,7 +50,7 @@ seek o = allowConcurrentOutput $ do
 		Batch -> batchInput Right (batchCommandAction . go)
 		NoBatch -> withKeyOptions
 			(keyOptions o) (autoMode o)
-			(Command.Move.startKey (fromToOptions o) False)
+			(Command.Move.startKey (fromToOptions o) Command.Move.RemoveNever)
 			(withFilesInGit go)
 			=<< workTreeItems (copyFiles o)
 
@@ -59,7 +59,7 @@ seek o = allowConcurrentOutput $ do
  - sending non-preferred content. -}
 start :: CopyOptions -> FilePath -> Key -> CommandStart
 start o file key = stopUnless shouldCopy $ 
-	Command.Move.start (fromToOptions o) False file key
+	Command.Move.start (fromToOptions o) Command.Move.RemoveNever file key
   where
 	shouldCopy
 		| autoMode o = want <||> numCopiesCheck file key (<)
diff --git a/Command/Drop.hs b/Command/Drop.hs
index 09385dddb..baeae66ee 100644
--- a/Command/Drop.hs
+++ b/Command/Drop.hs
@@ -202,8 +202,8 @@ requiredContent = do
 {- In auto mode, only runs the action if there are enough
  - copies on other semitrusted repositories. -}
 checkDropAuto :: Bool -> Maybe Remote -> AssociatedFile -> Key -> (NumCopies -> CommandStart) -> CommandStart
-checkDropAuto automode mremote (AssociatedFile afile) key a =
-	go =<< maybe getNumCopies getFileNumCopies afile
+checkDropAuto automode mremote afile key a =
+	go =<< getAssociatedFileNumCopies afile
   where
 	go numcopies
 		| automode = do
diff --git a/Command/Get.hs b/Command/Get.hs
index 4ebdc0cad..f4e3d47b5 100644
--- a/Command/Get.hs
+++ b/Command/Get.hs
@@ -68,7 +68,7 @@ start' expensivecheck from key afile ai = onlyActionOn key $
 			Nothing -> go $ perform key afile
 			Just src ->
 				stopUnless (Command.Move.fromOk src key) $
-					go $ Command.Move.fromPerform src False key afile
+					go $ Command.Move.fromPerform src Command.Move.RemoveNever key afile
   where
 	go a = do
 		showStartKey "get" key ai
diff --git a/Command/Mirror.hs b/Command/Mirror.hs
index a7d44d8ef..f0f0dc4f3 100644
--- a/Command/Mirror.hs
+++ b/Command/Mirror.hs
@@ -55,7 +55,7 @@ start o file k = startKey o afile k (mkActionItem afile)
 startKey :: MirrorOptions -> AssociatedFile -> Key -> ActionItem -> CommandStart
 startKey o afile key ai = onlyActionOn key $ case fromToOptions o of
 	ToRemote r -> checkFailedTransferDirection ai Upload $ ifM (inAnnex key)
-		( Command.Move.toStart False afile key ai =<< getParsed r
+		( Command.Move.toStart Command.Move.RemoveNever afile key ai =<< getParsed r
 		, do
 			numcopies <- getnumcopies
 			Command.Drop.startRemote afile ai numcopies key =<< getParsed r
diff --git a/Command/Move.hs b/Command/Move.hs
index cbb7d9a2b..df97b7969 100644
--- a/Command/Move.hs
+++ b/Command/Move.hs
@@ -1,6 +1,6 @@
 {- git-annex command
  -
- - Copyright 2010-2017 Joey Hess <id@joeyh.name>
+ - Copyright 2010-2018 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -28,6 +28,7 @@ cmd = withGlobalOptions [jobsOption, jsonOptions, jsonProgressOption, annexedMat
 data MoveOptions = MoveOptions
 	{ moveFiles :: CmdParams
 	, fromToOptions :: FromToHereOptions
+	, removeWhenOptions :: RemoveWhen
 	, keyOptions :: Maybe KeyOptions
 	, batchOption :: BatchMode
 	}
@@ -36,6 +37,7 @@ optParser :: CmdParamsDesc -> Parser MoveOptions
 optParser desc = MoveOptions
 	<$> cmdParams desc
 	<*> parseFromToHereOptions
+	<*> parseRemoveWhenOptions
 	<*> optional (parseKeyOptions <|> parseFailedTransfersOption)
 	<*> parseBatchOption
 
@@ -43,64 +45,75 @@ instance DeferredParseClass MoveOptions where
 	finishParse v = MoveOptions
 		<$> pure (moveFiles v)
 		<*> finishParse (fromToOptions v)
+		<*> pure (removeWhenOptions v)
 		<*> pure (keyOptions v)
 		<*> pure (batchOption v)
 
+data RemoveWhen = RemoveSafe | RemoveUnsafe | RemoveNever
+	deriving (Show, Eq)
+
+parseRemoveWhenOptions :: Parser RemoveWhen
+parseRemoveWhenOptions = 
+	flag' RemoveSafe
+		( long "safe"
+		<> short 's'
+		<> help "preserve numcopies"
+		)
+	<|> flag' RemoveUnsafe
+		(long "unsafe"
+		<> short 'u'
+		<> help "do not preserve numcopies (default)"
+		)
+	<|> pure RemoveUnsafe
+
 seek :: MoveOptions -> CommandSeek
 seek o = allowConcurrentOutput $ do
-	let go = whenAnnexed $ start (fromToOptions o) True
+	let go = whenAnnexed $ start (fromToOptions o) (removeWhenOptions o)
 	case batchOption o of
 		Batch -> batchInput Right (batchCommandAction . go)
 		NoBatch -> withKeyOptions (keyOptions o) False
-			(startKey (fromToOptions o) True)
+			(startKey (fromToOptions o) (removeWhenOptions o))
 			(withFilesInGit go)
 			=<< workTreeItems (moveFiles o)
 
-start :: FromToHereOptions -> Bool -> FilePath -> Key -> CommandStart
-start fromto move f k = start' fromto move afile k (mkActionItem afile)
+start :: FromToHereOptions -> RemoveWhen -> FilePath -> Key -> CommandStart
+start fromto removewhen f k =
+	start' fromto removewhen afile k (mkActionItem afile)
   where
 	afile = AssociatedFile (Just f)
 
-startKey :: FromToHereOptions -> Bool -> Key -> ActionItem -> CommandStart
-startKey fromto move = start' fromto move (AssociatedFile Nothing)
+startKey :: FromToHereOptions -> RemoveWhen -> Key -> ActionItem -> CommandStart
+startKey fromto removewhen = start' fromto removewhen (AssociatedFile Nothing)
 
-start' :: FromToHereOptions -> Bool -> AssociatedFile -> Key -> ActionItem -> CommandStart
-start' fromto move afile key ai = onlyActionOn key $
+start' :: FromToHereOptions -> RemoveWhen -> AssociatedFile -> Key -> ActionItem -> CommandStart
+start' fromto removewhen afile key ai = onlyActionOn key $
 	case fromto of
 		Right (FromRemote src) ->
 			checkFailedTransferDirection ai Download $
-				fromStart move afile key ai =<< getParsed src
+				fromStart removewhen afile key ai =<< getParsed src
 		Right (ToRemote dest) ->
 			checkFailedTransferDirection ai Upload $
-				toStart move afile key ai =<< getParsed dest
+				toStart removewhen afile key ai =<< getParsed dest
 		Left ToHere ->
 			checkFailedTransferDirection ai Download $
-				toHereStart move afile key ai
+				toHereStart removewhen afile key ai
 
-showMoveAction :: Bool -> Key -> ActionItem -> Annex ()
-showMoveAction move = showStartKey (if move then "move" else "copy")

(Diff truncated)

diff --git a/doc/git-annex-copy.mdwn b/doc/git-annex-copy.mdwn
index da48c5368..be082ffa0 100644
--- a/doc/git-annex-copy.mdwn
+++ b/doc/git-annex-copy.mdwn
@@ -24,6 +24,11 @@ Copies the content of files from or to another remote.
   Copy the content of files from the local repository
   to the specified remote.
 
+* `--to=here`
+
+  Copy the content of files from all reachable remotes to the local
+  repository.
+
 * `--jobs=N` `-JN`
 
   Enables parallel transfers with up to the specified number of jobs

diff --git a/doc/news/version_6.20171124.mdwn b/doc/news/version_6.20171124.mdwn
deleted file mode 100644
index 4b20f6fe7..000000000
--- a/doc/news/version_6.20171124.mdwn
+++ /dev/null
@@ -1,5 +0,0 @@
-git-annex 6.20171124 released with [[!toggle text="these changes"]]
-[[!toggleable text="""
-   * Display progress meter when uploading a key without size information,
-     getting the size by statting the content file.
-   * Fix build with dns-3.0."""]]
\ No newline at end of file
diff --git a/doc/news/version_6.20171214.mdwn b/doc/news/version_6.20171214.mdwn
deleted file mode 100644
index f79572b14..000000000
--- a/doc/news/version_6.20171214.mdwn
+++ /dev/null
@@ -1,22 +0,0 @@
-git-annex 6.20171214 released with [[!toggle text="these changes"]]
-[[!toggleable text="""
-   * Use youtube-dl rather than quvi to download media from web pages,
-     since quvi is not being actively developed and youtube-dl supports
-     many more sites.
-   * addurl --relaxed got slower, since youtube-dl has to hit the network
-     to check for embedded media. If you relied on --relaxed not hitting the
-     network for speed reasons, using --relaxed --raw will get the old level
-     of speed, but can't be used for urls with embedded videos.
-   * importfeed now downloads things linked to by feeds, even when they are
-     not media files.
-   * Removed no longer needed dependency on yesod-default.
-   * Allow exporttree remotes to be marked as dead.
-   * initremote, enableremote: Really support gpg subkeys suffixed with an
-     exclamation mark, which forces gpg to use a specific subkey.
-     (Previous try had a bug.)
-   * lookupkey: Support being given an absolute filename to a file
-     within the current git repository.
-   * A top-level .noannex file will prevent git-annex init from being used
-     in a repository. This is useful for repositories that have a policy
-     reason not to use git-annex. The content of the file will be displayed
-     to the user who tries to run git-annex init."""]]
\ No newline at end of file
diff --git a/doc/news/version_6.20180409.mdwn b/doc/news/version_6.20180409.mdwn
new file mode 100644
index 000000000..37a4c5b80
--- /dev/null
+++ b/doc/news/version_6.20180409.mdwn
@@ -0,0 +1,26 @@
+git-annex 6.20180409 released with [[!toggle text="these changes"]]
+[[!toggleable text="""
+   * Added adb special remote which allows exporting files to Android devices.
+   * For url downloads, git-annex now defaults to using a http library,
+     rather than wget or curl. But, if annex.web-options is set, it will
+     use curl. To use the .netrc file, run:
+       git config annex.web-options --netrc
+   * git-annex no longer uses wget (and wget is no longer shipped with
+     git-annex builds).
+   * Enable HTTP connection reuse across multiple files for improved speed.
+   * Fix calculation of estimated completion for progress meter.
+   * OSX app: Work around libz/libPng/ImageIO.framework version skew
+     by not bundling libz, assuming OSX includes a suitable libz.1.dylib.
+   * Added annex.retry, annex.retry-delay, and per-remote versions
+     to configure transfer retries.
+   * Also do forward retrying in cases where no exception is thrown,
+     but the transfer failed.
+   * When adding a new version of a file, and annex.genmetadata is enabled,
+     don't copy the data metadata from the old version of the file,
+     instead use the mtime of the file.
+   * Avoid running annex.http-headers-command more than once.
+   * info: Added "combined size of repositories containing these files"
+     stat when run on a directory.
+   * info: Changed sorting of numcopies stats table, so it's ordered
+     by the variance from the desired number of copies.
+   * Fix resuming a download when using curl."""]]
\ No newline at end of file

diff --git a/doc/forum/git-annex_move_does_not_appear_to_respect_numcopies/comment_1_5dba0ff04c86c7b2697a0750fd1e4b5d._comment b/doc/forum/git-annex_move_does_not_appear_to_respect_numcopies/comment_1_5dba0ff04c86c7b2697a0750fd1e4b5d._comment
new file mode 100644
index 000000000..52b85d16a
--- /dev/null
+++ b/doc/forum/git-annex_move_does_not_appear_to_respect_numcopies/comment_1_5dba0ff04c86c7b2697a0750fd1e4b5d._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="CandyAngel"
+ avatar="http://cdn.libravatar.org/avatar/15c0aade8bec5bf004f939dd73cf9ed8"
+ subject="comment 1"
+ date="2018-04-09T14:56:52Z"
+ content="""
+Well.. it actually says this in the source code for the move command:
+
+    - Note that unlike drop, this does not honor numcopies.
+    - A file's content can be moved even if there are insufficient copies to
+    - allow it to be dropped.
+
+So the behaviour seems to be known/intended. You told it to move the file, so it did.
+Perhaps it should output a warning if numcopies is not satisfied after the move?
+"""]]

diff --git a/doc/bugs/Assistant_temporary_delete_and_recommit_files.mdwn b/doc/bugs/Assistant_temporary_delete_and_recommit_files.mdwn
new file mode 100644
index 000000000..4106b1ecf
--- /dev/null
+++ b/doc/bugs/Assistant_temporary_delete_and_recommit_files.mdwn
@@ -0,0 +1,606 @@
+### Please describe the problem.
+Hi,
+
+I'm using git-annex assistant to sync my org-mode files between two hosts. Because these files are just plaintext, I configured git(-annex) to store the content inside git (and not annex). The setup is quite simple: On both hosts (laptop and server) there is a git-annex repository in `~/Org/Privat` which have a remote configured (over ssh) point to each other. I run "git-annex assistant" on both hosts and this works pretty well, except that `git-annex assistant` temporary delete and recommit files (probably during merge). This is no big deal because the file is still there but it unnecessarily messes up the `git log`. I'm not sure if this is a bug or just a side effect of how git-annex works.
+
+
+### What steps will reproduce the problem?
+
+    [Laptop]
+    mkdir -p ~/foo
+    cd ~/foo
+    git init
+    git annex init --version=6
+    git annex describe "laptop"
+    cat > .gitattributes << EOF
+    *.org annex.largefiles=nothing
+    *.org_archive annex.largefiles=nothing
+    EOF
+    git add .gitattributes
+    git commit -m "Versioning org-mode files with git"
+    git remote add server server:~/foo/
+
+
+
+    [Server]
+    git clone laptop:~/foo/ ~/foo/
+    cd ~/foo
+    git annex init "server"
+    git annex upgrade # update from v5 to v6
+    git annex assistant
+
+    [Laptop]
+    cd ~/foo
+    git annex assistant
+    echo "* this is a headline" >> test.org
+
+This is the basic setup. After editing test.org multiple times, the server get those changes, delete test.org and immediately recommit it. Unfortunately, I can't really reproduce the issue with the above instructions. I'm not sure if it's a timing problem, the content of the file matters or anything like that. Hopefully the attached `daemon.log` can provide more infos.
+
+
+### What version of git-annex are you using? On what operating system?
+    [Laptop, Debian stretch]
+    git-annex version: 6.20170101.1
+    build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Quvi
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+    local repository version: 6
+    supported repository versions: 3 5 6
+    upgrade supported from repository versions: 0 1 2 3 4 5
+    operating system: linux x86_64
+
+    [Server, Debian stretch]
+    git-annex version: 6.20180316
+    build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Testsuite
+    dependency versions: aws-0.14.1 bloomfilter-2.0.1.0 cryptonite-0.20 DAV-1.3.1 feed-0.3.11.1 ghc-8.0.1 http-client-0.4.31.1 persistent-sqlite-2.6 torrent-10000.0.0 uuid-1.3.12 yesod-1.4.3
+    key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
+    remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
+    local repository version: 6
+    supported repository versions: 3 5 6
+    upgrade supported from repository versions: 0 1 2 3 4 5
+    operating system: linux x86_64
+
+
+### Please provide any additional information below.
+daemon.log from server, interesting part:
+    [2018-04-09 11:37:44.985603501] Watcher: file deleted Eingang.org
+
+
+I'm also not sure where the `.#Eingang.org` comes from. (autosave-mode and backup stuff is disabled in Emacs). Note that I also ignore `.#` files:
+
+    *~
+    .#*
+    \#*#
+    .gitignore (END)
+
+[[!format sh """
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+[2018-04-09 11:34:41.651229076] main: starting assistant version 6.20180316
+[2018-04-09 11:34:41.653000679] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
+[2018-04-09 11:34:41.656349808] process done ExitSuccess
+[2018-04-09 11:34:41.656682497] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
+[2018-04-09 11:34:41.660412588] process done ExitSuccess
+[2018-04-09 11:34:41.660905012] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..19354e4a5c6bd694a449992d5bd0d4bae48eba3b","--pretty=%H","-n1"]
+[2018-04-09 11:34:41.664232695] process done ExitSuccess
+[2018-04-09 11:34:41.66500315] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
+[2018-04-09 11:34:41.666629464] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
+[2018-04-09 11:34:41.668907654] Cronner: You should enable consistency checking to protect your data. 
+[2018-04-09 11:34:41.673213064] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--stage","-z","--","."]
+[2018-04-09 11:34:41.67746018] process done ExitSuccess
+[2018-04-09 11:34:41.677940071] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--stage","-z","--","."]
+[2018-04-09 11:34:41.680803467] process done ExitSuccess
+[2018-04-09 11:34:41.682995824] chat: nice ["ionice","-c3","nocache","/usr/bin/git-annex","remotedaemon","--foreground"]
+[2018-04-09 11:34:41.698996042] TransferScanner: Syncing with laptop 
+[2018-04-09 11:34:41.714084077] TransferWatcher: watching for transfers
+[2018-04-09 11:34:41.71886081] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-tree","--full-tree","-z","--","refs/heads/git-annex","uuid.log","remote.log","trust.log","group.log","numcopies.log","schedule.log","preferred-content.log","required-content.log","group-preferred-content.log"]
+[2018-04-09 11:34:41.736468728] process done ExitSuccess
+[2018-04-09 11:34:41.737473438] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
+[2018-04-09 11:34:41.740351978] process done ExitSuccess
+[2018-04-09 11:34:41.740604847] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
+[2018-04-09 11:34:41.743608301] process done ExitSuccess
+[2018-04-09 11:34:41.743883152] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
+[2018-04-09 11:34:41.746600971] process done ExitSuccess
+[2018-04-09 11:34:41.746859505] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
+[2018-04-09 11:34:41.749864575] process done ExitSuccess
+
+  No known network monitor available through dbus; falling back to polling
+
+  No known volume monitor available through dbus; falling back to mtab polling
+[2018-04-09 11:34:41.75133072] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
+[2018-04-09 11:34:41.755913773] process done ExitSuccess
+[2018-04-09 11:34:41.756133988] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
+[2018-04-09 11:34:41.759413558] process done ExitSuccess
+[2018-04-09 11:34:41.759874669] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..19354e4a5c6bd694a449992d5bd0d4bae48eba3b","--pretty=%H","-n1"]
+[2018-04-09 11:34:41.763534204] process done ExitSuccess
+[2018-04-09 11:34:41.764491559] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-tree","--full-tree","-z","--","refs/heads/git-annex","uuid.log","remote.log","trust.log","group.log","numcopies.log","schedule.log","preferred-content.log","required-content.log","group-preferred-content.log"]
+[2018-04-09 11:34:41.768098522] process done ExitSuccess
+[2018-04-09 11:34:41.769538385] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","fetch","laptop"]
+[2018-04-09 11:34:41.774051138] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
+[2018-04-09 11:34:41.784145283] process done ExitSuccess
+[2018-04-09 11:34:41.784410291] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
+[2018-04-09 11:34:41.790119006] process done ExitSuccess
+[2018-04-09 11:34:41.790448451] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/master..refs/heads/synced/master","--pretty=%H","-n1"]
+[2018-04-09 11:34:41.803315234] process done ExitSuccess
+[2018-04-09 11:34:41.803726537] Merger: merging refs/heads/synced/master into refs/heads/master
+
+[2018-04-09 11:34:41.803885147] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/master"]
+[2018-04-09 11:34:41.806791552] process done ExitSuccess
+[2018-04-09 11:34:41.807106639] read: git ["--version"]
+[2018-04-09 11:34:41.809213293] process done ExitSuccess
+[2018-04-09 11:34:41.809579548] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","commit.gpgsign=false","merge","--no-edit","refs/heads/synced/master","--allow-unrelated-histories"]
+Aktualisiere 9d81724..1f72868
+Fast-forward
+ Eingang.org | 146 +++++++++++++++++++++++++++++++
+ Privat.org  | 284 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+ 2 files changed, 430 insertions(+)
+ create mode 100644 Eingang.org
+ create mode 100644 Privat.org
+[2018-04-09 11:34:41.869190999] process done ExitSuccess
+[2018-04-09 11:34:41.869616621] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
+[2018-04-09 11:34:41.87263735] process done ExitSuccess
+[2018-04-09 11:34:41.872886362] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
+[2018-04-09 11:34:41.876100082] process done ExitSuccess
+[2018-04-09 11:34:41.876526799] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..19354e4a5c6bd694a449992d5bd0d4bae48eba3b","--pretty=%H","-n1"]
+[2018-04-09 11:34:41.879793847] process done ExitSuccess
+(scanning...) [2018-04-09 11:34:41.880538361] Watcher: Performing startup scan
+[2018-04-09 11:34:41.880979369] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
+[2018-04-09 11:34:41.884247108] process done ExitSuccess
+[2018-04-09 11:34:41.884492946] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
+[2018-04-09 11:34:41.887468992] process done ExitSuccess
+[2018-04-09 11:34:41.88788752] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..19354e4a5c6bd694a449992d5bd0d4bae48eba3b","--pretty=%H","-n1"]
+[2018-04-09 11:34:41.891145787] process done ExitSuccess
+[2018-04-09 11:34:41.891582283] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
+[2018-04-09 11:34:41.892133862] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
+[2018-04-09 11:34:41.900436044] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
+[2018-04-09 11:34:41.903468755] process done ExitSuccess
+[2018-04-09 11:34:41.903676566] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
+[2018-04-09 11:34:41.906280325] process done ExitSuccess
+[2018-04-09 11:34:41.906524648] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
+[2018-04-09 11:34:41.908819698] process done ExitSuccess
+[2018-04-09 11:34:41.909024433] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
+[2018-04-09 11:34:41.911759505] process done ExitSuccess
+[2018-04-09 11:34:41.912398343] read: git ["--version"]
+[2018-04-09 11:34:41.914443754] process done ExitSuccess
+[2018-04-09 11:34:41.914752675] chat: git ["--git-dir=.git","--work-tree=.","check-ignore","-z","--stdin","--verbose","--non-matching"]
+[2018-04-09 11:34:41.917108907] Watcher: add Eingang.org
+[2018-04-09 11:34:41.91734644] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/master..refs/remotes/laptop/master","--pretty=%H","-n1"]
+[2018-04-09 11:34:41.920660043] process done ExitSuccess
+[2018-04-09 11:34:41.921055355] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.numcopies","annex.largefiles","--"]
+[2018-04-09 11:34:41.921800291] read: git ["--version"]
+[2018-04-09 11:34:41.925093152] process done ExitSuccess
+[2018-04-09 11:34:41.926328604] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
+[2018-04-09 11:34:41.928840733] process done ExitSuccess
+[2018-04-09 11:34:41.929206878] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
+[2018-04-09 11:34:41.934755521] process done ExitSuccess
+[2018-04-09 11:34:41.935152027] Merger: watching .git/refs
+[2018-04-09 11:34:41.958868627] Watcher: add Privat.org
+[2018-04-09 11:34:41.959579297] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--deleted","-z","--","."]
+[2018-04-09 11:34:41.962332644] process done ExitSuccess
+(started...) 
+[2018-04-09 11:34:41.962726051] Watcher: watching .
+Von laptop:Org/Privat
+   9d81724..1f72868  master        -> laptop/master
+   316555e..19354e4  git-annex     -> laptop/git-annex
+ * [neuer Branch]    synced/master -> laptop/synced/master
+[2018-04-09 11:34:41.997168884] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","symbolic-ref","-q","HEAD"]
+[2018-04-09 11:34:42.004185608] process done ExitSuccess
+[2018-04-09 11:34:42.004431346] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","refs/heads/master"]
+[2018-04-09 11:34:42.009686504] process done ExitSuccess
+[2018-04-09 11:34:42.010153632] process done ExitSuccess
+[2018-04-09 11:34:42.010482295] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
+[2018-04-09 11:34:42.013335466] process done ExitSuccess
+[2018-04-09 11:34:42.013718719] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
+[2018-04-09 11:34:42.019932467] process done ExitSuccess
+[2018-04-09 11:34:42.020329615] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..19354e4a5c6bd694a449992d5bd0d4bae48eba3b","--pretty=%H","-n1"]
+[2018-04-09 11:34:42.023886643] process done ExitSuccess

(Diff truncated)

diff --git a/doc/news/version_6.20180316/comment_1_81de8aa3859e65b944027f67bf5c8cc1._comment b/doc/news/version_6.20180316/comment_1_81de8aa3859e65b944027f67bf5c8cc1._comment
new file mode 100644
index 000000000..c8c952878
--- /dev/null
+++ b/doc/news/version_6.20180316/comment_1_81de8aa3859e65b944027f67bf5c8cc1._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Michael"
+ avatar="http://cdn.libravatar.org/avatar/86811fdafa094c610ec8ef8858a78dbf"
+ subject="prebuilt i386 still at old version?"
+ date="2018-04-08T01:44:24Z"
+ content="""
+https://downloads.kitenet.net/git-annex/linux/current/git-annex-standalone-i386.tar.gz - as of 2018-04-07 shows 6.20180227-g32d682dd8
+"""]]

diff --git a/doc/tips/fully_encrypted_git_repositories_with_gcrypt/comment_18_bdcb4bace530ac8b2018916893ae7e93._comment b/doc/tips/fully_encrypted_git_repositories_with_gcrypt/comment_18_bdcb4bace530ac8b2018916893ae7e93._comment
new file mode 100644
index 000000000..8153055da
--- /dev/null
+++ b/doc/tips/fully_encrypted_git_repositories_with_gcrypt/comment_18_bdcb4bace530ac8b2018916893ae7e93._comment
@@ -0,0 +1,87 @@
+[[!comment format=mdwn
+ username="itn@23fdfea9e5c13d8bb24e4cbde30d2a98293154c1"
+ nickname="itn"
+ avatar="http://cdn.libravatar.org/avatar/09ed2f8a3623192b3d38df2e420cd09d"
+ subject="Cloning and syncing leads to confusing "gcrypt: Failed to decrypt manifest!" errors."
+ date="2018-04-08T00:50:14Z"
+ content="""
+# Details
+
+I followed the instructions to create a new encrypted ssh-accessible remote (gcrypt, hybrid). As an additional detail, I have a key generated with instructions from <https://alexcabal.com/creating-the-perfect-gpg-keypair>). Everything worked until I tried to setup another machine (which has the same encryption GPG subkey available) to use the repo. After that, neither machine could no longer sync successfully. Here is the result of the commands on the second machine (partially omitted):
+
+    $ git clone gcrypt::ssh://server/user/annex
+    <output omitted>
+    $ cd annex
+    $ git annex sync
+    (merging origin/git-annex into git-annex...)
+    (recording state in git...)
+    commit  (recording state in git...)
+    
+    On branch master
+    Your branch is up-to-date with 'origin/master'.
+    
+    nothing to commit, working tree clean
+    ok
+    pull origin
+    gcrypt: Decrypting manifest
+    gpg: Good signature from \"...\" [ultimate]
+    ok
+    push origin
+    gcrypt: Decrypting manifest
+    gpg: Good signature from \"...\" [ultimate]
+    Counting objects: 6, done.
+    Compressing objects: 100% (5/5), done.
+    Total 6 (delta 1), reused 1 (delta 0)
+    gcrypt: Encrypting to: --throw-keyids --default-recipient-self
+    gcrypt: Requesting manifest signature
+    To gcrypt::ssh://server/home/user/annex   
+       494c59d..935583f  git-annex -> synced/git-annex
+    gcrypt: Decrypting manifest
+    gpg: decryption failed: No secret key
+    gcrypt: Failed to decrypt manifest!
+    ok
+
+Looking at the output, I was convinced that the following line explained the issue:
+
+    gcrypt: Encrypting to: --throw-keyids --default-recipient-self
+
+So I did everything again and took copies of the repo on the server after each interaction. I found out that when the remote is initialized, the encrypted \"session key\" contained the information on the recipients, but if the remote in synced with the cloned repo, the session key is modified.
+
+I took a look at the encrypted session key in the remote repo before and after (I cloned it directly without trying to decrypt). Here, I've replaced the real key id with \"DEADBEEFDEADBEEF\":
+
+    $ gpg --list-packets --list-only <session-key-before>
+    # off=0 ctb=85 tag=1 hlen=3 plen=524
+    :pubkey enc packet: version 3, algo 1, keyid DEADBEEFDEADBEEF
+            data: [4096 bits]
+    # off=527 ctb=d2 tag=18 hlen=2 plen=0 partial new-ctb
+    :encrypted data packet:
+            length: unknown
+            mdc_method: 2
+
+  
+    $ gpg --list-packets --list-only <session-key-after>
+    # off=0 ctb=85 tag=1 hlen=3 plen=524
+    :pubkey enc packet: version 3, algo 1, keyid 0000000000000000
+            data: [4092 bits]
+    # off=527 ctb=d2 tag=18 hlen=2 plen=0 partial new-ctb
+    :encrypted data packet:
+            length: unknown
+            mdc_method: 2
+
+This means that the recipient information is not included in the file. In practice this means that you need to check if you are the recipient by trying to decrypt the file. 
+Unfortunately, it seems that no key is actually tried. The file is decrypted successfully with:
+
+    gpg --try-all-secrets <session-key-after>
+
+# Solution
+
+As the core issue is that GnuPG fails to use the correct key if recipients are not known, we can tell gpg which keys to try. This can be done by editing ~/.gnupg/gpg.conf. Set the 'default-key' or alternatively, if the key you are using for this purpose is not your default key, set the 'try-secret-key'
+ 
+The other issue is that the recipients are removed from the session key, you can make sure that the correct recipients are kept if you set the following options for the remote in the cloned repo (.git/config):
+
+    gcrypt-participants = <insert-key(s)-here>
+    gcrypt-signingkey = <insert-key-here>
+    gcrypt-publish-participants = true
+
+I suspect that many users who have multiple keys may run into this issue. The previous comment also seems to be a variant of this.
+"""]]

diff --git a/doc/forum/git-annex_move_does_not_appear_to_respect_numcopies.mdwn b/doc/forum/git-annex_move_does_not_appear_to_respect_numcopies.mdwn
new file mode 100644
index 000000000..039ed005b
--- /dev/null
+++ b/doc/forum/git-annex_move_does_not_appear_to_respect_numcopies.mdwn
@@ -0,0 +1,36 @@
+I have a git-annex repository checked out onto two separate drives (connected to the same machine). git-annex is aware of other copies of the repository on other drives that are not currently accessible. The local repository version is 5, and I'm running git-annex version 6.20170818. I have configured numcopies, and it displays correctly on both machines:
+
+```
+$ git annex numcopies
+2
+```
+
+However, I can't figure out how to use `git annex move` to match the behavior I'd expect. If I run:
+
+```
+$ git annex move my_file.txt --to otherdrive
+```
+
+it will copy `my_file.txt` to otherdrive and drop it from my primary drive, even though it's only aware of two copies. `git annex drop`, however, behaves as I'd expect.
+
+    $ git annex drop my_file.txt
+    drop my_file.txt (unsafe)
+        Could only verify the existence of 1 out of 2 necessary copies
+
+        Rather than dropping this file, try using: git annex move
+
+        (Use --force to override this check, or adjust numcopies.)
+
+It appears that `move` is equivalent to `git annex copy && git annex drop --force`, rather than `git annex copy && git annex drop`. That's surprising to me, but the message here implies that it's the intended behavior. In any case, I can't figure out which arguments to pass to `git-annex move` to simulate the behavior of `git annex copy && git annex drop` - that is, I want to move a file to a different drive (to free up space), but I do *not* want to drop it if I cannot ensure the correct number of copies exist. 
+
+
+Passing `--numcopies 2` doesn't do what I want either:
+
+```
+$ git annex move --numcopies 2 --to otherdrive my_file.txt 
+```
+
+will still copy the file to otherdrive and drop my local copy, leaving me with only one copy.
+
+
+Is there a mode for `git annex move` which behaves the desired way: moving a file to a different drive, while still ensuring that I don't end up with fewer than N verifiable copies (where N is the value already configured via `git-annex-numcopies`)?

diff --git a/doc/devblog/day_494__url_download_changes.mdwn b/doc/devblog/day_494__url_download_changes.mdwn
new file mode 100644
index 000000000..3d542dd57
--- /dev/null
+++ b/doc/devblog/day_494__url_download_changes.mdwn
@@ -0,0 +1,25 @@
+To make git-annex faster when it's dealing with a lot of urls,
+I decided to make it use the http-conduit library for all url access by
+default. That way, http pipelining will speed up repeated requests to the
+same web servers. This is kind of a follow-up to the recent elimination of
+rsync.
+
+Some users rely on some annex.web-options or a .netrc file to configure
+how git-annex downloads urls. To keep that supported, when
+annex.web-options is set, git-annex will use curl.
+To use a .netrc file, curl needs an option, so you would configure:
+
+	git config annex.web-options --netrc
+
+I get the feeling that nobody has implemented resuming interrupted
+downloads of files using http-conduit before, because it was unexpectedly
+kind of hard and http-types lacks support for some of the necessary
+range-related HTTP stuff.
+
+Today's work was supported by the NSF-funded DataLad project.
+
+----
+
+Stewart V. Wright [announced recastex](http://git-annex.branchable.com/tips/Announcing_recastex_-___40__re__41__podcast__from_your_annex/),
+a program that publishes podcasts and other files from by git-annex to your
+phone.

diff --git a/Annex/Content.hs b/Annex/Content.hs
index 1816033ed..d14bc0642 100644
--- a/Annex/Content.hs
+++ b/Annex/Content.hs
@@ -939,16 +939,14 @@ saveState nocommit = doSideAction $ do
 
 {- Downloads content from any of a list of urls. -}
 downloadUrl :: Key -> MeterUpdate -> [Url.URLString] -> FilePath -> Annex Bool
-downloadUrl k p urls file = meteredFile file (Just p) k $
-	go =<< annexWebDownloadCommand <$> Annex.getGitConfig
+downloadUrl k p urls file = 
+	-- Poll the file to handle configurations where an external
+	-- download command is used.
+	meteredFile file (Just p) k $
+		go =<< annexWebDownloadCommand <$> Annex.getGitConfig
   where
-	go Nothing = do
-		a <- ifM commandProgressDisabled
-			( return Url.downloadQuiet
-			, return Url.download
-			)
-		Url.withUrlOptions $ \uo -> 
-			liftIO $ anyM (\u -> a u file uo) urls
+	go Nothing = Url.withUrlOptions $ \uo -> 
+		liftIO $ anyM (\u -> Url.download p u file uo) urls
 	go (Just basecmd) = anyM (downloadcmd basecmd) urls
 	downloadcmd basecmd url =
 		progressCommand "sh" [Param "-c", Param $ gencmd url basecmd]
diff --git a/Annex/Url.hs b/Annex/Url.hs
index 52469b58a..0d65a3eb2 100644
--- a/Annex/Url.hs
+++ b/Annex/Url.hs
@@ -1,5 +1,5 @@
 {- Url downloading, with git-annex user agent and configured http
- - headers and wget/curl options.
+ - headers and curl options.
  -
  - Copyright 2013-2018 Joey Hess <id@joeyh.name>
  -
diff --git a/Assistant/Upgrade.hs b/Assistant/Upgrade.hs
index a100e23b5..c63897150 100644
--- a/Assistant/Upgrade.hs
+++ b/Assistant/Upgrade.hs
@@ -36,6 +36,7 @@ import Utility.Tmp.Dir
 import Utility.UserInfo
 import Utility.Gpg
 import Utility.FileMode
+import Utility.Metered
 import qualified Utility.Lsof as Lsof
 import qualified BuildInfo
 import qualified Utility.Url as Url
@@ -322,8 +323,8 @@ downloadDistributionInfo = do
 	liftIO $ withTmpDir "git-annex.tmp" $ \tmpdir -> do
 		let infof = tmpdir </> "info"
 		let sigf = infof ++ ".sig"
-		ifM (Url.downloadQuiet distributionInfoUrl infof uo
-			<&&> Url.downloadQuiet distributionInfoSigUrl sigf uo
+		ifM (Url.download nullMeterUpdate distributionInfoUrl infof uo
+			<&&> Url.download nullMeterUpdate distributionInfoSigUrl sigf uo
 			<&&> verifyDistributionSig gpgcmd sigf)
 			( parseInfoFile <$> readFileStrict infof
 			, return Nothing
diff --git a/Build/BundledPrograms.hs b/Build/BundledPrograms.hs
index a6afdb763..a64206b7d 100644
--- a/Build/BundledPrograms.hs
+++ b/Build/BundledPrograms.hs
@@ -70,14 +70,6 @@ preferredBundledPrograms = catMaybes
 #ifndef mingw32_HOST_OS
 	, Just "sh"
 #endif
-#ifndef mingw32_HOST_OS
-#ifndef darwin_HOST_OS
-	-- wget on OSX has been problematic, looking for certs in the wrong
-	-- places. Don't ship it, use curl or the OSX's own wget if it has
-	-- one.
-	, ifset BuildInfo.wget "wget"
-#endif
-#endif
 	, BuildInfo.lsof
 	, BuildInfo.gcrypt
 #ifndef mingw32_HOST_OS
diff --git a/Build/Configure.hs b/Build/Configure.hs
index a0ddf4dbd..c82c9b7dd 100644
--- a/Build/Configure.hs
+++ b/Build/Configure.hs
@@ -6,14 +6,11 @@ module Build.Configure where
 
 import Build.TestConfig
 import Build.Version
-import Utility.PartialPrelude
-import Utility.Process
 import Utility.SafeCommand
 import Utility.ExternalSHA
 import Utility.Env.Basic
 import Utility.Exception
 import qualified Git.Version
-import Utility.DottedVersion
 import Utility.Directory
 
 import Control.Monad.IfElse
@@ -34,8 +31,6 @@ tests =
 	, TestCase "xargs -0" $ testCmd "xargs_0" "xargs -0 </dev/null"
 	, TestCase "rsync" $ testCmd "rsync" "rsync --version >/dev/null"
 	, TestCase "curl" $ testCmd "curl" "curl --version >/dev/null"
-	, TestCase "wget" $ testCmd "wget" "wget --version >/dev/null"
-	, TestCase "wget unclutter options" checkWgetUnclutter
 	, TestCase "bup" $ testCmd "bup" "bup --version >/dev/null"
 	, TestCase "nice" $ testCmd "nice" "nice true >/dev/null"
 	, TestCase "ionice" $ testCmd "ionice" "ionice -c3 true >/dev/null"
@@ -106,19 +101,6 @@ getGitVersion = go =<< getEnv "FORCE_GIT_VERSION"
 			error $ "installed git version " ++ show v ++ " is too old! (Need " ++ show oldestallowed ++ " or newer)"
 		return $ Config "gitversion" $ StringConfig $ show v
 
-checkWgetUnclutter :: Test
-checkWgetUnclutter = Config "wgetunclutter" . BoolConfig
-	. maybe False (>= normalize "1.16")
-	<$> getWgetVersion 
-
-getWgetVersion :: IO (Maybe DottedVersion)
-getWgetVersion = catchDefaultIO Nothing $
-	extract <$> readProcess "wget" ["--version"]
-  where
-	extract s = case lines s of
-		[] -> Nothing
-		(l:_) -> normalize <$> headMaybe (drop 2 $ words l)
-
 getSshConnectionCaching :: Test
 getSshConnectionCaching = Config "sshconnectioncaching" . BoolConfig <$>
 	boolSystem "sh" [Param "-c", Param "ssh -o ControlPersist=yes -V >/dev/null 2>/dev/null"]
diff --git a/Build/NullSoftInstaller.hs b/Build/NullSoftInstaller.hs
index 921a65aa3..0f56eef12 100644
--- a/Build/NullSoftInstaller.hs
+++ b/Build/NullSoftInstaller.hs
@@ -9,7 +9,8 @@
  - for that.
  - 
  - To build the installer, git-annex should already be built to
- - ./git-annex.exe and the necessary utility programs (rsync and wget)
+ - ./git-annex.exe and the necessary utility programs
+ - (specifically rsync)
  - already installed in PATH from msys32.
  -
  - Copyright 2013-2015 Joey Hess <id@joeyh.name>
diff --git a/CHANGELOG b/CHANGELOG
index 0e55c722b..4031dd2c7 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,6 +1,13 @@
 git-annex (6.20180317) UNRELEASED; urgency=medium
 
   * Added adb special remote which allows exporting files to Android devices.
+  * For url downloads, git-annex now defaults to using a http library,
+    rather than wget or curl. But, if annex.web-options is set, it will
+    use curl. To use the .netrc file, run: 
+      git config annex.web-options --netrc
+  * git-annex no longer uses wget (and wget is no longer shipped with
+    git-annex builds).
+  * Enable HTTP connection reuse across multiple files for improved speed.
   * Fix calculation of estimated completion for progress meter.
   * OSX app: Work around libz/libPng/ImageIO.framework version skew
     by not bundling libz, assuming OSX includes a suitable libz.1.dylib.
@@ -12,8 +19,6 @@ git-annex (6.20180317) UNRELEASED; urgency=medium
     don't copy the data metadata from the old version of the file,
     instead use the mtime of the file.
   * Avoid running annex.http-headers-command more than once.
-  * Enable HTTP connection reuse across multiple files, when git-annex
-    uses http-conduit.
   * info: Added "combined size of repositories containing these files"
     stat when run on a directory.
   * info: Changed sorting of numcopies stats table, so it's ordered
diff --git a/Command/AddUrl.hs b/Command/AddUrl.hs
index 951e5ba15..efe966861 100644
--- a/Command/AddUrl.hs
+++ b/Command/AddUrl.hs
@@ -27,6 +27,7 @@ import Types.KeySource
 import Types.UrlContents
 import Annex.FileMatcher
 import Logs.Location
+import Messages.Progress
 import Utility.Metered
 import Utility.FileSystemEncoding
 import Utility.HtmlDetect
@@ -260,9 +261,8 @@ downloadWeb o url urlinfo file =
 	go =<< downloadWith' downloader urlkey webUUID url (AssociatedFile (Just file))
   where
 	urlkey = addSizeUrlKey urlinfo $ Backend.URL.fromUrl url Nothing
-	downloader f p = do
-		showOutput
-		downloadUrl urlkey p [url] f
+	downloader f p = metered (Just p) urlkey (pure Nothing) $ 
+		\_ p' -> downloadUrl urlkey p' [url] f
 	go Nothing = return Nothing
 	-- If we downloaded a html file, try to use youtube-dl to
 	-- extract embedded media.
diff --git a/Command/ImportFeed.hs b/Command/ImportFeed.hs
index 4fdf63b2e..23ea75e13 100644
--- a/Command/ImportFeed.hs
+++ b/Command/ImportFeed.hs
@@ -33,6 +33,7 @@ import Logs.Web
 import Logs.File

(Diff truncated)

diff --git a/Remote/Helper/Http.hs b/Remote/Helper/Http.hs
index ebe0f2598..ee3107e46 100644
--- a/Remote/Helper/Http.hs
+++ b/Remote/Helper/Http.hs
@@ -15,6 +15,7 @@ import Utility.Metered
 import Remote.Helper.Special
 import Network.HTTP.Client (RequestBody(..), Response, responseStatus, responseBody, BodyReader, NeedsPopper)
 import Network.HTTP.Types
+import Network.HTTP.Conduit
 
 import qualified Data.ByteString.Lazy as L
 import qualified Data.ByteString as S
@@ -71,15 +72,5 @@ handlePopper numchunks chunksize meterupdate h sink = do
 httpBodyRetriever :: FilePath -> MeterUpdate -> Response BodyReader -> IO ()
 httpBodyRetriever dest meterupdate resp
 	| responseStatus resp /= ok200 = giveup $ show $ responseStatus resp
-	| otherwise = bracket (openBinaryFile dest WriteMode) hClose (go zeroBytesProcessed)
-  where
-	reader = responseBody resp
-	go sofar h = do
-		b <- reader
-		if S.null b
-			then return ()
-			else do
-				let sofar' = addBytesProcessed sofar $ S.length b
-				S.hPut h b
-				meterupdate sofar'
-				go sofar' h
+	| otherwise = runResourceT $
+		sinkResponseFile meterupdate zeroBytesProcessed dest WriteMode resp
diff --git a/Remote/S3.hs b/Remote/S3.hs
index 22f38ef59..f08ed6770 100644
--- a/Remote/S3.hs
+++ b/Remote/S3.hs
@@ -47,6 +47,7 @@ import Creds
 import Annex.UUID
 import Logs.Web
 import Utility.Metered
+import qualified Utility.Url as Url
 import Utility.DataUnits
 import Utility.FileSystemEncoding
 import Annex.Content
@@ -259,22 +260,9 @@ retrieve r info Nothing = case getpublicurl info of
 
 retrieveHelper :: S3Info -> S3Handle -> S3.Object -> FilePath -> MeterUpdate -> Annex ()
 retrieveHelper info h object f p = liftIO $ runResourceT $ do
-	(fr, fh) <- allocate (openFile f WriteMode) hClose
 	let req = S3.getObject (bucket info) object
 	S3.GetObjectResponse { S3.gorResponse = rsp } <- sendS3Handle' h req
-	responseBody rsp $$+- sinkprogressfile fh p zeroBytesProcessed
-	release fr
-  where
-	sinkprogressfile fh meterupdate sofar = do
-		mbs <- await
-		case mbs of
-			Nothing -> return ()
-			Just bs -> do
-				let sofar' = addBytesProcessed sofar (S.length bs)
-				liftIO $ do
-					void $ meterupdate sofar'
-					S.hPut fh bs
-				sinkprogressfile fh meterupdate sofar'
+	Url.sinkResponseFile p zeroBytesProcessed f WriteMode rsp
 
 retrieveCheap :: Key -> AssociatedFile -> FilePath -> Annex Bool
 retrieveCheap _ _ _ = return False
diff --git a/Utility/Metered.hs b/Utility/Metered.hs
index f20050223..54153404e 100644
--- a/Utility/Metered.hs
+++ b/Utility/Metered.hs
@@ -326,7 +326,7 @@ setMeterTotalSize :: Meter -> Integer -> IO ()
 setMeterTotalSize (Meter totalsizev _ _ _) = void . swapMVar totalsizev . Just
 
 -- | Updates the meter, displaying it if necessary.
-updateMeter :: Meter -> BytesProcessed -> IO ()
+updateMeter :: Meter -> MeterUpdate
 updateMeter (Meter totalsizev sv bv displaymeter) new = do
 	now <- getPOSIXTime
 	(old, before) <- swapMVar sv (new, now)
diff --git a/Utility/Url.hs b/Utility/Url.hs
index 33fdf84a5..079f21567 100644
--- a/Utility/Url.hs
+++ b/Utility/Url.hs
@@ -26,6 +26,8 @@ module Utility.Url (
 	assumeUrlExists,
 	download,
 	downloadQuiet,
+	downloadC,
+	sinkResponseFile,
 	downloadPartial,
 	parseURIRelaxed,
 	matchStatusCodeException,
@@ -34,6 +36,7 @@ module Utility.Url (
 
 import Common
 import Utility.Tmp.Dir
+import Utility.Metered
 import qualified BuildInfo
 
 import Network.URI
@@ -45,6 +48,7 @@ import qualified Data.ByteString.Lazy as L
 import Control.Monad.Trans.Resource
 import Network.HTTP.Conduit
 import Network.HTTP.Client (brRead, withResponse)
+import Data.Conduit
 
 #if ! MIN_VERSION_http_client(0,5,0)
 responseTimeoutNone :: Maybe Int
@@ -312,6 +316,111 @@ download' quiet url file uo = do
 		| quiet = [Param s]
 		| otherwise = []
 
+{- Download a perhaps large file, with auto-resume of incomplete downloads.
+ -
+ - By default, conduit is used for the download, except for file: urls,
+ - which are copied. If the url scheme is not supported by conduit, falls
+ - back to using curl.
+ -}
+downloadC :: MeterUpdate -> URLString -> FilePath -> UrlOptions -> IO Bool
+downloadC meterupdate url file uo = go `catchNonAsync` (const $ return False)
+  where
+	go = case parseURIRelaxed url of
+		Just u -> case parseUrlConduit (show u) of
+			Just req -> catchJust
+				-- When http redirects to a protocol which 
+				-- conduit does not support, it will throw
+				-- a StatusCodeException with found302.
+				(matchStatusCodeException (== found302))
+				(downloadconduit req)
+				(const downloadcurl)
+			Nothing
+				| uriScheme u == "file:" -> do
+					let src = unEscapeString (uriPath u)
+					withMeteredFile src meterupdate $
+						L.writeFile file
+					return True
+				| BuildInfo.curl -> downloadcurl
+				| otherwise -> return False
+		Nothing -> return False
+
+	downloadconduit req = catchMaybeIO (getFileSize file) >>= \case
+		Nothing -> runResourceT $ do
+			resp <- http req (httpManager uo)
+			if responseStatus resp == ok200
+				then store zeroBytesProcessed WriteMode resp
+				else return False
+		Just sz -> resumeconduit req sz
+	
+	alreadydownloaded sz s h = s == requestedRangeNotSatisfiable416 
+		&& case lookup hContentRange h of
+			-- This could be improved by fixing
+			-- https://github.com/aristidb/http-types/issues/87
+			Just crh -> crh == B8.fromString ("bytes */" ++ show sz)
+			Nothing -> False
+
+	-- Resume download from where a previous download was interrupted, 
+	-- when supported by the http server. The server may also opt to
+	-- send the whole file rather than resuming.
+	resumeconduit req sz = catchJust
+		(matchStatusCodeHeadersException (alreadydownloaded sz))
+		dl
+		(const $ return True)
+	  where
+		dl = runResourceT $ do
+			let req' = req { requestHeaders = resumeFromHeader sz : requestHeaders req }
+			resp <- http req' (httpManager uo)
+			liftIO $ print ("XXX", responseStatus resp)
+			if responseStatus resp == partialContent206
+				then store (BytesProcessed sz) AppendMode resp
+				else if responseStatus resp == ok200
+					then store zeroBytesProcessed WriteMode resp
+					else return False
+	
+	store initialp mode resp = do
+		sinkResponseFile meterupdate initialp file mode resp
+		return True
+	
+	downloadcurl = do
+		-- curl does not create destination file
+		-- if the url happens to be empty, so pre-create.
+		unlessM (doesFileExist file) $
+			writeFile file ""
+		let headerparams = map (\h -> Param $ "--header=" ++ h) (reqHeaders uo)
+		let opts =
+			[ Param "-sS"
+			, Param "-f"
+			, Param "-L"
+			, Param "-C", Param "-"
+			, Param "-o"
+			]
+		boolSystem "curl" $ addUserAgent uo $ concat
+			[ headerparams
+			, opts
+			, reqParams uo
+			, [File file, File url]
+			]
+
+{- Sinks a Response's body to a file. The file can either be opened in
+ - WriteMode or AppendMode. Updates the meter as data is received.
+ -

(Diff truncated)

diff --git a/doc/related_software.mdwn b/doc/related_software.mdwn
index a2448cf3a..1ea1ae3a4 100644
--- a/doc/related_software.mdwn
+++ b/doc/related_software.mdwn
@@ -50,4 +50,7 @@ designed to interoperate with it.
 * [git-annex-turtle](https://github.com/andrewringler/git-annex-turtle)
   is a macOS app providing Finder integration to git-annex.
 
+* [recastex](http://git-annex.branchable.com/tips/Announcing_recastex_-___40__re__41__podcast__from_your_annex/)
+  publishes podcasts and other files from by git-annex to your phone.
+
 See also [[not]] for software that is *not* related to git-annex, but similar.

diff --git a/doc/related_software.mdwn b/doc/related_software.mdwn
index 5be7dc2a4..a2448cf3a 100644
--- a/doc/related_software.mdwn
+++ b/doc/related_software.mdwn
@@ -4,11 +4,6 @@ designed to interoperate with it.
 * The [[git-annex assistant|assistant]] is included in git-annex,
   and extends its use cases into new territory.
  
-* [gitlab-shell](https://gitlab.com/gitlab-org/gitlab-shell) supports
-  git-annex. So git-annex can be used with repositories served by GitLab,
-  including gitlab.com, or deploy your own.
-  [See GitLab's announcement](https://about.gitlab.com/2015/02/17/gitlab-annex-solves-the-problem-of-versioning-large-binaries-with-git/)
-
 * Emacs Org mode can auto-commit attached files to git-annex.
 
 * [git annex darktable integration](https://github.com/xxv/darktable-git-annex)

diff --git a/doc/tips/Announcing_recastex_-___40__re__41__podcast__from_your_annex.mdwn b/doc/tips/Announcing_recastex_-___40__re__41__podcast__from_your_annex.mdwn
new file mode 100644
index 000000000..9e6d7654f
--- /dev/null
+++ b/doc/tips/Announcing_recastex_-___40__re__41__podcast__from_your_annex.mdwn
@@ -0,0 +1,15 @@
+Hi all,
+
+I've written a simple tool in Python to re-podcast from an annex:  [recastex](https://github.com/stewart123579/recastex)
+
+Starting with the [downloading podcasts](https://git-annex.branchable.com/tips/downloading_podcasts/) page, I've got a number of podcasts on my laptop, but they were not really synced to my podcast app on my phone.  Not a problem any longer.
+
+The app uses the metadata associated with *locally available* files to generate feeds for each of your "subscribed" podcasts - and collects anything else you have (like individual files) into a catch-all feed.
+
+It's designed with git-annex + limited network + privacy in mind separating the public internet queries from the things that can be done over git-annex.
+
+*(As the author of [git-annex-metadata-gui](https://git-annex.branchable.com/tips/a_gui_for_metadata_operations/) said...)*  I hope these can be useful to someone other than myself.
+
+
+
+

diff --git a/doc/bugs/proposal_for_timestamp_semantics/comment_2_8ba9e0c35e7761b99feb55bcdd6065cf._comment b/doc/bugs/proposal_for_timestamp_semantics/comment_2_8ba9e0c35e7761b99feb55bcdd6065cf._comment
new file mode 100644
index 000000000..b7edd53e4
--- /dev/null
+++ b/doc/bugs/proposal_for_timestamp_semantics/comment_2_8ba9e0c35e7761b99feb55bcdd6065cf._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="vrs+annex@ea5fa24dbb279be61a8e50adb638bf8366300717"
+ nickname="vrs+annex"
+ avatar="http://cdn.libravatar.org/avatar/74219abcec6eece8e2c9d4351c2c912c"
+ subject="comment 2"
+ date="2018-04-05T21:01:51Z"
+ content="""
+I have no opinion about what backend to use. If doing it via the metadata system significantly slows down things though and is generally awkward, why not build a separate subsystem?
+
+I don't know what you mean by \"look over all the merged changes and go off and frob timestamps\", but as long as n is on the order of [number of files changed in the commit], updating n files' timestamps sounds reasonable? There's the question of which timestamp has preference in a merge, but that sounds solvable.
+
+I made this a separate bug because it's a specific design proposal; I consider [[todo/does_not_preserve_timestamps]] a tracking bug/user story.
+
+proposal re [[/bugs/file_modification_time_should_be_stored_in_exactly_one_metadata_field/#comment-2ea94161228f0653917b91d4f999153f]]: File and symlink timestamps, after `git-annex-get` or `git-checkout`, are set to whatever's in the repo and then considered immutable. The user can of course change them with `touch`, but if the file is locked while that happens, that's considered a corruption like editing an object file and will be caught by `git-annex-fsck`.
+"""]]

diff --git a/doc/bugs/annex.genmetadata_should_default_to_true/comment_3_7322a9f7d8d01e8daf67bfb08ce7042c._comment b/doc/bugs/annex.genmetadata_should_default_to_true/comment_3_7322a9f7d8d01e8daf67bfb08ce7042c._comment
new file mode 100644
index 000000000..8d0efd3ec
--- /dev/null
+++ b/doc/bugs/annex.genmetadata_should_default_to_true/comment_3_7322a9f7d8d01e8daf67bfb08ce7042c._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="CandyAngel"
+ avatar="http://cdn.libravatar.org/avatar/15c0aade8bec5bf004f939dd73cf9ed8"
+ subject="comment 3"
+ date="2018-04-05T15:45:52Z"
+ content="""
+As far as I am aware, git-annex doesn't store *any* information about any files (including their content!) without fairly explicit instruction to do so.. this would be exception to that general behaviour.
+"""]]

diff --git a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_81cdf91b7720cb0a36bdff2815210435._comment b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_81cdf91b7720cb0a36bdff2815210435._comment
new file mode 100644
index 000000000..5dd975c70
--- /dev/null
+++ b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_81cdf91b7720cb0a36bdff2815210435._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="jhnichol@cce81d2a480707652a3340ea2f24b3dc4b1f808c"
+ nickname="jhnichol"
+ avatar="http://cdn.libravatar.org/avatar/2d05fd7e681bf4838bba8bab538ac65d"
+ subject="I also have this problem"
+ date="2018-04-05T15:25:22Z"
+ content="""
+I'm on OSX 10.11.6. I started with a homebrew install, and today I found the native .dmg app. Could there be a conflict between the homebrew install and the .app?
+I'm a first time user trying to test this program and not getting very far. I have a minimal understanding of git, and therefore shy away from the CLI. 
+"""]]

diff --git a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_599303cc8aef765a8090644c039ecec5._comment b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_599303cc8aef765a8090644c039ecec5._comment
deleted file mode 100644
index af70a384b..000000000
--- a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_599303cc8aef765a8090644c039ecec5._comment
+++ /dev/null
@@ -1,11 +0,0 @@
-[[!comment format=mdwn
- username="jhnichol@cce81d2a480707652a3340ea2f24b3dc4b1f808c"
- nickname="jhnichol"
- avatar="http://cdn.libravatar.org/avatar/2d05fd7e681bf4838bba8bab538ac65d"
- subject="comment 2"
- date="2018-04-05T15:23:42Z"
- content="""
-I'm a first time user trying to test this program and not getting very far. I have a minimal understanding of git, and therefore shy away from the CLI.
-I'm on OSX 10.11.6. I started with a homebrew install, and today I found the native .dmg app.
-Could there be a conflict between the homebrew install and the .app?
-"""]]

diff --git a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_599303cc8aef765a8090644c039ecec5._comment b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_599303cc8aef765a8090644c039ecec5._comment
new file mode 100644
index 000000000..af70a384b
--- /dev/null
+++ b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_599303cc8aef765a8090644c039ecec5._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="jhnichol@cce81d2a480707652a3340ea2f24b3dc4b1f808c"
+ nickname="jhnichol"
+ avatar="http://cdn.libravatar.org/avatar/2d05fd7e681bf4838bba8bab538ac65d"
+ subject="comment 2"
+ date="2018-04-05T15:23:42Z"
+ content="""
+I'm a first time user trying to test this program and not getting very far. I have a minimal understanding of git, and therefore shy away from the CLI.
+I'm on OSX 10.11.6. I started with a homebrew install, and today I found the native .dmg app.
+Could there be a conflict between the homebrew install and the .app?
+"""]]

diff --git a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_20204eb0e8a05333717c2a959ca5fb60._comment b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_20204eb0e8a05333717c2a959ca5fb60._comment
deleted file mode 100644
index 157102057..000000000
--- a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_20204eb0e8a05333717c2a959ca5fb60._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="jhnichol@cce81d2a480707652a3340ea2f24b3dc4b1f808c"
- nickname="jhnichol"
- avatar="http://cdn.libravatar.org/avatar/2d05fd7e681bf4838bba8bab538ac65d"
- subject="I also can't add remotes through the web assistant"
- date="2018-04-05T15:17:43Z"
- content="""
-I'm a first time user trying to test this program and not getting very far.
-I have a minimal understanding of git, and therefore shy away from the CLI.
-"""]]

diff --git a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_3_0ec03eb30ebe37d8b61159eab6cccc2c._comment b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_3_0ec03eb30ebe37d8b61159eab6cccc2c._comment
deleted file mode 100644
index 3ea17fca2..000000000
--- a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_3_0ec03eb30ebe37d8b61159eab6cccc2c._comment
+++ /dev/null
@@ -1,11 +0,0 @@
-[[!comment format=mdwn
- username="jhnichol@cce81d2a480707652a3340ea2f24b3dc4b1f808c"
- nickname="jhnichol"
- avatar="http://cdn.libravatar.org/avatar/2d05fd7e681bf4838bba8bab538ac65d"
- subject="I also can't add remotes through the web assistant"
- date="2018-04-05T15:17:53Z"
- content="""
-I'm a first time user trying to test this program and not getting very far.
-I have a minimal understanding of git, and therefore shy away from the CLI.
-- James
-"""]]

diff --git a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_3_0ec03eb30ebe37d8b61159eab6cccc2c._comment b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_3_0ec03eb30ebe37d8b61159eab6cccc2c._comment
new file mode 100644
index 000000000..3ea17fca2
--- /dev/null
+++ b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_3_0ec03eb30ebe37d8b61159eab6cccc2c._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="jhnichol@cce81d2a480707652a3340ea2f24b3dc4b1f808c"
+ nickname="jhnichol"
+ avatar="http://cdn.libravatar.org/avatar/2d05fd7e681bf4838bba8bab538ac65d"
+ subject="I also can't add remotes through the web assistant"
+ date="2018-04-05T15:17:53Z"
+ content="""
+I'm a first time user trying to test this program and not getting very far.
+I have a minimal understanding of git, and therefore shy away from the CLI.
+- James
+"""]]

diff --git a/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_20204eb0e8a05333717c2a959ca5fb60._comment b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_20204eb0e8a05333717c2a959ca5fb60._comment
new file mode 100644
index 000000000..157102057
--- /dev/null
+++ b/doc/bugs/Can__39__t_add_remotes_through_the_web_assistant/comment_2_20204eb0e8a05333717c2a959ca5fb60._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="jhnichol@cce81d2a480707652a3340ea2f24b3dc4b1f808c"
+ nickname="jhnichol"
+ avatar="http://cdn.libravatar.org/avatar/2d05fd7e681bf4838bba8bab538ac65d"
+ subject="I also can't add remotes through the web assistant"
+ date="2018-04-05T15:17:43Z"
+ content="""
+I'm a first time user trying to test this program and not getting very far.
+I have a minimal understanding of git, and therefore shy away from the CLI.
+"""]]

diff --git a/doc/bugs/file_modification_time_should_be_stored_in_exactly_one_metadata_field/comment_3_7bf423111829ec6b358c6925d012a3e2._comment b/doc/bugs/file_modification_time_should_be_stored_in_exactly_one_metadata_field/comment_3_7bf423111829ec6b358c6925d012a3e2._comment
new file mode 100644
index 000000000..8c5973f25
--- /dev/null
+++ b/doc/bugs/file_modification_time_should_be_stored_in_exactly_one_metadata_field/comment_3_7bf423111829ec6b358c6925d012a3e2._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="vrs+annex@ea5fa24dbb279be61a8e50adb638bf8366300717"
+ nickname="vrs+annex"
+ avatar="http://cdn.libravatar.org/avatar/74219abcec6eece8e2c9d4351c2c912c"
+ subject="comment 3"
+ date="2018-04-05T02:11:41Z"
+ content="""
+If pure functions were part of the design anyway, why not avoid adding derived fields and introduce functions like `year()` etc instead? The semantics would be obvious (you don't assign to a function call) and existing workflows involving the year field would keep working.
+
+As for editing the timestamp in general, I'll post an answer to <https://git-annex.branchable.com/bugs/proposal_for_timestamp_semantics/> later that should address this.
+"""]]

diff --git a/doc/bugs/annex.genmetadata_should_default_to_true/comment_2_0966f8bf944ded0a5199285ed3c12930._comment b/doc/bugs/annex.genmetadata_should_default_to_true/comment_2_0966f8bf944ded0a5199285ed3c12930._comment
new file mode 100644
index 000000000..cea41ad1d
--- /dev/null
+++ b/doc/bugs/annex.genmetadata_should_default_to_true/comment_2_0966f8bf944ded0a5199285ed3c12930._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="vrs+annex@ea5fa24dbb279be61a8e50adb638bf8366300717"
+ nickname="vrs+annex"
+ avatar="http://cdn.libravatar.org/avatar/74219abcec6eece8e2c9d4351c2c912c"
+ subject="comment 2"
+ date="2018-04-05T01:32:32Z"
+ content="""
+1. Not losing data (or alternatively having a great big warning in the manual, or requiring it as a configuration step) should be the default in software that manages files, especially software that advertises a backup usecase on its front page. I do think the current implementation could be improved, which is what I opened <https://git-annex.branchable.com/bugs/file_modification_time_should_be_stored_in_exactly_one_metadata_field/> for. If that's not enough, a simple binary format should do the trick at about the same overhead as regular filesystems while staying future-proof.
+2. Implementing timestamps with a field name that doesn't clash with existing fields should avoid this issue.
+"""]]

diff --git a/Annex/Content.hs b/Annex/Content.hs
index 627e3e518..1816033ed 100644
--- a/Annex/Content.hs
+++ b/Annex/Content.hs
@@ -947,8 +947,8 @@ downloadUrl k p urls file = meteredFile file (Just p) k $
 			( return Url.downloadQuiet
 			, return Url.download
 			)
-		Url.withUrlOptions $ \uo ->
-			anyM (\u -> a u file uo) urls
+		Url.withUrlOptions $ \uo -> 
+			liftIO $ anyM (\u -> a u file uo) urls
 	go (Just basecmd) = anyM (downloadcmd basecmd) urls
 	downloadcmd basecmd url =
 		progressCommand "sh" [Param "-c", Param $ gencmd url basecmd]
diff --git a/Annex/Url.hs b/Annex/Url.hs
index dc68d65fd..52469b58a 100644
--- a/Annex/Url.hs
+++ b/Annex/Url.hs
@@ -9,6 +9,7 @@
 module Annex.Url (
 	module U,
 	withUrlOptions,
+	getUrlOptions,
 	getUserAgent,
 ) where
 
@@ -24,20 +25,24 @@ getUserAgent :: Annex (Maybe U.UserAgent)
 getUserAgent = Annex.getState $ 
 	Just . fromMaybe defaultUserAgent . Annex.useragent
 
-withUrlOptions :: (U.UrlOptions -> Annex a) -> Annex a
-withUrlOptions a = Annex.getState Annex.urloptions >>= \case
-	Just uo -> a uo
+getUrlOptions :: Annex U.UrlOptions
+getUrlOptions = Annex.getState Annex.urloptions >>= \case
+	Just uo -> return uo
 	Nothing -> do
 		uo <- mk
 		Annex.changeState $ \s -> s
 			{ Annex.urloptions = Just uo }
-		a uo
+		return uo
   where
 	mk = mkUrlOptions
 		<$> getUserAgent
 		<*> headers
 		<*> options
+		<*> liftIO (U.newManager U.managerSettings)
 	headers = annexHttpHeadersCommand <$> Annex.getGitConfig >>= \case
 		Just cmd -> lines <$> liftIO (readProcess "sh" ["-c", cmd])
 		Nothing -> annexHttpHeaders <$> Annex.getGitConfig
 	options = map Param . annexWebOptions <$> Annex.getGitConfig
+
+withUrlOptions :: (U.UrlOptions -> Annex a) -> Annex a
+withUrlOptions a = a =<< getUrlOptions
diff --git a/Assistant/Restart.hs b/Assistant/Restart.hs
index 7d6575995..51ad7748a 100644
--- a/Assistant/Restart.hs
+++ b/Assistant/Restart.hs
@@ -95,7 +95,7 @@ newAssistantUrl repo = do
  - warp-tls listens to http, in order to show an error page, so this works.
  -}
 assistantListening :: URLString -> IO Bool
-assistantListening url = catchBoolIO $ exists url' def
+assistantListening url = catchBoolIO $ exists url' =<< defUrlOptions
   where
 	url' = case parseURI url of
 		Nothing -> url
diff --git a/Assistant/Upgrade.hs b/Assistant/Upgrade.hs
index d91b65657..a100e23b5 100644
--- a/Assistant/Upgrade.hs
+++ b/Assistant/Upgrade.hs
@@ -316,7 +316,8 @@ usingDistribution :: IO Bool
 usingDistribution = isJust <$> getEnv "GIT_ANNEX_STANDLONE_ENV"
 
 downloadDistributionInfo :: Assistant (Maybe GitAnnexDistribution)
-downloadDistributionInfo = Url.withUrlOptions $ \uo -> do
+downloadDistributionInfo = do
+	uo <- liftAnnex Url.getUrlOptions
 	gpgcmd <- liftAnnex $ gpgCmd <$> Annex.getGitConfig
 	liftIO $ withTmpDir "git-annex.tmp" $ \tmpdir -> do
 		let infof = tmpdir </> "info"
diff --git a/Assistant/WebApp/Configurators/IA.hs b/Assistant/WebApp/Configurators/IA.hs
index b819f66a2..c46fcf510 100644
--- a/Assistant/WebApp/Configurators/IA.hs
+++ b/Assistant/WebApp/Configurators/IA.hs
@@ -189,7 +189,8 @@ escapeHeader :: String -> String
 escapeHeader = escapeURIString (\c -> isUnescapedInURI c && c /= ' ')
 
 getRepoInfo :: RemoteConfig -> Widget
-getRepoInfo c = Url.withUrlOptions $ \uo ->
+getRepoInfo c = do
+	uo <- liftAnnex Url.getUrlOptions
 	exists <- liftIO $ catchDefaultIO False $ Url.exists url uo
 	[whamlet|
 <a href="#{url}">
diff --git a/CHANGELOG b/CHANGELOG
index e09542f5f..855c51c27 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -12,6 +12,8 @@ git-annex (6.20180317) UNRELEASED; urgency=medium
     don't copy the data metadata from the old version of the file,
     instead use the mtime of the file.
   * Avoid running annex.http-headers-command more than once.
+  * Enable HTTP connection reuse across multiple files, when git-annex
+    uses http-conduit.
 
  -- Joey Hess <id@joeyh.name>  Mon, 19 Mar 2018 23:13:59 -0400
 
diff --git a/Command/AddUrl.hs b/Command/AddUrl.hs
index 1a032a13e..951e5ba15 100644
--- a/Command/AddUrl.hs
+++ b/Command/AddUrl.hs
@@ -196,7 +196,8 @@ startWeb o urlstring = go $ fromMaybe bad $ parseURI urlstring
 		pathmax <- liftIO $ fileNameLengthLimit "."
 		urlinfo <- if relaxedOption (downloadOptions o)
 			then pure Url.assumeUrlExists
-			else Url.withUrlOptions (Url.getUrlInfo urlstring)
+			else Url.withUrlOptions $
+				liftIO . Url.getUrlInfo urlstring
 		file <- adjustFile o <$> case fileOption (downloadOptions o) of
 			Just f -> pure f
 			Nothing -> case Url.urlSuggestedFile urlinfo of
diff --git a/Command/ImportFeed.hs b/Command/ImportFeed.hs
index c51f5c6db..4fdf63b2e 100644
--- a/Command/ImportFeed.hs
+++ b/Command/ImportFeed.hs
@@ -150,7 +150,7 @@ downloadFeed url
 	| Url.parseURIRelaxed url == Nothing = giveup "invalid feed url"
 	| otherwise = do
 		showOutput
-		Url.withUrlOptions $ \ou ->
+		Url.withUrlOptions $ \uo ->
 			liftIO $ withTmpFile "feed" $ \f h -> do
 				hClose h
 				ifM (Url.download url f uo)
@@ -167,7 +167,8 @@ performDownload opts cache todownload = case location todownload of
 				then do
 					urlinfo <- if relaxedOption (downloadOptions opts)
 						then pure Url.assumeUrlExists
-						else Url.withUrlOptions (Url.getUrlInfo url)
+						else Url.withUrlOptions $
+							liftIO . Url.getUrlInfo url
 					let dlopts = (downloadOptions opts)
 						-- force using the filename
 						-- chosen here
diff --git a/Remote/Git.hs b/Remote/Git.hs
index 7bbdb6ea5..250aedd51 100644
--- a/Remote/Git.hs
+++ b/Remote/Git.hs
@@ -336,10 +336,11 @@ inAnnex rmt (State connpool duc) key
 	r = repo rmt
 	checkhttp = do
 		showChecking r
-		ifM (Url.withUrlOptions $ \uo -> anyM (\u -> Url.checkBoth u (keySize key) uo) (keyUrls rmt key))
-			( return True
-			, giveup "not found"
-			)
+		ifM (Url.withUrlOptions $ \uo -> liftIO $
+			anyM (\u -> Url.checkBoth u (keySize key) uo) (keyUrls rmt key))
+				( return True
+				, giveup "not found"
+				)
 	checkremote = 
 		let fallback = Ssh.inAnnex r key
 		in P2PHelper.checkpresent (Ssh.runProto rmt connpool (cantCheck rmt) fallback) key
diff --git a/Remote/S3.hs b/Remote/S3.hs
index 42dacc043..22f38ef59 100644
--- a/Remote/S3.hs
+++ b/Remote/S3.hs
@@ -1,6 +1,6 @@
 {- S3 remotes
  -
- - Copyright 2011-2017 Joey Hess <id@joeyh.name>
+ - Copyright 2011-2018 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -22,7 +22,7 @@ import qualified Data.ByteString as S
 import qualified Data.Map as M
 import Data.Char
 import Network.Socket (HostName)
-import Network.HTTP.Conduit (Manager, newManager)
+import Network.HTTP.Conduit (Manager)
 import Network.HTTP.Client (responseStatus, responseBody, RequestBody(..))
 import Network.HTTP.Types
 import Control.Monad.Trans.Resource
@@ -51,7 +51,7 @@ import Utility.DataUnits
 import Utility.FileSystemEncoding
 import Annex.Content
 import Annex.Url (withUrlOptions)
-import Utility.Url (checkBoth, managerSettings, closeManager)
+import Utility.Url (checkBoth, UrlOptions(..))
 
 type BucketName = String
 
@@ -295,7 +295,7 @@ checkKey r info Nothing k = case getpublicurl info of
 		giveup "No S3 credentials configured"
 	Just geturl -> do

(Diff truncated)

diff --git a/doc/todo/stop_using_curl_and_wget.mdwn b/doc/todo/stop_using_curl_and_wget.mdwn
new file mode 100644
index 000000000..2362d2d87
--- /dev/null
+++ b/doc/todo/stop_using_curl_and_wget.mdwn
@@ -0,0 +1,29 @@
+Currently git-annex uses wget and curl for downloading urls.
+Which is used depends on the situation, since both have their limitations
+and quirks.
+
+This often confuses users, who expect annex.web-options to only apply
+to whichever program git-annex was running, and put in an option that
+breaks the other program. Or, configure a netrc file, which wget uses by
+default, but curl does not.
+
+Also, using these external programs avoids keeping a http connection open
+and pipelining requests, so it makes mass url downloads a lot slower than
+if git-annex used http-conduit to do url downloads itself. [[users/yoh]]
+has requested http pipelining.
+
+For file: ftp: and more unusual urls, http-conduit can't support them.
+git-annex does support those urls, and people rely on that, so it would
+still need to use wget or curl for those.
+
+wget is also not shipped with git-annex on Windows or OSX, only curl is,
+and it would be good to only use one of the programs, not both, when
+handing those unusual urls.
+
+See also, [[support_.netrc_for_fsck_--from_web]]. That some users rely on
+git-annex using wget and a netrc file is kind of problimatic if switching
+to http-conduit which does not support it. Maybe require users to set
+`annex.web-download-command` if they want to make it use something that
+supports netrc?
+
+--[[Joey]]

diff --git a/doc/forum/released_finder_integration_for_macOS_called_git-annex-turtle/comment_5_d496549c4f8bf3500e8ac5c2f2ae0d6e._comment b/doc/forum/released_finder_integration_for_macOS_called_git-annex-turtle/comment_5_d496549c4f8bf3500e8ac5c2f2ae0d6e._comment
new file mode 100644
index 000000000..ed82fa19b
--- /dev/null
+++ b/doc/forum/released_finder_integration_for_macOS_called_git-annex-turtle/comment_5_d496549c4f8bf3500e8ac5c2f2ae0d6e._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 5"""
+ date="2018-04-04T18:17:02Z"
+ content="""
+I guess what would be helpful is if you can take the current git-annex.app
+and copy its files to a directory and then add in the
+git-annex-turtle.app's files in a way that combines them both.
+I don't currently have an OSX desktop to play with that, but if you can
+show me the changes that need to be made to the file tree in git-annex.app
+to merge turtle into it, I can update the build scripts.
+"""]]

diff --git a/doc/bugs/proposal_for_timestamp_semantics/comment_1_a54fc1bda24e04b15490d0eaa1838d28._comment b/doc/bugs/proposal_for_timestamp_semantics/comment_1_a54fc1bda24e04b15490d0eaa1838d28._comment
new file mode 100644
index 000000000..8fe2f552b
--- /dev/null
+++ b/doc/bugs/proposal_for_timestamp_semantics/comment_1_a54fc1bda24e04b15490d0eaa1838d28._comment
@@ -0,0 +1,19 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-04-04T18:01:01Z"
+ content="""
+This is lacking a lot of detail about how this would be accomplished. 
+
+Assuming it's to be accomplished using git-annex metadata, it seems likely
+to signficantly slow down some git-annex operations (which would need to do
+an expensive git-annex branch lookup).
+
+There's also the complication of merging a git-annex
+branch that contains changes to the timestamp metadata. Would git-annex
+need to look over all the merged changes and go off and frob timestamps?
+
+[[todo/does_not_preserve_timestamps]] already contains discussion of this
+topic. I'm not sure that it's productive to discuss it in two different
+places. (Nor does this really seem like a bug report.)
+"""]]

diff --git a/doc/bugs/annex.genmetadata_should_default_to_true/comment_1_215ca5e67a1bc04bcb8bea6062aebf3f._comment b/doc/bugs/annex.genmetadata_should_default_to_true/comment_1_215ca5e67a1bc04bcb8bea6062aebf3f._comment
new file mode 100644
index 000000000..48822f22a
--- /dev/null
+++ b/doc/bugs/annex.genmetadata_should_default_to_true/comment_1_215ca5e67a1bc04bcb8bea6062aebf3f._comment
@@ -0,0 +1,23 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-04-04T17:50:02Z"
+ content="""
+I think there are two potential problems with this change:
+
+1. Bloat. If someone is not going to use that metadata, they may not want
+   the overhead of attaching it to every file.
+
+2. It's entirely possible that a user has chosen to use year/month/day
+   metadata fields for their own purposes. Perhaps they are using it to
+   keep track of the original publication date of files, and so would not
+   want "wrong" values to be automatically added.
+
+And that's essentially the reason why all parts of git-annex that deal
+with specific metadata fields are optional, so there's no default 
+hard-coded semantics to the metadata fields.
+
+(With the exception of git-annex importfeed, which always
+stores the itemid from the rss feed in the metadata, but only because
+it's needed to detect buggy feeds that change their item urls.)
+"""]]

diff --git a/Annex/MetaData.hs b/Annex/MetaData.hs
index f14db0a3e..6280e8c81 100644
--- a/Annex/MetaData.hs
+++ b/Annex/MetaData.hs
@@ -22,7 +22,6 @@ import Annex.CatFile
 import Utility.Glob
 
 import qualified Data.Set as S
-import qualified Data.Map as M
 import Data.Time.Calendar
 import Data.Time.Clock
 import Data.Time.Clock.POSIX
@@ -41,28 +40,43 @@ genMetaData :: Key -> FilePath -> FileStatus -> Annex ()
 genMetaData key file status = do
 	catKeyFileHEAD file >>= \case
 		Nothing -> noop
-		Just oldkey -> 
-			whenM (copyMetaData oldkey key)
+		Just oldkey ->
+			-- Have to copy first, before adding any
+			-- more metadata, because copyMetaData does not
+			-- preserve any metadata already on key.
+			whenM (copyMetaData oldkey key <&&> (not <$> onlydatemeta oldkey)) $
 				warncopied
 	whenM (annexGenMetaData <$> Annex.getGitConfig) $ do
-		curr <- getCurrentMetaData key
-		addMetaData key (dateMetaData mtime curr)
+		old <- getCurrentMetaData key
+		addMetaData key (dateMetaData mtime old)
   where
 	mtime = posixSecondsToUTCTime $ realToFrac $ modificationTime status
 	warncopied = warning $ 
 		"Copied metadata from old version of " ++ file ++ " to new version. " ++ 
 		"If you don't want this copied metadata, run: git annex metadata --remove-all " ++ file
+	-- If the only fields copied were date metadata, and they'll
+	-- be overwritten with the current mtime, no need to warn about
+	-- copying.
+	onlydatemeta oldkey = ifM (annexGenMetaData <$> Annex.getGitConfig)
+		( null . filter (not . isDateMetaField . fst) . fromMetaData 
+			<$> getCurrentMetaData oldkey
+		, return False
+		)
 
 {- Generates metadata for a file's date stamp.
- - Does not overwrite any existing metadata values. -}
+ -
+ - Any date fields in the old metadata will be overwritten.
+ - 
+ - Note that the returned MetaData does not contain all the input MetaData,
+ - only changes to add the date fields. -}
 dateMetaData :: UTCTime -> MetaData -> MetaData
-dateMetaData mtime old = MetaData $ M.fromList $ filter isnew
-	[ (yearMetaField, S.singleton $ toMetaValue $ show y)
-	, (monthMetaField, S.singleton $ toMetaValue $ show m)
-	, (dayMetaField, S.singleton $ toMetaValue $ show d)
-	]
+dateMetaData mtime old = modMeta old $
+	(SetMeta yearMetaField $ S.singleton $ toMetaValue $ show y)
+		`ComposeModMeta`
+	(SetMeta monthMetaField $ S.singleton $ toMetaValue $ show m)
+		`ComposeModMeta`
+	(SetMeta dayMetaField $ S.singleton $ toMetaValue $ show d)
   where
-	isnew (f, _) = S.null (currentMetaDataValues f old)
 	(y, m, d) = toGregorian $ utctDay mtime
 
 {- Parses field=value, field+=value, field-=value, field?=value -}
diff --git a/Annex/MetaData/StandardFields.hs b/Annex/MetaData/StandardFields.hs
index b9ea47e2f..b2a3f3442 100644
--- a/Annex/MetaData/StandardFields.hs
+++ b/Annex/MetaData/StandardFields.hs
@@ -10,6 +10,7 @@ module Annex.MetaData.StandardFields (
 	yearMetaField,
 	monthMetaField,
 	dayMetaField,
+	isDateMetaField,
 	lastChangedField,
 	mkLastChangedField,
 	isLastChangedField
@@ -31,6 +32,12 @@ monthMetaField = mkMetaFieldUnchecked "month"
 dayMetaField :: MetaField
 dayMetaField = mkMetaFieldUnchecked "day"
 
+isDateMetaField :: MetaField -> Bool
+isDateMetaField yearMetaField = True
+isDateMetaField monthMetaField = True
+isDateMetaField datMetaField = True
+isDateMetaField _ = False
+
 lastChangedField :: MetaField
 lastChangedField = mkMetaFieldUnchecked lastchanged
 
diff --git a/CHANGELOG b/CHANGELOG
index 7c6d2eb84..cf43c0f0c 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -8,6 +8,9 @@ git-annex (6.20180317) UNRELEASED; urgency=medium
     to configure transfer retries.
   * Also do forward retrying in cases where no exception is thrown,
     but the transfer failed.
+  * When adding a new version of a file, and annex.genmetadata is enabled,
+    don't copy the data metadata from the old version of the file,
+    instead use the mtime of the file.
 
  -- Joey Hess <id@joeyh.name>  Mon, 19 Mar 2018 23:13:59 -0400
 
diff --git a/Types/MetaData.hs b/Types/MetaData.hs
index bc27c345f..279aacbb8 100644
--- a/Types/MetaData.hs
+++ b/Types/MetaData.hs
@@ -252,13 +252,15 @@ data ModMeta
 	= AddMeta MetaField MetaValue
 	| DelMeta MetaField (Maybe MetaValue)
 	-- ^ delete value of a field. With Just, only that specific value
-	-- is deleted; with Nothing, all current values are deleted.a
+	-- is deleted; with Nothing, all current values are deleted.
 	| DelAllMeta
 	-- ^ delete all currently set metadata
 	| SetMeta MetaField (S.Set MetaValue)
 	-- ^ removes any existing values
 	| MaybeSetMeta MetaField MetaValue
 	-- ^ set when field has no existing value
+	| ComposeModMeta ModMeta ModMeta
+	-- ^ composing multiple modifications
 	deriving (Show)
 
 {- Applies a ModMeta, generating the new MetaData.
@@ -279,6 +281,7 @@ modMeta m (SetMeta f s) = updateMetaData' f s $
 modMeta m (MaybeSetMeta f v)
 	| S.null (currentMetaDataValues f m) = updateMetaData f v emptyMetaData
 	| otherwise = emptyMetaData
+modMeta m (ComposeModMeta a b) = unionMetaData (modMeta m a) (modMeta m b)
 
 {- Avoid putting too many fields in the map; extremely large maps make
  - the seriaization test slow due to the sheer amount of data.
diff --git a/doc/bugs/if_annex.genmetadata_is_true__44___modification_metadata_is_imported_from_older_file_versions_after_unlock+add.mdwn b/doc/bugs/if_annex.genmetadata_is_true__44___modification_metadata_is_imported_from_older_file_versions_after_unlock+add.mdwn
index efd4ad492..f8a77f32d 100644
--- a/doc/bugs/if_annex.genmetadata_is_true__44___modification_metadata_is_imported_from_older_file_versions_after_unlock+add.mdwn
+++ b/doc/bugs/if_annex.genmetadata_is_true__44___modification_metadata_is_imported_from_older_file_versions_after_unlock+add.mdwn
@@ -20,3 +20,4 @@ output: `Copied metadata from old version of bar to new version. If you don't wa
 ### What version of git-annex are you using? On what operating system?
 6.20180227, various Linuxes.
 
+> [[fixed|done]] --[[Joey]]
diff --git a/doc/bugs/if_annex.genmetadata_is_true__44___modification_metadata_is_imported_from_older_file_versions_after_unlock+add/comment_1_b215c8ce0f2094a116672572951b19fc._comment b/doc/bugs/if_annex.genmetadata_is_true__44___modification_metadata_is_imported_from_older_file_versions_after_unlock+add/comment_1_b215c8ce0f2094a116672572951b19fc._comment
new file mode 100644
index 000000000..d598c7afc
--- /dev/null
+++ b/doc/bugs/if_annex.genmetadata_is_true__44___modification_metadata_is_imported_from_older_file_versions_after_unlock+add/comment_1_b215c8ce0f2094a116672572951b19fc._comment
@@ -0,0 +1,25 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-04-04T16:51:36Z"
+ content="""
+This happens in Annex.MetaData.genMetaData. First it copies
+metadata from the oldkey to the new key. Then it
+calls addMetaData on the dateMetaData of the file's mtime.
+
+In dateMetaData, there's a `filter isnew`, which makes
+it filter out any of the date fields that already exist
+in the metadata of the new key.
+
+This was done intentionally, see 
+[[!commit 8d5158fa3151be4c7fc698b96ed887b43ac48769]]
+But that's lacking an explanation of why it was done.
+
+Note that dateMetaData is also used in Command.ImportFeed
+to convert a itempubdate into year and month metadata.
+But changing its behavior to override old dates 
+would not change that code path.
+
+So, I don't see a problem with making this change, and have gone ahead and
+done it.
+"""]]

diff --git a/doc/bugs/file_modification_time_should_be_stored_in_exactly_one_metadata_field/comment_2_32285971869ec56460b3f0eee4edfa23._comment b/doc/bugs/file_modification_time_should_be_stored_in_exactly_one_metadata_field/comment_2_32285971869ec56460b3f0eee4edfa23._comment
new file mode 100644
index 000000000..68c177c83
--- /dev/null
+++ b/doc/bugs/file_modification_time_should_be_stored_in_exactly_one_metadata_field/comment_2_32285971869ec56460b3f0eee4edfa23._comment
@@ -0,0 +1,30 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2018-04-04T16:40:19Z"
+ content="""
+One use case for the separate fields is `git-annex view`,
+where they allow creating view branches with yyyy/mm/ subdirectories
+or artist/yyyy/ or whatever kind of thing the user wants.
+
+Part of the original metadata design, which never got implemented,
+was to allow pure functions on metadata, that added to the set of available
+metadata. So a unix timestamp could be what's stored in metadata, and then
+the individual fields (and more fields that are currently omitted to avoid
+bloat) derived from it.
+
+I'd be happy to see that part of the metadata design actually happen.
+There are some potential problems with it that need to be thought about,
+such as:
+
+* What should happen if the user runs `git annex metadata` to change
+  or delete such a derived metadata field?
+* If a unix timestamp is stored, it's harder for the user to make
+  some simple edit, such as adjusting the year of a file to the year they
+  want. Should a change to a derived year field propigate back to the unix
+  timestamp? (Would need reversible functions.)
+
+(BTW, the timestamps on metadata fields come for "free" in that the
+underlying metadata storage log necessarily contains timestamps of when
+each part of the metadata was changed.)
+"""]]

diff --git a/doc/bugs/filemanager_integration_broken/comment_1_aa5afe9af21e534a6c0683064296d9e6._comment b/doc/bugs/filemanager_integration_broken/comment_1_aa5afe9af21e534a6c0683064296d9e6._comment
new file mode 100644
index 000000000..3a4184791
--- /dev/null
+++ b/doc/bugs/filemanager_integration_broken/comment_1_aa5afe9af21e534a6c0683064296d9e6._comment
@@ -0,0 +1,30 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-04-04T16:26:07Z"
+ content="""
+Did you install git-annex from the nix package?
+
+The desktop file is supposed to be shipped with the OS's package of
+git-annex, in /etc so it's available for file managers to use as soon as
+it's installed. `make install` arranges for that to happen. 
+
+If nix is building
+git-annex in some other way, they may be leaving out such ancillary files.
+(And from what I know of nix, there would be some complication providing 
+files in /etc.)
+
+The other way the desktop file is installed was added for
+the git-annex standalone bundle, and installs it into the home directory
+of the user who starts the git-annex webapp. You've correctly
+identified the (undocumented) `GIT_ANNEX_PACKAGE_INSTALL=1` that informs
+git-annex that it's running in such a situation. But there's also a
+`GIT_ANNEX_APP_BASE`
+that it expects to point to where the standalone bundle is located
+(in order to use an icon file from there in the desktop file),
+and it doesn't install the desktop file when that's not set.
+
+I think that the nix package maintainer should be looped into this,
+so we can come up with some solution that works out of the box with those
+packages.
+"""]]

diff --git a/doc/install/OSX/comment_18_5eca3153048d2b3b4fa707d6b8a5ebeb._comment b/doc/install/OSX/comment_18_5eca3153048d2b3b4fa707d6b8a5ebeb._comment
new file mode 100644
index 000000000..31d475778
--- /dev/null
+++ b/doc/install/OSX/comment_18_5eca3153048d2b3b4fa707d6b8a5ebeb._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="achilleas.k@14be77d42a1252fab5ec9dbf4e5ea03c5833e8c8"
+ nickname="achilleas.k"
+ avatar="http://cdn.libravatar.org/avatar/ed6c67c4d8e6c6850930e16eaf85a771"
+ subject="comment 18"
+ date="2018-04-04T16:27:54Z"
+ content="""
+Right. Should have searched the bug reports. Thanks and sorry for the noise.
+"""]]

diff --git a/doc/preferred_content/standard_groups/comment_6_31fa43557e7585dec5dd105531b0fc10._comment b/doc/preferred_content/standard_groups/comment_6_31fa43557e7585dec5dd105531b0fc10._comment
new file mode 100644
index 000000000..6ca9aa8fe
--- /dev/null
+++ b/doc/preferred_content/standard_groups/comment_6_31fa43557e7585dec5dd105531b0fc10._comment
@@ -0,0 +1,18 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""re: How do I mark a specialremote (box.com) as being in the transfer group (cli)"""
+ date="2018-04-04T16:21:54Z"
+ content="""
+Up at the top of this page there's an example to change the group
+of the local repository and to make it use the standard preferred content settings for the
+group:
+
+    git annex wanted . standard
+    git annex group . manual
+
+The same commands can be used to configure a remote. Just put the name of the remote
+instead of the "." that is used the refer to the current directory's repository.
+
+    git annex wanted myremote standard
+    git annex group myremote transfer
+"""]]

diff --git a/doc/install/Android/comment_13_47a86c573b9673811f790f3219c5d212._comment b/doc/install/Android/comment_13_47a86c573b9673811f790f3219c5d212._comment
deleted file mode 100644
index fc59a6e98..000000000
--- a/doc/install/Android/comment_13_47a86c573b9673811f790f3219c5d212._comment
+++ /dev/null
@@ -1,9 +0,0 @@
-[[!comment format=mdwn
- username="cristianpolanco2015@04c392af1063e289871c7a1ae208794e1105ba65"
- nickname="cristianpolanco2015"
- avatar="http://cdn.libravatar.org/avatar/8787a59fc83d3a4d4ef0fd131c59e83d"
- subject="Play Store App Download Free"
- date="2018-03-29T15:27:19Z"
- content="""
-Download Google Play Store app free - Play Store https://playstore.zone/
-"""]]

diff --git a/doc/forum/Sync_Enabled__47__Disabled_Status/comment_1_cfe98bdd50cea0fd0e36a42892262084._comment b/doc/forum/Sync_Enabled__47__Disabled_Status/comment_1_cfe98bdd50cea0fd0e36a42892262084._comment
new file mode 100644
index 000000000..2649c7d57
--- /dev/null
+++ b/doc/forum/Sync_Enabled__47__Disabled_Status/comment_1_cfe98bdd50cea0fd0e36a42892262084._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-04-04T16:18:11Z"
+ content="""
+The webapp is changing the git configuration 
+"remote.<name>.annex-sync" to false to disable and back to true to enable.
+
+You can use `git config` to change that yourself. 
+
+However, if you have the assistant running in the background and make such
+a git configuration change, it won't notice the change, and will keep
+running with the old configuration. So you'll want to restart it by running
+`git annex assistant --stop; git annex assistant`
+"""]]

diff --git a/doc/forum/what_rules_is_git_annex_sync_--content_using_to_send_files_out/comment_3_368403625451415c66b9e62005d9ddea._comment b/doc/forum/what_rules_is_git_annex_sync_--content_using_to_send_files_out/comment_3_368403625451415c66b9e62005d9ddea._comment
new file mode 100644
index 000000000..43822f3ca
--- /dev/null
+++ b/doc/forum/what_rules_is_git_annex_sync_--content_using_to_send_files_out/comment_3_368403625451415c66b9e62005d9ddea._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2018-04-04T16:14:45Z"
+ content="""
+I just wanted to say thanks to andrew for a really excellent response here.
+I've answered plenty of such questions less well.
+I hope you answer many more questions about git-annex with such care and
+attention to detail.
+
+And, thanks to MatusGoljer for following up to confirm that you were
+misunderstanding -- and I agree that you should be able to adjust the
+preferred content expression to do what you want it to do.
+"""]]

diff --git a/doc/bugs/annex_metadata___40__not_--batch__39__ed__41___is_not_aware_of_files_added_via_addurls_--batch/comment_1_4169fceac6ef95400ddc79652558dc13._comment b/doc/bugs/annex_metadata___40__not_--batch__39__ed__41___is_not_aware_of_files_added_via_addurls_--batch/comment_1_4169fceac6ef95400ddc79652558dc13._comment
new file mode 100644
index 000000000..f02fd8669
--- /dev/null
+++ b/doc/bugs/annex_metadata___40__not_--batch__39__ed__41___is_not_aware_of_files_added_via_addurls_--batch/comment_1_4169fceac6ef95400ddc79652558dc13._comment
@@ -0,0 +1,20 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-04-04T16:01:21Z"
+ content="""
+I guess you expected annex.queuesize to be set to 1.
+However, that would mean every single file that addurl adds
+needs the whole git index file to be rewritten so other commands can
+immediately know about it. Which could be very slow, which is why that is
+not the default.
+
+What you can do is use `git annex addurl --batch --json` and observe the
+key that it reports it's added. Then pass that key into `git annex metadata --batch --json`
+to add metadata to the key, which will work before the file ever gets
+added to the git index, and much more efficiently than relying on the
+index.
+
+(Pretty sure this has came up before and that I suggested the same thing
+then.)
+"""]]

diff --git a/doc/install/comment_6_4ae37152bc9765cea921b436fb2785d8._comment b/doc/install/comment_6_4ae37152bc9765cea921b436fb2785d8._comment
new file mode 100644
index 000000000..c06c13ee2
--- /dev/null
+++ b/doc/install/comment_6_4ae37152bc9765cea921b436fb2785d8._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""re: git-annex-webapp missing from Fedora 27"""
+ date="2018-04-04T15:59:51Z"
+ content="""
+@dittigas git-annex can be built without the webapp,
+if the libraries it uses are not made available at build time.
+I suggest you get in touch with the Fedora maintainers and ask them to
+enable the webapp in their builds.
+"""]]

diff --git a/doc/bugs/does_not_handle_youtube_playlists.mdwn b/doc/bugs/does_not_handle_youtube_playlists.mdwn
index 1df49be37..213fa9632 100644
--- a/doc/bugs/does_not_handle_youtube_playlists.mdwn
+++ b/doc/bugs/does_not_handle_youtube_playlists.mdwn
@@ -40,3 +40,4 @@ git annex addurl --debug   6.49s user 0.32s system 18% cpu 35.834 total
 """]]
 
 
+> importfeed supports youtube playlists, once you look up their rss feed. [[done]] --[[Joey]]

diff --git a/doc/install/OSX/comment_17_06e8fc7a4670a42c249f90a3602c3523._comment b/doc/install/OSX/comment_17_06e8fc7a4670a42c249f90a3602c3523._comment
new file mode 100644
index 000000000..e4f3b1594
--- /dev/null
+++ b/doc/install/OSX/comment_17_06e8fc7a4670a42c249f90a3602c3523._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""re: dmg (possibly) requires rebuild"""
+ date="2018-04-04T15:56:38Z"
+ content="""
+@achilleas.k, that was [this bug](https://git-annex.branchable.com/Error_running_git-annex.app_on_OS_X_10.13.1___40__17B1003__41__/) which should be fixed
+now.
+
+Please file bug reports for issues with the OSX build, don't report them
+here.
+"""]]

diff --git a/doc/users/candyangel.mdwn b/doc/users/candyangel.mdwn
index 4c9aedadc..e40b7a693 100644
--- a/doc/users/candyangel.mdwn
+++ b/doc/users/candyangel.mdwn
@@ -3,8 +3,64 @@ I hang out in #git-annex on OFTC (come and chat/ask for help).
 [Getting mentioned by Joey](http://git-annex.branchable.com/devblog/day_277__thanks/) is definitely a (if not **the**) highlight of my involvement in the git-annex community. That really made my month!
 
 Things that I am (or plan to) work on related to git-annex (in some way):
+
 * FUSE filesystem for dynamic get/drop of content
 * FUSE filesystem for metadata change/views (like tagsistant)
 * Patchset for ikiwiki which allows it to be used with perlbrew libs
 * Support for modifying git-annex metadata in my file sorting app
 * Support for git-annex in my VR environment
+
+# sorting annex
+
+## diagram
+
+      +------+
+      |      | clone
+      | base ------------+---------------------------+
+      |      |           |                           |
+      +----|-+           |                           |
+         clone    +------|-----+              +------|-----+
+           |      |            |              |            |
+           |      | hd1-serial | < populate > | hd3-serial |
+           |      |            |              |            |
+           |      +------|-----+              +------|-----+
+           |             |                           |
+           |           clone                       clone
+           |             |                           |
+           |      +------|-----+              +------|-----+
+           |      |            |              |            |
+           |      | hd2-serial |   < get >    | hd4-serial |
+           |      |            |              |            |
+           |      +------|-----+              +------|-----+
+           |             |                           |
+    +------|------+      |                           |
+    |             |      |                           |
+    | collate.git -------+---------------------------+
+    |             |remote add >
+    +------|------+
+           |
+         clone
+           |
+      +----|----+
+      |         |
+      | sorting |
+      |         |
+      +---------+
+
+## explanation
+
+* *base* created and a mkannex.sh script is added to it which inits a git-annex repository with the desired settings
+* *base* is cloned to an empty drive as the "type" of files to be added (hd1, hd3)
+  * types are file, filelink, keyfile and keylink
+  * e.g. `git clone ${base_uri} keyfile`
+* mkannex.sh is run and repository is named ${drive_serial}_${type}
+* annex is populated with contents of $type, in directories created with `uuidgen -r` to prevent collision
+* populated repository (hd1, hd3) is cloned to another drive (hd2, hd4)
+* mkannex.sh is run and repository is named ${drive_serial}_${type}
+* contents are `git annex get`'d from the populated drives
+
+* *collate.git* cloned from *base* and mkannex.sh run, named "collate"
+* remotes added to collate.git for secondary drives (hd2, hd4)
+* *collate.git* syncs metadata, learning of all repositories and their contents
+* *sorting* is cloned from *collate.git* to do sorting activities
+  * read-only remotes added for the content drives as required

diff --git a/doc/git-annex-copy.mdwn b/doc/git-annex-copy.mdwn
index e817fd618..da48c5368 100644
--- a/doc/git-annex-copy.mdwn
+++ b/doc/git-annex-copy.mdwn
@@ -47,7 +47,7 @@ Copies the content of files from or to another remote.
   and always check if the remote has content. Can be useful if the location
   tracking information is out of date.
 
-* `--all`
+* `--all` `-A`
 
   Rather than specifying a filename or path to copy, this option can be
   used to copy all available versions of all files.
diff --git a/doc/git-annex-drop.mdwn b/doc/git-annex-drop.mdwn
index 0c07e150d..6975e4a08 100644
--- a/doc/git-annex-drop.mdwn
+++ b/doc/git-annex-drop.mdwn
@@ -35,7 +35,7 @@ safe to do so.
   the last repository that is storing their content. Data loss can
   result from using this option.
 
-* `--all`
+* `--all` `-A`
 
   Rather than specifying a filename or path to drop, this option can be
   used to drop all available versions of all files.
diff --git a/doc/git-annex-fsck.mdwn b/doc/git-annex-fsck.mdwn
index a9392935f..55134df6a 100644
--- a/doc/git-annex-fsck.mdwn
+++ b/doc/git-annex-fsck.mdwn
@@ -65,7 +65,7 @@ With parameters, only the specified files are checked.
   To verify data integrity only while disregarding required number of copies,
   use `--numcopies=1`.
 
-* `--all`
+* `--all` `-A`
 
   Normally only the files in the currently checked out branch
   are fscked. This option causes all versions of all files to be fscked.
diff --git a/doc/git-annex-get.mdwn b/doc/git-annex-get.mdwn
index 2f3981165..89c5388fb 100644
--- a/doc/git-annex-get.mdwn
+++ b/doc/git-annex-get.mdwn
@@ -58,7 +58,7 @@ or transferring them from some kind of key-value store.
   as git-annex does not know the associated file, and the associated file
   may not even be in the current git working directory.
 
-* `--all`
+* `--all` `-A`
 
   Rather than specifying a filename or path to get, this option can be
   used to get all available versions of all files.
diff --git a/doc/git-annex-inprogress.mdwn b/doc/git-annex-inprogress.mdwn
index 1a849e561..5e5390eb2 100644
--- a/doc/git-annex-inprogress.mdwn
+++ b/doc/git-annex-inprogress.mdwn
@@ -35,7 +35,7 @@ some file formats can be usefully streamed in this way.
   traverse. It's most efficient to specify a the file you are interested
   in, or to use `--all`
 
-* `--all`
+* `--all` `-A`
 
   Rather than specifying a filename or path, this option can be
   used to access all files that are currently being downloaded.
diff --git a/doc/git-annex-log.mdwn b/doc/git-annex-log.mdwn
index b2d9c0db9..1c65e4548 100644
--- a/doc/git-annex-log.mdwn
+++ b/doc/git-annex-log.mdwn
@@ -34,7 +34,7 @@ showing each repository they were added to ("+") and removed from ("-").
   The [[git-annex-matching-options]](1)
   can be used to specify files to act on.
 
-* `--all`
+* `--all` `-A`
 
   Shows location log changes to all files, with the most recent changes first.
   In this mode, the names of files are not available and keys are displayed
diff --git a/doc/git-annex-metadata.mdwn b/doc/git-annex-metadata.mdwn
index 7786d999b..5e86077cc 100644
--- a/doc/git-annex-metadata.mdwn
+++ b/doc/git-annex-metadata.mdwn
@@ -80,7 +80,7 @@ automatically.
   The [[git-annex-matching-options]](1)
   can be used to specify files to act on.
 
-* `--all`
+* `--all` `-A`
 
   Specify instead of a file to get/set metadata on all known keys.
 
diff --git a/doc/git-annex-mirror.mdwn b/doc/git-annex-mirror.mdwn
index d665a6e77..c66a0dd9f 100644
--- a/doc/git-annex-mirror.mdwn
+++ b/doc/git-annex-mirror.mdwn
@@ -36,7 +36,7 @@ contents. Use [[git-annex-sync]](1) for that.
   Enables parallel transfers with up to the specified number of jobs
   running at once. For example: `-J10`
 
-* `--all`
+* `--all` `-A`
 
   Mirror all objects stored in the git annex, not only objects used by
   currently existing files. 
diff --git a/doc/git-annex-move.mdwn b/doc/git-annex-move.mdwn
index ce932b198..0f0795889 100644
--- a/doc/git-annex-move.mdwn
+++ b/doc/git-annex-move.mdwn
@@ -30,7 +30,7 @@ Moves the content of files from or to another remote.
   Enables parallel transfers with up to the specified number of jobs
   running at once. For example: `-J10`
 
-* `--all`
+* `--all` `-A`
 
   Rather than specifying a filename or path to move, this option can be
   used to move all available versions of all files.
diff --git a/doc/git-annex-sync.mdwn b/doc/git-annex-sync.mdwn
index f4d644fd3..2e6912b15 100644
--- a/doc/git-annex-sync.mdwn
+++ b/doc/git-annex-sync.mdwn
@@ -94,7 +94,7 @@ by running "git annex sync" on the remote.
 
   This option can be repeated multiple times with different paths.
 
-* `--all`
+* `--all` `-A`
 
   This option, when combined with `--content`, makes all available versions
   of all files be synced, when preferred content settings allow.
diff --git a/doc/git-annex-whereis.mdwn b/doc/git-annex-whereis.mdwn
index 8ce8c6df1..9822c5703 100644
--- a/doc/git-annex-whereis.mdwn
+++ b/doc/git-annex-whereis.mdwn
@@ -31,7 +31,7 @@ For example:
 
   Show where a particular git-annex key is located.
 
-* `--all`
+* `--all` `-A`
 
   Show whereis information for all known keys.
 

diff --git a/doc/git-annex-inprogress.mdwn b/doc/git-annex-inprogress.mdwn
index 1a595cba1..1a849e561 100644
--- a/doc/git-annex-inprogress.mdwn
+++ b/doc/git-annex-inprogress.mdwn
@@ -25,16 +25,26 @@ some file formats can be usefully streamed in this way.
 
 # OPTIONS
 
-* file matching options
- 
-  The [[git-annex-matching-options]](1)
-  can be used to specify files to access.
+* `[path ..]`
+
+  The files or directories whose partially downloaded content you want to
+  access.
+
+  Note that, when no path is specified, it defaults to all files in the
+  current working directory, and subdirectories, which can take a while to
+  traverse. It's most efficient to specify a the file you are interested
+  in, or to use `--all`
 
 * `--all`
 
   Rather than specifying a filename or path, this option can be
   used to access all files that are currently being downloaded.
 
+* file matching options
+ 
+  The [[git-annex-matching-options]](1)
+  can be used to specify files to access.
+
 # EXIT STATUS
 
 If any of the requested files are not currently being downloaded,

diff --git a/doc/thanks/list b/doc/thanks/list
index 5b95a8ae2..2032312ae 100644
--- a/doc/thanks/list
+++ b/doc/thanks/list
@@ -95,3 +95,5 @@ Ethan Aubin,
 Pedro Luz, 
 John Pellman, 
 Duncan Holm, 
+Eric Drechsel, 
+Eric Prestemon, 

diff --git a/doc/bugs/howto_guarantee_a_single_instance_of_a_special_remote__63__/comment_5_098b2e2c51f70d495fe1587220173f55._comment b/doc/bugs/howto_guarantee_a_single_instance_of_a_special_remote__63__/comment_5_098b2e2c51f70d495fe1587220173f55._comment
new file mode 100644
index 000000000..49afab2c0
--- /dev/null
+++ b/doc/bugs/howto_guarantee_a_single_instance_of_a_special_remote__63__/comment_5_098b2e2c51f70d495fe1587220173f55._comment
@@ -0,0 +1,30 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 5"""
+ date="2018-04-03T19:17:32Z"
+ content="""
+PREPARE-FAILURE is documented as "the special remote cannot be used"
+and so I don't think it makes sense for git-annex to use the previously
+started instance of the program if a later one fails like that.
+
+It would need a new response to PREPARE. And some possibly not
+insignificant changes in Remote/External.hs to support it. In particular,
+Remote/External.hs currently delays sending PREPARE until the first time
+it uses a special remote, but this seems to need PREPARE to be sent
+earlier, when it starts up the special remote, so it can detect the new
+response and remember that it should not try to start up any more
+concurrent instances and instead use any already started instance.
+
+The best argument for doing it, I think, is if several different external
+special remote programs really only support a single instance running at a
+time, and if supporting that inside git-annex would be enough of a win,
+rather than making those programs do their own locking.
+
+Hmm, an external  special remote program can't just block its response to
+PREPARE when another instance is running, because it would never be able to
+un-block. So it seems they would have to use finer-grained locking when
+responding to eg TRANSFER. I don't know if anything other than datalad
+needs such locking (and IIUC datalad already got the necessary locking), 
+but it does seem like it would be worth adding an extra PREPARE response
+to avoid needing to so complicate external special remote programs.
+"""]]

diff --git a/doc/related_software.mdwn b/doc/related_software.mdwn
index ce65325f9..5be7dc2a4 100644
--- a/doc/related_software.mdwn
+++ b/doc/related_software.mdwn
@@ -23,6 +23,9 @@ designed to interoperate with it.
 * [DataLad](http://datalad.org/) uses git-annex to provide access to
   scientific data available from various sources.
 
+* [GIN](https://web.gin.g-node.org/) uses git-annex to manage data for
+  neuroscience.
+
 * The [Baobáxia](https://github.com/RedeMocambos/baobaxia) project
   built by the Brazilian [Mocambos network](http://www.mocambos.net/)
   is [using git-annex to connect isolated communities](http://www.modspil.dk/itpolitik/baob_xia.html).

diff --git a/doc/bugs/howto_guarantee_a_single_instance_of_a_special_remote__63__/comment_4_e90ba33f7506745582d18841ae9a9531._comment b/doc/bugs/howto_guarantee_a_single_instance_of_a_special_remote__63__/comment_4_e90ba33f7506745582d18841ae9a9531._comment
new file mode 100644
index 000000000..5dcc4d036
--- /dev/null
+++ b/doc/bugs/howto_guarantee_a_single_instance_of_a_special_remote__63__/comment_4_e90ba33f7506745582d18841ae9a9531._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2018-04-02T17:13:33Z"
+ content="""
+Ok so I misunderstood this, it's entirely about -J mode and external
+special remote processes. One is started per parallelism level, because
+otherwise nothing much would actually be done in parallel. 
+
+You wanted to start only one because reasons, which IIRC from talking
+with you were in the meantime fixed in your special remore program using
+locking.
+"""]]

diff --git a/doc/forum/Sync_Enabled__47__Disabled_Status.mdwn b/doc/forum/Sync_Enabled__47__Disabled_Status.mdwn
new file mode 100644
index 000000000..196a67899
--- /dev/null
+++ b/doc/forum/Sync_Enabled__47__Disabled_Status.mdwn
@@ -0,0 +1 @@
+In the webapp I can see which repos have syncing enabled or disabled, and toggle the property by clicking the current status. How can I do this in the terminal? I want to be able to see the current syncing status of the repository, and to change it, without opening the webapp.

diff --git a/doc/install/ArchLinux.mdwn b/doc/install/ArchLinux.mdwn
index b38571b98..6919c4cbe 100644
--- a/doc/install/ArchLinux.mdwn
+++ b/doc/install/ArchLinux.mdwn
@@ -2,19 +2,15 @@ There is now an [official git-annex package for Arch](https://www.archlinux.org/
 
 	pacman -S git-annex
 
-There are at least four non non-official packages for git-annex in the Arch Linux User Repository. Any of these may be installed manually per [AUR guidelines](https://wiki.archlinux.org/index.php/AUR_User_Guidelines#Installing_packages) or using a wrapper such as [`yaourt`](https://wiki.archlinux.org/index.php/yaourt) shown below.
+There are at least three non non-official packages for git-annex in the Arch Linux User Repository. Any of these may be installed manually per [AUR guidelines](https://wiki.archlinux.org/index.php/AUR_User_Guidelines#Installing_packages) or using a wrapper such as [`yaourt`](https://wiki.archlinux.org/index.php/yaourt) shown below.
 
-1. The simplest method is to use the [git-annex-bin](https://aur.archlinux.org/packages/git-annex-bin/) package based on the [prebuilt Linux tarballs](http://downloads.kitenet.net/git-annex/linux/current/). This package includes many of the binary shims from the pre-built package. Although common Linux system utilities have been stripped in favor of normal dependencies, the pre-configured Haskell libraries included out of the box make this an easy install. The disadvantage is the resulting installation is a bit on the heavy side at nearly 100M.
+1. A git-annex package is available in the haskell-core AUR <https://wiki.archlinux.org/index.php/ArchHaskell>
 
-       $ yaourt -Sy git-annex-bin
-
-2. A git-annex package is available in the haskell-core AUR <https://wiki.archlinux.org/index.php/ArchHaskell>
-
-3. A development package is available at [git-annex-git](https://aur.archlinux.org/packages/git-annex-git/) that functions similarly to the source package but builds directly from the HEAD of the git repository rather that the last official release.
+2. A development package is available at [git-annex-git](https://aur.archlinux.org/packages/git-annex-git/) that functions similarly to the source package but builds directly from the HEAD of the git repository rather that the last official release.
 
        $ yaourt -Sy git-annex-git
 
-4. A Cabal sandbox build is also available
+3. A Cabal sandbox build is also available
 
        $ yaourt -Sy git-annex-cabal
 

diff --git a/doc/bugs/Allow_automatic_retry_git_annex_get/comment_5_786cc686ac7049c19d572329641bbb6d._comment b/doc/bugs/Allow_automatic_retry_git_annex_get/comment_5_786cc686ac7049c19d572329641bbb6d._comment
index 0d351aef2..dfb25f922 100644
--- a/doc/bugs/Allow_automatic_retry_git_annex_get/comment_5_786cc686ac7049c19d572329641bbb6d._comment
+++ b/doc/bugs/Allow_automatic_retry_git_annex_get/comment_5_786cc686ac7049c19d572329641bbb6d._comment
@@ -19,4 +19,12 @@ be disabled for those.
 Hmm, a password prompt for eg ssh can also "stall"
 a transfer for a long time. Perhaps the thing to do is wait until
 the meter gets updated once, and only then start the stall detector.
+
+But what would the stall detector do when it does detect a stalled
+transfer? It should perhaps cancel the transfer action, but I don't know
+how to do that; the transfer action may have eg run a process, which would
+need to be canceled, or it may have a network connection open. Simply
+killing the transfer thread won't stop a process that it started.
+And when an external special remote is performing the transfer, there's
+nothing in that protocol to cancel a transfer.
 """]]

diff --git a/doc/bugs/Allow_automatic_retry_git_annex_get/comment_5_786cc686ac7049c19d572329641bbb6d._comment b/doc/bugs/Allow_automatic_retry_git_annex_get/comment_5_786cc686ac7049c19d572329641bbb6d._comment
new file mode 100644
index 000000000..0d351aef2
--- /dev/null
+++ b/doc/bugs/Allow_automatic_retry_git_annex_get/comment_5_786cc686ac7049c19d572329641bbb6d._comment
@@ -0,0 +1,22 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 5"""
+ date="2018-03-29T17:36:48Z"
+ content="""
+We have annex.retry etc configuration now to control retries of transfers.
+
+Stall detection would be good to have in git-annex. Currently, when a
+transfer is stalled, the progress meter stops updating, so displays
+an ETA that's wrong.
+
+Since the progress meter is already getting called periodically when
+a transfer is not stalled, a stall detector could perhaps be added
+into that, to abort stalled transfers. The tricky part being that a few
+types of transfers don't use the internal progress meters but rely on an
+external command's progress display, and the stall detection would need to
+be disabled for those. 
+
+Hmm, a password prompt for eg ssh can also "stall"
+a transfer for a long time. Perhaps the thing to do is wait until
+the meter gets updated once, and only then start the stall detector.
+"""]]

diff --git a/doc/bugs/Add_explicit_retries_for_curl__47__wget_on___34__transient__34___errors__63__.mdwn b/doc/bugs/Add_explicit_retries_for_curl__47__wget_on___34__transient__34___errors__63__.mdwn
index 4b9c21a75..bb6ba69f5 100644
--- a/doc/bugs/Add_explicit_retries_for_curl__47__wget_on___34__transient__34___errors__63__.mdwn
+++ b/doc/bugs/Add_explicit_retries_for_curl__47__wget_on___34__transient__34___errors__63__.mdwn
@@ -25,3 +25,5 @@ here are options for wget and curl which could help us out here:
 """]]
 
 Could git-annex add those options to curl/wget invocations for more robust access to the web remote?
+
+> [[done]] --[[Joey]]
diff --git a/doc/bugs/Add_explicit_retries_for_curl__47__wget_on___34__transient__34___errors__63__/comment_2_23fe5ddd3a77c20d66c6847bdb83c94b._comment b/doc/bugs/Add_explicit_retries_for_curl__47__wget_on___34__transient__34___errors__63__/comment_2_23fe5ddd3a77c20d66c6847bdb83c94b._comment
new file mode 100644
index 000000000..2072cc3ee
--- /dev/null
+++ b/doc/bugs/Add_explicit_retries_for_curl__47__wget_on___34__transient__34___errors__63__/comment_2_23fe5ddd3a77c20d66c6847bdb83c94b._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2018-03-29T17:24:35Z"
+ content="""
+I've implemented `remote.<name>.annex-retry`, `annex.retry`, 
+`remote.<name>.annex-retry-delay`, `annex.retry-delay` configs, so you can
+have full control over retry behavior for remotes.
+
+Since these are fully generic, not at the HTTP level, they'll
+make any and all transfer failures be retried, no matter why it failed.
+Which could be a good thing or a bad thing..
+"""]]

diff --git a/doc/install/OSX/comment_16_277249620801bd4305776a3d598bd074._comment b/doc/install/OSX/comment_16_277249620801bd4305776a3d598bd074._comment
new file mode 100644
index 000000000..daeda7175
--- /dev/null
+++ b/doc/install/OSX/comment_16_277249620801bd4305776a3d598bd074._comment
@@ -0,0 +1,24 @@
+[[!comment format=mdwn
+ username="achilleas.k@14be77d42a1252fab5ec9dbf4e5ea03c5833e8c8"
+ nickname="achilleas.k"
+ avatar="http://cdn.libravatar.org/avatar/ed6c67c4d8e6c6850930e16eaf85a771"
+ subject="dmg (possibly) requires rebuild"
+ date="2018-03-29T15:31:23Z"
+ content="""
+On macOS High Sierra (10.13.3) trying to use git-annex from the latest .dmg shows the following error:
+
+    > git annex version
+    dyld: Symbol not found: _inflateValidate
+      Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libPng.dylib
+      Expected in: /Applications/git-annex.app/Contents/MacOS/bundle/B
+     in /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libPng.dylib
+    error: git-annex died of signal 6
+
+I ran it by opening the .app directory (Show Package Contents) and running the `runshell` script to set up the environment:
+
+    > which git
+    /Applications/git-annex.app/Contents/MacOS/bundle/git
+    > which git-annex
+    /Applications/git-annex.app/Contents/MacOS/bundle/git-annex
+
+"""]]

diff --git a/doc/install/Android/comment_13_47a86c573b9673811f790f3219c5d212._comment b/doc/install/Android/comment_13_47a86c573b9673811f790f3219c5d212._comment
new file mode 100644
index 000000000..fc59a6e98
--- /dev/null
+++ b/doc/install/Android/comment_13_47a86c573b9673811f790f3219c5d212._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="cristianpolanco2015@04c392af1063e289871c7a1ae208794e1105ba65"
+ nickname="cristianpolanco2015"
+ avatar="http://cdn.libravatar.org/avatar/8787a59fc83d3a4d4ef0fd131c59e83d"
+ subject="Play Store App Download Free"
+ date="2018-03-29T15:27:19Z"
+ content="""
+Download Google Play Store app free - Play Store https://playstore.zone/
+"""]]

diff --git a/doc/forum/what_rules_is_git_annex_sync_--content_using_to_send_files_out/comment_2_50c1a19a7278e8748c2e6e789f6b418a._comment b/doc/forum/what_rules_is_git_annex_sync_--content_using_to_send_files_out/comment_2_50c1a19a7278e8748c2e6e789f6b418a._comment
new file mode 100644
index 000000000..efbf232b8
--- /dev/null
+++ b/doc/forum/what_rules_is_git_annex_sync_--content_using_to_send_files_out/comment_2_50c1a19a7278e8748c2e6e789f6b418a._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="MatusGoljer"
+ avatar="http://cdn.libravatar.org/avatar/8152eed1d594c570563ed46e7fd8356f"
+ subject="comment 2"
+ date="2018-03-28T23:16:51Z"
+ content="""
+Hi Andrew!
+
+Actually, you are entirely correct, I just misinterpreted the `smallarchive` formula.  What I was thinking it would do was it would upload the file in case there are not two copies in any combination of `archive` or `smallarchive` and as soon as there are two `archive` copies it would ideally drop the file if not in `*/archive/*` directory.
+
+But now I see that the formula does exactly what you say it would do, the whole long `and not` condition is additional to the first, which means a file will not get uploaded if not in those directories.
+
+I will play around with the expression, I'm sure it can be modified to do what I want.
+
+Thanks!  It was mostly me being silly, sometimes having other people re-state the obvious helps!
+"""]]

diff --git a/doc/bugs/does_not_handle_youtube_playlists.mdwn b/doc/bugs/does_not_handle_youtube_playlists.mdwn
new file mode 100644
index 000000000..1df49be37
--- /dev/null
+++ b/doc/bugs/does_not_handle_youtube_playlists.mdwn
@@ -0,0 +1,42 @@
+### Please describe the problem.
+
+Not really related to datalad -- thought to addurl youtube playlist
+
+youtube-dl seems might be capable of doing it 
+
+### Please provide any additional information below.
+
+[[!format sh """
+
+$> git annex addurl --debug 'https://www.youtube.com/playlist?list=PLBHioGD0U1Cjd-meZbEcz-9ZxK-mb50tZ'                                                            [2018-03-28 13:09:06.337738339] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
+[2018-03-28 13:09:06.34708705] process done ExitSuccess
+[2018-03-28 13:09:06.347202003] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
+[2018-03-28 13:09:06.35303272] process done ExitSuccess
+[2018-03-28 13:09:06.353205536] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..734b368eab4f39d8494671657977952b02a35d9a","--pretty=%H","-n1"]
+[2018-03-28 13:09:06.359101486] process done ExitSuccess
+[2018-03-28 13:09:06.360005167] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
+[2018-03-28 13:09:06.361006846] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
+addurl https://www.youtube.com/playlist?list=PLBHioGD0U1Cjd-meZbEcz-9ZxK-mb50tZ 
+[2018-03-28 13:09:06.409163359] call: wget ["-nv","--show-progress","--clobber","-c","-O","/tmp/testyt/.git/annex/tmp/URL--https&c%%www.youtube.com%playlis-5bc73cdf8dc8bd73b13addc290b160e6","https://www.youtube.com/playlist?list=PLBHioGD0U1Cjd-meZbEcz-9ZxK-mb50tZ","--user-agent","git-annex/6.20180316+gitg308f3ecf6-1~ndall+1"]
+/tmp/testyt/.git/ann 100%[===================>] 211.54K  --.-KB/s    in 0.04s   
+2018-03-28 13:09:06 URL:https://www.youtube.com/playlist?list=PLBHioGD0U1Cjd-meZbEcz-9ZxK-mb50tZ [216616/216616] -> "/tmp/testyt/.git/annex/tmp/URL--https&c%%www.youtube.com%playlis-5bc73cdf8dc8bd73b13addc290b160e6" [1]
+[2018-03-28 13:09:06.844196402] process done ExitSuccess
+[2018-03-28 13:09:06.845246148] read: youtube-dl ["https://www.youtube.com/playlist?list=PLBHioGD0U1Cjd-meZbEcz-9ZxK-mb50tZ","--get-filename","--no-warnings"]
+[2018-03-28 13:09:40.643440496] process done ExitSuccess
+[2018-03-28 13:09:40.644420015] call: youtube-dl ["https://www.youtube.com/playlist?list=PLBHioGD0U1Cjd-meZbEcz-9ZxK-mb50tZ","--no-playlist","--playlist-items","0","--max-filesize","9046842816"]
+[youtube:playlist] PLBHioGD0U1Cjd-meZbEcz-9ZxK-mb50tZ: Downloading webpage
+[download] Downloading playlist: Canonical Computation in Brains and Machines
+[youtube:playlist] playlist Canonical Computation in Brains and Machines: Downloading 0 videos
+[download] Finished downloading playlist: Canonical Computation in Brains and Machines
+[2018-03-28 13:09:41.925771815] process done ExitSuccess
+
+  youtube-dl did not put any media in its work directory, perhaps it's been configured to store files somewhere else?
+failed
+[2018-03-28 13:09:41.926669401] process done ExitSuccess
+[2018-03-28 13:09:41.926951446] process done ExitSuccess
+git-annex: addurl: 1 failed
+git annex addurl --debug   6.49s user 0.32s system 18% cpu 35.834 total
+
+"""]]
+
+

diff --git a/doc/forum/Init_a_directory_special_remote_with_already_existing_files/comment_1_f2a2f7309145d26d124c73ce6fcb32a5._comment b/doc/forum/Init_a_directory_special_remote_with_already_existing_files/comment_1_f2a2f7309145d26d124c73ce6fcb32a5._comment
new file mode 100644
index 000000000..47be769a4
--- /dev/null
+++ b/doc/forum/Init_a_directory_special_remote_with_already_existing_files/comment_1_f2a2f7309145d26d124c73ce6fcb32a5._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="andrew"
+ avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
+ subject="comment 1"
+ date="2018-03-28T00:59:41Z"
+ content="""
+I think if you don't mind scripting checkout Joey's comment in this thread: [\"Preseeding\" a special remote](https://git-annex.branchable.com/forum/__34__Preseeding__34___a_special_remote/).
+
+Otherwise maybe the following will work. You'll still need to download all of the files to hash them.
+
+UNTESTED:
+Lets say your Windows share data is at mnt/share/data
+
+ * create a new repo on your Windows share at mnt/share/repo1
+ * [git-annex-import](https://git-annex.branchable.com/git-annex-import/) all of your files into the new repo. This probably would download all of the files to generate the hash, then hopefully do a file rename avoiding the re-upload step.
+ * then create a new [directory remote](http://git-annex.branchable.com/special_remotes/directory/) for repo1 at say, mnt/share/remote1
+ * then do a [git-annex-move](https://git-annex.branchable.com/git-annex-move/) of all the files from mnt/share/repo1 to your new empty remote
+ * you could then make a clone of that repo locally, now you have a nice looking working tree locally
+ * then add mnt/share/remote1 as a new remote to your local clone
+
+If you don't care about hashing you could use the [WORM](http://git-annex.branchable.com/backends/) backend avoiding the initial download.
+
+I think, If you don't care about having a nice working tree you could probably use the web special remote and just leave all the files in their messy rsynced state. IE create a local repo, then do `git annex addurl --fast <path>` for each file on your Windows share. See joey's comment on addurl in this [DVD thread](http://git-annex.branchable.com/forum/Managing_a_large_number_of_files_archived_on_many_pieces_of_read-only_medium___40__E.G._DVDs__41__/)
+
+
+"""]]

diff --git a/doc/forum/what_rules_is_git_annex_sync_--content_using_to_send_files_out/comment_1_86428fc8286266d01cc51af8ae4aa335._comment b/doc/forum/what_rules_is_git_annex_sync_--content_using_to_send_files_out/comment_1_86428fc8286266d01cc51af8ae4aa335._comment
new file mode 100644
index 000000000..ce0a53992
--- /dev/null
+++ b/doc/forum/what_rules_is_git_annex_sync_--content_using_to_send_files_out/comment_1_86428fc8286266d01cc51af8ae4aa335._comment
@@ -0,0 +1,34 @@
+[[!comment format=mdwn
+ username="andrew"
+ avatar="http://cdn.libravatar.org/avatar/acc0ece1eedf07dd9631e7d7d343c435"
+ subject="numcopies"
+ date="2018-03-27T22:25:30Z"
+ content="""
+||| *When I add a new file to my client clone and do git annex sync --content is this respecting the … content of each remote or is it only trying to satisfy the numcopy?*
+
+Both.
+
+Running `git annex sync --content` will copy content to any remote where that content is wanted (IE look at the preferred content settings), and will drop files that are not wanted and don't violate numcopies totals (at the moment of the drop?).
+
+
+||| *One of the remotes is set as archive and two as smallarchive yet sync --content only ever copies to one so as to satisfy numcopies.*
+
+I don't believe that `git annex sync --content` is ever trying to “satisfy numcopies,” I would think of numcopies as more of a limit or restriction on when git-annex is allowed to drop content that is not wanted by a remote.
+
+
+
+
+||| *Shouldn't it always try to make two copies in archive or smallarchive?*
+
+Hmmmm. I would guess there is some issue with your archive or smallarchive expressions, or they aren't actually set (being used) or you have discovered an issue… You have overridden the standard groups, and can see your overrides with `git annex groupwanted archive` and `git annex groupwanted smallarchive`? And your remotes have `git annex group archive` and `git annex wanted groupwanted` set?
+
+So `(include=*/archive/* or include=archive/*) and` means you are only copying files in the archive directory, was that your intention?
+
+If so, the rest of your content expression seems like it should want 1 copy in an archive and 1 copy in a smallarchive, or 2 copies in 2 remotes marked archive or 2 copies in two remotes marked smallarchive.
+
+
+||| *Interestingly, it copies data to my backup*
+
+Right. If you are using a standard content group backup means “All content is wanted. Even content of old/deleted files.” This expression will want all content and never drop content.
+
+"""]]

diff --git a/doc/install/conda.mdwn b/doc/install/conda.mdwn
index 3e675dd4b..cd7911d1e 100644
--- a/doc/install/conda.mdwn
+++ b/doc/install/conda.mdwn
@@ -1,6 +1,6 @@
 On Linux, an up-to-date binary distribution of git-annex can now be
 installed with the [conda package manager](https://conda.io/docs/) 
-from with [conda-forge channel](https://conda-forge.org/).
+from the [conda-forge channel](https://conda-forge.org/).
 
 After installing conda, run:
 

diff --git a/doc/install.mdwn b/doc/install.mdwn
index 4005ed4cd..6255ee4b3 100644
--- a/doc/install.mdwn
+++ b/doc/install.mdwn
@@ -13,10 +13,11 @@ detailed instructions             | quick install
   [[ArchLinux]]         | `pacman -S git-annex`
   [[NixOS]]             | `nix-env -i git-annex`
   [[Gentoo]]            | `emerge git-annex`
-  [[Void]]            | `xbps-install git-annex`
+  [[Void]]              | `xbps-install git-annex`
   [[ScientificLinux5]]  |
   [[openSUSE]]          | `zypper in git-annex`
   [[Docker]]            | 
+  [[conda]]             | `conda install -c conda-forge git-annex`
 [[Windows]]                       | **beta**
 """]]
 
diff --git a/doc/install/comment_6_b095b0f6d7607f149ddc112da433319e._comment b/doc/install/comment_6_b095b0f6d7607f149ddc112da433319e._comment
deleted file mode 100644
index 61746a584..000000000
--- a/doc/install/comment_6_b095b0f6d7607f149ddc112da433319e._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="ilyawebmail@d5db0da13c02cda3860f534f3f02acdc914360b9"
- nickname="ilyawebmail"
- avatar="http://cdn.libravatar.org/avatar/34c90de0aa8b47144f32671827bd97d1"
- subject="conda install now available on Linux (no root privileges needed)"
- date="2018-03-27T18:33:36Z"
- content="""
-On Linux, an up-to-date binary distribution of git-annex can now be installed with the conda package manager ( https://conda.io/docs/ ) from with conda-forge channel ( https://conda-forge.org/ ).  After installing conda, do: \"conda install -c conda-forge git-annex\".
-
-"""]]
diff --git a/doc/install/conda.mdwn b/doc/install/conda.mdwn
new file mode 100644
index 000000000..3e675dd4b
--- /dev/null
+++ b/doc/install/conda.mdwn
@@ -0,0 +1,9 @@
+On Linux, an up-to-date binary distribution of git-annex can now be
+installed with the [conda package manager](https://conda.io/docs/) 
+from with [conda-forge channel](https://conda-forge.org/).
+
+After installing conda, run:
+
+	conda install -c conda-forge git-annex
+
+Note that this installation method does not need root permissions.

diff --git a/doc/devblog/day_493__two_new_special_remotes.mdwn b/doc/devblog/day_493__two_new_special_remotes.mdwn
index 6944cac81..478450ba8 100644
--- a/doc/devblog/day_493__two_new_special_remotes.mdwn
+++ b/doc/devblog/day_493__two_new_special_remotes.mdwn
@@ -5,10 +5,11 @@ In the meantime, Lykos has released
 a replacement for an older, unmaintained Google Drive special remote.
 
 Today I added a special remote that stores files on an Android device
-using `adb`. It supports `git annex exporttree`, so the files stored on 
+using `adb`. It supports `git annex export`, so the files stored on 
 the Android device can have the same filenames as in the git-annex
-repository. I have plans for implementing a `git annex importtree` and
-other features to make bi-directional sync with Android work well.
+repository. I have plans for making `git annex import` support
+special remotes, and other features to make bi-directional
+sync with Android work well.
 
 Of course, there is some overlap between that and the Android port,
 but they probably serve different use cases.