Recent changes to this wiki:

How to deal with files that change status from "precious, please keep n copies" to "junk, please delete it from everywhere you find it, now and forever".
diff --git a/doc/forum/Blacklisting_files___40__so_that_they_get_removed_any_time_a_copy_is_found__41__.mdwn b/doc/forum/Blacklisting_files___40__so_that_they_get_removed_any_time_a_copy_is_found__41__.mdwn
new file mode 100644
index 0000000..ca96a07
--- /dev/null
+++ b/doc/forum/Blacklisting_files___40__so_that_they_get_removed_any_time_a_copy_is_found__41__.mdwn
@@ -0,0 +1,55 @@
+Hello,
+
+I'm not sure this is mature enough as a wishlist item so I put it in the forum.  Please move if it is appropriate.
+
+# Context, general need
+
+I'm considering using git-annex to store photos, yet what I'm needing might be useful in more general cases.
+
+The need in one sentence: **deal with files that change status from "precious, please keep n copies" to "junk, please delete it from everywhere you find it, now and forever"**.
+
+# More concrete need
+
+I take many photographs and am interested in git-annex comfort to ensure a number of copies of files exist at any time.
+
+Sometimes I can sort photos and reject bad ones shortly after taking a roll, typically before I could `git annex add` them.  Sometimes I do it much later, after they have been included in `git-annex` and stored in several places.
+
+## Rationale 1: releasing storage space
+
+I'm worried about having a lot of space still used by photographs that I actually don't want to keep.
+
+It's not marginal.  For example, at 30Mb per RAW photo, 300+ photos in an ordinary afternoon take 10Gb, from which up to one half could turn out rejected.  Whole photo/video collection is already probably much over 1Tb and growing.
+
+So we're talking about 5Gb per shooting day to be freed and already probably 100+Gb to free from remotes, so definitely something to consider.
+
+## Rationale 2: rejecting files once and for all, not having to repeat it
+
+Once I rejected a photograph, I'd like to never have to `rm`, `forget`, `drop` them or whatever again.  Ideally it would just be dropped from any back-end (that supports removing information) at any time in the future, perhaps with just a generic command/script like `git annex purge_the_junk_files`.
+
+# Questions
+
+* Q1 Is there a simple way to somehow "blacklist" a particular file so that from that moment on, any `sync`-like operation of `git-annex` involving a remote will remove the data of this file in that remote.
+* Q2 UI considerations. In my dreams, files could just be moved to a "junk" sub-folder (using photo sorting tools like e.g. geeqie), then a batch command would assign them the "blacklisted" or "junk" status.
+* Q3 I don't mind at this point if there are traces of where (in the filesystem tree) the now-rejected files were.  Perhaps it's easier to perform a different operation, that is completely forgetting the history of blacklisted files in the spirit of `git filter-branch`.
+
+Perhaps most of this can be done with simple configuration and a helper script.
+
+# Additional information
+
+* I'm wondering if some simple scheme like an actual `git filter-branch` in the local `git-annex` repo then some cleanup command (`git annex fsck`) then push to remote would have the intended effect.  Since this involves rewriting history I don't know how `git-annex` would like it.  But that's just a thought-experiment at this point.
+* The number of files to blacklist can quickly go to a few hundreds later thousands.  This might rule out some very naive implementations.
+* I can hack a bash script to perform whatever appropriate command so that given a solution to Q1 I have a solution to Q2.
+
+# Search before you post
+
+I've found other topics but they don't seem to even remotely deal with the topic here.  E.g.:
+
+* [[https://git-annex.branchable.com/forum/Backing_up_photos_to_the_cloud/]]
+* [[https://git-annex.branchable.com/forum/best_practices_for_importing_photos__63__/]]
+* [[https://git-annex.branchable.com/tips/automatically_adding_metadata/]] 
+
+[[https://git-annex.branchable.com/git-annex-forget/]] seems to be orthogonal (forget history of everything, but not delete data)
+
+This might be more-or-less on topic but confusing to me at this point : [[https://git-annex.branchable.com/forum/How_to_get_git-annex_to_forget_a_commit__63__/]]
+
+Thank you for your attention.

diff --git a/doc/bugs/All_inodes_eaten.mdwn b/doc/bugs/All_inodes_eaten.mdwn
new file mode 100644
index 0000000..10c1780
--- /dev/null
+++ b/doc/bugs/All_inodes_eaten.mdwn
@@ -0,0 +1,41 @@
+### Please describe the problem.
+
+Adding new files to a gcrypt remote ends up eating all inodes.
+
+### What steps will reproduce the problem?
+
+I have a docs directory, 780 files in it, various types, various subdirectories. I add these to a git annex repo. Create a gcrypt remote on a usb drive and use assistant to copy. Eventually all sorts of stuff stops working since there are no inodes left.
+
+### What version of git-annex are you using? On what operating system?
+
+Arch Linux 64 bit, stable. Git annex 6.20160613-8
+
+### Please provide any additional information below.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+Remote bare repo looks as follows:
+
+    annex/objects - contains 4096 directories (000, 001, ...)
+
+    annex/objects/XXX - contains ~1380 directories (001, ...)
+
+    annex/objects/XXX/XXX - contains two or three directories, e.g. GPGHMACSHA1--3939af6b89c30015490ce9e19f9051bb9e9fe64e
+
+    annex/objects/XXX/XXX/GPGHMACSHA1-XXXX - contains one or two files of the same name
+
+Let's multiply that out: 4096 * 1380 * 2 * 2 = 22,609,920
+
+Plus . and .. for all the directories, and boom, all my 59 million inodes are gone, before I even manage to store my 780 files!
+
+Disk is about 20% full.
+
+
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+

diff --git a/doc/bugs/committing_an_edited_file_fails_with___34__hGetBuf__58___Invalid_argument__34__.mdwn b/doc/bugs/committing_an_edited_file_fails_with___34__hGetBuf__58___Invalid_argument__34__.mdwn
new file mode 100644
index 0000000..32b7f59
--- /dev/null
+++ b/doc/bugs/committing_an_edited_file_fails_with___34__hGetBuf__58___Invalid_argument__34__.mdwn
@@ -0,0 +1,33 @@
+### Please describe the problem.
+
+After unlocking a large file and editing it, the attempt to commit the change fails like so:
+
+git-annex: fd:16: hGetBuf: invalid argument (Invalid argument)
+
+### What steps will reproduce the problem?
+
+(it needs to be over 2G in size to reproduce the problem)
+
+### What version of git-annex are you using? On what operating system?
+
+git-annex version: 6.20160923 on macOS 10.12
+
+### Please provide any additional information below.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+$ dd if=/dev/zero of=largefile.txt bs=1048576 count=2048
+$ git annex add largefile.txt
+$ git commit -m 'added large file'
+$ git annex unlock largefile.txt
+$ echo foobar >> largefile.txt
+$ git commit -a -m 'edited large file'
+git-annex: fd:16: hGetBuf: invalid argument (Invalid argument)
+
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+

Added a comment: inodes....
diff --git a/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_11_bbd8b537e277d24df254ed058ad40e24._comment b/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_11_bbd8b537e277d24df254ed058ad40e24._comment
new file mode 100644
index 0000000..adff369
--- /dev/null
+++ b/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_11_bbd8b537e277d24df254ed058ad40e24._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="pot"
+ subject="inodes...."
+ date="2016-09-30T22:00:31Z"
+ content="""
+I think I may have tracked this down partly, I ran out of inodes on a removeable drive and started getting the same error. Will try and rectify underlying inode issue and then see.
+
+(secondary point, how on earth has my docs directory used up all 59 million inodes available to it when put into an annex???)
+"""]]

Added a comment: Bash on Windows
diff --git a/doc/todo/windows_support/comment_15_61d1281072de2bb418cf96fd8b563d45._comment b/doc/todo/windows_support/comment_15_61d1281072de2bb418cf96fd8b563d45._comment
new file mode 100644
index 0000000..9eae2b7
--- /dev/null
+++ b/doc/todo/windows_support/comment_15_61d1281072de2bb418cf96fd8b563d45._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="ztl8702"
+ subject="Bash on Windows"
+ date="2016-09-30T19:14:34Z"
+ content="""
+Currently there is a bug that stops all Haskell programs from running on Bash on Windows:
+https://wpdev.uservoice.com/forums/266908-command-prompt-console-bash-on-ubuntu-on-windo/suggestions/13572504-add-support-for-timer-create
+
+https://github.com/Microsoft/BashOnWindows/issues/307
+"""]]

devblog
diff --git a/doc/devblog/day_418__concurrent_externals.mdwn b/doc/devblog/day_418__concurrent_externals.mdwn
new file mode 100644
index 0000000..6dd6916
--- /dev/null
+++ b/doc/devblog/day_418__concurrent_externals.mdwn
@@ -0,0 +1,12 @@
+Realized recently that despite all the nice concurrency support in
+git-annex, external special remotes were limited to handling one request at
+a time.
+
+While the external special remote prococol could almost support concurrent
+requests, that would complicate implementing them, and probably need a
+version flag to enable to avoid breaking existing ones.
+
+Instead, made git-annex start up multiple external special remote processes
+as needed to handle concurrency.
+
+Today's work was sponsored by Josh Taylor on Patreon.

allow multiple concurrent external special remote processes
Multiple external special remote processes for the same remote will be
started as needed when using -J.
This should not beak any existing external special remotes, because running
multiple git-annex commands at the same time could already start multiple
processes for the same external special remotes.
diff --git a/CHANGELOG b/CHANGELOG
index 888e413..6f2627c 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -9,6 +9,11 @@ git-annex (6.20160924) UNRELEASED; urgency=medium
   * Add "total-size" field to --json-progress output.
   * Make --json-progress output be shown even when the size of a object
     is not known.
+  * Multiple external special remote processes for the same remote will be
+    started as needed when using -J. This should not beak any existing
+    external special remotes, because running multiple git-annex commands
+    at the same time could already start multiple processes for the same
+    external special remotes.
 
  -- Joey Hess <id@joeyh.name>  Mon, 26 Sep 2016 16:46:19 -0400
 
diff --git a/Remote/External.hs b/Remote/External.hs
index e736599..fb87f44 100644
--- a/Remote/External.hs
+++ b/Remote/External.hs
@@ -1,6 +1,6 @@
 {- External special remote interface.
  -
- - Copyright 2013-2015 Joey Hess <id@joeyh.name>
+ - Copyright 2013-2016 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -126,9 +126,8 @@ externalSetup mu _ c gc = do
 				INITREMOTE_SUCCESS -> Just noop
 				INITREMOTE_FAILURE errmsg -> Just $ error errmsg
 				_ -> Nothing
-			withExternalLock external $ \lck ->
-				fromExternal lck external externalConfig $
-					liftIO . atomically . readTMVar 
+			withExternalState external $
+				liftIO . atomically . readTMVar . externalConfig
 
 	gitConfigSpecialRemote u c'' "externaltype" externaltype
 	return (c'', u)
@@ -203,27 +202,28 @@ safely a = go =<< tryNonAsync a
  - While the external remote is processing the Request, it may send
  - any number of RemoteRequests, that are handled here.
  -
- - Only one request can be made at a time, so locking is used.
+ - An external remote process can only handle one request at a time.
+ - Concurrent requests will start up additional processes.
  -
  - May throw exceptions, for example on protocol errors, or
  - when the repository cannot be used.
  -}
 handleRequest :: External -> Request -> Maybe MeterUpdate -> (Response -> Maybe (Annex a)) -> Annex a
 handleRequest external req mp responsehandler = 
-	withExternalLock external $ \lck ->
-		handleRequest' lck external req mp responsehandler
+	withExternalState external $ \st -> 
+		handleRequest' st external req mp responsehandler
 
-handleRequest' :: ExternalLock -> External -> Request -> Maybe MeterUpdate -> (Response -> Maybe (Annex a)) -> Annex a
-handleRequest' lck external req mp responsehandler
+handleRequest' :: ExternalState -> External -> Request -> Maybe MeterUpdate -> (Response -> Maybe (Annex a)) -> Annex a
+handleRequest' st external req mp responsehandler
 	| needsPREPARE req = do
-		checkPrepared lck external
+		checkPrepared st external
 		go
 	| otherwise = go
   where
 	go = do
-		sendMessage lck external req
+		sendMessage st external req
 		loop
-	loop = receiveMessage lck external responsehandler
+	loop = receiveMessage st external responsehandler
 		(\rreq -> Just $ handleRemoteRequest rreq >> loop)
 		(\msg -> Just $ handleAsyncMessage msg >> loop)
 
@@ -234,26 +234,24 @@ handleRequest' lck external req mp responsehandler
 	handleRemoteRequest (DIRHASH_LOWER k) = 
 		send $ VALUE $ hashDirLower def k
 	handleRemoteRequest (SETCONFIG setting value) =
-		fromExternal lck external externalConfig $ \v -> 
-			liftIO $ atomically $ do
-				m <- takeTMVar v
-				putTMVar v $ M.insert setting value m
+		liftIO $ atomically $ do
+			let v = externalConfig st
+			m <- takeTMVar v
+			putTMVar v $ M.insert setting value m
 	handleRemoteRequest (GETCONFIG setting) = do
-		value <- fromExternal lck external externalConfig $ \v -> 
-			fromMaybe "" . M.lookup setting
-				<$> liftIO (atomically $ readTMVar v)
+		value <- fromMaybe "" . M.lookup setting
+			<$> liftIO (atomically $ readTMVar $ externalConfig st)
 		send $ VALUE value
 	handleRemoteRequest (SETCREDS setting login password) = do
-		fromExternal lck external externalConfig $ \v -> do
-			c <- liftIO $ atomically $ readTMVar v
-			let gc = externalGitConfig external
-			c' <- setRemoteCredPair encryptionAlreadySetup c gc
-				(credstorage setting)
-				(Just (login, password))
-			void $ liftIO $ atomically $ swapTMVar v c'
+		let v = externalConfig st
+		c <- liftIO $ atomically $ readTMVar v
+		let gc = externalGitConfig external
+		c' <- setRemoteCredPair encryptionAlreadySetup c gc
+			(credstorage setting)
+			(Just (login, password))
+		void $ liftIO $ atomically $ swapTMVar v c'
 	handleRemoteRequest (GETCREDS setting) = do
-		c <- fromExternal lck external externalConfig $
-			liftIO . atomically . readTMVar
+		c <- liftIO $ atomically $ readTMVar $ externalConfig st
 		let gc = externalGitConfig external
 		creds <- fromMaybe ("", "") <$> 
 			getRemoteCredPair c gc (credstorage setting)
@@ -286,11 +284,11 @@ handleRequest' lck external req mp responsehandler
 		send (VALUE "") -- end of list
 	handleRemoteRequest (DEBUG msg) = liftIO $ debugM "external" msg
 	handleRemoteRequest (VERSION _) =
-		sendMessage lck external $ ERROR "too late to send VERSION"
+		sendMessage st external (ERROR "too late to send VERSION")
 
 	handleAsyncMessage (ERROR err) = error $ "external special remote error: " ++ err
 
-	send = sendMessage lck external
+	send = sendMessage st external
 
 	credstorage setting = CredPairStorage
 		{ credPairFile = base
@@ -303,30 +301,28 @@ handleRequest' lck external req mp responsehandler
 	withurl mk uri = handleRemoteRequest $ mk $
 		setDownloader (show uri) OtherDownloader
 
-sendMessage :: Sendable m => ExternalLock -> External -> m -> Annex ()
-sendMessage lck external m = 
-	fromExternal lck external externalSend $ \h ->
-		liftIO $ do
-			protocolDebug external True line
-			hPutStrLn h line
-			hFlush h
+sendMessage :: Sendable m => ExternalState -> External -> m -> Annex ()
+sendMessage st external m = liftIO $ do
+	protocolDebug external True line
+	hPutStrLn h line
+	hFlush h
   where
 	line = unwords $ formatMessage m
+	h = externalSend st
 
 {- Waits for a message from the external remote, and passes it to the
  - apppropriate handler. 
  -
  - If the handler returns Nothing, this is a protocol error.-}
 receiveMessage
-	:: ExternalLock
+	:: ExternalState
 	-> External 
 	-> (Response -> Maybe (Annex a))
 	-> (RemoteRequest -> Maybe (Annex a))
 	-> (AsyncMessage -> Maybe (Annex a))
 	-> Annex a
-receiveMessage lck external handleresponse handlerequest handleasync =
-	go =<< fromExternal lck external externalReceive
-		(liftIO . catchMaybeIO . hGetLine)
+receiveMessage st external handleresponse handlerequest handleasync =
+	go =<< liftIO (catchMaybeIO $ hGetLine $ externalReceive st)
   where
 	go Nothing = protocolError False ""
 	go (Just s) = do
@@ -348,39 +344,43 @@ protocolDebug external sendto line = debugM "external" $ unwords
 	, line
 	]
 
-{- Starts up the external remote if it's not yet running,
- - and passes a value extracted from its state to an action.
- -}
-fromExternal :: ExternalLock -> External -> (ExternalState -> v) -> (v -> Annex a) -> Annex a
-fromExternal lck external extractor a =
-	go =<< liftIO (atomically (tryReadTMVar v))
+{- While the action is running, the ExternalState provided to it will not
+ - be available to any other calls.
+ -
+ - Starts up a new process if no ExternalStates are available. -}
+withExternalState :: External -> (ExternalState -> Annex a) -> Annex a
+withExternalState external = bracket alloc dealloc
   where
-	go (Just st) = run st
-	go Nothing = do
-		st <- startExternal external
-		void $ liftIO $ atomically $ do
-			void $ tryReadTMVar v
-			putTMVar v st
-
-		{- Handle initial protocol startup; check the VERSION
-		 - the remote sends. -}
-		receiveMessage lck external
-			(const Nothing)

(Diff truncated)
devblog
diff --git a/doc/devblog/day_417__cut_once.mdwn b/doc/devblog/day_417__cut_once.mdwn
new file mode 100644
index 0000000..419e279
--- /dev/null
+++ b/doc/devblog/day_417__cut_once.mdwn
@@ -0,0 +1,6 @@
+Did most of the optimisations that recent profiling suggested. 
+This sped up a `git annex find` from 3.53 seconds to 1.73 seconds.
+And, `git annex find --not --in remote` from 12.41 seconds to 5.24 seconds.
+One of the optimisations sped up git-annex branch querying by up to 50%,
+which should also speed up use of some preferred content expressions.
+All in all, a very nice little optimisation pass.

Make --json-progress output be shown even when the size of a object is not known.
diff --git a/CHANGELOG b/CHANGELOG
index c36fcb7..888e413 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -7,6 +7,8 @@ git-annex (6.20160924) UNRELEASED; urgency=medium
     "git-annex find --in remote" by over 50%.
   * Optimised git-annex branch log file timestamp parsing.
   * Add "total-size" field to --json-progress output.
+  * Make --json-progress output be shown even when the size of a object
+    is not known.
 
  -- Joey Hess <id@joeyh.name>  Mon, 26 Sep 2016 16:46:19 -0400
 
diff --git a/Messages/JSON.hs b/Messages/JSON.hs
index fb288b7..06bdd9a 100644
--- a/Messages/JSON.hs
+++ b/Messages/JSON.hs
@@ -95,17 +95,20 @@ complete v _ = add v (Just (HM.empty, True))
 
 -- Show JSON formatted progress, including the current state of the JSON 
 -- object for the action being performed.
-progress :: Maybe Object -> Integer -> BytesProcessed -> IO ()
-progress maction size bytesprocessed = emit $ case maction of
+progress :: Maybe Object -> Maybe Integer -> BytesProcessed -> IO ()
+progress maction msize bytesprocessed = emit $ case maction of
 	Just action -> HM.insert "action" (Object action) o
 	Nothing -> o
   where
 	n = fromBytesProcessed bytesprocessed :: Integer
-	Object o = object
-		[ "byte-progress" .= n
-		, "percent-progress" .= showPercentage 2 (percentage size n)
-		, "total-size" .= size
-		]
+	Object o = case msize of
+		Just size -> object
+			[ "byte-progress" .= n
+			, "percent-progress" .= showPercentage 2 (percentage size n)
+			, "total-size" .= size
+			]
+		Nothing -> object
+			[ "byte-progress" .= n ]
 
 -- A value that can be displayed either normally, or as JSON.
 data DualDisp = DualDisp
diff --git a/Messages/Progress.hs b/Messages/Progress.hs
index f21ccae..06f2531 100644
--- a/Messages/Progress.hs
+++ b/Messages/Progress.hs
@@ -30,12 +30,10 @@ import Data.Quantity
 {- Shows a progress meter while performing a transfer of a key.
  - The action is passed a callback to use to update the meter. -}
 metered :: Maybe MeterUpdate -> Key -> (MeterUpdate -> Annex a) -> Annex a
-metered othermeter key a = case keySize key of
-	Nothing -> nometer
-	Just size -> withMessageState (go $ fromInteger size)
+metered othermeter key a = withMessageState $ go (keySize key)
   where
 	go _ (MessageState { outputType = QuietOutput }) = nometer
-	go size (MessageState { outputType = NormalOutput, concurrentOutputEnabled = False }) = do
+	go (Just size) (MessageState { outputType = NormalOutput, concurrentOutputEnabled = False }) = do
 		showOutput
 		(progress, meter) <- mkmeter size
 		m <- liftIO $ rateLimitMeterUpdate 0.1 (Just size) $ \n -> do
@@ -44,7 +42,7 @@ metered othermeter key a = case keySize key of
 		r <- a (combinemeter m)
 		liftIO $ clearMeter stdout meter
 		return r
-	go size (MessageState { outputType = NormalOutput, concurrentOutputEnabled = True }) =
+	go (Just size) (MessageState { outputType = NormalOutput, concurrentOutputEnabled = True }) =
 #if WITH_CONCURRENTOUTPUT
 		withProgressRegion $ \r -> do
 			(progress, meter) <- mkmeter size
@@ -57,10 +55,10 @@ metered othermeter key a = case keySize key of
 		nometer
 #endif
 	go _ (MessageState { outputType = JSONOutput False }) = nometer
-	go size (MessageState { outputType = JSONOutput True }) = do
+	go msize (MessageState { outputType = JSONOutput True }) = do
 		buf <- withMessageState $ return . jsonBuffer
-		m <- liftIO $ rateLimitMeterUpdate 0.1 (Just size) $
-			JSON.progress buf size
+		m <- liftIO $ rateLimitMeterUpdate 0.1 msize $
+			JSON.progress buf msize
 		a (combinemeter m)
 
 	mkmeter size = do
diff --git a/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_9_32bfa465567eaad105a8336bc247cc2f._comment b/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_9_32bfa465567eaad105a8336bc247cc2f._comment
new file mode 100644
index 0000000..280cb31
--- /dev/null
+++ b/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_9_32bfa465567eaad105a8336bc247cc2f._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 9"""
+ date="2016-09-29T20:58:13Z"
+ content="""
+Gone ahead and made --json-progress be displayed when size is not known,
+although of course it then has to omit the total-size and percent-progress
+fields.
+"""]]

summary of progress
diff --git a/doc/todo/make_copy_--fast__faster/comment_10_1af4ac0d37c876912678522895c1656b._comment b/doc/todo/make_copy_--fast__faster/comment_10_1af4ac0d37c876912678522895c1656b._comment
new file mode 100644
index 0000000..868b103
--- /dev/null
+++ b/doc/todo/make_copy_--fast__faster/comment_10_1af4ac0d37c876912678522895c1656b._comment
@@ -0,0 +1,61 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 10"""
+ date="2016-09-29T18:33:33Z"
+ content="""
+* Optimised key2file and file2key. 18% scanning time speedup.
+* Optimised adjustGitEnv. 50% git-annex branch query speedup
+* Optimised parsePOSIXTime. 10% git-annex branch query speedup
+* Tried making catObjectDetails.receive use ByteString for parsing, 
+  but that did not seem to speed it up significantly.
+  So it parsing is already fairly optimal, it's just that a
+  lot of data passes through it when querying the git-annex
+  branch.
+
+After all that, profiling `git-annex find`:
+
+	        Thu Sep 29 16:51 2016 Time and Allocation Profiling Report  (Final)
+	
+	           git-annex.1 +RTS -p -RTS find
+	
+	        total time  =        1.73 secs   (1730 ticks @ 1000 us, 1 processor)
+	        total alloc = 1,812,406,632 bytes  (excludes profiling overheads)
+	
+	COST CENTRE            MODULE                  %time %alloc
+	
+	md5                    Data.Hash.MD5            28.0   37.9
+	catchIO                Utility.Exception        10.2   12.5
+	inAnnex'.checkindirect Annex.Content             9.9    3.7
+	catches                Control.Monad.Catch       8.7    5.7
+	readish                Utility.PartialPrelude    5.7    3.0
+	isAnnexLink            Annex.Link                5.0    8.4
+	keyFile                Annex.Locations           4.2    5.8
+	spanList               Data.List.Utils           4.0    6.3
+	startswith             Data.List.Utils           2.0    1.3
+
+And `git-annex find --not --in web`:
+
+	        Thu Sep 29 16:35 2016 Time and Allocation Profiling Report  (Final)
+	
+	           git-annex +RTS -p -RTS find --not --in web
+	
+	        total time  =        5.24 secs   (5238 ticks @ 1000 us, 1 processor)
+	        total alloc = 3,293,314,472 bytes  (excludes profiling overheads)
+	
+	COST CENTRE               MODULE                      %time %alloc
+	
+	catObjectDetails.receive  Git.CatFile                  12.9    5.5
+	md5                       Data.Hash.MD5                10.6   20.8
+	readish                   Utility.PartialPrelude        7.3    8.2
+	catchIO                   Utility.Exception             6.7    7.3
+	spanList                  Data.List.Utils               4.1    7.4
+	readFileStrictAnyEncoding Utility.Misc                  3.5    1.3
+	catches                   Control.Monad.Catch           3.3    3.2
+
+So, quite a large speedup overall!
+
+This leaves md5 still unoptimised at 10-28% of CPU use. I looked at switching
+it to cryptohash's implementation, but it would require quite a lot of
+bit-banging math to pull the used values out of the ByteString containing
+the md5sum.
+"""]]

Added a comment
diff --git a/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_8_2e11e383dd4e95e8a3df30039face3eb._comment b/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_8_2e11e383dd4e95e8a3df30039face3eb._comment
new file mode 100644
index 0000000..0de769a
--- /dev/null
+++ b/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_8_2e11e383dd4e95e8a3df30039face3eb._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4"
+ subject="comment 8"
+ date="2016-09-29T20:45:48Z"
+ content="""
+extracting size from a key I have done myself already ;)  but what if it is for a relaxed download - it still wouldn't show (based on the downloader's information)? (yet to build/try to discover myself)
+"""]]

Add "total-size" field to --json-progress output.
diff --git a/CHANGELOG b/CHANGELOG
index e94c1b0..c36fcb7 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -6,6 +6,7 @@ git-annex (6.20160924) UNRELEASED; urgency=medium
     copies of the environment. Speeds up commands like 
     "git-annex find --in remote" by over 50%.
   * Optimised git-annex branch log file timestamp parsing.
+  * Add "total-size" field to --json-progress output.
 
  -- Joey Hess <id@joeyh.name>  Mon, 26 Sep 2016 16:46:19 -0400
 
diff --git a/Messages/JSON.hs b/Messages/JSON.hs
index 6c22d95..fb288b7 100644
--- a/Messages/JSON.hs
+++ b/Messages/JSON.hs
@@ -104,6 +104,7 @@ progress maction size bytesprocessed = emit $ case maction of
 	Object o = object
 		[ "byte-progress" .= n
 		, "percent-progress" .= showPercentage 2 (percentage size n)
+		, "total-size" .= size
 		]
 
 -- A value that can be displayed either normally, or as JSON.
diff --git a/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_7_dc29d3603ce7ce436fac7fa6d6f80871._comment b/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_7_dc29d3603ce7ce436fac7fa6d6f80871._comment
new file mode 100644
index 0000000..2acb1e1
--- /dev/null
+++ b/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_7_dc29d3603ce7ce436fac7fa6d6f80871._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 7"""
+ date="2016-09-29T20:27:46Z"
+ content="""
+Added a "total-size" field for the size. Although, if a key's size is not
+known, git-annex does not currently do any progress displays for that key.
+"""]]

followup
diff --git a/Annex/DirHashes.hs b/Annex/DirHashes.hs
index 91c3e78..004536c 100644
--- a/Annex/DirHashes.hs
+++ b/Annex/DirHashes.hs
@@ -14,6 +14,7 @@ module Annex.DirHashes (
 	dirHashes,
 	hashDirMixed,
 	hashDirLower,
+	display_32bits_as_dir
 ) where
 
 import Data.Bits
@@ -74,7 +75,7 @@ hashDirLower n k = hashDirs n 3 $ take 6 $ md5s $ md5FilePath $ key2file $ nonCh
  -}
 display_32bits_as_dir :: Word32 -> String
 display_32bits_as_dir w = trim $ swap_pairs cs
-  where 
+  where
 	-- Need 32 characters to use. To avoid inaverdently making
 	-- a real word, use letters that appear less frequently.
 	chars = ['0'..'9'] ++ "zqjxkmvwgpfZQJXKMVWGPF"
diff --git a/doc/forum/git-annex_add_out_of_memory_error/comment_3_91af8300d640c34ff2466c89ec7a234c._comment b/doc/forum/git-annex_add_out_of_memory_error/comment_3_91af8300d640c34ff2466c89ec7a234c._comment
new file mode 100644
index 0000000..603409b
--- /dev/null
+++ b/doc/forum/git-annex_add_out_of_memory_error/comment_3_91af8300d640c34ff2466c89ec7a234c._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2016-09-29T15:33:00Z"
+ content="""
+Had another report of this, and there NFS seemed to be involved in the
+circumstances of the crash.
+"""]]

diff --git a/doc/forum/uuid_mismatch___59___expected_Just___40__UUID_...___41___but_remote_gitrepo_has_UUID_....mdwn b/doc/forum/uuid_mismatch___59___expected_Just___40__UUID_...___41___but_remote_gitrepo_has_UUID_....mdwn
index a70d651..2d1626e 100644
--- a/doc/forum/uuid_mismatch___59___expected_Just___40__UUID_...___41___but_remote_gitrepo_has_UUID_....mdwn
+++ b/doc/forum/uuid_mismatch___59___expected_Just___40__UUID_...___41___but_remote_gitrepo_has_UUID_....mdwn
@@ -9,6 +9,8 @@ Internal Server Error
 
 Then, I can see the structure of the hard links in the local directory but pointing to nothing ?
 
+FYI all sub directories in .git/annex/objects contain .map files only.
+
 Could you please help me ? 
 Those data are critical, thanks for your great app and your help!
 

diff --git a/doc/forum/uuid_mismatch___59___expected_Just___40__UUID_...___41___but_remote_gitrepo_has_UUID_....mdwn b/doc/forum/uuid_mismatch___59___expected_Just___40__UUID_...___41___but_remote_gitrepo_has_UUID_....mdwn
index e274337..a70d651 100644
--- a/doc/forum/uuid_mismatch___59___expected_Just___40__UUID_...___41___but_remote_gitrepo_has_UUID_....mdwn
+++ b/doc/forum/uuid_mismatch___59___expected_Just___40__UUID_...___41___but_remote_gitrepo_has_UUID_....mdwn
@@ -3,6 +3,7 @@ Hi,
 After reinstalling my laptop with the same GPG keys, I've got this error during re-syncing a remote crypted repo :
 
 ```
+Internal Server Error
 ; expected Just (UUID "e7fc2bc8-b630-5eeb-a0b3-b4806d7ab064") but remote gitrepo has UUID "e2f02805-eae7-5bb8-875e-c901ba9929b1" (":id:SCPxErEQVlqp+Hl6Su0f")
 ```
 

diff --git a/doc/forum/uuid_mismatch___59___expected_Just___40__UUID_...___41___but_remote_gitrepo_has_UUID_....mdwn b/doc/forum/uuid_mismatch___59___expected_Just___40__UUID_...___41___but_remote_gitrepo_has_UUID_....mdwn
new file mode 100644
index 0000000..e274337
--- /dev/null
+++ b/doc/forum/uuid_mismatch___59___expected_Just___40__UUID_...___41___but_remote_gitrepo_has_UUID_....mdwn
@@ -0,0 +1,14 @@
+Hi,
+
+After reinstalling my laptop with the same GPG keys, I've got this error during re-syncing a remote crypted repo :
+
+```
+; expected Just (UUID "e7fc2bc8-b630-5eeb-a0b3-b4806d7ab064") but remote gitrepo has UUID "e2f02805-eae7-5bb8-875e-c901ba9929b1" (":id:SCPxErEQVlqp+Hl6Su0f")
+```
+
+Then, I can see the structure of the hard links in the local directory but pointing to nothing ?
+
+Could you please help me ? 
+Those data are critical, thanks for your great app and your help!
+
+Guillaume

Added a comment: Using gitolite 3.6.6 mirror not working with annex
diff --git a/doc/tips/using_gitolite_with_git-annex/comment_24_a62f59295b3912dc38def5e493bdb55a._comment b/doc/tips/using_gitolite_with_git-annex/comment_24_a62f59295b3912dc38def5e493bdb55a._comment
new file mode 100644
index 0000000..c6e4c02
--- /dev/null
+++ b/doc/tips/using_gitolite_with_git-annex/comment_24_a62f59295b3912dc38def5e493bdb55a._comment
@@ -0,0 +1,23 @@
+[[!comment format=mdwn
+ username="git-annex@5b470e1f6ed6d30997d729f0a8b1c841dea886f1"
+ nickname="git-annex"
+ subject="Using gitolite 3.6.6 mirror not working with annex"
+ date="2016-09-28T18:12:56Z"
+ content="""
+I find out that I need to add the following lines to the gitolite.rc
+in the server side.
+
+'git-annex-shell ua'
+
+The signal repository works with gitolite as expected.
+However, the mirroring feature is not working for slaves.
+When I do
+
+git annex copy --to origin
+
+The master server store the annex file correctly.
+The file managed by annex is not syncing to the slave
+mirrors at all.
+
+
+"""]]

remove incorrect bit about multiple concurrent transfers, and improve description of protocol flow
diff --git a/doc/design/external_special_remote_protocol.mdwn b/doc/design/external_special_remote_protocol.mdwn
index 40f764c..8cb7ed3 100644
--- a/doc/design/external_special_remote_protocol.mdwn
+++ b/doc/design/external_special_remote_protocol.mdwn
@@ -56,17 +56,18 @@ and check that it's valid. git-annex responds with the configuration values
 	VALUE true
 
 Once the special remote is satisfied with its configuration and is
-ready to go, it tells git-annex.
+ready to go, it tells git-annex that it's done with the PREPARE step:
 
 	PREPARE-SUCCESS
 
-Now git-annex will tell the special remote what to do. Let's suppose
-it wants to store a key. 
+Now git-annex will make a request. Let's suppose it wants to store a key. 
 
 	TRANSFER STORE somekey tmpfile
 
-The special remote can continue sending messages to git-annex during this
-transfer. It will typically send progress messages, indicating how many
+The special remote can then start reading the tmpfile and storing it.
+While it's doing that, the special remote can send messages back to 
+git-annex to indicate what it's doing, or ask for other information.
+It will typically send progress messages, indicating how many
 bytes have been sent:
 
 	PROGRESS 10240
@@ -76,6 +77,8 @@ Once the key has been stored, the special remote tells git-annex the result:
 
 	TRANSFER-SUCCESS STORE somekey
 
+Now git-annex will send its next request.
+
 Once git-annex is done with the special remote, it will close its stdin.
 The special remote program can then exit.
 
@@ -107,8 +110,6 @@ The following requests *must* all be supported by the special remote.
   The filename will not contain any whitespace.  
   Note that it's important that, while a Key is being stored, CHECKPRESENT
   not indicate it's present until all the data has been transferred.  
-  Multiple transfers might be requested by git-annex, but it's fine for the 
-  program to serialize them and only do one at a time.  
 * `CHECKPRESENT Key`  
   Requests the remote to check if a key is present in it.
 * `REMOVE Key`  
@@ -215,7 +216,7 @@ while it's handling a request.
 ## special remote messages
 
 These messages may be sent by the special remote at any time that it's
-in control.
+handling a request.
 
 * `VERSION Int`  
   Supported protocol version. Current version is 1. Must be sent first

todo
diff --git a/doc/todo/external_special_remote_not_used_concurrently_with_-J.mdwn b/doc/todo/external_special_remote_not_used_concurrently_with_-J.mdwn
new file mode 100644
index 0000000..a515d35
--- /dev/null
+++ b/doc/todo/external_special_remote_not_used_concurrently_with_-J.mdwn
@@ -0,0 +1,9 @@
+When using an external special remote with -Jn, multiple transfers do not
+happen concurrently to the remote. A single process is
+started for the external special remote, and there's locking to only let
+one request be made of it at a time.
+
+This should not be hard to make to use a pool of Externals, starting up a
+new one if the pool is empty or all in use. --[[Joey]]
+
+[[users/yoh]] may want this for datalad?

devblog
diff --git a/doc/devblog/day_416__measure_twice.mdwn b/doc/devblog/day_416__measure_twice.mdwn
new file mode 100644
index 0000000..e3d9f00
--- /dev/null
+++ b/doc/devblog/day_416__measure_twice.mdwn
@@ -0,0 +1,8 @@
+Only had a couple hours today, which were spent doing some profiling of
+git-annex in situations where it has to look through a large working tree in
+order to find files to act on. The top five hot spots this found are
+responsible for between 50% and 80% of git-annex's total CPU use in these
+situations.
+
+The first optimisation sped up `git annex find` by around 18%.
+More tomorrow..

comment
diff --git a/doc/bugs/Metadata_charset_not_uniform/comment_1_bb6a2016801687ef38522611d7b6f2bc._comment b/doc/bugs/Metadata_charset_not_uniform/comment_1_bb6a2016801687ef38522611d7b6f2bc._comment
new file mode 100644
index 0000000..e0ac56c
--- /dev/null
+++ b/doc/bugs/Metadata_charset_not_uniform/comment_1_bb6a2016801687ef38522611d7b6f2bc._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2016-09-26T20:50:26Z"
+ content="""
+The metadata you get out should always be encoded the same as the metadata
+you put in. The encoding, or encodings used are up to you.
+
+Are you seeing metadata queries returning a different sequence of bytes
+than the sequence of bytes that were originally stored? If not, I don't
+think this is a bug.
+"""]]

more profiling
diff --git a/doc/todo/make_copy_--fast__faster/comment_9_f4d802a28b79905da0cb24af6cb65b0a._comment b/doc/todo/make_copy_--fast__faster/comment_9_f4d802a28b79905da0cb24af6cb65b0a._comment
new file mode 100644
index 0000000..9692ad2
--- /dev/null
+++ b/doc/todo/make_copy_--fast__faster/comment_9_f4d802a28b79905da0cb24af6cb65b0a._comment
@@ -0,0 +1,42 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""more profiling"""
+ date="2016-09-26T19:59:43Z"
+ content="""
+Instead of profiling `git annex copy --to remote`, I profiled `git annex
+find --not --in web`, which needs to do the same kind of location log lookup.
+
+	        total time  =       12.41 secs   (12413 ticks @ 1000 us, 1 processor)
+	        total alloc = 8,645,057,104 bytes  (excludes profiling overheads)
+	
+	COST CENTRE               MODULE                      %time %alloc
+	
+	adjustGitEnv              Git.Env                      21.4   37.0
+	catchIO                   Utility.Exception            13.2    2.8
+	spanList                  Data.List.Utils              12.6   17.9
+	parsePOSIXTime            Logs.TimeStamp                6.1    5.0
+	catObjectDetails.receive  Git.CatFile                   5.9    2.1
+	startswith                Data.List.Utils               5.7    3.8
+	md5                       Data.Hash.MD5                 5.1    7.9
+	join                      Data.List.Utils               2.4    6.0
+	readFileStrictAnyEncoding Utility.Misc                  2.2    0.5
+
+The adjustGitEnv overhead is a surprise! It seems it is getting called once
+per file, and allocating a new copy of the environment each time. Call stack:
+withIndex calls withIndexFile calls addGitEnv calls adjustGitEnv.
+Looks like simply making gitEnv be cached at startup would avoid most of
+the adjustGitEnv slowdown.
+
+(The catchIO overhead is a false reading; the detailed profile shows
+that all its time and allocations are inherited. getAnnexLinkTarget
+is running catchIO in the expensive case, so readSymbolicLink is
+the actual expensive bit.)
+
+The parsePOSIXTime comes from reading location logs. It's implemented
+using a generic Data.Time.Format.parseTime, which uses a format string
+"%s%Qs". A custom parser that splits into seconds and picoseconds
+and simply reads both numbers might be more efficient.
+
+catObjectDetails.receive is implemented using mostly String and could
+probably be sped up by being converted to use ByteString.
+"""]]

profiling
diff --git a/doc/todo/make_copy_--fast__faster/comment_8_c1f99493f5e5c362d5c39f048280b11b._comment b/doc/todo/make_copy_--fast__faster/comment_8_c1f99493f5e5c362d5c39f048280b11b._comment
new file mode 100644
index 0000000..e0f4987
--- /dev/null
+++ b/doc/todo/make_copy_--fast__faster/comment_8_c1f99493f5e5c362d5c39f048280b11b._comment
@@ -0,0 +1,45 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""profiling"""
+ date="2016-09-26T19:20:36Z"
+ content="""
+Built git-annex with profiling, using `stack build --profile`
+
+(For reproduciblity, running git-annex in a clone of the git-annex repo
+https://github.com/RichiH/conference_proceedings with rev
+2797a49023fc24aff6fcaec55421572e1eddcfa2 checked out. It has 9496 annexed
+objects.)
+
+Profiling `git-annex find +RTS -p`:
+
+	        total time  =        3.53 secs   (3530 ticks @ 1000 us, 1 processor)
+	        total alloc = 3,772,700,720 bytes  (excludes profiling overheads)
+	
+	COST CENTRE            MODULE                  %time %alloc
+	
+	spanList               Data.List.Utils          32.6   37.7
+	startswith             Data.List.Utils          14.3    8.1
+	md5                    Data.Hash.MD5            12.4   18.2
+	join                   Data.List.Utils           6.9   13.7
+	catchIO                Utility.Exception         5.9    6.0
+	catches                Control.Monad.Catch       5.0    2.8
+	inAnnex'.checkindirect Annex.Content             4.6    1.8
+	readish                Utility.PartialPrelude    3.0    1.4
+	isAnnexLink            Annex.Link                2.6    4.0
+	split                  Data.List.Utils           1.5    0.8
+	keyPath                Annex.Locations           1.2    1.7
+
+
+This is interesting!
+
+Fully 40% of CPU time and allocations are in list (really String) processing,
+and the details of the profiling report show that `spanList` and `startsWith`
+and `join` are all coming from calls to `replace` in `keyFile` and `fileKey`.
+Both functions nest several calls to replace, so perhaps that could be unwound
+into a single pass and/or a ByteString used to do it more efficiently.
+
+12% of run time is spent calculating the md5 hashes for the hash
+directories for .git/annex/objects. Data.Hash.MD5 is from missingh, and
+it is probably a quite unoptimised version. Switching to the version
+if cryptonite would probably speed it up a lot.
+"""]]

diff --git a/doc/bugs/Assistant_does_not_update_adjusted_branch.mdwn b/doc/bugs/Assistant_does_not_update_adjusted_branch.mdwn
index d666401..83fedbb 100644
--- a/doc/bugs/Assistant_does_not_update_adjusted_branch.mdwn
+++ b/doc/bugs/Assistant_does_not_update_adjusted_branch.mdwn
@@ -15,6 +15,7 @@ I would expect that the assistant is just doing this, that I don't have to trigg
 ### What steps will reproduce the problem?
 
 Synchronizing several git annex repositories (three) with the assistant, with the second one being in the adjusted branch:
+
 * Create a new file in the first repository (normal v5, autocommit=true)
 * The file is being commited automatically and synchronized into the 3rd repository (normal v5).
 * But the file does not appear in the 2nd one, being in the adjusted branch

diff --git a/doc/bugs/Assistant_does_not_update_adjusted_branch.mdwn b/doc/bugs/Assistant_does_not_update_adjusted_branch.mdwn
new file mode 100644
index 0000000..d666401
--- /dev/null
+++ b/doc/bugs/Assistant_does_not_update_adjusted_branch.mdwn
@@ -0,0 +1,89 @@
+### Please describe the problem.
+
+When using the assistant to synchronize several annex repositories, changes are not correctly or not at all propagated to repositories that are in the adjusted branch.
+
+* New files from other repositories are not created automatically in adjusted branch repos
+* Commited changes in files from other repositories are not reflected in in adjusted branch repos
+
+The manual states:
+>	To propigate  changes from  the adjusted branch  back to  the original
+>	branch, and to other repositories, as well as to merge in changes from
+>	other repositories, use git annex sync.
+
+I would expect that the assistant is just doing this, that I don't have to trigger the sync in the adjusted branch repo by myself.
+
+### What steps will reproduce the problem?
+
+Synchronizing several git annex repositories (three) with the assistant, with the second one being in the adjusted branch:
+* Create a new file in the first repository (normal v5, autocommit=true)
+* The file is being commited automatically and synchronized into the 3rd repository (normal v5).
+* But the file does not appear in the 2nd one, being in the adjusted branch
+
+Once you trigger
+    git annex sync
+in the 2nd repository, it seems the adjusted branch synchronizes with the master and the file appears.
+
+The same happens with changes in files.
+
+### What version of git-annex are you using? On what operating system?
+
+* git-annex version: 6.20160511-1~ubuntu14.04.1~ppa1
+* Linux 4.4.0-24-generic #43~14.04.1-Ubuntu x86_64
+* Linux Mint 17.3 Rosa
+
+### Please provide any additional information below.
+
+Assistant log from adjust branch repo.
+test7.txt is the new file created on other repo which didn't showed up immediately.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+(merging origin/git-annex into git-annex...)
+To /home/sven/Temporary/Annex1
+ + 9cc4eea...0e6c4ba git-annex -> synced/git-annex (forced update)
+(started...) 
+(merging synced/git-annex into git-annex...)
+(Merging into master...) Merge made by the 'recursive' strategy.
+ script.sh | 1 +
+ 1 file changed, 1 insertion(+)
+ create mode 100644 script.sh
+(Merging into adjusted branch...) 
+Updating 449f26f..09950df
+Fast-forward
+ script.sh | 1 +
+ 1 file changed, 1 insertion(+)
+ create mode 100644 script.sh
+[2016-09-25 16:32:28.594537] Committer: Committing changes to git
+(recording state in git...)
+[2016-09-25 16:32:28.631525] Pusher: Syncing with origin 
+To /home/sven/Temporary/Annex1
+   0e6c4ba..578f070  git-annex -> synced/git-annex
+   9754018..38c3683  master -> synced/master
+[2016-09-25 16:32:29.632224] Committer: Committing changes to git
+(recording state in git...)
+[2016-09-25 16:32:30.661338] Pusher: Syncing with origin 
+Everything up-to-date
+(merging synced/git-annex into git-annex...)
+(merging synced/git-annex into git-annex...)
+(recording state in git...)
+git-annex: Daemon is already running.
+[2016-09-25 16:36:58.360139] Committer: Adding test7.txt
+add test7.txt ok
+[2016-09-25 16:36:58.476985] Committer: Committing changes to git
+(recording state in git...)
+[2016-09-25 16:36:58.494964] Pusher: Syncing with origin 
+Everything up-to-date
+[2016-09-25 16:36:59.575623] Committer: Adding test7.txt
+add test7.txt ok
+[2016-09-25 16:36:59.578454] Committer: Committing changes to git
+(recording state in git...)
+[2016-09-25 16:37:00.506906] Pusher: Syncing with origin 
+Everything up-to-date
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+I'm using git-annex a lot for synchronizing stuff between work, home, external backup discs, and now that I see the autocommit=false flag I'll surely start to use the assistant.
+And the new adjust branch is perfect for one of my use cases, but not like it behaves right now.

diff --git a/doc/bugs/Metadata_charset_not_uniform.mdwn b/doc/bugs/Metadata_charset_not_uniform.mdwn
new file mode 100644
index 0000000..19f6aa4
--- /dev/null
+++ b/doc/bugs/Metadata_charset_not_uniform.mdwn
@@ -0,0 +1,64 @@
+### Please describe the problem.
+
+Metadata are not stored in a consistent format. It seems more like git-annex chooses the "smallest" charset able to hold the data, i.e. US-ASCII, unless there are latin1 characters, and only UTF-8 if there are UTF-8 characters that are not in latin1
+
+### What steps will reproduce the problem?
+
+    % git init
+    Initialized empty Git repository in /home/madduck/.tmp/cdt.GlIevu/.git/
+    
+    % git annex init
+    init  ok
+    (recording state in git...)
+    
+    % date > a
+    
+    % git annex add a
+    add a ok
+    (recording state in git...)
+    
+    % git annex metadata -s one=$(echo US-ASCII | iconv -tus-ascii) a
+    metadata a 
+      lastchanged=2016-09-25@13-18-57
+      one=US-ASCII
+      one-lastchanged=2016-09-25@13-18-57
+    ok
+    (recording state in git...)
+    
+    % git annex metadata -s two=$(echo lätin1 | iconv -tlatin1) a
+    metadata a 
+      lastchanged=2016-09-25@13-19-37
+      one=US-ASCII
+      one-lastchanged=2016-09-25@13-18-57
+      two=lätin1
+      two-lastchanged=2016-09-25@13-19-37
+    ok
+    (recording state in git...)
+    
+    % git annex metadata -s three=$(echo unicode… | iconv -tutf8) a  
+    metadata a 
+      lastchanged=2016-09-25@13-19-41
+      one=US-ASCII
+      one-lastchanged=2016-09-25@13-18-57
+      three=unicode…
+      three-lastchanged=2016-09-25@13-19-41
+      two=lätin1
+      two-lastchanged=2016-09-25@13-19-37
+    ok
+    (recording state in git...)
+    
+    % git annex metadata -g three a | iconv -tutf8                 
+    unicode…
+    
+    % git annex metadata -g two a | iconv -tutf8 
+    liconv: illegal input sequence at position 1
+    
+    % git annex metadata -g one a | iconv -tutf8 
+    US-ASCII
+    
+    % git annex metadata -g two a | iconv -flatin1 -tutf8 
+    lätin1
+
+### What version of git-annex are you using? On what operating system?
+
+6.20160808-1

Added a comment
diff --git a/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_7_8da1f5fc6656e92dc2e3dabd9808f2d1._comment b/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_7_8da1f5fc6656e92dc2e3dabd9808f2d1._comment
new file mode 100644
index 0000000..05fe101
--- /dev/null
+++ b/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_7_8da1f5fc6656e92dc2e3dabd9808f2d1._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="PaulK"
+ subject="comment 7"
+ date="2016-09-25T03:08:40Z"
+ content="""
+As an aside, just tried syncthing's arm verion, and I get a \"runtime: kernel page size (32768) is larger than runtime page size (4096)\" error. While their arm64 version also won't run.  
+
+Perhaps it's a similar issue with git-annex? Seems the page-size comes up in a number of contexts of people trying to get software running on the WD NAS.
+
+Is there any way to adjust  this page-size in the linker?
+
+Thanks
+"""]]

diff --git a/doc/forum/DVD-R-based_archive_--_lack_of_backward_compatibility_seems_like_a_showstopper.mdwn b/doc/forum/DVD-R-based_archive_--_lack_of_backward_compatibility_seems_like_a_showstopper.mdwn
new file mode 100644
index 0000000..bc89b13
--- /dev/null
+++ b/doc/forum/DVD-R-based_archive_--_lack_of_backward_compatibility_seems_like_a_showstopper.mdwn
@@ -0,0 +1,15 @@
+I've been planning to use git-annex to manage a collection of DVD-Rs (of photos and the like), something along [these lines](https://git-annex.branchable.com/forum/How_do_I_backup_my_data_to_blue_ray_disks__63__/), and especially this paragraph therein:
+
+> If the [media is] not rewritable [which in my case it won't be; see below], I might try making the git-annex repo in a temporary directory on the hard disk, and then generating the ISO from that once I've filled it up. Should work fine, I might set "remote..annex-readonly" to true in git repos that had such a disk as a remote to let git-annex know to not try to write to it.
+
+But then there's this bug report: [read-only filesystem on remote prevents auto-upgrade from v3 to v5, and prevents using a remote](https://git-annex.branchable.com/bugs/annex_get_fails_from_read-only_filesystem/), and [Joey's comment]((http://git-annex.branchable.com/bugs/annex_get_fails_from_read-only_filesystem/#comment-67f204c3a4312bbda8ace305dbe0afbf):
+
+> From a code complication POV, it's useful for git-annex to only support one version of repository at a time.
+
+My concept depends on being able to restore files from my archive, N years and M git-annex versions later, using `git annex get` (after `git annex whereis` or the like to find out which volume to mount).  If I can't count on the *get* to work -- and indeed, can pretty much count on it *not* to, after the next format-version bump -- this whole scheme sounds like a nonstarter.
+
+> As far as backwards compatablity goes, I don't anticipate ever removing the upgrade code from git-annex.
+
+This is an archive; therefore it will use write-once media -- if it were rewritable, it would be clobberable should something (including `git-annex upgrade`!) go wrong.
+
+So, is there a way around this?  Or am I (and others) out of luck on using git-annex to manage long-term archival storage?

Added a comment
diff --git a/doc/todo/transitive_transfers/comment_4_cb92e81c1ea00ca72f9a5ee729d627f4._comment b/doc/todo/transitive_transfers/comment_4_cb92e81c1ea00ca72f9a5ee729d627f4._comment
new file mode 100644
index 0000000..bf6887e
--- /dev/null
+++ b/doc/todo/transitive_transfers/comment_4_cb92e81c1ea00ca72f9a5ee729d627f4._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="erics"
+ subject="comment 4"
+ date="2016-09-25T00:33:56Z"
+ content="""
+> And there is a complication with running [`git annex copy --from --to`] at the same time as eg `git annex get` of the same file. It would be surprising for get to succeed (because copy has already temporarily downloaded the file) and then have the file later get dropped.
+
+A solution to this subproblem would transparently fall out of a facility for [logically dropping files](http://git-annex.branchable.com/todo/wishlist__58_____96__git_annex_drop_--relaxed__96__/#comment-9672d46a18a381971dd818cde6efbc33), which was briefly talked about a long time ago.  Just mark the file as *logically dropped*.  If the user `git annex get`s it while the copy-out is in progress, its status will change to \"present\", so *copy* will know not to physically delete it.
+
+(Of course there are race conditions involved, but I presume/hope that they're no worse than git-annex already has to deal with.)
+"""]]

Added a comment
diff --git a/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_6_8e05f8769870b3f1aec77fca06f37284._comment b/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_6_8e05f8769870b3f1aec77fca06f37284._comment
new file mode 100644
index 0000000..d56e90c
--- /dev/null
+++ b/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_6_8e05f8769870b3f1aec77fca06f37284._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="PaulK"
+ subject="comment 6"
+ date="2016-09-24T04:16:51Z"
+ content="""
+Nope, no difference. Same \"ELF load command alignment not page-aligned\" errors as before.
+"""]]

Added a comment: +may be "byte-target" field? ;)
diff --git a/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_6_25f9cd3d341c60a059f462a34c89d1d6._comment b/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_6_25f9cd3d341c60a059f462a34c89d1d6._comment
new file mode 100644
index 0000000..5b1134b
--- /dev/null
+++ b/doc/todo/interface_to_the___34__progress__34___of_annex_operations/comment_6_25f9cd3d341c60a059f462a34c89d1d6._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4"
+ subject="+may be &quot;byte-target&quot; field? ;)"
+ date="2016-09-24T02:43:21Z"
+ content="""
+THANK YOU for implementing this feature -- we will make use of it soon.
+But so we don't do reverse estimation from \"byte-progress\" and \"percent-progress\", and didn't have to get it from a key (which might not have it e.g. in case of URL relaxed backend) -- could you just include in each record the \"byte-target\" (if known) or something like that? ;)  thanks in advance!
+"""]]

cleanup
diff --git a/doc/news/version_6.20160419/comment_1_b7d20a64e3ddb99a30920e8f614abe6c._comment b/doc/news/version_6.20160419/comment_1_b7d20a64e3ddb99a30920e8f614abe6c._comment
deleted file mode 100644
index 9de1834..0000000
--- a/doc/news/version_6.20160419/comment_1_b7d20a64e3ddb99a30920e8f614abe6c._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="zfnupbos@a80a4f4cf5dde7ac4988ef0d077052c95fe1e058"
- nickname="zfnupbos"
- subject="version clarification"
- date="2016-05-04T01:05:52Z"
- content="""
-is this 6.20160419 or 6.20160428?
-
-It was released on the 28th, and 'git-annex version' shows git-annex version: 6.20160428-g1f253e8. But many places refer to it as 6.20160419.
-"""]]
diff --git a/doc/news/version_6.20160419/comment_2_88dd2807bda89ca9c61826e68140f7af._comment b/doc/news/version_6.20160419/comment_2_88dd2807bda89ca9c61826e68140f7af._comment
deleted file mode 100644
index 1033182..0000000
--- a/doc/news/version_6.20160419/comment_2_88dd2807bda89ca9c61826e68140f7af._comment
+++ /dev/null
@@ -1,14 +0,0 @@
-[[!comment format=mdwn
- username="joey"
- subject="""comment 2"""
- date="2016-05-04T17:19:17Z"
- content="""
-I should have changed the version number to 6.20160428, but I forgot to do
-so.
-
-The version shows up in `git annex version` because it uses the build
-date there, for autobuilds.
-
-So in short, both versions are equivilant, and both contain the security
-fix.
-"""]]
diff --git a/doc/news/version_6.20160511/comment_1_c7ad57b324c94d0a3eb00b9d624f7b66._comment b/doc/news/version_6.20160511/comment_1_c7ad57b324c94d0a3eb00b9d624f7b66._comment
deleted file mode 100644
index a48431d..0000000
--- a/doc/news/version_6.20160511/comment_1_c7ad57b324c94d0a3eb00b9d624f7b66._comment
+++ /dev/null
@@ -1,9 +0,0 @@
-[[!comment format=mdwn
- username="CandyAngel"
- subject="comment 1"
- date="2016-05-16T11:41:08Z"
- content="""
-I've very happy to report that the git-add, git-annex-fsck & 'git-annex-reinject --known' workflow is working great!
-
-Thank you for adding these :)
-"""]]
diff --git a/doc/news/version_6.20160613/comment_1_89cfca1174ea3e50a536053e73027296._comment b/doc/news/version_6.20160613/comment_1_89cfca1174ea3e50a536053e73027296._comment
deleted file mode 100644
index 67a0dc7..0000000
--- a/doc/news/version_6.20160613/comment_1_89cfca1174ea3e50a536053e73027296._comment
+++ /dev/null
@@ -1,9 +0,0 @@
-[[!comment format=mdwn
- username="CandyAngel"
- subject="comment 1"
- date="2016-06-21T08:46:50Z"
- content="""
-     Improve SHA*E extension extraction code.
-
-This sounds really likely to break things like [recovering files by readding](https://git-annex.branchable.com/tips/recover_data_from_lost+found/), because the key is going to be different, despite having the same backend..
-"""]]
diff --git a/doc/news/version_6.20160613/comment_2_c30ce151490345ac5283e3900c26f173._comment b/doc/news/version_6.20160613/comment_2_c30ce151490345ac5283e3900c26f173._comment
deleted file mode 100644
index 0df9710..0000000
--- a/doc/news/version_6.20160613/comment_2_c30ce151490345ac5283e3900c26f173._comment
+++ /dev/null
@@ -1,11 +0,0 @@
-[[!comment format=mdwn
- username="joey"
- subject="""comment 2"""
- date="2016-07-12T19:55:52Z"
- content="""
-In this case, the change only makes eg, "foo.ba__________r" not be considered
-to have an extension of ".bar". So it's unlikely to affect many files.
-
-I do try to keep the extension guessing code fairly stable to prevent breaking
-re-adding. But sometimes a bug does warrant changing it.
-"""]]

add news item for git-annex 6.20160923
diff --git a/doc/news/version_6.20160613.mdwn b/doc/news/version_6.20160613.mdwn
deleted file mode 100644
index db338b2..0000000
--- a/doc/news/version_6.20160613.mdwn
+++ /dev/null
@@ -1,34 +0,0 @@
-git-annex 6.20160613 released with [[!toggle text="these changes"]]
-[[!toggleable text="""
-   * Improve SHA*E extension extraction code.
-   * Windows: Avoid terminating git-annex branch lines with \r\n when
-     union merging and performing transitions.
-   * Remove Makefile from cabal tarball; man page building is now handled by
-     a small haskell program.
-   * sync --content: Fix bug that caused transfers of files to be made
-     to a git remote that does not have a UUID. This particularly impacted
-     clones from gcrypt repositories.
-   * Pass -S to git commit-tree when commit.gpgsign is set and when
-     making a non-automatic commit, in order to preserve current behavior
-     when used with git 2.9, which has stopped doing this itself.
-   * remotedaemon: Fixed support for notifications of changes to gcrypt
-     remotes, which was never tested and didn't quite work before.
-   * list: Do not include dead repositories.
-   * move --to: Better behavior when system is completely out of disk space;
-     drop content from disk before writing location log.
-   * Avoid a crash if getpwuid does not work, when querying the user's full
-     name.
-   * Automatically enable v6 mode when initializing in a clone from a repo
-     that has an adjusted branch checked out.
-   * v6: Fix initialization of a bare clone of a repo that has an adjusted
-     branch checked out.
-   * v6: Fix bad automatic merge conflict resolution between an annexed file
-     and a directory with the same name when in an adjusted branch.
-   * v6: Fix bad merge in an adjusted branch that resulted in an empty tree.
-   * v6: Fix bug in initialization of clone from a repo with an adjusted branch
-     that had not been synced back to master.
-     (This bug caused broken tree objects to get built by a later git annex
-     sync.)
-   * v6: Make lock and unlock work on files whose content is not present.
-   * v6: Fix update of associated files db when unlocking a file.
-   * v6: Make git clean filter preserve the backend that was used for a file."""]]
\ No newline at end of file
diff --git a/doc/news/version_6.20160923.mdwn b/doc/news/version_6.20160923.mdwn
new file mode 100644
index 0000000..d74ef61
--- /dev/null
+++ b/doc/news/version_6.20160923.mdwn
@@ -0,0 +1,34 @@
+git-annex 6.20160923 released with [[!toggle text="these changes"]]
+[[!toggleable text="""
+   * Rate limit console progress display updates to 10 per second.
+     Was updating as frequently as changes were reported, up to hundreds of
+     times per second, which used unncessary bandwidth when running git-annex
+     over ssh etc.
+   * Make --json and --quiet work when used with -J.
+     Previously, -J override the other options.
+   * addurl, get: Added --json-progress option, which adds progress
+     objects to the json output.
+   * Remove key:null from git-annex add --json output.
+   * copy, move, mirror: Support --json and --json-progress.
+   * Improve gpg secret key list parser to deal with changes in gpg 2.1.15.
+     Fixes key name display in webapp.
+   * info: Support being passed a treeish, and show info about the annexed
+     files in it similar to how a directory is handled.
+   * sync: Previously, when run in a branch with a slash in its name,
+     such as "foo/bar", the sync branch was "synced/bar". That conflicted
+     with the sync branch used for branch "bar", so has been changed to
+     "synced/foo/bar".
+   * Note that if you're using an old version of git-annex to sync with
+     a branch with a slash in its name, it won't see some changes synced by
+     this version, and this version won't see some changes synced by the older
+     version. This is not a problem if there's a central bare repository,
+     but may impact other configurations until git-annex is upgraded to this
+     version.
+   * adjust: Previously, when adjusting a branch with a slash in its name,
+     such as "foo/bar", the adjusted branch was "adjusted/bar(unlocked)".
+     That conflicted with the adjusted branch used for branch "bar",
+     so has been changed to "adjusted/foo/bar(unlocked)"
+   * Also, running sync in an adjusted branch did not correctly sync
+     changes back to the parent branch when it had a slash in its name.
+     This bug has been fixed.
+   * addurl, importfeed: Improve behavior when file being added is gitignored."""]]
\ No newline at end of file

Added a comment: thanks for considering this!
diff --git a/doc/todo/transitive_transfers/comment_3_e5c2ede203e7bdb5af8432df6c09268f._comment b/doc/todo/transitive_transfers/comment_3_e5c2ede203e7bdb5af8432df6c09268f._comment
new file mode 100644
index 0000000..7023cd2
--- /dev/null
+++ b/doc/todo/transitive_transfers/comment_3_e5c2ede203e7bdb5af8432df6c09268f._comment
@@ -0,0 +1,81 @@
+[[!comment format=mdwn
+ username="https://anarc.at/openid/"
+ nickname="anarcat"
+ subject="thanks for considering this!"
+ date="2016-09-22T12:43:11Z"
+ content="""
+> (Let's not discuss the behavior of copy --to when the file is not
+> locally present here; there is plenty of other discussion of that in
+> eg http://bugs.debian.org/671179)
+
+Agreed, it's kind of secondary.
+
+> git-annex's special remote API does not allow remote-to-remote
+> transfers without spooling it to a file on disk first.
+
+yeah, i noticed that when writing my own special remote.
+
+> And it's not possible to do using rsync on either end, AFAICS.
+
+That is correct.
+
+> It would be possible in some other cases but this would need to be
+> implemented for each type of remote as a new API call.
+
+... and would fail for most, so there's little benefit there.
+
+how about a socket or FIFO of some sort? i know those break a lot of
+semantics (e.g. `[ -f /tmp/fifo ]` fails in bash) but they could be a
+solution...
+
+> Modern systems tend to have quite a large disk cache, so it's quite
+> possible that going via a temp file on disk is not going to use a
+> lot of disk IO to write and read it when the read and write occur
+> fairly close together.
+
+true. there are also in-memory files that could be used, although I
+don't think this would work across different process spaces.
+
+> The main benefit from streaming would probably be if it could run
+> the download and the upload concurrently.
+
+for me, the main benefit would be to deal with low disk space
+conditions, which is quite common on my machines: i often cram the
+disk almost to capacity with good stuff i want to listen to
+later... git-annex allows me to freely remove stuff when i need the
+space, but it often means i am close to 99% capacity on the media
+drives i use.
+
+>  But that would only be a benefit sometimes. With an asymmetric
+>  connection, saturating the uplink tends to swamp downloads. Also,
+>  if download is faster than upload, it would have to throttle
+>  downloads (which complicates the remote API much more), or buffer
+>  them to memory (which has its own complications).
+
+that is true.
+
+> Streaming the download to the upload would at best speed things up
+> by a factor of 2. It would probably work nearly as well to upload
+> the previously downloaded file while downloading the next file.
+
+presented like that, it's true that the benefits of streaming are not
+good enough to justify the complexity - the only problem is large
+files and low local disk space... but maybe we can delegate that
+solution to the user: \"free up at least enough space for one of those
+files you want to transfer\".
+
+[... -J magic stuff ...]
+
+> And there is a complication with running that at the same time as eg
+> git annex get of the same file. It would be surprising for get to
+> succeed (because copy has already temporarily downloaded the file)
+> and then have the file later get dropped. So, it seems that copy
+> --from --to would need to stash the content away in a temp file
+> somewhere instead of storing it in the annex proper.
+
+My thoughts exactly: actually copying the files to the local repo
+introduces all sorts of weird --numcopies nastiness and race
+conditions, it seems to me.
+
+thanks for considering this!
+"""]]

Added a comment: simpler use case
diff --git a/doc/todo/transitive_transfers/comment_2_7255f5083283b0ae7e7fc41e127bd829._comment b/doc/todo/transitive_transfers/comment_2_7255f5083283b0ae7e7fc41e127bd829._comment
new file mode 100644
index 0000000..54d9c80
--- /dev/null
+++ b/doc/todo/transitive_transfers/comment_2_7255f5083283b0ae7e7fc41e127bd829._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="JasonWoof"
+ subject="simpler use case"
+ date="2016-09-22T00:40:07Z"
+ content="""
+Here's my use case (much simpler)
+
+Three git repos:
+
+desktop: normal checkout, source of almost all annexd files, commits, etc.. The only place I run git annex commands. Not enough space to stored all annexed files
+
+main_external: bare git repo, stores all annext file contents, but no file tree. Usually connected. Purpose: primary backups
+
+old_external: like main_external, except connected only occasionally.
+
+
+I periodically copy from desktop to main_external. That's all well and good.
+
+The tricky part is when I plug in old_external and want to get everything on there. It's hard to get content onto old_external that is stored only on main_external. That's when I want to:
+
+    git annex copy --from=main_external --to=old_external --not --in old_external
+
+Note that this would _not_ copy obsolete data (ie only referenced from old git commits)  stored in old_external. I like that.
+
+To work around the lack of that feature, I try to keep coppies on desktop until I've had a chance to copy them to both external drives. It's good for numcopies, but I don't like trying to keep track of it, and I wish I could choose to let there be just one copy of things on main_external for replaceable data.
+"""]]

Added a comment
diff --git a/doc/todo/Allow_globally_limiting_filename_length/comment_5_2b49a293b044d0c2fcfe1701c76424c4._comment b/doc/todo/Allow_globally_limiting_filename_length/comment_5_2b49a293b044d0c2fcfe1701c76424c4._comment
new file mode 100644
index 0000000..8c4c6f0
--- /dev/null
+++ b/doc/todo/Allow_globally_limiting_filename_length/comment_5_2b49a293b044d0c2fcfe1701c76424c4._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="interfect@b151490178830f44348aa57b77ad58c7d18e8fe7"
+ nickname="interfect"
+ subject="comment 5"
+ date="2016-09-21T22:49:55Z"
+ content="""
+OK, I'll try something like that.
+
+(Full disk encryption is still there; I think on one system I just have ecryptfs, because I want to be able to get in over ssh sometimes, and on one I have *both* FDE and ecryptfs on, because I enjoy performance penalties.)
+"""]]

devblog
diff --git a/doc/devblog/day_415__catching_up.mdwn b/doc/devblog/day_415__catching_up.mdwn
new file mode 100644
index 0000000..d2828a0
--- /dev/null
+++ b/doc/devblog/day_415__catching_up.mdwn
@@ -0,0 +1,19 @@
+Catching up on backlog today. I hope to be back to a regular work schedule
+now. Unanswered messages down to 156. A lot of time today spent answering
+questions.
+
+There were several problems involving git branches with slashes in their
+name, such as "foo/bar" (but not "origin/master" or "refs/heads/foo").
+Some branch names based on such a branch would take only the "bar" part.
+In `git annex sync`, this led to perhaps merging "foo/bar" into "other/bar"
+or "bar". And the adjusted branch code was entirely broken for such
+branches. I've fixed it now.
+
+Also made `git annex addurl` behave better when the file it wants to 
+add is gitignored.
+
+Thinking about implementing `git annex copy --from A --to B`.
+It does not seem *too* hard to do that, at least with a temp file
+used inbetween. See [[todo/transitive_transfers]].
+
+Today's work was sponsored by Thomas Hochstein on Patreon.

update
diff --git a/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_5_5fa9d10ebd639eb1a836866b99e90aab._comment b/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_5_5fa9d10ebd639eb1a836866b99e90aab._comment
new file mode 100644
index 0000000..a4784ce
--- /dev/null
+++ b/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_5_5fa9d10ebd639eb1a836866b99e90aab._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 5"""
+ date="2016-09-21T16:44:08Z"
+ content="""
+I've updated the [arm daily build](https://downloads.kitenet.net/git-annex/autobuild/armel/git-annex-standalone-armel.tar.gz) to use ld.gold temporarily. 
+
+Please give it a try and report back if it works and if so I'll
+make the change permanant.
+"""]]

comment
diff --git a/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_10_9d4fdddd7ab05de9dfa4cc90f1051ef5._comment b/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_10_9d4fdddd7ab05de9dfa4cc90f1051ef5._comment
new file mode 100644
index 0000000..2bb9552
--- /dev/null
+++ b/doc/bugs/Packfile_does_not_match_digest__58___gcrypt_with_assistant/comment_10_9d4fdddd7ab05de9dfa4cc90f1051ef5._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 10"""
+ date="2016-09-21T21:36:18Z"
+content="""
+@Don and @pot, did you actually see un-encrypted data end up in the gcrypt
+repository? Or by "same issue" do you mean you saw the same error message,
+perhaps due to a very different cause?
+
+@jgoerzen, when you say that the gcrypt reposotory had an "unencrypted
+index", are you referring to a regular git index file, or to a pack's .idx
+file?
+
+It would be very strange if a gcrypt repository had an index file.
+I've never been able to use git-remote-gcrypt with a non-bare repository,
+and bare repositories don't normally have index files.
+"""]]

Added a comment: re: regular ssh remote
diff --git a/doc/forum/Possible_bug_in_setting_up_Rsync.net_remote/comment_6_b90f349f00ea283d018681505b4feaf5._comment b/doc/forum/Possible_bug_in_setting_up_Rsync.net_remote/comment_6_b90f349f00ea283d018681505b4feaf5._comment
new file mode 100644
index 0000000..1232f31
--- /dev/null
+++ b/doc/forum/Possible_bug_in_setting_up_Rsync.net_remote/comment_6_b90f349f00ea283d018681505b4feaf5._comment
@@ -0,0 +1,35 @@
+[[!comment format=mdwn
+ username="dave@2ab82f485adf7e2ce787066e35f5f9789bff430b"
+ nickname="dave"
+ subject="re: regular ssh remote"
+ date="2016-09-21T21:30:24Z"
+ content="""
+I tried adding my rsync.net space as a regular (ssh) remote instead of as an rsync.net remote.  I got this in the webapp:
+
+    Failed to ssh to the server. Transcript: 
+
+(sic)
+
+This time the debug log shows this:
+
+    [2016-09-21 16:19:15.0106417] read: ssh-keygen [\"-F\",\"usw-s009.rsync.net\"]
+    [2016-09-21 16:19:15.2376417] process done ExitSuccess
+    [2016-09-21 16:19:15.2376417] chat: ssh [
+      \"-oNumberOfPasswordPrompts=0\",
+      \"-oStrictHostKeyChecking=yes\",
+      \"-n\",
+      \"-p\",
+      \"22\",
+      \"9553@usw-s009.rsync.net\",
+      \"sh -c 'echo '\\"'\\"'git-annex-probe loggedin'\\"'\\"';if which git-annex-shell; then echo '\\"'\\"'git-annex-probe git-annex-shell'\\"'\\"'; fi;if which git; then echo '\\"'\\"'git-annex-probe git'\\"'\\"'; fi;if which rsync; then echo '\\"'\\"'git-annex-probe rsync'\\"'\\"'; fi;if which ~/.ssh/git-annex-shell; then echo '\\"'\\"'git-annex-probe ~/.ssh/git-annex-shell'\\"'\\"'; fi;if which ~/.ssh/git-annex-wrapper; then echo '\\"'\\"'git-annex-probe ~/.ssh/git-annex-wrapper'\\"'\\"'; fi;cd '\\"'\\"'annex'\\"'\\"' && git config --list'\"]
+    [2016-09-21 16:19:17.4001417] process done ExitFailure 1
+
+
+For what it is worth:
+
+    $ ssh -oNumberOfPasswordPrompts=0 -oStrictHostKeyChecking=yes -n -p 22 9553@usw-s009.rsync.net sh -c 'echo hello'
+
+THAT fails with exit code 1.  But that's just because it's rsync.net, a restricted shell...  (The same command with just 'ls' works!)
+
+I'm back to thinking that it's actually trying to resolve the mangled string.  However, I've been known to be completely wrong before!
+"""]]

addurl, importfeed: Improve behavior when file being added is gitignored.
diff --git a/CHANGELOG b/CHANGELOG
index 5c44f41..f07f6fd 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -32,6 +32,7 @@ git-annex (6.20160908) UNRELEASED; urgency=medium
   * Also, running sync in an adjusted branch did not correctly sync
     changes back to the parent branch when it had a slash in its name.
     This bug has been fixed.
+  * addurl, importfeed: Improve behavior when file being added is gitignored.
 
  -- Joey Hess <id@joeyh.name>  Thu, 08 Sep 2016 12:48:55 -0400
 
diff --git a/Command/AddUrl.hs b/Command/AddUrl.hs
index 3a6ee75..80f3582 100644
--- a/Command/AddUrl.hs
+++ b/Command/AddUrl.hs
@@ -19,6 +19,7 @@ import qualified Types.Remote as Remote
 import qualified Command.Add
 import Annex.Content
 import Annex.Ingest
+import Annex.CheckIgnore
 import Annex.UUID
 import Logs.Web
 import Types.KeySource
@@ -157,7 +158,7 @@ performRemote r relaxed uri file sz = ifAnnexed file adduri geturi
 	geturi = next $ isJust <$> downloadRemoteFile r relaxed uri file sz
 
 downloadRemoteFile :: Remote -> Bool -> URLString -> FilePath -> Maybe Integer -> Annex (Maybe Key)
-downloadRemoteFile r relaxed uri file sz = do
+downloadRemoteFile r relaxed uri file sz = checkCanAdd file $ do
 	let urlkey = Backend.URL.fromUrl uri sz
 	liftIO $ createDirectoryIfMissing True (parentDir file)
 	ifM (Annex.getState Annex.fast <||> pure relaxed)
@@ -236,7 +237,7 @@ performQuvi relaxed pageurl videourl file = ifAnnexed file addurl geturl
 	geturl = next $ isJust <$> addUrlFileQuvi relaxed quviurl videourl file
 
 addUrlFileQuvi :: Bool -> URLString -> URLString -> FilePath -> Annex (Maybe Key)
-addUrlFileQuvi relaxed quviurl videourl file = stopUnless (doesNotExist file) $ do
+addUrlFileQuvi relaxed quviurl videourl file = checkCanAdd file $ do
 	let key = Backend.URL.fromUrl quviurl Nothing
 	ifM (pure relaxed <||> Annex.getState Annex.fast)
 		( do
@@ -285,21 +286,13 @@ addUrlChecked relaxed url u checkexistssize key
 		)
 
 addUrlFile :: Bool -> URLString -> Url.UrlInfo -> FilePath -> Annex (Maybe Key)
-addUrlFile relaxed url urlinfo file = stopUnless (doesNotExist file) $ do
+addUrlFile relaxed url urlinfo file = checkCanAdd file $ do
 	liftIO $ createDirectoryIfMissing True (parentDir file)
 	ifM (Annex.getState Annex.fast <||> pure relaxed)
 		( nodownload url urlinfo file
 		, downloadWeb url urlinfo file
 		)
 
-doesNotExist :: FilePath -> Annex Bool
-doesNotExist file = go =<< liftIO (catchMaybeIO $ getSymbolicLinkStatus file)
-  where
-	go Nothing = return True
-	go (Just _) = do
-		warning $ file ++ " already exists and is not annexed; not overwriting"
-		return False
-
 downloadWeb :: URLString -> Url.UrlInfo -> FilePath -> Annex (Maybe Key)
 downloadWeb url urlinfo file = do
 	let dummykey = addSizeUrlKey urlinfo $ Backend.URL.fromUrl url Nothing
@@ -400,3 +393,16 @@ adjustFile o = addprefix . addsuffix
   where
 	addprefix f = maybe f (++ f) (prefixOption o)
 	addsuffix f = maybe f (f ++) (suffixOption o)
+
+checkCanAdd :: FilePath -> Annex (Maybe a) -> Annex (Maybe a)
+checkCanAdd file a = ifM (isJust <$> (liftIO $ catchMaybeIO $ getSymbolicLinkStatus file))
+	( do
+		warning $ file ++ " already exists and is not annexed; not overwriting"
+		return Nothing
+	, ifM ((not <$> Annex.getState Annex.force) <&&> checkIgnored file)
+		( do
+			warning $ "not adding " ++ file ++ " which is .gitignored (use --force to override)"
+			return Nothing
+		, a
+		)
+	)
diff --git a/doc/bugs/reports_success_when_addurl_--batch__a_file_which_is_.gitignore__39__d.mdwn b/doc/bugs/reports_success_when_addurl_--batch__a_file_which_is_.gitignore__39__d.mdwn
index d4cdaaf..1fbd3bd 100644
--- a/doc/bugs/reports_success_when_addurl_--batch__a_file_which_is_.gitignore__39__d.mdwn
+++ b/doc/bugs/reports_success_when_addurl_--batch__a_file_which_is_.gitignore__39__d.mdwn
@@ -29,3 +29,8 @@ git-annex: user error (xargs ["-0","git","--git-dir=.git","--work-tree=.","--lit
 """]]
 
 [[!meta author=yoh]]
+
+> And it leaves the unstaged symlink behind too.
+> 
+> [[fixed|done]] to check ignore status before creating the file.
+> --[[Joey]]

Added a comment: re: mangling is normal
diff --git a/doc/forum/Possible_bug_in_setting_up_Rsync.net_remote/comment_5_6dd6819103814056d8b71356804be56a._comment b/doc/forum/Possible_bug_in_setting_up_Rsync.net_remote/comment_5_6dd6819103814056d8b71356804be56a._comment
new file mode 100644
index 0000000..a27494c
--- /dev/null
+++ b/doc/forum/Possible_bug_in_setting_up_Rsync.net_remote/comment_5_6dd6819103814056d8b71356804be56a._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="dave@2ab82f485adf7e2ce787066e35f5f9789bff430b"
+ nickname="dave"
+ subject="re: mangling is normal"
+ date="2016-09-21T21:12:16Z"
+ content="""
+... ok, here's the original message from the (web) UI:
+
+    ssh: Could not resolve hostname git-annex-.usw.2ds009.2ersync.2enet-9553_22_annex: Name or service not known
+
+I had figured the debug log might be helpful... Pardon me... :)
+"""]]

comment
diff --git a/doc/forum/git_annex_file_content_replaced_with_symlink_content/comment_3_cc703ef5015527425572002295568923._comment b/doc/forum/git_annex_file_content_replaced_with_symlink_content/comment_3_cc703ef5015527425572002295568923._comment
new file mode 100644
index 0000000..43f5107
--- /dev/null
+++ b/doc/forum/git_annex_file_content_replaced_with_symlink_content/comment_3_cc703ef5015527425572002295568923._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2016-09-21T20:44:26Z"
+ content="""
+Gold star for @jimparis!
+
+Another way to get out of this situation would be to edit the files,
+and remove the initial lines so it contains a single line that is
+the link target of the symlink (the bit with .git/annex/objects in it).
+
+Then, `git annex fsck` would be able to fix up the files.
+"""]]

comment
diff --git a/doc/forum/__34__git_add__34___vs___34__git_annex_add__34___in_v6/comment_1_285032a01ca754c539ce0634823db23c._comment b/doc/forum/__34__git_add__34___vs___34__git_annex_add__34___in_v6/comment_1_285032a01ca754c539ce0634823db23c._comment
new file mode 100644
index 0000000..f27c060
--- /dev/null
+++ b/doc/forum/__34__git_add__34___vs___34__git_annex_add__34___in_v6/comment_1_285032a01ca754c539ce0634823db23c._comment
@@ -0,0 +1,29 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2016-09-21T20:25:10Z"
+ content="""
+This is a documented behavior change in v6 mode. I'm willing to listen to
+arguments that the behavior change is a bad idea, but see below.
+
+To get the old behavior, you can use:
+
+	git -c 'annex.largefiles=exclude=*' add
+
+Unfortunately git won't let you alias git add to always pass that
+switch. But you could alias `git sadd` or something to use that switch.
+
+----
+
+The same behavior change also makes `git commit -a` use git-annex when file
+contents have changed, rather than the old hacky method which added the
+files to git and then undid that in a pre-commit hook. So there's quite a
+nice benefit there.
+
+And for that reason it doesn't make sense to add a configuration option to
+disable it, because such an option would break `git commit -a` of
+modified annexed files.
+
+Changing the gitattributes won't work because then the v6 repository won't
+get annexed files checked out properly.
+"""]]

comment
diff --git a/doc/todo/Allow_globally_limiting_filename_length/comment_4_918d87f7a961bfeb834cc28f7175ccd8._comment b/doc/todo/Allow_globally_limiting_filename_length/comment_4_918d87f7a961bfeb834cc28f7175ccd8._comment
new file mode 100644
index 0000000..ec18558
--- /dev/null
+++ b/doc/todo/Allow_globally_limiting_filename_length/comment_4_918d87f7a961bfeb834cc28f7175ccd8._comment
@@ -0,0 +1,23 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2016-09-21T20:16:44Z"
+ content="""
+If an update hook rejects a push, then `git annex sync` will just note that
+it was unable to push. It will sync in only one direction until the problem
+that prevents pushing gets resolved.
+
+It might try pushing to a different branch name than usual to get around
+some other problems that cause pushes to fail so be sure to have the update
+hook check pushes to all branches (except for the git-annex branch)).
+
+I don't know why you'd want to filter such a commit out of the git history.
+You could just fix it by renaming the problem file and make a commit on top
+of the problem commit. Just make the update hook only look at the diff
+between the old version of the branch and the new version, so it won't be
+tripped up by intermediate commits that violate its rules.
+
+(I know that Ubuntu uses encfs or something like that by default, but
+surely they have not removed the Debian installer's support for full
+disk encryption?)
+"""]]

response
diff --git a/doc/forum/manage_repository_connection_between_external_HDD_and_server/comment_1_ba8ed548bf228a7d1c82b68283f99a5b._comment b/doc/forum/manage_repository_connection_between_external_HDD_and_server/comment_1_ba8ed548bf228a7d1c82b68283f99a5b._comment
new file mode 100644
index 0000000..bbb509f
--- /dev/null
+++ b/doc/forum/manage_repository_connection_between_external_HDD_and_server/comment_1_ba8ed548bf228a7d1c82b68283f99a5b._comment
@@ -0,0 +1,25 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2016-09-21T20:06:28Z"
+ content="""
+I'm not entirely sure I understand the question, but think you're
+probably trying to make nhost be able to access data0 independant
+of whether the external HDD is plugged into nhost or zhost
+(or perhaps qhost, whatever that is?)
+
+The way to do that is simply to set up one remote for each place
+the HDD can be plugged into. If the HDD can be plugged in locally,
+make a remote pointing at the local mount point. If the HDD can be
+plugged into zhost, make a remote pointing at
+zhost:/run/media/mee/data0/NIdata/ and so on.
+
+git-annex will figure out that these different remotes are pointing
+at the same repository. When running `git annex get` or `git annex sync
+--content`, it will check if the HDD is mounted locally,
+and will use it there if so; if not it will fall back to trying
+to access it via the remotes that use zhost etc.
+
+So in summary, git-annex is designed to just work in this kind of scenario.
+Set up lots of remotes and let it figure out which ones to use.
+"""]]

response
diff --git a/doc/forum/Use_case__58___Main_machine__44___portable_drive__44___backup_machine/comment_2_0b778710b23eab1413256e289f0aa180._comment b/doc/forum/Use_case__58___Main_machine__44___portable_drive__44___backup_machine/comment_2_0b778710b23eab1413256e289f0aa180._comment
new file mode 100644
index 0000000..9a4530b
--- /dev/null
+++ b/doc/forum/Use_case__58___Main_machine__44___portable_drive__44___backup_machine/comment_2_0b778710b23eab1413256e289f0aa180._comment
@@ -0,0 +1,32 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2016-09-21T19:59:25Z"
+ content="""
+Use git-annex repositories on both the laptop and the thumb drive.
+
+You can start by setting up the repository on the thumb drive,
+and then git clone it to the laptop.
+
+Use `git annex direct` to make them use direct mode, so that
+files can be edited.
+
+You can run the git-annex assistant in the repository on the laptop
+(~/annex is the usual place for such a repository). This way
+changes to files there will automatically be committed to git.
+
+There's not currently a good way to have the assistant also commit
+any files that were added to the thumb drive, or merge in changes
+from the laptop to the thumb drive. Running the assistant in the repository
+on the thumb drive would take care of both, but fails when the thumb
+drive gets unplugged.
+
+What you can do is write a simple shell script to sync up the thumb drive.
+Something like this:
+
+	#!/bin/sh
+	set -e
+	cd /media/thumbdrive
+	git annex add
+	git annex sync -m "syncing thumb drive"
+"""]]

comment
diff --git a/doc/forum/Remote_type_to_make_portable_HDD_like_normal_but_indexed/comment_2_36c202f5883b66a431ae5e72c5ecede1._comment b/doc/forum/Remote_type_to_make_portable_HDD_like_normal_but_indexed/comment_2_36c202f5883b66a431ae5e72c5ecede1._comment
new file mode 100644
index 0000000..53f94b6
--- /dev/null
+++ b/doc/forum/Remote_type_to_make_portable_HDD_like_normal_but_indexed/comment_2_36c202f5883b66a431ae5e72c5ecede1._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2016-09-21T19:54:43Z"
+ content="""
+Suggest you use git-remote-gcrypt to encrypt the git repository stored on
+the backup drive. This will encrypt the whole git repository with
+your gpg key, and git-annex can also use it as a remote, and will encrypt
+the annexed files stored there too.
+
+See [[tips/fully_encrypted_git_repositories_with_gcrypt]].
+"""]]

update
diff --git a/doc/bugs/indeterminite_preferred_content_state_for_duplicated_file.mdwn b/doc/bugs/indeterminite_preferred_content_state_for_duplicated_file.mdwn
index a297a7a..086c21e 100644
--- a/doc/bugs/indeterminite_preferred_content_state_for_duplicated_file.mdwn
+++ b/doc/bugs/indeterminite_preferred_content_state_for_duplicated_file.mdwn
@@ -16,3 +16,8 @@ a wrinkle to using it for this. --[[Joey]]
 
 (See [[bugs/preferred_content_and_deduplication]] for a user bug report of
 the same thing, with some suggestions for workarounds too.)
+
+In the preferred content expressions for standard groups, the only
+place this bug can be triggered involves archive directories of
+repositories in the client group. A file both in the archive directory and
+in another directory has indeterminite status.

merge
diff --git a/doc/bugs/indeterminite_preferred_content_state_for_duplicated_file.mdwn b/doc/bugs/indeterminite_preferred_content_state_for_duplicated_file.mdwn
index 4f516a6..a297a7a 100644
--- a/doc/bugs/indeterminite_preferred_content_state_for_duplicated_file.mdwn
+++ b/doc/bugs/indeterminite_preferred_content_state_for_duplicated_file.mdwn
@@ -13,3 +13,6 @@ Such a map exists, in the keys database, but only in v6 mode repositories.
 So, this seems solvable in v6 repositories, but not in v5.
 Also, the associated files map may not be accurate at all times, so that's
 a wrinkle to using it for this. --[[Joey]]
+
+(See [[bugs/preferred_content_and_deduplication]] for a user bug report of
+the same thing, with some suggestions for workarounds too.)
diff --git a/doc/bugs/preferred_content_and_deduplication.mdwn b/doc/bugs/preferred_content_and_deduplication.mdwn
index e9244c4..fbd4aad 100644
--- a/doc/bugs/preferred_content_and_deduplication.mdwn
+++ b/doc/bugs/preferred_content_and_deduplication.mdwn
@@ -96,3 +96,8 @@ Linux lenard-hp 4.6.0-1-amd64 #1 SMP Debian 4.6.4-1 (2016-07-18) x86_64 GNU/Linu
 
 I'm just trying out git-annex and I like it so far. I'm using it to keep my personal photos on my home server.
 
+> There's already a bug report open about this:
+> [[bugs/indeterminite_preferred_content_state_for_duplicated_file]].
+> 
+> Closing as a duplicate, but thanks for a well-researched bug report.
+> [[done]] --[[Joey]]

comment
diff --git a/doc/tips/git-annex_on_NFS/comment_2_06f2ee5095cd35063a434560375e172e._comment b/doc/tips/git-annex_on_NFS/comment_2_06f2ee5095cd35063a434560375e172e._comment
new file mode 100644
index 0000000..53525fa
--- /dev/null
+++ b/doc/tips/git-annex_on_NFS/comment_2_06f2ee5095cd35063a434560375e172e._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2016-09-21T19:28:36Z"
+ content="""
+git-annex will probe to detect if the filesystem does not support FIFOs and
+disables `annex.sshcaching` in that case. It's done so since 2013. So I would
+be surprised if NFS had any problems with annex.sshcaching.
+
+`git config annex.pidlock true` will make git-annex avoid FCNTL locking,
+and so work on filesystems that don't support that. It should also
+avoid the ".nfs" files.
+
+It's not enabled by default on NFS because I don't currently have a good
+way to probe if a given directory is on NFS.
+
+Also, annex.pidlock makes git-annex significantly slower and less safe.
+But if you're using NFS, speed and safety must have already been
+de-prioritized.
+
+Seriously, my main advice for using git-annex on NFS is: 
+Don't. Make local clones of repositories and use git-annex to distribute
+the files around. Unless your institution forces you to use a networked
+filesystem to access gobs of disk space, and you need to have more files
+present in a repository than will fit locally.
+"""]]

Added a comment
diff --git a/doc/todo/Allow_globally_limiting_filename_length/comment_3_748d5b01e502370ed98937d0077d1ec5._comment b/doc/todo/Allow_globally_limiting_filename_length/comment_3_748d5b01e502370ed98937d0077d1ec5._comment
new file mode 100644
index 0000000..61e98df
--- /dev/null
+++ b/doc/todo/Allow_globally_limiting_filename_length/comment_3_748d5b01e502370ed98937d0077d1ec5._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="interfect@b151490178830f44348aa57b77ad58c7d18e8fe7"
+ nickname="interfect"
+ subject="comment 3"
+ date="2016-09-21T19:32:06Z"
+ content="""
+Also, I think Ubuntu is \"ecryptfs\" and not \"encfs\" anyway.
+"""]]

Added a comment: Pre Commit Hook
diff --git a/doc/todo/Allow_globally_limiting_filename_length/comment_2_cdb91ea855d9bcdeccf19ccb6d55ed10._comment b/doc/todo/Allow_globally_limiting_filename_length/comment_2_cdb91ea855d9bcdeccf19ccb6d55ed10._comment
new file mode 100644
index 0000000..dc36053
--- /dev/null
+++ b/doc/todo/Allow_globally_limiting_filename_length/comment_2_cdb91ea855d9bcdeccf19ccb6d55ed10._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="interfect@b151490178830f44348aa57b77ad58c7d18e8fe7"
+ nickname="interfect"
+ subject="Pre Commit Hook"
+ date="2016-09-21T19:20:04Z"
+ content="""
+I'm basically stuck with whatever home directory encryption Canonical deigns to give me in their setup wizard, given my time and attention budget. I've looked a bit at the security problems with it and they mostly seem to be that it's a bit leaky due to not hiding structures and sizes. Hiding contents is better than not hiding contents, so that's what I've got.
+
+Anyway, a pre-commit hook, or maybe an update hook, would be a great solution. I'd like one to be on the wiki somewhere as a useful tip for actually using git annex effectively across a bunch of non-ideal environments. It would be great if a \"git annex init\" could set it up for me, too.
+
+Any ideas for writing a pre-commit script that works on Linux, Mac, Windows, Android, and whatever weird embedded NAS things people might want to use it on? If I went with an update script over a pre-commit, that would make platform support less of a problem, but then you'd get Git Annex into weird situations when syncing.
+
+How would Git Annex react if I made a commit on one system, but then my central syncing repo's update script rejected the commit for breaking the rules on file names? If I have a commit that isn't allowed to be pushed to a particular remote, how would I use git annex to get it out of the history of any repos it might have already gotten to?
+"""]]

fix bugs in handing of deep branches with sync and adjusted branches
* sync: Previously, when run in a branch with a slash in its name,
such as "foo/bar", the sync branch was "synced/bar". That conflicted
with the sync branch used for branch "bar", so has been changed to
"synced/foo/bar".
* adjust: Previously, when adjusting a branch with a slash in its name,
such as "foo/bar", the adjusted branch was "adjusted/bar(unlocked)".
That conflicted with the adjusted branch used for branch "bar",
so has been changed to "adjusted/foo/bar(unlocked)"
* Also, running sync in an adjusted branch did not correctly sync
changes back to the parent branch when it had a slash in its name.
This bug has been fixed.
Eliminate use of Git.Ref.under and Git.Ref.basename; using
Git.Ref.underBase and Git.Ref.base make everything handle deep branches
correctly.
Probably noone was adjusting deep branches, and v6 is still experimental
anyway, so I'm not going to worry about the mess that was left by that bug.
In the case of git-annex sync, using a fixed git-annex with an old unfixed
one will mean they use different sync branches for a deep branch, and so
they may stop syncing until the old one is upgraded. However, that's only
a problem when syncing between repositories without going via a central
bare repository. Added a warning about this to the CHANGELOG, but it's
probably not going to affect many people at all.
This commit was sponsored by Riku Voipio.
diff --git a/Annex/AdjustedBranch.hs b/Annex/AdjustedBranch.hs
index ae5ad9a..4caf637 100644
--- a/Annex/AdjustedBranch.hs
+++ b/Annex/AdjustedBranch.hs
@@ -159,14 +159,14 @@ originalToAdjusted :: OrigBranch -> Adjustment -> AdjBranch
 originalToAdjusted orig adj = AdjBranch $ Ref $
 	adjustedBranchPrefix ++ base ++ '(' : serialize adj ++ ")"
   where
-	base = fromRef (Git.Ref.basename orig)
+	base = fromRef (Git.Ref.base orig)
 
 adjustedToOriginal :: Branch -> Maybe (Adjustment, OrigBranch)
 adjustedToOriginal b
 	| adjustedBranchPrefix `isPrefixOf` bs = do
 		let (base, as) = separate (== '(') (drop prefixlen bs)
 		adj <- deserialize (takeWhile (/= ')') as)
-		Just (adj, Git.Ref.under "refs/heads" (Ref base))
+		Just (adj, Git.Ref.underBase "refs/heads" (Ref base))
 	| otherwise = Nothing
   where
 	bs = fromRef b
diff --git a/CHANGELOG b/CHANGELOG
index 4f7674a..5c44f41 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -15,6 +15,23 @@ git-annex (6.20160908) UNRELEASED; urgency=medium
   * stack.yaml: Update to lts-7.0 (ghc 8)
   * info: Support being passed a treeish, and show info about the annexed
     files in it similar to how a directory is handled.
+  * sync: Previously, when run in a branch with a slash in its name,
+    such as "foo/bar", the sync branch was "synced/bar". That conflicted
+    with the sync branch used for branch "bar", so has been changed to
+    "synced/foo/bar".
+  * Note that if you're using an old version of git-annex to sync with
+    a branch with a slash in its name, it won't see some changes synced by
+    this version, and this version won't see some changes synced by the older
+    version. This is not a problem if there's a central bare repository,
+    but may impact other configurations until git-annex is upgraded to this
+    version.
+  * adjust: Previously, when adjusting a branch with a slash in its name,
+    such as "foo/bar", the adjusted branch was "adjusted/bar(unlocked)".
+    That conflicted with the adjusted branch used for branch "bar",
+    so has been changed to "adjusted/foo/bar(unlocked)"
+  * Also, running sync in an adjusted branch did not correctly sync
+    changes back to the parent branch when it had a slash in its name.
+    This bug has been fixed.
 
  -- Joey Hess <id@joeyh.name>  Thu, 08 Sep 2016 12:48:55 -0400
 
diff --git a/Command/Sync.hs b/Command/Sync.hs
index fd9d0b2..d7edac7 100644
--- a/Command/Sync.hs
+++ b/Command/Sync.hs
@@ -178,7 +178,7 @@ merge (b, _) mergeconfig commitmode tomerge =
 	autoMergeFrom tomerge b mergeconfig commitmode
 
 syncBranch :: Git.Branch -> Git.Branch
-syncBranch = Git.Ref.under "refs/heads/synced" . fromDirectBranch . fromAdjustedBranch
+syncBranch = Git.Ref.underBase "refs/heads/synced" . fromDirectBranch . fromAdjustedBranch
 
 remoteBranch :: Remote -> Git.Ref -> Git.Ref
 remoteBranch remote = Git.Ref.underBase $ "refs/remotes/" ++ Remote.name remote
diff --git a/Git/Ref.hs b/Git/Ref.hs
index 257c430..5b3b853 100644
--- a/Git/Ref.hs
+++ b/Git/Ref.hs
@@ -39,15 +39,6 @@ base = Ref . remove "refs/heads/" . remove "refs/remotes/" . fromRef
 		| prefix `isPrefixOf` s = drop (length prefix) s
 		| otherwise = s
 
-{- Gets the basename of any qualified ref. -}
-basename :: Ref -> Ref
-basename = Ref . reverse . takeWhile (/= '/') . reverse . fromRef
-
-{- Given a directory and any ref, takes the basename of the ref and puts
- - it under the directory. -}
-under :: String -> Ref -> Ref
-under dir r = Ref $ dir ++ "/" ++ fromRef (basename r)
-
 {- Given a directory such as "refs/remotes/origin", and a ref such as
  - refs/heads/master, yields a version of that ref under the directory,
  - such as refs/remotes/origin/master. -}
diff --git a/doc/bugs/sync_uses_conflicting_names_for_deep_branches.mdwn b/doc/bugs/sync_uses_conflicting_names_for_deep_branches.mdwn
index fa6dd0d..439bd55 100644
--- a/doc/bugs/sync_uses_conflicting_names_for_deep_branches.mdwn
+++ b/doc/bugs/sync_uses_conflicting_names_for_deep_branches.mdwn
@@ -3,4 +3,7 @@ changes to branch named "foo", but that same name is used to sync
 changes to a branch named "bar/foo".
 
 Also, the adjusted branch code uses "adjusted/foo(unlocked)" for
-both "foo" and "bar/foo".
+both "foo" and "bar/foo". And it fails to push changes back from there to
+"bar/foo", instead creating a "foo" branch.
+
+> [[fixed|done]] --[[Joey]]

bug
diff --git a/doc/bugs/sync_uses_conflicting_names_for_deep_branches.mdwn b/doc/bugs/sync_uses_conflicting_names_for_deep_branches.mdwn
new file mode 100644
index 0000000..fa6dd0d
--- /dev/null
+++ b/doc/bugs/sync_uses_conflicting_names_for_deep_branches.mdwn
@@ -0,0 +1,6 @@
+`git annex sync` uses a branch named "synced/foo" to sync
+changes to branch named "foo", but that same name is used to sync
+changes to a branch named "bar/foo".
+
+Also, the adjusted branch code uses "adjusted/foo(unlocked)" for
+both "foo" and "bar/foo".
diff --git a/doc/sync/comment_21_9c0368eb796f1191c22c186cbb06c642._comment b/doc/sync/comment_21_9c0368eb796f1191c22c186cbb06c642._comment
new file mode 100644
index 0000000..bd40a8a
--- /dev/null
+++ b/doc/sync/comment_21_9c0368eb796f1191c22c186cbb06c642._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""Re: Branch names containing slashes"""
+ date="2016-09-21T18:56:22Z"
+ content="""
+@kartynnik, that's a bug: [[bugs/sync_uses_conflicting_names_for_deep_branches]]
+
+Please file bugs there and not as comments here, it's too easy to lose
+track of a comment deep in a thread.
+"""]]

comments
diff --git a/doc/todo/transitive_transfers/comment_1_d244295696be5a8164db45f7fad1701a._comment b/doc/todo/transitive_transfers/comment_1_d244295696be5a8164db45f7fad1701a._comment
new file mode 100644
index 0000000..48995dc
--- /dev/null
+++ b/doc/todo/transitive_transfers/comment_1_d244295696be5a8164db45f7fad1701a._comment
@@ -0,0 +1,47 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2016-09-21T17:53:12Z"
+ content="""
+(Let's not discuss the behavior of copy --to when the file is not locally
+present here; there is plenty of other discussion of that in eg
+<http://bugs.debian.org/671179>)
+
+git-annex's special remote API does not allow remote-to-remote transfers
+without spooling it to a file on disk first. And it's not possible to do
+using rsync on either end, AFAICS. It would be possible in some other cases
+but this would need to be implemented for each type of remote as a new API
+call.
+
+Modern systems tend to have quite a large disk cache, so it's quite
+possible that going via a temp file on disk is not going to use a lot of
+disk IO to write and read it when the read and write occur fairly close
+together.
+
+The main benefit from streaming would probably be if it could run the
+download and the upload concurrently. But that would only be a benefit
+sometimes. With an asymmetric connection, saturating the uplink tends to
+swamp downloads. Also, if download is faster than upload, it would have to
+throttle downloads (which complicates the remote API much more), or buffer
+them to memory (which has its own complications).
+
+Streaming the download to the upload would at best speed things up by a
+factor of 2. It would probably work nearly as well to upload the previously
+downloaded file while downloading the next file.
+
+It would not be super hard to make `git annex copy --from --to` download a file,
+upload it, and then drop it, and parallelizing it with -J2 would
+keep both the --from and --to remotes bandwidth saturated pretty well.
+
+Alhough not perfectly, because two jobs could both be downloading while
+the uplink is left idle. To make it optimal, it would need to do the
+download and when done, push the upload+drop into another queue of actions
+that is processed concurrently with other downloads.
+
+And there is a complication with running that at the same time as eg `git
+annex get` of the same file. It would be surprising for get to succeed
+(because copy has already temporarily downloaded the file) and then have
+the file later get dropped. So, it seems that `copy --from --to` would need
+to stash the content away in a temp file somewhere instead of storing it
+in the annex proper.
+"""]]

comment, retitle
diff --git a/doc/bugs/autoenable__61__true_seems_to_not_work_any_longer.mdwn b/doc/bugs/autoenable__61__true_seems_to_not_work_any_longer.mdwn
index 3f6a602..86ecda2 100644
--- a/doc/bugs/autoenable__61__true_seems_to_not_work_any_longer.mdwn
+++ b/doc/bugs/autoenable__61__true_seems_to_not_work_any_longer.mdwn
@@ -75,3 +75,5 @@ I am a little confused though since we do test for this scenario in datalad and
 
 
 [[!meta author=yoh]]
+
+[[!meta title="autoenable not done for implicit init"]]
diff --git a/doc/bugs/autoenable__61__true_seems_to_not_work_any_longer/comment_1_457027471d09aa4ae8718c1508cfae1d._comment b/doc/bugs/autoenable__61__true_seems_to_not_work_any_longer/comment_1_457027471d09aa4ae8718c1508cfae1d._comment
new file mode 100644
index 0000000..0c8585d
--- /dev/null
+++ b/doc/bugs/autoenable__61__true_seems_to_not_work_any_longer/comment_1_457027471d09aa4ae8718c1508cfae1d._comment
@@ -0,0 +1,27 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2016-09-21T17:37:39Z"
+ content="""
+`git annex init` does handle autoenable. When you bypass explicit init,
+it does not do autoenabling.
+
+This is not a change AFAICS. The changelog entry for autoenable
+says that it's done by `git annex init`. Presumably your test suite
+does run `git annex init`.
+
+My original notes on why not to have automaitic init handle autoenable were:
+
+> There was also the question of what to do when git-annex auto-inits
+> in a clone of a repository. It wouldn't do for a command like
+> `git annex find`'s output to include any messages that might be shown
+> while auto-enabling special remotes as a result of an auto-init.
+> Since I can't guarantee enabling special remotes will be quiet, I've not
+> tried to auto-enable special remotes in this case.
+> 
+> I think I'd have to
+> exec a git-annex init process with stdout sent to stderr to implement
+> this in a safe way, and due to calls to ensureInitialized in Remote.Git,
+> which can auto-init a local remote, that gets particularly tricky. Best,
+> I feel, to wait and see if anyone needs that.
+"""]]

comment and close
diff --git a/doc/bugs/Pass_--verbose_option_to___39__git_fsck__39___from___39__git-annex_repair_--verbose__39__.mdwn b/doc/bugs/Pass_--verbose_option_to___39__git_fsck__39___from___39__git-annex_repair_--verbose__39__.mdwn
index 9ba8b38..7d0e7ba 100644
--- a/doc/bugs/Pass_--verbose_option_to___39__git_fsck__39___from___39__git-annex_repair_--verbose__39__.mdwn
+++ b/doc/bugs/Pass_--verbose_option_to___39__git_fsck__39___from___39__git-annex_repair_--verbose__39__.mdwn
@@ -10,4 +10,4 @@ Just run 'git-annex repair --verbose'
 
 not important, this is just a suggestion
 
-
+> [[done]] see comment --[[Joey]]
diff --git a/doc/bugs/Pass_--verbose_option_to___39__git_fsck__39___from___39__git-annex_repair_--verbose__39__/comment_1_04c49dd8c74c717e9401b0f11dec3e3c._comment b/doc/bugs/Pass_--verbose_option_to___39__git_fsck__39___from___39__git-annex_repair_--verbose__39__/comment_1_04c49dd8c74c717e9401b0f11dec3e3c._comment
new file mode 100644
index 0000000..84ccf28
--- /dev/null
+++ b/doc/bugs/Pass_--verbose_option_to___39__git_fsck__39___from___39__git-annex_repair_--verbose__39__/comment_1_04c49dd8c74c717e9401b0f11dec3e3c._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2016-09-21T17:23:18Z"
+ content="""
+git-annex is parsing the output of git fsck to see what sha1s it outputs,
+as those may have problems, --verbose would confuse that since it causes
+many more sha1s to be output to stdout.
+
+Passing --progress enables a progress display, but it is
+displayed on stderr, which is the same place git fsck displays
+some errors which git-annex also parses. Any attempt to untangle the
+progress and the error messages could be broken by changes to the git fsck
+output, so would not be a good idea.
+
+So while this is a good idea, it doesn't work out.
+"""]]

response
diff --git a/doc/install/fromsource/comment_62_4cf71016c3b42ec972760a938031cbf9._comment b/doc/install/fromsource/comment_62_4cf71016c3b42ec972760a938031cbf9._comment
new file mode 100644
index 0000000..05d1532
--- /dev/null
+++ b/doc/install/fromsource/comment_62_4cf71016c3b42ec972760a938031cbf9._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 62"""
+ date="2016-09-21T17:21:58Z"
+ content="""
+In the meantime, the stack.yaml has been updated to use lts-7.0, which
+includes concurrent-output-1.7.7, so that should solve that.
+"""]]

comment
diff --git a/doc/forum/Possible_bug_in_setting_up_Rsync.net_remote/comment_4_43f04bfbaeb4f04a5980ade59a36ae91._comment b/doc/forum/Possible_bug_in_setting_up_Rsync.net_remote/comment_4_43f04bfbaeb4f04a5980ade59a36ae91._comment
new file mode 100644
index 0000000..6b317e8
--- /dev/null
+++ b/doc/forum/Possible_bug_in_setting_up_Rsync.net_remote/comment_4_43f04bfbaeb4f04a5980ade59a36ae91._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2016-09-21T17:18:35Z"
+ content="""
+The error message "Permission denied (publickey,password,keyboard-interactive)"
+seems to be saying you entered the wrong password for rsync.net.
+"""]]

comment
diff --git a/doc/todo/Allow_globally_limiting_filename_length/comment_1_e38f78a1263b15258d2aa4472e7bc31b._comment b/doc/todo/Allow_globally_limiting_filename_length/comment_1_e38f78a1263b15258d2aa4472e7bc31b._comment
new file mode 100644
index 0000000..0d718bf
--- /dev/null
+++ b/doc/todo/Allow_globally_limiting_filename_length/comment_1_e38f78a1263b15258d2aa4472e7bc31b._comment
@@ -0,0 +1,23 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2016-09-21T17:11:06Z"
+ content="""
+Standard encfs warning: It's buggy and insecure. Don't use it.
+
+You can find many other problems caused by encfs on this site, and 
+<https://defuse.ca/audits/encfs.htm> has described security problems with
+encfs for years.
+
+It would not help for `git-annex add` to check some kind of filename limit,
+because it would not prevent you doing this:
+
+	git annex add smallenough
+	git mv smallenough oh-oops-my-name-is-too-long-for-encfs
+	git commit -m haha
+
+A git pre-commit hook can of course be written that blocks such commits.
+
+I am not inclined to complicate git-annex just to handle encfs given how
+broken encfs is.
+"""]]

lacking problem description
diff --git a/doc/forum/Possible_bug_in_setting_up_Rsync.net_remote/comment_3_753ff00c43be5f9bef6bcb529ad68bb9._comment b/doc/forum/Possible_bug_in_setting_up_Rsync.net_remote/comment_3_753ff00c43be5f9bef6bcb529ad68bb9._comment
new file mode 100644
index 0000000..ddb6db2
--- /dev/null
+++ b/doc/forum/Possible_bug_in_setting_up_Rsync.net_remote/comment_3_753ff00c43be5f9bef6bcb529ad68bb9._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2016-09-21T17:08:41Z"
+ content="""
+That mangling of the hostname is completely normal.
+
+Your problem description lacked a description of an actual problem.
+Is something not working, or were you put off by this ugly looking
+hostname? If something is not working, what is the error message?
+"""]]

close
diff --git a/doc/bugs/build_fails_on_debian_jessie.mdwn b/doc/bugs/build_fails_on_debian_jessie.mdwn
index 5dafa26..5727a84 100644
--- a/doc/bugs/build_fails_on_debian_jessie.mdwn
+++ b/doc/bugs/build_fails_on_debian_jessie.mdwn
@@ -5,3 +5,5 @@ Can't build on debian jessie using "make" - "apt-get build-dep git-annex" misses
 ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
 
 The stack build works very nicely, thanks.
+
+> The `debian-jessie-backport` branch should build. [[done]] --[[Joey]]

comment
diff --git a/doc/bugs/build_fails_on_debian_jessie/comment_5_c78b52b19ba1ab0f75ec0207539c652f._comment b/doc/bugs/build_fails_on_debian_jessie/comment_5_c78b52b19ba1ab0f75ec0207539c652f._comment
new file mode 100644
index 0000000..8bbdda0
--- /dev/null
+++ b/doc/bugs/build_fails_on_debian_jessie/comment_5_c78b52b19ba1ab0f75ec0207539c652f._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 5"""
+ date="2016-09-21T17:04:40Z"
+ content="""
+RichiH already has a debian-jessie-backport branch in the git-annex git
+repo, so I added a commit onto it that fixes up the shakespeare dependency.
+
+I don't know what the holdup is on getting that backport actually uploaded
+to Debian.
+"""]]

comment
diff --git a/doc/forum/partial_annex_repository/comment_2_e6a9584bacc3ec98c4757c82a195b414._comment b/doc/forum/partial_annex_repository/comment_2_e6a9584bacc3ec98c4757c82a195b414._comment
new file mode 100644
index 0000000..a7c0d1e
--- /dev/null
+++ b/doc/forum/partial_annex_repository/comment_2_e6a9584bacc3ec98c4757c82a195b414._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2016-09-21T16:54:24Z"
+ content="""
+Right, it's a feature of v6 mode that `git add` adds the file to the annex,
+unless annex.largefiles is configured to make it be added directly to the
+git repository.
+
+To switch your small files that are currently annexed to be added
+directly to git, the easiest way is:
+
+	git annex unannex $file
+	git -c 'annex.largefiles=exclude=*' add $file
+"""]]

comment
diff --git a/doc/todo/Workflow_guide/comment_1_f4242a3bdeffb90336f562f9ec182e5d._comment b/doc/todo/Workflow_guide/comment_1_f4242a3bdeffb90336f562f9ec182e5d._comment
new file mode 100644
index 0000000..40b109c
--- /dev/null
+++ b/doc/todo/Workflow_guide/comment_1_f4242a3bdeffb90336f562f9ec182e5d._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2016-09-21T16:52:10Z"
+ content="""
+I think this is a good idea for an extension to the walkthrough. I
+am probably the worst person to write it. How about, you write it, and I'll
+fact check and edit it.
+"""]]

comment
diff --git a/doc/forum/Hashes_instead_of_content_in_files/comment_4_67733580e8f8806b5036406e6712b6a1._comment b/doc/forum/Hashes_instead_of_content_in_files/comment_4_67733580e8f8806b5036406e6712b6a1._comment
new file mode 100644
index 0000000..72b0cab
--- /dev/null
+++ b/doc/forum/Hashes_instead_of_content_in_files/comment_4_67733580e8f8806b5036406e6712b6a1._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2016-09-21T16:51:02Z"
+ content="""
+Like I said, use `git annex get`. This will populate the unlocked files
+with the content instead of the hashes.
+"""]]

answer
diff --git a/doc/forum/Can_a_repo_be_marked_as___34__must_have_file__63____34__/comment_2_409d6a97f5c3d1f1fc5e932a2965e227._comment b/doc/forum/Can_a_repo_be_marked_as___34__must_have_file__63____34__/comment_2_409d6a97f5c3d1f1fc5e932a2965e227._comment
new file mode 100644
index 0000000..65ff9ed
--- /dev/null
+++ b/doc/forum/Can_a_repo_be_marked_as___34__must_have_file__63____34__/comment_2_409d6a97f5c3d1f1fc5e932a2965e227._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2016-09-21T16:46:16Z"
+ content="""
+Cost settings have nothing to do with this.
+
+[[Preferred_content]] can be configured to include specific files
+that you want to have in the repository, by matching on the filename.
+For example:
+
+	git annex wanted . "include=somefile or include=otherfile or include=*.mp3"
+
+Putting the repository in the backup group is similar to setting
+"include=*"
+"""]]

comment
diff --git a/doc/forum/Recover_repository_from_bup_repository/comment_1_6e3b1ff58fbdc10bcaca6d660eb1156f._comment b/doc/forum/Recover_repository_from_bup_repository/comment_1_6e3b1ff58fbdc10bcaca6d660eb1156f._comment
new file mode 100644
index 0000000..9b95472
--- /dev/null
+++ b/doc/forum/Recover_repository_from_bup_repository/comment_1_6e3b1ff58fbdc10bcaca6d660eb1156f._comment
@@ -0,0 +1,35 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2016-09-21T15:59:55Z"
+ content="""
+If your bup repository is a git-annex special remote, it contains only 
+the contents of annexed files, but not the rest of the git repository data
+(no filenames, no pointers to git-annex keys, etc).
+
+Also, the files in it may be encrypted in a variety of ways depending on
+how you set up that git-annex special remote. If encryption was used, most
+likely it needs information that was stored in the git repository to
+decrypt it.
+
+So, your best bet is to restore your git-annex repository from a backup or
+a clone of that repository. Then you can just `git annex enableremote` the
+bup special remote and use `git-annex get` as usual. If you didn't have a
+backup or a clone of the git-annex repository, then important 
+information is lost.
+
+Without the git-annex repository, you can manually get the contents of the
+files from the bup repository. In the bup repository, run `git branch`;
+this will print out the names of all the git-annex keys that were stored
+there. Then you can use `bup join` to extract the content of each key:
+
+	bup join -r /path/to/bup/repository $key > $key.content
+
+Without the git-annex repository, there's no record of what the filename
+was, but you can extract the content this way.
+
+But, if the key names start with "GPG" the the data is stored in bup
+encrypted and you are probably out of luck (although if you used
+encryption=pubkey when setting up the bup special remote, you
+can use gpg to decrypt the files after `bup join`).
+"""]]

update
diff --git a/doc/thanks.mdwn b/doc/thanks.mdwn
index 2d11ea9..490fcf1 100644
--- a/doc/thanks.mdwn
+++ b/doc/thanks.mdwn
@@ -18,7 +18,15 @@ git-annex development is partially supported by the
 [DataLad project](http://datalad.org/).
 
 Thanks also to these folks for their support: martin f. krafft, John Byrnes,
-Aurélien Couderc, and Jeffrey Chiu.
+Aurélien Couderc, Jeffrey Chiu, Amitai Schlair, Andreas, Anthony DeRobertis,
+Baldur Kristinsson, Boyd Stephen Smith Jr., Brock Spratlen, Christian Diller,
+Denis Dzyubenko, Eskild Hustvedt, Evgeni Kunev, FBC, Fernando Jimenez, Greg
+Young, Henrik RIomar, Ignacio, Jake Vosloo, James Valleroy, Jason Woofenden,
+Jeff Goeke-Smith, Jim, Jochen Bartl, Johannes Schlatow, John Carr, Josh
+Taylor, Josh Triplett, Kuno Woudt, Matthias Urlichs, Mattias J, Nathan Howell,
+Nick Daly, Nicolas Schodet, Ole-Morten Duesund, Remy van Elst, RémiV, Thom
+May, Thomas Ferris Nicolaisen, Thomas Hochstein, Tyler Cipriani, encryptio,
+Øyvind A. Holm
 
 ## 2013-2014
 

diff --git a/doc/forum/Recover_repository_from_bup_repository.mdwn b/doc/forum/Recover_repository_from_bup_repository.mdwn
new file mode 100644
index 0000000..473bfb6
--- /dev/null
+++ b/doc/forum/Recover_repository_from_bup_repository.mdwn
@@ -0,0 +1 @@
+I screwed up one of my repositories, so I decided to start from clean slate. I have a remote bup repository that has all of my previous repository data. Now, I want to get it back. How I can do this?

Added a comment
diff --git a/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_4_614da8cad46df5c71539cc5a8ed11831._comment b/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_4_614da8cad46df5c71539cc5a8ed11831._comment
new file mode 100644
index 0000000..6481abb
--- /dev/null
+++ b/doc/bugs/git-annex_won__39__t_execute_on_WD_My_Cloud_NAS/comment_4_614da8cad46df5c71539cc5a8ed11831._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="PaulK"
+ subject="comment 4"
+ date="2016-09-19T04:39:07Z"
+ content="""
+So, does anyone out there have access to this gold linker, and could they please try to link it for the armv7l architecture?
+Thanks.
+"""]]

Added a comment: Groups...
diff --git a/doc/forum/Can_a_repo_be_marked_as___34__must_have_file__63____34__/comment_1_14d4d3f84eacc36dcc55af783d189619._comment b/doc/forum/Can_a_repo_be_marked_as___34__must_have_file__63____34__/comment_1_14d4d3f84eacc36dcc55af783d189619._comment
new file mode 100644
index 0000000..adf1d1c
--- /dev/null
+++ b/doc/forum/Can_a_repo_be_marked_as___34__must_have_file__63____34__/comment_1_14d4d3f84eacc36dcc55af783d189619._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="kevin@f9ce8e46ad8c29b25093b82cf68fae38f54b89a1"
+ nickname="kevin"
+ subject="Groups..."
+ date="2016-09-18T20:35:00Z"
+ content="""
+The answer seems to be [groups](https://git-annex.branchable.com/preferred_content/standard_groups/).
+
+If I set the two repos I want to have content to be in the backup group, they'll always try to get every file the see.
+"""]]

diff --git a/doc/forum/Can_a_repo_be_marked_as___34__must_have_file__63____34__.mdwn b/doc/forum/Can_a_repo_be_marked_as___34__must_have_file__63____34__.mdwn
new file mode 100644
index 0000000..7eb024a
--- /dev/null
+++ b/doc/forum/Can_a_repo_be_marked_as___34__must_have_file__63____34__.mdwn
@@ -0,0 +1,5 @@
+Is there a way to mark a repo as needing to have a copy of a file?
+
+I know there's the `remote.<name>.annex-cost` setting but that's done just on a single copy of the repo.
+
+Obviously the problem here would be determining the path to get a file to a repo, so maybe this isn't possible.

promote the neurodebian backport since the official backport looks like it will never come
diff --git a/doc/install/Debian.mdwn b/doc/install/Debian.mdwn
index 55bbe3e..736ac7d 100644
--- a/doc/install/Debian.mdwn
+++ b/doc/install/Debian.mdwn
@@ -4,6 +4,14 @@ Debian unstable and testing are usually fairly up to date, so this should be eno
 
 	sudo apt install git-annex
 
+## Standalone backports for all releases
+
+If the version shipped with Debian is too old, 
+the [NeuroDebian team](http://neuro.debian.net/) provides a
+[standalone build package](http://neuro.debian.net/pkgs/git-annex-standalone.html)
+that is regularly updated and that should work across all releases of
+Debian.
+
 ## Debian 8.0 "jessie"
 
 	sudo apt install git-annex
@@ -30,11 +38,3 @@ Follow the instructions to [enable backports](http://backports.debian.org/Instru
 Follow the instructions to [enable backports](http://backports.debian.org/Instructions/).
 
 	sudo apt-get -t squeeze-backports install git-annex
-
-## Standalone backports for all releases
-
-If the version shipped with Debian is too old, 
-the [NeuroDebian team](http://neuro.debian.net/) provides a
-[standalone build package](http://neuro.debian.net/pkgs/git-annex-standalone.html)
-that is regularly updated and that should work across all releases of
-Debian.

diff --git a/doc/bugs/sync_deletes_files.mdwn b/doc/bugs/sync_deletes_files.mdwn
new file mode 100644
index 0000000..da3afad
--- /dev/null
+++ b/doc/bugs/sync_deletes_files.mdwn
@@ -0,0 +1,44 @@
+### Please describe the problem.
+
+Syncing two repositories causes many files that are on both systems to be deleted.
+
+My computers A and B have a directory with 1600 files taking 500MB.   I'm trying to sync A to B thus:
+
+git annex sync --no-commit --no-push
+
+This immediately deletes those 1600 files from B.   (Before I added -no-push it also deleted those files from A).
+
+Recovering from this with git reset --hard takes 13 minutes, so it's difficult to run experiments.   However I have spent about a full day on this over the last week.
+
+### What steps will reproduce the problem?
+
+This repository has evolved over time so I can't give a MWE.   However, I created the git repository before initing annex, and did some more work before adding the largefiles option.   It's probable that something inside annex is now badly confused.
+
+I don't expect you to solve my particular problem from this report.   However, sync has a major hidden problem. 
+
+### What version of git-annex are you using? On what operating system?
+
+20160511-1.
+the annex is v6.
+the OS is Ubuntu linux  4.4.0-38-generic 
+
+### Please provide any additional information below.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+
+# End of transcript or log.
+"""]]
+
+.git/annex has no log files, even in subdirs.
+
+There have been occasional reports going back years from other people reporting that sync destroyed massive amounts of their repositories.    That this bug persists suggests that the problem is subtle.
+
+I upgraded my annexes to v6 because the concept of locked and unlocked looks good.   Then, when researching my problem before posting this bug report, I found a comment that upgrading to v6 is not (now?) recommended because of possible bugs.   It would have been nice if this warning were more prominent.
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+The idea is great.  Preliminary tests were quite positive.   That's why I tried it on my big repositories.
+

I got the files to come back, but I don't want Android making these dumb delete-loads-of-newly-added-stuff commits
diff --git a/doc/bugs/Android_client_deletes_everything.mdwn b/doc/bugs/Android_client_deletes_everything.mdwn
index f4a3ef6..677bdaf 100644
--- a/doc/bugs/Android_client_deletes_everything.mdwn
+++ b/doc/bugs/Android_client_deletes_everything.mdwn
@@ -4,11 +4,11 @@ After not syncing my Android repo for a while, I tried to sync it. By some combi
 
 I then synced in my main direct mode crippled filesystem repo with the only copy of some files, and got a bunch of messages that Git Annex was deleting files I wanted to keep. I killed that sync with a ctrl+c.
 
-My problem is: how do I revert the offending commit and restore my files?
+My problem (***UPDATE**: solved) is: how do I revert the offending commit and restore my files?
 
-My other question is: how do I prevent this happening again? Is there a way I can pre-clear commits and not accept those that delete files without manual confirmation? Or should I just stop being mean to the Android client and hope it doesn't decide to delete things it shouldn't delete again?
+My other question (not yet solved) is: how do I prevent this happening again? Is there a way I can pre-clear commits and not accept those that delete files without manual confirmation? Or should I just stop being mean to the Android client and hope it doesn't decide to delete things it shouldn't delete again?
 
-I've tried a "git annex proxy -- git revert HASHOFBADCOMMIT", but (as I killed Git Annex before it got through recording that it had trashed my files), I just get:
+I've tried a `git annex proxy -- git revert HASHOFBADCOMMIT`, but (as I killed Git Annex before it got through recording that it had trashed my files), I just get:
 
 ```
 error: Your local changes would be overwritten by revert.
@@ -18,6 +18,8 @@ fatal: revert failed
 
 When syncing in a direct mode repo, does Git Annex happily delete the last copy of a file that appears to have been deleted somewhere else? Or does it save it until you manually clean up unused data, by moving it somewhere under .git?
 
+**UPDATE**: A `git annex sync`, of all things, in the direct mode repository seems to have brought the files and their contents back. It created a commit that undid the deleting commit, except for the deletion of a duplicate copy of one file, which I don't need to have.
+
 ### What steps will reproduce the problem?
 
 It's not entirely clear. Some combination of interrupting and restarting the Android app can make it think that files have been deleted when they really have never been created.

diff --git a/doc/bugs/Android_client_deletes_everything.mdwn b/doc/bugs/Android_client_deletes_everything.mdwn
new file mode 100644
index 0000000..f4a3ef6
--- /dev/null
+++ b/doc/bugs/Android_client_deletes_everything.mdwn
@@ -0,0 +1,43 @@
+### Please describe the problem.
+
+After not syncing my Android repo for a while, I tried to sync it. By some combination of starting up the assistant, killing it, running `git annex sync --content`, and killing *it*, I managed to get the Android client (operating in direct mode) to decide that I had manually deleted a whole bunch of files that had in fact just not been created yet, and to create a commit to that effect, which it promptly synced out to my other repos.
+
+I then synced in my main direct mode crippled filesystem repo with the only copy of some files, and got a bunch of messages that Git Annex was deleting files I wanted to keep. I killed that sync with a ctrl+c.
+
+My problem is: how do I revert the offending commit and restore my files?
+
+My other question is: how do I prevent this happening again? Is there a way I can pre-clear commits and not accept those that delete files without manual confirmation? Or should I just stop being mean to the Android client and hope it doesn't decide to delete things it shouldn't delete again?
+
+I've tried a "git annex proxy -- git revert HASHOFBADCOMMIT", but (as I killed Git Annex before it got through recording that it had trashed my files), I just get:
+
+```
+error: Your local changes would be overwritten by revert.
+hint: Commit your changes or stash them to proceed.
+fatal: revert failed
+```
+
+When syncing in a direct mode repo, does Git Annex happily delete the last copy of a file that appears to have been deleted somewhere else? Or does it save it until you manually clean up unused data, by moving it somewhere under .git?
+
+### What steps will reproduce the problem?
+
+It's not entirely clear. Some combination of interrupting and restarting the Android app can make it think that files have been deleted when they really have never been created.
+
+Alternately, to get the problem I really want to fix now, delete a file in one repo, sync the delete to a direct-mode repo with the only copy. Then somehow undo the delete from the direct mode repo and restore the content of the file.
+
+### What version of git-annex are you using? On what operating system?
+
+The PC has 6.20160903-g1c0b2b4 and Android has 6.20160714.
+
+### Please provide any additional information below.
+
+[[!format sh """
+# If you can, paste a complete transcript of the problem occurring here.
+# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
+
+
+# End of transcript or log.
+"""]]
+
+### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
+
+

info: Support being passed a treeish, and show info about the annexed files in it similar to how a directory is handled.
diff --git a/CHANGELOG b/CHANGELOG
index 4c5147c..4f7674a 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -13,6 +13,8 @@ git-annex (6.20160908) UNRELEASED; urgency=medium
   * Improve gpg secret key list parser to deal with changes in gpg 2.1.15.
     Fixes key name display in webapp.
   * stack.yaml: Update to lts-7.0 (ghc 8)
+  * info: Support being passed a treeish, and show info about the annexed
+    files in it similar to how a directory is handled.
 
  -- Joey Hess <id@joeyh.name>  Thu, 08 Sep 2016 12:48:55 -0400
 
diff --git a/Command/Info.hs b/Command/Info.hs
index f8a13eb..9def388 100644
--- a/Command/Info.hs
+++ b/Command/Info.hs
@@ -1,6 +1,6 @@
 {- git-annex command
  -
- - Copyright 2011-2014 Joey Hess <id@joeyh.name>
+ - Copyright 2011-2016 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -24,6 +24,7 @@ import Utility.DataUnits
 import Utility.DiskFree
 import Annex.Content
 import Annex.UUID
+import Annex.CatFile
 import Logs.UUID
 import Logs.Trust
 import Logs.Location
@@ -31,6 +32,7 @@ import Annex.NumCopies
 import Remote
 import Config
 import Git.Config (boolConfig)
+import qualified Git.LsTree as LsTree
 import Utility.Percentage
 import Types.Transfer
 import Logs.Transfer
@@ -136,7 +138,7 @@ itemInfo o p = ifM (isdir p)
 					Right u -> uuidInfo o u
 					Left _ -> ifAnnexed p 
 						(fileInfo o p)
-						(noInfo p)
+						(treeishInfo o p)
 	)
   where
 	isdir = liftIO . catchBoolIO . (isDirectory <$$> getFileStatus)
@@ -144,17 +146,33 @@ itemInfo o p = ifM (isdir p)
 noInfo :: String -> Annex ()
 noInfo s = do
 	showStart "info" s
-	showNote $ "not a directory or an annexed file or a remote or a uuid"
+	showNote $ "not a directory or an annexed file or a treeish or a remote or a uuid"
 	showEndFail
 
 dirInfo :: InfoOptions -> FilePath -> Annex ()
 dirInfo o dir = showCustom (unwords ["info", dir]) $ do
-	stats <- selStats (tostats dir_fast_stats) (tostats dir_slow_stats)
+	stats <- selStats
+		(tostats (dir_name:tree_fast_stats True))
+		(tostats tree_slow_stats)
 	evalStateT (mapM_ showStat stats) =<< getDirStatInfo o dir
 	return True
   where
 	tostats = map (\s -> s dir)
 
+treeishInfo :: InfoOptions -> String -> Annex ()
+treeishInfo o t = do
+	mi <- getTreeStatInfo o (Git.Ref t)
+	case mi of
+		Nothing -> noInfo t
+		Just i -> showCustom (unwords ["info", t]) $ do
+			stats <- selStats 
+				(tostats (tree_name:tree_fast_stats False)) 
+				(tostats tree_slow_stats)
+			evalStateT (mapM_ showStat stats) i
+			return True
+  where
+	tostats = map (\s -> s t)
+
 fileInfo :: InfoOptions -> FilePath -> Key -> Annex ()
 fileInfo o file k = showCustom (unwords ["info", file]) $ do
 	evalStateT (mapM_ showStat (file_stats file k)) (emptyStatInfo o)
@@ -192,27 +210,29 @@ global_fast_stats =
 	, transfer_list
 	, disk_size
 	]
+
 global_slow_stats :: [Stat]
 global_slow_stats = 
 	[ tmp_size
 	, bad_data_size
 	, local_annex_keys
 	, local_annex_size
-	, known_annex_files
-	, known_annex_size
+	, known_annex_files True
+	, known_annex_size True
 	, bloom_info
 	, backend_usage
 	]
-dir_fast_stats :: [FilePath -> Stat]
-dir_fast_stats =
-	[ dir_name
-	, const local_annex_keys
+
+tree_fast_stats :: Bool -> [FilePath -> Stat]
+tree_fast_stats isworktree =
+	[ const local_annex_keys
 	, const local_annex_size
-	, const known_annex_files
-	, const known_annex_size
+	, const (known_annex_files isworktree)
+	, const (known_annex_size isworktree)
 	]
-dir_slow_stats :: [FilePath -> Stat]
-dir_slow_stats =
+
+tree_slow_stats :: [FilePath -> Stat]
+tree_slow_stats =
 	[ const numcopies_stats
 	, const reposizes_stats
 	]
@@ -295,6 +315,9 @@ countRepoList n s = show n ++ "\n" ++ beginning s
 dir_name :: FilePath -> Stat
 dir_name dir = simpleStat "directory" $ pure dir
 
+tree_name :: String -> Stat
+tree_name t = simpleStat "tree" $ pure t
+
 file_name :: FilePath -> Stat
 file_name file = simpleStat "file" $ pure file
 
@@ -337,13 +360,19 @@ remote_annex_size :: UUID -> Stat
 remote_annex_size u = simpleStat "remote annex size" $
 	showSizeKeys =<< cachedRemoteData u
 
-known_annex_files :: Stat
-known_annex_files = stat "annexed files in working tree" $ json show $
-	countKeys <$> cachedReferencedData
+known_annex_files :: Bool -> Stat
+known_annex_files isworktree = 
+	stat ("annexed files in " ++ treeDesc isworktree) $ json show $
+		countKeys <$> cachedReferencedData
 
-known_annex_size :: Stat
-known_annex_size = simpleStat "size of annexed files in working tree" $
-	showSizeKeys =<< cachedReferencedData
+known_annex_size :: Bool -> Stat
+known_annex_size isworktree = 
+	simpleStat ("size of annexed files in " ++ treeDesc isworktree) $
+		showSizeKeys =<< cachedReferencedData
+  
+treeDesc :: Bool -> String
+treeDesc True = "working tree"
+treeDesc False = "tree"
 
 tmp_size :: Stat
 tmp_size = staleSize "temporary object directory size" gitAnnexTmpObjectDir
@@ -523,6 +552,36 @@ getDirStatInfo o dir = do
 			, return vs
 			)
 
+getTreeStatInfo :: InfoOptions -> Git.Ref -> Annex (Maybe StatInfo)
+getTreeStatInfo o r = do
+	fast <- Annex.getState Annex.fast
+	(ls, cleanup) <- inRepo $ LsTree.lsTree r
+	(presentdata, referenceddata, repodata) <- go fast ls initial
+	ifM (liftIO cleanup)
+		( return $ Just $
+			StatInfo (Just presentdata) (Just referenceddata) repodata Nothing o
+		, return Nothing
+		)
+  where
+	initial = (emptyKeyData, emptyKeyData, M.empty)
+	go _ [] vs = return vs
+	go fast (l:ls) vs@(presentdata, referenceddata, repodata) = do
+		mk <- catKey (LsTree.sha l)
+		case mk of
+			Nothing -> go fast ls vs
+			Just key -> do
+				!presentdata' <- ifM (inAnnex key)
+					( return $ addKey key presentdata
+					, return presentdata
+					)
+				let !referenceddata' = addKey key referenceddata
+				!repodata' <- if fast
+					then return repodata
+					else do
+						locs <- Remote.keyLocations key
+						return (updateRepoData key locs repodata)
+				go fast ls $! (presentdata', referenceddata', repodata')
+
 emptyKeyData :: KeyData
 emptyKeyData = KeyData 0 0 0 M.empty
 
diff --git a/doc/git-annex-info.mdwn b/doc/git-annex-info.mdwn

(Diff truncated)
Added a comment
diff --git a/doc/forum/Hashes_instead_of_content_in_files/comment_3_574964b83563df3afea9049754b1cfa2._comment b/doc/forum/Hashes_instead_of_content_in_files/comment_3_574964b83563df3afea9049754b1cfa2._comment
new file mode 100644
index 0000000..9f2b988
--- /dev/null
+++ b/doc/forum/Hashes_instead_of_content_in_files/comment_3_574964b83563df3afea9049754b1cfa2._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="Adam000"
+ subject="comment 3"
+ date="2016-09-15T16:03:42Z"
+ content="""
+Locking the files in the first repo did the trick. Ideally a solution is available permitting transfer of unlocked files.
+"""]]

Added a comment
diff --git a/doc/forum/Hashes_instead_of_content_in_files/comment_2_a111296eecf7d69ff30f6a82e9f1e6d4._comment b/doc/forum/Hashes_instead_of_content_in_files/comment_2_a111296eecf7d69ff30f6a82e9f1e6d4._comment
new file mode 100644
index 0000000..d9f4d5d
--- /dev/null
+++ b/doc/forum/Hashes_instead_of_content_in_files/comment_2_a111296eecf7d69ff30f6a82e9f1e6d4._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="Adam000"
+ subject="comment 2"
+ date="2016-09-15T15:57:59Z"
+ content="""
+Linux boxes. The repository is v6 and has files unlocked. I like to have my files unlocked so I can edit them. Is there a way to keep them unlocked, but transfer the real file content? Or, edit, then, as a batch operation, lock, sync, unlock?
+"""]]

diff --git a/doc/todo/Workflow_guide.mdwn b/doc/todo/Workflow_guide.mdwn
new file mode 100644
index 0000000..5cc971b
--- /dev/null
+++ b/doc/todo/Workflow_guide.mdwn
@@ -0,0 +1,15 @@
+I try to use git annex, but I frequently don't know if I'm doing things correctly, and my files are getting messed up.
+
+An expert guide to the full workflow would go a long way toward user-friendliness for me. The walkthrough currently has guides to a number discrete items in the workflow, but it doesn't give me a clear sense of the process.
+
+I'm always confused about when I'm supposed to be using pure git commands and when they should be git annex commands, when to commit, add, and sync --content, and when each of these is redundant.
+
+If possible, most helpful would be a guide to how you imagine the workflow from the beginning and including each step of the process, in the order you'd do it.
+
+I want to start keeping track of some files I have in a directory
+I want to copy them to a second computer.
+From a third place, I want to get them from the second computer.
+I change the files on one computer, and I want to make sure the changes get synced to the others.
+What are the commands you'd run at each step?
+
+Many thanks.

comment
diff --git a/doc/forum/Hashes_instead_of_content_in_files/comment_1_800d875c0d980614792207641cedb3ea._comment b/doc/forum/Hashes_instead_of_content_in_files/comment_1_800d875c0d980614792207641cedb3ea._comment
new file mode 100644
index 0000000..23ce342
--- /dev/null
+++ b/doc/forum/Hashes_instead_of_content_in_files/comment_1_800d875c0d980614792207641cedb3ea._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2016-09-15T04:53:16Z"
+ content="""
+Have you tried `git annex get`?
+
+The most common way this can happen is if using Windows or a FAT
+filesystem, or something else that does not support symbolic links.
+It also happens when using v6 repository mode with unlocked files.
+"""]]

diff --git a/doc/forum/Full_workflow_guide.mdwn b/doc/forum/Full_workflow_guide.mdwn
new file mode 100644
index 0000000..a9b22e5
--- /dev/null
+++ b/doc/forum/Full_workflow_guide.mdwn
@@ -0,0 +1,16 @@
+I try to use git annex, but I frequently don't know if I'm doing things correctly, and my files are getting messed up. 
+
+An expert guide to the full workflow would go a long way toward user-friendliness for me. The walkthrough currently has guides to a number discrete items in the workflow, but it doesn't give me a clear sense of the process. 
+
+I'm always confused about when I'm supposed to be using pure git commands and when they should be git annex commands, when to commit, add, and sync --content, and when each of these is redundant.
+
+If possible, most helpful would be a guide to how you imagine the workflow from the beginning and including each step of the process, in the order you'd do it. 
+
+1. I want to start keeping track of some files I have in a directory
+2. I want to copy them to a second computer. 
+3. From a third place, I want to get them from the second computer. 
+4. I change the files on one computer, and I want to make sure the changes get synced to the others. 
+
+What are the commands you'd run at each step?
+
+Many thanks.

diff --git a/doc/forum/Hashes_instead_of_content_in_files.mdwn b/doc/forum/Hashes_instead_of_content_in_files.mdwn
new file mode 100644
index 0000000..86614cf
--- /dev/null
+++ b/doc/forum/Hashes_instead_of_content_in_files.mdwn
@@ -0,0 +1 @@
+I just cloned from a bare repo. The files I got aren't symlinks, but when I look inside them, they're just text SHA numbers, not the real content of the files. How can I get the real content?

comment
diff --git a/doc/bugs/build_fails_on_debian_jessie/comment_4_667a58e3bef3ef908038d13e9ed8200e._comment b/doc/bugs/build_fails_on_debian_jessie/comment_4_667a58e3bef3ef908038d13e9ed8200e._comment
new file mode 100644
index 0000000..a277fcd
--- /dev/null
+++ b/doc/bugs/build_fails_on_debian_jessie/comment_4_667a58e3bef3ef908038d13e9ed8200e._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2016-09-14T16:20:25Z"
+ content="""
+Re shakespeare, IIRC the dependency on version 2.0 was only needed
+in order to remove a dependency on hamlet, which got removed upstream.
+So if you add back the dependency on hamlet, it should
+build with the older version of shakespeare.
+
+See git commit 2989cdfe3ed7524c00d16b21e8af9773c37497ff.
+Basically, you want to revert that commit.
+
+Sorry about this farce; I tragically can't see a way to express
+the actual dependency situation in git-annex.cabal and debian/control ..
+"""]]

comment
diff --git a/doc/todo/make_copy_--fast__faster/comment_7_3f52b6e19035d3c891356c6d98035987._comment b/doc/todo/make_copy_--fast__faster/comment_7_3f52b6e19035d3c891356c6d98035987._comment
new file mode 100644
index 0000000..c471325
--- /dev/null
+++ b/doc/todo/make_copy_--fast__faster/comment_7_3f52b6e19035d3c891356c6d98035987._comment
@@ -0,0 +1,58 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 7"""
+ date="2016-09-14T15:28:23Z"
+ content="""
+First, note that git-annex 6.20160619 sped up the git-annex 
+command startup time significantly. Please be sure to use a current
+version in benchmarks, and state the version.
+
+`git archive` (and `git cat-file --batch --batch-all-objects`) are just
+reading packs and loose objects in disk order and dumping out the contents.
+`git cat-file --batch` has to look up objects in the pack index files, seek
+in the pack, etc. It's not a fair comparison.
+
+Note that `git annex find`, when used without options like --in or --copies, 
+does not need to read anything from `git cat-file` at all. The
+`GIT_TRACE_PERFORMANCE` you show is misleading; it's just showing how long
+the git command is left running, idle.
+
+`git annex find`'s overhead should be purely traversing the filesystem tree
+and checking what symlinks point to files. You can write programs that do
+the same thing without using git at all (or only `git ls-files`), and
+compare them to git-annex's time; that would be a fairer comparison.
+Ideally, `git annex find` would be entirely system call bound and would use
+very little CPU itself.
+
+By contrast, `git annex copy` makes significant use of `git cat-file --batch`,
+since it needs to look up location log information to see if the
+--to/--from remote has the files.
+
+`git annex copy -J` already parallelizes the parts of the code that look at
+the location log. Including spinning up a separate `git cat-file --batch`
+processes for each thread, so they won't contend on such queries. So I
+would expect that to make it faster, even leaving aside the speed benefits
+of doing the actual copies in parallel.
+
+My feeling is that the best way to speed these up is going to be in one 
+of these classes:
+
+* It's possible that `git cat-file --batch` is somehow slower than it needs
+  to be. Perhaps it's not doing good caching between queries or has
+  inneficient seralization/bad stdio buffering. It might just be the case 
+  that using something  like libgit2 instead would be faster.
+  (Due to libgit2's poor interface stability, it would have to be an
+  optional build flag.)
+
+* Many small optimisations to the code. The use of Strings throughout
+  git-annex could well be a source of systematic small innefficiences,
+  and using ByteString might eliminate those. (But this would be a huge job.)
+  (The `git cat-file --batch` communication is already done using
+  bytestrings.)
+
+* A completely lateral move. For example, if git-annex kept its own
+  database recording which files are present, then `git annex find`
+  could do a simple database query and not need to chase all the symlinks.
+  But such a database needs to somehow be kept in sync or reconciled
+  with the git index, it's not an easy thing.
+"""]]

Added a comment: clarification
diff --git a/doc/forum/partial_annex_repository/comment_1_5e594914724a04e21087e1e4e57c2d30._comment b/doc/forum/partial_annex_repository/comment_1_5e594914724a04e21087e1e4e57c2d30._comment
new file mode 100644
index 0000000..a92c9e6
--- /dev/null
+++ b/doc/forum/partial_annex_repository/comment_1_5e594914724a04e21087e1e4e57c2d30._comment
@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="git-starter"
+ subject="clarification"
+ date="2016-09-13T20:27:22Z"
+ content="""
+My previous posting looks like I'm not aware of largefiles, which would mostly answer my question.    My problem was that, after reading some doc that might not have been fully updated for v6, I got confused about the difference between an unlocked file in the annex, and a file not in the annex.    
+
+In any case, since I added some small files before using largefiles, those small files were put in the annex.   How do I change them to be in the git repository not in the annex?
+
+Thanks.
+
+
+
+"""]]

diff --git a/doc/forum/partial_annex_repository.mdwn b/doc/forum/partial_annex_repository.mdwn
new file mode 100644
index 0000000..d83eac1
--- /dev/null
+++ b/doc/forum/partial_annex_repository.mdwn
@@ -0,0 +1,14 @@
+After I created a git repository, I added annex to it.
+
+Now, if I add a new file with 'git add', it goes into the annex.   However previously existing files are not in the annex.
+
+What are the implications of this?   Should I care?
+
+FWIW, most of the files are small, so I see no reason for the annex for them.
+
+How do I add a new file and not put it in the annex?
+
+How to I switch a file from one to the other?
+
+Thanks.
+

Added a comment
diff --git a/doc/bugs/build_fails_on_debian_jessie/comment_3_3f6e35e7b618bf1762e37611b13e26bb._comment b/doc/bugs/build_fails_on_debian_jessie/comment_3_3f6e35e7b618bf1762e37611b13e26bb._comment
new file mode 100644
index 0000000..5ab817b
--- /dev/null
+++ b/doc/bugs/build_fails_on_debian_jessie/comment_3_3f6e35e7b618bf1762e37611b13e26bb._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="jk@3f2b4ce16bbac41470815333505aa47b91b7a9a6"
+ nickname="jk"
+ subject="comment 3"
+ date="2016-09-13T07:05:48Z"
+ content="""
+A bit more detail: the version of git-annex in jessie-backports has shakespeare 2 as a dependency, but jessie-backports does not contain a package of that; also my attempts to build shakespeare 2 in a jessie{,-backports} enviroment fails with a long list of failed dependencies.
+"""]]