Please describe the problem.
I created somelargefile on host1. A file with that name appears in the right place on host2 but I can't use it:
host2:~/annex% cat somelargefile
cat: somelargefile: No such file or directory
What steps will reproduce the problem?
I am running http://downloads.kitenet.net/git-annex/linux/current/git-annex-standalone-amd64.tar.gz and I connected two hosts with the 'Local computer' workflow. I run assistant on both sides.
I make a file on either side, wait a minute for it to sync, and then observe this on the remote side:
lrwxrwxrwx 1 drewp drewp 178 Jul 9 23:58 somelargefile -> .git/annex/objects/Pw/vw/SHA256E-s8--bf87165d313027621936809a01da1994f9bd20ff9580c1380e7636e2443fe4ed/SHA256E-s8--bf87165d313027621936809a01da1994f9bd20ff9580c1380e7636e2443fe4ed
host2:~/annex% ls -l .git/annex/objects/Pw/vw/SHA256E-s8--bf87165d313027621936809a01da1994f9bd20ff9580c1380e7636e2443fe4ed/
total 4
-rw------- 1 drewp drewp 14 Jul 9 23:58 SHA256E-s8--bf87165d313027621936809a01da1994f9bd20ff9580c1380e7636e2443fe4ed.map
My "large file" is in fact 8 bytes long. The .map file contains the string "somelargefile\n"
What version of git-annex are you using? On what operating system?
git-annex version: 4.20130709-g339d1e0 build flags: Assistant Webapp Pairing Testsuite S3 WebDAV Inotify DBus XMPP local repository version: 3 default repository version: 3 supported repository versions: 3 4 upgrade supported from repository versions: 0 1 2
ubuntu 12.10 64-bit
Please provide any additional information below.
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
New lines in .git/annex/daemon.log when I add another new file:
(merging synced/git-annex into git-annex...)
Updating bac8b65..0422d27
Fast-forward
newfile2 | 1 +
1 file changed, 1 insertion(+)
create mode 120000 newfile2
[2013-07-10 00:08:38 PDT] Committer: Committing changes to git
(Recording state in git...)
[2013-07-10 00:08:38 PDT] Pusher: Syncing with host1.local_annex
To ssh://drewp@git-annex-host1.local-drewp_annex/~/annex/
a80b6f5..72c0865 git-annex -> synced/git-annex
Already up-to-date.
# End of transcript or log.
Everything you've described is expected behavior, when the content of the file has not yet been transferred to the repository.
There should be something in daemon.log about the transfer of the file. If there's an error message for the transfer, it will certainly show up there. You can enable debugging to get more information in daemon.log about file transfers.
receiver side logs again, with --debug mode on. I don't see anything weird.
I also noticed on host1 where the file was made, there is a git node in gitk that says 'Local uncommitted changes, not checked in to index' and this node appears to have all my file contents as diffs. So the problem is maybe not the transfer, but the failure of the originating side to commit the file contents properly?
It's normal for git diff and gitk to look like that in a repository that's using direct mode.
In your log, I see you added a file "test-2013-07-15e". And I don't see any indication that it tried to transfer it.
There are a few reasons that could happen. If you've "paused" syncing with a remote in the webapp, it could do that.
I think more likely would be if you've put the remote repository in a repository group that doesn't want the file. If it's configured to be in the Manual group or the Source group (or the Transfer, Public, or Small Archive group for that matter), the assistant won't try to send files to it. You can check this by editing the repository in the webapp. It should probably be set to be in the Client group.
I saw the 'syncing enabled'/'syncing disabled' switch in the webapp, which I think is what you're calling pause. That was always enabled.
I never adjusted the groups, but I see in the webapp that my remote is busted in other ways. It has no title, and settings->edit makes an error page. The edit link is like this:
http://127.0.0.1:38187/config/repository/edit/NoUUID?auth=4294a...333c
and the error is
I'll try re-configuring that remote. I wish the webapp could tell me more about what is broken with the remote repo and where I might go to repair it.
BTW "go enter it into the computer you want to pair with" is a very frustrating instruction to read. Every time I try to do a pairing and I encounter that, it's telling me "now it's time for you to guess what to do, so this is probably where you'll screw things up." I have no idea where to enter that secret phrase. I would also like to know if 'asdfasdf' is a good secret phrase or not, presuming the two computers are both on my internal network. Does the secret matter later or is it just used for a minute?
So far, I keep getting blank repo names with the NoUUID edit url, or I fail to accomplish a pairing at all.
Here's how little it takes for things to go wrong:
First host:
bang(pts/17):~/annex% git init
Initialized empty Git repository in /home/drewp/annex/.git/
bang(pts/17):~/annex% git-annex init
init ok
(Recording state in git...)
bang(pts/17):~/annex% git-annex webapp --listen 10.1.0.1:9999
http://10.1.0.1:9999/?auth=6f9a8b...
Second host:
dash(pts/34):~/annex% git init
Initialized empty Git repository in /home/drewp/annex/.git/
dash(pts/34):~/annex% git-annex init
init ok
(Recording state in git...)
dash(pts/34):~/annex% git-annex webapp --listen 10.1.0.229:9999
http://10.1.0.229:9999/?auth=f28bd56b456....
Browse to first URI. Add another repo. Local computer. Enter 'qwerty'.
Browse to second URI. Respond to pairing request. Enter 'qwerty' again.
View both dashboards: http://bigasterisk.com/post/git-annex-bad-pair.png
What settings does
.git/config
have for the git remote created by the local pairing process?Since I can't reproduce any problems with local pairing -- it works great on this network -- you will need to enable debugging, reproduce the bug, and send a debug log for me to debug this any further.
Like I sort of expected, this is missing a remote.$foo.annex-uuid setting. But I don't know why yet. I need a debug log to figure this out.
Can you enable debugging (run git-annex webapp --debug), reproduce the problem again, and send me the log?
Sorry, I meant to get those logs earlier but I got distracted.
One confounding thing in the previous runs is that I may have had accidental instances of git-annex assistant running on at least one of the machines. I killed them all for this next attempt, and named the dir annex2 just in case.
daemon.log files are now at http://bigasterisk.com/post/git-annex/logs1/
From the log on bang:
You said you're running from the standalone tarball. I think that's the problem, probably if you get git-annex-shell into your path it'll just work.
Can you paste the line git-annex added to "dash"'s .ssh/authorized_keys?
Thanks for finding that. I added git-annex-shell to my path and ran again. The repo lists look good now.
For the record, here's what I am getting in authorized_keys on dash:
When I went to the settings on each box, all 4 repository group select boxes were blank. I set them all to 'client'.
How do we make this long ticket worth your time by improving the code so people don't hit the same trap in the future? I think I would have looked in the right direction if, instead of saying nothing and not having a uuid to make the links work, the web ui said something like this:
Next, I ran "date > annex2/file1" on bang. The web consoles said they synced, but no file appeared on dash. There was a ~/annex2/.git/annex/objects/../../SHA256.../SHA256... file on dash with the right contents, but no symlink showed up in dash:~/annex2.
I ran "date > annex2/file2" on dash, and then I got both file1 and file2 on both boxes.
In the new logs at http://bigasterisk.com/post/git-annex/logs2/, the surprise is at [2013-07-27 22:10:49 PDT] where I thought dash would make a symlink called 'file1' but instead it reports no attempt or error. Later at [2013-07-27 22:12:08 PDT], while dealing with the upload of file2, dash finally makes the symlink for file1.
I'm more interested in fixing core problems, like why it was not able to find git-annex-shell, than in fixing bugs in the web UI that follow on from those problems.
When you're using the standalone tarball, it's supposed to set up
~/.ssh/git-annex-shell
to point to wherever you've unpacked the tarball. The assistant does this the first time you run it, and any time it sees the file is out of date. I've tested this, and it appears to be working ok. I don't understand what happened in your case to prevent this from happening.