Please describe the problem.
For some not obvious to me reason, git-annex used yt-dlp
fine for many files but then does not use for another, very odd:
here is two files from (pushed all to http://github.com/TrueTube/Andriy_Popik)
(git)smaug:/tmp/Andriy_Popyk/Andriy_Popyk[master]git
$> git annex whereis Чат_рулетка/2024-09-30-Зеки_-_основа_россии._Андрій_Попик.mkv Чат_рулетка/2024-10-01-Рассеянка__Андрій_Попик.mkv
whereis "\320\247\320\260\321\202_\321\200\321\203\320\273\320\265\321\202\320\272\320\260/2024-09-30-\320\227\320\265\320\272\320\270_-_\320\276\321\201\320\275\320\276\320\262\320\260_\321\200\320\276\321\201\321\201\320\270\320\270._\320\220\320\275\320\264\321\200\321\226\320\271_\320\237\320\276\320\277\320\270\320\272.mkv" (2 copies)
00000000-0000-0000-0000-000000000001 -- web
05689e10-d659-4239-a614-a6de95b11208 -- yoh@smaug:/mnt/btrfs/datasets/datalad/crawl-misc/TrueTube/popik
web: https://www.youtube.com/watch?v=Ikn4LnYDcKU
ok
whereis "\320\247\320\260\321\202_\321\200\321\203\320\273\320\265\321\202\320\272\320\260/2024-10-01-\320\240\320\260\321\201\321\201\320\265\321\217\320\275\320\272\320\260__\320\220\320\275\320\264\321\200\321\226\320\271_\320\237\320\276\320\277\320\270\320\272.mkv" (1 copy)
00000000-0000-0000-0000-000000000001 -- web
web: https://www.youtube.com/watch?v=5rFP3DlC2MU
ok
for one ($> git annex get --debug Чат_рулетка/2024-09-30-Зеки_-_основа_россии._Андрій_Попик.mkv
) works fine -- starts downloading using yt-dlp. But for the other:
$> git annex get --debug Чат_рулетка/2024-10-01-Рассеянка__Андрій_Попик.mkv
[2024-10-01 17:00:12.822269342] (Utility.Process) process [2784960] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"]
[2024-10-01 17:00:12.827363166] (Utility.Process) process [2784960] done ExitSuccess
[2024-10-01 17:00:12.828029045] (Utility.Process) process [2784961] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"]
[2024-10-01 17:00:12.832174221] (Utility.Process) process [2784961] done ExitSuccess
[2024-10-01 17:00:12.832951895] (Utility.Process) process [2784962] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..1c3a3eacf78934c71cdf560061207cdb13422fd0","--pretty=%H","-n1"]
[2024-10-01 17:00:12.835645728] (Utility.Process) process [2784962] done ExitSuccess
[2024-10-01 17:00:12.840540312] (Utility.Process) process [2784963] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"]
[2024-10-01 17:00:12.855549135] (Utility.Process) process [2784964] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","ls-files","--stage","-z","--error-unmatch","--","\1063\1072\1090_\1088\1091\1083\1077\1090\1082\1072/2024-10-01-\1056\1072\1089\1089\1077\1103\1085\1082\1072__\1040\1085\1076\1088\1110\1081_\1055\1086\1087\1080\1082.mkv"]
[2024-10-01 17:00:12.856270222] (Utility.Process) process [2784965] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)","--buffer"]
[2024-10-01 17:00:12.85694905] (Utility.Process) process [2784966] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"]
[2024-10-01 17:00:12.857415068] (Utility.Process) process [2784967] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"]
[2024-10-01 17:00:12.864931666] (Utility.Process) process [2784967] done ExitSuccess
[2024-10-01 17:00:12.865042169] (Utility.Process) process [2784966] done ExitSuccess
[2024-10-01 17:00:12.865124487] (Utility.Process) process [2784965] done ExitSuccess
[2024-10-01 17:00:12.865195432] (Utility.Process) process [2784964] done ExitSuccess
get "\320\247\320\260\321\202_\321\200\321\203\320\273\320\265\321\202\320\272\320\260/2024-10-01-\320\240\320\260\321\201\321\201\320\265\321\217\320\275\320\272\320\260__\320\220
\320\275\320\264\321\200\321\226\320\271_\320\237\320\276\320\277\320\270\320\272.mkv" (from web...)
[2024-10-01 17:00:12.898739526] (Utility.Url) Request {
host = "www.youtube.com"
port = 443
secure = True
requestHeaders = [("Accept-Encoding","identity"),("User-Agent","git-annex/10.20240927+git1-gf3403e9691-1~ndall+1")]
path = "/watch"
queryString = "?v=5rFP3DlC2MU"
method = "GET"
proxy = Nothing
rawBody = False
redirectCount = 10
responseTimeout = ResponseTimeoutDefault
requestVersion = HTTP/1.1
proxySecureMode = ProxySecureWithConnect
get "\320\247\320\260\321\202_\321\200\321\203\320\273\320\265\321\202\320\272\320\260/2024-10-01-\320\240\320\260\321\201\321\201\320\265\321\217\320\275\320\272\320\260__\320\220\320\275\320\264\321\200\321\226\320\271_\320\237\320\276\320\277\320\270\320\272.mkv" (from web...)
Verification of content failed
Unable to access these remotes: web
No other repository is known to contain the file.
(Note that these git remotes have annex-ignore set: origin)
failed
[2024-10-01 17:00:13.516843908] (Utility.Process) process [2784963] done ExitSuccess
get: 1 failed
What steps will reproduce the problem?
git clone http://github.com/TrueTube/Andriy_Popik
cd Andriy*
git config annex.security.allowed-ip-addresses all
git annex get --debug Чат_рулетка/2024-10-01-Рассеянка__Андрій_Попик.mkv
What version of git-annex are you using? On what operating system?
$> git annex version
git-annex version: 10.20240927+git1-gf3403e9691-1~ndall+1
I suspect that one file has a "yt:" url recorded and the other one does not. That's not displayed by
whereis
but you can see it if you look at the $key.log.web file in the git-annex branch.That typically would happen if you used
git-annex addurl --raw
, or added the url when yt-dlp was not working, although that doesn't explain how the youtube title got into the filename. (Maybe importfeed?)(I seemed to remember you having a similar problem before, and found https://git-annex.branchable.com/todo/yt-dlp__58___parse__47__handle___40__error__41_____34__Video_unavailable__34__/ which seems possibly relevant to how this could have happened.)