Finished up youtube-dl integration today, including all the edge cases in
addurl and honoring annex.diskreserve.
I changed my mind about git annex addurl --relaxed; it seems better for
it to be slower than before, but not have surprising behavior, than to be
fast but potentially surprising. If it's too slow, add --raw to avoid
using youtube-dl.
Merry Christmas! Thanks for adding this. I was falling back to yt-dl when quvi failed. I built from git last night and tested this. I have a few comments and questions.
I am using debian/stretch. Is yt-dl being used from system python package, or my virtualenv? (I have been preparing for py3 being default version in a couple of years.)
I have often used a terminal window and prefixed my input with "git-annex addurl" then drag links to the window for pasting. Often, I have to press the up-arrow and run the command again. The addurl behavior with urls that are already locally present, quvi responds "ok." However, when repeating a command using yt-dl, when the url is already local, yt-dl refuses to overwrite, yet returns "failed." I didn't know if you were aware of this. This isn't a show-stopper, but just something I noticed. I generally do "addurl" manually.
Playlist support. Currently youtube-dl will allow you to download a whole playlist. When I add a url "yt.com/watch?v=vidID&list=playlistID" only the video identified is downloaded. This is a good default behavior. When I add " yt.com/watch?list=listID" yt-dl successfully parses the whole list, but will only download one video from the list.
It uses whatever youtube-dl you have in PATH.
I was not aware of the changed behavior when adding an url that's already locally present. I've opened ?addurl youtube-dl behavior change to remember to look into it.
The reason it only downloads one video from a playlist or channel is that git-annex needs to associate an url with the annexed file, so it can later download that single file when git-annex get is used, and there's no clear way to associate an url with a particular file that youtube-dl downloaded from a playlist or channel or whatever. downloading podcasts has a way to use
git annex importfeedwith a youtube playlist though.importfeednow!I tested importfeed. It seemed to be stuck on
importfeed checking known urls. The first playlist I tried had 70+ videos, so I tested a shorter playlist and got the same response. I noticed that on the downloading podcasts page, only channels are mentions, but not playlists. However, you mentioned playlists in your response. I was able to work around the issue:for _id in `youtube-dl 'https://www.youtube.com/watch?v=ps40Uminvno&list=PLWnVxuqvY7Jj2Z8aeR-qKXACCXUN4Ouv_' --get-id` ; do echo "https://www.youtube.com/watch?v=${_id}" ; done | git-annex addurl --batch --jobs 2