Please describe the problem.
Adb special remote does not preserve remote timestamp on "import". Therefore files downloaded several days ago lose their real mtime when transferred to PC, and from that moment onward files on android will have mtime different from stored in annex/symlinks.
But all files already present in PC repo -- during "export" correctly propagate their mtime to android.
What steps will reproduce the problem?
git annex initremote android type=adb androiddirectory=/storage/0000-0000/books encryption=none exporttree=yes importtree=yes
git config remote.android.annex-sync false
git config remote.android.annex-tracking-branch main
git annex export main --to android
# ADD: download some "file" through android into "books"
git annex sync --content android
stat -c%y "file"
Modify: 2022-01-01 20:51:11.487603199 +0200
What version of git-annex are you using? On what operating system?
git-annex version: 8.20211118-g23ee48898 ArchLinux 5.15.12-arch1-1
Please provide any additional information below.
Most likely can be fixed by:
--- i/Remote/Adb.hs
+++ w/Remote/Adb.hs
@@ -202,6 +202,7 @@ retrieve' serial src dest =
showOutput -- make way for adb pull output
liftIO $ boolSystem "adb" $ mkAdbCommand serial
[ Param "pull"
+ , Param "-a"
, File $ fromAndroidPath src
, File dest
]
Have you had any luck using git-annex before?
Nice work fixing earlier notorious bug, it made the ADB special remote usable for me at least in PC -> android direction! 
https://git-annex.branchable.com/forum/Mixed_content_repos_with_import_and_export/#comment-764ac971faf756140055333649ffb94c
Git does not preserve timestamps. So this is kind of out of scope.
While git-annex will sometimes preserve timestamps when it's easy to do so, there are no guarantees. Most special remotes do not preserve any timestamps in either direction.
It might be that
adb pull -awould fix this. But that also preserves the permissions, and pulling in random permissions for annex object files could lead to arbitrary breakage (imagine a suid executable for example). So it would need to clear the permissions. The complexity does not seem worth trying to support a thing that is generally out of scope.Yes, git does not, but PC filesystem does. And when you annex files -- git-annex copies file own mtime to its symlink. Therefore you can sort and seek files based on the date of your first acquaintance (downloading) instead of the date of batch-sync.
I don't see a reason to sync any native executables with Android phone for usual mediafile-centered workflow with usual /storage location. Moreover, suid is of no concern for years due to virtual filesystem used in Android, which simply does not support suid or exec anyway:
Consider next workflow:
If adb does not preserve timestamps -- you lose "date" information, and core of the workflow crumbles to dust. And you can't use (name, size, mtime) as kind of UID for sync too.
To support this workflow I used rsync+adbfs, but it's a pain due to how you can't simply and efficiently rsync from phone into annex symlinks and back and be done with it. Moreover it's a separate different step, which could be eliminated if "git annex sync" properly worked with adb remote.
But you would have the same problem with that workflow if you were checking the files into a git repository not using git-annex, and pulling from that repository.
For that metter, you would have the same problem with that workflow if you were running
git-annex addand usinggit-annex getto transfer over a ssh connection.The mere fact that it might be possible to support that particular workflow with adb or some other type of remote does not mean that it's a bug for git-annex not to support it. Or that it's a good idea to complicate git-annex to partially support it.
That's entirely different. When you run
git addon a file, the file's mtime is left unchanged. It follows that, when yougit-annex adda file, the mtime of the file (now a symlink) should also be unchanged. That's entirely unrelated to remotes.This bug report is still open because I am still open to being argued out of my current position. But such an argument would have to engage with the things I discussed.
Ok, let's continue
I distinguish ADB from all other types of remotes -- because it's the actual source of new files -- not yet processed by user manually. And what you mentioned above -- are scenarios occuring on full-fledged work system, not on half-baked android phone.
When you sync Laptop with PC -- you must add files either on Laptop or on PC into git-annex first. Therefore you have an opportunity to do something with files first e.g. sort them into the folders by date, before adding them into git and losing that mtime information (which at that point is still useful, but not necessary).
When you sync PC with any "backup" remote -- they are pushed/pulled after files were added to git-annex. I.e. none of them adds new files, which user never seen before -- and process only "existing" ones.
But when you use ADB (or maybe Directory too -- however I still don't have a usecase for that) -- new files are added to ADB directly, avoiding user intervention. Because it's a pain to sort them on the phone immeditely without proper tools and scripts. And one of the purposes of using git-annex here -- is to fetch them to PC to sort properly on big screen. But fetching them without "pull -a" looses the necessary information. It's not that big of a problem for DCIM folder, as files contain dates in filenames, but it's an issue for Downloads (and separate folders of each chat app).
Therefore yes, ADB is different, it's involved into different workflow, and therefore deserves different treatment.
Still, it has a nice consequence of preserving mtime for files already present on PC.
And it allows me to scan whole filesystem and dump metadata into a separate file (e.g. "find -printf "%T@ %P\n"), to preserve the information "when I first seen/downloaded that file" for the future.
And it's very important information (at least for me), because it's easier to remember and link related things occuring in a similar timespan, than to sort files by types and then fruitlessly trying to link those fragmented and sparse datasets inside my head after several months or years.
@joey can we change android remote to use "pull -a" at least "for now"?
Let's try doing it in Linux Kernel or ZeroMQ way: enable/disable feature and wait for new bugreports. And if new bugreports were critical -- revert feature back. After all we can't predict everything from the get go.
I wish to preserve file original mtime (date of download) at least "somewhere" (e.g. manually dump find/CID into textual file). But doing "annex pull" from android remote loses this information completely. And this feature is critical for me -- unless some evidence will appear that having this feature more detrimental than not.
Concerns about SUID and other permissions are ungrounded, because Android virtual driver for /sdcard uses only 644/755 by design. But instead of speculating -- we may simply wait for new bugreports.
@joey, what do you say? Let's experiment and gather results from wider audience?