Recent comments posted to this site:

I stumbled over the same problem. With "git annex add" I can add the file. But in unlocked mode, I get the "out of memory" error when trying to commit. Is this also a known problem? Is there a workaround?
Comment by yves.noirjean Fri Jun 22 07:53:26 2018

Actually, I am still interested in a response.

I would prefer to use annex.addunlocked = true, which would use hardlinks.

Comment by yves.noirjean Thu Jun 21 16:15:05 2018

I'm setting up a repository using the following commands:

git annex init
export AWS_ACCESS_KEY_ID="[MY KEY]"
export AWS_SECRET_ACCESS_KEY="[MY SECRET]"
git annex initremote publics3 type=S3 encryption=none bucket=[BUCKET] exporttree=yes public=yes encryption=none
git annex export --tracking master --to publics3

I then add something and export it to s3 using:

git annex sync --content

I then go on to a different computer and clone the repository and run:

git annex init
git annex enableremote publics3
git annex get .

and receive this message:

# git annex get .
get IMG_2714.MOV (from publics3...) 
Set both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to use S3

No S3 credentials configured

Unable to access these remotes: publics3

Try making some of these repositories available:
     6bc91baa-5250-41e2-8195-d3c1ef3c86e3 -- .
     7f84dcff-5a74-42b0-b422-5b74490e4aa7 -- [publics3]

 (Note that these git remotes have annex-ignore set: origin)

failed git-annex: get: 1 failed

It is my understanding that setting the remote to "public=yes" should allow me to download the files from the remote repository without entering the credentials again. Am I missing something?

Thanks!

Comment by jared Thu Jun 21 13:47:27 2018
The issue has been resolved. The unix file permissions were not applied, but the NFS4 permissions. So git annex did not trust that files without write flags could not be changed. Therefore, it didn't move the files and create symlinks. This somehow lead to git allocating a lot of memory during commit.
Comment by yves.noirjean Mon Jun 18 11:25:18 2018

i retested a few times, and the actual tool i should have used is a link to the git-annex script in the root

sorry for the noise

Comment by git-annex.branchable.com Sat Jun 16 09:48:53 2018

Oh, Awesome, thanks! Great find. I'll write out some of the things I have tried:

Create a text file called hi.txt from Termux in /data/data/com.termux/files/home AKA $HOME. Launch Turbo Editor enable Use the Storage Access Framework in Preferences menu. Click Open a file | Open from Termux, click hi.txt, edit file, save. Now I can see changes in Termux. Nice.

From Termux create a new soft link, cd, ln -s hi.txt hilink.txt, then in Turbo Editor I can see both files, I can edit and save either, and see changes correctly in Termux.

I wasn't able to find the Termux $HOME folder in any File Manager I tried, nor any Gallery App.

Comment by andrew Fri Jun 15 15:59:04 2018

Thank you for clarifying. I've worked out a setup that seems to do what I want. The rules are a little messier, but not too bad:

group a1 = a 1 a1store
group a2 = a 2 a2store
group b1 = b 1 b1store
group b2 = b 2 b2store

wanted a1 = groupwanted
wanted a2 = groupwanted
wanted b1 = groupwanted
wanted b2 = groupwanted

groupwanted a1store = not copies=1:1 and not copies=a:1
groupwanted a2store = not copies=2:1 and not copies=a:1
groupwanted b1store = not copies=1:1 and not copies=b:1
groupwanted b2store = not copies=2:1 and not copies=b:1

numcopies default = 2
config annex.synccontent = true
Comment by ghen1 Wed Jun 13 17:18:49 2018

I've looked around for how this whole SAF thing works and where it is supported; it seems to me that it is indeed the way to go:

  • Termux added this with the very intention of making the home directory usable (initial issue).
    • That .JPG files are not image/jpeg is probably the fault of a library termux uses (reported)
  • Some applications can find files from a termux-home out-of-the-box, like the billthefarmer editor -- just that if it was just about editing text files, we could just have used ~/storage/shared and plain git.
  • Many applications accept files from there when opened from the file manager (VLC, Simple Gallery, Ghost Commander's picture viewer); I have not found one yet that does not.
  • A good way forward would be to encourage the authors of apps we care about to allow opening SAF locations directly; it appears to me that they sometimes add SAF support for a particular function (eg. to support USB-OTG or writing to external SD cards) and then limit it to that path because the usual other options appear to confusing. Requested support for it in Simple Gallery, let's see where that takes this.
Comment by chrysn Wed Jun 13 14:57:10 2018
Thanks! I think it's independent from adb, as it's the same serial number shown by dmesg.
Comment by lykos Wed Jun 13 10:46:25 2018

ad SDCardFS / FAT32: It seems that both the fuse wrapper around the SD card (be it actually formatted FAT32 because it's the external one, or ext4 as is the encrypted internal one on my device) and SDCardFS (which, it appears to me, is just Google moving the fuse wrapper into the kernel) won't expose the functionality we want.

ad adoptable storage: I think that's what's happening on my device (but am not sure), so if it is, the there's still the fuse wrapper / SDCardFS inbetween.

ad specific app: That'd still mean that git-annex needs to operate on crippled file system, and I don't want to rely on special solutions for each app.


What I've seen when just setting up a new device is that the Storage Access Framework (looks like Android's idea of gvfs / kvfs with a spike of flatpak-portals to me: a userland file system that operates on saf:/ and content:// URIs rather than files, probably does file operations by means of IPC and receiving file handlers) might help:

In the LineageOS built-in file manager, I don't see two "virtual SD cards" (the internal and the external one), but also a "Termux" "device" with proper logo and "0B free", which allows me to peek into the termux home directory (but not its usr directory). From that file manager I even see the photos inside a regular (symlinked) checkout, and can open them with the Simple Gallery app. (Provided they end with '.jpg' and not '.JPG', but that's probably a detail of the file manager).

I did not yet find another app that sees this "Termux" device; Ghost Commander only sees "Documents" and the internal and external SD cards; the "Dir" file manager exposes a SAF selection only when creating directories on external SD (and then shows the same selection as Ghost Commander), the Simple tools (Simple File Manager, Simple gallery) show "Internal", "SD Card" and "Root" in their own storage selection (might be pre-SAF; similar situation also for the Amaze file manager).

If we could reproduce what the builtin file manager does to access the files, that might be a solution here. Termux obviously already helps in this by exporting the home directory (since 0.34), but maybe they also can do more (like exporting XDG user directories for quick access?).

Comment by chrysn Tue Jun 12 19:01:09 2018