Sharing via Seekable Pipe or Stream with Another Android App?

You've posed a really difficult combination of requirements.

Lets look at your ideas for solutions:

Possible suggested ideas include:

• Some way to reconfigure createPipe() that results in a seekable pipe

• Some way to use a socket-based FileDescriptor that results in a seekable pipe

• Some kind of RAM disk or something else that feels like a file to the rest of Android but is not persistent

The first one won't work. This issue is that the pipe primitive implemented by the OS is fundamentally non-seekable. The reason is supporting seek that would require the OS to buffer the entire pipe "contents" ... until the reading end closes. That is unimplementable ... unless you place a limit on the amount of data that can be sent through the pipe.

The second one won't work either, for pretty much the same reason. OS-level sockets are not seekable.

At one level, the final idea (a RAM file system) works, modulo that such a capability is supported by the Android OS. (A Ramfs file is seekable, after all.) However, a file stream is not a pipe. In particular the behaviour with respect to the end-of-file is different for a file stream and a pipe. And getting a file stream to look like a pipe stream from the perspective of the reader would entail some special code on that side. (The problem is similar to the problem of running tail -f on a log file ...)

---

Unfortunately, I don't think there's any other way to get a file descriptor that behaves like a pipe with respect to end-of-file and is also seekable ... short of radically modifying the operating system.

If you could change the application that is reading from the stream, you could work around this. This is precluded by the fact that the fd needs to be read and seeked by QuickOffice which (I assume) you can't modify. (But if you could change the application, there are ways to make this work ...)

---

By the way, I think you'd have the some problems with these requirements on Linux or Windows. And they are not Java specific.

---

UPDATE

There have been various interesting comments on this, and I want to address some here:

1. The OP has explained the use-case that is motivating his question. Basically, he wants a scheme where the data passing through the "channel" between the applications is not going to be vulnerable in the event that the users device is stolen (or confiscated) while the applications are actually running.

   Is that achievable?

   • In theory, no. If one postulates a high degree of technical sophistication (and techniques that the public may not know about ...) then the "bad guys" could break into the OS and read the data from shared memory while the "channel" remained active.

   • I doubt that such attacks are (currently) possible in practice.

   • However, even if we assume that the "channel" writes nothing to "disc" there could still be traces of the channel in memory: e.g.

     • a still mounted RAMfs or still active shared memory segments, or

     • remnants of previous RAMfs / shared memory.

     In theory, this data could in theory be retrieved, provided that the "bad guy" doesn't turn of or reboot the device.

2. It has been suggested that ashmem could be used in this context:

   • The issue of there being no public Java APIs could be addressed (by writing 3rd-party APIs, for example)

   • The real stumbling block is the need for a stream API. According the "ashmem" docs, they have a file-like API. But I think that just means that they conform to the "file descriptor" model. These FDs can be passed from one application to another (across fork / exec), and you use "ioctl" to operate on them. But there is no indication that they implement "read" and "write" ... let alone "seek".

   • Now, you could probably implement a read/write/seekable stream on top of ashmem, using native and Java libraries on both ends of the channel. But both applications would need to be "aware" of this process, probably to the level of providing command line options to set up the channel.

   These issues also apply to old-style shmem ... except that the channel setup is probably more difficult.

3. The other potential option is to use a RAM fs.

   • This is easier to implement. The files in the RAMfs will behave like "normal" files; when opened by an application you get a file descriptor that can be read, written and seeked ... depending on how it was opened. And (I think) you should be able to pass a seekable FD for a RAMfs file across a fork/exec.

   • The problem is that the RAMfs needs to be "mounted" by the operating system in order to use it. While it is mounted, another (privileged) application can also open and read files. And the OS won't let you unmount the RAMfs while some application has open fds for RAMfs files.

   • There is a (hypothetical) scheme that partly mitigates the above.

     1. The source application creates and mounts a "private" RAMfs.
     2. The source application creates/opens the file for read/write and then unlinks it.
     3. The source application writes the file using the fd from the open.
     4. The source application forks / execs the sink application, passing the fd.
     5. The sink application reads from the (I think) still seekable fd, seeking as required.
     6. When the source application notices that the (child) sink application process has exited, it unmounts and destroys the RAMfs.

     This would not require modifying the reading (sink) application.

     However, a third (privileged) application could still potentially get into the RAMfs, locate the unlinked file in memory, and read it.

However, having re-reviewed all of the above, the most practical solution is still to modify the reading (sink) application to read the entire input stream into a byte[], then open a ByteArrayInputStream on the buffered data. The core application can seek and reset it at will.

I believe you're looking for StorageManager.openProxyFileDescriptor, function added in API 26. This will give you ParcelFileDescriptor, needed for your ContentProvider.openAssetFile to work. But you can also grab its file descriptor and use it in file I/O: new FileInputStream(fd.getFileDescriptor())

In function description is :

This can be useful when you want to provide quick access to a large file that isn't backed by a real file on disk, such as a file on a network share, cloud storage service, etc. As an example, you could respond to a ContentResolver#openFileDescriptor(android.net.Uri, String) request by returning a ParcelFileDescriptor created with this method, and then stream the content on-demand as requested. Another useful example might be where you have an encrypted file that you're willing to decrypt on-demand, but where you want to avoid persisting the cleartext version.

It works with ProxyFileDescriptorCallback, which is your function to provide I/O, mainly read pieces of your file from various offsets (or decrypt it, read from network, generate, etc).

As I tested, it's well suited also for video playback over content:// scheme, because seeking is efficient, no seek-by-read as is the option for pipe-based approach, but Android really asks relevant fragments of your file.

Internally Android uses some fuse driver to transfer the data between processes.

It's not a general solution to your problem, but opening a PDF in QuickOffice works for me with the following code (based on your sample):

@Override
public AssetFileDescriptor openAssetFile(Uri uri, String mode) throws FileNotFoundException {
    try {
        byte[] data = getData(uri);
        long size = data.length;
        ParcelFileDescriptor[] pipe = ParcelFileDescriptor.createPipe();
        new TransferThread(new ByteArrayInputStream(data), new AutoCloseOutputStream(pipe[1])).start();
        return new AssetFileDescriptor(pipe[0], 0, size);
    } catch (IOException e) {
        e.printStackTrace();
    }
    return null;
};

private byte[] getData(Uri uri) throws IOException {
    AssetManager assets = getContext().getResources().getAssets();
    InputStream is = assets.open(uri.getLastPathSegment());
    ByteArrayOutputStream os = new ByteArrayOutputStream();
    copy(is, os);
    return os.toByteArray();
}

private void copy(InputStream in, OutputStream out) throws IOException {
    byte[] buf = new byte[1024];
    int len;
    while ((len = in.read(buf)) > 0) {
        out.write(buf, 0, len);
    }
    in.close();
    out.flush();
    out.close();
}

@Override
public Cursor query(Uri url, String[] projection, String selection, String[] selectionArgs, String sort) {
    if (projection == null) {
        projection = new String[] { OpenableColumns.DISPLAY_NAME, OpenableColumns.SIZE };
    }

    String[] cols = new String[projection.length];
    Object[] values = new Object[projection.length];
    int i = 0;
    for (String col : projection) {
        if (OpenableColumns.DISPLAY_NAME.equals(col)) {
            cols[i] = OpenableColumns.DISPLAY_NAME;
            values[i++] = url.getLastPathSegment();
        }
        else if (OpenableColumns.SIZE.equals(col)) {
            cols[i] = OpenableColumns.SIZE;
            values[i++] = AssetFileDescriptor.UNKNOWN_LENGTH;
        }
    }

    cols = copyOf(cols, i);
    values = copyOf(values, i);

    final MatrixCursor cursor = new MatrixCursor(cols, 1);
    cursor.addRow(values);
    return cursor;
}

private String[] copyOf(String[] original, int newLength) {
    final String[] result = new String[newLength];
    System.arraycopy(original, 0, result, 0, newLength);
    return result;
}

private Object[] copyOf(Object[] original, int newLength) {
    final Object[] result = new Object[newLength];
    System.arraycopy(original, 0, result, 0, newLength);
    return result;
}