Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

android intent call and read response #2790

Open
myururdurmaz opened this issue Feb 16, 2022 · 24 comments
Open

android intent call and read response #2790

myururdurmaz opened this issue Feb 16, 2022 · 24 comments
Labels
enhancement New feature or request

Comments

@myururdurmaz
Copy link

Is your feature request related to a problem? Please describe:

on android side require some operations that provide os or other apps like barcode scan, capture video from camera

Is it possible to construct a solution with the existing API?

no as know

Describe the solution you'd like to see:

I want to call android intent and read response

@andydotxyz
Copy link
Member

Currently the only way to do this is to call out using CGo and use the Android NDK.
Open to better suggestions however :)

@MatejMagat305
Copy link
Contributor

at this point I have question: the function "RunOnJVM" is interna,l I understand why, but since move imports from go/src to readonly go/pkg is dificult add to program own ndk code run on your jvm..., will you plan how to solve it?

@andydotxyz
Copy link
Member

That is a good point @MatejMagat305 - this used to be possible but since we "inlined" the go-mobile code it isn't any more.
We should resolve this in some super clever way for v2.2.0 in the coming month(s).

@andydotxyz andydotxyz added blocker Items that would block a forthcoming release enhancement New feature or request labels Feb 19, 2022
@andydotxyz
Copy link
Member

This is too much work to do fast, and not really a blocker. So de-prioritising so we can get 2.2 out then move on with things like this.

@andydotxyz andydotxyz removed the blocker Items that would block a forthcoming release label May 6, 2022
@MatejMagat305
Copy link
Contributor

MatejMagat305 commented May 7, 2022

hi, I do not want to be rude or annoying (with silly pull request), and I understand why you de-prioritising this, but I have 2 solutions with relativly small modifications:

  1. add to package fyne/app: https://github.com/MatejMagat305/fyne/pull/3/files - only 23 lines of code,
  2. add small package for this: https://github.com/MatejMagat305/fyne/pull/2/files - althought it is 43 lines of code - it run func on other platform

the main deference is location of new code, but it is irelevant where it will be, I want pay attantion that the first one return not implemented error the second solution run function on other platform than android ...... (if my changes will be relevant I will create pull request)

@andydotxyz
Copy link
Member

Thank you for trying to help with this. Unfortunately this new API is not easy to understand - the parameters required cannot be discovered from the API usage. We have avoided using interface{} as much as possible for this reason. Also we try to avoid package level functions, preferring methods in interfaces so that the implementation can be provided by the platform and makes testing much easier.

@andydotxyz
Copy link
Member

As with so many things this is not technically very difficult, but designing the correct API to work for all platforms is very difficult - the semantics very for Android/iOS and others.

@andydotxyz
Copy link
Member

I have some work on a proposal that could help here fyne-io/proposals#7

@phemmer
Copy link

phemmer commented Jun 18, 2023

Any further update on this?
I am investigating using Fyne, and also need to be able to fire off an intent and wait for the result. It looks like RunNative() should indeed be able to fire it off. But startActivityForResult() calls onActivityResult() when finished, and while this function currently exists, it only handles a few hard-coded codes, and doesn't allow any sort of external registration.

It doesn't seem like iOS has any such equivalent, so this may just need to be a new method on AndroidContext. A simple API that makes sense to me is something where you pass a channel (that receives a struct with requestCode, resultCode, & data), the method allocates a requestCode for you (to avoid conflict with built-in codes like FILE_OPEN_CODE), and returns it. Plus a method to de-register.
Alternative would be for the method to allocate & return the channel as well. For most use cases there's not a whole lot of difference. But for some it could matter (user might want to use the same channel for multiple codes).

@andydotxyz
Copy link
Member

The Android "RunNative" from the proposal was released in 2.3. There is no particular API added for activity interaction but if you can use just NDK or the JNI it is now technically possible.

@phemmer
Copy link

phemmer commented Jun 18, 2023

Can you clarify. Because as I mentioned, onActivityResult() already exists, and does not support registering codes. So how is one supposed to be able to receive the callback? Are we supposed to create our own GoNativeActivity that replaces the one provided by Fyne?

@andydotxyz
Copy link
Member

I was not explaining how to do any specific thing. I was providing an update on the ticket as you requested - as I noticed the progress had not been reported here.

No, the API we have does not allow you to intercept or override how the bootstrapping application works.

@phemmer
Copy link

phemmer commented Jun 23, 2023

Sorry for the misunderstanding. When you said "if you can use just NDK or the JNI it is now technically possible." I interpreted that as saying the requested functionality this issue is for was technically possible. It sounded like the issue was considered resolved and that there was no further work to be done. I'm aware RunNative is implemented, and though the info is appreciated, I was more curious about the planned remaining work. Thanks

@andydotxyz
Copy link
Member

I was more curious about the planned remaining work.

I don't know what we would have to do to make this possible, but I do know that it has not been scheduled and I don't know of anyone working on it at this time.

@hkparker
Copy link
Contributor

I came across this issue while looking for a way to call ACTION_REQUEST_IGNORE_BATTERY_OPTIMIZATIONS, since my use case involves needing to maintain long running sockets in the background (I'll also probably want to find a way to call this function).

It's a little unclear to me what the state of this is. Is it correct to say that I can use #3385 to, in theory, interact with android to directly start an activity or call a function, but that there are no abstractions in Fyne yet to represent an android activity / intent?

@andydotxyz
Copy link
Member

Is it correct to say that I can use #3385 to, in theory, interact with android to directly start an activity or call a function, but that there are no abstractions in Fyne yet to represent an android activity / intent?

Yes. You can see it in action at https://fynelabs.com/2024/03/01/running-native-android-code-in-a-fyne-app/

@hkparker
Copy link
Contributor

I've been playing around with this (big thanks for the example up here: https://github.com/fynelabs/devicename). I'm not too familiar with the android subsystems tbh but I've come across an issue where I appear to be using the wrong context for a call (based on JNI DETECTED ERROR IN APPLICATION: can't call java.lang.Object android.app.Activity.getSystemService(java.lang.String) on instance of android.app.Application. Based on what I've been reading I think I need to be in a native activity's android_app struct, but I'm not entirely sure. Working off someone else's example, they are able to call Context->App->activity->clazz, and I don't have that Context yet.

The context I'm using comes from using the JNI to call getApplication on currentActivityThread. Perhaps I'm supposed to be using the AndroidContext.Ctx that gets passed into C as a uintptr_t, but in that case I'm not sure how to cast that into the appropriate stuct.

Anyway, posting this here just in case anyone else has run into a similar thing. A bit difficult to describe without sharing a lot of code context, I'm working out of a fork of that example though so perhaps I can push that up if there's interest.

@andydotxyz
Copy link
Member

The error says that you are calling an Activity method on an Application instance - so check that you are passing the right object to the method execution.

@hkparker
Copy link
Contributor

hkparker commented Oct 19, 2024

Yes, I should be passing my activity context but I don't know how to get my activity context from the JNI yet. This isn't really a fyne problem (though I was curious if this is something Fyne provides on the AndroidContext object), so I'll just keep looking around and getting more familiar with android, see if I can't figure out how to get a reference to that.

Progress at least: JNI DETECTED ERROR IN APPLICATION: can't call android.content.Context android.content.ContextWrapper.getApplicationContext() on instance of java.lang.Class<android.app.Activity>

@hkparker
Copy link
Contributor

Alright, sorry to continue to be a bit off topic here, but I did figure this out and it might be helpful to others who get stuck and stumble onto this issue. I learned from looking at the internal android driver code in Fyne that I was right that the ctx that gets passed around is the android activity context that I needed for my call, and that I could cast it into a jobject with just a simple (jobject)ctx.

So with that, here's some quick and dirty code that should allow you to interact with the power manager using the Fyne run native example :)

const char *batteryOptimizationsIgnored(uintptr_t java_vm, uintptr_t jni_env, uintptr_t ctx) {
	JNIEnv *env = (JNIEnv*)jni_env;

        jclass classNativeActivity = (*env)->FindClass(env, "android/app/NativeActivity");
        jclass classPowerManager = (*env)->FindClass(env, "android/os/PowerManager");

	jmethodID idNativeActivity_getAppContext = (*env)->GetMethodID(env, classNativeActivity, "getApplicationContext", "()Landroid/content/Context;");

        jmethodID idNativeActivity_getSystemService = (*env)->GetMethodID(env, classNativeActivity, "getSystemService", "(Ljava/lang/String;)Ljava/lang/Object;");
        jstring POWER_SERVICE = (*env)->NewStringUTF(env, "power");
        jobject PowerManager = (*env)->CallObjectMethod(env, (jobject)ctx, idNativeActivity_getSystemService, POWER_SERVICE);

	jmethodID idPowerManager_isIgnoringBatteryOptimizations = (*env)->GetMethodID(env, classPowerManager, "isIgnoringBatteryOptimizations", "(Ljava/lang/String;)Z");
        jstring jPackageName = (*env)->NewStringUTF(env, "fynelabs.devicename");
        jboolean isIgnoredBool = (*env)->CallBooleanMethod(env, PowerManager, idPowerManager_isIgnoringBatteryOptimizations, jPackageName);

        (*env)->DeleteLocalRef(env, jPackageName);
        (*env)->DeleteLocalRef(env, POWER_SERVICE);

	if (isIgnoredBool == JNI_TRUE) {
		return "ignored";
	}
	return "not ignored";
}

@andydotxyz
Copy link
Member

Sweet, thanks for sharing. I guess that doesn't quite resolve the headline issue title though right? I think calling and listening to intent calls will be more complex?

@hkparker
Copy link
Contributor

Not yet. I'm currently working on calling an intent (the ignore_battery_optimization action I mentioned earlier) and think I'll have something to share soon there, using the JNI like I did for the power manager stuff. I'm not sure how listening would work, and I don't think I'll need to figure that out for my purposes, but hopefully I can share a working example of calling an intent at least, and that can be a starting point for building something more generalized.

@hkparker
Copy link
Contributor

Alright, here's some code successfully starting an intent via the JNI in Fyne:

const char *requestIgnoreBatteryOptimizations(uintptr_t java_vm, uintptr_t jni_env, uintptr_t ctx) {
	JNIEnv *env = (JNIEnv*)jni_env;

        jclass intentClass = (*env)->FindClass(env, "android/content/Intent");
        jmethodID newIntent = (*env)->GetMethodID(env, intentClass, "<init>", "()V");
        jobject intent = (*env)->NewObject(env, intentClass, newIntent);

        jmethodID methodSetAction = (*env)->GetMethodID(env, intentClass, "setAction", "(Ljava/lang/String;)Landroid/content/Intent;");
	jstring actionString = (*env)->NewStringUTF(env, "android.settings.REQUEST_IGNORE_BATTERY_OPTIMIZATIONS");
        jobject intentActivity = (*env)->CallObjectMethod(env, intent, methodSetAction, actionString);

	jclass uriClass = (*env)->FindClass(env, "android/net/Uri");
	jmethodID uriParse = (*env)->GetStaticMethodID(env, uriClass, "parse", "(Ljava/lang/String;)Landroid/net/Uri;");
	jstring packageName = (*env)->NewStringUTF(env, "package:devicename.fynelabs");
	jobject uri = (*env)->CallStaticObjectMethod(env, uriClass, uriParse, packageName);

        jmethodID methodSetData = (*env)->GetMethodID(env, intentClass, "setData", "(Landroid/net/Uri;)Landroid/content/Intent;");
	(*env)->CallObjectMethod(env, intent, methodSetData, uri);

	jclass contextClass = (*env)->GetObjectClass(env, (jobject)ctx);
        jmethodID startActivityMethodId = (*env)->GetMethodID(env, contextClass, "startActivity", "(Landroid/content/Intent;)V");
        (*env)->CallVoidMethod(env, (jobject)ctx, startActivityMethodId, intentActivity);
	return "";
}

pretty rough example as before, since I haven't done any error checking or made sure the naming scheme was consistent yet. But it works!

As far as getting the result of an activity, I'm not sure, that looks a bit more complicated. But hopefully this serves as a little proof of concept for how intents can be fired off in Fyne.

@hkparker
Copy link
Contributor

Reading an intent response might require modifying onActivityResult, and having Fyne users be able to declare intents they want to listen for, and having Fyne store those entries and matching them and calling the user defined code.

Something like

myApp.HandleIntentResponse(requestCode, resultCode int, func(Intent data) { 
        // Fyne got an intent response that matched my request and result codes, now I can act on the data...
})

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants