[feature request] Load a setting file from an external script

[dpellegr]

Hi,

I would like to load a setting file from an application external to the browser (eg a python script) which manages a directory of them. What would be the easiest way to accomplish this?

Many thanks!

[sereneblue]

Hi,

I would like to load a setting file from an application external to the browser (eg a python script) which manages a directory of them. What would be the easiest way to accomplish this?

Many thanks!

Hi @dpellegr,

I'll be writing a detailed guide for developers about this use case because I think it's pretty useful for certain purposes and there's a few people who have asked something similar in the past.

I figure you're probably most interested in switching between browser profiles and any other additional settings would stay consistent? I've been down this route and it works pretty well. In fact, this is somewhat how Chameleon is tested! You can find some examples of NodeJS/Selenium here.

Here's a short TLDR of how it works:

1. In your script, launch Firefox with Chameleon installed.
2. Navigate to this URL: about:debugging#/runtime/this-firefox.
3. Find the Internal UUID of Chameleon using JS.
4. Navigate to moz-extension://INTERNAL_UUID/popup.html (this URL will change in v0.20.0)

Using JS/preferred browser automation library, you should now be able to interact with the popup page. This is the page shown when you click the Chameleon icon.

To load settings go to moz-extension://INTERNAL_UUID/import.html. You can upload your settings on this page.

[dpellegr]

Hi, sorry for the late reply. This is indeed pretty helpful! Thank you!

[dpellegr]

Hi @sereneblue,

I have spent a couple of days trying to find a way to interact with a moz-extension:// page from JS. I also tried to record the commands with selenium, but there seems to be nothing that can be done to interact with internal browser pages apart from using (virtual) mouse and keyboard.

At this point I think that the best course of actions would be to plant a runtime.onConnectExternal somewhere in Chameleon so that commands can be sent straight to it. That would be pretty much a dev-oriented feature, but I think that it could turn out quite handy, allowing to integrate chameleon in other projects.

If you agree, I am happy to follow this up aiming at producing a pull request.

Thank you!

[sereneblue]

@dpellegr

Don't worry, I've got you covered. ;) I wrote a Python script that gets you to the the Chameleon popup page and selects the Random Desktop option:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from time import sleep

firefox_binary = '/usr/bin/firefox' # or firefox-developer-edition
driver = webdriver.Firefox(firefox_binary=firefox_binary, executable_path='./geckodriver')
extension_path = '/full/path/to/chameleon-ext.xpi'
driver.install_addon(extension_path, temporary=True)

driver.get('about:debugging#/runtime/this-firefox')

# wait for extensions to appear
wait = WebDriverWait(driver, 5)
element = wait.until(EC.element_to_be_clickable((By.CLASS_NAME, 'card')))

# internal uuid for extension pages
internal_uuid = driver.execute_script("""
    let extensions = Array.from(document.querySelectorAll('.card'));
    let chameleon = extensions.filter(el => el.querySelector('span').title == 'Chameleon')[0];
    let metadata = Array.from(chameleon.querySelectorAll('.fieldpair__description')).map(e => e.innerText);
    return metadata[2];
""")

POPUP_PAGE = f'moz-extension://{internal_uuid}/popup.html'

# navigate to chameleon popup page
driver.get(POPUP_PAGE)

# select Random Desktop profile option
el = driver.find_element_by_id('profileRandomDesktop')
el.click()

sleep(10)

driver.quit()

I tested with Python 3.8, FF Dev 75 and FF74, and Selenium 3.141. Please let me know if you're still experiencing issues.

[dpellegr]

@sereneblue Thank you! Yep, that worked perfectly. However Selenium is a huge and noisy weapon. One of the requirements of my project is avoiding it. I think that I will look anyway into a small chameleon driver listening on javascript. I'll let you know if it turns out nicely.

[sereneblue]

@dpellegr What library/tool are you using?

[dpellegr]

For learning/fun I wrote my own browser extension which:

1. Includes a native app which listen to an IPC socket (Linux) and forwards the messages to background.js.
2. Does a bit of routing in the background.js sending various requests either to the tabs or to other extensions.
3. Sends the responses back to the native app which forwards them on the IPC socket.

As the socket is language agnostic, this allow any program to communicate with the internals of the browser in a transparent and pretty stealth way.

Anyway, I have been looking in the chameleon source code (v0.20 yay!!) and so far with about 10 lines, I managed send messages to the browser.runtime.onMessage.addListener callback in background.ts. From there I should be able to do pretty much anything, right?

BTW should I test on the alpha2 release or on the develop branch?

[sereneblue]

Interesting, I've explored doing something similar for the about:config checklist. Unfortunately, you can't change the preferences file while Firefox is in use.

You should check out the src/popup/App.vue and src/store/actions.ts files to learn more about messaging and to see how it all works. The background.ts file does contain everything you'd need to control Chameleon.

You should test on the develop branch. It usually will contain some fixes/features that aren't in the latest dev build.

[dpellegr]

Everything looks great from here! These are the edits that I introduced:

develop...dpellegr:develop

Then in my background.js I just do:

var port = browser.runtime.connectNative("MyNativeApp");

/*
Listen for messages from the app.
*/
port.onMessage.addListener( (req) => {
  console.log("Received request: " + req.target +" "+ req.action);
  if ( req.target === "tab" ) {
    if ( req.action === "get-src" ) {
      getPageSource(port.postMessage);
    }
  } else
  if ( req.target === "chameleon" ) {
    var chameleonPort = browser.runtime.connect(
      "{3579f63b-d8ee-424f-bbb6-6d0ce3285e6a}"
    );
    chameleonPort.onMessage.addListener((resp) => {port.postMessage(resp);});
    chameleonPort.postMessage(req);
  }
});

Clearly one needs to know how to shape the requests, but for just importing the settings (as I originally desired) that is pretty straightforward.

[sereneblue]

@dpellegr Glad to see you've got everything sorted out.

[dpellegr]

Do you think that you could consider pulling my additions to background.ts into the baseline? I know that very few people will profit from it, but for me maintaining my modded fork (including packaging and signing) would be a bit daunting.

An important step in this direction could be adding a GUI entry to set the authorized variable. For the time being, empty means that anyone can communicate with chameleon, while one may want to add just a few extension ids.

[sereneblue]

@dpellegr

Sure, I'll consider it. I'll be writing a developer guide soon so I'm happy to take a look to see if this is something that can also be added to Chameleon.

Personally, I'd prefer developers control Chameleon using the popup page since it provides consistency across libraries like Selenium/Puppeteer/etc. This seems to be an interesting use case though. I don't mind adding it to the dev builds of Chameleon but I'll have to get a bit familiar with the workflow first.

[dpellegr]

Great! Selenium is an excellent tool but many websites frown upon it. This method instead is pretty much undetectable in addition to being more slender and direct.
Anyway the basics are presented here: #301 (comment) but if you have questions don't hesitate to ask.
I would also be happy to contribute to the guide!

[sereneblue]

Thanks. Once I'm done with the developer's guide, feel free to check it out and if there's any improvements that can be made, feel free to submit a pull request. The wiki is located in a branch here.

Selenium is convenient because there's usually a library in every language you'd want to use, but I'd reach for Puppeteer/Playwright instead.They're very good alternatives that avoid the issues brought with Selenium. You may find this blog post useful.

[sereneblue]

@dpellegr I've added some code that should allow the dev build to be controlled by another extension here. Please let me know if there are any issues. I've also updated the developer guide.

[dpellegr]

Wonderful! It looks perfectly integrated.

The only issue that I see is that no confirmation is sent back, indeed on my side I wait for that to proceed.

This:

chameleon/src/lib/chameleon.ts

Line 811 in 6a61610

browser.runtime.sendMessage(request);

should probably become:
browser.runtime.sendMessage(request, null, port.postMessage);

Then here:

chameleon/src/background.ts

Line 24 in 6a61610

browser.runtime.onMessage.addListener((request: any, sender: any, sendResponse: any) => {

You should make sure that all the actions that could be sent by another extension (or at least the ones exemplified in the wiki) invoke the callback, even if in a silly way such as sendResponse("done")

[sereneblue]

@dpellegr Thanks! I've updated the above functions based on your suggestions.

[dpellegr]

Hi sereneblue,

Do you think that a dev build is suitable for being published on https://addons.mozilla.org ?
I am asking because I need a signed xpi, so for me it keeping it private or publish is the very same effort, but you may be against this or prefer to publish it with your account in order to better keep it updated in the future...

Thank you in advance for letting me know!

[sereneblue]

Hi @dpellegr,

The dev builds on the releases page are signed.

[dpellegr]

Great, thank you!

…

On Fri, May 8, 2020, 00:05 sereneblue ***@***.***> wrote: Hi @dpellegr <https://github.com/dpellegr>, The dev builds on the releases <https://github.com/sereneblue/chameleon/releases> page are signed. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#301 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB3UFLXQEYXXMS6N6U3R6C3RQMWA7ANCNFSM4LNKCRRQ> .

[dpellegr]

Sorry to bother you again, but I have started to migrate from my custom build and unfortunately we are not there yer :(

This line seems wrong:

chameleon/src/lib/chameleon.ts

Line 844 in 3be8d6e

browser.runtime.sendMessage(request, null, port.postMessage);

as calling sendMessage directly (without a message handler) the arguments get completely different meanings: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/runtime/sendMessage

This should be the necessary fix (without the console.log line): master...dpellegr:master

However even with the fix in place, on sendMessage(request) I get the infamous Error: Could not establish connection. Receiving end does not exist. I do not understand its origin. Build, build:dev, unloading, reloading, moving to a (re)fresh non-about: tab... I have been on this for 6 hours, nothing helped.

[sereneblue]

@dpellegr I've updated the dev build with your change. I'll do some testing on my own to see if I can get a connection to Chameleon from an extension.

[dpellegr]

I am observing additional instabilities in the latest releases (I tried 20.3 and 20.4), like the pop-up sometimes disappearing when browsing it. In the console I get messages like:

Error: Could not establish connection. Receiving end does not exist. background.js:7:371108
TypeError: PrecompiledScript.executeInGlobal: Argument 1 is not an object. 3 ExtensionContent.jsm:567:25
Error: Promised response from onMessage listener went out of scope background.js:7:380461

It is really hard to track it down to a specific sequence of actions, but I believe that this could be related to my communication issue.

Also, with previous v20 releases, I remember having few icons reporting the status of the protection at the bottom of the popup home page. Now I do not see them any longer.

[sereneblue]

@dpellegr The fingerprint detection panel has been removed from Chameleon. It caused a number of issues with sites.

I was able to send a message to Chameleon using the following background script:

async function onClick() {
	await browser.runtime.sendMessage(
		'{3579f63b-d8ee-424f-bbb6-6d0ce3285e6a}',
		{
			action: 'updateProfile',
			data: 'win4-ff',
		},
		null
	);

	await browser.runtime.sendMessage(
		'{3579f63b-d8ee-424f-bbb6-6d0ce3285e6a}',
		{
			action: 'implicitSave'
		},
		null
	);
}

browser.browserAction.onClicked.addListener(onClick);

I've rewritten the external listener in 3db7049. Instead of using ports, it's much easier to use runtime.sendMessage to send the message directly to Chameleon. onMessageExternal reuses the same message handler that Chameleon does so there shouldn't be any issues. I've also added a new action implicitSave that saves the settings currently being used. Between that, getSettings, and save, there's a lot of flexibility for settings management. I'll update the docs soon and will also push out a v0.20.5-beta1 release so that you can test with a signed addon.

[dpellegr]

Check these out: https://github.com/dpellegr/RedBlue They are two super simple extensions which I temporary load from about:debugging#/runtime/this-firefox.

When clicking on their icons, Blue sends a message to Red (which receives it properly), in the same way Red sends a message to Chameleon v0.20.5 which for me keeps failing.

BLUE: sending msg background.js:6:11
RED: received message from [email protected] background.js:2:11
RED: message from blue background.js:5:13

RED: sending msg background.js:15:11
Error: Could not establish connection. Receiving end does not exist. background.js:7:373258

Actually it seems to me that the message gets sent properly, but Chameleon internally fails to handle it.

[sereneblue]

@dpellegr Chameleon is actually handling the message. You can verify this by checking the response.

browser.browserAction.onClicked.addListener(async () => {
  // Send a message to chameleon
  console.log('RED: sending msg');
  let response = await browser.runtime.sendMessage(
    '{3579f63b-d8ee-424f-bbb6-6d0ce3285e6a}',
    {
      action: 'updateProfile',
      data: 'win4-ff',
    },
    null
  );
  console.log(response); // done

  await browser.runtime.sendMessage(
    '{3579f63b-d8ee-424f-bbb6-6d0ce3285e6a}',
    {
      action: 'implicitSave'
    },
    null
  );
});

Changes to settings need to be saved if you want the UI to be updated or for settings to be persistent.

Error: Could not establish connection. Receiving end does not exist. background.js:7:373258 was actually caused by the listeners in Chameleon. The error doesn't break functionality. The next dev build will contain a fix.

[dpellegr]

You were right. Everything seems fine from here.
It has been very nice to collaborate with you.
Great thanks!