Unable to create new session using a custom profile

[soosrol]

System

• Version: 0.19.1
• Platform: Windows 10
• Firefox: 57.0
• Selenium: 3.7.1 (Java)

Testcase

1. Set up Selenium hub + node
2. Prepare Firefox profile on the node (optionally install an extension)
3. Create a browser on the grid with passing in the args to load the profile from local file system:

FirefoxOptions fo = new FirefoxOptions();
fo.addArguments("-profile", "./firefoxprofile");
DesiredCapabilities c = DesiredCapabilities.firefox();
c.setCapability(FirefoxOptions.FIREFOX_OPTIONS, fo);
RemoteWebDriver wd = new RemoteWebDriver(hubUrl, c);

Expected result

1. Browser opened
2. Profile is loaded from the node's local file system
3. Session is created
4. Browser can be interacted with using the RemoteWebDriver object from the tests

Actual result

1. Browser opened
2. Profile is loaded from the node's local file system
3. RemoteWebDriver.createSession() never returns, session is not created

Notes

• The Firefox profile was created on the same node, with the same version of Firefox
• There is only one extension installed, but I tried it without that, same result
• Starting Firefox through selenium grid works as expected without fo.addArguments("-profile", "./firefoxprofile");
• Firefox does not hang when started, it can be interacted with manually.
• Launching Firefox with firefox.exe -profile "path/to/profile" also works.

Stacktrace

10:50:23.300 INFO - Got a request to create a new session: Capabilities {acceptInsecureCerts: true, androidServerPort: 7100, appInstall: true, browserName: firefox, moz:firefoxOptions: {args: [-profile, ./firefoxprofile], log: {level: trace}, prefs: {}}, serverPort: 5769, uuid: f39b5203-64ca-4b0f-b4c2-b0c..., version: }
10:50:23.300 INFO - Trying to create a new session on test slot {firefox_binary=C:\Program Files\Mozilla Firefox\firefox.exe, seleniumProtocol=WebDriver, browserName=firefox, maxInstances=1, platform=WIN10}
11:00:23.363 INFO - Error forwarding the new session new session request for webdriver should contain a location header or an 'application/json;charset=UTF-8' response body with the session ID.
org.openqa.grid.common.exception.GridException: new session request for webdriver should contain a location header or an 'application/json;charset=UTF-8' response body with the session ID.
        at org.openqa.grid.internal.TestSession.updateHubIfNewWebDriverSession(TestSession.java:426)
        at org.openqa.grid.internal.TestSession.forward(TestSession.java:248)
        at org.openqa.grid.web.servlet.handler.RequestHandler.forwardNewSessionRequestAndUpdateRegistry(RequestHandler.java:90)
        at org.openqa.grid.web.servlet.handler.RequestHandler.process(RequestHandler.java:113)
        at org.openqa.grid.web.servlet.DriverServlet.process(DriverServlet.java:84)
        at org.openqa.grid.web.servlet.DriverServlet.doPost(DriverServlet.java:68)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
        at org.seleniumhq.jetty9.servlet.ServletHolder.handle(ServletHolder.java:841)
        at org.seleniumhq.jetty9.servlet.ServletHandler.doHandle(ServletHandler.java:535)
        at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
        at org.seleniumhq.jetty9.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
        at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
        at org.seleniumhq.jetty9.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
        at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
        at org.seleniumhq.jetty9.servlet.ServletHandler.doScope(ServletHandler.java:473)
        at org.seleniumhq.jetty9.server.session.SessionHandler.doScope(SessionHandler.java:1564)
        at org.seleniumhq.jetty9.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
        at org.seleniumhq.jetty9.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
        at org.seleniumhq.jetty9.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.seleniumhq.jetty9.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
        at org.seleniumhq.jetty9.server.Server.handle(Server.java:564)
        at org.seleniumhq.jetty9.server.HttpChannel.handle(HttpChannel.java:317)
        at org.seleniumhq.jetty9.server.HttpConnection.onFillable(HttpConnection.java:251)
        at org.seleniumhq.jetty9.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
        at org.seleniumhq.jetty9.io.FillInterest.fillable(FillInterest.java:110)
        at org.seleniumhq.jetty9.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
        at org.seleniumhq.jetty9.util.thread.Invocable.invokePreferred(Invocable.java:128)
        at org.seleniumhq.jetty9.util.thread.Invocable$InvocableExecutor.invoke(Invocable.java:222)
        at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:294)
        at org.seleniumhq.jetty9.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:199)
        at org.seleniumhq.jetty9.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672)
        at org.seleniumhq.jetty9.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590)
        at java.lang.Thread.run(Unknown Source)

Trace-level log

C:\SeleniumGrid>java -Dwebdriver.chrome.driver="./chromedriver.exe" -Dwebdriver.gecko.driver="./geckodriver.exe" -jar selenium-server-standalone.jar -role node -hub http://lpselgrid3001.noclab.local:4444/grid/register/ -nodeConfig nodeconfig.json
10:49:54.363 INFO - Selenium build info: version: '3.7.1', revision: '8a0099a'
10:49:54.363 INFO - Launching a Selenium Grid node
2017-11-15 10:49:55.050:INFO::main: Logging initialized @1133ms to org.seleniumhq.jetty9.util.log.StdErrLog
10:49:55.113 INFO - Driver class not found: com.opera.core.systems.OperaDriver
10:49:55.128 INFO - Driver provider class org.openqa.selenium.safari.SafariDriver registration is skipped:
 registration capabilities Capabilities {browserName: safari, platform: MAC, version: } does not match the current platform WIN10
10:49:55.175 INFO - Using the passthrough mode handler
2017-11-15 10:49:55.206:INFO:osjs.Server:main: jetty-9.4.5.v20170502
2017-11-15 10:49:55.222:WARN:osjs.SecurityHandler:main: [email protected]@1c9d8e2{/,null,STARTING} has uncovered http methods for path: /
2017-11-15 10:49:55.238:INFO:osjsh.ContextHandler:main: Started o.s.j.s.ServletContextHandler@1c9d8e2{/,null,AVAILABLE}
2017-11-15 10:49:55.253:INFO:osjs.AbstractConnector:main: Started ServerConnector@13eba10{HTTP/1.1,[http/1.1]}{0.0.0.0:5555}
2017-11-15 10:49:55.253:INFO:osjs.Server:main: Started @1331ms
10:49:55.253 INFO - Selenium Grid node is up and ready to register to the hub
10:49:55.253 INFO - Starting auto registration thread. Will try to register every 5000 ms.
10:49:55.253 INFO - Registering the node to the hub: http://lpselgrid3001.noclab.local:4444/grid/register
10:49:55.363 INFO - The node is registered to the hub and ready to use
2017-11-15 10:50:23.331:INFO:osjshC.ROOT:qtp25671582-16: org.openqa.selenium.remote.server.WebDriverServlet-691363: Initialising WebDriverServlet
10:50:23.347 INFO - Found handler: org.openqa.selenium.remote.server.commandhandler.BeginSession@b3462b
10:50:23.347 INFO - /session: Executing POST on /session (handler: BeginSession)
10:50:23.425 INFO - Capabilities are: Capabilities {acceptInsecureCerts: true, androidServerPort: 7100, appInstall: true, browserName: firefox, firefox_binary: C:\Program Files\Mozilla Fi..., moz:firefoxOptions: {args: [-profile, ./firefoxprofile], binary: C:\Program Files\Mozilla Fi..., log: {level: trace}, prefs: {}}, serverPort: 5769, uuid: f39b5203-64ca-4b0f-b4c2-b0c..., version: }
10:50:23.441 INFO - Capabilities {acceptInsecureCerts: true, androidServerPort: 7100, appInstall: true, browserName: firefox, firefox_binary: C:\Program Files\Mozilla Fi..., moz:firefoxOptions: {args: [-profile, ./firefoxprofile], binary: C:\Program Files\Mozilla Fi..., log: {level: trace}, prefs: {}}, serverPort: 5769, uuid: f39b5203-64ca-4b0f-b4c2-b0c..., version: } matched class org.openqa.selenium.remote.server.ServicedSession$Factory (provider: org.openqa.selenium.firefox.GeckoDriverService)
1510771823519   geckodriver     INFO    geckodriver 0.19.1
1510771823519   geckodriver     INFO    Listening on 127.0.0.1:35467
1510771824050   mozrunner::runner       INFO    Running command: "C:\\Program Files\\Mozilla Firefox\\firefox.exe" "-marionette" "-profile" "./firefoxprofile"
1510771826113   geckodriver::marionette TRACE     connection attempt 0/600
1510771828253   geckodriver::marionette TRACE     connection attempt 1/600
1510771830410   geckodriver::marionette TRACE     connection attempt 2/600
1510771832550   geckodriver::marionette TRACE     connection attempt 3/600
[...]

[whimboo]

So when you log into the node, can you see that Firefox is getting started at all? I would expect at least the Firefox log to be present in the trace log. I assume you followed https://firefox-source-docs.mozilla.org/testing/geckodriver/geckodriver/TraceLogs.html?

[andreastt]

I am also surprised to not find any stdout output from Firefox after the Running command: … log line. If you see connection attempt going up to 600 and then timing out, it means geckodriver is either unable to launch Firefox or that it is unable to connect to the Marionette remote protocol in the Firefox that gets started.

[whimboo]

@soosrol, does Firefox in those failing cases show the robot in the location bar? Also have a look at the browser console (tools menu) if there are some errors reported for Marionette.

Regarding the logs, usually all output should (for the time being) end up in the same log file. But I don't know anything about the remove driver to give a qualified answer in how it works.

[soosrol]

@whimboo, yes, it has the robot in the address bar. Also checked the browser console, and as far as I can tell, it doesn't have any relevant errors. See attached screenshots.

[2mf]

@soosrol I'm loading a profile in a few cases but most cases do not use profiles at all.
What I'm doing is running Firefox headless or within Xvfb inside a docker container.

[whimboo]

@soosrol interesting! It reminds me on a case we occasionally see in our automated tests but which is so seldom that I wasn't able to hit it so far.

Is that reproducible for you all the time? Does it also happen when you are not creating a new profile but let geckodriver do it? I would like to know what the minimal testcase is for you.

Also I would like to see a real trace level log. Would be fantastic if you can find out where this is stored on the hub.

Btw. is that a new issue you are facing with the 57 release, or did it already exist with 56?

[soosrol]

@whimboo I can reproduce it every time. Everything works flawlessly if I don't specify a profile to load. This has been the same ever since I first tried it 2-3 Firefox versions ago and since Selenium 3.4 and geckodriver 17 or so...

What's really strange, is that I only encounter this issue if I pass in the command line arg for loading the profile from local drive. If I generate/load the profile within my tests by using

FirefoxProfile profile = new FirefoxProfile(new File("path/to/profile"));
FirefoxOptions options = new FirefoxOptions();
options.setProfile(profile);
capabilities.setCapability(FirefoxOptions.FIREFOX_OPTIONS, options);

the profile gets loaded and and the session is started without an issue.

You may ask why I'm not doing it this way, and the answer is simple: I am running my tests on Selenium grid on multiple threads and this would require me to have selenium transfer the profile upon every browser start/restart. My custom profile contains a browser extension, that is ~20MB in size. Having a profile of this size transferred by selenium grid is not just very slow, but also very fragile, as after 2-3 tries I always get Java heap space exception, regardless of the allowed memory in JVM settings.

[whimboo]

Thank you for the update @soosrol. It looks great to have a reduced testcase now. One thing you could help us with is, can you please try if you can get the same issue reproduced when not running the test on the hub but locally? Does that produce the same failure?

Otherwise, @andreastt do you have an idea? Without having trace logs for that problem, I'm meanwhile out of ideas.

[soosrol]

@whimboo: I've tried running the same comparison (starting with/without the profile arg) on my local machine and I got the exact same results. I think this rules out most of the environmental causes, as my local machine is a Mac, unlike my grid that runs on Windows VMs.

See attached logs for both sessions:
Without profile arg: good_ff.log
With profile arg: bad_ff.log

Selenium version: 3.7.1
Geckodriver version: 19.1
Firefox version: 57.0 (64-bit)

[whimboo]

Amazing to see. It means we are getting closer! Given those results I took the opportunity and had a look myself. Here the results...

When trying to use an already existent profile the profiles key in moz:firefoxOptions cannot be used. It indeed only supports base64 encoded profile data. But as our documentation says: To have geckodriver pick up an existing profile on the filesystem, please set the args field to {"args": ["-profile", "/path/to/your/profile"]}

So I tried to use args and passed-in the path to a specific profile directory. Firefox launches, by using this profile but we are not able to make any connection to Marionette even it is running. After the connection failure the test quits without killing the Firefox instance. I will continue diving deeper into that...

@soosrol can you please also test the args key. Do you get the correct profile loaded? Or do you have the same connection problems?

[whimboo]

The problem with not being able to make a connection to Marionette for the given profile path is, that by default Marionette uses port 2828, but geckodriver takes a random free port. Right now I do not see a way to allow pass-in a fixed port to geckodriver when not starting it manually. Checking the code I can only see that the command line argument --marionette-port gets evaluated.

So whether we need a capability under moz:firefoxOptions to let users specify a port for Marionette, or geckodriver would have to update the prefs.js file first with the wanted port before starting Firefox. @andreastt what do you think?

[soosrol]

@whimboo: Yes, the args works, that is how I am loading the profile. The correct profile gets loaded both on Windows/Mac, regardless if I do it with the grid or locally.

In my opinion you may have just found the root cause with the marionette port!

When I don't specify the profile and let it be generated for the session, I can see on about:config that the port was set via prefs.js. However, when loading my profile with -profile, it shows as default just like you said.

At this point I'm trying to see if I can somehow force the geckodriver --marionette-port command line parameter from either the code or some config.

[whimboo]

I will file a bug soon so that it can be fixed.

As workaround you could start geckodriver manually by passing in --marionette-port 2828 or whatever you have in the profile setup. Sadly there is no other way possible right now.

[whimboo]

Bug 1421766 is now taking care of it.

Thank you for reporting! We really wonder why no-one else has seen this yet.

[stewart-r]

Any chance someone could clarify the workaround steps for this? I'm using the selenium standalone-firefox-debug docker container and selenium remote webdriver on .NET

I tried specifying --marionette-port 2828 in options.AddArgument and can see from the container logs that it is being passed to the CLI:

1512734718509   mozrunner::runner       INFO    Running command: "/usr/bin/firefox" "-marionette" "--profile /home/seluser/.mozilla/firefox/5vgqsnyv.default/" "--marionette-port 2828"

but I still have the same behaviour. I'm not at all sure how to check which port is in the profile setup? :-/ I'm not too familiar with selenium or geckodriver and feel like I am missing the point.

[stewart-r]

Ah i understand now... In case anyone else was struggling in the same way as I was the --marionette-port command line option is applied when starting geckodriver not starting firefox.

So workaround steps are:

1. rename /usr/bin/geckodriver to /usr/bin/geckodriver2
2. use @soosrol 's script to point to the renamed executable file and save this script in the original /usr/bin/geckodriver location

Many thanks for the workaround, guys! All back up and running now :-)

[baptx]

@dncpax in case it helps, the workaround I used is to create a custom WebExtension and load it with Selenium WebDriver instead of loading a whole Firefox profile, for example to have cookies before doing an HTTP request: SeleniumHQ/selenium#8693 (comment)
The advantage is that it will be more lightweight, you only load what you need. But profile support could be useful also. Without profile support, if you need to save cookies / localStorage / sessionStorage, you would need to to it manually (writing and reading data in a file).

Edit: I also don't know what is @soosrol 's script mentioned by @stewart-r at #1058 (comment), more information are welcome.

[dncpax]

@baptx thanks for replying. My goal is to login automatically so Im using a profile for that which has autoauth extension installed and configured. With this bug I'm stuck. I also use selenium-side-runner to avoid scripting tests. So I'm really stuck... This is such an old bug...

[baptx]

@dncpax the workaround I use should work for you also, if you load the WebExtension instead of the profile (and write / read session ID stored in e.g. cookies to a file if you want to stay logged in next time you execute the script).

[dncpax]

@baptx that would be wonderful... but I'm working with windows auth... that's the only reason I need a profile and the autoauth extension. Would this work with a webextension?

[hornschorsch]

A fix for this problem is above in this thread. No idea why it isn't implemented.

[dncpax]

@hornschorsch same here... I know everyone does what he/she can, but someone posted 10lines of working code, and somehow it wasn't picked up... not to rant or anything...

[baptx]

@dncpax What do you mean by windows auth? It is probably possible to do it without profile if you write and read profile data to a file. Where is the 10 lines of working code exactly?

[dncpax]

windows integrated authentication...

[globalcitizen]

Verified

This bug has been open for 4 years.

Still seeing this issue on geckodriver-0.30.0 with selenium-webdriver-4.0.3 .

• Same desire (use a prior session with cookies).
• Same result (selenium never gains control of the browser, but otherwise the session works fine).

Further investigation

• Only occurs when passing the existing profile with -P <profilename>
• Does not occur when passing the existing profile with -p </path/to/profile>

Fix

• Pass your profile with -p </path/to/profile> instead of -P <profile_name>

[saifahn]

@globalcitizen Does using -p over -P actually use the profile correctly? Thanks :)

[globalcitizen]

@saifahn Sorry test environment long since destroyed.

[ToxicFrog]

@globalcitizen Does using -p over -P actually use the profile correctly? Thanks :)

I've experimented with this and it does not appear to -- -p and -P are aliases for each other, --profile has no short equivalent. So when running with -p path/to/profile/dir, it looks for a profile with that name, doesn't find one, and ends up running in an ephemeral profile instead -- you can verify this by running it in non-headless mode and looking at about:profiles.

If you use -P profile-name, -p profile-name, or --profile path/to/profile, it loads the profile fine, but port autonegotiation doesn't work, as discussed above.

[whimboo]

Can someone who still sees this problem please provide the code that you are using with Selenium? When using a custom profile you will have to make sure to instruct geckodriver to use the appropriate port for Marionette which should be passed to FirefoxDriver() (at least for the Python binding) as service_args.

[ToxicFrog]

@whimboo here's how I'm using it, via etaoin:

(wd/firefox
   {:args ["-P" "test.profile"]
    :args-driver ["--log" "trace" "--marionette-port" port]
    :log-stdout "/dev/tty"
    :log-stderr "/dev/tty"
    :size [1920 2160]
    :path-browser firefox-bin
    :locator "css selector"})

Note that by "doesn't work" earlier, I don't mean "it doesn't load the profile", I mean "it loads the profile but doesn't autonegotiate a port to use for Marionette, so unless you manually set one in prefs.js or user.js ahead of time, it'll always pick 2828 and you can only have one instance running at a time".

In the above code, I've constructed an appropriate user.js by hand and port is passed in via the command line, but this doesn't scale; absent a fix for this, I'm probably going to have to write something that figures out where the profile directory is, selects a port, and writes an appropriate user.js before launching geckodriver with --marionette-port (hoping that nothing else has stolen that port in the intervening time).

[whimboo]

@ToxicFrog thank you for the details. For now I don't have a proper solution yet, but I can say that we are working on it. But please bear with us that it might take some time to finish it. For now here a workaround:

1. Prepare your profile with the preference marionette.port set to 0.
2. Let your automation start Firefox directly without using geckodriver by passing --marionette --profile %profile_path% and other arguments if needed
3. Read the file %profile_path%/MarionetteActivePort for the port Marionette is using (available since Firefox 95)
4. Use geckodriver with --connect-existing --marionette-port %port% to connect geckodriver to Marionette

We are aware that this is not optimal but at least a start so that scaling won't be a problem for you anymore. With the above steps each instance of Firefox and Marionette will pick up a free and unique system port.

Ongoing work can be tracked via: https://bugzilla.mozilla.org/show_bug.cgi?id=1240830#c16 and ff. For now only step 1 has been done.

[whimboo]

Good news. Recently we landed a patch for geckodriver which by default would allow Marionette to use a system allocated port, and geckodriver then reads that port from the Firefox profile. Anyone with the problem on this issue could you please check this Nightly build of geckodriver. Just click the green gd-s job for your platform, and download the geckodriver.tar.gz or geckodriver.zip from by clicking the link in the lower pane.

When testing please don't force a Marionette port via the --marionette-port argument of geckodriver because that would turn off the new code path. Also you have to make sure to at least use a Firefox 95 release.

Please let us know if it works or not. If it's the latter please attach a trace log. Thanks!

[ToxicFrog]

I've been otherwise occupied this week but I'll try that out next time I'm tinkering with this, which should hopefully be soon. Thanks!

[ToxicFrog]

No luck.

In short: geckodriver starts up, and starts firefox; firefox does not write the MarionetteActivePort file, and geckodriver spins endlessly waiting for it to appear until its parent process watchdogs and kills it.

The runtime profile directory it's looking for in /run/user does exist.
The profile has a blank user.js in the firefox config directory, and the user.js in /run/user has marionette.port set to 0.

It looks like geckodriver is doing the right thing but FF isn't writing the MarionetteActivePort file; any advice on next steps?

1644624687959   geckodriver     INFO    Listening on 127.0.0.1:45335
1644624688072   webdriver::server       DEBUG   -> POST /session {"desiredCapabilities":{"loggingPrefs":{"browser":"ALL"},"moz:firefoxOptions":{"binary":"/nix/store/pjxk10j2cbj9ld1xz78wq1m1jggylcy2-firefox-96.0.3/bin/firefox","args":["-width","1920","-height","2160","--new-window","https://[redacted]/","--headless","-P","ca.ancilla.hangbrain"]}}}
1644624688072   webdriver::command      WARN    You are using deprecated legacy session negotiation patterns (desiredCapabilities/requiredCapabilities), see https://developer.mozilla.org/en-US/docs/Web/WebDriver/Capabilities#Legacy
1644624688072   geckodriver::capabilities       DEBUG   Trying to read firefox version from ini files
1644624688072   geckodriver::capabilities       DEBUG   Trying to read firefox version from binary
1644624688144   geckodriver::capabilities       DEBUG   Found version 96.0.3
1644624688145   mozrunner::runner       INFO    Running command: "/nix/store/pjxk10j2cbj9ld1xz78wq1m1jggylcy2-firefox-96.0.3/bin/firefox" "--marionette" "-width" "1920" "-height" "2160" "--new-window" "https://chat.google.com/" "--headless" "-P" "ca.ancilla.hangbrain" "-no-remote"
1644624688145   geckodriver::marionette DEBUG   Waiting 60s to connect to browser on 127.0.0.1
1644624688145   geckodriver::browser    TRACE   Failed to open /run/user/8509/rust_mozprofileKu4Rax/MarionetteActivePort
1644624688145   geckodriver::marionette TRACE   Retrying in 100ms
*** You are running in headless mode.
1644624688246   geckodriver::browser    TRACE   Failed to open /run/user/8509/rust_mozprofileKu4Rax/MarionetteActivePort
1644624688246   geckodriver::marionette TRACE   Retrying in 100ms
Crash Annotation GraphicsCriticalError: |[0][GFX1-]: glxtest: Unable to open a connection to the X server (t=0.12545) [GFX1-]: glxtest: Unable to open a connection to the X server
Crash Annotation GraphicsCriticalError: |[0][GFX1-]: glxtest: Unable to open a connection to the X server (t=0.12545) |[1][GFX1-]: glxtest: libEGL initialize failed (t=0.125485) [GFX1-]: gl
xtest: libEGL initialize failed
1644624688346   geckodriver::browser    TRACE   Failed to open /run/user/8509/rust_mozprofileKu4Rax/MarionetteActivePort
1644624688346   geckodriver::marionette TRACE   Retrying in 100ms
[...many more such messages redacted...]
Crash Annotation GraphicsCriticalError: |[0][GFX1-]: glxtest: Unable to open a connection to the X server (t=0.12545) |[1][GFX1-]: glxtest: libEGL initialize failed (t=0.125485) |[2][GFX1-]: RenderCompositorSWGL failed mapping default framebuffer, no dt (t=0.790276) [GFX1-]: RenderCompositorSWGL failed mapping default framebuffer, no dt

The X11/GL errors show that FF is starting up successfully, and are normal -- it's running headless on a machine with no X server. I can see that Geckodriver and Firefox are both running:

$ pgrep -laf gecko
1504017 geckodriver --port 35947
$ pgrep -laf firefox
1517150 /nix/store/pjxk10j2cbj9ld1xz78wq1m1jggylcy2-firefox-96.0.3/bin/.firefox-wrapped --marionette -width 1920 -height 2160 --new-window https://[redacted]/ --headless -P ca.ancilla.hangbrain -no-remote
$ firefox --version                           
Mozilla Firefox 96.0.3
$ geckodriver --version     
geckodriver 0.30.0 (b9121c1a4330 2022-01-27 23:34 +0200)

[whimboo]

Interesting case. So you are specifying a profile name from the profile manager when using "-P","ca.ancilla.hangbrain". But geckodriver doesn't know about the profile location and as such is checking the wrong directory. Can you please check by using the profile path via "-profile", "%path_to_profile%"?

For the -P issue I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1755261.

[ToxicFrog]

I can do that as a one-off, but not in the general case since the mapping from profile name to profile directory isn't straightforward
If I use -CreateProfile "foo path/to/foo", can I then subsequently use -profile path/to/foo? In that case all the profile names are predictable and I can use this.

In any case, I'll give it a shot and report back.

[whimboo]

@ToxicFrog can you explain why you need -CreateProfile? You can start Firefox with -profile path to let it populate the folder with a fresh profile. That way it doesn't pollute the profile manager entries. Note that by using the profile manager the path to the profile is platform dependent.

[ToxicFrog]

I'd completely forgotten about that -- I think that would work better for my use case anyways. I'll try that out and report back, thanks.

[ToxicFrog]

It works! It even works with an existing profile, as long as you don't have marionette.port set in it.

[whimboo]

It works! It even works with an existing profile, as long as you don't have marionette.port set in it.

This is great to hear! So yes, marionette.port should not be set or have a value of 0. Thanks for the verification.

If there is still a problem related to this feature please file a new issue. Otherwise we will ship this new feature with the next geckodriver 0.31.0 release.

[whimboo]

For the -P issue I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1755261.

FYI this bug is now fixed and we marked the usage of -P as deprecated. Instead of the profile name the profile path has to be used.

[MagdelineNg]

@whimboo Hi, I am referencing this issue to automate the existing Firefox instance by connecting to the marionette.port. According to this issue and many others, it seems like the default port is 2828 for marionette, but I realised the selenium driver connects to the port shown in the image instead (and that marionette.port variable changes every time I relaunch the Firefox browser). Is there a way to fix the marionette port when launching Firefox?

Current workflow:

1. Launching Firefox

'C:\Program Files\Mozilla Firefox\firefox.exe' - marionette 
'C:\Program Files\Mozilla Firefox\firefox.exe' - marionette -start-debugger-server=2828 #also tried this but marionette.port still does not reflect 2828

2. Launching Selenium GeckoDriver

driver = webdriver.Firefox(executable_path=GeckoDriverManager().install(), service-args=['--marionette-port', '65493', '--connect-existing'])   #i don't want to hardcode the port, is there a way to get the Firefox marionette.port variable?
print(driver.page_source)

[whimboo]

First note that -start-debugger-server is not an option that controls Marionette. Instead it is related to DevTools. So you can omit that argument.

Regarding connecting to an existing instance of Firefox this is not yet fully supported. Especially not with random ports given that geckodriver cannot read the MarionetteActivePort file from the user's Firefox profile. But if you know which profile is used you could read the port yourself and feed it as argument to the Selenium driver.