The browser just appears white #272

robkinyon-roivant · 2019-02-21T15:20:19Z

Describe the bug
When running withokta, the browser windows appears, showing location in title bar, connecting to AWS header, and powered by Okta footer, and a blank space in the middle where the signin should load (EDIT: clarification).

To Reproduce
Steps to reproduce the behavior:

Run withokta with OKTA_BROWSER_AUTH=true

Expected behavior
The browser should load the signin completely and allow login.

Screenshots
UPDATE: forgive me for the edit, @robkinyon-roivant. I think it is useful for people to have an accurate picture of the symptom:

Additional context
Only impacts Windows users.
macOS is not affected.
This started February 20th, 11pm ET. It is likely related to the 2019.02.0 release Okta did on February 19th.
Running with OKTA_BROWSER_AUTH=false works.

AlainODea · 2019-02-21T17:18:03Z

@robkinyon-roivant I haven't experienced this issue. We're on Version 2019.02.0 (Tuesday's release).

Does visiting OKTA_AWS_APP_URL manually still work?
Are there any further details in the console log?
Were there any network/firewall/proxy changes around 11pm ET in your environment?

bogeylnj · 2019-02-21T19:04:48Z

No idea if at all related....But...., our tool worked yesterday and stopped today, as well. We have been troubleshooting all morning to figure out/resolve the cause.

Using BrowserAuthentication, the WebEngine opens a window, but it sits at "Signing In..."

the only thing we "may" have uncovered, is that it works for a couple Users that are on java 1.8.0_162 (our 1.8.0_181+ versions we have installed stopped working today)

robkinyon-roivant · 2019-02-21T20:31:55Z

@AlainODea Everything works properly if we set OKTA_BROWSER_AUTH to false and authenticate with the text method. This implies it's not a network issue.

AlainODea · 2019-02-21T20:45:20Z

@AlainODea Everything works properly if we set OKTA_BROWSER_AUTH to false and authenticate with the text method. This implies it's not a network issue.

I'm at a loss here. I don't experience this issue. It may be IWA related. I'm not sure how the IWA interaction may have changed in Tuesday's release. I don't have IWA deployed.

duhaas2015 · 2019-02-21T21:58:09Z

@bogeylnj and I have narrowed it down to a version a java. If you run the code with version 8/161 of java it works, use anything newer it doesn't. Issues for us seem to line up with when okta upgraded our cell to 2019.02.0. @robkinyon-roivant would be curious to see if you tested with older version of java if you see the same results we are.

https://i.imgur.com/Q82OBqE.gifv

bogeylnj · 2019-02-21T22:42:57Z

@AlainODea - I was also able to find a way to get more output from the WebEngine ("get javascript messages"). Just putting it here in case it gives you an idea. But, I'll be digging into it some more.

I confirmed messages like this don't get thrown out on java 162, but do on later versions.

ConsoleListener: Cannot load script https://ok2static.oktacdn.com/assets/js/widget/testscript.8b00a0599e8d731970eae85a11c92d4a.js?v=1. Failed integrity metadata check.[at 0]
ConsoleListener: Cannot load script https://ok2static.oktacdn.com/assets/js/saasure.min.983d1588099b17439374cd1e9d11239c.js?v=1. Failed integrity metadata check.[at 0]
ConsoleListener: Cannot load script https://ok2static.oktacdn.com/assets/js/sections/user/routing.77e1bbe6e9f5f13ec333a5b6f091135e.js?v=1. Failed integrity metadata check.[at 0]
ConsoleListener: ReferenceError: Can't find variable: $[at 138]

Probably worth another PR to ensure those messages are exposed, as well.

AlainODea · 2019-02-21T23:34:20Z

@bogeylnj you are a gen. Thank you!

We aren’t affected because we disabled the CDN. Long story...

It looks like the WebView is caching outdated versions of the CDN or signin page somehow or that regular browsers are serving outdated cached CDN content that happens to match the integrity checks.

We ran into an issue like this a few weeks ago that impacted customers in some geos. It was like Okta’s CDN was outdated.

AlainODea · 2019-02-21T23:48:40Z

In early January, we encountered an issue where the CSS and JS wouldn’t load in IE11 if you weren’t using the CDN. It seems like Okta did something intended to fix that but caused a separate issue for CDN users. We were getting an issue because without the CDN the JS and CSS did not have Content-Type headers.

bogeylnj · 2019-02-22T00:09:51Z

A viscous cycle :)

It seems as though the latest Okta Update pushed, to our cell last night, added the following integrity check attributes. Capturing the "Signing In..." HTML response received from launching OKTA_AWS_APP_URL shows these integrity attributes set:

<SCRIPT src="https://ok2static.oktacdn.com/assets/js/widget/testscript.8b00a0599e8d731970eae85a11c92d4a.js?v=1" 
    crossorigin="anonymous"
    integrity="sha384-Lhsa8qaDx4gNblcOUQPJSmKlvBQat00JSwBIs9hfhSBfyHikHbioeI+69/7jB5dP" ... type="text/javascript"/>

These are what I think the JavaFX WebView cannot handle (best detail around it). I'm not sure there is a "good" workaround if Okta can't remove the integrity attributes ...outside of the fragile "work around" described in the link.

I also happen to have OKTA_AWS_APP_URL response files saved off from 20190214 (while debugging why >1.8.0_181 required our Users to enter un/pw manually). And, I can confirm that those integrity attributes were not present. So, this explains why given 0 changes in our on-premises environment, things still broke today.

Hope this helps!

AlainODea · 2019-02-22T01:39:15Z

I might be able to devise a hack to filter out the integrity checks. It seems like the best temporary solution is to use an older JRE.

I’m surprised the JRE impacts this as I bundled OpenJFX. Perhaps Java 11 (which removes JavaFX and was the trigger for me bundling OpenJFX) is a suitable workaround since it should fall back to the WebKit implementation embedded in this code.

AlainODea · 2019-02-22T01:48:47Z

I think this hack is worth a shot:
https://stackoverflow.com/a/53617671/154527

I’ll throw together a preview release with that once I check the hack thoroughly for security implications and do some manual sanity tests.

In the meantime, I recommend that folks use jdk1.8.0_161.

bogeylnj · 2019-02-22T02:01:11Z

Yeah, that is the workaround I provide in the link above and describe as fragile. It is susceptible to the same problem - if the response changes again, it could break the hack... unless you have inside info on guaranteed DOM elements that won’t change (I.e., how you implement the replaceAll won’t have a future negative impact)

AlainODea · 2019-02-22T04:16:24Z

I spent the last couple of hours trying to tailor the workaround to fit. I was mainly focused on targeting HTML responses based on Content-Type and replacing ** integrity=** with ** integrityDisabled=** and .integrity = with .integrityDisabled =. I couldn't get it to work. The page came up without CSS and the JS didn't load properly either. The modified login page HTML looks fine, but the WebView won't run the JS and won't log any errors.

The workaround is not usable in Java 11 either due to JPMS restrictions on using internal Sun/Oracle classes.

An entirely gruesome hack would be to run a local proxy server that rewrites the responses. This is essentially

This is caused by a WebView regression in the JRE. There is nothing practical I can do to fix this. I welcome a contributed fix here if someone gets one working.

I recommend one of the following fixes:

Run it with JRE 1.8 update 161 (make the path to java in withokta explicit should suffice if you used the install scripts)
Ask Okta Support to disable the Okta CDN on your Org (this works for me on JRE 1.8 update 202)

bogeylnj · 2019-02-22T04:19:01Z

Thanks for taking the time! We'll see what we can get done tomorrow (if time permits) and reply back.

duhaas2015 · 2019-02-22T04:27:09Z

@AlainODea was it as simple as opening a ticket with Okta and asking for CDN to be disabled? Was going to see what behavior looks like against our okta sandbox that’s running a newer version. Will report back Friday.

_{Sent with GitHawk}

AlainODea · 2019-02-22T11:35:43Z

@duhaas2015 pretty much. One of my colleagues noted that the login page wasn’t loading for a customer. We asked if oktacdn could have vanity domains. Okta said no. We asked them to disable it.

AlainODea · 2019-02-22T17:51:12Z

Confirmed impacting us as well in Amazon WorkSpaces. Does not appear to affect macOS.

bogeylnj · 2019-02-22T17:54:29Z

I have the hack in place here as a workaround (just built a new artifact for folks to test/confirm). I can push a PR to you for the JS stdout and send you a link to the hack code I used so you can compare what might be different in your workaround code to get it working.

AlainODea · 2019-02-22T18:02:18Z

@bogeylnj that would be extremely helpful. Thank you.

duhaas2015 · 2019-02-22T18:08:23Z

I have the hack in place here as a workaround (just built a new artifact for folks to test/confirm). I can push a PR to you for the JS stdout and send you a link to the hack code I used so you can compare what might be different in your workaround code to get it working.

Thanks @bogeylnj

AlainODea · 2019-02-22T18:50:53Z

I have multiple independent confirmations internally that uninstalling JRE and JDK and installing JDK 1.8 update 161 fixes this issue. Thank you, @duhaas2015.

@bogeylnj if you have that stdout snippet and hack snippet I'll work on integrating them into a draft release for folks to try.

bogeylnj · 2019-02-22T22:30:02Z

@AlainODea, et al.

Here is a link to okta-awscli code utilizing the workaround supplied by Mickaël Guessant on SO.
bogeylnj@d9b7836

AlainODea · 2019-02-23T17:02:02Z

I am still working on this. It’s quite involved to build a reliable multi-org, cross-platform fix that handles the various login pages without breaking somewhere.

The Okta login page is not rendered when OKTA_BROWSER_AUTH=true. This started happening after Okta released 2019.02.0, which introduced subresource integrity checks on certain JavaScript resources. This bug appears to be Windows specific. I could not reproduce it on any macOS Mojave configuration regardless of JDK 1.8 or JDK 11 version I tried. The root cause is that JavaFX WebEngine on Java 1.8.0_162 or later on Windows does not correctly handle all subresource integrity checks and will refuse to load certain referenced resources like CSS and JavaScript. - Strip subresource integrity directives from DOM before it gets to the JavaFX WebView (this is a hack, and an ugly one) Resolves oktadev#272

AlainODea · 2019-02-24T01:04:20Z

@robkinyon-roivant I have just published a snapshot release with a candidate fix:
https://github.com/oktadeveloper/okta-aws-cli-assume-role/releases/tag/v1.0.9-SNAPSHOT

I have only lightly tested this, but it works with Java 1.8 update 202 on Windows while v1.0.8 clearly doesn't.

@bogeylnj @duhaas2015 please give this pre-release a try and let me know if it fixes the issue for you.

au457415 · 2019-02-24T15:01:36Z

Interesting. I setup Okta-aws for the first time 2 weeks ago and ran into intermittent issues with OKTA_BROWSER_AUTH = true. When I rolled out to our team I had in my instructions to just use OKTA_BROWSER_AUTH=false. I didn't trouble shoot too much but it seemed to work at first and if you switched AWS accounts in the config.properties I believe that is when the failures started. I don't know as I didn't take notes on that part and was really just trying to get everything to work. Once I saw OKTA_BROWSER_AUTH=false always worked I just moved on.

When it failed it would just hang (permanently) at the Desktop SSO popup screen.

AlainODea · 2019-02-24T17:24:14Z

@au457415 have you tried the fix here: #272 (comment)

mrbelowski · 2019-02-25T10:52:06Z

Running on Windows 7 with the 1.0.9-SNAPSHOT build and Oracle JDK 1.8.0_201-b09, I see the same behaviour as I saw on the previous version (blank page with browser auth enabled). I get this in the shell console:

WebConsoleListener: Cannot load script https://[my provider URL]/assets/js/widget/testscript.8b00a0599e8d731970eae85a11c92d4a.js?v=1. Failed integrity metadata check.[at 0]
WebConsoleListener: Cannot load script https://[my provider URL]/assets/js/mvc/loginpage/initLoginPage.pack.5fee764041262a1a7a369d93b8ff43b7.js?v=1. Failed integrity metadata check.[at 0]

I've also tried running this with OpenJDK version 11.0.1 and Oracle JDK 11.0.2, and building from the latest Git source. All with the same result

AlainODea · 2019-02-25T15:00:36Z

@mrbelowski the fix is not on master.

Please try v1.0.9-SNAPSHOT which has a fix based on the prerelease branch.

mrbelowski · 2019-02-25T15:09:50Z

I tested from the 1.0.9-SNAPSHOT release, and the version built from source was from the prerelease branch

AlainODea · 2019-02-25T15:14:31Z

@mrbelowski okay. That's very concerning. I'll download the prerelease and retest. Thank you for letting me know.

AlainODea · 2019-02-25T15:36:12Z

@mrbelowski I just tested on Windows 10 and Windows 2008 Data Center and it worked, including Duo push MFA.

I do get quite a few logged integrity issues loading scripts:

WebConsoleListener: Cannot load script https://example.okta.com/assets/js/vendor/lib/fingerprint2.min.68ab45bd98459cb766f3ab26d086e5f5.js?v=1. Failed integrity metadata check.[at 0]
WebConsoleListener: Cannot load script https://example.okta.com/assets/js/jquery-1.12.4.min.e93c5a2265fbe2a3e96fe19159fc9a84.js?v=1. Failed integrity metadata check.[at 0]
WebConsoleListener: Cannot load script https://example.okta.com/assets/js/vendor/lib/fingerprint2.min.68ab45bd98459cb766f3ab26d086e5f5.js?v=1. Failed integrity metadata check.[at 0]
WebConsoleListener: Cannot load script https://example.okta.com/assets/js/jquery-1.12.4.min.e93c5a2265fbe2a3e96fe19159fc9a84.js?v=1. Failed integrity metadata check.[at 0]
WebConsoleListener: Cannot load script https://example.okta.com/assets/js/widget/testscript.8b00a0599e8d731970eae85a11c92d4a.js?v=1. Failed integrity metadata check.[at 0]
WebConsoleListener: Cannot load script https://example.okta.com/assets/js/jquery-1.12.4.min.e93c5a2265fbe2a3e96fe19159fc9a84.js?v=1. Failed integrity metadata check.[at 0]
WebConsoleListener: Cannot load script https://example.okta.com/assets/js/widget/spin.min.545405225c13b2f42ab103fca31a49b1.js?v=1. Failed integrity metadata check.[at 0]
WebConsoleListener: ReferenceError: Can't find variable: $[at 161]
WebConsoleListener: Cannot load script https://example.okta.com/assets/js/app/sso/interstitial.474dce61acfac4a4d016921943cf2a68.js?v=1. Failed integrity metadata check.[at 0]

It looks like the scripts that are loaded also set subresource integrity attributes on the DOM elements they generate. That is very difficult to correct for. My attempts to do that broke the login page entirely for me.

This ultimately needs an upstream fix from Oracle since they are the ones who broke this.

AlainODea · 2019-02-25T15:38:07Z

@mrbelowski are you replacing ~/.okta/okta-aws-cli.jar with the JAR downloaded from the v1.0.9-SNAPSHOT? That is what is needed to deploy the hotfix.

mrbelowski · 2019-02-25T15:43:22Z

yes I am (or replacing it with the jar from the /target folder when building from source).

I've changed nothing at my end since i posted originally, yet now it works with exactly the same warning messages as you've posted. Incredibly confusing, but at least it's working

The Okta login page is not rendered when OKTA_BROWSER_AUTH=true. This started happening after Okta released 2019.02.0, which introduced subresource integrity checks on certain JavaScript resources. This bug appears to be Windows specific. I could not reproduce it on any macOS Mojave configuration regardless of JDK 1.8 or JDK 11 version I tried. The root cause is that JavaFX WebEngine on Java 1.8.0_162 or later on Windows does not correctly handle all subresource integrity checks and will refuse to load certain referenced resources like CSS and JavaScript. - Strip subresource integrity directives from DOM before it gets to the JavaFX WebView (this is a hack, and an ugly one) Resolves oktadev#272

AlainODea · 2019-02-25T18:19:09Z

@mrbelowski I have reports from others that the original fix didn't work. Reviewing the different behavior of their Okta Org, I've expanded the fix, retested it, and re-released it. Please give it another try. It should allow login with no integrity metadata check failures.

The Okta login page is not rendered when OKTA_BROWSER_AUTH=true. This started happening after Okta released 2019.02.0, which introduced subresource integrity checks on certain JavaScript resources. This bug appears to be Windows specific. I could not reproduce it on any macOS Mojave configuration regardless of JDK 1.8 or JDK 11 version I tried. The root cause is that JavaFX WebEngine on Java 1.8.0_162 or later on Windows does not correctly handle all subresource integrity checks and will refuse to load certain referenced resources like CSS and JavaScript. - Strip subresource integrity directives from DOM before it gets to the JavaFX WebView (this is a hack, and an ugly one) Resolves #272

AlainODea · 2019-02-25T19:11:19Z

@robkinyon-roivant @duhaas2015 @mrbelowski @au457415 v1.0.9 is released with a workaround for the blank webpage on login.

tbenshalom · 2019-02-26T06:35:21Z

Hi,
I tried v1.0.9 and got same problem - white page.
I'm running on windows 10, with java: Java(TM) SE Runtime Environment (build 1.8.0_201-b09).

Does anyone has success with the new release?
Thanks...

mrbelowski · 2019-02-26T12:07:24Z

I just re-tested and it works for me without the warning messages.

I can get it to display a blank screen if I change the config so that the OKTA_ORG and OKTA_AWS_APP_URL are different domains - i.e. this does NOT work now (but it did up until last week):

OKTA_ORG=https://[corporate-okta-url-1]
OKTA_AWS_APP_URL=https://[corporate-okta-url-2]/home/amazon_aws/[blah blah]

but if the org and aws url use the same domain it DOES work (up until the most recent 1.0.9 build this printed some errors to the command window) - i.e.

OKTA_ORG=https://[corporate-okta-url-1]
OKTA_AWS_APP_URL=https://[corporate-okta-url-1]/home/amazon_aws/[blah blah]

AlainODea · 2019-02-27T17:56:03Z

@tbenshalom do you get anything logged in your terminal when you attempt this? It should show integrity metadata failures or something to indicate the cause when it is blank like that.

Make sure to replace ~/.okta/okta-aws-cli.jar with the v1.0.9 JAR file. If both JARs are present it will not work correctly. Reinstalling is also an option as the installer will get the v1.0.9 release.

tbenshalom · 2019-02-28T09:57:37Z

@AlainODea I reinstall the tool and have the following log in console:
WebConsoleListener: Cannot load script https://ok7static.oktacdn.com/assets/js/widget/testscript.8b00a0599e8d731970eae85a11c92d4a.js?v=1. Failed integrity metadata check.[https://viber-mgmt.okta.com/login/login.htm?fromURI=%2Fhome%2Famazon_aws%2F0oa28dulgiYyyeQ46356%2F272:0]
WebConsoleListener: Cannot load script https://ok7static.oktacdn.com/assets/js/mvc/loginpage/initLoginPage.pack.86be4cd312dec3b2d181b08ea1d495fc.js?v=1. Failed integrity metadata check.[https://viber-mgmt.okta.com/login/login.htm?fromURI=%2Fhome%2Famazon_aws%2F0oa28dulgiYyyeQ46356%2F272:0]

AlainODea · 2019-02-28T11:41:27Z

@tbenshalom is your OKTA_AWS_APP_URL on a different domain from OKTA_ORG?

If so, #277 (now merged to master) should fix your issue. It hasn’t been released, so you’ll need to build it yourself or wait for me to release 1.0.10 with the fix.

tbenshalom · 2019-02-28T12:26:21Z

@AlainODea No, they are same domain...

AlainODea · 2019-02-28T16:13:11Z

@tbenshalom do you have a means to chat privately? If I have your OKTA_AWS_APP_URL I can take a deeper look to see why the SRI stripping workaround doesn't work for you.

tbenshalom · 2019-03-03T13:12:08Z

@AlainODea sure. Please contact me to [email protected].
Thanks

AlainODea self-assigned this Feb 21, 2019

AlainODea removed their assignment Feb 22, 2019

AlainODea added help wanted bug labels Feb 22, 2019

AlainODea mentioned this issue Feb 24, 2019

🐛 The browser just appears white #274

Merged

AlainODea closed this as completed in #274 Feb 25, 2019

This was referenced Feb 26, 2019

OKTA_ORG and OKTA_AWS_APP_URL must match for Subresource Integrity workaround to work #276

Closed

🐛 SRI hack doesn't work for multi-org #277

Merged

AlainODea removed the help wanted label Mar 3, 2019

The browser just appears white #272

The browser just appears white #272

Comments

robkinyon-roivant commented Feb 21, 2019 • edited by AlainODea Loading

AlainODea commented Feb 21, 2019

bogeylnj commented Feb 21, 2019 • edited Loading

robkinyon-roivant commented Feb 21, 2019

AlainODea commented Feb 21, 2019

duhaas2015 commented Feb 21, 2019

bogeylnj commented Feb 21, 2019 • edited Loading

AlainODea commented Feb 21, 2019 • edited Loading

AlainODea commented Feb 21, 2019

bogeylnj commented Feb 22, 2019

AlainODea commented Feb 22, 2019

AlainODea commented Feb 22, 2019

bogeylnj commented Feb 22, 2019 • edited Loading

AlainODea commented Feb 22, 2019

bogeylnj commented Feb 22, 2019

duhaas2015 commented Feb 22, 2019

AlainODea commented Feb 22, 2019

AlainODea commented Feb 22, 2019

bogeylnj commented Feb 22, 2019

AlainODea commented Feb 22, 2019

duhaas2015 commented Feb 22, 2019

AlainODea commented Feb 22, 2019

bogeylnj commented Feb 22, 2019

AlainODea commented Feb 23, 2019

AlainODea commented Feb 24, 2019

au457415 commented Feb 24, 2019

AlainODea commented Feb 24, 2019

mrbelowski commented Feb 25, 2019 • edited Loading

AlainODea commented Feb 25, 2019

mrbelowski commented Feb 25, 2019

AlainODea commented Feb 25, 2019

AlainODea commented Feb 25, 2019

AlainODea commented Feb 25, 2019

mrbelowski commented Feb 25, 2019

AlainODea commented Feb 25, 2019

AlainODea commented Feb 25, 2019

tbenshalom commented Feb 26, 2019

mrbelowski commented Feb 26, 2019

AlainODea commented Feb 27, 2019

tbenshalom commented Feb 28, 2019

AlainODea commented Feb 28, 2019

tbenshalom commented Feb 28, 2019

AlainODea commented Feb 28, 2019

tbenshalom commented Mar 3, 2019

robkinyon-roivant commented Feb 21, 2019 •

edited by AlainODea

Loading

bogeylnj commented Feb 21, 2019 •

edited

Loading

bogeylnj commented Feb 21, 2019 •

edited

Loading

AlainODea commented Feb 21, 2019 •

edited

Loading

bogeylnj commented Feb 22, 2019 •

edited

Loading

mrbelowski commented Feb 25, 2019 •

edited

Loading