Errors on Windows

[JeremyRand]

Testing with Firefox 24 on Windows 8 (latest code from master), the Settings dialog shows no notaries, all checkboxes are unchecked, and clicking OK doesn't close the dialog. Firefox 23 on Linux Mint 15 works fine. The following debug output is displayed under Windows.

*** LOG addons.xpi: startup
*** LOG addons.xpi: Skipping unavailable install location app-system-local
*** LOG addons.xpi: Skipping unavailable install location app-system-share
*** LOG addons.xpi: checkForChanges
*** LOG addons.xpi: No changes found
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\util\ConvergenceUtil.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\ctypes\NSPR.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\ctypes\NSS.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\ctypes\SSL.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\ctypes\SQLITE.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\sockets\ConvergenceListenSocket.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\sockets\ConvergenceClientSocket.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\sockets\ConvergenceServerSocket.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\ctypes\Serialization.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\ssl\CertificateManager.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\ssl\CertificateInfo.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\proxy\HttpProxyServer.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\proxy\PatternList.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\component
s\LocalProxy.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\ssl\PhysicalNotary.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\ssl\Notary.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\component
s\SettingsManager.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\component
s\ConnectionManager.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\ssl\NativeCertificateCache.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\component
s\DatabaseHelper.js
Convergence.core: Loaded!
Convergence.core: Loading: C:\Users\Jeremy\AppData\Roaming\Mozilla\Firefox\Profi
les\wp22l9tm.default\extensions\[email protected]\chrome\co
ntent\util\ConvergenceUtil.js
Convergence.core: Loaded!
Convergence.core: Failed to find nspr4 in installed directory, checking system p
aths.
Convergence.core: Failed to find nspr4 in system paths, trying explicit FreeBSD
path.
Convergence.core: |
Error initializing ctypes: Error: couldn't open library /usr/local/lib/libnspr
4.so, @file:///C:/Users/Jeremy/AppData/Roaming/Mozilla/Firefox/Profiles/wp22l9tm
.default/extensions/[email protected]/components/Convergenc
e.js -> file:///C:/Users/Jeremy/AppData/Roaming/Mozilla/Firefox/Profiles/wp22l9t
m.default/extensions/[email protected]/chrome/content/ctype
s/NSPR.js:40
@file:///C:/Users/Jeremy/AppData/Roaming/Mozilla/Firefox/Profiles/wp22l9tm.def
ault/extensions/[email protected]/components/Convergence.js
:87
Convergence@file:///C:/Users/Jeremy/AppData/Roaming/Mozilla/Firefox/Profiles/w
p22l9tm.default/extensions/[email protected]/components/Con
vergence.js:33
@resource://gre/modules/XPCOMUtils.jsm:271
ConvergenceContentPolicy@file:///C:/Users/Jeremy/AppData/Roaming/Mozilla/Firef
ox/Profiles/wp22l9tm.default/extensions/[email protected]/c
omponents/ConvergenceContentPolicy.js:31
@resource://gre/modules/XPCOMUtils.jsm:271
Convergence.core: |
Initializing error: Error: couldn't open library /usr/local/lib/libnspr4.so ,
@file:///C:/Users/Jeremy/AppData/Roaming/Mozilla/Firefox/Profiles/wp22l9tm.defau
lt/extensions/[email protected]/components/Convergence.js -

file:///C:/Users/Jeremy/AppData/Roaming/Mozilla/Firefox/Profiles/wp22l9tm.defa
ult/extensions/[email protected]/chrome/content/ctypes/NSPR
.js:40
@file:///C:/Users/Jeremy/AppData/Roaming/Mozilla/Firefox/Profiles/wp22l9tm.def
ault/extensions/[email protected]/components/Convergence.js
:87
Convergence@file:///C:/Users/Jeremy/AppData/Roaming/Mozilla/Firefox/Profiles/w
p22l9tm.default/extensions/[email protected]/components/Con
vergence.js:33
@resource://gre/modules/XPCOMUtils.jsm:271
ConvergenceContentPolicy@file:///C:/Users/Jeremy/AppData/Roaming/Mozilla/Firef
ox/Profiles/wp22l9tm.default/extensions/[email protected]/c
omponents/ConvergenceContentPolicy.js:31
@resource://gre/modules/XPCOMUtils.jsm:271
Convergence.ui: CertificateStatus constructor called : C:\Program Files (x86)\Mo
zilla Firefox\nss3.dll
Convergence.ui: Constructed!
Convergence.ui: Getting current tab status...

A quick Google suggested this:

https://blog.mozilla.org/addons/2013/06/03/compatibility-for-firefox-22/

"Fold NSPR and NSS into mozjs (for Windows) or libxul (for other platforms). If your add-on has binary components or you build against Mozilla code for other reasons, this is possibly important. I’ve already been contacted a couple of times about failed builds because of this."

Not sure if this is related, but it might be a place to start.

[mk-fg]

Yeah, looks quite like that bug 648407 you referenced, thanks for looking into it.
(a bit dated) Aurora on linux seem to install both libxul and nss/nspr still, but I guess they removed the latter on Windows already.

I don't have Win8 handy to test it right off the bat, but maybe will find Win7 machine to check if it's reproducible there - probably should be, given that change seem to affect all Windows versions.

[JeremyRand]

Thanks mk-fg. Yeah, I'm pretty sure this occurs on Win7 as well; I just happened to be on Win8 when I collected the debug output. Any idea how hard this would be to fix?

[mk-fg]

Without still reproducing the actual issue (sorry about that), I think it should be rather easy.

Mozilla has a long tradition of wrapping and re-exporting symbols from other .so's, look at nss for example, which is in essence just a wrapper with a few extra extension methods around libssl (openssl).

So now when nss and nspr got "folded" into libjs/libxul, I imagine all that'd have to change in ffi code is checking if libxul is available (probably as a fallback) and using all the same constants, calls and signatures we already have from there, as I imagine they shouldn't change at all, just same code linked into different lib.

[mk-fg]

Finally looked into the issue today and have what looks like a bad news.

There was quite long discussion of the change on mozilla bug 648407 and two major highlights there are:

• "Being system-class libraries, NSPR and NSS have been held out as very stable and platform-neutral." "Only on Linux, where we are not planning to make this change." "Remember that we would not do this on Linux at all" (link)

  So apparently on linux, nss/nspr and such will stay as they are, no need for any changes here (but in cd2ac1d, I added trying to use libxul anyway, which works as wrapper as I expected in my comment above).

• "Right now, all symbols will be available. However, in the future, many fewer symbols will be available because I am planning to remove all the symbols that aren't being directly used by Gecko. (I already have a patch to do this). I will post on dev.tech.crypto soon about this." (link).

  If I understand that correctly, this seem to be really bad news for windows - basically they make all the stuff that allows convergence to work - which it builds proxy from - unavailable.
  Or, maybe, "all the symbols that aren't being directly used by Gecko" is some acceptable subset that convergence doesn't use either.

So, with that in mind, I added cd2ac1d which checks libs in order of "mozjs - xul - ", where "fallback" is pre-fold name and keeps old platform-specific hacks in ctypes init code.

Bad news, again, is that it totally fails on windows (and works fine on linux) - it has mozjs.dll and xul.dll, but apparently both are missing at least some of the nss/nspr symbols we use, or have them renamed somehow, as js-ctypes complain.

It might be really bad if these symbols (e.g. related to cert validation, hinted as being first for removal in that mozilla bug) are gone in newer FF builds on windows, which would require either separate nspr/nss builds and bundling these with addon, switching to another lib for same functionality on windows (which I definitely won't do), or complaining to mozilla until they restore what's necessary.

Better outcome might be that I missed something and everything's fine and available.

I think next step should be examining mozjs.dll with something like "readelf" (but for windows binaries, which are not ELF) and checking if everything's there.
Might look into it in the near future, but someone with more windows xp should probably be in a better position to do it.

[r3h0]

This is very bad news indeed. I installed Convergence a couple of years ago to play around with it, and I just recently re-discovered it. I'm on Windows 7 with Firefox 25.0, and I'm experiencing the same issue: no notaries listed after installing. It doesn't seem to be functioning at all.

@mk-fg, this probably isn't the place to ask about this, but I'm interested in the idea behind Convergence. I'm curious to know why Moxie abandoned the project, and whether there's some flaw in the design that either prevented it from catching on or kept the infosec community from getting behind it. I know that's pretty vague, and I'm not sure why I expect you to have the answer, but I see you have been maintaining the most recent code for Convergence. I don't know; I think I'm just disappointed that it's not working.

[mk-fg]

@r3h0
Note that I still need to look at windows dll and make sure there is a problem.
And I guess just picking nss3 and nspr dlls from older ff and bundling them with addon might be a simple and not-so-bad solution.

As to why Moxie stopped working on it - you can ask him on twitter, iirc I've seen him reply there on the subject, but don't remember the details.
I think he just switched to work on TACK and other solutions to the same problem.

Approach Convergence uses has distinct advantages over post-factum notification like Perspectives does (somewhat similar thing that should work, as it's much simplier - gets cert after FF fetches stuff) - e.g. doesn't allow browser to hand over cookies before checks, but also has more overhead (running local proxy), delay and much more low-level and intrusive.

If a goal is to make thing as accessible, performant and easy to use as default CA system in FF for everyone, I think it's very hard to reach with Convergence, might not be worth the effort, but if one knows what they're doing, afaik it works... well, except for where this issue applies ;)

[r3h0]

@mk-fg, thanks for taking the time to reply. I'm bummed that Convergence isn't working right now, but I share your reservations on getting the average user to adopt something like it. I appreciate that you and @JeremyRand have taken the time to maintain Convergence as far as you have. I'll watch this issue in case you get around to fixing this Windows-Firefox problem.

[JeremyRand]

@r3h0, my interest in Convergence is actually for an alternate DNS system (Namecoin) which can securely embed TLS fingerprints in the domain records in a decentralized way (unlike the centralized DNSSEC). So I'm mainly using it to verify fingerprints according to custom rules, in an easy-to-install package. But, if I come up with improvements which are useful for general Convergence usage, I will definitely submit a pull request to @mk-fg.

[mk-fg]

@r3h0, @JeremyRand, a question for you:
If tomorrow I'll commit, say, five .dll files (nss3.dll, nspr4.dll, etc) into this github repo, would you be ok with that?
It's a dumb-easy instant fix, but the problem I see here is "what's in these libs?"

Lighter solution might be having addon say "Hello user! I need these five libs from firefox-21.zip (link) to function, please unpack them from that zip to this dir...".
There's also work done by Tor people on reproducible Firefox builds, which I think should include nss/nspr libs, so these can be built independently and checksum-matched with what's in the repo, but not sure if that work applies to windows.

I was contacted yesterday by person who proposed the solution with libs (built according to this) and patches for git-am in the attachement, so I'm inclined to merge that, but with two documented ways to build the addon - one with bundled libs and big warning about implications of that, and other one producing xpi with just .js sources for linux or windows with a note on where you can get the libs and link on how to build them.

[JeremyRand]

@mk-fg, including pre-built .dll files sounds reasonable to me, as long as it's made clear how to build them, and easy to drop in custom-built .dll files in case the user really wants to build them by themself. Thanks for your work on this.

[mk-fg]

Before bundling stuff, still wanted to look at the symbols in the libs that get shipped with the ff and found that nss3.dll has all the latest stuff from nspr4, ssl3, sqlite3, so apparently they just got folded there, not to mozjs.dll.

Hence embarassingly trivial fix in aa54df5 for the whole issue - basically just check if separate lib is available on windows, if not - use nss3.dll, done.

objdump from binutils seem to handle windows PE binaries just fine - if I wasn't lazy to check it from the start, whole thing would've taken like a dozen of minutes to fix.
Also shame on all you people for being too lazy to check and point that out too ;)