Python 3 compatibility?

[olebole]

Hi,

I am working on the Python 3 compatibility of gavodachs, which has newvow as dependency. Therefore, I am wondering whether nevow is Python 3 compatible resp. if there are any short-term plans to do so?

(Debian has only a Python 2 package, that's why I am asking)

[mithrandi]

I don't think there are any plans to port Nevow to Python 3 at this stage. The project is mostly in "maintenance mode", and I suspect that a port would be quite tricky given some hairy things that are done in Nevow's internals. For gavodachs you may have an easier time porting from nevow to twisted.web (and twisted.web.template) than porting nevow itself to Python 3.

[wthie]

Aloha
just made some progress on this, there is a fork over at https://github.com/msdemlei/nevow.git which has advanced by the work of Markus to the point where a good part of nevow is operational. I'm currently working on getting node.js to digest the test code and made some progress in this area too. My main objective is getting athena to py3 as a lot of my work is based upon it.

[exarkun]

@wthie Do you have any plans to contribute your changes upstream? It would be nice to avoid folks duplicating your efforts and it would be nice if the "nevow" package (on pypi, etc) had Python 3 support instead of folks having to find a random fork on github.

(cc @pythonhacker @heartsucker)

[wthie]

I definitely do want to push my changes upstream, but I will need some time to reorient myself. Mahalo, Werner

…

On 3/24/19 03:59, Jean-Paul Calderone wrote: @wthie <https://github.com/wthie> Do you have any plans to contribute your changes upstream? It would be nice to avoid folks duplicating your efforts and it would be nice if the "nevow" package (on pypi, etc) had Python 3 support instead of folks having to find a random fork on github. (cc @pythonhacker <https://github.com/pythonhacker> @heartsucker <https://github.com/heartsucker>) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB2z7Q2EC1uMm5uUEDNXmfzTqqKQFDGsks5vZ4THgaJpZM4O1rok>.

[olebole]

@wthie -- do you coordinate with @msdemlei with the patches? Would be great if a fork could be avoided.

[sandrotosi]

@wthie did you make any progress in getting your changes ready for upstream nevow? Debian is in the process of removing Python2, and nevow is still py2 only, which makes it at risk of being removed.

[wthie]

Sorry not ready yet and very little time to work on it :( Werner I did not do any work on formless which sits at $$$ trial formless.test FAILED (errors=6, successes=12) The remaining errors are all around athena/JavaScript support. For the ones I fixed already node.js works pretty well.   $$$ trial nevow.test FAILED (skips=20, expectedFailures=1, failures=1, errors=7, successes=827) $$$ trial nevow.test | grep -B2 ERROR RequestWrapperTests test_attributes ... [OK] test_contentLength ... [ERROR] -- test_getDependenciesNoModules ... [OK] JSGenerationTestCase test_generateTestScript ... [ERROR] -- testSerialize ... [OK] testStringlikeRoundtrip ... [OK] test_customSerialization ... [ERROR] -- test_fragmentSerialization ... [OK] test_lineTerminators ... [OK] test_undefined ... [ERROR] -- nevow.test.test_nit NevowInteractiveTesterTest test_errorRendering ... [ERROR] test_gatherError ... [OK] test_getSuite ... [ERROR] test_portOption ... [OK] test_portOptionDefault ... [OK] test_run ... [ERROR] -- nevow.test.test_loaders.TestDocFactoriesCache.test_reloadAfterPrecompile =============================================================================== [ERROR] -- nevow.test.test_compression.RequestWrapperTests.test_contentLength =============================================================================== [ERROR] -- nevow.test.test_consolejstest.JSGenerationTestCase.test_generateTestScript =============================================================================== [ERROR] -- nevow.test.test_json.JavascriptObjectNotationTestCase.test_customSerialization =============================================================================== [ERROR] -- nevow.test.test_json.JavascriptObjectNotationTestCase.test_undefined =============================================================================== [ERROR] -- nevow.test.test_nit.NevowInteractiveTesterTest.test_errorRendering =============================================================================== [ERROR] -- nevow.test.test_nit.NevowInteractiveTesterTest.test_getSuite =============================================================================== [ERROR]

…

On 1/31/20 15:35, Sandro Tosi wrote: @wthie <https://github.com/wthie> did you make any progress in getting your changes ready for upstream nevow? Debian is in the process of removing Python2, and nevow is still py2 only, which makes it at risk of being removed. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101?email_source=notifications&email_token=AAO3H3PFBK4J7KJ7IXAXJILRATGUZA5CNFSM4DWWXISKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKQP3FQ#issuecomment-580976022>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAO3H3LLT3UAGYQCNWCPGK3RATGUZANCNFSM4DWWXISA>.

[wthie]

Aloha

With Matthias Demleitner having done the bulk of the work of porting nevow to Python3 and me tackling the nevow/athena part I now have the tests down to

FAILED (skips=4, expectedFailures=1, failures=1, successes=1037)

which pretty much compares to what I see running the tests on Python2

PASSED (skips=4, expectedFailures=2, successes=1043)

I have attached the two output files produced when running

trial nevow

I understand that I'm possibly the last one to use nevow but I have a few successful projects in production which rely nevow/athena and the Livepage implementation. These projects I will now subsequently port to Python3 as well.

My question to the few still on this mailing list is - how do I handle the merge back into the nevow project on github? I'm asking because this work was not done in manageable junks with reviews but more in a single battle style relying completely on the validity of the test suit.

Mahalo, Werner

res2.txt
res3.txt

[mithrandi]

I think putting your whole branch up in a PR is where we should start; from there we can look at cherry-picking some of the changes into smaller branches, with the help of tools like pyupgrade.

[wthie]

Mahalo Tristan will work on the PR then. I've already ported my project using nevow/athena which was surprisingly easy and am now testing, MySQLDb currently being the roadblock. Werner

…

On 11/4/20 21:54, Tristan Seligmann wrote: I think putting your whole branch up in a PR is where we should start; from there we can look at cherry-picking some of the changes into smaller branches, with the help of tools like pyupgrade. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#101 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAO3H3KO3TWDDFRDG3CEYT3SOJK3BANCNFSM4DWWXISA>.

[vai-brma]

We have a large project running with Python 2.7 and twisted/nevow. We are under pressure to upgrade to Python 3. Replacing nevow would be a huge amount of work.
Is there a chance that nevow will be Python 3 compatible this year?
thanks,

[wthie]

From my own application porting experience so far I'm currently mired in the flatteners having used a mixture of different loaders in my projects. This mixture has proven so far to be quite difficult to manage due to the string/byte array conversions and checks for proper formats in the nevow code base, seeing exceptions has become the new normal.

I'm under quite some time pressure with my projects but will try to do the PR before the end of the year, hopefully with a clearer idea of how the flattener code could be improved.

Werner

[exarkun]

Can I suggest once more that you submit whatever porting work you've already completed? Even if you port all of Nevow next week, the chances of a single PR porting all 16000-some lines of Nevow is going to be meaningfully reviewed anytime soon.

As far as replacing Nevow goes, Tahoe-LAFS has recently finished replacing Nevow with Twisted.Web. It was not a small project but it was also doable. Even if it seems untenable I'd suggest considering that route. At the end, you not only end up with something Python 3 compatible, but you drop a dependency and you get Twisted.Web's improved flattener and the benefit of all the rest of Twisted Web's maintenance (only some of which you can benefit from via Nevow due to Nevow taking over large parts of how requests are handled). A lot of what you get from Nevow is just misfeatures anyway, and moving to Twisted Web is a good way to be sure you're not using those.

[msdemlei]

On Wed, Dec 02, 2020 at 11:58:46AM -0800, Werner Thie wrote: From my own application porting experience so far I'm currently mired in the flatteners having used a mixture of different loaders in my projects. This mixture has proven so far to be quite difficult to manage due to the string/byte array conversions and checks for proper formats in the nevow code base, seeing exceptions has become the new normal.

On porting experience: I've ported my Virtual Observatory server package DaCHS (which has some extra trickery because it's deployed by third parties who have templates that I didn't want to break; also, it contains formal as a semi-exposed interface) to be based on twisted web directly. Doing this, the one thing that, in the end, caused the most trouble was that I often returned resources from nevow renderHTTP; finding a pattern that covers all of these various use cases I found rather confusing, and I think if I did it again, I'd put in some glue that would still let me return resources from render(). Oh, and of course the data attribute really is something I had to maintain. Anyway, mainly because of my attempt to maintain the interfaces I expose to my users, I ended up writing something like a nevow compatibility layer (nevowc.py), and regrettably I can't do without monkeypatching twisted.web a bit (twistedpatch.py); with that, porting is still quite an effort, but user-visible changes were rather minimal. The code is in https://svn.ari.uni-heidelberg.de/svn/gavo/python/trunk/gavo/formal (with porting instructions in README). I *could* imagine tearing the two files out and making them into something resembling a package if there's interest; I'll certainly maintain them for the next 10 years or so.

[exarkun]

The linked repo seems private, fwiw.

[msdemlei]

On Thu, Dec 03, 2020 at 04:10:25AM -0800, Jean-Paul Calderone wrote: The linked repo seems private, fwiw.

Oh, sorry -- that's the authenticated endpoint. The public, non-authenticated endpoint is http://svn.ari.uni-heidelberg.de/svn/gavo/python/trunk/gavo/formal

[wthie]

PR is up

[vai-brma]

Thanks for all your suggestions and effort Mr. wthie. I have a few options to consider now.

[perkinslr]

I'm sorry I somehow missed this go by. I ported the majority of epsilon, axiom, and nevow to python3.5 back in 2016. Since this project was in maintenance mode, and my port was not exactly clean (the goal was to make it work correctly, not make it pretty), I did not open an issue/PR at the time for python3 support. Nevertheless, I've been using it in production since, and it's been rock-solid stable. I can probably open a PR for it, at least that would help people find it to get up and running.