[omniorb] Add new port

[CaeruleusAqua]

OmniORB (4.2.3):
Description: OmniORB is a robust high performance CORBA ORB for C++ and Python. It is freely available under the terms of the GNU Lesser General Public License (for the libraries), and GNU General Public License (for the tools). omniORB is largely CORBA 2.6 compliant.

[CaeruleusAqua]

I am not able to make this simply run, omniorb builds a code generator in the build processes that is an embedded Python interpreter. The main problem is that the interpreter needs the Python standard libraries. I tried to force it to use the libraries shipped with vcpkg_find_acquire_program, but it failed.
Maybe because it only contains precompiled *.pyc files and is only 32 bits.

On my local vcpkg fork, I added the python libraries and interpreter to the python3 package. To install them under ${VCPKG_ROOT_DIR}/installed/${TARGET_TRIPLET}/tools/python3

This problem should apear in every program which is embedding python using the python3 package.

Please give me some hints, how to process!

From 402fc4e9865baa6ca2de7ac99f4e50f22bec2908 Mon Sep 17 00:00:00 2001
From: Julin Scholle <[email protected]>
Date: Fri, 20 Jul 2018 17:21:30 +0200
Subject: [PATCH] also install python py libs

---
 ports/python3/portfile.cmake | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/ports/python3/portfile.cmake b/ports/python3/portfile.cmake
index 576a35e8a..3186f2e34 100644
--- a/ports/python3/portfile.cmake
+++ b/ports/python3/portfile.cmake
@@ -53,7 +53,7 @@ else()
 endif()
 
 vcpkg_build_msbuild(
-    PROJECT_PATH ${SOURCE_PATH}/PCBuild/pythoncore.vcxproj
+    PROJECT_PATH ${SOURCE_PATH}/PCBuild/python.vcxproj
     PLATFORM ${BUILD_ARCH})
 
 if (VCPKG_LIBRARY_LINKAGE STREQUAL static)
@@ -76,6 +76,13 @@ if (VCPKG_LIBRARY_LINKAGE STREQUAL dynamic)
     file(COPY ${SOURCE_PATH}/PCBuild/${OUT_DIR}/python${PYTHON_VERSION_MAJOR}${PYTHON_VERSION_MINOR}_d.dll DESTINATION ${CURRENT_PACKAGES_DIR}/debug/bin)
 endif()
 
+file(COPY ${SOURCE_PATH}/PCBuild/${OUT_DIR}/ DESTINATION ${CURRENT_PACKAGES_DIR}/tools/python${PYTHON_VERSION_MAJOR})
+file(COPY ${SOURCE_PATH}/tools DESTINATION ${CURRENT_PACKAGES_DIR}/tools/python${PYTHON_VERSION_MAJOR})
+file(COPY ${SOURCE_PATH}/Lib DESTINATION ${CURRENT_PACKAGES_DIR}/tools/python${PYTHON_VERSION_MAJOR})
+file(COPY ${SOURCE_PATH}/Include DESTINATION ${CURRENT_PACKAGES_DIR}/tools/python${PYTHON_VERSION_MAJOR})
+file(COPY ${SOURCE_PATH}/PCBuild/${OUT_DIR}/python${PYTHON_VERSION_MAJOR}${PYTHON_VERSION_MINOR}.lib DESTINATION ${CURRENT_PACKAGES_DIR}/tools/python${PYTHON_VERSION_MAJOR}/libs)
+file(COPY ${SOURCE_PATH}/PCBuild/${OUT_DIR}/python${PYTHON_VERSION_MAJOR}${PYTHON_VERSION_MINOR}_d.lib DESTINATION ${CURRENT_PACKAGES_DIR}/tools/python${PYTHON_VERSION_MAJOR}/libs)
+
 # Handle copyright
 file(COPY ${SOURCE_PATH}/LICENSE DESTINATION ${CURRENT_PACKAGES_DIR}/share/python${PYTHON_VERSION_MAJOR})
 file(RENAME ${CURRENT_PACKAGES_DIR}/share/python${PYTHON_VERSION_MAJOR}/LICENSE ${CURRENT_PACKAGES_DIR}/share/python${PYTHON_VERSION_MAJOR}/copyright)
-- 
2.16.1.windows.1

[cenit]

Look at the duktape port for example

[CaeruleusAqua]

The duktape port installs PIP, that doesn't help me,

I'm not talking about any extra packages, I'm talking about the Python standard library (like the utf8 and iso8859 encodings).

The Python version provided by vcpkg_find_acquire_program is a Python-3.7.3-embed-win32 which only ships the precompiled standard libraries. Which don't work with the python3 port (which only installs the python3 dll/so and headers).

The libs can simply be installed with the python3 port, but I'd have to change that.
The patch above is installing them into ${CURRENT_PACKAGES_DIR}/tools/python
The question is whether this is welcome.

[NancyLi1013]

@CaeruleusAqua thanks for your PR first. I noticed the failures:

x64-windows

CMake Error at cmake/modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find Python (missing: Python_EXECUTABLE Python_LIBRARY
  Python_INCLUDE_DIR Interpreter Development) (found version "3.7.3")
Call Stack (most recent call first):
  cmake/modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  cmake/modules/FindPython.cmake:169 (find_package_handle_standard_args)
  C:/vsts/_work/2/s/scripts/buildsystems/vcpkg.cmake:263 (_find_package)
  CMakeLists.txt:18 (find_package)

x64-linux

CMake Error at cmake/modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find Python (missing: Python_LIBRARY Python_INCLUDE_DIR
  Development) (found version "2.7.15")
Call Stack (most recent call first):
  cmake/modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  cmake/modules/FindPython.cmake:169 (find_package_handle_standard_args)
  /ci/myagent/_work/1/s/scripts/buildsystems/vcpkg.cmake:263 (_find_package)
  CMakeLists.txt:22 (find_package)

It seems that different python versions are required on Windows and Linux. Maybe you need to find python3 on Windows in this way:

vcpkg_find_acquire_program(PYTHON3)
get_filename_component(PYTHON3_DIR "${PYTHON3}" DIRECTORY)
vcpkg_add_to_path("${PYTHON3_DIR}")

I install python2 by "sudo apt-get install python-dev"and can build this port successfully on my local with make on Linux.

Hope this will help you some.

[CaeruleusAqua]

Is anyone here reading my comments??????? vcpkg_find_acquire_program(PYTHON3) is not an option!

I don't use vcpkg on Linux, but omniORB works with Python 2 and 3, on Linux and Windows.
Usually find_package(Python) prefers python3, it probably couldn't find Python 3 on your system and falls back to python2.

[CaeruleusAqua]

It now runs at least on the Windows-dynamic.
I have now adjusted the Python port accordingly.

From my point of view there are some basic problems,
the findPython scripts in cmake are completely incompatible with vcpkg. Unfortunately, only the release version of the python lib is found, even in a debug configuration. The problem doesn't only apply to this port, I refer you to my boost-python issue #8356.

Unfortunately it is not enough to manually overwrite the library paths, because the script does much more, e.g. retrieve the python runtime and library paths from the interpreter.

I think we need a more general solution for all ports that use Python.

[invy]

@CaeruleusAqua I've built omniorb for vcpkg using msys without cmake. Indeed, embedded python is not an option.
I've also extended python to build the python3 tool.
There is the problem I'm currently facing with FindPython cmake module (or rather with python3 dynamic library): python3.exe cannot find corresponding dll, so you need to specify path accordingly to installed/bin directory.
On Linux using the system wide installation of python3 worked, on windows this is of course is not an option.

[JackBoosY]

@CaeruleusAqua Please resolve these conflicts.

Thanks.

[JackBoosY]

/azp run

[ras0219-msft]

This must be removed before merging.

[ras0219-msft]

Thank you for the PR and sorry for the delays in processing!

As you've said above, using Python and C++ is tricky and doubly so inside vcpkg.

Can you shed a bit more light on a few aspects of the OmniOrb build? Specifically:

I am not able to make this simply run, omniorb builds a code generator in the build processes that is an embedded Python interpreter.

Is this code generator only used during the build of omniorb or is it shipped for downstream projects to use as well? If it is only used during omniorb, then it might actually be ideal for us to acquire python via vcpkg_find_acquire_program() (or otherwise) and use that python for the code generator.

However, any modules that are intended to be loaded by downstream users of python should be built against our python3 port. I know that it's confusing to have multiple copies of python available during the build, but it's clearer if we pretend like we're always cross-compiling -- shipped modules need to use the (hypothetically) arm64 copy of python (the python3 port), but code generators should use the host copy of python (vcpkg_find_acquire_program(PYTHON3)).

If it is too difficult to offer this distinction, it may be best to simply always use vcpkg_find_acquire_program(PYTHON3) and build against that copy -- the python3 port inside vcpkg should be ABI compatible for users that eventually want to install it and use it. This will not enable cross-compilation, but we can't make that work at all without multiple copies of python anyway.

[invy]

@ras0219-msft python3 interpreter is not enough to build omniorb. It also needs headers and libraries. You cannot also mix different version (interpreter program and vcpkg port).
On linux you need python3 interpreter for idl compiler to work anyway.

[CaeruleusAqua]

The code generator is used both when building omniorb and afterwards. But I understand why you want i have two Python versions. The idl compiler (code generator) is completely written in Python, under windows it is embedded in a c++ program to make it easier to use. I think I could try to use the Python version at least in the bootstrap process (with vcpkg_find_acquire_program(PYTHON3)) But for the then shipped IDL compiler we still need the Python headers and libraries. Also, the build system had to be be heavily modified (which I can do, but takes time)

[invy]

On linux omniorb users will need to have python interpreter anyway...

[BillyONeal]

Hello @CaeruleusAqua; it's been ~8 months since Robert suggested a path forward are you still interested in working on this?

[PhoebeHui]

Closing this PR since it seems that no progress is being made. Please reopen if work is still being done.