Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RFC: Document Cygwin-to-MinGW cross compilation #8902

Merged
merged 4 commits into from
Nov 6, 2014
Merged

RFC: Document Cygwin-to-MinGW cross compilation #8902

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ec1cdaf
check for and error early with cygwin or msys-target compilers
tkelman Nov 5, 2014
616153c
Document Cygwin-to-MinGW cross compilation
tkelman Nov 5, 2014
5576084
add a call to env in the git troubleshooting alias
tkelman Nov 5, 2014
123de5a
set XC_HOST to `uname -m`-w64-mingw32 by default for cygwin builds
tkelman Nov 5, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions Make.inc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,10 @@ CMAKE ?= cmake
# Figure out OS and architecture
BUILD_OS := $(shell uname)

ifneq (,$(findstring CYGWIN,$(BUILD_OS)))
XC_HOST ?= $(shell uname -m)-w64-mingw32
endif

ifeq ($(XC_HOST),)
CROSS_COMPILE=
HOSTCC = $(CC)
Expand Down Expand Up @@ -400,6 +404,10 @@ ifeq ($(ARCH),)
override ARCH := $(shell $(CC) -dumpmachine | sed "s/\([^-]*\).*$$/\1/")
ifeq ($(ARCH),mingw32)
$(error "the mingw32 compiler you are using fails the openblas testsuite. please see the README.windows document for a replacement")
else ifeq (cygwin, $(shell $(CC) -dumpmachine | cut -d\- -f3))
$(error "cannot build julia with cygwin-target compilers. set XC_HOST to i686-w64-mingw32 or x86_64-w64-mingw32 for mingw cross-compile")
else ifeq (msys, $(shell $(CC) -dumpmachine | cut -d\- -f3))
$(error "cannot build julia with msys-target compilers. please see the README.windows document for instructions on setting up mingw-w64 compilers")
endif
ifeq ($(BUILD_OS),Darwin)
## Mac is a rather amazing 64-bit user-space on 32-bit kernel architecture, so to determine arch we detect
Expand Down
81 changes: 43 additions & 38 deletions README.windows.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ The 64-bit (x86_64) binary will only run on 64-bit Windows and will otherwise re

# Line endings

Julia uses binary-mode files exclusively. Unlike many other Window's programs, if you write '\n' to a file, you get a '\n' in the file, not some other bit pattern. This matches the behavior exhibited by other operating systems. If you have installed msysGit, it is suggested, but not required, that you configure your system msysGit to use the same convention:
Julia uses binary-mode files exclusively. Unlike many other Windows programs, if you write '\n' to a file, you get a '\n' in the file, not some other bit pattern. This matches the behavior exhibited by other operating systems. If you have installed msysGit, it is suggested, but not required, that you configure your system msysGit to use the same convention:

git config --global core.eol = lf
git config --global core.autocrlf = input
Expand Down Expand Up @@ -148,52 +148,57 @@ or edit `%USERPROFILE%\.gitconfig` and add/edit the lines:

5. In case of the issues with building packages (i.e. ICU fails to build with the following error message ```error compiling xp_parse: error compiling xp_make_parser: could not load module libexpat-1: %```) run ```make win-extras``` and then copy everything from the ```dist-extras``` folder into ```usr/bin```.

## Building on Windows with MinGW-builds/MSYS
## Cygwin-to-MinGW cross compiling

### The MSYS build of `make` is fragile and may not reliably support parallel builds. Use MSYS2 as described above, if you can. If you must use MSYS, take care to notice the special comments in this section.
Julia can be also compiled from source in [Cygwin](http://www.cygwin.com), using versions of the MinGW-w64 compilers available through Cygwin's package manager.

1. Install [7-Zip](http://www.7-zip.org/download.html).
1. Download and run Cygwin setup for [32 bit](http://cygwin.com/setup-x86.exe) or [64 bit](http://cygwin.com/setup-x86_64.exe). Note that you can compile either 32 or 64 bit Julia from either 32 or 64 bit Cygwin. 64 bit Cygwin has a slightly smaller but often more up-to-date selection of packages.

2. Install [Python 2.x](http://www.python.org/download/releases). Do **not** install Python 3.
2. Select installation location and download mirror.

3. Install [MinGW-builds](http://sourceforge.net/projects/mingwbuilds/), a Windows port of GCC. as follows. Do **not** use the regular MinGW distribution.
1. Download the [MinGW-builds installer](http://downloads.sourceforge.net/project/mingwbuilds/mingw-builds-install/mingw-builds-install.exe).
2. Run the installer. When prompted, choose:
- Version: the most recent version (these instructions were tested with 4.8.1)
- Architecture: x32 or x64 as appropriate and desired.
- Threads: win32 (not posix)
- Exception: sjlj (for x32) or seh (for x64). Do not choose dwarf2.
- Build revision: most recent available (tested with 5)
3. Do **not** install to a directory with spaces in the name. You will have to change the default installation path. The following instructions will assume `C:\mingw-builds\x64-4.8.1-win32-seh-rev5\mingw64`.
3. At the "Select Packages" step, select the following:
1. `git` (under `Devel` category)
2. `make` (under `Devel` category)
3. `curl` (under `Net` category)
4. `patch` (under `Devel` category)
5. `python` (under `Interpreters` or `Python` category)
6. `gcc-g++` (under `Devel` category)
7. `m4` (under `Interpreters` category)
8. `cmake` (under `Devel` category)
9. `p7zip` (under `Archive` category)
10. `mingw64-i686-gcc-g++` and `mingw64-i686-gcc-fortran` (for 32 bit Julia, under `Devel` category)
11. `mingw64-x86_64-gcc-g++` and `mingw64-x86_64-gcc-fortran` (for 64 bit Julia, under `Devel` category)

4. Download and extract the [MSYS distribution for MinGW-builds](http://sourceforge.net/projects/mingwbuilds/files/external-binary-packages/) (e.g. msys+7za+wget+svn+git+mercurial+cvs-rev13.7z) to a directory *without* spaces in the name, e.g. `C:/mingw-builds/msys`.
4. At the "Resolving Dependencies" step, be sure to leave "Select required packages (RECOMMENDED)" enabled.

5. Download the [MSYS distribution of make](http://sourceforge.net/projects/mingw/files/MSYS/Base/make) and use this `make.exe` to replace the one in the `mingw64\bin` subdirectory of the MinGW-builds installation.
5. Allow Cygwin installation to finish, then start a "Cygwin Terminal" (or "Cygwin64 Terminal") from the installed shortcut.

6. Run the `msys.bat` installed in Step 4. Set up MSYS by running at the MSYS prompt:
6. Build Julia and its dependencies from source.
1. Get the Julia sources
```
git clone --recursive https://github.com/JuliaLang/julia.git
cd julia
```
*Tips:*
- If you get an `error: cannot fork() for fetch-pack: Resource temporarily unavailable` from git, add `alias git="env PATH=/usr/bin git"` to `~/.bashrc` and restart Cygwin.

```
mount C:/mingw-builds/x64-4.8.1-win32-seh-rev5/mingw64 /mingw
mount C:/Python27 /python
export PATH=$PATH:/mingw/bin:/python
```
2. Set the `XC_HOST` variable in `Make.user` to indicate MinGW-w64 cross compilation

Replace the directories as appropriate.
For 32 bit Julia
```
echo 'XC_HOST = i686-w64-mingw32' > Make.user
Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it be reasonable to just autodetect this from uname, and just assume that cygwin == cross-compile-with-mingw-for-cygwin-arch?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe as a default for when BUILD_OS contains CYGWIN? I commonly build 32-bit Julia from 64-bit Cygwin, but wouldn't be opposed to setting XC_HOST by default for same-bitness - as long as it's easily overridable.

Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, then you could do something like set ARCH=i686 to switch builds, so that it doesn't look much like a cross-compile in the end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are a couple of small places where I do need special behavior for BUILD_OS containing CYGWIN, or BUILD_OS being WINNT would do things that are msys-specific. I don't quite want to turn cygwin into WINNT here. It isn't quite as much of a cross-compile in that you are able to run the compiled executables so we could try to turn on dependency self-checks for this particular combination, but I think that's the only downside in current cross-compilation. There's also the issue of Cygwin's git not working with the package manager, same as in MSYS2, hopefully we'll fix that with the libgit2 work soon enough.

Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's fine. we can automatically guess that if BUILD_OS==CYGWIN, then OS==WINNT (essentially just auto-detecting that the user wants a cross-compile, since we can't to a native cygwin compile anyways)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uname and therefore BUILD_OS are CYGWIN_NT-#.# so it should use a findstring not an exact match, but yeah. Looking at

julia/Make.inc

Lines 163 to 164 in 49083be

ifneq (,$(findstring mingw,$(XC_HOST)))
override OS := WINNT
we already set OS to WINNT for any mingw-containing values of XC_HOST. I'll put in a cygwin-only default setting of XC_HOST there at line 155.

For posterity if anyone's curious why libuv won't build for Cygwin or msys targets, see joyent/libuv#845 (comment) - the Cygwin posix dll doesn't appear to have any async io capabilities.

```
For 64 bit Julia
```
echo 'XC_HOST = x86_64-w64-mingw32' > Make.user
```

7. Download the Julia source repository and build it
```
git clone https://github.com/JuliaLang/julia.git
cd julia
make
```
*Tips:*
- The MSYS build of `make` is fragile and will occasionally corrupt the build process. You can minimize the changes of this occurring by only running `make` in serial, i.e. avoid the `-j` argument.
- When the build process fails for no apparent reason, try running `make` again.
- Sometimes, `make` will appear to hang, consuming 100% cpu but without apparent progress. If this happens, kill `make` from the Task Manager and try again.
- Expect this to take a very long time (dozens of hours is not uncommon).
- If `make` fails complaining that `./flisp/flisp` is not found, force `make` to build FemtoLisp before Julia by running `make -C src/flisp && make`.

8. Run Julia with _either_ of:
3. Start the build
```
make -j 4 # Adjust the number of cores (4) to match your build environment.
```

7. Run Julia with _either_ of:
- Using `make`
```
make run-julia
Expand All @@ -205,7 +210,7 @@ or edit `%USERPROFILE%\.gitconfig` and add/edit the lines:
usr/bin/julia
```

## Cross-compiling
## Cross-compiling from Unix

If you prefer to cross-compile, the following steps should get you started.

Expand Down