Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kernel_4x4_haswell.S build error on scientific linux 6 #7482

Closed
floswald opened this issue Jul 1, 2014 · 8 comments
Closed

kernel_4x4_haswell.S build error on scientific linux 6 #7482

floswald opened this issue Jul 1, 2014 · 8 comments
Labels
building Build system, or building Julia or its dependencies

Comments

@floswald
Copy link

floswald commented Jul 1, 2014

I cannot build Julia from source on an hpc system. It errors with

../kernel/x86_64/dgemm_kernel_4x4_haswell.S:2548: Error: no such instruction: vpermpd $ 0xb1,%ymm3,%ymm3'

I tried to set the option as suggested here #7476 but that didn't change anything. my version is

cat /proc/version
Linux version 2.6.32-279.19.1.el6.x86_64 ([email protected]) (gcc version 4.4.6     20120305 (Red Hat 4.4.6-4) (GCC) ) #1 SMP Tue Dec 18 17:22:54 CST 2012
@timholy
Copy link
Member

timholy commented Jul 1, 2014

Does the solution in #7240 fix it?

@floswald
Copy link
Author

floswald commented Jul 1, 2014

It DID fix it! thanks!

@floswald floswald closed this as completed Jul 1, 2014
@tkelman
Copy link
Contributor

tkelman commented Jul 1, 2014

If you're trying to build for a heterogeneous cluster with a variety of processors, you're going to need OPENBLAS_DYNAMIC_ARCH enabled. Did you actually apply the patch from #7476 to your local copy? It hasn't been merged into master yet so you would need to manually replicate the additional lines of code in your local makefile.

@floswald
Copy link
Author

floswald commented Jul 1, 2014

@tkelman I did apply your patch into my make.user locally. that didn't change the failing build.
doing your trick from #7240 however worked:

OPENBLAS_DYNAMIC_ARCH=0
JULIA_CPU_TARGET=core2

this build works on my cluster. Are you saying that it should not? I'm unsure how the core2 target interacts with this issue, just that if I don't set this, I get an arch mismatch error. Entirely possible that I'm just lucky here - please let me know!

@tkelman
Copy link
Contributor

tkelman commented Jul 1, 2014

JULIA_CPU_TARGET is related to the code generation in the precompiled Julia system image, rather than openblas. I'm very surprised OPENBLAS_DYNAMIC_ARCH = 0 works. If you had compiled Julia on a newer machine with a newer instruction set, then openblas compiled on that machine would not work on an older processor. You're saying setting OPENBLAS_NO_AVX = 1 and applying my patch (to deps/Makefile, as well as setting the option in Make.user) was still giving the same openblas error message? Did you clean out the openblas build between changing this setting?

By "apply the patch", I mean specifically adding these 3 lines to deps/Makefile, around line 790:

# Allow disabling AVX for older binutils
ifeq ($(OPENBLAS_NO_AVX), 1)
OPENBLAS_BUILD_OPTS += NO_AVX=1
endif

@floswald
Copy link
Author

floswald commented Jul 1, 2014

i did actually wipe the entire julia directory and clone a fresh copy between both trials. but i'm not sure if I applied your patch correctly. so I just have to set echo OPENBLAS_NO_AVX = 1 > Make.user and copy those 4 lines into my deps/Makefile? I'm tempted not to try this again, since it works now. i'll build a second version if I have time later on.

@tkelman
Copy link
Contributor

tkelman commented Jul 1, 2014

so I just have to set echo OPENBLAS_NO_AVX = 1 > Make.user and copy those 4 lines into my deps/Makefile?

Yes, that should do it. If NO_AVX=1 was properly getting sent to OpenBLAS and you were still getting the same no such instruction error, then I'd consider it a bug somewhere in OpenBLAS's build system.

@tkelman
Copy link
Contributor

tkelman commented Jul 1, 2014

Relevant cross-reference: OpenMathLib/OpenBLAS#401

Disabling all AVX instructions could hurt performance on Sandy Bridge, Bulldozer, and Piledriver, but the problematic vpermpd instruction is actually AVX2 and only used on Haswell so far.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
building Build system, or building Julia or its dependencies
Projects
None yet
Development

No branches or pull requests

4 participants