ccall does not support C99 complex numbers

[ViralBShah]

I fixed complex dot(), but it still does not work correctly. I can't figure out what might be wrong. The only thing I can think of is that the C interface probably does not know how to return complex numbers.

[JeffBezanson]

This is a difficult one. The problem is that when structs or C99 complex numbers are involved, the calling convention changes and no longer depends only on the size of the value in question. On x86-64, I think this would be fixed by using a proper struct. On 32-bit I don't believe there is a standard ABI for these types. LLVM does not directly support the C99 complex types, so we could try to borrow from clang. Then we would need to make Complex64 and Complex128 special so the code generator knows they correspond to _Complex. Not a fun situation.

In other words, a "full" C99 interface needs to include complex as an explicit feature; the type can't be implemented in pure julia. However, so far we haven't specified which C standard our C FFI supports :)

[StefanKarpinski]

Ugh. That's a shame. Oh, well.

[ViralBShah]

Is it safe to mark this one as "wontfix"? We just avoid calling BLAS routines for complex dot().

[StefanKarpinski]

LLVM does not directly support the C99 complex types, so we could try to borrow from clang. Then we would need to make Complex64 and Complex128 special so the code generator knows they correspond to _Complex.

Given that this datatype is part of the C99 standard, I think this would actually be a reasonable solution.

[JeffBezanson]

We can add types CComplexFloat and CComplexDouble to be used in ccall, which can then be converted to our types. Or we could make Complex128 and Complex64 built-in, but I'm not sure of all the implications of tying our complex numbers to the C99 type.

[StefanKarpinski]

I think that what Jeff was talking about was making ccall specially aware of the Complex64 and Complex128 types: i.e. converting to and from them automatically. That seems sane to me for now (i.e. until we can have Complex{Float32} and Complex{Float64} replace Complex64 and Complex128). ccall already handles a bunch of Julia types specially, converting them automatically to corresponding C types (e.g. ASCIIStrings and UTF8Strings).

[StefanKarpinski]

Oh, I misread that post as from Viral. Obviously, I'm not going to argue with Jeff about what he meant. Regardless of whether it's what you meant, can't we do that?

[JeffBezanson]

If Complex{Float64} replaced Complex128 then it would need to be the special case instead. I guess it feels a little strange to have a particular instance of a parametric type be special.

[StefanKarpinski]

I don't think it's all that strange. We sort of do the same thing in essence by dispatching the methods for *(::Array{Float64,2}, ::Array{Float64,2}) to BLAS while the general case is handled by a generic Julia method. But I see what you're saying. I guess I see this is ccall knows how to "translate" some types, but not all types. Complex64 / Complex{Float32} and Complex128 / Complex{Float64} are just specific types that have a natural translation to a C(99) type.

[ViralBShah]

Usually, very few library functions return one complex value as in dot(). I think it is safe to not put effort into this one. Its just that it took me a while before I realized what was going on.

Perhaps ccall should just throw a warning when it sees things that are unlikely to work?

-viral

On Jun 28, 2011, at 2:05 AM, StefanKarpinski wrote:

I don't think it's all that strange. We sort of do the same thing in essence by dispatching the methods for *(::Array{Float64,2}, ::Array{Float64,2}) to BLAS while the general case is handled by a generic Julia method. But I see what you're saying. I guess I see this is ccall knows how to "translate" some types, but not all types. Complex64 / Complex{Float32} and Complex128 / Complex{Float64} are just specific types that have a natural translation to a C(99) type.

Reply to this email directly or view it on GitHub:
#85 (comment)

[ViralBShah]

openlibm has a bunch of functions that return complex numbers, which we may want to use. For example, csinh. How do we call these? It would be nice if Complex128 mapped to C's double complex.

[JeffBezanson]

Yes, it would be nice. But we have our own implementation of complex sin in terms of real functions. If their complex sin performs some tricks we need, we should try to port them to the julia code.

[ViralBShah]

Apart from this, I cannot think of any other use for this feature. Can we close this?

-viral

On 09-Jan-2012, at 2:27 AM, [email protected] wrote:

Yes, it would be nice. But we have our own implementation of complex sin in terms of real functions. If their complex sin performs some tricks we need, we should try to port them to the julia code.

---

Reply to this email directly or view it on GitHub:
#85 (comment)

[JeffBezanson]

I guess this can be considered a duplicate of handling C structs in general. The only extra feature is handling c99 complex on IA32, where there is no standard ABI for it, and we would have to do something special. That is a fairly marginal feature however. Up to you.

[ViralBShah]

I am closing this one and marking it wontfix.

[JeffBezanson]

Do we want to be able to call zdot and cdot?

[StefanKarpinski]

I think those are fairly important. It seems very premature to close this as "won't fix".

[ViralBShah]

BLAS 1 is not such a big deal. But yes, would call them if it worked.

-viral

On 18-Aug-2012, at 1:10 AM, Jeff Bezanson [email protected] wrote:

Do we want to be able to call zdot and cdot?

—
Reply to this email directly or view it on GitHub.

[ViralBShah]

The discussion seems to suggest that it is not clear what should be done. Please reopen if we can implement.

-viral

On 18-Aug-2012, at 1:18 AM, Stefan Karpinski [email protected] wrote:

I think those are fairly important. It seems very premature to close this as "won't fix".

—
Reply to this email directly or view it on GitHub.

[JeffBezanson]

I think we can definitely have it on x86_64, then from there see what happens on IA32.

[andreasnoack]

Please ignore my reference.

[stevengj]

I just ran into this issue when trying to write a Julia interface for some C99 special-function evaluation code (http://ab-initio.mit.edu/Faddeeva) that is too complicated to contemplate rewriting in Julia.

At the very least, the ccall interface should exit with an error if the return type is Complex, rather than silently producing erroneous results. (Note that this contradicts the documentation of ccall, which says that "any bits type" is supported. The documentation should be fixed too if Complex is not supported.)

Furthermore, it does not seem to be just return types that cause a problem -- even complex arguments seem to be problematic when passed by value. For example, the following code gives rather unexpected (nondeterministic!) results on x86_64 (Debian GNU/Linux, gcc 4.4.5):

#include <complex.h>
#include <stdio.h>
complex double foo(complex double x) {
     printf("%g+%gi\n", creal(x), cimag(x));
     return x;
}

compile with: gcc -std=c99 -fPIC -shared -Wl,-soname,libfoo.so.0 -o libfoo.so foo.c

In Julia:

julia> foo(x::Complex128) = ccall(("foo","libfoo"),Complex128,(Complex128,),x)
julia> foo(x::Number) = foo(complex128(x))
julia> foo(1+3im)
3+1i
1.0 + 3.0im
julia> foo(1.+3.im)
0+0i
0.0 + 0.0im
julia> foo(1.+3.im)
3+3i
3.0 + 3.0im

[JeffBezanson]

Yes, you're right that this should be fixed in the docs and, ideally, supported.
The issue is that our bits types actually look like n-bit integers in ABI terms, and (at least in amd64) c99 complex is a struct. So you can actually pass any bits type, provided the native signature actually matches. Not saying the above behavior is good, just explaining what is happening. We will fix it by making Complex128 a struct, which will not require any julia code changes but will make this work.

[JeffBezanson]

Noting that this is now supported via ComplexPair, which has not yet been merged with Complex128, but will be.

[StefanKarpinski]

This is pretty exciting – we've wanted to do this for a very long time. Our mandelbrot benchmark might get substantially faster.

[stevengj]

Please leave this issue open until Complex128 and Complex64 are supported or merged with ComplexPair, since until then this seems a bit hard to use in conjunction with typical complex data in Julia.

[vtjnash]

@stevengj The conversion between Complex128 and ComplexPair should be relatively automatic in most use cases. Currently, Complex128 is needed for making arrays, where as ComplexPair is needed for ccalls. Eventually, however, Complex128 will be turned into an alias for ComplexPair (when it is possible to declare that instances of ComplexPair are immutable, and thus can be copied)

[stevengj]

I see; the main thing until then is to update the documention, but we have issue #2145 for that.