Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

method ambiguity when writing columns with eltype Union{DateTime, Missing} #70

Closed
ExpandingMan opened this issue Jan 10, 2018 · 1 comment · Fixed by #71
Closed

method ambiguity when writing columns with eltype Union{DateTime, Missing} #70

ExpandingMan opened this issue Jan 10, 2018 · 1 comment · Fixed by #71

Comments

@ExpandingMan
Copy link
Collaborator

ExpandingMan commented Jan 10, 2018 •
edited
Loading

See the following

using DataFrames
using Feather

df = DataFrame(A=Union{DateTime,Missing}[DateTime(), DateTime(), missing], B=rand(3), C=rand(Int64,3))

Feather.write("test1.feather", df)

gives

ERROR: LoadError: MethodError: Feather.writecolumn(::Base.AbstractIOBuffer{Array{UInt8,1}}, ::Type{DateTime}, ::Array{Union{DateTime, Missings.Missing},1}) is ambiguous. Candidates:
  writecolumn(io, ::Type{T}, A::Array{Union{Missings.Missing, T},1}) where T in Feather at /home/expandingman/.julia/v0.6/Feather/src/Feather.jl:352
  writecolumn(io, ::Type{DateTime}, A) in Feather at /home/expandingman/.julia/v0.6/Feather/src/Feather.jl:349
Possible fix, define
  writecolumn(::Any, ::Type{DateTime}, ::Array{Union{DateTime, Missings.Missing},1})
Stacktrace:
 [1] close!(::Feather.Sink{DataFrames.DataFrameStream{Tuple{Array{Union{DateTime, Missings.Missing},1},Array{Float64,1},Array{Int64,1}}}}) at /home/expandingman/.julia/v0.6/Feather/src/Feather.jl:470
 [2] #write#46(::Bool, ::Dict{Int64,Function}, ::Array{Any,1}, ::Function, ::String, ::DataFrames.DataFrame) at /home/expandingman/.julia/v0.6/Feather/src/Feather.jl:523
 [3] write(::String, ::DataFrames.DataFrame) at /home/expandingman/.julia/v0.6/Feather/src/Feather.jl:522
 [4] include_from_node1(::String) at ./loading.jl:576
 [5] include(::String) at ./sysimg.jl:14
while loading /home/expandingman/src/scrap.jl, in expression starting on line 6

Investigating fixes...
ExpandingMan pushed a commit to ExpandingMan/Feather.jl that referenced this issue Jan 10, 2018
Fixed JuliaData#70.
2ec0117
This was referenced Jan 10, 2018
Merged
Closed
@quinnj quinnj closed this as completed in #71 Feb 3, 2018
quinnj pushed a commit that referenced this issue Feb 3, 2018
@ExpandingMan @quinnj
Fixed #70. (#71)
03a19c4
quinnj pushed a commit that referenced this issue Apr 10, 2018
@ExpandingMan @quinnj
Overhauled to Arrow Back-End and Better Memory Safety (#78)
a2558d1 
* Fixed #70.

* Initial cleanup.

* Split into multiple files.

* Moved Arrow.jl to its own directory.

* Fixed method ambiguity in getmetadata.

* Initial implementation with arrow backend.

* Fixed errors; now works for basic bits types and strings.

* Now correctly implement datetime.

* Rewrote column constructors to be reasonable and sane.

* Dict encoding now working.

* Started writing sink.

* Reads now work with new version of Arrow.

* Continuing to work on sinks.

* Everything works except dictionary encoding, which is currently completely fucked on write side.

* Most column types now supported.

* Most functionality now properly implemented.

* Finally supports bools!

* Trying to fixed DictEncoding but it's still fucked up.

* Finally completely fixed DictEncoding.

* Fixed unit testing.

* Removed old reference files.

* Added some materialize methods.

* Removed old comment about DictEncoding being fucked up.

* Removed old reference file fileio.jl

* Removed spurious comment.

* Removed spurious comment.

* Tried to fix appveyor yaml.

* Tried to fix appveyor yaml.

* DictEncoding now works for non Int32, cleaned up some things.

* Added a materialize method for a DataFrame.

* Updated for 0.7.

* Fixed scary metadata bug.

* Replaced uninitialized with undef.

* Fixed file potential file validation bug.

* Updated for new Arrow locator interface.

* Cleaned up Source functions a bit.

* Started adding extra tests.

* Removed references to pre 0.6 in README.

* Added more unit tests.

* Removed explicit Arrow clone commands from travis and appveyor (now that it's registered).

* Small fixes.

* Fixed breaking test due to poor type inference on 0.6.
@jacobadenbaum
Copy link

I am still having this problem when using Feather on a dataframe with columns of dates or of Bools. See the adapted version of the above example (which also fails for me) using the most recent version of Feather.jl and DataFrames.jl:

using DataFrames
using Feather

df = DataFrame(A=Union{Bool,Missing}[true, false, missing], 
               B=rand(3),
               C=rand(Int64,3))

Feather.write("test1.feather", df)

gives

ERROR: LoadError: MethodError:
Feather.writecolumn(::Base.AbstractIOBuffer{Array{UInt8,1}},
::Type{Bool}, ::Array{Union{Bool, Missings.Missing},1}) is ambiguous.
Candidates:
    writecolumn(io, ::Type{T}, A::Array{Union{Missings.Missing, T},1})
        where T in Feather at
        /Users/jacob/.julia/v0.6/Feather/src/Feather.jl:352
    writecolumn(io, ::Type{Bool}, A) in Feather at
        /Users/jacob/.julia/v0.6/Feather/src/Feather.jl:334
Possible fix, define
writecolumn(::Any, ::Type{Bool}, ::Array{Union{Bool, Missings.Missing},1})
Stacktrace:
[1] close!(::Feather.Sink{DataFrames.DataFrameStream{Tuple{Array{Union{Bool,
    Missings.Missing},1},Array{Float64,1},Array{Int64,1}}}}) at
    /Users/jacob/.julia/v0.6/Feather/src/Feather.jl:470
[2] #write#46(::Bool, ::Dict{Int64,Function}, ::Array{Any,1},
    ::Function, ::String, ::DataFrames.DataFrame) at
    /Users/jacob/.julia/v0.6/Feather/src/Feather.jl:523
[3] write(::String, ::DataFrames.DataFrame) at
    /Users/jacob/.julia/v0.6/Feather/src/Feather.jl:522
[4] include_from_node1(::String) at ./loading.jl:576
[5] include(::String) at ./sysimg.jl:14

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix writecolumn method ambiguity ExpandingMan/Feather.jl
2 participants
@jacobadenbaum @ExpandingMan