Proposed roadmap for DataArrays revision

[johnmyleswhite]

Here's a list of what I would consider the most important changes to make to this package:

• Find a better abstraction for iteration than the Each* types
• Add an indexing version of isna: isna(da, inds)
• Straighten out and/or remove promotion rules
• Ensure systematic test coverage of codebase
• Get unique and levels definitions right
• Clean up @data and @pdata macros
• Move autocor, percent_change and reldiff to Stats.jl
• Remove databool, datafloat and dataint
• Stop exporting get_indices, index_to_level and level_to_index
• Stop exporting getpoolidx
• Drop pdatabool, pdatafloat and pdataint
• Clean up definition of reorder / move it to DataFrames.jl
• Merge existing rep functionality into Base.repeat
• Add optional category ordering to PooledDataArray
• Move xtab and xtabs to Stats
• Move linalg.jl functions to a separate package
• Try to find a way to remove baseval hack
• Clean up operators code
• Add keyword args to basic statistical functions that must skipna or dropna
• Replace all row* and col* functions with slice-indexing interface
• Do something with gl

[prcastro]

What about deleterows! for DataArrays and DataVectors? It should help to get the same functions on DataFrames, doesn't it?

JuliaData/DataFrames.jl#595

[nalimilan]

The concept of row makes sense for data frames, but for arrays the Julian convention is to consider rows as dimension 1, without any special-casing (cf. next-to-last point above). So deleterows! wouldn't fit into this scheme; one could consider something like deletecoord!(array, dimension, coord), but I'm not sure it would be very useful.

[johnmyleswhite]

We shouldn't support any operations on DataArray that don't exist for Array, unless they directly refer to missingness.

[prcastro]

But the delerow! makes sense for DataFrames, doesn't it?

[johnmyleswhite]

Yeah, DataFrames should support methods to delete columns and rows.