Skip to content

Error The data type type List .. has no natural order when using arrow::compute::lexsort_to_indices with list and more than one column #5454

@alamb

Description

@alamb

Describe the bug

lexsort_to_indices a ListArray itself works fine

However, lexsort_to_indices with a ListArray and another array results in an error:

To Reproduce
Here is a reproducer:

    let string_array: ArrayRef = Arc::new(StringArray::from(vec!["a", "b", "c", "d", "e"]));
    let data = vec![
        Some(vec![Some(0), Some(1), Some(2)]),
        None,
        None,
        Some(vec![Some(3), None, Some(5)]),
        Some(vec![Some(6), Some(7)]),
    ];
    let list_array: ArrayRef = Arc::new(ListArray::from_iter_primitive::<Int32Type, _, _>(data));

    // lex_sort works fine on ListArray
    let sort_cols = vec![
        SortColumn {
            values: list_array.clone(),
            options: None,
        },
    ];
    arrow::compute::lexsort_to_indices(&sort_cols, None).unwrap();

    // but not on list array with string array
    let sort_cols = vec![
        SortColumn {
            values: list_array,
            options: None,
        },
        SortColumn {
            values: string_array,
            options: None,
        },
    ];
    // XXXXX This fails with 
    // called `Result::unwrap()` on an `Err` value: InvalidArgumentError("The data type type List(Field { name: \"item\", data_type: Int32, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }) has no natural order")
    arrow::compute::lexsort_to_indices(&sort_cols, None).unwrap();

The test should pass and not error

Additional context
Reported by @JasonLi-cn on apache/datafusion#9410

Metadata

Metadata

Assignees

No one assigned

    Labels

    arrowChanges to the arrow cratebug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions