Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[question]when using the datafusion reading csv in rust project, it went wrong #9652

Closed
colommar opened this issue Mar 17, 2024 · 5 comments
Closed

Comments

@colommar
Copy link
Contributor

part of my rust project

  ctx.register_csv("example", "src/b.csv", CsvReadOptions::new().has_header(true)).await?;
  let df = ctx.sql("select * from example").await?;
  df.show().await?;

top 3 lines of src/b.csv

date,time,name,num,target,money,pay_type,status
2024.03.15,12:02:02,POS消费,20240315115719080408,A食堂1楼大餐厅,14.50,现金,交易成功
2024.03.15,10:03:20,POS消费,20240315100247068511,北区西点房,4.00,现金,交易成功

error

Error: ArrowError(ParseError("Error while parsing value 20240315100247068511 for column 3 at line 2"), None)
error: process didn't exit successfully: `target\debug\rg.exe` (exit code: 1)

Could anyone try to help me to figure this problem out?

@haohuaijin
Copy link
Contributor

haohuaijin commented Mar 17, 2024

related to #3174, maybe you can avoid this problem by specifying the schema instead of letting Datafusion infer the schema.

@colommar
Copy link
Contributor Author

colommar commented Mar 17, 2024

related to #3174, maybe you can avoid this problem by specifying the schema instead of letting Datafusion infer the schema.

I think maybe surface the underlying error here is a good choice.
As I have found the in this, Could you help me find which issue to contribute this?

Or maybe there is some way to solve this problem? Could you give me some ideas.

@haohuaijin
Copy link
Contributor

the related issue in arrow-rs is apache/arrow-rs#2580

@colommar
Copy link
Contributor Author

the related issue in arrow-rs is apache/arrow-rs#2580

Thanks so much!

@alamb alamb closed this as completed Sep 18, 2024
@alamb
Copy link
Contributor

alamb commented Sep 18, 2024

I think the question was answered so we can close the issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants