Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dates not detected in arrow tables #269

Open
Fil opened this issue Oct 15, 2024 · 1 comment
Open

dates not detected in arrow tables #269

Fil opened this issue Oct 15, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@Fil
Copy link
Collaborator

Fil commented Oct 15, 2024

A bug where dates are shown as numbers in the table; it should have been covered by #263, but in this case we have field.type.unit==0 instead of 1.

The bug can be seen in Framework with the following page:

---
sql:
  launches: data/launches.csv
---

# Rocket launches 🚀

```sql echo
SELECT * FROM launches;
```

Capture d’écran 2024-10-15 à 18 36 21

I can't find the documentation for what this "unit 0" means. The source code https://github.com/apache/arrow/blob/main/js/src/fb/time.ts seems to indicate that these are seconds. If I change

return field.type.unit === 1 ? "date" : "number"; // millisecond
to accept 0 as a date, the dates appear correctly in the table.

Capture d’écran 2024-10-15 à 18 35 06

@Fil Fil added the bug Something isn't working label Oct 15, 2024
@Niluge-KiWi
Copy link

Ended up here from a data source duckdb -> parquet -> observable framework Plots: it worked on 2024-03-31 (framework 1.5.1, duckdb python 0.10.1): both duckdb SQL DATE & TIMESTAMP were of type js Date in the browser document. Now only TIMESTAMP is: DATE is a number (framework 1.13, duckdb 1.1)/

(I failed to reproduce the old build: I lost my local _npm cache, and the implicit npm:apache-arrow import gets the latest version, and I failed to force another version: it loads both and only the latest is used, and explicit imports were not supported on framework 1.5, and with 1.6 and explicit import apache-arrow after npm install apache-arrow@15 still gets both versions loaded and latest used..
-> is there really a way to force an old apache-arrow?)

Reduced repro to:

```js
Inputs.table(await sql`SELECT current_date as thedate, current_localtimestamp() as thetimestamp`)
```

Image

-> it would be great to support the DateUnit 0: Day

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants