Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serialize common numpy data types in write_json #1119

Merged
merged 3 commits into from
Jan 5, 2023

Conversation

victorlin
Copy link
Member

@victorlin victorlin commented Dec 30, 2022

Description of proposed changes

There is a possibility that data to be written by this function contains numpy data types that are un-serializable by the default JSONEncoder¹. Make them serializable using a custom subclass.

Note that defining a default function for un-serializable values is another approach², but using a custom subclass is more complete since it allows for proper fall-back to the default encoder which automatically raises meaningful TypeErrors.

¹ https://docs.python.org/3/library/json.html#json.JSONEncoder
² https://stackoverflow.com/a/65151218/4410590

Related issue(s)

Fixes #1060

Testing

Added pytest which fails when reverting the fix.

Checklist

  • Add a message in CHANGES.md summarizing the changes in this PR that are end user focused. Keep headers and formatting consistent with the rest of the file.

There is a possibility that data to be written by this function contains
numpy data types that are un-serializable by the default JSONEncoder¹.
Make them serializable using a custom subclass.

Note that defining a default function for un-serializable values is
another approach², but using a custom subclass is more complete since it
allows for proper fall-back to the default encoder which automatically
raises meaningful TypeErrors.

¹ https://docs.python.org/3/library/json.html#json.JSONEncoder
² https://stackoverflow.com/a/65151218/4410590
@victorlin victorlin requested a review from a team December 30, 2022 21:08
@victorlin victorlin self-assigned this Dec 30, 2022
@codecov
Copy link

codecov bot commented Dec 30, 2022

Codecov Report

Base: 67.99% // Head: 67.98% // Decreases project coverage by -0.01% ⚠️

Coverage data is based on head (1e3d8a8) compared to base (f8f15ff).
Patch coverage: 63.63% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1119      +/-   ##
==========================================
- Coverage   67.99%   67.98%   -0.02%     
==========================================
  Files          57       57              
  Lines        6646     6656      +10     
  Branches     1634     1637       +3     
==========================================
+ Hits         4519     4525       +6     
- Misses       1823     1825       +2     
- Partials      304      306       +2     
Impacted Files Coverage Δ
augur/utils.py 71.13% <63.63%> (-0.48%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@victorlin victorlin merged commit 1d61850 into master Jan 5, 2023
@victorlin victorlin deleted the victorlin/add-numpy-json-encoder branch January 5, 2023 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

Object of type int64 is not JSON serializable
2 participants