Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google Spanner dependency is too old; cannot deal with JSON type code #21198

Closed
damccorm opened this issue Jun 4, 2022 · 1 comment · Fixed by #23365
Closed

Google Spanner dependency is too old; cannot deal with JSON type code #21198

damccorm opened this issue Jun 4, 2022 · 1 comment · Fixed by #23365

Comments

@damccorm
Copy link
Contributor

damccorm commented Jun 4, 2022

Apache Beam is using google-cloud-spanner version 1.19.1, while current version of google-cloud-spanner is 3.11.1.

I was attempting to use ReadFromSpanner when I ran into  a ValueError - it was getting a type code 11 from Spanner and couldn't parse it. 

Here is (some) relevant code from v3.11.1 of Spanner: 


class TypeCode(proto.Enum):    
  r"""``TypeCode`` is used as part of [Type][google.spanner.v1.Type]
to
  indicate the type of a Cloud Spanner value.    

  Each legal value of a type can be encoded
to or decoded from a JSON   
  value, using the encodings described below. All Cloud Spanner values
   
  can be ``null``, regardless of type; ``null``\ s are always encoded    
  as a JSON ``null``.

  """
  TYPE_CODE_UNSPECIFIED = 0    
  BOOL = 1    
  INT64 = 2    
  FLOAT64 = 3    
  TIMESTAMP
= 4    
  DATE = 5    
  STRING = 6    
  BYTES = 7    
  ARRAY = 8   
  STRUCT = 9    
  NUMERIC
= 10    
  JSON = 11

and the same from v1.19.1:


_sym_db.RegisterEnumDescriptor(_TYPECODE)
TypeCode = enum_type_wrapper.EnumTypeWrapper(_TYPECODE)
TYPE_CODE_UNSPECIFIED
= 0
BOOL = 1
INT64 = 2
FLOAT64 = 3T
IMESTAMP = 4
DATE = 5
STRING = 6
BYTES = 7
ARRAY = 8
STRUCT
= 9
NUMERIC = 10 

There is no support for type code 11, aka JSON in Apache Beam right now.

Imported from Jira BEAM-13270. Original Jira may contain additional context.
Reported by: LauraFP.

@Abacn
Copy link
Contributor

Abacn commented Jun 22, 2022

The native spannerio implementation in python sdk has not been actively maintained. May use cross-language transform apache_beam.io.spanner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants