Python3.8 support issue

[niuzaisheng]

In python3.8 ast package has change that make asdl can‘t work normally.

Because of:
https://docs.python.org/3.8/library/ast.html#

Changed in version 3.8: Class ast.Constant is now used for all constants.
Deprecated since version 3.8: Old classes ast.Num, ast.Str, ast.Bytes, ast.NameConstant and ast.Ellipsis are still available, but they will be removed in future Python releases. In the meanwhile, instantiating them will return an instance of a different class.

Do you have plans to support such changes in python3.8? If not, I will use a lower version of python. Thank you!

[Vimos]

I tried to use the ast in the link you shared. But I am still getting issues with keyword. Any hint over this?

## ASDL's 5 builtin types are:
identifier, int, string, object, constant

mod = Module(stmt* body, type_ignore *type_ignores)
    | Interactive(stmt* body)
    | Expression(expr body)
    | FunctionType(expr* argtypes, expr returns)

    ## not really an actual node but useful in Jython's typesystem.
    | Suite(stmt* body)

stmt = FunctionDef(identifier name, arguments args,
                   stmt* body, expr* decorator_list, expr? returns,
                   string? type_comment)
      | AsyncFunctionDef(identifier name, arguments args,
                         stmt* body, expr* decorator_list, expr? returns,
                         string? type_comment)

      | ClassDef(identifier name,
         expr* bases,
         keyword* keywords,
         stmt* body,
         expr* decorator_list)
      | Return(expr? value)

      | Delete(expr* targets)
      | Assign(expr* targets, expr value, string? type_comment)
      | AugAssign(expr target, operator op, expr value)
      ## 'simple' indicates that we annotate simple name without parens
      | AnnAssign(expr target, expr annotation, expr? value, int simple)

      ## use 'orelse' because else is a keyword in target languages
      | For(expr target, expr iter, stmt* body, stmt* orelse, string? type_comment)
      | AsyncFor(expr target, expr iter, stmt* body, stmt* orelse, string? type_comment)
      | While(expr test, stmt* body, stmt* orelse)
      | If(expr test, stmt* body, stmt* orelse)
      | With(withitem* items, stmt* body, string? type_comment)
      | AsyncWith(withitem* items, stmt* body, string? type_comment)

      | Raise(expr? exc, expr? cause)
      | Try(stmt* body, excepthandler* handlers, stmt* orelse, stmt* finalbody)
      | Assert(expr test, expr? msg)

      | Import(alias* names)
      | ImportFrom(identifier? module, alias* names, int? level)

      | Global(identifier* names)
      | Nonlocal(identifier* names)
      | Expr(expr value)
      | Pass | Break | Continue

      ## XXX Jython will be different
      ## col_offset is the byte offset in the utf8 string the parser uses
      attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset)

      ## BoolOp() can use left & right?
expr = BoolOp(boolop op, expr* values)
     | NamedExpr(expr target, expr value)
     | BinOp(expr left, operator op, expr right)
     | UnaryOp(unaryop op, expr operand)
     | Lambda(arguments args, expr body)
     | IfExp(expr test, expr body, expr orelse)
     | Dict(expr* keys, expr* values)
     | Set(expr* elts)
     | ListComp(expr elt, comprehension* generators)
     | SetComp(expr elt, comprehension* generators)
     | DictComp(expr key, expr value, comprehension* generators)
     | GeneratorExp(expr elt, comprehension* generators)
     ## the grammar constrains where yield expressions can occur
     | Await(expr value)
     | Yield(expr? value)
     | YieldFrom(expr value)
     ## need sequences for compare to distinguish between
     ## x < 4 < 3 and (x < 4) < 3
     | Compare(expr left, cmpop* ops, expr* comparators)
     | Call(expr func, expr* args, keyword* keywords)
     | FormattedValue(expr value, int? conversion, expr? format_spec)
     | JoinedStr(expr* values)
     | Constant(constant value, string? kind)

     ## the following expression can appear in assignment context
     | Attribute(expr value, identifier attr, expr_context ctx)
     | Subscript(expr value, slice slice, expr_context ctx)
     | Starred(expr value, expr_context ctx)
     | Name(identifier id, expr_context ctx)
     | List(expr* elts, expr_context ctx)
     | Tuple(expr* elts, expr_context ctx)

      ## col_offset is the byte offset in the utf8 string the parser uses
      attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset)

expr_context = Load | Store | Del | AugLoad | AugStore | Param

slice = Slice(expr? lower, expr? upper, expr? step)
      | ExtSlice(slice* dims)
      | Index(expr value)

boolop = And | Or

operator = Add | Sub | Mult | MatMult | Div | Mod | Pow | LShift
             | RShift | BitOr | BitXor | BitAnd | FloorDiv

unaryop = Invert | Not | UAdd | USub

cmpop = Eq | NotEq | Lt | LtE | Gt | GtE | Is | IsNot | In | NotIn

comprehension = (expr target, expr iter, expr* ifs, int is_async)

excepthandler = ExceptHandler(expr? type, identifier? name, stmt* body)
                attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset)

arguments = (arg* posonlyargs, arg* args, arg? vararg, arg* kwonlyargs,
             expr* kw_defaults, arg? kwarg, expr* defaults)

arg = (identifier arg, expr? annotation, string? type_comment)
       attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset)

## keyword arguments supplied to call (NULL identifier for **kwargs)
keyword = (identifier? arg, expr value)

## import name with optional 'as' alias.
alias = (identifier name, identifier? asname)

withitem = (expr context_expr, expr? optional_vars)

type_ignore = TypeIgnore(int lineno, string tag)

When trying over parsing the code

py_code = """pandas.read('file.csv', n_rows=100)"""

The error is listed in detail as follows.

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-28-f8e859182bac> in <module>
      9 
     10 # convert the python AST into general-purpose ASDL AST used by tranX
---> 11 asdl_ast = python_ast_to_asdl_ast(py_ast.body[0], grammar)
     12 print('String representation of the ASDL AST: \n%s' % asdl_ast.to_string())
     13 print('Size of the AST: %d' % asdl_ast.size)

~/Data2/en_nlp/Text2SQL/code/asdl/lang/py/py_asdl_helper.py in python_ast_to_asdl_ast(py_ast_node, grammar)
     40             if field_value is not None:  # sometimes it could be 0
     41                 if grammar.is_composite_type(field.type):
---> 42                     child_node = python_ast_to_asdl_ast(field_value, grammar)
     43                     asdl_field.add_value(child_node)
     44                 else:

~/Data2/en_nlp/Text2SQL/code/asdl/lang/py/py_asdl_helper.py in python_ast_to_asdl_ast(py_ast_node, grammar)
     48             if grammar.is_composite_type(field.type):
     49                 for val in field_value:
---> 50                     child_node = python_ast_to_asdl_ast(val, grammar)
     51                     asdl_field.add_value(child_node)
     52             else:

~/Data2/en_nlp/Text2SQL/code/asdl/lang/py/py_asdl_helper.py in python_ast_to_asdl_ast(py_ast_node, grammar)
     31     # assert py_node_name.startswith('_ast.')
     32 
---> 33     production = grammar.get_prod_by_ctr_name(py_node_name)
     34 
     35     fields = []

~/Data2/en_nlp/Text2SQL/code/asdl/asdl.py in get_prod_by_ctr_name(self, name)
     48 
     49     def get_prod_by_ctr_name(self, name):
---> 50         return self._constructor_production_map[name]
     51 
     52     @property

KeyError: 'keyword'

[XiaoXiaoYi123]

Have you solved this problem?