search: Search is not working with special character.

[PXke]

It is impossible to do search with 2.1 with special character like in "Gård"

Stacktrace is :

Stacktrace (most recent call last):

  File "invenio/ext/legacy/__init__.py", line 119, in __call__
    response = self.app.full_dispatch_request()
  File "flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "flask_restful/__init__.py", line 265, in error_router
    return original_handler(e)
  File "invenio/base/wrappers.py", line 133, in handle_user_exception
    return super(Flask, self).handle_user_exception(e)
  File "flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "invenio/base/decorators.py", line 203, in decorator
    return f(*args, **argd)
  File "invenio/modules/collections/decorators.py", line 56, in decorated
    return method(collection, *args, **kwargs)
  File "invenio/modules/search/views/search.py", line 370, in search
    recids = Query(p).search(collection=collection.name)
  File "invenio/modules/search/api.py", line 56, in search
    query = self.query
  File "werkzeug/utils.py", line 71, in __get__
    value = self.func(obj)
  File "invenio/modules/search/api.py", line 47, in query
    tree = pypeg2.parse(self._query, parser(), whitespace="")
  File "pypeg2/__init__.py", line 665, in parse
    raise parser.last_error

Some pointers for the resolution:

In [9]: from invenio.modules.search.api import parser

In [13]: import pypeg2

In [14]: pypeg2.parse("toto", parser(), whitespace="")
Out[14]: Main(Query([SimpleQuery(ValueQuery(Value(SimpleValue('toto'))))]))

In [15]: pypeg2.parse("Gård", parser(), whitespace="")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-15-64494a74f7c6> in <module>()
----> 1 pypeg2.parse("Gård", parser(), whitespace="") ...

In [16]: from __future__ import unicode_literals, print_function

In [17]: pypeg2.parse("Gård", parser(), whitespace="")
Out[17]: Main(Query([SimpleQuery(ValueQuery(Value(SimpleValue(u'G\xe5rd'))))]))

From the pypeg documentation :

Caveat: pyPEG 2.x is written for Python 3. You can use it with Python 2.7 with the following import (you don't need that for Python 3):

from __future__ import unicode_literals, print_function

Unfortunately I haven't been able to fix it, in the time that was allocated for that.

[tiborsimko]

Confirmed to exist in maint-2.1 branch.

A test case for invenio-demosite:

• http://localhost:28080/search?p=author:"李白"

gives the following traceback:

web_1    |   File "/code/invenio/base/decorators.py", line 203, in decorator
web_1    |     return f(*args, **argd)
web_1    |   File "/code/invenio/modules/collections/decorators.py", line 56, in decorated
web_1    |     return method(collection, *args, **kwargs)
web_1    |   File "/code/invenio/modules/search/views/search.py", line 370, in search
web_1    |     recids = Query(p).search(collection=collection.name)
web_1    |   File "/code/invenio/modules/search/api.py", line 56, in search
web_1    |     query = self.query
web_1    |   File "/usr/local/lib/python2.7/site-packages/werkzeug/utils.py", line 71, in __get__
web_1    |     value = self.func(obj)
web_1    |   File "/code/invenio/modules/search/api.py", line 47, in query
web_1    |     tree = pypeg2.parse(self._query, parser(), whitespace="")
web_1    |   File "/usr/local/lib/python2.7/site-packages/pypeg2/__init__.py", line 665, in parse
web_1    |     raise parser.last_error
web_1    | TypeError: exceptions must be old-style classes or derived from BaseException, not NoneType