Add Tipue Search

[d0ugal]

Successor to #60. Rebased and updated.

[d0ugal]

I wouldn't call this finished, but it roughly works ;)

[d0ugal]

I don't know why this is showing up here - these changes are already merged.

[tomchristie]

Excellent!

[tomchristie]

Screenshot of @d0ugal's current progress on this...

Rough & ready, but working!
I guess you'll be going for a modal popover, similar to DRF style?

[d0ugal]

I was actually copying Docker's approach: https://docs.docker.com/jsearch/?q=install

However, a modal would make some things a bit easier.

[d0ugal]

Starting to look better..

[d0ugal]

The Read the docs theme too..

[d0ugal]

There are a few bugs I'm still working on, but it's starting to shape up well.

[tomchristie]

Very nice!

[d0ugal]

Having some issues getting the code is search.py to work under 2 and 3. So I've updated the Tox/Travis run to better highlight the failures (might revert it later if it feels too verbose).

I think i might need to look for an alternative to https://docs.python.org/2/library/htmlparser.html - that is the root of the issue. It appears to have changed more than a simple relocation in Python 3.

[d0ugal]

Alrighty, I think I'm at a point now where feedback would be useful. I'm not aware of any glaring issues. Search is working fine in the MkDocs and ReadTheDocs themes - interesting to note that they use two different strategys for making this work (MkDocs is modal, Read the docs uses a different page as a modal didn't fit the theme).

One outstanding question: Before we just dumped one file in the site_dir (404.html), now we potentially create tipuesearch_content.js and search.html too. Should we put these under something that wont conflict? i.e. a directory named mkdocs?

[fedelibre]

I've checked out your pull request and tested it on a project and I get the traceback below.
Have you tested on a project containing unicode characters? Or do you think there's something wrong with my installation?

$ mkdocs serve
Traceback (most recent call last):
  File "/usr/local/bin/mkdocs", line 9, in <module>
    load_entry_point('mkdocs==0.11.1', 'console_scripts', 'mkdocs')()
  File "/usr/local/lib/python2.7/dist-packages/mkdocs-0.11.1-py2.7.egg/mkdocs/main.py", line 60, in run_main
    main(cmd, args=sys.argv[2:], options=dict(opts))
  File "/usr/local/lib/python2.7/dist-packages/mkdocs-0.11.1-py2.7.egg/mkdocs/main.py", line 33, in main
    serve(config, options=options)
  File "/usr/local/lib/python2.7/dist-packages/mkdocs-0.11.1-py2.7.egg/mkdocs/serve.py", line 93, in serve
    build(config, live_server=True)
  File "/usr/local/lib/python2.7/dist-packages/mkdocs-0.11.1-py2.7.egg/mkdocs/build.py", line 240, in build
    build_pages(config)
  File "/usr/local/lib/python2.7/dist-packages/mkdocs-0.11.1-py2.7.egg/mkdocs/build.py", line 207, in build_pages
    table_of_contents, meta, config
  File "/usr/local/lib/python2.7/dist-packages/mkdocs-0.11.1-py2.7.egg/mkdocs/search.py", line 22, in add_entry_from_context
    parser.feed(content)
  File "/usr/lib/python2.7/HTMLParser.py", line 117, in feed
    self.goahead(0)
  File "/usr/lib/python2.7/HTMLParser.py", line 155, in goahead
    if i < j: self.handle_data(rawdata[i:j])
  File "/usr/local/lib/python2.7/dist-packages/mkdocs-0.11.1-py2.7.egg/mkdocs/search.py", line 145, in handle_data
    self.section.text.append(data.rstrip('\n'))
AttributeError: 'unicode' object has no attribute 'text'

[d0ugal]

Ah, good point. I probably havent tested with unicode. I have only fully tested with the MkDocs documentation.

[d0ugal]

@fedelibre I just tried with a project containing Unicode (I borrowed the markdown from https://github.com/dongww/php-book). It worked without any issues.

Can you show me the project you tried with? It's hard for me to fix when I can't reproduce it yet.

[tomchristie]

@d0ugal also side question: does this use the same format as generated with JSON builds?

[fedelibre]

@d0ugal I've tried a simple project from scratch and it works fine.
The project I'm using is personal and not on public websites. I can send you a link to it privately by email. I don't know how to debug it and have no idea about the possible problems.

[d0ugal]

@tomchristie They use a similar, but different structure. I was hoping to speak to Eric and see if they could make use of this, if so we could sepricate the json command at some point... It appears Tipue isn't flexible

[d0ugal]

@fedelibre I'll make a change to output some information - what file it is failing on. This might provide you with enough info or you can just send me that file perhaps.

[fedelibre]

@d0ugal I think that I found the reason of the error: if a markdown file doesn't start with an header (# Title) it fails. You can easily reproduce the error now.

[d0ugal]

aha, thanks!

[d0ugal]

@fedelibre That last commit should fix it for you.

[fedelibre]

Thanks, I can run the server and compile the site. But I have three issues:

1. IIUC, the search results contain a link to the headers of each file where the searched string is found. Highlighting the searched word when we click on any of the results would be a very nice feature (the only one I'm really missing now).
2. Is the search meant to work only in "server mode"?
   If I test the mkdocs website, the search works fine when running the server. But if I open the built site/index.html, I see that the URI generated by the search is missing the complete file name:
   file:///home/fede/src/mkdocs/site/?q=installation
   So I see just the list of files.
3. Finally, in my private project the search doesn't work at all. Whatever text I enter in the search box, I'm redirected to the page before the results are listed. On the terminal I can see just:
   code 404, message File not found
   I thought it could be related to the missing headers at the beginning of some files, but it's not (I've checked with find . -name '*.md' | xargs head -n 1 and fixed a couple of files). Any idea about the reason of the error? Let me know, if you want to investigate I can send my files privately.

[d0ugal]

1. IIUC, the search results contain a link to the headers of each file where the searched string is found. **Highlighting the searched word** when we click on any of the results would be a very nice feature (the only one I'm really missing now).

Good idea, I'm not sure how best to go about this but I'll give it some thought.

2. Is the search meant to work only in "server mode"?
If I test the mkdocs website, the search works fine when running the server. But if I open the built site/index.html, I see that the URI generated by the search is missing the complete file name:
    `file:///home/fede/src/mkdocs/site/?q=installation`
So I see just the list of files.

This should now work, fixed a minor issue in cb3a5f6.

3. Finally, in my private project the search doesn't work at all. Whatever text I enter in the search box, I'm redirected to the page before the results are listed. On the terminal I can see just:
    `code 404, message File not found`
    I thought it could be related to the missing headers at the beginning of some files, but it's not (I've checked with `find . -name '*.md' | xargs head -n 1` and fixed a couple of files). Any idea about the reason of the error? Let me know, if you want to investigate I can send my files privately.

Hmm, this is odd. I think I need to find a few online public documentation projects and test this out.

I also plan to greatly improve the unit test situation, that should hopefully help identify some issues too.

[d0ugal]

@jniggemann From a quick chat on IRC. Partly adding here to remind me to look as I go on holdays soon :)

<dubiosos> How can I install the search functionality provided by d0ugal (see https://github.com/tomchristie/mkdocs/pull/222). I did pip install git+https://github.com/d0ugal/mkdocs@search and triggered a new build (mkdocs build --clean), but there is no search.html created
<d0ugal> dubiosos: what theme?
<d0ugal> ReadTheDocs will have a search.html, mkdocs does it in a modal and wont
<d0ugal> None of the other themes support it yet
<dubiosos> I do use the rtd theme
<dubiosos> I did include include_search: true in my mkdocs.xml file, but I am not sure if it is needed though
<d0ugal> dubiosos: it is
<d0ugal> dubiosos: I'm not sure what is going wrong
<dubiosos> I uploaded my current working branch at https://github.com/dirkk/basex-rtd/tree/search
<dubiosos> If you could take a look, I would be very grateful!

[tobegit3hub]

Is it ready to merge? It's a large patch and we're all looking foreword to use it. I hope we can merge it and improve it from time to time. @d0ugal 👍

[tomchristie]

Reminder to self - figure out integrating this with the Django REST framework theme.

[d0ugal]

I was planning on doing that, first I want to settle my doubts with Tipue.

[d0ugal]

To help move this along I plan to rebase and split the change up. I want to land the index generation first and then implement it in themes - disconnecting the index code from the client side search implementation.

[samuelcolvin]

This seemed to be on the brink of being merged in early December but now seems to have relapsed.

Is there any ETA for it being merged or has it been replaced by a Lunr.js approach?

[d0ugal]

@samuelcolvin I've been quietly working on a large refactor of this implementation and I've been getting close. So far I'm happier with the results but I still have work to do - I don't think it is quite fully functional yet.

See #457 for the new work being done here. I'll close this PR for now, I didn't want to rebase on this one as there is some interesting discussion etc. here.

[gustavosoares]

hello. what is the currently status for searching? is it working? I've just upgraded to version 0.12.2 and it is not working using readthedocs theme. I could not find anything related to search in http://www.mkdocs.org/

[d0ugal]

@gustavosoares Search will be released with 0.13.0 track progress at #502