Skip to content

Commit

Permalink
Auto merge of #2863 - puremourning:java-async-diagnostics, r=micbou
Browse files Browse the repository at this point in the history
[READY] Java support with asynchronous diagnostics and messages

# PR Prelude

Thank you for working on YCM! :)

**Please complete these steps and check these boxes (by putting an `x` inside
the brackets) _before_ filing your PR:**

- [x] I have read and understood YCM's [CONTRIBUTING][cont] document.
- [x] I have read and understood YCM's [CODE_OF_CONDUCT][code] document.
- [x] I have included tests for the changes in my PR. If not, I have included a
  rationale for why I haven't.
- [x] **I understand my PR may be closed if it becomes obvious I didn't
  actually perform all of these steps.**

# Why this change is necessary and useful

This change is required for a better user experience when using native
java support

This implements an asynchronous message system using a long-poll request
to the server.

The server provides an endpoint /receive_messages which blocks until
either a timeout occurs or we receive a batch of asynchronous messages.
We send this request asynchronously and poll it 4 times a second to see
if we have received any messages.

The messages may either be simply for display (such as startup progress)
or diagnostics, which override the diagnostics returned by
OnFileReqdyToParse.

In the former case, we simply display the message, accepting that this
might be overwritten by any other message (indeed, requiring this), and
for the latter we fan out diagnostics to any open buffer for the file in
question.

Unfortunately, Vim has bugs related to timers when there is something
displayed (such as a "confirm" prompt or other), so we suspend
background timers when doing subcommands to avoid vim bugs. NOTE: This
requires a new version of Vim (detected by the presence of the
particular functions used).

NOT_READY because:

- the submodule commit points at my repo and requires ycm-core/ycmd#857 to be merged
- my spider sense suggest i have more testing to do...

Notes:

- Part 3 (I think) of the Java support PRs. This one actually adds the minimal changes for working java support
- There are about 2 or 3 other PRs to come to add things like automatic module imports, etc.

[Please explain **in detail** why the changes in this PR are needed.]

[cont]: https://github.com/Valloric/YouCompleteMe/blob/master/CONTRIBUTING.md
[code]: https://github.com/Valloric/YouCompleteMe/blob/master/CODE_OF_CONDUCT.md

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/valloric/youcompleteme/2863)
<!-- Reviewable:end -->
  • Loading branch information
zzbot authored Feb 10, 2018
2 parents 26a5e3c + 690a763 commit 0d49557
Show file tree
Hide file tree
Showing 11 changed files with 1,226 additions and 73 deletions.
251 changes: 216 additions & 35 deletions README.md

Large diffs are not rendered by default.

43 changes: 38 additions & 5 deletions autoload/youcompleteme.vim
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ let s:pollers = {
\ 'server_ready': {
\ 'id': -1,
\ 'wait_milliseconds': 100
\ },
\ 'receive_messages': {
\ 'id': -1,
\ 'wait_milliseconds': 100
\ }
\ }

Expand Down Expand Up @@ -71,6 +75,29 @@ function! s:Pyeval( eval_string )
endfunction


function! s:StartMessagePoll()
if s:pollers.receive_messages.id < 0
let s:pollers.receive_messages.id = timer_start(
\ s:pollers.receive_messages.wait_milliseconds,
\ function( 's:ReceiveMessages' ) )
endif
endfunction


function! s:ReceiveMessages( timer_id )
let poll_again = s:Pyeval( 'ycm_state.OnPeriodicTick()' )

if poll_again
let s:pollers.receive_messages.id = timer_start(
\ s:pollers.receive_messages.wait_milliseconds,
\ function( 's:ReceiveMessages' ) )
else
" Don't poll again until we open another buffer
let s:pollers.receive_messages.id = -1
endif
endfunction


function! youcompleteme#Enable()
call s:SetUpBackwardsCompatibility()

Expand Down Expand Up @@ -451,6 +478,7 @@ function! s:OnFileTypeSet()

call s:SetUpCompleteopt()
call s:SetCompleteFunc()
call s:StartMessagePoll()

exec s:python_command "ycm_state.OnBufferVisit()"
call s:OnFileReadyToParse( 1 )
Expand All @@ -464,6 +492,7 @@ function! s:OnBufferEnter()

call s:SetUpCompleteopt()
call s:SetCompleteFunc()
call s:StartMessagePoll()

exec s:python_command "ycm_state.OnBufferVisit()"
" Last parse may be outdated because of changes from other buffers. Force a
Expand Down Expand Up @@ -801,6 +830,10 @@ endfunction

function! s:RestartServer()
exec s:python_command "ycm_state.RestartServer()"

call timer_stop( s:pollers.receive_messages.id )
let s:pollers.receive_messages.id = -1

call timer_stop( s:pollers.server_ready.id )
let s:pollers.server_ready.id = timer_start(
\ s:pollers.server_ready.wait_milliseconds,
Expand Down Expand Up @@ -828,11 +861,11 @@ endfunction


function! s:CompleterCommand(...)
" CompleterCommand will call the OnUserCommand function of a completer.
" If the first arguments is of the form "ft=..." it can be used to specify the
" completer to use (for example "ft=cpp"). Else the native filetype completer
" of the current buffer is used. If no native filetype completer is found and
" no completer was specified this throws an error. You can use
" CompleterCommand will call the OnUserCommand function of a completer. If
" the first arguments is of the form "ft=..." it can be used to specify the
" completer to use (for example "ft=cpp"). Else the native filetype
" completer of the current buffer is used. If no native filetype completer
" is found and no completer was specified this throws an error. You can use
" "ft=ycm:ident" to select the identifier completer.
" The remaining arguments will be passed to the completer.
let arguments = copy(a:000)
Expand Down
31 changes: 26 additions & 5 deletions python/ycm/buffer.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,17 +27,23 @@
from ycm.diagnostic_interface import DiagnosticInterface


DIAGNOSTIC_UI_FILETYPES = set( [ 'cpp', 'cs', 'c', 'objc', 'objcpp',
'typescript' ] )
DIAGNOSTIC_UI_ASYNC_FILETYPES = set( [ 'java' ] )


# Emulates Vim buffer
# Used to store buffer related information like diagnostics, latest parse
# request. Stores buffer change tick at the parse request moment, allowing
# to effectively determine whether reparse is needed for the buffer.
class Buffer( object ):

def __init__( self, bufnr, user_options ):
def __init__( self, bufnr, user_options, async_diags ):
self.number = bufnr
self._parse_tick = 0
self._handled_tick = 0
self._parse_request = None
self._async_diags = async_diags
self._diag_interface = DiagnosticInterface( bufnr, user_options )


Expand All @@ -60,9 +66,18 @@ def NeedsReparse( self ):
return self._parse_tick != self._ChangedTick()


def UpdateDiagnostics( self ):
self._diag_interface.UpdateWithNewDiagnostics(
self._parse_request.Response() )
def UpdateDiagnostics( self, force=False ):
if force or not self._async_diags:
self.UpdateWithNewDiagnostics( self._parse_request.Response() )
else:
# We need to call the response method, because it might throw an exception
# or require extra config confirmation, even if we don't actually use the
# diagnostics.
self._parse_request.Response()


def UpdateWithNewDiagnostics( self, diagnostics ):
self._diag_interface.UpdateWithNewDiagnostics( diagnostics )


def PopulateLocationList( self ):
Expand Down Expand Up @@ -105,5 +120,11 @@ def __init__( self, user_options ):

def __missing__( self, key ):
# Python does not allow to return assignment operation result directly
new_value = self[ key ] = Buffer( key, self._user_options )
new_value = self[ key ] = Buffer(
key,
self._user_options,
any( [ x in DIAGNOSTIC_UI_ASYNC_FILETYPES
for x in
vimsupport.GetBufferFiletypes( key ) ] ) )

return new_value
97 changes: 97 additions & 0 deletions python/ycm/client/messages_request.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Copyright (C) 2017 YouCompleteMe contributors
#
# This file is part of YouCompleteMe.
#
# YouCompleteMe is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# YouCompleteMe is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with YouCompleteMe. If not, see <http://www.gnu.org/licenses/>.

from __future__ import unicode_literals
from __future__ import print_function
from __future__ import division
from __future__ import absolute_import
# Not installing aliases from python-future; it's unreliable and slow.
from builtins import * # noqa

from ycm.vimsupport import PostVimMessage

from ycm.client.base_request import ( BaseRequest, BuildRequestData,
JsonFromFuture, HandleServerException )

import logging

_logger = logging.getLogger( __name__ )

# Looooong poll
TIMEOUT_SECONDS = 60


class MessagesPoll( BaseRequest ):
def __init__( self ):
super( MessagesPoll, self ).__init__()
self._request_data = BuildRequestData()
self._response_future = None


def _SendRequest( self ):
self._response_future = self.PostDataToHandlerAsync(
self._request_data,
'receive_messages',
timeout = TIMEOUT_SECONDS )
return


def Poll( self, diagnostics_handler ):
"""This should be called regularly to check for new messages in this buffer.
Returns True if Poll should be called again in a while. Returns False when
the completer or server indicated that further polling should not be done
for the requested file."""

if self._response_future is None:
# First poll
self._SendRequest()
return True

if not self._response_future.done():
# Nothing yet...
return True

with HandleServerException( display = False ):
response = JsonFromFuture( self._response_future )

poll_again = _HandlePollResponse( response, diagnostics_handler )
if poll_again:
self._SendRequest()
return True

return False


def _HandlePollResponse( response, diagnostics_handler ):
if isinstance( response, list ):
for notification in response:
if 'message' in notification:
PostVimMessage( notification[ 'message' ],
warning = False,
truncate = True )
elif 'diagnostics' in notification:
diagnostics_handler.UpdateWithNewDiagnosticsForFile(
notification[ 'filepath' ],
notification[ 'diagnostics' ] )
elif response is False:
# Don't keep polling for this file
return False
# else any truthy response means "nothing to see here; poll again in a
# while"

# Start the next poll (only if the last poll didn't raise an exception)
return True
3 changes: 2 additions & 1 deletion python/ycm/diagnostic_interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,8 @@ def _DiagnosticsCount( self, predicate ):


def _UpdateLocationList( self ):
vimsupport.SetLocationList(
vimsupport.SetLocationListForBuffer(
self._bufnr,
vimsupport.ConvertDiagnosticsToQfList( self._diagnostics ) )


Expand Down
142 changes: 142 additions & 0 deletions python/ycm/tests/client/messages_request_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
# Copyright (C) 2017 YouCompleteMe Contributors
#
# This file is part of YouCompleteMe.
#
# YouCompleteMe is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# YouCompleteMe is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with YouCompleteMe. If not, see <http://www.gnu.org/licenses/>.


from __future__ import unicode_literals
from __future__ import print_function
from __future__ import division
from __future__ import absolute_import
# Not installing aliases from python-future; it's unreliable and slow.
from builtins import * # noqa

from ycm.tests.test_utils import MockVimModule
MockVimModule()

from hamcrest import assert_that, equal_to
from mock import patch, call

from ycm.client.messages_request import _HandlePollResponse
from ycm.tests.test_utils import ExtendedMock


def HandlePollResponse_NoMessages_test():
assert_that( _HandlePollResponse( True, None ), equal_to( True ) )

# Other non-False responses mean the same thing
assert_that( _HandlePollResponse( '', None ), equal_to( True ) )
assert_that( _HandlePollResponse( 1, None ), equal_to( True ) )
assert_that( _HandlePollResponse( {}, None ), equal_to( True ) )


def HandlePollResponse_PollingNotSupported_test():
assert_that( _HandlePollResponse( False, None ), equal_to( False ) )

# 0 is not False
assert_that( _HandlePollResponse( 0, None ), equal_to( True ) )


@patch( 'ycm.client.messages_request.PostVimMessage',
new_callable = ExtendedMock )
def HandlePollResponse_SingleMessage_test( post_vim_message ):
assert_that( _HandlePollResponse( [ { 'message': 'this is a message' } ] ,
None ),
equal_to( True ) )

post_vim_message.assert_has_exact_calls( [
call( 'this is a message', warning=False, truncate=True )
] )


@patch( 'ycm.client.messages_request.PostVimMessage',
new_callable = ExtendedMock )
def HandlePollResponse_MultipleMessages_test( post_vim_message ):
assert_that( _HandlePollResponse( [ { 'message': 'this is a message' },
{ 'message': 'this is another one' } ] ,
None ),
equal_to( True ) )

post_vim_message.assert_has_exact_calls( [
call( 'this is a message', warning=False, truncate=True ),
call( 'this is another one', warning=False, truncate=True )
] )


def HandlePollResponse_SingleDiagnostic_test():
diagnostics_handler = ExtendedMock()
messages = [
{ 'filepath': 'foo', 'diagnostics': [ 'PLACEHOLDER' ] },
]
assert_that( _HandlePollResponse( messages, diagnostics_handler ),
equal_to( True ) )
diagnostics_handler.UpdateWithNewDiagnosticsForFile.assert_has_exact_calls( [
call( 'foo', [ 'PLACEHOLDER' ] )
] )


def HandlePollResponse_MultipleDiagnostics_test():
diagnostics_handler = ExtendedMock()
messages = [
{ 'filepath': 'foo', 'diagnostics': [ 'PLACEHOLDER1' ] },
{ 'filepath': 'bar', 'diagnostics': [ 'PLACEHOLDER2' ] },
{ 'filepath': 'baz', 'diagnostics': [ 'PLACEHOLDER3' ] },
{ 'filepath': 'foo', 'diagnostics': [ 'PLACEHOLDER4' ] },
]
assert_that( _HandlePollResponse( messages, diagnostics_handler ),
equal_to( True ) )
diagnostics_handler.UpdateWithNewDiagnosticsForFile.assert_has_exact_calls( [
call ( 'foo', [ 'PLACEHOLDER1' ] ),
call ( 'bar', [ 'PLACEHOLDER2' ] ),
call ( 'baz', [ 'PLACEHOLDER3' ] ),
call ( 'foo', [ 'PLACEHOLDER4' ] )
] )


@patch( 'ycm.client.messages_request.PostVimMessage',
new_callable = ExtendedMock )
def HandlePollResponse_MultipleMessagesAndDiagnostics_test( post_vim_message ):
diagnostics_handler = ExtendedMock()
messages = [
{ 'filepath': 'foo', 'diagnostics': [ 'PLACEHOLDER1' ] },
{ 'message': 'On the first day of Christmas, my VimScript gave to me' },
{ 'filepath': 'bar', 'diagnostics': [ 'PLACEHOLDER2' ] },
{ 'message': 'A test file in a Command-T' },
{ 'filepath': 'baz', 'diagnostics': [ 'PLACEHOLDER3' ] },
{ 'message': 'On the second day of Christmas, my VimScript gave to me' },
{ 'filepath': 'foo', 'diagnostics': [ 'PLACEHOLDER4' ] },
{ 'message': 'Two popup menus, and a test file in a Command-T' },
]
assert_that( _HandlePollResponse( messages, diagnostics_handler ),
equal_to( True ) )
diagnostics_handler.UpdateWithNewDiagnosticsForFile.assert_has_exact_calls( [
call ( 'foo', [ 'PLACEHOLDER1' ] ),
call ( 'bar', [ 'PLACEHOLDER2' ] ),
call ( 'baz', [ 'PLACEHOLDER3' ] ),
call ( 'foo', [ 'PLACEHOLDER4' ] )
] )

post_vim_message.assert_has_exact_calls( [
call( 'On the first day of Christmas, my VimScript gave to me',
warning=False,
truncate=True ),
call( 'A test file in a Command-T', warning=False, truncate=True ),
call( 'On the second day of Christmas, my VimScript gave to me',
warning=False,
truncate=True ),
call( 'Two popup menus, and a test file in a Command-T',
warning=False,
truncate=True ),
] )
Loading

0 comments on commit 0d49557

Please sign in to comment.