Epic: Support dictionary/data lookup in MacVim #1311
Labels
Epic
Large tasks to be split up into smaller chunks
Comments
ychin
added a commit
to ychin/macvim
that referenced
this issue
Oct 11, 2022
This adds support for looking up data under the mouse cursor. Usually it will bring up a dictionary, but other times it could be a Wikipedia article, Siri knowledge, etc. Apple doesn't really have a good name for it, other than "looking up data", "quick look" (a confusingly similar name with the other Quick Look OS feature), or "show definition". You can activate this by doing Ctrl-Cmd-D when the mouse is over a cursor. If you have a trackpad, you can also either activate this using Force click or three-finger tap (depends on your system preference settings). Note that for Force click, this could potentially make it impossible to use the MacVim `<ForceClick>` mapping in Vim, which allows you to map a force click to a Vim command (macvim-dev#716). This is handled by having a new setting (under a new "Input" preference pane which will have more populated later) that allows you to choose whether to use Force click for data lookup or Vim's `<ForceClick>` mapping. If you have configured to use three-finger taps though this setting wouldn't do anything, and `<ForceClick>` is always send to the Vim mapping. Also, this is lacking a lot of features that a normal macOS application would get, e.g. looking up selected texts (e.g. if you have "ice cream", you may want to select the whole thing to look up the phrase, rather than just "ice" or "cream"), data detector, and much more (e.g. custom API support). They will be done later as part of macvim-dev#1311. Fix macvim-dev#1191 Part of Epic macvim-dev#1311, which contains other items to be implemented. Technical details below: The way the OS knows how to look up the data and present it is by talking to the NSTextInput/NSTextInputClient. Previously MacVim implemented NSTextInput partially, and didn't implement the critical firstRectForCharacterRange:actualRange and characterIndexForPoint: functions. First, in this change we change from NSTextInput to NSTextInputClient (which is the newer non-deprecated version), and implement those functions, which allows the OS to query the text storage. By default, the OS sends a quickLookWithEvent: call to us whenever the lookup happens but for some odd reason this isn't automatic for Force clicks, presumably because some apps want to handle Force clicks manually (this is why some apps only work for three-finger taps but not Force clicks for lookups). This isn't documented but I found references in iTerm/Firefox, and basically we just need to manually handle it and send off quickLookWithEvent: when handling Force clicks. For implementing the NSTextInputClient properly, the main issue is making sure that can work properly with input methods / marked texts, which is the other primary purpose for this class (other than inputting keys). For data lookups, I'm represending the grid as a row-major text (with no newline/space in between) and expose that to the OS. This already has some issue because it doesn't handle Vim vertical splits well, as MacVim doesn't really have access to detailed Vim text buffers easily (unless we do a lot of calls back-and-forth). This means wrapped texts won't be looked up properly, which I think is ok. Also, the OS APIs deal with UTF-8 indices, so we can't just convert row/column to raw indices and have to do a lot of character length calculations (especially for wide chars like CJK or emojis) to make sure the returned ranges are consistent and valid. For marked texts though, this presents a challenge because Vim doesn't really have a strong enough API to communicate back-and-forth about the marked positions and whatnot (it only let the GUI know where the current cursor is), and it's hard to implement APIs like `markedRange` properly because some marked texts could be hidden or wrapped (if you implement some of these functions improperly Apple's input methods could start misbehaving especially when you use arrow keys to navigate). In the end I kept the original implementation for treating the marked texts as a range starting from 0, *only* when we have marked text. Kind of a hack but this makes sure we work both in marked text mode (i.e. when inputting texts) and when doing lookups. For simplicity I made it so that you can't do data lookups when in marked text mode now. Data detection: Note that the default implementation is quite bare, and lacks a lot of smart data detection. For example, if you put your mouse over a URL, it won't properly select the whole URL, and addresses and dates for example also won't get grouped together properly. This is because these require additional implementation (e.g. using NSDataDetector) instead of coming "for free", and will be handled later. In fact, Apple's WebKit and NSTextView cheats by calling an internal API framework called "Reveal" (which you can find out by intercepting NSTextView's calls and/or looking at WebKit's source code) which is much more powerful and supports looking up package tracking, airline info, and more, but it's not available to third-party software (that's why Safari's lookup is so much better than Chrome/Firefox's). This isn't tested right now. Future task needs to add XCTest support to properly test this as there are a lot of edge cases involved here.
ychin
added a commit
to ychin/macvim
that referenced
this issue
Oct 11, 2022
This adds support for looking up data under the mouse cursor. Usually it will bring up a dictionary, but other times it could be a Wikipedia article, Siri knowledge, etc. Apple doesn't really have a good name for it, other than "looking up data", "quick look" (a confusingly similar name with the other Quick Look OS feature), or "show definition". You can activate this by doing Ctrl-Cmd-D when the mouse is over a cursor. If you have a trackpad, you can also either activate this using Force click or three-finger tap (depends on your system preference settings). Note that for Force click, this could potentially make it impossible to use the MacVim `<ForceClick>` mapping in Vim, which allows you to map a force click to a Vim command (macvim-dev#716). This is handled by having a new setting (under a new "Input" preference pane which will have more populated later) that allows you to choose whether to use Force click for data lookup or Vim's `<ForceClick>` mapping. If you have configured to use three-finger taps though this setting wouldn't do anything, and `<ForceClick>` is always send to the Vim mapping. Also, this is lacking a lot of features that a normal macOS application would get, e.g. looking up selected texts (e.g. if you have "ice cream", you may want to select the whole thing to look up the phrase, rather than just "ice" or "cream"), data detector, and much more (e.g. custom API support). They will be done later as part of macvim-dev#1311. Technical details below: The way the OS knows how to look up the data and present it is by talking to the NSTextInput/NSTextInputClient. Previously MacVim implemented NSTextInput partially, and didn't implement the critical firstRectForCharacterRange:actualRange and characterIndexForPoint: functions. First, in this change we change from NSTextInput to NSTextInputClient (which is the newer non-deprecated version), and implement those functions, which allows the OS to query the text storage. By default, the OS sends a quickLookWithEvent: call to us whenever the lookup happens but for some odd reason this isn't automatic for Force clicks, presumably because some apps want to handle Force clicks manually (this is why some apps only work for three-finger taps but not Force clicks for lookups). This isn't documented but I found references in iTerm/Firefox, and basically we just need to manually handle it and send off quickLookWithEvent: when handling Force clicks. For implementing the NSTextInputClient properly, the main issue is making sure that can work properly with input methods / marked texts, which is the other primary purpose for this class (other than inputting keys). For data lookups, I'm represending the grid as a row-major text (with no newline/space in between) and expose that to the OS. This already has some issue because it doesn't handle Vim vertical splits well, as MacVim doesn't really have access to detailed Vim text buffers easily (unless we do a lot of calls back-and-forth). This means wrapped texts won't be looked up properly, which I think is ok. Also, the OS APIs deal with UTF-8 indices, so we can't just convert row/column to raw indices and have to do a lot of character length calculations (especially for wide chars like CJK or emojis) to make sure the returned ranges are consistent and valid. For marked texts though, this presents a challenge because Vim doesn't really have a strong enough API to communicate back-and-forth about the marked positions and whatnot (it only let the GUI know where the current cursor is), and it's hard to implement APIs like `markedRange` properly because some marked texts could be hidden or wrapped (if you implement some of these functions improperly Apple's input methods could start misbehaving especially when you use arrow keys to navigate). In the end I kept the original implementation for treating the marked texts as a range starting from 0, *only* when we have marked text. Kind of a hack but this makes sure we work both in marked text mode (i.e. when inputting texts) and when doing lookups. For simplicity I made it so that you can't do data lookups when in marked text mode now. Data detection: Note that the default implementation is quite bare, and lacks a lot of smart data detection. For example, if you put your mouse over a URL, it won't properly select the whole URL, and addresses and dates for example also won't get grouped together properly. This is because these require additional implementation (e.g. using NSDataDetector) instead of coming "for free", and will be handled later. In fact, Apple's WebKit and NSTextView cheats by calling an internal API framework called "Reveal" (which you can find out by intercepting NSTextView's calls and/or looking at WebKit's source code) which is much more powerful and supports looking up package tracking, airline info, and more, but it's not available to third-party software (that's why Safari's lookup is so much better than Chrome/Firefox's). This isn't tested right now. Future task needs to add XCTest support to properly test this as there are a lot of edge cases involved here. Fix macvim-dev#1191 Part of Epic macvim-dev#1311, which contains other items to be implemented.
ychin
added a commit
to ychin/macvim
that referenced
this issue
Oct 11, 2022
This adds support for looking up data under the mouse cursor. Usually it will bring up a dictionary, but other times it could be a Wikipedia article, Siri knowledge, etc. Apple doesn't really have a good name for it, other than "looking up data", "quick look" (a confusingly similar name with the other Quick Look OS feature), or "show definition". You can activate this by doing Ctrl-Cmd-D when the mouse is over a cursor. If you have a trackpad, you can also either activate this using Force click or three-finger tap (depends on your system preference settings). Note that for Force click, this could potentially make it impossible to use the MacVim `<ForceClick>` mapping in Vim, which allows you to map a force click to a Vim command (macvim-dev#716). This is handled by having a new setting (under a new "Input" preference pane which will have more populated later) that allows you to choose whether to use Force click for data lookup or Vim's `<ForceClick>` mapping. If you have configured to use three-finger taps though this setting wouldn't do anything, and `<ForceClick>` is always send to the Vim mapping. Also, this is lacking a lot of features that a normal macOS application would get, e.g. looking up selected texts (e.g. if you have "ice cream", you may want to select the whole thing to look up the phrase, rather than just "ice" or "cream"), data detector, and much more (e.g. custom API support). They will be done later as part of macvim-dev#1311. Technical details below: The way the OS knows how to look up the data and present it is by talking to the NSTextInput/NSTextInputClient. Previously MacVim implemented NSTextInput partially, and didn't implement the critical firstRectForCharacterRange:actualRange and characterIndexForPoint: functions. First, in this change we change from NSTextInput to NSTextInputClient (which is the newer non-deprecated version), and implement those functions, which allows the OS to query the text storage. By default, the OS sends a quickLookWithEvent: call to us whenever the lookup happens but for some odd reason this isn't automatic for Force clicks, presumably because some apps want to handle Force clicks manually (this is why some apps only work for three-finger taps but not Force clicks for lookups). This isn't documented but I found references in iTerm/Firefox, and basically we just need to manually handle it and send off quickLookWithEvent: when handling Force clicks. For implementing the NSTextInputClient properly, the main issue is making sure that can work properly with input methods / marked texts, which is the other primary purpose for this class (other than inputting keys). For data lookups, I'm representing the grid as a row-major text (with no newline/space in between) and expose that to the OS. This already has some issue because it doesn't handle Vim vertical splits well, as MacVim doesn't really have access to detailed Vim text buffers easily (unless we do a lot of calls back-and-forth). This means wrapped texts won't be looked up properly, which I think is ok. Also, the OS APIs deal with UTF-8 indices, so we can't just convert row/column to raw indices and have to do a lot of character length calculations (especially for wide chars like CJK or emojis) to make sure the returned ranges are consistent and valid. For marked texts though, this presents a challenge because Vim doesn't really have a strong enough API to communicate back-and-forth about the marked positions and whatnot (it only let the GUI know where the current cursor is), and it's hard to implement APIs like `markedRange` properly because some marked texts could be hidden or wrapped (if you implement some of these functions improperly Apple's input methods could start misbehaving especially when you use arrow keys to navigate). In the end I kept the original implementation for treating the marked texts as a range starting from 0, *only* when we have marked text. Kind of a hack but this makes sure we work both in marked text mode (i.e. when inputting texts) and when doing lookups. For simplicity I made it so that you can't do data lookups when in marked text mode now. Data detection: Note that the default implementation is quite bare, and lacks a lot of smart data detection. For example, if you put your mouse over a URL, it won't properly select the whole URL, and addresses and dates for example also won't get grouped together properly. This is because these require additional implementation (e.g. using NSDataDetector) instead of coming "for free", and will be handled later. In fact, Apple's WebKit and NSTextView cheats by calling an internal API framework called "Reveal" (which you can find out by intercepting NSTextView's calls and/or looking at WebKit's source code) which is much more powerful and supports looking up package tracking, airline info, and more, but it's not available to third-party software (that's why Safari's lookup is so much better than Chrome/Firefox's). This isn't tested right now. Future task needs to add XCTest support to properly test this as there are a lot of edge cases involved here. Fix macvim-dev#1191 Part of Epic macvim-dev#1311, which contains other items to be implemented.
ychin
added a commit
to ychin/macvim
that referenced
this issue
Oct 11, 2022
This adds support for looking up data under the mouse cursor. Usually it will bring up a dictionary, but other times it could be a Wikipedia article, Siri knowledge, etc. Apple doesn't really have a good name for it, other than "looking up data", "quick look" (a confusingly similar name with the other Quick Look OS feature), or "show definition". You can activate this by doing Ctrl-Cmd-D when the mouse is over a cursor. If you have a trackpad, you can also either activate this using Force click or three-finger tap (depends on your system preference settings). Note that for Force click, this could potentially make it impossible to use the MacVim `<ForceClick>` mapping in Vim, which allows you to map a force click to a Vim command (macvim-dev#716). This is handled by having a new setting (under a new "Input" preference pane which will have more populated later) that allows you to choose whether to use Force click for data lookup or Vim's `<ForceClick>` mapping. If you have configured to use three-finger taps though this setting wouldn't do anything, and `<ForceClick>` is always send to the Vim mapping. Also, this is lacking a lot of features that a normal macOS application would get, e.g. looking up selected texts (e.g. if you have "ice cream", you may want to select the whole thing to look up the phrase, rather than just "ice" or "cream"), data detector, and much more (e.g. custom API support). They will be done later as part of macvim-dev#1311. Technical details below: The way the OS knows how to look up the data and present it is by talking to the NSTextInput/NSTextInputClient. Previously MacVim implemented NSTextInput partially, and didn't implement the critical firstRectForCharacterRange:actualRange and characterIndexForPoint: functions. First, in this change we change from NSTextInput to NSTextInputClient (which is the newer non-deprecated version), and implement those functions, which allows the OS to query the text storage. By default, the OS sends a quickLookWithEvent: call to us whenever the lookup happens but for some odd reason this isn't automatic for Force clicks, presumably because some apps want to handle Force clicks manually (this is why some apps only work for three-finger taps but not Force clicks for lookups). This isn't documented but I found references in iTerm/Firefox, and basically we just need to manually handle it and send off quickLookWithEvent: when handling Force clicks. For implementing the NSTextInputClient properly, the main issue is making sure that can work properly with input methods / marked texts, which is the other primary purpose for this class (other than inputting keys). For data lookups, I'm representing the grid as a row-major text (with no newline/space in between) and expose that to the OS. This already has some issue because it doesn't handle Vim vertical splits well, as MacVim doesn't really have access to detailed Vim text buffers easily (unless we do a lot of calls back-and-forth). This means wrapped texts won't be looked up properly, which I think is ok. Also, the OS APIs deal with UTF-8 indices, so we can't just convert row/column to raw indices and have to do a lot of character length calculations (especially for wide chars like CJK or emojis) to make sure the returned ranges are consistent and valid. For marked texts though, this presents a challenge because Vim doesn't really have a strong enough API to communicate back-and-forth about the marked positions and whatnot (it only let the GUI know where the current cursor is), and it's hard to implement APIs like `markedRange` properly because some marked texts could be hidden or wrapped (if you implement some of these functions improperly Apple's input methods could start misbehaving especially when you use arrow keys to navigate). In the end I kept the original implementation for treating the marked texts as a range starting from 0, *only* when we have marked text. Kind of a hack but this makes sure we work both in marked text mode (i.e. when inputting texts) and when doing lookups. For simplicity I made it so that you can't do data lookups when in marked text mode now. Input method: This change also fixes a quirk in input method as a driveby change. Previously the logic for calculating the rect for where the candidate list was quite broken, but now it's calculated correctly using the desired range and the current cursor position. This matters when say using Japanese IM and using the left/right arrow to jump to different sections of the text. If the desired range is in a wrapped line, the new logic would attempt to pin it to the left-most column of where the cursor is in the range. Data detection: Note that the default implementation is quite bare, and lacks a lot of smart data detection. For example, if you put your mouse over a URL, it won't properly select the whole URL, and addresses and dates for example also won't get grouped together properly. This is because these require additional implementation (e.g. using NSDataDetector) instead of coming "for free", and will be handled later. In fact, Apple's WebKit and NSTextView cheats by calling an internal API framework called "Reveal" (which you can find out by intercepting NSTextView's calls and/or looking at WebKit's source code) which is much more powerful and supports looking up package tracking, airline info, and more, but it's not available to third-party software (that's why Safari's lookup is so much better than Chrome/Firefox's). This isn't tested right now. Future task needs to add XCTest support to properly test this as there are a lot of edge cases involved here. Fix macvim-dev#1191 Part of Epic macvim-dev#1311, which contains other items to be implemented.
|
I wonder if there will be an option to disable this feature. It would be even better if I can control what to disable. e.g. if I'd like to disable macOS dictionary lookups leaving URL recognition. |
|
Why would you need to disable it? This only triggers per user interaction. If you don’t like the built in dictionary you can just not trigger the lookup which is quite hard to accidentally do. |
|
Yep, I don't want to trigger is dictionary lookup. Dictionary for Polish, primary language I use beside English during writing a code is so poor and I there's no way to install a custom one, so I don't use this feature in macOS at all as it's completely useless. Additionally, I like MacVim to be a editor with lightweight OS integration when I don't need it. Also I like Vim itself for an ability to customise it. As a possible solution I propose make operations noop. I agree, that custom integration is a good thing, but not the macOS dictionary. |
|
I still don’t understand under what circumstance would this be an issue for you? Can you be more specific? I’m not asking why you don’t like the built in dictionary (note that this allows looking up other data like general knowledge articles, not just definitions of words). I'm asking why we would need an option to turn this off. Only known ways I know to trigger this:
I just don’t understand why you would care if you never use this feature. Most macOS applications support this so this is a pretty basic OS integration. I think options make sense when there is a reason for them but not when the feature is completely opt in to begin with. |
ychin
added a commit
to ychin/macvim
that referenced
this issue
Oct 15, 2022
Add a new Vim script function `showdefinition()` that allows Vim script to call back to macOS's data lookup feature and show the definition / URL preview / etc for any text, at a designated row/col position. If the row/col are not provided this function will just show it at the cursor. Also, add a new autoload/macvim.vim for utility functions to call showdefinition() for selected texts and the word under cursor. Make a new right-click popup menu "Look Up" call that when there are selected texts in visual mode to make the lookup functionality easier to access for users without a trackpad (since Ctrl-Cmd-D is a little obscure and unwieldy to use). For the utility functions, it was a little hard to determine how to get the text under visual selection without yanking (we don't want to pollute the register here), and just implemented a function to take care of all the edge cases including visual/block/line modes and selection=exclusive. It could be useful in other situations. As a side refactor, change the message handler in MacVim from if/else to switch case. In optimized builds, they both essentially optimize to the same thing, but for debug builds, the if/else statements have to step through one by one, and switch case just makes more sense for a giant message ID lookup like this. Part of Epic macvim-dev#1311
ychin
added a commit
to ychin/macvim
that referenced
this issue
Oct 15, 2022
Add a new Vim script function `showdefinition()` that allows Vim script to call back to macOS's data lookup feature and show the definition / URL preview / etc for any text, at a designated row/col position. If the row/col are not provided this function will just show it at the cursor. Also, add a new autoload/macvim.vim for utility functions to call showdefinition() for selected texts and the word under cursor. Make a new right-click popup menu "Look Up" call that when there are selected texts in visual mode to make the lookup functionality easier to access for users without a trackpad (since Ctrl-Cmd-D is a little obscure and unwieldy to use). For the utility functions, it was a little hard to determine how to get the text under visual selection without yanking (we don't want to pollute the register here), and just implemented a function to take care of all the edge cases including visual/block/line modes and selection=exclusive. It could be useful in other situations. As a side refactor, change the message handler in MacVim from if/else to switch case. In optimized builds, they both essentially optimize to the same thing, but for debug builds, the if/else statements have to step through one by one, and switch case just makes more sense for a giant message ID lookup like this. Part of Epic macvim-dev#1311
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It is useful to support the ability to look up a piece of selected / highlighted text in MacVim and show a OS popup that gives you more information. #1191 was the original issue, but this work has ballooned as this becomes more ambitious and need to span multiple commits. Using this epic to track the work and elicit ideas / brainstorm.
showdefinition()so that we can programmatically invoke this.docfolder for technical details and general guidelines. Would be a nice time to update thesrc/MacVim/READMEfile which hasn't been updated forever.The text was updated successfully, but these errors were encountered: