-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Shortcode search #9
Conversation
- Data looks different: `icon` changed to `icon_${style}`
- noto-emoji is now noto, since idk if dashes will be happy - main.py should now get icon data from the correct place
- twemoji seems to work rly well w/ url naming scheme - noto/blobmoji don't quote work as well
See discussion in commit 8115b76 for more information
Also, DB Updated with new specs thanks to EmojiSpider modifications! :D
- Fixed scraper's incorrect var refs - Added Fallback Emoji Style Setting to alleviate emoji version differences - Added System Emoji display setting to further alleviate emoji version differences
Also, gave up on trying to make the extension icon dynamic. It's now hardcoded to twemoji
- Default Emoji style is now twemoji to support recognizability betwee the extension icon (also now twemoji) - Default fallback is noto, because that works better with twemoji than apple IMO
If the EmojiSpider can't get the resource, it will not insert the path of where the image would be into the database. This way, when we try to figure out whether to use the fallback, we'll know based on whether or not we get a None from row['icon_{ICON STYLE}']
Name-encoding bugAs you can see, the blood type emojis don't have any shortcodes associated. However, the shortcodes are on emojipedia, and should be present. It appears this is happening because the CLDR Shortnames (as listed on unicode.org) use punctuation that isn't correctly url-encoded, so when the spider tries to access them, it accesses a dead link. The full name of |
I'm not 100% sure on how I want UX to work here, but I had a few ideas. We'll use the 😄 emoji as an example: A: How to tell if the user is searching normally or searching 'by shortcode'
B: What to display on the search listing when searching by shortcode
Right now, I'm partway through implementing a setup with A.1 + B.1, but I'm honestly liking A.2 + B.2 better. I just don't quite know how I'd implement B.2. |
Implementing B.2 - Ideas
So, idea 1 would provide a more robust implementation of this feature, but idea 2 would be way easier, and we'd make less heavy architecture changes when doing so. |
B.2 Implementation 1 - Searching the new DB structureHere's the current query string that's used by SELECT em.name, em.code, em.keywords,
em.icon_apple, em.icon_twemoji, em.icon_noto, em.icon_blobmoji,
skt.icon_apple AS skt_icon_apple, skt.icon_twemoji AS skt_icon_twemoji,
skt.icon_noto AS skt_icon_noto, skt.icon_blobmoji AS skt_icon_blobmoji,
skt.code AS skt_code
FROM emoji AS em
LEFT JOIN skin_tone AS skt
ON skt.name = em.name AND tone = ?
WHERE shortcodes LIKE %?%
LIMIT 8 An individual entry retrieved w/ this query would return a With B.2 Impl 1's proposed changes, the search query would likely have to be something more like this: SELECT em.name, em.code, em.keywords,
em.icon_apple, em.icon_twemoji, em.icon_noto, em.icon_blobmoji,
skt.icon_apple AS skt_icon_apple, skt.icon_twemoji AS skt_icon_twemoji,
skt.icon_noto AS skt_icon_noto, skt.icon_blobmoji AS skt_icon_blobmoji,
skt.code AS skt_code, sc.code AS shortcode
FROM emoji AS em
LEFT JOIN skin_tone AS skt
ON skt.name = em.name AND tone = ?
LEFT JOIN shortcode AS sc
ON sc.name = em.name
WHERE shortcode LIKE ?%
LIMIT 8 Not 100% if this actually works, since I had to familiarize myself with how |
At this point, I think I'm probably ready to draft A.2 + B.2.1 as a full solution. |
See convo starting at Ulauncher#9 (comment) for details
See Ulauncher#9 (comment) for details.
Also added some nice comments, and Implemented a bit of A.2. See the following thread for details: Ulauncher#9 (comment)
I'm noticing something super funky in the DB. It basically has no data on skin tone variations. Not super sure why. |
This bug seems like it might have been present since I first updated this with the EmojiSpider, since the page that it scrapes from doesn't even list modifier emojis anymore
Hi @MyriaCore I wasn't able to review your PRs yet. I hope I can find some time for that soon. |
Thanks @gornostal! Please review #8 first, as this PR contains code from there. |
skin tone modifiers should work perfectly now
I personally like the 'startswith :' approach better
At this point, unless I find some more bugs, the implementation is pretty much done. This PR should be ready for review. |
Thanks a lot for these improvements and sorry it took so long! |
Closes #7. Blocked by #8 (since this PR includes commits from #8). Review blocks merge of #8.
While this MR is being reviewed, users can install my fork via Ulauncher's plugin install system, since the
shortcode-search
branch has been merged intomaster
as of MyriaCore#2.New Features / Changes
EmojiSpider.py
now scrapes pages on emojipedia.org to retrieve shortcode information:
. Otherwise, CLDR names will be searched as per usual.Technical Information
I've added a
shortcode
table to theemoji.sqlite
database: