Add out of the box emoji handling

[tooaverage]

I think it'd be be helpful to have basic emoji handling. And eventually understanding positive

😃
/negative

😡
/neutral

🐩
emoji.

Can assign to me

😇

[tmbo]

@tooaverage yes that would be an awesome contribution 🎆 😃

[tooaverage]

Weo @tmbo What is the best/leanest way to integrate this?

[amn41]

one pretty lean solution would be to replace the emoji with words, for example taking the short name or keywords http://unicode.org/emoji/charts/full-emoji-list.html

[gildastone]

Hello @tooaverage, any update on this issue?

I trained the model using both unicode (\U0001f37a) and the actual emoji (🍺) within the training data, and set synonyms to map to my named entity. No success (tried just in case).

[nicksahler]

This project seems to have built a dictionary of vectors for emoji -

is this useful?

https://raw.githubusercontent.com/uclmr/emoji2vec/master/pre-trained/
(the paper this belongs to is also great)

Is there a way I can assign these vectors to emoji tokens if they're found?

[amn41]

spacymoji would be helpful here: https://pypi.python.org/pypi/spacymoji/1.0.0

[giorgobiani]

Any progress on this? 😊

[tmbo]

We have not been working on this, but it would be a great contribution 😉

[parthsharma1996]

Might be a little off-topic but how are emoji's handled currently by the NLU? I would like the include emojis in our current chatbot's training data.

Are they currently simply ignored while predicting the intent?

[tmbo]

ok i think that depends on the intent classifcation component used:

• for svm + spacy: the emojis will be ignored because they do not have a word vector assigned and hence don't contribute anything to the sentence representation
• for the embedding policy, actually I am not sure <- @Ghostvv

[parthsharma1996]

@tmbo In the tensorflow_embedding pipeline my guess is that the vectors for the emojis should also be learned from the training data, since that pipeline learns the word embeddings from only the training data anyway (thus able to handle OOV words).
Guesses aside, would be interesting to know how it actually happens though

[giorgobiani]

I tried with tensorflow_embedding but no luck so far, but I was trying with actual emojis. Have to try with codes/synonyms

[akelad]

I think the tensorflow embedding policy also ignores them, because it only looks at words with a certain amount of characters. You can change that in the token_pattern parameter of the intent_featurizer_count_vectors though

[Ghostvv]

Yes, it depends whether a python string that stores emoji is falling under token_pattern or not

[kirtisynap19]

Hey @Ghostvv for now I am using token_pattern as "(?u)\b\w+\b" for words. However, I also want to add a capability to recognize emoji, what should be the token_pattern then?
Secondly, what would the training data be like for emojis, "👍" or "U+1F44D"?

[Ghostvv]

@kirtisynap19 in this case token pattern should correspond to regex that also picks emojis

[kirtisynap19]

@Ghostvv thanks for your response. And what goes in the training data? Is it '👍' or 'U+1F44D' or 'u"\U0001F44D"' or "\uD83D\uDC4D"?

Token pattern: (?u)(\b\w+\b|(\u00a9|\u00ae|[\u2000-\u3300]|\ud83c[\ud000-\udfff]|\ud83d[\ud000-\udfff]|\ud83e[\ud000-\udfff]))

[Ghostvv]

@kirtisynap19 I'm not sure, try both. You can print vocabulary of CounntVectorizer to check

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[mbslet]

Hi, this issue is available? @lucasdutraf, @Henrike100, and I to work on it! :)

[saikiran2603]

@mbslet Are you working on this change ? if not, i would like to pick it up !

[chiranshu14]

Has there been any progress on this? Or should I just do rule-based intent classification for this? :)

[giorgobiani]

We've used pretrained emoji embeddings (emoji2vec) and wrote custom featurizer. After you just add it to your pipeline and provide relevant examples (as you do for the text). It also should be noted that it's better to do text preprocessing before featurizing it, because if you have something like "Hello there 😈😈😈" where text and emoji are mixed, the final classification result is not always good. So what we do for example, is that we strip emojis if they are with text and classify only text and if there's no text and only emojis, we classify emoji, which gives consistent results. Overall it may sound hard, but it's not, it's just a cycle of trial and error while you get the version that works for you best.

[sync-by-unito]

➤ Maxime Verger commented:

💡 Heads up! We're moving issues to Jira: https://rasa-open-source.atlassian.net/browse/OSS.

From now on, this Jira board is the place where you can browse (without an account) and create issues (you'll need a free Jira account for that). This GitHub issue has already been migrated to Jira and will be closed on January 9th, 2023. Do not forget to subscribe to the corresponding Jira issue!

➡️ More information in the forum: https://forum.rasa.com/t/migration-of-rasa-oss-issues-to-jira/56569.

[mishra011]

We've used pretrained emoji embeddings (emoji2vec) and wrote custom featurizer. After you just add it to your pipeline and provide relevant examples (as you do for the text). It also should be noted that it's better to do text preprocessing before featurizing it, because if you have something like "Hello there 😈😈😈" where text and emoji are mixed, the final classification result is not always good. So what we do for example, is that we strip emojis if they are with text and classify only text and if there's no text and only emojis, we classify emoji, which gives consistent results. Overall it may sound hard, but it's not, it's just a cycle of trial and error while you get the version that works for you best.

Please explain it with an example. That will be helpful. Thanks @giorgobiani