Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Enhancement: Improve "Trim Incomplete Sentences" detection. #824

Closed
Spacellary opened this issue Feb 5, 2024 · 0 comments · Fixed by #886
Closed

Feature Enhancement: Improve "Trim Incomplete Sentences" detection. #824

Spacellary opened this issue Feb 5, 2024 · 0 comments · Fixed by #886
Labels
approved The feature or enhancement is approved OR the bug is confirmed.

Comments

@Spacellary
Copy link

Spacellary commented Feb 5, 2024

Is your feature request related to a problem? Please describe.
When enforcing a shorter max_new_tokens limit sometimes it cuts replies in a way that leaves a lonely * or " at the end of the message, this can happen in 2 ways, sometimes it's in the same line, sometimes it's in a new line immediately after a complete sentence.

Describe the solution you'd like
Punctuation should be trimmed with the rest of the incomplete response if preceded by a (as appearing outside of edit mode) whitespace. Similar to SillyTavern/SillyTavern#1780.

Describe alternatives you've considered
I have tried adding possible matching entries to the list of Stop Sequences but this doesn't seem to be related to the issue or have any effect.

Additional context
Examples of undesirable responses due to trimming leaving some characters behind, notice at the end of responses:

*User's heart races, torn between his desire for her and his sense of self-worth. But deep down, he knows that he's in too deep, and there's no going back.*
*
*Her smile broadens at your words of gratitude.* "That's good. Now let's forget about all the troubles and just enjoy our time together, alright?" *She suggests with a playful wink, indicating she has more surprises in store for her loving boyfriend.* "But first, how about I make you some of your favorite dinner?" *

Screenshots:
Screenshot_2024-02-05-00-07-50-807_com kiwibrowser browser-edit

Screenshot_2024-02-05-00-08-43-836_com kiwibrowser browser-edit

@Spacellary Spacellary changed the title Feature Enchantment: Improve "Trim Incomplete Sentences" detection. Feature Enhancement: Improve "Trim Incomplete Sentences" detection. Feb 14, 2024
@sceuick sceuick added the approved The feature or enhancement is approved OR the bug is confirmed. label Feb 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved The feature or enhancement is approved OR the bug is confirmed.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants