You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Take a look at @felipemello1's awesome RFC looking at how we would plan to support even longer context models: #1244.
We started taking a look at it, but then de-prioritized the work in favor of onboarding new modalities b/c we found that with our memory optimizations we could easily get to 64K context length.
What use case are you trying to work on? We could revisit our prioritization.
Take a look at @felipemello1's awesome RFC looking at how we would plan to support even longer context models: #1244.
We started taking a look at it, but then de-prioritized the work in favor of onboarding new modalities b/c we found that with our memory optimizations we could easily get to 64K context length.
What use case are you trying to work on? We could revisit our prioritization.
In my opinion, everyone is chasing after O1. In this process, the training on long texts in SFT (Supervised Fine-Tuning) and RL (Reinforcement Learning) is necessary.
Long-text scenarios are quite common, and it would be of great help if they could be supported.
The text was updated successfully, but these errors were encountered: