More actions
Increasing background length or number of keywords or length
Increasing the length of the background or the amount of physical traits, identity traits, personality traits, or clothing keywords (which are already set at 17 each) would slow down chat performance for each inference. Our current context is already substantial, including 30 chat history entries, 10 memories (embeddings), 17*4 traits at 80 characters each, plus the role-play and user profiles. This results in a context size of around 2.5K to 4K tokens, depending on the persona and history length.
Given that we're running high-quality language models on high-end GPUs (Tesla H100 80GB), expanding these traits would require more processing power. Unfortunately, we currently don't have the financial resources to increase the number of these GPUs, as Soulkyn is still not profitable at the time of writing.