Token Limit Error in logs.json (Constant Completion Tokens)

[Karthik-277]

1. Whatever changes i tried to do in settings.yaml file, I always get token limit error. Input token ranging from 2000-2300, but Completion token is fixed to 2000 always. How can i reduce the completion token size or input token size as well if you have any idea??
   the text file is not that large as well. Contains only a few paragraphs of texts.

2. Tried changing the max token values and played around with the tokens_per_minute and requests_per_minute and still didnt get anywhere.

3. This is the current params I've used. Tried with different values as well but ended with same error.
   max_tokens: 290
   tokens_per_minute: 1 # set a leaky bucket throttle
   requests_per_minute: 1 # set a leaky bucket throttle
   max_retries: 5
   max_retry_wait: 120.0
   concurrent_requests: 10 # the number of parallel inflight requests that may be made

Please help me out if you have faced any issue like this. or how to solve this if you have any ideas. Thank you!

[AlonsoGuevara]

Hi @Karthik-277
This was caused due to a bug on configuration that was ignoring settings for tpm and rpm, this has been fixed in this PR: #373

If you're using the source, we recommend pulling the latest main, if not, the fix will be included in our next release.
Will keep this issue open until a new release is done