Skip to content

Issues: huggingface/text-generation-inference

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

105 Open 1,100 Closed
105 Open 1,100 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Shard process was signaled to shutdown with signal 4 rank=0 Error: ShardCannotStart
#2238 opened Jul 16, 2024 by fhamborg
2 of 4 tasks
2
Nightly load test results
#2235 opened Jul 15, 2024 by Hugoch
2
Model offloading (preparing for Llama3 405b)
#2229 opened Jul 15, 2024 by flozi00
3
CPU and Memory Utilization for TGI
#2227 opened Jul 12, 2024 by snps-ravinu
1
Gibberish generated with deepseek-ai/deepseek-coder-6.7b-base
#2223 opened Jul 11, 2024 by zch-cc
2 of 4 tasks
3
OpenAI supports top_p = 0.0 and top_p = 1.0 but TGI fails with a validation error with either of these values.
#2222 opened Jul 11, 2024 by michael-newsrx
2 of 4 tasks
1
@ErikKaum
4
Support for no_repeat_ngram_size generation parameter
#2219 opened Jul 11, 2024 by njbrake
1
3
OPENAI Format Completion endpoint does not include finish reason and logprobs
#2211 opened Jul 9, 2024 by zch-cc
2 of 4 tasks
1
Description of Tools format within Documentation
#2210 opened Jul 9, 2024 by RonanKMcGovern
2 of 4 tasks
2
Tgi crash on multi GPUs bug Something isn't working
#2207 opened Jul 9, 2024 by RohanSohani30
2 of 4 tasks
5
How to clean the TGI guidance cache?
#2200 opened Jul 8, 2024 by EdisonE3
@ErikKaum
2
error warming up cohere/aya-23-35b model: error in reshape_and_cache function with TGI 2.1.1
#2199 opened Jul 8, 2024 by Jason-CKY
2 of 4 tasks
load eetq model
#2198 opened Jul 7, 2024 by meitalbensinai
1
OPTForCausalLM doesn't give prefix to OPTModel in opt_modelling.py
#2197 opened Jul 7, 2024 by sadra-barikbin
1
LLava NEXT 34B does not start: RuntimeError: shape mismatch: value tensor of shape
#2195 opened Jul 5, 2024 by ktrapeznikov
3 of 4 tasks
2
Queue size increases indefinitely
#2192 opened Jul 5, 2024 by QLutz
2 of 4 tasks
2
Phi-3 mini 128k produces gibberish if context >4k tokens
#2185 opened Jul 4, 2024 by jphme
2 of 4 tasks
1
DistServe support
#2183 opened Jul 4, 2024 by icyxp
1
Is there any possibility to support Phi-3-vision-128k-instruct?
#2174 opened Jul 3, 2024 by farzanehnakhaee70
1
Gemma2 fails due to missing model.embed_tokens.weight
#2162 opened Jul 2, 2024 by sebastian-nehrdich
2 of 4 tasks
Understanding how max-batch-prefill-tokens works
#2160 opened Jul 2, 2024 by varad0309
Can tgi surpport non tokenizer model？
#2150 opened Jul 1, 2024 by Strive-for-excellence
1
DeepSeek Coder V2: sharded is not supported for AutoModel
#2146 opened Jun 30, 2024 by freegheist
2 of 4 tasks
2
Error "EOF while parsing an object..." with tool_calls
#2145 opened Jun 30, 2024 by ishelaputov
2 of 4 tasks
6
Could not import Flash Attention enabled models: cannot import name 'FastLayerNorm'
#2144 opened Jun 29, 2024 by Hhhh8
2 of 4 tasks
1
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.