Chenjie Luo ce8ce2229c [NVBUG: 5617733] Update LLM generate API for modelopt LLM eval (#498) ...

## What does this PR do?

**Type of change:** ? Bug fix

**Overview:** ?

1) Remove kv_cache_config in the generate API. It's no longer used in
the code as well. We just estimate KV cache usage from other parameters
2) Add max_seq_len in the generate API to better estimate the real KV
cache usage.
3) Assume default lm_eval max input sequence length to be 4096

Signed-off-by: Chenjie Luo <chenjiel@nvidia.com>

2025-11-04 11:08:03 -08:00

..

llm

[NVBUG: 5617733] Update LLM generate API for modelopt LLM eval (#498)

2025-11-04 11:08:03 -08:00

__init__.py

Update 0.23.0 - OSS release

2025-01-29 01:45:47 +05:30