Update TensorRT-LLM

[kaiyux]

• Model Support
  • QWEN (external contribution - thank to @Tlntin and @zhaohb)
  • Replit Code V-1.5 3B (external contribution Add GQA support to MPT (and GPT) models #205 - thanks to @bheilbrun)
• Features
  • MPT MQA/GQA support (external contribution Add GQA support to MPT (and GPT) models #205 - thanks to @bheilbrun)
  • Falcon INT4 AWQ support
  • Encoder-decoder models
    • BF16 support
    • Multi-GPU support, including tensor parallelism, pipeline parallelism and hybrid parallelism.
    • T5 family support w/ gated activation, including T5/mT5/Flan-T5 (developed in parallel with Add support for Flan-T5 models #351 - thanks to @mlmonk and @nqbao11)
    • Remove padding support
  • Limited support for LoRA (limited to QKV projection and only exposed in Python runtime for now) in the GPT model. More support will be released in the coming weeks.
  • C++ runtime
    • Gather generation logits support
    • Add options to return logProbs and cumLogProbs
  • CUTLASS 3.x upgrade
  • Experimental Python runtime API
• Bug fixes
  • Tensor names error RuntimeError: Tensor names (host_max_kv_cache_length) in engine are not the same as expected on main branch #369
  • Bloom weights split issue world_size = 2 will raise error "array split does not result in an equal division" #374
  • SmoothQuant multi-GPU failure smooth-quant with tp=2, and build llama 7b with tp=2, pp=2 failed #267
  • GenerationSession crash issue GenerationSession crashes if stream keyword argument is not None #202
  • Typo when calling PyNVML API [BUG] code bug #410
• Memory optimization
  • Add utility to load model weight by shard to reduce peak CPU memory pressure.
• Performance
  • Add reproduction steps for benchmarked performance results.
• Infra
  • NGC docker version upgraded from 23.08 to 23.10.
• Documentation
  • Add memory usage documentation
  • Add instructions to run LLaMA that use Position Interpolation for long sequence lengths(32K)