UPSTREAM PR #16899: vulkan: Fix multi_add invalid descriptor usage

[DajanaV]

Mirrored from ggml-org/llama.cpp#16899

This is similar to what I did for the K/V bindings in the FA shader, just a lot more bindings. I did it this way to be able to keep using a single descriptor set layout, for simplicity. In the long run I think we should switch all shaders to use a single binding with N array elements, to make it possible to do indexing. But that's a bigger change than I want to do to fix this bug.

I removed the code that disabled this path for Intel, presumably it'll work after this.

[loci-agentic-ai]

Access the complete analysis in the LOCI Dashboard

Based on the performance analysis data from the previous analysis, I'll provide a comprehensive summary focusing on the critical functions and their impact on key performance indicators.

LLaMA.cpp Performance Analysis Summary

Critical Function Performance Changes

Response Time Degradations

• Worst Impact: unicode.cpp__ZZ15unicode_tolowerjENKUlRKSt4pairIjjEjE_clES2_j (operator function)
  • Change: +0.08% (19 ns vs 19 ns base)
  • Location: Unicode processing within tokenization pipeline
  • Impact: Affects text preprocessing in llama_tokenize() workflow

Throughput Degradations

• Primary Impact: Same unicode operator function
  • Self-time increase: +0.08%
  • Function role: Unicode lowercase conversion for token processing

Bottleneck Analysis

• Most Significant: _ZSt10_ConstructISt6vectorI21llama_grammar_elementSaIS1_EEJRKS3_EEvPT_DpOT0_ (_Construct)
  • Change: +0.13% (20 ns vs 20 ns base)
  • Location: STL vector construction for grammar elements
  • Context: Grammar processing within tokenization

Key Performance Indicator Impact Analysis

1. Tokens Per Second

Impact Assessment: Minimal degradation expected

Affected Functions:

• llama_tokenize(): Indirectly affected through unicode processing degradation
  • Unicode operator function shows 0.08% increase in processing time
  • Grammar element construction shows 0.13% bottleneck increase

Quantified Impact:

• Based on reference data (7% tokens/sec reduction for 2ms llama_decode slowdown)
• Current unicode function degradation: 0.015 ns increase
• Estimated impact: <0.001% reduction in tokens per second
• Reasoning: Unicode processing represents small fraction of total tokenization time

Critical Functions Status:

• llama_decode(): No performance data available in current analysis
• llama_encode(): No performance data available in current analysis
• Core inference functions appear unaffected by measured changes

2. Power Consumption

Binary-Level Analysis:

Impacted Binaries:

• build.bin.libllama.so: -0.82 nJ decrease (305,211.61 nJ vs 305,212.44 nJ base)
• build.bin.libggml-base.so: No change (90,434.19 nJ)
• build.bin.libggml-cpu.so: No change (151,692.17 nJ)
• build.bin.libggml.so: No change (6,339.24 nJ)

Net Effect: Negligible power consumption improvement (-0.0% overall change)

3. Quantization Efficiency

Analysis: No direct impact detected

Reasoning:

• Performance changes isolated to unicode processing and STL operations
• Core quantization functions (llama_model_quantize(), quantization backends) not affected
• No changes in GGML tensor operations or quantization algorithms

4. Memory Usage

Affected Areas:

• Grammar Element Allocation: STL vector construction bottleneck (+0.13%)
  • Function: _Construct for llama_grammar_element vectors
  • Impact: Slight increase in memory allocation overhead
  • Scope: Grammar processing during tokenization setup

Memory Management Functions:

• Core memory functions (llama_memory_clear(), llama_memory_seq_rm()) not directly impacted
• KV cache operations appear unaffected

5. Batch Processing

Analysis: No measurable impact

Reasoning:

• Batch processing functions (llama_batch_init(), llama_decode() with batches) not affected
• Performance changes limited to preprocessing stages
• Parallel token processing efficiency maintained

Root Cause Analysis

Unicode Processing Degradation

• Assembly Analysis: Identical instruction sequences between versions
• Likely Cause: Compiler optimization differences or memory layout changes
• Function Behavior: No logical changes detected

Grammar Construction Bottleneck

• Pattern: STL template instantiation performance variation
• Context: Grammar element vector allocation during tokenization setup
• Scope: Initialization phase, not runtime inference

Actionable Recommendations

Immediate Actions

1. Build Environment Validation

   • Verify identical compiler versions and optimization flags between builds
   • Ensure consistent build environment to eliminate measurement noise

2. Unicode Processing Optimization

   • Consider inlining unicode operator functions to eliminate function call overhead
   • Review compiler optimization flags specifically for unicode processing modules

3. Grammar Element Management

   • Evaluate custom allocators for llama_grammar_element vectors
   • Consider pre-allocation strategies to reduce dynamic allocation overhead

Code-Level Optimizations

1. Function Inlining

   // Consider marking unicode operators as inline or constexpr
   inline FLOAT_TYPE unicode_operator(...) { /* implementation */ }

2. Memory Allocation Strategy

   // Pre-allocate grammar element vectors with expected capacity
   std::vector<llama_grammar_element> elements;
   elements.reserve(expected_size);

Build System Improvements

1. Optimization Flag Review

   • Ensure -O3 or equivalent optimization levels for unicode processing
   • Consider profile-guided optimization (PGO) for hot paths

2. Compiler-Specific Tuning

   • Evaluate architecture-specific optimizations (-march=native)
   • Review template instantiation optimization settings

Performance Impact Summary

Overall Assessment: Changes represent measurement noise rather than functional regressions

Critical Insights:

• Core inference functions (llama_decode(), llama_encode()) remain unaffected
• Power consumption shows slight improvement despite localized degradations
• Tokenization preprocessing shows minimal overhead increases
• Batch processing and quantization efficiency maintained

Priority Focus: Build environment consistency and compiler optimization validation rather than algorithmic changes, as performance variations appear to be compilation-related rather than code logic changes.