🐛 [Bug]: cache middleware: runtime error: index out of range [0] with length 0 #3072
Comments
|
Thanks for opening your first issue here! 🎉 Be sure to follow the issue template! If you need help or want to chat with us, join us on Discord https://gofiber.io/discord |
|
@canbusio We need an example to replicate this, from your snippet we can't tell what is Cam yo share the runtime error logs |
|
Does it make sense to have a maxLen 0 rule in the route? |
Hi~ I tried again with auth.ValidateLogin codes bellow: func ValidateLogin(c fiber.Ctx) bool {
return true
}And this "index out of range [0] with length 0" error can be reproduced. By the way, when I remove "CacheInvalidator: auth.ValidateLogin", it works fine. The full run time error log is like this: |
|
@ReneWerner87 cache config for home page only, if no maxLen 0, other page like /c/12345 would be mathed. |
|
But you can use app.All("/", cache.WebCache_Home)with the same effect |
|
@ReneWerner87 The https://github.com/gofiber/fiber/blob/main/middleware/cache/heap.go#L86 |
|
Ok, then we need an adjustment |
|
Should be something like this? func (h *indexedHeap) remove(idx int) (string, uint, error)
if idx < 0 || idx >= len(h.indices) {
return "", 0, ErrIndexOut0fRange
}
return h.removeInternal(h.indices[idx])
}
|
|
Can I work on this issue? |
|
@brunodmartins Sure, basically the implementation of heap.go needs some updates. A lot of the functions are not checking // Remove entry by index
func (h *indexedHeap) remove(idx int) (string, uint) {
h.mu.Lock()
defer h.mu.Unlock()
if idx < 0 || idx >= len(h.indices) {
return "", 0 // Return empty values if the index is out of bounds
}
return h.removeInternal(h.indices[idx])
}This requires also updating the actual |
|
Thanks! I will review the code now! Any doubts I will return here =) |
|
@brunodmartins This is a draft I have for package cache
import (
"container/heap"
"sync"
)
type heapEntry struct {
key string
exp uint64
bytes uint
idx int
}
// indexedHeap is a regular min-heap that allows finding
// elements in constant time. It does so by handing out special indices
// and tracking entry movement.
//
// indexdedHeap is used for quickly finding entries with the lowest
// expiration timestamp and deleting arbitrary entries.
type indexedHeap struct {
// Slice the heap is built on
entries []heapEntry
// Mapping "index" to position in heap slice
indices []int
// Max index handed out
maxidx int
// Mutex for concurrent access
mu sync.Mutex
}
func (h indexedHeap) Len() int {
return len(h.entries)
}
func (h indexedHeap) Less(i, j int) bool {
return h.entries[i].exp < h.entries[j].exp
}
func (h indexedHeap) Swap(i, j int) {
h.entries[i], h.entries[j] = h.entries[j], h.entries[i]
h.indices[h.entries[i].idx] = i
h.indices[h.entries[j].idx] = j
}
func (h *indexedHeap) Push(x any) {
h.mu.Lock()
defer h.mu.Unlock()
h.pushInternal(x.(heapEntry)) //nolint:forcetypeassert // Forced type assertion required to implement the heap.Interface interface
}
func (h *indexedHeap) Pop() any {
h.mu.Lock()
defer h.mu.Unlock()
return h.popInternal()
}
func (h *indexedHeap) popInternal() any {
n := len(h.entries)
if n == 0 {
return nil
}
entry := h.entries[n-1]
h.entries = h.entries[:n-1]
return entry
}
func (h *indexedHeap) pushInternal(entry heapEntry) {
h.indices[entry.idx] = len(h.entries)
h.entries = append(h.entries, entry)
}
// Returns index to track entry
func (h *indexedHeap) put(key string, exp uint64, bytes uint) int {
h.mu.Lock()
defer h.mu.Unlock()
idx := 0
if len(h.entries) < h.maxidx {
// Steal index from previously removed entry
// capacity > size is guaranteed
n := len(h.entries)
idx = h.entries[:n+1][n].idx
} else {
idx = h.maxidx
h.maxidx++
h.indices = append(h.indices, idx)
}
// Push manually to avoid allocation
h.pushInternal(heapEntry{
key: key, exp: exp, idx: idx, bytes: bytes,
})
heap.Fix(h, h.Len()-1)
return idx
}
func (h *indexedHeap) removeInternal(realIdx int) (string, uint) {
h.mu.Lock()
defer h.mu.Unlock()
if realIdx < 0 || realIdx >= len(h.entries) {
return "", 0 // Return empty values if the index is out of bounds
}
x := heap.Remove(h, realIdx).(heapEntry) //nolint:forcetypeassert,errcheck // Forced type assertion required to implement the heap.Interface interface
return x.key, x.bytes
}
// Remove entry by index
func (h *indexedHeap) remove(idx int) (string, uint) {
h.mu.Lock()
defer h.mu.Unlock()
if idx < 0 || idx >= len(h.indices) {
return "", 0 // Return empty values if the index is out of bounds
}
return h.removeInternal(h.indices[idx])
}
// Remove entry with lowest expiration time
func (h *indexedHeap) removeFirst() (string, uint) {
h.mu.Lock()
defer h.mu.Unlock()
if len(h.entries) == 0 {
return "", 0 // Return empty values if there are no entries
}
return h.removeInternal(0)
} |
|
@gaby I have found why the idx does not exist. The main reason is that on the given scenario by @canbusio , the cache always expire, due to the function ValidateLogin return always cache.go // Get entry from pool
e := manager.get(key)
// Lock entry
mux.Lock()
// Get timestamp
ts := atomic.LoadUint64(×tamp)
// Invalidate cache if requested
if cfg.CacheInvalidator != nil && cfg.CacheInvalidator(c) && e != nil {
e.exp = ts - 1
}
// Check if entry is expired
if e.exp != 0 && ts >= e.exp {
deleteKey(key)
if cfg.MaxBytes > 0 {
_, size := heap.remove(e.heapidx)
storedBytes -= size
}
}The call to manager.go if it, _ = m.memory.Get(key).(*item); it == nil { //nolint:errcheck // We store nothing else in the pool
it = m.acquire()
return it
}To fix the issue, I changed the |
|
@brunodmartins That's probably one issue, but the |
Signed-off-by: brunodmartins <[email protected]>
Signed-off-by: brunodmartins <[email protected]>
Signed-off-by: brunodmartins <[email protected]>
… length 0 (#3075) Resolves #3072 Signed-off-by: brunodmartins <[email protected]>
Bug Description
Use cache middleware to cache home page "/".
Use CacheInvalidator to skip cache, got error: runtime error: index out of range [0] with length 0.
How to Reproduce
1.use cache middleware to cache home page "/".
2.do not visit "/" in browser, so there will not be any cache of it.
3.visit "/login" page to login, then CacheInvalidator would return true now.
4.visit "/".
5.got error.
Expected Behavior
When restart the program, not every page is cached, when user logged in visit no cache page, it should be open normal.
Fiber Version
v3.0.0-beta.3
Code Snippet (optional)
Checklist:
The text was updated successfully, but these errors were encountered: