[BugFix] Don't scan entire cache dir when loading model#13302

simon-mo merged 2 commits intovllm-project:mainvllm-project/vllm:mainfrom

njhill:dont-scan-cachenjhill/vllm:dont-scan-cacheCopy head branch name to clipboard