minus-squareCaptnBook@feddit.orgtoSelfhosted@lemmy.world•Faster Ollama alternativelinkfedilinkEnglisharrow-up2·11 hours agoVllm unfortunately doesn’t support switching the model without a restart. linkfedilink
minus-squareCaptnBook@feddit.orgtoSelfhosted@lemmy.world•Faster Ollama alternativelinkfedilinkEnglisharrow-up1·edit-211 hours agoIt’s not, by far. But vllm or SGLang don’t support switching the model… such a shame. linkfedilink
Vllm unfortunately doesn’t support switching the model without a restart.