plopslop

technology

rss
newest firstBack to technology

Hardware-Agnostic Models in vLLM

PyTorch24d ago

Featured projects TL;DR To achieve state-of-the-art performance at the frontier, vLLM is changing its internal implementation in ways that make it incompatible with fullgraph torch.compile.