Efficient LLM Serving for RAG (Cache-Craft)
Chunk-cache decomposition and reuse for faster prefill.
- Developed a key–value reuse mechanism that decomposes prefill states into reusable chunks.
- Achieved ~2.7× reduction in prefill compute costs and latency with comparable output quality.
Links and artifacts can be added here (demo, slides, code, product integration notes).