KV Cache

What is a KV Cache?

A digital cheat sheet of prior context and attention used during an AI chat to avoid repeat calculations and allow for faster output generation.