Key-Value Cache

What is a Key-Value Cache?

A digital cheat sheet of prior context and attention used during an AI chat to avoid repeat calculations and allow for faster output generation.