Prefill Stage

What is the prefill stage?

The faster initial phase of AI inference that translates user queries into tokens before generating answers.