LLMs: The Important Ingredients

An overview of the pieces coming together to make LLMs work.

Xander Dunn, 27 Jan 2023

This is a list of the ingredients I think are important right now for the improvement of LLMs. It's an overview of the literature on improving various aspects of Large Language Models. Reading a sampling of these will give a good understanding of current LLM capabilities and where engineering and research efforts are focused.

It's well known that papers often leave out important information for reproducing their results. This is often intentional and sometimes unintentional, sometimes information in the paper is mistakenly wrong. So, while this will give a good overview of the field, there is still a considerable gap between paper comprehension and being able to implement these methods to achieve similar results.