Meta, the parent company of Facebook, is making strides in artificial intelligence research. Their AI research team, known as Meta FAIR, is developing a Self-Taught Evaluator for large language models (LLMs). This innovative technology has the potential to streamline the development of custom LLMs for enterprises, reducing both time and human resource requirements.
In a recent paper published by Meta FAIR, the team introduced the concept of Self-Taught Evaluators, which allows LLMs to generate their own synthetic training data for evaluation purposes. Traditionally, LLM evaluators, known as LLM-as-a-Judge, are trained on large amounts of annotated data provided by humans. This process is costly and the data can become outdated as the model improves over time, as explained by the researchers.
