Context

TRL currently includes 15 trainers, which vary significantly in usage and maintenance requirements.

Key points:

1. Maintaining all trainers in their current state imposes a high maintenance cost and reduces our ability to quickly respond to user feature requests.
2. As we prepare the library for V1, we need to clearly define which trainers are considered stable versus experimental, with the latter receiving less/no attention from maintainers.
3. Some trainers have required bug fixes and refactoring for a while but have received limited attention due to low usage. Currently, these trainers remain in the codebase but are largely ignored in practice. While this works for rapid alpha development, it is not sustainable for a stable V1 release.

Proposal

We suggest a radical approach: remove most of the trainers from the main codebase.

Some may still live in trl.experimental module based on feedback from the community.

EDIT: We will first move most trainers in trl.experimental, and discuss later which one should be promoted, which one should stay in experimental, and which one should be removed.

Goal

• Reduce maintenance overhead
• Allow maintainers to focus on the stable core of the library and new features
• Provide a clearer distinction between stable and experimental trainers
• Simplify the codebase

Questions / Discussion

• Which trainers do you think should remain in the stable codebase?
• Any other suggestions or concerns regarding this plan?

Current plan (edited based on feedback)