Features Backlog #46
Description
Parked Ideas 🚗
- Testing social stereotypes for MLMs (paper)
- Adding support for cloud provider models to be tested
- Support an installation for airgapped environments
- Need to generate a spec of what data the model expects and can safely run with (for example, if a model has only been validated on females aged 18 and up, then the model should not be used on people outside that demographic group) - see https://ianwhitestone.work/hello-great-expectations/
- Toxicity tests (swear words, offensive answers)
- Data leakage tests (PHI)
- Adversarial attacks tests
- Freshness tests (replace _2023_name)
- Runtime tests
- Question answering
- Text generation
- Summarization
- Paraphrasing
- Translation