[Summer 2021] Create model verification tests

[r0rshark]

Currently we have the integration tests which make sure we do not have regression for a limited set of important categories however we do not have a way to make sure that all the models are valid.

The idea is to create a model validation testing system for the Opensource models and run it in the Github CI. This would require

1. Create a folder in pyre-check/tools/pysa_integration_tests/verify_models with

• a requirements.txt file which will contain the library dependencies needed for the model defined
• (probably) a main.py file with an empty main function (just use to run pysa on this folder with the environment created by)
• Readme explaining the folder content

2. A python script which

• Creates a python environment installing the libraries in the requirements.txt
• Create a Pyre connection with specific configuration options pointing the source_directory to the pyre-check/tools/pysa_integration_tests/verify_models and including the environment created in the previous step. This can be done by creating a temporary .pyre_configuration.local file and specifying "source_directories" configuration. We may run into issue since there is a top level .pyre_configuration file. In this case we can create a temporary top level folder and store the temporary .pyre_configuration file there:

temporary_file_path = pyre_root_dir / hashlib.sha1(
            ",".join((targets or []) + (source_directories or [])).encode()
        ).hexdigest()

• Call the get_invalid_taint_models
• Parse the input and allow filtering out based on the model regex or filesystem path. The configuration with the information of which models/filepath to filter out can be a file called verify_models.conf in JSON format

[0xedward]

Linking respective PR - facebook#467