Support inference with RenyiELBO for local latent variable models

[OlaRonning]

Addresses #3104

TODO

• Add Local latent variable model test case
• Correct RenyiELBO logsumexp reduction dim for LLV model

[OlaRonning]

Should I ignore notebooks when using black? When running black[jupyter]==22.6.0, I get 48 files changed.

[fritzo]

Yes could you please ignore notebooks? The easiest way to do so is to run black through make format. Alternatively we could set up a pyproject.toml file with the black exclusion list

[OlaRonning]

Yes could you please ignore notebooks?

Sure, no problem.

The easiest way to do so is to run black through make format. Alternatively we could set up a pyproject.toml file with the black exclusion list.

Looking at the format target in the Makefile, it seems it's only black that needs a section in the TOML file. I'll configure a pyproject.toml and update the Makefile.

[martinjankowiak]

how much time do these tests take?

[OlaRonning]

how much time do these tests take?

In CI :

	Time [sec]
test_renyi_nonreparameterized	50.67
test_renyi_reparameterized	36.59
test_renyi_vectorized	36.32

So all are in the top ten slowest for batch 2.

[OlaRonning]

@martinjankowiak, can you suggest a plating structure where we need to compute the dependence (i.e., all_dims - sum_dims != num_particles_vectorized)? I've been playing around with the current implementation but cannot come up with one.

[martinjankowiak]

@OlaRonning will this still give results for a local latent variable model like a VAE with a data plate? why doesn't this logic sum out the data plate dimension?

[OlaRonning]

@martinjankowiak, sorry for the late reply.

why doesn't this logic sum out the data plate dimension?

From my understanding, this corresponds to including the data plate dimension in sum_dims, or have I misunderstood something?