Reduce status of C backend to "just another backend"

[jessegrabowski]

Description

This is a proposal, discussion strongly requested.

The C backend is the original backend from the Theano days, and pre-dates the notion of a backend. As a result, there are lots of C concepts interwoven in the code. A simple example is a COp, an Op that has a C implementation. These are sprinkled through the code base in tensor.

My proposal is to put C on the same level as numba/jax/etc. This would entail:

• Depreciating the COp entirely
• Removing any loose references to C code or C implementation in other places (elementwise vectorization for example, though Ricardo is already fixing this)
• Replace COps with C_funcify dispatches that live in the linker/c folder
• Move C-specific blas code to relevant linker/c files

This would do a few things. First, clean up our core library code, and keep it as slim and "easy to read" as possible. All of the codegen and C code strings can be hidden away in there. It will also make it more obvious what is and is not covered by the C backend, in case someone ever comes along and wants to try to work on it.

[ricardoV94]

Absolutely, but it will be a major endeavor even in the age of LLMs

[jessegrabowski]

Nothing worth doing is easy. I think the trick will be to see if we can lay some groundwork so it can be tackled Op by Op over time