Restructure test suite into 4-tier directory layout: unit, universal, component and integration

[ntkathole]

Is your feature request related to a problem? Please describe.

Our current test organization relies on pytest markers to separate unit and integration tests, but this system is failing:

• test-python-unit runs pytest sdk/python/tests (the entire tests directory) and filters by markers — any integration test missing @pytest.mark.integration silently runs as a "unit" test
• At least 3 integration test files (test_mcp_feature_server.py, test_hybrid_online_store.py, test_dbt_integration.py) are missing the marker and currently run during unit CI
• Unit tests import heavyweight dependencies (ray, pyspark) that slow down the entire suite and require special environment setup
• The conftest.py marker-filtering logic in pytest_collection_modifyitems is a house of cards — markers are opt-in and people forget them

Describe the solution you'd like

Replace marker-based filtering with a folder-based 4-tier layout:

tests/
  unit/ → Fast, no heavy deps, pure logic
  universal/ → Cross-backend parameterized tests (same test, many backends)
  component/ → Backend-specific tests (needs ray/spark/etc installed)
  integration/ → Needs external services (K8s, Docker, Keycloak, cloud)

Key benefits:*

• pytest tests/unit — path-based, no marker gymnastics, no leakage
• pytest tests/component/ray — runs only ray-specific tests
• pytest tests/universal --config=duckdb — runs universal tests against DuckDB
• The folder is the categorization — no markers to forget
• Ray CI = component/ray + universal(ray config). Clean and composable.

Sub-tasks

• Create directory skeleton and move feature_repos/
• Move ray tests to component/ray/
• Move spark tests to component/spark/
• Move universal offline store tests to universal/offline_store/
• Move universal online store tests to universal/online_store/
• Move universal registration and materialization tests to universal/
• Reclassify misplaced tests (unit ↔ integration boundary fixes)
• Refactor root conftest.py — move fixtures, remove marker filtering
• Update Makefile targets for new directory layout
• Update CI workflows for new directory layout

Migration Principles

1. Each sub-task is a self-contained PR that can be merged and tested independently
2. No functional changes to tests — only file moves and import path updates
3. Old Makefile targets should remain as aliases during transition (deprecated, then removed)
4. Run the relevant CI after each PR to verify nothing broke

Thanks to @tokoko for idea and kicking off discussion.