2.10 Changes Data Layer Without Migration

[nzjrs]

PR #2818 added autoCollapse to StepDict.to_dict(), so sql_alchemy.py now includes it in the INSERT. No migration was added though, so existing databases blow up

  sqlite3.OperationalError: table steps has no column named autoCollapse

modes in v2.9.4 had the same issue but at least got a callout in the release notes with an ALTER TABLE instruction. This one just silently breaks.

The underlying problem is that sql_alchemy.py builds SQL dynamically from to_dict() keys so any new field on StepDict becomes a column reference with no check that the column exists. This will keep happening every time someone adds a field. Please add a comment to StepDict to remind maintainers that migrations will be needed upon change.

[dosubot]

You're right that this is a recurring issue. The dynamic SQL construction in sql_alchemy.py automatically includes any non-None keys from to_dict() in INSERT statements, which means any new field added to StepDict can silently break existing databases.

For immediate relief, you can run:

ALTER TABLE steps ADD COLUMN IF NOT EXISTS "autoCollapse" BOOLEAN;

Or for Prisma:

ALTER TABLE "Step" ADD COLUMN IF NOT EXISTS "autoCollapse" BOOLEAN;

Your observation about the modes field in v2.9.4 having the same pattern (documented in the CHANGELOG as a breaking change with ALTER TABLE instructions) is correct—PR #2818 added autoCollapse to StepDict.to_dict() but didn't include migration instructions in the release notes.

The suggestion to add a comment to StepDict reminding maintainers about migrations seems reasonable given there's no automated migration system in place.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[justus-wendroth]

I just setup chainlit with the SQLAlchemy Data Layer and ran into this issue. The documentation for the data layer probably also should be updated (https://docs.chainlit.io/data-layers/sqlalchemy) :)

[christopheroflaherty]

Same with the official datalayer: https://github.com/Chainlit/chainlit-datalayer

[github-actions]

This issue is stale because it has been open for 14 days with no activity.