Implement subcommand and/or write documentation for killing the https certificates stored in consul.
The idea is simple: when you change the ACME caServer the old certificates are not deleted automatically.
The simplest solution might be a curl -XDELETE $CONSUL_URL/v1/kv/liquid/traefik/acme or something along those lines.