The first working node's hard drive is damaged and unrecoverable. #8451
Description
The first working node's hard drive is damaged and unrecoverable. It's currently impossible to forcibly disable or remove this node, nor can a new node be added. The version is Citus 11, and PG is 14.
postgres=# SELECT citus_disable_node('192.168.10.218', 5432); ERROR: disabling the first worker node in the metadata is not allowed DETAIL: Citus uses the first worker node in the metadata for certain internal operations when replicated tables are modified. Synchronous mode ensures that all nodes have the same view of the first worker node, which is used for certain locking operations. HINT: You can force disabling node, SELECT citus_disable_node('192.168.10.218', 5432, synchronous:=true); postgres=# SELECT citus_disable_node('192.168.10.218', 5432, synchronous := true); ERROR: cannot remove or disable the node 192.168.10.218:5432 because because it contains the only shard placement for shard 102808 DETAIL: One of the table(s) that prevents the operation complete successfully is public.ads HINT: To proceed, either drop the tables or use undistribute_table() function to convert them to local tables postgres=# SELECT undistribute_table('public.ads'); NOTICE: creating a new table for public.ads NOTICE: moving the data of public.ads ERROR: connection to the remote node 192.168.10.218:5432 failed with the following error: server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. could not send SSL negotiation packet: Success CONTEXT: SQL statement "INSERT INTO public.ads_2561830478 (id,company_id,campaign_id,name,image_url,target_url,impressions_count,clicks_count,created_at,updated_at) SELECT id,company_id,campaign_id,name,image_url,target_url,impressions_count,clicks_count,created_at,updated_at FROM public.ads" postgres=# SELECT citus_disable_node('192.168.10.218', 5432, synchronous:=true); ERROR: cannot remove or disable the node 192.168.10.218:5432 because because it contains the only shard placement for shard 102808 DETAIL: One of the table(s) that prevents the operation complete successfully is public.ads HINT: To proceed, either drop the tables or use undistribute_table() function to convert them to local tables postgres=# select * from pg_dist_placement limit 10;