Allow LiveViews to be adopted

[josevalim]

One of the issues with LiveView is the double render when going from dead render to live render and the fact we lose all state on disconnection.

This issue proposes for us to render a LiveView (a Phoenix.Channel really) upfront and then it gets "adopted" when necessary. In a nutshell:

• On disconnect, we keep the LiveView alive for X seconds. Then on reconnect, we reestablish the connection back to the same LiveView, and we just send the latest diff

• On dead render, we already spawn the LiveView, keep it alive until the WebSocket connection arrives and "adopts" the LiveView, so we just need to send the latest diff

However, this has some issues:

• If the new connection happens on the same node, it is perfect. However, if it happens on a separate node, then we can either do cluster round-trips on every payload, copy only some assigns (from assigns_new) or build a new LiveView altogether (and discard the old one).

• This solution means we will keep state around on the server for X seconds. This could perhaps be abused for DDoS attacks or similar. It may be safer to enable this only on certain pages (for example, where authentication is required) or keep the timeout short on public pages (e.g. 5 seconds instead of 30).

On the other hand, this solution should be strictly better than a cache layer for a single tab: there is zero copying and smaller payloads are sent on both connected render and reconnects. However, keep in mind this is not a cache, so it doesn't share across tabs (and luckily it does not introduce any of the caching issues, such as unbound memory usage, cache key management, etc).

There are a few challenges to implement this:

• We need to add functionality for adoption first in Phoenix.Channel

• We need to make sure that an orphan LiveView will submit the correct patch once it connects back. It may be we cannot squash patches on the server. We would need to queue them, which can introduce other issues

• We may need an opt-in API

While this was extracted from #3482, this solution is completely orthogonal to the one outlined there, as live_navigation is about two different LiveViews.

[elliottneilclark]

After dead view renders the http response we have all of the assigns. If we spend the cycles to start a LV process off the critical path of sending the response then the cost of extra work is not user facing. So when the websocket connection comes in we already have the LiveView started and the assigns are cached. This would trade a bit of memory and non-critical CPU usage for faster websocket connect.

However we don't want to leak memory forever, and it's possible that the websocket never comes. So there would need to be some eviction system in place (time, memory, etc)

I don't know the Erlang VM well enough, but is it possible to convince the VM to move ownership of structs (specifically assigns) rather than copying if there are no live references? That would be another way to make spinning up LV processes even cheaper. Rather than copying assigns to a new LV process, since we are using them after we have the bytes of the response ready, we can drop all references.

[josevalim]

@elliottneilclark there are some tricks we could do:

• On HTTP 1, we send connection close but we keep the process around to be adopted as a LiveView later. No copying necessary. This may require changes to the underlying webservers.

• On HTTP 2, each request is a separate process, so we can just ask for it to stick around.

Outside of that, we do need to copy it, the VM cannot transfer it (RefCounting is only for large binaries). But we can spawn the process relatively early on. For example, we spawn the process immediately after the router, so none of the data mounted in the LiveView needs to be copied, only the assigns set in the plug pipeline that are accessed by the LiveView are copied (using a similar optimization as live_navigation).

[elliottneilclark]

Outside of that, we do need to copy it, the VM cannot transfer it (RefCounting is only for large binaries).

That makes sense; large binaries are equivalent of huge objects in JVM with different accounting.

[simoncocking]

it's possible that the websocket never comes

Our experience is that this happens only on public pages which are exposed to bots / search engines / other automatons. So in our situation:

It may be safer to enable this only on certain pages (for example, where authentication is required)

this is exactly what we'd do.

If the new connection happens on the same node, it is perfect. However, if it happens on a separate node, then we can either do cluster round-trips on every payload, copy only some assigns (from assigns_new) or build a new LiveView altogether (and discard the old one).

We have some LiveViews that do some pretty heavy lifting on connected mount, so we'd need some way to guarantee that this work wouldn't be repeated if the LV was spawned on a different node to that which receives the WebSocket connection.

[josevalim]

I just realized that the reconnection approach has some complications. If the client crashes, LiveView doesn't know if the client has received the last message or not. So in order for reconnections to work, we would need to change LiveView server to keep a copy of all responses and only delete them when the client acknowledges it. This will definitely make the protocol chattier and perhaps affect the memory profile on the server. So for reconnection, we may want to spawn a new LiveView anyway, and then transfer the assigns, similar to push_navigate.

This goes back to the previous argument that it may be necessary to provide different solutions for each problem, if we want to maximize their efficiency.

We have some LiveViews that do some pretty heavy lifting on connected mount, so we'd need some way to guarantee that this work wouldn't be repeated if the LV was spawned on a different node to that which receives the WebSocket connection.

This is trivial to do if they are in the same node, it is a little bit trickier for distinct nodes. For distinct nodes, you would probably need to opt-in and say that a LiveView state is transferrable, which basically says that you don't rely on local ETS or resources (such as dataframes) in your LiveView state.

[bcardarella]

There may be an option here by pulling from a prior work, although I don't know if the solution is entirely what @josevalim has in mind.

Back in Ember the SSR framework is called Ember Fastboot. Ember's own Data library, Ember-Data, introduced a double-render issue in that on the server it would request the data, render the template, then that template was sent to the client. The app would boot, then make the same request again. The solution was called the Shoebox

The Shoebox worked by encoding the Ember-Data content during the SSR, and injecting a <script> tag with the data sent back to the client. Ember would then load data from the shoebox first rather than make the API request for data from the server (our version of double-render)

So what if this is what LiveView did? On the dead render it also injected certain data from the assigns into the template. When the Liveview connection is made inject that data as a query param. If the QP is present skip the mount function and render a diff back to the client to hydrate the app.

The upside here is that we could skip the double-render if the mount is skipped. The trade off is that there may be data that we don't want to expose to the client.

I could see this working with something like this:

def mount(params, session, socket) do
  data = MyApp.some_data_operations()
  {:ok, shoebox_assign(socket, data: data)}
end

This way there could be an acceptance from the developer that they have to opt-into this rendering benefit. Not assiging to the special keys or with a specific assign function would be existing double-render behavior.

Furthermore, it would be on the developer to ensure that whatever data they are embedding would need to be encodable and shouldn't be exposing sensitive information.

This doesn't entirely solve the problem for every use case but considering how difficult this problem has been to solve on the server I do wonder if we'll ever find a "one solution for all" here.

[hubertlepicki]

The Shoebox worked by encoding the Ember-Data content during the SSR, and injecting a <script> tag with the data sent back to the client.

This data, I assume would not be server-side only, it would not be sensitive? In LV, we would load say User schema there, we don't want to expose it to the client. We would probably store the data somewhere and then pass the key to the data to the client, that would be passed back to us on re-connecting, instead of passing the whole data.

But I like the idea of LiveViews sticking around for X amount of time. I assume they could receive messages over pubsub and otherwise, and we would eliminate the race condition where events can be missed between original mount and first connected mount.

To add to @josevalim options on what to do when a LiveView process connects via Web Socket, we could also:

1. Try to always connect to the same pod, i.e. "sticky sessions". This is possible but requires load balancer config and these can be widely different. I suspect this would be the easiest solution, but a headache to turn on for sys admins.

2. We could attempt to migrate the LV process from the original node/pod to the one where WebSocket connected to, preserving its internal state. I guess that's a can of worms and opens doors for another race condition, but should be doable. I have used Swarm and then Horde for something like that before.

Another cool feature that LV processes that "stick around" would give us is possibly better handling of recovery of state after connection drops. It'd be the same functionality, allowing for not only form state to be restored, but also internal state of LV like user_id that's never exposed on the form.

[bcardarella]

@hubertlepicki as I said in my comment:

Furthermore, it would be on the developer to ensure that whatever data they are embedding would need to be encodable and shouldn't be exposing sensitive information.

[bcardarella]

Tonight I met someone on the Open Riak team and I was curious how they handle state inconsistencies in cluster situations, which seems like the same problem the double mount needs to solve.

In Riak they replicate across nodes in the cluster and when a request for a resource is made a quorum of nodes needs to agree on what that data is before it is sent back.

Would a similar solution solve the mount problem? As I understand it the problem is in a cluster situation where state is created on a node for the dead render that may not be the node that liveview connects to.

[josevalim]

Our concern is not related on how to track the state on the server/cluster, that's doable, but the interactions between client and server. The issue is both that LiveView doesn't really control the client (we must assume someone may have malicious intent) nor the server state (it is controlled by the user/developer but it may, for example, have data that is not transferable, such as ETS tables, Nx/Explorer data, etc).

[MarthinL]

Apologies. When the consequences of multiple mount/3 calls (on different servers) became significant to me I didn't make the connection to this issue and if I correctly follow along with the thinking above my own thoughts on the matter also seems orthogonal if not inverted.

I can't say much about increased attack surfaces, but it seems to me the least impactful and demanding way to address this, at the core of it: Instead of terminating the process after the HTML is rendered and dispatched, wait for a message or a timeout before dying. If the message arrives before the timeout, reply with a (serialised) copy of the state and/or when the timeout occurs simply terminate.

Subsequent HTTP level requests (which may be the usual second request to set up the web-socket or yet another request issued to restore a broken web-socket) would send the message that the first (or previous) process is waiting for to do a handover of the (latest) state from which to carry on its processing rather than call mount/3 again to build it from the database.

The part I obviously left out is how any of these process instances know whether they should build the state by calling mount/3, or request it from somewhere else. I've left that for last because I'm not 100% sure if LiveView processes are currently registered or not. If they already are, it should be no big deal for the process responding to requst to load a LiveView page to check the registry before registering itself. The session cookie probably already contain sufficient identifying information or something might need to be added. If LiveView processes are not currently registered at all, it might introduce too much additional overhead for this as a default behaviour, and it might have to be an explicit option to be applied if and when it warrants the resources.

By the way, I saw mention of public pages vs authenticated pages in the discussion above, and thought at this point it appropriate to add that I've steered clear of LiveView for public pages. I use LiveView exclusively in combination with a duly authenticated active user, but I don't know how others do it or even if serving public pages was ever considered part of the scope for LiveView. I've assumed not.

Assuming then there is sufficient wiggle-room for every LiveView process to be registered, I am essentially suggesting:

1. That each LiveView process becomes a globally (named) registered process, if it isn't one already,
2. That the process of serving a LiveView page starts by testing (using the name calculated from the available identifiers) if thre is an existing process for the session or not,
3. If no registered process is found, the loading process can assume it's doing an original page load and call mount/3 as normal.
4. If a registered process is found, that process is asked (by message) for the loaded state, instead of calling mount/3 to build it.
5. Once a process has handed over the state to another process (perhaps with confirmation) it terminates and the process to which it handed over updates the registration to point to itself.
6. When a web-socket reset is detected server-side, the process does not terminate immediately but enters the same exit routine an initial load process uses, i.e. it waits for a handover request from a reconnecting process or a timeout to occur.
7. With overlaps only during the handover process there would, at any time, be one a single process per session per cluster holding the session's state in local variables, same as there is today. We're simply arranging a handover procedure so rather than each building it from scratch the series of processes servicing the same session and web-socket forms a chain of custody for the state variables that last exactly as long as the session itself despite network and environmental disruptions such as reconnects and restarts.

Note: The same approach covers de-duplication during initial setup and re-connection of web-sockets any number of times after that.

Note: Keeping state external in something like reddis can be effective but I expect would be very network heavy in clusters (a.k.a. chatty nodes). I'm not quite sure how expensive or chatty named process lookups would be in comparison, but basic web-socket behaviour would ensure that no other nodes or even processes get involved beyond initial setup and web-socket reconnection. As long as the process (reading the next web-socket packet) remembers and confirms its own identity there would be no additional traffic to consult the cluster-wide process registry.

I also saw mention of load-balancing concerns in the original conversation. As it happens, my focus was drawn to this matter while chasing down undesirable side-effects of a load-balancer setup involving session affinity and liveness probes taking much longer to pick up node failures than either side of the web-socket. It might still not be exhaustively considered, but the approach I propose should, as far as I've been able to see, play rather nicely with load balancing. Specifically, it would side-step the need for session affinity at the load balancer level which would not only reduce complexity (in its current form, LiveView's double-load doesn't only cause a performance hit, it also makes it incredibly tough to succussfully configure most common load balancers to get the web-socket to terminate on the same node as the original request. The end-result being that a lot of load-balancing flexibility and actual balancing of load gets sacrificed) but also boost scaling efficacy a lot (because the load-balancer is allowed to make better decisions which stays effective).

Phoenix/Elixir is a far richer environment that what most micro-services are written in, and it would be a waste not to make clever use of its facilities to reduce dependency on external load balancers.

In principle I'm in favour of separating concerns and it might seem like thinking about load balancing in this context is more conflating than separating concerns. Bear in mind though that web-sockets (and if I'm not mistaken, the slow-poll fallback used by Channel when web-sockets don't work as well) by their very nature as TCP connections, already assert precedence over load balancer decisions. The conflation of concerns is a given, and the best we can hope for is to work towards making LiveView apps better neighbours in clustered deployments.

In practice, web-sockets (TCP connections, Channels) enforce their own session affinity, which is great, until a node gets overloaded while its peers experience some crisis and stays overloaded once they restarted because all the sockets now terminate on that node. Naturally occuring page loads and reconnects eventually evens it out in the end but that can take a while. Though it would require additional logic, what I'm proposing as solution would also pave the way for reconnections becoming more seamless and as a result a plausible way for stressed nodes to transparently shed load when less busy nodes are available.

Anyway, there you have it, my 2c about where my mind was heading when I thought I'd be solving this problem by myself for myself. If it has value you'd like to explore further, or weaknesses I've missed, talk to me.

P.S.
My nodes are clustered using libcluster. I've presumed that to be the standard way, that inter-cluster traffic is handled very differently, that inter-node/intra-cluster traffic is via high-speed, low latency network (LAN) and that as a result the BEAM mesh libcluster creates isn't the mess it could be if those conditions weren't true.
Therefore I'm essentially assuming that, even at scale, (cluster-wide) global names would be cheap enough for this purpose.

If all goes to plan, the global registry, and potentialy inter-node messaging would only be called upon roughly as frequent as calls to the database. Mileage could vary for applications built with heavy local (to the node) caching to reduce database query times. For such (essentially monolithic applciations) global names would also be scoped to a single node and as such present no real challenge. Not my circus though.

[Edit] P.P.S.
Oh, just one more thing (~ Lt. Columbo). When a session times out on the client, it should be allowed to also time out on the server. Ideally, if it's a timeout and not a network interruption, it should be an orderly shutdown of the socket triggering both sides to consider the session either closed or suspended. When the user eventually returns and a reconnect initiated, the server would know to treat the reconnect as it would an initial connection since there would be no registered process with the right name/credentials. The chain of custody of over the loaded state variables would continue as if the session had been open all along. That prevents keeping state in memory without any benefit. By the time the session resumes, perhaps days later, it's unlikely that what was kept in memory all that time remained current enough to still be cheaper to calculate deltas than to reload fresh. Each application and each LiveView could maybe define its own threshold for that.

[MarthinL]

Our concern is not related on how to track the state on the server/cluster, that's doable, but the interactions between client and server. The issue is both that LiveView doesn't really control the client (we must assume someone may have malicious intent) nor the server state (it is controlled by the user/developer but it may, for example, have data that is not transferable, such as ETS tables, Nx/Explorer data, etc).

I don't know if my suggestion fully address the your concerns about the interactions between client and server, but I suspect it just might. Let me know what difficulties I've left unaccounted for and I shall attempt to address them. The plan was not to impose my way onto this. I've spent some energy on the problem before I knew or realised there was a relevant open issue on the same topic, so my objective was merely to share my work in progrress in case it could further the cause.

[MarthinL]

The existence of the swarm library came to my attention. Swarm is about process registration but also seem to get involved with process management which is where there appears to be some overlap with the approach I'm proposing. It's either an opportunity to leverage prior art or conflicting interests, which should at least be considered, I'd guess.