Design syntax and semantics to indicate persistent storage of per-table-entry data across packets, modifiable in data plane

[jafingerhut]

Personnel

• Owner: @jafingerhut
• Supporters: @mariobaldi

Design

• Document: Discussion about a few possible ways such a feature could be designed: https://github.com/jafingerhut/p4-guide/tree/master/data-plane-writable-per-table-entry-state

Implementation

• p4-spec: A draft of a PR to the language spec that is currently only intended to help define this feature for a proposed experimental implementation in p4c: Data plane writable per table entry state v1 #1239
• p4c:

Process

• LDWG discussed: First time to introduce this issue to the LDWG was 2022-Nov-07 meeting. This was primarily to introduce the idea, and collect comments. It will definitely need additional discussion before it is ready to write any changes to the spec.
• LDWG approved:
• Merged into p4-spec:
• Merged into p4c:

=======================================

[jafingerhut]

I'm not wedded to defining a new direction for action parameters, but it seemed to me like a semi-natural way to express this. This explanation will explain the syntax in that way, but if you have another syntax that achieves the same goal that you believe has advantages, please propose it.

As a very short example, suppose there was a new direction other than the existing in, out, inout directions defined in the language spec today, and it was called rmw, which I chose for the sake of example because it is the initials of "read, modify, write". The intent is that this direction would only be defined and supported as the direction of an action parameter. I cannot imagine any use yet for using this new direction for other things like functions, extern functions, extern methods, sub-parsers, or sub-controls, because the new behavior is intended specifically for actions of tables.

The new rmw direction would also not be defined for actions that are not associated with tables, as again, I cannot imagine a useful way to define the new feature in that situation.

When an action a1 is an action of a table t1, and one or more of a1's parameters has direction rmw, it means:

• like a directionless parameter of an action of a table as defined today in the language spec, the initial value comes from the control plane software, via whatever API call it made when adding the table entry.
• unlike a directionless parameter of an action, for any parameters with direction rmw it is legal within the action body to assign a value to that parameter.

When a1 is executed because an entry of the table was matched, and the entry had a1 configured as its action, the body of the action would be executed. When the body of the action was finished being executed, the final values of all rmw parameters would be "written back" to the entry, modifying their values. Any later packet matching that same entry would see the modified values as written by the last packet that caused the action to be executed for that same table entry.

This would require some definition of what happens in concurrent execution scenarios, but the intent is that effectively all or as much of the action body as required would effectively "lock the table entry E1" during its execution, and any changes made to rmw parameters by one execution of the action on entry E1 would be visible to any later execution of the action on that same entry E1. That or something similar could I think be used to define the correct behavior, but leave open the precise implementation mechanism as any mechanism that achieves the same behavior.

Example of syntax with new direction rmw:

action a1 (bit<8> foo, bit<16> bar, rmw bit<32> max_seq_seen) {
    hdr.ethernet.etherType = bar;
    usermeta.x = foo;
    if (hdr.tcp.seqNo > max_seq_seen) {
        max_seq_seen = hdr.tcp.seqNo;
    }
}

Example of a different syntax mentioned by Nate Foster and/or Mihai Budiu, with NO new direction rmw, but instead define a new modifier for local variables inside of action bodies state, which is similar to C/C++'s static, except that the state thus defined is created independently for each table entry that has been added with the enclosing action name:

action a1 (bit<8> foo, bit<16> bar, bit<32> initial_max_seq_seen) {
    // logically, the following initialization assignment only occurs on the first time that this
    // action is invoked, per table entry that uses this action.
    state bit<32> max_seq_seen = initial_max_seq_seen;
    hdr.ethernet.etherType = bar;
    usermeta.x = foo;
    if (hdr.tcp.seqNo > max_seq_seen) {
        max_seq_seen = hdr.tcp.seqNo;
    }
}

[jafingerhut]

@mariobaldi Created this issue for the P4 language work group, to see if I can get the ball rolling. For the background of others, this feature request originates in the architecture work group, and seems to me to be a natural way to express data plane modification of per-table-entry state.

[jafingerhut]

Note: TNA defines a way to do this using DirectRegister and RegisterAction externs, plus abstract methods.

https://github.com/barefootnetworks/Open-Tofino/blob/master/share/p4c/p4include/tofino1_base.p4#L621-L636

I will check with Tofino compiler developers whether this proposal is "just another syntax" for this behavior, easy to support, or if there are any differences in expressive power that I am not yet aware of. The syntax of this proposal definitely seems noticeably easier to read and understand, to me personally.

I believe that one noticeable difference between this proposal and using a DirectRegister extern is that with this proposal, if you want three fields A, B, C to be rmw for action a1 of a table, but you want 1 field D to be rmw for action a2 of the same table, it is quite natural to express that using the proposed syntax. My understanding from talking with Vladimir Gurevich is that at least today's TNA definition of DirectRegister only lets you express that all of the actions for the same table must have the same set of rmw fields as each other.

[jnfoster]

Directions were originally intended to indicate calling conventions -- a defensible language design choice. (Ab)using directionless parameters to indicate the control-plane parameters was always controversial. Experience has shown that it is confusing to new P4 developers -- not ridiculously so, but it's not natural. In my opinion, it was a mistake. (At the time P4_16 was being developed, we considered having two sets of braces, which would have been arguably more natural, but some on the LDWG had parenthephobia.)

Anyway, I really like the idea of having syntax for initializing per-table-entry state for direct mapped objects. But I wonder if further abusing direction parameters is the best way to go, or if we are continuing down a bad syntactic path.

[jfingerh]

I am certainly open to other syntactic ideas, certainly. I know there is a preference/guidance against using annotations for things that significantly change the meaning of the program if they are removed, so this seems like a poor use case for them. If you or others have thoughts on syntax besides a new direction besides in/out/inout, please do describe them.

[jafingerhut]

@jnfoster I recall you mentioning something like this about directionless parameters for actions being a choice you wish had been made differently. Perhaps I misunderstood, but I thought you meant something like "I wish instead of action parameters having no direction keyword meant 'comes from the control plane', instead we had an explicit direction keyword that meant 'comes from the control plane'". But perhaps you had something else in mind?

[apinski-cavium]

Here are my thoughts and even how I would have done the original control plane based arguments.
Having directionless meaning two different things is definitely confusing.
I would have used roentry and rwentry as the two direction keywords for actions (I am not 100% set on these two names).
I can think of how we can still introduce roentry and still be backwards compatible (and even if they use that as a type name) and have an option to warn for directionless arguments for actions (maybe even by default).

The grammar becomes
direction typename argname
ROENTRY argname

Note I am ok with not adding a specific direction and leaving directionless because it is almost water on the bridge now.

[vgurevich]

What would be a difference between such an action and a control or a function?

[jafingerhut]

@vgurevich asks: "What would be a difference between such an action and a control or a function?" It would be in an action, and thus could be an action of a table. Controls and functions cannot be actions of a table. Controls cannot be invoked from within actions in the current language definition.

[jfingerh]

@apinski-cavium I agree that if we want to preserve backwards compatibility with very very commonly written P4 code, we should not make a change where we mandate that directionless parameters to actions must have a new direction keyword in front of them. I do think something like your roentry could be introduced in the language spec, and if someone uses it in their program, its meaning would be identical to what a directionless action parameter is today (or perhaps a subset of that meaning, e.g. it MUST come from the control plane, never the data plane, could be considered for such a new direction keyword).

[mihaibudiu]

The reason directionless has two meanings is historical, to support p4-14, where there were no functions, and an action could be used both as a function and as an action. The same action could be called from two different contexts: other actions or tables.

[mihaibudiu]

I would actually use a declaration and not a parameter:

action a(parameters) {
   state bit<32> value = initializer;
}

[jafingerhut]

I have generalized the name of this issue to be closer to the problem statement, and the title no longer mentions the specific implementation approach of adding a new kind of action parameter direction.

I have created a more explicit example in the style of this comment: #1177 (comment) and added it this comment above: #1177 (comment)

@mariobaldi Please take a look at the two syntax examples at the end of this comment, particularly the last one, to see what you think: #1177 (comment)