Expanded computed field capability

[PeterBaker0]

Is your feature request related to a problem? Please describe.

The current computed is limited to numerical types in input and output. Ideally we would offer more flexibility here to support use cases that e.g. Survey123/Fulcrum can support.

Describe the solution you'd like

Ability to compute over strings, numbers and booleans and output strings, numbers and booleans. This would then allow us to build a more general expression engine based on this work, for informing things such as boundary conditions for sliders (e.g. XLSForm constraints).

An example of a field (pseudo code) might be:

fireRisk := if vegType = 'hedge' then (if vegHeigh < 3 then 'Low' else if vegHeight <5 then 'Medium' else 'High') else 'Medium'

[Caedous]

A couple of questions to help me work out the direction this needs to go:

1. With strings, numbers and booleans in play, what should mixed-type operations do? My inclination is to be strict and reject them at compile/eval time (e.g. {vegType} * 2 where vegType is a string is an error, not NaN), rather than inherit JavaScript's coercion rules. Related: should + concatenate strings, or should concatenation be a separate operator so + stays unambiguously numeric?
2. The field currently declares type-returned: faims-core::Number statically in the spec. If an expression can return a string or boolean, we need to decide how that's expressed. Options I can see are (a) the author declares the return type in the field config and we validate the expression against it, or (b) the field's return type becomes a union. (a) keeps validation and downstream consumers well-defined, but adds a config step. Any preference?
3. The example uses if ... then ... else. The engine currently usesthe ternary ({vegHeight} < 3 ? 'Low' : 'Medium') via jsep's grammar. Reusing the ternary is much cheaper than adding an if/then/else form (which means custom parsing). Is the ternary acceptable, or is the friendlier syntax a requirement?
4. Is there an idea that this engine eventually backs conditions/validations as well?

Also noting the current field references fields in braces ({Field-ID}) to handle IDs containing hyphens.

[PeterBaker0]

@Caedous thanks - good qns

With strings, numbers and booleans in play, what should mixed-type operations do? My inclination is to be strict and reject them at compile/eval time (e.g. {vegType} * 2 where vegType is a string is an error, not NaN), rather than inherit JavaScript's coercion rules. Related: should + concatenate strings, or should concatenation be a separate operator so + stays unambiguously numeric?

Survey123 is interesting case studies. Lots of talk online about unwanted coercion because of ambiguous return types. Let's try and do this well, let's throw at compile time in designer if we are doing ambiguous operations. Let's use discrete operators for concatenation. Is that feasible? So each operator should be unambiguous. +/-/* etc numeric only. && || etc are boolean. < > etc work across suitable types (everything except boolean) and so on.

The field currently declares type-returned: faims-core::Number statically in the spec. If an expression can return a string or boolean, we need to decide how that's expressed. Options I can see are (a) the author declares the return type in the field config and we validate the expression against it, or (b) the field's return type becomes a union. (a) keeps validation and downstream consumers well-defined, but adds a config step. Any preference?

I think we have two fields - ComputedNumber and ComputedText, then we have a clear return typef or each. They have identical implementations except one compile time enforces a return type of string, and one number. I'm worried the union type will be risky in terms of data processing. It also might mess up our export flows where columns need to know data types for headers e.g. Geopackage.

The example uses if ... then ... else. The engine currently uses the ternary ({vegHeight} < 3 ? 'Low' : 'Medium') via jsep's grammar. Reusing the ternary is much cheaper than adding an if/then/else form (which means custom parsing). Is the ternary acceptable, or is the friendlier syntax a requirement?

Sorry was just meant as a pseudocode example - ternary is fine.

Is there an idea that this engine eventually backs conditions/validations as well?

Also noting the current field references fields in braces ({Field-ID}) to handle IDs containing hyphens.

That's fine.

[Caedous]

Awesome, thanks for that. I am thinking of using & as the operator for concat if thats ok as I believe this follows the convention in XLSForm? I'll also rename the ComputedField to ComputedNumber so that we can distinguish between ComputedNumber and ComputedText.

[PeterBaker0]

Sounds good. And boolean operators are still in scope e.g. && || ?

I think ComputedNumber/ComputedText makes sense yep.

Just checking - computedNumber would allow strings/boolean as input - but the output would need to be validated as a number. E.g. (ignore syntax just example)

if vegType == 'test' || booleanVariable then return 3 else return 5.

This should be a valid ComputedNumber.

Where as

if vegType == 'test' || booleanVariable then return 3 else return 'Not valid'.

Should be invalid? If so, that sounds good.

[Caedous]

Yep, && || ! are in scope, boolean only operands. Your first example is a valid ComputedNumber as string and boolen inputs are fine anywhere in the expression, only the overall result is validated against the field's type. The second example is rejected at compile time.