python code generator ease of use enhancements

[leonwilly]

First the python code generator works fantastic. I appreciate all of the work that went into writing that plugin. I am assuming the code generators are proprietary as I am unable to find them in your GitHub profile or statecharts repository.

While the implementation for a simple statemachine is fairly straight forward and simple. The more complex the statechart becomes with submachines and loosely coupled state machines the more redundant writing the run time becomes. I would love to contribute to them if I can I have over 5 years of professional experience with Python. If not I'll suggest what I would consider improvements below. Please keep in mind this is constructive criticism and as I stated early I am great full for my personal use license as I am well aware of the amount of time that went into developing this project. Also I have no idea how much knowledge you and your team have pertaining to Python. I'm sure these ideas were discussed and decided against for valid reasons.

example 1:

# Implementation of statechart example_statemachine.

import warnings
# implemented interfaces:
from .example_statemachine_interface import SCI_Interface
# to store states:
from enum import Enum

class Example_Metaclass(type):
     sci_interface = SCI_Interface
     def __new__(mcs, name, bases, dct, sci_interface=SCI_Interface):
         return super().__new__(mcs, name, bases, dict(dct, sci_interface=SCI_Interface)

     def __call__(mcs, *args, **kwargs):
        # here we do the setup for the interface
        instance = super().__call__(*args, **kwargs)
        instance.sci_interface = mcs.sci_interface(instance)
        return instance

class Example(metaclass=Example_Metaclass):
    # ... normal generated code

user code

class Example_Interface(Example_SCI_Interface): pass
class Example_Implementation(Example, sci_interface=Example_Interface): pass

This particular example uses Metaclass key word arguments thus only valid in Python version 3.x and above. Since most of the sci_interface code is implemented in the generated statemachine class with the exception of operations and non-builtin sct types. This would be fairly straightforward to implement.

I'll add more suggestions to this issue throughout this coming week

[leonwilly]

You can delete this I'm writing a custom generator. In the meantime I just created a decorator that does the above.

[terfloth]

I'm happy to read that you like the Python code generator. It will be part of the core product with release 4 and and as such will be continuously maintained. It supports all modeling features including super steps, multi state machines etc..

We always appreciate feedback and suggestions for improvement like yours - thank you. We will discuss in the team if we want to make use of meta classes and I will keep this issue open as long this is ongoing.

[leonwilly]

Great news thank you! In the meantime I wrote a simple decorator that wraps the construction and initialization of state machines. Decorators would be a great way to implement interface operations. Similar to C++ declaration and definitions. Since the statemachine already imports and creates the generic interface. You could use decorators to overwrite properties and operations. Right now the SCI_Interface creates a property called operationCallback to which the operations are supposed to be implemented in. But seeing as operations are declared under the interface I believe operations should be part of the interface and not a secondary property. Having the operations directly declared inside the interface itself avoids Pythons '.' lookup performance penalty.

Example:

example_statemachine_interface.py

class SCI_Example:
    # Interface
    
    def foo(self):
        raise NotImplemented('Interface operations must be defined in a subclass')


class SCG_Example(SCI_Example):
    # Generic interface for creation

    @classmethod
    def foo(cls, callback=None):
        if callback is None:
            raise NotImplemented("Generic interface operations must be defined using a decorator")
        setattr(cls, 'foo', callback)

user definition

from example_statemachine import Example, SCG_Example

@SCG_Example.foo
def foo(self): # arguments could be added if needed
    # do something
    return # something if needed

# ... normal state machine runtime below

The fact that the statemachine creates an interface object violates the interface paradigm. As I am sure your aware, interface objects shouldn't be created directly. You could easily solve this by a generating Generic interface class that allows using decorators to override its methods.

I am not implying this is the best solution just an alternative that could be easily added to the existing generator.

[rherrmannr]

hey, @gabewillen
thanks for your detailed feedback! You're right, there are several things that we can improve in the generated code. Besides patterns, there is a lot to do for naming conventions e.g. public, protected and private members with _ or __ prefix.

The last weeks we've done great progress and now support every feature, as @EventDriven/@CycleBased semantics, @SuperSteps, Observables, EventBuffering (Internal, In event Queues) and much more. Thus, the python generator will is feature complete and will be fully maintained and part of YSCT 4.0 - and not labs anymore.

The next 2 weeks I will do a refactoring and improve the generated code - so, your feedback is welcome!

Besides this, we try to generate code which is compliant with MicroPython.

PS: There is a new beta version available, which support all features: https://www.itemis.com/en/yakindu/state-machine/download-options/beta/

[leonwilly]

@rherrmannr I am already using the beta version it is fantastic. I can tell a lot of work went into adding imports, deep c++ integration and submachine simulation.

While micropython is excellent and with Adafruits CircuitPython written on top of that it is opening python up to embedded developers and opening up embedded development to hobbyist who are overwhelmed learning C / C++. But restricting the generated code to such a small subset of python really limits performance and features. As of right now micropython is still in its infancy and although its gaining in popularity it has a small number of ports and pales in comparison in usage to standard Python.

My opinionated advice would be to add a flag in the .sgen file for micropython support. I would also look into using locks and concurrent.futures ThreadExecutor to add real (as real as a thread gets in Python) region concurrency while protecting the event queue and run to completion paradigm. As your aware the beauty of model generated source code is the ability to add / remove features without affecting the expected behavior. The trade off for me is I am a perfectionist when it comes to developing and for some reason I find model layout tedious as I spend to much time making transition lines and container layout look organized. This framework is fantastic and that is coming from someone who has read the UML statemachine specification dozens of times, along with Miro Samek's "Practical UML Statecharts". I have used Quantum Leaps statemachine framework extensively and written an in house C++17 compile time statemachine framework targeted towards embedded devices. Once I learned about statemachines I just can't go back to writing spaghetti if/else code.