read vs watch_poll

[aalvarez89]

Hello, I needed to inquire regarding the functionality of a watch object.
My objective was basically subscribing to a data point and getting a live feed of that data point's value.
I'm not sure what am I doing wrong but after creating the watch object, I have to constantly ping again the server with the watch_poll() function, so how is this different than just calling .read() on that same data point?
Is there actually a way to get a live feed with one call?
Thanks in advance

[sjlongland]

That is indeed how Haystack real-time works.

You subscribe to a point, this tell the server to begin polling that point in the background.
You then poll the server for updates.

The .read() call will simply return (in curVal) the last known value, which if the point is not being actively polled, could be hours old, there's no guarantee the value will be "fresh" in any way shape or form.

Some implementations of Haystack have alternate mechanisms (e.g. WideSky has a socket.io v2 based true real-time mechanism), but this is outside the Haystack specification.

https://project-haystack.org/doc/docHaystack/Ops#watchPoll

[aalvarez89]

@sjlongland Understood, that makes a lot of sense, then what is the callback on .watch_poll() for? or how do you properly use it?
Thanks!

[sjlongland]

Same as the other functions, if you're using an asynchronous HTTP client (the default is to use a synchronous one), it'll be called when the result is received instead of calling .wait() on the operation object.

operation = session.watch_poll(watchobj)
operation.wait()
# do something with operation.result

Using callback actually hooks the .done signal on the operation object as a convenience.

https://github.com/ChristianTremblay/pyhaystack/blob/master/pyhaystack/client/session.py#L663-L664

The callback returns the operation object as an argument. That API was developed for Python 2.7, where there's several competing asynchronous frameworks:

def _on_done(operation, **kwargs):
    # do something with operation.result
    pass
session.watch_poll(watchobj, callback=_on_done)

In Python 3.5 and above, the operation object returned by calls actually is await-able, so I'd recommend using that instead:

operation = await session.watch_poll(watchobj)

[aalvarez89]

@sjlongland thank you very much for this, it really clarified a bunch of things.
I'd like to ask you if you know what could I be doing wrong when I call session.watch_sub
since I get this:
AttributeError: 'PostGridOperation' object has no attribute 'id'

(currently Im doing this call on a raw http call and its working fine but I'd like to have some consistency but I cant seem to find some sort of example of this call)

[sjlongland]

Sounds like a bug to be honest, would you have a full backtrace so we can see where it is happening?

[aalvarez89]

python hs_live.py

<PostGridOperation failed>

Traceback (most recent call last):
  File "~/hs_live.py", line 39, in <module>
    call_watch_poll(ws_id, True, poll_cb)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^

  File "~/hs_live.py", line 35, in call_watch_poll
    session.watch_poll(watch_id, refresh, callback)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "~/venv/lib/python3.13/site-packages/pyhaystack/client/session.py", line 232, in watch_poll
    return self._on_watch_poll(watch=watch, refresh=refresh, callback=callback)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "~/venv/lib/python3.13/site-packages/pyhaystack/client/session.py", line 515, in _on_watch_poll
    watch = watch.id
            ^^^^^^^^

AttributeError: 'PostGridOperation' object has no attribute 'id'

@sjlongland

[sjlongland]

Okay, so it's been quite some time since I've looked at this code (my workplace used to use Python a lot, but in 2016 pivoted away from that to JavaScript).

Two things stick out:

• PostGridOperation is a state machine that handles encoding, submission, retrieval and decoding of HTTP POST requests and responses. It itself isn't meant to be used directly: you're supposed to retrieve the result of the operation once it's done; either await it if it's Python3, or call .wait() to synchronously block on it, then read the .result property.
• The watch_id is not the whole grid returned by watchSub, but rather, the watchId field given in the metadata.

The watch_sub, watch_poll and watch_unsub are low-level functions that merely encode the arguments into a grid to be submitted to the Haystack server, and return the result as a Grid object. There isn't a higher-level abstraction at this stage as there hasn't been a requirement for one so far (real-time doesn't get a lot of love).

So if you call watch_sub, and the server responds with a grid like this:

ver:"3.0" watchId:"w-2bfe7e77-8a7c55ab" lease:1min
id,cur,curStatus,curVal,equipRef,kind,point,unit
@2b80f96a-9dcc2202 "RTU-1 ReturnTemp",M,"ok",68.2°F,@2b80f96a-ee1d1270 "RTU-1","Number",M,"°F"
@2b80f96a-7b16eb30 "RTU-1 ZoneTemp",M,"ok",74.1°F,@2b80f96a-ee1d1270 "RTU-1","Number",M,"°F"

The thing you pass to watch_poll is the string "w-2bfe7e77-8a7c55ab":

ws_id = get_watch_poll_grid() # I can't see the code that sets `ws_id`
watchsub_res = await ws_id

watchpoll_res = await session.watch_poll(watchsub_res.metadata["watchId"], refresh, callback)

That would give you a grid object in watchpoll_res. Let's assume the server gave us this:

ver:"3.0" watchId:"w-2bfe7e77-8a7c55ab"
id,cur,curStatus,curVal,equipRef,kind,point,unit
@2b80f96a-9dcc2202 "RTU-1 ReturnTemp",M,"ok",68.2°F,@2b80f96a-ee1d1270 "RTU-1","Number",M,"°F"
@2b80f96a-7b16eb30 "RTU-1 ZoneTemp",M,"ok",74.1°F,@2b80f96a-ee1d1270 "RTU-1","Number",M,"°F"

You'd be able to iterate over the rows to pick out the value and status of each point:

for row in watchpol_res:
    print ("point %s (ID %s) has value %s (status %s)" \
        % (row["id"].value, row["id"].name, row["curVal"], row["curStatus"]))

[aalvarez89]

@sjlongland Thanks for the clarification, this has worked really well. I was able to subscribe to multiple points at the same time which was my objective. I do have an inconvenience, whenever I sub to data points that emit enum or string values, I am able to see the current value on the watch_sub's response but I don't see them being tracked on my watch_poll's response. I have tried with boolean, numeric values and it works perfectly, Is there any reason/fix for this?

[sjlongland]

I do have an inconvenience, whenever I sub to data points that emit enum or string values, I am able to see the current value on the watch_sub's response but I don't see them being tracked on my watch_poll's response

The code in pyhaystack is very minimal in the low-level functions, basically it just takes care of authenticating and managing the session, and serialising/deserialising grids. Higher level things like interpreting the values sent are up to the application using it.

As such, the content of the watch_sub()'s response is little more than a re-packaging of what the server sent. This would be a server bug and not in the scope of pyhaystack.