Raster images showing under Windows, not under Linux

[nve-sak]

I have a Shiny app plotting some images on a basemap using LeafMap; developed under Windows and moved to Ubuntu Server to be deployed using Shiny-server. With Python 3.12 on Windows and Python 3.13 on Ubuntu, I cleaned the virtual env on Windows and pip-froze a new requirements.txt which was installed in a new virtual env under Ubuntu without hickups. I excluded pywin32; whereas the Ubuntu install added ptyprocess and pexpect to the list in requirements.txt. Otherwise the package lists and versions are identical.

The Shiny app runs nicely both under shiny-server on Ubuntu and under Shiny's debug-server on Windows, but the Ubuntu app does not show the images on top of the base maps. The first plot is from Ubuntu, the second from Windows:

Any idea what may cause this difference?
The Linux side has Ubuntu server only, no window system, so I am a bit limited in testing (with my linux knowledge, at least). But the functions fetching the images work and returns what is expected.

[giswqs]

See #677

[nve-sak]

Thanks again :-) My situation isn't quite the same as in #677, since I don't use VScode on Ubuntu. I gave the provide-a-port idea a shot anyway:

The Shiny app actually shows six maps, each with a raster layer which is deleted and recreated each time I change a date. On Windows, I noticed that the URL stayed the same both among the six and over date changes, so I just pre-added a dummy raster, extracted its port number, deleted the dummy raster and supplied that port number to each subsequent add_raster().

Under Windows, this worked just as fine as before.

Under linux, it crashed with an 'address already in use' exception on the line where I supply the port to add_raster():

Using Python 3.13.0 at /srv/shiny-server/nveapps/HydMetGeo-testbenk/venv/bin/python
INFO: Started server process [135000]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:41525 (Press CTRL+C to quit)
INFO: ('127.0.0.1', 42124) - "WebSocket /websocket/" [accepted]
INFO: connection open
Address already in use
Port 40897 is in use by another program. Either identify and stop that program, or start the server with a different
port.
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/srv/shiny-server/nveapps/HydMetGeo-testbenk/venv/lib/python3.13/site-packages/werkzeug/serving.py", line 75
9, in init
self.server_bind()
~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/http/server.py", line 136, in server_bind
socketserver.TCPServer.server_bind(self)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
File "/usr/lib/python3.13/socketserver.py", line 473, in server_bind
self.socket.bind(self.server_address)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 98] Address already in use

Is there a way within Python to get a non-used port number to supply to add_raster, or is the #677 solution not applicable here?

[giswqs]

Do you have jupyter-server-proxy installed?

[nve-sak]

No, but I guess that's doable. Will it work under ubuntu server?

[giswqs]

This is a Localtileserver issue. You can explore the GitHub repository to see there are similar issues https://github.com/banesullivan/localtileserver/issues

[nve-sak]

Thanks again, I'll bring the question over there. This is somewhat beyond my competence, sorry for not catching all advice well :-)

[ninsbl]

Hi,
I have been looking into @nve-sak 's issue.
My understanding of the problem is that shiny does not use the jupyter-proxy-server (an does not run the jupyter-server either). Thus localtileserver on localhost does not get exposed to the clients browser.

A workaround for shiny would be to deploy a reverse proxy in addition (e.g. like in the small Flask app below), that is able to route incoming requests to localhost while dynamically mapping the port to whatever localtileserver is running on.

The following example (set up in a docker container with shiny and a minimal leafmap example shiny-app) does work. However, I have not tested it with several layers / or maps and I have not investigated security issues yet (so I would not try this on a web-facing server)...

import os
from flask import Flask, request, redirect, Response
import requests
app = Flask(__name__)
SITE_NAME = "http://localhost:"

@app.route("/")
def index():
    return "Flask is running!"

@app.route("/<int:port>/<path:path>", methods=["GET"])
def proxy_port(port, path):
    if request.method=="GET":
        global SITE_NAME
        SITE_NAME = f"http://localhost:{port}"
        with requests.Session() as s:
            resp = s.get(f"{SITE_NAME}/{path}", params=request.args, stream=True)
            excluded_headers = ["content-encoding", "content-length", "transfer-encoding", "connection"]
            headers = [(name, value) for (name, value) in resp.raw.headers.items() if name.lower() not in excluded_headers]
            return  Response(resp.content, resp.status_code, headers)

@app.route("/<path:path>", methods=["GET"])
def proxy(path):
    if request.method=="GET":
        global SITE_NAME
        with requests.Session() as s:
            resp = s.get(f"{SITE_NAME}/{path}", stream=True)
            excluded_headers = ["content-encoding", "content-length", "transfer-encoding", "connection"]
            headers = [(name, value) for (name, value) in resp.raw.headers.items() if name.lower() not in excluded_headers]
            return  Response(resp.content, resp.status_code, headers)


if __name__ == "__main__":
    port = int(os.environ.get('PORT', 8888))
    app.run(debug=False, host='0.0.0.0', port=port)

P.S.: by setting debug=True one can track the network traffic / requests...

In the shiny-app the tile client is then set up like this (assuming that the flask-proxy is running on host "my-flask-proxy.com" and port 8888):

client = TileClient("/data/test.tif", client_host="my-flask-proxy.com", client_port=8888)

[nve-sak]

Thanks a lot, @ninsbl :-), this looks promising!
I'll not be at work until January, but I am looking forward to testing this.
In case someone is more impatient, here is a minimal Shiny app built around @giswqs' example from #650:

import os
# os.environ['LOCALTILESERVER_CLIENT_PREFIX'] = 'proxy/{port}'
if 'PROJ_LIB' in os.environ:
    del os.environ['PROJ_LIB']
from localtileserver import TileClient, get_leaflet_tile_layer
import rasterio
from ipyleaflet import Map
from shiny import App, ui
from shinywidgets import output_widget, register_widget

# Open a rasterio dataset
dataset = rasterio.open('https://open.gishub.org/data/raster/srtm90.tif')
data_array = dataset.read(1)

# Do some processing on the data array
data_array[data_array < 1000] = 0

# Create rasterio dataset in memory
memory_file = rasterio.MemoryFile()
raster_dataset = memory_file.open(driver='GTiff',
                                  height=data_array.shape[0],
                                  width=data_array.shape[1],
                                  count=1,
                                  dtype=str(data_array.dtype),
                                  crs=dataset.crs,
                                  transform=dataset.transform)

# Write data array values to the rasterio dataset
raster_dataset.write(data_array, 1)
raster_dataset.close()


app_ui = ui.page_fluid(  output_widget("map"),  )    

def server(input, output, session):
    # Read the dataset from memory
    dataset_reader = rasterio.open(raster_dataset.name, mode='r')

    # Create a TileClient and a Leaflet map
    client = TileClient(dataset_reader)
    t = get_leaflet_tile_layer(client, colormap='terrain', vmin=100, vmax=3000)
    m = Map(center=client.center(), zoom=client.default_zoom)
    m.add_layer(t)
    register_widget(f"map", m)

app = App(app_ui, server)