Unable to convert bed to States Data (multivec) file using clodius bedfile-to-multivec

[haotingjia0]

Hi Developers,

I am trying to convert a bed file to States Data (multivec) file, ingest the multivec file to my higlass server and try to display the tracks on the website. However, the multivec, the output of clodius bed_to_multivec, cannot be loaded correctly on my Higlass website.

I was using the example bed file and row info file as the input of clodius bed_to_multivec and trying to get the example output using the following command:
clodius convert bedfile-to-multivec states_format_input_testfile.bed.gz \ --format states \ --row-infos-filename states_format_test_row_infos.txt \ --assembly mm9 \ --starting-resolution 200 \ --num-rows 10
I used diff to check if my output states_format_input_testfile.bed.multires.mv5 differs from the example output and they are different. And after I ingested my output to my Hilglass webpage, it keeps loading and cannot display the track.

Can you check what is the reason. I used pip3 install clodius to install and the convert command of my clodius is bedfile-to-multivec instead of what the tutorial command bedfile_to_multivec. Is my issue due to the clodius version difference? Thank you very much.

Best,
Haoting

[pkerpedjiev]

Hi, sorry for the delayed response. Could you take a screenshot of your browser console and post it here?

Also what version of clodius are you using? You can find out by doing this:

python -c "import clodius; print(clodius.__version__)"

[haotingjia0]

Thank you. Here is a screenshot and my version is 0.18.1.
The upper track is from ingesting the original example successfully.
The lower track is from ingesting the multivec i got using clodius convert.

[pkerpedjiev]

Sorry, could you post a screenshot of the browser console? If you're on a mac, you can open it using i. Otherwise you can find a link in your browser menu. It should look like this:

[haotingjia0]

Thank you. Here is the console screenshot of the tracks condition.

[pkerpedjiev]

Are you running higlass locally, could you try to load that dataset again and then paste the last few lines of the ~/hg-data/log/hgs.log file?

I'm pretty sure I've reproduced the issue locally but want to make sure.

[haotingjia0]

Thank you. Here is my error after loading.

29/May/2022 21:36:50] ERROR [django.request:222] Internal Server Error: /api/v1/tileset_info/
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/python3.8/dist-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.8/dist-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/django/views/generic/base.py", line 71, in view
return self.dispatch(request, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/rest_framework/views.py", line 505, in dispatch
response = self.handle_exception(exc)
File "/usr/local/lib/python3.8/dist-packages/rest_framework/views.py", line 465, in handle_exception
self.raise_uncaught_exception(exc)
File "/usr/local/lib/python3.8/dist-packages/rest_framework/views.py", line 476, in raise_uncaught_exception
raise exc
File "/usr/local/lib/python3.8/dist-packages/rest_framework/views.py", line 502, in dispatch
response = handler(request, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/rest_framework/decorators.py", line 50, in handler
return func(*args, **kwargs)
File "/home/higlass/projects/higlass-server/./tilesets/views.py", line 681, in tileset_info
tileset_infos[tileset_uuid] = hgmu.tileset_info(
File "/usr/local/lib/python3.8/dist-packages/clodius/tiles/multivec.py", line 262, in tileset_info
tileset_info["row_infos"] = [r.decode("utf8") for r in row_infos]
File "/usr/local/lib/python3.8/dist-packages/clodius/tiles/multivec.py", line 262, in
tileset_info["row_infos"] = [r.decode("utf8") for r in row_infos]
AttributeError: 'str' object has no attribute 'decode'

[pkerpedjiev]

As I suspected. I just created a PR that should fix this: #145

I'll try to get it merged and issue a new release tomorrow.

[pkerpedjiev]

Could you try v0.10.5 of the higlass-docker container? It should be fixed there.

[haotingjia0]

Thank you. But I still have problem using higlass-manage. I updated clodius to v0.19.0. I repeated steps of clodius bed-to-multivec and used higlass-manage ingest. The issue is still the same. Should I use higlass-docker or is there a way i need to do to update higlass-manage? Thank you.

[haotingjia0]

The problem is solved after using docker pull higlass/higlass-docker:latest and using higlass-manage start. Thank you very much for fixing it. I will try to make a multivec from my data to see if it works.

[pkerpedjiev]

Wonderful!

Another way to do that is to specify the docker container version in higlass-manage: higlass-manage start -v v0.10.5