Skip to content

Multiproc & Dynamics Compatibility#27

Open
manye31 wants to merge 9 commits intomainfrom
feature/dynamics_data
Open

Multiproc & Dynamics Compatibility#27
manye31 wants to merge 9 commits intomainfrom
feature/dynamics_data

Conversation

@manye31
Copy link
Contributor

@manye31 manye31 commented Jan 20, 2026

Note

Some of this might not be conflicting with the new dataset branch (im def not passive aggressively finishing this PR now :^)

Features:

  • Dynamics datatypes (Racepak)
  • Add flag: --fill_missing_stamps which gives optional behavior to pull timestamp from bag if no timestamp in topic (such as steering_angle)
  • Add an optional optional arg to topics which are not always present e.g. racepak, RTK topics.
  • Tried to add more helper functions to make script slightly less monolithic
  • A crude multiprocessing functionality. See more below.
  • Update viz with the multiprocessing functionality

Multiprocessing

Taking advantage of each topic's independence from one another, I distribute each topic to an independent process to process the data into KITTI format. Future work should also try to add multiple processes for a single, more computationally demanding topic like perception data and split up the frames to process.

I kept the ros2bag_2_kitti.py alone, and made a new ros2bag_2_kitti_multiproc.py script.

This multi-proc is relatively a drop in set of code with 2 primary functions: display_progress_monitor and process_topic_wrapper. The process_topic_wrapper is just a wrapper function around the generic bag reading code we already have, to be used with multiprocessing. The display_progress_monitor is just a nicer CLI UI to show proc progress.

Here's an example of the CLI output:
image

@manye31 manye31 force-pushed the feature/dynamics_data branch from 0e9a58f to e122664 Compare January 22, 2026 14:52
@manye31 manye31 marked this pull request as ready for review February 18, 2026 21:54
@manye31 manye31 changed the title Dynamics Compatibility Multiproc & Dynamics Compatibility Feb 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant