Skip to content

JP-4285: Cube Build updates - #10433

Merged
melanieclarke merged 33 commits into
spacetelescope:mainfrom
jemorrison:cb_check
Jun 10, 2026
Merged

JP-4285: Cube Build updates #10433
melanieclarke merged 33 commits into
spacetelescope:mainfrom
jemorrison:cb_check

Conversation

@jemorrison

@jemorrison jemorrison commented Apr 7, 2026

Copy link
Copy Markdown
Collaborator

See comment below:needs update to calspec2 parameter reference file. Update done.

Resolves JP-4285

Closes #10362

This PR addresses how output filenames are created in cube_build. The 'pipeline' parameter was removed from cube along with the option to create 'single' output_type cubes. The 'single' type cubes had been used early in the mission in outlier detection to try and flag bad pixels. In the mode outlier detection called 'cube_build' to create a set of cube for each band. Each input file was mapped to the single band, all the single band s3d files where median combined and blotted back to the detector frame where bad pixels were flagged. The method of outlier detection for IFU data was replaced did not flag bad pixels well and was replaced soon after launch. The hooks to make this types of 'single' ifu cubes was left in cube_build. This pr removed the unnecessary code.
For calspec2 the internal 'step' routines create the default IFU cube names.
For calspec3 cube - cube build uses the asn name and information on the bands to create a string for the output name.

REMAINING ISSUE update to Parameter reference file.
In cube_build the 'output_type' has now been defaulted to 'band'. Only when running calspec2 on MIRI data do
we want the 'output_type' parameter = 'multi'
To run the calspec2 pipeline on miri data - the parameter reference file needs to be updated to
set 'output_type' = multi.
We could hard code it in the calwebb_spec2 code - but it is probably better to have a parameter reference file.
For miri the current calwebb_spec3 parameter reference file is set to
class: jwst.cube_build.cube_build_step.CubeBuildStep
name: cube_build
parameters: {output_type: band, skip: false}

If NIRSPec wants cubes other than 'band' from calspec3 then they should also create a parameter reference file
for calspec3.

Tasks

  • If you have a specific reviewer in mind, tag them.
  • add a build milestone, i.e. Build 12.0 (use the latest build if not sure)
  • Does this PR change user-facing code / API? (if not, label with no-changelog-entry-needed)

Regression test passing: https://github.com/spacetelescope/RegressionTests/actions/runs/26462438602
- [ ] Do truth files need to be updated ("okified")?
- [ ] after the reviewer has approved these changes, run okify_regtests to update the truth files

@jemorrison
jemorrison requested a review from a team April 7, 2026 18:00
@jemorrison
jemorrison marked this pull request as draft April 7, 2026 18:00
@codecov

codecov Bot commented Apr 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.00000% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.86%. Comparing base (2976f24) to head (df50c70).

Files with missing lines Patch % Lines
jwst/cube_build/cube_build_step.py 64.28% 5 Missing ⚠️
jwst/cube_build/data_types.py 66.66% 1 Missing ⚠️
jwst/cube_build/ifu_cube.py 96.77% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #10433      +/-   ##
==========================================
+ Coverage   86.59%   86.86%   +0.26%     
==========================================
  Files         374      374              
  Lines       40471    40305     -166     
==========================================
- Hits        35047    35010      -37     
+ Misses       5424     5295     -129     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jemorrison

Copy link
Copy Markdown
Collaborator Author

I running the regression test to see if I need to fix any tests.

@jemorrison

Copy link
Copy Markdown
Collaborator Author

@drlaw1558 One other thing while we are cleaning up the interface for cube build. Currently the parameter 'linear_wave' only works for NIRSpec data. This allows the user to switch between linear and non-linear wavelengths dimensions. The default value is True. For MIRI data, cube_build looks at the data that is to make the cube and decides if it should be linear or non-linear. In general, ifu cubes of a single band are linear and multi channel or multi band cubes can be non-linear. Cubes from spec2 are non-linear and cubes from spec3 are linear. But the user can run cube_build offline and create non-linear cube from spec3 associations by selecting the bands or channels for the cube.
Do you want to keep cube build determining if the ifu cubes have a linear or non-linear wavelength or move control to the 'linear_wave' parameter ?

@melanieclarke melanieclarke left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a very nice simplification of the code!

Can you please add some unit tests for the expected filenames in various cases?

Also, I think we will need a breaking change note, since this removes parameters from the spec.

Comment thread changes/10433.cube_build.rst Outdated
Comment thread jwst/cube_build/ifu_cube.py Outdated
Comment thread jwst/cube_build/ifu_cube.py Outdated
Comment thread jwst/cube_build/ifu_cube.py Outdated
Comment thread jwst/cube_build/ifu_cube.py Outdated
Comment thread jwst/cube_build/ifu_cube.py Outdated
Comment thread jwst/cube_build/cube_build_step.py Outdated
@jemorrison

Copy link
Copy Markdown
Collaborator Author

Currently the calwebb_spec2.py file sets output_type = 'multi' for MIRI MRS data and for NIRSpec it is set to 'band'
Cube_build_step defaults output_type to band - so these changes should not cause any problems.

@jemorrison

Copy link
Copy Markdown
Collaborator Author

@melanieclarke I wrote a unit test for testing suffix names but it uses from unittest.mock import MagicMock
I don't think other unit tests use MagicMock so if you don't want it done this way let me know.
I needed a way to call ifu_cube and not have a ton of set up just to test the filename suffix.

@jemorrison

Copy link
Copy Markdown
Collaborator Author

@melanieclarke https://github.com/spacetelescope/RegressionTests/actions/runs/27244907925
There are regression test failure but not related to this PR (they are imaging failures)

@melanieclarke melanieclarke left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test failures are unrelated, from a bad CRDS context. I think everything looks good now, all comments addressed, so I will go ahead and get this merged. Thanks for all the hard work on this one!

@melanieclarke
melanieclarke merged commit 310c5f7 into spacetelescope:main Jun 10, 2026
29 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Clean up cube_build interface

5 participants