ZAP - KCWI

[gannonjs]

I'm currently looking at using your code ZAP on some KCWI data. Messing about with some test data I am running into problems where it seems the FITS header isn't being correctly parsed by the program [see below]. Do you know of anyone using ZAP for KCWI data?

It seems to be a problem with fits.open in astropy.io:

I realise that this may not be related to ZAP but will be grateful for any help you are able to provide.

[saimn]

Hi,
I don't know about anyone using ZAP for KCWI data, but I would be interested to see of it works.
The issue is that the code is assuming that the data lies in the first extension:

zap/zap/zap.py

Lines 342 to 347 in ed07445

	def __init__(self, musecubefits, pca_class=None, n_components=None):
	self.musecubefits = musecubefits
	with fits.open(musecubefits) as hdul:
	self.ins_mode = hdul[0].header.get('HIERARCH ESO INS MODE')
	self.cube = hdul[1].data
	self.header = hdul[1].header

This could easily be fixed by adding a parameter to specify the HDU name or index.
Do you know if there is some public data that I could use for a quick test ?

[gannonjs]

Thanks for the quick reply,
I am unsure of any public data for KCWI given how new it is but I do have access to a few frames that turned out to be highly uninteresting. I have reduced two through the KCWI pipeline and put them in a dropbox for you:

https://www.dropbox.com/sh/e5yirpylmimn7d4/AABnmc1MtxAYMK6809F8lRtIa?dl=0

_NOSKY variants are the cubes without the default KCWI sky subtraction [for my science I will probably look into doing filled field sky subtraction using offset sky frames]. Thanks for the help

[saimn]

Thanks @gannonjs , I tried with your test cubes and added in the code a detection of the instrument in order to read and write to the correct extensions. Is it good to suppose that KCWI data are always in the primary HDU ? Otherwise I will need to add a keyword to specify the HDUs indices.

So with this change (you can try the 'kwci' branch) it seems to work. I created quickly a mask for the edges, with null values, but to get better results the artifacts at the beginning and end of the cube should also be masked. Zap does not currently have an option to skip a given wavelength range, so the easy way to proceed for now would be give it a cropped cube, or a cube where these wavelengths are filled with NaNs.

[gannonjs]

Thank you very much for how quickly you have helped me with this. I am only just starting to use KCWI so am unable to comment fully on supposing the KCWI data is in the primary HDU though I have put multiple different obs from different nights through the pipeline and each seems to run now so I suspect it is a fair assumption.

I have run a few cropped cubes through ZAP now and am getting promising results. [though need to play around with the parameters more]

[saimn]

Thanks for testing the branch. Let's go with this for now, we can still revisit how to load KCWI data in the future if needed.
The most important parameter is the mask, in order to be sure to exclude the sources signal from the eigenvectors. Then the number of remaining sky spaxels is important. If you have dedicated sky exposures, zap can use these to compute the eigenvectors. This does not seems to work so well for MUSE because the sky exposures are usually too short, but I don't know for KCWI.