kis_tools issueshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues2021-03-22T08:26:49Zhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/271clean GRIS header2021-03-22T08:26:49ZCarl Schafferclean GRIS headerAfter discussing with Manolo and Carlos while working on [the slit orientation issue](https://leibniz-kis.atlassian.net/browse/SR-99?atlOrigin=eyJpIjoiMTE2ZjAzN2EwNjlhNDc5MmFiYTdmMDYzZTNkOGYyOTkiLCJwIjoiaiJ9) I found out that some of the...After discussing with Manolo and Carlos while working on [the slit orientation issue](https://leibniz-kis.atlassian.net/browse/SR-99?atlOrigin=eyJpIjoiMTE2ZjAzN2EwNjlhNDc5MmFiYTdmMDYzZTNkOGYyOTkiLCJwIjoiaiJ9) I found out that some of the values we are propagating in the GRIS header are not correct, they might:
* be historic values such as `SLITORIE` which do not have a well-defined or obvious function
* be values that apply only to observations but not single slit positions (e.g. `AZIMUT`, `ELEVATION`)
* be unreliable due to different ephemerides code used for producing them/not be compliant with the telescope's definitions (again `AZIMUTH`)
For fully consistent headers, we would need to go through each value, assess the definition and validity, document that and throw out all confusing values.https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/257CTYPE inconsistency in header for data in spectroscopic mode2021-02-11T17:33:36ZVigeesh GangadharanCTYPE inconsistency in header for data in spectroscopic modeFor dataset in spectroscopic mode, the first axis seems to be always wavelength.<br>
But this is not consistent with the CTYPE-_i_ keywords later.
e.g.
```
SIMPLE = T / conforms to FITS standard ...For dataset in spectroscopic mode, the first axis seems to be always wavelength.<br>
But this is not consistent with the CTYPE-_i_ keywords later.
e.g.
```
SIMPLE = T / conforms to FITS standard
BITPIX = 32 / array data type
NAXIS = 3 / number of array dimensions
NAXIS1 = 1010 / Length of data axis 1
NAXIS2 = 484 / Length of data axis 2
NAXIS3 = 1 / Length of data axis 3
FILENAME= 'gris_20160613_120408_l1s_008_120_0029.fits' / Name of file
EXTNAME = '20160613_008_120_0029' / Unique HDU name
POINT_ID= '20160613_008' / Unique (re-)pointing ID
OBS_TRGT= 'Sunspot(s)' / Observation Target
WCSNAME = 'Helioprojective Cartesian' / nan
CTYPE1 = 'HGLN-TAN' / Type of coordinates along axis 1
CUNIT1 = 'arcsec ' / Units along axis 1
CRPIX1 = 0 / Reference pixel
CRVAL1 = 297.671 / Value at reference pixel on axis 1
CDELT1 = 0.080792 / Sampling along axis 1
CSYER1 = 1900 / Systematic Error along axis 1
CTYPE2 = 'HGLT-TAN' / Type of coordinates along axis 2
CUNIT2 = 'arcsec ' / Units along axis 2
CRPIX2 = 0 / Reference pixel
CRVAL2 = 408.258 / Value at reference pixel on axis 2
CDELT2 = 0.080792 / Sampling along axis 2
CSYER2 = 1900 / Systematic Error along axis 2
CTYPE3 = 'WAVE ' / Type of coordinates along axis 3
CUNIT3 = 'Angstrom' / Units along axis 3
CRPIX3 = 0 / Reference pixel
CRVAL3 = 10823.076 / Value at reference pixel on axis 3
CDELT3 = 0.017998 / Sampling along axis 3
CSYER3 = 0.017998 / Systematic Error along axis 3
```WCS fixesCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/259GRIS Coordinates without cross correlation2021-02-11T17:32:42ZCarl SchafferGRIS Coordinates without cross correlation
In general we want to have a full cross correlation of coordinates fro GRIS maps. This is the preferred method of getting coordinates for WCS keywords. Back when we did this, we assumed SLITPOS_X and SLITPOS_Y to give coordinates in a ...
In general we want to have a full cross correlation of coordinates fro GRIS maps. This is the preferred method of getting coordinates for WCS keywords. Back when we did this, we assumed SLITPOS_X and SLITPOS_Y to give coordinates in a system that is aligned with the solar axis and for that matter with the HMI system.
Looking at the location guesses from the Mercury transit of last year, it seems very likely, that there is an additional rotation between the systems that we don't account for. This would also affect our uncertainty estimates for non-correlated files. See https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/199
![transit](/uploads/9e4bb8c70cef34cb3b68c6fa0964b603/transit.gif)WCS fixeshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/279gris header: AO_MODES=02021-10-11T16:01:08ZCarl Schaffergris header: AO_MODES=0Corrected modes by ao are set to 0 in the sample file I sent to Oslo, we should check where that Value comes from and whether it's correct. If we don't know, we should probably switch to NAN or something similar.Corrected modes by ao are set to 0 in the sample file I sent to Oslo, we should check where that Value comes from and whether it's correct. If we don't know, we should probably switch to NAN or something similar.https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/258GRIS Header Observer Metadata2021-02-23T10:01:48ZCarl SchafferGRIS Header Observer Metadata![Clipboard_-_November_26__2020_2_54_PM](/uploads/100ec7c094c94884841d2cfe7fccea67/Clipboard_-_November_26__2020_2_54_PM.png)
We should add the HGLN_obs and dsun_ref keywords to the header to silence the astropy warning![Clipboard_-_November_26__2020_2_54_PM](/uploads/100ec7c094c94884841d2cfe7fccea67/Clipboard_-_November_26__2020_2_54_PM.png)
We should add the HGLN_obs and dsun_ref keywords to the header to silence the astropy warningWCS fixesCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/256GRIS header: RSUN_REF should be in meters2020-12-08T08:33:32ZVigeesh GangadharanGRIS header: RSUN_REF should be in metersIt seems like the RSUN_REF is in arcsec, but should be in meters ([Thomson et al 2010](https://ui.adsabs.harvard.edu/abs/2010A&A...515A..59T/abstract)).<br>It seems like the RSUN_REF is in arcsec, but should be in meters ([Thomson et al 2010](https://ui.adsabs.harvard.edu/abs/2010A&A...515A..59T/abstract)).<br>https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/280gris headers changes2021-10-11T16:36:09ZCarl Schaffergris headers changes* [x] Origin: KIS
* [x] Timeunit for exposure times/ change time values to seconds* [x] Origin: KIS
* [x] Timeunit for exposure times/ change time values to secondshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/254Gris WCS: CTYPE should be HP instead of HG2021-02-11T17:34:08ZCarl SchafferGris WCS: CTYPE should be HP instead of HGExcempt from rocketchat:
> Vigeesh:
> Question about GRIS data header
> ```
> WCSNAME = 'Helioprojective Cartesian' / nan
> CTYPE1 = 'HGLN-TAN' / Type of coordinates along axis 1
> CUNIT1 = 'arcsec ' / Units alon...Excempt from rocketchat:
> Vigeesh:
> Question about GRIS data header
> ```
> WCSNAME = 'Helioprojective Cartesian' / nan
> CTYPE1 = 'HGLN-TAN' / Type of coordinates along axis 1
> CUNIT1 = 'arcsec ' / Units along axis 1
> CRPIX1 = 0 / Reference pixel
> CRVAL1 = -393.014 / Value at reference pixel on axis 1
> CDELT1 = 0.027174 / Sampling along axis 1
> ```
> HGLN-TAN according to Thompson et al refers to Stonyhurst heliographic longitude
> so, I'm a bit confused about why it is called "Helioprojective Cartesian"?
> also the CDELT1 is in HGLN-TAN or HPLN-TAN?
>
> schaffer
> 10:56 AM
> You're right, that is inconsistent. I believe CTYPE should be HPLT-TAN, at least that is what is being done under the hood. I'll open an issue and correct this in all files sometime in the near future.
* [x] fix code
* [x] re-format files
* [ ] re-upload files to archiveWCS fixeshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/151headers: Btype for spec runs2020-04-21T16:44:14ZCarl Schafferheaders: Btype for spec runs`Btype` shouldn't be `Stokes Spectra along slit` for spectroscopic run![Selection_052](/uploads/0fcb137a42b58bb1191f44e549d5c167/Selection_052.png)`Btype` shouldn't be `Stokes Spectra along slit` for spectroscopic run![Selection_052](/uploads/0fcb137a42b58bb1191f44e549d5c167/Selection_052.png)https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/249headers: EXPTIME2020-12-10T12:11:27ZCarl Schafferheaders: EXPTIME`EXPTIME`is required by solarsoft but strongly discouraged by SOLARNET
discussion:
> hola!
> is it true that we do not have an entry for 'exposure time' in the header metadata?
>
> 9:22 AM
> morning 🙂
> no, we have that
>
> 9:23 AM
>...`EXPTIME`is required by solarsoft but strongly discouraged by SOLARNET
discussion:
> hola!
> is it true that we do not have an entry for 'exposure time' in the header metadata?
>
> 9:22 AM
> morning 🙂
> no, we have that
>
> 9:23 AM
> ok. aparently Lucia mentioned in the questionaire that we do not have it
>
> 9:25 AM
> what we don't have is the
> EXPTIME
> keyword. SOLARNET discourqages using it, as it is unclear whether it means single frame exposure or total exposure time
> what we do instead for GRIS (as recomended by solarnet) :
> ```
> XPOSURE = 300.0 / [ms] Accumulated exposure time
> TEXPOSUR= 100.0 / [ms] Single-exposure time
> NSUMEXP = 3 / Number of summed exposures
> ```
>
> i think for GSJC we added the EXPTIME keyword specifically for lucia
>
> 9:28 AM
> ok.
> i believe what LK meant is that 'exptime' is/was widely used and i guess SolarSoft requires it.
> When Morten was around there was some agreement with LK to include duplicates of some keywords.
> I bet 'exptime' should be one of them..
>
> 9:28 AM
> yes it was
>
> 9:28 AM
> exactly
>
> schaffer 9:26 AM
> i think for GSJC we added the EXPTIME keyword specifically for lucia
> 9:29 AM
> is there a specification or definition document on which keywords solarsoft expects?
> we could try to check whether it conflicts with SOLARNET once we start nailing down the standards
>
>
> 9:31 AM
> i don't know. should be checked.
> it is also true that SOLARNET should clarify how to deal with this issue..WCS fixeshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/269headers gris nmaps2021-04-22T10:02:39ZCarl Schafferheaders gris nmapsCheck if NMAPS is corrected in cases of trailing observations?
@yakobchuk reported that it doesn't always match for the split files.Check if NMAPS is corrected in cases of trailing observations?
@yakobchuk reported that it doesn't always match for the split files.WCS fixesCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/291IFU header_validation2022-04-21T10:49:01ZCarl SchafferIFU header_validation`kis_tools.gris.ifu_fits_file.IFUFitsFile.get_cleaned_header` overwrites the parent implementation and omits the call to `validate_headers`
Asess whether this is ok`kis_tools.gris.ifu_fits_file.IFUFitsFile.get_cleaned_header` overwrites the parent implementation and omits the call to `validate_headers`
Asess whether this is okhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/242kis_tools git info2020-09-21T11:06:28ZCarl Schafferkis_tools git infoIf KIS tools is installed locally, it is impossible to update git_info dynamically. I need to either remove it or check it in at each commit.
Even better: add the generation to the install script and add it to MANIFEST.inIf KIS tools is installed locally, it is impossible to update git_info dynamically. I need to either remove it or check it in at each commit.
Even better: add the generation to the install script and add it to MANIFEST.inCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/267LARS: add wavelenght fix as shell script2020-12-17T12:03:23ZCarl SchafferLARS: add wavelenght fix as shell scriptLARS doesn't add wavelength info to the header by default. We need to add it post-fact by querying it from the data. There is an IDL routine but python would be nicer. Add a cmdline script to add the InfoLARS doesn't add wavelength info to the header by default. We need to add it post-fact by querying it from the data. There is an IDL routine but python would be nicer. Add a cmdline script to add the Infohttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/207module: header tools2020-07-16T10:14:48ZCarl Schaffermodule: header toolsWrite a module that provides header checking from templates as well as well as eventually header generation for different instruments.
Should have different tests such as:
* [ ] Compare planned list of keywords to actual list of keywo...Write a module that provides header checking from templates as well as well as eventually header generation for different instruments.
Should have different tests such as:
* [ ] Compare planned list of keywords to actual list of keywords
* [ ] Check typing of values
* [ ] Check FITS comment lenghts in templates
* [ ] Host definitive templates for different instrumentshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/299'scanedge' in SCANCNTR keyword2022-01-28T15:50:32ZCarl Schaffer'scanedge' in SCANCNTR keywordSometimes gris coordinates do not reference off of the slit center. Add a `NotImplemented` error for these cases and open an issue for futher investigation at a future point in time. There are not sufficiently many examples to warrant a ...Sometimes gris coordinates do not reference off of the slit center. Add a `NotImplemented` error for these cases and open an issue for futher investigation at a future point in time. There are not sufficiently many examples to warrant a lot of time spent.
* [ ] Ask manolo, berke or lucia about this... or reiner?Carl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/260send copy of gris header to Oslo2021-04-20T09:30:56ZCarl Schaffersend copy of gris header to OsloSend a copy of the GRIS header to stein vidar ([mail](mailto:s.v.h.haugan@astro.uio.no) to check up on our SOLARNET compliancy.
Questions to ask:
* [x] Degenerate axis for WCS in slit?
* [x] EXPTIME and DATE-OBS in SOALRSOFT (https://ww...Send a copy of the GRIS header to stein vidar ([mail](mailto:s.v.h.haugan@astro.uio.no) to check up on our SOLARNET compliancy.
Questions to ask:
* [x] Degenerate axis for WCS in slit?
* [x] EXPTIME and DATE-OBS in SOALRSOFT (https://www.lmsal.com/solarsoft/ssw_standards.html), we will add them redundantly
* [x] any other commentsWCS fixeshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/308Stepsize value might be wrong for old data2022-04-26T07:25:07ZCarl SchafferStepsize value might be wrong for old dataThe Cassda-Gui prints the following message on loading old datasets e.g. 20150603_008:
![image](/uploads/8a990cd860cc47c77c487519629b6bed/image.png)
This implies that the step size value in the header might not be correct. InvestigateThe Cassda-Gui prints the following message on loading old datasets e.g. 20150603_008:
![image](/uploads/8a990cd860cc47c77c487519629b6bed/image.png)
This implies that the step size value in the header might not be correct. Investigatehttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/307use updated tag list for header translation2022-03-31T08:28:31ZCarl Schafferuse updated tag list for header translationThe heqader translator has a `to_OBS_TRGT`method that is not being used. It should be modified to be used. Also the `.csv` file in kis_tools/gris/resources containing the target tag information should be extended with Sanis tags from 202...The heqader translator has a `to_OBS_TRGT`method that is not being used. It should be modified to be used. Also the `.csv` file in kis_tools/gris/resources containing the target tag information should be extended with Sanis tags from 2022:
[missing_targets_main_tag_added.csv](/uploads/6ce45a8503f63c1443b1388488a48dcf/missing_targets_main_tag_added.csv)https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/150WCS Coordinates: check consistency2019-11-07T15:24:16ZCarl SchafferWCS Coordinates: check consistencyWrite tests that ensure that the corrected coordinates within a Gris Split file correspond to those expected from the cross correlation (or are at least close enough)Write tests that ensure that the corrected coordinates within a Gris Split file correspond to those expected from the cross correlation (or are at least close enough)