kis_tools issueshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues2020-05-12T09:46:26Zhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/188Gris PhD Embargos2020-05-12T09:46:26ZCarl SchafferGris PhD EmbargosWrite list of observers per day to NBG, she will check which dates should be embargoed.Write list of observers per day to NBG, she will check which dates should be embargoed.Demleitner VisitCarl SchafferCarl Schaffer2020-05-11https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/303skewed boxes in location previews for 2021 data2022-03-14T17:44:20ZCarl Schafferskewed boxes in location previews for 2021 data![image](/uploads/6155a8922f12d28ec42010d5fbb93f41/image.png)
![image](/uploads/e1245cbcd549c057d94ccb78cf144c80/image.png)
![image](/uploads/307d603ecb23984b5c8f376750ea45b2/image.png)![image](/uploads/6155a8922f12d28ec42010d5fbb93f41/image.png)
![image](/uploads/e1245cbcd549c057d94ccb78cf144c80/image.png)
![image](/uploads/307d603ecb23984b5c8f376750ea45b2/image.png)Carl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/302previews: respect aspect ratio step size2022-02-21T12:21:22ZCarl Schafferpreviews: respect aspect ratio step sizeSome observations use a step size perpendicular larger than the resolution along the slit. This leads to skewed preview images, as the pixels would essentially need to be non square.
matplotlib supports this, but it also needs to be che...Some observations use a step size perpendicular larger than the resolution along the slit. This leads to skewed preview images, as the pixels would essentially need to be non square.
matplotlib supports this, but it also needs to be checked through the entire chain of calculations and plotting:
The preview plots should use the wcs keywords to determine the values for the aspect ratio and spatial extent of FOVs.
Example:
![image](/uploads/48b580236f2776f341cb7d7f290d906a/image.png)
https://archive.sdc.leibniz-kis.de/SDCDetailServlet?Instrument=gris&ObjectId=607c9dcd18262e0426a2b2a2
Also see the corresponding issue on [Jira](https://leibniz-kis.atlassian.net/jira/servicedesk/projects/SR/queues/custom/10/SR-173)https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/299'scanedge' in SCANCNTR keyword2022-01-28T15:50:32ZCarl Schaffer'scanedge' in SCANCNTR keywordSometimes gris coordinates do not reference off of the slit center. Add a `NotImplemented` error for these cases and open an issue for futher investigation at a future point in time. There are not sufficiently many examples to warrant a ...Sometimes gris coordinates do not reference off of the slit center. Add a `NotImplemented` error for these cases and open an issue for futher investigation at a future point in time. There are not sufficiently many examples to warrant a lot of time spent.
* [ ] Ask manolo, berke or lucia about this... or reiner?Carl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/297GRIS data is flipped depending on step-angle2022-04-21T10:46:49ZCarl SchafferGRIS data is flipped depending on step-angleDepending on how the slit data is scanned, images are flipped w.r.t. HMI.
We should consider serving flipped data to the end-users instead of what we're doing right now.
* [ ] Check whether this is already done somewhere
* [ ] Evaluat...Depending on how the slit data is scanned, images are flipped w.r.t. HMI.
We should consider serving flipped data to the end-users instead of what we're doing right now.
* [ ] Check whether this is already done somewhere
* [ ] Evaluate how much effort a fix would be
* [ ] decide on actionCarl SchafferCarl Schaffer2021-11-30https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/296Misleading Labels for GRIS Previews2022-02-01T16:08:40ZCarl SchafferMisleading Labels for GRIS PreviewsThe labels of the GRIS slit previews imply alignment along the X and Y axes of the Helioprojective coordinate system, while they are actually relative coordinates within the map. Make this clearer.
Also the colorbar of the plots does no...The labels of the GRIS slit previews imply alignment along the X and Y axes of the Helioprojective coordinate system, while they are actually relative coordinates within the map. Make this clearer.
Also the colorbar of the plots does not seem to match up with the data. Investigate and label/omit.
Checklist:
* [ ] Axis Lables
* [ ] Colorbar
* [ ] Re-run Preview generation
* [ ] Upload new Previews
## Excerpt of conversation:
> yakobchuk
> 4:00 PM
> Hi Carl! What's up? Don't remember if I raised this: I think gris image preview labels are confusing, if not wrong. X/Y coords on top indicate center of the frame in helio-projective coord.sys., while deltaX and deltaY labels refer to instrumental relative coords. Here is a good example: https://archive.sdc.leibniz-kis.de/SDCDetailServlet?Instrument=gris&ObjectId=5ee10fdf871689ed8458baad
> The spot is clearly stretched along Y (LAT) (see whole sun image), while on the preview image it goes along deltaX. What do you think?
>
>
> yakobchuk
> 4:21 PM
> Then regarding the colorbar: what are these numbers? I see they are much smaller than intensity data raw counts I get in grisview. Also, I think colorbar label is missing showing what is this (average intensity? polarization?) and in which units.
>
> schaffer
> schaffer
> 9:38 AM
> Hey Taras,
>
> you're completely right concerning the images, this is an issue that hadn't occured to me earlier, but it just came up to me a couple of days ago while checking the IFU coordinates. I agree that the names x and y are misleading as they don't represent actual coordinates. For slit data we should probably move to something like "parallel/horizontal" to slit. Philip also once proposed having two arrows on the image indicating the directions of Solar North and Disk center. I'll open an issue on kis_tools 👍 Thanks for the feedback.
>
> The colorbar we should probably omit or label as arbitrary units: The image data is taken from manolos .cm files, I'm not sure whether any preprocessing is done there beforehand, but there might well be something that causes a discrepancy to the values in the split files. I'll also add it to the issue.https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/287Add interface to gristagger2021-09-30T10:30:05ZCarl SchafferAdd interface to gristaggerthe gristagger utility in the bin folder is undocumented even though it's pretty useful. Add a cleaner interface based on the kis_tools interface structure and add documentation.the gristagger utility in the bin folder is undocumented even though it's pretty useful. Add a cleaner interface based on the kis_tools interface structure and add documentation.https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/286Add interface to grisplot2021-09-30T10:30:06ZCarl SchafferAdd interface to grisplotthe grisplot utility in the bin folder is undocumented even though it's pretty useful. Add a cleaner interface based on the kis_tools interface structure and add documentation.the grisplot utility in the bin folder is undocumented even though it's pretty useful. Add a cleaner interface based on the kis_tools interface structure and add documentation.https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/282wcs wavelength retrieval shifted by one pixel2021-05-06T14:48:42ZCarl Schafferwcs wavelength retrieval shifted by one pixelPhilip pointed out to me that the wcs retrieval of wavelengths might be off by one pixel. We narrowed the problem down to the definition of the `CRPIX`keyword. We set it to `0` meaning to indicate the beginning of the array, but using th...Philip pointed out to me that the wcs retrieval of wavelengths might be off by one pixel. We narrowed the problem down to the definition of the `CRPIX`keyword. We set it to `0` meaning to indicate the beginning of the array, but using the `WCS.wcs_pix2world(pixels,0)` method to retrieve the value yields a wavelength array with larger by `CDELT`than we would expect.
We either need to set the CRPIX keyword to 1 or use 1 for the origin value of the retrieval method.
It is unclear which of these solutions should be preferred.
Links:
* [related issue on github](https://github.com/astropy/astropy/issues/9989)
* [wcs paper](https://www.aanda.org/articles/aa/pdf/2002/45/aah3859.pdf)
* [astropy docs for WCS.wcs_pix2world](https://docs.astropy.org/en/stable/api/astropy.wcs.WCS.html#astropy.wcs.WCS.wcs_pix2world)Carl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/280gris headers changes2021-10-11T16:36:09ZCarl Schaffergris headers changes* [x] Origin: KIS
* [x] Timeunit for exposure times/ change time values to seconds* [x] Origin: KIS
* [x] Timeunit for exposure times/ change time values to secondshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/279gris header: AO_MODES=02021-10-11T16:01:08ZCarl Schaffergris header: AO_MODES=0Corrected modes by ao are set to 0 in the sample file I sent to Oslo, we should check where that Value comes from and whether it's correct. If we don't know, we should probably switch to NAN or something similar.Corrected modes by ao are set to 0 in the sample file I sent to Oslo, we should check where that Value comes from and whether it's correct. If we don't know, we should probably switch to NAN or something similar.https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/269headers gris nmaps2021-04-22T10:02:39ZCarl Schafferheaders gris nmapsCheck if NMAPS is corrected in cases of trailing observations?
@yakobchuk reported that it doesn't always match for the split files.Check if NMAPS is corrected in cases of trailing observations?
@yakobchuk reported that it doesn't always match for the split files.WCS fixesCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/262location plot: 20150417 run 2 Uncertainties in locplot should be small2021-03-23T09:13:50ZCarl Schafferlocation plot: 20150417 run 2 Uncertainties in locplot should be smallThe plot is made with large uncertainties, even though there is a cross correlation result with HMI. Check!The plot is made with large uncertainties, even though there is a cross correlation result with HMI. Check!WCS fixesCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/259GRIS Coordinates without cross correlation2021-02-11T17:32:42ZCarl SchafferGRIS Coordinates without cross correlation
In general we want to have a full cross correlation of coordinates fro GRIS maps. This is the preferred method of getting coordinates for WCS keywords. Back when we did this, we assumed SLITPOS_X and SLITPOS_Y to give coordinates in a ...
In general we want to have a full cross correlation of coordinates fro GRIS maps. This is the preferred method of getting coordinates for WCS keywords. Back when we did this, we assumed SLITPOS_X and SLITPOS_Y to give coordinates in a system that is aligned with the solar axis and for that matter with the HMI system.
Looking at the location guesses from the Mercury transit of last year, it seems very likely, that there is an additional rotation between the systems that we don't account for. This would also affect our uncertainty estimates for non-correlated files. See https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/199
![transit](/uploads/9e4bb8c70cef34cb3b68c6fa0964b603/transit.gif)WCS fixeshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/258GRIS Header Observer Metadata2021-02-23T10:01:48ZCarl SchafferGRIS Header Observer Metadata![Clipboard_-_November_26__2020_2_54_PM](/uploads/100ec7c094c94884841d2cfe7fccea67/Clipboard_-_November_26__2020_2_54_PM.png)
We should add the HGLN_obs and dsun_ref keywords to the header to silence the astropy warning![Clipboard_-_November_26__2020_2_54_PM](/uploads/100ec7c094c94884841d2cfe7fccea67/Clipboard_-_November_26__2020_2_54_PM.png)
We should add the HGLN_obs and dsun_ref keywords to the header to silence the astropy warningWCS fixesCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/257CTYPE inconsistency in header for data in spectroscopic mode2021-02-11T17:33:36ZVigeesh GangadharanCTYPE inconsistency in header for data in spectroscopic modeFor dataset in spectroscopic mode, the first axis seems to be always wavelength.<br>
But this is not consistent with the CTYPE-_i_ keywords later.
e.g.
```
SIMPLE = T / conforms to FITS standard ...For dataset in spectroscopic mode, the first axis seems to be always wavelength.<br>
But this is not consistent with the CTYPE-_i_ keywords later.
e.g.
```
SIMPLE = T / conforms to FITS standard
BITPIX = 32 / array data type
NAXIS = 3 / number of array dimensions
NAXIS1 = 1010 / Length of data axis 1
NAXIS2 = 484 / Length of data axis 2
NAXIS3 = 1 / Length of data axis 3
FILENAME= 'gris_20160613_120408_l1s_008_120_0029.fits' / Name of file
EXTNAME = '20160613_008_120_0029' / Unique HDU name
POINT_ID= '20160613_008' / Unique (re-)pointing ID
OBS_TRGT= 'Sunspot(s)' / Observation Target
WCSNAME = 'Helioprojective Cartesian' / nan
CTYPE1 = 'HGLN-TAN' / Type of coordinates along axis 1
CUNIT1 = 'arcsec ' / Units along axis 1
CRPIX1 = 0 / Reference pixel
CRVAL1 = 297.671 / Value at reference pixel on axis 1
CDELT1 = 0.080792 / Sampling along axis 1
CSYER1 = 1900 / Systematic Error along axis 1
CTYPE2 = 'HGLT-TAN' / Type of coordinates along axis 2
CUNIT2 = 'arcsec ' / Units along axis 2
CRPIX2 = 0 / Reference pixel
CRVAL2 = 408.258 / Value at reference pixel on axis 2
CDELT2 = 0.080792 / Sampling along axis 2
CSYER2 = 1900 / Systematic Error along axis 2
CTYPE3 = 'WAVE ' / Type of coordinates along axis 3
CUNIT3 = 'Angstrom' / Units along axis 3
CRPIX3 = 0 / Reference pixel
CRVAL3 = 10823.076 / Value at reference pixel on axis 3
CDELT3 = 0.017998 / Sampling along axis 3
CSYER3 = 0.017998 / Systematic Error along axis 3
```WCS fixesCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/256GRIS header: RSUN_REF should be in meters2020-12-08T08:33:32ZVigeesh GangadharanGRIS header: RSUN_REF should be in metersIt seems like the RSUN_REF is in arcsec, but should be in meters ([Thomson et al 2010](https://ui.adsabs.harvard.edu/abs/2010A&A...515A..59T/abstract)).<br>It seems like the RSUN_REF is in arcsec, but should be in meters ([Thomson et al 2010](https://ui.adsabs.harvard.edu/abs/2010A&A...515A..59T/abstract)).<br>https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/254Gris WCS: CTYPE should be HP instead of HG2021-02-11T17:34:08ZCarl SchafferGris WCS: CTYPE should be HP instead of HGExcempt from rocketchat:
> Vigeesh:
> Question about GRIS data header
> ```
> WCSNAME = 'Helioprojective Cartesian' / nan
> CTYPE1 = 'HGLN-TAN' / Type of coordinates along axis 1
> CUNIT1 = 'arcsec ' / Units alon...Excempt from rocketchat:
> Vigeesh:
> Question about GRIS data header
> ```
> WCSNAME = 'Helioprojective Cartesian' / nan
> CTYPE1 = 'HGLN-TAN' / Type of coordinates along axis 1
> CUNIT1 = 'arcsec ' / Units along axis 1
> CRPIX1 = 0 / Reference pixel
> CRVAL1 = -393.014 / Value at reference pixel on axis 1
> CDELT1 = 0.027174 / Sampling along axis 1
> ```
> HGLN-TAN according to Thompson et al refers to Stonyhurst heliographic longitude
> so, I'm a bit confused about why it is called "Helioprojective Cartesian"?
> also the CDELT1 is in HGLN-TAN or HPLN-TAN?
>
> schaffer
> 10:56 AM
> You're right, that is inconsistent. I believe CTYPE should be HPLT-TAN, at least that is what is being done under the hood. I'll open an issue and correct this in all files sometime in the near future.
* [x] fix code
* [x] re-format files
* [ ] re-upload files to archiveWCS fixeshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/247gris run: Coord save from other runs2020-10-01T15:19:35ZCarl Schaffergris run: Coord save from other runsGrisRuns `coord_saves` property returns saves for other runs, this shouldn't happen.
![Selection_164](/uploads/f46e62d437aea9f49bf6dd63ea358011/Selection_164.png)GrisRuns `coord_saves` property returns saves for other runs, this shouldn't happen.
![Selection_164](/uploads/f46e62d437aea9f49bf6dd63ea358011/Selection_164.png)https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/243translate header: don't run git info for every file2020-09-21T11:06:29ZCarl Schaffertranslate header: don't run git info for every filegit info is currently being run for every file during translation. this is unnecessary overheadgit info is currently being run for every file during translation. this is unnecessary overheadCarl SchafferCarl Schaffer