kis_tools issueshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues2019-11-07T15:32:06Zhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/31mapgen check observation mode for map gen2019-11-07T15:32:06ZCarl Schaffermapgen check observation mode for map genlimit to single maps for nowlimit to single maps for nowregenerate mapshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/20mapgen map truncated2018-10-02T12:40:02ZCarl Schaffermapgen map truncated![Selection_003](/uploads/fef4d8cf8c07d48c8bab36c078add7a8/Selection_003.png)![Selection_003](/uploads/fef4d8cf8c07d48c8bab36c078add7a8/Selection_003.png)regenerate mapshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/189Questions for Demleitner Visit2019-11-05T13:43:44ZCarl SchafferQuestions for Demleitner VisitQuestions:
- How to interface DACHS to our files for scanning
- use MongoDB or file system as storage and URL target (MongoDB preferred due to being more structured than our file archives)
- What sorts of interfaces do we need to prep...Questions:
- How to interface DACHS to our files for scanning
- use MongoDB or file system as storage and URL target (MongoDB preferred due to being more structured than our file archives)
- What sorts of interfaces do we need to prepare
- What infrastructure do we need to prepare
Implement GRIS embargo? Are there still datasets that need to be protected due to PhD theses?Demleitner VisitCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/188Gris PhD Embargos2020-05-12T09:46:26ZCarl SchafferGris PhD EmbargosWrite list of observers per day to NBG, she will check which dates should be embargoed.Write list of observers per day to NBG, she will check which dates should be embargoed.Demleitner VisitCarl SchafferCarl Schaffer2020-05-11https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/275location plot: wavelngth part in plot title2021-04-14T12:32:51ZCarl Schafferlocation plot: wavelngth part in plot title![image](/uploads/1e3678de2b4670afb87fad5038b9904b/image.png)
not sure where this number comes from, remove?![image](/uploads/1e3678de2b4670afb87fad5038b9904b/image.png)
not sure where this number comes from, remove?WCS fixesCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/273update manuals in confluence2021-03-25T07:33:04ZCarl Schafferupdate manuals in confluencechanges have been made to some of the tools, check in confluence whether everything is still up to datechanges have been made to some of the tools, check in confluence whether everything is still up to dateWCS fixesCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/272translate_header: illegal p0_angle2021-03-25T07:36:29ZCarl Schaffertranslate_header: illegal p0_angleHeader translation throws warning when p0 angle is not defined e.g. set to `''` in the header and ends up with a 1000" uncertainty afterward.
Need to check if this is desired behavior or not.Header translation throws warning when p0 angle is not defined e.g. set to `''` in the header and ends up with a 1000" uncertainty afterward.
Need to check if this is desired behavior or not.WCS fixeshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/269headers gris nmaps2021-04-22T10:02:39ZCarl Schafferheaders gris nmapsCheck if NMAPS is corrected in cases of trailing observations?
@yakobchuk reported that it doesn't always match for the split files.Check if NMAPS is corrected in cases of trailing observations?
@yakobchuk reported that it doesn't always match for the split files.WCS fixesCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/262location plot: 20150417 run 2 Uncertainties in locplot should be small2021-03-23T09:13:50ZCarl Schafferlocation plot: 20150417 run 2 Uncertainties in locplot should be smallThe plot is made with large uncertainties, even though there is a cross correlation result with HMI. Check!The plot is made with large uncertainties, even though there is a cross correlation result with HMI. Check!WCS fixesCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/260send copy of gris header to Oslo2021-04-20T09:30:56ZCarl Schaffersend copy of gris header to OsloSend a copy of the GRIS header to stein vidar ([mail](mailto:s.v.h.haugan@astro.uio.no) to check up on our SOLARNET compliancy.
Questions to ask:
* [x] Degenerate axis for WCS in slit?
* [x] EXPTIME and DATE-OBS in SOALRSOFT (https://ww...Send a copy of the GRIS header to stein vidar ([mail](mailto:s.v.h.haugan@astro.uio.no) to check up on our SOLARNET compliancy.
Questions to ask:
* [x] Degenerate axis for WCS in slit?
* [x] EXPTIME and DATE-OBS in SOALRSOFT (https://www.lmsal.com/solarsoft/ssw_standards.html), we will add them redundantly
* [x] any other commentsWCS fixeshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/259GRIS Coordinates without cross correlation2021-02-11T17:32:42ZCarl SchafferGRIS Coordinates without cross correlation
In general we want to have a full cross correlation of coordinates fro GRIS maps. This is the preferred method of getting coordinates for WCS keywords. Back when we did this, we assumed SLITPOS_X and SLITPOS_Y to give coordinates in a ...
In general we want to have a full cross correlation of coordinates fro GRIS maps. This is the preferred method of getting coordinates for WCS keywords. Back when we did this, we assumed SLITPOS_X and SLITPOS_Y to give coordinates in a system that is aligned with the solar axis and for that matter with the HMI system.
Looking at the location guesses from the Mercury transit of last year, it seems very likely, that there is an additional rotation between the systems that we don't account for. This would also affect our uncertainty estimates for non-correlated files. See https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/199
![transit](/uploads/9e4bb8c70cef34cb3b68c6fa0964b603/transit.gif)WCS fixeshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/258GRIS Header Observer Metadata2021-02-23T10:01:48ZCarl SchafferGRIS Header Observer Metadata![Clipboard_-_November_26__2020_2_54_PM](/uploads/100ec7c094c94884841d2cfe7fccea67/Clipboard_-_November_26__2020_2_54_PM.png)
We should add the HGLN_obs and dsun_ref keywords to the header to silence the astropy warning![Clipboard_-_November_26__2020_2_54_PM](/uploads/100ec7c094c94884841d2cfe7fccea67/Clipboard_-_November_26__2020_2_54_PM.png)
We should add the HGLN_obs and dsun_ref keywords to the header to silence the astropy warningWCS fixesCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/257CTYPE inconsistency in header for data in spectroscopic mode2021-02-11T17:33:36ZVigeesh GangadharanCTYPE inconsistency in header for data in spectroscopic modeFor dataset in spectroscopic mode, the first axis seems to be always wavelength.<br>
But this is not consistent with the CTYPE-_i_ keywords later.
e.g.
```
SIMPLE = T / conforms to FITS standard ...For dataset in spectroscopic mode, the first axis seems to be always wavelength.<br>
But this is not consistent with the CTYPE-_i_ keywords later.
e.g.
```
SIMPLE = T / conforms to FITS standard
BITPIX = 32 / array data type
NAXIS = 3 / number of array dimensions
NAXIS1 = 1010 / Length of data axis 1
NAXIS2 = 484 / Length of data axis 2
NAXIS3 = 1 / Length of data axis 3
FILENAME= 'gris_20160613_120408_l1s_008_120_0029.fits' / Name of file
EXTNAME = '20160613_008_120_0029' / Unique HDU name
POINT_ID= '20160613_008' / Unique (re-)pointing ID
OBS_TRGT= 'Sunspot(s)' / Observation Target
WCSNAME = 'Helioprojective Cartesian' / nan
CTYPE1 = 'HGLN-TAN' / Type of coordinates along axis 1
CUNIT1 = 'arcsec ' / Units along axis 1
CRPIX1 = 0 / Reference pixel
CRVAL1 = 297.671 / Value at reference pixel on axis 1
CDELT1 = 0.080792 / Sampling along axis 1
CSYER1 = 1900 / Systematic Error along axis 1
CTYPE2 = 'HGLT-TAN' / Type of coordinates along axis 2
CUNIT2 = 'arcsec ' / Units along axis 2
CRPIX2 = 0 / Reference pixel
CRVAL2 = 408.258 / Value at reference pixel on axis 2
CDELT2 = 0.080792 / Sampling along axis 2
CSYER2 = 1900 / Systematic Error along axis 2
CTYPE3 = 'WAVE ' / Type of coordinates along axis 3
CUNIT3 = 'Angstrom' / Units along axis 3
CRPIX3 = 0 / Reference pixel
CRVAL3 = 10823.076 / Value at reference pixel on axis 3
CDELT3 = 0.017998 / Sampling along axis 3
CSYER3 = 0.017998 / Systematic Error along axis 3
```WCS fixesCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/254Gris WCS: CTYPE should be HP instead of HG2021-02-11T17:34:08ZCarl SchafferGris WCS: CTYPE should be HP instead of HGExcempt from rocketchat:
> Vigeesh:
> Question about GRIS data header
> ```
> WCSNAME = 'Helioprojective Cartesian' / nan
> CTYPE1 = 'HGLN-TAN' / Type of coordinates along axis 1
> CUNIT1 = 'arcsec ' / Units alon...Excempt from rocketchat:
> Vigeesh:
> Question about GRIS data header
> ```
> WCSNAME = 'Helioprojective Cartesian' / nan
> CTYPE1 = 'HGLN-TAN' / Type of coordinates along axis 1
> CUNIT1 = 'arcsec ' / Units along axis 1
> CRPIX1 = 0 / Reference pixel
> CRVAL1 = -393.014 / Value at reference pixel on axis 1
> CDELT1 = 0.027174 / Sampling along axis 1
> ```
> HGLN-TAN according to Thompson et al refers to Stonyhurst heliographic longitude
> so, I'm a bit confused about why it is called "Helioprojective Cartesian"?
> also the CDELT1 is in HGLN-TAN or HPLN-TAN?
>
> schaffer
> 10:56 AM
> You're right, that is inconsistent. I believe CTYPE should be HPLT-TAN, at least that is what is being done under the hood. I'll open an issue and correct this in all files sometime in the near future.
* [x] fix code
* [x] re-format files
* [ ] re-upload files to archiveWCS fixeshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/253location plot: package to nice function2020-11-18T14:45:47ZCarl Schafferlocation plot: package to nice functionThere is too much logic in the gris_locplot script in the bin/ folder. Move that to a function.There is too much logic in the gris_locplot script in the bin/ folder. Move that to a function.GRIS Pipeline Automationhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/249headers: EXPTIME2020-12-10T12:11:27ZCarl Schafferheaders: EXPTIME`EXPTIME`is required by solarsoft but strongly discouraged by SOLARNET
discussion:
> hola!
> is it true that we do not have an entry for 'exposure time' in the header metadata?
>
> 9:22 AM
> morning 🙂
> no, we have that
>
> 9:23 AM
>...`EXPTIME`is required by solarsoft but strongly discouraged by SOLARNET
discussion:
> hola!
> is it true that we do not have an entry for 'exposure time' in the header metadata?
>
> 9:22 AM
> morning 🙂
> no, we have that
>
> 9:23 AM
> ok. aparently Lucia mentioned in the questionaire that we do not have it
>
> 9:25 AM
> what we don't have is the
> EXPTIME
> keyword. SOLARNET discourqages using it, as it is unclear whether it means single frame exposure or total exposure time
> what we do instead for GRIS (as recomended by solarnet) :
> ```
> XPOSURE = 300.0 / [ms] Accumulated exposure time
> TEXPOSUR= 100.0 / [ms] Single-exposure time
> NSUMEXP = 3 / Number of summed exposures
> ```
>
> i think for GSJC we added the EXPTIME keyword specifically for lucia
>
> 9:28 AM
> ok.
> i believe what LK meant is that 'exptime' is/was widely used and i guess SolarSoft requires it.
> When Morten was around there was some agreement with LK to include duplicates of some keywords.
> I bet 'exptime' should be one of them..
>
> 9:28 AM
> yes it was
>
> 9:28 AM
> exactly
>
> schaffer 9:26 AM
> i think for GSJC we added the EXPTIME keyword specifically for lucia
> 9:29 AM
> is there a specification or definition document on which keywords solarsoft expects?
> we could try to check whether it conflicts with SOLARNET once we start nailing down the standards
>
>
> 9:31 AM
> i don't know. should be checked.
> it is also true that SOLARNET should clarify how to deal with this issue..WCS fixeshttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/241Move Settings module2020-09-23T15:37:34ZCarl SchafferMove Settings moduleMove the settings module to the generic folderMove the settings module to the generic folderremove importer utilitiesCarl SchafferCarl Schafferhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/238ChroTel: Coord Bug2020-06-04T12:25:19ZCarl SchafferChroTel: Coord BugChrotel Provides strange coordinate when imported into the archive.
![image](/uploads/9203f9fe883c66675f0783217a5808d2/image.png)Chrotel Provides strange coordinate when imported into the archive.
![image](/uploads/9203f9fe883c66675f0783217a5808d2/image.png)kharon v1.0.0https://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/23220150603 translate header breaks2020-05-27T16:04:43ZCarl Schaffer20150603 translate header breaks```
(sdc) [09:40 schaffer@itchy sdc] $ gris_translate_header /dat/sdc/gris/20150603/level1_split/
Modifying /dat/sdc/gris/20150603/level1_split/gris_20150603_082644_l1p_004_001_0001.fits: 9%|â–Š | 227/2646 [00:37<03:29, 11.54it/s...```
(sdc) [09:40 schaffer@itchy sdc] $ gris_translate_header /dat/sdc/gris/20150603/level1_split/
Modifying /dat/sdc/gris/20150603/level1_split/gris_20150603_082644_l1p_004_001_0001.fits: 9%|â–Š | 227/2646 [00:37<03:29, 11.54it/s]Traceback (most recent call last):
File "/home/schaffer/anaconda3/envs/sdc/bin/gris_translate_header", line 7, in <module>
exec(compile(f.read(), __file__, 'exec'))
File "/dat/schaffer/projects/kis_tools/bin/gris_translate_header", line 7, in <module>
run_translate_header(sys.argv[1:])
File "/dat/schaffer/projects/kis_tools/kis_tools/headers/translate_header.py", line 309, in run_translate_header
main(f, outfile=f, overwrite=True)
File "/dat/schaffer/projects/kis_tools/kis_tools/headers/translate_header.py", line 221, in main
slitpos_cards = wcs_generator.slitpos_cards()
AttributeError: 'NoneType' object has no attribute 'slitpos_cards'
Modifying /dat/sdc/gris/20150603/level1_split/gris_20150603_082644_l1p_004_001_0001.fits: 9%|â–Š | 227/2646 [00:37<06:40, 6.04it/s]
```reimport with kharonhttps://gitlab.leibniz-kis.de/sdc/kis_tools/-/issues/23120140503 Translate header breaks2020-06-04T13:52:08ZCarl Schaffer20140503 Translate header breaksHeader translation does not work for 'gris_20140503_120949_l1p_011_001_0250.fits' possibly also not for successive files. Needs investigationHeader translation does not work for 'gris_20140503_120949_l1p_011_001_0250.fits' possibly also not for successive files. Needs investigationreimport with kharon