Skip to content

GitLab

  • Menu
Projects Groups Snippets
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • sdc_wiki sdc_wiki
  • Project information
    • Project information
    • Activity
    • Members
  • Analytics
    • Analytics
    • Value stream
  • Wiki
    • Wiki
  • Activity
Collapse sidebar
  • sdc
  • sdc_wikisdc_wiki
  • Wiki
  • Standards_and_references
  • Observation Segmentation

Last edited by Carl Schaffer May 18, 2020
Page history

Observation Segmentation

Observation Segmentation

The basic data unit for our archive is an observation. Within the database, observations are represented by an observation document with a well defined structure.

A set of one or more science-ready data files and all directly related data products.

The properties shared by an observation are used to find the observations within the archive. This means that the definition of observations needs to be granular enough to ensure that only strongly related measurements are grouped within in one observation. On the other hand observation descriptions also need to be generic enough that a query is able yields a small enough amount of results to not overwhelm a potential user.

Files within one observation should share common properties, such as proximity in time or wavelength, while sufficient distance in such parameters warrants the definition of a new observation. See the exmple of a GRIS observation data set from 2019:

observation

Case Study for Observation Segmentation: BBI

The data aquisition framework of BBI does not explicitly mark files as belonging to certain observations. Nevertheless the concept of observations can be imposed on the data and reconstructed from the files. BBI observes a feature on the sun for a period of time specified by the user, cycling through four different wavelength filters every couple of seconds. When a new feature is focused with the telescope, there will be a brief period during which the instrument does not record images. These breaks can be used to separate observations, making a collection of images with consecutive timestamps and identical wavelengths the group of files used to build observations for bbi.

Below we have included some exemplary observation metadata, where two targets were observed in four wavelengths. We have scaled the time and wavelength variables and made a scatter plot of all 600 files taken on that day. By eye it is almost trivial to distinguish the different observations, and a simple clustering algorithm allows us to automatically group the observations, the result of the clustering is encoded in the color of the scattered points.

bbi_automated_clustering

Observation Definitions for Some Examplary Instruments

For the currently integrated instruments GRIS, BBI, ChroTel and LARS we have defined observations as follows:

Instrument Observation Definition Observation Naming Scheme
GRIS full scan of a feature, possibly with multiple repetitions gris_YYYYMMDD_OBSNUMBER
BBI consecutive images of a feature sharing a common wavelength filter bbi_YYYYMMDD_OBSNUMBER_WL[wavelength in nm]
ChroTel single full disk measurement in one wavelength chrotel_YYYYMMDD_HHMMSS_[ca|he|ha]
LARS single measurement of an integrated spectrum lars_l12_20170514-092649_clv5380_mu085_n
Clone repository
  • Restheart API
  • Starting Page for Data Providers
  • Starting Page for Users
  • Starting page for Developers
  • Who's who?
  • dev_manuals
    • Logging and Debugging Messages
    • Setup Webapp Development
  • developers
    • pipeline orchestration
  • for_developers
    • how_to
      • Command Line Interfaces
      • Setting up with KIS GitLab
  • Home
  • knowldge
    • WOSSL 2020
  • old_sidebar
  • standards_and_references
    • BBI Header
    • ChroTel Data Quality Categories
View All Pages