ESA EO Operations Framework (EOF)

– CSC –

Ground Segment Architecture

APPROVAL

CHANGE LOG

CHANGE RECORD

DISTRIBUTION

Table of Contents

2.1. Background 7

2.2. Scope 10

2.3. Purpose of this document 11

2.4. Document context 11

2.5. Applicable and Reference Documents 12

2.5.1. Applicable Documents 12

2.5.2. Reference Documents 12

2.6. Acronyms and Definitions 13

2.7. Document organisation 16

4.1. EOF Architecture Principles 18

4.2. EOF High Level Architecture 20

4.3. EOF Ground Segment Architecture components 23

4.3.1. Mission Planning 23

4.3.2. Data Acquisition 27

4.3.2.1. X-Band Acquisition 27

4.3.2.2. EDRS Acquisition 32

4.3.2.3. Ka-Band Acquisition 34

4.3.2.4. Mission Data Acquisition Service (MDAS) for CO2M 38

4.3.3. Production and routine quality control 38

4.3.3.1. Systematic Production 39

4.3.3.2. On-Demand Production 46

4.3.3.3. Reprocessing 49

4.3.4. Auxiliary Data Gathering 51

4.3.5. Instrument data processing algorithm and Operational Processor maintenance 54

4.3.6. Calibration and Validation 56

4.3.7. Precise Orbit Determination 58

4.3.8. Data Preservation 61

4.3.9. Data Access 66

4.3.10. Reference System 71

4.3.11. Operations Traceability service 75

4.3.12. End-to-End Operations Performance Monitoring 76

4.3.13. Services Operations Coordination 80

4.3.14. Security Monitoring 81

4.3.15. Benchmarking Service 82

4.3.16. Flight Operations Services 83

EXECUTIVE SUMMARY

The overall system architecture for the Copernicus Space Component (CSC) and its evolution have been defined on the basis of user requirements coordinated by the European Commission. The Long-Term Scenario (LTS) describes the main elements of this architecture and is maintained and evolved in an iterative process in close interaction with the European Commission (COM), EUMETSAT and Copernicus Participating States.

ESA needs to guarantee the continuity of the on-going operations with the maximum level of performances for the flying Copernicus Sentinels while facing the technical and financial challenges to adapt to the evolutions of the CSC architecture.

A transformation process for the CSC GS was established in 2018 in based on a clear set of principles and a target architecture. The principles (management and architectural) are referred hereafter as the ESA EO Operations Framework (EOF).

The EOF encompasses all the activities necessary to successfully deliver the expected level of CSC operations entrusted to ESA (i.e. establishment and maintenance of the new baseline, procurement actions, operations management, reporting, etc.).

The EOF is documented throughout a complete package describing and specifying the applicable operational concepts as well as the architecture and procurement evolutions to be adopted for establishing the future CSC operations baseline (in particular with respects to the future Expansion Missions, associated with the necessary cost information to size the proposed approach and potential trade-offs).

The EOF implementation is based on service architecture with well-identified components that exchange data through Internet respecting defined interfaces. A service presents a simple interface to its consumer that abstracts away the underlying complexity. Combined with deployments on public cloud infrastructure, the service approach shall offer large adaptability to evolution of the operational scenarios in particular for what regards scalability.

Since 2019, the Copernicus ground segment operations have been transferred to cloud based environments (in anticipation of the enlargement of the Copernicus Sentinel missions and in response to the ever-increasing demand for Copernicus data) and the service-oriented approach for each component of the CSC ground segment operations has been strengthened to enhance competitiveness, prevent industrial and technical lock-in and introduce the necessary operational flexibility and transparency to allow the adaptation of the Copernicus Ground Segment to future challenges.

Within this context, this document outlines the Ground Segment Architecture as targeted and as implemented following the Ground Segment transformation phase during the period 2019-2022. This transformed architecture is referred as EOF Copernicus Space Component GS Architecture in this document or simply as EOF CSC GS Architecture.

This Architecture is a representation of the overall CSC Operations in terms of functional decomposition and operational data flows views. The high level architecture corresponds to the first layer of implementation and supports the implementation and the execution of all the activities related to the corresponding operational services and related procurements. The architecture reflects the fundamental principles supporting all aspects of the system, making it robust, scalable, efficient in cost and sustainable for the next 10 years without relying on unknown technological progress whilst being sufficiently flexible to benefit from any relevant future evolutions.

Introduction

Background

The Copernicus Space Component has been established as one of the largest and most proficient Earth Observation infrastructure in the world. With seven high-performance satellites put in orbit the system has evolved at a breath-taking pace.

The Copernicus operations have generated more than 70 PBytes of data. Central to the programme, the current CSC fuels the operational Copernicus Services but also provides a reliable and growing data stream for numerous new applications and services.

The current CSC has also triggered large application oriented technological developments, such as in the domain of Big Data management, as exemplified by the Data and Information Access Services (DIAS) and further evolved in Copernicus Data Space Ecosystem (CDSE).

The intense use and increased awareness for the potential of Copernicus have generated great expectations for an evolved Copernicus system.

User and observation requirements have been identified, structured and prioritized in a continuous reflection process led by the EC. Future missions have been discussed in the context of four observational capability families:

• Microwave Imaging Family,
• Optical Imaging Family,
• Topographic Measurement Family
• Spectroscopic Atmosphere Measurement Family.

Two sets of missions have emerged, that correspond respectively to the enlargement of the present measurements through the introduction of new missions and to a more progressive improvement of the current measurement capabilities, mostly by means of new generation of similar instrumentation compared to the ones currently deployed.

The CSC operations entrusted to ESA for Sentinel-1, -2, -3 A&B and Sentinel-5P deliver outstanding results, with more than 700 PB of delivered Sentinel data to registered users around the globe (2024/03).

Figure 1: Dissemination statistics (March 2024)

The ESA and EUMETSAT operations frameworks encompass all the activities necessary to successfully deliver the expected level of entrusted CSC operations and reflect the primary goal of securing the entry into operations of the Sentinels Missions, with their unprecedented data volume and user uptake.

The CSC operations encompass the Space Segment operations with the necessary post-launch spacecraft maintenance activities, the Flight Operations Segment services to ensure the spacecraft operations and the Ground Segment operations further addressed hereafter.

In particular, the tasks and functions of the current and future CSC Ground Segment operations include:

• Ground Segment Operations Management
  • Overall configuration management
  • End-to-end system & performance monitoring
  • Security monitoring and maintenance
• Space Segment and Flight operations service: spacecraft operations and post-launch spacecraft engineering support.
• Mission services
  • Mission planning service: instrument sensing & satellite downlinks operations planning & optimization.
  • Acquisition service: satellite data acquisition services (e.g. X-band, relay satellite).
  • Precise orbit determination service
• Data services
  • Systematic Production and routine quality control service
  • On-demand Production services
  • Reprocessing services
  • Long term data archive service: long-term data preservation and curation
  • Data access service: data distribution & user interface allowing access to any mission User Level Data and dedicated interfaces for streamlined access to selected datasets
  • Calibration & Validation Services: monitoring of instrument performance and data quality, calibration and validation, algorithms maintenance and external cal/val data provision.
  • Support to international agreement with data access services
  • Auxiliary data provision
  • Data traceability
  • User support: helpdesk, training, tools, communication
• Mission Management services
  • Mission specific Observation strategy
  • Satellite contingency management

The ESA Copernicus Ground Segment Operation Framework also includes the implementation and operations of the interfaces with services identified through specific agreements, deriving from the collaborative programme component or upon EC request. In particular these interfaces cover:

• Support to collaborative local acquisition stations
• Provision of a network of data access relays
• Coordination of the interfaces for data and information provision from other ground segments and Copernicus Services.

The operational experience gathered over the first years of operations has demonstrated that the sizing and performances are largely dependent on the user demand. As a result, a refined approach has been introduced for the management of the future CSC operations and reflected in the baseline presented in this document. While ensuring a robust and highly performing solution, introducing a wider flexibility wherever CSC operations may be impacted by the user scenario.

Scope

The ESA Operations Framework documentation for the CSC is made of the following set of documents:

• The Operations Concept Document;
• The Ground Segment target specifications defining the scope of the ESA Copernicus operations, its associated performance and the traceability to the EC requirements;
• The associated Ground Segment architecture describing the main functional blocks and their interfaces in particular for what concern the external interfaces;
• The system technical budget, providing an estimate of the expected operations load and generally operations context sizing;
• The risk folder;
• The cost models establishing the key parameters driving the operation cost;
• The high level roadmap, establishing the objectives for the future evolution.

The EOF documentation seeks to:

• Support the continuous evolution process for the CSC operations entrusted to ESA;
• Devise novel schemes for Ground segment operation management and industry involvement;
• Serve as an information vehicle to EU and Copernicus participating countries for iterative consolidation.
• Document the ESA – EUMETSAT coordinated technical baseline.
• Prepare the CSC GS Operations to manage the upcoming Expansion Missions and to ensure the operations capacity and performance (e.g. linked to increasing volume of data)

It is noted that the data warehouse in charge of the provision of third party mission data on behalf of Copernicus Services is not subject to this document even if it may benefit from the projects and services developed as part of the Copernicus Ground Segment.

Purpose of this document

The purpose of this document is to maintain an up to date description of the Architecture supporting the services and functions of the current and future CSC Ground Segment operations.

Document context

The present document is part of the CSC operations baseline issued by ESA in the overall context of the Copernicus Programme documentation baseline as illustrated in the figure below:

Figure 2: Copernicus Documentation Tree

Applicable and Reference Documents

Applicable Documents

1. ESA EO Operations Framework – CSC – Ground Segment Specifications [ESA-EOPG-EOPGC-RS-1]

Reference Documents

1. ESA EO Operations Framework – CSC – High Level Roadmap [ESA-EOPG-EOPGC-PL-6]
2. ESA EO Operations Framework – CSC – System Technical Budget [ESA-EOPG-EOPGC-TN-09]
3. ESA EO Operations Framework – CSC – Copernicus Missions Observation Strategy Implementation [ESA-EOPG-EOPGC-TN-08]
4. ESA EO Operations Framework – CSC – Master ICD [ESA-EOPG-EOPGC-IF-06]
5. Concept for the Cooperation of ESA and EUMETSAT regarding Copernicus Evolutions co-signed by J. Aschbacher (ESA) & A. Ratier (EUMETSAT), 2017
6. ESA EO Operations Framework (EOF) – CSC – Copernicus Sentinel Data Portfolio [ESA-EOPG-EOPGC-TN-12]
7. CSC Operations – ESA Framework – Data Quality Functional Baseline [ESA-EOPG-EOPGM-TN-01]
8. ESA EO Operations Framework (EOF) – CSC – Glossary [ESA-EOPG-EOPGC-TN-13]
9. ESA EO Operations Framework (EOF) – CSC – Operations Concept Document [ESA-EOPG-EOPGC-TN-19]
10. Copernicus Space Component Sentinel User Level Data Portfolio [ESA-EOPG-EOPGC-TN-20]
11. ESA EO Operations Framework (EOF) – CSC – Data Access Services Portfolio [ESA-EOPG-EOPGC-TN-57]
12. ESA EO Operations Framework (EOF) – CSC – Ground Segment Operational Configuration [ESA-EOPG-EOPGC-TN-62]

Acronyms and Definitions

AIP Archive Interface (delivery) Point

API Application Programming Interface

AUX Auxiliary

AUXIP Auxiliary Interface (delivery) Point

CADU Channel Access Data Unit

CADIP CADU Interface (delivery) Point (replacing X-Band Interface Point XBIP)

Cal/Val Calibration/Validation

CAMS Copernicus Atmosphere Monitoring Service

CFI Customer Furnished Item

CFDP CCSDS File Delivery Protocol

CHIME Copernicus Hyperspectral Imaging Mission for the Environment

CMEMS Copernicus Marine Environment Monitoring Service

CNES Centre National D’Études Spatiales

COM European Commission

CRISTAL Copernicus Polar Ice and Snow Topography Altimeter

CSC Copernicus Space Component

CVIP Calibration & Validation Interface (delivery) Point

DD Data Distribution

DFEP Demodulator and Front End Processor

DIAS Data and Information Access Services

DP Documentation Package

E2E End-to-End

EC European Commission

ECMWF European Centre for Medium-Range Weather Forecasts

EDRS European Data Relay Satellite System

EDRSGEO European Data Relay Satellite System Geostationary Earth Orbit

EIP EDRS Interface (delivery) Point

EO Earth Observation

EOF ESA Copernicus Ground Segment Operations Framework

EOP Earth Observation Programme

ESA European Space Agency

ESOC European Space Operations Centre

EU European Union

EUM EUMETSAT

FOCC Flight Operations Control Centre

FOS Flight Operations Service

FTP File Transfer Protocol

GNSS Global Navigation Satellite System

GPS Global Positioning System

GS Ground Segment

HKTM Housekeeping Telemetry

ICD Interface Control Document

IF Interface

ILRS International Laser Range Satellite

IP Interface (delivery) Point

IPF Instrument Processing Facility

KMF Key Management Facility

LEOP Launch and Early Orbit Phase

LGS Local Ground Station

LSTM Land Surface Temperature Monitoring

LTA Long Term Archive

LTS Long Term Scenario

MCS Mission Control System

MFF Mulitannual Financial Framework

MKMF Master Key Management Facility

MOC Mission Control Operations

MP Mission Planning

MPIP Mission Planning Interface (delivery) Point

MW Microwave

NG Next Generation

OBSM On-Board Software Maintenance

ODP On-Demand Processing

OM Operations and performance Monitoring

OTR Operations Traceability

PA Product Assurance

POD Precise Orbit Determination

PODIP Precise Orbit Determination Interface (delivery) Point

PR Processing

PRIP Production Interface (delivery) Point

QA Quality Assurance

QC Quality Control

RS Reference System

RSIP Reference System Interface (delivery) Point

S-1/2/3/5P Sentinel-1/2/3/5 Precursor

SAR Synthetic Aperture Radar

SAT Satellite

SLA Service Level Agreement

SRA Sentinels Resource Allocation

SSRS System-specific Security Requirement Statement

TC Telecommand

TLE Two Line Element

TM Telemetry

TT&C Telemetry, Tracking and Command

VA Vulnerability Assessment

WAN Wide Area Network

XB X-Band Station

Document organisation

Section 1 is an executive summary of the ESA EO Operations Framework (EOF) as applied to Copernicus Space Component Ground Segment.

Section 2 provides the introduction and the scope, the Applicable Documents and organisation of the Ground Segment Architecture document

Section 3 introduces the highest level decomposition of the CSC Ground Segment Architecture

Section 4 provides the ground segment architecture principles and detailed description of all GS elements.

Annex A provides a traceability to the Ground Segment Specifications [AD-1]

All relevant assumptions for the Architecture are set out in the Ground Segment Specifications [AD-1] and System Budget [RD-2]

ESA EO Framework – CSC – Ground Segment CONTEXT

The following figure illustrates the highest-level context of the CSC Operations relevant to the ESA Framework with a synoptic view of the key internal and external interfaces:

Figure 3 High level context of the CSC Operations.

An overview of each element and main interface is provided in the following sections and a further description is provided for the Ground Segment in the following chapters.

The Copernicus Space Segment ensures the instrument observations and their transfer to ground. It is composed by the dedicated Copernicus satellites, including the current Sentinel satellites and the future Expansion and Next Generation satellites. It encompasses as well the necessary post launch satellite maintenance activities and is complemented by the specific satellite commissioning facilities.

The Flight Operations Segment ensures the Copernicus spacecrafts operations, monitoring and control, including in particular:

• • Mission Control System operations (including mission scheduling)
  • Telemetry, telecommand and tracking operations within the S-band frequency
  • Satellite orbit maintenance, debris monitoring.

The Ground Segment ensures the overall handling of the data acquired by the instruments on board of the of the Copernicus satellites, including in particular:

• Implementing the mission observation scenario in line with the satellite, ground resources and constraints;
• Acquiring the Sentinel data, processing, archiving and making resulting User Level Data available;
• Guaranteeing that operational User Level Data meet the expected quality;
• Monitoring and reporting on the end-to-end operations performance.

The web site https://sentinels.copernicus.eu provides reference information on the various Copernicus missions and most relevant application domains.

EOF High Level Architecture

The EOF High Level Ground Segment Architecture is a representation of the overall CSC Ground Segment Operations in terms of functional and operational data flows views. The high level architecture corresponds to the first layer of implementation and supports the implementation and the execution of all the activities related to the corresponding operational services and related procurements.

The architecture reflects the fundamental principles supporting all aspects of the system, making it robust, scalable, efficient in cost and sustainable for the next 10 years without relying on unknown technological progress whilst being sufficiently flexible to benefit from any relevant future evolutions.

EOF Architecture Principles

The early Copernicus operation phase, has demonstrated that the user scenario is so unpredictable that any architecture relying on a rigid description of such scenario is subject to a major risk. The new EOF GS architecture responds to the need to deliver reliable Copernicus Sentinel operations and prepare for the future Expansion and Next Generation operations, within a context foreseeing a major capacity increase and subject to unpredictable and potentially major user scenarios evolutions. In this respect, the key principles followed to establish the target architecture are highlighted hereafter.

• The operations are based on an architecture of operational services for which no dedicated infrastructure is to be procured or provided by ESA;
• Operational services are defined wherever possible based on the input and output interfaces, associated services scope and performance.
  • The definition of the different operational services is mission independent.
• Operational services map whenever possible standard functions directly procured to industry.
• Interfaces between operational services are streamlined and standardized, minimizing the Sentinel specific interfaces. Provider or consumer of an interface may be changed transparently.
• Operational services and associated interfaces are to the maximum extent operated in parallel in two distinct implementations or configurations, with a pre-established sharing of the operational load to maintain the services operations autonomy. This parallelism is a technical and performance risk mitigation.
• Operational services provide an interface to enable continuous monitoring of the service operations performance and to contribute to the end-to-end operations monitoring.
• The operational services should favour the use of open source and cloud-based environments for the execution of the service operations.
• All acquired Sentinel data is systematically processed to a predefined set of User Level Data with free and open on-line access to users.
• All mission data is available free and open to all users, combining immediate on-line access to data and deferred on-line access, including the possibility for on-the-fly processing (configurable per mission, User Level Data type)
• Optimise the Long Term Archiving data volumes, reducing the archive volume and enhancing data curation.
• Allow adaptation of operational resources to the user demand and to the available financial resources, maintaining a direct and transparent relationship with associated operations performance.

EOF High Level Architecture

The following sections give a global and end-to-end vision of the EOF GS architecture through a description of its main architectural components and their boundaries versus external elements. It provides a high level description of foreseen interfaces.

To meet its objectives, the CSC EOF architecture includes the following main functionalities:

• Data Acquisition
• Mission Planning, including the definition of the instrument sensing and downlink plan
• Systematic and on-demand production
• Essential Copernicus data preservation
• Data Access
• Gathering of auxiliary data from calibration and validation activities and other auxiliary data sources necessary to generate data meeting the quality specifications
• Precise orbit determination
• Data calibration and validation
• Routine data quality control
• Operations performance monitoring

Some of these functions are common to all Sentinel missions while others are specifically implemented and operated for each Sentinel mission.

The functions composing the Ground Segment architecture are implemented in the form of operational services, complying to a set of applicable input and output interfaces and to the corresponding operational performance requirements.

Considering the principles enounced above, the Ground Segment architecture includes multiple operational instances for most of the architectural elements. Each operational instance for the same architectural element complies with the same interfaces, functional and performance requirements and typically supports a percentage of the overall operations capacity associated to the architectural element.

For the purpose of architecture and design definition, the interfaces between functions are classified as:

• Data flow interfaces, for interfaces carrying Sentinel data or auxiliary information required for processing activities
• Monitoring and control interfaces, for interfaces carrying reporting information (necessary for the performance monitoring at element level or end to end) or operations control information
• Scheduling interfaces, for interfaces related to mission planning, including stations planning, satellite scheduling, etc.

Data flow interfaces, for systematic data transfer between services, are based on the concept of small data cache areas, referred to as data “interface delivery points” (IP) hereafter. Each function or service generating a systematic or routine data flow to be further managed by one or more services, is making the output data available in an interface delivery point located on a cloud-based environment, which is logically considered as part of, and under the responsibility of, the data source service.

The high level architectural decomposition is provided in Figure 4, including whenever necessary multiple instances of the same service to follow the principles indicated earlier. The number of instances illustrated for each service in the figure is not intended to represent a precise number of instances but the fact that multiple instances are present for the same service or function.

The CSC operations encompass different functionalities to be operated in full coherence among them, with the needs of the various Sentinel missions and towards the CSC users for delivering the expected outcome. The necessary overarching management needs are captured by the Ground Segment Management function corresponding to the ESA management of the CSC Ground Segment.

Figure 4 High level decomposition of the ESA Copernicus Ground Segment Architecture.

It is highlighted that:

• The interface delivery points are located in a cloud-based environment and hold a short-term data rolling storage, allowing the consumer services to systematically retrieve the data for the next step.
• The data transfer between services is nominally performed through internet or locally within the same cloud environment.
• The cost of the delivery point is managed by the provider, and sized for the number of clients authorized by ESA to retrieve the data (access control to prevent abuse of the service beyond foreseen sizing resides on the provider)
• It will be possible to establish point to point network connectivity by the consumer in order to benefit from higher performance throughput/availability over internet in case deemed necessary for achieving the committed service provision quality, for which the cost is born by the consumer.

The various architectural components are addressed in the next chapter.

EOF Ground Segment Architecture components

The elements composing the CSC ESA Ground Segment Architecture are addressed in the following sections as follows:

• Mission planning
• Data Acquisition
• Production and routine quality control
• Auxiliary data gathering, from calibration and validation activities or other data sources
• Data Preservation (Long Term data Archiving)
• Data Access
• Reference System
• Operations traceability
• E2E Operations performance monitoring
• Services Operations coordination
• Benchmarking Service
• Security monitoring

An overview of the Flight Operations Services is also included for completeness.

Mission Planning

The mission planning function is fulfilled by the Mission Planning Service(s) in the form of operational service provision making use of mission-specific systems maintained by ESA and provided as CFI to the service operator.

Pre-defined observation plans are implemented for all Sentinel missions in order to guarantee continuous data flow to the operational services

The Sentinel Mission Observation Scenarios are described in the Copernicus Sentinel Observation Strategy Implementation [RD-3] and undergo regular revisions for adaptation to the user needs. The Implementation defines a stable observation plan for the benefit of the operational users according to a priority scheme.

Functional scope

The mission planning function implements the observation scenario of each Sentinel mission, in line with the observation strategy implementation and with the available satellite and ground resources and constraints.

The mission planning function includes a transversal planning system for the managing and de-conflicting of downlink conflicts between the various Sentinel satellite units, the Sentinels Resource Allocation system (SRA).

Missions Scope & Operations highlights

The mission planning system is specific for each mission while the element managing the de-conflicting between the flying Copernicus missions is common to all missions.

The ESA CSC Ground Segment includes the following Mission Planning elements:

• One Sentinel-1 mission planning system, in charge of implementing the Sentinel-1 observation scenario, in line with the spacecraft operational constraints and available downlink resources.
• One Sentinel-2 mission planning system, in charge of implementing the Sentinel-2 observation scenario, in line with the spacecraft operational constraints and available downlink resources.
• One Sentinel-5P acquisition planning system, interfacing the Spacecraft Operations and Control for implementing the Sentinel-5P observation scenario, in line with the spacecraft operational constraints and available downlink resources
• One Sentinels resource allocation system (SRA), resolving conflicts between Sentinels to prevent downlink interferences and ensure coordinated usage of downlink resources, interfacing with each Sentinel Mission planning system accordingly.
• One Mission planning interface point (MPIP), also referred as Exchange Data Server, acting as a temporal repository of scheduling information generated by the various mission planning systems accessible to the CGSs for retrieving the associated acquisition schedule.

The Mission Planning operations are ensured by the Mission Planning Service(s), operating one or more mission-specific planning systems. The present operational configuration of Mission Planning Services and operated missions is documented in [RD-12].

The Sentinel-3 mission planning system is maintained and operated by EUMETSAT and thus not considered part of the ESA Copernicus Ground Segment. The MPIP is supporting EUMETSAT operations whenever required.

Interfaces & Context

The Mission Planning system for each Sentinel mission interfaces with

• the FOS to provide the instrument and downlink schedule for uplink to the satellite and receive satellite orbit information and other necessary information (resource consumption etc.) for further planning processes.
• the SRA, to receive EDRS link schedule, orbit information and possibly banned stations passes for a given given orbit as a consequence of a downlink conflict resolution.
• each Acquisition Service, through the MPIP, to provide schedule information necessary for acquiring the allocated passes
• the E2E Operations Performance Monitoring System, in order to provide the MP information required for the E2E operations monitoring (e.g. planned instrument acquisitions, planned downlinks, planned emergency acquisitions, etc.).
• The Services Operations coordination service, typically for anomaly management.

The context of the MP operations and main interfaces is illustrated in Figure 5. Detailed interface information is provided in the [RD-4].

Multiple Acquisition Services and associated Ground Stations will contribute to the Copernicus Missions operations. The number of stations shown in the figure is purely illustrative. The operational configuration is documented in Error! Reference source not found..

The Mission Planning may interface as well with potential Collaborative Local Ground Stations through the same MPIP.

Figure 5 Mission Planning element context and main interfaces.

Performance

Mission Planning performances are Sentinel specific, the timelines required for provision of planning information to Acquisition Stations and FOS being set out in operational procedures.

Infrastructure and CFIs

The service operations make use of the MP software, maintained and provided by ESA as a CFI to the corresponding MP service operator. The service includes all necessary infrastructure to operate the relevant mission planning system(s), with no dedicated Copernicus infrastructure deployment.

The service includes also a delivery point, the MPIP, deployed and operated by the MP service provider on a public cloud infrastructure, accessible to a set of consumers authorised by ESA.

Data Acquisition

The data acquisition function is fulfilled by Acquisition Service in the form of operational service provision.

The CSC Acquisition service is tailored to the technical baseline of each CSC satellite family

and includes in particular:

• X-Band Acquisition service, in line with the Sentinels satellite downlink operations technical baseline
• EDRS Acquisition service, in line with the Sentinel-1 and Sentinel-2 satellites downlink operations technical baseline
• Ka-Band Acquisition service, foreseen to fulfil the needs of some Expansion missions

Multiple Ground Stations may be necessary to jointly fulfil the acquisition needs of a given satellite unit and in any case, multiple Ground Stations are necessary to fulfil the overall acquisition needs. The same Ground Station may serve one single or several Sentinel missions, depending on its acquisition capacity and the various mission needs. Each Acquisition Ground Station contributing to the CSC acquisition operates autonomously with respect to the others, the allocation of downlink operations for each Ground Station being performed by the Mission Planning system of each Sentinel after being de-conflicted by the Sentinels Resource Allocation (SRA) system.

X-Band Acquisition

Functional Scope

The scope of the X-Band Acquisition function is defined with the aim of minimising the need for Copernicus specificities (e.g. hosting, e.g. processing to Level-0 data). Accordingly, the functional scope is focussed on the Copernicus missions data acquisition, data down-conversion, demodulation, satellite CADU data stream generation and delivery.

The X-Band Acquisition service covers the following functions:

• Regular retrieval of station scheduling information from the Mission Planning Interface Point (MPIP), as made available by the mission planning element.
• X-Band acquisition of Sentinel satellites payload and HKTM data based on associated acquisition schedule, including down-converter and demodulation (HKTM is downloaded in every orbit in X-Band and once a day in S-Band).
• Generation of Sentinel CADU data stream for the scheduled passes.
• Transfer of generated Sentinel CADU stream to the X-Band Interface Point (CADIP) within required timeliness
• Maintenance of the CADIP, ensuring its availability for consumers and handling of the CADU rolling policy
• Pass performances monitoring and reporting

Missions Scope & Operations highlights

The X-Band Acquisition is a common element for the Copernicus missions: The functional scope of the X-Band Acquisition is defined in order to be generic and fulfil the acquisition needs for all flying Copernicus missions requiring the X-Band downlink operations.

Multiple Acquisition Ground Stations are necessary in the GS architecture to jointly satisfy the downlink needs of all flying missions. Each Acquisition Ground Station operates autonomously with respect to the others, the allocation of downlink operations for each Ground Station being performed by the Mission Planning system of each Sentinel after being de-conflicted by the Sentinels Resource Allocation (SRA) system.

The same Ground Station may serve one single or several Sentinel missions, depending on its acquisition capacity and the various mission needs.

Interfaces and Context

Each Acquisition Ground Stations interfaces with:

• Sentinel satellites for X-Band payload and HKTM data reception
• the Mission Planning Systems, through the MPIP, to retrieve station planning information and orbit files necessary for performing the data acquisition
• The E2E Operations Performance Monitoring System, providing reporting information on the acquisition operations to support the E2E monitoring
• The production systems for each Sentinel through the station CADIP
• The Services Operations coordination service, typically for anomaly management

The context of the Acquisition Ground Stations operations and main interfaces is illustrated in Figure 6. The figure shows the context and interfaces for one Acquisition Ground Station, which applies equally to each of the Ground Stations contributing to the Sentinel operations. Detailed interface information is provided in the [RD-4].

The figure indicates the scope of each Acquisition Service, including the operations and management of the CADIP in a cloud-based environment, as well as the exchanges with the MPIP in line with the associated interfaces and performance.

The output of the Acquisition function corresponds to the acquired satellite data stream (in the form of CADU data as defined in [RD-4]) for each downlink pass and for each operated satellite unit. The acquired satellite data stream is transferred, as part of the Acquisition function, to the X-Band Interface Delivery Point (CADIP). There is one CADIP per each Ground Station, where data for each pass and operated satellite is made available for retrieval by the production services.

Each CADIP is operated in a public cloud environment as part of the corresponding Acquisition Ground Station service.

A number of production services access each of the CADIP for retrieving the satellite data and performing the subsequent data processing activities.

N.B. In this context, EUMETSAT acts as a production service, requiring access to the acquired Sentinel-3 satellite data stream. An ad-hoc interface, backwards compatible with the legacy interface for receiving the Sentinel-3 data, is maintained with EUMETSAT.

Figure 6 Acquisition Ground Stations context and main interfaces. N.B. Ad-hoc interfaces with EUMETSAT are not illustrated.

Operations

X-Band Acquisition services are managed autonomously by service providers with the Ground Stations responding to the service management requirements and service level agreements established by ESA. Acquisition services are assigned to Ground Stations according to standard ESA procurement practices, qualified services are managed through a frame contract which allows the service providers to perform interface testing and bid for the downlink capacities required and specified by ESA. In any switch-in switch-out of Ground Stations a brief phase-in period of a few weeks is nominally foreseen to ensure against interruptions in the overall acquisition service. Service switch-over is closely coordinated and overseen by the Service operations coordination.

The acquisition service encompasses all activities from the scheduling of the on-ground data acquisition facilities to the delivery of the received satellite data stream. The service scope requires delivery of acquisition operations through streamlined interfaces. The necessary infrastructure and resources are considered as being inherently part of the service.

The X-band operations are based on the main following principles:

• The X-band Ground Station receives from the Mission Planning, in line with the specification set by ESA, all necessary elements (including orbital information and start & stop times) to perform the acquisition service.
• The X-band Ground Station tracks the satellite and receives the downlinked data between start & stop times.
• The X-band Ground Station delivers the received data in CADU format to a data delivery point deployed on a public Cloud. (Responsibility for the necessary network connection and set up of the delivery point is fully with X-band service provider)
• The X-band Ground Station delivers the Station Acquisition Report.
• The authorised Copernicus Ground Segment service providers access the data delivery point and pick up the CADU data for further processing. The CADU data are available for pick-up for at least one week (i.e. the data delivery point features a one week rolling archive).

Performance

The Acquisition operations performance is monitored through a Service Level Agreement based on a number of performance indicators, fully harmonised for all Acquisition Stations, including:

• Tracking Quality: A successfully tracked pass is a pass in which the RF signal is in “lock” status for the whole duration of the pass, i.e. between the start and stop of data download set by ESA.
• Acquisition Quality: A successful acquisition is an acquisition in which the ratio between the sum of the RS uncorrectable frames over all frames is lower than 10^-6 for the duration of the acquisition service (between the start and stop of data download set by ESA) not affected by tracking anomalies.
• Acquisition Reliability: the number of consecutive failed passes per mission, whereby a failed pass one in which less than 70% of the time between the start and stop of data download set by ESA has been successfully tracked and acquired.
• Data Delivery Timeliness: the time interval between the data start signal and the delivery of the last CADU file at the data delivery point
• Data Completeness: the ratio between the number of files made available at the delivery point and the expected number of files

The Acquisition Service sizing for a given ground station is established by the service provider based in particular:

• on the overall number of downlink passes to be acquired on a monthly basis, including all satellite units operated at the station, assuming that all satellite overpasses are planned.
• Volume of acquired satellite data to be made available on the CADIP;
• Timeliness requirements from acquisition to data availability on CADIP;
• Distributed volume of satellite data to the overall consumers.

The variation of the sizing parameters may impact on the service performance and cost. The relationship between the operations sizing, performance and cost is captured through a parametric model provided by the Service operator as part of the competitive service operations procurement process.

Infrastructure and CFIs

The service includes all necessary infrastructure to operate the function, with no dedicated Copernicus infrastructure deployment and no hosting of elements required to operate other functions (e.g. no hosting of network or production infrastructure).

No ESA CFIs are included in this function (e.g. no DFEP-like CFI, no Copernicus WAN connection).

The service includes a delivery point on a public cloud infrastructure, accessible to a set of consumers authorised by ESA..

EDRS Acquisition

Functional Scope

The EDRS service covers the following functions:

• Regular definition of the EDRS Links available for the data downlink planning.
• Optical Sentinel-EDRSGEO Link operations and Ka-Band Sentinel payload and HKTM data acquisition based on associated link schedule, including down-converter and demodulation
• Generation of Sentinel CADU data stream for the scheduled passes
• Transfer of generated Sentinel CADU stream to the EDRS Interface Point (EIP)
• Maintenance of the EIP, ensuring its availability for consumers and handling of the CADU rolling policy
• Pass performances monitoring and reporting

Missions Scope & Operations highlights

The EDRS Acquisition is a Sentinel-1 and Sentinel-2 element.

EDRS downlink visibilities are settled by the SRA in interface with the EDRS Mission Control Operations (MOC) and used by each Sentinel mission planning as part of the available downlink resources, together with the X-Band available passes.

Interfaces and Context

The EDRS Service interfaces with:

• The Sentinels Resource Allocation (SRA) system, exchanging the EDRS/Sentinels Link Schedule
• The production systems for each Sentinel through the EDRS Interface Point (EIP) (N.B. For EDRS, the EIPs are currently located within each production service and managed by the production services)
• The E2E Operations Performance Monitoring System providing reporting information on the acquisition operations to support the E2E monitoring
• The Services Operations coordination service, typically for anomaly management

The output of the EDRS function corresponds to the acquired satellite data stream (in the form of CADU data as defined in [RD-4]) for each downlink pass and for each operated satellite unit. The acquired satellite data stream is transferred, as part of the EDRS function, to the EDRS Interface Delivery Point (EIP).

N.B. There is currently one EIP per each Production system, where data for each pass and operated satellite is made available.

Ka-Band Acquisition

The observation requirements of the Copernicus missions currently in operations (i.e. Copernicus Sentinel-1,2,3,5P,6) are compatible with a downlink of payload data in X-Band. In contrast, the increased volume of data generated by some of the Copernicus Expansion (COP EX) missions requires Ka-Band to fulfil the demanding downlink capacity.

The COP EX missions will also implement the CFDP protocol for the transmission of the payload data between the satellite and the ground.

Functional Scope

The scope of the Ka-Band Acquisition function is identical to the X-Band Acquisition function and it covers the following functions:

• Regular retrieval of station scheduling information from the Mission Planning Interface Point (MPIP), as made available by the mission planning element.
• Ka-Band acquisition of Sentinel satellites payload and HKTM data based on associated acquisition schedule, including down-converter and demodulation (HKTM is downloaded in every orbit in Ka-Band).
• Generation of Sentinel CADU data stream for the scheduled passes.
• Transfer of generated Sentinel CADU stream to the Ka-Band Interface Point (CADIP) within required timeliness
• Maintenance of the CADIP, ensuring its availability for consumers and handling of the CADU rolling policy
• Pass performances monitoring and reporting

Missions Scope & Operations highlights

The Ka-Band Acquisition is a common element for the Copernicus missions: The functional scope of the Ka-Band Acquisition is defined in order to be generic and fulfil the acquisition needs for all flying Copernicus missions requiring the Ka-Band downlink operations.

Multiple Acquisition Ground Stations are necessary in the GS architecture to jointly satisfy the downlink needs of all flying missions. Each Acquisition Ground Station operates autonomously with respect to the others, the allocation of downlink operations for each Ground Station being performed by the Mission Planning system of each Sentinel after being de-conflicted by the Sentinels Resource Allocation (SRA) system.

The same Ground Station may serve one single or several Sentinel missions, depending on its acquisition capacity and the various mission needs.

Interfaces and Context

Each Acquisition Ground Stations interfaces with:

• Sentinel satellites for Ka-Band payload and HKTM data reception
• the Mission Planning Systems, through the MPIP, to retrieve station planning information and orbit files necessary for performing the data acquisition
• The E2E Operations Performance Monitoring System, providing reporting information on the acquisition operations to support the E2E monitoring
• The production systems for each Sentinel through the station CADIP
• The Services Operations coordination service, typically for anomaly management

The context of the Ka-Band Acquisition Ground Stations operations and main interfaces is identical to context of the X-Band Acquisition Ground Station illustrated in Figure 6.

The output of the Ka-Band Acquisition function corresponds to the acquired satellite data stream (in the form of CADU data as defined in [RD-4]) for each downlink pass and for each operated satellite unit. The acquired satellite data stream is transferred, as part of the Acquisition function, to the Ka-Band Interface Delivery Point (CADIP). There is one CADIP per each Ground Station, where data for each pass and operated satellite is made available for retrieval by the production services.

Each CADIP is operated in a public cloud environment as part of the corresponding Acquisition Ground Station service.

A number of production services access each of the CADIP for retrieving the satellite data and performing the subsequent data processing activities.

Operations

Ka-Band Acquisition services are managed in the same way than X-Band Acquisition services.

In any switch-in switch-out of Ground Stations a brief phase-in period of a few weeks is nominally foreseen to ensure against interruptions in the overall acquisition service. Service switch-over is closely coordinated and overseen by the Service operations coordination.

The acquisition service encompasses all activities from the scheduling of the on-ground data acquisition facilities to the delivery of the received satellite data stream. The service scope requires delivery of acquisition operations through streamlined interfaces. The necessary infrastructure and resources are considered as being inherently part of the service.

The Ka-band operations are based on the main following principles:

• The Ka-band Ground Station receives from the Mission Planning, in line with the specification set by ESA, all necessary elements (including orbital information and start & stop times) to perform the acquisition service.
• The Ka-band Ground Station tracks the satellite and receives the downlinked data between start & stop times.
• The Ka-band Ground Station delivers the received data in CADU format to a data delivery point deployed on a public Cloud. (Responsibility for the necessary network connection and set up of the delivery point is fully with X-band service provider)
• The authorised Copernicus Ground Segment service providers access the data delivery point and pick up the CADU data for further processing. The CADU data are available for pick-up for at least one week (i.e. the data delivery point features a one week rolling archive).
• The CFDP protocol is not managed by the Ka-Band station, the file reconstruction is performed by the processing facilities.

Performance

The Ka-Band Acquisition operations performance is monitored in the same way than the X-Band Acquisition service through a Service Level Agreement based on a number of performance indicators, fully harmonised for all Acquisition Stations, including:

• Tracking Quality: A successfully tracked pass is a pass in which the RF signal is in “lock” status for the whole duration of the pass, i.e. between the start and stop of data download set by ESA.
• Acquisition Quality: A successful acquisition is an acquisition in which the ratio between the sum of the RS uncorrectable frames over all frames is lower than 10^-6 for the duration of the acquisition service (between the start and stop of data download set by ESA) not affected by tracking anomalies.
• Acquisition Reliability: the number of consecutive failed passes per mission, whereby a failed pass one in which less than 70% of the time between the start and stop of data download set by ESA has been successfully tracked and acquired.
• Data Delivery Timeliness: the time interval between the data start signal and the delivery of the last CADU file at the data delivery point
• Data Completeness: the ratio between the number of files made available at the delivery point and the expected number of files

The Acquisition Service sizing for a given ground station is established by the service provider based in particular:

• on the overall number of downlink passes to be acquired on a monthly basis, including all satellite units operated at the station, assuming that all satellite overpasses are planned.
• Volume of acquired satellite data to be made available on the CADIP;
• Timeliness requirements from acquisition to data availability on CADIP;
• Distributed volume of satellite data to the overall consumers.

The variation of the sizing parameters may impact on the service performance and cost. The relationship between the operations sizing, performance and cost is captured through a parametric model provided by the Service operator as part of the competitive service operations procurement process.

Infrastructure and CFIs

The service includes all necessary infrastructure to operate the function, with no dedicated Copernicus infrastructure deployment and no hosting of elements required to operate other functions (e.g. no hosting of network or production infrastructure).

No ESA CFIs are included in this function (e.g. no DFEP-like CFI, no Copernicus WAN connection).

The service includes a delivery point on a public cloud infrastructure, accessible to a set of consumers authorised by ESA.

Mission Data Acquisition Service (MDAS) for CO2M

The CO2M Mission is the first COP EX missions to be launched by end of 2025.

The CO2M Mission Data is downlinked in the Ka-band.

In accordance with the “CSC Operations – ESA Framework – Operations Concept”, the CO2M Ka-Band mission data acquisition will be ensured by the CSC Acquisition Service, that will be aligned with the Ka-Band Acquisition concept described in chapter 4.3.2.3.

The CO2M Acquisition service will be procured in line with the CO2M satellite(s) downlink specifications and link budget. It will be operated on the basis of the station acquisition planning information to be received regularly from the Flight Operations.

The MDAS Service is to operate the CFDP protocol for the transmission of the payload data between the satellite and the ground in Class 1.

The MDAS Service will be provided by a subset of the Acquisition Stations (AcqS) network procured by ESA as part of the CSC GS Acquisition Service.

Production and routine quality control

The CSC Production operations encompass all activities necessary to process any acquired Sentinel satellite data, generating user level data, engineering data or HKTM, meeting the quality specifications and making it available for archiving and for further user access.

The production operations cover the following operational functions, addressed in more detail in the following sections:

• Systematic Production, gathering the data stream acquired by the Sentinel satellites and made available at pick-up points by the Acquisition services; processing it into a set of pre-defined set of data meeting the expected specifications; and deliver them systematically and fully to the production delivery point within a given timeliness.
• On-demand Production, allowing the generation of higher level of data (typically Level-1/2) from archived lower level of data (typically Level-0) available in the Long Term Archive (LTA). On-demand product is nominally triggered as a result of a user request through the Data Access function for a user level data not available for user access at the time of the request..
• Reprocessing, aiming at the bulk (re-)generation of past User Level Data (typically Level-1/2) from archived lower level data (typically Level-0).

The list of User Level Data that may be generated by the CSC operations is provided in [RD-6].

Production operations rely on the use of data processors developed and maintained by ESA, and made available as CFIs to the services involved in production operations (the precise list and scope of the data processors CFIs remains mission dependent):

• Data processors, allowing transforming input satellite data into output User Level Data according to validated algorithms (typically Level-0, Level-1 and Level-2 data processors)
• Routine Quality Control elements, allowing the automated routine quality control of all generated User Level Data as part of the systematic production data flow.

CFIs are provided to the Production service operators via the Reference System as outlined in section 4.3.10, providing access to the software, documentation and configuration files.

The Routine Quality Control performed as part of the production operations allows screening all generated User Level Data for detecting production related issues or data quality related issues that are identifiable in a fully automated manner.

The Routine Quality Control complements the expert quality control performed as part of the calibration and validation activities, and allows the fast identification of production and/or quality issues whenever their nature allows an automated detection.

Systematic Production

In line with the Copernicus missions operations concept, all data acquired by the Copernicus missions satellites is systematically processed to a set of pre-defined User Level Data types and made available to users within a given timeliness.

The systematic production is the element in charge of generating a pre-established set of User Level Data for each Sentinel satellite according to the associated User Level Data and timeliness baseline, starting from the satellite CADU stream acquired at the Acquisition Ground Stations contributing to the satellite operations, and performing an automated routine quality control of the resulting User Level Data.

The Systematic Production operations are fulfilled by the Production Service in the form of operational service provision. Multiple Production Service contribute jointly to the overall CSC systematic production needs.

Functional Scope

The systematic production and routine quality control operations cover the following main functions:

• Retrieval of CADU data stream from the CADIP, as delivered by each Acquisition service and of CADU data steam from the EDRS Service (whenever relevant)
• Retrieval of auxiliary data from AUX data gathering
• Systematic production from the CADU stream of the Level-0/1/2 User Level Data in line with the Sentinel mission specificities (including CADU assembly as necessary, generation of Level-0 data, generation of the Level-1 data, generation of Level-2 data whenever relevant)
• Operations and management of the Production Interface Point (PRIP), including a short rolling storage (few days) to support retrieval from the various consumers
• Provision of the systematically generated data to the PRIP within the required timeliness and performance
• Automated routine quality control of the generated User Level Data
• Production Performance Monitoring and Reporting

Missions Scope & Operations highlights

The systematic production is mission specific, since the data processors and output data remains mission and instrument specific. Nevertheless, the associated data flow and operational management is not considered Sentinel specific.

The outputs of the systematic production operations are made available by the Production Service on the Production Interface Delivery Point (PRIP) and encompass:

• user level data; typically, Level-0 Level-1 and Level-2
• engineering data, intended for instrument performance monitoring and calibration activities;
• satellite housekeeping data, intended for spacecraft operations support; typically, ancillary and housekeeping telemetry data

Following the principle of redundancy, at least 2 services for each Sentinel mission production are foreseen as part of the Ground Segment architecture, jointly contributing to the overall mission data production without duplication (e.g. sharing implemented based on Sentinel satellite unit). The production services for each Sentinel mission comply to the same interfaces, performance and expected service outputs in order to ensure full transparency from user perspective and to be fully interchangeable.

Each Sentinel production service operates autonomously with respect to other potential services contributing to the production operations for the same mission. The sharing of the production operations among the various Sentinel production services is managed by ESA as part of the operations procurement activities and each production service is configured accordingly.

The data production requires the use of auxiliary information, including instrument and products calibration and validation parameters provided by the Calibration and Validation activities as well as information from other auxiliary data providers, gathered by the Auxiliary Data Gathering Service.

Interfaces & Context

Each Systematic Data Production service interfaces with:

• Each Data Acquisition service relevant for the mission being operated, through the corresponding CADIPs
• The POD element, through the POD Interface Point (PODIP), to retrieve the POD information required for the production of the relevant Sentinel mission
• The Auxiliary Data Gathering Service, through the Auxiliary Interface Point (AUXIP), to retrieve the auxiliary information from the Calibration and Validation function or other auxiliary data providers
• The consumers via the Production Interface Point (PRIP) operated by the Production Service provider, allowing access to the outputs of the production operations. The PRIP consumers include in particular:
  • The Data Access Service, making available to users the User Level Data generated by the Production Service
  • The Long Term Archive Service(s), ensuring the archiving of the data generated by the Production Service in line with the long term data preservation scenario
  • The Calibration & Validation function, for access to engineering level data required for specific Calibration and/or Validation activities
  • The Operations Coordination Desk, for the E2E operations performance monitoring
  • The Flight Operations Services, for access to HKTM for spacecraft monitoring and control purposes^[1].
• The Traceability Service, for the registration of the trace associated to any User Level Data published on the PRIP.
• The Services Operations Coordination Service, typically for anomaly management or coordination of the introduction of new Data Processors.
• Where appropriate, the Mission Planning (via MPIP, not shown in figure 6) to determine the planned data takes, such that the production service may autonomously monitor the completeness of its production.

The context of the “Systematic Production” operations and main interfaces is illustrated in Figure 7. The figure shows the context and interfaces for one Systematic Production service, which applies equally to each of the Systematic Production services contributing to the Sentinel operations. Detailed interface information is provided in the [RD-4].

The figure indicates the scope of each of the Systematic Production services, including the operations and management of the functions in a cloud-based environment. The illustrated number of PRIP consumers is not exhaustive. Additional consumers may be authorised to access the PRIP for specific purposes, e.g. for Calibration & Validation activities to access to specific engineering data generated by the Production service and used for calibration or instrument monitoring purposes that may otherwise not be made available by the Data Access service.

Figure 7 Systematic Data processing context and main interfaces (for each Systematic Data Processing instance). N.B. Only one instance of the services implementing the various functions is illustrated (i.e. only one LTA, one Data Acquisition Service is shown).

Considering that multiple instances of the Systematic Production will co-exist in operations, ensuring overall the production of all Sentinel mission data and that multiple Data Acquisition services and LTA services will also operate in parallel serving multiple missions, multiple interfaces will therefore be available between these services as illustrated in the figure below. It is to be noted that the number of instances is only provided for illustration purposes, it is not relevant for the Architecture definition may vary according to the outcome of the procurement process.

The present configuration of the service is documented in [RD-12].

Figure 8 High level interfaces between the Systematic Data Production and the Data Acquisition, Data Access and LTA

Operations

Production services are managed autonomously by service providers responding to the service management requirements and service level agreements established by ESA. Production services are assigned to service providers according to standard ESA procurement practices, qualified services are managed through a frame contact which allows the service providers to access processing software and documentation (Data Processors and the Routine Quality Control software) in order to size operations and evaluate the orchestration requirements of the particular mission. Service switch-over between contracted services is closely coordinated and overseen by the Service operations coordination, access is provided to CADIP and auxiliary data in order to ensure the quality of production service before full nominal operations. Conformance tests are performed against the PRIP implementations before integration tests are performed by the various PRIP clients confirming the readiness.

Routine operations of the Production Services are monitored via the reporting and monitoring interfaces of the service which are integrated in the E2E operations monitoring, as well as through the dedicated dashboards provided by each service.

Production Services deploy updated processing software according to the needed deployment dates as notified by ESA, and are capable of roll-back by simple configuration.

The Data Processors are regularly updated typically to enhance the data quality, correct any potential anomalies or adapt to on-board instrument evolutions. Data Processors and static Auxiliary/Ancillary files that are considered part of the processing baseline are provided via the Reference System.

In case an operational qualification phase is required for a new version of a data processor, a temporary parallel production with the current and the new processor version may be operated in order to verify the operational readiness (performance, reliability and quality) of a new processor version before it substitutes the previous version in operations. The data flow from the temporary operations are published on a separated PRIP instance with the respect to the operational one.

Production services are responsible for prompt reporting of anomalies in the production due to failures of the Data Processors. Any unexpected behaviour is to be reported according to the anomaly coordination procedures established by ESA as soon as it is concluded by the service provider that these anomalies are not the consequence of an improper use of the data processor (e.g. incorrect selection or management of the auxiliary data, inadequate available resources, improper triggering of the processor).

Performance

The Production operations performance is monitored through a Service Level Agreement based on a number of performance indicators, harmonised for all production services, including in particular:

• Completeness of production according to quality factors (correct configuration, correct selection of auxiliary files etc.)
• Timeliness of production: the time between the availability of the source CADU data at the acquisition interface point and the publication time of the data on the PRIP
• Availability of PRIP (correct functioning and within performance limits)
• Data Processor Anomaly reporting: anomaly notification time

The Production sizing is defined based on key operational parameters, including e.g.:

• Volume of acquired satellite data to be processed
• Production volume to be generated
• Production timeliness
• Distributed volume to the overall consumers

The variation of the sizing parameters may impact on the service performance and cost. The relationship between the operations sizing, performance and cost is captured through a parametric model provided by the Service operator as part of the competitive service operations procurement process.

Such model allows a transparent prediction of the Production operations cost and performance as a function of the sizing variations.

Infrastructure and CFIs

The service operates fully on a public cloud infrastructure and encompasses any necessary public cloud based infrastructure to operate the function.

The Data Processors and the Routine Quality Control elements are provided as CFIs by ESA, as described in section 4.3.3.

On-Demand Production

The on-demand production element allows generating higher-level User Level Data (typically Level-1/2) from archived lower level data (typically Level-0) as a result of a user request to the Data Distribution service for a data item not immediately available on the Data Distribution storage (e.g. a past User Level Data item rolled out from the Data Distribution storage, or a user Level Data item from a different processor or from an different processing baseline).

The On-demand Production operations are fulfilled by the On-demand Production Service, operated as part of the Data Distribution service, to allow optimising the data retention strategy according to the user scenarios observed and as an important element of a disaster recovery scenario in case of major unavailability of historic data storage.

The on-demand production is operated in a public cloud environment to provide a highly scalable and adjustable capacity in response to the user demand.

Functional Scope

The on-demand production covers the following main functions:

• Operations of the interface allowing the on-demand production to be triggered via the Data Access Service or other authorised elements.
• Management of on-demand production requests queues.
• Retrieval from the LTA AIP of Level-0, auxiliary data or other data required for the processing
• Generation of the requested higher level User Level Data (typically Level-1 or Level-2) according to the required processing baseline.
• Operations and management of the delivery point with a short rolling storage (few days) to support retrieval from the various consumers.
• Provision of the User Level Data to the delivery point within the required timeliness and performance
• Automated routine quality control of the generated User Level Data
• Performance Monitoring and Reporting.

Missions Scope and Operations highlights

The on-demand production is mission specific.

The on-demand production function allows generating a higher level User Level Data with any previous qualified processing baseline.

Interfaces & Context

The On-demand production element interfaces with:

• Each authorised consumer (typically brokered through the Data Access service) to receive on-demand production requests
• Each authorised consumer (typically brokered through the Data Access service), through the delivery point, to make available the requested data
• Each LTA, through the AIP, for the retrieval of lower level data and auxiliary data required for processing
• The E2E Operations Performance Monitoring element, through the Data Access Service, to provide the reporting information required for the E2E operations monitoring.
• The Traceability service, for the generation and registration of the trace for the generated products.
• The Services Operations coordination service, typically for anomaly management.

The context of the On-Demand Processing operations and main interfaces is illustrated in

Figure 9. The figure shows the context and interfaces for the On-demand Production service. Detailed interface information is provided in [RD-4].

The figure below indicates the scope of each On-Demand Production service in a cloud-based environment, as well as the exchanges with the AIP in line with the associated interfaces and performance (only one instance of the AIP is represented).

Figure 9 On-demand Production context and main interfaces.

Performance

The On-demand Production operations performance is monitored through a Service Level Agreement based on a number of performance indicators including in particular:

• Timeliness and quality of on-demand production
• Volume of on-demand production requests

The On-demand Production sizing is defined based on key operational parameters, including e.g.:

• Timeliness to satisfy a given volume of on-demand requests
• Scalability capacity

The variation of the sizing parameters may impact on the service performance and cost. The relationship between the operations sizing, performance and cost is captured through a parametric model provided by the Service operator as part of the competitive service operations procurement process.

Such model allows a transparent prediction of the Production operations cost or performance as a function of the sizing variations.

Infrastructure and CFIs

The service operates fully on a public cloud infrastructure and encompasses any necessary public cloud based infrastructure to operate the function.

The Data Processors and the Routine Quality Control elements are provided as CFIs by ESA, as described in section 4.3.3.

Reprocessing

The Reprocessing operations allow the bulk (re-)generation of past higher level data (typically Level-1/2) from archived lower level data (typically Level-0).

The reprocessing function may be operated in a public cloud environment or on a private infrastructure according to the reprocessing needs, performance, capacity and effort.

The Reprocessing operations are fulfilled by the Production Services according to the specific reprocessing needs.

Functional Scope

The reprocessing covers the following main functions:

• Retrieval from the LTA AIP of Level-0, auxiliary data or other data required for the processing
• Generation of the requested higher level User Level Data (typically Level-1 or Level-2) according to the required processing baseline.
• Provision of the data to within the required schedule and performance
• Automated routine quality control of the generated User Level Data

Missions Scope and Operations highlights

The reprocessing is considered a specific case of Production service.

Reprocessing operations are performed on a case by case basis, according to the needs for massive re-generation of a specific mission User Level Data types over a specific mission time frame with a given processing baseline. The reprocessing operations are defined based on:

• The mission and data type(s) to be reprocessed
• The timeframe of the mission production to be reprocessed
• The source user level data to be used as input (typically L0) and the associated auxiliary information
• The target user level data (typically L1 and/or L2)
• The data processing baseline to be used (data processing software and configuration files)
• The schedule for the reprocessing campaign, defining the reprocessing rate (ratio between the months of data to be processed and the months necessary for their reprocessing).

Interfaces & Context

Each Reprocessing operations interface with:

• Each LTA, through the AIP, for the retrieval of lower level products and auxiliary data required for reprocessing.
• The LTA, for the archiving of the reprocessed data as necessary for the specific reprocessing activity.
• The Data Access Service, for making reprocessed data available to users as necessary for the specific reprocessing activity.
• The Traceability service for the generation and registration of the traces for the reprocessed data.

The interface for the delivery of the reprocessed data follow a standard PRIP interface, with a sufficient retention time to allow access by consumers. Ad-hoc interfaces may be considered on a case by case basis depending on the specific reprocessing activity.

Figure 10 Reprocessing context and main interfaces

Performance

The Reprocessing operations performance is mainly driven by the associated reprocessing

rate.

The relationship between the reprocessing performance and cost is captured through a parametric model provided by the Service operator as part of the competitive service operations procurement process.

Infrastructure and CFIs

The service encompasses any necessary infrastructure to operate the function, based on a public cloud infrastructure or on a private infrastructure.

The Data Processors and the Routine Quality Control elements are provided as CFIs by ESA, as described in section 4.3.3.

Auxiliary Data Gathering

The processing of the acquired satellite data requires a number of auxiliary information, not included in the satellite telemetry. This encompasses different types of auxiliary information from different sources:

• Regularly updated auxiliary files related to the instrument and processing configuration information, updated regularly on a need basis. These files are typically generated as part of the Cal/Val operations and used for the Level-1 production and Level-2 production.
• Dynamic auxiliary information generated by external entities (e.g. ECMWF, Ifremer, etc.), systematically renewed with a known frequency. This information is often available in a heterogeneous format and requires reformatting before it can be used for the Copernicus production operations.
• Dynamic auxiliary data from FOS (e.g. orbital info), systematically renewed with a known frequency.
• Dynamic auxiliary orbit and attitude information generated by the POD service, systematically renewed with a known frequency.
• Static and semi-statis auxiliary files, required by the data processor an updated rarely through the mission.

Production operations require access to these various types and sources of auxiliary data. In order to optimise the effort related to the auxiliary data management and ensuring coherence among all services requiring the use of auxiliary information, a single auxiliary data collector and reformatting is considered within the CSC GS Architecture.

The Auxiliary Data Gathering (ADG) Service acts as a concentrator and single point of access for auxiliary data for all systematic production operations within the CSC GS.

In case timeliness to specific auxiliary data is of essence for some specific production operations (e.g. access to POD for Sentinel-1 production operations), the relevant auxiliary files may overpass the ADG.

Static and semi-static auxiliary data are instead managed in configuration control with the processing software (through a dedicated environment within the Reference System).

Functional Scope

The Auxiliary Data Gathering Service covers the following key functions:

• Collection of auxiliary data from the different sources (e.g. Cal&Val component, FOS, ECMWF, Ifremer, etc.) in line with the operational needs of the various Copernicus missions
• Reformatting of the external auxiliary data in line with the CSC GS interfaces
• Gathering of auxiliary data generated by the Cal&Val service for production operations
• Repository of auxiliary data interface point (AUXIP)
• Operations and management of the Auxiliary Interface Point (AUXIP), including a rolling repository to support retrieval of auxiliary data from the various consumers (e.g. systematic production service, on-demand production service, Data Access and LTA service). By configuration, for some auxiliary data types, the full historic set will be maintained within the AUXIP without a rolling policy.
• Performance Monitoring and Reporting

Missions Scope and Operations highlights

The Auxiliary Data Gathering Service is common to all flying Copernicus missions. The data collection interfaces, and possible reformatting are mission specific.

Interfaces & Context

The Auxiliary Data Gathering Service interfaces with the following elements:

• The various external Auxiliary Data Providers (including FOS)
• The Cal & Val Services (for gathering specific Aux Data)
• A set of AUXIP consumers (including in particular the various Production instances, the various LTA instances, and the Data Access Service), accessing the auxiliary information handled by the service
• The End to end Operations Performance monitoring
• The Traceability service, for the generation and registration of the traces for the reformatted files and the verification of the traces for the input files whenever relevant.
• The Services Operations coordination service, typically for anomaly management

Figure 11 Auxiliary Data Gathering context and main interfaces.

Performance

The Auxiliary Data Gathering operations performance is monitored based on a number of

performance indicators including in particular:

• Completeness of auxiliary data production according to quality factors (correct configuration etc.)
• Timeliness of production
• Availability of AUXIP

Instrument data processing algorithm and Operational Processor maintenance

The operational processing of the satellite data stream into Level-0 and Level-1 User Level Data for the Sentinel satellites is based on well-defined processing algorithms for which the operational implementation and maintenance depends strongly on the target operational environment and operational requirements (i.e. processing performance, scalability, optimisation for a target computing environment, etc.).

Functional Scope

The scope of this function is to maintain the algorithms and processor implementation ensuring maximum suitability with the processing environment and operational needs while guaranteeing the User Level Data quality standards.

Accordingly, the Instrument and data processing algorithm and operational processor maintenance encompasses the following key functions:

• Maintenance of the Level-0 and Level-1 processing algorithms
• Maintenance of the Level-0 and Level-1 data format
• Maintenance of the Level-0 and Level-1 instrument data processors, including all necessary information to operate the processor (e.g. including the associated interfaces, the orchestration definition, the user manual, etc.). An Open Source approach is targeted as baseline for all processors.
• Assessment of the Level-1 data quality
• Full validation of any new versions of the data processors

Missions Scope and Operations Highlight

The processing algorithms and processors are mission and instrument dependent.

The operational implementation approach is however common to all Copernicus missions and suited in particular for a cloud-based operational environment, in line with the strategy adopted for the operations of the Copernicus missions data production.

The outcomes of this function constitute a CFI for the services involving data production operations.

Interfaces & Context

The function interfaces with the Reference System, in charge of making available any CFI elements provided by ESA.

This function includes the representative operational validation of any new processor versions before release for operational use by the services in charge. For this purpose, this function interfaces with the Reference System that provides the environment and functionalities to the “Instrument data processing algorithm and operational processor maintenance” for deploying and validating the new processor version in a representative environment.

The function interfaces also with the services in charge of the production operations, providing processor related support if necessary, with the Services Operations coordination, to handle any production anomalies related to the operational processor and with the “Instrument and data calibration and validation function”.

Performance

The Operational Processor performance are mission dependent and relate to the instrument calibration accuracies required to fulfil the mission objectives and the computational efficiencies required to achieve these accuracies within the timeliness constraints.

Infrastructure and CFIs

The existing Level-0 and Level-1 processing algorithm and the currently operational processors are made available as a CFI to this function.

The environment for the operational validation of new processor versions is made available by the Reference System. The Reference System provides the environment and functionalities allowing the “Instrument data processing algorithm and operational processor maintenance” to perform a representative deployment and validation of any new processor versions before release for operational use by the services in charge.

Calibration and Validation

The aim of the Calibration and Validation (Cal&Val) function is to monitor the instrument performance and perform all expert activities necessary to ensure that User Level Data meet the calibration and validation specifications.

The Calibration and Validation operations are fulfilled by the Mission Performance Cluster services in the form of operational services provision.

Functional Scope

The Cal&Val function encompasses all expert activities to monitor the instrument performance, to characterise the User Level Data quality, to assess the achievement of the expected quality levels according to the product specifications, to perform all necessary activities to derive and maintain the calibration information, to perform all necessary activities to derive and maintain the validation information and to characterise the User Level Data quality and its calibration and validation levels.

The function includes the scientific algorithm definition and maintenance, in particular the definition and maintenance of the Level-2 algorithms and the implementation and maintenance of the Level-2 data processors.

A detailed elaboration of the core functions involved is provided in [RD-7]

Missions Scope and Operations Highlight

The systematic screening and routine quality control of all Level-0 and Level-1/2 User Level Data is performed as part of the production operations, allowing an optimised screening of all generated data in synergy with the production.

The outcome of this systematic screening, including extracted metadata and/or routine quality control results, is maintained as part of the E2E Operations Monitoring function, offering an operational access interface to authorised consumers, typically the Cal&Val function.

Interfaces & Context

Being a mission specific service, the associated operational interfaces and performance may be also specific, however a number of interfaces and performance measures will be common to all services.

Each Cal&Val element interfaces in particular with:

• The Data Accees, for accessing to the User Level Data to be quality checked or used for specific calibration and/or validation activities,
• The Systematic Production element, for accessing to any engineering data not available through the Data Access and necessary for the quality related activities,
• The Reference System, using the environment made available by the reference system for the integration and testing of new Level-2 data processors, verification of new processor versions and pilot production of new User Level Data.
• The Reference System, for the provision of data processors and associated information, to be made available as ESA CFI to services involved with the data production operations.
• The Auxiliary Data Gathering, providing processing auxiliary information.
• The Services Operations coordination service, typically for anomaly management.
• The FOS for on-board instruments configuration.

Performance

The Calibration and Validation performances are mission dependent and relate to the parameter retrieval accuracies required to fulfil the mission objectives and the computational efficiencies required to achieve these accuracies within the timeliness constraints.

Infrastructure and CFIs

The existing Level-2 algorithms and operational processors implementation are made available as CFI.

The environment for the operational self-qualification of new processor versions before delivery to the production function is made available by the Reference System.

Precise Orbit Determination

The Precise Orbit Determination (POD) function is in charge of the generation, provision and validation of precise orbital data and auxiliary data files for the flying Copernicus Sentinel Satellites.

Functional Scope

The POD encompasses the following functionalities:

• generation, provision and validation of precise orbital data and auxiliary data files for the relevant flying Copernicus missions (at present forSentinel-1, Sentinel-2 and Sentinel-3)
• the assessment of the GPS (Global Positioning System) receiver sensor performance on-board the Copernicus satellites, the quality of the generated data
• interface and liaising with Copernicus POD Quality Working Group
• interface and liaising with the International Laser Range Satellite (ILRS) community
• interface and liaising with the GPS POD data providers
• interface and liaising with partner agencies (CNES, EUMETSAT)

Missions Scope and Operations highlights

The POD function is a common to all flying Copernicus mission, used where a high precision orbit determination is required for the geolocation accuracy of the Copernicus User Level Data. This is the case for the Sentinel-1, Sentinel-2, Sentinel-3 and Sentinel-6 operations.

The POD operations are based on the data from the GNSS receiver on board of the Sentinel satellites, downlinked to ground as part of the regular operations and extracted by each Systematic Production function from the overall satellite data stream. The extracted GNSS data is made available in the PRIP as input to the POD function, together with any other satellite data necessary to generate the POD auxiliary files.

Interfaces and Context

The POD interfaces with the following elements:

• Each PRIP to get the GNSS receiver information from the downlinked data and other information as necessary.
• FOS for information on manoeuvres, mass history files, restituted and predicted orbits, unavailability reports
• CNES for information in S3 DORIS and additional POD data
• A set of PODIP consumers (including in particular the various Systematic Production instances, the various LTA instances, and the Data Access Service), accessing the auxiliary information generated by the POD service
• The Traceability Service
• The Services Operations coordination service, typically for anomaly management

The context of the POD and main interfaces is illustrated in Figure 12. Detailed interface information is provided in the [RD-4].

Figure 12 Precise Orbit Determination context and main interfaces.

Performance

The POD operations performance is monitored based on a number of performance

indicators including in particular:

• Timeliness and quality of the generated POD outputs
• Availability of PODIP

The POD sizing is defined based on key operational parameters, including e.g.:

• Timeliness to produce the orbit and auxiliary files information required for the subsequent production operations
• Number of satellites requiring routine generation of POD files

The variation of the sizing parameters may impact on the service performance and cost. The relationship between the operations sizing, performance and cost is captured through a parametric model provided by the Service operator as part of the competitive service operations procurement process.

Such model allows a transparent prediction of the Production operations cost or performance as a function of the sizing variations.

Infrastructure and CFIs

The service operates fully on a public cloud infrastructure and encompasses any necessary public cloud based infrastructure to operate the function.

Data Preservation

The data preservation function ensure the storage of the essential mission data. It is fulfilled by the Long Term Archive element in the form of operational service provision.

Functional Scope

The Long Term Archive (LTA) service is defined to cover the following main functions:

• Data Ingestion and safe storage
• Data Query, Retrieval and delivery on an Archive Interface delivery Point (AIP)
• Data curation and archive Integrity
• Performance Monitoring and Reporting
• Implementation and operations of the AIP, ensuring its availability for consumers and handling of cache policy

The LTA interface includes all the features necessary to support the above LTA functionalities. In particular, all the features to query the stored data, query performance information for monitoring purposes and to retrieve the selected data.

The LTA interface allows precise query capabilities for standard EO oriented metadata, with a detailed definition of the indexed fields for each data type.

Missions Scope & Operations highlights

The LTA service scope is common to all missions: The functional scope of the LTA and the associated interfaces are defined in order to be Sentinel-generic and fulfil the archiving needs for any Copernicus mission.

Multiple LTAs service instances are however necessary in the GS architecture to ensure geographical redundancy with separated safe storage for all Sentinel missions. The same LTA may serve one single Copernicus mission or several Copernicus missions, depending on its capacity and cost efficiencies.

The content of each LTA (in terms of Sentinel mission and data types) and the overall data flow configuration (data flows between the various Sentinel production services and the various LTA services) is defined and maintained by ESA as part of the E2E operations management [RD-12]. Each LTA operates autonomously with respect to the other LTAs. However, the LTAs monitor their own completeness, and any missing products are downloaded from one of the other LTAs in case it is no longer available on the PRIP. An E2E view is also gathered through the E2E operations monitoring and the traceability service.

Interfaces & Context

Each LTA interfaces with:

• Each Systematic Production element, through the Production Interface Point (PRIP), to retrieve the Sentinel data to be stored in the LTA (as well as with the On-Demand Production and the Reprocessing elements as relevant per mission and reprocessing scenario).
• The POD element, through the POD Interface Point (PODIP), to retrieve the POD information to be stored in the LTA
• The Data Access and Production Services, through the Archive Interface Point (AIP), to provide access to the data stored in the LTA
• The E2E Operations Performance Monitoring element, to provide the reporting information required for the E2E operations monitoring
• The Traceability service, for the verification of the traces of the input data and registration of the traces of the ingested data
• The Services Operations coordination service, typically for anomaly management

As per general principle, each LTA will manage an Archive Interface Point (AIP) on a cloud-based environment to provide access to the data requested from the LTA. Data retrieved from the LTA may correspond to User Level Data as a result of a user demand through the data access service or to the lowest level data necessary to generate higher-level data (e.g. as a result of a user request or as part of a reprocessing activity).

Access to the LTA query and retrieval interfaces is restricted to consumers authorised by ESA as part of the E2E operations management function.

Authorised consumers include:

• The production services, in order to retrieve the lowest level data and auxiliary data for reprocessing or on-the-fly processing activities
• The data access service, in order to retrieve any data from the LTA required to satisfy a user access request
• Other services like the calibration & validation, in order to retrieve data necessary for the specific campaigns or cal/val activities.

For the LTAs providing the same data for redundancy purposes, the load balancing is performed by the consumer through the available LTA interfaces. The LTA interfaces include the information of LTA operations load (e.g. pending retrieval queue size, current average retrieval time) necessary by the consumer to select for each query/retrieval operations the target LTA according to the respective load.

The context of the LTA operations and main interfaces is illustrated in

Figure 13. The figure shows the context and interfaces for one LTA, which applies equally to each of the LTAs contributing to the Copernicus operations. Detailed interface information is provided in [RD-4].

The figure indicates the scope of each LTA service, including the operations and management of the AIP in a cloud-based environment, as well as the exchanges with the PRIP, and PODIP in line with the associated interfaces and performance.

Figure 13 LTA context and main interfaces

Operations

LTA services are managed autonomously by service providers responding to the service management requirements and service level agreements established by ESA. LTA services are assigned to service providers according to standard ESA procurement practices, qualified services are managed through a frame contact which allows the service providers to access sample data to prepare for operations. Service switch-over between contracted services is closely coordinated and overseen by the Service operations coordination, access is provided to data from prior LTA operations in order to ensure any necessary transfer of historic data. Conformance tests are performed against the AIP implementations to ensure precise implementation of the data models and extraction of metadata from the data for archival such that query results do not differ from one LTA to another.

Routine operations of the LTA Services are monitored via the reporting and monitoring interfaces of the service and integrated in the E2E operations monitoring, as well as the dedicated dashboards for each service.

Performance

The LTA operations performance in monitored through a Service Level Agreement based on a number of performance indicators, fully harmonised for all LTAs, including:

• Archival Ingestion Timeliness (completeness of the ingestion within nominal period of data availability on the PRIP)
• Preservation Performance (concerning any data loss events detected)
• Retrieval Performance (concerning data extraction from the archive)
• Service Availability (correct performance of all functions)

The LTA operations sizing is defined based on key operational parameters, including e.g.:

• The ingestion rate & performance, according to the Sentinel mission operations daily production and the associated configuration for the data to be stored in the LTA (regardless the number of the Interface Delivery Points acting as providers)
• The retrieval rate & performance, according to the volume that it shall be retrieval from the LTA (defined as a function of the ingested volume), regardless of the number of consumers accessing the AIP

The variation of the sizing parameters may impact on the service performance and cost. The relationship between the operations sizing, performance and cost is captured through a parametric model provided by the Service operator as part of the competitive service operations procurement process.

Such model allows a transparent prediction of the LTA operations cost or performance as a function of the sizing variations.

Infrastructure and CFIs

The LTA service includes any necessary infrastructure necessary to operate the function, with no dedicated Copernicus infrastructure deployment and no hosting of elements required to operate other architectural elements (e.g. no hosting of network or production infrastructure).

No ESA CFIs are included in this function. Open Source reference parser supports the precise definition of EO metadata for extraction within a standardized data model.

The service includes a delivery point on a public cloud infrastructure, accessible to entitled consumers.

The infrastructure for the implementation of the long term archiving is not imposed by the CSC GS architecture. It is represented in this document outside the “Cloud environment” since the use of a public cloud environment is not imposed (i.e. the selection of the infrastructure for the LTA operations is to be defined by the LTA service provider).

Data Access

The Data Access element provides end user access to the Copernicus missions User Level Data. The Data Access element is embedded in a cloud computing ecosystem (herein called the Copernicus Data Space) that allows the provision of computing environments (IaaS, PaaS etc.) to customers to process hosted data with high efficiency.

The GS architecture foresees a that the Data Access services serves all Copernicus missions and provides access to the Copernicus Contributing Missions Core datasets.

The characteristics of the service and its criticality of the Data Access service are such that the redundancy principle followed for the CSC GS Architecture is implemented here by ensuring that the sreivce is deployable on two separated and independent cloud infrastructure, with a rapid reconfiguration from the primary to secondary cloud in response to any disaster situation. The Data Access element relies on a unified user management user management service, providing user access to services through a common identity management. As an example, the provision of an account for the Copernicus Data Access Services shall open access to complementary Copernicus Data Space services like IaaS, PaaS or SaaS based on the same digital identity.

Since 2024 the Data Access element of the Copernicus Ground Segment is provided through the Copernicus Data Space Ecosystem: https://dataspace.copernicus.eu

Functional Scope

The Data Access element addresses the following key functions:

• Data Distribution (a.k.a. Data Retreival) function allowing download of user Copernicus mission User level data for any user.

The Data Distribution function is based on immediate synchronous access to a certain configurable data set, including most recent data and past data with a configurable time span, and asynchronous access to other data e.g. via data archive or a triggering of on-demand processing.

• Streamlined Access function allowing direct / interactive processing of selected datasets.

The role of the Streamlined Access function is to provide an open and free streamlined and harmonised access to dedicated Sentinel User Level Data collections, allowing local processing through a Platform as a Service paradigm (e.g. Notebook server).

The Data Access Service covers the following elements:

• User Registration, Terms and Conditions acceptance
• Data Retrieval, Ingestion and Publication
• Metadata Extraction
• Quicklook Generation (if needed) / Publication
• Data Discovery
• Data Download functions
• Data Transformation functions
• Data Deletion functions
• Catalogue Summary functions
• Storage Management
• Performance Monitoring and Reporting
• Streamlined data access interfaces
• Data visualisation interfaces

Missions Scope & Operations highlights

The Data Access is common to all flying Copernicus missions, providing at present data access for all Sentinel missions operated by ESA in the form of an operational service provision. The Data Access service provides configurable access performances to specific use typologies (Open Access, Copernicus service, Collaborative GS, International partners, etc.), the performances/quota allocated to each use typology is set out in the Data Access Services Portfolio [RD-11]

User registration is managed according to the user typologies, self-registration (web-based) for the Open Access, operator managed registration for the other services. Access policies are also configured accordingly (parallel download quota, etc.). User information are managed according to the relevant personal data policy regulations, and in principle aim to request and retain the minimum set of user information necessary to fulfil the data access operations and reporting requirements.

The Data Access function ensures the immediate availability of a configurable amount of data (from one year to the whole mission_. Data not anymore available for immediate access remain available to users through the same data access interfaces by triggering either the retrieval e.g. from a data archive or an on-demand processing. Data thus retrieved are maintained in a cache for configurable period of time.

Data Access operations are supported by Graphical User Interface and an API.

In addition to the common data access services for all Copernicus missions and data sets, ad-hoc data access services may be operated to satisfy specific needs from the Copernicus Services or other Entrusted Entities, in particular:

• Dedicated FTP access to Sentinel-1 data required by CMEMS
• Dedicated FTP access to Sentinel-1 data following an emergency/security activation by the CEMS or Security Service.
• Dedicated access to Sentinel-5P data required by EUMETSAT for further dissemination via EUMETCAST.
• Dedicated access to processing auxiliary files and orbit information.

As far as possible dedicated access points shall be decommissioned in favour of standard services.

Interfaces

The Data Access Service interfaces with:

• Each Systematic Production element, through the Production Interface Point (PRIP), to retrieve the Sentinel User Level Data to be made available to users. Two PRIP are expected for each Sentinel mission (i.e. one per satellite unit).
• The POD element, through the POD Interface Point (PODIP), to retrieve the POD information to be made available to users. One PODIP is expected, common to all Sentinel missions.
• Each LTA, through the AIP, to retrieve and make available to users User Level Data from the LTA where available or to retrieve data as an input for the On-Demand production.
• The E2E Operations Performance Monitoring element, to provide the reporting information required for the E2E operations monitoring.
• The Auxiliary Data Gathering, for the retrieval of any auxiliary data to be made available to users or to retrieve data as an input for the On-Demand production.
• Users, providing access to the Sentinel User Level Data, CCM core datasets and auxiliary files.
• The Traceability Service, for the registration and verification of traces.
• The Copernicus Contributing Mission Data Access System (PRISM) for the retrieval of Copernicus Contributing Missions core data sets to be made accessible for download through the Copernicus Data Access Service.

The context of the Data Access operations and main interfaces is illustrated in Figure 14. Detailed interface information is provided in the [RD-4].

The figure includes the on-demand production service as described in 4.3.3.2, and operated as part of the Data Access service the operations and management of the Data Distribution function in a cloud-based environment, as well as the exchanges with the PRIP, PODIP, and AIP in line with the associated interfaces and performance.

It is to be noted that the interface definition is common for the various Interface Delivery Points (PRIP, PODIP, AIP), such that the multiplicity of Interface Delivery Points is not increasing the complexity of the Data Access implementation and operations, although it requires a careful configuration management.

Figure 14 Data Access context and main interfaces. Only one instance of the Systematic Production and the LTA are illustrated for simplicity.

Performance

The Data Access operations performance is monitored through a Servce Level Agreement based on a number of performance indicators including in particular:

• Service Availability
• Data Availability Timeliness, Completeness and Integrity of publication

The Data Access sizing is defined based on key operational parameters, including e.g.:

• Yearly published data volume & number of items
• Volume of data available for immediate user access
• Distributed data volume
• Number of active users
• Required data access timeliness
• Data access performance parameters
• Amount of User activity quantified according to the specific data acces function

The variation of the sizing parameters may impact on the service performance and cost. The relationship between the operations sizing, performance and cost is captured through a parametric model provided by the Service operator as part of the competitive service operations procurement process.

Such model allows a transparent prediction of the Data Access operations cost or performance as a function of the sizing variations.

Infrastructure and CFIs

The service operates fully on a public cloud infrastructure and includes any necessary public cloud based infrastructure to operate the function. The Data Access service provides connectivity for data access through public (commodity) internet as well as high bandwidth research networks, in particular GÉANT^[2].

ESA CFIs for processing are included in this function for the on-demand processing.

Reference System

The Reference System is a key element of the Ground Segment architecture, responding to multiple needs and aiming mainly at:

• preventing the industrial lock-in for the production and data download operations by implementing and maintaining an open-source solution for these functions
• operating an “under-scaled” production service, cloud-based, able to be scaled up to support an operational load should it be necessary, and offering a reference production and quality control environment (with well defined access and sizing conditions)
• providing a ready-to-use representative environment for the integration of new User Level Data and their pilot production phase
• serving as potential representative CSC ground segment operational environment for the pre-launch activities related to the new satellite units/new Copernicus missions.

As a secondary service, The Reference System ensures the publication of the historical, new and updated versions of the Sentinel Data processors packages which are used in various components of the CSC (e.g. Production Services, Data Access).

Functional Scope

The Reference System covers the following main functionalities and scope:

• Implementation and maintenance of an operational open source cloud-based solution for the (scaled-back) production, routine quality control and data download
• Offering the interfaces for triggering the production of any Sentinel User Level Data, from fresh or archived data, and from any processing baseline
• Operating an “under-scaled” production service, cloud-based, able to be scaled up to support an operational load should it be necessary, and offering a reference production and quality control environment (with well defined access and sizing conditions)
• Offering the tools and functionalities allowing a streamlined pilot integration of new data processors and the associated pilot production activities
• Implementing and maintaining the tools and functionalities allowing the streamlined integration of new data flows for new Sentinel units, including associated data processing and routine quality control
• Providing a suitable environment to support the pre-launch CSC ground segment preparation activities for new Sentinel units/new Copernicus missions.
• Providing the possibility to trigger the processing of any historical User Level Data with any former processor version no longer part of the operational baseline.
• Managing the publicly accessible repositories for all RS software related documentation, open source code, and executables / containerised deployment images available to the various Copernicus Ground Segment operational services and to the Collaborative Ground Segment partners community in general.
• Operatng a dedicated Web portal and API for the publication of the Sentinel Data Processors packages, in native form and as RS addons, accessible towards duly authorized entites (e.g. Procution Service operators).

Missions Scope

By its nature, the Reference System is common to all Copernicus missions.

Interfaces & Context

The Reference System interfaces with:

• Auxiliary Data Gathering Service for accessing to the auxiliary data required for the processing
• Each Data Acquisition service, through the CADIP, providing the possibility to access as necessary the satellite CADU data stream for reference production activities or new processor validation, or pilot production for new User Level Data, etc.
• The EDRS Ground Segment, through the EDRS Interface Point (EDIP)
• The POD element, through the POD Interface Point (PODIP), to retrieve the POD information required for the reference production needs
• Each LTA for reference production purposes

The reference production and data access functionalities provided by the Reference System are available to entitled CSC operational services (e.g. Cal&Val, Production services, Operations Management and control functions), and are not meant to be directly available to Copernicus users.

The context of the Reference System and main interfaces is illustrated in Figure 15. Detailed interface information is provided in the [RD-4].

Figure 15 Reference System context and main interfaces.

Operations

The Reference System operates an online repository for the publication of the Copernicus missions Data Processors received from ESA. The Data Processors are published with access to documentation, executables and configuration files test data etc. The Processors are made available to authorized entities.

The Data Processors are integrated within the Reference System to support a (scaled) production. The scaled production is operated to allow a sampled systematic production that provides a reference for the data quality that is also expected from the nominal Production Services. Access to this scaled production is for authorized users only. The integration may also support the qualification of new processors

The Reference System itself is published to users under open source conditions.

Performance

The Reference System plays a key role in guaranteeing the availability of a production and data access solution capable to scale up for supporting an operational role in the routine operations.

The monitoring of the Reference System performance resides in particular on the capability of the Reference System to demonstrate the readiness to support the production operations for any on-going Sentinel mission and the scalability capabilities.

Infrastructure and CFIs

The Reference System is the source of any CFI elements made available by ESA to the Ground Segment operational services.

The Reference System is implemented and operated on a cloud-based environment.

Operations Traceability service

In an operational context with multiple providers and solutions involved in the production, archive and access to Copernicus Sentinel User Level Data, it becomes essential to ensure the data traceability, certifying its multiple transformations (e.g. systematic production, reprocessing, download from different access points, etc.) and allowing users to verify the “supply chain”, univocally identifying all core User Level Data.

The Traceability Service manages trace records for all user level data produced, archived and distributed within the EOF. Trace records contain cryptographic hash values for the user data items exchanged through the interface delivery points within the CSC GS elements. The hash values are calculated in such a way as to provide reproducible values even for the data packages (tar/zip), hash values are also calculated for key elements within the data package. The trace records include standard metadata items that identify the user level data. The trace records are digitally signed by the GS element providers to provide verifiable identification of the originating source.

The Traceability Service is interfaced via API for the reception of trace records from the various CSC GS elements and the retrieval/verification of trace records by consumer services and end users. The Traceability Service also offers a GUI for simple verification of hash values.

Functional Scope

The Traceability Service covers the following main functionalities and scope:

• Copernicus operations key attribution system for trace signature
• Implementation and maintenance of an operational open source cloud-based solution for the operations traceability
• Operations of the data certification and traceability service

Missions Scope and Operations highlights

By its nature, the Traceability Service is common to all Copernicus missions.

Interfaces & Context

The Traceability Service interfaces with:

• Production services, for the registration of signed trace records for new user level and engineering data
• Auxiliary data gathering function, for the registration signed trace records for new auxiliary data
• LTA services, for the verification of authenticity of data before archive and the registration of signed records following the archival
• Data Access services, for the verification of authenticity of data before publication and the registration of signed records following any re-packaging
• Copernicus Sentinel data users, for the verification of authenticity of data

Infrastructure and CFIs

The Traceability Service is implemented and operated on a cloud-based environment. The Traceability Service provides a reference implementation of the cryptographic hash calculation algorithm that is itself based on state of the art Secure Hash Algorithms (SHA) standards available within common libraries. The reference implementation may be used by the GS services or re-implemented within the workflow of the service according to opportunity for performance optimisations.

End-to-End Operations Performance Monitoring

The E2E Operations performance monitoring aims at gathering the operations performance of the various services contributing to the CSC GS operations in order to build a comprehensive overall end-to-end view of the operations performance at satellite unit level, mission level and CSC GS level.

The E2E Operations performance function is fulfilled by the E2E Performance Monitoring Service in the form of operational service provision.

Functional Scope

The E2E Operations Performance Monitoring covers the following main functionalities and scope:

• Implementation, maintenance and operations of an open-source cloud-based system for the E2E CSC Ground Segment Operations Performance Monitoring
• Monitoring of the E2E operations of each operational service, each Sentinel satellite unit, each Sentinel mission and the overall CSC GS
• Provision of a comprehensive monitoring and reporting interactive dashboard for CSC operations management, allowing access to the full operations performance information (at service level, at satellite unit level, at mission level, etc.)
• Provision of a comprehensive monitoring and reporting interactive dashboard for Copernicus users and stakeholders, providing up to date information on any operational issues at mission level, satellite level and User Level Data (including plans, unavailabilities, data quality information, operations performance information, etc.)

Missions Scope and Operations highlights

By its nature, the E2E Operations Performance Monitoring is common to all Copernicus missions and covers the complete operational chain, from planning to acquisition and data access, including notably the data production, quality control and archiving.

Each CSC GS operational service provides an operations dashboard and/or an interface to query operations performance monitoring information.

The E2E Operations Monitoring element interfaces with all the services and functions contributing to the CSC operations in order to build a comprehensive performance monitoring view.

Interfaces & Context

The E2E Operations Performance Monitoring interfaces with the following elements to collect the reporting information required for the operations monitoring and reporting:

• The Mission Planning service.
• Each Systematic Production services.
• The On-demand Production services, via the Data Access service.
• Each Data Acquisition service
• The EDRS service
• The POD service
• The Auxiliary Data Gathering service
• The Data Access Service, for the retrieval of the operations monitoring of data publication and data distribution and access performances
• Each LTA service, for the retrieval of LTA operations performance monitoring
• The Traceability service
• The Satellite operations and control, as the source for satellite and instrument unavailabilites

The context of the E2E Operations Performance Monitoring and main interfaces is illustrated in Figure 16. Detailed interface information is provided in the [RD-4].

Figure 16 E2E Operations Performance monitoring context and main interfaces.

Performance

The E2E Operations Performance Monitoring is monitored through a Service Level Agreement based on a number of performance indicators including in particular:

• Dashboard and monitoring & reporting interfaces availability
• Completeness and quality of the monitoring & reporting
• Monitoring and reporting timeliness

The E2E Operations Performance Monitoring sizing is defined based on key operational parameters, including in particular the number of satellite units in operations.

Infrastructure and CFIs

The E2E Operations Performance Monitoring is implemented and operated on a cloud-based environment as part of the service delivery.

Services Operations Coordination

Service Operations Coordination consists of the engineering processes necessary to analyse the performances of the E2E CSC Ground Segment Operations, coordinate the various issues that may impact on the multiple Ground Segments Services, coordinate the response to the users inquiry or complaint, ensuring the correct level of information is easily available to the users with reference to the CSC operations and outputs.

Functional Scope

The Services Operations Coordination function encompasses the following main functions:

• PA/QA elements for
  • coordinated anomaly management,
  • coordinated maintenance management,
  • coordinated configuration control.
• Coordination helpdesk operations to trigger 2^nd line investigations within the various service elements.
• Operations and maintenance of the Coordinated Anomaly Management System, providing an open environment shared among all entities contributing to the CSC operations, for the coordinated management of service anomalies affecting the E2E operations, management of service maintenance activities and configuration control.
• Users support and web information publishing service.

Missions Scope and Operations highlights

By its nature, the Services Operations Coordination is common to all Copernicus missions and covers the complete operational chain.

Interfaces & Context

The Services Operations Coordination interfaces with all services and elements contributing to the CSC operations according to dedicated operational procedures. The Services Operations Coordination supports the switch-in and switch-out of GS service elements and the control of the interface point configurations, including the coordination of client registrations for each of the GS internal elements interconnectivity (no centralised identity access management is foreseen for the GS internal elements)

Performance

The performance of the Services Operations Coordination is monitored through a Service Level Agreement based on a number of key performance indicators including in particular:

• Timeliness for the management of the coordinated anomalies and maintenance activities
• Timeliness for the reporting

The Services Operations Coordination sizing is defined based on key operational parameters, including e.g.:

• Number of satellite units in operations
• Number of operational services involved in the CSC operations

Security Monitoring

The Security monitoring is implemented as part of each individual service operations, with a particular focus on the security control of the service interface point.

A Security management layer implements the top level information security management function of the CSC GS. It is composed of a central security management function for security auditing, risk assessment, and security incident handling.

Functional Scope

The Security management covers the following main functionalities and scope:

• Assess and audit the implementation and management of Security implementation including Vulnerability Assessment (VA) and System hardening process for all GS Services / elements.
• Security Incidents recording and response coordination with GS Services and ESA representatives.

Missions Scope and Operations highlights

The Security monitoring, prevention, detection, response and maintenance function covers all CSC GS components and Sentinel missions.

Benchmarking Service

The Benchmarking Service provides an independent analysis of the performance of the user facing GS data access elements, assessing the quality of experience from a user perspective, as well as a measure of comparison against similar services operated by other providers.

Functional Scope

The Benchmarking service performs routine testing of data access elements to provide:

• Summary reports of quality of experience indicators and performance metrics.
• Comparative reports including end-to-end scenario implementations
• Analysis of performances from distinct geographic locations
• Temporal stability of performances

The test suites developed and operated by the Benchmark Service are made available as open source software to allow any third party repeat the benchmark tests, to compare to the published performances.

Missions Scope and Operations highlights

The Benchmarking service is applied to data access of all Sentinel missions.

Performance

The performance of the Benchmark is monitored based on a number of performance indicators including in particular:

• Completeness of test campaigns
• Delivery of reports

Infrastructure and CFIs

The Benchmarking Service is implemented and operated on a cloud-based environment, there are no CFIs to or from the Service.

Flight Operations Services

The FOS main responsibility encompasses the spacecraft monitoring and control, including execution of all platform activities and the commanding of the payload schedules

Functional Scope

The main Sentinels FOS components are:

• The Copernicus Flight Operations Control Centre (FOCC) at ESOC, including the following main elements:
  • The Sentinels Mission Control System, supporting hardware and software, Telecommand coding and transfer, Housekeeping Telemetry (HKTM) data archiving and processing tasks essential for controlling the mission, as well as all FOCC external interfaces, including On-Board Software Maintenance (OBSM) facilities; The Mission Control System (MCS) is also able to manage the required inputs and calculations to interface via on-board laser links using the Optical Communication payloads with e.g. EDRS
  • The Sentinels Mission Planning System (today part of the Mission Control System), supporting command request handling and the planning and scheduling of spacecraft/payload operations;
  • The specific Sentinel Spacecraft Simulators, supporting procedure validation, operator training and the simulation campaign before each major phase of the missions;
  • The Sentinels Flight Dynamics and the Collision Assessment Probability Systems, supporting all activities related to attitude and orbit determination and prediction, collision probability and avoidance systems, preparation of slew and orbit manoeuvres, spacecraft dynamics evaluation and navigation.
  • The Sentinels Master Key Management Facility (MKMF) and the Key Management Facility (KMF), responsible for the generation, storage, and distribution of cryptographic cipher keys and for their own monitor and control. Moreover, their Telecommand Encryption module is responsible for generating the Authentication Field for all telecommands and for encrypting the Packet Data Field of those telecommands identified in the Operational DataBase as requiring it.
• A General-Purpose Communication Network, providing the services for exchanging data with any other external system during all mission phases
• The Ground Station and Communications Network performing telemetry, telecommand and tracking operations within the S-band frequency. A single S-band ground station in Kiruna is being used throughout all mission phases as prime stations, complemented by additional TT&C stations as LEOP, backup and contingency stations, including terminals in Troll, Svalbard, Alaska, Inuvik and Australia.

Missions Scope and Operations highlights

The Flight Operations Segment and associated Services has been developed as an underlying infrastructure that is common to all Sentinel missions, augmented with any specific delta when applicable to one mission only (e.g. interfaces with EDRS only exist for Sentinel-1 and Sentinel-2, exclusion of security features for Sentinel-5P, etc). The FOS covers the complete satellite control and command operational chain, from planning to on-board activities monitoring and control, including notably the satellite HKTM processing, quality control, and archiving. The FOS also implements satellite health-checks and trends and aging analysis supported by Industry. Sentinels satellite House-Keeping data is available to external users through the external FOS services.

By design, the Sentinels FOS is able to support any operational phase of the missions’ lifetime, including preparation, Launch and Early Orbit Phase, Commissioning, Operational and Disposal Phase. So far, the Sentinels FOS has flawlessly supported 7 Sentinels LEOPs, including their commissioning phases and hand-over to EUMETSAT as applicable.

Interfaces & Context

The Sentinels FOS interfaces with a number of entities, in order to perform its operational duties:

• the Mission Planning, to receive from them the instrument and downlink schedules for uplink to the satellite and to send them the satellite orbit information from FOS.
• the POD service, for precise orbit determination services
• the Post-Launch Maintenance Service, for the maintenance of the satellite and associated anomalies tracing and resolution and as main interface with Industry
• the Systematic Production Service for HKTM retrieval following processing of the downlinked CADU data
• the Services Coordination element

Performance

The FOS is monitored based on a number of performance indicators including e.g.:

• Availability of Mission Control system
• Availability of S-band TM/TC acquisition service, time from satellite HKTM reception up to processing by the Mission Control System
• Time to detect and report any satellite Anomaly
• Time to detect and recover a Satellite Safe-Mode during any mission Phase
• Number of planned and unplanned FOS unavailability
• Provision of a Backup Control Centre located in ESA Kiruna
• Completeness and quality of the satellite HTKM archive
• Orbit maintenance and ground track keeping

The FOS sizing is defined based on key operational parameters, including e.g.:

• 24/7 operational support
• Number of satellites in operations
• Number of required passes per day
• Timeliness for re-planning requests
• Short- and Long-Term archiving data preservation

Infrastructure and CFIs

The Sentinels infrastructure features a mix of hosted infrastructure, concentrated at ESOC (main operations centre) and in ESA Kiruna (host of the Copernicus Sentinels Back-up Control Centre), together with some infrastructure externalised (e.g. archives). The FOS is planning the transition of a number of non-operational services to be hosted on-the-cloud.

Annex A: Traceability to EOF Specifications

Traceability is provided towards CSC Operations – ESA Framework – Ground Segment Specifications v1.1 issued 18/01/2021

1. HKTM will be initially pushed to the FOS in line with the legacy interfaces. The PRIP interface remains available in addition. ↑

2. GÉANT is a membership organisation representing the National Research and Education Networks (NRENs) across the continant of Europe. GÉANT also operates the pan-European GÉANT backbone network as well as providing a number of other services to the research and education community, in Europe and globally. ↑