This repository provides access to the Checks by Monitoring (CbM) project code and technical documentation developed by the Joint Research Centre (JRC) to support Member States (MS). Here the Guidance and Tools for CAP (GTCAP) JRC group shares a CbM system layout and code examples to demonstrate how Paying Agencies (PA) can process and use Sentinel data to check aid applications for the common agricultural policy (CAP). The JRC CbM is based on a cloud infrastructure solution that is modular and built exclusively on open source components.
The technical description of the system components and the instruction on how to set up and use a CbM system are available in the JRC CbM TECHNICAL DOCUMENTATION.
A general overview of the system architecture and scope is provided in the JRC CbM GENERAL DOCUMENTATION.
JRC CbM ARCHITECTURE is made of two layers. The first is the BACKEND SERVER that provides the end-points to retrieve data and includes the physical infrastructure and the routines that generates the information used by the analysts. The second is the FRONTEND COMPONENT that is manipulated by the user and provides access to the data generated by the backend through standard Application Programming Interface (API) and the functions and tools to analyse the data.
In this repository we share the code developed for both the backend and the frontend, structured according to three levels:
- Setting up the infrastructure (target users: system administrators)
- Develop analytical functionalities (target users: analysts)
- Apply the functionalities (target users: final users)
The scripts and documentation are organized in the following sections/subfolders:
- api: Modules to build a RESTful API for CbM
- cbm: CbM Python library
- docker: Docker image files
- docs: Documentation pages with example codes
- ipynb: Jupyter Notebook examples
- scripts: Python scripts for signal processing (marker detection, parcel extraction routines and time series calendar view)
- tests: Test scripts for generic functionalities
In the framework of the Outreach project, an operational cloud infrastructure (based on CreoDIAS) has been created by GTCAP with the backend component developed and managed by JRC and offered to Member States to experiment the functionalities of the system. Here, MS can use dedicated API to explore and analyse Sentinel data extracted for their declared parcels using Python and Jupiter Notebooks.
A PA/MS that wants to developed and managed its own CbM system must have as a prerequisite:
- Computing resources
- Copernicus Analysis Ready Data (CARD) (Sentinel 1 and 2 data for the study area)
- Agricultural parcel data (typically, declared parcels from the Land Parcel Identification System (LPIS) and the Geospatial Aid Application (GSAA))
The first two requisites can be achieved using one of the five Copernicus Data and Information Access Services (DIAS) available (CREODIAS, WEKEO, SOBLOO, MUNDI, ONDA).
There are several steps to set up the core components for CbM that require different types of technical expertise.
- Setup server applications
- Docker (containerization system)
- PostgreSQL database with PostGIS extension
- Jupyter (interactive analysis and visualization environment)
- Restful API (intermediate layer to access and use Copernicus data and the database)
- Adding data to the database
- Parcels data
- CARD Metadata and other setting data
- Process Sentinel data to derive relevant information
- Parcel stats extraction routines
- Machine learning algorithms
- Analytical routines (e.g. markers detection)
- Analyzing and reporting
- Using Jupyter Notebooks to explore and analyze data
- Generate reports and other outputs to classify aid applications
This repository is open for contribution, particularly from PA experts. Please read "Creating a pull request" for details on the process for submitting pull requests.
- Guido Lemoine - glemoine62
- Konstantinos Anastasakis - konanast
See also the list of contributors who participated in this project.
This project is licensed under the 3-Clause BSD license - see the LICENSE file for details.
Copyright (c) 2022, European Commission, Joint Research Centre. All rights reserved.