Wiki (Help)

Orchestrator

How to use COPA ?

With COPA you can create Workflows which are composed of a series of processes that may occur sequentially, in parallel, with specified dependencies to enable one step to be executed after another, or a mixture of both.

Pre-requisites : log in to the MAAP platform with your credentials

1- Once logged, click on Tools in the menu, then click on Orchestrator. A new window is opened.

To create a workflow, click on create a new diagram at the bottom middle of the page

Use case : I want to download subset data from edav front-end and upload this data to s3

2- To see available algorithms you can click on the search bar Fin and Algorithm and then on the magnifying glass

3- Then if you click on start

4-Click on append task icon

Then the block will appear

5-To add an algorithm in the task search your algorithm, click in the task block and then on the algorithm

Then your algorithm will be displayed in the block

6-To add onother algorithm depending on your workflow use case tou can click on the task block and add another one with another algorithm with the same process (NB: the name of the algorithm to transfer data on s3 is now "transfer-from-copa-s3" and not s3-transfer as you can see on the screens below).

7-Then at the end of your workflow add click on the Append EndEvent block

8- Once it’s done fill the name of the worflow which must be in lowercase without special caracters and click on save to save the workflow.

9- Then this page will appears, click on validate Run Workflow

10- Algorithms configuration

Subset-data configuration :

Go on the MAAP front-end select your subset on dataset and paste Url in the subsetData parameter, then set the name of your subset in the filename parameter

S3-transfer configuration :

maap-scientific-data/shared/TOBE_COMPLETED

If you want to save your data on s3 complete the s3 destination parameter :

Example : maap-scientific-data/shared/myname/subsetdata

NB: if you don’t complete the pathor if you replace the beginning of the path “maap-scientific-data/shared” you will not be able to download data. Your path must start with maap-scientific-data/shared/.

11- Finally click on Valide and Run to run the workflow

12-

If you want to download your data you can use the maap-s3 script In jupyter or on your laptop

Notebook : https://repo.uat.maap-project.org/esa-common/demo-scripts/-/blob/master/maap-s3/maap-s3.ipynb

0 Comments