How to use COPA ? - Help
How to use COPA ?
With COPA you can create Workflows which are composed of a series of processes that may occur sequentially, in parallel, with specified dependencies to enable one step to be executed after another, or a mixture of both.
Pre-requisites : log in to the MAAP platform with your credentials
1- Once logged, click on Tools in the menu, then click on Orchestrator. A new window is opened.
To create a workflow, click on create a new diagram at the bottom middle of the page
Use case : I want to download subset data from edav front-end and upload this data to s3
2- To see available algorithms you can click on the search bar Fin and Algorithm and then on the magnifying glass
3- Then if you click on start
4-Click on append task icon
Then the block will appear
5-To add an algorithm in the task search your algorithm, click in the task block and then on the algorithm
Then your algorithm will be displayed in the block
6-To add onother algorithm depending on your workflow use case tou can click on the task block and add another one with another algorithm with the same process (NB: the name of the algorithm to transfer data on s3 is now "transfer-from-copa-s3" and not s3-transfer as you can see on the screens below).
7-Then at the end of your workflow add click on the Append EndEvent block
8- Once it’s done fill the name of the worflow which must be in lowercase without special caracters and click on save to save the workflow.
9- Then this page will appears, click on validate Run Workflow
10- Algorithms configuration
Subset-data configuration :
Go on the MAAP front-end select your subset on dataset and paste Url in the subsetData parameter, then set the name of your subset in the filename parameter
S3-transfer configuration :
If you want to save your data on s3 complete the s3 destination parameter :
Example : maap-scientific-data/shared/myname/subsetdata
NB: if you don’t complete the pathor if you replace the beginning of the path “maap-scientific-data/shared” you will not be able to download data. Your path must start with maap-scientific-data/shared/.
11- Finally click on Valide and Run to run the workflow
If you want to download your data you can use the maap-s3 script In jupyter or on your laptop