HTC Execute Flavour

Many computational tasks in Earth Observation and environmental modelling apply the same processing algorithm independently across thousands or millions of input data items. These parallel tasks are best served by High Throughput Computing (HTC).

The European Weather Cloud (EWC) provides a centrally-managed EWC HTC Batch Processing service based on HTCondor, a specialized workload-management system designed for compute-intensive jobs.

The principal advantage of this EWC service is access to a much larger, resource pool than any single tenant could provide alone. Capacity is drawn from:

1. Capacity made available by EWC itself
2. Spare VMs contributed by EWC tenants for shared usage

EWC users can submit jobs from their VMs (i.e. those configured by the HTC Batch Processing Submit Flavour). Jobs are placed in the queue, matched to available resources, executed, and results are returned to VM where the job was submitted from.

This template is EWC users wishing to contribute their compute resources to the pool of resources as HTCondor Execute Nodes:

Functionality

The template is designed to:

• Configure a pre-existing virtual machine running RockyLinux, with private IP address, and a minimum recommended 32GB of RAM, as:
  • an HTCondor execution node, enrolled into the EWC centrally-managed pool of shared compute resources
  • a Docker runtime able to pull both public images from DockerHub and private images from the EWC Container Registry
  • a Singularity runtime, similar to Docker but with scientific computing and security in mind

Prerequisites

• Request authorization keys:
  • Write to EWC Support (support@europeanweather.cloud), making sure to provide your EWC tenancy name alongside your request for VPN, HTCondor and Harbor access.
• Verify the htcondor OpenStack Security Group exists in your EWC tenancy

  💡 You may create Security Groups via this EWC Community Hub Item if pre-required ones are missing.

Usage

Deploy via EWCCLI

⚠️ By default, deploying via EWCCLI is only possible from within your EWC private network. You may override by passing the --external-ip flag upon deployment. Caution is advised as this weakens security on the deployed VM.

1. Setup working environment

pip install ewccli

2. Configure access credentials

ewc login

3. Deploy

💡 To lean about EWCCLI deployment customization, checkout the EWC User Stories: I want to use the ewccli documentation pages.

ewc hub deploy htc-execute-flavour

Deploy via native tooling (Ansible)

1. Setup working environment

• Install git (version 2.0 or higher )

• Install python (version 3.9 or higher)

• Install ansible (version 2.15 or higher)

• If you plan to configure an existing VM, ensure it meets the minium requirements before proceeding:

  • VM Image: RockyLinux 8 or 9
  • VM Plan: 8 CPU cores, 64GB RAM, 30GB Disk
  • Network: Private
  • Security Groups: htcondor
  • Floating IP: Optional (not recommended from a security standpoint)

  Otherwise, provision a new VM with above specifications before continuing (see EWC Getting Started: Provision a VM for details).

2. Clone the repository

git clone https://github.com/ewcloud/ewc-ansible-playbook-flavours-and-provisioning.git

2.1. Change to the specific Item's subdirectory

cd playbooks/htc-execute-flavour

2.2. Checkout an specific Item's version

⚠️ Make sure to replace x.y.z in the command below, with your version of preference.

git checkout x.y.z

3. Download Ansible dependencies

💡 By default, Ansible Roles are installed under the ~/.ansible/roles directory within your working environment.

Download the correct version of the Ansible dependencies, if you haven't done so already:

ansible-galaxy role install --force -r requirements.yml

4. Specify the target host and SSH credentials

Create an inventory file, to specify address/credentials that your local working environment should use to connect to the target VM.

Copy into the file one of the two snippets below, and replace the placeholders (i.e. values enclosed in < > characters):

• To connect within the EWC private network

  # inventory.yml
  ---
  ewcloud:
    hosts:
      target:
        ansible_python_interpreter: auto
        ansible_host: <add the PRIVATE IP address of the target host>
        ansible_ssh_private_key_file: <add the path to local SSH private key file>
        ansible_user: cloud-user
        ansible_ssh_common_args: -o StrictHostKeyChecking=no
  

OR

• To connect from the public internet to a EWC public IP address

  # inventory.yml
  ---
  ewcloud:
    hosts:
      target:
        ansible_python_interpreter: auto
        ansible_host: <add the PUBLIC IP address of the target host>
        ansible_ssh_private_key_file: <add the path to local SSH private key file>
        ansible_user: cloud-user
        ansible_ssh_common_args: -o StrictHostKeyChecking=no
  

OR

• To connect from the public the public internet to a EWC private IP address

  ⚠️ This requires an SSH Bastion to be already provisioned within your EWC tenancy.

  # inventory.yml
  ---
  ewcloud:
    hosts:
      target:
        ansible_host: <add the PRIVATE IP address of the target host>
        ansible_ssh_user: cloud-user
        ansible_ssh_private_key_file: <add the path to local SSH private key file>
        ansible_python_interpreter: auto
  
  all:
    vars:
      ansible_ssh_common_args: >-
        -o StrictHostKeyChecking=no
        -o UserKnownHostsFile=/dev/null
        -o ProxyCommand="ssh
                        -o StrictHostKeyChecking=no
                        -o UserKnownHostsFile=/dev/null
                        -o BatchMode=yes
                        -W %h:%p
                        -i <add the path to local SSH private key file>
                        cloud-user@<add the PUBLIC IP address of the SSH bastion>"
  
  

5. Configure and apply the template

• Interactive Mode

  By running the following command, you can trigger an interactive session that prompts you for the necessary user inputs, and then applies changes to your target EWC environment:

  ansible-playbook -i inventory.yml htc-execute-flavour.yml
  

OR

• Non-Interactive Mode

  💡 To learn more about defining variables at runtime, checkout the official Ansible documentation.

  You can also run in non-interactive mode by passing the --extra-vars or -e flag, followed by a map of key-value pairs; one for each and every available input (see inputs section below). For example:

  ansible-playbook \
    -i inventory.yml \
      -e '{
        "headscale_preauthkey":"<redacted>",
        "headscale_login_server":"http://headscale.batchprocessing.eumetsat.ewcloud.host:80",
        "htcondor_cm_external_ip":"100.64.0.32",
        "htcondor_password":"<redacted>",
        "harbor_auth_token":"<redacted>"
      }' \
    htc-execute-flavour.yml
  

Inputs

Dependencies

💡 Upon execution, a SBOM (SPDX format) is auto-generated and stored in the VM's file system root directory (see /sbom.json).

Operation

Checkout the following how-to guides to learn about usage of the Item after setup:

• How to submit jobs