Rocket is a heterogeneous HPC cluster that currently consists of 50 compute nodes, featuring almost 12000 cores, a bit over 5 terabytes of memory and 64 GPUs, interconnected by high-speed low-latency Infiniband networking. The cluster also utilizes two General Parallel File systems, which in total provide more than 10 petabytes of usable storage space.

This cluster is available to University of Tartu, and to third parties via ETAIS. After requesting and receiving an account, one can access the cluster via SSH at rocket.hpc.ut.ee or via OpenOndemand.

Overview

The main part of the Rocket cluster consists of:

  • 40 nodes with AMD CPUs (called ares 1-20, artemis 1-20)
  • 10 nodes with Nvidia GPUs (falcon 1 to 6, pegasus 1 and 2, firefly 1 and 2)
  • 2 head-nodes (login1.hpc.ut.ee, login2.hpc.ut.ee)

In addition to these nodes, there are a few GPFS filesystem servers which will provide fast storage for the entire cluster.

All the machines mentioned above are connected to a fast Infiniband fabric.

In addition to Infiniband, all aforementioned machines are also connected to a regular ethernet network for easier access. Machines are connected together with 1/10/25/40 Gbit/s Ethernet in order to provide fast access from these machines to outside of the cluster network, to the University central network and beyond, depending on necessity.

All nodes in the Rocket cluster are running the latest RHEL 9.

You can submit your computational tasks to the cluster using SLURM.

Ares 1-20 – high-density AMD nodes

  • 2x AMD EPYC 7702 64-Core Processor (128 cores total)
  • 1 TB RAM
  • 8 TB of local SSD storage
  • Infiniband:
    • 1x 100 Gbps link

Artemis 1-20 – AMD nodes,

  • 2x AMD EPYC 7763 64-Core Processor (128 cores total)
  • 1 TB RAM
  • 8TB local SSD storage
  • Infiniband:
    • 1x 100 Gbps link

Firefly 1-2 – nodes with NVIDIA H-200 GPUs:

  • 2x AMD EPYC 9575F 64-Core Processors (256 cores total)
  • 1.5 TB RAM
  • 28 TB of local SSD storage
  • 4x NVIDIA H-200 with 141 GB of vRAM each
  • Infiniband:
    • 1x 100 Gbps links

Pegasus and Pegasus 2 – nodes with NVIDIA Tesla A-100 GPUS:

pegasus.hpc.ut.ee

  • 2x AMD EPYC 7642 48-Core Processors (192 cores total)
  • 512 GB RAM
  • 1.6 TB of local SSD storage
  • 4x NVIDIA Tesla A-100 with 40 GB of vRAM each
  • Infiniband:
    • 1x 200 Gbps link

pegasus2.hpc.ut.ee

  • 2x AMD EPYC 7713 64-Core Processors (256 cores total)
  • 2 TB RAM
  • 15 TB of local SSD storage
  • 8x NVIDIA Tesla A-100 with 80 GB of vRAM each
  • Infiniband:
    • 9x 100 Gbps links

Falcon 1-6 – nodes with NVIDIA Tesla V-100 GPUs, purchase funded by Institute of Computer Science:

  • 2x Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (48 cores total)
  • 512 GB RAM
  • 5 TB of local SSD storage
  • Infiniband:
    • Falcon 1 - 3 – 2x 40 Gbps links
    • Falcon 4 - 6 – 5x 100 Gbps links
  • 44x NVIDIA Tesla V-100 GPUs:
    • Falcon 3 GPUs have 16 GB of vRAM.
    • Falcon 1 - 2, 4 - 6 GPUs have 32 GB of vRAM.

The following storage branches are mounted to all machines in the Rocket cluster:

  • /gpfs/space – 4.7 PB
    Declustered RAID based GPFS specific very high performance disk storage with a transparent Flash tier.

  • /gpfs/helios – 5.9 PB
    Declustered RAID based GPFS specific very high performance disk storage with a transparent Flash tier.

Pricing

The table below indicates the prices of our services for the structural units of University of Tartu and users outside of the University. For additional information please check our pricing.

HPC Compute servers
  • CPU 0.012 EUR/core h or
  • Memory 0.012 EUR/6GB/h.
  • GPU 0.5 EUR/GPU/h.

  • Memory usage is calculated by 6 GB segments.
    Price depends on the use of which resource is greater. Accounting of the use the computing servers is based on the amount of memory (with 1 unit being equal to 6 GB of RAM/h) and processors (with 1 unit being equal to 1 core/h) allocated to the user’s job(s).

  • When determining the order of starting compute jobs, jobs with higher priority are given preference.
Calculate your costs
Storage space
  • Storage 80 EUR/1TB/year
  • In case of 2x replicated data, 160 EUR/1TB/year.
  • A copy stored on tape costs 30 EUR/1TB/year.
  • Replicated + tape stored data costs 190 EUR/1TB/year.

  • Usable protocols are Samba, NFS, dsmc (TSM command line tool) and direct usage from HPC cluster.
    If looking for simpler access protocols, S3 is a better option.
Calculate your costs
Administrator’s hourly rate
  • Rate 60 EUR/h

  • Will be applied in case the desired software requires unusually long and complicated installation process.