Mimer

The Mimer storage system is intended for workflows with a very high demand for IOPS, for example in Artificial Intelligence and Machine Learning applications.

Mimer is based on a specialised solution for all-flash from WEKA IO and consists of a 0.6 PB flash tier and a 7 PB tier of spinning disks.

The storage is primarily intended for use on Alvis, but will be available from other systems as well for example for data staging..

Applications for group storage are managed through the SUPR Storage rounds. The group storage is mounted under /mimer, and a compute project is required for access.

Flash and non flash

All allocations on Mimer are based on the total size of the resource (i.e. all-flash + bulk part). Since the size of the all-flash part is limited this means that there is no guarantee that all files are on the flash part at all times. As the all-flash part fills up, files will automatically be migrated to the bulk tier to free up flash space for new data.

As soon as data is accessed again it will however be staged back again, therefor make sure not to base benchmarks on a first run (if data is "old" on the system).

Mimer hardware

The storage servers are delivered by Lenovo, and consists of:

Mimer server

  • All-flash tier; 12 SR630v2 servers with:

    • 2 x Intel Xeon Gold 6338 32c 2.0GHz Processor, 384 GB memory
    • 2 x M.2 5300 480GB SSD (Mirrored for OS)
    • 10 x Intel P5500 7.68TB NVMe PCIe 4.0
    • 2 x Mellanox ConnectX-6 HDR Infiniband adapters
    • 1 x Mellanox ConnectX-6 Lx 10/25GbE Ethernet adapter
    • A total of 922 TB raw / 634 TB usable capacity
  • Bulk tier; 14 SR630v2 servers:

    • 2 x Intel Xeon Silver 4314 16c 2.3GHz Processors, 256 GB memory
    • 2 x M.2 5300 480GB SSD (Mirrored for OS)
    • 3 x 800GB NVMe PCIe 4.0 (for Ceph journal and database)
    • 1 x Mellanox ConnectX-6 100 Gb 2-port Ethernet adapter
    • 1 x Mellanox ConnectX-6 10/25 Gb 2-Port Ethernet adapter
    • Connected (2 servers to one JBOD) to a D3284 JBOD with 84x 14TB SAS HDD Drives
    • A total of 8232 TB raw / 6860 TB usable capacity

Mimer JBOD