The MareNostrum 5, supercomputer in which Portugal, through the Foundation for Science and Technology, holds a 5% stake, financed by the Recovery and Resilience Plan, will enter in production phase.
Those interested in submitting expressions of interest to access the MN5 production phase must complete and submit the simplified form, available at expression of interest form, until September 9th.
Prior to this phase, 11 national users with prior experience were sent to the Barcelona Supercomputing Center, the location of MareNostrum 5, pioneering the testing and optimization of their projects on the two largest partitions of this supercomputer.
Thales Silva, a researcher in the Lasers and Plasmas Group at the Institute of Plasmas and Nuclear Fusion, part of the Instituto Superior Técnico, was one of the first to use the supercomputer, through experimental access still in pre-production.
At a time when this opportunity opens up to other projects, the researcher with 7 years of experience using high-performance computing, who has already used 10 different supercomputers in Europe, shares your testimony about MareNostrum 5.
Before this supercomputer, Thales accessed the predecessor MareNostrum 4, which it describes as its “favorite supercomputer to date.” In this machine, the highlights are the short time in which the computer is not accessible, the clear communication with users when there are problems, the agility and usefulness when it is necessary to resort to user support and the organization of the software stack”.
“All the systems I’ve used have responded well to most of these aspects, but few compare to MareNostrum 4. As for MareNostrum 5, my perception is that this quality has been maintained,” he says.
1. How would you describe your experience with Marenostrum 5 and what resources did you use?
The experience was very positive, especially considering it was an experimental access with the machine still in pre-production.
I used resources from the general partition (focused on CPUs) and the partition that uses accelerators (GPUs) for a total of 2 million core hours on CPU and 100 thousand GPU hours on GPU.
2. What are the projects for which you used Marenostrum 5?
Our primary goal was to test the OSIRIS code for plasma simulations. Our group has been a co-developer of this code for over 20 years, along with groups from the University of California, Los Angeles, and, more recently, the University of Michigan.
The code is used by dozens of groups and universities around the world and is recognized by the scientific community as the benchmark for plasma simulations. Therefore, it is important that we test its performance on new machines to ensure that it is utilizing all the features of the most modern architectures.
These tests give the OSIRIS community confidence in using the code on MareNostrum 5, and allow us to extrapolate expected performance to other systems with similar architecture and processing units, thus helping other OSIRIS users who are preparing requests for computing time on other computers.
OSIRIS also has a newer version that uses GPUs to accelerate the most demanding parts of the code. The opportunity to test this version, especially in a context where we can use hundreds of GPUs in parallel to verify the code's scalability, was one of the objectives we achieved.
Additionally, our project has a highly relevant scientific component in plasma physics. Plasma is the fourth state of matter, in which temperatures are so high that the image of an atom as electrons trapped around ions (nucleus) no longer exists: the particles are so energetic that there are no direct bonds between electrons and the nucleus. The Sun, for example, is in this state of matter. While it is not a common state on Earth, the truth is that more than 99% of the (visible) matter in the universe is in the plasma state.
Many fundamental questions remain unanswered regarding this matter, particularly how the energy contained in plasma is distributed among its various particles. We have collaborated on this topic with the University of Oxford, and we have performed large-scale simulations on MareNostrum 5 to corroborate theoretical results that explain these patterns in energy distribution.
Another scientific component in which we used MareNostrum 5 is related to the emission of coherent radiation by plasmas. Plasmas are disorganized, and the radiation emitted is not coherent (each particle emits independently of the others). However, there are processes that can cause plasma particles to emit high-intensity coherent radiation. Our group, as part of student Pablo Bilbao's doctoral thesis, is preparing a series of papers that connects the radiation observed on Earth with coherent emission by astrophysical plasmas.
Another parallel project, in which I used MareNostrum 5's resources, involves creating conditions analogous to those of these astrophysical plasmas that can emit coherent radiation in the laboratory. This way, we can better understand the connection between the radiation we observe on Earth and the coherent radiation emitted by these plasmas. A highly relevant aspect of this work is the combination of two technologies: ultra-intense lasers and high-performance computing.
3. What are the benefits of using this supercomputer's capacity for your projects?
Regarding the scientific part, we can run large-scale simulations for the projects mentioned above efficiently.
A unique factor was that MareNostrum 5 also allowed us to test, for the first time, the GPU version of OSIRIS on H100 GPUs. This was very useful for comparing the results and the pros and cons compared to the A100 GPUs that are more common in supercomputers today.
4. What improvements would you like to see implemented?
Maybe some small details, like showing how much compute time and storage the project has used or is using on the login screen.
There are other features (some commands that can be useful to check the computer status, how many nodes are idle in the partition, etc.) that are disabled for now, but I think this is because it is in pre-production.
5. What did you think of the application process and how do you intend to apply again?
The application was quite simple and convenient. MareNostrum 5 will probably always be at the top of my list in the coming years as the supercomputer I intend to use to prepare applications. The same also applies to Deucalion.
Having both computers available through national competitions is extremely important for our group, which requires these advanced computing resources for a significant part of our research. In the last RNCA competition, our group obtained three projects with computing time on Deucalion and MareNostrum.
6. Would you recommend this experience to other researchers?
Without a doubt, and I've even done it: Prof. Jorge Vieira competed on time for MareNostrum 5 in the last RNCA competition after I told him that I was using the computer during that pre-production period and that the performance of our code (OSIRIS) was excellent.
More information about the application for the MareNostrum 5 production phase at https://rnca.fccn.pt/marenostrum-5/
