A Global Research Project
“The technology is one thing, but the patients and the research, those are what it is all about”, Dan Søndergaard, HPC Chief Consultant at GenomeDK, says to start us off. “Our business is to provide capacity for both inventory and computing primarily to scientists and hospitals. We have a broad spectrum, but we specialize in the data-heavy arenas and specifically life sciences. It is all about data, both being stored and being used for computing and calculations”, he says and adds:
“We have been around for years now, and we quickly became involved in research. Around 2015, we started in the clinical arena. Around that time, we were a spearhead project for this type of collaboration in the north, and we still are. We cover the entire country, and we now have more than 700 users who access and share resources in a secure manner. We have a steady collaboration with AU and the government entity referred to as Central Denmark Region regarding clinical research, but also as it relates to actual production. That means that every day, we run patient data through GenomeDK which naturally creates large requirements for both data and operational security.
One Step into the Future
Thus, the work of Dan and GenomeDK in the server room is of great importance to us all. That being said, we need to take a look behind the curtain to grasp the perspectives and opportunities created by GenomeDK’s new Lenovo installation. “We are constantly expanding our capacity to meet the needs of our users. However, the great increase of the need for computing power means that we are challenged in the physical space in the server room. This is where the very dense installation makes a difference,” Dan Søndergaard explains.
“First off, it protects our future and by making this purchase alone, we have quadrupled our current capacity and we are prepared for as much as six of these. We have purchased one rack, one CDU, and one Cooling Distribution Unit. Furthermore, the combination of extremely compact hardware and the large cooling capacity in the CDU means that we can expand far into the future”.
Karl Hansen, HPC and AI Sales Manager at Lenovo has no doubt that Dan and GenomeDK’s choice of system has great potential: “They have taken it a step further in the technology they have chosen. Dan and the team have thought ahead and secured their future with this installation”, he says and adds: “The special aspect is the investment in a CDU, in water distribution. This provides a number of advances, such as there being no fans in the servers and that alone provides a saving of close to 20% of the power usage as well as an improved efficiency by using water cooling rather than air cooling of the equipment because GenomeDK has chosen a so-called DWC solution, an abbreviation for Direct Water Cooled. In such a system, water is used to cool all components and it takes up a fraction of the space of an air-cooled solution”, he says.
At the onset, very little is green about IT. As an overall industry, we must acknowledge that, but that just adds to the reasons for making an effort when we can. That is also the case for both GenomeDK and Lenovo. “In the old days, a rack cabinet would use maybe 5 kW and today, it uses 80kW or more which is why there is a great focus on reducing the usage,” Karl Hansen confirms. This is a fact which Dan and GenomeDK agrees with: “Thanks to a massive undertaking in preparations by our team and by our new facility management, we were able to ensure that we had enough water and power available. It is very high capacity,” he says and adds: “Our room truly does consume a great deal of electricity and therefore, we are doing everything in our power to minimize the energy consumption.”
Karl Hansen from Lenovo agrees: ”By using Lenovo Neptune, which is the name of this technology, there are several benefits. There is a better usage of electricity and the solution has a better performance in that consolidated operations preserves more power and today, electricity has become a critical expenditure. As an example: if you purchase a HPC cluster priced at DKK 30 million, it will cost approximately DKK 15 million in power supply for the entire lifespan of the unit.”
Good Things Come in Small Packages
In a time where things almost cannot seem to get too big, the HPC development is moving in the opposite direction. Clearly, as is evident in the pictures in this case study, the Genemo DK Lenovo installation is anything but small. However, measured by the HPC yardstick, it is quite modest and that is a good thing and very intentional. “We can no longer build compact servers with regular CPU’s without using water cooling. Merely 1 to 2 years ago, we could build a chassis in two U’s which is a rack unit of 4.5 cm with room for 4 server nodes. The new generation of CPU’s uses more than 500 watt and cannot be cooled by air any longer in such a compact unit. That has increased the space needed in the server rooms and furthermore, the expenditure on cables and the network has been challenging. Therefore, Dan and GenomeDK chose water cooling. This way, we can still create more compact and close infrastructures and it is also 3000 times more efficient to remove heat with water than with air. That is just a law of physics” Karl Hansen confirms.
Dan Søndergaard puts the way GenomeDK uses HPC IT into words: ”We receive several petabytes of IT each year and by law, that must be stored for 15 years. That results in ongoing large data migrations and maintenance. And we can handle that”, he says. That could point toward using vast amounts of equipment. But Dan has also thought ahead on this front: “We do things differently than in other HPC systems. We have never purchased a system, used it for five years, and then discarded it because we simply do not believe in that way of doing things. Instead, we ensure that we can expand ongoingly as capacity demands it. Just as we have done with our new system. That also means that our many users do not experience large changes or manual data migrations. Finally, it is also thinking long-term and more sustainable because the capacity does not sit around without being used.”
Already Getting Noticed
At Danoffice IT, the case of GenomeDK is something very special. This is purely because of the technical skills applied and partly because of the valuable gain to GenomeDK – but also in part because of the nature of the collaboration. “Water-cooled HPC systems are a critical focus area for us at Lenovo,” Karl Hansen says. “We have had our Neptune water cooling on the market for more than 10 years and we are truly delighted with the collaboration we have with Danoffice IT. They are our preferred collaborative partner with it comes to HPC. Our contact Palle Gram is truly doing wonderful work and we trust each other. This is a good example of people doing business with people. And we have already drawn attention to ourselves because we were able to deliver this water-cooled system to Dan and GenomeDK”, says Karl Hansen, HPC and AI Sales Manager at Lenovo.