Roco’s Beginner’s Guide to the Real Cost of HPC
Hello Again, Humans!
I’ve been having another educational adventure.
This time, Jorge has been teaching me about TCO, that’s Total Cost of Ownership, for anyone wondering whether it stands for Tea, Cake and Office Gossip.
Sadly, it does not.
Although I still think that meeting deserves a place in the calendar.
Now, Jorge has been leading me through Red Oak’s approach to TCO.
I am very much the curious robot with the clipboard, asking why the system with the smallest number on the quotation does not always deliver the lowest overall cost.
Apparently, buying the hardware is only the beginning.
My circuits were intrigued.
So, What Is Total Cost of Ownership?
When organisations compare new AI or high-performance computing systems, the first figure they usually notice is the purchase price.
That makes sense. It is right there at the bottom of the quotation, often in bold, doing its very best to attract attention.
But Total Cost of Ownership looks beyond what a system costs to buy.
It considers all the costs associated with owning and operating that system throughout its operational lifetime.
That can include:
- Power and cooling
- Software licences
- Storage and networking
- Installation and integration
- Support and maintenance
- People and specialist skills
- Upgrades and expansion
- Downtime
- The expected service life of the system
In other words, the price on the quotation is not necessarily the final answer.
The most important question is often not “How much does the system cost?” but “How much useful work can it deliver for that investment?
When a Bargain Stops Being a Bargain
Let’s imagine two systems can both run the workloads your organisation needs.
System A has the lower purchase price.
System B costs a little more.
At first glance, System A appears to be the obvious winner.
But what happens if it uses more power?
What if it takes longer to complete the same work?
What if it needs extra software, more support or an expensive upgrade much sooner than expected?
What if your organisation outgrows it before anyone has finished congratulating themselves on securing such a brilliant deal?
Suddenly, the lower-priced system may not look quite so economical.
It reminds me of those printers that appear to cost less than a family takeaway, only for the replacement ink to require its own finance agreement.
Jorge has assured me that TCO is not about automatically choosing the most expensive option.
That would not be strategic consulting.
TCO is not simply about finding the cheapest option either. It is about understanding which option delivers the best value for the outcomes the organisation needs.
Sometimes, the lower-priced system really will provide the best value.
TCO simply helps organisations understand whether the saving is genuine or whether some of the costs are hiding further down the road.
The Shiny Hardware Is Only One Part of It
I understand why people get excited about the hardware.
Processors. Accelerators. GPUs. Impressive performance figures. All those impressive specifications.
Very appealing.
But Jorge has been teaching me that AI and HPC infrastructure must work as a complete environment.
Powerful compute delivers limited value if the surrounding infrastructure cannot provide data quickly enough.
That would be like fitting me with rocket-powered wheels and then asking me to drag the office photocopier behind me.
Impressive engineering.
Limited progress.
Compute, storage, networking and software all need to work together. The system also needs sufficient power, cooling and support.
If one part of the environment cannot keep up, it can slow everything else down.
That may result in longer processing times, wasted resources and expensive equipment that is not being used to its full potential.
These problems are not always obvious when you are comparing specifications and quotations.
They tend to introduce themselves later, once the system has been installed and everyone has stopped admiring the boxes.
Start With the Work, Not the Shopping List
One of the most useful things Jorge has explained is that a good TCO conversation should begin with the workloads.
What does the organisation actually need the system to do?
How quickly does the work need to be completed?
How much data is involved?
How are those requirements likely to change over the next few years?
It can be tempting to begin with a list of hardware and ask suppliers to price it.
But if that list is based on outdated assumptions, incomplete information or an overly optimistic forecast, even a competitive quotation may lead to the wrong system.
A lower purchase price does not help much if the infrastructure struggles to deliver the results the organisation needs.
That is why Red Oak looks at the complete environment before recommending a solution.
The aim is not to buy the largest or fastest system available, but the system that delivers the right capability efficiently
It is to design an environment that suits the workloads, the budget and the organisation’s longer-term plans.
Even I know that adding more flashing lights than necessary is not a recognised procurement strategy.
I suggested it. Jorge looked at me over the top of his spreadsheet.
Ask a Better Question
When comparing systems, the first question is often:
“How much does it cost?”
Jorge has taught me that a more useful question is:
“What will it cost us to achieve the results we actually need?”
That leads to some important follow-up questions:
- What will the system cost to power and cool?
- Will additional software or licences be required?
- What support and expertise will be needed?
- Can the infrastructure grow as demand increases?
- Will upgrades cause disruption?
- What would slower performance or downtime cost the organisation?
- How long is the system expected to remain useful?
These questions help technical, procurement and finance teams look at the same decision from different angles.
They also help turn a technical recommendation into a clearer business case.
Instead of simply saying:
“This option has a more suitable architecture,”
you can explain:
“This option should complete our workloads more efficiently, cost less to operate and reduce the risk of an early replacement.”
Same recommendation.
Far fewer puzzled faces around the meeting-room table.
Can TCO Predict Everything?
No.
Even Jorge cannot predict the future, and he is extremely clever.
Energy prices can change. Workloads can grow. New technology can emerge. Business priorities can shift.
Humans can also change their minds halfway through a project, which I understand happens occasionally.
A TCO assessment is not supposed to predict every future expense down to the last penny.
Its purpose is to make sensible assumptions, identify costs that might otherwise be overlooked and compare different options on a fairer basis.
It is better to examine the likely costs before making the investment than to discover them one by one afterwards.
Hope is a wonderful thing.
It is less useful as a budgeting method.
Look Beyond the Price Tag
My main takeaway from Jorge’s TCO lessons is this:
Do not judge an AI or HPC system by the purchase price alone.
Look at what it will cost to deliver the performance and results your organisation needs throughout its useful life.
The right solution may not be the cheapest one on the first day.
It should, however, provide the right performance, operate efficiently, work with the rest of the environment and support the organisation as its requirements develop.
I am still learning, so I will leave the detailed modelling and calculations to Jorge.
I will continue asking questions, taking notes and trying not to become distracted by spreadsheets with too many tabs.
Still wondering what might be hiding behind the price tag? A good TCO assessment can help uncover the full picture.
This TCO adventure has really caught my attention.
Understanding how organisations can make better decisions and achieve their goals more efficiently has made me want to learn much more.
It has also taught me that there is more to an investment than the first number on a quotation.
I think I’ll ask Jorge to help me explore this fascinating world further in future blogs.
There is so much more to discover about how business decisions, technical challenges, cost analysis and new ideas all come together.
Jorge has also explained that TCO is only one part of the story.
Other areas, such as analysing workloads to improve efficiency, understanding the sustainability impact of HPC and AI, and exploring new approaches to technology, can help organisations discover even more opportunities.
My clipboard is ready. I have a lot more questions to ask.
I’m off to ask Jorge whether Tea, Cake and Office Gossip can be included in the next TCO calculation.
Until next time!
Roco
Roco
HPC Whizz, Professional Pixel Wrangler
Red Oak Consulting
