Not quite the same as working for a Formula One team…
I previously spoke about the unique challenges involved in working in a secure environment, and working at my new place of work was different again, in so many ways.
I was now working for a sports team, where the aim is not about helping your company to succeed and prosper by keeping its customers happy; it’s all about winning.
Winning races, winning the drivers’ world championship and most importantly, winning the constructors’ world championship.
A different kind of team
What took me a little by surprise was the huge emphasis on winning, and how the whole team wins a race, not just the driver or the race team, but everyone.
When you are part of a race team, everyone contributes to the car and driver being successful.
The engineers and mechanics in the factory that make and assemble all the parts for the car.
The Aero department that design the car to make it fast and stick to the track, and the FEA department that design all the structural parts and suspension components.
But it is not only these departments that count, it is all the other departments as well, such as the restaurant staff, the security department, the IT department, marketing, the list goes on with everyone celebrating when we win.
I should, however, mention that although winning is the goal, there are other aspects of working for an F1 team that set it apart from working for a regular customer-based business which I was used to.
More about this later.
To continue this theme, my first day involved a glass of bubbly.
Not that I am so special as my arrival was toasted, but I had joined just after a race win, and the whole factory was invited to a post-race debrief, with a glass of something to celebrate.
Being an F1 fan myself, there were times when I felt a little starstruck seeing and being introduced to senior staff that I was used to watching on TV.
Glancing out of the window next to my desk and seeing an engineer walk past carrying a front wing.
Hearing strange noises coming out of the building opposite and discovering it was the mechanics practicing wheel changes.
These were all things that I was becoming accustomed to.
But there were many more occasions where the difference between working for an F1 team and having a regular job were apparent.
I mentioned the winning mentality earlier, and this was not just about our team striving to do better and improve; it also meant constantly looking at performance and where improvements could be made.
The pace never really stopped.
What happened during race weekend was presented in detail by a senior member of the race team, warts and all.
HPC behind Formula One
Race weekends brought their own level of intensity.
What happened at the circuit mattered to everyone back at the factory and, while you could enjoy being part of it, you also had to be mindful that you had a job to do.
Mine was working as an HPC support engineer.
This involved looking after the HPC systems supporting various aspects of car design and, increasingly, car setup and strategy.
My first few weeks were spent getting to know my colleagues, learning about all of the systems and processes, and also getting to know all the staff in the other departments that depend on HPC such as the Aero and FEA departments.
While I was getting up to speed (pun intended), the team were doing well on the track.
I was fortunate to join a successful team; we had two very good drivers, a fast and reliable car and during a race week there was a level of excitement and a little tension as to how well the race would go.
The IT department looks after the systems used across the organisation, including departments with specialist requirements such as Aerodynamics, FEA, strategy and car setup, as well as the race operation itself.
The race side has its own set of IT requirements, and a lot of technology is needed to support each race weekend, both at the circuit and back at the factory.
One crucial aspect is the communication between the race team at the circuit and the engineers and technicians supporting them from the factory.
With limits on how many people can be trackside, the factory-based teams provide additional analysis and support during practice, qualifying and the race itself.
The HPC function was a small team, so everyone had to know the systems well and be prepared to get involved when something needed attention.
BAU activities usually involved managing HPC-related tickets.
These could be anything from an HPC issue, failed or slow nodes, or a storage issue.
By far, our biggest ‘customer’ was the Aero department as, in terms of HPC, CFD had the largest requirement.
Their workflow involved pre- and post-processing, the solver and meshing, all of which place different demands on the infrastructure.
As an HPC department, we worked closely with the Aero development team to make sure aerodynamicists could run their work and that the underlying HPC environment was available and performing as it should.
Keeping HPC performing
With all the systems we had to manage, monitoring was a key activity.
We had systems in place to alert us when something failed, or even if a machine was simply slower to respond than expected.
The performance of the CFD environment was critical.
Formula One regulates the amount of computing resource that can be used for certain aerodynamic work, so getting the best possible performance from the available HPC resource really matters.
To wring every clock cycle of power out of the HPC, it had to be running in optimum condition, meaning all of the subsystems had to be healthy with no ‘slow nodes’ or any other issue impacting parallel performance.
Regular benchmark jobs were used to keep an eye on performance over time.
If a node had an issue that had not been picked up by the normal monitoring, a change in benchmark performance could still give us a clue that something was wrong.
Where we had hardware issues, we would work with the hardware vendor to get the problem resolved, whether that meant an engineer attending site or a replacement part being supplied.
Change control was also important.
Some of the systems were complex and specifically tuned for performance, so making an unrecorded change could easily make fault finding much harder later on.
Any significant change therefore needed to be considered carefully, with a clear plan for what would happen if it did not work as expected.
A firmware update is a simple example.
Before making the change, you need to know that you can return the system to its previous working state if something goes wrong.
It could sometimes feel rather onerous, but in an environment where performance and availability mattered so much, the discipline made sense.
During a race week the appetite for unnecessary change became even lower.
The last thing anyone wanted was for an avoidable IT issue at the factory to have an impact on the team at the circuit.
My time in Formula One also spanned the COVID era.
Like many organisations, we suddenly had to enable people to work from home wherever their role allowed.
With very little notice, the IT teams worked quickly to make that possible.
For our department, the priority was making sure aerodynamics and FEA engineers could still access the resources they needed and work as efficiently as possible away from the factory.
The network team increased the capacity needed to support remote working, while we concentrated on keeping the HPC resources available to the people who depended on them.
Once everyone was working remotely, our role was to make sure they could continue to use those resources and to fix or assist when they could not.
Back to consultancy
I have made my time with the team sound like one big rollercoaster of fun, but there were times when the work was hard and, for varying reasons, the hours were long.
If an IT issue had a wide impact, everyone would pitch in and help until it was resolved or there was a clear plan for resolution.
During those times, particularly in IT, everyone made themselves available, no matter what time of the day or night it was.
As I am in the twilight of my career, those long hours and pressure situations were starting to affect me in a way that my younger colleagues were not.
After a period of thought and deliberation, I eventually came to the sad and difficult decision that my time with this wonderful, exciting and successful Formula One team should end.
Although I did not feel ready to retire.
I thought, perhaps I should look for something not quite as high-speed, high-octane and high-adrenaline as Formula One offers.
Maybe see if I can go back to consultancy?
As it turned out, consultancy was exactly where I ended up.
I returned to Red Oak Consulting, bringing with me not only the experience I had gained throughout my career in HPC, but also everything I had learned from working in such a unique and high-pressure Formula One environment.
 It felt like the right place to bring that experience back into consultancy and use it to help customers with their own HPC challenges.
Â
Â
Paul Ingram
Senior Principal Consultant
Red Oak Consulting
