A sponsored thought-piece from CSIRO, Platinum Sponsor of the 2024 ATSE New Fellows Showcase and Awards Gala Dinner

Written by Elanor Huntington, Executive Director - Digital, National Facilities & Collections, CSIRO and Stefan Hajkowicz, Chief Research Consultant, Analytics and Decision Sciences, CSIRO

Generative AI – as exemplified by ChatGPT – burst into our collective imagination in 2023, triggering a flurry of GenAI theatre as well as genuine engagement. It’s part of a seismic shift in the nature and reach of AI. But not because I have an app that will remove the drudgery of writing emails, fail to write good Haiku and give me a recipe for a Mojito.

It’s what lies underneath – “foundation models”. These vast, powerful, fine-tunable and reusable models are what powers ChatGPT and its cousins. They’ve emerged as a new class of infrastructure. And we already depend upon them. 

Australia’s geographic sprawl has long underpinned our national appetite for infrastructure – largely to connect us through roads, train lines, ports and across the skies. We have an emotional attachment to this infrastructure because it brings us home to our families after work, it takes us across borders to loved ones at special times through the year, and it underpins the daily human interactions we took for granted until the recent pandemic, when we realised just how vital those connections are.

Perhaps less often, we think of the iconic research infrastructure that has underpinned each leap forward in our societies and each inch forward in each of our scientific careers. Everyday Australians may not know about infrastructure like the Australian Centre for Disease Preparedness in Geelong where we study pathogens ranging from COVID-19 to avian influenza and beyond, but many of them will know Murriyang, our iconic Parkes radio telescope, courtesy of the film ‘The Dish’ and our role in landing the first human on the Moon.

Personally, we each might have a piece of research infrastructure that shaped our careers. For Elanor, working on building and using a piece of major research infrastructure as part of the global research effort needed to figure out if we could build gravitational wave detectors was a transformative experience. It has shaped her whole career and worldview on infrastructure. Research infrastructure is about the convening power of a big problem and a big solution; it forces us to think 20, 30... 50 years into the future. So, what infrastructure should we be thinking about now to set us up for the next 20, 30, 50 years?

Today, a new class of infrastructure is emerging that is no less important to the connections we hold dear, but harder to see than a road or a train line, and more embedded in our lives than we might like to believe. It’s the digital infrastructure that sits behind technologies like Generative Artificial Intelligence, or GenAI, which we can see in platforms like ChatGPT. We call these ‘foundation models’. At the moment, it can be easy to dismiss the influence of ChatGPT as a novelty – but it’s also carving out roles in healthcare, the criminal justice system, education, and even our science. Foundation models, like our more traditional infrastructure, would support all Australian researchers as well as underpinning many vital international research partnerships, which lift Australia’s capabilities.

What makes foundation models so powerful? They transfer general model insights from original training data to highly diverse and specialised tasks. They’re a general-purpose technology that can perform wide-ranging tasks in practically all industry sectors. Access to an AI foundation model takes away one of the most significant blockages to AI creation and adoption, which is all the hard, expensive work associated with building an AI model for a specific task. AI foundation models give people who would not otherwise have the resources to do so the ability to create AI products and services. It’s already started and is accelerating at an eye-watering pace.

The emergence of AI foundation models as infrastructure opens up the possibility for Australians – industry, academia, community and government – to exert agency over one of the most consequential inventions of our time. In the 20th Century we built wealth and prosperity in Australia by investing in railways, airports, dams and highways. Digital technology is giving us a new economy. And it’s changing our lifestyles. In the 21st Century we can build wealth with AI foundation models. They’re a new and much needed type of infrastructure.

Read CSIRO's Artificial Intelligence foundation models report.