Prof Penny Endersby has been chief executive of the UK Met Office since December 2018.
She took the reins at the UK’s climate and weather service after more than two decades working in the science and technology department of the Ministry of Defence.
Endersby has led the Met Office during a critical period which has seen record-breaking heat in the UK, an intensification of extreme weather around the world and a resurgence in attacks on climate science.
At the same time, advances in artificial intelligence (AI) have started to transform climate modelling and the Met Office has switched on a cloud-based “supercomputer” dedicated to improving weather and climate science.
- On how working on defence is like climate change: “There are more scientific parallels than you would think. Armour modelling is computational fluid dynamics modelling, like weather modelling.”
- On her previous interest in climate change: “I am naturally a sort of geeky, evidence-based person. I actually had kept 10 years of rain gauge records in Wiltshire – just because I was interested – before I ever thought about this job.”
- On how the Met Office informs UK climate-change preparations: “Our core bit is on preparing other people to make the decisions on what to expect. So we do the science of the climate projections that says: ‘Where might we be in 2030, 2050, 2100 under different emission scenarios?’”
- On the aspect of UK climate change that concerns her most: “The one that probably keeps me awake at night is the flash flooding – the surface water flooding from very localised torrential rainfall events, because those are the hardest to model.”
- On the Paris Agreement’s 1.5C temperature limit: “It is theoretically possible we stay within 1.5C, but it’s going to require action that’s never been seen and doesn’t look like it’s coming.”
- On geoengineering: “There’s nothing regulating [it] globally. So other people may do it – whether we advocate [for] it or not. So, we do think it is the right thing to do to understand what the impacts of it could be [so as] to be able to detect it if other people do it.”
- On carbon removal techniques: “The best way to get carbon out of the atmosphere is still a tree…The next easiest thing is direct carbon capture…The other techniques – I think they’re worth investigating, but they’re not going to be available at scale in the times that we need.”
- On budget cuts to the US weather and climate service: “I think the actual impact on funding, so far, has not been as severe as some of the news stories have said – because their role is just as fundamental to the operation of the state as ours is.”
- On the reach of the IPCC’s big assessment reports: “I think, in many ways, our best hope now lies in the global financial system. They’re not very altruistic, but they are very rational and they do use the best evidence… And, if the money sends them into different investments or different insurance strategies – that is still going to be based on the evidence that comes out of the IPCC.”
- On the evolution of climate misinformation: “I think the climate data is now so stark, that anyone who looks at the data at all can see that we’re in unprecedented times. [But] what has happened, to my grief and distress, has been people now attacking the trusted sources of data. And in the UK – that’s us.”
- On how the Met Office deals with misinformation: “We have had to become – per force – experts in countering misinformation and disinformation, and, really, to an extent, quite thought-leading in government and in convening cross-government networks to deal with this.”
- On social media attacks on the Met Office: “The aggression that some of our media-facing people face online [and] the really vile hostility which often quite junior staff are dealing with – that definitely is something where we have to take care of and look after people.”
- On personal attacks on social media: “It can be painful, but you really have to rise above it. And when it’s a woman, there’s always a generous salting of misogyny in there as well.”
- On claims about the inaccuracy of Met Office temperature readings: “Just to be clear, the claims are baseless…Everything we do is to the required standards.”
- On potential applications for AI in climate science: “AI for climate change is relatively new – and the challenge is always, what’s the training data set? Because you don’t have the training data set for climate that hasn’t happened yet.”
- On the Met Office’s work with AI: “Our strategy is to go for the best blend of conventional and AI modelling – but we’re still working out what we think that best blend will be.”
- On the Met Office’s first cloud-based supercomputer: “It needs to be [in the cloud], because the amazing, fantastic, wonderful data that we have – [and] on which all of these products and AI is built – is now so big you can’t move it. We have about half an exabyte of data.”
Carbon Brief: Hi Penny. Thanks for joining us. To start off, I wanted to ask: previous to becoming the chief executive of the Met Office, you worked for the science and technology arm of the Ministry of Defence. How did that role prepare you for your current position – and in what ways is working on climate change like defence?
Penny Endersby: I think the whole of that 25-year career as a scientific civil servant prepared me for my current position. And I started off being a researcher myself in armour and explosives, and then leading scientists in larger and larger chunks. My final role was as head of the cyber information division of the Defence Science and Technology Laboratory (DSTL). I was on the board of DSTL as well.
There are more scientific parallels than you would think. Armour modelling is computational fluid dynamics modelling, like weather modelling. I was responsible for big data and AI. I was responsible for space. So, there was lots of crossover. But the main thing was leading the experts and the people who are passionately committed to making lives better through science. That was the biggest crossover with what I do now.
CB: Why did you want to work for the Met Office?
PE: I have to say being chief executive of the Met Office was completely my dream job. I had actually applied to be chief executive of DSTL, very much on a punt, not long before. I got further than I expected – I got down to the last four, but I wasn’t successful. I had good feedback that went [along the lines of]: “Yeah, try again sometime.”
And then the Met Office [job] came up, and it had just had everything I wanted: a mission that I really cared about, doing something valuable, the scientific content, staying a civil servant and working for the government. [And] not being in London, because I am a great lover of the country. I moved from Wiltshire to Devon to take this job. [It was also] a promotion.
It just was the complete package. I gave getting the job everything.
CB: Why did you start becoming interested in climate change? I don’t know if you remember a particular moment or event?
PE: I was interested in [the] weather [and] the natural world from childhood. So I have always been a naturalist. I am naturally a sort of geeky, evidence-based person. I actually had kept 10 years of rain gauge records in Wiltshire – just because I was interested – before I ever thought about this job.
Although I was concerned about climate change and I knew the basics, I didn’t really study it until I got to this job and then was leading the organisation with the Hadley Centre [the Met Office’s climate research centre] in it, with amazing climate scientists and amazing climate science. So that seven years has been a journey of building my knowledge and my expertise.
CB: The Met Office has a mandate to help people make decisions and stay safe and thrive through its weather services. So could you just tell us a little bit about how the Met Office is equipping citizens for the current and future climate?
PE: You are right, that is our purpose – helping you make better decisions to stay safe and thrive. And it is weather and climate.
So on the “staying safe” side, that goes from everything from severe weather warnings in the shorter term, through to [longer-term] making decisions about what flood defences you are needing in the future. And that’s not [directed at] citizens – that’s [directed at] policymakers.
And on the “thrive” side, as well, it might be as simple as, “am I going to go for a run or hang washing out?” [when it comes to] weather timeframes. On climate timeframes, it is about making sure that we have got built infrastructure that enables us thrive in a changing climate, whether that is houses that don’t overheat or green spaces that cool down our urban centres – all those things.
CB: And in terms of preparing for that infrastructure, could you explain a bit more about how the Met Office actually does that?
PE: So our core bit is on preparing other people to make the decisions on what to expect. So we do the science of the climate projections that says: “Where might we be in 2030, 2050, 2100 under different emission scenarios?” [We ask] what does that look like globally and in the UK? We need to know that for our food security and border security and energy security – and nationally, in detail, in the UK.
And then, it is other people who will take that information and decide what to do with it. So, on the adaptation side, we’re really informing other people’s decisions.
CB: Other people being the government…?
PE: It could be local authorities. We have local authority climate dashboards for local authorities to look at how climate change is going to [impact them]. I did notice that Bermondsey, where we are right now, is right at the peak of the bit of London that is expected to be under water – the floodiest bit of London going forward and the hardest to protect.
And then it could be big national decision-makers. I’ll give you a completely different example. If we’re going to have a renewable energy system in the future, the weather we [are having] today is the worst possible weather for renewable energy. It could be cold as well, but it is dull and still. So, [there is] not much wind, not much solar. How long could that wind drought go on and how long could it persist when it’s also cold and there’s a high heating demand? So, we do the modelling that helps the National Energy System Operator plan for how much capacity they need in different weather scenarios and climate scenarios in the future.
CB: What aspects of current or projected climate change in the UK concern you most?
PE: I think the biggest concern is the flooding. And it’s all the sorts of flooding – [including] the coastal flooding from higher sea level [and] the river flooding from extreme rainfall.
The one that probably keeps me awake at night is the flash flooding – the surface water flooding from very localised torrential rainfall events, because those are the hardest to model. The smaller the scale of a phenomenon, the more difficult it is for us to model well in advance.
And yet we know – and we have seen – we have had a few very near misses in this country with the kind of things that affected Valencia or Germany, or that terrible Texas event – very rapid river rises from torrential downpours.
And it’s not just [about the Met Office] getting the rainfall [warnings] right. Our managing director for this area described [these preparations] as a “team sport”. So, we have to get the rainfall right, the hydrological people, jointly with us, have to get the flooding right, the whole of the response system has to respond to understand what’s going to happen. People have to get out and often in maybe only a couple of hours. Whereas for a great big winter storm, we might be giving seven or eight days notice.
CB: Thinking a bit more globally, a new report from a research group based at [the University of] Exeter suggests that a climate tipping point for warm water corals has already been crossed. What does the prospects of these tipping points mean for the work of climate scientists?
PE: It presents us with lots of new challenges, I think.
If there was one thing that shocked me most, going back to the beginning of my time at the Met Office, it was just how long we have known about the fundamentals of climate science – like, 150 years. I had not realised that our understanding of the greenhouse effect and the rough idea of climate sensitivity went back so far. And we have been really well able to understand and project that, really for my whole lifetime. The good climate modelling goes back to about 1970.
But tipping points changed the whole equation for climate science. And, of course, we’re only just beginning to observe them. So, there, we don’t have the track record of projecting it and checking back against what’s happened in reality. So they’re some of the hottest topics – I dare say you might come on to the AMOC [Atlantic Meridional Overturning Circulation]. But all of those areas are very, very active areas of research and areas where the science is not so settled.
CB: The 1.5C warming threshold of the Paris Agreement is looking more fragile than ever. 2024 was the first year-long breach of the limit. How likely is it, in your opinion, that [the rise in] temperatures are limited to 1.5C? And when do you think that could happen?
PE: Well, it is theoretically possible we stay within 1.5C, but it’s going to require action that’s never been seen and doesn’t look like it’s coming. So, we think the opportunity to stay within that first Paris threshold is vanishingly small now – and, if we continue at the current rate, we have only got a few years to go before we cross it long-term.
And, actually, the Met Office has done quite a bit of work looking at how you identify that threshold without waiting for 10 years of averages to go: “Yes, [the limit] was [breached] 10 years ago.”
So then we are into, well, how far can we limit it? Because obviously it’s not a cliff edge. That’s where we think – and I still think – that’s where the more dangerous impacts of climate change kick in. But between 1.5C and 2C – there’s a huge difference. And at 2C, 3C [and] God forbid 4C, all of those impacts multiply. So, it’s how we stay as close [to 1.5C of warming] as we can.
CB: There are growing calls for solar radiation modification and other forms of geoengineering to be considered to tackle climate change. I wanted to get your take on geoengineering as a climate strategy.
PE: The Met Office doesn’t take any particular stance on geoengineering. I need to make it really clear – because we get lots of conspiracy theories – and we do none. We have some very limited modelling to understand what it might look like and what it might do.
I think I might like to draw a distinction between my personal view and the Met Office here. But the only form of geoengineering that actually solves the problem is taking the carbon back out of the atmosphere. Solar radiation modification – it is a masking technique. It doesn’t stop ocean acidification. And once you start, you’ve got to go on, because if you stop, you can get catastrophic, very rapid, catching up.
So, in as much as we advocate anything, it would always be the mitigation techniques [actions that reduce emissions of greenhouse gases] we already know.
Having said that, there’s nothing regulating [geoengineering] globally. So, other people may do it – whether we advocate it or not. So, we do think it is the right thing to do to understand what the impacts of it could be [so as] to be able to detect it if other people do it and understand what they might be doing. It will be a political decision whether it’s a last resort thing to do.
CB: You mentioned carbon removals just now. Scientists at a recent conference on climate overshoot stressed that the 1.5C goal, if breached, needed to be “met from above” with the help of these technologies that remove emissions from the air. How feasible is that, in your view?
PE: The best way to get carbon out of the atmosphere is still a tree. So, some of these are nature-based solutions. And, then, the next easiest thing is the direct carbon capture – so, catch [the carbon] where it is, don’t try to get it back.
The other techniques – I think they’re worth investigating, but they’re not going to be available at scale in the times that we need. It is an entirely good and valid topic for research, but it shouldn’t be a substitution.
CB: Changing topic a bit. The US government has attacked climate science and is cutting national weather and climate services, including access to satellite data. How is this impacting both weather forecasting and climate research at the Met Office?
PE: So we retain a really close collaboration with NOAA [the US National Oceanic and Atmospheric Administration] who are important partners for us. We continue to have access to all of those US satellites that are gathering data. We exchange that through the World Meteorological Organization’s (WMO’s) data exchange.
And the EUMETSAT is the way we gather European satellite data – and the UK is a part of that, and that is all still exchanged.
I think what we are finding is that any government – irrespective of its political stance – needs the fundamentals of what a national meteorological agency can do. Everybody needs a weather forecast, everybody needs warnings, everybody needs aviation, transport, everybody needs defence.
And, so, there’s certainly been a lot of instability in NOAA, while these things work through. And, obviously there’s a government shutdown at the moment. But I think the actual impact on [NOAA] funding, so far, has not been as severe as some of the news stories have said – because their role is just as fundamental to the operation of the state as ours is.
CB: And, a secondary question to that is, are these events or geopolitics impacting the morale at all at the Met Office? And, as a boss, how do you address that, or try to mitigate that if so?
PE: I think the general Met Office staff are really very interested in their science and the mission and not so politically focused. I spent a lot of time thinking about what are the things that we do that will have value under any government and into the future and how we make sure that we can continue to deliver our great services to the government. And we do that with our executive and board.
What does impact morale is some of the misinformation we’ve seen, where we get people attacking the integrity of our observations or the integrity of our scientists. The aggression that some of our media-facing people face online [and] the really vile hostility which often quite junior staff are dealing with – that definitely is something where we have to take care of and look after people.
CB: In the UK, we’ve seen the Conservatives and Reform describe the UK’s net-zero target as “arbitrary”. And, in the US, we’ve seen the president describe climate change as a “con”. So, I wanted to ask you, do you feel that the Met Office and other influential climate science centres have a responsibility to publicly rebut or respond to these claims?
PE: No, absolutely not. The Met Office is a civil service organisation, so we have a very strong mandate to remain impartial and serve the government of the day. We have a government that is really committed to net-zero and being a green energy superpower and we will do everything we can for that. If a democratically elected government comes with a different mandate, our role is to provide that government with the best science to make the best policy decisions they can. It is not to tell them that their policies are wrong.
CB: I want to talk a bit about the IPCC [Intergovernmental Panel on Climate Change] now and the seventh assessment cycle. The IPCC has appointed five Met Office scientists to its newest cycle. What would you like to see the cycle focus on?
PE So I’m not sure I’m really the best person to answer this. Clearly, they are going to be focused on what happens between and above 1.5C. How do we replace that Paris threshold if it has gone? And therefore, increasingly, the adaptation [topic].
Where I actually get more involved is, as a delegate to the WMO and on the Early Warning for All [initiative]. And obviously, the more extreme climate-driven weather events we see, the more crucial early warning is to protect populations. And that will come out of the projections from the IPCC.
CB: I wondered if you had any thoughts on the IPCC’s big assessment reports. Is this model the best way for it to be effective? Or do you think there’s a better option?
PE: I do firmly believe that good decisions have to be based on good evidence. And the IPCC is as good a gold standard as you could have for benchmark evidence in any field.
So that whole process of collecting all the evidence from across researchers across the globe, factoring in what’s effectively a multi-model ensemble, looking at the error bars coming to consensus – it is too slow, right? Because consensus always lags. What the best guess would be – the consensus was always behind it. But I think it is an important model. I’m reminded of Simon Sharpe’s book Five Times Faster [which states that] it is not just [about] the scientists, it is then all the other, the other responses, as well, that need to respond.
I think, in many ways, our best hope now lies in the global financial system. They’re not very altruistic, but they are very rational and they do use the best evidence. They are actually not remotely interested in the politics, because they will look at where the money sends them. And if the money sends them into different investments or different insurance strategies – that is still going to be based on the evidence that comes out of the IPCC.
CB: After the US pulled its officials from attending the last IPCC meeting in China, how could a reduced US contribution impact the work of the IPCC in the upcoming cycle?
PE: I really can’t speak to that. I don’t know enough about it to give you a sensible answer.
CB: All right, let’s come on to misinformation. You mentioned it already. To start off: how is climate misinformation changing in your view?
PE: I’ve really seen this change in the course of this job. I think I was naive when I took this job. One of the things I said in my pitch for the job was: I believe I will be in post in the period when the person in the street in the UK becomes aware of the impact of climate change.
And I think I was right – we have had the first 40C and extreme wildfire events, extreme floods. People have seen it. But, I thought that that would lead to a rational response of people going: “Oh, gosh, they were right all along – we need to do something about it.”
That didn’t happen. What I have seen, therefore, is it was still more or less possible to go “meh, you’ve got the science wrong and you don’t really understand it” seven years ago.
I think the data is now so stark, that anyone who looks at the data at all can see that we’re in unprecedented times. [But] what has happened, to my grief and distress, has been people now attacking the trusted sources of data. And in the UK – that’s us.
We see increasingly statements about all kinds of rubbish, everything from, “you’re hiding the sensors in aircraft jets exhaust to show fake heating – you’ve gone back and changed the past” [to] “you’re measuring temperatures on the ground instead of properly in the Stevenson screen”. [There are] just a raft of things that take us a lot of time and taxpayers money to rebut.
There’s a name for this law and I’ve forgotten it [Brandolini’s law] – but there’s a law that says that, basically, misinformation can be produced in seconds, but takes days to rebut. And this is very, very true.
And, so, we have had to become – per force – experts in countering misinformation and disinformation and, really, to an extent, quite thought-leading in government and in convening cross-government networks to deal with this.
And although I guess I’m speaking to the converted and I’m offering you some rebuttal. That isn’t the best way; the people who firmly believe these things, they’re not remotely interested in your rebuttal.
The best way to maintain the extraordinary trust the Met Office currently enjoys is actually to be putting out the good stories. Here is the plethora of amazing ways we collect weather observations, from deep ocean buoys to space to marine gliders to measurements in the Arctic – whatever it is. Put out the positive stories, alongside the people stories. At your weather station – wherever you are in the UK – some expert person from the Met Office comes along and checks all the instruments and takes them back for calibration and makes sure the grass is the regulation height and assesses the surrounding area to make sure that things aren’t encroaching and so on.
Putting out those positive stories of how we do it, is a better use of our time and energy and more effective, we think, than rebutting the people who are not acting in good faith.
CB: My next question was going to be about your strategy for dealing with online climate misinformation. And you’ve talked about focusing on the positive story…
PE: Preinoculation in misinformation terms.
CB: I wondered if there was anything else you could share about your strategy and perhaps how it’s evolving as climate misinformation is?
PE: The other thing we found is it’s really useful to get independent voices in. So now, when we do put out stories, particularly when we put one up that we know it’s going to be a red rag. [For instance] it has been the hottest UK year in history. It has been the warmest summer. This was the first year that was above 1.5C for the average globally – which was something we forecast, by the way. We also do the WMO state of the climate and the annual to decadal five-year projections.
Whereas we might have just put those out with the WMO, now we tend to pre-share that information with other trusted parties – whether that is in academia or the Royal Met Soc [the Royal Meteorological Society], or whether it is NOAA or NASA or whoever – and have quotes nicely lined up for journalists, so that they can take them. That kind of independence is useful. I think all the UK scientific bodies are looking at how we can strengthen that network across government, so that we can speak to our areas of expertise when they cross over with other people’s.
CB: I wanted to ask you about AI-generated content as well. Is that something that you’ve seen specifically?
PE: Obviously, we do lots of good work in AI, which we might also come to.
CB: I was talking still in the context of climate misinformation – do you have a strategy for addressing that particular type of content?
PE: Yes. We have even seen Met Office deepfakes. So our presenters [as] deepfakes put out misleading information. And I’m not sure we do have a strategy for this.
The other thing we do, but it’s not so much for the AI, is deal with the clickbait. You know [for example]: “Exact date UK to be wiped out by a wall of snow.” And we do put quite a lot of stories out going: “Have you seen a thing that goes, it’s going to be a heatwave and a wall of snow?” [We] try to help people understand how to tell [a] good source from a bad source. But the level of protection you have, legally, against those things is not very strong.
CB: And striking that balance between, as you were saying earlier, responding to certain claims, but not giving them more air…
PE: We don’t want to make them more salient.
CB: How do you judge, essentially, when it is worth a public response and when it isn’t?
PE: Partly on how much is in its echo chamber versus where it has widened out. Generally, we find that within the echo chambers, it is just not worth a public response. The Met Office has a million followers on the main social media platforms and we have people putting out things who have 20, so if you rebut the 20[-follower] person with your million, everybody sees the 20-person story. That’s not the right thing.
And we have had to change our blocking policy as well, which is a shame, because we had a really [light-touch] blocking policy. We only blocked the obscene and threatening, but we found that our big audiences are being used to gain a platform for misinformation, particularly around geoengineering. And we have had to say we can’t, we can’t live with that. So we block more liberally than we did.
CB: You already brought up the attacks we’ve seen on the accuracy of Met Office temperature readings and data. I wanted to ask, have you been surprised by those stories and what is your general response to those claims?
PE: Just to be clear, the claims are baseless. We’ve dealt particularly with the ones that say the WMO is critiquing our data. You may have seen now the statement from the secretary general of the WMO going “we have the highest confidence in the quality and validity of Met Office data”. So, that was one [claim] where we did source a deliberate rebuttal.
But, generally, I’d just like to reassure you that everything we do is to the required standards, the WMO to ISO9001 [quality management] standards, assured externally and internally. And then the [Met Office] independent public weather service customer group also assures the quality of the science and the outputs [and] the accuracy of the warnings.
The worst thing is if people start to believe [the claims] – and then they don’t take action on warnings when [they are] there to protect their lives.
CB: You have been the subject of quite a lot of attacks on social media. I wanted to ask how you manage that on a personal level?
PE: It can be painful, but you really have to rise above it. And when it’s a woman, there’s always a generous salting of misogyny in there as well. I have tended now not to go and read these things, because they prey on your mind and there’s nothing you can do. We do monitor for actual threats, which we would have to act on.
CB: And you mentioned a lot of colleagues as well were facing [attacks on social media]?
PE: Yes. The personal attacks tend to be on the most senior people, [on] me or the chief scientist [Prof Stephen Belcher]. But, obviously, the person who is managing our social media feed still gets a mouthful of abuse when they’re reading and responding. It is not aimed at them personally, but they are still a human being – and maybe not a very senior or experienced one.
CB: And moving on from social media, I wanted to talk a bit about the media more generally. So have you seen a change in the way that the media covers climate change?
PE: It is around the fringes. I expect you know that we’ve just signed a new partnership deal with the BBC, who are extremely rigorous in how they cover climate change. And one of the things…we’re really excited to work on them. Last time we worked with the BBC, Verify [a service where journalists share their evidence-gathering] didn’t exist. They, too, have had to invest a whole lot of effort in how you counter misinformation – and they have some really leading thinking. We’re excited to work with Verify on weather and climate information.
But, I think it’s the “wilding” of the social media landscape that’s changed.
CB: What climate change topics would you like to see the media cover more?
PE: I think there is lots of coverage across all the topics. I can’t say the information isn’t out there. It is how it is picked out and the way that our social algorithms segregate it. [For] anyone who wants to find out, there is good information out there on almost any topic – because media is always looking for stories, right?
The problem is once you’ve moved yourself into a bubble where you don’t want to see it. And you can see [this] if you watch US media coverage of a weather disaster. Even when it’s highly likely to be climate change-related, they don’t say so. The people presumably watching those channels don’t make that link.
CB: I wanted to talk a bit about AI in a different way now. So, how is AI transforming climate forecasting at the Met Office?
PE: The Met Office has been working with AI for several years – and before the big generative AI shift. We do that in many aspects of our work. AI for climate change is relatively new – and the challenge is always “what’s the training data set”? Because you don’t have the training data set for climate that hasn’t happened yet.
But we are using it to look at some of the opportunities. For example, in what we would call downscaling, which is a technical term, but basically going from a coarse-resolution model – which climate change tends to be, because we need to run them over such a long time and they’re very computer hungry – to see if we can use AI to replicate something that is more like the fine scale of our weather models.
And, more generally, we are enthusiastic, but not naive, adopters of AI, I would like to say. We do now have our own AI weather model, FastNet, which we developed jointly with the Turing Institute. We’re looking at the opportunities for AI and our products and services – so could you fuse it with, say, transport data to say: “Well, the weather’s here and the trains or the planes are there – where are the impacts going to be?” And go straight to the results.
And we use it quite widely in our everyday work as well. So, increasingly, I think 1,000 people in the Met Office are using [Microsoft] Copilot and 97% of people who have a licence use it for just making our everyday work more efficient.
I expect you’re going to come on and ask about the challenges of validation and trust, and if you’re not, I’ll go on that way.
CB: Can you tell me a bit about the challenges that come with using AI?
PE: AI can hallucinate, right? The rule we have in the Met Office is you may use AI for any purpose that is efficient in your job, like to write your code for you. You must declare you’ve used it. So if I use it to summarise a board paper, it’s wonderful. I get a 10-page board paper – I’m not having that – [and use] Copilot [to get it] down to five pages and bring it back. But you must declare it and you are still responsible for the accuracy of what you produce. So, if there’s a bug in your code, or it has actually changed your board paper so it now says something different, that’s still your problem.
Where we are really exploring things is – we rigorously validate anything we use operationally and we’re not really using AI operationally yet. And we have extremely tried and trusted techniques to do that. And every time we upgrade a weather model, we put it through a whole series of checks and balances to make sure it really is better than the last one – and if it isn’t, we don’t implement it.
The techniques we use for conventional modelling have limitations for AI.
AI, you train to replicate. You optimise it for a particular thing [and] it will do that wonderfully. But then, if it has very low average errors, it may still miss the extremes. And if what you do is compare average error, it will look like it’s better than the conventional model. But if, actually, that’s because it smooths everything out and it has missed the extremes – when you really want to know, it’s going to be wrong.
So, what we’re increasingly working on is on running our own AI models. And we’re looking at the other market leaders – the European ones, the [Google] DeepMind one. And we’re continuously evaluating them against the leading conventional models and looking at what the full suite of metrics is you need, if we wanted to feed our app from an AI model, rather than from a numerical one – a physical model. What are the thresholds it would need to pass before we were confident to do that?
CB: That’s really interesting. I wanted to talk a bit more generally about climate modelling as well. We’ve talked about AI, but where are the other exciting innovations, and also perhaps where are the gaps that still need to be addressed?
PE: They are kind of two sides of the same coin, I think. We would love to be able to do the kind of kilometre-scale [modelling] we do for weather for climate. Computationally by conventional methods, it’s just unaffordable and it doesn’t even look close – and with Moore’s law breaking down, it almost looks like you’ll never get there. AI, potentially, could close that gap. And that’s where that downscaling problem that I gave you, came in.
Others of the exciting things, I think they are around particularly the tipping points and the adaptation and the attribution. Some of the live areas of research for us when you see
CB: When you say Moore’s law is breaking down, what do you mean?
PE: The amazing improvement of weather forecasting since the 50s has been built on the fact that computing power doubles every couple of years for the same cost. So you can get more and more transistors on a chip and the supercomputer gets more and more powerful. We can reduce the scale of our weather models and improve the resolution, and can give you more accurate weather. That’s gone on for basically – [it has] improved the weather forecast at a “day a decade” over that period.
We’ve reached about the physical limit of how many transistors you can fit on a chip and supercomputers are no longer basically giving us improved modelling accuracy for free. And, so, we have to use different techniques now to find a way to continue to improve the accuracy of the weather forecast and maintain that “day a decade” improvement. And we’re confident we can and AI will be part of that mix.
So, our strategy is to go for the best blend of conventional and AI modelling – but we’re still working out what we think that best blend will be.
CB: And I know the Met Office recently switched on its new supercomputer, so I wondered if you could just tell us a bit about what’s new and how it might impact your research.
PE: This is a big step up in [computing power] and it’s also… The Met Office, I think we’re on about our 14th supercomputer. The first one we blagged some time on a Lyons tea room computer back in the 50s. But, then, numbers two to 13 we’ve owned and they’ve been based in our headquarters – wherever we’ve been – and we’ve operated them ourselves.
This one is different. Microsoft owns and operates it for us. And that’s a step to the whole thing being fully in the cloud, fully in Azure Cloud. And it needs to be because the amazing, fantastic, wonderful data that we have – [and] on which all of these products and AI is built – is now so big you can’t move it. We have about half an exabyte of data. So, the data needs to be next to the computer to be processed.
And, so, this computer is really exciting. We’re about to implement the first, what we call a parallel suite, but the first big model upgrade. Using it will let us do finer-scale, better microphysics – particularly cloud microphysics – [and] better precipitation. Because we’re running the parallel suite – which isn’t live yet, that’s why it’s a parallel suite – we can see the improvement we’re getting just from that first step forward.
And, then, we’ve got a whole series of scientific upgrades planned over the next few years to continue to improve our forecasting in weather and climate.
CB: Brilliant. That’s everything. Thank you very much.
The post The Carbon Brief Interview: UK Met Office chief executive Penny Endersby appeared first on Carbon Brief.