The UK is setting up a new publicly funded agency, ARIA, with a remit to pursue technological breakthroughs that could yield huge benefits for society at large.

ARIA’s managers will have considerable discretion to invest for the long term in high-risk/high-reward projects, and the tight rules that usually bind such programmes will be relaxed.

The model is promising, but three questions must be answered which are explored in this article.

• Where does ARIA sit on the UK’s research and innovation funding spectrum? We explore the potential for using the existing Challenge Funds as a ‘funnel’ for ARIA missions.
• Will the amount invested be sufficient, and how will ARIA priorities be chosen to be compatible with the real-world business of innovation, in particular given the focus on risk?
• How do we judge whether it’s succeeding? We highlight the need to focus on critical leading indicators of success, and the processes around ARIA, at least in the early days.

Spurred by the fear of lagging behind the Soviet Union following Moscow’s “Sputnik moment”, the US launched its Advanced Research Projects Agency (ARPA), now renamed the Defense Advanced Research Projects Agency (DARPA), in 1958. DARPA championed high risk tolerance, collaboration between academia and industry, and a long investment horizon in research and development (R&D) funding. DARPA was a pioneer of what is now referred to as mission-oriented The approach not only helped land humans on the moon but also underpinned technological breakthroughs whose spillover benefits paved the way for technologies including the internet and GPS.

• The UK is now looking to get in on the action, as the legislation to launch an Advanced Research and Invention Agency (ARIA) has just received Royal Assent. Former DARPA Deputy Director Dr Peter Highnam was recently appointed as ARIA’s first CEO, while the search for Chair is currently underway.
• ARIA will be sponsored by the Department for Business, Energy & Industrial Strategy (BEIS), with public investment of around £800m until 2024-25 (here). The intent is ambitious: “ARIA will exclusively focus on projects with potential to produce transformative technological change, or a paradigm-shift in an area of science. While it is anticipated that most programmes may fail in achieving their ambitious aims, those which succeed will have profound and positive impact on society.” (here)

Drawing on Frontier’s experience in evaluating innovation policy over many years, we explore three issues relating to ARIA:

• Where does ARIA fit in the policy landscape?
• Is it ‘big enough’?
• How will we know if it works?

Aiming for the high notes: Where does ARIA fit in?

ARIA aims to foster an agile, high-risk/high-reward investment environment and a light-touch approach to project clearance. The intention is that ARIA will mimic DARPA in relying on expert programme managers to run funding portfolios over three to five years to support ‘missions’, high-risk research and innovation (R&I) goals, with considerable discretion over the allocation of the funding. The managers will be closely involved with the projects and be able to stop and change direction where required. ARIA will be industry/sector agnostic, akin to Germany’s SPRIN-D and Japan’s Moonshot R&D programmes, rather than defence-focused.

ARIA looks to carve out a unique position in UK R&D funding. Much direct public backing for innovation is currently funnelled through UK Research and Innovation (UKRI), with business-focused support primarily delivered through Innovate UK, one of UKRI’s nine councils. ARIA will sit outside of UKRI. This separation could foster a more agile, longer-term innovation perspective, avoiding the need for ARIA to follow existing governance arrangements. UKRI is subject to annual budget planning and regular Spending Review procedures which demand regular evidence of impact even in the shorter-term. None of these necessarily gel well with the stated design principles of ARIA, as noted in a recent Public Accounts Committee report.

That said, it will be critical for ARIA to have a well-defined role to avoid potential confusion or duplication. Spelling out its relationship with UKRI will be vital. One model that would connect UKRI and ARIA, and provide longer-term funding support, is to use existing UKRI investments such as Challenge Funds as a ‘funnel’ for ARIA; projects showing signs of success after a few years’ funding would then become ARIA missions with additional, more autonomous investment. This could help address some concerns that Challenge Funds have too short a time span (normally three to four years) to tackle genuinely ambitious and transformative innovation challenges.

Frontier has been working extensively on the evaluation of Challenge Funds with UKRI. They are ambitious, highly varied projects and so having clear, measurable criteria to help select which Funds go on to receive further support through ARIA would be critical to the success of this model, drawing on the potential innovation, economic and societal impacts and the demonstrable need for further public support.

A sense of scale: is ARIA big enough?

How much public investment ARIA ‘needs’ to succeed is clearly hard to define. With a budget of £800m spread over a few years, ARIA is much smaller than Innovate UK, which had a budget of almost £1bn in 2021/22. It is also smaller than DARPA, which invests $3.5bn annually. That said, ARIA is seeking to occupy a particular niche on the R&I spectrum , rather than providing wide-ranging support, and the UK is a much smaller economy than the US.

Another way of looking at it is whether ARIA helps the UK to get closer to its target of spending 2.4% of national income on R&D by 2027, compared with 1.7% in 2019 (see chart). Hitting this target, which is designed to help reverse the UK’s lacklustre performance in productivity growth, will be daunting, requiring an extra £10bn a year in R&D spending in today’s terms. Even assuming that the entire ARIA budget is additional (rather than displacing other planned public R&D outlays), and that a generous leverage rate of additional private expenditure is crowded in as a result, ARIA on its current scale will clearly close the funding gap only a small part of the way.

We can also try and understand how many different ‘missions’ ARIA might be able to support at current planned funding levels. For example, many of the larger and more complex Challenge Funds have funding pots well in excess of £200m over a three- to four-year period, suggesting that a substantive mission may need around £50m or so in annual investment. If about £200m per year is available for R&I through ARIA, we could infer that such a sum might support about four large-scale missions at a time. The number could be larger if funding were spread more thinly, but this would seem to run counter to the intention to focus on high-risk and transformative investments.

• R&D as a % share of national income, 1985 to 2019 and comparisons to 2027 target

Frontier Economics calculations based on ONS statistics.

Given these difficult trade-offs between the number and size of missions to support, it will be particularly important for ARIA to understand what is needed to move innovation forward in high risk areas. Having programme managers with expertise in the practical business of innovation is likely to be critical here. Other work we have done exploring how businesses innovate has suggested a real need to think about business models, culture and collaboration, all likely to be important factors in the design of a high-risk, high-reward innovation programme like ARIA.

Music to our ears: how do you evaluate ARIA?

Given the desire for ARIA to operate in an agile and autonomous way, and a stated intent to take more risk, an obvious public policy issue is how to factor evaluation into its design and delivery. Some sort of evaluation of ARIA will be necessary given the scale and nature of the investment, as made clear in the Magenta Book: “…a high-risk, high-status policy breaking new ground is likely to require a large-scale evaluation”. Thinking about this upfront in the early design and implementation of ARIA will make it a more effective process. If a future BEIS secretary of state wished to expand ARIA, evidence that it is ‘working’ would be needed. But what could that mean in the context of this kind of intervention?

A key objective, true of all evaluation but particularly relevant for ARIA, is that the assessment should be complementary to delivery and not be seen as getting in the way. This will require co-operation between programme managers and evaluators in terms of access to data, evidence and stakeholders. An agile funding model will demand similarly lean processes around performance and project monitoring, capturing only key information needed to support decision-making.

There will also need to be consensus on what is being evaluated. Frontier regularly designs and implements evaluations of long-term innovation programmes such as Catapult Centres and the Strength in Places Fund. Our experience suggests that, rather than an emphasis on value for money and short-term economic impact, which would be counter to the high-risk and long-term model of ARIA, early evaluation should focus on different questions and definitions of success, and draw heavily on expert engagement to provide evidence.

Given the emphasis on agility, any early evaluation of ARIA should be proportionate, and focus on understanding the unique features of the scheme. Some evaluation issues relate to the processes around ARIA and whether they are working as intended: is it indeed occupying a distinct niche in UK R&I and avoiding adding complexity? Is it performing in line with its intended design principles? Do programme managers have genuine autonomy, free from narrow political considerations? Is failure truly tolerated, and when failures do occur how are the lessons identified and learned? Any focus on ARIA’s early impact should be centred on a clear theory of change (what is ARIA trying to do, and how does it achieve it?) and prioritise leading indicators of success linked to that theory: are the relevant R&I challenges being identified and addressed? Is this being done in a joined-up way that improves on previous practice? Are innovative solutions to the missions being developed and adopted?

Conclusion

ARIA is an ambitious initiative, investing sizeable sums of public money in potentially transformative research and innovation.

ARIA will enjoy a large degree of autonomy in executing its high-risk/high-reward projects, but its relationship with UKRI must be well defined to avoid duplication. There are opportunities to join up existing UKRI funding to ARIA, but clear frameworks will be needed to support this alignment.

Another tricky problem is how to evaluate whether ARIA is achieving its goals. For a ground-breaking agency, conventional assessment yardsticks may not be applicable. An early focus on qualitative evidence around process and leading indicators of success may be the most pragmatic approach, focusing on the unique and innovative features of ARIA compared with other programmes.

Download the full article here: