What did Caesar Augustus ever do for us?

As all those Christmas nativity plays remind us, the census has a long history. But as the detailed results of the 2020-21 batch of national censuses begin to be proudly rolled out by statistical authorities around the world, the question arises: in the age of Big Data, with analytical tools giving us real-time information down to postcode level, is this ancient and expensive decennial head count still really needed?

 It all began, so they say, with the Babylonians, in about 4,000 BC. Then, the Romans loved censuses - and so did the Chinese, conducting a remarkably precise headcount of 57.67 million people at about the same time as Caesar Augustus was rolling his censuses out around the Mediterranean. About a millennium later, Duke William famously used one to exploit his conquest of Britain. But modern census-taking in the West didn’t really get under way until the late 18th century.

The Spanish had a go in 1768, but left the job to their bishops, which wasn’t a success. Pre-revolutionary France had a stab at it in 1772 and the United States a post-revolutionary go in 1790. (This produced a total population figure of 3.9m, which Washington chose to believe was a serious underestimate. But in his “free” nation, the census count was divided between “whites” and “slaves”, and it’s anybody’s guess how well the latter were counted.)

In the UK, the first “modern” census didn’t come until after the Act of Union, in 1801. And since it would be decades more before Germany even existed as a single country, censuses continued to be local in the area until 1875. But now almost every country conducts a national census of some kind or other every five or (more often) ten years, although Afghanistan hasn’t managed to do so for nearly four decades, and Lebanon hasn’t dared have one since 1932.

The European Union obliges its members to carry out a census each time the year ends in a one, although it does not dictate the way they do so. As in the UK, the results from the latest are just beginning to be published. Since a batch of other countries, notably the US, carry out censuses when the year ends in a zero, a wealth of new data from these too has, in combination, enabled the UN to update its estimates of world population levels and trends.

So the UN has recently informed us that Planet Earth passed the 8 billion population mark on 15th November; that amongst the world’s most populous countries India will surpass China during 2023; and that Japan has dropped out of the top ten. It also tells us that while populations are shrinking in 61 countries or areas, global population is still likely to rise to over 10 billion by the end of the century.

In some countries, we still need traditional censuses just to count heads. But not all still use massive decennial surveys. A number of northern European countries base their census returns on administrative data, while France carries out a rolling census of sample surveys which arguably provide more up-to-date information with less fuss. These other methods, of course, work best in countries with good registration data, often based on identity cards, such as Germany.

Britain has resiled from that route: a first step towards ID cards, taken in 2006, was reversed, after a change of government, in 2011. Perhaps growing attention on electoral fraud (justified or not) will prompt further thought, particularly now the technologies are available to support a virtual ID card, not just another piece of plastic for everyone to lose.

Meanwhile it’s important to keep asking: what’s a census actually for? Early census-takers were more about get than give: they were the tax-collector’s enforcers rather than Santa’s little helpers (although the Egyptians did use censuses for planning food and labour supply).

Tax continued to be an important motive, but it wasn’t just about listing property to expropriate. After German unification, for example, the census was important to ensure an equitable distribution of tolls from the new customs union, and the distributive purpose of census data has become increasingly important in modern government.

Ask too much, however, and you’ll get an adverse reaction. The basic census information asked of each household - age, sex and working status, and later housing - would seem uncontentious enough, but what voters find difficult or intrusive varies between cultures and over time. And questions citizens don’t like may cause unrest and will certainly yield data of doubtful quality.

In mid-nineteenth century Britain, for example, asking women their age was tricky - and found to yield significant underestimates. Today gender has become trickier. And the farther you get from these basics the more contentious it can become. In the UK, a late twentieth-century experiment with questions about household income was such a failure it never got incorporated in the main census. Americans, by contrast, seem quite happy to fill that box in, with what accuracy is of course another question.

And then of course there’s the question of religion, which is, for example, so important yet so sensitive in Lebanon that it has seemed easier to duck census-taking altogether. It’s a question with a poor history. In Japan, for centuries, there was a special census designed to identify illegal Christians; in the UK in penal times, there was a similar use of parish registers to dig out secret Roman Catholics. The United States statistical authorities have robustly refused to include a question, on the grounds that religion is no business of the state. But the UK has, for no very obvious official purpose, included a “voluntary” direct question since 2001.

The results, published for the 2021 latest census in November 2022, show that in England and Wales the proportion of the population declaring themselves to be Christian had fallen from 71.7% cent to 46.2% in 20 years. Surprisingly, however, in a clearly secular age, this still exceeded the proportion declaring themselves to have no religion, although that rose from 14.7% to 37.2 % over the same period. And self-declared Christians amounted to nearly quadruple the number of adherents of all other religions put together, of which the largest groups were Muslim (6.5%) and Hindu (1.7%).

The geographical spread was fairly predictable, with up to two-thirds declaring themselves Christian in some rural areas and many fewer in cities (making the incumbent Christian Church of England’s tendency to abandon so many rural parishes for urban ministry seem even stranger). And it’s intriguing that Christianity should have faded so much faster in Wales than England (while perhaps not everyone would have guessed that the most secular place in England would be Brighton and Hove). But is this information really relevant to government policy?

Questions on ethnic self-identity have been asked in the UK since 1991, although a number of other countries avoid these too. Here there’s a more arguable reason: to help measure the impact of public policies on different ethnic groups. But the answers can attract misleading headlines. Media coverage of newly-released data from the UK census has, for example, focused on the increase in the number of non-white majority cities, which now include Manchester as well as Birmingham, Leicester and London. But meanwhile, the aggregates demonstrate that England and Wales as a whole are still overwhelmingly “white”, with 81.7% identifying themselves that way; while the details are more interesting than the broad ethnic blocks that attract news reports.

Notably, “other whites”, a category selected by many East Europeans, were by 2021 up to 6.5% of the population, more than the 4% made up of all those who identified themselves as “Black, Black British, Black Welsh, African or Caribbean”. And these were also outnumbered by the proportion of households containing more than one ethnic group (now over 10%). Add the 3% reporting mixed or multiple ethnic identity and there’s some hope that true diversity will start to grind away at the obsession with differences. Meanwhile the number of forms for everything from Covid tests to job applications that ask for your ethnic identity must raise a question as to whether we really need these census data as well.

Ethnicity data also are poured over in the US - where the 2020 Census showed that amongst under-18s “people of color” outnumbered “whites” for the first time, and the white population is falling. But categorisations too vary with cultures, as well-meaning American firms can find in Europe if they identify “Hispanics” in the region as “people of color”. Sometimes, it all makes one sympathise with the anti-apartheid campaigner in South Africa, who when asked to state his race on an immigration form, simply put “human”.

For questions on ethnicity have a dark history that helps explain why the Germans are historically suspicious: the answers given in the 1939 census were put to terrible use by the Nazi regime. So it’s not surprising that German policy-makers tend to play down census-taking and make the best use of what they collect anyway. France carries some of the same scars, and doesn’t collect ethnic data in its census either.

Meanwhile in the UK, a decision will have to be taken as to whether to conduct another full census in 2031. The decision hung in the balance after the 2011 census, but the Office for National Statistics has been busy publicising the success of the 2021 version, in which priority was for the first time given to online completion. Compliance was higher than expected at 97% (did we all have too little to do during lockdown?).

That shows an impressive degree of trust, and, spread over 10 years, the cost per person still looks pretty low. And the real meat of the information collected, such as the data on housing conditions, has yet to be published.

But the trouble with the decennial census is that it’s the original Christmas tree. Once it has been decided to hold one, lobby groups pressure the statistical authorities (and an even easier target, parliamentarians) to hang just one more question on its branches, since the marginal cost is tiny and the appetite for information vast.

The cost to the UK of the 2021 census was not far short of £1 billion, which can only be justified by the production of data that significantly corrects other sources, and is of a kind where accuracy is needed for public policy planning, rather than just a mine of not-many people-know-that social facts. If the state doesn’t need them it shouldn’t be asking. For while voting isn’t mandatory in most countries, filling in a census almost always is - and making some questions “voluntary” doesn’t completely disperse the compulsory feel around them.

Meanwhile, not all information is best gleaned from a decennial census. Some is worthless if it is more than a year or two old, while other data points have a long shelf-life. Before committing to another old-style census, It would be better to task statistical authorities with reviewing the different ways to collect the information that is really needed for public policy-making, and considering what is already available. It’s quite salutary to count up the number of different interactions you’ve had with government in the past year which required you to provide loads of personal information. Something to do, perhaps, while sitting on your donkey (who needs trains?) on the slow journey back to the old home for Christmas.

Guest Author: Baroness Sarah Hogg, Former Frontier Economics Chairman