Japan’s Statistics Scandal: The Need for New Approaches

In January 2019 it came to light that the Ministry of Health, Labor, and Welfare had been using improper methodology for its Monthly Labor Survey from 2004 through 2017. Rather than collecting data from a sample of smaller companies and from all companies with 500 or more employees, the ministry surveyed a smaller sample of large firms located only in Tokyo, thus skewing national wage averages and leading to a range of faulty economic data produced on the basis of this information. We spoke with Hayakawa Hideo, formerly head of the Bank of Japan’s Research and Statistics Department, about this scandal, which has shortchanged millions of Japanese workers who failed to receive deserved benefits and has triggered a raft of fresh investigations into the government’s approach to gathering statistics.

A Disgraced Statistics Office

INTERVIEWER  How would you summarize the statistics scandal?

HAYAKAWA HIDEO  Simply put, this was a breaking of the rules: an arbitrary change to the methodology of the Monthly Labor Survey, which surveys wages and other labor statistics, making it a sampling survey rather than a more complete investigation of the labor market. There’s nothing wrong with sampling surveys per se, of course, so long as they are performed after proper notification. But in this case, we had a failure to make needed revisions  to the data covering the years from 2004 to 2017.

Sampling surveys can of course be useful. Say you want to calculate average wages, and you survey the entire manager class of a company, but only one in ten employees. In that case, the figure for employees needs to be multiplied by ten to get the company total. This is basic statistics, but it wasn’t done this time around, which came as a real shock. This was an amateurish, elementary error. How can a compiler of statistics not know this?

Another problem was trying to cover up the situation while knowing that the sample surveys weren’t being properly adjusted. This is a problem with compliance. Now we have a situation where the figures gathered since January 2018, under the proper methodology, have been compared with the flawed, previous figures, producing an upward swing in the year-on-year growth rate. The Statistics Office should have announced that it had made a mistake and provided a correction. It’s outrageous that this has not taken place.

This wrongdoing spotlights problems with the competence and ethics of the Statistics Office. I have my doubts about the appropriateness of the debate on fabricated data taking place in the Diet these days, given that Japan’s statistics have been growing increasingly wobbly for some time, but now we’ve reached a real low. What the Diet should address is how to correct the current situation where unreliable statistics are being generated.

INTERVIEWER  Wages are an important indicator in discussing Abenomics.

HAYAKAWA  Abenomics has been discussed based on such economic data as prices, the economic growth rate, the number of employees, and wages. Wages are an important element in measuring trickle-down effects, where improved corporate earnings propagate to jobs and worker pay. Since the growth rate of wages has been revised downward following the discovery of these irregularities, this will no doubt affect the assessment of Abenomics.

Economists had already noticed in January 2018 that the growth rate of wages was high, which they set aside as an outlier. This being the case, the downward revision of the growth rate of wages does not come as a surprise. My feeling is that through this affair we finally understand the reason for the aberrant figure. Until the summer of 2018, the Cabinet Office was boasting that employment was growing quickly based on the official figures. From around autumn, however, they were acknowledging that something was wrong. A trickle-down effect is not occurring.

The Bureaucracy’s Disdain for Statistics

INTERVIEWER  What explains the poor quality of statistics?

HAYAKAWA  One reason for the deterioration of statistics in Japan is the disdain they receive from the government. Career bureaucrats are assigned to the statistics departments of central government agencies for a period of about a year, and their efforts there make little difference in advancing their careers. Also, bureaucrats are under pressure to reduce head counts and expenditures. The first cuts they make are in areas that are unlikely to benefit their careers, and statistics departments have come under the axe. It’s hard to say for sure, but it’s possible that the statistical irregularities were spotted early on, but precedent was followed to avoid discord with predecessors.

INTERVIEWER  Some observers point out that statistical methods have fallen behind the times.

HAYAKAWA  When I joined the Bank of Japan in 1977, I thought that Japanese statistics were the best. At that time, the United States used retail sales to measure personal consumption in calculating the gross domestic product. While the Americans measured GDP only in terms of goods, Japan had a more complete view, since the Family Income and Expenditure Survey also surveyed services. Things, however, have changed over time. The Family Income and Expenditure Survey assumed that full-time housewives were keeping a record of household accounts—something that seldom happens anymore. As full-time housewives decrease and the number of dual-income families continues to rise, wives no longer know what their husbands’ salaries are going for.

Fostering Specialists and Big Data

INTERVIEWER  What is your prescription for the disdain being accorded statistics departments?

Some twenty years ago, when Murayama Shōsaku headed the Research and Statistics Department, the Bank of Japan made a major change in its statistics policies. Previously, since you couldn’t effectively build your career simply by compiling statistics, people viewed their job as analyzing results and reporting it to important people before formal publication. Under Murayama, however, such advance reporting was halted, efforts were made to improve statistics, and the department was tasked with increasing precision as a whole. These changes also aimed to enhance information management in the organization. One output was a marked shortage of certain sorts of human resources. While there are many PhDs in statistics in the United States, Japan has few specialists in the field. The BOJ placed people in professional positions and worked to develop statistics professionals over time—women in particular.

It’s easy to call for a central statistics bureau, where the statistics departments scattered around Japan’s ministries and agencies are brought under one roof, but without professionals to staff it, such a bureau won’t do much to improve the situation. Money and people need to be allocated to develop specialists. This problem is most serious for the Cabinet Office. The Department of National Accounts, which compiles figures on Japan’s GDP, is seriously understaffed.

INTERVIEWER  It sounds like Japanese statistics have fallen behind the times. What does the country need to do to improve their precision?

While the Family Income and Expenditure Survey worked well in the past, the environment has changed. Sticking to the same methods will only degrade the survey. What should be done is to take advantage of the progress of digital technology. For example, in surveying consumer prices, the Ministry of Internal Affairs and Communications basically collects data from stores for a limited number of items. In the case of hamburgers, only the cheapest, most basic product is surveyed. If the price of a Big Mac or cheeseburger changes, this will not be reflected in statistics.

Regarding this issue, the United States has digitalized statistical data, and it is able to compile statistics in great detail. This is an age where data is available instantly in large volume without having housewives keep a record of household accounts. In the case of the Family Income and Expenditure Survey, the Ministry of Internal Affairs and Communications provides survey respondents with paper and a calculator. It would be more efficient to collect responses from individuals rather than at the household level and to do so through a smartphone app. Once we enter a cashless age, settlement data can also be surveyed.

It would not do for statistics to be changed every year, but they will deteriorate if the same methods are continued decade after decade. When the times change, methodology should change in response. How can we debate policy properly if we’re using a crooked ruler to measure the numbers being impacted by it? Too much energy has been going toward maintaining the system. The fact that this issue has come before the Diet is an opportunity for reform.

(Originally published in Japanese on March 5, 2019. Reporting and text by Mochida Jōji of Nippon.com.)