New paper on revenue forecasting accuracy and the independence of forecasters

Thiess Buettner and Bjoern Kauder have released a paper in the September 2010 edition of the journal Fiscal Studies on government revenue forecasting practices and the consequences of forecasting performance. The object of analysis is the one-year ahead deviation of the forecast from the actual outcome, and various descriptive statistics. Revenue forecast errors for twelve OECD countries (including two competing institutions in the United States, the Congressional Budget Office which deals with the legislative branch of government, and the Office of Management and Budget, which is a part of the executive branch) are examined using panel data estimation techniques.

The countries examined are: Austria, Belgium, Canada, France, Germany, Ireland, Italy, Japan, the Netherlands, New Zealand, the United Kingdom and the United States. Curiously, however, Australia is not considered.

The authors acknowledge that forecasting is 'a difficult task'. Nevertheless economic and fiscal forecasts are an essential part of public accounting. Accuracy of forecasts creates credibility on the part of the forecasters. This forms the basis of revenue that a government expect to receive and what they expect to spend. Only revenue forecasting performance is considered in the quantitative analysis. The authors cite four main difficulties encountered when forecasting revenues:

• macroeconomic developments
• functioning of tax law and its enforcement
• policy changes in tax law and
• structural changes in the economy.

As a brief aside, it should be noted that, in the Australian context, all of these difficulties are heightened at present.

The first stage of the analysis tests for the presence of statistical biases, that is, systematic over- or under-estimation of revenue forecasts as measured by the level of the one-year ahead revenue forecast error. This set of tests is aimed at establishing whether there is a ‘zero average’ forecast error, which would tend to indicate no significant bias up or down in preparing revenue forecasts. The authors only find significant instances of a consistent downward or pessimistic bias in two countries – Canada and Italy. Unexpected changes in GDP had a significant impact. Overall, they conclude that there is little evidence for systematic biases between forecasting institutions but macroeconomic uncertainty can have a significant effect on revenue forecasting performance.

The second stage of the analysis tests the precision and accuracy of revenue forecasts. Three main categories of determinants that show considerable variation across countries are considered. The first is conditions facing forecasters when preparing forecasts. The authors attempt to capture this using:

• the time span between the official forecast and the beginning of the fiscal year being forecasted (e.g. in Australia the Commonwealth Budget is handed down in early May, yet the fiscal year does not begin until 1 July – a difference of around 2 months)
• differences in tax structures, as shown by the number of taxes (ranked smallest to largest) that comprise 50 per cent of total revenue – it is anticipated that being dependent on a large number of relatively small, narrow-based taxes would increase the difficulty of forecasting and
• The one-year ahead GDP forecast error.

Institutional factors such as the degree of independence from the executive branch of government when preparing revenue forecasts are also considered. Three institutional revenue forecasting characteristics are combined to form an index of institutional independence from the executive branch of government. These are:

• use of what the authors term external ‘research institutes’ – these are agencies that sit outside the executive – in preparing the revenue forecasts
• the relative involvement of external (including central bank) and government experts in the revenue forecasting process and
• whether an internal or external (to the executive) macroeconomic forecast is used in preparing revenue forecasts.

Another institutional factor is considered: the existence of a ‘supra national body’ – the European Commission and more particularly the Ecofin Council that discusses and standardises member states revenue forecasts may have an impact on performance. In particular, the authors consider whether arrangements under the European Union Stability and Growth Pact may have had an impact on forecasting performance.

Macroeconomic uncertainty is captured by a measure of volatility in economic growth, the standard deviation of GDP forecast errors. The authors test a variety of different specifications of the variables listed above. As an example of some of the different specifications, different weights are used for the components of the independence index and tax structures are delineated by tax type (i.e. personal, corporate, etc). They range in complexity from merely considering time span and macroeconomic uncertainty up to and including all of the above listed variables. In sum, they find that the index of independence exhibits a statistically significant association with forecasting performance. This finding is robust (at least at the ten per cent level of significance) for every specification where the independence index is included.

Thus, the authors conclude:

The quantitative analysis shows that the cross-country differences in the performance of revenue forecasting are first of all related to uncertainty about macroeconomic development ... Controlling also for differences in the timing of forecasts, we find that that precision of revenue forecasts increases with the independence of forecasts from possible government manipulation

Given the current debate over the establishment of a Parliamentary Budget Office in Australia, this research tends to validate the push for increased oversight by the legislature and less involvement of the executive branch of government in the revenue forecasting process (notwithstanding the fact that Australia was not part of the data panel).