Forecasting

banner

"There are two kinds of forecasters:

Those who don't know, and those who don't know they don't know''

John Kenneth Galbraith

(Wall Street Journal, January 22, 1993)

“There are known knowns; there are things that we know that we know.

We also know there are known unknowns; that is to say we know there are some things we do not know.

But there are also unknown unknowns, the ones we don't know we don't know”

Donald Rumsfeld

Forecasting Science and Theory of Forecasting

Forecasting is a kind of decision making for defining the most expected scenario from alternatives of future prospects. Although it is not explicitly mentioned, many of our decisions are forecasting the future of a system, organization, human behavior etc. Forecasting is one of the most used functions of human recognition which works unconsciously. We predict arrival time, departure time, cost of daily consumables and behavior of a friend and so on. However, there are some popular forecasts which usually have particular media coverage such as weather forecasts, economic forecasts and the story of fortune tellers based on a magical crystal ball!

The reputation of forecasting is deteriorated with improper practices and poor knowledge about the theory of forecasting. There are several common misconceptions about forecasting and the method of forecasting process. Therefore, it is strongly needed to illustrate what forecasting is and how it works.

Exordium: Philosophy of Forecasting

Forecasting is not a new topic, and it is frequently discussed with the physical phenomena. One of the critical questions about the forecasting is predictability debate. Before approaching to forecast, people need to clarify whether it is achievable. In the similar circumstance, the difference between the nature (physics) and human make a strong impact on our perception of forecasting. Human sciences e.g. economics are subject to the uncertainty of mankind which requires an extraordinary effort for forecasting.

Physical phenomena are usually based on robust, clear and repetitive rules. One can easily predict destiny of a ball if it is released from a table: It drops to floor. We are unconsciously aware of several physical rules including gravity. However, when it comes to a complex problem such weather forecasting, there is a huge number of forces and systems working together. In such cases, we are unable to make a precise and error-free forecast. On the other hand, the movements of earth, moon, sun and most part of astronomic motion can be predicted precisely even in seconds of time.

Economic phenomena as the most popular field for forecasters has somewhat different dynamics. In the core of the problem, we, human, exist, and our nature is quite complicated. The models of our behavior can change every day. We may dislike something today which we liked it yesterday. Therefore, most of the econometric models need to be re-estimated and re-thought frequently. No model can serve for several years.

The use of forecasts and the state shift are some other critical topics. Forecasts are usually required for developing broader policies or strategies in micro level. Based on the forecast, decision makers define a policy to achieve a desired outcome. Therefore, policies are employed for changing the direction of a phenomena from the expected (non-intervention) future state to desired future state. The major function of policy here is shifting between states by using policy/strategy instruments.

graph

As a result of policy actions (intervention), the nature of phenomena changes, and now it is not same as estimated at the time of forecasting. If policies are somewhat successful, forecasted future state will not become a reality. If someone criticizes the failure of the forecast, that would be ignorance of the function of a forecast. Unless a forecast is kept exclusive for a person or a small group of people, future state will always change, and it will be replaced with a new one. The theory of information asymmetry emphasizes that if an information is hold by exclusive users, then that will create an advantage for them. Based on this, we can roughly say that a publicly known forecast probably fails since decision makers replace their position and direction. In addition to that, if most of the decision makers are familiar with the state-of-art forecasting methods, they will probably conclude similar predictions which in turn causes similar strategies. Finally, forecasts will fail even in exclusive terms since the method is common and symmetrical. That is the current situation in the financial markets. Although forecasts are exclusive, methods are well known, and even many trader companies employ quants with PhDs from Harvard or MIT to find a less known way of forecasting.

When human action is in the problem, there is no straight and accurate way of prediction whether a forecaster runs complex mathematical functions and simulations with an extraordinary programming talent.

Assumptions behind the classical forecasting methods

The major assumptions behind forecasts are the repetitive nature of history and the stationary nature of decision makers. The recursive history perspective is very old topic, and it is one of the main themes of Muqaddimah (Ibn Khaldun) which is the first written publication of the scientific method to social sciences and the philosophy of history. In modern times, Peter Turchin extended and mathematically presented the theory, and even he developed models of historical dynamics. Cliodynamics is branch of history dealing with the mathematical modeling of historical fluctuations and metamorphosis. According to the theory, the most part of the history is a kind of cyclic movement while names, titles or instruments are changed. For example, civilization has a cyclic behavior and every civilization movements and developed empires have a life span from birth to mature and death.

The current business cycle theory also supports the theory of recursive history and both macro and micro level economic systems are subject to cycles of upturns and downturns i.e. recovery and recession. However, the vital point of the theory is the size and schedule of cycles. Based on the recursive history assumption, every forecaster deals with finding a proper method to define particulars of coming cycles. For longer periods, forecasters study long term cycles and for shorter terms they look for short term cycles.

Another critical assumption is the stationarity of decision makers which refers to the identical reactions to identical impacts (Reader may confuse with the rationality assumption which is a quite different matter and will be discussed later). The recursive history assumption somewhat includes the stationarity of decision makers. History repeats itself since decision makers behave identically in addition to the identical circumstances. Under these assumptions, we recognize the historical pattern and replicate it to find future direction.

From the economic perspective, the utilization of decision making processes is a common issue and it is known as rationality of decision maker. It is very similar to the stationarity assumption with a slight difference. In stationarity assumption, we assume that all economic agents behave identically even when these selections and decisions are inadequate. If some decision makers make faulty preferences, we assume that this faulty decision will be repeated as well. However, rationality assumption is about the finding the optimum decisions among a number of options in every cases. Econometric models are usually based on rationality assumption which means the economic agents of the intended marketplace rationally defined and defines optimum economic preferences. Therefore, the proposed model is estimated under the rational circumstances and the future state is expected to be rationally managed. Based on these assumptions (many assumptions mean it is probably impractical), history will repeat itself in the rational people’s world. However, rationality assumption is strongly criticized in the last few decades and the irrationality concept (also bounded rationality) is a growing topic in the field.

Why micro economic forecasts are usually inferior in business practice?

It is related with the publicity of forecasts. As it is discussed in Exordium, the asymmetric information refers to the possibility of arbitration in case of private market intelligence. If an economic forecast is publicly available, every decision makers consider this evidence and revise the direction of their investment. Although macroeconomic and long term forecasts are able to predict cycles roughly, micro-economic and short-term fluctuations are affected by the short term position changes of agents. Once a forecast is publicly available, every agent moves to a new position, and then the decision space of desired future will dramatically change. From that time, the particulars of marketplace is not same as it is assumed/estimated in the modelling stage.

Although the model delivers proper predictions, it will never be accurate since it is common information. Many international organizations (e.g. World Bank, IMF) publish macro-economic forecasts while it is very difficult to find publicly available forecasts for industrial markets. Usually we pay charges for them and we wish that a few people reach to these predictions. Otherwise they are worthless and “inaccurate” as a result!

Methods of Forecasting

Forecasting methods have two major divisions: Quantitative methods and qualitative methods (i.e. objective vs. subjective). Econometric modeling, time series analysis, neural networks and other methods of mathematical solutions are quantitative methods, and they are very useful when there is a repetitive nature. For example, seasonal time series methods are quite accurate for prediction of ice-cream sales or the volume of harvest. However, they are strictly limited to the historical pattern, and it is impossible to embed pure subjective factors, expectations or political aspects. Therefore, qualitative methods are utilized distinctly or in addition to the quantitative methods. Judgmental forecasting is a typical subjective forecasting method, and it is frequently used for exposing herd behavior and other psychological trends. Expert guided adjustment of quantitative methods is an alternative solution to gain advantages of both. Judgmental forecasts are also limited to biases and heuristics of decision makers. For this reason, a special care is needed to handle and manage judgmental forecasts.

The use of computer intelligence, neural network models and fuzzy sets for uncertainty problem is a growing section of forecasting science, and these methods contributes to the improvement of processes, randomness and complexity (i.e. chaos systems). The existing literature has several good applications of computer intelligence which are superior to the conventional time series analysis and the orthodoxy of econometrics.

Fundamentals of Forecasting

In a brief list, we may figure out some fundamental needs and common failures of forecasting studies:

(1) Data control and preparation: Stationarity control | Data transformation if needed

(2) Sampling (Partitioning): Defining the in-sample period for training (estimation) and the post-sample period for real forecasting test (i.e. testing period)

(3) Benchmark Selection: Conventional methods and other potential competitors of the proposed forecasting method should be selected as benchmark. Since this step is subjective, one may intend to select inferior methods to highlight the proposed one. A computer intelligence method should not only tested against similar one, but it should also be compared with the conventional time series methods.

(4) Accuracy Metric Selection: There are several accuracy metrics used frequently while they are quite biased (e.g. MAPE, RMSE). For example, Mean Absolute Percentage Error (MAPE) is one of the most biased error metrics. Assume that we have two actual value, 1 and 5, and then our model generates same predictions for them, 3 and 3, which means same absolute error for both, 2 and 2. MAPE for these forecasts are |1-3|/1=2.0 (200%) and |5-3|/5=0.4 (40%). Although absolute errors are same, MAPE metrics have a huge difference. If forecasting model systematically predict less (undervalue), then we will find it superior. Please be aware of illusion of accuracy gain. Mean Absolute Scaled Error (MASE) is a relatively better choice for accuracy control.

(5) Residual Control: Residuals should be checked against whether a remaining pattern exists. A figure showing the residual series is usually enough for illustrating the white noise control. White noise testing procedures (e.g. serial correlation test) may also clarify whether residuals are really irregular oscillations.

Ethics of Forecaster

Forecasting methods and procedures are usually subject to expert consultation, and there is a strong potential of unethical use or presentation. Subjective selections and arbitrary preferences may help to validate and rationalize the scholar’s work (self-serving bias), then the outcome would not be useful and practical. By this way, forecasting study turns to be an entertainment rather than a professional effort.

Forecasters are strongly encouraged to criticize their work and profession if they concern about the practical meaning of their efforts.