On the usefulness of forecasts
Select the answers to the following questions. You can choose one or more areas of interest. The economic sphere presents the possibility of inserting monetary values to have the result of a simple cost-benefit analysis. The resulting evaluations will appear below.
Here we propose a method to evaluate the usefulness of weather forecasts with the aim of representing a tool to think about the benefit of the user. The method is inspired by the multicriteria analysis but it is very simplified because it was designed to introduce the decision maker to the problem of the choice to use or not the forecasting service. The analysis has a significant part of subjectivity because the evaluation is based on judgments expressed by the same decision maker, but is oriented by objective criteria for each area: greater satisfaction for the personal area, greater profit for the economic one, greater security for the social one and, finally, greater protection for the environment one. Further details are provided below.
Evaluation method details
The method followed a multi-criteria logic using criteria, attributes and alternatives:
criteria:
• Psychological criterion: satisfaction of the decision maker;
• Economic criterion: maximization of the decision maker’s profit;
• Social Criterion: increase in security and knowledge for the community to which the decision maker belongs (risk reduction);
• Environmental criteria: increasing the protection of the environment in which the decision maker lives (risk reduction).
Attributes:
• Psychological aspect: sentimental or pleasure, fun and entertainment (index of importance from 0 to 10);
• Economic aspect: profit (index of importance and monetary index estimated in euros);
• Social aspect: warning for the community, scientific research or journalistic investigation to improve the living conditions of people and protection of human lives (index of importance);
• Environmental aspect: protection and conservation of the natural environment, landscape and artistic heritage (index of importance).
Alternatives:
• H0 – Not use of the forecast;
• H1 – Use of the forecast with correct result;
• H2 – Use of the forecast with incorrect result.
The unit of measurement that has been used is the following scale of importance, by which the decision maker can answer the questions and estimate the extent of the events.
Classes of importance of the events:
| 0 | Null |
| 1 | Minimal or insignificant |
| 2 | Very little |
| 3 | Little |
| 4 | Limited |
| 5 | Medium |
| 6 | Considerable |
| 7 | Relevant or significant |
| 8 | Big |
| 9 | Great |
| 10 | Maximum |
Weather forecasts have a probability of success that is complementary to that of failure. These probabilities vary according to the temporal distance, so first of all it is required to choose the time interval between the present and the expected time. The longer the distance, the greater the uncertainty of the forecast. For daily forecasts the probability of success is very high in the first 5 days (from 97% to 90%) but then drops rapidly; for the medium to seasonal forecasts the probability of success does not come as high as for daily forecasts, but does not fall so rapidly with time, thus preserving an interesting usefulness value.
The risk to the decision maker can be considered as the product of the probability of failure of the forecast for the magnitude of the negative events. In this way the result is classified according to the following table.
Risk index:
| 0 | Null | Probability and/or importance of the negative effects null |
| (0 – 0.5) | Very low | |
| [0.5 – 1) | Low | |
| [1 – 2) | Medium | |
| [2 – 3) | High | |
| [3 – 4) | Very high | |
| [4 – 10] | Maximum | Probability > 45% and importance of negative effects maximum |
For each area of interest, a potential usefulness index is calculated based on the probabilities of the forecast and the judgment values expressed by the decision maker. The used formula follows the cost-benefit analysis, calculating the difference between the risk in the case of success of the prediction and that in the case of failure.
In the case of the cost-benefit analysis of the economic context, the average cost of the forecast is included in the costs. Finally, the overall judgment is calculated with the average of the usefulness values of the various areas of interest; this too is expressed according to the following table.
Index of potential usefulness of the forecast:
| 3 – 10 | Very useful | Possible benefits higher the risk |
| 2 – 3 | Useful | |
| 1 – 2 | Partially useful | |
| 0 – 1 | Minimally useful | |
| -1 – 0 | Not recommended | Risk higher than possible benefits |
| -10 – -1 | Certainly not recommended |
Examples of evaluation of the choices:
1 – You want to choose the holiday period to make a holiday in 3 months. A good forecast of the period of interest could give a sufficient (medium) contribution to the choice (H1), although the possibility of a false expectation could result in a discreet dissatisfaction (H2). The result is that the risk we run is medium and potentially the forecast is useful. Comment: even if the importance given to the possible negative consequences tends to increase the risk, however, potentially the forecast has a good usefulness.
2 – You want to organize a Sunday out of the city in 10 days. It is thought that the forecasting service can give a big help to decide what to do (H1), however a forecast error can cause a great dissatisfaction (H2). As a result, the risk is maximum and the final judgment indicates that the forecast is: minimally useful. Comment: 10 days for a detailed daily forecast are many, especially for those who have particularly high demands, so the risk is indicated at the highest level and the potential usefulness is only minimal. A solution to increase usefulness and reduce risk could be to request updates of the forecast.
3 – After rainy days, the local administration considers the weather forecast service of maximum importance to reduce the risk of flooding (H1) in 10 days. However, in case of an error the negative consequences would be of maximum gravity (H2), so the social risk is maximum and the forecast is considered to be only partially useful. Comment: in dangerous situations it is always good to be prepared in advance, forecasts can be helpful in prevention but only partially. Thanks to constant updates, their usefulness can be greatly increased.
4 – A farmer with economic interests requires weather information 5 months in advance. Considering that it can receive a big benefit from a correct forecast (H1), although he has a considerable worry about the uncertainty of seasonal weather (H2), he can rely on a potentially useful forecast even if with a high risk. Comment: for the agriculture to know in advance the meteorological conditions is very important, so generally the forecasts are useful also for long-term and, above all, if you follow constant updates. These updates increase costs but guarantee greater benefits.
5 – A company producing electricity from renewable sources wants to estimate the amount of energy to be sold for the next 3 months and, knowing that the weather conditions are of great importance for its production, it wants to invest in a meteorological forecast that can bring a benefit of maximum importance (H1), having a minimum concern in case of error (H2), since it would not have an economic loss compared to normal. In this case the risk is very low and the forecast very useful. Comment: situations of this type are the best to get the benefits offered by long-term forecasts.
6 – The institution responsible for the management and protection of a natural park needs weather informations for the next 2 months due to the real fire hazard, having a big confidence in the ability of weather forecasts (H1 ) and worrying only a little in case of their error (H2), because it is always on alert. In this example, too, the risk is low and the forecast is very useful. Comment: environmental monitoring can only receive more or less useful contributions from weather forecasts, even greater if accompanied by constant updates.
