A Student's Guide to Cost-Benefit Analysis for Natural Resources
Lesson 10 - Conducting Sensitivity Analysis
Introduction
Sensitivity analysis should be part of any CBA. It attempts to determine how net benefits will change if specified parameters deviate from their anticipated values.
Sometimes we have expectations about how the values may deviate. But sometimes we do not. This gives rise to the concepts of risk vs. uncertainty.
Risk vs. uncertainty:
a) Risk - probability distribution of outcomes is known
b) Uncertainty - probability distribution of outcomes is unknown
But keep in mind, these two terms are frequently used interchangeably. Boardman, and others, seems to make a differentiation between risk and uncertainty. Many management science texts do not necessarily differentiate between the two.
Types of Sensitivity Analysis
A. Partial - the most commonly used approach, uses alternative values for individual key parameters. This is frequently applied to discount rates. That is, use several different discount rates and see if these alone will have a positive/negative impact on the NPV. A form of uncertainty analysis.
B. Worst & Best (extreme) Case analysis - determines if various scenarios (i.e., combinations of plausible assumptions about key parameters) will reverse the sign of net benefits. To do this:
i) Specify one worst case scenario (i.e., the combination of variables that could contribute to the lowest, possibly negative, NPV).
ii) Then, specify one best case scenario (i.e., the combination of variables that could contribute to the highest NPV).
The choice of scenarios will usually be made jointly by the analyst and the decision-maker.
Worst case analyses could be preceded by partial sensitivity analysis to see which variables are important to include in the scenarios. A form of uncertainty analysis.
C. Simulation - the study of a real system by using a model that replicates the behavior of the system. Simulation models use mathematical expressions and computers to replicate and predict the system’s behavior.
Simulation is one of the most widely used methods in management science. It is used in a wide variety of applications: automobile traffic flow, inventory systems, manufacturing processes, air traffic, etc.
Simulation can also be used in CBA to conduct sensitivity analyses. The general method is sometimes referred to as risk analysis (there we go with that term!). Risk analysis treats key CBA parameters as random variables with known or assumed probability distributions.
Risk analysis simulation example: The state wants CBA for a planned a campground. (This example is adopted from Anderson, Sweeney and Williams, An Introduction to Management Science, 1997, Ch 13.)
Certain costs:
a) RVD value - $125/day
b) administrative cost -$100,000/year
c) vehicle cost - $50,000/year
Uncertain (Random) costs:
d) labor cost -
|
cost |
prob. dist. |
|
$16/RVD |
0.1 |
|
$17/RVD |
0.2 |
|
$18/RVD |
0.4 |
|
$19/RVD |
0.2 |
|
$20/RVD |
0.1 |
|
SUM |
1.0 |
e) campground maintenance cost - $40 to $50/ RVD; uniform probability distribution
f) RVDs: μ= 7500 RVDs; σ = 1500 (normal probability distribution)
[68% of pop. is μ " 1 std, dev.; 95% is μ " 2 std. devs.; 99% is μ " 2.58 std. devs.]
Simulation Model of the CBA
1. Flow chart - if the problem is complex develop a flow chart
2. Formulate the problem:
TB = $125 x (RVDs)
TVC = labor cost/RVD x (RVDs) + Maintenance cost/RVD x (RVDs)
TC = TVC + admin cost + vehicle cost
Net Benefits = TB - TC
Simulating the CBA with Random Numbers & Probability Distributions
We must also determine values of:
1) labor cost/RVD
2) maintenance cost/RVD
3) RVDs
We use computer-generated random numbers and their probability distributions to determine values of the random variables.
EXCEL function RAND( ) placed in a cell will generate a random number: 0 < rand < 1.
We must know (or guess at) what these distributions are.
We will demonstrate 3 methods of Simulation:
Method 1. Cumulative Probability Function - labor cost
|
labor cost |
Prob. Dist. |
Cum. Prob. |
if Rand is |
|
$16/RVD |
0.1 |
0.1 |
0<0.1 |
|
$17/RVD |
0.2 |
0.3 |
0.1<0.3 |
|
$18/RVD |
0.4 |
0.7 |
0.3<0.7 |
|
$19/RVD |
0.2 |
0.9 |
0.7<0.9 |
|
$20/RVD |
0.1 |
1.0 |
0.9<1.0 |
|
SUM |
1.0 |
|
|
Value of = RAND( ) determines labor cost. Which row does it indicate?
EXCEL random number trial yielded = 0.550748; thus the value of labor cost is $18/RVD.
Method 2. Uniform Probability Distribution - maintenance cost/RVD; an equal probability of for all outcomes.
Solve: maintenance cost/RVD = a + rand(b-a)
where:
rand = a random no. between 0 and 1 =RAND( )
a = smallest possible value of maintenance cost
b = largest possible value maintenance cost
Note: this says that (Prob. range is proportional to cost range: 0 : 1 : $40 : $50
EXCEL trial yielded: $40 + .321034 ($50-$40)= $40
= $43.21 maintenance cost/RVD
this is maint. cost/RVD generated as a random variable
Method 3. Normal Probability Distribution - RVDs
Assume that this population has: μ = 7500 RVDs and σ = 1500
EXCEL method for determine a randomly generated value from a normal probability distribution:
=NORMINV(RAND)( )mean, std. dev.)
=NORMINV(RAND( ) 7500, 1500)
An EXCEL trial yields a random RVD = 5442 RVDs.
Method 4. Solve the Simulation Model
Placing the 3 random variables in to our simulation model yields:
TB = $125 x 5442 RVDs = $680,250
TVC = $18 labor cost/RVD x 5442 RVDs
$43.21 Maintenance cost/RVD x 5442 RVDs
= $235,148.82
TC = TVC + $100,000 admin. cost + $50,000 vehicle cost
= $235,148.82 + $150,000
= $385,148.82
Net Benefits = TB - TC = $680,250 - $385,148.82
Net Benefits = $295,101.18
Method 5. Monte Carlo Simulation
Monte Carlo simulation - multiple simulation trials where each trial is an independent event. Repeated trials enable the development of a frequency distribution of net benefit results so that the analyst can determine: 1) central NPV values (i.e., mean and median), 2) and the NPV range. MC simulation has been in use with CBA since 1968; thus, it has shown usefulness, hence staying power.
The utility of the Monte Carlo simulation method rests in the law of large numbers - that is, as the number of trials gets very large (perhaps 500 to 1000 trials) the estimated mean, median & range will converge on their true underlying values. (Assuming that the prob. distributions are correct.)
There are software programs to do Monte Carlo simulation, however it can be done with an EXCEL spreadsheet.
Steps in Monte Carlo simulation analysis are:
i) specify a probability distribution for each parameter that you want to assume is random.
ii) a trial: for each parameter, take a random draw from the distribution to obtain values for computing net benefits (as we have just done).
iii) repeat the trial many times (maybe 1000 times) computing the net benefits for each trial, compute the average = the E(net benefits). Because the random numbers are uncorrelated, then the average is the E-value.
See Anderson, Sweeney and Williams for an EXCEL Spreadsheet Monte Carlo example. You will use these EXCEL functions:
1) =VLOOKUP(RAND( ),$A$3:$C$7,3)
2) =NORMINV(RAND (),7500,1500)
3) use Tools - Histogram
4) use Tools - Descriptive Statistics (for: mean, median, std. dev., range)
Monte Carlo caveat - in order to be useful, a MC simulation model must be realistic and it must be understood.
III. Expected Value
(Under construction)
Links: