data <-read.csv('expert_scores.csv') #raw sensitivity data from FSCVA Portal
species.data.list <- split(data, data$Stock.Name) #generate a list of data.frames, where each species has its own dedicated data frame of scores from all scorers in the list
species.sensitivity <- lapply(species.data.list, calculate.sensitivity, bootstrap = F) #calculate attribute scores and total sensitivity Sensitivity Calculation
Analysis
Sensitivity scores were collected using the Fish Stock Climate Vulnerability Assessment Portal, and exported as a csv. Scores were calculated for each attribute and these scores were used to calculate a final exposure score for the species. Unlike exposure, and similar to the original analysis, a single value is produced for each species.
Attribute Scores
Attribute scores were calculated using a weighted average following Equation 1 in Morrison et al 2015. Briefly, the number of tallies in each of the four bins is multiplied by a weight, summed, and divided by the total number of tallies (which would be 5 scorers per species x 5 tallies per scorer = 25). The weights that correspond to each bin are:
| Bin | Weight |
|---|---|
| Low | 1 |
| Medium | 2 |
| High | 3 |
| Very High | 4 |
The function attribute.score performs the weighted average. This function is based on code provided by the Tyler Loughran and the Atlantic Highly Migratory Species Management Division, Office of Sustainable Fisheries. The workflow is outlined below. Individual attribute scores are also accompanied by a data quality score, which are provided by expert scorers for each species and attribute to help identify data gaps. This is calculated using the data.quality.score function.
Total Sensitivity Scores
Following the original CVA methods, total sensitivity is calculated with a logic rule. This is the same logic rule used to calculate total exposure from individual variable exposure scores:
| Overall Score | Numeric Score | Logic Rule |
|---|---|---|
| Low | 1 | All other scores |
| Medium | 2 | 2 or more attributes >= 2.5 |
| High | 3 | 2 or more attributes >= 3.0 |
| Very High | 4 | 3 or more attributes >= 3.5 |
The total sensitivity score is calculated using the logic.rule function. Certainty was calculated by resampling the tallies with replacement across the four bins, and recalculating the weighted average attribute score, then the total sensitivity. Following the original CVA, certainty is represented as the proportion or percent of bootstrapped total sensitivity scores that match the expert-derived total scores. The certainty associated with the total sensitivity score is calculated with the bootstrap.certainty function.
Workflow
Calculating Scores
A single function, calculate.sensitivity, combines both the attribute score calculation done by attribute.score and the total sensitivity score calculation done by logic.rule in a single function. This function is modeled on provided code from the Southeast Fisheries Science Center.
calculate.sensitivity returns a vector, containing the individual attribute scores and the total score. When used with lapply as in the example above, a list of vectors will be produced, with one vector per species in the provided species.data.list.
Calculating Data Quality
The function data.quality.score calculates the mean data quality score for each attribute and species. It produces a vector similar to calculate.sensitivity, with an entry for each attribute, and works with the lapply family of functions.
data <-read.csv('expert_scores.csv') #raw sensitivity data from FSCVA Portal
species.data.list <- split(data, data$Stock.Name) #generate a list of data.frames, where each species has its own dedicated data frame of scores from all scorers in the list
data.quality <- lapply(species.data.list, data.quality.score) #calculate data quality scoresCalculating Certainty
Certainty was quantified using the bootstrapping methods described above. For the default number of iterations (10,000), this function takes just under a minute (~45 seconds) per species on a standard workstation.
Both attribute.score and logic.rule, and subsequently calculate.sensitivity have the ability to also perform the bootstrap calculation.
data <-read.csv('expert_scores.csv') #raw sensitivity data from FSCVA Portal
species.data.list <- split(data, data$Stock.Name) #generate a list of data.frames, where each species has its own dedicated data frame of scores from all scorers in the list
sensitivity.bootstrap <- lapply(species.data.list, calculate.sensitivity, bootstrap = T, samples = 10000) #calculate sensitivity with bootstrapping When bootstrap = TRUE, a data.frame, rather than a vector, is produced. This data.frame will have a number of rows equal to the number of samples, with each row representing one of the iterations. Columns will contain the attribute scores and total sensitivity score for that iteration. When used with lapply as in the example above, a list of data.frames will be produced, with one data.frame per species in the provided species.data.list.
The function bootstrap.certainty calculates the final certainty value (AKA the proportion of bootstrap iterations that match the expert scores). This function will calculate the proportion of bootstrapped total sensitivity scores correspond to the total sensitivity score (1-4). It adds a column called ‘Certainty’ to the vector generated by calculate.sensitivity(bootstrap = FALSE).
The entire workflow can be run using lapply and mapply functionality on the entire raw dataset from the FSCVA portal using only a few lines of code:
data <-read.csv('expert_scores.csv') #raw sensitivity data from FSCVA Portal
#create list of data.frames for each species/stock
species.data.list <- split(data, data$Stock.Name)
species.sensitivity <- lapply(species.data.list, calculate.sensitivity, bootstrap = F) #calculate sensitivity w/o bootstrap
sensitivity.bootstrap <- lapply(species.data.list, calculate.sensitivity, bootstrap = T) #calculate sensitivity with bootstrap to calculate certainty later
#get certainty
sensitivity.certainty <- mapply(bootstrap.certainty, sensitivity.bootstrap, species.sensitivity, SIMPLIFY = F)
sensitivityDF <- do.call(rbind, sensitivity.certainty) #convert list of vectors to a data.frame
write.csv(sensitivityDF, './sensitivity.csv') #save results
## make data.quality spreadsheet
dq <- lapply(species.data.list, data.quality.score) #calculate mean data quality scores
attributeDQ <- do.call(rbind, dq) #convert list of vectors to a data.frame
write.csv(attributeDQ, './data_quality.csv') #save resultsThe resulting sensitivity data.frame shows the scores for each attribute, plus the total sensitivity and corresponding certainty:
X Adult.Mobility Complexity.in.Reproductive.Strategy
1 Acadian redfish 2.72 1.84
2 Alewife 1.60 3.25
3 American conger 1.45 2.15
4 American eel 1.15 2.05
5 American lobster 1.90 1.40
6 American plaice 2.75 1.45
Dispersal.of.Early.Life.Stages
1 2.04
2 2.80
3 1.35
4 1.10
5 1.80
6 1.45
Early.Life.History.Survival.and.Settlement.Requirements Habitat.Specificity
1 2.20 1.72
2 3.28 2.56
3 2.25 2.55
4 2.45 2.05
5 2.35 1.85
6 2.15 1.80
Other.Stressors Population.Growth.Rate Prey.Specificity
1 1.92 3.52 1.96
2 2.68 2.24 1.68
3 2.24 2.00 1.65
4 2.45 2.70 1.10
5 2.70 3.45 1.15
6 1.25 2.45 2.10
Sensitivity.to.Ocean.Acidification Sensitivity.to.Temperature Spawning.Cycle
1 1.12 1.80 2.60
2 1.56 2.16 3.16
3 1.25 1.70 2.90
4 1.15 1.40 2.50
5 3.85 2.70 2.90
6 1.90 1.65 2.40
Stock.Size.Status Total.Sensitivity Certainty
1 1.84 2 0.9998
2 2.52 3 1.0000
3 2.40 2 0.6946
4 2.55 2 0.7234
5 2.20 3 1.0000
6 2.95 2 0.9882Not shown here, but the data quality data frame has a similar layout, with a row for each species and a column for each attribute. There is no total sensitivity or certainty column.