On January 28 and 29, the Delta Science Program convened an independent science review panel for the third phase of the review of the Bay Delta Conservation Plan’s Effects Analysis contained in Chapter 5 of the plan documents.
The Effects Analysis evaluates the biological outcomes of the BDCP, or the impact of the plan’s actions on covered species; it is distinctly different than the environmental impact document, which evaluates the impact that the project would have on the environment and resources. Using a wide-range of analytical tools, primarily models, the BDCP’s actions are analyzed to determine how the Delta’s ecosystems, natural communities, and covered species might respond, and then presents conclusions on the outcomes from implementing the conservation strategy. In producing the analysis, 68 different models were used, including hydrologic and hydrodynamic models, temperature models, water quality models, biological lifecycle models, habitat models, and conceptual models. For more on the Effects Analysis, visit this page of the BDCP Road Map: Effects Analysis: Analyzing the biological impacts of the plan
On the first day of the meeting, Dr. Peter Goodwin, Lead Scientist with the Delta Science Program, gave a brief introduction:
“
At the request of the agencies responsible for the Bay Delta Conservation Plan, the Delta Science Program was asked to convene an independent science review panel to assess the scientific soundness of the BDCP’s effects analysis. The Effects Analysis is an absolutely critical component of the BDCP, and it’s intended to provide the best scientific assessment of the likely effects of BDCP actions on the species of concern, and the ecological processes within the Bay Delta ecosystem.
“I’d like to stress what this panel is about. This review is all about the science of BDCP, and only the science. It’s not about the policy or decisions about implementation.
“The importance of the role of science in guiding the development and review of the BDCP is very well established in state law under the Delta Reform Act which is overseen by the Delta Stewardship Council, and in several statements by Governor Brown and the Secretaries of the U.S. Departments of Interior and Commerce.
“We would also like to acknowledge the role of Sue Fry and Reclamation staff and Laura King Moon and her staff at DWR for really seeking out the spirit as well as the letter of the law. They have made it very clear they want the highest quality scientific review to guide future refinements in the analysis, and both agencies, along with their consultants, ICF, have been very responsive to the requests of the science program as well as those requests from the panel.
“Just to put this project in perspective, we all know that this is a massively complex open system. We also know that this system is not in a static state of equilibrium; it’s highly dynamic and responding to multiple stressors and multiple management actions, so this is an incredibly difficult problem. And of course we do know a lot about the Bay-Delta system, as this is one of the most studied estuaries in the world. But there is considerable uncertainty and it’s very challenging to predict outcomes in such an open system, particularly when we make a major perturbation that is outside what the system has experienced in the past.
“As scientists, when we conduct reviews, we’re trained and our tendency is to focus on the areas which are either poorly understood or inadequately analyzed. However, we’ve been asking more of this particular panel. This scientific analysis has been a massive undertaking, and the analysis framework and methodology is likely to form the basis for understanding the future of the Delta ecosystem, irrespective of if BDCP goes ahead or not. So the panel is asked to comment on the robustness of the analytical framework, as well as the strengths and challenges of the analysis approach. Where are the greatest uncertainties and have they been quantified adequately?
“This is very important as managers in the future will have to determine whether there is sufficient flexibility and contingencies within the adaptive management framework to compensate for the uncertainties or unanticipated consequences. …”
Dr. Goodwin noted that this was the third phase of the review, a process that began with the first review in October of 2011. He noted that the panel’s findings will be used to refine the BDCP Effects Analysis, as well as inform other reviews of the BDCP, including the Delta Stewardship Council, the California Department of Fish and Wildlife, the Delta Independent Science Board, regulatory agencies and the public.
Dr. Goodwin thanked the panel for their ‘dedication, flexibility, willingness, and very considerable effort that they have put into the reviews.
The first day of the review, ICF gave presentations on the Effects Analysis. The presentations from ICF are not covered, but I have provided this handy guide so that you can find the parts that interest you:
- Click here for the webcast: BDCP independent science panel review of the Effects Analysis, phase 3
- Click here for ICF’s power point presentation.
| Seg #1 | Introduction by Dr. Peter Goodwin; introduction of panel members | ||
| Seg #2 | Steve Culberson, US Fish & Wildlife Service | ||
| Seg #3 | Mike Tucker, USFWS | ||
| Seg #4 | Carl Wilcox, DFW | ||
| Seg #5 | Laura King Moon, DWR | ||
| Seg #6 | ICF Introduction | Slides 1 – 5 | Jennifer Pierre, ICF |
| Seg #7 | Impacts to Greater Sandhill Cranes and Refinements to the Conservation Strategy Currently Being Explored | Slides 6 – 25 | Ellen Berryman,Terrestrial biologist, ICF |
| Seg #8 | Effects of Clifton Court Forebay Changes | Slides 26 – 29 | Marin GreenwoodAquatic Ecologist, ICF |
| Seg #9 & #10 | Assessment of Far-field Hydrodynamic Changes in the North Delta and the Effects on FishHydrodynamics ModelingAnalyses of reverse flows at Georgiana Slough | Slides 30 – 62 | Chandra Chilmakuri, CH2M Hill, member of the modeling team |
| Seg #11 | North Delta intake Far-Field Effects on Fish;Restoration Effects | Slides 63 – 78Slides 79 – 93 | Marin GreenwoodAquatic Ecologist, ICF |
| Seg #12 | Assessment of Upstream Effects, Specifically in the Feather and Sacramento RiverEffects Downstream of Plan Area | Slides 94 – 108Slides 109 – 111 | Rick Wilder, ICF |
| Seg #13 | Decision Tree Analysis | Slides 112 – 120 | Jennifer Pierre |
| Seg #14 | Development of the Fish Net Effects | Slides 121 – 124 | Marin Greenwood,Aquatic Biologist, ICF |
| Seg #15 | Discussion | ||
| Seg #16 | Discussion | ||
| Seg #17 | Public Comment: Burt Wilson, Public Water News Service | ||
| Seg #18 | Public Comment: Doug Obegi, NRDC | ||
| Seg #19-22 | Second day: Panel’s Initial Findings |
On the second day, the science panel members met in the morning to discuss the presentations, and later on that day, presented their initial findings to the ICF panel. (The initial findings were presented in this power point.) The Chair of the independent science panel, Dr. Alex Parker, lead the presentation & discussion.
Dr. Parker started by saying that today the panel would be giving their initial comments and impressions, which are somewhat lacking in the specifics at the moment, but that more detail would be contained in the panel’s report. “Our first impression was that there were many positive things,” he said. “In particular, a lot of the phase 1 and phase 2 comments and suggestions have been addressed, at least to some degree. We were pleased to see the additional analyses in some cases, and an improved literature review for the Bay-Delta. We are pleased with that overall.”
Dr. Parker said that nearly all of the actions or conservation measures have the potential to benefit the species of concern, and the panel acknowledges the monumental effort that is going into BDCP to meet those intended goals. “Of course, the question is will the species of concern benefit to the degree that is described in the Effects Analysis, and that’s largely our job to help evaluate the effects,” he said, noting that he would start with the broad areas of concern, and then discuss the panel charge questions.
Chapter 5 is very large and complex; it is meant to be a summary document, said Dr. Parker. However, the panel felt that they ultimately had to go to the associated appendix for the detail and analysis, and often there wasn’t a sufficient road map to find the information we were looking for. In some cases, it also meant additional chapters, such as Chapter 3, which aren’t under the panel’s purview, he said. “That was a challenge for us.”
Some of the organization is pretty repetitive and to some extent, that may be unavoidable, but it makes processing the document very challenging, he said. Overall, in chapter 5, the panel thought there were some places where it was very clear the sheer number of multiple authors and a lack of consistency in streamlining the document as well as inconsistencies in how the document was put together, he said.
“So much of the analysis is dependent upon the understanding of uncertainty, and in some cases it makes real assessment of the effects virtually impossible for us at this time,” said Dr. Parker. “Overall there was fairly inconsistent treatment of uncertainty throughout the effects analysis, within appendices, and in the net effects. … Within chapter 5, the uncertainty may have been stated but not really linked with the overall outcome, whereas within the various appendices, there was greater emphasis placed on the uncertainty, although inconsistently.”
The panel recognizes that the success of BDCP hinges on a commitment to effective adaptive management, he said. “Although it doesn’t fall under our purview in this review, adaptive management and really linking the effects analysis with the adaptive management was lacking with virtually no mention of it within Chapter 5,” said Dr. Parker.
The effects analysis for many of the biological objectives is incomplete right now, and the lack of specificity with the location and configuration of the Restoration Opportunity Areas made it difficult to understanding what the effect or potential effect is, he said.
Dr. Parker then turned to the specific questions the panel was charged with answering. The first question is how well does the Effects Analysis meet its expected goals? “This is where we have real trouble with accessibility to the information,” said Dr. Parker. “There’s a sense that much of the information is probably there – it’s almost certainly there, but even up until and through yesterday, we really needed the dialog with you directly to find some of that information. … We just need better linking throughout – better cross-referencing and better indexing throughout because without it, we are left with this sense of ‘trust us.’ Without the level of detail, we’re really hunting.”
The second question was how clearly are the methods described? “Again, this comes down to some of the cross-referencing and linking, the detail methods are often just simply difficult to find,” he said, noting that some things, such as how models are linked or coupled together, are not obvious from the reading. “It’s placing a real burden on the reader.”
Within the net effects, the approach to evaluate the relative importance of each attribute is mentioned, but it’s not explicitly incorporated into the net effects analysis, and of course this would confound the interpretation of overall net effects, said Dr. Parker.
Panel member Dr. Greg Ruggerone clarified that in reading Chapter 5, it was mentioned that the relative importance of each attribute would be described, but the impact analysis and the summary table does not have evaluate the relative importance of each of the attributes. “This confounds a roll-up of the overall net effects,” he said, noting that in the previous draft, there was such a chart. One of the scientists with ICF said that the information was there, but it was in the text. “It’s hard when you have 15 stressors to describe the relative importance of each one of those in text,” said Dr. Ruggerone. “It would be helpful to have it in a tabular form.”
The panel was asked if the effects analysis is reasonably and scientifically defensible. Dr. Parker noted that they had several points to address on this question. The first comment the panel had was that the net effects per species-specific fish is highly uncertain, he said. “This comes down to the sense that the individual attributes for each fish species are probably reasonable, but the net effects resulting from these individual attributes is questionable,” said Dr. Parker.
“That’s obviously an area that we’ve been challenged with from the beginning,” said Jennifer Pierre of ICF. “Any input or advice in help in how you would with that integration … that is certainly one of the biggest challenges we have with the fish net effects, so we are very open and would very much appreciate the input on that. It’s very difficult to do.”
“Unless you weight the attributes, you can’t determine the overall uncertainty,” said John Skalski, a panel member. “An attribute that has high importance, even one of them, that has high uncertainty, makes the whole program uncertain. On the other hand, you might have a lot of attributes that are very insignificant with a lot of uncertainty but it has no effect on the overall assessment. Uncertainties don’t average; sometimes they multiply, but it depends on your model, so unless we know the weight of the attributes and how certain you are of those individual components, we can’t put the pieces together.”
“There’s a feeling that simplistic assumptions were incorporated into the Delta passage model for application for net effects, with assumptions based on tag studies of large hatchery and migrating fish and the assumption that shorter residence time for fish and the Delta equates with higher survival,” said Dr. Parker.
“I don’t want to say that this wasn’t acknowledged,” offered science panel member Charles “Si” Simenstad. “The thing is that it carries through to the broader analysis of in-Delta survival, and it would be really wise at least to balance the migrators with the foragers and talk about the distinct differences in those, even though the mortality estimates or the survival estimates are less dependable and less available for those fry-type foragers. I think you need to show the distinct difference amongst the stocks and those life history types so that those then relate more to the restoration, because so much of your restoration actions are aimed towards the foragers, which you have the least information on. I’m not suggesting that you don’t use those data, but I think it’s important to show what little data there is to describe the other components.”
The panel felt that there was a lack of consideration of a variety of scenarios, including moderate or worst case scenarios, with predominantly optimistic scenarios in modeling, said Dr. Parker. He cited food web models and turbidity as examples that should look at a range of scenarios and bracket the results. “There was a general comment overall that there should be more model bracketing with some of these ecosystem-based models,” he said.
Restoration of tidal wetlands is highly uncertain or a very long process, and yet restoration is assumed to be 100% perfect for meeting goals, said Dr. Parker. Science panel member Nancy Monsen offered some clarification on this point. “When you make a flooded habitat, there are a lot of varieties of things that could happen,” she said. “You could end up with clams in there and those clams could eat up all your phytoplankton, so then is it effective habitat? Or have you reached the goal you wanted with that habitat? You may end up with SAV in there. That’s going to put you on a different trajectory, so is it 100% effective? When you put these in there, we don’t know that they are actually going to produce what we want them to produce,” adding that it will also take awhile for these things to establish.
Science panel member Charles “Si” Simenstad noted that there were some analyses that gave an idea of trajectories in terms of time development, but other analyses showed trajectories which were mostly appear to be acquisition of wetlands. “It really would be worthwhile to develop a small section that consolidates, for example, the PWA analysis with what do you expect to see in terms of trajectory and development to what you might consider a functioning tidal wetland,” he said. “Because there are a lot of contingencies in depth, subsidence, sediment supply and things like that, so that is where a couple of scenarios or more would be helpful in bracketing what the time frame is before those come online.”
“We can appreciate the reason behind some of the lack of specificity in the location and configuration of Restoration Opportunity Areas, but it’s certainly going to limit conclusions throughout many aspects of the effects analysis,” said Dr. Parker. “For instance, the implications for connectivity and hydrodynamics, this suggestion that by restoring large areas of tidal wetlands, we’re going to produce a lot of organic matter subsidy for the food web. That’s very hard to evaluate when we have no sense of whether these wetlands will be hydrodynamically connected to the larger bay or Delta.”
Another example is Mildred Island, offered science panel member Nancy Monsen. “We know that the exchange in the north part of Mildred Island is very different than the south part of Mildred Island, so as you develop these restoration areas, it’s going to be unique to where you are at, what the bathymetry looks like, where are those connections to the channels, how big are those connections, and what kind of tidal interaction do you have between the interior of that area and the channel. Those are very important details, and because we don’t know where those locations are and what those connections are, it makes this effects analysis more difficult to tease apart.”
The panel also felt that the physical changes at Clifton Court Forebay need more evaluation before implementation. “There is an overall lack of documentation in the construction appendix on the impacts to fish and also the justification for this barrier – which comes at considerable cost,” said Dr. Parker. “What are the benefits? There was some suggestion it felt a bit like an afterthought.”
Science panel member Nancy Monsen said that the construction appendix did not include a discussion of how installing a wall in the middle of Clifton Court Forebay was going to affect the fish. “There was also a suggestion that there won’t be fish in the north part of the court. I’m not sure that’s going to happen, and I’m not sure it’s really necessarily to have a barrier between the north and the south, whether they can just drain into one location. I don’t understand what the benefit for fish is for that barrier to be there.” She said that maybe Clifton Court Forebay needed to be expanded as an engineering necessity, but the details are lacking in the documentation.
“It’s not that either side won’t have fish, it’s that the side that comes from the tunnels won’t have the listed fish,” said one of the ICF representatives. “They will be screened out, so that water will be basically listed-fish free. It means that you don’t have to run it through the salvage operations, which is a massive time and effort. The more water you run through, the harder it is, so if you have a large amount of water that you don’t have to run through [the salvage facility], that really saves a lot of effort. Then you have the other part which is coming in from the Delta and it will have listed fish which will have to run through the salvage operation, so I think that’s the reason for separating the two.”
“I think it’s just valuable to write up what your thoughts are behind that and to explain, even to the general public, your justification for wanting that facility there, because people will see this as a big expense to do that construction, so you really need to be able to justify that,” said Ms. Monsen. “For instance, to say that we don’t have to put it through the screening facility is very important because that long-term will save you money. But it’s important for people to understand that, and it was not obvious to me.”
Dr. Parker said that he struggled with the contaminants appendix, noting that Appendix 5F didn’t really deliver on the indirect effects, with the exception of Microcystis. “Just as an example, the herbicide used as part of conservation measure 13, there’s some discussion about the impact to microcystis, but I wonder about those indirect food web effects and whether that could be explored a little bit more fully.”
The panel overall feels that the effects of changes to the conservation measures should include San Francisco Bay, said Dr. Parker. “Reality is that there is connectivity, and the BDCP will likely have impacts downstream, particularly the loss of suspended sediments, and that impact to salt marsh in the face of sea level rise. … At this time we feel like it should be included.”
The panel felt there was an overall lack of synthesis and interacting effects in feedbacks, such as the restoration placement effects in terms of hydrodynamics, Dr. Parker said, citing examples of impacts of the construction of restoration habitat on Microcystis and residence time, or how nutrient dynamics might influence Microcystis.
“Presently, the food web analysis is interesting,” said Dr. Parker. “The conceptual model looks pretty good, but the food web right now is partitioned into phytoplankton versus detritus, rather than really treated as an integrated system in the analysis.” He noted that there could be contributions from other sources, so this is something that needs to be considered further.
The effects analysis is incomplete for the biological objectives, Dr. Parker said, noting that only a limited fraction of objectives can be evaluated at this time. This highlights the uncertainty in achieving the objectives as well as the need for monitoring and adaptive management to ensure that these objectives will be achieved.
Dr. Greg Ruggerone said that per a table from the beginning of chapter 5 that for covered fish species, only 11 or 28% of the biological objectives could be evaluated in your opinion, 38% of them were partially evaluated, but 33% were not evaluated at all. “This is unfortunate, because the biological objectives are where the project wants to go. You want to be able to achieve those objectives and yet we don’t have the information in the basin to evaluate at this point in time whether or not those objectives might be achieved, so therefore this raises uncertainty in the overall effectiveness of the plan and highlights the need for monitoring and adaptive management as the project continues.”
Another part of that question 3 was how clearly are the net effects results are conveyed in the text, figures, and tables, said Dr. Parker. He discussed a particular example where the graphic representation of the net effects could have been presented differently and could be improved.
“There is no quantitative assessment of take, and the take and the benefits are not scaled to each other to assess the net effects,” said Dr. Parker. Science panel John Skalski offered further clarification on this point. “The problem is the qualitative scale of high, low, to moderate, is fine as long as you operate on the same scale, but if you’ve got take that, for instance, is designed primarily for one type of group of fish, the migrators, and you’ve got benefits designed for another group, those scales are very likely not the same,” he said. “It was a challenge for us to try to figure out what the net effects were and what you would basically plug in to an assessment of HCP benefit if you’re looking at stressors on one component or a restricted component, and benefits to another. That’s specific for salmonids, but not necessarily for some of the other species and certainly not for the terrestrial species.”
There is a mismatch of potential benefit from conservation action and how they are assessing it, said Dr. Parker, noting that this is a broad comment that addresses the communication of uncertainty. “Overall the broad consensus was that the level of detail within Chapter 5, the summary sections, that level of uncertainty is often downplayed, so there is this sense that prediction is there’s going to be a net benefit but without adequately capturing the level of uncertainty around that prediction,” he said. “Within the associated appendices, there’s more explicit discussion but again, it’s disconnected from those summary pages.” He suggested that the uncertainty be more completely addressed in the summaries.
There are two sorts of problems, Dr. Parker said. One is the inconsistency within from multiple authors, and then the inconsistencies with the way uncertainty is reported, he said. “Appendix 5F reports these nice sections, concise sections of research needs, which really highlights data gaps and uncertainty, and those are really helpful – I certainly found those to be really helpful,” he said. “However, it was inconsistent across appendices. We saw it there and nowhere else.”
The next slide addressed restoration. “With respect to uncertainty, I think what this slide is really reflecting is that within restoration-wide range of systems that are slated for restoration, the trajectory or the ability to achieve restoration on short time scales is not very good. And there’s a real difference across these different landscapes in terms of restoration.”
Someone from the ICF panel (video did not clearly show who) asked for clarification on the third bullet on the slide which read ‘Some are difficult, eg vernal pools, grasslands, others are relatively easy for functions’.
(Science panel member, not shown): “The functions in certain places are extremely difficult to restore. For example, riparian forests, you can get a forest, and you can get shade, and you can get a lot of the functions of it. It’s not necessarily fully restored but you can get a lot of the things you are looking for. … What we’re really saying is that everything is treated relatively equally but some systems are very difficult and others aren’t.”
“I think that’s a fair comment,” responded David Zippin of ICF International. “We could do a better job, perhaps, of explaining our intent with the grassland restoration measure, for example, which isn’t very common – it is difficult. And we acknowledge, even if it’s not stated clearly, that we don’t expect to get a completely native grassland at the end. It’s certainly going to be a mix of natives and non-natives, but we think it would still function perfectly fine for at least the covered species that we’re intending it to support. I think we have more modest goals than it may appear, certainly by the title. So we can certainly clarify that. With the vernal pool restoration, that’s actually something we do have a lot of experience with in the Central Valley and again for the target species, we’re fairly confident we can support them. Also our acreage targets are quite modest there, too, and that’s another way we’ve tried to limit that uncertainty, given the challenges with that restoration, so we can do a better job of explaining that.”
Dr. Parker said there was also a lot of uncertainty related to suspended sediments and around the loss of sediment under the new configuration, and the acknowledgement that there will be less suspended sediments and making up for that.
Science panel member Nancy Monsen offered some further clarification. “As you are adaptively managing the north Delta operations, that turbidity is one of those things that you adaptively manage for,” she said. “You recognize that there is 8 to 9% sediment that we’d like to keep in the Sacramento, if possible, and if there’s something you can do with pump operations – if you see a pulse of sediment that possibly goes down, maybe you back off the pumping for that period of time while the sediment goes down. That’s just an example of how you could adaptively manage sediment in the system.”
The next question in the panel’s charge is how well does the effects analysis describe conflicting models and analyses and how are they interpreted?, said Dr. Parker. He noted that there is an overall lack of consideration of propagation of errors or sensitivity analyses in these linked models. He asked science panel member Nancy Monsen to elaborate on this point.
Ms. Monsen indicated this was something she had given a lot of thought to over the years. Her experience with the USGS’s CASCADE program, where they linked watershed models to CalSIM to the Delta model to phytoplankton and temperature, gives her some perspective on this point. “We recognized that is a difficult thing and each model has error associated with it,” said Ms. Monsen. “As you plug one model into the next model, you’re dealing with the propagation of those errors. In the next month, I’m going to go back and look at some of the conclusions that CASCADE made about how to deal with that, so I can give you recommendations of how to deal with that error, but we have to recognize that the error does exist as we put one model on top of the other model.”
She referred to the point discussed yesterday, where the 2D and 3D models aren’t quite lining up with the 1D model. “Here’s somewhere where there’s an error, and you have to ask the question, well, is that error is important for the question that we’re asking?,” said Ms. Monsen. “As I understand it, there was a presentation to the ISB last week about looking at the CalSIM II model and the assumptions made with climate change and what was done in 2009, and here again, they are going to be different, so you have to ask yourself why, and are my ultimate conclusions hinging on that model? These are very complicated models and there are a lot of knobs to tweak, so you really have to figure out what is the question that you’re wanting to answer with these really complex models. Because you’re taking these complex models and then distilling down into some very specific questions.”
Another one of our questions is how well does the effects analysis link to the adaptive management plan and associated monitoring programs?, said Dr. Parker. “Given the high level of uncertainty that we’ve been talking about, everyone is well aware of the success of the BDCP does hinge on effectiveness of monitoring and an adequate adaptive management framework,” he said. “It’s noted but not well described in Chapter 5. It’s really outside of the actual scope of the review, but in order for us to understand the effects analysis, we had to go to these other sections.”
The next slide stated “Adaptive management process does not specify how responses to BDCP by populations of concern will be detected and linked to conservation actions.” Dr. Parker asked a member of the science panel to clarify. (Video does not show who responded.) Science panel member says, “The real question is how, with a lot of uncertainty about population estimates, mortality rates and survival rates, do you establish actions that are based on … Usually in adaptive management, there is a conceptual model, and if you reach this level, there’s a trigger that you move to a different resolution or a different approach. How do you bridge that gap that you’ve got? Especially with something like Delta smelt. How do you know that they are responding to your actions, and that the level of response, is sufficient to meet your biological objectives, or do you need to take another action. Even something supposedly as well populated as salmon data has the same problems, especially in the Delta, so I guess the real question is can you anyway more specific or at least describe how you’re going to get around that gap.”
The next slide dealt with salmonid restoration. “There wasn’t much attempt to bring in information from the literature on, for example, salmonid responses to habitat restoration, so we thought we might share some of the results,” said science panel member Dr. Greg Ruggerone. “There’s been a lot of habitat restoration for salmonids through their range, unfortunately, there’s not many studies that actually look at the response of salmonids to that habitat changes, but fortunately Phil Roni with NOAA fisheries and with his colleagues, have published two recent review papers. One of them dealt with habitat restoration for coho and steelhead throughout Puget Sound, and this evaluation was based on empirical data and simulation modeling of those data.”
Dr. Ruggerone continued: “I thought the conclusion from their paper was quite relevant here, so I’ll just read the conclusion from their abstract. They said: ‘Given the large variability in fish response, such as changes in density or abundance to restoration, 100% of the habitat would need to be restored to be 95% certain of achieving a 25% increase in smolt production for either species.’ Ultimately, they concluded that ‘our study demonstrates considerable restoration is needed to produce measurable changes in fish abundance on a watershed scale.’ So this just raises the question that we’re asked a lot in the Columbia River Basin by policy makers … and we ask the question here, with the BDCP, is the amount of restoration that’s being proposed, is it enough to achieve the biological objectives set forth in recovery?”
The last slide stated that ‘it is unnecessary to extend an uncertain but potentially positive effect to statements about species conservation, especially under future climate scenarios.’ Dr. Ruggerone added additional clarification. “This gets to the specific statements where we thought there was a bit of an overstatement, and they are kind of subtle overstatements of benefits,” he said. He noted that the first quotation regarding winter Chinook is taken from the summary of Chapter 5. “The quotation is ‘the magnitude of benefits for winter run Chinook salmon at the population level cannot be quantified with certainty,’ and that’s a good statement. Nonetheless, it continues, the overall net effect is expected to be a positive change that has the potential to (I underlined it) increase the resiliency and the abundance of winter run Chinook salmon relative to existing conditions. And I see the logic, but when you can’t quantify the magnitude whatsoever, then to suggest that it might have an ability to increase the resiliency of the population in front of climate change and so forth may be stretching it a bit.”
Dr. Ruggerone continued. “Another statement in the same area of the report: ‘the BDCP should help conserve the species in the plan area and help it cope with expected climate change, and again here, I see where it’s coming from, you’re concluding a net overall effect but in terms of conserving the species, especially for salmonids, this is just a narrow window in their life history. There is a lot going on upstream and a lot going on in the ocean as well, so to the extent that it can actually help conserve the species is perhaps a bit of an overstatement. And then the last quote example is, ‘the BDCP may lead to more robust Chinook salmon populations with resiliency and diversity, necessary to cope with a changing environment.’ Again, that, if it’s able to cope with a changing environment, especially given some of the scenarios under climate change, that implies a fairly large beneficial effect which you really haven’t concluded from the impact analysis for the BDCP, so the thought is that it’s a bit of an overstatement.”
Dr. Parker indicated that it was the last slide and therefore the conclusion of the panel’s presentation.
Dr. Zippin then offered an explanation for the second bullet, which stated “The BDCP should help conserve the species in the Plan Area and help it cope with expected climate change….” He said, “As I’m sure you know, California Department of Fish and Wildlife has to make findings for every one of our covered species that we have provided for the conservation and management of those species in our plan area. That is actually the regulatory finding they need to make, so I don’t know the context of this particular excerpt, but in many cases, we did try and reach that conclusion when we thought it was justified, using those regulatory terms,” he said. “So we can be clearer in the document when we are using the word ‘conserve’ in that regulatory context, which is limited to just the plan area, so I think that would address your comment about the fact that this is a narrow window of a species life history and we certainly recognize that. And we’ll also try not to use that word in the broader context, which some might interpret it as being, and so I think that would help.”
“I agree,” acknowledged Dr. Ruggerone. “It’s tough wording.”
One of the members of the ICF panel noted that one of the bullets was missed on a previous slide. Paging back to the slide, the bullet said ‘In cases where multiple models (i.e. IOS and OBAN) are used a comparison of model outputs is lacking.’
“With regard to the salmon assessment using IOS and OBAN models, for example, you’d have a number of water years, and then you have the fourteen different scenarios, and you calculate expected adult returns or in-Delta salmon survival, and then you’d average across the years and compare the means basically between the models,” said science panel member Charles “Si” Simonstad. “If you look between models, I believe the adult returns are off by a factor of 5, which is a big difference in models, but then you mentioned, don’t take it in the absolute sense, take it in the relative sense, and I think that’s important, but I think the one thing you could do would be to rank your fourteen scenarios from best to worst with the different models to see how consistent they are.”
“I have a question on our treatment or lack of treatment in Chapter 5 of adaptive management since that permeates so many of the conclusions, especially when there’s a high degree of uncertainty,” said David Zippin with ICF International. “And I would encourage the panel to provide specific suggestions on how we might do that. In previous drafts of the document, I think we probably overdid it, and now we’ve sort of swung the other way, omitted it completely, so there has to be a happy medium somewhere in the middle. … We opted to be conservative and rely on adaptive management very little in chapter 5, but perhaps there’s a way we could acknowledge it that’s defensible.”
“My quick thought is that maybe just a section within chapter 5 towards the end recognizing the uncertainty in the effects of these attributes and then their net overall effect,” offered science panel member Dr. Greg Ruggerone. “And then briefly describing and referencing the other sections of the BDCP on the monitoring would be going on to address those uncertainties, and then briefly going into the adaptive management framework, how it’s going to respond unanticipated findings.”
“I’m sitting here, contemplating the difficulty with which the services and fish and wildlife together are going to have adopting such a program without specifics,” said Steve Culberson with the U.S. Fish and Wildlife Service. “I know that there are other people in other rooms discussing how that might go, but what I think might be helpful would be an example exercise … if you began a program of tidal marsh restoration and you pick a candidate suite of locations and connectivities that are going to be hydrodynamically important, and you discover a third of the way through your first phase or restoration that you’ve expended 90% of tidal energy that you expected to be available throughout the course of the 65,000 or 100,000 acre restoration program, what course is now open for the adaptive management program? … If you discover at a certain point, that you are out of tidal energy, you’d better have another strategy for getting at what you think the fish are going to need, besides flow. And I’m not sure that I’m real satisfied yet at the level of discourse that we’ve had in the scientific community, let alone in the project proponent and regulatory world about how that should go.”
Nancy Monsen said tidal energy can be modeled. “You’re going to have 2D and 3D models, so I don’t think you are going to get to a case where you are actually going to build something where you’ll be surprised on tidal energy. I think you can have the confidence that the multi-D models will be able to show you a lot of what that connectivity is going to look like. I don’t know if you’re going to get into that scenario with tidal energy.”
“I think this is the candidate that you could work through an example because I think you could come up with a list that would get you 90% expenditure of your tidal energy in the first 10,000 acres, and then the tough part of the adaptive management is ‘and then what,’” said Mr. Culberson. “The adaptive management should have decision points, alternatives if we expend 90% of our tidal energy before we’re 90% of our restoration, then what does the program look like. That’s what I’m getting at. So my concern is maybe an emphasis on more detail in the adaptive management part, because all of the things we said were deficient, I suspect we’re not going to get answers to, so the real nub here for me is the adaptive management plan. … It isn’t necessarily going to be in the effects analysis, I get that, it might be outside of our charge or scope here, but I think that’s the real key to the success of this program. And the one I choose is tidal energy because I think that’s tractable, and then what.”
“That’s where you really have to think about the order in which you are restoring so that if tidal energy is something that you think is important that you need to allocate or budget, then you have to think about that when you’re getting into the nitty gritty because it comes down to you’re going to get your hands dirty and you’re going to have to be pretty open about where specifically am I looking at and adapting it in your modeling,” said Ms. Monsen. “Luckily, tidal energy – it’s physics. Physics is a lot more tractable than biology.”
“A rigorous adaptive management plan would have a conceptual model that poses those decision points that are linked to triggers and thresholds,” said science panel member Charles “Si” Simonstad. “One thing that was a little confusing to me was in the monitoring and research component, the tables had compliance monitoring, effectiveness monitoring and research, and most of the uncertainties were allocated to research. In fact, it should be sort of explicitly incorporated as a major element of adaptive management to try to resolve those with monitoring and incorporate those into alternative approaches if they pass those trigger points. So in some respects, there’s a potential need for a table, a conceptual model or some other diagram that suggests for the major uncertainties, how would you approach that with a rigorous adaptive management program. What would be the candidate triggers, what would be the candidate thresholds, what would be the candidate alternatives that you would have to move to under that condition?”
Next Step: The science panel will produce a final report which is due to the Delta Science Program by March 10.
For more information …
- Click here for the webcast: BDCP independent science panel review of the Effects Analysis, phase 3
- Click here for ICF’s power point presentation.
- Click here for the independent science panel’s initial findings.
