This is part two of three-part coverage of the May 29th meeting of the Delta Stewardship Council. In part one, consultant Larry Roth from ARCADIS reviewed how the Bay Delta Conservation Plan addresses the goals and objectives of the Delta Reform Act and the Delta Plan. In part three posting tomorrow, the Council reviews and finalizes their comments.
Report of the independent panel’s review of the BDCP’s Effects Analysis
Sam Harader from the Delta Science Program began with some background information on the independent science review of the Effects Analysis. The BDCP’s Effects Analysis is a critical component of the plan as it is the scientific assessment of what the effects of implementing the BDCP’s actions would have on covered species and the Delta’s ecosystem. It also provides the fish and wildlife agencies the information they will need to make the necessary findings to issue incidental take permits and authorizations as well as comply with other regulatory requirements. The review was done at the request of the Department of Water Resources and the Bureau of Reclamation, and it was intended to provide a scientific review to help to strengthen the scientific quality of the effects analysis, he said.
The review was done in three phases, he said. The first phase of the review, conducted in the fall of 2011, looked primarily at the conceptual foundation and framework to give some early feedback. The second phase, occurring over the summer of 2012, was an interim review to provide feedback before the final effects analysis was prepared. Earlier this year, the panel completed the review of the completed draft Effects Analysis, the one that is currently available for public review.
The three phases of the review were completed over the course of three years, with the same seven panel members who represented a broad range of scientific expertise including hydrodynamics, fish biology, statistics modeling, aquatic ecology, food webs, plant communities, and terrestrial ecology. The panel members were Dr. Charles Simenstad, a research professor at the University of Washington; Luke George from Colorado State University; Dr. Nancy Monsen from Stanford University; Dr. Tom Parker from California State University, San Francisco; Dr. Greg Ruggerone Natural Resources Consulting; Dr. John Skalski with the University of Washington; and last but not least, Dr. Alex Parker with the California Maritime Academy.
Dr. Alex Parker then presented the findings of the independent science panel’s review.
“By way of background, I’m a microbial biogeochemist, which means I am interested in looking at how phytoplankton and natural populations of bacteria respond to the environment, and my particular interest is in these highly disturbed systems, urbanized estuaries, systems like San Francisco Bay,” he said.
Dr. Parker said that the panel started looking at the document in November of 2011 and finished up in March of this year, so they were able to see the process as it unfolded, which was really helpful in evaluating the work along the way. There were also opportunities to interact with the consultants ICF along the way, which was also very helpful, he added.
“We were tasked with really drilling down into the details of this effects analysis and to try to understand, using the best available science, what the likely net effects to covered species would be of implementing the plan,” he said. “The effects analysis is vast in its scope. The actual chapter that deals with the effects analysis is 745 pages, but you really need eight associated technical appendices which total another 4500 pages or so in order to really begin to understand the potential effects of implementation for covered species.”
Dr. Parker said he would just provide the highlights of their analysis and the major themes that emerged as a result of their review. “We are heartened to see that the Delta Independent Science Board review of the draft BDCP and the EIR/EIS echoed a lot of our concerns, and I think that probably highlights for folks the areas where attention needs to be paid.”
He said there were four themes that emerged for the panel:
- The first is a real disconnect between the assessments of scientific certainty or uncertainty that is reflected in the Effects Analysis chapter versus what is in technical appendices, he said. “This was a concern to us because we know that with a set of documents this vast, most people are going to read the Effects Analysis and not the technical appendices. There’s a real concern that the effects analysis doesn’t adequately address that level of uncertainty around virtually all of the conclusions that are made.”
- The implementation of the BDCP and its effects are highly uncertain, so the way to address this is through adaptive management, he said. “It is part of the plan; however the Effects Analysis needs to really clearly articulate the uncertainty in order to have an effective adaptive management process and at present, that simply doesn’t exist within the main document.”
- The third theme is a lack of systems approach in terms of looking at the Bay-Delta system and the Plan, and how conservation measures will interact with each other, and along with this is an integration of quantitative assessment of the net effects for the covered species, he said.
- Finally, overall there was a concern of just how this very detailed document lacks a lot of structure, making its interpretation very difficult, he said.
Dr. Parker then gave some further discussion. “The BDCP is vast; it’s of huge magnitude, and there is a lot of natural and anthropogenic (or human-caused) complexity to the Bay Delta ecosystem,” he said. “We have its long time horizon and climate change, and as a result, any of the effects or presumed effects of implementation carry with them a high degree of uncertainty. There’s just no way around that. And so the Effects Analysis needs to provide clear guidance about what those uncertainties are, where our information gaps lay, and then also inform adaptive management in terms of what information should we be monitoring along the way, what are the appropriate triggers so that we know when we are on the wrong track and then course correct – move in a different direction. That was largely lacking.”
“The challenge is in the technical appendices,” he said. “The details seem to be there, and the uncertainty seems to be there, but it just didn’t really translate to what we presume most people will be reading and that’s really problematic because from the outset, for adaptive management to be effective, it needs to identify and really truly embrace the uncertainty. That’s how adaptive management and iterative learning works.”
Dr. Parker gave an example, drawing from the section that he took the lead on reviewing, Technical Appendix 5-D. “It provided a great deal of information,” he said. “The scientific literature was there; the information was there although sometimes not well organized, but in the actual effects analysis, there was only a page and a half devoted to contaminants in the 745-page document, giving the impression that the best case scenario probably is the likely outcome, and that was difficult for the panel.”
“This question of uncertainty really comes down to how to move forward with the implementation of the plan – if it is to move forward, and you need to really understand the uncertainty in order to implement adaptive management in a meaningful, rigorous way,” he said. “Instead of having that process in place, you’re left with the sense that really passive learning is how this is going to unfold. In other words, you don’t reach an outcome then decide what to do at that time … instead, the approach from an active adaptive management perspective should present the information gaps and the unknowns, provide workable hypotheses, but then also the metrics to test those hypotheses and course-correct early with alternative hypotheses ready to go.”
Dr. Parker noted that chapter 5 does have a statement that says that these expectations represent a working hypothesis of the relationships between actions, stressors, and biological performance, and that really is at the heart of adaptive management. “These are hypotheses that carry a high degree of uncertainty, so all these uncertainties need to be explicitly stated for adaptive management to be effective in moving forward. They did mention adaptive management; I don’t want to suggest that they didn’t, but it wasn’t adequately integrated, and important information that can be gleaned from the effects analysis wasn’t clearly enough stated, we believe, to inform the other chapters, including adaptive management.”
The lack of a whole ecosystem approach in the analysis concerned all of us, he said. “The effects analysis ended basically in Suisun Bay, and outside of the plan area, the assumption that you’re not going to have downstream effects on the whole ecosystem – the Delta, the Bay, and even the coastal ocean are not connected systems seemed awfully presumptuous to us. I think it reflects a general concern that perhaps better integration of how all of these different conservation measures work together to influence a net effect for covered species needs to be more deeply considered.”
Another place where this is clear to us is with respect to hydrodynamics modeling, Dr. Parker said. “Hydrodynamics is basically the movement of water, and this is a master variable in the system,” he said. “If we want to have any conversation about circulation patterns, temperatures, submerged aquatic vegetation, contaminants, nutrients – we need to have reasonable modeling of the hydrodynamic system, and because we don’t know where the restoration opportunity areas are necessarily defined in all cases – these are places where major conservation and restoration activities will take place – they were limited in what they could model in terms of hydrodynamics. That wasn’t adequately acknowledged throughout, and again, raises high level of uncertainty in the ultimate analysis.” He also noted there were some counterintuitive results from some of the hydrodynamic modeling that was done there, but there wasn’t sufficient information to really understand where those results came from.
With respect to the net effects, there was an evolution of development of trying to rank net effects and balance lots of different interacting processes. and to some extent, the scientific information wasn’t there to do that in a robust way, Dr. Parker said. “The effects analysis does identify a quantitative or semi-quantitative approach for looking at the net effects, balancing all of these different potential processes, plusses or minuses for species health, but then they don’t implement it, they don’t apply it, and instead default to basic professional judgment, which is essentially a hypothesis about how the system would work.”
The scope of the Effects Analysis is impressive and they have done a fair job of assembling all or many of the necessary pieces of information, he said. “However, there isn’t sufficient cross referencing between the technical appendices and the actual document itself, and it leaves the impression of sort of a trust us message because you can’t find the information. You’re convinced it’s there, but it sometimes very difficult to find the relevant information necessary to make a determination about likely effects.”
“We do believe that the Effects Analysis needs to be a standalone document, and in its present shape, we don’t think it reflects that,” he said. “Some, but not all, of the content problems could be diminished with simple better reporting.”
“The Effects Analysis is a real opportunity to take all of these information gaps that are identified and the scientific priorities and better integrate that with Delta science overall,” he said. “Clearly as we move forward with potential implementation of the plan, we are going to be doing a lot of learning along the way, and the Effects Analysis, if synthesized correctly, could really help guide that process along in a more meaningful way, and should integrate better with the Delta Science Plan.”
“And with that … “
For more information …
- Click here for the report by the Independent Review Panel for the BDCP’s Effects Analysis
- Click here for the agenda for the May 29th meeting of the Delta Stewardship Council.
Report of the Independent Science Board on their review of the BDCP’s EIR/EIS
Marina Brand from the Delta Science Program and Dr. Tracy Collier and Dr. Jay Lund from the Delta Independent Science Board then briefed the Council on the results of their review of the BDCP’s EIR/EIS.
Marina Brand, Program Manager for the Delta Science Program, began by giving some background information for the review. “The 2009 Delta Reform Act created the Independent Science Board (ISB) to provide oversight of the scientific research, monitoring, and assessment of programs that support adaptive management of the Delta,” she said. “In addition, the Delta Reform Act required that the Independent Science Board review the BDCP draft EIR/EIS. The Reform Act also required that they submit this review to the Council and to also to the Department of Fish and Wildlife.”
“In addition, on November 15, 2012, this Council approved a charge to the ISB to guide its review of the DCP EIR/EIS, and the charge asked the ISB to comment of four primary aspects,” she said. “The first was the completeness, structure, and effectiveness of the description of the alternatives. The second one was the appropriateness of the approach, analysis, tools, and modeling used to conduct the impact analysis. The third aspect was the adequacy of the proposed monitoring and description of the adaptive management plan, including whether or not the adaptive management goals are achievable, and then the last aspect was to evaluate the scientific basis and clarity of some very specific regulatory questions.”
The Independent Science Board imitated its review in 2012 with the administrative draft documents of the BDCP and the draft EIR/EIS and produced four memos between June of 2012 and June of 2013 that were submitted to the Natural Resources Agency, the Department of Water Resources, and the Department of Fish and Wildlife as well as the Council, she said.
The ISB reviewed not only the draft EIR/EIS, but also the draft BDCP and the phase 1, 2, and 3 reports prepared by the independent review panel on the effects analysis, she said, noting that the ISB members assigned themselves specific chapters based on each member’s area of expertise. And then that individual was considered the lead reviewer and they were supported by a subset of ISB members, with additional support provided by Dr. Michael Healey who led the review of the adaptive management component..
“Although a few of the chapters of the draft EIR/EIS were not reviewed by the ISB, the Board firmly believes that review of the additional chapters would not change its overall conclusions,” she said.
The review included lengthy discussion at several public meetings held since the documents were released and two meetings with Laura King Moon and members of the BDCP consultant team, as well as presentations from others. In addition to that, the ISB received numerous public comments at the meeting, she said. The ISB’s report is comprised of an overall summary of its primary findings and two appendices. Appendix A is basically the ISBs responses to the Council’s charge to them to conduct the review, and Appendix B consists of comments on individual chapters of the draft EIR.
She then turned it over to Dr. Tracy Collier to present the Independent Science Board’s findings.
“When we looked at this, we really viewed the BDCP as a rare opportunity to really try to merge science with management in a very complex system in a way that is going to look at infrastructure and human resources – environmental, regulatory, institutional, and financial aspects of Delta management,” began Dr. Collier, Chair of the Delta Independent Science Board. “We undertook the review with being mindful that this is a not-common occurrence and that we should really put our all into it. We really are mindful of the need to be constructive, so when we identify problems, we’re trying to identify issues with the documents in way so that they can be addressed so that it can be improved, so that’s the basis, the mindset which we brought to this review.”
“In the documents that we reviewed, we found a lot of things to admire,” Dr. Collier said. “There were a lot of really positive things. We have a list of it in our overview. I won’t go through all of them, but things like the background descriptions of the Delta environment, the context for CEQA and NEPA, are really impressive. They are detailed and yet they are clear. There’s a lot of information that has been pulled together that needs to be valued.”
Dr. Collier said they thought the presentation of the alternative water conveyance designs for CM1 was comprehensive and well balanced, and many of the resource chapters are extensive, containing tremendous, often overwhelming amounts of information. In the analysis of mitigation or avoidance minimization measures, while the ISB thought the set of models was limited, the ones that were used were employed effectively, he said.
“There’s frequent reference to the importance of adaptive management, but there were some issues of whether we think it can be effectively implemented with the current mindset, but there is the clear intention to make adaptive management the centerpiece of the project,” he said. “There are in-depth evaluations of individual species, most notably two of the salmonid species, but again, that leads into one of our concerns that there’s not this whole systems effect of looking at the interactions among species as much as it should be. Climate and sea level rise are included in some of the analyses and that was good to see that, but at the same time, there are issues with how climate change is viewed in the overall project.”
“I have to admit, as someone who has spent probably too much time researching the Delta, that I don’t think there’s anybody on earth who couldn’t learn something from that massive work,” added Dr. Jay Lund. “I found a lot of things that were new to me and I’m sure anybody, no matter what angle they are coming at or how many years they’ve spent in the Delta, will find something they can learn that they didn’t know.”
Dr. Collier said that the ISB has eight major concerns:
1. Many of the impact assessments hinge on overly optimistic expectations about the feasibility, effectiveness, or timing of the proposed conservation actions, especially habitat restoration: “Is the scientific basis for the analyses and for the draft EIR/EIS, is it sufficient, is it good enough to support the decisions that are going to have to be made? There are overly optimistic expectations about the feasibility, effectiveness, and timing of the mitigation measures of the conservation actions, especially habitat restoration.”
2. The project is encumbered by uncertainties that are considered inconsistently and incompletely; modeling has not been used effectively to bracket a range of uncertainties or to explore how uncertainties may propagate: “The project has uncertainties encumbering it and one of the problems is that the level of uncertainty that is there is inconsistently applied, so some have more uncertainty, some have less, but it’s not been used,” he said. “They haven’t used modeling to effectively bracket a range of uncertainties, or how to explore how uncertainties may propagate through the system as they compound and cascade.”
3. The potential effects of climate change and sea-level rise on the implementation and outcomes of BDCP actions are not adequately evaluated: “When we asked DWR about that, their response was that the EIR/EIS process is to look at the effect of the project on the environment, but not the effect of the environment on the project. And while that may be the legal context, that’s just a big science issue that applies to levee failures, floods, and invasive species – the effects of those actions on the project itself, we think needs to be considered.”
4. Insufficient attention is given to linkages and interactions among species, landscapes, and the proposed actions themselves: “We don’t think there’s enough attention given to that, he said.
5. The analyses largely neglect the influences of downstream effects on San Francisco Bay, levee failures, and environmental effects of increased water availability for agriculture and its environmental impacts in the San Joaquin Valley and downstream. “In the analyses, they largely neglect the influences of downstream effects on San Francisco Bay. They don’t consider it because its outside the project area and the defined scope.
They don’t consider the effects of increased or altered water reliability on agriculture outside the region and what that might do to water quality issues, etc.”
6. Details of how adaptive management will be implemented are left to a future management team without explicit prior consideration of (a) situations where adaptive management may be inappropriate or impossible to use, (b) contingency plans in case things do not work as planned, or (c) specific thresholds for action: “The details of adaptive management and how it’s going to be implemented are just not there,” said Dr. Collier. “There needs to be more specific discussion about adaptive management. There will be situations where adaptive management is not appropriate or not even achievable, and that needs to be explicitly recognized. But we can’t leave everything to an adaptive management team at some point in the future to determine how we’re going to do this; we need to make more specific recommendations now or as soon as possible.”
7. Available tools of risk assessment and decision support have not been used to assess the individual and combined risks associated with BDCP actions: “We don’t see evidence that a comprehensive risk assessment has been really applied in the system, and there’s not been the use of currently available scientific tools for decision support or how to support decision making based on technical information. There are methods for doing that and we think those could be beneficially applied to this process.”
8. The presentation, despite clear writing and an abundance of information and analyses, makes it difficult to compare alternatives and evaluate the critical underlying assumptions. “There’s a lot of good writing and a lot of good content, but we suffered equally as well as the effects analysis panel with the inability to get to the information we needed,” he said. “This is one where we tried really hard not to be really cranky in our review comments because we have said this many times over the last couple of years. It would benefit so much the ability of people who want to review it, which was our main concern, but also to actually use the information in a constructive way. We are bothered that has not been done more … We haven’t seen any attempts to undertake some of the suggestions that we made early on in the process.”
Dr. Collier said they build on the concerns in the overview of the report, as well as give some recommendations. “Some of our recommendations for going forward and improving the scientific framework of the overall BDCP is to really work on developing these adaptive institutional, regulatory, scientific, human and financial capacities. The concept is there, the concept is supported, but the willfulness or the ability to actually implement adaptive management in this wide range of scenarios that you have … it’s to be determined later, it seems to us.”
“We think BDCP science needs to be integrated with the Delta Science Plan,” he said. “It really provides a framework for trying to effectively use adaptive management in supporting decision making and management actions, so we think that it needs to be a concerted effort, and much like in the state Water Action Plan they now say, the Delta Science Plan, we will do our best to adhere to that, we need to see that same level of commitment for BDCP.”
Another one of the ISB’s recommendations is to get some pilot projects with appropriate monitoring and evaluation underway as soon as possible, he said. “This issue of the habitat and the huge uncertainties around the 140,000 acres of protected or restored habitat and whether that can mitigate some of the impacts. Everybody realizes how difficult that is and how hard that’s going to be make a clear call as to how effective that’s going to be, but we need to get pilot projects going as soon as possible.”
With respect to using risk based decision analyses, the ISB recommends using some of the technical tools that are out there that allow one to grapple with some of these really horribly complex issues in making decisions and try to apply them and see if that helps to inform the way the decisions come out, he said.
“Finally, learn from the current drought. If there’s ever a time when we need adaptive management and nimbleness and the ability to think and respond quickly, this is it,” he said. “How are we doing, what would work better for dealing with this situation, and then figure out if we can take some of those lessons and apply them to BDCP.”
Discussion highlights
Dr. Jay Lund: “In one of our conversations yesterday, we were talking about adaptive management and how everyone seems to be using the term differently, and the most cynical interpretation of a lot of the use of adaptive management is promising to fix it later. There seems to be a fair number of occasions when adaptive management is used in that sense. This problem is too important for the state and the ecosystem and the locals to have that kind of an attitude. My own disappointment with the document is that they pay lip service to adaptive management and they have a diagram that goes in circles like everybody else does, but this is a real hard thing to do. It’s going to require serious money. I joked yesterday that if everyone was taxed a dime every time they said adaptive management, we might have enough funding for this. It illustrates to me the difficulty of getting it done, that that hasn’t been specified, and to me it raises a real threat on the likely effectiveness of this project. If you cannot muster a functional adaptive management program, so it’s a real challenge to the BDCP folks, to muster an effective adaptive management program and to define what that is. Come up with the financial resources, the expertise, and the administrative authorities that would allow you to have the hope of it being effective. Otherwise it really amounts to promising to fix things later.”
Dr. Tracy Collier: “My personal view on this is that we’re not going to come up with an adaptive management plan that’s going to be ready to go when we start turning dirt. But we do need to put in the time to figure out where we’re going to get the resources for the monitoring, where are we going to get the resources for the staff time to do the evaluations, how do we determine contingency plans or do we have alternatives that we’re going to implement if things go this direction versus this direction, and I just don’t see that in there. I don’t see that it’s there … maybe it’s not part of the EIR/EIS, but it still needs to be part of the thinking about the project is what are we going to do, where are we going to build the resources so that we can actually gather information, analyze it and assess it, and act on it, both management wise and policy wide, as well as a science-basis wise. … There’s not enough thought in what we saw in the documents to say we are planning for how to really support adaptive management, even though we don’t’ know exactly what it’s going to look like, we don’t see the preparation taking place to try to resource it and put that into place. But there are people talking about it, that’s good, but we need to do more than be talking as we go down the road to project implementation.”
For more information …
- Click here for the report from the Delta Independent Science Board on the BDCP and its Draft EIR/EIS
- Click here for DWR’s Laura King Moon’s response to the Independent Science Board.
- Click here for the agenda for the May 29th meeting of the Delta Stewardship Council.
- Click here for part one, How the Bay Delta Conservation Plan addresses the goals and objectives of the Delta Reform Act and the Delta Plan
