An expert review panel evaluates the methodology behind the development of the Delta Levee Investment Strategy
Note: This post is quite long; it is available for downloading as a pdf (with essential graphics only) here: DLIS Workshop PDF
The 2009 Delta Reform Act created the Delta Stewardship Council and directed it to do a number of things, among them to develop a way to prioritize investments in the Delta levee system.
In order to accomplish this task, the Delta Stewardship Council has been working with a team of consultants at ARCADIS, the Rand Corporation, and ESA on a project to develop a methodology and a decision making tool that will consider the assets protected by the Delta’s levees, the threats to the levees, and the multiple beneficiaries of levee investments.
The outcomes of the project include a computer-based decision making tool and a final report proposing a Delta levee investment and risk reduction strategy and outlining a suite of investments that best address state goals and priorities. Ultimately, the strategy is expected to be used to update the Delta Plan’s regulatory policies and recommendations, and may also be submitted to the legislature to help guide its decisions regarding funding for Delta levees.
As part of the progress on the project, the Delta Science Program convened a scientific review panel on May 19 and 20th to review the methodology being developed and its supporting scientific basis. The seven panel members hail from all over the country, and even one from the Netherlands; some panel members are familiar with the Delta and some not. The panel members are (click here for panel member bios):
- James K. Mitchell, Sc.D., Virginia Polytechnic Institute and State University (Emeritus) (Panel Chair)
- Kenneth A. Rose, Ph.D., Louisiana State University (Lead Author)
- Nathalie E.M. Asselman, Ph.D., Deltares
- John P. Bolte, Ph.D., Oregon State University
- Susan L. Cutter, Ph.D., University of South Carolina
- Martin W. McCann, Jr., Ph.D., Jack R. Benjamin & Associates, Inc.
- Ari M. Michelsen, Ph.D., Texas A&M University
In the first day of the workshop, the ARCADIS team presented the components of the methodology; afterwards, the panel members and the ARCADIS team answered questions and discussed issues. On the second day, panel members provided some preliminary feedback. A formal finalized report with more detailed recommendations from the panel is due sometime in mid to late July.
The workshop began with Cindy Messer, Deputy Executive Officer of Planning for the Delta Stewardship Council, who provided some background for the panel members. “The Delta is a very large area, and it’s a very complex area, both physically and politically,” she said. “It houses many valuable assets; it provides many critical services related to things like water supply and ecosystem functions; it has infrastructure related to energy, transportation, and fuel lines; it has a very strong agricultural base, and economy, and it also houses many cultural assets, values, and legacy communities, to which all of these are protected to some degree by the levees that are in this region. And it is home to half a million people, most of those living in cities along the outer boundaries of the Delta, but there are a smaller percentage living well within the heart of the Delta, in rural areas, farms and small communities. All of this complexity, the assets and the values are something that the Council has been taking into consideration as we develop this levee investment strategy.”
The Delta Stewardship Council was established in 2009 as part of a suite of bills that covered issues and needs pertaining to water conservation, ecosystem restoration and protection, and better groundwater management; the legislation also addressed risk reduction as well as emergency preparedness and response measures, Ms. Messer explained. Specifically the Delta Reform Act focused on restructuring governance for the Delta region which is the bill that established the Stewardship Council and also made some other changes related to state agencies in this region, she said.
“The Council was tasked through the Delta Reform Act to develop an investment strategy,” she said. “We did that in this initial version of the Delta Plan, and essentially the effort we’re going through now is to update that strategy and to bring a bit more focus to it and to turn it into a long term investment plan and strategy.”
Ms. Messer explained that the vision for the Delta Levee Investment Strategy is that it’s a means to identify what is at risk in the Delta, and where improvements need to be made to reduce that risk; it’s a means to address the ongoing need for levee improvements in the face of not having enough funding to raise every portion of the Delta’s levees to a desired level of protection; and it’s a means to prioritize investments of these limited funds now, and given the uncertainty of funding in the future, to have a strategy that will allow us on an ongoing basis to prioritize investments as new funding sources become available. The vision is that the framework is flexible to adapt to changing environmental conditions, political conditions, or financial conditions; it can incorporate new information and new data as it becomes available, and that it works with other existing state agency programs for investing in the Delta’s levees, she said.
One of the outcomes of this process is to produce a report that will go to the legislature that will outline the strategy and include recommendations that would allow them to fully implement the strategy, identify the gaps in information, and determine where resources are needed for both structural and non-structural improvements that will lead to a feasible and cost-effective strategy for reducing risk overall in the Delta, Ms. Messer said.
Their approach has been very open and transparent, Ms. Messer said, and has included public meetings, posting technical memos and appendices, and vetting information with stakeholders and technical experts. At each key step they’ve engaged both with the public and with the technical experts to get information out and to receive comments back. “We are using the best available existing information for this effort, and thus our desire to vet this through some of the technical experts and stakeholders that are interested in this particular effort,” she said.
“Our approach is based on the statutory guidelines of protecting people, property, and state interests, and state interests at this point include the coequal goals, as well as Delta as a place,” she said. “This is a collaborative process, so we’re working with key partner agencies such as the Department of Water Resources, the Central Valley Flood Protection Board whose jurisdiction lies with project levees, the Delta Protection Commission who has land use jurisdiction in the primary zone in the Delta and who are also working with us a bit on the cost allocation side of this effort, and of course we’re working with local flood management agencies, the reclamation districts, local governments, and stakeholders.”
Ms. Messer said they have scheduled the review at this time as it was early in the process and they wanted to make sure they were headed in the right direction. 
“We wanted to make sure that the approach makes sense, is sound, robust, and that it can indeed by flexible and adaptable as we look towards the future, given the boundaries and some of the constraints around the timeline, around the budget, around the information that we’re using.”
Dave Mraz next gave some brief comments on behalf of the Department of Water Resources, thanking the panel for providing guidance in the effort that will hopefully bring transparency, logic, and predictability to prioritizing the levees within the Delta. “It’s a huge task and we’ve been doing it by various means over the last 30 years, trying to keep the system up, and it’s probably time we have a more logical and visible way of making decisions that protect 25 million people, their water supply, and about 4 million acres of agricultural land,” he said.
Dr. Rainer Hoenicke then briefly addressed the panel, noting that science was recognized not just by the governor but also by President Obama as one of the guiding principles behind the work being done at the Delta Stewardship Council and the Delta Science Program. 
“The Delta Science Program’s vision is that all Bay Delta water and environmental policy is founded on the highest caliber science, and we want to make sure that we provide the best possible unbiased scientific information for water and environmental decision making in the Bay Delta system,” he said. “We don’t do the science ourselves – what we do is essentially support research; we facilitate special studies, we also keep track of monitoring results and synthesize all of that into something that is actually usable by policymakers, and we do facilitate independent peer review. One of our key mandates is to communicate and coordinate science, so we have a reputation as an honest broker of science – the parties, no matter what interest group they are a part of, they tend to look to the Delta Science Program as an honest broker; therefore we try to be as transparent and as accountable as we can possibly be.”
“Today we’re here because we want to make sure we have a mechanism to check whether we are using our team that is developing strategy is using the best available science, whether they missed anything, whether we should adjust things as we move forward,” he said.
DELTA RISKS: Dave Mraz
Department of Water Resources
Dave Mraz then began the presentations with some background on Delta levees and the risks posed to them. He began by noting that he’s been working in the Delta for 15 years trying to understand the complexities of the system. “I think I have an idea and I’m not sure that I grasp it all, but it’s really hard to distill into a short presentation,” he said.
“The Delta is about 1% of the state of California, but it has a huge impact on people, economy, habitat, and flooding,” he said. “Probably one of the most important facts to the state is that it really depends on who you are as to what’s important in the Delta. If you’re a person that has a house in Stockton, there’s one Delta that most important to you. If you’re a farmer on one of the islands, there’s probably a few different things that you consider. If you’re delivering water from Northern California to Southern California where it’s scarce, there are other things, so what are the most important things in the Delta? I’m looking from the perspective of the state of California, trying to capture those things that are of interest to the state as a whole, and the water supply looms very large in that.”
“Water that comes through the Delta provides drinking water, agricultural water, municipal and industrial water for a large portion of the state of California, and that water is responsible for about 25% of the gross national product of California so it’s really important,” he said, presenting a chart showing where Delta water is delivered around the state. “There are about 4 million acres that are irrigated and about 25 million people that receive some portion of their municipal and industrial water, so those are by anyone’s guess, truly state and national concerns.”
A second important fact about the Delta is that it is the hub of California’s water system – not just exports, but the natural water system too, he said. “We have the Sacramento River with all of its tributaries coming from the north into the Delta, and we have the San Joaquin River with additional tributaries coming from the south and flowing north into the Delta, so about 47% of all the water that falls on the state of California passes through the Delta,” he said.
Mr. Mraz pointed out that the system has undergone massive change. “When people came into it in the 1840s and earlier, they saw a river system and a Delta that was lined with tules that was so dense, you had a hard time getting through it. You had a hard time on horseback even seeing over the top of the vegetation. There are stories about taking several days to a week to get from San Francisco into Sacramento so that they could run off to the goldfields. It was a very complex place.”
At the time, it was viewed as useless swampland, and so the federal and state governments passed legislation to give or sell the land for people to use, he said. “Under manifest destiny, we had to tame the land, take it over, and make it productive, so we started doing that, some might say with a vengeance,” he said. “We have taken that system and developed a marvel of engineering where we take the water that’s supplied in the north and move it 1000 miles to where it’s needed in the south.”
“However, that system has very significantly changed the hydraulics, hydrodrynamics, and the habitat of California, but specifically of the Delta,” he said. “We now have a system that has more than 400 species of exotics that are wreaking havoc on native species and we’ve lost quite a few varieties of native fish. I think the most endangered right now are winter-run salmon and Delta smelt. We’re counting those by the individual fish, not so much by large units.”
Mr. Mraz said that one of the ways the Delta has been changed is that the islands are subsided. “They were peat soils and many of them remain peat, though some are not,” he said. “The act of taking over the land, cultivating it and making it productive introduced oxygen into that peat soil. When it’s oxygenized, the microbes can take it over and it goes off as a gas. There’s a mechanical disruption; it breaks it up into small pieces and there are particles that fly off. If you were out in the Delta yesterday, you undoubtedly saw someone who was farming, and there’s a big cloud of smoke after each one of the tractors. That smoke is Delta soil. It has gone off at a rate that is pretty surprising when you get down and you look at it. Right now, some of those islands are as deep as 20 feet below sea level.”
“This levee system that we have that protects the Delta is rather unique,” Mr. Mraz said. “If they weren’t where they are, they’d be dams in the state of California, but there is an exemption in the law that says these levees are not dams. If you look at the picture on the slide, the real thin ribbon of soil is the only thing that protects that island from flooding and becoming a lake, and that’s the case for about 70 islands in the Delta. We call them islands; I think in the Netherlands, they call them polders, they are really not islands the way that you think of.”
The Delta soils are unique, he said. “If you don’t know how to manage them, they can get the better of you. Another thing we have are natural hazards. This has been the cause of more than one levee failure in the Delta. You get beavers on one side and squirrels on the other, and when they come together, they reduce the amount of seepage path and can cause a leak. A leak in this particular case would erode out very quickly and become a levee failure.”
“This is a tidal system so there is water around it 100% of the time, so if it goes fast enough, you lose an island,” he said. “It’s not like the upland system where you lose an island or you lose an area, the water comes in floods some land, and then the water recedes and flows back out the breach. In the Delta, the water flows in and it has to be pumped back out.”
Mr. Mraz said that hydrologic flooding is probably the largest hazard in the Delta. “Even though we’re in the middle of a drought right now, we still think about the atmospheric river,” he said. “When we see that atmospheric river developing, we know that the Delta is going to get hit; we know there are things that we have to do to protect the islands from flooding.”
“The other thing that’s a little bit unique about the Delta is that we are in a highly seismic area,” he said. “We’ll get some argument from people about how the Delta soils behave with respect to seismicity, but suffice to say that we’ve got some major faults that are near-field to Delta levees, and if we get a major rupture on some of these faults, we can expect that there’s going to be multiple failures at one time.”
“Why do I say that? This is an embankment that was shaken during the Loma Prieta earthquake, a moderate 6.9 event,” he continued. “This is probably 60 miles away, and the damage is significant in and of itself, but then you look at the areas here that are just disrupted. That would be a through-seepage path in the Delta, and if you think about these circular levees and the fact that they are wet 100% of the time – if you don’t have somebody there right away to treat it, this transverse cracking is going to generate high-seepage, high volume erosion and failure, so in addition to the failures that are caused directly by the earthquake by differential settlement such as you see here, you’re going to have other failures that occur a little bit later, so we need to be able to respond to these. We need to recognize that the potential is there, then we need to develop means to account for them, and means to respond to them quickly.”
It would be clarified later in the presentation by Larry Roth, speaking on Dave Mraz’s behalf, that the levee in the picture is not in the Delta, but is located in Moss Landing, California, which is about equidistant in the other direction than the earthquake as the Delta. “But the important point is that we haven’t had any known failures in the Delta of Delta levees from seismic activity in the past,” said Mr. Roth.
He presented a chart showing sea level rise, and noted that he couldn’t be convinced of sea level rise until he saw this graph. “This is a graph of sea level at the Golden Gate, and I can’t argue with it. It’s coming up, so the question is, how much and how fast.”
He then played an animation of a fly-in to the Delta for the panel. He noted that about 300,000 second-feet of salt water comes in on the tides every day, and while the Sacramento and the San Joaquin rivers have a lot of capacity, by comparison to the tides and the large volume that comes in from the ocean, they’re really not that big.
Mr. Mraz said that it is the public-private partnerships between the state and the reclamation districts that maintain the levees, especially during the winter stormy season. “We have reclamation districts on each one of the islands, and those people are out there during conditions like this, making sure that the levee is protected. If there is an area that’s being overtopped and eroding, they are actively putting sandbags in place and putting floodwalls in place. If they see seepage on the backside, they’re putting sandbag rings and working to make sure that the system does not fail. The public-private partnership in this case is absolutely the best bang for the buck for the state to go ahead and maintain these levees. We pay the bills but the reclamation districts are the ones that actually do the work. We get an excellent deal.”
INTRODUCTION/TOLERABLE RISK
Larry Roth
Principal Engineer and Project Manager for ARCADIS
Larry Roth, the lead for the project team, then began the presentation of the methodology behind the decision tool being created for the Delta Levee Investment Strategy.
“This peer review presentation was essentially established at about the midpoint of our project progress,” he said. “The idea is to be able to present our methodology to the peer review panel, have it vetted, and then be able to move forward.”
He then outlined the upcoming presentation. The team will first present a rational basis for assessing relative risks in the Delta, and then show how the risk can be assessed in ways that can be readily updated based on new and changing information. He reminded that they are relying on existing information, so as new information becomes available, the results will need to be updated. They will describe how new or proposed levee improvement projects can be readily measured or compared for their ability to reduce risk to property and lives, to water supply, to the ecosystem, and to the Delta as a place, as well as their cost-effectiveness. They will also describe the cost allocation methods for how these improvements can be paid for.
“The approach assesses how investments will reduce risk in the Delta and at what cost,” Mr. Roth said. “We have a series of seven steps, including inventory the assets and identify hazards, evaluate the risks essentially without new investment, be in a position to rank islands and tracts by risk, and using this information, evaluate levee investments that rank those levee investments for risk reduction and for cost. Then evaluate the risks in the Delta with both state and local share levee investment, and then finally contribute, with the Stewardship Council, to define a Delta levee investment strategy.”
“Another way to look at this is we begin by identifying risks, the risk framework for the Delta, then evaluate risk under current conditions, get into position to be able to identify a proposed projects that can be used to reduce that risk, evaluate those projects for their cost effectiveness and their ability to reduce risk, and from those projects identify portfolios that could be recommended for investment,” he said. “With portfolios identified, we have methodology to allocate costs and that all contributes to the development of a levee strategy.”
Mr. Roth then began with a discussion of tolerable risk. He explained that risk is determined by asking what are the hazards and how likely they are to occur, how will the infrastructure perform in the face of those hazards, and who and what are in harm’s way.
“Put very simply, risk is probability times consequences,” he said. “The probability of a levee breach and a subsequent flood in the Delta are due to many factors. There is certainly the potential for high water levels in the Delta, for seismic activities, and for the condition of the levees and the potential for continuing deterioration. These present threats to lives and property, to physical assets, agricultural lands and crops, but also to the ecosystem and to the water supply -that’s water supply as export and also as in-Delta water supply.”
Mr. Roth said that they relied on the publication, Guiding Principles for the Nation’s Infrastructure, published by the American Society of Civil Engineers after Katrina. “The intent of this publication was to take a look at infrastructure failures and see if there are any common threads, and in fact, the publication says there were,” he said. “It first says we should be reminded that critical infrastructure must hold paramount the safety, health, and welfare of the public it serves, and certainly the Delta levees do constitute critical infrastructure.”
“The four common guiding principles that were identified were the need to exercise sound leadership, to use a systems approach, the ability to adapt to change, and then to understand, manage, and communicate risk,” he said. “Today we’ll focus on the latter of those, which is how do you manage risk. From a historical perspective, our approach has generally been to try and eliminate risk, and we’ve emphasized design standards that are really based on levels of protection. Out of necessity, perhaps we’ve focused on hazards and not paid enough attention to the consequences. In our project, we’d like to be able to develop a methodology to reduce risk to tolerable levels, and then manage that risk by advising how to make cost-effective investments.”
Mr. Roth then presented a graph originally produced by General O’Reilly of the Corps of Engineers. “We begin in just about any critical infrastructure situation with a level of risk that needs to be managed,” he said. “There are a variety of ways we can buy down that risk. We can do things such as zoning and building codes, be better risk communicators and so on, but important, particularly in areas like the Delta, levees and floodwalls will be of great concern in their ability to buy down risk here. I also want to emphasize that at the very end, no matter how effective we are, we’re going to have residual risk that we need to deal with.”
In the past, design standards have been used to manage some of the risk, he said. “We have disaster rehabilitation guidelines in the Hazard Mitigation Plan or HMP that was the geometry for levees that was agreed to by FEMA and the Department of Water Resources,” he said. “Then there’s a similar set of geometries put forth by the Corps of Engineers called PL 84-99. These are really geometries, although in the case of PL 84-99, at least there are other requirements to be in that particular program. We also have common levee design standards; FEMA talks in federal regulation 6510 about the 100 year storm or the 1% event, and then more recently in the state of California, the legislature has mandated a 200 year level of protection for urban areas in the state.”
“I want to emphasize that while these are design standards that look at protection levels, but they don’t really recognize the residual risk that occur from larger events, and none of them are really safety standards,” Mr. Roth said.
So why not ‘appropriate levels of protection’? There are many reasons, Mr. Roth said. “What is appropriate and who is going to define that?” he said. “When we look at levels of protection, it really focuses on the hazard at the risk of ignoring the consequences; it also implies that risk can be eliminated. Most of the standards that derive from this are basic geometry that don’t say much about or anything about actual levee performance.”
The concept of tolerable risk represents a global shift from the idea that floods can be controlled to the concept that with tolerable risk, those risks can be better managed, he said. “It’s based on the premise that risk cannot be totally eliminated,” he said. “Tolerable risk is defined simply as the level of risk that people are willing to live with in order to secure certain benefits. There’s a range of risk tolerability all across the scale, from those that are totally unacceptable to those risks that are broadly acceptable; the idea is to try and find the sweet spot somewhere in the tolerable to broadly acceptable range.”
“This approach enables us to take a comprehensive look at probabilities and consequences and use those to informed decisions about how that risk could be reduced,” he said. “It supports policy setting and decision making, and it enables us to evaluate trade-offs between different approaches to reduce risk and focus really on levee improvements. It’s really very useful in allocating scarce resources, and we deal with scarce resources in looking at improvements in the Delta, tolerable risk offers advantages in clarity, transparency, efficiency, and consistency.”
“The principles include that risk cannot be ignored, and that absolute safety cannot be guaranteed,” he continued. “It allows us to do our risk management looking at equity and at efficiency, it allows us to look at individual risks versus societal risk, and it provides a framework that enables us to continuously review our risk profile. The goal is to reduce risk in all situations to as low as reasonably practicable, or ALARP.”
Mr. Roth said that this methodology leads to risk mapping, and he provided an example of a map expected annual damages in the Netherlands. “We’re following this general approach,” he said. “Our approach will lead to the development of similar maps for the Delta.”
“You can think of these as layers where we can look at the potential for life loss, for flood damage expressed as EAD, the risk to water supply, the risk to ecosystem functions, and the risk to agriculture; then we’ll present a way where those can be rolled up in various ways to look at aggregate risks throughout the Delta.”
“So that’s essentially how we intend to develop and portray risk information for use in this DLIS project,” concluded Mr. Roth.
THE PLANNING FRAMEWORK
Nidhi Kalra, PhD
Information Scientist, The RAND Corporation
Nidhi Kalra then discussed the framework for the tool. “What we’re here to do is to try and help the state come up with a set of investments it can make in the Delta that balance its very diverse risks, while also taking into consideration that resources are limited, and that we have to allocate those scarce resources into a smart set of investments,” she said. “Our approach is to base the investments on the way in which they buy down risk, and compare that to how much this buy-down of risk is going to cost, so I want to give you an overview of what the methodology and the framework looks like to do that.”
It’s a challenging project as the Delta is an extremely complex place with an uncertain future, she said. “The investments that the state will make in the coming years will last for decades and will shape the Delta potentially for even longer, so we have to take a very long view when we think about what the investments can be,” she said. “The data are always evolving. The data that we have at any moment reflects what was collected in the past, and conditions on the ground are always changing, so we inherently are playing a game with whatever the data is available. Our role here is to be aggregator of the best available data and to try and use that intelligently.”
“The big part of the Delta being a complex place is that we have a diverse range of interests, not just in the Delta but throughout the state and in fact, national stakeholders who are interested in what’s happening in the Delta, so we’re trying to balance a wide-range of concerns,” she added.
The goal is to have a methodology and a process that is credible, transparent, collaborative, and uses the best available data, she said. “One that allows anyone in the public to see what we’re doing, takes into account a variety of different viewpoints, and takes a balanced approach to decision making,” Ms. Kalra said.
The process starts by inputting the best available data that has been collected on the islands, hazards, projects, beneficiaries, and costs; they then draw upon the data to get into a iterative process of analysis and discussion following the best practices by the National Research Council on how to do complex decision making in these conditions, Ms. Kalra explained.
For discussion, they are soliciting stakeholder’s preferences across the different interests, such as how does agricultural interests compare to water supply and ecosystem, what does risk reduction look like, what are the sensitivities of different risk reduction methods to the various uncertainties, and various different ways of looking at the data, she said. They are also looking at the tradeoffs of making investment A versus investment B, or a portfolio of investments versus another portfolio of investments; this is coupled with analysis of the effects and the cost effectiveness of different investments and portfolios of investments that together achieve a broad range of goals that no individual investment would be able to do, she said.
There are three results they are looking to achieve. “One is the list of islands that have particularly high risk in different categories which is crucial,” she said. “Our aim is to help the state take action, so then we are looking for a list of ranked investments of what are the investments that achieve the greatest buydown of risk and at what cost. The third thing we’re looking for is what group or portfolio of investments can help balance the interest in risk reduction across a variety of areas.”
When looking at investments, the question to ask is how does the levee system perform in reducing risks to lives, properties, and state interests, and how would this change with specific investments, she said. “Our goal is to come up with a variety of metrics or really specific measures we can use to assess the difference in risk with and without potential investments. The challenge is that there are so many different interests throughout the Delta, and we want to get as many metrics as we need to get an adequate view of the different risks, but if we have so many – it is possible to have hundreds of metrics, then it becomes very difficult to use them for decision making. So we have tried to be judicious in our selection of metrics, such that together, they cover the variety of interests in the Delta.”
“The ones that we have chosen are ones that have sufficient data to develop credible metrics; there’s a scientific basis for developing the metrics, they are not overly subjective but are things people can look at and see how we calculated it, so you’ll see a selection of metrics that don’t cover every interest, but hopefully they cover every category of interest,” she said.
She then briefly reviewed the metrics:
Expected Annual Fatalities: How many lives would we expect to lose on average annually due to flooding. “This is constructed as expected annual, since we don’t know when a flood will occur and we’re looking at it over time across all different levels of flood to see what are the average effects.”
Expected Annual Damages: The expected annual damage to structural, agricultural and other assets that we would expect to incur. “This includes crop losses and agricultural losses, but we’ll also separate out the land effects on agriculture, since that’s particularly important.”
Expected Annual Water Supply Disruption Score: “Water supply is a very important concern, so we have a measure of water supply disruption. What is the likelihood of a disruption from a levee breach in the Delta or a set of levee breaches?”
Expected Annual Change in Habitat: “We’re looking at ecosystem in terms of habitat. How much habitat area or miles of natural channel margin would be affected due to flooding.”
Expected Annual Agricultural Land Loss: “Agriculture is particularly important, so we’re very interested in pulling out of the analysis what is the effect on agricultural land and what are the losses that might be expected from flooding.”
They also evaluate the efficiency of the investments, Ms. Kalra said. “We’re interested in the investments themselves and their cost, and we’re interested in are they efficient investments,” she said. “The cost is just not in dollars or how much do we have to pay for the investment, but also what is the effect of a particular levee project on habitat. Setback levees for example would create a certain amount of habitat, but at the same time, they may have a consequence on agricultural land, so we’re looking at the variety of ways in which the implementation of a levee investment project would affect the different state interests.”
These investments will have long term consequences, so the risk is assessed at three points in time: the present moment, 2030, and 2050.
There are a number of reasons why risks may change, Ms. Kalra pointed out: rising sea levels, population changes in the secondary zone, construction of water conveyance, the implementation of ecosystem restoration projects that will change risk, and the levee conditions themselves, whether levees are maintained or whether they are degraded and what those effects will be.
“These are the changes we’re looking at initially, and as we do sensitivity analysis, this list may change to include more things,” she said.
INVENTORYING ASSETS AND IDENTIFYING BENEFICIARIES
Jessica Ludy
Water Resources Planner, ARCADIS
Jessica Ludy then discussed the process of inventorying assets and identifying beneficiaries. “Our charge was to use best available existing information to support our analysis, so what comes along with that is a responsibility to also identify the general data gaps and the uncertainties associated with the data, as well as the limitations that poses to our analysis or to our methodology,” she said. “So what is involved, in addition to collecting various data sets from our partner agencies, is making public our data inventory, and discussing and validating these data with our stakeholders and our technical experts. This enables us to get a sense from those folks with the expertise in the area whether or not it looks about right, or whether or not we’ve got some outliers that we could use to look into more detail.”
She noted that the picture on the slide is the result of a risk communication effort by a number of state and federal agencies that shows the high water mark from the 1986 flood. “It is one of the great efforts going on by the state of California right now to communicate risk to people living behind levees,” she said.
“All of the data we’re using has been used in other state and Delta related studies, and what we are finding is that the quality varies with age and level of detail,” she said. “However, it does so far seem to support assessing relative risks in the Delta, and our methodology does enable us to readily incorporate new and updated information. It also enables a wide range of sensitivity analyses, so for example, how might part of the quality or coarse level of detail of parcel data or evacuation data affect our outcomes, particularly on state level investments.”
Ms. Ludy noted that they used parcel data to get the inventory of how many structures are on each island. “If there are more structures on an island than our data set, this would likely increase the consequences of a flooding event, and if there are fewer, than this would decrease the consequences,” she said. “So what our methodology allows us to do is go a couple steps out and say, does an increase or decrease in the number of structures actually affect the investments that the state might make in the end, and if it does, then we know we need to go back and refine the data, get more detail, and perhaps validate it better. If it turns out that state investments in particular area are not that sensitive to a particular data set, then maybe we don’t need to go back and refine it.”
One of the challenges has been incomplete evacuation data sets, Ms. Ludy noted. “It’s been very challenging to find available data on evacuation routes and likely these would definitely affect state investments,” she said. “If you were really concerned with improving evacuation effectiveness, you would need to know where these are, so we know these are the types of areas where we’re really going to have to dig deeper or reconcile.”
Ms. Ludy said they are working with our partner agencies and the Stewardship Council to acquire improved data sets where necessary, as well as working with the Council to develop a process for incorporating the new data when it becomes available.
The first charge was to define the geographic scope for the analysis and coming up with a single list for all the islands and tracts in the Delta. “We decided partly as by mandate from the legislation that we were required to consider all areas within the legal Delta and Suisun Marsh, and then we included within that, every area that is subject to flooding from the .2% chance or 500 year flood as delineated by FEMA,” she said.
Despite the vast number of studies that have been going on since the 1960s and 70s, there was no single agreed upon list of the Delta tracts and islands, so the process required consolidating and reconciling numerous lists, she said. “We worked with partner agencies and solicited feedback from a number of the reclamation districts to ensure that we had our lists right, and ultimately we settled upon a list of 176 total islands and tracts that will be analyzed, 170 of which are dry and six of which are currently flooded.”
The goal is to use existing data to compile an inventory of assets on each Delta island and to identify data gaps, limitations, and uncertainties, she said. “This asset inventory is the basis for the flood exposure analysis; it feeds into the flood risk analysis and expected annual damages calculations.”
Ms. Ludy said they followed standard protocols for the types of assets that are considered when doing a flood exposure analysis:
- Lives and property: Parcel data, population count, energy and utilities infrastructure, critical facilities, transportation and navigation, wastewater treatment plants, and public facilities, such as schools.
- Delta as a place: Agricultural land, crops, and crop value, public lands, cultural resources and recreational facilities such as marinas.
- Ecosystem: Habitat area
- Water supply: Conveyance, intakes, and in-Delta infrastructure
The data is then pulled together and organized by island or tract, she said. “We have a spreadsheet for every single island with sort of a head count of everything that’s on there,” she said.
“If you live in the Delta or if you’re familiar with this region, you are probably aware that the Delta is in fact a system and it acts as a system, and within that to make it even more complex and complicated, there are a number of subsystems and interrelationships and connections within the Delta,” Ms. Ludy said. “Just a few of the type of system assets and systems that we have in the Delta are critical infrastructure, transportation networks, energy infrastructure networks, the ecosystem, water supply and quality, and evacuation routes. It is definitely a challenge to capture a complex interrelationships, particularly when your base unit of analysis is on an independent island by island basis.”
“What we’ve talked about so far really involves looking at the number of assets on an island and assessing direct impacts of flooding to an island, but that doesn’t necessarily capture the relationships with neighboring islands,” she said. “But we are looking at a few systems so far. We have the ecosystem and the water supply system, and the GIS approach we’re using allows us to look at the data we have on a map and be able to identify critical links and points where if they were disrupted, it might cause a system effect, and similarly, identify investments that might reduce risk to a system.”
With respect to identifying beneficiaries, there are a lot of different people and uses that benefit from Delta levees, she said. “Our goal was to identify all of those entities that that fit, and these benefit categories that we develop will become part of the cost allocation procedure,” she said. “So we cast a really wide net in thinking about benefits and beneficiaries, based on this idea that all who benefit from the Delta’s levees should really contribute to maintenance and improvement of the Delta levees. We’ve started with the asset data to get a sense of owners, operators, and users from the assets, and then also getting stakeholder input.”
Ms. Ludy said they would be coordinating with the Delta Protection Commission, who is simultaneously working on Delta levee assessment district and is also looking at the beneficiaries and benefits.
HAZARDS AND VULNERABILITIES
Hollie Ellis
Senior Vice President, Shannon and Wilson
Hollie Ellis on the Arcadis team then discussed the hazards and vulnerabilities of the Delta levees. They reviewed the literature to determine the causes of levee failures, and then categorized them according to natural and human actions. He noted it is less important how the potential threats were categorized but that all of the potential threats to the levees are captured.
Natural hazards
Hydrologic/hydraulic: Hazards include high volume inflow, high flow velocity, high head differential, river morphology changes, and rapid drawdown. “Some of these may not be the most important in the Delta, but we examined all of these again to make sure that we really understood all of the potential threats and could identify the ones that were really most important to the Delta,” Mr. Ellis explained.
Climatic change: Sea level rise; greater head differential
Wind: Wave run up and storm surges
Geologic/geotechnical: Soft or organic soils below levee embankment or on the landside; earthquake induced liquefaction. “The soil conditions under these levees and the soils that the levees themselves are made out of are less than ideal from an engineering point of view, but they are what they are and we deal with them accordingly,” he said. “In some cases, the potential liquefaction introduced by earthquake, some would categorize that as a threat, some would categorize that as failure mechanism, but we’re not interested if we categorized them correctly but did we capture everything.”
Ecologic: Animal burrows; vegetation type or location
Human actions
Permanent: Encroachments, channel dredging, deferred maintenance, and upstream water management
Temporary: Wakes, impact from ships or debris, fires, footpaths, etc.
“We build things up against the levee and cut into the toe of the levee and do maintenance or not,” he said. “These human activities also have an impact on the performance of these levees. Some are permanent, for example, an encroachment; some are periodic, for example, levee maintenance that doesn’t happen every day but happens on an as-needed basis.”
“We looked at all of these hazards and what had been concluded in the past studies and reports, and we determined that these are the most significant threats to the levees in the Delta,” Mr. Ellis explained. “From that point forward, our metrics and our analysis are based on analyzing these particular threats, current and into the future.”
Mr. Ellis then explained how the vulnerability of the levees are evaluated. “The concept of risk is probability times consequence, and it allows us to compare 2, 3, or 170 different islands that all have widely different probabilities of a levee breach occurring and widely different consequences should the levee fail, but this risk calculation puts everything at the same scale so that we can compare one island to another, or the consequences of one island to another,” he said.
To evaluate the probability of a levee failure, they looked at the DRMS study. Mr. Ellis noted the curve on the upper right shows the return period in years of a certain inflow into the Delta in cubic feet per second, which determines the probability of water inflow of a certain amount in any given year. Tidal influence is also a major factor in controlling the water levels in the Delta because as the tide rises, the water level in the Delta has to rise as well, and as the tide falls, the water level in the Delta falls as well, he explained.
“In the DRMS report, there is a series of equations that relate the total inflow and the tide to water level at certain locations,” he said, noting that the slide shows 15 red dots that are the basis for the multiple regression that was done. “We used those equations to then predict what the probabilistic distribution of water levels were at each and every of these 170 islands and tracts would be. I show an example there for the Lisbon District, the blue curve is the curve that we use in our analysis based on the DRMS equations and the dots are from the Central Valley flood protection prediction for the same location, just to show that the two different methods are giving essentially the same potential distribution of water levels.”
“Wind is a very significant contributor to water levels that might impinge or even overtop the levees, and so relying on information that’s in the DRMS report, the upper left curve shows the probability distribution of wind speed for the different climate conditions that occur in the Delta,” Mr. Ellis said. “The table on the upper right relates wind speed and fetch and the length of open water in front of the Delta to the amount of water that will rise up on the Delta – the wave run up, so I combined those two to obtain the curve on the lower left for a particular location. This shows me the annual probability that I would get a particular wave run-up of a certain type, and of course that depends on what my assumption is of the fetch length is. I show this for a 2 mile or 4 mile fetch length.”
“Then I combine that in the lower right with the inflow and tide effect,” he said. “We have water flowing into the Delta, we have the tide flowing in and out at the western edge of the Delta, plus the effect of the wind wave run-up, and in the absence of any wind, I would use the distribution represented that reddish curve on the left. Then depending on what the fetch length is that’s in front of the levee, I would use the distribution – either the black or the blue shown to the right there. So that way I account for all of the factors that influence the height of the water that impinges on the levee.”
“Now we come to the levee fragility curves that relate the probability that the levee will fail, given that there’s a certain height of water on the levee,” he said. “We dug out whatever fragility curves we could find from previous studies. This happens to be some curves from the Central Valley Flood Protection Plan study; this is a typical set of curves that show as the water gets higher and higher on the levee, the greater the likelihood of failure.”
He noted that the graph on the right shows overtopping fragility. “A well built levee will hold back the water when the water is all the way to the levee crest, but as the water rises above the levee crest, at a certain time, you get erosion and failure and that is what the overtopping fragility is related to.”
Mr. Ellis said that there are levee fragility curves for about 15% of the 170 leveed islands and tracts, they had to develop levee fragility curves for the remainder. “These have been developed in part based on our knowledge of what the levee foundation conditions are, what the levee soils are, and the geometry and the height of the levee in particular using a method based on a Corps of Engineers approach for developing levee fragility curves when you don’t have geotechnical explorations and geotechnical analyses,” he said.
Mr. Ellis gave an example of how this was done. “For example, we assume that there is 0 probability of failure when the water level is at or below the average elevation of the island or tract. We assume it’s 100% probability of failure when the water reaches 3 feet above levee crest, and then we estimate a couple of other points on the fragility curve, based on what the general shape of known fragility curves … We use that shape to inform these estimated fragility curves in cases where we don’t have on in the literature.”
Mr. Ellis then discussed how the probabilities of failure are calculated, noting that these probabilities are used in all of their methods, whether it’s expected annual fatalities, or the ecosystem function. “So for the probability of levee failure or levee breach due to flood, we have a stage recurrence curve, which would include the tide effect, the water inflow effect, and the wind, if it’s appropriate to that island,” he said. “I integrate that curve to get the stage occurrence curve for each range of potential water levels impinging on the levee. That multiplied by the probability that the levee would fail at that level gives me the probability of levee failure for each potential river stage. And I sum all of those overall potential river stages and get the annual probability of levee failure due to flood effect, due to hydraulic effects.”
It’s a similar process with respect to earthquake failure, he said. “From the USGS, I get a Peak Ground Acceleration exceedence probability for each and every island and tract; it’s a cumulative probability distribution function, similar to a stage recurrence function. I integrate that to obtain a probability of a particular range of PGAs occurring, and again from the DRMS report, the seismic fragility which relates the probability of levee failure to the occurrence of a particular ground acceleration.”
“To calculate the probability of a levee breach in an earthquake, we differentiate the recurrence curve to get an occurrence curve multiplied times the fragility curve; this gives me the probability the levee would fail, given a small range of accelerations, and sum that over all accelerations to get an annual probability of failure due to seismic events.”
METRICS: EAD and EAF
Hollie Ellis
Senior Vice President, Shannon and Wilson
Mr. Ellis then discussed how the Expected Annual Fatalities (EAF) and the Expected Annual Damages are calculated.
To calculate Expected Annual Fatalities, the first thing to do is figure out who is exposed. Mr. Ellis explained that they obtained the resident population of the 170 islands and tracts from the census list; used USDA statistics to determine a weighted annual average effective population; determined a recreation population as well as a traveling population. “Of course all of these people are not exposed to the floodwaters,” he noted. “There’s sufficient warning and evacuation and so the population that actually gets exposed to the water is much, much smaller.”
He then presented the equation, noting that it is used in a number of different places, such as the Corps flood damage assessment model and the FEMA hazards model. “This equation and variations of it are the basis of those systems, so we’re not inventing something new here; we’re adapting a common practice,” he said. “What we do is multiply the probability of levee failure times the portion of the population that’s actually exposed to the flood water; that proportion is the total population reduced by those who are willing and able to evacuate, and that’s a matter of the effectiveness of the warning systems.”
“It’s also a function of the type of levee failure,” Mr. Ellis continued. “If it’s a slow rise flood situation such as winter or spring runoff, there’s probably plenty of warning time. If it’s a sunny day failure or seismic failure, of course the warning time is much, much less. And then finally multiplied by a mortality function that relates the depth inundation to the likelihood of mortality. And this mortality function is based on work that was done post-Katrina, the situation here in the Delta is similar in the sense that a lot of the islands are bowls, they will fill up, there’s no place for the water to runoff, and these mortality functions are similar to work that’s being done in Europe, particularly in the Netherlands.”
This calculation is made for each potential river stage or channel stage that impinges on the levee, and summed over all potential stages to get an Expected Annual Fatalities; this is done for both the hydraulic hydrologic potential for failure and for the seismic potential for failure, he said, noting that hydraulic failure and seismic failure are considered to be independent events so the probabilities can be summed.
The calculation for Expected Annual Damages (EAD) uses essentially the same equation, except the factors are a little bit different. “We’re now using the same probability of levee failure, but looking at the asset values, say someone’s home, multiplied by the proportion of the value of the structure and contents of the home that would be damaged by an inundation of a certain depth,” he said. “So if the levee fails at low water, we have 1 or 2 feet of inundation, you’re going to get one damage value for that home. If the water level is going to be very high, you’re going to get a greater damage to the home.”
The cost of rehabilitation of the island is included, because the flood waters won’t recede, but instead has to be pumped out. “There is a cost for doing that, and a different cost for each potential inundation depth at the island,” he noted. “We sum the overall potential river stages and seismic events, and sum the two of them to get total expected annual damages, and we do this for each and every island and tract. For the rehabilitation cost component, we’ve assumed that there’s a fixed cost of repairing the levee, and then a variable cost of pumping the water out of the island that’s proportional to the volume of water that floods the island, and that’s the fixed costs plus dewatering on the far right on the equation.”
ESTIMATED ANNUAL WATER SUPPLY DISRUPTION SCORE (EAW)
Alex Trahan
Coastal Engineering Specialist, ARCADIS
Alex Trahan then discussed how the metric Estimated Annual Water Supply Disruption Score is determined. He began by pointing out that water supply disruption and water supply reliability are a complex topic, difficult to succinctly and clearly define and even more so to quantify. “We’ve gone through a lot of iterations on this to figure out what we can use from other studies and how we can draw from work that’s been done before to express water supply and water supply reliability in a way that is useful and leads to good planning ability,” he said.
The primary problem in the Delta with water supply is salinity and salinity intrusion, he said. “The Bay and the Pacific Ocean are a huge source of salt water, and there’s a great concern that salt water will sneak its way into the Delta and the salinification will change the nature of the habitats and agriculture in the Delta,” he said. “The western Delta islands are the gatekeeper between the Bay and the Delta, so we wanted to be sure we incorporated them when we talk about water supply risk in the Delta. We also wanted to be sure we captured the through-conveyance corridor which connects fresh water from the Sacramento River in the north to the Clifton Court Forebay pumps in the south. We’re talking about a lot of water here, about 80, 85% of the water that humans take from the Delta, so this is really an important corridor and important idea to be sure we’re recognizing and incorporating as we consider water risk.”
Mr. Trahan said the basic form of the metric is similar to the EAD and EAF, although there’s one critical difference in that it’s more in the form of a conditional probability then in the form of a risk. “We have the probability of a levee breach multiplied by the probability of a disruption, given that there’s a levee breach; the second term, the probability of a disruption given that there’s a levee breach, is a complex idea.”
This begins with determining the flooded volume for a given island, or how much water will be pulled in, he said. “The easiest way to understand flooded volume as a proxy for water supply disruption is to think of when the island opens up, the water has to come from somewhere, and the Pacific Ocean and the Bay are a huge source of saltwater. So when you open up a large volume and its start pulling water, a lot of that water is apt to come from this salty source and lead to salinification of the Delta. And that’s why flooded volume offers a proxy for the risk we’re seeing here.”
“However, it doesn’t capture everything,” he acknowledged. “It depends on where you are in the Delta, and it depends on the hydrodynamics of the location of your island. If you have an island that’s in the far north of the Delta very far from the bay and it fills with water, chances are a lot of that water will be fresh water, and it won’t pull a lot of salt water from the Bay. However, if you flood Sherman Island, you are right there at the bay and you’re apt to pull a lot of salt water into the Delta. That’s not captured by simply considering volume.”
Mr. Trahan said they determined three factors that were important to account for in the metric to move beyond the simple concept of total flooded volume. “True flooded volume is a measure of the accommodation space of the water you’d need to fill that island, and there are three influence factors. The first is proximity to the through-conveyance corridor that is important for conveyance of freshwater to all Delta exports and it serves as a proxy for the quality of water in the Delta. Second, we looked at salinity influences; this is more hydrodynamics driven, it’s less linear, it’s more complex, and so this was a way for us to capture the things that you can’t see by just location and distance between two points about how water is moving through the Delta. The third factor is in-Delta infrastructure, or water supply infrastructure that’s in the Delta for water that’s provided to the Delta and surrounding communities. Each of those three were given a weight of one-third … we multiply that by the true flooded volume to get an effective flooded volume.”
He then walked through each of the influence factors. “There is a through-conveyance corridor which runs from the freshwater Sacramento River down to the Clifton Court forebay pumps; the shortest distance between an island’s boundary and that corridor is measured; and the closer it is, the higher the score, up to a third,” he said.
“We also looked at salinity modeling done by DWR for breaches of individual islands and breaches of groups of islands together; what they reported in that study was the percent change in salinity at the Clifton Court Forebay pumps, so we used that again as a proximity for salinity changes in the Delta as whole,” he said. “Then we also looked at in-Delta infrastructure, so you can see there are aqueducts and water conveyances through the Delta, and the islands that have that infrastructure are marked with one-third. You sum up the scores in each of the categories to get a total influence factor. So then you have your true flooded volume multiplied by your influence factor to give you an effective flooded volume.”
We still have to get from the effective flooded volume to the probability of disruption, and that is done using a logistic function, Mr. Trajan said. “Logistic functions are relatively common in environmental situations. There is a period of stability, followed by a period of change followed by a period of saturation. During the period of stability, the system basically corrects for itself, so if you have small changes, it doesn’t change the functioning of the whole system. But then you get this tipping point, after which the system can no longer accommodate those changes that are being made … so you get rapid change. Then there’s a second point at which you’ve saturated your system – if I flood even more volume, it isn’t going to make it any more likely than 100% that we’ll have a disruption.”
“So we use that as the form of our distribution which we then fit to the data we have at hand, so we’re actually fitting our scenario in the Delta with this function,” he explained. “We did that using three points: We used a very low probability point, a very high probability point, and a point in the middle. Our low probability point was if there’s no flooded volume, the chance of there being a significant disruption is very low – not zero, but very low. If we were to flood all of the western islands, all of the central islands, and all of the Old River islands, the chance of a disruption is very high – that would leave a direct corridor from the bay to the Clifton Court pumps and would open up a rainbow of saltwater that could flow through and then spread its way out through the rest of the Delta, so we’re pretty sure if that were to happen, there would be a disruption to water supply in the Delta.”
“The third point is a little trickier,” he continued. “That one we had to define somewhere in the middle of that curve, and we did that based on western island failures. There’s an assumption that the western islands are quite important to salinity in the Delta and there are many studies supporting that. So what we did is we said if all of them failed, we’ll look back at the salinity study and say what are changes based on individual breaches and what are the changes based on group breaches, and we’ll sum all that up to represent the effect of losing all those islands. It came to an 83% change, and so we said 83% chance of failure if all the western islands go. So that gave us our three points, we were able to fit our distribution, and then we could work our way from true flooded volume, though that influence factor, to an effective flooded volume, and then from the effective flooded volume, we could look at the likelihood of disruption.”
“And basically that’s how we went through the water supply assessment,” Mr. Trahan concluded.
ECOSYSTEM METRICS
Ramona Swenson, PhD
Restoration Ecologist, ESA
Dr. Ramona Swenson then discussed the ecosystem metric, Expected Annual Change in Habitat (EACH). “The effect on the ecosystem will be measured by the quantity and quality of the different habitat types that we are maintaining, creating, or losing due to flooding or levee investments, so here we’ve calculated Expected Annual Change in Habitat, and like the others, it’s related to the probability of a levee breach times the net change in habitat due to flooding.”
Habitat types can encompass a range of values depending on the landcover, and can include both natural communities and agricultural communities, she pointed out. “With the existing land cover in the Delta, we gave a habitat quality value or rating of these based on the number of species that might be supported, its priority in conservation plans, or in recognition of losses of these habitats,” she said.
“The Delta is largely agricultural land cover, and for some of these agricultural types, they also provide habitat for certain wildlife,” she said. “For example, corn when flooded in the winter can provide wintering habitat for cranes and waterfowl; and rice has a habitat value for waterfowl as well as giant garter snakes, so we tried to capture that by giving some habitat related values for these particular agricultural types, usually annual crops, and recognizing that certain other agricultural types may not have a habitat specific value.”
“We also wanted to capture the fact that tidal marsh has been very highly valued in the Delta,” she added. “The Delta used to be mostly tidal marsh, but now we have less than 2% left, and it’s a very high priority for restoration in many plans and programs.”
Looking at the potential change in habitat due to flooding, the top diagram shows existing habitat with a channel to the left and a levee, and then on the inside of the levee, typically land cover that might be agriculture, she explained. “The dip is to recognize that most of our islands are bowls and not really islands, and if we had a levee breach as shown in the bottom diagram, we would expect that the levee, the islands would fill up with water and you would have a proportion of habitat that could be subtidal or on the rim of the island if it was at the appropriate elevations; you might have some intertidal habitat where you might get some emergent marsh developing; and you might also have some riparian habitat, either on the levee slope itself or maybe on elsewhere on the island, depending again on elevation.”
So we looked at existing habitat and asked what would the potential change in habitat type be should an island flood, she said. “This is driven by elevation, so we took elevations that were mapped as part of the Delta Plan, and we estimated what the likelihood for those habitats are. The deeper blue shows areas that would be subtidal which is below mean low low water. Those different bands are in bands of 1 meter, so the deepest blue is greater than 3 meters, greater than 9 feet below mean low water. Then we take a habitat rating value for those, again relative to species it might support or its priorities.”
Dr. Swenson noted that seasonal floodplain habitat and intertidal habitat were given a very high value, while subtidal habitat value potentially varies, depending on its depth and other factors, such as would it be likely to become overcome with invasive aquatic weeds or good habitat for warm water fishes that may prey upon native fishes. “Because there are differing thoughts on that, we left that as varied, and this is an opportunity for users to dial in certain ratings there, depending on the information that we have,” she said.
In addition to looking at the direct effects of flooding, we can also look at the effects of our projects on habitat by measuring the change in the area due to projects, she said. 
Those changes could include projects that reduce habitat, such as placing rip rap on the waterside of the levee, or projects that increase habitat, liking building a setback levee and potential gains in riparian or intertidal habitat.
EXPECTED ANNUAL AGRICULTURAL LAND LOSS
Jessica Ludy
Water Resources Planner, ARCADIS
“Agriculture is very important to the Delta, not only economically but also it is part of what helps define the Delta’s character; it is part of its heritage and a very critical component of what describes the Delta as an evolving place,” began Jessica Ludy. “As such we have two very simple metrics for looking at these effects that are very similar to the way that we measure changes in habitat area.”
“The effects on agricultural land will be measured by the area of agricultural land that is lost due to both project investments and due to flooding,” she said. “This first metric looks simply at the change in agricultural land as the sum of agricultural land that is lost due to the projects.”
The metric for the amount of agricultural land that would be lost due to flooding is called the Expected Annual Agricultural Land Loss. “It’s defined by the product of the probability of levee breach multiplied by the area in acres of agricultural land loss due to flooding,” she said. “This just addresses agricultural area; it does not go into detail about cropland or the type of agriculture that is lost, but the value of crops that are lost are captured in the expected annual damages calculation that Hollie described earlier. And the caveat on this particular metric is that it assumes that an island or a tract is not rehabilitated, and remains permanently flooded.”
BRINGING IT ALL TOGETHER
David Groves, PhD
Senior Policy Researcher, RAND Corporation
Dr. David Groves then discussed how all the information is brought together across the islands, under current conditions and without investment, and then ultimately under different investments.
He began by presenting the outputs of the tool, which can be divided up into three kinds:
List of high risk islands: “We’re going to be able to identify which islands are most at risk, which ones are moderately at risk and so on, based on the different risk metrics,” he said. “Now we know both from theory and from the work that we’ve done to date that the islands are at risk for different reasons and if we were to simply try and rank islands based on risk to EAD, that would be very different from the ranking of islands due to water supply risk metric; so what we’re doing in this stage of the analysis is understanding first what is at risk to what and how might that information combined with our preferences about how we want to balance across risks and does that identify some islands that are risky across multiple risk metrics, and so this kind of analysis that comes out of this process will help us focus our attention on those high-risk islands for developing investments to analyze.”
List of ranked investments: “What it costs to reduce risk is going to differ depending on the island and so we need to analyze how different investments will reduce that risk,” he said. “This process helps us understand how do investments reduce risk and if we combine that with cost, we can then have a metric of cost effectiveness. Then we can basically rank our investments by their ability to reduce risks at lowest cost, and again, this is potentially a very interactive kind of analysis because it depends on how you rank different risks in terms of how different investments are going to rise to the top of the list.”
Groupings or portfolios of investments: “We want to think about how these investments work together, how they perform when it’s not possible to do all the investments you want to do, so in other words, when its constrained by a budget, so this process is going to help us develop for the council a series of portfolios of investments,” he said.
Portfolios are created by looking at how individual investments reduce risk, combining that with preferences across those risks (which can be modified) as well as combining that with additional constraints, such as limited funding or tolerable levels of risk. “The process will help develop a portfolio of investments for each of these different conditions. Then we can look across all these different portfolios and weigh their tradeoffs in terms of outcomes as well as costs, etc. and then that’s really the key ingredients that are needed for putting down on paper the strategy of how we’re going to invest in levees given available funding.”
Dr. Groves then turned to the process behind development of the outputs. “We’re developing a planning tool which is really an aggregator of all of the risk information that’s being developed,” he said. “This planning tool is an interactive decision support tool, it’s designed to help bring the analysis to the council, to the stakeholders, and the public. It’s developed in a platform that can be made available on the web to different users so it really invites interaction and real involvement with the results.”
He then presented some screenshots of an early version of the planning tool. “One of the things that the tool does is it gives us a way of visualizing a variety of different kinds of information,” he said. “The tool helps us look at rankings of island risk interactively, so this is an example of how we might combine results for several different risk metrics, EAF, EAD, EAW, etc. and create and aggregate risk score based on different user-specified weights. The purpose of this is not to come up with a definitive list of risk ranking, but rather help people understand how different weightings of risks lead to different island being highlighted as high-risk.”
“The planning tool also then similarly supports a ranking and prioritization of investments,” he said, noting the graphic is just fictitious information for demonstration purposes. “We’re looking at abilities to rank investments based on these different metrics, based on user input in terms of preferences of different types of risk reduction.”
“The planning tool includes an optimization engine, which helps us develop optimal portfolios for different sets of preferences; this is just an example of how we might create three portfolios of options based on emphasizing different risk reduction goals.”
“Then lastly, once we’ve identified those portfolios, we can then look at key tradeoffs in terms of the risk reduction performance of those portfolios,” he said, noting that the slide is using fictitious data that we developed, but shows how each portfolio has different characteristics in terms of the risk metrics as well as cost.
COST ALLOCATION METHODOLOGY
George McMahon, PhD
Senior Water Resources Engineer, ARCADIS
George McMahon then discussed how costs would be allocated.
The purposes or categories for which costs will be allocated are defined as project objectives that have been authorized in law: Flood reduction, water supply reliability, and ecosystem protection, restoration, and enhancement, as well as the costs that are incurred in developing the project are the costs that have to be allocated, he said.
In determining how costs should be allocated, there are questions that need to be addressed, he said. “One is how is the federal interest measured, and that’s going to be important in determining the federal share in project levees,” he said. “What informs federal-state cost sharing; how much of the state’s share should be allocated to local management agencies; and then going on down to the end users, how much of the local shares should be allocated to the end users or beneficiaries.”
Dr. McMahon noted that the beneficiaries can be individual property owners, industries, water purveyors, agriculture or businesses. “They all benefit from the levee improvements in various ways, and basically aligned along those three purposes,” he said.
“The costs that we’re trying to allocate are capital costs and operations, maintenance, repair, rehabilitation and replacement, or OMRR&R costs,” he said. “You have the big bucket that represents total project cost. That big bucket is split in some way; There is a federal-state share, the remaining state share that then some is allocated to local management agencies, and then the local management agencies may choose to allocate some of their remaining portion to individual beneficiaries, or users. So if you add up the net federal share, the net state share, the net LMA share and the net share to be distributed among the beneficiaries, you’ll exactly equal the total project cost – not a penny more, not a penny less, so there’s not profit motive here.”
He then discussed the different types of projects that costs would be allocated for:
Project levees
Project levees are about one-third of the Delta levees and are those that are included in the State Plan of Flood Control. “The main distinction between those and non-project levees is that capital costs are shared between the federal and state government. There is federal allocation, and then the non-federal share is distributed on down by the state through local agencies. Maintenance costs are not shared by the federal government; they are shared between the state and local agencies.”
Historically, federal shares vary between 50 and 75%; the non-federal share is typically 70% state, 30% local, and then OMRR&R outside the Delta are delegated to the local agencies and within the Delta they are supported by the subvention policy, he explained. Under the current policy, the federal share can be up to 65% for ecosystem restoration; the minimum state share of the non-federal portion is 50%, and the state’s share can be increased from 50 to 90% depending upon different aspects of the project, such as whether it serves a disadvantaged community or includes ecosystem enhancement or other multi-benefit features that may not be actually measured. A state will share up to 80% for setback levees.
“This doesn’t really give any rationale for selecting where in this range how should they be allocated, it just says what has happened in the past or what current policy is, and there can be some conflicts or overlaps between these current policies,” he noted.
Non-project levees
Those are the 2/3rds of the levees that are not included in the State Plan of Flood Control. DWR provides assistance to local agencies from a base level share of 50% up to 90% based on what other mix of benefits are included with the project, and up to 75% maintenance cost-sharing. “I don’t quite understand where these practices overlap and where they conflict, but clearly we need to have something to fill in between the lines here. So on the non-project levees, we’re starting off with a federal share of 0, and so the non-federal share can vary by all these different criteria. 
He noted that there are differences between the primary zone and the secondary zone, and the state share can be increased based on different enhancements or contributions to public purposes.
For operations and maintenance expenses, the subvention policy is laid out in the state water code, and subject to availability of state funds, he said. There’s kind of a deductible of up to 75% after local agencies have expended $1000 per mile, he explained. The water code requires a determination or that the local agencies provide information on ability to pay, and DWR can use ability to pay for determining reimbursement, he said, noting that the ability to pay requirement is going to expire in 2018, afterwards it will simply be 75% after local agencies have expended $1000 per mile.
“There are multiple steps in the cost allocation methodology,” he said. “The first step should inform of guide the determination of federal interests and what the federal-state share of total cost should be, and for the non-project levees, it can also help to parameterize or set limits on the state interests, balance state interests and determine appropriate levels of local state-LMA sharing of non-federal costs, and that can be capital costs and operations and maintenance costs as well.”
“So the first step should provide a basis or a foundation for both project and non-project levee cost allocation, fortunately, we didn’t have to reinvent the wheel here,” Dr. McMahon said. “Because California is very closely intertwined and partners with the federal government on water projects, the state has adopted the federal principles and guidelines for water resources development basically in total. … Incorporated within that policy is the first step of what we’re recommending for the cost allocation policy, which is Separable Costs-Remaining Benefits (SCRB) method. … Basically it’s a fairly simple procedure; there are 8 steps.”
The important thing to note is that there are different categories and costs are allocated by purpose, he said. “The total project costs are made up of separable costs, which are the costs of including one of those three purposes, and there are joint costs, which are the costs of facilities that serve all purposes; those costs are allocated in proportion to what’s called remaining benefits received by each purpose. Benefits in this case are not the economic benefits necessarily – they are either economic benefits or justifiable costs, which is the least cost, single purpose alternative that provides those same benefits to that purpose. So benefits are limited; they are either the actual benefits or the least cost single purpose alternative, and if its done properly, basically, the multi-purpose project always has economies of scope and scale that a single purpose alterantive would not, and so from a fairness and efficiency standpoint, the SCRB method assures that at least at the outset, that all purposes will share equitably in the benefits of multipurpose development, and that no purpose will subsidize any other purpose. Efficiency is the idea that it will cost less to participate in the multi-purpose project than it would be to go it alone for that particular purpose to develop an alternative that serves just that purpose.”
He then presented an example from Appendix B in the DWR manual which illustrates the application of SCRB to the Hamiilton City flood protection project. “The interesting thing is that this was originally a single purpose flood protection project, and it couldn’t pay it’s own way, so to speak, so ecosystem enhancement or restoration was added as a purpose; the benefits of ecosystem restoration were limited to the costs of the least cost single purpose alternative. Going through the procedure, the thing to point out is looking at line K, the total cost allocated to flood damage reduction and ecosystem restoration, those two costs that are allocated to those purposes are less than line F, the limited benefits or the constrained benefits of each purpose, so in this case, those two purposes benefited from participating in this project and those two purposes were better served than if they had developed single purpose alternatives. They wouldn’t have had a project at all.”
“SCRB can also guide the determination of state interests by purpose, because when allocating costs downward to the LMAs, those agencies can receive benefits in different forms – some benefit from flood risk reduction, some benefit from water supply reliability, and they may benefit from more than one, so there can be different state LMA shares depending upon purpose, Mr. McMahon said. “That can inform that cost sharing, and then of course the ability to pay, which as long as it’s in effect, can also be used to temper those allocations.”
The manual outlines two approaches: a benefits-based approach which is limited more or less to flood risk reduction, and a financial approach that might be more suitable because it allows looking at state and LMA interests in particular, so they could vary in different proportions, he said.
How the LMA is going to distribute the remaining share to their constituents is their decision, Dr. McMahon said. “There are a couple approaches that we outlined in the technical memorandum that we’re not making any recommendation on. They could simply assimilate it all and either incorporate it as user fees or taxes; they could facilitate negotiation among different users to determine how they might agree to distribute the costs; or they could let trading go on and let each user or beneficiary trade concessions to defray their share of the costs.”
“In summary, the objectives are fairness primarily, so we’re trying to allocate only the costs incurred, so we want to allocate costs in proportions of benefits received; we don’t want any purpose to subsidize other purposes,” Dr. McMahon said. “There are four steps; the first step is the SCRB cost allocation by purpose and then that will inform the subsequent three steps listed here. Step 1 also informs the federal and state sharing, it informs the State-LMA sharing, and then it can constrain or set boundaries on what costs remain to be distributed between the local management agencies and the individual users.”
SUMMARY AND WHAT’S NEXT
Larry Roth
Principal Engineer and Project Manager for ARCADIS
Larry Roth then wrapped up the presentation. “We’ve established, we believe, a rational basis for establishing relative risks in the Delta, and that this assessment of relative risk can be readily updated essentially by plugging in new or changing information as it becomes available, so it’s a living system,” he said. “With this approach, proposed levee improvement projects can be readily measured, evaluated and compared for their ability to reduce risks to metrics of concern: lives and property, water supply, ecosystem, Delta as a place, and they can be measured and compared for their cost effectiveness. And we’ve also assessed current cost allocation methods.”
“Just to give you a quick view of the path ahead for us, we intend to adjust this methodology as recommended by the peer review process,” he said. “We’re just about to begin the identification of conceptual projects that we will use in the analysis; these conceptual projects will be first selected based on areas where risk according to the metrics we’ve developed, looks to be highest within the Delta, but we also want to be able to compare that against the goal or objective of achieving a PL 84-99 level of protection throughout the Delta and be able to compare that for cost and effectiveness in reducing risk.”
“After identification of these various conceptual projects, we’ll identify portfolios,” Mr. Roth said. “We’ll then take the portfolios and recommend a cost allocation method, using all of this information working with Council staff and their activities in the general Delta levees issue, develop a Delta levees investment strategy. That investment strategy will inform new policies and regulations in the Delta Plan, and under California law, those policies and regulations will have to be assessed for their impacts using the CEQA processes, and that will then inform the complete Delta levees investment strategy.”
“That concludes our presentation. Look forward to upcoming discussions.”
REVIEW PANEL PRESENTS PRELIMINARY FINDINGS
The afternoon of the first day was spent with the review panel and the Arcadis team answering questions and discussing issues. The panel then returned on the second day to give their preliminary findings.
(Note: Without a video, it was hard to identify who was speaking, other than review panel members and ARCADIS team members. Specific names are given where identifiable.)
Jim Mitchell, the review panel chair, began by noting that prior to the meeting, they reviewed the background material and the technical memos; at the public meeting yesterday, ARCADIS and their team of consultants presented the information, which they had an opportunity to discuss afterwards. The panel reconvened again this morning, where they formulated and discussed some of their preliminary general conclusions. They will meet again in closed session and submit a report to the Delta Science Program in the next 30 days.
He said their initial reaction is that they are looking at a very promising methodology and tool that will be useful in dealing with a challenging problem. “The whole Delta issue is a tough one, there’s no getting around it,” he said. “There are some constraints on what the results will be used for, there are some constraints on you folks that are doing the preparations, and we appreciate that you’re operating with very difficult task with limited time and limited budget, and that you’re charting new ground with some of what you are doing.”
The tool will be useful because there is a clear need for it, given the importance of the Delta and the Delta’s resources both to the Delta as a place, the state of California, and the nation as a whole, he said. He acknowledged that it’s mid-point in the development of the tool and that it is not yet complete. “Our understanding is that what the Stewardship Council is asking us for are our views on where you stand, and our suggestions on things that we can do to help you identify areas of strength and areas of weakness, so that’s the role we think we are to play.”
Mr. Mitchell said they had some overarching issues they would be discussing this afternoon, which they will expand on in their report, as well as answer the specific charge questions the panel has been given. In the report, they will give their recommendations.
Definitions
“We think it would be helpful if there were better documentation and more clearly defined terminologies, particularly with terms like tolerable risk, disruption of the water supply and what does that really mean, as well as the metrics that are being used and how you balance one against the other,” he said. “These definitions can be used throughout … I think you’re report has got to be clear, specific, and one that defines the boundaries of what you’re doing. What is your task, where does your responsibility end in the methodology that you’re being asked to develop, and when does it pass into the hands of the decision makers. There are of course many issues that come up … the inputs to the decisions themselves, but it’s going to be in everybody’s interest to have the boundaries clearly defined.”
Science panel member: “Some of the terms are vague and they need to be very clearly defined in terms of working what that metric means, and then as you move down to the decision support process, to be able to evaluate, does this ranks as number 5, number 3, what does that mean. What is the measure of it.”
Larry Roth with Arcadis responds that they recognize this is an issue, and asks if the team could provide a list of items they feel need more definition. “Just working within our multi-disciplinary team, definitions are often a problem amongst ourselves, we recognize, but if you could shed some light at some point about defining the list of terms, that would be helpful,” he said.
Data
Data forms the basis for a lot of the methodology, said Mr. Mitchell. “Things that we want to be sure are looked at are critical data gaps. We know that you haven’t got all the data that you need to do all the things that you want to do; we also know that you have been constrained to work with the data that’s already existing. One thing though that we’re not sure that in every instance you have provided us with the best existing data, and that’s the data that should be used. Along the same lines, the existing data maybe needs some QA QC to be sure that all passes the test of reasonableness.”
“I think the sense of the panel was the credibility of the calculations is both the calculations and what goes in, and we would like to see an ability to trace back through metadata or other methods where this data came from, because as you know every time data gets passed, some information is lost about it,” a member of the review panel said. “The equations always look better by themselves than with the caveats that go with it and the nuances, and you can’t recover some of that, but it should be acknowledged and documented as part of being transparent.”
Dr. Susan Cutter pointed out that there could be issues in the data source that you used so there is a need to go back and really look at that input data. She noted that it might be useful to do some ground truthing of a representative sample of the data. “As best I can remember, there’s no discussion in the document about your assessment of the quality of the data that you used,” she said. “We all know that data have issues, so it’s better to be up front about what those caveats are in your data, rather than assuming the reader knows how good or how bad the datasets are that you are using. And also, it’s important to document the procedures that you used to do the QA/QC.”
She also said it is important to be explicit about the metadata, which is the data about the data. “Where you got it, how you modified it, what year, what units it was measured in, the kinds of things we see in GIS all the time which is tell us about your data in a very systematic way; there are data standards that you can use to write up that metadata that would then comply across the board with federal levels.”
A panel member noted that it’s also important to document any adjustments made to the data to make it consistent spatially and temporally. “How did you adjust those data to put them into consistent units,” he said. “Depending on the source, they are in different units and different measures … at the end, the decision support system has to be able to go back and point to the results in terms of the original data, the quantification of those measures.”
Uncertainty
It’s important that the uncertainty be characterized, said Jim Mitchell. “There’s uncertainty everywhere, but some of you know more about than others and that’s a vital aspect of it all, so perhaps it would be useful to consider how to propagate data uncertainty through the calculations in the planning tool,” he said. “There are various ways to do this. I’m certainly no expert on uncertainty and its characterization, but where you put the variabilities in and how you propagate them can have a big impact on the output. And here’s where some sensitivity and scenario analyses can help inform in the uncertainty analysis.”
One of the members of the panel said they would be providing some recommendations about uncertainty, sensitivity, and scenario analyses in the upcoming report. “As you know they are not the same; together they can be extremely powerful but they also could not address some of the questions if they are not done in a systematic and coordinated way.”
Aggregation and scale – systems approach
Jim Mitchell then turned to aggregation, scale, a systems approach and multiple island issues, which he said were important to the success of the overall scheme. He suggested looking at methods for binning and ranking the outputs as well as inputs and island aggregations. He noted that one of the charges is to come up with a scheme to aggregate groups of islands into ‘prioritization groups’. “So it’s certainly time to start looking at how that’s going to be done, and there may be a generic approach to grouping that can be adopted by the users of the tool when you’re all done so they can do it with respect to certain specific things that they might be interested in,” he said.
“The grouping goes both to prioritization across the indices or objectives as well as within objectives, so that you can look at the ranking or grouping within this one criteria, so then you can use that as one means of looking at how does this meet the various objectives or criteria,” said one of the panel members.
Jim Mitchell said there need to be methods for characterizing the interactions, the responses and consequences among islands. “Things that happen here can influence what happened there,” he said. “I know that your work is focused pretty much on individual islands and tracts up to this point, but there are things that happen for groups.”
Jim Mitchell also said that the analysis might be expanded to include local, regional, and national benefits and consequences. “An island isn’t really an island,” he said. “Things happen here that have impacts that extend pretty broadly throughout the rest of the state with regard to the water supply, and the importance of all of that to the state is enormous and the importance of the state on a national scale is very important, so don’t think too locally.”
Review panel member Dr. Nathalie Asselman said that for the total number of fatalities, events and for the disruption, it would be very important to have some feeling of which islands might fail under similar conditions simultaneously. “We were thinking it would be good to look at different types of events – for instance, somewhere in the Delta, there might be a 1 in 500 year water level occurring, but it might be caused by high river flows and no storm surge, or it might be caused by moderate river flows in combination with a storm surge, and depending on what event that might lead to this 1 in 500 year water levels, and different combinations of islands might fail. I think it’s very important to get a feeling of which islands are likely to fail during a certain type of event, so that you can use this information to get a better feeling about the disruption and about its total number of fatalities that you need for the tolerable risk.”
Dr, Ari Michelson pointed out that the export of water can be extremely important. “That’s where you go from local, the individual island or the Delta, to regional to statewide impact, and at least in the charges and information we’ve read, it includes national,” he said. “The national might be something that could help bring in the Corps of Engineers or other support or funding sources on, and that’s really important, because incorporating the measures for what impact levees within the Delta might have on export of water to the rest of the state could be one of the major impacts, both economically and physically. If you start looking at Central Valley agricultural impact and the population impact in Southern California, it could well swamp the Delta impacts in some cases, and so it’s really critical.”
Dr. Susan Cutter notes that another way of thinking about aggregation is to look at low probability, high consequence islands versus high probability, low consequence. “The consequences could be differentiated simply on the basis of population, but it could also be differentiated on the category of ecosystem services, or something like that, and that’s another way of slicing rather than doing it island by island. And I think the idea is how can you come up with this mix of aggregation that gives you the best kind of result in this kind of prioritization process.”
Larry Roth from Arcadis notes that this is analysis is part of the work they have yet to do.
One of the panel members asks how the primary and secondary zone is included in the calculations. “Whether a particular island is primary or secondary, we didn’t see how that affected the calculations of anything.”
Larry Roth from Arcadis responded that the predominant factors are property and damage. “We don’t expect significant changes to the primary zone but there may be development in the secondary zone. Whether it is actually implemented or not, it’s still at issue. In some respects, the Delta primary zone is unique in that what we see today is going to be pretty close to what we see in 2030 and in 2050 in terms of population and to that extent, we’ve incorporated that … ”
“If you have an identical island that’s in the primary zone and it’s an identical island in the secondary zone, you’d get the same calculations out; there’s nothing that flags it as being in the secondary zone,” said the panel member.
Larry Arcadis said the secondary zone is predominantly it’s land use designation as opposed to anything else. “There is no other significance other than land use, which is a major factor … “
Jim Mitchell said that the panel thought more attention was necessary to the consistency of measurement units and the temporal dynamics. Dr. Ari Michelson provided an example. “One example is measuring annual damages as the current crop versus agricultural land as the lost benefit across time; one of them is dealing with a very short period of time as a measure, and the other is dealing with essentially a permanent loss. There needs to be some consistency in the measurement units of how that’s handled. One way of doing that monetarily is using present value, and that would capture the long-run in both cases, but it needs to be clear and consistent in terms of the analysis of what kind of measures you are using and are they comparable.”
The Arcadis panel suggests an equivalent annual series.
“That would be one way,” replied Dr. Ari Michelson. “This is a work in progress, but these are items or issues that need to be addressed before you get down to the decision support system of comparing these issues and items.”
Use of indices and representation of risk
“You’ve made a judgment that you’re looking at five or six different indices, and presumably you are summing the ranks across those indices with some subjective weighting system in your final tool that is still under development based on that one diagram we saw yesterday,” said Dr. Susan Cutter. “The philosophical question that we had is, is it really important to have rank from 1 down to 170? Or is it better to simply bin the data into top third, middle third, bottom third, and to see how the islands and tracts compare across these broader framings? Because ultimately, what is the difference between a 1 and a 3?”
“Methodologically, it’s perhaps easier to justify a binning system or categorization system than it is to justify the difference between rank number 1 and rank number 4,” continued Dr. Cutter. “And so this gets at that philosophical question of why the decision was made to go into the absolute ranking as opposed to some sort of binning process where you could still use index values and you would simply rank where these are in index 1 and you would get a 1 to 3 … rather than 1 to 170. I think it might make it easier for the end user to see very quickly that this was high in this and low in this one so if I adjust my subjective evaluations of this, the importance, I can make this one come up higher or lower.”
One of the review panel members said there was a lot of discussion on both the use and aggregation of indices, because there are other methods. “The literature, I’ll characterize it as mixed, but often the aggregation of indices presents some real difficult issues in terms of ending up with a decision support tool that’s useful and will end up being something that decision makers can take away,” he said. “At some point, the decision makers is going to need to be able to go back and be provided through this tool or through access to the data, what does this mean in terms of a quantifiable measure, as all of the indices have some sort of quantification. It could be a number, or in some cases, it could be high, low, medium in terms of habitat or there are some other measures that are characterized as high, low, medium, and so that translation needs to be made so that you can go back to it.”
A member of the Arcadis team said that it was unfortunate they could not demonstrate how the tool works, because if he understood the question correctly, that’s exactly what the tool does in real time. “The user can change those weights and in real time, see how they move. Now your suggestion to bin things versus rank may turn out to be very significant; may not, though, if it turns out that in 170, you have 5 or 10 that are clearly different and everyone else is pretty much the same,” he said. “We’re certainly looking at that, but that is really fundamental to how we want to use the tool and make it available to all users.”
One of the Arcadis panel members notes that there are questions about transparency of converting various quantified and other types of measures into an index, rather than reflecting what that measure actually is. “There are a number of methods that can be used without going to an index that still lead you to identification of prioritizing levees,” he said. “One example is if it’s damages, you can have the damages ranked in a column by a level of damages, and then the same thing in terms of habitat with whatever measure, and put down the order of ranking; then the weight can be applied to the top of that, to say the economic damages are the most important to me, so I’m going to rank this first and then weight each of the others, whether the economic damages get 50% and habitat gets 35% and so on. Then you can rank across the rankings, and it still provides the original information in that table is showing what the quantification is, how many lives is this, instead of an index.”
Dr. Susan Cutter noted that the choice of metrics seem to reflect some of the goals of the science plan in term’s of what’s important in the Delta, but there doesn’t seem to be a metric that captures Delta as a place. “Are you thinking of adding an additional metric that captures those cultural and historic attributes of the place? You have the data in the asset sheets at the end so we know the base information is in the database, but you haven’t taken the next step and created the metric that includes that.”
An Arcadis team member acknowledged there was not a separate metric. “But damages to historical places, state parks, legacy towns that are in the asset inventory are included in the Expected Annual Damage calculation, so in that regard, they are included in the metric.”
“But that’s damage,” countered Dr. Cutter. “A sense of place is more than just economic damage. It’s the presence or absence of these things, the presence or absence or recreational opportunities, things of those sort.”
David Groves on the Arcadis team acknowledged they have data that could be used to create a metric for that, but hasn’t been developed yet. “If you recommend we do it … we’re balancing a lot of different things so this is one of those pieces that we could do, especially if the Council believes we should expand our metrics to include that.”
Cindy Messer, Deputy Executive Officer of Planning for the Delta Stewardship Council, said that it is one of the discussions they have been having with the Council over the last few months. “We’ve been discussing what are the state interests, what is put under that category, and how do we address some of these more perhaps sometimes tangible but maybe sometimes not, values and assets,” she said. “Right now so far we’ve stayed focused on the ones we’ve built, and that discussion isn’t over yet.”
Ms. Messer said they have been thinking about what the primary drivers are for levee investment projects. “Let’s take recreation for example,” she said. “When we look historically at investments the state has made in levee improvement projects, recreation was never a primary driver. Maybe a secondary benefit, so that’s one area that right now has differentiated it. But when we get to that next stage after the strategy is developed and we turn to our partner agencies that actually provide the funding and help implement the projects, that may be the place where we say we’re going to put multiple benefits with these projects and here are some of the more of the Delta as a place that should be considered in those. That’s the thinking so far, but it’s not completely set in stone yet.”
Dr. Cutter suggests designating them as primary and secondary. “You’re not ignoring the co-benefits and the co-goals of the Delta but rather these are the primary and there’s some secondary ones we want to think about.”
David Groves notes that the system is flexible and database driven, so if they decide to add a new metric, it’s easy to accommodate.
One of the members of the panel notes that there is the issue of validation and verification of the results. “In regards to fatalities, historically at least, there haven’t been any fatalities, so how does that get addressed? I’m not suggesting it be dropped, but it may in terms of weighting, be dropped simply because it’s a pretty minimal impact compared to some of the others. One of the public comments yesterday was disputing some of the data that had been developed on Roberts Island. Just going back, not necessarily the entire dataset because that would be enormous, but at least spot checking some of these things. How accurate are you, and does the data you have as input, is that accurate, and then the output, does this make sense?”
A member of the Arcadis team agrees on the spot checking of data, but as far as validating the results, I think all we have to go on is historical evidence. “For example, for a certain level … say 500,000 cubic-feet per second of inflow in ’97, there were 11 failures in that particular year, not necessarily from that one event, but there are ways that we can use the fragility curves and the stage recurrence curves that we have to run a Monte Carlo analysis and see if our current inputs produce that kind of result for that level of flooding, so there will be a number of spot checks like that.”
A member of the panel says he recognizes that it’s a work in progress, but these things along with the assumptions need to be documented. Going back to indices, he said. “Working with decision support systems, often you end up with a number of mathematical equations and lots of different inputs and these types of support systems, you turn the crank and out pops a set of results, and typical statement is the answer is 42. And then you have to go back, what does this mean and so that the DSS tool has to translate back to quantifiable number of bodies, agricultural land, etc, so I just want to make sure I highlight the importance of that.”
The Arcadis team concurred. “The way we view Decision Support Tools is that it’s not a tool to provide an answer, it’s a way of bringing analytics and best available science into a deliberation process,” he said. “The goal of the decision support tool is not to tell the Council what to do but just give it access in a more useful way, what is a variety of technical and scientific information that we have.”
One of the panel members asked how engaged the Arcadis team has been with stakeholders in the designing of this tool.
As part of the project, a communications and outreach plan was developed that includes outreach to stakeholders at various levels, responded a member of the Arcadis panel. He noted that they have identified agency stakeholders and met with them at a minimum of once a month; they have established a number of technical resource groups that are by open invitation; they have had meetings where they’ve shown various aspects of data gathering and the beginnings of the development of the tool; they have held a series of public meetings, as well as smaller group meetings, and they try to accommodate anybody that has requested to meet with them.
In the next couple of weeks, there will be having a series of public meetings on the notice of project for the environmental analysis, followed very quickly with public meetings for the next roll out of information, the Arcadis team member continued. “We have a series of four public meetings throughout the course of development of this particular project, and then there will be first be additional meetings associated with the environmental process. When the tool is ready to be shared, we will make it available at public meetings so that people can use it, try it, and we’ll be rolling it out similarly with the technical resource group.”
“In my mind, that’s been one of the real challenges of the project,” the Arcadis team member said. “There’s a tremendous number of interested people and trying to gather them together efficiently and make it worthwhile for their time to gather them and tap into whatever particular expertise or data they might offer, it’s been a bit of a challenge, but we’ve tried diligently to include as much stakeholder feedback as we can. It is very important.”
Larry Roth from Arcadis said that the importance of the levees in the Delta takes on a particular perspective, depending which hat you put on. “But we have a very precise set of guidance and that is prioritizing state investments to reduce risks to state interests in the Delta,” he said. “That has been the sideboards in which we’re trying to operate, and so some of the things such as state interests are being provided to us by the Council, and we’re attempting to respond with metrics or indices to address those state interests. As Cindy said, they are still in the process. We’re looking precisely to advise the state as to how it should it’s prioritize its investments in the Delta, which is only a very narrow slice of the entire Delta situation when it comes to levees.”
“My understanding is the tool is not supposed to be limited to just the impacts in the Delta, but to state assets and investments, and that’s one of the reasons why that export of water is so important,” he said. “It’s not just the Delta; it’s regional and state and possibly national associated with that; it is the importance of identifying the specific beneficiaries. Are they within an island, portions of the Delta, or groups; it is the region down to Stockton, Southern California, those beneficiaries, whether it’s by user group or by spatial location, it’s important to identify those.”
“We do recognize it’s the state writ large,” acknowledged Larry Roth.
“I would add the ecosystem restoration and the benefits derived from that well gets to that broader scale,” said Cindy Messer. “To add a couple of things to what Larry was saying, it’s definitely been a tough part of this project; we’re doing our best and we are engaging with many groups. The Council meetings are monthly, they are webcasted, they are open to the public, and we update the Council every month on the progress of the project and the milestones and decision points that they need to weigh into. … Not to make any excuses, we’re trying to be very cognizant on how we navigate through, competing with a whole lot of other high priority issues and projects and tasks, and agency activities out there. I think we’re doing a reasonably good job; I wish we had another twice the number of people we have just to get out there more, but we’re definitely using all the different venues we can think of to be able to engage folks. The idea is that we may not all agree at the end of the day, but we certainly want anyone who is interested or has a vested interest in this to understand what we’re doing, why we’re doing it, how we’re doing it.”
Ms. Messer also said she appreciated the comments about documentation, transparency, and the need to be able to go back and see how the calculations were made. “I think that’s a critical piece, especially as we move past this first phase and we look back 5 years from now and we need to do some updates, just being able to understand how we got to the end point this time around.”
Dr. Susan Cutter noted that in the discussions yesterday, they talked at length about water supply disruption and the metric used to capture that, which is salinity. She asked the team if they had any additional thoughts on the metric.
A member of the Arcadis team said they were working to clarify the factors that determine whether an island flooding contributes to the risk of a supply disruption. “As the metric is currently configured, it’s a combination of islands that are considered in close proximity to the supply corridor; islands that have salinity effects if they were to flood, both positive and negative; and then lastly, islands that have critical infrastructure on them, so I think actually, it’s a bit more accurate to say our current metric considers salinity as well as critical infrastructure. For instance, we’re concerned with the islands on which the statewide aqueduct travels on, and that’s not a salinity issue. We’re working to make sure that when we think about the factors that influence whether an island floods and has an impact on risk, making sure that those are clear, and transparent and understandable and relate back to what people understand as the critical islands. It may be that we rethink about how we come up with an aggregate score to determine whether an island is important or not when it floods, so that’s one of the areas where we’re looking at.”
“The second area is continuing our thought process on multiple islands and ensuring that as we’re thinking about individual island failure, we’re doing it within condition on the likelihood that it’s triggered by an event that’s flooding other islands simultaneously,” he continued. “We’re working through that to make sure that we’re both doing it properly, and then if we are, communicating that.”
“This is a good example of where we have made not only a good faith effort but have succeeded in our outreach,” said Larry Roth from Arcadis. “We’ve established good lines of communication with those entities and organizations that are most affected by this and we’re gaining their feedback and their input. It is a work in process, but we’ve met many, many times with the key water purveyors; we’ll continue to do so, and I think that will be a mark of real success in terms of outreach and using stakeholder feedback to advise us and inform us on that particular issue.”
Dr. Susan Cutter notes that this brings up a point about metric versus index. “This is a case where you’re creating an index rather than a metric … There’s nothing wrong with that, I’m just pointing it out.”
Dr. Ari Michelson said he disagrees. “I think the index really loses a lot of what the impact is, and this goes back to the beneficiaries. I would strongly encourage efforts be made to quantify the water quality as well as the water quantity impacts, not just within the Delta but the Delta and beyond, the Central Valley and Southern California. By quantifying that if those impacts are shown to be significant, it can really be used as strong justification to providing funding for Delta levees, and that’s the kind of support would be very helpful for the Stewardship Council as well as the other stakeholders, so the use of an index of disruption within the Delta I think reduces the effectiveness of that metric … it makes it not a metric in many ways.”
A member of the Arcadis team said the challenge is in estimating the specifics of disruption due to an event that affects the islands. “It’s a challenge to come up with an approach that can determine how much the flooding of an island or series of islands actually reduces water deliveries to a particular user. We think that given the data the we have, that that’s a pretty big leap to make which makes it very difficult to say you have a risk of a reduction of a certain amount of acre-feet being delivered to a particular user and therefore that has a particular value … I think this is a challenging kind of analysis to do with the kinds of data we have, which are the probabilities of flooding of islands due to earthquakes and floods. But I take your point that we do want to move beyond a very abstract disruption risk and make it so it is possible to understand how does that compare to EAD risk and other types of risk and so one of the things we were discussing this morning is not overaggregating through that … not combining the salinity, the infrastructure, and the conveyance factors into a single one but instead breaking out by user group, because then even though we may not be able to say you have a one in five chance of losing 20,000 acre-feet over a certain period of time, we could say this island has this percent chance under this scenario to have an impact, have some sort of disruption … “
“It’s more likely a group of islands … “ notes the panel member.
“Yes, and so by breaking it out by the different users,” responded the Arcadis team member. “I think it’s easier to relate that to something tangible, right now I can see it’s a single risk to water supply and that makes it difficult to know, is that EBMUD supply, is it SoCal supply and how much value does it have? It depends on which one you’re talking about. So we definitely take your point, and we’re striking that balance between what can we do faithfully with the data we have and credibly, and how can we make the metric most useful to ultimately prioritize risk and investment.”
The review panel’s report is expected in mid to late July.
For more information …
- Click here for meeting materials, presentations, and links to listen to the audio for both days of the workshop.
- Click here for more information on the Delta Levee Investment Strategy.
Get the Notebook blog by email and never miss a post!
Sign up for daily emails and get all the Notebook’s aggregated and original water news content delivered to your email box by 9AM. Breaking news alerts, too. Sign me up!
