Lies, Damn Lies, and... (Another Look at LEED Energy Efficiency)

Interesting, a LEED building would be hard pressed to beat one with no occupants and no energy usage.

Energy Intensity Index: On a delivered energy basis, the overall commercial sector energy intensity in 2004 was about the same as in 1985. The declines observed in 1991 and since 2001 are primarily the result of economic recessions. In these periods, vacancy rates of commercial office and retail space increased and the utilization of occupied space fell.

Trying to measure the performance of a building using one single parameter is like trying to measure a piece of music ( Zeppelin, Chopin ) by measuring its volume only. Easy enough to do, but what does it mean ?

If the controversy that has unfolded in this blog is an indication of anything, we need to start with what we measure and how we measure it ?

It’s interesting to note too that the Daylighting folk are working on new metrics to replace the well worn yet fundamental cornerstone of daylighting - the daylight factor. Do we need to undertake a similar exercise with new metrics for building performance ?

And while we are thinking about new and more meaningful metrics we should look at a bigger picture - for example why are building owners and building tenants not that concerned by the energy used per unit area of their building ? ( this seems to be a reoccurring theme to many of the previous comments, including examples elsewhere of the LEED building that was , to everyone’s surprise, " hemorrhaging energy" )

The answer may well be that building owners or building tenants have other performance metrics in mind when they evaluate a building - perhaps they think more in terms of productivity of the building occupants ? If the people in the building are profitable and productive who cares if they are waste more energy ? For more thinking along these lines you may want to read up on the work done by Adrian Leaman and Bill Bordass ( not that they advocate the wasting of energy, but that they consider the economic performance and occupant usage of buildings as well ).

So, perhaps high performance buildings have nothing to do with energy used per unit area - they should be measured by how well the occupants can perform a given task ? A suitable metric could be :

kW / occupant / average $ profit generated by the company

Of course one could argue that buildings don’t have any effect on the productivity and profitability of the occupants ?

Another view, when the bigger picture is of concern, could be these metrics :-

energy used for transport of occupants & for building operation / occupant

grams carbon generated by transport of occupants & by building operation / occupant

Why should we not introduce the amount of carbon generated ( or energy used) by transport systems used to get the building occupants to and from home ? This approach may reward in the future the densified cities with efficient public transport and good telecommunications for tele-working, and make a REAL difference to the amount of carbon in the atmosphere ?

Who knows, maybe in a 100 years LEED will be more known for its work on LEED accredited neighborhoods and cities than for the work they did on individual buildings ?

Lastly, the scary thing about all this uncertainty about building performance and its measurement is that someone somewhere is probably trading carbon credits based on published building energy savings …..

I have continued to enjoy the thoughtful and perceptive comments in this discussion. I agree with Rob Watson that every building is a unique engineered object. As we know, the unique engineered object then gets used by a specific set of people--a population of owners, managers and users that changes over time. So we have a complex social technical system that has a unique trajectory in time that will affect LEED EB evaluations.

A number of partiipants have discussed the data comparisons that kicked off this topic and we've reviewed the meaning of averages and medians, the shapes of batches of data, hidden factors that may confuse comparisons, etc.--important statistical ideas.

In addition to the statistical ideas we've reviewed, there's a whole set of statistical methods and perspectives that can help designers, engineers and building owners learn about their new, renovated or existing buildings. Techniques for experimental design (from simple one factor tests to more sophisticated multi-factor experiments) , grounded in explicit development of meaningful measurements, help people learn more rapidly and more effectively than passive observation.

When owners are asked to live in a building for a year to understand how to operate it (as I overheard engineers telling an owner's representative last week in a walk-through of a new building's energy control system), the owners with staff or consultants who can run simple tests and study relevant performance variables will be ahead of their peers. The designers can learn from explicit experiments, too, so their next designs are even better.

There are many ways to learn more about experimental design techniques, you can get a taste at the NIST website, http://www.itl.nist.gov/div898/handbook/pri/pri.htm

best regards

Kevin Little

How about these simpler metrics for building energy efficiency based on the architectural elements and the climate:
- Joules* per sq ft per heating degree day for heating for the year
- Joules per sq ft per cooling degree day for cooling for the year
- Joules per sq ft per year for lighting
Full credit would be included for design elements that reduce the energy loads including all the normal things like insulation but also including advantage from passive solar. Various software packages can give these values for a design. After built and occupied, these values can come from measurements. Comparisons would be straightforward for before and after and among buildings within similar regions of the country. Energy efficient buildings will have lower values for the above measures. Water heating efficiency measures could be treated separately because in most cases water heating is decoupled from the building envelope performance per se.

The building’s mechanical systems could be rated according to the number of Joules consumed to meet the load above, that is, an inverse efficiency measure.

Comments?

*or BTUs or kwhrs or pick you favorite energy unit.

Currently I am employed by a large MEP design consulting firm and exclusively working in building commissiong. After working in the HVAC industry for the past 19 years I have spent most of my time on the project sites performing many functions such as controls testing, test and balance, HVAC intallation observations, etc. in new and existing buildings. I have seen, readjusted, fixed and assessed many existing buildings which quite frankly were performing as dogs. I am not trying to say I am an expert, just trying to show where my background is.

Recently the LEED process has become popular and I see some terrible mistakes made by the Owners and A/E designers (not our firm of course) in pursuing a LEED certification. I have heard the LEED rating system 'called a design guide', watched the owners purchase/chase points by using systems with no payback such as PV, seen watered down versions of the commissioning process which netted little or no value to the owner, seen large administration fees paid to 'sustainability consultants' who did little more that fill out the LEED templates, promise and sign contracts saying we are going to deliver a LEED 'Silver' building, watched our local city, state, and federal governments mandate new construction be built to LEED standards and I could go on and on with examples of work our industry professionals performed that did not hit the mark. LEED is not a level playing field -the success of the program is highly dependent on where it is and who is designing the project. What is to keep these new 'high performing' buildings from becoming a performance dog?

I have been telling owners for years that your designers need to focus on the energy use systems in the projects and establish the energy efficiency goals, water reduction goals, lighting use and reduction goals. Once the design criteria is established (in LEED language this is the OPR) the design team can focus and more importantly remain focused on this criteria. Once these goals are established and we have established an initial design it is time to perform an inital evalutation and develop the LEED points matrix. Since LEED is a measurement tool - not a design guide - we can get an idea of how 'good' we designed the building.

This approach helps to focus on the energy efficient design and avoid saying we are going to deliver the owner a LEED Silver or Gold building. In the event the team does not deliver the LEED 'silver' as promised the goals of efficiency and the resulting paybacks are in place. After all not all projects benefit from bicycle racks. Please don't misread me - I believe we have to change the way we design and construct buildings for all the right reasons. The USGBC's goal through the LEED process is to transform the market and I think we can safely say they have done so. Good feedback and information on this site.

This is a very interesting thread, though one that has a bit of an "angels dancing on the head of a pin" quality to it. The fact is that every building is a unique engineered object and "all things being equal" never are and never will be. Having been involved with dozens of LEED buildings, I can say unequivocally that they are better buildings from an energy perspective, not to mention all of the other green elements included in the system. By focusing narrowly on operational energy consumption we mistakenly assume that it's the only way buildings consume resources, but LEED buildings are also location efficient, water efficient and, perhaps, more material resource efficient (now there's something we REALLY know nothing about.) than average and healthier (if a bit noisier) indoors. The latter (people) is where the big bucks are, completely swamping the piddling amount of money going to operations.

LEED is a market transformation tool and that job is beginning to be realized. We have peoples' attention and now that it has some meaning we have the ability to influence things for the better. No one can argue that LEED has stood still WRT performance, continually improving energy efficiency requirements as the science and markets develop.

For example, the vast bulk of the buildings analyzed by NBI went through LEED using versions 2.0 & 2.1, which only focused on "regulated" energy an no "process" energy and also did not include the Appendix G modeling rules. Building "process" energy is not understood at all and represented a huge (and now closed) loophole in early LEED buildings Pre Appendix G ASHRAE was never intended for the way it was used and therein lies much of the problem we're seeing with predicted vs. actual performance so early in the game. Earlier modeling was much less precise than what we can do now, and the fact that Version 2.2 requires ALL energy to be evaluated as opposed to "regulated" energy closed a huge loophole.

Now that LEED has some traction, the increasing stringency that is working into the system will drive real improvements that may or may not have been present early on--three cheers for the call to disaggregate energy consumption analysis (maybe it will actually cost enough to make it worthwhile--mostly a problem with 18th Century ego-nomics, not LEED). Again, my experience with early LEED projects indicates that better equipment and lower power densities were installed in these buildings; that these buildings would be operated less efficiently than their non-LEED counterparts defies logic. Dispositive? No, but certainly no less a valid conclusion than that drawn by Mr. Gifford, damned lies and statistics aside.

No one doubts that we could make the LEED engine go faster by decoupling it from the market train. Then we'd be like Green Building Challenge: a great system with a few good buildings that NO ONE has ever heard of or cares about. The following quote I heard attributed to Gandhi, but have not verified it, sums it up: A leader who is 100 paces ahead of his followers is revered and called a visionary; one who is 1000 paces ahead is stoned and called a heretic.

My guess is that we will see the center of gravity in LEED shift toward LEED EB, which does required measured energy use & that NC will continue to be an entryway into the system, but won't even necessarily be the main gate over time.

Mr. Gifford does a valuable service when he points out areas of improvement and new directions that are needed in LEED and buildings in general. And, indeed, many of the issues he has identified are in the process of being rectified and implemented.

Unfortunately, he does nothing useful when he makes blanket statements about LEED being "harmful to the environment [and] to the prosperity of our country." Reading things like this, one can't help but think that he just wants some attention (and maybe a hug).

Sustainability by subtraction.

For any resource, consumption precludes conservation, and conservation precludes consumption. Oddly enough, “green” seems coupled to consumption rather than conservation. It is coupled by the notion that consuming one resource allows us to conserve another, in theory. Sort of like—we had to burn the oil to save it. We need to get back to conservation. Here are a few of my bumperstickers:
Put it back, it’s not yours.
Consumption. The Fun way to conserve.
Consumption is theft. (Proudhon)
Consumption is conservation. (Orwell)
All we’re sustaining is more consumption.
Maybe next planet.

As I try to follow the bailout discussions, what strikes me is that the gov’t could wind up with a big equity stake in the building stock in the US. So say we commandeer those discussions and make some lemonade. OK, it’s a stretch, but... What would a sustainable neighborhood/town/city look like? It would be consolidated, with shorter utility service lines and transportation spokes, higher density, shorter vacancy terms, etc. Would it be worth taking an exurb development and turning it back into farmland? (It’s frightening to think how little salvage value there is in new housing construction.) Gov’t must balance precarity of its citizens against scarcity of resources. At present citizens are left precarious as resources are squandered. Oh well. If the emphasis was put on resource scarcity over citizen precarity, then a gov’t could imagine selective removal of the most wasteful property in order to get consumption down. Sustainability by subtraction. This may be really dumb—destroying property as the likelihood of economic and climate refugees increases—but at least it gets us out of thinking that consumption is the key to sustainability.

This is an important discussion. If the critical measurement right now is greenhouse gases per person, as the California study Reuben Deumling cited underlines (& Marc Rosenbaum concurs), then almost all our data sources are too coarse. As we evaluate alternate interventions, we also need to impose a time period (set with the advice of climate scientists) within which all the direct and indirect impacts have to break even and move into positive territory vs the status quo. This evaluation has to include transportation, and what is called the "embodied energy", and I prefer to call the "energy history", of the intervention.

This kind of analysis is missed almost completely by LEED in its present state, and not to my knowledge adequately addressed by Life Cycle Assessment tools like Sima-Pro, Envest or Athena .

Imposing a relatively short "payback" period will greatly favor the kind of limited fixes on existing buildings that Henry Gifford knows so much about. I would love to be better informed, but the greenhouse gas break even periods I have seen for new green construction, either greenfield or demo and rebuild, goes out at least 15 years and often 25 years or more, which may be too late. Our scary footprint, in North America, is in our existing buildings.

Creative thinking about how to green those buildings and fill them with overlapping, symbiotic uses, reinforcing existing infrastructure (transit, sewers, etc) will increase the program/people use per square foot and decrease the greenhouse gas/person. If someone can figure out how to avoid a building project by housing a use in an existing building, then they should get a platinum rating for that decision.

Greenwashing & Energy Efficiency Washing

Greenwashing - a term used to describe the perception of consumers that they are being misled by a company regarding the environmental practices of the company or the environmental benefits of a product or service. It is a deceptive use of green PR or green marketing.
The term is generally used when significantly more money or time has been spent advertising being green (that is, operating with consideration for the environment), rather than spending resources on environmentally sound practices.

A new definition for you.

Energy efficiency washing - a term used to describe the perception of consumers that they are being misled by a company regarding the energy efficiency benefits of a product or service. It is a deceptive use of energy efficiency PR or energy efficient marketing.
The term is generally used when significantly more money or time has been spent advertising being energy efficient, rather than spending resources on energy efficiency practices.

From the USGBC; LEED Leadership in Energy and Environmental Design is a third-party certification program and the nationally accepted benchmark for the design, construction and operation of high performance green buildings.

Note: high performance and the median vs. mean data (at top) it would appear that the USGBC misleads it's consumers into thinking that they are getting an energy efficient product or service, energy efficiency washing. Then again if the USGBC (U.S. Green Building Council) defines itself by the LEED rating system then they have also been practicing greenwashing.

The CBECS data are available for download and it didn't take long to calculate the median EUI. For the overall CBECS sample, the (sample and size weighted) mean EUI is 93 kBtu/sqft but the median EUI is 77 kBtu/sqft. So, should we conclude that CBECS buildings are 17% more efficient than CBECS buildings? No, that's not a typo. This finding should not surprise anyone familiar with statistics -- in positively skewed distributions, the mean is larger than the median. If we look at just CBECS office buildings, the mean is also 93 and the median is 78. If we look at office buildings built since 2000, the mean is 93 and the median is 73. It is pretty clear that it is biased to compare the median from the LEED sample to the weighted mean from the CBECS sample.

Before jumping to conclusions based on these median numbers from CBECS, people should realize that such comparisons are still lacking (although much better than biased median vs mean). A brief review of the CBECS data shows that EUI varies considerably and that these variations appear to be related to many factors such as building type/use, building size, climate, vintage, etc. Without matching on these other factors, we can't tell whether differences are due to LEED design or just differences in other factors related to energy usage. It is not clear that the CBECS data can really provide an appropriate comparison group for the analysis and any attempt would certainly have to be much more detailed than what's been done thus far. I don't think we have the evidence to draw any strong conclusions LEED building performance except than that it doesn't look remarkably different from typical commercial buildings.

You really need to use a consistent technique. I understand that LEED wants to sell itself, but you risk sacrificing your credibility.

If CBECS uses a square foot weighted mean EUI, then LEED should do the same in order to compare LEED performance to CBECS performance. Comparing the median LEED EUI with the mean CBECS EUI (sq.ft. weighted ) is NOT a valid comparison. We can make excuses, but it is just not a meaningful or intelligent comparison.

The energy services industry is really plagued with this kind of false advertising, like many other industries. The question is what role will LEED fulfill? I challenge LEED to be more honest, transparent and credible...

I think the fundimental reason LEEDS buildings are not performing as expected is that the theroredical baseline is to low. ASHRAE 90.1 minimum criteria can result in all glass buildings. An energy efficient green house is an oximoron unless you are growing food. LEEDS building have been built with all glass facades and therefore are fundimentally not energy efficient or sustainable. Therefore all parties should be working to bring the baseline up to a level that there is a potential to suceed and make the minimum standard equal to the LEEDS Extisting Buildings prereq of EnergyStar 69 or 75 so that there is a potential for sucess.

As a solution I suggest all LEEDS buildings should be required to meet LEEDS EB when they are 90% occupies or risk lossing the designation. This will stop Arichitecs and Engineers fudging the numbers to statify thier clients that want there cake and eat it. This will take all the failures off the list and give everyone somthing to learn from. Lets make performance the criteria for sucess and learn from those who do not perfrom.

I have looked at ASHRAE's new 30 plus criteria for small buildings. They seem to have missed the boat agian. IF ASHRAE is going to led they should be held to a standard of performance or some other criteria must be set.

I think we are all missing the point. Even if you believe every word of the NBI study, it is a pretty poor showing for buildings that are supposed to be at the top of energy conservation.
LEED 2009 should help this, as well as the proposed ASHRAE standards which call for a minimum 30% savings.
Forgetting the quality of the study, we need to find a way to track PERFORMANCE of buildings which I see as the real value of Gifford's article.

I have just begun a research program to document and analyze the energy use of buildings that have been rated green (LEED or other green rating program) to determine whether they are operating according to the standards to which they were built. This is a research project through the Green Building Program in the College of Natural Resources Conservation where I am currently a faculty member. We are now developing tools and procedures for conducting energy audits that we will use to analyze the energy consumption of buildings that have been certified two to five years after they were built.

This will tie into further research into new buildings that have been rated "LEED certifiable" as part of the Boston Green Building Code, to determine whether the green building legislation is producing better buildings (more efficient, more durable, healthier) or if new metrics for building codes are required.

This discussion has brought forward the very questions that we are asking about whether so-called high performance buildings are indeed high performing.

In my UK experience most buildings fall short, and in my recent sabbatical here I have seen little to tell me that the US is much different. I have written about this in things like The Credibility Gap - downloadable from the publications section of http://www.usablebuildings.co.uk

The UK Building Research Establishment found similar things for housing.
http://www.eeph.org.uk/uploads/documents/partnership/Building%20Regs%20C...

I think it is not so much the Code that has to be ratcheted up, but practice to be ratcheted up so that buildings perform more as intended. This needs better attention to details that often overlooked - particularly regarding build quality, robustness, usability, manageability and control systems that actually work properly.

The ENPER-TEBUC EU research project into building energy regulations in Europe found that in some countries, as regulations moved energy efficiency forward actual performance was moving in the opposite direction as the industry couldn't cope.

The problem is that - as a rule - neither designers, builders, nor regulators know enough about how buildings actually work. So if you are not careful, what is intended to be a code improvement actually makes things more difficult and so more likely to be done worse.

So what we really need to be regulated are:
- Clarity of design intent.
- Energy predictions that count everything, make the assumptions clear.
- Clear communication of the components of energy performance, with transparency between expectations and outcomes.
- Management of design expectations through specification, construction and commissioning and on into use.
- Appropriate metering and documentation.
- Involvement with operators, during design, commissioning and after handover.
- Routine in-use performance evaluation and feedback as part of the standard process.
We are developing the Soft Landings procedure to assist with this. Info also available on http://www.usablebuildings.co.uk ... click the Soft Landings Graphic on the left hand side.

Keep it simple, do it well, follow it through and tune it up. Only when you know how to do that, start to be clever.

With good wishes

Bill

Bill Bordass, the Usable Buildings Trust, London, England

Hi

Its good to see so many pationate responses but I think you need to put this all in perspective.

LEED does not equal a sustainable and more efficient building.
It is only a tool that can help put a focus on a number of areas that would otherwise not be addressed in a normal project.

What it does do is focus everyone including those controlling the $ (who like to make cuts wherever possible) to make sure a lot of important aspects are maintained. If good guidance is provided in combination with a LEED rating there is great potential. If a LEED rating is only pursued to chase points and sign off a certificate then it has no great outcome in reality.

A LEED rating does not mean a sustainble and effcient building in operation. It is an evaluationof the building design and construction only.

LEED and any energy modelling is a compartitive analysis only. It only measures the relative performance under a theoretical set of parameters. So generally speaking a building that scores a higher LEED energy score should have the potential to score better than one with a lower score (not always but generally).

Of cousre in operation there is a lot more to do to commission and operate the building effectively and also to have it optimised to suit the actual part load operation.

So if you want to set a good target set a LEED rating that aims to use only the parts of LEED that makes sense and adds other aspects not covered by LEED that can help improve the project. Then set real energy targets to achieve in operation including a 6-12 month fine tuning period before monitoring begins to allow the buidling to become occupied and tested under real operating conditions and for defects to be resolved.

We regulary complete quite in depth energy modelling and monitor the buildings in operation. What we find is that the modelling is about 20% better than predicted for the intiial phase of the project which then through a course of fine tunnig for 12 months can be brough down to less than 10% above the prediction (which is based on ideal operating parameters)

All of this takes a huge amount of effort but is worthwhile in the long run.

Regards
Julian

With spec office buildings at least:
LEED-Core & Shell is good as a first step
LEED-CI if added, is a bonus second step
LEED-EB/O&M with measured results is the best (but in this economy may not happen for years - many office buildings are currently sitting empty without tenants)

I hope that as public pressure, municipal pressure (through codes) and pressure from tenants increases (as we've been seeing in the marketplace - the LEED buildings now get leased before non-LEED)... more and more building Owners will be pushed toward LEED-O&M (with actual energy savings for the tenants) with their new and old buildings.

This is about educating everyone to increase our expectations - and not settling or getting too comfortable with current standards. LEED has to evolve and get better with future versions (as it is intended) - and AIA's 2030 Challenge for Carbon Neutrality is a huge step, as long as the metrics are understood and performance can be evaluated to ensure buildings are reaching those goals.

Great work to everyone who is pushing forward.

LEED stands for Leadership in Energy and Environmental Design
Where is the Leadership in Energy part
Leadership - NO - by not addressing this problem / issue head on
Energy - NO
Environmental - Remains to be seen
Design - a lot of people design stuff - myself included
To sum things up a possible 2 out of three is that certified or silver rating.
and a name change to Liars in Energy and Environmental Design.
The DOE has the appropriate programs in place to rate the houses and buildings.
Stick with the DOE and Energy Star for they are the only ones showing and Leadership and verifiable data.
Thanks,

It is quite possible to demonstrate that all the LEED programs are not performance standards for Energy Efficiency or Energy Conservation, without having a degree in statistical analysis. One has only to look at their checklists and you will see that the sum of the points that have nothing (or only peripheral relationships) to operational energy add up to more than enough points to get a Platinum LEED rating. Operational energy is the vast majority (in the range of 2/3rds to 3/4s) of the lifecycle energy and environmental impacts created by buildings, as is readily learned by even a short period of using software like BEES for doing lifecycle analysis. I assert that as long as it is possible to achieve the highest rating possible without doing more than the prerequisite bare minimum Energy Star level of energy performance (which is only about 30% better than illegal), the various levels of LEED certification do not tell you much of anything about the relative energy or environmental performance of buildings.

A band-aid approach to resolving this problem would be to actually require increasing levels of operational energy performance for each level of "higher" LEED certification, or to take operational energy out of the points system entirely and require a certain operational energy (Energy & Atmosphere Category in LEED) performance level for each LEED certification level, plus a certain amount of points coming from the other categories at the designer's discretion.

But Henry and the others above are right in the assertion that the incremental improvement/ checklist approach to sustainable design is flawed in its essence, as it doesn't encourage the kind of climate and site specific integrated systems thought process that ensures that the whole performs better than the sum of the parts, nor does it take us to the highest performance levels as fast as possible. There are alternatives to LEED that do address these core flaws by actually being quantifiable highest energy conservation performance standards with rigorous design and construction processes that encourage systems thinking founded in building science and use performance metrics from existing buildings to continually refine the performance standard. The pre-eminient example being the Passive House Standard, which is how I met Henry - we were both part of the first class studying to become Certified Passive House Consultants in the United States (Marc Rosenbaum was another), and both succeeded, though nearly half of our class dropped out of this intensive training program.

The Passive House Standard was developed over a decade ago in Germany, www.passivehouse.com, and has continually been refined since with the performance studies of completed buildings. There are over 15,000 projects certified to this totally voluntary building energy efficiency standard worldwide now, and growing exponentially. PHIUS is the US administrator of the P.H.Standard, www.passivehouse.us.

The building energy consumption quotas propagated by the P.H.Standard are based on life-cycle economics & building science, not an arbitrary concept of how to "move the marketplace in the right direction". At 15 kwh/sq. meter annual energy consumption for space heating and cooling, the required amount of space conditioning energy has dropped so low that it can be delivered by the minimum required ventilation air for the building's occupants, allowing for the down-sizing of much of the cost in the building's mechanical system, which can be used to offset much of the increased costs of the better building envelope that makes meeting the P.H. Std possible. The P.H.Std also penalizes excessive sf/occupant ratios, so it does create a metric that addresses the root issues of global climate change. And, though not required by the P.H.Std, practitioners typically take their conservation ethic into every aspect of building design for which there is a LEED category (and maybe some LEED hasn't thought of yet), innovating products that help designers to regularly save more land, water, pollution, embodied energy, and produce higher indoor environmental quality than almost any LEED building has here in the USA.

The P.H.Standard regulates building performance while leaving building systems and strategies used to achieve that performance at the discretion of the design team and building owner. It provides tools and training for replicating other's success without squashing innovation. It provides quality assurance checks throughout the design, construction, and commissioning process, essential for the protection of all parties involved. If by some miracle the designers & contractor managed to complete all the documentation required for certification (which is less than LEED and targeted on the essentials of the building's performance) yet still build something that didn't actually meet the P.H.Standard, the owner would know it in short order because their mechanical system would not be adequate to meet the space conditioning requirements for comfort, and their energy bills would provide strong supporting evidence for contractual breach in a court of law.

I want to be a building science practitioner who can replicate the highest energy efficiency performance in all my projects in the range needed for averting global climate change, which is about a 90% reduction in energy consumption of all sectors, and that is the range of energy savings that the Passive House Standard achieves in building space conditioning energy. And I think it only fair that owners of buildings be able to verify their building's performance as easily as they can check their car's mpg, and hold me accountable. I encourage LEED to evolve towards the real Leadership in Energy and Environmental Design that the Passive House Institute has exemplified, or to stop claiming that their program produces higher energy efficiency, when as currently written, it makes no such requirements for certification.

LEED as a checklist is deficient. What is perhaps most disturbing is a perception by many that LEED is a sustainability metric. We are however very near the beginning of a design paradigm shift. Old design thinking is being examined and found wanting. For instance, buildings that have entirely glass envelopes in the climate where I live (and I would argue any climate) are simply a bad design idea. Square buildings in our climate (or likely any climate) are a bad idea. Yet both these designs can receive any of the LEED certifications, at the same time that I would be observing them as failures.

I think that most owners are interested in sustainability in design as long as one can make the ROI argument. Energy efficiency (as in the case of LEED) can be a good distance removed from the goal of sustainability.

I have many local examples of buildings that were designed to low energy use standards, including full energy metering broken down by systems in the buildings, and two to three years after the buildings were built I asked for the energy use data to see how they were performing. I was told by the Facilities Maintenance folks “sorry, don’t have any data, we don’t have anyone who can spend the time to collect it” and other problems like “well those meters never worked from day one” – they blame poor commissioning, or a litany of other “reasons” why the meters were never maintained/calibrated/monitored.

Hence I as a mechanical engineer get very little feedback other than complaints – typically it’s too hot if the room temp is over 23C, and too cold if the room temp is below 22C, even though the Building Standards we were tasked to design to stipulated no mechanical cooling unless a process required it, and the design indoor temperature range was to be between 20C and 26C.

I’ve come to believe that most building owners spend more $$ to get their BMW maintained than they do on their Building Systems Maintenance. Granted there are some building owner/operators who do a pretty good job of running their buildings, but even some large institutional organizations I know basically operate their buildings in a crisis management mode, with no resources allocated to proper preventative maintenance or operational monitoring, other than Excel Spreadsheets of costs vs how many rolls of toilet paper the budget will allow.

Geoff McDonell, P.Eng., LEED AP
Senior Mechanical Engineer
OMICRON

I am intrigued by this study and somewhat frightened by the idea that its apparent conclusion - that LEED buildings are no more or only slightly more efficient than others - might deter folks from embracing LEED as a means to generally improve building performance. Certainly, we need to know that the mechanisms we adopt to theoretically improve our built environment are actually living up to our expectations but, it would be very easy to turn the massive effort to reform an industry on its ear, with the mantra that LEED building are no better that non-LEED buildings! The State of Ohio, Ohio School Facilities Commission has recently mandated LEED Silver Certification for all state funded school buildings. Practicing architects in Ohio are following that dictate - most with the resistance of a second grader at receiving his /her first two hour homework assignment.

Whether or not these practitioners have embraced the spirit of LEED or are chasing points, a lot of folks in the industry are asking good questions and earlier in the design process.

Not having studied the original data sheets referred to in this discussion, I haven't the right to comment on their veracity or interpretation. However, there are such vast differences in building performance based on location, design and occupant culture that I would be cautious in accepting the validity of any general data that lumped non-coextensive sets together. As I read the various thoughts on these pages over the short time this topic has been active, I have, perhaps inappropriately, wondered where Richard Feynman is when we need him to remind us that the average weight of the planets is not something that is necessary for young people to learn or know? I am not part of the culture of statisticians for whom the language used to describe the comparisons of these sets is comprehendible, but the absence of affirming language for me remains unsettling.

There are some questions that as a building design practitioner, I would like to know.

1. How similar in performance are LEED buildings by building type, HVAC system, climatic region, period of operation, and certification level? Operating dollars/SF would be a measure our clients, Superintendents and Treasurers of K - 12 schools, would appreciate.

2. How do LEED buildings stack up with those designed using the Green Globes process?

As our engineering teams confront the need to improve on the "standard" model (I really don't want to go there!), they are running headlong into their preconceptions and that is totally healthy! and expensive, I might add!

I must admit, that when I see buildings in the literature promoted as LEED Gold or Silver, I am occasionally taken aback by massive S/E/W facing unshaded curtain walls, willy-nilly orientation and semi-functional daylighting. Yet, I can see that such buildings when compared with even grosser designs could look sterling to an owner.

For the buildings that we presently have on the boards, several hundred thousand SF in fact, the limiting factor to performance is in my mind not technical but sociological. It is what we know to be true about our process, our clients, our code officials, and our building occupants which constrains us. When you don't trust teachers to open or close an operable window, that choice may be taken away. When a "construction factor" of 11% is proscribed as a reasonable upper limit, you don't contemplate requesting a variance for rammed earth or hay bale construction. If a team of 20 year veterans knows only VAV, then consideration of displacement ventilation as a legitimate alternative may fall on deaf ears.

When the acceptable standard for operating cost is $1.60+ /SF/YR, $1.40 looks like a vast improvement.

This discussion has focused really on the energy aspect of LEED vs. CBECS. This is understandable, since it is the most easily quantifiable (and marketable) aspect. Unfortunately, as I think has been adequately discussed, we really don't know yet how they compare. What we do know is that there hasn't yet been a good comparison done, though Bill Rose has started with what he has. Given that medians are simply values with a certain ranking in a distribution (the middle one), and averages are the numbers which, when multiplied by the sample size give the sum of the elements in the distribution, any similarity between them is purely coincidental (i.e. they don't have anything to do with one another). So is any particular difference coincidental.

But should we be worried about it if LEED buildings aren't any more energy efficient? I think that in some ways yes, in some ways no.

The answer can be "no" if the intent of the design was not to save building energy (the first E) but rather focus on the environmental aspect (the second E). There are certainly a lot of points to be gained that have nothing to do with the building's utility bills, so why should we expect buildings that pursue those points to have lower EUIs?

On the other hand, if the buildings were intended to use less energy, then it is a problem, and we would be wise to understand why the problem exists. There are a number of possibilities. One is that the design is not a finished building, and there are plenty of chances for even good building components to be put together in a way that compromises the energy benefits. Consider a building with a lot of leakage. That is like leaving some of the windows open. So how well does more insulation or a better conditioning system work if the windows are open? What about conditioning system airflow? If it is off then the efficiency of the system is likely to also be off.

Then there is the possibility that the problems are with the occupants, not the building. There could even be some takeback, where the occupants figure that since they have this better building they can afford to be more lax on other energy use (or have more windows).

These are just speculations of course, but both lead to the conclusion that a design doesn't say whether a building uses less energy; we only know if it there is verification. And that this verification needs to be done in such a way as to minimize the extent to which we are evaluating the occupants instead of the building.

Again, it really is not yet clear whether LEED buildings use less energy, or whether they even should be expected to given the current structure of the program. However, if we all agree that reduced energy use should be a primary goal, then I believe that the only way to assure this is to verify performance. So to Christopher Sweetnam-Holmes' question, "would we really want a LEED system that had much higher integrity, if only a few bothered to use it?", I say absolutely. A quality assurance program that does not assure quality is of dubious merit. It may allow more buildings to fit in, in the short term, but eventually people in buildings without improved performance will notice that all they bought was a warm fuzzy feeling. That will be bad publicity, LEED will become a joke, and that will be the end of it. Think I am overblowing it? Think about solar in the 1970s. Let's not let building energy efficiency go down that path.

My friend (A LEED Certified Architect) built a house (Platinum Certified) for himself up here in the Midwest. A pretty cool modern design, lots of glass (52%), steel beams, huge cantilever, and steel studs, concrete w/radiant heat floors through out. This house is located in the burbs so it doesn't fit in in very well - but it is neat none the less. Even considering the use of Open Cell Polyurethane foam at the cantilever, roof line and the exterior framed walls the question arises as to if this home would pass a RES Check (the answer is a resounding NO!). One of the problems with the home is the 52% glass and the steel beams are left exposed on both the inside and outside (i.e. no insulation- just the steel and a bit of paint) The village never asked for a RES Check because it was a LEED project assuming the Leadership in Energy part not Liars in Energy. Every time I see my friend I ask him how his frost is doing and he says "not too bad.". Frost on the inside of a home, in this day and age - what a joke. I pity the person that will buy this LEED home from him (when he decides to sell). Hopefully the USGBC will; get it's act together, admit their failures, and revamp their entire system or stick with what they are good at which is the Environmental E aspect leaving the Leadership and the Energy aspects to the professionals.

Disaggregate. It always provides enhanced understanding. And for a metric, yes, please, let's relate energy use to occupancy loading, in energy use per person-year. We can also report btu/sf-yr as an interesting way to show that it is not the only or best metric when we have data using both metrics on reasonably comparable and reasonably different occupancies of the same building type.

So, I ask:

Why, then, are we still relying so heavily on btu/sf-yr when we know it can be so misleading?

What do we have to do to instigate a change in the metrics used? Is LEED the place to start? Get designers thinking more about use during design?

First, we don't know plug loads well. Second, we aren't good predictors of how the buildings we design will be used. I now recommend separating plug circuits, lighting circuits, and HVAC circuits by panel so they can be separately metered and accumulated by the controls software. Every green building I have worked on with Kubala Washatko (in particular, the Schlitz Audubon Nature Center, the Urban Ecology Center and the Aldo Loepold Legacy Center - all SE2 award winners) had actual program use greater than predicted use (I would like to think this is a result of good design). While your observation that the Aldo Leopold Legacy Center has a small number of occupants is accurate, they actively rent the Seed Hall (no mechanical systems, only natural ventilation and a wood burning stove) from late spring thru early fall. The energy use by area estimates are based on conditioned area of the main building, not the additional of the Seed Hall. We did not model occupancy of the Seed Hall in our energy analysis.

The whole issue of heating with wood is fraught with environmental correctness. While I insisted on accounting the CO2 emissions from wood burning stoves in a carbon neutral balance, many of the staff at Aldo Leopold Foundation heat with wood and felt it was better than drawing hydrocarbons from the lithosphere. Many biologists would argue that carbon in biomass is already in the biomass/atmosphere/ocean cycle and what we must avoid is drawing ancient carbon biomass from the lithosphere and combusting it to the atmosphere. I wanted the wood pile to be 1 cord. The staff were thinking along the lines of 3 cords. We settled on 2 cords for the whole complex.

We focused on occupant controlled lighting rather than dimming controls at ALF. LEED, ASHRAE and ISNEA (I assume) do not recognize occupant controls while recognizing automatic controls. While the installed, non-dimming lighting at the ALF is 1.39 watts/sf, neither LEED nor any other agency recognizes that occupants may actually have operable brains and be capable of turning lights off when the sun is up. Furthermore, guests visiting buildings with lights off will have a different experience vs dimmed lights. I contend that experts cannot recognize dimmed lights while a young child can see that lights are off.

Understanding the energy balance is critical to assessing the overall performance of buildings. Dis-aggregation is critical to determining the actual building energy use.

But consider this, if low energy consumption per square foot is what we measure, are we blocking designs that stress multiple occupancy of workspaces? We should measure kBtu/occupant/yr not kBtu/sf/yr.

Allow me to throw another log on the fire (or the sustainable equivalent).

Three weeks ago I had the honor of judging Wisconsin’s SE2 (sustainability and energy efficiency) awards, put on by the Focus on Energy Program. This is the fifth year of the program, a combination engineering and architectural awards. One great thing about the program that I wish others would do... SE2 more or less requires submitting a year’s worth of actual energy use data, and this plays a huge role in winning projects. The smartest entrants explain the data, such as noting a change occurring due to the activation of a piece of process equipment.

Last year, we gave the top award to the wonderful Leopold Center, a true green LEED platinum almost-net-zero energy building with essentially 0 BTU/SF/YR. This year, a couple of projects challenged the jury. The top award (to be announced shortly) went to a project with annual energy use of over 150,000 BTU/SF /YR. One might ask, in this important efficiency program, WHAT GIVES?????

The answer I hope is clear... we use energy in buildings for more than thermal comfort and light. There isn’t a lot of data to guide the SE2 jury when assessing the difference between the relatively empty, low use hours Leopold building and a highly populated, long working hours building full of computers and process loads. We use and interpret CBECS as best we can, but we concluded that it is not possible to measure and disaggregate building energy use in manner allowing simple comparisons. In other words, turn the Leopold Center into a fine arts museum and retail store in downtown Chicago and the same building won’t be anywhere near net zero ­ but it will still be a great building.

I feel that we’ve done a good job of sorting out the greenwash (of which some project entries are totally made) and appropriately rewarding projects. But another of the contestants this year designed a building as technically correct within a modest budget as possible and should have been our big winner ­ but their inability to explain a so-so 70,000 BTU/SF/YR use sunk their ship, so to speak. Had they disaggregated the data, they might have explained that the building was using about half of the energy and the rest was process. If credible, that effort would have moved them into contention for the top award, but without disaggregation, it seemed to be just another fair building. Reading from BuildingGreen, it seems that this issue may be a part of the wide data spread. From my own work for Southern California Edison, I measured office building plug loads from 1/2 watt per sf to over 5 w/sf ­ - sometimes in the same building but different tenants.

Am I missing something or is this a real contributor to the discussion?

James R Benya, PE, FIES, FIALD, LC
Principal
Benya Lighting Design
West Linn OR

This jumped out at me: "...Very few architects, mechanical engineers and energy simulators actually look at the measured performance of their projects after completed..." Since it is the building OWNERS who most benefit from post-occupancy verification, it stands to reason that THEY would pay for the required metering, calibration, post-commissioning, etc, etc. However, it is STILL the rare owner who understands both the short- and long-term benefits of M & V.

Architects and engineers are usually too far down the food chain to have much influence on whether and how their buildings' performance will be measured. I'd be curious if this snag is on the radar of the Architecture 2030 folks.

Julie E. Gabrielli, NCARB, LEED
Gabrielli Design Studio LLC
Baltimore, MD

This is just a wild speculation, but could it be that non-LEED buildings are also built to exceed ASHRAE 90-1? LEED is a minute sample of the built environment. Many design professionals (all?) know how to and want to make good buildings and code is simply the worst building that is legal to build. Just because a building isn't certified, doesn't mean it's not worthy of certification. Perhaps most buildings do perform better than code. Perhaps LEED has dragged the vast majority of buildings along toward higher performance. Perhaps LEED has transformed the market. Maybe Gifford's findings verify this heartening assertion.

Of course we don't know... better performance monitoring is needed. It's time to know how our buildings perform and to learn from the feedback.

Thanks to all for a stimulating conversation.

Two follow-up items to the analysis presented above.

Cathy Turner of NBI and Brendan Owens of USGBC have responded positively to my request for data. Such willingness is a real credit to both organizations.

In the second analysis I showed the consequences of using the data selectively rather than in total. That was not meant to be the final word on this approach. I hope it is seen as opening the door rather than closing the door to further analysis. “Massaging” the data may sound suspect, but it is appropriate where the data sets have major differences. I would not be surprised to see someone review the data, make explicitly-stated balanced adjustments, provide justifications, and conclude that LEED buildings are x% better in energy intensity than average (CBECS) buildings.

There's been some excellent discussion about this post on the Society for Building Science Educators (SBSE) email list. Parts of that exchange also appear as comments here. Until we can get permission from the authors to aggregate all of those posts in this location, take a look at that list's archive:
https://www.lists.uidaho.edu/pipermail/sbse/2008-September/

Actually, take a look at that list's archive in any event.

In August 1980, the Government Accounting Office conducted a review of the federally funded solar programs that were completed and underway, and found that they failed to deliver the expected, and promised, benefits. Many of us are old enough to know that federal funding for building energy research dried up for the next 20 years, prompted in large part by the GAO finding. So this issue — does LEED deliver the expected benefits? — is one of critical importance to all of us.

I’ve reviewed the data the best I can in a short period of time. Yesterday I requested the USGBC energy intensity data from Brendan Owen and have not received a response yet. I spoke yesterday with Mark Frankel as well. He was very cooperative, and offered some important information. So at this moment I do not have the NBI data, and not through any denial on their part. In order to move quickly, I am presenting data that I reconstituted from NBI presentation graphs (+/- 1 kBtu/sf/yr approximation), data which fits exactly to NBI presentation of quartiles for high and medium energy type buildings from their dataset. I’ll call this dataset NBI1. What is notably lacking in my reconstituted data set is an assignment of data type to each individual building.

CBECS provides Public Use data that lists all of the survey responses for all of their respondents. The 2003 CBECS survey found 395 respondents for buildings with a construction date of 2000-2003, and for which energy consumption data is available. Let me call this dataset CBECS1. CBECS, in order to arrive at national estimates of energy use, applies a different weighting factor to each of the respondents, depending on the geographic population each one represents. This weighting factor has the curious effect, for 2000-2003 buildings, of dropping a raw average energy intensity (kBtu/sf/yr) of 97.8 to an adjusted (national) energy intensity of 79. This adjustment arises because, apparently, and for some reason, low energy intensity buildings in their sample represent greater populations of buildings than do high energy intensity buildings. In my analysis, I used non-adjusted energy intensity straight from the respondents to CBECS.

Chart:
http://www.buildinggreen.com/live/images//comparison1.png

Suppose we compare NBI1 to CBECS1 and plot them datapoint-for-datapoint, ordered by increasing EUI. What we note is that the two distributions have essentially the same shape. A t-test indicates that the two samples are statistically indistinguishable from one another. Having both data sets allows us to calculate the mean, medians and quartiles of both data sets. Results are shown in the following table. The medians are practically identical. The means are somewhat different, with NBI showing higher energy intensity.

Table:
http://www.buildinggreen.com/live/images//comparison2.png

The aim here must be to perform the fairest possible comparison. An argument can be made that the comparison above is fair. CBECS uses a statistically valid sampling technique in order to extrapolate to the entire population of US commercial buildings. The NBI dataset is a sample of convenience, but one that represents a high proportion of the entire population of LEED-certified buildings.

The argument may also be made that the two datasets can be made more fair by excluding any building types that are found in only one dataset and not in the other. Exactly which building types to include and which to exclude, given imprecise definitions, unknown definitions and imprecise groupings, can be argued. Nevertheless, I have conducted a second analysis which excludes vacant buildings (n=5) and non-refrigerated warehouses (n=56) from the CBECS data (call it CBECS2) and which seeks to exclude data centers from NBI data (call it NBI2). I say “seeks to exclude” because I do not have the building type data for NBI available to me. So I excluded 4 of the 6 datapoints for that type (EUI = 78, 507, 523.5, 555); these are datapoints that seem to belong to the data center type, as seen in the quartile information provided for that type in the NBI report. “Data centers” is a category in CBECS but there are no instances of data centers in the dataset.

The results of this second analysis are shown in the following chart:
http://www.buildinggreen.com/live/images//comparison3.png

The means, medians and quartiles are shown in the following table:
http://www.buildinggreen.com/live/images//comparison4.png

In this case, by dropping non-matching data types, NBI appears to have lower energy intensity than CBECS. Again, by t-test, the two datasets are similar, but closer to significance than the previous comparison.

These two data sets, CBECS and NBI, are valuable. It is important to make the best use possible of both, as they have come at considerable expense to the funding agencies. They should not be discounted or ignored simply because a comparison of them presents challenges. In this analysis I have sought to create two comparisons, both of which have arguable merit. They bracket the question of energy intensity comparison. In the first case the two sets resemble one another visually, match in quartiles and medians, and show an 8 EUI energy intensity edge in mean value to CBECS over NBI. In the second case, NBI appears to have overall better performance graphically and using quartiles, and has a 16 EUI energy intensity edge in the mean value. Both comparisons show, by t-test, similar sample populations.

A few discussion points:
The ranked distribution, as shown in the two graphs, is non-linear, showing increasing slope at higher intensities. With such distributions the mean is higher than the median. Compare, for example, the mean to the median in each of the four data sets. This should make it clear that a comparison of a median in one data set to a mean in another conveys very little information other than the shape of the ranked distribution curve. Gifford is correct, the NBI reliance on this comparison is flawed and misleading.

CBECS1 contains data from 395 buildings from dates 2000-2003. This is three times the number of buildings in NBI1. NBI has no basis to opt for 1990-2003 rather than 2000-2003 dataset. Gifford is correct in his criticism of this choice by NBI.

CBECS adjusts raw data in order to reach national values. Gifford’s claim that NBI data show a 30% energy penalty hinges on the CBECS adjustment. My analysis uses non-adjusted values, because the adjustment has no meaning in terms of analysis of the individual respondents in the sample. The penalty claim is correct if based on the CBECS adjustment. I have questions about the CBECS adjustment.

If certain data are to be excluded from the analysis, then a solid basis must be provided for such exclusion. The basis provided by NBI is not sufficient. I have sought to provide a fairer basis for exclusion in the second analysis, namely, exclusion of any data types that do not appear in the other data set. (I make no claim that this second analysis is exhaustive — it is not, especially in that I was unable to exclude 2 of the four data center values, and my NBI values are “reconstituted”.) NBI excluded data from their comparison while retaining the same building type in CBECS — laboratories, for example. Such exclusions cannot be supported. Again, Gifford’s criticism of NBI is warranted.

My tentative, provisional conclusion is that the CBECS dataset and NBI dataset are similar. That means that LEED buildings perform about like average buildings, but adjustments in the analysis by building type tend toward small improvements in LEED performance.

I would recommend to NBI, for followup studies, that they seek to replicate the conditions of study developed by CBECS. This will improve the ability to conduct fair comparisons. The CBECS survey form is available at their website.

ASHRAE 90.1 should pay heed to these findings. That standard is used as the basis for LEED and other programs. A failure of these programs to show significant improvement over average buildings reflects on the standard.

Finding that LEED buildings perform about like average buildings should not be cause for dismay. The industry is slow to take steps that will lead to actual improvement. My opinion is that any and all programs for building energy efficiency must address airtightness. We may have reached the limit of energy efficiency in buildings where airtightness is ignored.

Finally, I’d like to thank USDoE (EIA, CBECS), USGBC, NBI, Henry Gifford and others who have put the focus on measured energy performance. I sincerely hope that this discussion leads to more and better measurement, analysis and distribution of findings. And thanks to Paul Francisco for conducting the t-tests.

Bill Rose
Building Research Council
University of Illinois at Urbana-Champaign

On the issue of taking down the plaque if the building doesn't perform well, isn't the deal these days supposed to be that you get a LEED-NC certification and then you must recertify under LEED-EB by year five. If you don't do that, isn't the building at that point no longer LEED certified?

Just two quick points from a residential guy:

1. we worked pretty hard to get the quality process into the LEED for Homes program. It's far from perfect, but it is a good start on the notion that quality is an integral part of high performance buildings, and every green home must be a high performance home.

2. I think that energy bill guarantees cut the mustard. Yes, they can be just marketing hype if the targets are really loose, and yes, guaranteeing just space conditioning instead of all loads is just solving part of the problem. But they can inextricably link quality and energy efficiency. I don't know how or if they could work on commercial buildings, but I would think we could figure that out.

And yes, it has always troubled me that LEED's name ends with Design. That is not how high performance and small environmental footprints work.

I have enjoyed the thoughtful discussion.

I think we can agree that we need more than one measure of energy use to get a useful picture of performance--at least on a building level (and more on subsystem levels where we can) I want to know: energy intensity; energy use, not normalized for size of building; and some measure of climate impact (e.g. tons carbon or CO2/year). Of course, there ought to be something about energy effectiveness, too, in terms of energy units per desired service provided (light, temperature and humidity levels, air quality...)

The Energy Star site www.energystar.gov in addition to providing the Energy Star rating for certain classes of buildings promotes a method for energy management of buildings that applies to LEED and non-LEED buildings alike. Performance data are neccessary but not sufficient to achieve good performance; deployment of an effective energy management system is more about people than hardware.

Data logging, aggregation, storage and access costs have dropped a lot in 10 years since the early days of LEED. If you don't need your meters connected to a building controls system, stand-alone, web-enabled data loggers that talk to pulse meters are now on the market for less than $1000, with no recurring storage or access charges for typical interval steps.

Mike Utzinger mentioned getting professional students to evaluate performance, supported by local professional societies. Good suggestion! Simple evaluations of energy use in buildings (and interventions to reduce wasted energy) can be done by high school students, with a bit of education and support.

Kevin Little, Ph.D. (statistics)
Informing Ecological Design, LLC

Instead of debating on how to establish a set of universal metrics of greenness in buildings, perhaps we should be reflecting on the fundamental impossibility of such a task, as this decades long conversation makes clear.

Then it is far easier to relegate tools like LEED to their appropriate role.

No question LEED can provide a good set of guidelines for architects and design teams to consider. Most architects would be even better served by a modest understanding of building science, mechanical engineering, building envelope design, construction, economics and other fundamentals that seem so obviously lacking in the way architects are generally trained here in the US.

No question LEED can be a great marketing tool. None of us can deny the success USGBC has had in that regard.

Concern comes in the inevitable mission creep and power grab that all successful endeavors seem to evolve. It is really amazing how USGBC has so quickly amassed such significant regulatory power over our built environment

Recently I was asked by a city planning office to respond to an RFP for the “energy consultant” component of a master plan for a 5 million square foot urban redevelopment effort. The RFP was very much focused on issues regarding LEED. Among other tasks the consultant will be charged with determining is “Should a specific standard (such as silver, gold, or bronze) be required for development?”. In declining the opportunity, I sent the city planner Henry’s article.

Here in Rhode Island, a new Green Building Council has recently formed with one of the first items on the agenda being legislative.

While I don’t question the potential value of LEED as a tool that architects and developers voluntarily utilize to help improve the design and marketability of their buildings, LEED is not appropriate as a government mandate.

Recognizing the impossibility of achieving the goals of this decades long conversation among many of the most brilliant and expert people in the building industry, is it really wise to simply turn the question over to marketers and regulators to carve arbitrary answers into law?

The important question really becomes how to restrain the abuse of a tool which might otherwise be useful.

Another area where there is a major gap in follow through is with the use of carbon dioxide sensors. While more projects are having them included, only a few seem to understand the important need to make sure that the sensors used are actually calibrated and performing as intended. I'm hearing horror stories, such as the one in Maine where distributed sensors were used in a new school - they were 700 ppm out of calibration, leading to the call for more air in January, wasted energy, and over-drying of the interior spaces. One option for avoiding these problems is use of the shared-sensor, centralized monitoring approach along with frequent review of the data to maximize its diagnostic value. All to often, it seems to be Spec and Forget.

That is interesting. But all new FEDERAL construction is supposed to be 30% more energy efficient than the ASHRAE 90.1-2004 baseline per the Energy Policy Act of 2005. So what is the subset of buildings that he is looking at?

(I'm not saying that ALL new federal construction performs as predicted because I don't know that anyone has studied that.)

When I was a child, my father bought a car for its energy efficiency just after the months of the 1973 oil crisis. My father, being my father, implicitly did not trust the rated miles per gallon advertised; so he kept a detailed chart clipped to the visor where he recorded gallons of gas purchased and city and highway mileage driven. He divided this out every month to see if his energy performance equaled what was advertised. He figured if it was off 10% or so – that was within a margin of reasonable user error. More than that would be either something terribly wrong with the engine or tires or simple consumer fraud.

The question here is whether the LEED label (Leadership in Energy and Environmental Design) is consumer fraud. I am no statistician but in order to assess whether this is true we have to fairly isolate some variables and comparisons without loosing sight of the bigger picture and goals.

The first requirement for a green building and the primary responsibility of the design team should be low kBTU/sf/year (we could also take it back to source energy or carbon). Experienced teams know most of the steps to get there but the LEED process of completing a detailed DOE2 model is not typically useful in this effort. (I once waited a full year after construction completion for a final $30,000 LEED energy model as the EAc1 points slipped from 8 to 4 and still ended up at 350 kBTU/sf/yr including massive plug loads – tell me what value that added besides 4 LEED points? 30% better than bad is still sometimes bad - and the owner knows it!)

In my opinion, using upfront energy models during design is more useful, not because they are totally accurate, but because they are educative on what variables matter for each unique project. A LEED requirement for including that upfront process during SD would demand that architects be responsible to deliver designs with low energy loads and not unfairly delegate that job to engineers to service high loads with oversized, overpriced, and overcomplicated equipment. That is not a responsible or affordable approach.

The baseline for any building energy model comparison cannot be ASHRAE 90.1 which is a system that optimizes components. And it is quite possible to put efficient components together in a terribly stupid and inefficient way. In addition, the LEED Energy Modeling Protocol does not allow many valid passive strategies on the basis that they rely on occupant intelligence (what!). And the baseline for comparison should not be the Energy Star dataset either. Although this has a basis in history and reality, there is nothing to suggest that this history is good especially with the wild card of user behavior -which could be bad. What we need is a database of what is good, better, and best design – for building type, climate, building size, separated by load types. That can only come from a massive study of the best that has been achieved real and/or theoretically assuming reasonably intelligent and participating occupants.

Until we have this, we should do what we can do: an upfront energy model, a final ASHRAE model if LEED is required, and an Energy Star Requirement for 2 years – all looking at kBTU/sf/yr and carbon and setting our own aggressive baselines if necessary.

Given a low energy design and checking the engine and tires, the owner and occupants (with design team help) are responsible for the kBTU/occupant/year metric – which should be the second requirement for a green building and a component of a personal footprint. This will exempt the hedge fund millionaire from getting to call his 10,000 sf second home ‘sustainable’. But we must be sensitive here that different populations (excluding hedge fund millionaires) have legitimately variable thermal, lighting, and ventilation comfort zones which might be widened or narrowed. We can call this additional kBTU/occupant standard LEED&U – Leadership in Energy & Environmental Design and Usage.

Before LEED, I found energy research and green architectural design to be quite rewarding work. I started LEED consulting because I honestly thought the system would help me push and educate others to do more. After 6 years and best efforts at establishing goals before counting points, I can tell you that I believe the rating system has degenerated the process into a humiliating exercise of chasing irrelevant credits. It is not true, nor has it ever been true, that a meaningful goal can be motivated or reached by anything other than a meaningful intent – loopholes will be found. But it is unfortunately true that a meaningful intent can be sabotaged by a substitute reward – like a plaque. I once took 3 days to trace and calculate every stitch of recycled content in a desperate attempt to achieve points for Silver. I have witnessed bicycle racks replace high performance glazing as if the choices were equal. When I asked a colleague a few years ago, how she achieved EQc4.4 for UF-free composite woods when there were fire-rated composite wood doors – she simply said that she omitted those cutsheets from the LEED documentation. In other words, she lied.

I don’t know if all aspects of the LEED label are meaningless, but the process sure seems like it and it is certainly not providing value proportional to the effort. We all have a professional responsibility to address it and try to correct it before we are guilty as accomplices of possible consumer fraud.

Michele Helou, Associate AIA, LEED AP

Interesting discussion. My take on LEED for new construction is that if an owner chooses to pursue the energy points, they will probably end up with a building that is capable of performing better than most. The actual energy consumption depends primarily on the operation. The fractured and watered down LEED commissioning requirements are no guarantee that the building design intent will be carried through or that the staff will be properly trained and have good documentation for the systems they are tasked to run, assuming that they even have an in-house staff. I agree with an earlier comment that those buildings who have at least tracked their energy use are more likely to perform better.

Both the article and the pdf upon which it is based are thought provoking. I do wish, however, that when we get into arcana, we take the time to keep thoughts straight. There is absolutely no connection in logic between energy efficiency and energy consumption. I grew up in a very inefficient house, but we didn't consume much energy. I now work in an energy efficient building, but we consume massive quantities of energy. Which building is greener?

Efficiency simply allows one to use lots of energy in an efficient manner. Low energy consumption is environmentally friendly. When low consumption and efficiency intersect, we're better off as a species, when we have bright people conflating concepts and forgetting to straighten out their epistemological underpinnings, we get glimpses of ideas, but nothing coherent. If LEED is to work, the goal is less consumption of fossil fuel energy. If we want to keep windows open while we bustle about all buttoned up expending energy to keep warm - that's good too. Remember, efficiency is nothing but an remove of non-essential energy expenditure. For example: people are always looking for the most effecient method of executing people "humanely." That doesn't make death any less offensive, nor does it make the process humane.

Once energy is expensive enough, Owners will use their LEED buildings more wisely. Until then, fools will be fools, regardless of the plaque on the wall.

I think what is missing from this discussion is the purpose of LEED. Instead we have people drawing conclusions based on studies and examinations that do not have enough reliable data to make detailed conclusions that would make it into a peer reviewed journal. This same of 'scientific' analysis used by Gifford is the approach used by global warming deniers who use the facts to support their thesis.

At its essense- LEED is a quality assurance system. It shares many similarities with the various ISO standards. What is common to all quality assurance systems is that the focus is on creating a culture of quality, and encouraging the important items affecting quality to be discussed and strategized.

What is important to note about quality assurance systems is that they significantly increase the probability of a meeting the quality standard, however they can not guarantee it 100%.

The fact that a system such as LEED cannot guarantee success is what is so contentious.
It is disappointing to eco-builiding passionate people to think that someone could put the 'green' mark on a building, and it not really be 'green'
In order to reduce the possibility of these buildings not being what they promised, LEED would have to change to integrate more of a control, and regulatory approach. However, if the LEED system was changed to be more of a control/regulatory system this would be more expensive, more onerous to comply with, and would likely be much less successful in prompting widescale change in the building industry, because fewer would take on the challenge to begin with.

So far LEED has been very successful in changing the way the building industry thinks about what quality is- now quality increasingly means LEED- and I think everyone can agree that LEED buildings are better buildings than those that are not.
So the question is would we really want a LEED system that had much higher integrity, if only a few bothered to use it?

Thanks, Reuben, for this link. It makes absolutely the right point - that relative metrics of energy efficiency miss the point of environmental protection, and only setting absolute consumption metrics begin to get us where we need to go. Simple example - the LEED Platinum 5,000 sf house vs. the normal 1,000 sf house - which has more impact? Ultimately, in the interests of environmental protection and social equity (which includes inter-species equity) we need to measure impact per person rather than impact per widget or sf.

Always interesting to encounter (the rare) critiques of energy efficiency advocates' familiar claims. I would also add what I think is a related point, but one that was not emphasized in the attached review, namely that energy efficiency is but a means to an end. More and more it is treated as an end in itself, and LEED, I think, along with the Energy Star New Homes programs, and other such efforts, go far toward suggesting this. The BTU/sq ft metric is already one step removed from the parameter with greatest relevance to climate change. I recommend reading the following report along similar lines - Is Efficiency Enough: Towards a New Framework for Carbon Savings in the California Residential Sector, by Mithra Moezzi and Rick Diamond of Lawrence Berkeley Nat'l Lab. http://repositories.cdlib.org/lbnl/LBNL-58580/

I'm also guessing that the responders were self-selecting for projects where someone keeps track of energy use, which in my own experience is an indicator that there is someone at the facility that cares and know something about energy use.

I find the median argument uncompelling, because what the climate change is dependent upon is energy used, so actual energy use per square foot over the entire dataset is what matters, not the average of the EUIs of a set of buildings. Whatever the analysis, total energy of the dataset divided by total square footage of the dataset is the meaningful number - it's the average EUI of all the buildings in the dataset. This is not based on statistician's sleight-of-hand, it's simply the most straightforward way to assess the impact of the energy use of the dataset.

I also have proposed in the past that the LEED certification not be awarded until two years of occupancy have occurred. This would easily shift the focus from the checklist and credit-mongering to both resource consumption and occupant satisfaction.

I've been an environmental building consultant on a number of buildings that have been certified at levels from Silver to Platinum, both new construction and renovation. Some of these buildings are top performers in energy use and some aren't so great. My experience is that once an owner gets the plaque, there isn't much interest in why the building uses the energy that it does - the PR has been garnered and they are on to something else.

If folks involved with LEED cared deeply about the climate change crisis, they would shift the system asap to one where actual performance was the basis of the rating. They would decouple the energy points from the flawed ASHRAE 90.1 standard and base them on how close the building gets to the lowest potential energy use for the building type (yes, labs and schools are different.) This forces the setting of actual EUI objectives by building type and climate zone, which IMO is the only way we will have meaningful progress to reducing the extraordinary impact buildings have on energy use and emissions.

I agree with Henry and others that the 2000-2003 CBECS data set is appropriate for comparison. I consider Mark Frankel's argument that some LEED buildings are renovations spurious; the far majority are new construction, and even those few renovations tend to have new lighting, new mechanical equipment, even envelope upgrades. Further to Mark Frankel’s argument, the buildings in the 2000-2003 (4 years, not 3) set is large enough – certainly larger than the LEED set, and I find the assertion that EUI’s for part-decades in CBECS are lower than full decades dubious. Further, in comparing full-decade averages, new buildings do not in fact use just as much as old ones. Decade averages from the 1920’s to the present have varied from around 80 to 100, I’m sure due to many factors. One can only make the best guess that whatever factors led to an EUI of 80 for the post-2000 period had a corresponding influence on LEED buildings.

As far as median vs. mean, I see three important points. First, although median can be a useful representation of centrality, if it is to be used for the LEED set, it must also be used for the CBECS set. It is never justified to compare median to mean. However, second is that, as Marc Rosenbaum points out, climate change depends on the total energy use of buildings, which is logically linked to mean (mean times total square footage is total energy use); not so for median. We want to know the total effect of LEED, not the typical impact, nor the impact that LEED would have had if it hadn’t been for those few hogs, darn it... But third, the problem with using mean is that we don’t know the energy use of all LEED buildings – only 121 of them – so we can’t completely justify a logical extension to total energy use. Therefore the only remaining rationale for using the median would be to neutralize outliers in a small sample of the entire set. However, the 121 buildings in the set are 22% of all LEED buildings – that seems to me sufficiently large to dampen outlier effects. It’s not a randomized sample, but the bias of the selection process is unknowable and can only be noted as such.

Ostensibly, the median was chosen as a way to overcome the skewing of outliers. However, as Nadav Malin points out, the “outliers” can be distinguished from the rest of the data set by their building type. The “outliers” were not a symptom of an insufficiently small data set per se, but a disproportionate number of energy intensive buildings in the set. Therefore using the median is missing the point; what is needed is rather a segregation of the data set by type. Nadav Malin points out that the NBI study did this (on page 37 of the final report). However, even that graph uses median values for LEED vs mean CBECS. Given that the skewing by high-energy types has been eliminated, it should compare mean to mean.

Further, the data should be weighed on a per-area basis, not only for the sake of an apples-to-apples comparison but also so that, as per Marc Rosenbaum’s comment, the data links logically to total effect on resource depletion, climate change, etc.

No comment from NBI or USGBC has yet addressed the fact that they advertise 25-30% savings over non-LEED buildings, despite the fact that the NBI study concluded only 24% savings for its core analysis. I think it would benefit us all to see a single graph (similar format to the one on page 37 of NBI’s final report) that compares EUI’s for LEED and CBECS; area to area, type to type, mean to mean, vintage to vintage. That would start us off with the most reliable information we can get on how well LEED buildings are performing, and would likely lead to other more useful analyses (why did these perform well, and those not well?). It would not be the desirability of the results but their correctness that would make them useful to improving the energy efficiency of our built environment. It’s easy to lose sight of this; let’s all do our best to remember it.

This debate becomes a real serious concern with governments mandating LEED on public buildings and starting to mandate it for private development also.

If we are going to be regulating this stuff, shouldn't we be using tools like the model energy code, try to improve that code, use real scientific metrics and then actually enforce the code.

What started out as a marketing effort of the building materials industry is really not the ideal basis for regulating the construction of buildings.

As everyone here knows, building good buildings is not the easiest of endeavors and doesn't lend itself to simplified universal solutions. Rating "greenness" by counting points is really a very arrogant presumption to start with.

I find the scatter of the data in the NBI study to be the critical issue, not the arguments of mean vs median. Mark Frankel attempted to determine whether either the LEED commissioning or the LEED daylighting point had a significant effect on measured energy performance. No significant difference was reported. The data set is only robust enough to talk about trends between two populations (argument about whether mean or median is appropriate and how the populations are characterized further points to problems inherent in population sets where the variance is large relative to differences between mean or median). The problem with the scatter is that LEED certifies design, not performance. Very few architects, mechanical engineers and energy simulators actually look at the measured performance of their projects after completed. Frankly, the scatter represents the scatter in the quality of the design teams and/or the commitment of the owners. If LEED were to certify based on the second year's data (as Marc Rosenbaum suggests), the design teams would be more concerned about actual performance.

In addition, owners of LEED certified buildings should consent to building use data being made publicly available. Knowledge of what is possible in high performance buildings is critical to design teams. I worked on a LEED Gold project which was included in the NBI study (and performed as it was designed). In addition to knowledge of the performance of previous designs, the team I participated with that designed the Aldo Leopold Legacy Center made use of the NREL report on six high performance buildings and the WEB reported performance Ordway Research Campus in Woods Hole as we set goals and developed the design.

Marc's point about comparing designs to ASHRAE 90.1 is absolutely correct. The Aldo Leopold Legacy Center was designed to be net zero based on energy generated on site. The team spent roughly $6,000 on a simulation model of a 90.1 "comparison" building even though net zero was the actual bar we were measuring against. LEED also does not permit modeling intelligent building occupants. Rather than using dimming controls, our lighting strategy provides multiple on/off switches with a daylighting design not requiring lights on sunny and partly cloudy days (provided owners actually turn lights off). LEED does not allow modeling occupant light usage in the Design Base Case simulation. Anyone walking into a daylit building with the lights off will have a more viseral experience "the lights are off in this building" than walking into a daylit with the lights on, but dimmed.

The first year performance of the Aldo Leopold Legacy Center was not Net Zero, as designed, but it was close. Detailed first year performance is being evaluated and we hope to be net zero in the second year. This evaluation is occuring because I am a faculty member (in architecture) and can afford the time and interest to do the work. As Marc points out, interest in the LEED and energy performance issues goes away when the plaque is on the wall and the four years of design, construction and first year of occupancy have come to pass. Frankly, keeping the onus of the cost for evaluation on the owner may not be the best way to quickly get high performance, low-emission buildings built. As our goals are social rather than individual, post occupancy evaluation should be supported by the DOE and carried out by students in local professional schools (architecture and engineering) with oversight by the local professional societies. NREL, NBI and USGBC could assist in developing standard evaluation metrics allowing for building type and climate zone variation. Buildings should be required to be sub-metered for plug loads, daylighting and HVAC systems to receive any LEED certification. Then will we begin to get truly useful data sets and improvements in building performance.

This is a great dialogue and I appreciate the work done by NBI and the subsequent comments. One of the most surprising findings is how the predicted performance of the code baseline buildings was close to the average performance of national building stock (CBECS). The NBI study points out, "This information has significant implications for policies and programs that use ASHRE 90.1-1999 as a baseline for driving increasing levels of building performance/carbon reduction and suggests the relationship between the stringency of this standard and standard (unregulated) building practice need significant further study and calibration." How well do you think we understand the energy implications of the ASHRAE 90.1 code? I think there is a lot of merit to Marc Rosenbaum's comments to design and compare buildings to actual EUI objectives based on building type, location etc.

Tom Hudson, PE, LEED AP
Green Building Services
Portland, OR

I have now read Nadav Malin's analysis, the March NBI study and their follow-up FAQ. I still appreciate Henry Gifford's bringing to light the shortcomings that he sees in the LEED process, and the important proposals he makes to modify it.

One strong conclusion from all of this investigation, no matter how one looks at the data, is that there are many LEED buildings which are significantly under-performing in energy efficiency.

Look at the poor performance relative to EnergyStar among the 60 buildings where that can be assessed. The NBI study says: “Nearly half of LEED buildings had Energy Star ratings of at least 75, meeting the qualification level for an EPA-certified Energy Star building.”

But that's not good! It is alarming, that, with all the effort the LEED process (presumably) entails, that less than half of these LEED buildings qualify for an EnergyStar rating! Shouldn't a “LEED” building end up in the top 25th percentile of buildings in its category?

Gifford points out that the NBI study should have compared LEED buildings only against recently-built buildings, and that that would make a big difference. NBI authors claim that the 410 buildings in the CBECS dataset which were built in 2000-2003 are too small a sample for comparison (and, it is true that CBECS data does not break out the recent group by detail). But that recent 410 group seems to be significantly more efficient than older buildings. Wouldn't that be a fairer general comparison in order to ascertain whether the LEED process is actually working the way it should? (And even if some of the LEED buildings are retrofits of older buildings, shouldn't they still be trying to retrofit to beyond-average new-construction standards?)

Because it seems that an equally plausible possibility is that buildings are gettting more efficient without LEED, and LEED is on average only a nominal difference!

Whatever more info might come from this data, I would hope that LEED/USGBC will dig to see whether and how well the under-performing buildings really were commissioned and managed, etc. And, I would hope they would push to see what's going on with the other 78% who didn't report at all!

At the end of any such analysis, I imagine that the conclusion will be that the best performing buildings are those where designers and builders and installers, and owners and managers, and, most important, maintenance staff, took all of this seriously, with or without "advanced" equipment. And, that the worst performing would be those where the rating was sought solely out of a mis-directed, if not cynical, desire for status. This would be a triple shame, when one realizes that energy conservation efforts, when done well, are actually very good for the bottom line!

One more comment about explaining variability in buildings, although it may confound all of the above. When “building scientists” look at buildings from afar, we are virtually ignorant of what goes on inside them. We tend to treat them literally as shells, connected to power sources, that operate, more or less independently from the “activity” that they house.

We then are prone to missing the fact that energy-use “variance” might be the result of the very essence of the “activity” going on inside!

A little brain exercise: imagine two “identical” commercial buildings -- whose construction, locale, systems, orientation, are exactly the same. And let's even say that they are both the same “type” -- office building, say, and the companies within them are even in the same industry. Now, what is the difference between them? Well, in our society, two different companies operating in the same niche will “naturally” attempt to be different from each other. One will be “fancier,” the other “leaner,” one will work harder, one later, one will have perks, the other will have parties, etc., etc. This is the nature of the competitive free marketplace, if not humanity. The competitive nature of business will tend to differentiate two otherwise identical enterprises, and their energy use will thus also tend to differ.

When we later go back to look at these two buildings, which “should” look the same, we will likely be confounded to explain their difference because we cannot see inside.

I don't have a good answer for this conundrum, but it is an issue at the heart of any building-comparison exercise.

One premise of incentive programs like LEED, of course, is that the LEED certification should be yet a further opportunity for competition!

But, if at times the only outcome of the LEED process is to LEAD to a plaque and not to actual dramatic energy improvements, then it is a sham, and all of us in the energy community NEED a quick corrective.

The bottom line is that Henry Gifford is helping to show that the LEED program is deficient in some important regards. The LEED program has boomed in the last five years or so, with hundreds of people hoping that they are making significant environmental/energy improvements in buildings, and it is high time for some tough scrutiny.

It is my fervent hope that the criticisms will be handled constructively so that the LEED program, or some alternative, will be made better. Those of us involved in responsible energy conservation over the years rely on good building science, and it is the scientific method to learn from experience.

Fred Davis Corporation
Energy-Efficient Lighting Wholesalers
----going green since 1983----
120 N. Meadows Rd., Medfield, MA 02052
800-497-2970
508-359-3610
fred@freddaviscorp.com