As we discussed in our previous review of humorous and horrifying events we actually witnessed in pharmaceutical facilities, “Pigeons In The Plant: 10 Real-Life Pharma Facility Blunders (And How To Avoid Them)”, our industry is flawed like any other. However, our impact on our global society is far more important because of our mission, to save lives. Indeed, the mistakes we make can be quite catastrophic.

This final series of events relates more to the engineering side of the business, what we do, what we tell others, and the culture we create. As an industry, we require solid risk-based thinking, common sense, and a sense of what is best practice. As in the previous article, each of these mini case studies has a message. These are all true and have occurred, as bizarre as it may seem.

Drawings Cost Money — We Don’t Need No Stinking Drawings

One of the clichés’ in our industry goes along the lines of how frugally CMOs run their businesses. Frugality is good, but not to the level that you injure your operation. A few years ago, an engineering firm was designing several modifications to the clean utilities at a CMO’s ageing aseptic plant. The complexity of the piping was significant enough, plus the added effort for the electrical, drain lines, architectural (wall, floor penetrations), etc., made it an important project as it integrated into many of their systems. When the design was completed, the engineering firm was asked to supervise the construction (done by a local construction firm). As commonplace in any industry, the engineer then provides “as built” or “record” drawings as a key part of the delivery. In this case, the engineering firm was asked to take out all the drawings from the proposal as the CMO would not be needing them. The engineering firm pressed the CMO, telling them that the wall enclosure would cover much of the work, and it would be a necessity for their qualification. The CMO’s engineer told the firm “drawings are expensive, and we don’t need them”!

Saving money is great, but when it impacts GMP documentation and the ability to locate lines, power, and utilities for future projects and site viability, that is foolishness. During a subsequent construction near the area discussed above, I witnessed the construction crew vigorously demolishing an apparent “legacy hollow wall” and an interstitial area, which flooded out the main material handling corridor because they hit an undocumented water line.

Cats In The Warehouse?

In a large eastern metropolitan area, there is a CMO site wedged in an industrial area. When performing a mock GMP audit, we were checking the material management systems, warehouse management procedures, and related systems. We asked to see the actual warehouse and walk through it as part of the audit. As we walked through the GMP released cage, we were caught off guard when we saw a midsize, orange-striped cat leap from one pallet to the next. It happened so fast, I was startled and asked the host what it was. The host simply said, “That was a cat!” I asked, “What is an animal doing in a GMP warehouse?” I was answered in a matter of fact manner that the cats are part of the pest control program due to the prolific nature of the inner-city rat problem. Just about then, I tripped over something at the corner of the rack area, jutting out slightly into the aisle, positioned between pallets. It was a litter box! Yes, a litter box, with all the expected contents. I told the host that regardless of all the risk assessments, many cats and litter boxes are not acceptable in a GMP warehouse.

Why pest management programs are such a challenge, we cannot understand. The perimeter of the building and all entry points must be considered and managed. In some extreme cases, the dock areas need double doors or an airlock type system to prevent animals, vermin, and other creatures great and small from coming in contact with drug products. An inner-city pest management professional needs to be consulted.

I (Herman) worked in Pfizer Brooklyn in Bedford Stuyvesant in the 1970s, so I understand and know that a serious pest management program can be effective, but we did not use cats!

That’s A Very Interesting Floor Pattern

A newly built midwestern biotech plant had an epoxy terrazzo floor installed from a brand-name franchise. The radiant blue hue began to deteriorate over the next three months into a series of white concentric circles, with material flaking/shedding from the white circular areas. The contractor that installed the floor had left the premises after an initial acceptance, and we had to call the head office and invoke their warranty. The installer’s first salvo when seeing the defective floor was, “It’s your fault for cleaning often and with harsh chemicals.” Then, we had to show them the specification saying the flooring is highly chemically resistant. Most brand-name floors installed correctly with proper slab preparation and a little care will last for 20 years. Floors are notorious for not performing, primarily due to the installer not mixing the resin correctly or applying it to an unprepared or cold floor or not having their employees properly gowned (see “7th Time Is The Charm” below). Ultimately, the head office’s inspection determined the wrong floor was applied and it was done so incorrectly, and they agreed to mill down the existing floor and reapply it until we were satisfied.

Inspection of the floors and all construction components before acceptance, and monitoring them after, is critical, as well as getting reputable installers/builders. This is done by auditing and gathering references, not by low bid. Low bids increase your risk, and the consequences in delays can be career-limiting.

Home Depot Caulk Is The Best!

An aseptic fill site in the Mid-Atlantic area had a very unusual occurrence: a settlement crack in the foundation. This crack came directly in the EU Grade C corridor, causing the floor to split from one wall across to the next, and then up the wall. The crack at the one wall was about ½ inch, and the opening at the other end was about 2 inches. The tear up the wall was overall ½ inches and ran up the wall for about 4 feet, until it was no longer visible. While this is rare, it is not unheard of. Remediation is typically done by segmenting a “wall-to-wall” space about 1 to 2 feet across, breaking out the old resin floor, grinding the floor below, and having the floor installer put in this minor section. They would then cove the floor up the wall and, after that, the wall can be repaired in a similar sectioned manner. These repairs could be done on a weekend with minimal interruption of operations.

This facility’s engineering staff came up with the fastest solution yet: go to Home Depot, buy a case of the popular silicone caulk, and fill all the gaps with this nice tacky goop. They implemented this scheme and all they got was a sticky mess all over (including the floor), trapping all the known particulates, fibers from personnel booties, and cart wheel bottoms. It looked horrible and was a very visible example of their ineptness.

The pharmaceutical industry is unique in that cleanliness, precision, and validation are the basis of how we function. How the engineering staff could have come up with this as an alternate to the proper fix is indicative of the mentality that encourages cost cutting at all levels. It also brings into question how operations and QA accepted this solution. This facility was the recipient of a warning letter and then had two changes of ownership. The poor condition of the plant overall reduced its market value and made it an easy acquisition target for an organization willing to invest and recapitalize it. The entire facility has been completely redone.

7th Time Is The Charm

Floors are a popular topic because some pharmaceutical engineering professionals believe a floor is a floor, whether it is an EU Grade A or the kitchen at home.  A startup biotech company was building a new facility and among the many challenges with the facility was a “ballroom” concept clinical production suite with an epoxy floor sold as a brand name but provided by various and sometimes dubious installers. Once the initial installation was done, upon inspection, it had an appearance like the “surface of the moon,” with pits, valleys, and ridges. This was rejected and the installer was called in to repair the floor. The floor was ground down and reapplied unsuccessfully, as the grinding down was not complete before reapplication. Subsequently, the floor was reapplied four more times, until finally, the biotech’s facility engineer observed the application being done late at night. She noticed the installers were wearing dirty street clothes and filth-laden work boots. When asked if they had ever installed a cleanroom floor before and whether they knew you need to be cleanroom gowned to prevent hair and dirt from entering the epoxy surface, they said “yes” without blinking an eye. It never occurred to them that they were not doing so, or they just lied. They were ordered off the premises, and a new installer was engaged, with proper gowning and oversight during normal working hours. The seventh application was successful, as the installer knew how to execute the mix and perform a clean install. Trying to get a company that made the initial mistakes – over several tries – to correct itself and to do the right procedure is generally a fruitless adventure.

Aging Biotech and Aseptic Facilities – Renovation for Survival

Working with an aseptic or biological facility? Learn the facility design features that put the “c” in cGMP from aseptic filling facilities and systems experts, Herman Bozenhardt and Erich Bozenhardt. They’ll also cover the modern technologies of facility redevelopment that can extend facility life by 10-20 years (and reduce operating risks!).

The Magical Properties Of Yellow Tape (HVAC)

Over the years we have seen many magic tricks on TV and in live shows; however, I always was intrigued by the magic of yellow tape – or the even more powerful yellow and black striped tape. One U.S.-based CMO recognized the need to conform to EU as well as FDA requirements, but it had a tough problem with the lack of airlocks for both personnel and materials. Between CNC to Grade D, there was an airlock (actually a storage room with gowning racks), but there was no discernable airlock from EU Grade D to the many C suites. As a solution, the company put a yellow tape rectangle segment that looked like an airlock outline on the floor adjacent to the door. When you needed to enter from the D space, you stepped in the “tape airlock,” put on another bootie (third one), sanitized your hands, and stepped across the line into EU Grade C!

The more powerful tape is the black and yellow striped version. A major pharmaceutical company had shoe-horned a high-volume filling line into an old building. Their fundamental problem was not providing enough room for a pathway to go from a busy, congested CNC hallway into the Grade C area. Realizing they had only enough room for two doors (or one airlock), they decided to place the yellow and black tape on the floor midpoint across the airlock and up both walls to the ceiling. In addition, they had a placard on the wall showing D/C, with the slash in black and yellow. Instant airlock! I asked them how that airlock was qualified and what the ACR (air change rate) was, and I received blank stares. Airlocks and Annex 1 are actually spelled out in the regulations. Not having a design that accommodates the regulations is not an escape clause. This case called into question the CMO’s design, compliance, and how they validated this and what criteria they used.

More Magical Properties Of Yellow Tape (Virtual Cages)

That black and yellow tape from the previous case study has global applications. In an aseptic facility in the Eastern U.S., the magical tape was used instead of a product quarantine cage. I observed open trays of unlabeled, bright stock 10 R flint vials stacked 4 feet high on a pallet in a pedestrian walkway in the plant, with only an 8½ x 11” sheet of paper designating the lot number and product code taped to one of the trays. Directly adjacent to these two pallets were more stacks of vials, all of which appeared to be clear 10 R flint vials with a different lot number. These were all unsecured, uncontrolled, unlabeled, and vulnerable to future mislabeling.

Apparently, the facility’s warehouse was overwhelmed with product and had no physical space to secure the unreleased product, so the hallway (where visitors and employees alike traveled) was deemed good enough to store the vials, since it was protected by the black and yellow striped tape.

This facility was under consent decree just prior to this observation. The facility went through no renovation, was shut down, and is today a large warehouse that no longer functions as a manufacturing plant.

If It’s Good Enough For My Car, It’s Good Enough For The Aseptic Facility

Often, we make spot decisions on what is good or not for construction within an aseptic facility based upon need, timing, and materials available. We perform “mental risk assessments” all the time and designate impact vs. non-impact equipment and systems. Sometimes the use of the most casual of materials or methods can have a dire effect. In one U.S. facility with substantial aseptic capability, a new compounding process needed a small continuous supply of chilled water. This chilled water would flow through the jacket of a tank containing an intermediate material used as a component of the final solution. The engineering of the new process did not identify nor properly address the need for the source of the water to interface at the point of use connection in the pipe chase. In order to remedy this omission in the engineering, the plant engineer decided that the use of a rubber hose (actually several hose lengths) would be an adequate low-risk solution. Since he did not have a length of hose long enough, he used a common automotive hose coupler and hose clamps (automotive screw style) to complete the piping. He also wanted to make sure there was no leaking at the coupling, so they tightened it as tight as they could. The solution worked initially, and the quick thinking by the engineer was hailed as an economic success – or was it? What about the engineering change document?

After approximately two to three months of on and off use, flow and no flow, and varying temperature, the hose clamp tore through the hose at the coupling and an unchecked volume of water poured through the pipe chase, eventually finding its way onto the ceiling of the aseptic fill room. This filled the ceiling, drenched the HEPAs, and ran down the walls of the entire suite, rendering it inoperable for several months.

In this as in other case studies, we make decisions daily and have a choice to get it done cheaply and fast or correctly. Correctly is usually the best economic choice. All internal utilities need to be in welded piping.

That Red Stuff On The Stainless Steel Is Just From The Cleaning Chemicals

A major pharmaceutical company in the U.S. has a fill finish site with several excellent isolator lines and above-average capability from an operations standpoint. During an engineering readiness audit, it was observed that almost all the vial trays were severely rouged to the point that they were red in color. In addition, many of the vessel surfaces in the filling suite and compounding suite were also rouged. The interesting aspect was that nobody seemed to notice or care about the red-stained stainless steel on the site. Finally, at an operational level interview, I asked one of the supervisors why there was an extensive amount of rouging and why they did not de-rouge and passivate it. The answer was very simple: the “red stuff is what you get when you clean all the equipment; it’s all fine.” At this plant, rouging was a good thing, as it meant the equipment was well cleaned over the years. This is the problem with either false information or uninformed folklore that is passed down from management; people believe it and don’t question it. As an industry and as a society, we need to question, challenge, and understand what we are doing. Subsequently, the facility had a major metallurgical audit and a substantial de-rouging effort, and a re-passivation program was put in place. Self-auditing, understanding metallurgy, and keeping stainless steel shiny are important.


In summary, we – the engineering community – must always use good judgement and especially GMP judgement, strive for excellence, and not try to be “Mr. Fixits” using any tools at our disposal. We need to make use of communities of practice such as the ISPE and PDA to learn and reach out for advice. We should never make hasty decisions and have the attitude that what we are doing is “good enough,” because our decisions not only impact our products today, but doing so also sets a bad precedent that will be handed down over the years in our plants. These attitudes, opinions, and our personal example will, as we have seen here, eventually catch up with us and could cause a catastrophic failure.

We need to be the stewards of our own methods, engineering, and conduct so we can set a standard of success in our plants and business. If we don’t do it, who will?

In the end, let’s ask ourselves what have we contributed to our business and what good did we do for humanity?

About The Authors:

Herman Bozenhardt has 43 years of experience in pharmaceutical, biotechnology, and medical device manufacturing, engineering, and compliance. He is a recognized expert in the area of aseptic filling facilities and systems and has extensive experience in the manufacture of therapeutic biologicals and vaccines. His current consulting work focuses on the areas of aseptic systems, biological manufacturing, and automation/computer systems. He has a B.S. in chemical engineering and an M.S. in system engineering, both from the Polytechnic Institute of Brooklyn. He can be reached via email at  and on LinkedIn.

Erich Bozenhardt, PE, is the process manager for IPS-Integrated Project Services’ process group in Raleigh, NC. He has 13 years of experience in the biotechnology and aseptic processing business and has led several biological manufacturing projects, including cell therapies, mammalian cell culture, and novel delivery systems. He has a B.S. in chemical engineering and an MBA, both from the University of Delaware. He can be reached at via email at and on LinkedIn.

Leave a Reply