Part 3 – Everything I Know I Learned After College

Consider the following posted requisition for hire:

Senior / Principal Quality Lead

“ … This position is responsible for supplying quality leadership throughout the organization.  The Quality Lead will interact with and influence other cross functional team members, including R&D, manufacturing, regulatory, clinical, etc.

Responsibilities include:

  • representing Quality on new product introduction projects
  • developing, assessing, and validating test methods
  • assuring compliance with internal SOP’s and external regulations
  • leading Root-Cause and CAPA efforts, Risk Management, Risk Reviews, Material Review Boards, and Change Control needs
  • Representing the Customer, identifying their needs, and reducing customer complaints
  • Driving and fostering a quality environment and mindset through the business: provides training, guidance, leadership, and expertise to the R&D, quality, and manufacturing organizations.

Skills needed:

Competency in statistical techniques: Gauge R&R, SPC, process capability, sampling plans and sample size determination, DOE, ANOVA, regression, Lean

  • Design FMEA, process FMEA
  • Test method and protocol development for verification and validation studies, and evaluation of test results
  • Demonstrated ability to drive change
  • Strong analytical and problem solving skills
  • Ability to multi-task

Experience

  • Practical knowledge of FDA Quality Systems Regulations, ISO 13485 (medical device quality management systems), ISO 14971 (Risk Management), IEC 62366 (usability engineering)
  • Previous experience working in an FDA regulated environment
  • Bachelor’s degree in an engineering discipline or related STEM field, and 1-5 years experience.

… “

Ok – this is not an actual requisition duplicated verbatim.  But it is a synthesis of real job postings this past year, all of them seeking hires in the medical device and combination device arena.  The “posting” is representative of a great many of the advertised job descriptions and criteria against which candidates are evaluated.

Let’s first look at the needs that appear to drive this job description.  In a previous article (“A Quality System is Not Enough”) we indicated that the best practices of systems engineering, Six Sigma, and project management all supply the tools, and practices using those tools, to support development of high-quality products.  The job description above contains the responsibility to “represent the customer and to identify their needs”.  These activities of identifying stakeholders and evaluating their needs come directly out of project management, Six Sigma, and systems engineering.  This is also the critical first step in the FDA and ISO design controls, which call for identifying what it is that the new product needs to do.  These needs statements are the “Design Inputs (DI).”  It makes sense that for new product development and design…this skill is needed.

Next in the requisition are a series of needs around verification and validation, along with associated test method development and evaluation of results.  First, we note that the activities of verification and validation are integral to systems engineering and project management – though they are often confused.  A useful graphic to illustrate the difference between verification and validation is the “V diagram” from systems engineering:

Follow the arrows from the top left: first define user needs, use those to define system requirements, then sub system design requirements, and so on.  After you do the design work to the lowest levels, work your way up the right-hand side: verify that what you actually made meets the component, sub-system, and system design requirements.

As we will discuss in a later article, the design requirements that are “verified” are (typically) posed in technical language: a dimension is 5 mm, a weight is 12.3g, a volume is 7.8 ml, and so on.  Some non-technical language may be present at the “higher” system levels.  At “lower” levels the language becomes more specifically technical … but at all levels the requirements are always measurable.  The needed verifications can be done in the lab with (as defined by the test protocol) micrometers, balances, MTS machines, etc.  They can also be accomplished by: inspection of drawings; inspection of the physical product; demonstration of function; or analysis via mathematical models or other calculations.  Verification can, and should, be an ongoing activity: it takes place throughout the design process – not just as a single final test.

The language used to define User Needs, however, is a whole different thing.  A user will not come out and say “I want this widget to be 5.6 cm in diameter with a maximum weight of 0.23 kg.”  They say something more like, “I want to be able to comfortably hold this in my hand.,” or “I want to be able to turn the knob in an operating room while wearing sterile gloves.”  A design is validated by giving the user the final product (or a suitable substitute) and asking, “Does this do what you need it to do?”  (The execution of usability engineering is more rigorous than the previous statement, but hopefully the gist of the point comes through).

So – validation is about confirming that the design output (DO), or the device itself as actually built, meets the user’s needs and expectations.  Verification is different: it does not involve human factors, rather it demonstrates that each product specification, at each “level” of the design, is met.  Verification requires that laboratory-based tests, inspection, or analyses, be developed, executed, and interpreted.  It is perhaps crucial for the business needs of an organization to take note that a specification could be developed that passes verification, but fails validation if the design fails to meet the user’s needs.

Returning to our requisition, this position clearly needs skills in test method development.  The methodologies used to perform these activities are mostly statistical methods that are integral to the body of knowledge contained within Six Sigma (gauge R&R, sampling plans and sample size determination, capability analysis, regression, etc.).  Less obvious, but still present in the job description, are tools that assist in effectively determining the ability of a design to meet the stakeholders’ needs (DOE, ANOVA, regression, root-cause analysis, and FMEA).  Again, the use of all of these methodologies makes sense in satisfying the regulatory “requirements.” All of these methodologies are contained in the bodies of knowledge of Six Sigma and systems engineering.

Continuing the evaluation of the requisition, we see a series of responsibilities and skills often considered “soft” skills: “representing quality throughout the organization” (i.e. influence management), “fostering a quality mindset throughout the organization” (i.e. change management), “providing training, guidance, leadership” (i.e. instruction and mentorship), “participating in Material Review Boards” and “change control needs” (i.e. critical thinking).  These skills and methodologies are not taught by systems engineering, Six Sigma, or project management (although project management does discuss and foster them under “develop” and “manage” the project team).  Neither are these skills taught in an engineering education: they are learned over time through experience, and occasionally through hard-learned lessons.  For some, these skills are never learned.

Not explicitly stated in the job description, but definitely present, is a need for something we seldom talk about: confidence, strength, maturity of personality, and a willingness to engage in productive conflict.  In sum: an ability to challenge an organization, its constituent functions, and its leadership, to think differently and to consider options (whether design or process options) other than those with which one is familiar or would otherwise prefer.  This is the ability to see through the fog and identify the core element to execute – and to teach others how to do the same.  In short – to lead.

These skills are definitely not taught in the engineering disciplines.  The authors would argue that the ability to play a leadership role comes from hard-learned experience and time on the job.  Six Sigma and Design for Six Sigma actually teach methodologies that support an individual executing a leadership role in a team.  Although many of the decision-making tools taught in Six Sigma and Design for Six Sigma are taught as “engineering” tools … in reality, when effectively applied, they are really methodologies to lead people to consensus in a team or organization.  Examples of such tools include: prioritization/selection matrices, RACI matrices, affinity diagrams, “house of quality” incidence matrices, process flow charts, fishbone/Ishikawa diagrams, 5-why techniques, FMEA techniques, Pugh concept selection matrices, the analytic hierarchy process, and many more.  Think about it: even graphical display of data (Pareto charts, histograms, scatter plots, regression analysis, etc.) and statistical methodologies, with their agreed-upon significance levels, p-values, etc. are methodologies for building consensus in the face of potentially confusing information.

Finally, in the job description there are requirements of practical knowledge of regulatory and ISO requirements, combined with previous experience working in an FDA regulated environment.

All of this is asked for with 1-5 years of experience after leaving school.  We are not distorting the experience called for – check the job requisitions that are now being posted.

This example posting, and many like it, is for a “Quality Lead.”  What it is really describing however, are many of the activities and skill sets involved in systems engineering, project management, and Six Sigma.

Here is the critical point: the skills, methodologies, and experiences central to designing and manufacturing high-quality product, are NOT taught, to any significant level, in either the undergraduate or graduate engineering curricula.  Also not taught in those curricula are the skills needed to do so in a manner compliant to regulatory expectations.  Practical knowledge and effective execution of this set of skills is only developed through practical work experience.

Everything I know to effectively navigate and execute engineering product development and produce high-quality product I learned after college.

One of the authors has had opportunity to seek input on the following question: “How much experience is needed to effectively execute the skill set and responsibilities required by this job description, or ones similar to it?”  The question was posed to people in the medical device industry with a wide range of experience and job level – varying from Senior/Principal engineer up through the Director and VP level.  The responses were quite uniform:

At least 15 years.

Yet, almost uniformly, postings look for people to fill this role who have less than five years of experience.  Anyone with more experience is automatically considered “over qualified” and not even considered.

Think about it.  In our hiring practices we are specifically excluding from these critical roles the very people who have the practical experience and skill sets needed to produce high-quality product.  Instead, we expect those skills from people right out of college, where the skills are not taught, and with far too little real-world experience to have acquired and therefore to use them.

To make things worse, organizations frequently look askance at any or all of the disciplines of project management, Six Sigma, or systems engineering.  Why?  The reasons are complicated.  Sometimes these disciplines try to create an “empire” within an organization, try to impose too much overhead or constraints on an organization, and truly do slow down organizational execution.  Sometimes the organization rejects the rigor called for because they “know what their design is and just want to go ahead and build it” (a perspective which is, by the way, antithetical to the Design Controls requirements – so such an organization should step very carefully).  Sometimes the organization is just resistant to change and does not embrace the “new” disciplines (regardless of the fact that for decades these have long been world-wide accepted best practices!).

Whatever the reason, organizationally we often do not acknowledge the critical learning coming from project management, Six Sigma, and systems engineering.  Because of this we also do not seek to sustain and propagate them within an organization.

What can we do to change this mismatch between the true quality goals of our organizations and our expectations of who we will hire to achieve those goals?

As we said earlier, the skills to achieve those quality goals reside in the disciplines of systems engineering, project management, and Six Sigma.  The authors do not believe there is a need to create each of these functions in their totality within an organization.  Many organizations, especially small ones, do not need or cannot afford that.  Also, in any sized organization there is real danger if any of those disciplines act in a way that their sole purpose is to promote and sustain themselves, rather than act in a way that promotes the overall health of the organization (this is a topic of a later article).

Rather, we are talking about recognizing, embracing, and fostering the individual teachings and practices embodied by these disciplines.  There is great overlap between the disciplines of systems engineering, project management, and Six Sigma.  But, with regard to producing high-quality product, they also each have unique benefits to the overall objective.  Systems engineering and project management supply focus and structure around identifying the stakeholders, clearly documenting their needs, and identifying how the system can be defined and broken down to sub systems in order to meet those needs.  Design for Six Sigma focuses on identifying a best design to meet stakeholder needs and requirements (requirements are not the same as design – also a topic of a later article). Six Sigma and design for Six Sigma together supply the statistical/analytical horsepower to: quantify the capabilities of proposed design and manufacturing processes to satisfy the requirements; to distinguish between competing design concepts (and in so doing supply the very mechanisms needed to perform design verification and validation); and to monitor and control designs and manufacturing processes over time through control charting methodologies.  Lean, although we have not focused on this practice above, supplies the process flow charting and related graphical and visual methodologies to build team consensus on the understanding of exactly what our processes are (and thus how they can be improved) and how they are performing.

We state again, the clear majority of these methodologies, of which there are literally hundreds, are not taught during college-level training.  We are not going to get them from candidates with just a few years of experience.

So, how to proceed?  At a minimum, seek and hire candidates who are certified by a reputable body at a black-belt level (or equivalent) in these disciplines.   There is a growing trend in Universities and Community Colleges to offer certification training through their Continuing Education programs.  Many companies are taking advantage of this to send selected employees for such certification. The certification gives some level of confidence that these candidates carry with them an appropriate understanding, knowledge and practice in these methodologies.  Do not hire them to create a “systems engineering group … or a “project management group” … or a “Six Sigma process improvement group.”  Hire them with a mandate to use, and to demand the use of, the appropriate methodologies at the appropriate time and place.  Put them in a position of organizational influence and oversight so that they can influence, model, and coach others in the use of these methodologies.

For somewhat larger organizations, we recommend identifying individuals who show interest in and passion for these methodologies.  Sponsor them (in terms of both funding and time) to train to a black belt level (or equivalent) through a reputable outside organization.  Again, do not use them to create additional functions, but use them to identify and use the appropriate methodologies at the right time and place.  Above all, support their activities within the organization.

The largest organizations should seriously consider institutionalizing the learning and internal teaching of these skill sets.  Where this has been done, the use of these methodologies has born great fruit.  The practice of “internalization” of these engineering methodologies is something that we have lost over the years.  The authors believe this loss is a significant cause of the discordance between implementation of “quality systems” as a compliance-focused mechanism versus an organization’s use of fundamental and powerful engineering methodologies to produce high-quality product.

In the first article these authors posted, we made the point that product quality is an outcome of effective interaction between organizational functions, and cannot be “owned” by any one function.  For an organization to effectively wield the methodologies of systems engineering, project management, design for Six Sigma, and Six Sigma, the appropriate “pieces” (i.e. specific methodologies) must be used at the correct time.  Some of them need to be used up front during evaluation of the needs of the stakeholders, others during translation of those needs into product requirements, others during the development of a design that satisfies product requirements, others when the design (and associated manufacturing processes) are verified and/or validated, and still others during manufacturing and post-market surveillance.

We need to empower the individual functions in the organization to use these engineering methodologies appropriately.  A quality system by itself, because it cannot foresee and define reactions and procedures for all eventualities, cannot therefore define what is “appropriate” to a sufficiently fine level of detail.

Rather, the practices an organization puts in to place on a daily basis, used within the framework of the quality system, but not defined by the quality system, define how the quality system is “used.”  It is the discrete engineering practices that ultimately lead to high-quality product.

© DPM Insight, LLC 2017 All Rights Reserved

About the Authors:

Cushing Hamlen

Over 27 years of experience in industry, including 20 years with Medtronic, where he worked and consulted with many organizational functions, including research, systems engineering, product design, process design, manufacturing, and vendor management.  He has also worked with development, regulatory submission, and clinical trials of combination products using the pharma (IND) regulatory pathway.  He has been extensively involved with quality system (FDA and ISO) use and design, and is particularly concerned about effective understanding and use of product requirements and design controls.  He has formally taught elements of systems engineering, design for Six Sigma, Lean, and Six Sigma.  Cushing has degrees in chemistry and chemical engineering, is certified as a Project Management Professional, is certified as a Master Black Belt in Lean Sigma, and is the owner/member of DPMInsight, LLC (www.dpmillc.com).

 

Bob Parsons

Over 26 years of experience in leading Quality Assurance, Validation and remediation efforts in FDA regulated Medical Device and Pharmaceutical industry. Experience includes product development life cycle management from initial VOC through New Product Introductions (NPI), sustainable manufacturing, and end of life product management.  Technical expertise in quality system gap assessment, system enhancement, alignment and implementation of all quality elements including design controls, risk management, purchasing controls, change control and post-market surveillance.  Regulatory experience includes; ISO 13485, 9001 and 14971 certification, providing guidance for FDA PMA/510K and CE clearance, designated Management Representative, company representative and lead during FDA and ISO audits, 483 and warning letter resolution with experience working within consent-decree environments.  Bob currently supports various organizations in remediation and compliance projects through Raland Compliance Partners (RCP) www.raland.com .

 

Michael B. Falkow

Michael Falkow is a Quality Specialist with Raland Compliance Partners. He has served as a regulatory compliance and quality assurance executive with multi-facility/international companies and was an FDA Compliance Officer and Senior Investigator/Drug Specialist.  Michael has subject matter expertise for quality and regulatory compliance, quality auditing, quality assurance, quality control, supplier evaluation and certification, and compliance remediation.  He has been approved by FDA as a GMP certifying authority and is qualified to provide Expert Witness testimony for GMPs.

Currently – Adjunct Professor at Mercer County Community College – teaching courses on Clinical Development for Certificate Program in Clinical Research as part of Drexel University’s Masters Degree in Clinical Development.

Part 2 – A Quality System is Not Enough

About twenty years ago the FDA re-wrote 21 CFR Part 820, the “Quality System Regulation” (QSR), to require that manufacturers institute and maintain quality systems.  This re-write included the critical element of design controls.  Even before that re-write of Part 820, the practices around design controls were embodied in ISO standards.  Nevertheless, it is still common for design and manufacturing organizations to re-tool their quality systems because of product quality issues – and those organizations pay a significant amount for help to create or completely re-write their quality system, often on an urgent or emergent basis.  These revisions frequently focus on design control practices.  An extensive market exists around this need – and several of the authors have been part of this market for quite some time.

Yet, issues with product quality persist: companies continue to pay the prices associated with poor quality, or they succumb to market and regulatory pressures and do not survive.  Clearly there is room for improvement in practices that actually lead to high-quality product.

The authors believe that, to understand how this situation can be changed, it is important that we first understand what quality systems are and where they came from.

It is common and tempting to view a quality system on a point-by-point basis.  We pull out one of the little books that list the regulations (21 CFR 210/211, 21 CFR 820, 21 CFR 11, etc.), and shuffle the pages to Section XXX.YYY.  We read one sentence and ask “are we satisfying that requirement?”  Worse, we go Section XXX.YYY, read it, and say “this does not say I have to do ‘activity Z’ (i.e. some other activity) … so I won’t do it.”  Come on, admit it … many of you have been part of those or similar discussions.  This is a misleading “forest for the trees” approach to creation, management, and use of quality systems.  When we focus on the minutiae of specific phrases, and debate fine points in the statements’ meaning, we lose sight of the objective of those statements.  Worse, we don’t clearly envision from the beginning what the quality system is trying to accomplish.  In addition, we lose sight of the fact that the QSR regulations are minimum requirements – and in so doing we don’t achieve the desired goal of the quality systems.

So, what are “quality systems?”  Where did they come from?  Information about this exists in many practices that surround us: Project Management, Six Sigma, Design for Six Sigma, and Systems Engineering.  All of these disciplines embody world-wide, accepted best practices. Yet, they are often regarded as disparate systems and, unfortunately, are often thought to conflict with each other in execution.  In reality, these practices share many common elements.

At the beginning of any product design effort, Project Management, Six Sigma, Systems Engineering, and 21 CFR Part 820 all call on you to do the same things: figure out what the customers need; define the disease the product is intended to treat or cure; identify what the customers will buy; and then carefully and clearly define the specific set of needs you choose to address in your product.  And – importantly – do so before you actually design the product so you are not justifying your design after the fact.

Specifically, Project Management calls for: identification of stakeholders, collection of project requirements, and definition of project scope (PMBOK[1] , 5th ed.).  Six Sigma calls for: identification of the customer, seeking input from the customer, documentation of Customer Needs, and setting of Specifications.   Systems Engineering calls for: identification of Stakeholders and definition of Stakeholder Requirements.  Here is what the CFR says about this: “Each manufacturer shall establish and maintain procedures to ensure that the design requirements relating to a device are appropriate and address the intended use of the device, including the needs of the user and patient” (21 CFR 820.30(c)).

All of these disciplines are saying the same thing.

The next step in the product development process is to determine how the pieces of the project (or device) fit together and to identify which “pieces” are most important.  In this development step, all the disciplines identified above are once again calling for the same actions – namely, to define the product with traceability of the functions of its subsystems to the higher-level needs of the stakeholders (to make sure the design covers the intended functions of the product).  We also are called to assure all requirements have been met.

Specifically, Project Management says: create Work Breakdown Structure (i.e. the work that represents the pieces of the project or product) that achieves the requirements, and Sequence the Activities.  Design for Six Sigma says: develop a high level design, identify the Critical to Quality output, drive to a detailed design (i.e. components and subsystems).  Six Sigma also calls for: identification of Critical to Quality output.  Systems Engineering says: Architect the system, its sub-systems, the interfaces between them and trace the design outputs to the inputs.  The CFR says: “Each manufacturer shall establish and maintain procedures for defining and documenting design output in terms that allow an adequate evaluation of conformance to design input requirements” (21 CFR 820.30(d)).

Our next example is the design verification/validation step of the product development process.  Here, we once again see that the disciplines we have identified above are calling for the same activity.  Specifically, Project Management calls for: documentation and traceability of requirements, followed by validation of deliverables against those requirements.  Design for Six Sigma explicitly calls to: Analyze and Verify the design (in the DMADV[2] process).  Systems Engineering calls for: Verification and Validation of a system based on established and traced requirements.  The CFR says: “Each manufacturer shall establish and maintain procedures for verifying the device design.  Design verification shall confirm that the design output meets the design input requirements.” (21 CFR 820.30(f)).  Similar language is used in Part 820 around the need for Validation.   Project Management does not draw quite as clear a distinction between verification and validation … but it does try.  It puts the onus for validation on the customer, and for verification on the project team).

Without going in to detail, the parallels continue in: the practice of identifying, supporting (i.e. funding), and training a team (21 CFR 820.25 Personnel); the practice of identifying, selecting, and managing vendors (21 CFR 820.50 Purchasing Controls); the practice of monitoring and controlling designs and processes (21 CFR 820.70); and the practice of controlling changes to product or processes (21 CFR 820.30(i).  All of these regulatory “requirements”, and more, have parallels in world-wide accepted best practices.  In most cases the regulatory agencies likely did not create these “requirements” out of thin air: The requirements were most likely drawn directly from these accepted best practices.

There are four points to take away from this.  The first point is that these seemingly disparate practices … are not so disparate after all.  They are trying to accomplish the same objective.  The second point is that the conflict or suspicion that often exists between people performing these practices in the workplace is senseless, because these practices are trying to accomplish the same goals.  The third point is that we cannot “cherry pick” the facets of each of these best practices to suit our preferences – doing so ignores the balances inherent in each of the practices, and gives rise to the conflicts that people practicing them frequently experience.  The fourth point is that to succeed in producing high-quality product we need a holistic understanding of what these practices are trying to accomplish.  We need to see the “forest” and not focus on the individual “trees.”

So what is the objective of these practices … what is the “forest?”  The answer is quite simple, actually.  The goal of these practices is to:

  • concisely define and design products that clearly meet the needs of customers/stakeholders at an acceptable level
  • to manufacture and distribute those products in a state that continues to meet the needs of the stakeholders at an acceptable level
  • to do both of the above in a predictable and reproducible manner

The regulatory expectations only add one thing:

  • provide reasonable evidence that you are doing the first three things.

In other words, demonstrate that you built the product you said you were going to.

The Good Manufacturing Practices (cGMP’s), which are often considered separately, are focused on achieving the “predictably and repeatably” goal.

We get into trouble executing to a “quality system” because we tend to focus so much on meeting the letter of each statement in the regulations that we lose track of the regulations’ context and intent.  In the vignette in the first article in this series, we relate in lines 9 – 12 that the quality system has been developed to meet the contents of the regulations, and the development team is trained to adhere strictly to its procedures.  The problem with this approach, which is commonly taken, is that our “quality system” in practice becomes a “compliance system.”  We do what the system “says,” and often nothing more – despite the fact that the regulatory agencies clearly state that the regulations represent a minimum amount of rigor.  We ignore (or, as in lines 11-12 of the vignette, are not allowed to act on) what makes scientific sense or is otherwise defensible based on the specific needs of the current activities; but this is exactly counter to the intent of the regulations.  In the end we lose track of and do not execute the very activities and evaluations that produce high-quality product.  Instead, we create waste and frustration.

We need to recognize and embrace the understanding that a few regulatory statements cannot capture all the meaning and fine points of the rich collection of practices contained in the engineering and project management bodies of knowledge.  It is therefore not wise to assign responsibility for high product quality to the relatively few statements in regulations, or to a single function within an organization.  If instead, as much as possible throughout the organization, we really understand and execute what these best practices recommend, improved product quality will naturally follow, as will regulatory compliance.

It is likewise critical that each one of the individuals practicing in the areas mentioned above do not “cherry pick” their focus.  One of the most useful concepts coming out of Project Management is the “Triple Constraint” or “Project Management Triangle.”  This states that any project is constrained by time, cost, and scope – and that quality results from an appropriate balance between these constraints.  This is a truism.  Wishful thinking or demands on the development team can not change this interdependence.  Therefore, to achieve high product quality, Project Management cannot ignore the need to clearly define and manage Scope in favor of cost and schedule.  Likewise, Systems Engineering and Design for Six Sigma cannot ignore the demands of budget and schedule in favor of their work on scope (i.e. requirements definition and system design).  To be selective in executing within each discipline is to neglect the very internal teachings of each of those disciplines.

So, the solution to our problem lies at least partially in the following: the question of how to achieve high-product quality needs to be changed from “how can we change our quality system” to: “how can we effectively and flexibly execute within and around the framework of our quality system.”  These are two very, very, different statements.  The former requires that we open a small book and start discussing fine points of meaning of individual phrases.  The latter requires that we learn and institutionalize, within our organizations, the rich bodies of knowledge contained in the practices of Project Management, Systems Engineering, and Design for Six Sigma.

© 2016 DPMInsight, LLC all right reserved.

About the Authors:

Cushing Hamlen

Over 27 years of experience in industry, including 20 years with Medtronic, where he worked and consulted with many organizational functions, including research, systems engineering, product design, process design, manufacturing, and vendor management.  He has also worked with development, regulatory submission, and clinical trials of combination products using the pharma (IND) regulatory pathway.  He has been extensively involved with quality system (FDA and ISO) use and design, and is particularly concerned about effective understanding and use of product requirements and design controls.  He has formally taught elements of systems engineering, design for six sigma, Lean, and six sigma.  Cushing has degrees in chemistry and chemical engineering, is certified as a Project Management Professional, is certified as a Master Black Belt in Lean Sigma, and is the owner/member of DPMInsight, LLC (www.dpmillc.com).

Bob Parsons

Over 26 years of experience in leading Quality Assurance, Validation and remediation efforts in FDA regulated Medical Device and Pharmaceutical industry. Experience includes product development life cycle management from initial VOC through New Product Introductions (NPI), sustainable manufacturing, and end of life product management.  Technical expertise in quality system gap assessment, system enhancement, alignment and implementation of all quality elements including design controls, risk management, purchasing controls, change control and post-market surveillance.  Regulatory experience includes; ISO 13485, 9001 and 14971 certification, providing guidance for FDA PMA/510K and CE clearance, designated Management Representative, company representative and lead during FDA and ISO audits, 483 and warning letter resolution with experience working within consent-decree environments.  Bob currently supports various organizations in remediation and compliance projects through Raland Compliance Partners (RCP) www.raland.com .

 

Michael B. Falkow

Michael Falkow is a Quality Specialist with Raland Compliance Partners. He has served as a regulatory compliance and quality assurance executive with multi-facility/international companies and was an FDA Compliance Officer and Senior Investigator/Drug Specialist.  Michael has subject matter expertise for quality and regulatory compliance, quality auditing, quality assurance, quality control, supplier evaluation and certification, and compliance remediation.  He has been approved by FDA as a GMP certifying authority and is qualified to provide Expert Witness testimony for GMPs.

Currently – Adjunct Professor at Mercer County Community College – teaching courses on Clinical Development for Certificate Program in Clinical Research as part of Drexel University’s Masters Degree in Clinical Development.

 

[1] A Guide to the Project Management Body of Knowledge (PMBOK Guide), fifth edition, Project Management Institute, 2013.

[2] DMADV: “Define, Measure, Analyze, Design, Verify”

Part 1 – Setting the Stage and Initial Thoughts

 

Well-publicized media stories about recalls and regulatory agency action (483’s, warning letters, and consent decrees), have raised the general public’s and medical device manufacturers’ awareness of the critical need for “product quality” in designing and producing medical devices.  Efforts have been made to educate companies and employees on regulatory expectations and the need for stringent quality in design, development and manufacturing.  Also taught is the need for effective execution of Corrective and Preventative Action (CAPA) processes to correct and prevent reoccurrence of lapses in product quality.  Despite this, failure to demonstrate both efficacy of corrective actions and improvement of product quality remain ongoing issues.

Beyond the damaging stories visible to the outside world, gaps in product quality create waste.  They cause undue work for employees (which also affects morale and retention), unacceptable scrap rates, and repeated failure of verification & validation.  They also cause high customer support costs and lead to expensive “tiger teams” created to fix the problem.   These and other costs all affect the company’s bottom line.

Dealing with regulatory actions creates significant distraction from business goals and activities (while incurring significant costs).  Other costs resulting from poor quality (e.g. increased inspection of incoming components, increased supplier oversight or friction with those suppliers, increases in field support personnel, increased warrantee expenses, etc.) are less often discussed, but nevertheless can easily become existential issues to the organization.

A number of organizational functions typically try to address quality issues: Development, Operations, Supplier Management, Quality, Regulatory, PMO (and others).  These functions and the people within them are well intentioned and hardworking – yet they often work in isolation from each other, and thus have differing perspectives and incentives.  This can, and does, interfere with the design and production of a high-quality product.

Let’s present a vignette[1] to illustrate how this occurs (note: line numbers have been added for use in reference in later articles in this series).

The well-respected company, My Quality Device, Inc. (MQD), has an internally developed idea for a new medical therapy.  An internal champion has proposed a general purpose medical device called the Internal Organ Monitoring and Diagnostic Device “IOMDD.”  After spending several years developing and presenting a nearly final design working prototype for the device, the champion has garnered intense interest and financial support from upper management.  A development team has been assembled and funded, and a required launch date for this new potentially lucrative product has been specified.

MQD has an existing quality system suitable for development of the IOMDD.  The quality system has been well thought out and put in place by personnel highly experienced in the content of the regulations.  The development team has been trained in the quality system, and instructed to adhere strictly to its procedures.

As development of the product proceeds, the development team feels uncomfortable.  As they refine the design at its lowest levels, they end up in disagreement over what the specific output levels and detection sensitivities of some of the subassemblies should be.  They even disagree over what some of the physiological signals are that should be detected.  They spend hours and days in meetings arguing about these points.  Ultimately, the internal champion simply says “do it this way.”  Moreover, as they carefully follow the quality system, the development team is sometimes forced to reach conclusions that seem illogical and against common sense.  They even encounter circumstances in which different aspects of the quality system seem contradictory – and they are not sure what to do.  Upper management is extremely clear regarding schedule and launch date, and is putting a lot of pressure on the development team to meet their contracted deliverable dates.  So, the team marches on, defines criteria for the design as well as specifications and supplier sources for the components, and hands the design off to manufacturing.

Manufacturing quickly discovers that many of the defined components are going to be obsolete within the next year.  They rush to define replacements, or do a large (and expensive) “last buy” to lay in enough stock to last for several years of manufacturing.  They also discover that the prices for components and subassemblies from the vendors are much more than expected.  The suppliers indicate that the design tolerances, in combination with the materials specified, are very tight and difficult to manufacture.  There is therefore a very high scrap rate, which increases the cost of the in-specification parts that can be shipped to MQD.  MQD puts intense pressure on the suppliers to reduce cost of the components.  They do so … but the working relationship sours; communication between MQD and the suppliers becomes curt and infrequent.  Occasionally, bad parts or bad lots of components are received anyway, and are not detected in the receiving process.  In a reaction to this the receiving processes are re-written and more resources are hired to support the additional receiving steps and inspections.  On top of all that, internal manufacturing finds that their scrap rates are very high: “the IOMDD is very difficult to assemble reliably”.  Finally, during a regulatory audit, the auditor finds several critical manufacturing tolerances that are not monitored or controlled.  A 483 with several findings is issued.

In the end, the IOMDD launches later than contracted for, and at a significantly higher cost of goods sold than originally forecast.  At first the IOMDD does not meet its sales quotas.  The customers complain that it is too expensive – so the average selling price is lowered.  Sales pick up somewhat, but MQD starts hearing that the IOMDD performs functions the customers do not need or want, and that it is difficult to make it perform desired functions. Additionally, MQD starts receiving complaints that IOMDD are breaking and they are being returned.  A “help center” is opened to coach customers on how to use the product, and costs for running this center increase alarmingly.  Failed products are returned and replaced at no cost to the customer.  Frequently, the customer simply wants a refund.

After a few years, it is clear that revenues are down, and internal indirect expenses are high. As a result, profit is far below expectations of the shareholders.  Senior management and Finance conclude that they need to maintain the revenue stream of MQD, so 25% of R&D, Regulatory, and Quality personnel are let go.  The next year, the situation is still bad – so another 25% of the personnel of each of these non-revenue producing functions are laid off.  The year after that, the Board of Directors meets and concludes that the senior management of the company is not meeting their expectations, and they are asked to resign.

The year after that MQD files for bankruptcy, and shortly afterward goes out of business.

Is this an unrealistic “perfect storm” of things gone wrong?  Perhaps.  But the story illustrates how many typical company behaviors can lower the quality in products as they are designed.  It also illustrates what can happen when high-quality is not present in marketed product.  In their collective experience, the authors have seen many of these typical, but ultimately damaging, behaviors occur in organizations: you have likely seen some of these behaviors in your own organization.  Sometimes all of these behaviors do indeed occur during a single product’s development.  In those cases, a scenario like that of MQD may occur.

It is our belief that organizations as a whole are missing four key points when it comes to designing and producing high-quality product:

  1. Product quality is an outcome of effective interaction between organizational functions. It is not “owned” by any one function, and cannot be “put into” a product by any one function.
  2. Few organizations have a position that coordinates organizational functions and to which the functions are responsible (other than a VP or SVP, who is typically preoccupied with business-level issues). Because organizational functions have different perspectives and incentives, they are, at best, not coordinated in objectives.  At worst, they operate with conflicting objectives.  Some companies attempt to remedy the situation by implementing broad PLM systems.  But these implementations are often not properly managed, and the effect of a poorly-managed implementation is worse than none at all.  In short:  no one is minding the store.
  3. Many functions operate without clear and intentional consideration of how their decisions affect the company’s bottom line. For example: development can produce designs that are not manufacturable or maintainable; manufacturing can miss opportunities to feed information back to development; and quality or regulatory can create procedures that are overly burdensome and perhaps not even executable.
  4. Quality is, to paraphrase, the ability of a product to safely perform its desired function. Meeting this goal is at the heart of Design Controls as defined by the FDA and ISO.  Failure to clearly define the desired function leads to project scope creep, project delays, failed V&V, and more.  Failure to concisely define and limit the collection of desired product functions leads to “too much”: too much of or too complicated a design; too many design outputs to verify and control; too stringent a set of requirements to demand of a supplier, etc.  “Too much” can – and will – “handcuff” the organization.  This is organizational and operational distraction at its source: with that distraction, something essential will always get “dropped,” creating the quality and cost issues at the heart of this discussion.

Solutions

There appear to be a number of potential actions to correct the situation described above.  However, these actions are not the ones that organizations have previously taken.  After all, if we keep trying the same thing and it does not work, clearly we need to try something different!  The authors are interested in starting this discussion with the intent of helping organizations understand and find solutions to the issues that prevent them from consistently producing high-quality products.

Toward that goal, we will be publishing a series of articles to expand on and illuminate specific topics associated with the organizational issues described above.  One hint of things to come: although the authors are all deeply ensconced in the design and use of “quality systems,” we do not feel that the problematic issues and organizational behaviors can be solved by creating or modifying those quality systems.  It is our belief that the solutions to these problems are already known, and lie in the nexus of recognized best practices in systems engineering, design for six sigma, six sigma, and project management.  Further, the issues run deeper into organizational structure, hiring practices, and training practices.  We hope that as we move forward with this exploration, you will all join in the discussion and contribute your insights.

© 2016 DPMInsight, LLC all right reserved.

(This article, and others in the series, are available and archived at http://www.dpmillc.com/reflections.html.

About the Authors:

Cushing Hamlen

Over 27 years of experience in industry, including 20 years with Medtronic, where he worked and consulted with many organizational functions, including research, systems engineering, product design, process design, manufacturing, and vendor management.  He has also worked with development, regulatory submission, and clinical trials of combination products using the pharma (IND) regulatory pathway.  He has been extensively involved with quality system (FDA and ISO) use and design, and is particularly concerned about effective understanding and use of product requirements and design controls.  He has formally taught elements of systems engineering, design for six sigma, Lean, and six sigma.  Cushing has degrees in chemistry and chemical engineering, is certified as a Project Management Professional, is certified as a Master Black Belt in Lean Sigma, and is the owner/member of DPMInsight, LLC (www.dpmillc.com).

 

Bob Parsons

Over 26 years of experience in leading Quality Assurance, Validation and remediation efforts in FDA regulated Medical Device and Pharmaceutical industry. Experience includes product development life cycle management from initial VOC through New Product Introductions (NPI), sustainable manufacturing, and end of life product management.  Technical expertise in quality system gap assessment, system enhancement, alignment and implementation of all quality elements including design controls, risk management, purchasing controls, change control and post-market surveillance.  Regulatory experience includes; ISO 13485, 9001 and 14971 certification, providing guidance for FDA PMA/510K and CE clearance, designated Management Representative, company representative and lead during FDA and ISO audits, 483 and warning letter resolution with experience working within consent-decree environments.  Bob currently supports various organizations in remediation and compliance projects through Raland Compliance Partners (RCP) www.raland.com .

 

Michael B. Falkow

Michael Falkow is a Quality Specialist with Raland Compliance Partners. He has served as a regulatory compliance and quality assurance executive with multi-facility/international companies and was an FDA Compliance Officer and Senior Investigator/Drug Specialist.  Michael has subject matter expertise for quality and regulatory compliance, quality auditing, quality assurance, quality control, supplier evaluation and certification, and compliance remediation.  He has been approved by FDA as a GMP certifying authority and is qualified to provide Expert Witness testimony for GMPs.

Currently – Adjunct Professor at Mercer County Community College – teaching courses on Clinical Development for Certificate Program in Clinical Research as part of Drexel University’s Masters Degree in Clinical Development.

[1] Disclaimer: the company and product portrayed in this vignette is totally fictitious: it is a compilation of observations made by the authors, and does not represent any company currently or previously in business.