Part 3 – Everything I Know I Learned After College

Consider the following posted requisition for hire:

Senior / Principal Quality Lead

“ … This position is responsible for supplying quality leadership throughout the organization.  The Quality Lead will interact with and influence other cross functional team members, including R&D, manufacturing, regulatory, clinical, etc.

Responsibilities include:

  • representing Quality on new product introduction projects
  • developing, assessing, and validating test methods
  • assuring compliance with internal SOP’s and external regulations
  • leading Root-Cause and CAPA efforts, Risk Management, Risk Reviews, Material Review Boards, and Change Control needs
  • Representing the Customer, identifying their needs, and reducing customer complaints
  • Driving and fostering a quality environment and mindset through the business: provides training, guidance, leadership, and expertise to the R&D, quality, and manufacturing organizations.

Skills needed:

Competency in statistical techniques: Gauge R&R, SPC, process capability, sampling plans and sample size determination, DOE, ANOVA, regression, Lean

  • Design FMEA, process FMEA
  • Test method and protocol development for verification and validation studies, and evaluation of test results
  • Demonstrated ability to drive change
  • Strong analytical and problem solving skills
  • Ability to multi-task


  • Practical knowledge of FDA Quality Systems Regulations, ISO 13485 (medical device quality management systems), ISO 14971 (Risk Management), IEC 62366 (usability engineering)
  • Previous experience working in an FDA regulated environment
  • Bachelor’s degree in an engineering discipline or related STEM field, and 1-5 years experience.

… “

Ok – this is not an actual requisition duplicated verbatim.  But it is a synthesis of real job postings this past year, all of them seeking hires in the medical device and combination device arena.  The “posting” is representative of a great many of the advertised job descriptions and criteria against which candidates are evaluated.

Let’s first look at the needs that appear to drive this job description.  In a previous article (“A Quality System is Not Enough”) we indicated that the best practices of systems engineering, Six Sigma, and project management all supply the tools, and practices using those tools, to support development of high-quality products.  The job description above contains the responsibility to “represent the customer and to identify their needs”.  These activities of identifying stakeholders and evaluating their needs come directly out of project management, Six Sigma, and systems engineering.  This is also the critical first step in the FDA and ISO design controls, which call for identifying what it is that the new product needs to do.  These needs statements are the “Design Inputs (DI).”  It makes sense that for new product development and design…this skill is needed.

Next in the requisition are a series of needs around verification and validation, along with associated test method development and evaluation of results.  First, we note that the activities of verification and validation are integral to systems engineering and project management – though they are often confused.  A useful graphic to illustrate the difference between verification and validation is the “V diagram” from systems engineering:

Follow the arrows from the top left: first define user needs, use those to define system requirements, then sub system design requirements, and so on.  After you do the design work to the lowest levels, work your way up the right-hand side: verify that what you actually made meets the component, sub-system, and system design requirements.

As we will discuss in a later article, the design requirements that are “verified” are (typically) posed in technical language: a dimension is 5 mm, a weight is 12.3g, a volume is 7.8 ml, and so on.  Some non-technical language may be present at the “higher” system levels.  At “lower” levels the language becomes more specifically technical … but at all levels the requirements are always measurable.  The needed verifications can be done in the lab with (as defined by the test protocol) micrometers, balances, MTS machines, etc.  They can also be accomplished by: inspection of drawings; inspection of the physical product; demonstration of function; or analysis via mathematical models or other calculations.  Verification can, and should, be an ongoing activity: it takes place throughout the design process – not just as a single final test.

The language used to define User Needs, however, is a whole different thing.  A user will not come out and say “I want this widget to be 5.6 cm in diameter with a maximum weight of 0.23 kg.”  They say something more like, “I want to be able to comfortably hold this in my hand.,” or “I want to be able to turn the knob in an operating room while wearing sterile gloves.”  A design is validated by giving the user the final product (or a suitable substitute) and asking, “Does this do what you need it to do?”  (The execution of usability engineering is more rigorous than the previous statement, but hopefully the gist of the point comes through).

So – validation is about confirming that the design output (DO), or the device itself as actually built, meets the user’s needs and expectations.  Verification is different: it does not involve human factors, rather it demonstrates that each product specification, at each “level” of the design, is met.  Verification requires that laboratory-based tests, inspection, or analyses, be developed, executed, and interpreted.  It is perhaps crucial for the business needs of an organization to take note that a specification could be developed that passes verification, but fails validation if the design fails to meet the user’s needs.

Returning to our requisition, this position clearly needs skills in test method development.  The methodologies used to perform these activities are mostly statistical methods that are integral to the body of knowledge contained within Six Sigma (gauge R&R, sampling plans and sample size determination, capability analysis, regression, etc.).  Less obvious, but still present in the job description, are tools that assist in effectively determining the ability of a design to meet the stakeholders’ needs (DOE, ANOVA, regression, root-cause analysis, and FMEA).  Again, the use of all of these methodologies makes sense in satisfying the regulatory “requirements.” All of these methodologies are contained in the bodies of knowledge of Six Sigma and systems engineering.

Continuing the evaluation of the requisition, we see a series of responsibilities and skills often considered “soft” skills: “representing quality throughout the organization” (i.e. influence management), “fostering a quality mindset throughout the organization” (i.e. change management), “providing training, guidance, leadership” (i.e. instruction and mentorship), “participating in Material Review Boards” and “change control needs” (i.e. critical thinking).  These skills and methodologies are not taught by systems engineering, Six Sigma, or project management (although project management does discuss and foster them under “develop” and “manage” the project team).  Neither are these skills taught in an engineering education: they are learned over time through experience, and occasionally through hard-learned lessons.  For some, these skills are never learned.

Not explicitly stated in the job description, but definitely present, is a need for something we seldom talk about: confidence, strength, maturity of personality, and a willingness to engage in productive conflict.  In sum: an ability to challenge an organization, its constituent functions, and its leadership, to think differently and to consider options (whether design or process options) other than those with which one is familiar or would otherwise prefer.  This is the ability to see through the fog and identify the core element to execute – and to teach others how to do the same.  In short – to lead.

These skills are definitely not taught in the engineering disciplines.  The authors would argue that the ability to play a leadership role comes from hard-learned experience and time on the job.  Six Sigma and Design for Six Sigma actually teach methodologies that support an individual executing a leadership role in a team.  Although many of the decision-making tools taught in Six Sigma and Design for Six Sigma are taught as “engineering” tools … in reality, when effectively applied, they are really methodologies to lead people to consensus in a team or organization.  Examples of such tools include: prioritization/selection matrices, RACI matrices, affinity diagrams, “house of quality” incidence matrices, process flow charts, fishbone/Ishikawa diagrams, 5-why techniques, FMEA techniques, Pugh concept selection matrices, the analytic hierarchy process, and many more.  Think about it: even graphical display of data (Pareto charts, histograms, scatter plots, regression analysis, etc.) and statistical methodologies, with their agreed-upon significance levels, p-values, etc. are methodologies for building consensus in the face of potentially confusing information.

Finally, in the job description there are requirements of practical knowledge of regulatory and ISO requirements, combined with previous experience working in an FDA regulated environment.

All of this is asked for with 1-5 years of experience after leaving school.  We are not distorting the experience called for – check the job requisitions that are now being posted.

This example posting, and many like it, is for a “Quality Lead.”  What it is really describing however, are many of the activities and skill sets involved in systems engineering, project management, and Six Sigma.

Here is the critical point: the skills, methodologies, and experiences central to designing and manufacturing high-quality product, are NOT taught, to any significant level, in either the undergraduate or graduate engineering curricula.  Also not taught in those curricula are the skills needed to do so in a manner compliant to regulatory expectations.  Practical knowledge and effective execution of this set of skills is only developed through practical work experience.

Everything I know to effectively navigate and execute engineering product development and produce high-quality product I learned after college.

One of the authors has had opportunity to seek input on the following question: “How much experience is needed to effectively execute the skill set and responsibilities required by this job description, or ones similar to it?”  The question was posed to people in the medical device industry with a wide range of experience and job level – varying from Senior/Principal engineer up through the Director and VP level.  The responses were quite uniform:

At least 15 years.

Yet, almost uniformly, postings look for people to fill this role who have less than five years of experience.  Anyone with more experience is automatically considered “over qualified” and not even considered.

Think about it.  In our hiring practices we are specifically excluding from these critical roles the very people who have the practical experience and skill sets needed to produce high-quality product.  Instead, we expect those skills from people right out of college, where the skills are not taught, and with far too little real-world experience to have acquired and therefore to use them.

To make things worse, organizations frequently look askance at any or all of the disciplines of project management, Six Sigma, or systems engineering.  Why?  The reasons are complicated.  Sometimes these disciplines try to create an “empire” within an organization, try to impose too much overhead or constraints on an organization, and truly do slow down organizational execution.  Sometimes the organization rejects the rigor called for because they “know what their design is and just want to go ahead and build it” (a perspective which is, by the way, antithetical to the Design Controls requirements – so such an organization should step very carefully).  Sometimes the organization is just resistant to change and does not embrace the “new” disciplines (regardless of the fact that for decades these have long been world-wide accepted best practices!).

Whatever the reason, organizationally we often do not acknowledge the critical learning coming from project management, Six Sigma, and systems engineering.  Because of this we also do not seek to sustain and propagate them within an organization.

What can we do to change this mismatch between the true quality goals of our organizations and our expectations of who we will hire to achieve those goals?

As we said earlier, the skills to achieve those quality goals reside in the disciplines of systems engineering, project management, and Six Sigma.  The authors do not believe there is a need to create each of these functions in their totality within an organization.  Many organizations, especially small ones, do not need or cannot afford that.  Also, in any sized organization there is real danger if any of those disciplines act in a way that their sole purpose is to promote and sustain themselves, rather than act in a way that promotes the overall health of the organization (this is a topic of a later article).

Rather, we are talking about recognizing, embracing, and fostering the individual teachings and practices embodied by these disciplines.  There is great overlap between the disciplines of systems engineering, project management, and Six Sigma.  But, with regard to producing high-quality product, they also each have unique benefits to the overall objective.  Systems engineering and project management supply focus and structure around identifying the stakeholders, clearly documenting their needs, and identifying how the system can be defined and broken down to sub systems in order to meet those needs.  Design for Six Sigma focuses on identifying a best design to meet stakeholder needs and requirements (requirements are not the same as design – also a topic of a later article). Six Sigma and design for Six Sigma together supply the statistical/analytical horsepower to: quantify the capabilities of proposed design and manufacturing processes to satisfy the requirements; to distinguish between competing design concepts (and in so doing supply the very mechanisms needed to perform design verification and validation); and to monitor and control designs and manufacturing processes over time through control charting methodologies.  Lean, although we have not focused on this practice above, supplies the process flow charting and related graphical and visual methodologies to build team consensus on the understanding of exactly what our processes are (and thus how they can be improved) and how they are performing.

We state again, the clear majority of these methodologies, of which there are literally hundreds, are not taught during college-level training.  We are not going to get them from candidates with just a few years of experience.

So, how to proceed?  At a minimum, seek and hire candidates who are certified by a reputable body at a black-belt level (or equivalent) in these disciplines.   There is a growing trend in Universities and Community Colleges to offer certification training through their Continuing Education programs.  Many companies are taking advantage of this to send selected employees for such certification. The certification gives some level of confidence that these candidates carry with them an appropriate understanding, knowledge and practice in these methodologies.  Do not hire them to create a “systems engineering group … or a “project management group” … or a “Six Sigma process improvement group.”  Hire them with a mandate to use, and to demand the use of, the appropriate methodologies at the appropriate time and place.  Put them in a position of organizational influence and oversight so that they can influence, model, and coach others in the use of these methodologies.

For somewhat larger organizations, we recommend identifying individuals who show interest in and passion for these methodologies.  Sponsor them (in terms of both funding and time) to train to a black belt level (or equivalent) through a reputable outside organization.  Again, do not use them to create additional functions, but use them to identify and use the appropriate methodologies at the right time and place.  Above all, support their activities within the organization.

The largest organizations should seriously consider institutionalizing the learning and internal teaching of these skill sets.  Where this has been done, the use of these methodologies has born great fruit.  The practice of “internalization” of these engineering methodologies is something that we have lost over the years.  The authors believe this loss is a significant cause of the discordance between implementation of “quality systems” as a compliance-focused mechanism versus an organization’s use of fundamental and powerful engineering methodologies to produce high-quality product.

In the first article these authors posted, we made the point that product quality is an outcome of effective interaction between organizational functions, and cannot be “owned” by any one function.  For an organization to effectively wield the methodologies of systems engineering, project management, design for Six Sigma, and Six Sigma, the appropriate “pieces” (i.e. specific methodologies) must be used at the correct time.  Some of them need to be used up front during evaluation of the needs of the stakeholders, others during translation of those needs into product requirements, others during the development of a design that satisfies product requirements, others when the design (and associated manufacturing processes) are verified and/or validated, and still others during manufacturing and post-market surveillance.

We need to empower the individual functions in the organization to use these engineering methodologies appropriately.  A quality system by itself, because it cannot foresee and define reactions and procedures for all eventualities, cannot therefore define what is “appropriate” to a sufficiently fine level of detail.

Rather, the practices an organization puts in to place on a daily basis, used within the framework of the quality system, but not defined by the quality system, define how the quality system is “used.”  It is the discrete engineering practices that ultimately lead to high-quality product.

© DPM Insight, LLC 2017 All Rights Reserved

About the Authors:

Cushing Hamlen

Over 27 years of experience in industry, including 20 years with Medtronic, where he worked and consulted with many organizational functions, including research, systems engineering, product design, process design, manufacturing, and vendor management.  He has also worked with development, regulatory submission, and clinical trials of combination products using the pharma (IND) regulatory pathway.  He has been extensively involved with quality system (FDA and ISO) use and design, and is particularly concerned about effective understanding and use of product requirements and design controls.  He has formally taught elements of systems engineering, design for Six Sigma, Lean, and Six Sigma.  Cushing has degrees in chemistry and chemical engineering, is certified as a Project Management Professional, is certified as a Master Black Belt in Lean Sigma, and is the owner/member of DPMInsight, LLC (


Bob Parsons

Over 26 years of experience in leading Quality Assurance, Validation and remediation efforts in FDA regulated Medical Device and Pharmaceutical industry. Experience includes product development life cycle management from initial VOC through New Product Introductions (NPI), sustainable manufacturing, and end of life product management.  Technical expertise in quality system gap assessment, system enhancement, alignment and implementation of all quality elements including design controls, risk management, purchasing controls, change control and post-market surveillance.  Regulatory experience includes; ISO 13485, 9001 and 14971 certification, providing guidance for FDA PMA/510K and CE clearance, designated Management Representative, company representative and lead during FDA and ISO audits, 483 and warning letter resolution with experience working within consent-decree environments.  Bob currently supports various organizations in remediation and compliance projects through Raland Compliance Partners (RCP) .


Michael B. Falkow

Michael Falkow is a Quality Specialist with Raland Compliance Partners. He has served as a regulatory compliance and quality assurance executive with multi-facility/international companies and was an FDA Compliance Officer and Senior Investigator/Drug Specialist.  Michael has subject matter expertise for quality and regulatory compliance, quality auditing, quality assurance, quality control, supplier evaluation and certification, and compliance remediation.  He has been approved by FDA as a GMP certifying authority and is qualified to provide Expert Witness testimony for GMPs.

Currently – Adjunct Professor at Mercer County Community College – teaching courses on Clinical Development for Certificate Program in Clinical Research as part of Drexel University’s Masters Degree in Clinical Development.

Leave a Reply

Your email address will not be published. Required fields are marked *