ASQ Certification Exam Pass Rates - Per QCI

[holly21]

I recently attended a local ASQ meeting where the guest speaker was Bill Wortman of the Quality Council of Indiana. He shared some information concerning certification exam pass rates that I thought was interesting. I was surprised at the variance from exam to exam.

I know that ASQ doesn’t typically publish this information, so after checking with the moderator, I’m happy to post and share. Below I’ve listed the certification, the average pass rate, and the time period over which the average is calculated.


CMI 58% 3/95 – 3/05
CQT 50% 3/95 – 3/05
CQM 62% 3/95 – 3/05
CRE 51% 3/95 – 3/05
CSSBB 76% 10/01 – 3/05
CQE 43% 12/94 – 12/04
CQA 70% 6/98 – 12/04
CSQE 60% 6/99 – 12/04
CQIA 84% 12/00 – 12/04
CCT 72% 6/03 – 12/04

[Jim Wynne]

Using ASQ's unique methods of data interpretation, I think it's safe to say that all of those who passed must now have better jobs, and CSSBBs must be much smarter than CQEs.

[Wes Bucey]

Originally Posted by holly21

I recently attended a local ASQ meeting where the guest speaker was Bill Wortman of the Quality Council of Indiana. He shared some information concerning certification exam pass rates that I thought was interesting. I was surprised at the variance from exam to exam.

I know that ASQ doesn’t typically publish this information, so after checking with the moderator, I’m happy to post and share. Below I’ve listed the certification, the average pass rate, and the time period over which the average is calculated.


CMI 58% 3/95 – 3/05
CQT 50% 3/95 – 3/05
CQM 62% 3/95 – 3/05
CRE 51% 3/95 – 3/05
CSSBB 76% 10/01 – 3/05
CQE 43% 12/94 – 12/04
CQA 70% 6/98 – 12/04
CSQE 60% 6/99 – 12/04
CQIA 84% 12/00 – 12/04
CCT 72% 6/03 – 12/04

There are lots of reasons folks don't pass the ASQ certification exams.

I regret that ASQ does not apparently explore the "why" behind the apparently high failure rates of some of the exams.

These numbers "seem" accurate to me.

Govind has been on a crusade of late in trying to disseminate information about the exams to help folks study "smarter" in preparing for the exam.

It is important to know the exams are not graded on a curve, but on an absolute scale. ASQ does sometimes give everyone credit for questions which are found to be "ambiguous" or "wrong" after the exam has been taken.

Anecdotally, I hear most folks just were not prepared for the relentless effort needed to complete the test within the time period. Rehearsal with timed tests is paramount, regardless of how much knowledge you have of the BOK.

A second problem is the number of folks dealing with English as a second language. (In many cases, even regional "English" from Alabama, Canada, Great Britain, Australia, India can foment problems) - this springs back to time problems in that the effort of assimilating the nuances of English used in the exam takes time.

I would guess there are many folks who are completely naive in the way they approach the test. I recall seeing posts in ASQ Forums similar to:
"I am sitting for the CQE next month. Which books should I study?"

Folks - even I would hesitate to wait until a month before an exam to start asking about books to study. These tests are NOT like the SAT - they don't test your native intelligence, they test your knowledge and ability to apply the Body of Knowledge. Think of most of them as the final exam after a particularly rigorous graduate level course in school. If a school term lasts three months, you should probably expect to be reviewing and studying for the equivalent of that same school term. If you aren't conversant with the BOK, the fact these are open book tests won't help you one bit.

I have a lot of complaints about the certification system, but the fact so many folks fail a test is not in my list of complaints.

[Lynda Young]

I have a problem with the percentages that are listed. I have written both the CQE and CQM in each I obtained a 70 + on both. The rating I received was 540 with 550 being the passing grade on their "bell curve". Am I missing something?

[ralphsulser]

Wes said:Anecdotally, I hear most folks just were not prepared for the relentless effort needed to complete the test within the time period. Rehearsal with timed tests is paramount, regardless of how much knowledge you have of the BOK.

This is absolutely correct-actually the discipline one must be prepared to practice, practice, practice.

When I passed the CQE on the first time that what I did, and the info was that only 50% pass the first time. This was Dec. 1986

[Lynda Young]

Originally Posted by ralphsulser

Wes said:Anecdotally, I hear most folks just were not prepared for the relentless effort needed to complete the test within the time period. Rehearsal with timed tests is paramount, regardless of how much knowledge you have of the BOK.

This is absolutely correct-actually the discipline one must be prepared to practice, practice, practice.

When I passed the CQE on the first time that what I did, and the info was that only 50% pass the first time. This was Dec. 1986

I agree that that can be a problem, I was ready for the for the efford and did not have a problem with the timing in either exam, I finished with extra time. It is the use of the "Bell curve" that I feel is wrong.

[Wes Bucey]

The issue of curves, cut scores, etc. is a frequent topic of discussion among ASQ members. Here are some excerpts from recent posts in the ASQ Forums moderated by our own Govind which address this topic. There was some "misinformation" that was answered by Sally Harthrun, the lady who manages certification for ASQ - her comment is in blue:

Gregory Gay

Posts: 60

Re: Letter: Study as if Exams Are Closed Book
Posted: Jan 18, 2005 5:55 PM Reply


Just a little about cut scores
There is not a minimum on any section
if you do really good on one section,
you can do really bad on another
it is the total score that counts
no one question is any more important than any other question. questions are developed by sections of the bok and there maybe more questions in one section than in another.

an exam cycle is about 5 years and the cut score is developed on the first exam in that cycle. each exam after the first is adjusted up or down depending on if any questions on the new exams are in error. There is a mini exam made up of part of the first exam in each of the following exams in the cycle. this mini exam is scored to equate the first and new exam to help determine if the cut scores remains stable. It is quite a process, but not that hard.

It is worth note that there is a cut score not a passing rate or a grade on the curve. If 100 percent of the people taking the exam reach the cut score they all pass. Although it is a "secret" what the cut score is, it is not hard to determine what the cut score is when you know a few people that have taken the exam and some passed and some failed.

We are talking about a certification exam and it does not make much sense to certify people passing at 60 percent. And although I would like to see high scores it just does not make sense to fail a certification attempt at 89 percent. Even at 80 percent you are missing 20 questions per 100 questions asked. That seem like a lot when you are working on zero defects, and 3 parts per million in the day job.

I tell people to get through 80 percent correct. It is a good goal and study guide. Cut score could be higher or lower than 80 percent, however if you are studying at least the 80 percent level in all sections your strong sections will carry you through the exam.

What I tried to do was study at the 95 percent level in my weakest areas (why study what you are good at), and also study the hardest in the areas that most people miss. Your score will be higher than theirs. So although there is not a curve, you should still pass. People have a lot trouble with statistics. I do also, so I studied that section the hardest to try to get 95 percent in stats.

What is really fun after you have obtained a certification is helping develop the new exams. I think I learned more on the subject matter creating exams than preparing to take them. Each 5 years the exam cycle starts over and ASQ is always looking for certified individuals to keep the exams current.

What is really crazy, I know of an individual that takes CQE by recert exam not points. And has done it for a few 3 year cycles. I think he like pain. (willy)

Best of luck in your certification efforts. It is not as hard as it might seem if the prep is there.

Gregory


indeed, is a minimum pass
> > mark for the 10 sections of the BOK.
> > Questions pertaining to each section must be
> passed
> > at a 75% rate. Exams are not separated into
> sections
> > but questions can be attributed to one section or
> > more.
>
> Govind is correct about the "cut score". What
> qualifies as a passing score is determined AFTER the
> exams are scored each time based on a VERY closely
> guarded secret formula. Because the questions change
> for each exam there are questions which are thrown
> out after the scoring is done when the scorers
> realize the question was a bad question or there were
> multiple right answers or the right answer was not
> one of the possible answers. They also evaluate the
> score distribution to determine if the current exam
> was more difficult or easier than normal.
> Incidentally, the "cut score" practice is a commonly
> y accepted practice in the ETS (Educational Testing
> Service) world.


@@@@@@@@@@@@@@@@@@@@@

Sally Harthun, the Manager of Certification

Posts: 8

Re: Letter: Study as if Exams Are Closed Book
Posted: Jan 19, 2005 10:27 AM Reply


Hi Bill - I would be happy to answer this question. For all of our exams, it is an overall score that must be obtained. You can do very poorly in one section of the BOK, but if you do well enough in the other areas to obtaining the passing grade you pass. We do not limit the number of people who pass by section or any other method. Anyone who obtains the passing score passes - it is just that simple. I know that people do, indeed, believe there is some type of "limit" that we place on the number of people who pass, but that simply is not true. We tried to address that rumor when we put together the "Ten Myths of Certification" that is listed on the web site. Thanks for allowing me to address this once again.

The ten myths:

Originally Posted by http://www.asq.org/cert/pdf/TopTenMythsOfCert.pdf

Top 10 Myths of Certification
Myth: If an answer is obvious it must be a trick question.
FACT: Just because an answer is obvious to you doesn’t mean it is obvious to everyone. Don’t let the fact that this is a test question get in the way of your knowledge. Answer it and move on. Don’t read more into the question in an effort to make it harder.

Myth: Guessing wrong can hurt your score more than leaving an answer blank.
FACT: There is no penalty for guessing, and you have a 25% chance of getting it right. Although some tests use “formula scoring” methods, ASQ certifications do not. You get one point for each question you answer correctly and zero points for those you get wrong or leave blank.

Myth: The passing score for all ASQ exams is 70%.
FACT: The passing score for each ASQ exam is established as a minimum performance standard during the cut-score process.

Myth: Taking a section refresher course or buying ASQ exam prep material is a sure way to pass.
FACT: Section refresher courses and the self-directed products are excellent ways to prepare for the examinations, but using them does not guarantee that you will pass. Refresher courses are meant to renew your knowledge, not to instruct you in areas that aren’t familiar to you. Questions from the self-directed products will assist you in becoming familiar with how to answer certain questions, but they are not the same questions that you will see on the exams. Individual study is also a critical element for success.

Myth: If you do poorly on one area of the body of knowledge, you automatically fail the test.
FACT: Your total score on the examination determines whether you pass or fail, not your score on any one portion of the test. Even in the certified quality manager’s exam it is possible to pass if you do poorly on the constructed response portion, as long as your overall score is at or above the passing grade.

Myth: ASQ limits the number of people who pass.
FACT: Anyone who meets or exceeds the passing score (cut point) passes the examination. ASQ does not set a passing rate.

Myth: The grading of the constructed response portion of the certified quality manager exam is very subjective.
FACT: The constructed response portion of the exam is designed to test the candidate’s ability to respond to realworld situations. The responses are scored by certified quality managers who have been trained in the evaluation techniques used for the scoring process. In addition, all the scorers judge the papers against a standard of performance that is specific to each essay question. That standard is neither arbitrary nor subjective, but is developed on the basis of sound quality practices, as described and prescribed in major textbooks in the field of quality. The essay questions are pretested on a group of certified quality managers, so the development of the question includes a reality check to make sure that the committee’s expectation of performance matches actual responses.

Myth: It takes a long time to receive exam results.
FACT: ASQ works very hard to provide exam results as quickly as possible and is very aware that the examinees are anxious to learn whether they have passed or failed. As a direct result of bringing the exam development in house, ASQ has recently reduced the exam turnaround time from eight weeks to two weeks, and to three and one-half weeks for the quality manager results. There are many steps that have to be completed, verified, and checked prior to sending out the results. The answer sheets are sent back to ASQ headquarters and scanned, and statistics must be run and reviewed by the test development staff and appropriate volunteers. The results are then scored, verified, and uploaded to a computer before any result letters can be generated. ASQ is continuously working to reduce this cycle time.

Myth: Test questions are deliberately tricky.
FACT: ASQ goes through an extensive process to ensure that examination questions are as accurate, clear, and concise as possible.

Myth: I can’t learn from my mistakes if I don’t get my scored test back.
FACT: Because of its policy to reuse examination questions, ASQ cannot release copies of the examinations. Releasing tests would give the retake applicants an unfair advantage over candidates taking the examinations for the first time. The integrity of the examination process is of paramount importance to ASQ. Besides, it would not support the underlying premise of the certification program for candidates to just study the questions they got wrong, as it would not ensure that they would understand the material any better. It is more appropriate for the retake candidates to use the diagnostic information to identify the areas where they are weak and improve their knowledge in those areas.

[Govind]

Pass rate:
First of all, I would like to thank Holly taking the initiative to post the information. One thing you may want to check with Mr.Bill Wortman is if he also included the results of exams conducted during Special administration like Conferences. Very few people sit in these exam (mostly single digits/exam). Even if one or 2 people fail, or Pass, this % will skew the overall % on either end. Including these special administration data may not be a good idea. Iam sure Mr.Wortman would have taken care of this, as he is very familiar with various certifications for many years.

Scores and process:
ASQ cut off score for various exams have always come up. It is a closely guarded secret. It is not known outside the Certification offerings. The process of obtaining such cut score (which is described in any certification brochure Page 5- Grading Process).

ASQ takes very careful review of candidates who sit for Cut score process so that they do not skew the score and/or add bias. Equaters are created to compare the difficulty between exam-to-exam and statistically analyzed. Cut score adjustments are made if indeed one is different than other based on performances of candidate groups.

ASQ takes a look at every exam question based on historical performance and see if the discriminator is significant enough between top quintile to the bottom quintile. Poorly performing questions are critiqued and reviewed at Item review.

Exam preparation:
Wes, thanks for pointing the right direction from previous discussions and ASQ Brochure. Very good inputs on your post regarding exam failures. Many exam failures can be attributed to lack of preparation and practice. Certification exam require between an average 125~150 hours of good preparation. Wes is right. Trend look like, many wait until the last minute to even enquire about references and when they get too close to exam, either they reschedule or try their luck.

Almost every week, someone call me or send mail asking about certification preparation. Sometimes I find that people try to register in exams not very appropriate for them. Right selection of exam based on their job requirement, experience, career progression is also very important. This also can cause failure, as they may be ready to face the exam with mere self-study or refresher courses. I have provided them with suggestions with Pros-Cons.

Language also is a probably a reason when it comes to non-native speakers. But the impact could be mainly on Certification like CQMgr and CQA where you can expect more of non-mathematical, non-statistical questions.

BTW, Good inputs from all the contributors. Please continue with our thoughts.

Regards,
Govind.