Cost of Quality (CoQ) generally is between 15-40% of Sales

Marc

Hunkered Down for the Duration with a Mask on...
Staff member
Admin
#21
And another voice:

Subject: Re: Defining "cost of quality" /Pfrang/Kohn
Date: Mon, 7 Jun 1999 11:36:37 -0600
From: ISO Standards Discussion

From: Brian Charles Kohn
Subject: RE: Defining "cost of quality" /Pfrang/Kohn

> From: (Doug Pfrang)
> Can anyone explain to me what precisely is meant by the expression, "Cost of
> Quality?"

There are a lot of perspectives, and even whole books on the subject.

> 1. "Cost of Quality" suggests that "quality" is something separable from
> other activities in the plant, as in "Cost of Labor," "Cost of Electricity,"
> or "Cost of Raw Materials." I do not see how "quality" can be separated
> like this, since it seems to me more an inherent characteristic of
> everything and everyone in the facility.

The way I understand Cost of Quality is in the manner that it can indeed be
separated out. CoQ is typically divided into four aspects:

1) Cost of Appraisal
2) Cost of Prevention
3) Cost of Internal Failure
4) Cost of External Failure

The Cost of Appraisal is the cost of all those activities within your operation strictly devoted to inspecting, measuring or otherwise appraising whether the product or service you produce meets the requirements. Imagine the division between work done: Sometimes you'll be screwing parts together; sometimes you're looking to make sure they were screwed together.

The Cost of Prevention is the costs associated with evaluating and implementing processes and systems to find better, more robust ways of doing things. Think about it this way: Sometimes you have people who are thinking about how to better screw things together so that the folks who are looking to make sure they were screwed together right don't find as many that weren't, and more importantly the customer doesn't find as many that weren't.

The Cost of Internal Failure is the cost associated, typically, with the materials and work that went into building something that you end up having to scrap because it doesn't meet specifications.

The Cost of External Failure are the costs associated with providing warranty or goodwill service, and perhaps even more important, the cost associated with having a customer experience a failure of your product or service.

> The expression "cost of quality"
> suggests that "quality" is something distinct and external to everything
> else, which seems nonsensical.

Quality Professionals, like myself, love to say that, because it keeps myopic managers from focusing on the very measurable costs and ignoring the almost-unmeasureable benefits of quality control, quality assurance and quality improvement. However, the reality is that there are measurable costs associated with these activities, and if the cost is higher than the benefit (which, admittedly, we can't possibly measure accurately in advance) then good business sense would dictate that we re-evaluate the quality program and make it more effective.

> 2. "Cost of Quality" suggests that "quality" has this downside called
> "cost," when it seems to me that "cost" is more likely to result from a
> "lack of quality."

The key word you use here is "seem." What you really need to do is "know." As I've indicated earlier, it is very hard to know that more cost is likely to result from lack of quality *activities*. (Remember, in the case of Cost of Appraisal and Cost of Prevention, we are talking about the cost of activities, not of poor quality.)

> To me, the company first decides what level of
> performance, reliability and capability (i.e., "quality") it is going to
> build into its products, processes and services, and then it decides how its
> facility and processes are going to achieve those goals. Because the firm,
> presumably, incurs whatever expenses result from the initial marketing
> decisions it made regarding the placement of its products, processes and
> services, the resulting expenses are therefore driven by the company's
> strategy, not by "quality." Accordingly, "cost of quality" does not appear
> to have any meaning to me in this context.

Perhaps the words "Cost of Quality" aren't precise, but they have a very valid place in the discussion. Since the words were coined over a decade ago, it is a bit late to try to change them now.

> 3. Now, what might have meaning is when a company is producing a product
> that has a certain level of performance, reliability and capability (i.e.,
> "quality"), and the company decides it wants to increase this level of
> "quality." Perhaps it is going to redesign its product, or buy better
> components and raw materials, or improve its facility with new production
> equipment, or restructure its processes. Of course, these changes cost
> money, but they still do not seem to be a "cost of quality," because they
> are an INVESTMENT (presumably made after some ROI calculation to show that
> they are justified) that the company makes in the hope of achieving some
> greater financial reward in the future (e.g., greater selling price, fewer
> returns, lower warranty costs, etc.). Therefore, "cost of quality" still
> does not seem to have any obvious meaning in this context either -- the
> company is making a short-term investment to achieve a higher long-term
> return.

The question is what is the investment for? If it is to reduce the cost of poor quality (i.e., the Cost of Internal Failure or the Cost of External Failure) then the investment is indeed a Cost of Quality, specifically the Cost of Prevention. Investments are indeed costs of a sort. The ROI calculation you mention is definitely the right way to go.

However, as I mentioned earlier, sometimes the benefits are unquantifiable. How much benefit do you derive from 10% less dissatisfied customers? No matter how much market research you do, you never can really know what the impact will be; heck, sometimes you go into an "investment" like this with only a *forecast* of the physical improvement you expect to yield. There is often a "technology risk" associated with such investments; they sometimes don't pan out at all.

> 4. Another scenario is when a company is operating under a quality control
> scheme (rather than a quality assurance scheme), where it is trying to
> "inspect quality into" its products. To such a company, "increasing
> quality" might mean hiring more inspectors to create more "quality." Of
> course, this costs money, but this is still not a "cost of quality,"
> because, in fact, it is simply a higher cost of labor that has been traded
> off (presumably) for an increase in production or a reduction in the number
> of returns. Again, it is merely a short-term investment to achieve a higher
> long-term return.

Again, you're getting mired in the words. It is surely a Cost of Quality to add more inspectors. It increases the Cost of Appraisal. You're simply pointing out that it is, too, an investment. Fine. You have to break some eggs to make an omelet, but you STILL have to break the eggs, i.e., spend the money--incur the cost.

> 5. Finally, if "cost of quality" is to be quantified in terms of dollars
> and treated as an expense, how does this calculation take into account the
> financial gains that result from the quality initiatives?

This is an accounting question, and where this issue gets really muddy. I won't even attempt to go into it without getting my CPA first... <grin>

> Again, it seems
> absurd to allocate "cost" to something with the ambiguous name of "quality,"
> but then to allocate the revenue somewhere else. Looking at "cost" in
> isolation just seems distorted.

This is almost always what is done. At the companies I've worked with, the costs associated with making the kinds of investments you're talking about are often represented as expenses, increased operating costs, etc. The benefits can only very rarely be directly attributed to specific investments. Sad.

Brian

****************
EDITOR'S NOTE: Shortened url so page does not wrap to far out. Software html error.


[This message has been edited by Marc Smith (edited 20 November 2000).]
 
Elsmar Forum Sponsor

Marc

Hunkered Down for the Duration with a Mask on...
Staff member
Admin
#22
Subject: Re: Q: Defining "cost of quality" /Pfrang
Date: Tue, 8 Jun 1999 10:10:41 -0600
From: ISO Standards Discussion

From: (Doug Pfrang)
Subject: Re: Q: Defining "cost of quality" /Pfrang

Thank you all for your replies and for giving me some resources to investigate. I have followed up with some of the references that were suggested and found them to give excellent definitions of cost of quality.

In fact, you were so good at answering my original question, that I now have a more perplexing follow-up question. I understand the trade-off between "cost of control" and "cost of failure," and I see how they can be used to compute a "cost of quality" metric, but now what I don't understand is why it makes any sense to compute these metrics, because I don't see how anyone can actually use these metrics to make management decisions.

The difficulty I anticipate is as follows: Let's say I want to minimize my "cost of quality." The question is, how do I find the minimum? When I go into my factory and I tally up my actual COQ -- i.e., when I compute my "cost of control" and "cost of failure," and add them together -- I get only one number, which means I only have a single data point on my COQ curve. How do I plot the rest of the curve, or find its minimum, using only one data point? Moreover, if I look for other data points, such as by computing my COQ on different dates over an extended period of time, what can I do with these data points? Even if they show a trend, indicating that I am going in the direction of higher or lower COQ, does this really tell me anything useful? Let's say the trend is downward, which would seem to be a good thing because it seems to indicate I am heading in the direction of minimizing my COQ. But maybe the trend is downward because my shipments are down (and thus I have fewer inspections and fewer failures), I have laid-off half my employees (and therefore I have less training), or I have stopped introducing new products (and thus I have less quality system activity and fewer customers calling with questions). Even if I compute these metrics as a percentage of sales, the COQ metric does not necessarily tell me anything useful because the changes in this metric over time might be the result of factors unrelated to the company's quality initiatives. Such factors could include changes in the economy (cyclical changes, inflation, etc.), general improvements in technology (i.e., better price/performance), loss of key personnel, etc. Moreover, notice how many different things can affect the COQ equation: the calculation responds to so many different inputs that a change in the aggregate number doesn't say anything about where those changes came from. Thus, if it begins moving higher, then even if I think I have a problem I would not know where that problem might be located. Accordingly, the COQ metric seems to offer me no new insights about my company that I would not already have by looking at standard company performance and financial metrics. Moreover, unlike standard company performance metrics, COQ cannot even be compared to others in the industry, because the method of computation is not standardized.

To my surprise, Feigenbaum avoids these difficulties entirely, while Gryna, to his credit, points out the problem of minimizing COQ and then admits that he has no answer to it.

In other words, regardless of how COQ is defined, even the experts cannot seem to explain how it gives meaningful insight for purposes of making management decisions.

Am I missing something, or is COQ just a decoration that looks nice in the quarterly meeting speech by the head of QA? Does anyone actually use COQ and, if so, how exactly do they use it?

Geez, give me a little information and I can be a real pest.

Thanks everyone!

-- Doug Pfrang
 
J

John C

#23
Doug,
No, you're missing nothing. It is "just a decoration that looks nice in the quarterly meeting speech by the head of QA". I think your long, convoluted analysis is a good example of what we are likely to be drawn into. Best to keep clear of it and spend our time where we have clear, undisputable goals.
To quote Brian: "if the cost is higher than the benefit then good business sense would dictate that we re-evaluate the quality program and make it more effective".
But we already know that we need to re-evaluate the quality program and make it more effective (and more efficient, of course).
My mother used to say;
"It's only a fool, that asks the road he knows".
There is such a thing as the law of diminishing returns but CoQ exercises aren't designed to pick up the precise engineering data normally needed for finding this point.
It's safe to assume that sensible process engineering, based on yields, is always necessary and will provide positive returns.
rgds, John C
 
B

Batman

#24
Doug-
Years ago, from my past as the COQ reporter, I found that the COQ report had value, when used and presented in a format that was useful. It was NOT just a report. The president got pie charts for relative comparisons, overall trends, etc. The supervisors used this report (along with other information) for any needed root cause investigations, continuous improvement efforts, etc. The trends and data were used in part as "company level data."

Some of the items we tracked were:
Scrap (who doesn't) as % of sales; Late delivery charges - expedited shipments, etc.; Customer Returns $$ value and PPM,
Efficiencies as $$, Uptime in $$, many more.


I accumulated the COQ data and sent out the reports. We had several departments, each department was given the overall report and their individual report. It included "the four" with many subcategories. I found that unifying some of the data was beneficial to the trend analysis.
For instance (just an example):
Department A had an upward trend in % of sales scrap. Department B had a downward trend in their scrap as % of sales. Department A had ongoing increases in sales, and it was discovered that he had a couple of issues with bringing in untrained folks on weekends to keep up. It was also discovered that some of his equipment was breaking down more often due to increased use. It was easier to go to the president with the COQ trend chart and the supporting information to request more resources.

Department B was also finding increasing sales. These sales were NEW business, not just increased old business. His department and his technical support folks were congratulated for lowering % scrap by bringing on new business well, and correcting things in a more timely manner. This superisor was paying attention to the business plan, the development of the strategy, and the monthly COQ reports that were not so kind two years earlier.

So, while I agree in part with John C's evaluation of COQ, I think that if you are doing nothing in this regard, then starting a good COQ can get some focused on the problems, and aid in decision making. It cannot be used to compare your company to another (unless you are privvy to some info), but if you know there are positive and negative trends, it can help with the WHERE of something, and it can steer you to the WHY, WHERE, and WHO. If there is a "bad" trend, detailed information should be available for any follow-up. "Good" trends could also be examined, perhaps to apply lessons learned in one department to another.

By the way, John C, I like the higher level considerations you mentioned.

As far as setting goals, some come to mind - 25% decrease in scrap as a percent of sales, 10% increase in training, 100% On Time deliveries...
 
J

John C

#25
Batman,
Sorry to keep on about this, but persistence is my greatest virtue;
The examples you described were predominently "cost of scrap as a percentage of sales". Well that's as good an example as any:
What does "cost of scrap as a percentage of sales" mean? Is it the cost of the replacement items? The cost of that plus the administrative costs of scrapping the old items? All of that plus ordering, holding and fitting the replacement parts? All of that plus the cost of doing overtime or extra work to fulfill the orders which were missed because of the fallout due to the scrap? or, etc, etc, up to the really unknowable costs which are the ones which hurt so much they can be terminal.
If you've got scrap, it's because your yield is down by so much percent. And if your yield is down, you know. Or, if you don't know, then it's because you are not controlling your main activity - you haven't got the basics, whether it be line fallout, untrained operators, unanswered queries or complaints, whateve. If you've got scrap, you know you've got all those problems and accumulating costs and you know that you have to do something about it. Not measure, do something. Investigate the cause, not the second, third and fourth and subsequent effects - it's too late to do anything about them. Stop the scrap, prepare a source of trained backup operators, apply preventive (CofQ must be the antithesis of preventive quality!) control as far as possible and, where not possible, immediate and direct action. If your managers don't know that, then that is another problem. Investigate it and find the root cause. Who hired managers who want indirect information when the immediate need is obvious to the janitor? What sort of management school did they go to?
People keep saying 'the managers want it' or 'it is better to hand off this sort of info to the managers'. Well, we all know that, 'without management commitment and involvement, ISO 900x will not work'. We've heard that so many times. Well, their involvement will not be much use if it is the wrong sort of involvement. The Quality Professionals have usurped a big slice of management responsibility and have put their own slant on it. They have a lot to answer for.
rgds, John C

[This message has been edited by John C (edited 17 June 1999).]
 
B

Batman

#26
Hi John
Persist away!
I agree in most of what you say, generally the Cost of Quality data is useful to some in different ways. It was not the only data we used. I was offering Doug Pfrang a method of compensating for some of the items he stated affect the COQ report. Even if sales go up or down, the % of sales ignores this. That is not to say that declining sales is not an issue, just that a percentage report can have value.

The managers used the information to help make resource decisions. The COQ report was only one of other data that was evaluated each week / month. We always have scrap, is it going up? is my continuous improvement projects working? which part(s) do I work on? Which department needs the most (and of course limited) technical resources? That was what the COQ report was used for. The persident could see where the profit impact was occuring most. Of course you can't report on everything, so we picked what was important and meaningful to the business.

I guess there are many ways to approach the problem of gathering "company level data" that is meaningful. I think COQ as described above is one good way, especially if the receivers really understand what the data means; certainly it is better than nothing.
 
J

John C

#27
Batman,
Well, I've got to hand it to you for persistence, as well. When neither side will move then the question of who is right becomes irrelevant. One thing is sure in life; In the long run, it's better to be wrong along with all the others, rather than the single, solitary individual who is right.
Sure, the CofQ is one ecellent way to do the job - An awful lot better than nothing. It's usually not what you do but the way that you do it (and that's what gets things done), so I've heard. The Astecs and the Incas build magnificent civilisations without ever inventing the wheel and it didn't do them an ounce of harm.....
except disaster, destruction and eventual annihilation, of course.
Imagine what their war chariots would have done to Cortez, if they had had wheels instead of sledge runners. Oh, and horses...well... er....???....
I think I've given it my best shot, so I'll give up.
Give 'em hell, Batman,
and regards,
John C

[This message has been edited by John C (edited 18 June 1999).]

[This message has been edited by John C (edited 18 June 1999).]
 

Marc

Hunkered Down for the Duration with a Mask on...
Staff member
Admin
#28
Subject: Re: Q: ISO Cost Effectiveness /../Ohri/Smith
Date: Mon, 18 Oct 1999 11:50:15 -0600
From: ISO Standards Discussion <[email protected]>

From: "Gary V. Smith" <[email protected]>
Subject: Re: Q: ISO Cost Effectiveness /../Ohri/Smith

> From: Edith Ohri <[email protected]>
> Subject: RE: Q: ISO Cost Effectiveness /../Ohri/Smith/Ohri
>
> Right, but those "hard numbers" are coming through lots of interpretations of
> the idea of quality, therefor their authenticity is far from being self
> evident. For example, let's take measuring dimensions:
> First, all chances are that the cost would catch only the control and
> re-works, but not the illusive error prevention.
> Second, the physical qualities are only one part of the product quality and
> focusing on partial cost figures may cause neglecting other qualities,
> such as good design and integration.
> What's worse, I have a feeling that the illusive cost/benefit of good design
> worth an order of magnitude more then the measurable qualities.
>
Dear Edith,

In the case of a "hard" product the customer establishes boundaries which they have determined gives the product the desired form, fit and function. These boundaries are usually some class of dimensions and/or attributes which gives the product the desired quality. If planned efficiently quality will be designed into the product by having determined early in the process how the boundaries, i.e., specifications, are to be met. Whether the product is a wheelbarrow or a 747 jet all the parts must be designed to fit together well, look right and function properly at a reasonable cost and delivered on time.

A good cost of quality program, whether manual or software based, captures as many of the factors in this process as possible. The cost of quality program that I use has upwards of 100 separate cost categories that cover all contingencies within the major classifications of the cost of prevention, appraisal, and internal and external failures.

However, a great deal is made of the perception of quality, but this is not always a good gauge of quality. For example, a Rolex watch and a Rolex knockoff. Cosmetically they both look like Rolex watches and at first perception they operate the same, but the difference in the quality in the design will become apparent to the practiced eye. An authentic Rolex could cost $5000 and the knockoff could cost $100. All else being cosmetically equal the perception is that the $5000 watch is better than the $100 watch. But they will both tell time, although probably not equally as well. The accuracy of the watches is determined by the form, fit and function of the internal parts and the cost of making them all work properly. How accurate does the customer need the watch to be? Are you a train conductor or a beachcomber?

Anyway, the cost of (poor) quality can be as accurately measured as is necessary to provide the desired indicators. The user has to determine what is important.

Gary Smith
 
#29
Juran states that cost of poor quality is but one of several tools that can be used to seek improvement in thew system and should bwe used as such ,on a project by project basis. Crosby bulit his reputation on Cost of Quality (Along with zero defects).

Why don't more people use it ?,primarily they are afraid of it. And for good reason. Cost of quality is measured in DOLLARS, and nothing is of more interest to upper level management than DOLLARS.
Cost of quality is also and excellent tool for measuring the effectiveness of management,i.e., High (or low) cost of quality is a direct result of how good (or bad) the system is managed. It is the only tool, that I know of , that is directly owned and controllable by management.






------------------
Sam Goody
 

barb butrym

Quite Involved in Discussions
#30
people are afraid to visualize the wasted dollars.......and they show very clearly what the weakness is...SO once told...the culprit is soon found....and depending on the culture of the company...could produce very painful actions. the old 'be careful what you wish for....' syndrome.

AND alot of people just don't get it

[This message has been edited by barb butrym (edited 01 November 1999).]
 
Thread starter Similar threads Forum Replies Date
S Difference between COQ (Cost of Quality) vs COPQ (Cost of Poor Quality)? Misc. Quality Assurance and Business Systems Related Topics 8
S COQ (Cost of Quality) Looking for forms, templates Quality Manager and Management Related Issues 5
Q COQ (Cost of Quality) - Would someone review this doc and offer feedback? Benchmarking 10
J Quality Cost Principles? COQ - Cost of Quality Misc. Quality Assurance and Business Systems Related Topics 5
A Cost of Quality (COQ) Information Misc. Quality Assurance and Business Systems Related Topics 5
C Which Cost of Quality (COQ) Model is best? Misc. Quality Assurance and Business Systems Related Topics 5
D Cost of Quality (COQ) Excel .xls spreadsheet Excel .xls Spreadsheet Templates and Tools 1
B Quality Manager wants a Cost of quality (COQ) procedure in our QMS Document Control Systems, Procedures, Forms and Templates 8
P Acronym COPQ (Cost of Poor Quality) and COQ (Cost of Quality) - Differences Definitions, Acronyms, Abbreviations and Interpretations Listed Alphabetically 26
S Cost of Quality (COQ) Work Sheet or template Document Control Systems, Procedures, Forms and Templates 23
R Cost of Quality (COQ): How do I measure it? As a % of Sales? Misc. Quality Assurance and Business Systems Related Topics 9
S Philip Crosby's Cost of Quality - James Harrington's Book Addressing COQ Book, Video, Blog and Web Site Reviews and Recommendations 18
D Cost of Quality (COQ) - Breakdown of Costs Discussion QS-9000 - American Automotive Manufacturers Standard 3
Marc Cost Of Quality (COQ) Software - Defining "cost of quality" Quality Assurance and Compliance Software Tools and Solutions 2
N Guidance - Cost of Good Quality Misc. Quality Assurance and Business Systems Related Topics 2
C Budgetary cost to obtain ASME NQA-1 Quality Program Certification Quality Management System (QMS) Manuals 0
optomist1 Automotive News The Cost of Inspecting In Quality IATF 16949 - Automotive Quality Systems Standard 7
S Cost of Poor Quality (CoPQ) - Scrap element of calculation includes lost margin? Misc. Quality Assurance and Business Systems Related Topics 7
B Include cost of quality (and other wastes) in an ISO 9001 system? ISO 9000, ISO 9001, and ISO 9004 Quality Management Systems Standards 5
C What is your Cost of Poor Quality (COPQ)? Misc. Quality Assurance and Business Systems Related Topics 8
W How much would it cost to setup a Quality System for a Repackager or Relabeler? Quality Manager and Management Related Issues 1
M Reducing Cost of Poor Quality - Your opinion requested Quality Manager and Management Related Issues 14
Geoff Cotton What is the benchmark for Cost of Quality in a Heavy Engineering organization ? Benchmarking 2
B Quality problems that cost Millions of dollars (or euros, or pounds, etc.) Quality Manager and Management Related Issues 7
M How Does Your Organization Measure Cost of Quality? Benchmarking 14
M Which metric is better for Cost of Poor Quality (COPQ)? Quality Tools, Improvement and Analysis 17
W Cost of Quality - Electronics Manufacturing The Reading Room 14
M Supplier Rating Form with Quality, Cost, Delivery wanted Document Control Systems, Procedures, Forms and Templates 4
P Quality Engineer's Role in Cost Management (including UMC) Misc. Quality Assurance and Business Systems Related Topics 3
R Cost of Quality in Aerospace Industry AS9100, IAQG, NADCAP and Aerospace related Standards and Requirements 10
R Development Rejection Costs in COPQ (Cost of Poor Quality) Calculations Misc. Quality Assurance and Business Systems Related Topics 6
R Cost of Quality Benchmarking Information for the Electronics Industry Benchmarking 1
M Cost of Quality - Is it good or bad and why - Critical Review Research Philosophy, Gurus, Innovation and Evolution 28
1 Cost of Poor Quality form or template wanted Quality Tools, Improvement and Analysis 1
S COPQ (Cost of Poor Quality) Vs PONC (Price of Non Conformance) - Explanation wanted Quality Manager and Management Related Issues 5
M Cost of Poor Quality Reports IATF 16949 - Automotive Quality Systems Standard 15
M Justifying the Cost of Quality? Money that we spend on the quality system Misc. Quality Assurance and Business Systems Related Topics 24
S How to develop a Quality Cost Summary Report Quality Manager and Management Related Issues 4
M Cost of Software to Calculate the Cost of Quality Quality Assurance and Compliance Software Tools and Solutions 10
K Generic worksheet for calculating Supplier Chargeback and Cost of Poor Quality (COPQ) Supplier Quality Assurance and other Supplier Issues 2
C Corrective Action Importance Level - Cost and Quality - The Johari window Nonconformance and Corrective Action 4
N Personnel COPQ (Cost of Poor Quality) Quality Manager and Management Related Issues 6
H Cost of Quality Can Open Management's Eyes Quality Tools, Improvement and Analysis 15
A KPI (Key Process Indicator) - Cost of Non Quality Benchmarking 8
S COPQ (Cost of Poor Quality) - Can we add Premium Freight? Misc. Quality Assurance and Business Systems Related Topics 6
S COPQ - Cost of Poor Quality - COPQ course for non-financial people Misc. Quality Assurance and Business Systems Related Topics 1
N Identification of COPQ (Cost of Poor Quality) aspects Quality Tools, Improvement and Analysis 2
M Quality Cost Control procedure wanted Quality Manager and Management Related Issues 4
G Quality Group Salaries as Percent of Sales (as part of the Cost of Quality) Misc. Quality Assurance and Business Systems Related Topics 8
J Quality Cost Summary Report/Procedure? Cost of Quality Misc. Quality Assurance and Business Systems Related Topics 4

Similar threads

Top Bottom