How do I analyze the Sound Quality Testing of vacuum cleaners?

[tahirawan11]

Hi

I would like to perform a blind test to rank the 'sound quality' of five different vacuum cleaners. The study shall be performed by running each vacuum cleaner and then asking a voting panel to rank the perceived sound quality from 1 to 10 (1 being worst and 10 being best). Also i would like to have 3 repeats (with randomization) of the rankings so i have more confidence in the results. Can anyone tell me how should i analyse the results? Also if there are any suggestions on how to conduct this study?

Thanks

 

[John Broomfield]

Re: How do I analyze the Sound Quality Testing of vacuum cleaners ?

Tahira,

You could start with ASTM F1334 - 14

Standard Test Method for Determining A-Weighted Sound Power Level of Vacuum Cleaners

Here is a similar British Standard:

BS EN 60704-2-1:1995
Household and similar electrical appliances. Test code for the determination of airborne acoustical noise. Vacuum cleaners


John

 

[Mike S.]

Re: How do I analyze the Sound Quality Testing of vacuum cleaners ?

Sound level is relatively easy to measure quantitatively.

Sound quality is much more likely to be subjective/qualitative.

Which do you really want to know?

 

[PaulJSmith]

Re: How do I analyze the Sound Quality Testing of vacuum cleaners ?

Agreed. 'Sound quality' will be entirely subjective. Everyone's ears hear differently all along the audio spectrum, with some being more or less sensitive to certain frequency ranges. This individual sensitivity will color each person's opinion differently than others who are hearing the exact same thing. Distance and environmental factors (the listening room) will also affect the experience.

When you say you will "run" each vacuum, do you mean just turn it on and let it run at idle, or will it be running under various loads to represent different surfaces (carpet pile thickness/length)? Will it include debris? All of these things will change the sound of each unit.

Have you considered a simple audio spectrum analysis?

 

[tahirawan11]

Re: How do I analyze the Sound Quality Testing of vacuum cleaners ?

yes, I agree that sound quality is very subjective and 'noise level' can be measured easily in 'db' but still the customer will prefer one 'sound' over the other; although they may have similar 'noise level'.

i plan to do a simple study where the vacuum cleaner will be run in idle for few minutes and then the 'customer panel' (who are ordinary persons and not specialist to judge sound) will rank it. The goal of the study is to learn which 'sound' sounds better to ears and then the design can be modified to achieve the most pleasant sound.

No, i have not planned to do a simple audio spectrum analysis but i have never used it but can you tell me what i can get by doing one?

I would like to know if it is fair to take the average sound rank from three runs and then rank them from good to bad. also do i need to use any statistical tool (t-test) to show the significance of results?

/tahir

 

[Eredhel]

Re: How do I analyze the Sound Quality Testing of vacuum cleaners ?

One easy metric is to EQ it. You could use a graphic EQ and see how people feel about lower and higher pitch idles and use. Plus you'll have hard data with the frequency numbers. I'm sure there are some quick and easy mobile apps like garage band where you can just put an EQ on, record, and watch the spikes.

 

[tahirawan11]

Re: How do I analyze the Sound Quality Testing of vacuum cleaners ?

I have completed the first round of sound quality testing. The sound quality has been subjectively ranked by a group of test panel on a likert scale. After which i have performed a Kruskal-Wallis test to test for significance. Based on the analysis i can conclude that there is a statistical difference between the sound quality of the different vacuum cleaners and vacuum 1 has the best quality followed by Vacuum 4. Any comments if my analysis is correct ??

I plan to do a second round of tests after making some changes in the design of vacuum cleaner and then re-run the test. The time i plan to do 2 repeatable and record the graphic EQ which will be used to detect any correlation between the results and the high/low spikes.

/Tahir

 

[Steve Prevette]

Personally I avoid Likert Scales. Since you actually asked the question using verbal descriptors, you may want to try this method:

http://www.efcog.org/wg/esh_es/Statistical_Process_Control/docs/Survey_Analysis.pdf

 

[John Broomfield]

Personally I avoid Likert Scales. Since you actually asked the question using verbal descriptors, you may want to try this method:

http://www.efcog.org/wg/esh_es/Statistical_Process_Control/docs/Survey_Analysis.pdf

Steve,

I read your alternative to Likert Scales.

What does "Very Effective" mean?

Cheers,

John

 

[Steve Prevette]

Steve,

I read your alternative to Likert Scales.

What does "Very Effective" mean?

Cheers,

John

Actually, that is left to the survey responder. But it is the highest positive response the person may choose.

The original paper contains all sorts of suggested categorical scales. In that case, the phrases were built around "Effective" and "Very Effective" was the highest selection. Personally, I tend to use Strongly Disagree, Disagree, Neutral, Agree, and Strongly Agree.

There is some psychology around the choice of very vs. strongly, in fact, some of the scales tend to give uniformly distributed results, and others thend to be more bell-shaped.