Nutek, Inc. Quality Engineering Seminar, Software, and Consulting ( Since 1987) Site: OEC 
Overall
Evaluation
Criteria
(OEC) 
The need for an overall evaluation method arises when there are more than one objective that a product or process is expected to satisfy. Situations of this nature are more common in many areas of our lives. Consider the educational system of rating students. Students are evaluated separately in each of the courses they take. But for comparison purposes, it is the Grade Point Average (GPA) which is used most often. Do you ever wonder where the performance numbers like 5.89, 4.92, etc. come from for the Figure Skating competition? Obviously these numbers are averages of all the judges' scores on a scale of 0 to 6. Each of the judges, of course, use the same range of evaluation numbers to evaluate different aspects of the performance like style of skating, how high they jump, how well they land, etc. The scores one judge assigns to a performer come from averaging his/her scores in all separate criteria of performance. Such use of an average or overall criteria of evaluation is quite common in many activities. It is not so common in engineering and science however. (Read an article of OEC, click here) If an overall index is so common in sporting, educational, and social events, why is it so rare in engineering and scientific studies? It so primarily, because of three difficulties: 

Example of Criteria with Different QC: Determine the best player
Player 1 Player 2 QC Note: Example
experiment file included with Qualitek4
POUND.Q4W .
To examine how OEC is formulated, run Qualitek4, be in the
Experiment Configuration Screen, then select

Are these two players of equal caliber as the total scores suggest? Is the additions of the scores meaningful and logical? Obviously the total of scores have no meaning, as these players to do not perform equally. To make the total score meaningful, one of the numbers must be adjusted such that all QC's are aligned one way (either BIGGER or SMALLER). A logical and meaningful way to combine the two scores will to first change the QC of the Golf score by subtracting from a fixed number, say 100 (an arbitrary number, expected highest possible for 9 holes of Golf), then add it to the score of the Basketball. The new total score then becomes: Total score for Player 1 = 30 + (100 45) = 85 Total score for Player 2 = 20 + (100 55) = 65 The number 85 and 65 now indicate the relative merit of the players, Player 1 ( 85 ) Player 2 (65).

Multiple objectives are quite frequent in engineering
applications. No matter the project, be it a product optimization, process study,
or problem solving, the desire to satisfy more than just one objective is quite common.
Because the criteria involved are different, the experimental results (be it DOE/Taguchi or
otherwise) are generally analyzed one criteria at a time. This
approach, of coarse, does not guarantee that the best design obtained for one criterion,
will also be desirable for the other criterion. What is needed is a properly
formulated Overall Evaluation Criteria (OEC) number representing the performance of the
test sample. Thus, when there are multiple criteria of evaluations, lack of such
formulation poses a major hurdle for analysis of DOE results.
OEC Formulation The method of OEC formulation and computation can be studied by considering the cake
baking.

EVALUATION CRITERIA DESCRIPTION Criteria Worst Value (w) Best Value (b) QC Weighting (Rw) 1.Taste 0 12 Bigger 55%(Rw1) 2. Moistness 25 gm 40 gm Nominal 20% (Rw2) 3. Consistency 8 2 Smaller 25% (Rw3)

The evaluation criterion were defined such that TASTE was measured on a scale of 0 to
12 (bigger is better), MOISTNESS was indicated by weight with 40 gm (target value)
considered the best value (nominal is the best) and CONSISTENCY was measure in terms of the
number of voids seen (smaller is better).
Assume that the cake sample for trial#1, the readings are (T, M, C): Taste T = 9, Moistness M = 34.9, and Consistency C = 5 Then OEC for the cake sample is can be expressed as: OEC = [ (90)/(120) ] x Rw1 + [ 1  (34.9  40)/(4025) ] x Rw2 + [1  (52)/(82)] x Rw3 
Explanation of data reduction:
Numerator (9  0 ) represents (reading  worst value)
in case of BIGGER QC Numerator (40  34.9 ) represents ( reading  target
value) in case of NOMINAL Numerator (5  2 ) represents (reading  best value) in case of SMALLER QC Denominators (12  0, 40  25, and 8  2) represent differences between the best and the worst values for all QC. The worst value in case of NOMINAL is the worst deviant of the data extremes from the target. 
Note:
Before all criteria of evaluations can be combined, their
QC's must all be the same. The second expression is modified to change the NOMINAL QC,
first SMALLER by finding the deviation from the nominal, then to BIGGER. The third
expression is modified to change the SMALLER QC to BIGGER. The numerator in each term is
calculated by subtracting the smaller magnitude (or target value in case of NOMINAL) from
the reading, then taking the absolute (x) value. The denominator is always the range of
data spread, which is positive difference between the best and the worst reading for the
criteria.
Since an L8 orthogonal array described the baking experiment, there are eight cakes baked with one sample per trial condition.. The OEC calculated above (OEC = 66) represents the result for trial#1 . There will be seven other results like this. The eight values (OEC's) will then form the result column in the orthogonal array. The process will have to be repeated if there were more repetitions in each trial condition. Quality Characteristic for OEC  Depending on the QC of the constituting evaluation criteria, the QC of the OEC can be either BIGGER or SMALLER. I the above case, all criteria evaluations were aligned to be of bigger QC as the first criterion with 55% relative weighting has Bigger QC. All versions of Qualitek4 prior to version 6.5 allowed QC of OEC to be either Bigger or Smaller as the relative weight of the criteria dictated (automatically set). With the release of Version 6.5 of Qualitek4 software will always assign Bigger is better QC to all OEC values regardless of the QC of the criteria it is made of. [Download and try new OEC capabilities of Qualitek4 software from http://Nutekus.com/wpq4w.html

Observe that the evaluation (T, M and C) in each case is first modified to show the
positive difference between the reading and the smaller magnitude of the best/worst values
(or target for NOMINAL), then divided by the allowable spread of the data. This is
done to get rid of the associated units (Normalization). Subtraction of the fractional
reading from 1, as done for the second and the third criteria, is to change the quality
characteristics to line up with the first criteria, that is, BIGGER IS BETTER. Each
criteria's fractional value (y/ymax) is also multiplied by the corresponding weighting and
added together to produce a net result in numerical terms. The manipulation above
normalizes OEC formulation to produce numbers between 0  100. Author of OEC Concept: Ranjit K. Roy, Ph.D.,P.E. REFERENCES:
2. A PRIMER ON THE TAGUCHI METHOD by R. Roy. 3. Report# 1. Multiple Criteria of Evaluations for Designed Experiments
