|Nutek, Inc. Quality Engineering Seminar, Software, and Consulting ( Since 1987) Site: Marketing DOE|
Design of Experiments (DOE)
for Marketing & Advertising Applications
To Our Marketing
Optimizing your marketing effort is similar to improving the taste of a cake by adjusting the ingredients like sugar, flour, butter, egg, etc. in the baking process. The materials you send out soliciting response or web site you design to attract potential customers, all depend on how you combine different factors that make your ad and those that influence conversion rate. With little effort in experimenting, you can determine how to combine these factors to maximize your returns. The technique employed is a new and creative problem solving approach that has been successfully utilized in engineering and manufacturing over last half a century to optimize the products and process designs. You can now benefit from this proven technique to economically optimize your marketing returns (design and layout of literature, texts, web site, etc) following an structured and data-driven approach.
ABOUT TAGUCHI DOE Design of Experiment (DOE) is a statistical technique
to study effects of multiple variables. This technique was first
introduced in England in 1920’s by Sir R. A. Fisher. Fisher wanted
to study the influence of rain, water, fertilizer, sun shine, etc.
on agricultural produce. Later, in the 1950’s a Japanese scientist,
Dr. Genechi Taguchi, standardized the application of the DOE for benefit
of manufacturing professionals. Since 1980’s the Taguchi version of
the DOE technique has been in use by the US manufacturing
industries with remarkable success.
|The design of experiment (DOE) technique has been successfully used in optimizing engineering and manufacturing process designs for decades. Its use in optimizing advertising response has just begun.|
BACKGROUND: The DOE technique has been successfully applied in all phases of engineering products and processes to improve quality, reduce variation, and satisfy customers for last several decades. Examples of doubling mail solicitation response and increased conversion rates in internet marketing has been documented by many in the field. We support our clients who are involved in maximizing response solicitation, landing page optimization, improving pay-per-click advertisements, etc. and have greatly benefited from the use of the DOE technique. We support your application effort from "cradle to grave".
Details of services we provide:
(Click here to review: List of Nutek Client Companies)
BY TODD DAVIS,
Dec. 22, 2005
It's an axiom the advertising industry hasn't been able to shake,
since advertising is emotional and often unpredictable beyond never
anticipating results greater than a single percentage point.
But scientist Ranjit Roy and automotive marketing veteran Michelle Morrissett want to change that perception. They call it MetaMarketing.
Using statistical analysis known as the "Taguchi method" to test the performance of advertising before it runs, the two say MetaMarketing has the potential to dramatically improve advertising efficiency similar to the gains achieved by applying the practice to engineering.
"This helps narrow the bull's-eye instead of scattering advertising all around."
A direct-mail trial run this summer for Hank Graff Chevrolet in Davison resulted in a 60 percent gain in responses over traditional advertising, the Fouresquare principals said.
Morrissett said future applications could include Internet marketing programs, advertising agencies and other retail advertisers.
New applications, new results
Roy, through his Bloomfield Hills-based company, Nutek Inc., has been teaching the theories of Japanese scientist Genichi Taguchi to engineers and managers of Fortune 500 companies since 1987.
Taguchi's "design of experiment" methodology provides the framework for performing multiple test parameters simultaneously. Traditional testing methods compare two sources of influence at a time - repeated for validation - before comparison with another set of data.
It has become an accepted form of quality control and validation for ISO/QS-9000 and Six Sigma practitioners around the globe, Roy says.
"It's applicable in many areas - from cars to Campbell's soup," he said. "Many who followed it were so preoccupied with manufacturing, it wasn't applied to other avenues."
Having worked in marketing for Ford Motor Co. for 16 years, Morrissett also realized the potential Taguchi's could have on the industry.
"(Design of experiment) is a tried and true discipline in many environments," she said. "Now we're applying it to the marketing and advertising arena."
For the pilot program at Graff Chevrolet, Morrissett and Roy analyzed a variety of marketing and advertising criteria that they gathered through a series of focus groups with sales representatives and customer surveys.
Roy crunched the data in a program based on Taguchi's method, narrowing 128 potential advertising options to eight.
Morrissett created eight ads based on the results of the program, mailing 1,000 direct-mail pieces in August based on each option.
The most-effective ad was a letter with two colors, a picture of Hank Graff in the upper right-hand corner and not much text. The returns demonstrate that an easy-to-understand offer and brand recognition work, Morrissett said.
According to dealership general manager Chris Graff, the results, delivered in October, were "astronomical."
"Statistically, they proved one of the eight ads was far more effective than the others," he said.
Morrissett compared 5,481 target customers to customers - identified through Graff's customer database - who purchased new cars during the test period. She said 2,519 were eliminated from the count due to duplication and technical difficulties obtaining data from the dealership's computer system.
Of the target customers, 59 bought new cars from Graff Chevrolet during the campaign. That's less than a 1 percent response rate overall, but an improvement of 61.2 percent over Graff's own previous best campaign.
Compared to the dealership's "average performing campaign," the ads resulted in a 135 percent improvement, Graff said.
Still, Graff said he remains a skeptic, holding off on final judgment.
"We're going to run that (same) sample again to validate the results," he said.
Once a "best method ad" is identified, a client can reuse the advertising and marketing data as long as they want, Morrissett said.
"The bull's-eye always moves," she said.
"It's reasonable to re-test after 12-18 months as market conditions change."
The costs can vary, depending on the complexity of the preparation and research, according to Morrissett, but are expected to be 5 percent-10 percent of the prospective campaign budget.
Todd Davis is a freelance writer.
MetaMarketing Studies Help Auto Dealer Achieve Improved Sales
Date: 11/22/05 Press Release by Automation Alley, Michigan.
Nutek, Inc., a Bloomfield Hills of Michigan-based quality engineering consulting firm and Fouresquare LLC, a Troy of Michigan-based marketing & business development firm are pleased to share their first test case of scientifically designed advertisement for an automobile dealership. Under the name Metamarketing, the consulting companies initiated a study to optimize the direct marketing mail strategy of Hank Graff Chevrolet of Davison, Michigan, one of the top 5 largest Chevrolet dealerships in the State of Michigan, and achieved the following results.
The newly launched MetaMarketing strategy uses Taguchi Experimental Design technique to run live marketing/advertising experiments. This proprietary service is offered by Nutek in collaboration with Fouresquare LLC of Troy, Michigan. The results identify optimal elements (delivery type, ad content, colors, etc.) that will provide the greatest ROI for every advertising dollar spent.
Chris Graff, General Manager at our pilot dealership has agreed to give personal testimonials to businesses considering utilizing our Metamarketing service. We will provide his name and contact information to you for references upon request. In his own words, Its difficult to believe the results are real, but that's what the numbers show.
Nutek, Inc. is an engineering consulting company founded by its president Dr. Ranjit K Roy in 1987. It specializes in training and implementation of the Taguchi experimental design technique for product and process design improvement by the manufacturing industries. Experimental design technique has a long proven track record in manufacturing processes. The technique is expected to be equally effective in scientifically identifying what works the best. We are looking for data that tells us how the customer response is affected by a long list of uncontrollable factors says Nutek's president Dr. Ranjit Roy.
Fouresquare LLC specializes in marketing with a key focus on branding and imaging. As part of their on-going evolution, a new marketing response optimization technique called Metamarketing was developed. This new process gives us the ability to open the boundaries related to traditional marketing campaign design. Metamarketing is a mathematically based process that statistically analyzes and optimizes the performance of marketing tools being used. They include but are not limited to; direct mail, newspaper, web site, point of purchase materials. Michele Morrissett, President of Fouresquare LLC has 20 years of experience in the automotive industry. Her skill at effectively consolidating marketing materials ranging from printed materials to auto shows, while ultimately reducing costs was gained during her time at one of the Big Three. Most advertisers today do not always have a quantitative measure of the response. With use of DOE it is possible to statistically determine what works and what does not. says Ms. Morrissett.
1. DOE Test Samples for Marketing Applications - How many is too many? - Download this white paper (PDF) by Dr. Roy and David Bullock, that explains the methodology in clear terms that business people can readily understand.
(Read the article below to learn about how the Design of Experiment (DOE) technique may apply to your activities)
K. Roy, Ph.D., P.E. (M.E.)
Since the early 1980s, Design of Experiments (DOE) has been used by industries world-wide to improve product and process designs. DOE can save significant amounts of time and money for its users because it streamlines the process of selecting the best of many possible choices, and does so with validity. It is one of the techniques prescribed by the ISO/QS-9000 and TS 16949 standards for quality improvement.
DOE is about choices
Everyday we make choices and generally, the more the choices the better. Consider some of the common choices we make every day. My wife keeps four popular brands of soft drinks in the kitchen refrigerator. In this case, I have one item (a soft drink) with four available options. In making selections like this we face two issues: (1) the number of choices available, and (2) how the desirable option is selected.
The number of options available to choose from can become overwhelming if we face more than three items, or factors, each of which offers multiple substitution possibilities. For large numbers of factors, each of which has a large number of options (called “levels”), the combinations/permutations formula lets you calculate the total possibilities. At this point, simply realize that the number of total possibilities rapidly becomes very large. Seven factors with two levels in each produces 128 possibilities. 15 factors with two levels in each produce 32,768 possibilities. When the possibilities are known, how do you decide which one is best for you? The most factually revealing method is to test each and every possibility – that is, if you can afford the time and cost. Often, you can’t. And, in that situation, you will need a more formal selection methodology.
Selecting desirable options
Choices and outcome possibilities in business and scientific investigations are endless. Whenever performance (output/results) is subject to influence from more than two factors, overlooking all the possibilities may cause some opportunities to be lost. But can we always afford to look at all possibilities?
The problem compounds with the number of possibilities, which range from eight possibilities for three factors (which can be inputs, ingredients, or variables) to over 32,000 possibilities for 15 such factors that offer only two choices each. Finding the best among all possible options is like trying to find a needle in a haystack. Testing all conditions is almost never an option. The situation is similar to polling efforts for presidential elections. It is never practical to talk to all voters. Pollsters make their predictions after talking with a necessarily limited number of voters whom they assume to constitute a reliable sample. Statistical science tells us that the larger the sample the higher the accuracy of prediction.
When looking for the most desirable among a huge number of possibilities, help comes from the DOE technique. It defines the smallest sample (a few conditions among all the possibilities) that you will need to look at, and what those combinations are. When you have designed an experiment, you will have answers to crucial questions such as what is the minimum number of tests you must conduct in order to assure a reliable result, and how must these tests be conducted?
DOE enables you to predict the behavior of the factors involved in an experiment as part of determining the most desirable option(s). In the election polling example, the sample size is much smaller than the number of all voters, so some kind of statistical calculation is needed to predict (an estimation of expected value) the final results. Similarly, in DOE, statistical calculations are needed because only a fraction of all possibilities generally can be looked at.
Where is DOE used?
DOE produces the most benefit when it is used in the early stages of product design. It can be used with simulation and “virtual” engineering, as well as later when design has progressed to the point where prototypes are built and manufacturing processes are being structured. DOE-based investigations can be created by using the simulation model, or by using physical hardware. Most manufacturing processes offer opportunities to perfect or at least significantly improve performance with DOE. No matter which phase of engineering (concept, design, or production) uses it, DOE is most practical and effective for applying a finishing touch or for fine-tuning something that is already working at least reasonably well.
The number of individual combinations you need to test depends on the number of factors included in the prerequisite study. As a result of that study, you will find that the number of combinations (or, configurations) needed to investigate will typically range between four configurations for three factors with two choices in each and 16 configurations for fifteen factors with two choices in each. Usually, you want to repeat tests with each combination to check for variability from sample to sample. This, of course, increases the number of test samples for the investigation, but is vastly more efficient than studying each of hundreds or thousands of possible combinations.
DOE’s application includes three major tasks: (1) experiment planning, (2) experiment design, and (3) analysis of results. Among these, design and analysis are routine tasks similar to all sizes of experiments. These are areas in which technical advice is relatively easy to obtain and where automated software is appropriate. The most important step, however, is planning. This is what impacts the outcome the most. Your knowledge of the project comes into play here. This is where you will decide, generally as a group and by consensus, how you should go about judging the experiment’s performance, what your yardstick of the results should be, which factors you believe will most influence the results, and the choices that you will allow in the factors.
Dr. Genichi Taguchi formulated the process of laying out standardized sets of tests to suit different experimental situations. Dr. Taguchi provided guidelines in the form of tables of numbers. These rectangular tables of numbers help experimenters prescribe the minimum number of tests to be carried out for a given situation; that is, the number of factors and their choices to be included in the study.
One such table of numbers (known as L-4 array), shown in Fig. 1, can be used to investigate and select the most desirable configuration comprising three factors with two choices in each. In the popcorn example, four conditions prescribed by DOE that will point to the most desirable parameter combination can be laid out by replacing the character notations A, B, and C with the factors in the table (Fig. 2). Choices for each parameter are shown below.
Figure 1. L-4 Array
In the popcorn example, to carry out the experiment, the machine is operated under the four test conditions. A simple analysis of the results (for example, the amount of popped or unpopped kernels) helps determine the best settings for the machine. The predicted condition is then confirmed by testing a few more samples just in case some mischievous variables sneaked into the tests.
Figure 2. Test Lay Out
For the popcorn vendor, the bottom line is that his machine is set up for optimal performance on the basis of only four tests. As the number of possible combinations increases, potential savings from using DOE grow dramatically. Shift from the popcorn vendor to the automotive parts manufacturer whose profit margin has been shaved by competition to a dollar per part. He contracts to deliver a million parts. If his design engineers fail to foresee and prevent problems on the production line, or worse, if failures occur once his customer delivers vehicles containing faulty parts, the results can be disastrous. Nutek Inc.