Summary of my knowledge about statistics or hypothesis test in particular

Reason for summary
I feel it's time to stop looking into more statistics or hypothesis test in particular. Anyway I'm not going to work as a statistician. I should position myself as marketing analyst with solid knowledge of statistics and capable of conducting complicated hypothesis test for marketing test.

List of my knowledge about hypothesis test

• parametric testing
• one sample test (z test and t test)
• two sample test (mean test and standard deviation test)
• one way ANOVA
• two way ANOVA (full factorial design and fractal factorial design - Taguchi method)
• non parametric testing (normal distribution test)

The next question is what is the value to marketing. Maybe it is easy to explain it in design of experiment. hypothesis test is one step of design of experiment

Marketing campaign can be considered as one type of experiment with input as marketing mix (4P, price, place, promotion, and product), output as goals (revenue, profit, market share and etc) and market itself as black box. After you have insight about the market, you might find a few key factors. You will try to find the optimal combination to maximize your goal with minimum cost. That's where hypothesis test comes in and help it.

Application,
One interesting application might be in Google ads optimization. Here is an very interesting youtube video to find the best ads with different heading, first description line, second description line and url. One thing to note is that, in practice, people care about the optimal level of the factor, rather than know the factor really has impact on the experiment.

P.S. From tomorrow, I need to focus more on marketing strategy and cases instead of techniques in statistics. Don't be a nerd.

Buzz words in job requirement - Taguchi method

The first time I heard "Taguchi method" is in the six sigma course. It's not covered by the intense course. But the professor did mention it several times.

I kept on searching for details about the method. It seems not well defined even in Wikipedia. Maybe the term itself is not clear or accurate. The exact term might be "Taguchi method in design of experiments". If that is the case, I think "Taguchi method" is just orthogonal analysis in fractal factorial design.

the Idea
In simple English. "Taguchi method" is the method to use smallest runs of experiment, which neglects effect of combination of factors.

If effect of combination of factors is not our interest, we can use 8 runs of experiment to represent an experiment with up to 7 facors, which is far more effective than full factorial case, 2⁷=128

Design of experiment
I'm not sure if should use this as a heading. It covers lots of topics and Fisher even wrote a book about it in 1935. Here I will copy the rule of thumbs about options in design of experiments from one online document
as the summary of this blog.

When do we use which method?

• Option 1: Factorial Design
• Small numbers of variables with few states (1 to 3)
• Interactions between variables are strong and important
• Every variable contributes significantly
• Option 2: Taguchi Method
• Intermediate numbers of variables (3 to 50)
• Few interactions between variables
• Only a few variables contributes significantly
• Option 3: Random Design
• Many variables (50+)
• Few interactions between variables
• Very few variables contributes significantly

Buzz words in job requirement - fractal factorial design

Here is the buzz word I used in my experience as project officer in charge of an experiment. To relate the experience to marketing test, I have to use to some buzz words. "fractal factorial design" seems complicate, and the professor of six sigma mentioned the course. So, it must be useful somewhere in some way.

First of all, definition from Wikipedia.

In statistics, fractional factorial designs are experimental designs consisting of a carefully chosen subset (fraction) of the experimental runs of a full factorial design. The subset is chosen so as to exploit the sparsity-of-effects principle to expose information about the most important features of the problem studied, while using a fraction of the effort of a full factorial design in terms of experimental runs and resources.

There are key terms here, "fraction" and "full factorial design".

"full factorial design" is the way to list all the combination of change in each factors to take all factors and combination and factors into consideration.
For an experiment with 2 factors, source of errors/difference in observation comes from X₁, X₂ and X₁X₂

For an experiment with 3 factors, source of errors/difference in observation comes from X₁,X₂,X₃,X₁X₂,X₁X₃,X₂X₃,and X₁X₂X₃

It's simple. But when number of factors increases, runs of experiment increase exponentially. it's 2ⁿ. In some cases, it's extremely expensive to conduct the full factorial design.

That's where "factual" comes. Number of runs does not increase when new variable comes in, but some combination of variable need to be compromised. Here is the example.

Conclusion of fractal factorial design, full factorial design is the dumb but expensive method. fractal factorial design is the cheap method with the trade off of cost.

P.S. I need to related it to the context of marketing. The technique can be used in design of marketing campaign test. For traditional marketing campaign, fractal factorial design might be necessary because it's expensive. But for online marketing campaign test, it seems full factorial design is possible.

P.S.2 Here is a youtube video that might be useful to explain fractal factorial design.

Buzz words in job requirement - ANOVA part 2

I am afraid there has to be ANOVA part 2 if change of more than one factor is conduct simultaneously. Another reason is that the title with part 2 is so cool. It sounds like a Kill Bill 2, Harry Potter and deathly hallows part2.

1. What problem can cool ANOVA part 2 can solve?
In one-way ANOVA, samples are different due to change of one variable. What if there are more than one factor/variable? In this case, there are 2-by-2 matrix, and each cell contains one sample with several observations.
In the context of marketing, imagine the marketer design a campaign with input of ads spending and discount level. He/she is not sure about how to fine tune the parameter, so he/she set different levels for each factor. impact of market is recorded. The question is if market really respond differently as factors change.

2. Idea of two-way ANOVA
Assume there are only two factors. And assume means of these samples are identical. Errors of each observation comes from four sources, factor A, factor B, interaction between A and B, randomness.

SS_A, SS_B and SS_AXB are all compared with SS_within. If they are relatively large, they causes difference more than randomness.

3. Formula of two-way ANOVA
Of course, there should be formula when there is no analyst supporting you. You can find formula here.

Buzz words in job requirement - ANOVA

One of the popular buzz word is ANOVA, which is acronym of analysis of variance.

1. What type of problem can be solved by ANOVA?
There are several samples, which could be generated by different methods. ANOVA can test if means of these sample are different. In the context of marketing, several campaigns are run among different countries and impact of the campaigns (revenue maybe) is recorded. ANOVA can test if impact of campaigns differs among different countries.

2. ANOVA v.s. t test of means by two groups
If there are only two groups, t test can be used instead of ANOVA for simplicity.

3. The idea of ANOVA
If means of the sample groups are assumed to identical, then error of each observation is due to two reasons. One is the error within sample, the other is between samples.

when SS_A is relative larger than SS_withinwithin, we can say that means of different samples are different.

4. Formula of one-way ANOVA
Formula is always boring. You can find it here.