Most marketers know about Split testing - the process is one that everyone should know about.  But I have been seeing more and more is the Taguchi Multivariate testing...

I was wondering if anyone has some resources that I can refer to on this...

Ah ha! Someone else who's interested!

I've been involved with Taguchi Multivariate Tests thru the company I work for. We use an outside marketing company for some of our marketing...funny how that works huh?

Anyways, what are you trying to learn about?

I suggest checking out this site for starters.

Right now I'm just trying to get a grasp of how it works.  I know its results, but I do not know the functions at work specifically.  The idea looks great, though.

Yeah. Its pretty cool. Testing the slightest changes to make a big difference....

Its kinda like the chaos theory...

Interesting results, too.

I am an OLD timer in marketing an just LOVE a fad…what I mean is that there is always some bloody naming convention for the same old sh_t.. go figure though, it’s done by marketers right? Lol

Anyways you can make up as many variances of split testing as your little heart desires, that’s the fun of it actually. Then add some other elements into the pie such as USP testing, branding elements and it goes on forever.
Split testing shouldn’t be limited to ‘mere’ conversions. We adapt our A/B software for all kinds of purposes. I love the stuff myself….

One small issue people should consider is SE’s. It can be a tricky thing to avoid the duplicate content penalties and some minor ‘indexing issues’. This often means the testing is done OUTSIDE of the view of the SEs (limiting the bots) which presents it’s own set of marketing challenges.

So Taguchi-shmaGoochie, learn and put it into practice. Grab a couple different A/B testing applications for your site and see where it leads. That’s from the cranky old fella at least who has heard enough ‘naming’ conventions over the last 15+ years and only cares about what works. I find each industry/market needs a different approach to split testing and as such should try different tools and methods to find what works….

Great to have another Analytics/Conversions chat going ,,,, 3rd in as many days… woo woo .. Great for my next company…. (conversions)….

Actually it does beg the question of discussing MST before even a Split Testing discussion... lol...cart before the horse for the uninitialted no?

OH, I also noticed an important point on the site, TRAFFIC. You certainly need larger numbers for MST than a traditional ST program. A low traffic site COULD use the approach, but the time it would take to GET the numbers may be too long to accurately get a 'snapshot' of the user group.
I am not certain of its full effectiveness on limited viewing. Which also touches on the need for a targeted PPC for pages without a core traffic organically. This limits the user views and thus further dilutes the end data.

So as with everything, it woks for some situations and not for others.

OK as a follow up, for those whom care, I just got a recommendation from some one who would know this far better than I and the advice for Split Testing and Indexing is to ensure that the Googlebot is included in the control group.

This means to EXCLUDE via robots.txt, your testing page. Just ensure the test page is not indexed. You want to ensure the control group page IS indexed of course so that your ST activities do not adversely affect your SEO efforts...

thegyspsy wrote:

I am an OLD timer in marketing an just LOVE a fad…what I mean is that there is always some bloody naming convention for the same old sh_t.. go figure though, it’s done by marketers right? Lol

Anyways you can make up as many variances of split testing as your little heart desires, that’s the fun of it actually. Then add some other elements into the pie such as USP testing, branding elements and it goes on forever.
Split testing shouldn’t be limited to ‘mere’ conversions. We adapt our A/B software for all kinds of purposes. I love the stuff myself…

From my understanding, Taguchi testing is not just another fad...in fact, its pretty heavily used in industrial practice.  If you want to see a stats-laden industry, just look at heavy industry.  Again, from my understanding, Taguchi is MUCH more than just a simple A/B split test.  It does acheive somewhat the same things, but it gives you the ability to identify many more things in a much shorter amount of time with a smaller time-frame and less traffic than A/B testing is able to provide.

Now I know you say you are and old time, but I wouldn't say so.  You've been around for a while, but you certainly aren't an old timer.  From the little research I've done, Taguchi deserves some investigation.  If it allows you to test more variables in a shorter amount of time with the effectiveness (if not greater effectiveness) than A/B testing, then why not explore it?

Oh no.. ya missed me there.... I was merely talking about 'naming conventions' , that's all. It is really better stated as Multi-Variance Split Testing etc... I just gag at things like 'Taguchi' that all.

It deserves more than consideration, it deserves to be understood. I actually spend a great deal of time (when available) on that website ( http://www.marketingexperiments.com ) so I love it.

What the rest of my rant was, is just the usual multiple data sources and the fact that you DO need a larger traffic base as it is merely SPLIT TESTING variables separately. To do that you must juxtapose;

Control Group V Page A (single element change)
Control Group V Page B (single element change)
Control Group V Page C(single element change)
Control Group V Page D (single element change)

So we need 4X the traffic as we would for a simple A/B test. ..see?

Ahhh...right, gotcha.  I referred to it as Taguchi because that's how I got introduced to it.  Multi-variate does seem a little more appropriate.

I am beginning to understand the basic concepts of Multi-variate testing.  I still haven't had the solid hour that I want to spend reading up on it, but I'm getting a hold of some of the basic concept.

SiteReference wrote:

Ahhh...right, gotcha.  I referred to it as Taguchi because that's how I got introduced to it.  Multi-variate does seem a little more appropriate.

I am beginning to understand the basic concepts of Multi-variate testing.  I still haven't had the solid hour that I want to spend reading up on it, but I'm getting a hold of some of the basic concept.

It’s not really about MVT but did ya read the games I play with my Home page?
http://forums.site-reference.com/topic/ … es-part-3/

I love this stuff.. that's why I am starting the new company I was telling U about for analytics/convversions etc.. SEO is fun..but it only get's them to the door... it's not the 'end-game' for me. The analytics/conversion stuff is the real meat...

You dont need 4 times the traffic, though. If you take your example and have:

Control Group V Page A (single element change)
Control Group V Page B (single element change)
Control Group V Page C(single element change)
Control Group V Page D (single element change)

Then you rotate those pages equally; 25% of the traffic to each page. Since each page still gets the same amount of traffic, you are now testing 4 DIFFERENT element changes on the SAME amount of traffic.

MarkCCDC wrote:

You dont need 4 times the traffic, though. If you take your example and have:

Control Group V Page A (single element change)
Control Group V Page B (single element change)
Control Group V Page C(single element change)
Control Group V Page D (single element change)

Then you rotate those pages equally; 25% of the traffic to each page. Since each page still gets the same amount of traffic, you are now testing 4 DIFFERENT element changes on the SAME amount of traffic.

Ha ha ha..lost U huh?

It's not about IDENTICAL traffic. I mean you need 4X as much traffic to set the same base line as a standar ST. it is true my friend... do the logic (or just read the page YOU posted...they talk about it there..lol)

Yes, if you were trying to set the same base line as a STANDARD ST you would need 4 times the traffic. But if you have 4 different pages with a small element change on each, split the traffic 25% to each page. Then you can tell the effectiveness of each page.

A common change is the headline on your page. Create 4 different pages, each with a different headline. Split your traffic EQUALLY (it must be equal or it doesnt work), 25% to each of the 4 pages. Look at the conversion rates, CTR, or whatever it is you are trying to improve.

Test 1 - Headline Change:

Page A: 0.85% CR @ 25% Normal Traffic
Page B: 1.4% CR @ 25% Normal Traffic
Page C: 0.79% CR @ 25% Normal Traffic
Page D: 0.99% CR @ 25% Normal Traffic

This tells you that whatever headline change you made to Page B caused that page to convert better than the other 3.

The next test you would do would be to keep Page B, and create 3 more pages from Page B with a DIFFERENT change. So, now you would have:

Test 2 - Logo Change:

Page B: 1.4% CR @ 25% Normal Traffic
Page E: 1.1% CR @ 25% Normal Traffic
Page F: 1.8% CR @ 25% Normal Traffic
Page G: 1.4% CR @ 25% Normal Traffic

So, this shows you that with the new headline from Page B AND the logo you had on Page F gives you the best conversion rate.

Small, simple tests can help you increase your CR, CTR, or whatever you are trying to improve.

I am new to a lot of this, but that sounds a lot more like a split test rather than a MVT test - am I right?

Well, its not an A/B Split test, but in order to make it a MULTI Variable Split Test, then take the examples I explained, make a few different changes on each version of a page (instead of just one change at a time).

A/B/C/D MVST is the most common test that I have seen - it seems to be the most up and coming thing.

Next year there will be something new

SiteReference wrote:

I am new to a lot of this, but that sounds a lot more like a split test rather than a MVT test - am I right?

That's all a MVT is really. It's just testing MULTIPLE factors as opposed to a simple A/B testing. As the example above shows, we have the Control group and then 4 parameters being tested. So a A/B/C/D test instead of an A/B test...lol

This is why it requires more traffic than a standard ST as you need to test MORE variables with the SAME amount of visitors. The number of visitors to the PAGE in question won't change but the parameters being tested DO.

So, the more variables in the MVT the longer it takes to get a base line in comparison to the same page and a simple A/B testing (Split Testing).....

Dave wrote:

So, the more variables in the MVT the longer it takes to get a base line in comparison to the same page and a simple A/B testing (Split Testing).....

Which is why I posted the A/B/C/D Test as a one element change. That way, splitting the same traffic into 25% increments and sending each 25% to a different page gives you a snapshot of how each page would do with 100% traffic; thus requiring no more traffic to do the tests.

MarkCCDC wrote:

Well, its not an A/B Split test, but in order to make it a MULTI Variable Split Test, then take the examples I explained, make a few different changes on each version of a page (instead of just one change at a time).

A/B/C/D MVST is the most common test that I have seen - it seems to be the most up and coming thing.

Next year there will be something new

I agree totally with 'Next year there will be something new'. Read learn...move on. Great tools and great fun but just another piece of the puzzle


"make a few different changes " -- Actually there is the problem that makes MVT tasking for smaller traffic sites. You have to change ONE variable on each test page. That is the ONLY way to tell what individual changes make.

It's kind of the CORE purpose of MVT. If you changed some text, a background color AND an image, how would you know WHICH change made the difference? You wouldn't. So the MVT test makes ONE change per page and tests the control group against the variations...

Hence my first example of the A/B/C/D Test of ONE element change.

If you got a million page views a day, creating 4 sites with 3 changes on each would still constitue as a MVT, so my explanation is still correct. We're just thinking about LTS and STS (Large Traffic Sites and Small Traffic Sites).

But yes, you are very correct in saying that more than one change on a page wouldnt tell you what made the difference.

This is my 300th post!