What would happen if we changed our web site such that no free demos of any games were offered? I.e. what if the site were designed with the singular goal of convincing visitors to buy right away? So instead of having a sales funnel that looks like vistor->downloader->customer, we'd just have visitor->customer. Download links from all other sites would still work, so we'd still use the shareware model elsewhere, but if someone came to our site from a search engine, the site would try to close the sale right then and there. We'd have more detailed sales pages for each game, but no freebies.

I've long pondered what would happen if we did this, and I've done a few experiments over the past year that all suggest the possibility that sales would increase. But this is a big change that can't be easily done with a simple A/B split like some more modest experiments. It requires a lot of changes throughout the site.

Some of the advantages of having no links to demo downloads on the site:
- The experiment has a finite downside but a potentially unlimited upside. If it fails, it means bad sales for a few days and some wasted time on site redesign; the site can be reverted to its original form in a matter of minutes. But if it succeeds, it could mean higher sales permanently.
- Less reliance on the conversion rate of developer demos. If a demo doesn't convert well, we have to work through the developer to change it, and that's not always practical. But a poorly converting sales page can be tweaked more easily.
- Less tech support. No local demo downloads means we do more tech support for customers and less for downloaders who never buy.
- Lower bandwidth costs.
- Better ability to focus on serving customers instead of downloaders who never buy. People who never intend to buy or who aren't capable of buying will simply leave.
- Fewer problems with cracks/piracy. No demos to crack here.
- Solves problem of demoitis, where customers who might otherwise buy a game get distracted by all the free demos and end up downloading another freebie instead of buying.
- The conversion rate of visitors to customers could increase substantially. The sales funnel is shorter and has fewer steps, so perhaps fewer people will drop out along the way.
- Site design is simpler and more unified. All product pages are designed with the singular goal of selling a game instead of the dual goal of selling a game while also encourage a freebie download.
- People can still download demos from other sites like download.com, so we still benefit from those free demos bringing in new customers. But when they get to our site, it would try to convince them to buy instead of downloading something else for free.
- We gain the option of selling full version games with no demo at all. For all our Silver deal published games, we wouldn't even need a demo from the developer. Thus, our QA cost would be much less.

For the downsides, here's what I see:
- We'd have to target different search engine keywords to bring in better quality traffic. "Free game downloads" wouldn't work if we don't offer any.
- We might get a surge in complaints from people looking for free demos. However, this might be an advantage, since it gives us the chance to push our unconditional guarantee.
- The experiment may fail, but again, this is a temporary condition and can be corrected. Worst case, we lose some time and some money, but those are finite losses compared to the potential upside.

Some web sites mainly push for the demo download, hoping that the demo will close the sale. Other sites primarily push for the sale, while still offering prominent demo download links. But I really wonder what would happen if the site doesn't give visitors the option of the free download. After all, this is how most retail games are sold. What would this change do to the conversion rate? I've seen some suggestive results from other developers, but none of them sell games.

I don't know the answer. But I've decided that I must find out. I think there's enough of a chance that it could work, and the risk of loss is minimal. So later this week, all free download links will be removed from our site, and the site design will be changed to reflect it. If the experiment fails, we should find out rather quickly, and we can change everything back. Even if I think there's only a 10% chance it will work, those are still good odds, since the potential upside means permanently higher sales. So I think it's a good bet.

Has anyone else here tried or considered trying anything like this?

I like it...

The analogy of not having downloads to test would effectly back them against a wall to buy or leave, with no middle ground. Either buy something that looks appealing, that you've already tried out perhaps somewhere else or just leave. No more wasted bandwith.

This is interesting as there isn't any middle ground anyways in reality. You can buy the game OR never buy it, something like -checking out the demo forever indefinitely - doesn't exist ultimately. You will buy it or not in the end.

This adds leverage and maximizes your unconditional 60 day money back guarantee and could very well bring in more sales if the 60 days guarantee is truly an effective incentive for a user to buy a game online anyways.

_Shareware_ this wouldn't be anymore, it would be like "pitchware", make the convincing sales pitch through words and guarantees, screenshots.. and hope they cross over with their wallets to the BUY side or just leave the site.

Try it out and give us the results.

Wish we could do that. I do, however, believe that the competition is concentrating on just getting their feet in the door by free demos. Essentially a timed demo would give them such an "unconditional" guarantee.

I could almost envision a time when one could offer a "standard" version of the game for free and have them purchase expansion packs for $9.99 each. Gets them addicted to minimal levels, but then drives them to get more. Gives you the chance to really out FREE GAMES.

This is really interesting Steve and I look forward to the experiment.

If I try to put myself in the shoes of a potential customer, I find myself asking, "What am I getting out of this change at Dexterity.com?". Am I getting higher quality games? Cheaper prices? A longer return policy?

As a customer, I would feel like you're taking something away and I would want it replaced. I want the negative (from a customer's perspective) of not having a demo to try-before-I-buy to be replaced with some sort of new positive (for me the customer).

When I was a developer/owner in the retail CD game industry, it always kind of irritated me how the gamers started "demanding" demos for every game (remember back when there were very few game demos for retail games?). Of course, as a consumer I didn't mind. :) But, because of that, I would love to see this experiment succeed.

Keep us posted!
-Scorpio

I have wanted to know the answer to this question for a long time, Steve. I can't wait to see the results.

Some of your pros and cons I do not necessarily agree with, but who cares- only the numbers will matter and speculation is pointless.

I hope you have a good way to tell whether users have downloaded a demo of a purchased game, otherwise the initial numbers will be skewed.

For example, let's say in an average week, there are 100 dweep sales- all from people who downloaded the demo. Now the first week you make the switch to no downloads, many people who previously downloaded the demo will purchase, let's say 90 do. Then 50 people buy the game sight-unseen from the site, creating 140 sales for the week. That's better than 100 sales, but only in the short term. Because as time goes by, there will be less people purchasing from the demo...

Just something to keep in mind- but I am guessing you considered that already.

Hi,

I'm thinking in using this no-demo approach for selling the games of my small company, Moraldo Games. Actually, I have very few alternatives for this, as I think the highest value of my games would be immediately lost if I gave away a free demo.

The point is that the games we do are small, yet powerful in our opinion for being highly innovative (or that is what we try to do!). We believe a very specific public that's looking for games that are very different to what is known would buy our games by paying not for the fun only, but mostly for having a chance to see 'that new thing'.

We think we can tell that public what the games are about while not killing their curiosity, but if we gave them a free demo for playing, they would lose the most important reason for buying the game in the first place: knowing the way that new experience feels!

That is the main idea behind our strange games... currently we are working in the development of the first one of the series, Randomedia Concept. We haven't had a chance to check whether this kind of business works or not, but in the next months we should be making our first final release so I might be giving you some feedback very soon!

For those interested, there is a screenshot of the game in the next url: http://www.hhm.com.ar/varios/sshot9.png

The same games could be sold in some other extra ways, for example, in the retail CDs that contain many games in one (classic 50-in-1 CDs and the alike)... and with some differences to make the games simpler, they could also be sold as shareware if everything else fails (though the conversion would kill most of the innovative concepts, of course).


Any thoughts?

Best regards,

I'll be happy to share the results of this experiment. I agree with Mike that the short-term results may differ from the long-term results. If the short-term results are negative though, that indicates that the long-term may only get worse. We get enough sales every day that it shouldn't take more than a few days to know if the short-term results at least look promising. But it would have to continue for several weeks to really have a sense of the long-term effect. It's possible the effect may be very large, such as a doubling or halving of sales.

I think we're in a good position to try this experiment now, since we already have a good flow of traffic, and since all the product sales pages are database-generated, we can change a single template and have most of the pages reflect the changes instantly.

Basically all we're doing is trying a purer form of direct marketing / direct sales, an approach that clearly works well for other products, including software. But I sometimes think that games are a different animal... that maybe games don't necessarily follow the same rules. When your product's purpose is to entertain, it can be harder to express the emotional benefits with text and images alone. But I still think it can be done. A video preview (like a movie trailer) might be a good alternative to a free demo.

The original shareware model was developed before the internet was in vogue, and this model established itself during the age of BBSes and CompuServe, when shareware was mostly sold via mail order. But now that developers sell primarily online and we have multimedia-capable web sites in our arsenal, I wonder if the try-before-you-buy model is a suboptimal choice for selling via a web site. I still feel it's a terrific way of getting distribution, as demos on other sites can clearly generate a lot of traffic and sales. But when someone comes to the developer's site directly, perhaps it's best not to pull out the freebie links so readily.

As I see it, it's impossible for this experiment to fail. Either I'll permanently increase our sales and have a learning experience, or I'll just have a learning experience. :)

While I've never actually tried eliminating the demo download completely, I have tried "increasing the focus" on the buy aspect. Right to the point of hiding the demo download link (middle of the page, small text etc).

I found sales increased slightly, but the return rate went up as well. The return hassle didn't justify the revenue increase. The return rate went up because:

#1 Users didn't know if the game would run on their system. They purchased it, and it didn't.

#2 Users didn't like the game for one of many reasons. (Enough to keep it for more then 30 days).

Granted, most of my current games were originally destined for retail. The system requirements were planned higher (too high), with the focus on making the games look interesting for box shots, not "long term value".

As I replace my current games with new ones designed exclusively for online distribution, I will be returning to the "push the sale, hide the demo" model. I've found it to work better.

For Steve: Eliminating the demo altogether might not be the best idea. I suggest simply "hiding" the demo link and pushing the sale more. If you get positive results, you can remove the demo entirely and test some more.

Michael Sikora
Guardian Games (http://www.guardiangames.com)

I'm looking forward to see how this experiment turn out. (I hope you'll share it with the rest of us :D )

It's strange because I have really mixed feeling about something like this.

As a gamer, I feel like it's not a good idea because as Micheal mentionned, player will not have the chance to try the game and therefore will feel less guilty to ask for a refund or might even be aggresive toward the game saying that it's not what they thought it was while with a free demos they can usually only blame themselves and some of them will rarely ask for a refund because of that.

On the other hand, as a producer, it sounds like a great idea because it would definitively get rid of those who cause the most trouble usually (those who never buy) and could indeed increase sale.

It's sadly not until a couple of months that you'll probably be able to tell for sure which one is the best.

My only wish is that if it fails, it won't drove away some of your regular customer.

Nick

One thing I really appreciate about you Steve is the fact that you are always ready to be a "wind dummy". In hang and paragliding circles that is the fellow who is the first to rush off the hill to test how the weather is shaping up. That is the fellow who has to climb the hill most often, as the weather is rarely perfect on the first try. But that is also the person who gets a lot of exprience and is usually the one with most flight hours clocked. And really good muscles from climbing the hill .. :)

I have been toying with similar ideas and am really interested to hear how things shape up.

There are many games (especially in the 'casual' market) that I can't see myself EVER paying for if I hadn't had the opportunity to become hooked and dependent on the game through the demo.

Other kinds of games, I would be happy to try without a demo if there's a nice friendly-looking returns policy. :) Non time-waster games, if that makes sense... RPG, Adventure, certain kinds of platform I'd be willing to try out based solely on screenshots, info, and if possible, little animations / game movies to show how it plays. And I'm a sucker, so it'd probably have to either totally not run on my machine or personally offend me before I'd ask for a refund.

Still, most games that come across this board I would never consider buying for a second without having gotten addicted to it through free play.

I think it's a very good experiment. And please tell as soon as possible the results of the short term test. Thanks in advance!

It's not really an experiment since it's been done succesfully by several online companies already.

GarageGames : torque
Phelios : ptk
int13 : pocket pc games
and another game company but i forgot the name. ( ate too much mad cow :( i'm losing memory arghhh )

the good thing about this is that it will be more honest to your potential customers, I think. Dexterity's site actually mentions "Free PC Games Downloads" which is not actually the case since it's only the Demo which is free. That will get rid of deceptive advertising and surely results in more satistaction for the customer.

patrice.

I often put myself in the customer's position: would I go to a site that sells new games without even being able to try? no, of course :)

It is different if you put the demo on other sites: but which sites?
Pls tell me, I wasn't able to upload the demo on any other site, all were using a link to my site :confused:

Apart for that, the experiment could work. To be honest isn't really an experiment: with full commercial games works already this way since 2-3 years.
You see demos of the games on CD magazines or websites, and if you like it you can go to the homepage of the publisher and order a CD directly from them (or in the upcoming HL2, you can download the game).

Personally, my instinct is that cutting out the demo download would work best for games with web-playable teasers. The current model there seems to be:

web version -> upsells trial download -> upsells purchase

The web version is almost deliberately crippled, in order to prevent people from getting their fix on it after the trial expires. Cutting out the trial download could work well, here, since there's still a demo of sorts available. Customers will know if they like the game's mechanics before purchasing a more content-rich version. Of course, it could backfire completely in this situation, since potential customers have a known free game and might be too risk averse to pay for anything more.

If it does produce more sales, an interesting question is how succesful it would be for other sites; in particular, for small shops trying out 1st-party sales.

I was mulling an experiment that solved a few of the aforementioned problems with compatibility and so on. I was considering making Alien Flux into a demo attract mode game. I'd record several games within the game and then the demo itself simply played through them on attract mode like the arcade games of old. You couldn't actually play it yourself, you'd have to just watch and decide to spend your money there and then based on what you saw.

The advantages of this method:

1. Customer knows the software works
2. Customer knows what the game looks and plays like
3. Customers who pay for things up front are often more determined to like them
4. No further downloads required - they've already downloaded the full game so it can just be unlocked

The disadvantages:

1. Customer still doesn't know how it feels to play
2. Probably essential to have purchase GUI built in to the game; when you decide you want to play you don't wander out of the arcade and give 50p to a bloke over the road do you?
3. Likelihood of increased returns
4. The technical troubles of recording an upselling demo are non-trivial. You can't just play through normally from level 1 to level 10, you have to pick moments of interest. Etc.

But I'm not thinking of trying this yet until I've got a bunch of games.

Cas :)

Steve,

Didn't you try this when you first published Aargon Deluxe Gold here a year ago? What were the results then?

Originally posted by APE
Steve,

Didn't you try this when you first published Aargon Deluxe Gold here a year ago? What were the results then?

Yes, that was one of the experiments, but it wasn't a really good test, since most of the early sales were made to our previous customers and newsletter subscribers. Also, it was our first release after about 8 months, and it was a virtually perfect fit for our previous Dweep/Fitznik customers. Shortly after the game's launch, a demo was added.

I already know that we can convert newsletter subscribers and previous customers very nicely with no free demo. What I don't know is whether we can do this less qualified general site traffic. And that's what I want to find out.

Originally posted by Jack_Norton
It is different if you put the demo on other sites: but which sites?
Pls tell me, I wasn't able to upload the demo on any other site, all were using a link to my site :confused:


The demo files would still be hosted on our site, so other download sites would still link to us for the files. However, we wouldn't post links to those demo files on our site.

I was poking around real arcade yesterday, checking out the different games. I don't trust real client applications very much so I’ll probably never install their arcade, but one thing they did have, was for each game you could watch a real movie stream of the gameplay. I found that was enough information for me to decide if I liked the game or not and make a purchase decision. One of the benefits of this movie is that you can show content from all over the game not just the first 5 levels.

I've got a totally awesome movie of Alien Flux being played which was used at JavaOne to wow punters. It's a bit big though at around 57mb...

Cas :)

Originally posted by Dexterity
What would happen if we changed our web site such that no free demos of any games were offered?

It's an interesting idea, Steve, and I'd sure like to know the results. I can't argue with any of your points, as they *could* be true. The only way to know is to test it.

I've considered doing it myself, although on a separate site. Doing it on the primary site would be a big job, although if you've got everything set up the way you do it wouldn't be so hard.

The problem with watching streams of the gameplay is that this wouldn't work well for puzzle games whereas I can see it working well for action games. Watching a puzzle game being played just isn't as exciting as trying to solve the puzzle yourself. That and it just sounds boring for most games. Perhaps if there was some commentator explaining how the game works it might work.

Anyways, as to the topic at hand I'd have to say there are two camps. Those that aren't interested in buying games and are simply looking for fun and those who come looking for demo games to try. You can sometimes convert those in the former camp, by wowing them with the gameplay. The idea you're discussing cuts out those in the first camp but it sounds like it might increase purchases in the second camp. I'm one of the first camp myself (well that and research), although I have been convinced a few times (although very few).

That said, I just have to say bravo if you try the idea out.

I agree with the last post. You are going to lose money on people who are looking for a free game but then get hooked. However you might get more people who are willing to spend. I can't wait to se what happens.

I think this is something that is more pratical for some place like Dexterity that all ready has traffic and a history. Even if this works for Dexterity I still wonder if it will work for a smaller site, or for a first game.

You may want to try putting a little compliation of the game playing into a small media file (avi, mpeg, wmv) instead of offering a demo download. Maybe that could be the second "part" of your experiment. Instead of offering a portion of your game you can still give people an idea of how the game plays.

I agree with BlueWaldo, this may work great for Dexterity because Dexterity has a really good 'rep' for having fun games that are worth your money, but for a smaller game company with 1 or 2 games and no 'rep', I can't see customers risking their money on them. Another downside is, many more people may get more use out of your 60 day money back guarantee, so even if the first month looks good, you may get a lot of returns the following month. With that being said, I'm still very interested to see what happens... Good Luck Steve!

Am I the only person who absolutely hates this idea? :mad:

Honestly, I hope you find out it doesn't work at all, because I see nothing good in it for the consumer. It also seems like something of a bretrayal of the Shareware heritage to me! Maybe that's a little harsh... I realize you are in business to make money, but think about it from your customers' point of view.

A successful restaurant, once they have established their popularity, may increase their profits substantially by buying inferior ingredients to what they initially used... but is it the "right" thing to do? Are they still meeting the same standards they set for themselves? I could probably think of a better analogy... :rolleyes:

Anyway, I can't imagine doing this for any of my games... the concept of Shareware is too important to me.

I can't say I like the idea myself, since it means I don't get to play any demos, and I sure wouldn't buy any shareware game this way. I do find it interesting just because I want to know what will happen. But otherwise, I hope the idea doesn't work or catch on.

If this does work though, hopefully the demos from download sites will stay.

I'd expect that our return rate will increase a bit if no free demos are offered, but I don't expect it will be too great. The return rate is already pretty low, so a doubling or tripling or quadrupling of it would still be no problem.

As for a betrayal of the shareware model, I don't think I'd be betraying the model. I built my business on this model. Just a few years ago I was President of the Association of Shareware Professionals, responsible for helping to promote the concept of shareware. Shareware is a marketing method though, and like all marketing methods, it has a certain scope within which it works. Currently I think it's being stretching to act as a selling method too. When someone comes to your web site, you don't need a marketing method; you need a selling method. Your marketing has already worked -- the potential customer has entered your store. Now it's time to close the sale. I think freebies should be used to bring people to your site, not to give hand-outs to people who've already arrived.

Sometimes the mechanism that brings people in the door isn't the same mechanism that closes the sale. I'm aiming to get the best of both worlds: shareware for off-site marketing and distribution combined with great sales copy and enticements on-site for closing sales.

Think of any retail business, like auto dealerships. Newspaper ads, TV commercials, and word of mouth bring people to the through the front door. But once there, commissioned salespeople work hard to close sales. Different mechanisms are used for marketing and selling. My thinking is to use shareware mainly as a marketing method to bring people back to dexterity.com, while the web site focuses on closing sales.

You may want to try putting a little compliation of the game playing into a small media file (avi, mpeg, wmv)


Although this would be quite a bit easier than making a self-playing demo running with the game engine, I do think that doing it with the game engine would be a much better solution, because:

a) The download will likely be much smaller, the demo can be longer, and the visual quality much better.

b) You have the option of unlocking it with a code, if that's the way you want to do it.

c) Any compatibility problems come to light before purchasing.

Wouldn't this only work for well-established shareware businesses? It wouldn't necesserily work for someone like me... I've had one game out (well that i can count) and it sold a few copies... But me not putting up the links to the demo wouldn't probably work all right... (of course now I no longer have a domain name) but still...

The more this idea is discussed, the more I like it. I think Steve is hitting the point, the site should close the sale, not do marketing. I am running a new business with a single product and I am amazed by the amount of people that buy the game without downloading the demo. I might try this myself, to give a different datapoint.

Steve, I am directed to download.com but I want to see the experimental look of the site. Can you help me?

ergas

I am willing to try something different but somewhat related to this thing: in future version of USM (sales have started to come, yes!!! ;)) I plan to add interesting features, like for example some 3D cut-scene of the match.
But those will be available ONLY in the registered version.
I'll modify the demo so that at the end of it the user see some screenshots of those cut-scenes, but nothing more.
This way I could get some sales only by players' curiosity :p

(of course I'll try to do my best not to disappoint them when they'll buy the game!! :)

For puzzle games for casual players maybe this will work, but for other genres, I tend to agree that there is nothing in it for the consumer. Id never buy a game I had doubts about even if they had a returns policy, beacuse as a hardened consumer I would presume its involved and difficult to get my money back.
I've always been suspicious of games that don't have a demo, as I'm wondering what it is they are tryiong to hide. Thats why I didnt buy the Sims or Black and White. Gamers are used to trying before they buy these days.
And I'm not sure the comparison with an auto delaer actually works. After all, how many of us have bought a car without a test drive?

Originally posted by cliffski
I've always been suspicious of games that don't have a demo, as I'm wondering what it is they are tryiong to hide.

Unless I misunderstood, people can still download and try out the games for free. They use a search engine or download site to find and download the demos. But they will no longer visit the Dexterity site for this.

The purpose of dexterity.com is just to make the sale. Most people have already played the demo and decided they like it. To buy, they visit the Dexterity site. There's nothing there anymore to distract them from buying.

So basically, Steve is offloading the 'try' part to other channels and focusing his own site just on the 'buy' part.

But I may be wrong.

I also don't know how well this would work. Some thoughts: Do many potential customers download directly from download sites? Do these sites provide enough incentive for the customer to click the download link? Or do they visit your website for additional information first (this is what I do, if only to make sure I download the very latest version). If so, not providing that info may lose you some sales.

You also can't have people end up on your site if they use Google to look for free downloads. (Hmm, this may actually save you money since you no longer have to bid on search engine keywords ;))

Maybe it makes sense to set up a separate site for each of your games where you offer information about the game, the free demo download, and a Buy Now link. This is what the various download sites and search engines point to. You also have another website -- your store -- where people can only buy the games, not download free demos. This is where the Buy Now links point to. One is optimized to hook people, the other for closing the sales.

Originally posted by ergas
Steve, I am directed to download.com but I want to see the experimental look of the site. Can you help me?


I haven't started the experiment yet. Some minor changes have already been made, but it will be another few days before the site is completely changed. And even after the demo links are removed, there will be more changes to improve the site's ability to sell.

If you're redirected to download.com, it's because you're from a banned country or banned IP range unfortunately. You should be able to use a tool like anonymizer.com to see it though.

Originally posted by cliffski
For puzzle games for casual players maybe this will work, but for other genres, I tend to agree that there is nothing in it for the consumer. Id never buy a game I had doubts about even if they had a returns policy, beacuse as a hardened consumer I would presume its involved and difficult to get my money back.
I've always been suspicious of games that don't have a demo, as I'm wondering what it is they are tryiong to hide. Thats why I didnt buy the Sims or Black and White. Gamers are used to trying before they buy these days.
And I'm not sure the comparison with an auto delaer actually works. After all, how many of us have bought a car without a test drive?

The key here is that in order for this to work, each product has to have a strong unique selling proposition (USP). Dweep's has always been, "A defenseless furry creature trapped in a hostile world." There's something about the dichotomy of a cute, fluffy, squeaky creature surrounded by things that could blow him to bits (bombs, lasers, etc) that makes the game sound fun to a lot of players. I think a detailed web page can indeed make this kind of game sound fun and unique, enough to get people to buy it sight unseen. Have you ever talked a friend pr coworker into buying a game sight unseen? How'd you do it? You made it sound fun, right? Same concept here.

But if you're only selling a generic FPS that's no better than any other, then the problem isn't the sales method. It's the product itself; there's just very little market for it. The best salesperson in the world won't get too far with a poor product.

Originally posted by Matthijs Hollemans
So basically, Steve is offloading the 'try' part to other channels and focusing his own site just on the 'buy' part.


Precisely.


Maybe it makes sense to set up a separate site for each of your games where you offer information about the game, the free demo download, and a Buy Now link. This is what the various download sites and search engines point to. You also have another website -- your store -- where people can only buy the games, not download free demos. This is where the Buy Now links point to. One is optimized to hook people, the other for closing the sales. [/B]

This would defeat the purpose. Getting a visitor to my site -- any site I own -- is like having a customer walk into a retail store. This isn't the time to give them a free sample and send them home. It's time to sell.

The purpose of the free demos is to bring people to the store/site in the first place. Once the customer arrives, the free demo has already served its purpose. My thinking is that free demos should be used primarily for marketing. Once a visitor comes into the store, offering a free demo is probably not the best way to close a sale.

Think of how this concept might be applied for a retail store. A store might advertise a promotion in the local paper, offering to give a freebie to each visitor to the store on a particular day. But the key is the advertising. It's not the freebie given away at the store that works -- it's the promotion of the freebie in other locations (newspapers, TV, radio, etc). That promotion and the offer of the freebie bring people into the store, not the freebie itself. But imagine if the store just gave away a freebie to everyone who walked in, but they never advertised the freebie outside the store. What good does it do them? They're just giving away value that people may be willing to pay for, and it could actually cost them sales.

Another problem with giving someone a demo at your web site is that you're sending the potential customer out the door. I think this is very dangerous, since as we all know, most of these people never return. Anything you give them to download where they'll likely close their browser to view it is dangerous IMO. My plan is to redesign the Dexterity site with a singular purpose. Whenever someone arrives on the site, the goal is to close a sale right then and there. And the idea is to offer such compelling benefits and such enticing offers that it's hard to resist the urge to buy a game. If someone leaves the site, they might come back, but I'm assuming they never will.

100 potential customers walk into your store. 99 of them walk out, probably never to return. I'm beginning to realize that this is a deplorable situation. To me, that isn't a failure of the marketing function. You just got 100 people into your store; that's excellent. But why did you close only 1% of them? Were they all just unqualified prospects? If they were all unqualified, then why did they enter your store in the first place? Or were a good chunk of them capable of buying a game, and you failed miserably to persuade them to buy? I think our problem is the latter... that maybe we're failing miserably at getting people to buy because we're using the wrong mechanisms to try to close sales (i.e. sending people out the door with a free demo).

Of course, we can debate this issue from all the different angles, and it will only make us dizzy in the end. The only way to know for sure is to dive in and do it.

Regarding 1% of 100%, there's another issue that we've come to realise with our game (because the game itself has an integrated chat room):

More and more we see people just looking for a free game period who do not have the capability to buy something on the net or some of them even if they have that capability are not interested in buying something from the net period (we refer to them as Free Loaders) and go from one free game to another.

They're basically the "Window Shopper" of the web industry. Those who enter into Zellers or Wal-Mart just to check the games and don't buy anything.

It seems like we're seeing more and more of this genre in the Web/Shareware gaming industry as the big publisher but a lot of money into advertising to disguise the lack of originality of most of their product.

And of course, the big game site will usually play follow the money as well (that's why I think we're hearing so much about the N-Gage while I think most gamer would agree that it's more like a gadget than a gaming console, Nokia probably paid big money to have these game site in their pocket) so Indie company are given less and less places. Just to give you an example, take Savage. An original concept but it's obvious that they lack the kind of marketing budget big game site required so we don't hear a lot about the game (but I'm sure they'll managing anyway since they already have a good community).


Of course, our game has only been for about 6 months so maybe it's just me but I remember that 3 or 4 years ago, it felt like the web-gamer community was more open to Indie games then they are right now.

Regardless of that, Dexterity is another proof that there's still a market for good original Indie game but by checking the forum it seems that it's becoming more and more difficult to reach a good market on the web and the more I think about it, the more I realise that removing the Demo will at least remove the non-buyer from the site.

Will it decrease sale, I doubt it. Will it increase sale significantly .. that, as I said earlier, I am not certain.

I could be wrong, but right now, that's what the all pictures of Web-gaming looks like from my perspective.


Nick

I've been thinking about this for a while now (well, since Steve P. made the initial post :)) and my conclusion is that this is overall a good idea for two reasons (in addition to the excellent reasons Steve P. has given). One reason is that I believe there are two types of people looking for games -- one type just wants something free to kill time and will never buy (the bulk of download.com users from what I gather), the other one is looking for a fun game and will buy one if he/she finds one. This may be a bit too black and white for some people, but I really don't think that someone who's subconsciously (or even consciously) set on not paying for a game can be enticed by even a very good demo to buy the game.

Not offering a demo on your site will disappoint the first type of person, but will at the same time increase the amount of sales you get from the second type. Since the first type would have probably not bought anything anyway, your sales will go up regardless of the fact that fewer people are downloading the demo. This is because instead of procrastinating buying the game after downloaing it (or possibly not downloading it at first, and forgetting about it and the site as Steve P. said, and never coming back) a customer would buy right away.

The second reason is that when you have a lot of games on a site (which means a lot of demos) a potential customer (someone who's type 2) might download a game and like it. But he or she might remember that there are more demos to try, and thus come back and try other games. In the end this customer might buy or might not because of the distraction of other demos.

As a consumer, my initial reaction was "GAH! Where's the demo!?!" but after thinking about it for a bit, I realized that I usually end up being a type 1 (someone who's just looking for free fun) when I'm downloading a demo. I don't need a demo to figure out if I'm interested in the game or not if there are screenshots and a good description. For the record though, I would not buy a game without a demo if it didn't have a good return policy. But with a 60 day return policy that's more than plenty of time to decide if you like the game or not. And chances are you will if the description of the game was very accurate. You will know that this is something you will like based on the screenshots and description. It might seem far fetched, but it's really not. You can't of course, know if the game is too hard or the controls are too hard without a demo but that's something that the developer should be addressing as you're looking at the screenshots. :)

I think I might try this experiment too then. You'll be able to see the perspective from a no-name-one-horse-startup, which will be enlightening. But buggered if I'm keeping a 60-day money back guarantee. If I'm going to make it operate like a retail store I'm going to use retail store terms. So it's 7 days money back guarantee.

One thing I'll be doing different is I won't be using the site to sell the game. I'll be getting the game to sell itself. It will still be the primary vehicle for sale as before, except it won't be playable, and all the necessary code to purchase the game online immediately will be built in to the game.

<edit: it has occurred to me this is remarkably like ReflexiveArcade's tactic. Most of their selling comes from within the game application, not the website>

Cas :)

Hello Steve,

I've been trying a similar experiment on my site for the last year.

Here are the results so far:

In 2000 I release a web game called Mancala Snails.
Please see: http://www.rocketsnail.net/mancala/

In 2002 I release a download demo of the game. This version was your standard try-before-you-buy demo.

In 2003 I decided to remove the download demo to save bandwidth. Now players can play the game online, or they can PAY to download a full version of the game. Surprisingly my sales continue to grow.

Personally I think you should create a single level web demo of Dweep for your site. This would allow players to sample the game without a large download, or installation.

Here is an example of web demo for the game Lost Vikings by Blizzard.
http://www.blizzard.com/blizzclassic/vikingsdemo.shtml

Good Luck with your experiment.

Lance

RocketSnail Games
http://www.rocketsnail.com

My question still stands... wouldn't this only work for a well-established developer?

If this holds I won't be able to recommend dexterity.com as a site with great indie games anymore.

Recommending particular games on dexterity.com will become more difficult too as providing a link to the demo download will become necessary (lest people google for the game and end-up on dexterity.com looking for the demo).

100 potential customers walk into your store. 99 of them walk out, probably never to return. I'm beginning to realize that this is a deplorable situation. To me, that isn't a failure of the marketing function. You just got 100 people into your store; that's excellent. But why did you close only 1% of them? Were they all just unqualified prospects? If they were all unqualified, then why did they enter your store in the first place? Or were a good chunk of them capable of buying a game, and you failed miserably to persuade them to buy? I think our problem is the latter... that maybe we're failing miserably at getting people to buy because we're using the wrong mechanisms to try to close sales (i.e. sending people out the door with a free demo).
[/B]

99 customer leave the store with a miniaturiased version of the store ( a game download is a tiny store by itself since it has a buy button... )

Maybe you need a better convertion rate for your miniaturized stores ?

pat.

Originally posted by Kai-Peter
I am amazed by the amount of people that buy the game without downloading the demo.

I would have bought your game right away, but the demo did not run on my computer. I think it's because I run Mac OS. ;)

Why would I have done so? First, I liked the idea so much. I loved to run my own SpaceStation (BTW you rmember SpaceMax from Bluebyte (or Starbyte)?). Second I saw some screenshots. Then my brain made a calculation: a) the idea is original b) the graphics are brilliant c) the website is in shape. Thaz means to me: this guy is not a sucker, thus is game will not suck. I love it I want to have it.

So, no thought of "where is the demo?" and the like. If the website and the screenshots build up enough trust, I don't need any demo.

Remains the fact that it still does not run on my OS...

Originally posted by formfarbeminze

So, no thought of "where is the demo?" and the like. If the website and the screenshots build up enough trust, I don't need any demo.


So much for the qualitative evidence, now we just need to look at the results from the full scale experiments .. :)

I might port to MacOS one day, I really like the Unixy feel of it.. :)

There is a big thread on the regnow marketing forum about this very topic. The general concensus there was that removing the demo lead to increased sales. However, the people engaging in the discussion were all selling applications, not games.

I think probably the biggest stumbling block in this sales experiment will be the fact that so many people are coming to the site after searching for things like "free games", "free game downloads", "free game demos", "game downloads", etc. Converting these people is going to be a much tougher sell than the average person just browsing along looking for a fun game.

Is this experiment also going to include a change in keywords, and if so how much time are you going to factor in for that?

Converting people looking for "free games" was never going to be easy (or maybe even possible). We should consider the fact that they were actually looking for "free" and not "trial" - ie. they never had any intention of paying for anything, hence the "free" part in their search.

This is a function of the marketing aspect. You need to find people who want to pay for games to come to the site, not people looking for freebies. If only people googled for "expensive downloadable games"...

Cas :)

Originally posted by Dexterity
The key here is that in order for this to work, each product has to have a strong unique selling proposition (USP). Dweep's has always been, "A defenseless furry creature trapped in a hostile world." There's something about the dichotomy of a cute, fluffy, squeaky creature surrounded by things that could blow him to bits (bombs, lasers, etc) that makes the game sound fun to a lot of players. I think a detailed web page can indeed make this kind of game sound fun and unique, enough to get people to buy it sight unseen. Have you ever talked a friend pr coworker into buying a game sight unseen? How'd you do it? You made it sound fun, right? Same concept here.

Steve, I think that you are a little bit wrong with your thinking. Your idea in this message: I have good USP so I will make nice selling message of site and people will buy it. The problem is here: you can overpromise and underdeliver, while generally all sales people recommend: underpromise and overdeliver. I would say: good sales proposition does not guaranty quality of game. Although it could guaranty “theoretical fun” which is not really implemented in program. But I have talked about general product here. If we would talk about Dweep: fun delivered by this game is guarantied by 4 days of existence of the game and it’s user opinions.

I also think that one of the possible problem of the 99 people going away is that they already saw your games on other site, so there's nothing new really.

Turning to original internally developped content and getting rid progressively of the published games will bring in a lot more income ( no 70% loss anymore ( or x% royalties) , at your point it's probably cheaper to pay the development upfront than publish )

pat.

Originally posted by RedClaw
I think probably the biggest stumbling block in this sales experiment will be the fact that so many people are coming to the site after searching for things like "free games", "free game downloads", "free game demos", "game downloads", etc. Converting these people is going to be a much tougher sell than the average person just browsing along looking for a fun game.

Is this experiment also going to include a change in keywords, and if so how much time are you going to factor in for that?

My thinking is that the people coming to the site looking for "free games" and similar search terms are not the best qualified traffic. I'm envious of sites that can target such great keywords as "solitaire" or "blackjack," since then you're getting people looking for a specific type of game. Our games are not the kind that people will think to look for except in a general way, unless they already know about the game from a friend. Still, in some areas that uniqueness has been an advantage.

So yes, we will be changing the keywords we target. Initially the traffic is still going to come from old keywords for a while. But we can track the sales/traffic ratios from different keywords to see how that changes.

Even beyond this we also need to change the way we advertise. We can't keep advertising free downloads, so we have to change our ads too. As an example, we have a new Dweep ad running on the left sidebar of every page of http://www.sharewarejunkies.com -- instead of promoting the free download, it offers a $5 discount for Shareware Junkies visitors.

Originally posted by princec
Converting people looking for "free games" was never going to be easy (or maybe even possible). We should consider the fact that they were actually looking for "free" and not "trial" - ie. they never had any intention of paying for anything, hence the "free" part in their search.


One thing about keyword phrases that contain the word "free" is that they bring in loads of traffic. So while they don't necessarily convert as well as other terms, they tend to make it up in volume.

Worst case, if we stray from our old keywords, and the new choices don't work out for us, we can eventually reclaim our old positioning by changing everything back. We've done this before when doing keyword experiments. For some search engines like Google, the positional changes can happen really fast.

Originally posted by Dexterity

As an example, we have a new Dweep ad running on the left sidebar of every page of http://www.sharewarejunkies.com -- instead of promoting the free download, it offers a $5 discount for Shareware Junkies visitors.
It's interesting. I never advertised on pages of download sites. Can you share some stats about that? Are sharewarejunkies your affiliate? It means that they run it for free and then get percentage of sales?

Thinking further between the lines I'm really not happy the way the search engines have got the whole deal sewn up on the internet. There is so much emphasis on search engine ranking; and once again if you take it to its logical conclusion: you're no longer in control of your own business, they are (something that was brought home to me when google actually kicked my ads).

If you're relying on search engines for people to find your site ultimately you will fail, or end up in an increasingly abstract battle with algorithms.

You need to get the right people to your site by other means; and then execute plan B - sell, sell, sell. The more I think about it the more I really do think that the mainstream media - not the internet - is probably the first way to do this that pops into my head. But it's so prohibitively expensive. There needs to be a new guerilla way.

Cas :)

The problem with guerilla marketing is that, if it is successful, it doesn't remain "guerilla" very long. It becomes a busy channel, and people start charging money to improve your access to that channel.

Originally posted by Coyote
The problem with guerilla marketing is that, if it is successful, it doesn't remain "guerilla" very long. It becomes a busy channel, and people start charging money to improve your access to that channel.
The same became with download sites I guess? Before they were free, now they started to charge money for listings (download.com, tucows.com)
BTW, can you explain in two words what is it guerilla marketing? I red a lot about it in forums. But I still did not understand the idea.

Think guerilla warfare. When you have an inferior fighting force against a superior one, guerillas can still succeed in dishing out a great deal of damage by using unconventional methods.

Guerilla Marketing is pretty much the same thing - avoiding the expensive routes dominated by the guys with the big budgets, and instead trying to get the message out through unconventional means. And like I said - once one of these unconventional proves effective, it quickly becomes expensive. I suppose at one point television advertising was fairly cheap & unconventional --- heck, a single company would sponsor several TV shows all by itself.

Originally posted by RomanV
It's interesting. I never advertised on pages of download sites. Can you share some stats about that? Are sharewarejunkies your affiliate? It means that they run it for free and then get percentage of sales?

No stats to share on this ad -- this campaign only started yesterday. They aren't an affiliate; we pay for the ad out of our marketing budget. If it converts well (i.e. generates a profit), we'll continue it; otherwise, we'll dump it. We have a pretty decent backend business though, so an ad can actually lose money on the front end, and we can often make a backend profit by selling other games down the road to those new customers. For example, if we pay $1000 for advertising that generates only $500 in sales, that's not necessarily a losing proposition for us. We'll likely recoup the whole $1000 within a few months from additional sales to those new customers, not to mention additional referral sales. Without a decent backend though, it's a lot harder to make a profit from advertising. If all you've got going for you is a one-time sale, most advertising will probably just lose money.

I think it would be best to try to get visitors searching for something like "buy games". There probably aren't that many, but I'ld rather have a few of those come to my site than a whole lot of people searching for "free games".

What would people look for in a search engine if they do have money to spend, but don't really know what to buy? (Just like going shopping in real life without planning to buy anything specific).

Originally posted by Mike Wiering
What would people look for in a search engine if they do have money to spend, but don't really know what to buy? Gift ideas.

Originally posted by Dexterity
... advertising that generates only $500 in sales ....

I've always been curious about this because I can't seem to find a good method (other than ask the customers) to find out what proportion of people coming from site X have bought or put money into a game ?

Is there another way than ask the customer ?

Nick

When I go looking for a new game to buy, I usually think in terms of what genre I might like to play and whether I'd want a PC or console game. Then I'll go check review sites to see what's gotten good reviews in that genre. I rarely download a demo first, even if one is available, and I put a lot of weight on recommendations from other people. A couple weeks ago I looked for a 3D multiplayer FPS for XBox, and I ended up picking Return to Wolfenstein. I've found that the multiplayer XBox live experience is pretty fun. I play with the handle WWWoDWEEPoCOM (I can't resist the extra marketing opportunity), so if you ever see me online, be sure to drop me some extra ammo and health packs. :)

While I know that many games are bought as impulse purchases, I think targeting impulse buyers directly is pretty tough, like trying to grasp a slippery fish. That's essentially what I've been trying to do for the past few years, and it has certainly been effective to some degree. But I figure that if I put a stronger effort into convincing the ideal target customers to buy (such as people who look for logic puzzle games), then that shouldn't hurt the impulse crowd. I think impulse buyers of our games must have at least some overlap with our ideal target audience, so if we address the ideal audience's wants and desires, there should be enough there to persuade impulse buyers too. They key is that all the product benefits have to be explained in plain simple language, so we don't lose the casual players by excessively targeting the hardcore puzzle gamers.

Best of luck to Steve and anyone else who's going to give this a go. It's an interesting idea and I'd love to see how it pans out. Personally, my way of thinking, at least as far as it applies to my own work, is that the demo itself, with its (hopefully) compelling gameplay, juicy teaser of the full version's features, and oh-so-tempting "buy it now!" button would be the primary instrument for closing a sale so I don't necessarily think people leaving the site with just a demo is a bad thing providing the demo can work its magic on them. But there's only one way to find out...

I have doubts whether this method can work.

Even AAA games put demos out and they have much better known reputation than shareware developers. For example: Half-life 2 comes out, reaction follows -> "I buy it, don't have to even see screenshots!!".

Dweep 3D comes out -> "Dweep 3D? never heard it... Whadda heck is this? No demo?"

I see CHANCES that this can work when you have build a solid reputation being number 1 shareware producer and having lots of "hype/advertising/reviews/you-name-it" on your product. You have to be FAMOUS.

Just my opinions - everyone is free to act they like. But for me, this sounds like another "adware bandwagon" ;)

Originally posted by Dexterity
[B]No stats to share on this ad -- this campaign only started yesterday. They aren't an affiliate; we pay for the ad out of our marketing budget. For example, if we pay $1000 for advertising that generates only $500 in sales, that's not necessarily a losing proposition for us. We'll likely recoup the whole $1000 within a few months from additional sales to those new customersB]
Steve, sorry, but I would ask one more question then: what kind of advertisement was it? (fixed pay per month, pay per click, pay per impressions?)

When a developer turns a publisher they spend most of their waking time thinking "How do I make more money than what I am currently making". All big publishers that we loathe today were once pioneering developers that went down the slippery road precisely because of this question. Is Dexterity going down the same route?

Not having a demo for the customer to try on my website simply violates my ethic of being an independent artist.

The whole thing sounds a little like Retail to me.
Steve, have you gone to the dark side?

HHhhhHH... Steve, I am your father.... HHhhhHHH <vader says as he pulls off his mask, only to reveal he is really Andre' LaMothe>

-Bruno

Personally, my sympathies lie with the indie developer - not limited to the shareware publisher, or the online distributor. So if this or any other method works for Steve or anyone else, then it's all good by me.

Originally posted by jaggu
When a developer turns a publisher they spend most of their waking time thinking "How do I make more money than what I am currently making".

No, that's what any business owner asks themselves. Has nothing to do with being evil. :) Publisher or developer, if you want to stay in business and/or expand your business, then that's what you ask. There's nothing wrong with figuring out ways to make more money. It doesn't have to affect quality. Think of it this way -- if you worked really hard on a game for a year, would you rather be making $100 a month from it or $3000? If you don't ask yourself how you can make more money from the game, you will never get to a point where it's selling $3000 a month.

And there are plenty of ways you can increase sales. Steve P. is just trying to figure out what's the easiest way to increase them. There're a hundred things you can do but some are exponentially more cost effective than others. If you don't like the business side, once you're successful enough just hire a CEO or a Marketing Director and you can be free to work on games. :)

Originally posted by RomanV
Steve, sorry, but I would ask one more question then: what kind of advertisement was it? (fixed pay per month, pay per click, pay per impressions?)

Fixed pay per month.

Originally posted by jaggu
When a developer turns a publisher they spend most of their waking time thinking "How do I make more money than what I am currently making". All big publishers that we loathe today were once pioneering developers that went down the slippery road precisely because of this question. Is Dexterity going down the same route?

Not having a demo for the customer to try on my website simply violates my ethic of being an independent artist.

The question I ask more often is, "How can we increase our service to our customers?" And I think one way to do that is to pull back on servicing those people who will likely never become customers, thus being able to devote more free time to our customers.

A few months back, when Steve announced that he was going to concentrate on marketing his current lineup and therefore release fewer *new* games, I worried that this would negatively impact return traffic to dexterity.com (over time). Common sense would seem to dictate that if a customer (or potential customer) isn't finding new content, there's little incentive for them to keep coming back to the site.

Could this potential new direction also have a negative impact on return visits? Consider the following scenario:

Mary Q. Public downloads a bunch of FREE games from download.com. She likes Dweep Gold SOOO much she actually decides to register it, and in the process finds the dexterity.com website. Seeing all the great games listed there she bookmarks the site. A week or two later she returns to the Dexterity website and finds three or four games that she wants to "try out" (remember, this person got the game from download.com -- she's accustomed to the "try before you buy concept"). Unfortunately, she can't figure out how to download the game. She looks everywhere for a link, figuring that she must be missing the "free download" area. After a few minutes Mary gives up in frustration, cursing the poor website design. She comes back to the site a month later but nothing has changed so she deletes the bookmark and never returns.

That's a pretty extreme example, of course, but it certainly seems possible, doesn't it? Am I missing something? (Just playing devil's advocate here...)

I say, enough speculation already. Time to let the experiment begin and speak for itself! :cool:

<edit>

ahem..... i hope it doesn't expire either!:)

Originally posted by Dan MacDonald
I say, enough speculation already. Time to let the expirement begin and speak for itself! :cool:

Expirement? I hope Dexterity doesn't expire... ;)

Jack

Steve,

It looks like you completely activated your new site now.
But why do you write everywhere: "PC Game Downloads"? In this context
you should write PC Game purchases.
"Top 10 PC Game Downloads" - "Top 10 PC Game Purchases"

Well I guess they are downloads. You just have to buy them first :)

Originally posted by simonh
Well I guess they are downloads. You just have to buy them first :)
I buy a game or a download?

The experiment has already started, but it's too soon to report any results. I managed to figure out a simple way to conduct this experiment as an A-B split test, which pleases me greatly, since the results will be much more accurate. I had to make some basic changes to the site first, but now there's a 50-50 test going on, linked to our cookie-based visitor tracking and sales database.

Every unique visitor to our site gets assigned a unique visitor number, stored as a cookie. If you have cookies disabled, you won't get assigned a number, but this appears to be less than 1% of our visitors, and we can still track sales from these visitors as a group simply by the absence of the cookie. This tracking is nothing new for us; hundreds of thousands of these visitor cookies have already been assigned.

So to conduct an A-B split test, all I do is add a bit of PHP code to certain pages to alter the page content based on the visitor number. The visitor number doesn't change unless someone clears their cookies, so if the person returns to the site on the same PC at a later time, they'll see the same version of the page they saw before. I've been doing A-B split tests for a long time now (I think I first mentioned applying them to web sites at the 2000 Shareware Industry Conference), testing things like different page layouts, promotions, headlines, etc. By checking the database, I can see which option in each test leads to the best sales.

Right now the test affects all the product pages on our site. For instance, if you go to www.dexterity.com/dweep and look at the buttons on the top and bottom of the page, half of you will see two buttons (download free demo and order now), and the other half will only see the order now button. Which version of the page you see is determined by whether your visitor cookie is even or odd.

So over time we'll hopefully see one group of visitors outperform the other, since we already track visitor numbers in our sales database. By doing this kind of A-B split, we don't have to worry so much if we get a surge or dip in overall sales caused by some other factor, assuming that it would affect both test groups equally.

By tracking visitor cookies, I can also segment our visitors somewhat. For instance, I can determine if our previous customers do better with one version of the site, while first-time visitors do better with a different version.

There's still the problem of dealing with people who've already downloaded demos before the experiment began, but those people should be split evenly between even/odd visitor numbers, so we should still be able to determine which overall approach works best, although the magnitude of the change may take a while to converge.

Interesting stuff, Steve. A/B splits for testing out page effectiveness is a great idea. I'm looking forward to hearing your results. Have you been following any of the thread on ASP regarding using a Taguchi matrix to track N parameters with M-way splits?

When is this very interesting experiment finished? When will you submit the final results and the conclusions of these tests?

Originally posted by elund
Interesting stuff, Steve. A/B splits for testing out page effectiveness is a great idea. I'm looking forward to hearing your results. Have you been following any of the thread on ASP regarding using a Taguchi matrix to track N parameters with M-way splits?

No, I haven't been using the ASP newsgroups much lately. Is that a similar type of experiment?

Originally posted by johnson
When is this very interesting experiment finished? When will you submit the final results and the conclusions of these tests?

Probably in about a week I'd have some preliminary results to share. The length of the experiment depends on how long it takes for the results to begin to converge.

Steve, I just wanted to say that I think your approach to things is so freakin' awesome. Most people would have either kicked the idea around and then forgot about it, or changed the whole site around and lose the ability to get truly meaningful results.

However, by splitting all of your visitors into two groups and running both sites at the same time, you get real, quantifiable numbers. That's brilliant!

Ok, it's probably not an original idea, but it really resonated right now. I'm really curious to see how this turns out, more from an implementation standpoint than what the experiment is trying to show.

Originally posted by Dexterity
No, I haven't been using the ASP newsgroups much lately. Is that a similar type of experiment? Taguchi is one of those Quality gurus. I hadn't heard of him until I read the post in the ASP newsgroups. From what I understand, he has a design methodology for examining the effects of modifying a number of independent changes, using fewer tests, by doing them in combinations. The post in ASP was asking if anyone had used the Taguchi method for designing advertising experiments.

You have one parameter you are testing, which has two levels: try before buy / buy only. Let's say you wanted to test out this plus six other unrelated ideas on the home page (red/blue buy button, More Info link before Buy link or after, right sidebar or left sidebar, etc). If you tried just changing to a red buy button, and it resulted in more sales, you'd keep it. Then if you added moving the sidebar to the right side, and sales went down, how sure can you be that the combination with the red button didn't effect your outcome? You'd have to test all 7 parameters in combination will all others (7!) to be sure. With Taguchi you'd do it with say 8 tests (or 8-way split).

There's a lot in there about noise reduction and ensuring good parameter selection. I'm not big into statistics but this looks really neat. :) I do some reading on heuristics and problem solving, so I'd think Taguchi should help removing local maxima problems. Here's one link that gives some info on it:

http://www.orszulik.free-online.co.uk/

I only brought this up because I think this is exactly the kind of gritty thing you'd be the first to sink your teeth into. :)

Sounds interesting. Normally I only test one variable at a time, and most experiments usually last only a few days or weeks, so I don't quite need stronger methods just yet, but it's nice to know there's a way to test more variables without having to subdivide the visitors into large numbers of groups.

Incidentally, it was Jay Abraham's material that first convinced me of the value of testing. I highly recommend this article on his site about the 10 biggest marketing mistakes:
http://www.abraham.com/articles/Ten_Marketing_Mistakes.html

Mistake #1 in that article is "not testing." Mistake #4 is "not having a back-end," which is something that's become a huge part of our business over the past year.

The web is a terrific medium for testing because you can measure the results in real-time. And you can run the test for as long as it takes to understand those results and terminate the test whenever you feel it's done. It's not a big commitment, and it takes very little time to conduct a test.

There's just so much uncertainty with online marketing. Some people say that 30-day software trials are the best; others swear by 10-day or 14-day or even 60-minute trials. But which is really the best for your particular products? You could just guess. But if you guess wrong, maybe you're only getting half the sales you would otherwise. That's a big gamble. And what if you guess wrong in other areas too? Is it possible that there's a minor 10-minute change you could make to your web site right now that would increase your sales by 25% permanently? The only sure-fire way to know is to test. It's not as cool and exciting as guessing, but it's a lot more accurate, and at least then you'll be able to move forward knowing what works and what doesn't instead of trying to go by the feel of your sales. Eventually I want to integrate a lot more testing capabilities into our products, so we can see exactly what factors matter and by how much. I think it's the only way to be sure we make improvements where they matter most and avoid wasting time on areas that are already close to optimal.

I think for this test to be done correctly you need to run the site changes for several months to completely offset the risidual effects of demos already downloaded. This seems to me like a test that could do well initially but poorly in the long run.

Hi Steve,

Well you can always half your risk by creating a php page that redirects every second customer to a buy only page and track the impact with cookies. So if someone comes to your site and they are in the group of people that can download a demo you would know it from the cookie if they buy the game, and if they can only buy then the cookie could tell you later if they return to buy. Of course not everybody will surf and buy from the same machine, but I would gues that most people do.

If the search sites could redirect a customer to a specific page depending on the key words, you would have a much better way of measuring the results without any risk. So one set of key words would take you to a page on your site offering no demo download, only purchase options and another set of keywords would redirect a customer to a demo, try before you buy page.

Maybe you could do this by adding another domain that is very close to your current domain like www.dexterity.net and then use that domain name for the experiment.

My to-cents about your experiment and risk reduction.

Cornel

Originally posted by svero
I think for this test to be done correctly you need to run the site changes for several months to completely offset the risidual effects of demos already downloaded. This seems to me like a test that could do well initially but poorly in the long run.

True to some extent. I've been finding that most people who download a demo make their purchase decision within the first three days though.

I can still dig deeper into the data to look at comparisons within specific subgroups, such as first-time visitors to the site. We record things like pages visited, demos downloaded, etc. So there's a lot of data to look at to help break the tie. For instance, I can determine of all the people who visited our Dweep product page within a certain period of time, what % of them bought the full version.

Originally posted by Dexterity
True to some extent. I've been finding that most people who download a demo make their purchase decision within the first three days though.

I can still dig deeper into the data to look at comparisons within specific subgroups, such as first-time visitors to the site. We record things like pages visited, demos downloaded, etc. So there's a lot of data to look at to help break the tie. For instance, I can determine of all the people who visited our Dweep product page within a certain period of time, what % of them bought the full version.

My fear is that a bunch of people who already have demos come back to the site and then have an easier or more direct time buying due to the new lacks of download clutter thus increasing sales in the short term. If most people really do decide in 3 days then maybe it's not so important. Another factor I think is important is how many downloads/trials come from your site vs download sites. If very little of your download trials come directly off the site then it may indeed work better to not host downloads on the site itself. On the other hand there's no rule that says someone clicking from the demo need see the same page as someone just visiting from google or whatever, so that benefit could be achieved anyway by pointing in demo links to pages without extra download buttons etc...

Originally posted by Dexterity
What would happen if we changed our web site such that no free demos of any games were offered? I.e. what if the site were designed with the singular goal of convincing visitors to buy right away?

Did this experiment change? because the first thing that happens when I click a "more info" button on the homepage is my being greeted with a "Free Download" button...

Is the current state of the page a half step towards realizing the above quote? or did the experiment change a little in impementation?

Originally posted by Dexterity
The experiment has already started, but it's too soon to report any results. I managed to figure out a simple way to conduct this experiment as an A-B split test, which pleases me greatly, since the results will be much more accurate. I had to make some basic changes to the site first, but now there's a 50-50 test going on, linked to our cookie-based visitor tracking and sales database.

...

Right now the test affects all the product pages on our site. For instance, if you go to www.dexterity.com/dweep and look at the buttons on the top and bottom of the page, half of you will see two buttons (download free demo and order now), and the other half will only see the order now button. Which version of the page you see is determined by whether your visitor cookie is even or odd.


Read Dexteritys earlier post for a complete answer.

Ahh thank you. My bad, skim reading will be the death of me. :)

Looks like it will take a while for the results to converge. The experiment has been going for almost a week, and the even/odd distribution is still too close to identify a winner. I'll just keep the experiment going until a winner can be determined.

This sounds great. Can't wait to hear the results!

Rofl. Believe it or not, for some reason I hadn't found the time to go to the Dweep page until now (wanted to see if I got the demo and order buttons or just the order button). I got just the order button, and since I was there I read the description. Immediately I was thinking, "I've gotta try this game!" Then I remembered I couldn't download, so I'll just wait (since I'm outta cash). I'd delete cookies but that'd be unfair to the experiment ;)

Well, you can always cheat and download it from download.com or any other shareware site too. :)

I also wonder if certain games will sell better with demos while others whill sell better without. Dweep is the kind of game I think will still sell well w/o a demo, but block-puzzle games like Penguin Puzzle may be harder to sell this way, since it can be harder to make them look interesting with just text and images. But in order to get enough data to make conclusions about individual games, we'd have to run this experiment for many weeks.

I think any game with impressive screenshots and original ideas will sell well without a demo. Though, there's games that don't look really "fun" until seen in action. Those games, in my opinion, are the ones that need demos. Dweep is definitely on my must-have list :P

Oh, and I think playing the demo for a game like Dweep would probably take a lot of the fun out of the game when I buy it, no matter that my first inclination was to try the demo :D

Originally posted by programmer_ted
I think any game with impressive screenshots and original ideas will sell well without a demo. Though, there's games that don't look really "fun" until seen in action.

What about the word of mouth? Maybe won't loose sales, but surely loose popularity, which might turn to a drastic loss in sales in the future. This is the point where I get stuck. Any ideas?

ergas

Woohoo! 100th post here. Sorry for the OT :D

Any news, Steve?

Originally posted by programmer_ted
Woohoo! 100th post here. Sorry for the OT :D

Any news, Steve?

Nothing new in the last 48 hours... still too close to call a winner. That's sort of interesting in itself though.

Originally posted by Dexterity
Nothing new in the last 48 hours... still too close to call a winner. That's sort of interesting in itself though.
Too close to call...I take that to mean your sales are about the same. If so, how has your bandwidth usage changed?

For those people who download directly from our site, bandwidth usage has dropped by half as expected.

Hi Steve,

Did you ever come to a convergence on your data? What were the results of your testing?

Anthony

Yes, there was a later thread about that IIRC.

Originally posted by Dexterity
Yes, there was a later thread about that IIRC.
What do you mean? What is IIRC? Could you give direct URL on this thread?

I think IIRC = If I Remember Correct

I guess this is the thread:
http://www.dexterity.com/forums/showthread.php?s=&threadid=1579

Originally posted by freeman
I think IIRC = If I Remember Correct

Ok I did not know that. I am not a native speaker.

I guess this is the thread:
http://www.dexterity.com/forums/showthread.php?s=&threadid=1579
But Steve gave results on 18 October in this thread. It would be interesting to see up to date results.
But as far as I understood from mentioned thread, Steve started some new "hidden" experiment.