Landmark and the Price of a Badly Defined Beta

There has been an argument over what “beta” means when it comes to software for as long as I have been part of the industry, which is pushing on 25 years now.

The baseline definition for me has always been that your software is feature complete and you feel it is ready to ship, but now you are going to take some time to get people outside the development group to look at things.  This can be surprisingly important and an eye opening experience, as when you have worked with a piece of software for months at a stretch, your brain becomes adjusted to the way it works.  You stop seeing the flaws and you become invested in the project vision.

And then you hand it to some fresh eyes who will, often almost immediately, tell if what you have been slaving over makes a lick of sense.  It can be a sobering moment when somebody, after five minutes with your product, makes a suggestion for a fundamental change that, upon reflection, seems obvious.  Plus they tend to catch all those quirks that the team has simply learned to work around to the point of developing a blind spot, those bugs that “everybody” knows about yet somehow never quite made it to the bug tracking database.

That is the idea in my book.  I have fought for that ideal now and then.  But I have been through the wringer enough times to know that fight can be futile.  So I have been through internal betas (where we learn how little the rest of the company cares) schedule betas (the schedule says we’re beta as of today so we are) political betas (we’re going beta today because if we don’t, somebody in senior management will look bad) survival betas (we’re going beta because if we don’t they’ll cancel the project and lay us all off) and the occasional investor beta (I gave your company money so install your product on my son’s laptop… and put more RAM in there as well… I don’t care, strip your lab machines if you have to).

But in all of that there is still a certain level feature availability before we hand the software over to fresh eyes, if for no other reason that a fresh perspective is a perishable commodity and you don’t want to waste it on things you should have caught yourself.  Once people have been in your beta a bit they will become fixated on things that are important to them and tend to not notice anything else.  Long betas introduce beta fatigue, as I am going to guess SOE is finding out with Landmark.

Landmark was in alpha for a stretch and then went into “closed beta” a few months back, which meant “paid beta” so far as I could tell.  I was invited in for a couple of seven day runs at the product and, as the joke goes, there wasn’t much “there” there.  I suspect that SOE is feeling interest wain as the software goes on and on with small but important changes but no real end in sight.  So while they fleeced convinced some people to pay money to get into the software early, I am going to guess that even the most hard core fan has some limit and really need more people online and active to test.

Which is why I suspect we got this sale today over at Steam.


Yes, Landmark has been marked down to Steam Summer Sale levels of discount.  That is the basic Settler Pack, but the other tiers are available too, including upgrades if you are already invested.

All packages marked down

All packages marked down

I was a tad miffed that people were getting Planetary Annihilation for three bucks less than my Kickstarter pledge back during the Steam Summer Sale.  How would I feel if I was in for a hundred for the top tier Trailblazer Pack and then, still during closed beta, they offered up the same deal for $33.99?  I wonder if any of those early adopters will pipe up?

And given the caveats, I am not sure that $33.99 is a good deal from where I sit.  The warning on Steam as part of their Early Access disclaimer:

ATTENTION: Landmark is in Closed Beta. That means we are still adding core feature sets and that updates are happening weekly. Everything in the game is currently subject to change, which includes the possibility of wipes.

Please make sure to read the Landmark Blueprint, which provides a list of planned feature updates and timing estimates.

We are using an Open Development process to create this game, which means that you are encouraged to interact directly with the development team via the Steam Community, Twitter, Reddit, Twitch and our Forums. If you are interested in helping to create a game from the ground up, Landmark offers that opportunity.

For more information on the Landmark development process, click here.

The Landmark blueprint forum thread shows a list of features and says that they will be unveiling some new things at SOE Live in a couple of weeks.  But there is a long list of features, including almost everything that might turn Landmark into a game as opposed to a wanna-be Minecraft prototype, waiting to be implemented. (But they got the Station Cash store running muy pronto!)  There is certainly no obvious “okay, it is worth my time” point on their blueprint as yet.

While I am sure that for the devs actually working on the project, these changes are coming as fast as they can manage them, from the outside the pace can feel very different.  If you’ve been playing around with Landmark for six months or more at this point there is probably a good chance that your interest has faded somewhat, or that your focus has narrowed to a few things.  There certainly haven’t been a lot of blog posts about Landmark lately, and bloggers as a group tend to be more enthusiastic about their games than the average play.  SOE has gotten a mention here and there due to handing out seven day passes, but people who were on fire early on have been pretty quiet these days.

So, while I am not ready to claim that Landmark is DOA, it could be easily inferred that SOE needs some more people actually coming in to play, to start from scratch, to get involved, and to be enthusiastic about the game.  And for just under seven bucks I am slightly tempted.  But there still doesn’t seem to be enough there yet, and the game is going to be free to play eventually anyway.  So I will probably pass.

SOE has a chance to revive interest at SOE Live, though that can be a double edged sword as well.  They got a lot of people interested in EverQuest Next at the last SOE Live but haven’t said much about it since, and SOE has something of a history of sporadically building up enthusiasm with their customer base only to go silent for long stretches.

10 thoughts on “Landmark and the Price of a Badly Defined Beta

  1. Jenks

    Not just a beta, but a beta for a f2p game. I have no idea what a dollar is supposed to get you in a free game, and no one else really does either. It makes the whole situation that much more convoluted.

    Were your “trailblazer flag, mastercraft bracer, noble’s regalia, and tech commander gear (etc)” really worth $100? I don’t think I would take anyone’s complaint about this sale seriously unless they first admitted that they made a terrible purchase decision (sorry Keen).


  2. guest

    “How would I feel if I was in for a hundred for the top tier Trailblazer Pack and then, still during closed beta, they offered up the same deal for $33.99?”

    Put it in a subscription perspective, which I know you are no stranger to: (100 – 33)/6 = 11,11

    That’s about how much per month these people have paid for the privilege of early access (… to a highly unfinished game, but that’s beside the point), so not all that far from the price of all-access, for whichever benefit that will provide in Landmark.


  3. Wilhelm Arcturus Post author

    @guest – “to a highly unfinished game, but that’s beside the point”

    I would say that the game being highly unfinished and likely to remain so for the foreseeable future would be completely the point. Shunting that aside and making a comparison against a live subscription game seems silly.

    If nothing else, this is planned to be a free to play game, so you could make the argument comparing anything except the top tier against paying nothing at all.

    And, in the end, the only thing the top tier grants you is early access to the “live” game at some point post-wipe, so it comes down to $99.99 for early claim selection and some cosmetic items, which you could get right now for $33.99.


  4. Alicia Aishai

    The idea of paying for the beta of a F2P was a non starter for me. I got a 7-day beta code from Gamespot a few weeks ago and gave it a shot, which validated my view that I was wise not to pay for this.

    I played a game which doesn’t seem to know which direction it wants to take and everything seems half done. Farming is so-so, exploration is pointless most of the time, building requires a ton of farming and is ok but clearly could be better. I am not sure what adding combat or those huge caves are going to do for the game. Just make the farming part more tedious?

    SOE seems months away to ship a final product. I will give it another shot when the game is finished … and free. I remember a time where people had free access to unfinished games and had to pay for playing the finished games…


  5. Wilhelm Arcturus Post author

    @Alicia Aishai – I am with you. I probably came across too negative. I think there is probably something interesting there, and will certainly go back and try Landmark when it is “done,” using the term as it applies to MMOs.

    Another issue, which I have mentioned elsewhere, is the self-selecting beta testers. If you buy into what they are calling beta, that does make you invested in the game and probably a better tester over the longer term. I get why SOE did that, though I still feel they might have gone there too early. But I also wonder if that will make the usual beta community, which tends to be different from the community of a live game, more or less supportive of a specific vision articulated early on and more or less resistant to change as the project develops.

    Many are the tales of beta testers disillusioned by the final product. Does that get stronger if you had to pay to get in?


  6. bhagpuss

    I bought one Trailblazer pack and one Explorer pack back in February and I thought it was excellent value. I very much got my money’s worth. Here’s why:

    Mrs Bhagpuss’s birthday is in February and because we’ve just come out the other side of Christmas, for which I put in a good deal of effort to come up with original and unprompted surprise gifts, I tend to ask her what she wants for her birthday rather than trying to guess. The same happens for my birthday, which is just not long before Christmas.

    Unfortunately (or perhaps fortunately…) both she and I are of an age now where there isn’t actually very much that either of us wants that we don’t already have and when something does come along we’d most likely just buy it anyway. So, for both of us, coming up with a valid answer to the question “what do you want for your birthday?” involves a non-trivial amount of effort and some years the answer really is “I can’t think of anything”.

    This year I asked her if she’d like the Trailblazer Pack and she said she would. Gift buying and gift receiving “problem” happily resolved. What’s more, unlike some previous gifts which were admired but never used (the mountain bike comes immediately to mind), the Trailblazer pack got used extensively. For about six weeks she played little else.

    As for my Explorer’s Pack, I played a good deal too. I satisfied my curiosity and got a series of blog posts out of it, which was exactly what I’d hoped to get. As you point out, no-one’s much writing about, or interested in, Landmark right now so while I might have saved myself some money waiting I wouldn’t have been writing about it when it was “hot”, which I very much enjoyed doing.

    I knew exactly what I was buying when I paid my $160. I was giving my wife a birthday present she wanted and which I thought, correctly, she’d very much enjoy and I was paying to share that experience and to record it for posterity. I may not play Landmark much in the future but you can bet I will read and re-read those blog posts and the comment threads they generated with great pleasure many times over the coming years.

    So, I knew exactly what I was buying and why I was buying it, I consider it money very well spent and excellent value. That’s my subjective view.

    Looked at objectively, were the Packs worth the money? HAHAHAHHAHAHA!!!! Like hell they were! I read the descriptions before I paid and I was never under any illusions I was buying anything of objective value. Total and complete waste of money that would have been. You’d have to be gullible in the extreme to believe otherwise. What, pay $100 for access to the Alpha test of a game that’s going to be Free to Play, with a bunch of self-evidently meaningless, pointless, worthless trinkets thrown in to make it look like there was *something* on the table? Give me a break!

    On the substantive issue of whether Landmark should be on Steam…no. Clearly it isn’t ready. It isn’t any kind of a “game” and it’s going to confuse or disappoint almost anyone who buys it I would imagine. Also, if they really do add combat at or around SOE Live then you can anticipate a totally non-working, chaotic mess for at least 4-6 weeks while they try to get even the basic functionality in place. All right when a whole boatload of brand new, probably entirely uninformed, Steam players arrive to “play” the new game they just bought.

    What could possibly go wrong?


  7. Jenks

    You are a good man, Bhagpuss. Enjoying what you have instead of coveting that of others. A great example for all.



  8. Gripper

    To throw my $0.02 in – I did buy the $100 deal, and have a “little” bit of a heartache that suddenly my expenditure of $100 is no worth about $40 bucks.

    But I will say a bit just to clarify why I did this – to this day I regret my decision to not get the collectors edition of WOW – the first one and missed my chance with the little diablo pet. Seems silly I know but I really miss that and have every collectors edition since then.

    So when I saw the chance to purchase this – I thought – I like EQ2 – and since this seems to be the EQ Next “jump” I wanted to make sure I was in on the ground floor.

    I have done the same thing with Wildstar, MWO, Star Citizen etc.

    The thing that surprises me is that Landmark was not what I thought it was when they announced it etc – I thought I was getting the jump for EQNext – but instead landed in a Minecraft type of game, which does not interest me much.

    So I chalk it up to throw on the pile of things I should have known better to do, but I dont really have any real emotions except to think to be honest is that SOE is hurting and the whole Landmark experiment (it has to be right? Where is EQNext?) is not doing well as a whole.

    Right now I am sticking with the WOW and thats about it – even gave up my Eve online for now – Eve takes to much “organized” time to juggle family and gaming time right now.


  9. Toldain

    It’s my sense that one of the goals of Landmark was to create a stable of artists conversant with the build tools of Landmark who would create content for sale in Everquest Next. At my last look, that seems to be successful.

    The one contest I participated in had lots of entries that blew my attempt out of the water (except for the giant toilet, which did not really impress me.)

    For my $100 bucks, I got to bring in some old friends and fool around a fair bit. Probably not the best value as far as a game goes, but like Gripper says, it’s kind of like getting a collectors pack.


Comments are closed.