The Allure of the Thin Client

For a couple of decades various companies have been trying to get us back to the thin client model of computing.  Oracle has suggested this loudly more than a few times and Google ponders it now and again, with things like Stadia being based on the idea.  Also, if you work for a big company I assure you that your IT department has wet dreams about taking away all your laptops and desktops and making you work on some sort of thin client appliance.  IT at my company keeps pushing Citrix virtual desktops as the solution to every problem.

I say “back to” because I am old enough to remember when dumb terminals and terminal emulators were a mainstay of computing.  In addition to my time spent in the computer lab in college, the online games I played back in the 80s and into the mid-90s, things like Stellar Emperor or Gemstone on GEnie, MegaWars III on CompuServe, and Sonjourn/Toril MUD, were all built on that model.

Star Trek in vt52 emulation

As personal computers came along and started growing in computing power, much of the heavy lifting was put on that end of the equation.  Air Warrior rendered its very primitive visuals on the player end, and shooters and action games like Marathon and Diablo made the user’s system do the graphical work while just data about inputs and positioning were shared.   This meant that in the low bandwidth of the time… I played Air Warrior on a 2400bps modem… the back and forth between client and server was kept to a minimum.

So the end user client became fat.  Eventually so much data was stored at the user end by the late 90s that EverQuest had a little test module app that let you run around a mini-zone to test your 3D card, but you could rename many of the game maps and run around the main, if empty, world if you knew what you were doing.  You couldn’t zone or do much, but if you wanted to explore it was a boon, and you were not even connected to Sony while you did it.  And that is the way that many MMORPGs and other online games went, keeping data on the server and letting the end user machine do the graphical work.

For video game developers there are many benefits to going with a thin client, of keeping all that data on their servers.

For starters, the downloads and patching at the user end are kept to a minimum.  This has often been viewed as a point of friction that keeps players from trying out new games.  The holy grail is for a player to just be able to play without any sort of download, something CCP has been experimenting with recently with EVE Online and their EVE Anywhere browser beta.  If you can just play the game on any computer, then your potential market is greatly expanded.

It also makes updates easy, since things only have to get pushed as far as the servers.  Very little need be pushed to the player’s machine.  New content just appears or is unlocked without a download.  You also get all your settings and configurations as you move from device to device.

It is also a major boon for security.  If all the key files are on the company’s secure servers, then secrets can be kept.  We are familiar with every new pet, item, mount, or NPC being spoiled for us in WoW by the race to datamine any pre-patch update.  And, of course, addons, illicit or benign, and hacks are kept at bay.  This all falls under Raph Koster’s admonishment that “the client is in the hands of the enemy.”  Overall the environment is more secure.

Finally, all the end user issues that come from the wide variety of PC configurations, a huge problem for many applications, are largely eliminated.  A thin client stops caring about processors and video cards and operating systems and the like.  Your game can theoretically run on somebody’s TV or refrigerator.

All in all there is a lot of upside.  Control! Security! Ease of access!

Sign me up today!

So why isn’t every new online game in a thin client in the cloud?

Since I used the word “cloud” there, I am going to take a moment to point out that cloud computing is not the same thing, or required for, a thin client, though when people who should know keep conflating the two things I get how you might be confused on that.  Thin clients are as old as computing.  See my reference to dumb terminals at the top of this post.  And cloud computing is, simply put, a scalable server architecture with redundancy built in… though, again, some things that get referred to as “the cloud” are better labeled “somebody else’s computer” and not used as examples of the technology.  Talking about cloud computing as though that means any remote computer is a simplification that renders the term meaningless.  If that is your frame, then every online game is “cloud” and there is nothing special about it.

Anyway, why is Google’s Stadia something of an outlier rather than the norm for the industry, at least when it comes to 3D rendered world-based games?  (Because you kind of have to count early RuneScape, Club Penguin, Star Wars: The Clone Wars Adventures, Nation Geographic Adventures, and all those Cartoon Network games if you don’t put a barrier somewhere.  So I am speaking of high end games where some level of realistic graphic fidelity is a requirement.)

And maybe Stadia is a bad example in that it is attempting to be a virtual console that can play titles that were not otherwise designed for such an environment, but if you take it off the list we don’t have a lot of other big name examples.  Well, at least no successes.  A few companies have tried to do what Stadia is doing in the past and have ended up failing.  But given that it is common as shit unless you want render a ton of polygons, why isn’t already a common thing?

Part of the issue is likely due to the cost of the infrastructure.

The problem is that if you’re going to take over all the rendering functions of the remote device, you essentially have to do all the processing that the end user’s PC or console was going to do.  If you want to run that all yourself or you want to use somebody else’s data center, that still means a lot of extra hardware.  The company basically has to pay to run your client rather than letting you run it on your own hardware.

For example, EVE Online has a minimum system requirement of a 2.0 GHz dual core processor and a modest GPU.  If it went entirely thin client, if EVE Anywhere was the only way to access New Eden, you would have to have the equivalent of 20,000 minimum spec PCs in processing power on hand just about all the time, scaling up to 35,000 or even 40,000 at prime time on weekends.

You can probably get away with less processing power for most operations, but you would most assuredly want to put more processing power behind GPU support unless you want the whole game to run in potato mode for everybody all the time.

In a modern cloud architecture where you can bring capacity online easily and only pay for what you are using at a given time, you can keep the costs down somewhat, but everybody playing is incurring a cost, and somebody has to pay for it.  I don’t think it will be like my days back in college where your online account had an allocated budget to spend on processing time (which inevitably got squandered on Rogue), but the company is going to have to find some way to pay for using their processing power rather that your own.  Expect to pay more.

And, while the company saves on bandwidth when it comes to things like pushing patches to every client, the need to pipe high quality video at an appropriate resolution and quality will more than offset that.

Meanwhile, latency and connection quality issues will become a much more visible, something that Google’s Stadia demonstrated.  These are issues in current games like WoW, but you often don’t see it because the client with all the assets and world data will keep you walking, running, riding, or flying along while it tries to catch up after any data blip.  But if you lose connectivity for a bit and far end is only routing video to you, everything stutters or stops quite noticeably.  And even when the is able to get to you but the network traffic is slow, you’ll see your video quality degrade.

Also, if you live some place with restrictive bandwidth caps you’ll find streaming all that video might put you in danger of exceeding them.  You need high speed and lots of bandwidth to play a thin client game at the quality level you’re used to with an equivalent fat client title.  But you can play the thin client title on your refrigerator, so there is that.

But, if the game decides to take full advantage of the potential platform independence aspect of a thin client, if they’re going to support your high end desktop PC with the 34″ ultra wide screen monitor AND that refrigerator screen, there is likely going to have to be some sort of compromise on quality and UI.  So even if bandwidth and network hiccups aren’t dragging down your quality, the game itself, optimized to some happy medium, might not deliver the same satisfying, high definition experience that you would get if your own gaming rig was doing the work rather than some standardized system on a remote server.  Oh, and I keep using the term “thin client,” but for most uses you can substitute in “web browser,” though a light app is also possible. (Though with that comes the temptation to fatten it up.)

Finally, if the thin client game shuts down… see MetaPlace… you have nothing left but memories and credit card bills.  All of the major pirate /private server projects to restore online games that have been closed rely heavily on people being able to get a hold of a copy of the fat client.  All the graphics and a lot of the data is stored there, which is how so many of these rudimentary projects get stood up so quickly.  The world is in the client, you just need to get a system with the right responses going to get basic walking around the world functionality running.

The thin client idea is an attractive proposition for the dev side.  It simplifies a lot of things for them, gives them better security, and hands them all the control.  Done right in a cloud environment, it could even solve the first day server load issues if they can scale successfully.

But somebody is going to pay for the additional cost, your experience may be degraded if you do not have an ideal internet connection, if the studio wants to run their title across platforms and devices you may find the experience and interface less than you desire, and the whole thing becomes a virtual world that can disappear, never to be seen again, as fast as any virtual good.

SuperData Charts Gaming Revenue Highs as We All Stay Home

SuperData Research has their monthly chart out for March 2020.  The results and accompanying data are not all that surprising.  With people all over the world stuck at home digital purchases peaked according to the company.

• Spending on digital games reached $10.0B in March, the highest monthly total ever. Individuals are turning to games as a reliable entertainment option during the COVID-19 crisis and are using online multiplayer to keep in touch with others. Total digital revenue was up 11% year-over-year from March 2019 ($9.0B).

The charts reflect what was most popular last month.

SuperData Research Top 10 – March 2020

On the PC end of the chart the usual top four held on to the summit for yet another month, though League of Legends fell back to second place behind Dungeon Fighter Online.

The first new entry on the list is Doom Eternal, which has gotten a lot of buzz on both PC and console.  CS:GO, which comes and goes from the bottom of the list saw a nice jump as did Borderlands 3, which made the list when it launched, subsequently falling off as many buy to play titles do.

Half-Life: Alyx came in at number eight, which might seem low for something in the Half-Life series, but as a VR title making the cut is impressive.  VR remains a niche element in the market.

And at the bottom of the list are World of Warcraft and World of Tanks.  Both titles have seen more players.  We’ll see if things like WoT‘s 10 year anniversary celebration and WoW‘s throwing a 100% xp bonus at players will boost their standings for April.

On the console column, surprising nobody, Animal Crossing: New Horizons stands at the top.  I would be hard pressed to find a general news outlet that hasn’t reported on it and my Twitter feed was probably 20% mentions of the game the week it launched.  Even SuperData gives it a special mention:

• Animal Crossing: New Horizons sold more digital units in a single month (5.0M) than any console game in history. The Nintendo-published title broke the console record for monthly digital game sales previously held by Call of Duty: Black Ops IIII. Animal Crossing: New Horizons also roughly matched the first-month digital sales of Super Smash Bros. Ultimate and Pokémon Sword and Shield put together. The game’s combination of social features and a relaxing setting likely appealed to individuals stuck at home. Closures of brick and mortar stores also meant that a higher share of consumers purchased the game digitally compared to past Switch titles.

After Animal Crossing, FIFA 20 continued its run near the top of the list, followed by MLB: The Show 20, which is filling in the gap for baseball fans bereft of a season so far in 2020.  If you can’t have a real season you can run your own, something that people have been doing with a variety of baseball titles.  Out of the Park 21 on PC is popular with the hardcore fans doing that sort of thing.

And at the mobile end of the chart Honour of Kings continued on at the top. (It is one of the most popular games in China and is popular on the streaming front there as well.)  Most of the list carried over from last month, Roblox and Mafia City being the only two titles not on the February chart.  My benchmarks for the list, Candy Crush Saga and Pokemon Go were in third and fifth place respectively.

This is where I usually compare the SuperData charts to what NPD has listed for the month, as NPD includes physical retail sales.  However, NPD hasn’t posted their numbers for March to their site yet, so I will have to give that a pass for now.  I’ll put them in if they do get posted, but right now they still have February listed.  Retail might be causing them problems I suppose.

Instead I will jump to the usual close, which is the bullet points included with the SuperData chart, minus the one I injected in the post above:

• Premium console and premium PC earnings jumped as lockdowns took effect. Premium console revenue rose 64% from February to March ($883M to $1.5B) and premium PC revenue rose 56% during the same period ($363M to $567M). These game types tend to be most popular in North America and Europe, where COVID-19 prevention measures expanded dramatically in March.
• Gamers continued to play and spend on mobile titles even as they stayed home. Mobile games revenue was up 15% year-over-year and reached $5.7B during March. Earnings for a number of major mobile titles also grew during the month. For example, Pokémon GO revenue rose to $111M in March (up 18% month-over-month) after publisher Niantic made tweaks to the game to make it easier to play without physically moving.
• The addition of Warzone to Call of Duty: Modern Warfare resulted in monthly active user numbers for the title jumping 159% month-over-month to reach an all-time high of 62.7M. While most modes in Modern Warfare require an upfront purchase of the game, Warzone, a battle royale mode in the style of games like Fortnite and Apex Legends, is free-to-play.
• Doom Eternal from id Software sold 3.0M digital units in March, more than three times what Doom sold (957K) during its launch in May 2016. The latest entry in the seminal franchise benefited from strong reviews and the positive reception of its predecessor. However, as a primarily single-player game, Doom Eternal will likely have a shorter revenue tail than other multiplayer shooters that monetize through the regular sale of in-game content.
• Half-Life: Alyx performed modestly by the standards of AAA games but was a blockbuster by the standards of virtual reality (VR) exclusive titles. A total of 860K gamers played the PC VR title in March. The game had a limited addressable audience, as there was an install base of fewer than 4M PC-compatible VR headsets at the end of 2019. Direct purchases of Half-Life: Alyx generated $40.7M in revenue, and hundreds of thousands of free copies of the game were also bundled with devices like the Valve Index headset to boost interest in VR.

Taking the Eye of Terror to Lonetrek

Every so often I go on a main fleet op just to remind myself why I don’t go on main fleet ops any more.  Plus I happened to still be sitting in 1DQ1-A from the Reavers race (still in my Gnosis) and Pittsburgh2989, who is in Liberty Squad, was the FC.  So when the ping went out festooned with no fewer than three animated parrots I joined up.

I thought if Pitts was here we might not be having a Liberty Squad op.  I was wrong, but I only figured that out when it was too late.

The call was for Baltecs, a fleet composition built around the Megathron battleship.  I actually had a few options for this, including a Megathron left over from last year’s war, sitting in my hangar.  I decided to go with Logi though, hopping into an Oneiros and putting on the Emergency Response SKIN.

Oneiros in the hangar

I could have gone with a Guardian… I have a few of those sitting around as well… but being free of the cap chain for an evening had its appeal.  So I got in the logi channel while the fleet came together.

There are any number of reasons why I avoid main fleet ops.  They tend to be ponderous operations, full of strangers who will talk on and on in a fleet of 150 people about not being able to find a something in the hangar.  There are a variety of FCs, each with their own quirks and pet peeves.  And they fly battleships, which are slow to get places.  This is why you join SIGs and squads, to fly with a regular group… usually in cruisers.

While things were getting sorted I undocked and sat on tether, wondering where we were headed.

Hanging on the Keepstar in 1DQ1-A

Of course, the battleship slow travel problem isn’t always problem.  Not when you have the reconstituted Eye of Terror jump bridge network running from Period Basis all the way up into Cloud Ring.  When the fleet got going Pitts essentially gave us three way points, each corresponding to an Ansiblex jump gate.

We were headed up to Cloud Ring and the trip was essentially “take a normal gate, take an Ansiblex, take a normal gate, take an Ansiblex, take a normal gate, take an Ansiblex, take a normal gate” and we were way up north, passing a series of Keepstars along the way.

Also, who puts a Keepstar AT THE SUN?

In Cloud Ring we took a gate into low sec and sat on a titan for a while.  The word filtered down to us that we were there to jump on anybody who formed up against an op already in progress.  We lingered there for a while… for longer than it took us to make the trip there for sure… until it was decided that nobody was going to show up.  So a cyno was lit and the bridge went up.

Time to go

We landed in Nannaras, which is way up in Lonetrek, a location that seems like a long way from home.

Obligatory fleet landing screen shot

Looking at DOTLAN navigation, that is about 40 gates away from where we started.

Up the western edge of New Eden

But with the Ansiblex network it felt like we hadn’t gone that far at all.

As for what was in Nannaras, there was a Pandemic Horde Fortizar there in its final timer.

A kill mail to be had

We warped up within range of it and joined in the shoot.

Somebody was there to gun it, so bombs were launched our way repeatedly.  This led to a lot of people hitting the “need shields” button in the fleet window, an annoyance to those of us in the logi wing because Baltecs are an armor doctrine fleet.  There was nobody around to rep shields.

But even announcing that on comms didn’t stop people from broadcasting for shields.

I actually had some shield repair drones.  Since there was little else to do for the moment, I put those out and repped some of the pricier ships.

Shield rep drones on a Bhaalgorn

However, I missed a bomb launch… there was something nobody was announcing on comms… and it wiped out my drones in one go.  So that was the end of that.  A few people were unlucky enough to drop into armor due to bombs, but otherwise it was a simple shoot.  I put out a combat drone towards the end… among other benefits, the Oneiros has a big drone bay… to get in some hits on the Fortizar just to get on the kill mail.

The small ships in the fleet also warped in to get their hits in as well, which prompted the gunner to turn on the point defense system.  That meant I didn’t have to worry about recalling my Warrior II drone, as it died pretty quickly.  But the Fortizar followed suit shortly there after, and I was on the kill mail.  Priorities.

Another dramatic explosion scene

From there we had to actually slow boat back to the Ansiblex gate network.  But we were only seven jumps or so away.  And once we arrived we were let off the leash to free burn home.  As somebody pointed out in comms, the first thing that happens is that the logi, which are all cruisers, leave the slower warping battleships behind leaving the potential for comedy/disaster.

But nobody bothered us and we were all quickly back in Delve and docked up.

Not a bad op, for main fleet.  Comms were subdued.  There were relatively few Game of Thrones spoilers mentioned and only one fan theory brought up. (The “Due to a time travel loop Bran IS the Night King” theory.)  I am not saying that comms should be silent, but there are people who seem to forget they have an audience of 150+ people.

Even with a good main fleet op I’ll be happy to be back in a Liberty Squad fleet when I have the chance.

Daybreak Has Always Been Owned by Eastasia

In a bit of a futile effort somebody at Daybreak Game Company is attempting to either edit history or cover up a serious mistake.

That eye is, and always has been, Jason Epstein’s

As we know, back in 2015 Sony Online Entertainment was purchased by an investment group called Columbus Nova.  We know this because that is exactly what SOE told us, both in press releases as well in messages to the community.  Both John Smedley and Jason Epstein specifically mention Columbus Nova and having SOE join their portfolio.

People dug into just who the hell Columbus Nova was, finding out that it was owned by Renova, which in turn was owned by Russian Viktor Vekselberg.

This comes into play, as reported over at Massively OP, because of sanctions being imposed on certain key Russians, Viktor included.

So the question came up as to what happens to Daybreak.  Since Daybreak is setup as an LLC, probably nothing in the short term, no matter who owns it.

Daybreak seems to have gone paranoid on the question though.  When asked for a statement by Massively, Daybreak said:

Daybreak Game Company has no affiliation with Columbus Nova. Jason Epstein, former member of Columbus Nova, is and has always been the primary owner and executive chairman of Daybreak Game Company (formerly Sony Online Entertainment) which he acquired from Sony in February 2015.

When prodded about the fact that this flew in the face of everything we knew up until now, Daybreak followed up again stating that everything that they said about Columbus Nova in the past was wrong and that Jason Epstein himself bought SOE and has owned it himself ever since.

Meanwhile, somebody at Daybreak has been trying to clean up, fixing their Wikipedia article to correct an error with a very specific notation, “Removing inaccuracies. The company has no affiliation to Columbus Nova. Jason Epstein is and has always been the primary owner and executive of Daybreak Game Company.”  This started on April 6, 2018, more than three years after the acquisition.

Turns out @DaybreakGames edited their own Wikipedia page on April 6th and April 9th to remove any reference to Columbus Nova. Daybreakpr and an anonymous editor with an SOE IP address. https://t.co/Pml3PmhpWB

— EQ2Wire (@EQ2Wire) April 24, 2018

So, somehow for the last three years nobody at Daybreak bothered to correct this mistaken belief that they had been acquired by Columbus Nova, even when opportunities arose, like when Russel Shanks left and Ji Ham of Columbus Nova took over the reigns as president of the company.  Then, suddenly, when sanctions might be an issue the company suddenly remembers that Columbus Nova was never even part of the deal, as though a company and the person they claimed purchased it would make such an error of material fact and neither notice nor correct it for more than three years?

Sounds like horse shit to me,

Anyway, I thought I ought to put a pin on the date that Daybreak started inadvertently started drawing attention to itself by coming up with an odd retroactive tale about its acquisition.

A Decade on the Road to Mordor

Ten years ago today Lord of the Rings Online officially went live.

Happy Anniversary

LOTRO was one of the first MMORPGs to go through beta and launch while I was blogging, and certainly the first one I jumped into at launch during the reign of the blog. (I was watching Vanguard during beta as well, but was dissuaded by my experiences there.)  One of my earliest posts was a response to the idea of such a game trying to tell the story of the books.

Yahoo Headline 2007, before Yahoo became malware

The path for LOTRO from beta through launch and to today has been somewhat symbolic of the post-World of Warcraft era for MMORPGs.  It was driven along initially on the wave of WoW-subscription number induced euphoria, where the sky seemed to be the limit.

Then there was launch and the rush to play and the problem of the game simultaneously being too much and not enough like WoW.  There were some fun little quirky features, like titles you could earn for actions… though the fact that those were secret (or at least not documented or traceable anywhere in the UI) meant most players never knew about them.  It is hard to sell yourself as different when you hide things I suppose, and a decade later I don’t think I have a single one of those special titles yet.

Meanwhile, for those who rushed ahead, there was bemusement as content fell of a good ten levels before the cap.  Subscriptions fell off, complaints mounted, changes and updates went in, and Turbine went forward with plans for an expansion; The Mines of Moria.  That was an amazing and ambitious expansion.  I pre-ordered that as well back in 2008, though I did not actually get into the content for a few years and it was several years after that before I was out of the far side of those caverns.

LOTRO was also a leader in the conversion to a free to play model citing a huge boost in players and revenue to accompany the change.  They were also ahead of the curve when it came to the grim reality of such conversions.  Player expectations as to what “free” really means can be harsh, that initial surge of new players never lasts, and once you start down the cash shop path, forever will it dominate your destiny and development time.

More expansions came and again Turbine was in the lead with pre-orders and special editions with cosmetic fluff thrown in to entice more money out of players.  But eventually expansions stopped and all revenue focus went into the cash shop and new items and new currencies showed up.

Then they began to fall behind the curve of the industry.  They were not too late to the game with insta-level boosts, but they failed to grasp that people want such boost to get to the latest content, the NEW stuff.  Turbine thought boosting people to the first expansion and level 50 was enough.  That was a worst of all possibilities idea for me.  Moria was nowhere near the current content and the first 50 levels that get you there are some of the best, most charming, most memorable content in the game.  Skipping The Shire and Bree and the Forsaken Inn and Weathertop and Rivendell and the beautiful scenery of Middle-earth to head down into a cave for ten levels seems like insanity.

Eventually they figured that out and now you get jumped to Rohan with your insta-level boost, but I remain dubious about the idea as a general plan.  A seasoned player advancing an alt might take full advantage of the jump, but a new player is likely to be out of their depth.

And then there was the spin-off, where LOTRO and its sister, Dungeons & Dragons Online were folded into their own company by Warner.  Spinning of MMORPG divisions has been a thing, from Daybreak Games and its Norrathian legacy to Broadsword Games which keeps Ultima Online and Dark Age of Camelot going.

It has been a strange trip these last ten years.

But all of that was in the future back in April of 2007.  Ten years ago today I was in Middle-earth playing the game, having pre-ordered it.  That was back when pre-ordering could be done by buying a pre-order box off the shelf at Fry’s.  I still had to follow up and buy a retail box, but the pre-order box got me a head start and kept my account active for a while until I got the real game key.

And then, of course, there was the choice to be made as to which founder’s bonus to take.

Founders Choices

I went with the $199 lifetime subscription over the $9.99 reduced monthly subscription price which, looking back over a decade and more of MMORPG time, was probably the most sound investment I have ever made.  I have received more value for that money than I ever expected.

At the eight year anniversary I mocked the game a bit, pointing out that even by the longest measure… Frodo is warned by Gandalf to leave the Shire through to Bilbo and Frodo departing from the Grey Havens… the events of end of the Third Age in Middle-earth being reproduced in the game only took three and a half years.

At that point Minis Tirith was on the horizon still.  Two years later… again, more time than it took Frodo to get from the Shire to Mount Doom… and Mordor is in sight.  Update 20 brings players to the battle of the Black Gate.

Gandalf blazing in bleached white

I read about that and the tenth anniversary events and such and I feel like I should log in and take a look.  But then I read about the mix of joy and frustration with the anniversary events and remember that I am, as always, stuck behind several layers of content and I pass on to something else.

Look, could you just point me towards Mordor? I’m a bit behind.

This is ever the problem with MMORPGs that evolve through expansions and updates and levels and rigid layers of content.  I think I have patched up a few times since I opted for the Blessing of the Valar level boost, but every time I log in I look in my bags and cannot figure out what half the stuff in there is (the icons haven’t gotten any clearer in ten years while my eyesight has gotten worse) and the legendary weapon that the boost handed me… which isn’t ready to use, you have to find a vendor and go through some gyrations to make it work… and I wonder if I might just be better off rolling up yet another new character and playing through the first 30-40 levels yet again.  I think I must be close to 20 characters past level 30 at this point.

But Middle-earth yet abides, waiting for my return.  Some day that ring will be tossed into Mount Doom.  Of course, then we will have to get back to the Shire.  The tale isn’t done until the Shire has been scoured.