Julian Lombardi: Second Life’s Technical Problems “Inevitable”
by Urizenus Sklar on 12/03/07 at 9:16 pm
Second Life not Scalable?
Image stolen from Lombardi’s blog.
Julian Lombardi, one of the six original architects of Croquet and executive director of the Croquet Consortium AND the original designer of ViOS (Visual internet Operating System) has posted a rather sobering assessment of the scalability prospects for Second Life. Lombardi observes that ViOS went though similar scalability problems back in the day. His description sounds scarily like the current situation in SL:
Our strategy back at ViOS, Inc. was to simply re-tune the system and put up more servers as the loads increased – hoping for the best. That approach would work well for Intranet applications that serviced relatively small numbers of clients. It even worked well for ViOS’ initial user base of around 15,000 unique users. Problem was that once we had several thousand simultaneous Viosians tooling about in the landscape, they began to overload our interactivity servers, resulting in performance problems and service interruptions.
The prospects for Second Life?
Second Life … has a very similar technical architecture to that of ViOS – a vintage twentieth century client-server architecture with with single points of failure, inertia, and control. It’s been interesting to watch Linden Lab’s struggle with the inevitable technical problems faced by Second Life as a result of its recent popularity, constrained architecture, and non-scaling technical approach.
Yeah, it’s been interesting all right. Damned fascinating.
Wayfinder Wishbringer
Mar 12th, 2007
A man wanted to build a factory. So he purchased land to suffice to his expected needs, without foreseeing that his factory might need to grow. He spent a lot of money building the factory, and initially it worked. But then as production demands increased, demands on his factory assets increased as well. Unfortunately, he had no room to expand, because there was no more available land, and the foundation he had built– although sufficient for a one-story building, was not going to suffice for a multiple-story building. His only choice was to either completely tear it down and rebuild (which would allow his competition the time they needed to take over his market) or he could stress his assets to the maximum to try and keep up. This of course upset his employees and his customers– who were by now receiving inferior products.
His initial product was excellent. In fact, many of his initial customers thought him a genious. However, now his customers were upset and angry and just waiting for his competitors to come along with a better-designed product that would meet their needs.
To make matters worse, when viable suggestions were made to the factory owner of ways that he could alter his production methods and fix the problem, he ignored all those suggestions, convinced that his initial plans and visions were the right ones. He got to the point he didn’t care about his customers at all, and spent all day on the phone trying to woo investors. He catered to specific customers with a lot of money, not realizing that he was driving away the vast majority of customers that paid his daily bills. He alienated the people who had helped him form the factory in the first place and unfortunately, didn’t even care that he did so.
Eventually, the inevitable happened. Something in his factory busted and like a stack of dominoes, the factory came crashing down. No one was willing to jump in and bail him out because he had lost most of the friends that were originally dedicated to him and his vision. Eventually, the factory was nothing more than a worn-out hull, barely limping along while the competition thrived all around him… and soon the last light bulb burned out and the factory was no more.
Trevor F. Smith
Mar 12th, 2007
Wow, that’s pretty harsh.
Trevor F. Smith
Mar 12th, 2007
I’m sure it’s no coincidence that the Croquet folks are announcing a platform release tomorrow.
Gorean Furry
Mar 12th, 2007
Turning a simple sentence into a 385-word drawl. Well done Wayfinder.
Kimberly Rufer-Bach
Mar 12th, 2007
It’s interesting to see this guy discussing Second Life’s technical architecture as if he actually knows how it works. I don’t recall anyone named JulianLombardi Linden. He certainly doesn’t know what plans Linden Lab has or what they’re working on right now.
On one hand, there’s this disparaging commentary from some negative-campagining SL competitor. On the other, there’s Cory Ondrejka. No contest there. Cory knows what he’s doing.
Crissa
Mar 12th, 2007
Yeah, big comparison. 15K unique vs millions.
Next he’s going to say that Eve can’t exist because it has five times the unique users online than SL and ten times the number online as they had in the db total.
What he didn’t have was more than one revenue model.
Prokofy Neva
Mar 13th, 2007
Well, Kimberly, we don’t have to get into the Cory O. fan club here, or take it on faith from the oh-so-special FIC who have some insider information about the Lindens’ plan (or want you and their prospective clients to *think* they do).
We can ask questions about the architecture of Second Life, which is, well, straining at the cracks and seams mightily.
However, putting up Croquet as the option to take over the lead is preposterous. Croquet is even more geeky than SL, if that’s possible. You can’t even enter the world without knowing programming languages. It has like rabbits and boxes as the avatars — and while they are now putting up some interesting projects, these are projects run by sequested feted geeks with grants, not open public experiments in mass creativity like Second Life.
See, for all this snottiness we have to endure from the Web 1.0 Worthies, they can never explain that basic contradiction: the stuff that’s open source is as closed as a Kremlin vault *unless you know the code or unless you know the people to get in on the project*. You can sneer that anyone can go “learn this” and anybody can go suck up to the right geeks-with-grants, but hey, I simply think you’ll be sounding pretty lame there, contrasting THAT process with an open, free-of-charge experience like Second Life, where anyone can hit the ground and consume and make content from the first day.
By the time Croquet becomes user-friendly for dummies, the geeks in it will have made all the decisions about the world they’ll make it without us. Sure, they’re saying, “We’re taking our time, we’re making it right…but we’re making it without you.”
As I posted to the guy linked in Lombardi’s blog, at least you have to give Philip Rosedale credit, he said fairly early on, “Come on in everybody, the water’s fine!”.
Still, he has to stay up late now and do some real homework on justifying this architecture stuff. I’m not going to be put off by facile comments like “20th century system” because, um, bridges are also from previous centuries but we don’t knock them, either, eh? Doors and windows, too.
I think probably the Lindens will be forced to open up separate grids. That will make people like Kimberly happy as she and her clients can be free of blingtards.
Kimberly Rufer-Bach
Mar 13th, 2007
Prokofy wrote: “Well, Kimberly, we don’t have to get into the Cory O. fan club here, or take it on faith from the oh-so-special FIC who have some insider information about the Lindens’ plan (or want you and their prospective clients to *think* they do).”
“Fan club”? “FIC”? “Insider information”? Huh? I do run a business, and it involves creating content for clients in Second Life. Of course I believe Cory, the Linden Lab CTO, knows what he’s doing, or I wouldn’t be betting on his product. When I first came to SL a few years back I would not have done so, but after watching Linden Lab in action, repeatedly solving technical challenges in innovative and flexible ways, I became convinced. I give Cory, the CTO, credit for that.
“I think probably the Lindens will be forced to open up separate grids. That will make people like Kimberly happy as she and her clients can be free of blingtards.”
I imagine that’s something Linden Lab is considering, but it seems like something they’d work very hard to avoid. I’ve certainly had inquiries from prospective clients who wanted to know if they could have sims separate from the main grid for various reasons. However, most of my clients would choose to have their sims be a part of the Main Grid, and so would I.
“Blingtards”? I have bling on my sneakers, and I wish I owned them in real life. This is getting off topic, though.
Nacon
Mar 13th, 2007
Duh… haven’t you noticed that their last two blogs about SL’s server problem had comments closed?
They knew and covering their head up in their ass.
From what I heard… rumors that is, LL may go with separate grids idea by making two worlds… where you travel in space, thus.. a 3rd temp-grid in between for space junkie & cowboys to dance around.
What I can foresee… a L$ fee to move your account between the grids.
Khamon
Mar 13th, 2007
“By the time Croquet becomes user-friendly for dummies, the geeks in it will have made all the decisions about the world they’ll make it without us. Sure, they’re saying, ‘We’re taking our time, we’re making it right…but we’re making it without you.’” – Prokofy
This statement confuses me. It’s like saying “by the time we get to the hospital, medical professionals and insurance agents have already decided how treatments will be conducted” or “by the time we get to the grocery store, the owners and stockpeople will have already decided how the products will be arranged and the colour of the carts” or “by the time we get to college, some educational professionals will have already gleaned through the material and formulated solid course offerings” or ” by the time we take that cruise, some engineers and directors will have already built the ship and provisioned it with limited activity opportunities” or “by the time we read the news, on scene reporters and their editors will have already trimmed what they don’t want us to know and phrased it all according to their own ideals.”
If the masses don’t want, or don’t have time, to learn how to build and operate virtual worlds, we’ll have to use ones built and operated by people who do. That’s not snotty; everything works that way. The only inherent evil is having no choice of worlds so that we’re forced to live under one single All Hail The Central Lab. Yet, other platforms are consistently attacked for not being Second Life. Trevor has to post defensive comparisons between Ogoglio and SL every other month because peoople push this one track mind on him. I imagine Lombari’s doing the same.
FlipperPA Peregrine
Mar 13th, 2007
I think Croquet’s lack of popularity and usability speak a lot more volumes about the team working on that product than Linden Lab’s scalability. Remember the pile-on for the Benshee, when we struggled to get 4,000 all logged in at once, and the grid barely survived? Well, it’s handling 30,000 at once now, only about what, 18 months later? Granted, there are still far too many bumps in a road, but that’s an impressive track record of improving scale.
nimrod yaffle
Mar 13th, 2007
Flipper, I think the load after the Log-a-thon was all the Benshees people were rezing (some with many hundreds rezed per person!).
FlipperPA Peregrine
Mar 13th, 2007
Hehehehehe, could be! Those things were somewhat lag creating.
dildo baggins
Mar 13th, 2007
>>but that’s an impressive track record of improving scale.
LOL! Are you seriously an *executive* at a major dev company and coming out with inane statements like that?
Try checking the ratios of signons to active users, churn rate due to crap experience, and loads of other more intelligent useful metrics and you will come to the conclusion that the track record of improving scale is pathetic.
Look, we know you have a job to do as a big cheese in the metaversal development and +ve spin etc etc blah blah but don’t be a dick and make dumb statements like that…you just make yourself look like a dork. What you going to do next..start wearing gay hats on TV?
Khamon
Mar 13th, 2007
One *does* have to grant that the Lindens are terrible examples to new residents who study their building and scripting techniques. Their textures are massive, even when applied to smaller prims. The number, and tortured types of tiny detail prims in builds such as the welcome centers, and superlagacious scripts, display utter lack of regard for frame rates. It seems as though they’ve no concept of the platform’s limitations. That’s silly though; of course they know. They just don’t care that their own creations lag the world and teach people bad habits.
Khamon
Mar 13th, 2007
One *does* have to grant that the Lindens are terrible examples to new residents who study their building and scripting techniques. Their textures are massive even when applied to smaller prims. The number of tortured, tiny detail prims and lagacious scripts in builds such as the welcome centers and library items, display utter lack of regard for frame rates. It seems as though they’ve no concept of the platform’s limitations. That’s silly though; of course they know. They just don’t care that their own creations lag the world and teach people bad habits.
Khamon
Mar 13th, 2007
Okay I’ve no idea why that posted before I was finished editing it. But if there’s a mod around, please feel free to delete the first copy. thxbye
Prokofy Neva
Mar 13th, 2007
“Fan club”? “FIC”? “Insider information”? Huh? I do run a business, and it involves creating content for clients in Second Life. Of course I believe Cory, the Linden Lab CTO, knows what he’s doing, or I wouldn’t be betting on his product. When I first came to SL a few years back I would not have done so, but after watching Linden Lab in action, repeatedly solving technical challenges in innovative and flexible ways, I became convinced. I give Cory, the CTO, credit for that.
*Bursts out laughing*. You’re as FIC as they come, Kim Anubis, please, spare me. We all realize how important you, your clients, and well, everything your winged little fairy hands touch are, but we mortals will go on being a *teensy bit* more critical of Cory O et. al. than you. I run a business and pay tier and care about Second Life too, but I don’t see why I have to pretend the performance sucks.
There is a real controversy around this that you are pretending doesn’t exist merely because you’re taking one side in it, and that is the decision to go open-source and fete up libsecond life and focus on programming new features and chasing big corporate interest rather than stabilizing and enhancing performance for more garden-variety users. That may be a rational venture capitalists’ decision, but in the long run, it may flop and cost even the loyalist customer base, I don’t know.
>I think probably the Lindens will be forced to open up separate grids. That will make people like Kimberly happy as she and her clients can be free of blingtards.”
>I imagine that’s something Linden Lab is considering, but it seems like something they’d work very hard to avoid.
Why? Because they are hippies holding hands and singing “We Are the World?” not for long, really, when they finally have to make the thing profitable and focus on how to do this in the quickest possible way.
>I’ve certainly had inquiries from prospective clients who wanted to know if they could have sims separate from the main grid for various reasons. However, most of my clients would choose to have their sims be a part of the Main Grid, and so would I.
You need to get out more. Most of the businesses in Second Life now trying to operate sims, show up, run events, and do stuff, as well as the educational organizations, are frustrated with everything from lag to capacity to griefing to lack of control over controversies like ageplay. They would LOVE to have a separate grid to themselves, and LL would LOVE to charge for this and filter out the blingtards and the hoochie-haired and fetishists. Maybe if they make the performance bad enough, those people will go away?
The idea that businesses want to incorporate into the world is one of the bigger myths that the businesses themselves and the metaversal development companies are perpetrating. Most of the companies don’t do any outreach, they don’t penetrate into the rest of the world — it’s too down in the weeds, too time-consuming, and probably not warranted. There are a few exceptions like IBM, but most stay on their islands, those rare times they show up. They are prototyping something for a future, better-functioning metaverse, not *living in it as it is now, today.*
>”Blingtards”? I have bling on my sneakers, and I wish I owned them in real life. This is getting off topic, though.
Maybe you should return to your roots, hon.
Khamon,
That’s quite the interesting statement. I never realized those, never really studied those builds. I remember I used to think it was ridiculous that Waterhead involved a telehub that made your avatar rez and bump his head hard and see stars while coming out. Headbangers’ Ball. Strange way to welcome to SL — but perhaps a subliminal message.
And that message is: “we’re just playing at building this and prototyping it for some future metaverse, we’re not actually interested in living in it now, we hope we’ll get better jobs elsewhere”.
As for your other uber-supreme-tekkie comment, my God, you do need to get out more.
>This statement confuses me. It’s like saying “by the time we get to the hospital, medical professionals and insurance agents have already decided how treatments will be conducted”
Um, no. Because at the local, state, and national level, citizens have had an opportunity to discuss, vote on, and decide issues like whether there will be universal health coverage and what insurance will cover. They will have had a chance to research and study alternative treatments on the Internet, with social groups, and with other doctors in a second opinion. If a patient dies under treatment, the media will investigate it and cover it. There’s a whole rich texture of civil society and governance that simply doesn’t pertain.
If you only let doctors set up the entire health care system without input from patients and others with interests, like government regulators, you’d get that sort of unfair situation you have in geek-led worlds. And indeed, to the extent that the medical profession claims special knowledge and arrogance (as it has in some quarters), to the extent it’s problematic and becomes a target of Congressional committees and such.
>or “by the time we get to the grocery store, the owners and stockpeople will have already decided how the products will be arranged and the colour of the carts”
No, because that implies that Second Life is just a rack with services or a store; instead, it’s a world where a whole series of choices get made — and have been made for years.
If I had to chose between, say, having more stability or having the Lindens code up yet more patches that hide people online or mute ever more new rows and tables of people, guess what I would pick? But I don’t even get to comment on this, nor do most people. Features are decided at the whim of coders, who aren’t answerable even to overall Linden leadership at times.
>or “by the time we get to college, some educational professionals will have already gleaned through the material and formulated solid course offerings”
another REALLY BAD analogy. Because it makes it sound like SL is merely a book shelf, or a bunch of Internet links, a hodge-podge of whatever anybody puts up, or the result of some sort of committee of libraries.
But it isn’t. The Lindens decide, for example, that HTML on a prim isn’t as much of a priority as voice. Voice beats text, even though we already have voice in a place called “real life” and on a thing called “the telephone” and even “Skype”. These kinds of political decisions are constantly made, and usually under the guise of not being political, but actually are heavily politicized.
SL is bad enough, but you could posit that the Croquet people have made their geek thing in total sequestered geekitude. They’ve only summoned their own kind to get involved on suffrance. They think this is fine, because they think, well, you need coders and designers, you don’t need ordinary people or doctors or lawyers, they do their job, we do ours. This is the sort of corporativist approach that leads to authoritarian societies.
And BTW, the problem with tekkies these days is that they think they pwn all the professions because every single profession now depends on them because it’s all online and in dbases. and that makes them think they’re experts on everything from surgery to monetary policy, when in really, they are annoyingly limited folks who parse everything on a yes/no toggle and have trouble looking up above their screens.
Well, here, you're positing that something like SL or Croquet is like a big Hollywood movie, or a big CBS news story, or a cruise to the Bahamas -- a prepackaged set of media entertainment. And perhaps that's how it should indeed be viewed. But I aspire to more. I think it's a world, a place, and it has to have democracy in it and especially given its user-content claims, has to have a very "nothing without us/about us" sort of stance.
>If the masses don’t want, or don’t have time, to learn how to build and operate virtual worlds, we’ll have to use ones built and operated by people who do.
Um, Khamon, really people like you in the next generation seriously scare me.
The masses cannot perform brain surgery or figure out a hospital budget. But they can participate in the public policy discussion about what the health care industry can cover and what it can’t. If you don’t think things like health insurance and stem-cell research are subjects that the public should discuss and be involved in, well, I understand the problem better now. You really do believe society is something best run by elite technicians and everyone else should just STFU.
And…the whole point of SL is that it is suppose to be YOUR WORLD/YOUR IMAGINATION. Of course, you’re secretly revealing now that what you think that REALLY means is “GEEK WORLD/GEEK IMAGINATION’ and that Philip was only calling to his own like you and Jarod. The rest of us showed up by accident and we should find our way to the door now.
>That’s not snotty; everything works that way.
Hell, no, it doesn’t. HELL NO. I don’t let computer geeks and the bureaucracy they have spawned in awful places from the city school district to the police station to the health care clinic rule my life — and others don’t, either.
>The only inherent evil is having no choice of worlds so that we’re forced to live under one single All Hail The Central Lab.
Well, good! They’ve taken more care than YOU wish to take to have more participation from more kinds of people. So…why aren’t YOU in Croquet, Khamon!
>Yet, other platforms are consistently attacked for not being Second Life. Trevor has to post defensive comparisons between Ogoglio and SL every other month because peoople push this one track mind on him. I imagine Lombari’s doing the same
Well, it’s just that people making claims to the challenge of SL should live up to that challenge and not be something else completely.
I mean, if Croquet were to say, “You know, we have this little geekworld thing going that we can’t let people into because we’re taking it slow and going geek with it, but you know, down the road it will be really cool and then we’ll let you in,” that might be interesting — but they don’t really say that.
I continue to submit that when you make a world, you have to start out with all kinds of people. That should have been done at the get-go with all these worlds.
Kimberly Rufer-Bach
Mar 13th, 2007
My “winged little fairy hands”?
As cute as that is, Prokofy, I have no idea where you came up with it. I’m not the developer known for her wings. I’m the one wearing the top hat. It’s just one of the many things in your post here that indicates that you have no clue who I am, what I’m up to, and what I think. It’s funny that you decided to pretend otherwise at such length.
I didn’t say anything about whether or not SL has performance issues. Everyone knows about them, so it wasn’t necessary to mention them. The only one pretending otherwise here is you.
I believe Linden Lab will continue to fix them and that their product is scalable.
The decision to open source the SL client was a good one, for a variety of reasons. I think, based on past releases, that Linden Lab is working on both addditional features and stabilizing and enhancing Second Life performance. But really, no one but the Lindens themselves know for certain what percentage of their effort is currently being spent on either of these things — including you.
“That may be a rational venture capitalists’ decision, but in the long run, it may flop and cost even the loyalist customer base, I don’t know.”
No, you don’t. However, we all know SL would have already flopped — gone broke — if Linden Lab hadn’t received venture capital.
Anyway, as for your comments about “most businesses in Second Life” . . . my company isn’t “most businesses” and neither are my clients. Clearly, you aren’t familiar with my company’s work. That’s not a big surprise. I haven’t focused much on marketing.
There are other solutions to social problems (like griefing) that are preferable to creating separate grids. Some want their projects to be hosted separate grids for other reasons, and perhaps they’ll get them. Most of my clients wouldn’t want that, and I don’t. You can post disbelievingly, Prok, and try to label me as some sort of wing’d proletariat-hating immortal. But you don’t know what you’re talking about.
Khamon
Mar 13th, 2007
Thank You Prokofy, I’m no longer confused.
Nacon
Mar 13th, 2007
Of course it’s scalable…. just going on a long painful path at it.
flakfizer
Mar 13th, 2007
I hope theory of TeaTime messaging in Croquet will turn to be true. What I find inapropriate is claiming scalability as advantage before actual proof of that. The architecture is really beautiful, but practice and real world tests may show the negative side. With scalability issues being the major problem of SL right now, such claims are hitting under the belt. I do hope Croquet holds its promise.
And Prokofy, Croquet is geeky not only for the regular users, but for the average software developers too! Smalltalk isnt one of the popular and spread programing languages, again claiming learning curve of a totally different approach to programing + APIs and techniques to be under 2 weeks is really hillarious. I have watched presentations by Lombardi and my first impression is that he is flying in the sky most of the time believing in untested technology. One thing we know – Second Life is changing its internal organization because its architecture is last century, but that is even better then not knowing if an untested theory holds true.
YHelloThar
Mar 14th, 2007
http://www.encyclopediadramatica.com/index.php/Urizenus_Sklar
LOL THE TRUTH ABOUT THE “GUY” THAT WROTE THIS
I HERD YOU LEIK THE PENIS
Nacon
Mar 14th, 2007
Heheheh, that’s a amusing one.
Anywho, Flakfizer has made a very good strong point there. Thus time will tell with Linden Labs and new developments slowly coming in.
Ex:… Steve Linden on FPS rate.
Prokofy Neva
Mar 14th, 2007
>It’s just one of the many things in your post here that indicates that you have no clue who I am, what I’m up to, and what I think.
“Kimberly,” that’s always been your theme song, ever since I began publicly criticizing you, instead of listening in awe to you inworld for hours like your other ex-fans lol.
You don’t have wings, but you appear as a tiny bunny or something, and you have various fairy stuff out at your store. I believe you even invented one of those following fairies that can be so annoying. So I simply remember you as fairy-like and magical. It’s definitely not mixing you up with any “other” big clunky thunder-thighed purple-winged thing, no.
“Kimberly,” the kind of venture capital that LL has is something that isn’t really “venture capital” in the sort of usual cut-throat Silicon Valley way, it strikes me. Of course, I’m looking at this from afar, from the East Coast. The capital used to start Second Life comes from a) Philip Rosedale himself, the idea guy b) the ebay guy, Pierre Omidyar, who is also a philanthropist and who might have just as easily done all this as a philanthrophy project, but he just wanted to attract more buzz and involvement no doubt; and c) Mitch Kapor, another very visionary, ideological sort of fellow who doesn’t just look at the bottom line.
I’m very familiar with your resume as you bruit it about everywhere, “Kimberly”. BTW, I’m putting your RL name in scare quotes because I believe you to behaving not like a RL person with their RL name, but a person in a virtual world with an anonymous identity that enables you to be mean-spirited and arrogant to other people.
You seem to forget that I spend many of my waking hours in Second Life staring at things you have made in Carlysle and Hyle. They’re fine for what they are. No doubt you have done good work for your clients elsewhere.
It’s good you’re putting yourself on the record as saying that you don’t want to be on a separate grid, nor do your clients. It will be good to contrast and compare that to whatever justification you make for moving to the separate grid, becaue that’s what you and others who wish to make money in Second Life will be forced to do.
Kimberly Rufer-Bach
Mar 14th, 2007
Sure, I’ll go on record saying that most of my clients (not all) would not want to move to a separate grid like you’ve described. And, as I’ve said, I wouldn’t want to, either. However, if clients I wanted to work with were locating their sims on another grid, I’d set up their content there, similar to the way my company currently develops content in the Teen Grid for clients.
It’s funny that you’d want to stock up this info for later reference, just in case. You must think I’m a public figure of influence and importance. Or maybe it’s that you’re still upset over our personal falling out a couple years back, and are just hoping for future opportunities to flame me.
Slade
Mar 15th, 2007
Predicting technical dificulties with a server side system is like predicting that you will have problems with your car. It’s stating the inevitable. Here are some simular prodictions: “you will live in intresting times”, “some one close to you will die.”, “you will have some luck in the near future.”, “Whatch out for men with big forheads.”