Op/Ed: Sky Isn’t Falling – Ground Is Rising — Fast
by Pixeleen Mistral on 11/01/07 at 12:49 am
Watermelon and gaffer-tape bodge
by Inigo Chamerberlin
Well, here I am, Wednesday 10th January 5 pm PST, stuck on my private estate, like I have been for the last 3 days. Objects are going off-world and NOT being returned like there’s no tomorrow, the packet loss (which according to Neotrace is all in the Linden Lab network) is monumental.
I can’t teleport anywhere, if I log out and log back on, slowly, to a named sim, like as not I arrive with various attachments jammed up my ass and can’t move, can’t even look around properly because my camera’s jammed. When I do get to move, average 4 relogs, I find the place littered with ‘ghosts’.
The ‘new’ search is so slow it’s pitiful, none of my friends are online – or maybe ‘presence’ has gone down again, you simply wouldn’t know. Can’t even look at the Linden forums – try and go to a section and mostly get a blank page, posting in ‘Linden Answers’ simply fails.
There is nothing about all this on the Linden Blob.
No response from ‘Request Help’.
We’ve been officially informed Bug Reports do not, never did in fact, go to Support
There doesn’t seem to be any way of contacting ‘Support’ now either.
What IS going on?
Anyone who wasn’t familiar with LL’s approach to their customers might think LL had abandoned ship.
Have they? Well, this brave and oh so generous ‘making the SL client open source’ could easily be interpreted as LL washing their hands of a piece of software that they simply can’t fix.
People with far more technical expertise than I have aren’t impressed with the source; I’ve heard it described as ‘a mess, full of inactive components, un-commented, undocumented, a hodgepodge of broken code, a heap of crap’. Just the sort of thing in fact that you might want to get rid of.
Assuming the server side software is no better, and there’s no reason to suppose is isn’t – and every reason to suppose it is, especially seeing the way the latest ‘update’ has turned out. What now?
For fairly obvious reasons they can’t make the server software open source, they pretty obviously can’t fix it themselves, so, IS there a way out – besides the door the Linden fan club used to be so keen on advising people who were waving warning flags not to let hit them on the ass on the way out?
I just don’t know. It looks pretty hopeless from here I’m afraid.
LL, as usual of course, is in full ‘fingers in ears chanting ‘lalalala – we can’t HEAR you’ mode – like pretending things aren’t massively broken AGAIN will make them ‘not broken’.
In a few days they’ll throw together another watermelon and gaffer-tape bodge to try and reanimate the terminally sick grid. For how long this time?
And just how long do they think this fiasco can carry on? How long before the patient flat lines for good?
The FACT is that the grid becomes unstable over the 10k concurrent user mark, past 15k it becomes difficult to use, at 20k it becomes so nasty that anyone with half a brain simply doesn’t bother.
I know, I know – what am I doing logged on as I write this. Well, maybe I only have half a brain? But what am I doing? Well, nothing really, there’s not a lot I CAN do, but paying the money I do each month it seems a pity not to at least try and use it. Besides, logging off and leaving it to the freeloaders feels like letting them ‘win’…
The FACT is that LL have consistently failed to produce a remotely stable update, whether combined (server/client), or server only for, er, it has to be well over a year now.
The last time I remember things running quite well was back before 1.7… And it was right after that that we started having ‘pile-ons’, which of course, though we didn’t realise it at the time, were aimed towards free membership and ultimately, the infamous 06/06/06 introduction of completely unregulated free registration.
I just hope to God all this ‘nearly 2.5 million (fake) users’ bullshit is a prelude to a sell off, and soon.
Because if someone with the necessary backing and expertise doesn’t take this mess on really pdq I seriously fear for the future of Second Life.
Just a thought
Jan 11th, 2007
Don’t know what you’re babbling on about – I haven’t had any major problems in quite a while.
The only problem I have ever had occurred when the servers which handle your presence were kicking off a bit more regularly. since then not a single glitch.
Spankubux
Jan 11th, 2007
yeah, what are you going on about? I logged in for the first time in a few days tonight and actually spend several hours online doing my thing.
Do you just make things up for publishing purposes?
Lewis Nerd
Jan 11th, 2007
You probably got caught up in a script kiddie’s attempt to see what his modified open source client could do. Expect plenty more.
Animetia Bellman
Jan 11th, 2007
I think the problem migh tbe on your end frankly. I have had 0 problems with SL wince it came back up toaday after the brief teleport hiccup. I have been visiting the SL forums and website all day as well … no problems there either. I’ve been able to post and browse just fine. Something tells me you should check things out just a bit more before making such an overly dramatic posting and calling it “news”.
Warspite Fackler
Jan 11th, 2007
The Neotrace claim is correct.
Just by surfing webpages and forums the specific network performance ingame cannot be benchmarked
Yes this is not news, but in fact it’s a counterpost to the not very techsawy posts that were recently made by the lindens “adressing problems, taken steps, dismembered a chicken to the dark gods of ethereal transportation…”.
LL is more and more turning into a Dilbert environment… no wait, they’re already beat me to that.
Inigo Chamerberlin
Jan 11th, 2007
Interesting thought Lewis – I wondered a bit about that too. Oh well…
Inigo Chamerberlin
Jan 11th, 2007
Animetia – if you trouble to look at the title of this piece you’ll see it’s clearly labeled ‘Op/Ed’, that, in case you hadn’t realised, stands for ‘Opinion/Editorial’- which is what the article is, My opinion.
The only person claiming this is presented as ‘news’ is you I’m afraid – nice try though.
And as for the problem being at my end – ever seen or used Neotrace?
Obviously not. It shows exactly where PL issues are originating in real time, and last night, and the last couple of nights, the problems were entirely within LL’s facilities and the associated Level 3 network.
Depending on the SIM I was in, I was hitting Huston or SF, and interestingly, seeing I’m in England, Huston, which is nearer, was much worse for PL than SF.
Everything from here was pretty good, until I hit Level 3 in the ‘states, from there on, downhill all the way to the sim server.
You know, this is just like the old forums – a bunch of technically illiterate LL fans telling anyone with the temerity to draw attention to LL’s shortcomings that the problem was all at their end.
Well get this – it didn’t work then, it doesn’t work now, and things have got worse, not better.
Inigo Chamerberlin
Jan 11th, 2007
‘Dilbert environment’
Damn, why didn’t *I* think of that Warspite?
OK folks, so, votes for ‘Dilbert Linden’?
Which lucky Linden deserves the accolade?
Prokofy Neva
Jan 11th, 2007
Inigo,
I’ve had Verizon swear the same thing to me, too, even sending me the traces showing it’s on LL’s side. I’ve had Phoenix Linden send me a trace claiming it is on Verizon’s side, I’ve called up Verizon even spending hours on the phone talking to people, and pasting Phoenix’s claims, and getting only puzzlement or laughter. I don’t expect Verizon to admit it’s wrong, but the way some of the Verizon people talked, I began to think it really wasn’t a Verizon cover-up.
What’s particularly hilarious is that I’m supposed suffering packet loss from Verizon on X sim, but as soon as I fly to Y sim, my red bars magically disappear. I mean, if packet loss is on my side, how could it simply disappear during the same log-on session merely by changing from say, one server that is in California perhaps, and another in Texas? or even moving between two Cali servers I know are both in Cali as they are older? ?
Because I have a big system with now some 1200 people in it in 12 groups across 58 sims, I can see whether the problem is “on my side” or is on “all of our sides”.
I find that typically, in a bad day like yesterday I will have the following:
o one of my own builds or content sites completely disappearing, borking, or damaging — like yesterday the entire plaza of the Flamingo Court disappeared. It’s not that it isn’t rezzing, it’s not in lost and found, and clearing cache will not find it again. It’s just gone, part of a weak spot in the build that simply got eaten by the servers
o at least one if not 6 tenants will call and say, hey, why did you return my furniture? or why is this other guy’s furniture now in my lap? The *rearrangement* of items and builds on servers is one of those untold stories of SL. I log on, and I find a picnic table and hotdog up in the air. How can that be? It’s not on “anyone can move” and it’s my object. It’s just — flung up in the air. And the tenant sometimes mercifully will find her stuff by using clear cache — but really, must we all clear our caches daily for this stuff
o By and large I still get enjoyment and a business profit from SL. That’s because I spread my cards over many, many tables. If you are on one big sim or one big continent, or only on your 4096 m2 and that’s it for your SL experience, then you are fucked.
o I routinely keep several projects half finished just to reroute around them and not waste my time. I know that Ross is chronically at 3 FPS (!!!) so I can’t do things like save notecards in helper FAQs boxes so I leave that. Or I sit on Tuliptree, a better sim, prepare all my notecards and scripts for Ross, then go on a bomb run to Ross to drop off the products, hoping they will stay on the ground and that even rezzing them will be possible — yesterday I sat in Refugio unable even to rez an object.
o What I can see from my little bird’s eye view that the people on one sim, not 96 m2 from others, can have a completely different experience of SL, and even become angry and hostile if you get angry at LL yourself.
o As I commented back to Doc Nolan, she had a bad set of sims, her 2-3 sims were fucked from birth, they should have been DNR’d. I had land on one of those exact same sims which has never been “right”. I disagree with her approach and the one Wayfinder Wishbringer used to use on the Lindens which was to rant and rave endlessly that their performance is in the toilet, when it was only in the toilet on that one sim for reasons unknown. Any Linden looking across their system could characterize as 70 percent good running and healthy a system seriously flawed for 30 percent of the sims, and look good.
o What to do about this? Well, when I purchased phone service from ATT and they never get it to work in my building, which has several complications, and the building is suffering outages, I switch to Verizon who is more adept at working the issue. You can’t do that with LL obviously. But I do think you can demand a replacement land or sim that does work.
o Only when a consumer movement gets going that is loud and strong that documents and aggregates and forcefully presents to the Lindens the chronic problems on bad sims will it be taken seriously.
Prokofy Neva
Jan 11th, 2007
BTW, I’m well aware that sims are shared among servers, that in fact you may have a different server over the life of your land ownership. But it’s my distinct impression, based merely on my field observations for two years, that certain sims themselves — their terraforming, Linden content, and boundaries — as a package — are in themselves weak, so that even on top of a good server, they are messed up. I don’t know why this is so, I only know *that* it is so.
Seola Sassoon a.k.a Random Writer
Jan 11th, 2007
I’m with you 100% Inigo.
A sim I work in, two days ago, in 4 hours crashed 9 times. Last night, I couldn’t tp for 3 hours despite Linden’s saying it’s resolved.
The last issue, when they said ‘oops, it’s broken again’…. then ‘resolved’, I lost some shoes AFTER “resolution”. I couldn’t tp 40 minutes after “resolution”. I did the channels to tell them it’s still happening and was basically laughed at, at which point I got a lil miffy and said… just because people aren’t reporting it en masse, doesn’t mean it’s not happening. Hell, most are just logging off, or probably having so many problems they can’t bring anything up to tell you!
Inigo, as for the op/ed piece, it’s as if I wrote it. Everything you said is exactly what I think, what I’ve seen, and what’s going on.
I’m thinking some of these people who didn’t experience problems, aren’t coming online at roughly 1pmSL time – about 8pm SL time, which seems teh crux of the difficulties for me.
Seola Sassoon a.k.a Random Writer
Jan 11th, 2007
P.S. Also, those not experiencing problems also aren’t trying to build or create…
Textures are now officially borked… my prim links are losing permissions, and half the time they disappear altogether, never to be seen again. Repeats are messed up because of lag, and only when you set it about 50 times, then pick up the item and drop it again, you can hope it saved and will look right…
My attachments also are stuck pending for 30 minutes+ when I try to attach newly created.
And honestly, quite a few merchant transactions I’ve been doing are going stale.
Prokofy Neva
Jan 11th, 2007
I’m wanting to hear now whether the proximity of a server really means anything. I mean, if you are near to Texas or California, what that is worth. Could it be if the Lindens and some of their special friends are in California, they don’t feel the lag? But surely the Internet isn’t set up like that as if it were a tin- can phone with a wire? And why would say, somebody in Ohio or wherever twixt Cali and Texas not get better service than those far away from both?
Just a thought
Jan 11th, 2007
distance, status of the lines being used, improperly routed packets …. the list of factors is endless – and can be pinned on no one.
Your ISP is going to tell you it’s LL, LL is going to tell you it’s your ISP. guess what? Both of them are only half right.
Macphisto Angelus
Jan 11th, 2007
Just my 2 cents..
I live in california. About 3ish hours away from San Francisco. I experience all the stuff mentioned here more often then not lately. My provider is Comcast. So, different provider and close proximity = same problems. What does that tell you as far as where the issue originates?
Inigo.. I am sorry to see you are STILL dealing with this stuff. You have been having nothing but problems with that Island since you got it. I remember your past posts and am amazed you still have this island. I would have gave up long ago on the thing.
It is nuts that LL hasn’t done something for you yet.
Inigo Chamerberlin
Jan 11th, 2007
I wasn’t commenting on the island really. Ok, stuff goes ‘off world’ way too often. All that means is that the Asset Server stopped believing in the items in question. Just like stuff that vanishes from inventory from time to time. Sometimes it returns, sometimes it doesn’t. This is SL – shit happens – daily. I’m not entirely sure it’s a problem solely caused by the asset server cluster either. I believe a big chunk of it’s caused by SL’s internal networking systems. Poor network architecture and a poor choice of networking protocols will screw up best designed systems.
At present my sim’s running a bit slow, I know why too. I have too many physical objects wandering about. But that’s how I choose to run the sim, it’s non-commercial, purely for my pleasure, so it doesn’t matter, as long as I get 30 or so personal fps I’m happy.
What’s driving me wild is day after day with no TP – it’s a nice island, but I like to get out now and then!
And then to discover, when I do escape, the rest of the grid suffering massive PL, severely limited TP’s, extreme border crossing problems – even walking! – ghosts, 1.7 speed object rezing and texture rezing, etc, etc
All this while LL blandly assure us it’s all ‘fixed’! Fixed how? Like a cat or a dog get’s ‘fixed’?
That and the complete lack of any attempt now to provide any form of useful support. And then they wash their hands of the client!
Sorry, I’m ranting again, but I’m angry and amazed that a company run by a supposedly intelligent man can come apart so comprehensively, so quickly. Yes, it’s been predicted – I’ve done my fair share of that – but you know? It’s kind of taken me by surprise…
Tonight’s a little better though. Only took one try to get within 4 sims of where I wanted to go on the mainland. AND I managed to fly the rest. However, 4 border crossings resulted in me arriving bald, with a katana up my ass (that smarts), my cloak around my waist, and my belt around my knees…. And that’s with everything ‘fixed’ remember.
What was it Zero said? Something like ‘you’ll hardly notice’? Well maybe he should drop by and bend over so I can stick that katana somewhere he’ll ‘hardly notice’!
Bah, I should shut up and stop whining – except… if we don’t make waves, are the Dilberts – sorry, Lindens – ever going to fix the grid? Most notably the asset server and DB? Somehow I doubt it. In fact I don’t think they can… Still, you have to try, don’t you?
There’s been discussion about making the sim server software open source too, so people could host their own sims.
I can’t see that – for two reasons. Firstly LL would loose a big chunk of their revenue stream pretty quickly. Secondly, there would still have to be a ‘grid’ to facilitate inter-sim transport and to keep track of ‘assets’. And those are two things the grid is failing miserably at. Hosting your own sim server you’d STILL find yourself paying LL for a service that simply can’t perform as it should! Open source server software would result in a loose cluster of sims, isolated from each other 90% of the time.
That’s a non-starter I’m afraid.
Hmmm, I’ll be quiet now. Message sent and understood I hope?
Scott
Jan 11th, 2007
Second lifers are paying for a service that is being trashed by giant companies, linden lab sitting back, and doing barely anything.
I dont know why i am paying for it, the service is dreadfull.
Warspite Fackler
Jan 12th, 2007
http://www.dilbert.com/comics/dilbert/archive/dilbert-20070112.html
Scott Adams must be playing SL.
Inigo Chamerberlin
Jan 12th, 2007
Leading the developers team maybe?
Seola Sassoon a.k.a Random Writer
Jan 12th, 2007
And to add to the conundrum….
Last night, I had a few people in my place dancing. Of course it was couple dancing so we were using the slow dance poseballs.
Sim I live in crashed… and guess what? The poseballs disappered. GONE! Poof and vaminosed away. I asked the sim owner to check for errant items… nada. In fact, I am now 4 prims UNDER my limit, when maxed before.
And where I work, supposedly a Linden came out to check the sim out and it’s horrific crashing… the Linden blamed it on couple dancing poseballs. And there’s only like 3 out. It never caused any problems before, so if it is those slow dances and salsas… WTH did LL do to make them crash sims?
Macphisto Angelus
Jan 12th, 2007
Seola,
This is the second time in a week I heard poor sim performance blamed on poseballs by a Linden.
I wish I could remember where the first one was to link. :/
I wonder if it is the same Linden saying that or just a new thing to say to people developed in the Lab?
Seola Sassoon a.k.a Random Writer
Jan 12th, 2007
I just don’t understand, how after all these years with clubs having those poseballs, even some that have been there for years, are the reason all the sudden it’s crashing sims.
Since I don’t the the sim I work in (and I’m not gonna say so as to seem a shameless plug) isn’t owned by me, I didn’t get the specific Linden unfortunately.