Phoenix Power Outage Cripples Second Life – Blended Frappuccino Blamed
by Pixeleen Mistral on 30/04/10 at 3:45 am
Geeks with a taste for blended frappuccinos may have been to blame for the Second Life crash that crippled the virtual world over the last 24 hours, according to Oskar Linden’s comments in the AWGroupies chat tonight. Oskar told the Herald that trouble started when the "phoenix datacenter had a complete power outage" and the standby power failed. Oskar claims to have warned against this problem, saying "I told them not to run the blender and the espresso machine at the same time, but geeks love their blended frappuccino’s".
As the news spread across the grid, residents began pooling their L$s, hoping to raise sufficient L$ spacebux to purchase Starbucks gift cards for the Phoenix-based sysadmins and avert a repeat of last night’s datacenter disaster. Some observers suspect that the virtual world disater might have been averted had the operators simply left the frappuccino creation to the professionals – or at least those with approved third party espresso machines and blenders.
Oskar also shed light on why some players have complained of being logged off SL with a message that they are already logged in from a different location. According to Oskar, even the Linden game gods are experiencing the issue, which happens when "one failed login got queued, then you relogged in after closing the viewer - [and the] 2nd attempt goes in, but the first was queued".
Will the grid stabilize in time for the traditional friday night user generated drama content? We hope so – and beg the data center staff to heed Oskar’s warning against making your own frappuccinos.
Iridian Pelliot: Anyone else getting logged off by SL for logging in in a different location?, I just got logged out, I’m logging in in the same location I always do, which is from home, laughs.
Pixeleen Mistral: SL told you you are being logged out for logging on from a different location?
Iridian Pelliot: It did, I was like huh?
Latif Khalifa: someone got your passwordPixeleen Mistral: this sounds like something a bank or credit card company would do – not a game
Pixeleen Mistral: did you get a screen shot of the message?
Latif Khalifa: Iridan, I’d change my password just in case
Iridian Pelliot: Ugg, so I need to change my password?
Iridian Pelliot: No, I didn’t, Pixel, didn’t think about it.
Iridian Pelliot: Okay, will do, Latif.Pixeleen Mistral: pictures or it didn’t happen
Oskar Linden: the multi login thing happens when one failed login got queued
Oskar Linden: then you relogged in after closing the viewer
Oskar Linden: 2nd attempt goes in
Latif Khalifa: today the grid is so fscked up so its perhaps another glitch, but it’s better to be safe than sorry
Oskar Linden: but the first was queued
Oskar Linden: we’ve been getting that today too
Iridian Pelliot: I didn’t ask, or expect you to believe me, Pixel, I just asked if it happened to anyone else, never had it happen before.
Iridian Pelliot: Okay, thanks, Oskar.Pixeleen Mistral: Oskar what happened to the grid?
Oskar Linden: phoenix datacenter had a complete power outagePixeleen Mistral: yikes – and the backup power failed too?
Oskar Linden: apparently
Han Sontse: hmmm someone is going to be in trouble….Pixeleen Mistral: rebooting a data center is not fun
Oskar Linden: I told them not to run the blender and the espresso machine at the same time
Moon Metty: hehehe
Oskar Linden: but geeks love their blended frappuccino’s
xstorm Radek: sounds more like some one must have messed with the system there
Techwolf Lupindo: Good thing the grid status page was hosted out of band.
Oskar Linden: speaking of…
Oskar Linden runs downstairs to get some coffee
pefton
Apr 30th, 2010
OH GOD IT MUST BE AN EMERALD CONSPIRACY I SWEAR
IntLibber Brautigan
Apr 30th, 2010
The espresso thing is a cover story. The truth is that Linden Lab has been put on 24 hour suspension by the Userbase Governance Team for griefing the entire grid repeatedly, defrauding customers, and repeated violations and illegal adhesion amendments to their Terms of Service contract with their customers. These punishments will continue…
Senban Babii
Apr 30th, 2010
@IntLibber
There *should* be a userbase governance team! I’ve said for a long time now that what SL needs is a consumer association to prevent abuses and ensure fair treatment because right now, everything is designed to look after the Flab and the content creators at the cost of the general userbase. If the Flab want to treat us as consumers then they have to realise that as consumers we have reasonable expectations of treatment and reasonable expectations of fair usage. I’m sure someone with more experience than I can wrap better words around it but it boils down to a watchdog body to prevent runaway abuse of the consumer base by the corporation.
It’s all well and good for the Flab to throw their TOS around and say if you don’t like it, leave but sooner or later they have to realise that they’ll have to deal with the userbase if they want to survive. The same thing happened in the real world and sooner or later, if the Flab want to be successful, they’ll have to accept it in SL too.
Apologies for going off on a Prok XD
Gundel Gaukelei
Apr 30th, 2010
“But boss, what happens if the failover is failing?”
“Close your eyes – then you have a rough impression”
Tux Winkler
Apr 30th, 2010
Would be interesting if someone phoned a neighbouring building to the datacenter to see if they to had power outage. I bet not!
I see on all the forums people are blindly congratulating LL for it not being their fault. Ahem, but it is. Their actions have bought this upon them.
/me supports the UGT!
Free the Woodbury nn ( <== insert actual figure here )
UGT supporter
Apr 30th, 2010
The UGT should get a sim or two don’tcha think Intlibber?
Pappy Enoch
Apr 30th, 2010
They am three theories, I done found out.
I done me some snoopin’ at the Welcome Center & I even run down (well, not like my sister Jezz, in her car) Jack Linden durin’ that feller’s office hours. I done asked if’n he knowed what caused it.
Jack Linden: Pappy, yes.. Ops engineers are working on the problems
Jack Linden: Okays folks, I need to run off to my next meeting. Thanks for hanging on for me, sorry again for being late
Pappy Enoch: thank’ee sire
I done meant to say “sir” but I had me one o’ them slips like that German head-shrink feller with the cigar done said that time.
So ol’ Jack didn’t know diddly so I run him right off! I are still bettin’ on some “wisdom from the (fake) streets.” The follerin’ am a randum sample (well, I done picked the funny ones):
Theory #1: Some dumbass stepped on the cord
Pappy Enoch: any theories for the Herald?
Pappy Enoch brought his tin-foil hats
Bronson Blackadder: someone stepped on the power cord that powers the servers
Theory #2: Boobs n’ burps
AlphaQueen Ceriano: Maybe it was boobquake
Pappy Enoch: blame Emerald and them jigglin’ ones
AlphaQueen Ceriano: That and someone important burped, I think
Pappy Enoch: I’ll tell the editor that the boobs done did it
AlphaQueen Ceriano: boobs and burping’ll do it every time
Theory #3: That-there hacker dude who done bragged about it.
Nikki Ormstein: Pappy, you catch that hacker dude in here who was bragging about some sick crap, i poofed
Pappy Enoch: I’m trying to git that done! We’ll git him at the Herald! We don’t sleep till we gits a story!
Desdemona Paramour: “pappy”. that’s a fun word to say.
Pappy Enoch: yep all the gals says that boo hoo hoo. I ain’t got me no paramour boo hoo hoo.
Curious
Apr 30th, 2010
This whole story sounds fishy Operating a espresso machine caused this? I wonder if the local paper in Phoenix mentions a powerfailure. Hmm and the back up generaters didnt work either? I think they needed to shut the grid down for some recoding for todays new TOS and what they thought would be a quick on and off ended taking a lot longer than they planned.
Ted
Apr 30th, 2010
“the “phoenix datacenter had a complete power outage” and the standby power failed. ”
Time for another data center. I’ve never seen a data center go down and backup completely fail. Usually they run monthly checks on the backup generators and secondary systems to ensure this never happens. Most have an emergency plan in place just for such an issue.
To have power down for more than a few minutes at most is not the norm.
Levity Monday
Apr 30th, 2010
The good news is that Pappy lives, despite his retard sister’s attempt to blow up him and his outhouse.
Pappy Enoch
Apr 30th, 2010
The moral o’ this story: don’t drink no city-folk coffee. Stick to Shine.
My new com’py aims to issue them Linden flyin’ monkeys a cornpetitive bid fo’ a new data centur. It are run by fools like me, so you knows it am foolproof:
Enoch Holler, VA.
When the power goes cause somebody were drunk and run a truck into the Dominion Power pole outside, we keeps Ike Bugtussle (who ain’t right in the head and loves a moo-cow) pumpin’ the pedals on an ol’ Stingray bike cornected to the generatur.
I just brings Bossy the Cow in front of that bike an tells Ike “she’s runnin’ awful fast–hop on yo’ bike if’n you wants some lovin, ol’ son!”
We ain’t lost power yet.
Gaara Sandalwood
Apr 30th, 2010
Well this must explain the message I got last night. And to think I messaged Pix in a panic about it. Hehe. Anyway, at least this may mean my passwords are safe.
Pappy Enoch
Apr 30th, 2010
“The good news is that Pappy lives, despite his retard sister’s attempt to blow up him and his outhouse.”
Don’t be two hard on po’ Jezz, just becuz I done got all the brains–an’ looks–in the famberly.
Ted
Apr 30th, 2010
Well, there is good news as well, let’s not forget the good news. As of today the “30th”, there will be no more issues within SL. Only approved clients are allowed to connect, and all of the content and their creators are now protected. So we shouldn’t be hearing anymore of “Someone stole my prims”, or “The Lab should do more to protect my works”.
Everything should be splendid here on out and everyone should be happy. Emerald has joined the ranks of accredited third party clients, so this resolves all the issues for those that want to use one.
Should be no more complaints now as of today. Everyone should be pretty happy and content. Which of course is a good thing.
Tux Winkler
Apr 30th, 2010
/me takes Ted’s words with his 7.5 tonne delivery of salt – XD
Alex Ponebshek
Apr 30th, 2010
Right Ted. Only LL-approved clients are allowed to connect. And since Linden Lab runs psychic servers, that means only LL-approved clients will connect.
Tux Winkler
Apr 30th, 2010
Ok, day one of the new 3PVP. My views and observations.
1. Network throughput is maxed for a few minutes upon connection
2. CPU maxed (has not dropped after 10 mins)
3. You are not blocked for using a unlisted viewer (therefore non-compliant)
4. Lag is bad
5. No super-de-duper bells and whistles welcoming residents to the new world
So what has changed? Well CPU has gone from 40% to 100%, network is hit hard on connection (packet analysis not done yet), and lag is as bad if not worse than ever.
And as of yet the formation of the UGT in world hasn’t happened (Int?) – XD
Antonius Misfit
Apr 30th, 2010
@Tux:
“3. You are not blocked for using a unlisted viewer (therefore non-compliant)”
That’s false. Being listed in the viewer directory is not a requirement to being TPV policy compliant. The viewer directory is only meant as a promotion tool where the developers who list “self-certify” compliance. And none of those viewers have been audited by LL for compliance, either.
Quoted from the SL Official blog( http://blogs.secondlife.com/community/community/blog/2010/04/27/enforcing-the-new-third-party-viewer-policy ):
“We know that there are many compliant viewers in use — beyond the seven that are currently listed on the Third-Party Viewer Directory — and we will not interrupt their access to Second Life.”
What will be blocked are the truly malicious viewers out there, such as the ones listed in the Onyx viewer reference page(though Onyx itself is non-compliant, but LL is giving it a pass).
Curious
Apr 30th, 2010
yeah the key words here are “allowed” and ” self-certify” I will take emerald’s being a safe viewer with Ted’s “7.5 tonne delivery of salt”
Orion
Apr 30th, 2010
Don’t count your chickens on the whole UGT thing. Normal pattern of SL doom:
1 – LL does something to piss a bunch of people off.
2 – Everyone effected screams, whines, and cries for about a week – month at max.
3 – Half the people who were effected either leave, or say they’re gonna leave but never do.
4 – The ones left just roll over and stick their butts in the air while the Lindens have their way.
Add in a few random lawsuits and protest groups that are usually so disorganized that they fail – and there ya have it.
Gundel Gaukelei
Apr 30th, 2010
By now, noone who cares about quality service is left in SL and Everyone left is suffering from Stockholm Syndrome.
xstorm Radek
Apr 30th, 2010
Second Life is a great 3D community with many people from around the world.
It not a game like other companys out there but a nice rounded community filled with many people from many walks of life.
Taking all that in account to think that any company any place can be taken down so fast from non back up power systems with a alarm has to make you ask what if next time its data for a hospital and your the person on the OR table with the doctor looking to comfirm with the robot system thats going to open up your heart ?
How safe are your data base ?
Judge Joker
Apr 30th, 2010
@Orion
As you’re so knowledgeable on the subject, perhaps you would like to post a list of all the “protest groups that are usually so disorganized that they fail”.
IntLibber Brautigan
Apr 30th, 2010
Judge I think that was a commentary on the SOS group that protested the homestead tier hike. Those in the know from the period recall the only real ‘disorganization’ was due to two factors, Lindens and their fanbois spreading FUD within the group, and the old group chat borking that made such a huge group malfunction, which generated its own drama and splitting dynamics.
In the end we did force LL to cut back their planned tier hike, and we did preserve the prim limits that we wanted, and our protests were very successful in filling up the welcome areas with avatars so no new avatars could enter SL while we controlled those sims (which obviously led to LL banning people in a bid to get those newbs in, a rather curious method of population growth, by executing the older population, a practice LL seems to keep doing).
Other protest actions in SL: the old prim tax revolt, the anti-
FN protests, as well as the protest conflicts between the business community and the faux-Arbor Project over in world advertising.
James Larken Smith
Apr 30th, 2010
Get a life folks. He obviously was joking about a cappuccino machine actually causing the outage, geez. And Kennedy was killed by the JLU…..
Tux Winkler
Apr 30th, 2010
Wow James, what page in the wiki is that?
XD
Jumpman Lane
May 1st, 2010
I kNEW it was the JLU and not that sap from the watchmen!
Judge Joker
May 1st, 2010
I think this video speaks for it’s self, Spiderbait – Fucken Awesome!
http://www.youtube.com/watch?v=dwzWajyWzsw
GreenLantern Excelsior
May 1st, 2010
Phoenix has a data center? Oooh!!
Danziel Lane
May 1st, 2010
You didn’t patrol there, GLE?
That explains it all.
Well, at least for some hours the peace was kept in the grid.
To all peacekeepers: for peace in the grid, disable logins.
Senban Babii
May 1st, 2010
@Judge Joker
“I think this video speaks for it’s self, Spiderbait – Fucken Awesome!”
Was that Pedobear dancing at 0.16 O.o
marilyn murphy
May 1st, 2010
did something happen?
Obvious Schism
May 1st, 2010
Wasn’t me. It was already like that when I got there.
The Hulk
May 1st, 2010
Hulk SMASHED computer machine building and puny humans!
Was not JLU Green Flashlight Man and puny DC heroes!
Horny Hulk chase hot geek woman and trip on stupid cord! Then Hulk crush stupid data center! Then Hulk hurl APCs at puny policemen.
Hot geek woman get away from Hulk. Hulk come back next week.
Levity Monday
May 1st, 2010
@Pappy defending his sister even if she did try to blow up his outhouse.
That’s awfully sweet of you, big man, but you stil might want to check that squat of yours. Chances are poor little Jezz did succeed in detonating that TNT.
Between the TNT and the excretions of a man who takes to Shine, who knows — the explosive strength might have been enough to be the real cause of this grid failure.
The Hulk
May 1st, 2010
Hulk know real cause…fat man drink Shine he make with run-off from reactor-machine….check link Hulk use…hah! Flying Monkeys have no chance now!
http://www.slprofiles.com/picturedetail.asp?id=325905&t=1
Jumpman Lane
May 1st, 2010
“Horny Hulk” gawd that is so fun EEEEE hehehe i love it!
The Hulk
May 2nd, 2010
Jumpman like Hulk!
Hulk look at Jumpman’s butt and think funny thoughts haha.
Jumpman Lane
May 3rd, 2010
@ The Big Gay Hulk
Sap!
hehehehehe
Deadlycodec
May 3rd, 2010
Mmmmm….mocha…
Sockpuppet Pointer
May 3rd, 2010
Anybody else besides me think that IntLibber and Judge Joker are the same person? Only Int has that level of paranoia. Stuff like “The sun rises every morning. People die every day. Therefore the sun makes people die.” Not exactly like that, but that sort of thing.
Kiddoh
May 3rd, 2010
I like how you pointed out your own logical fallacy, Sockpuppet Pointer.
IntLibber Brautigan
May 3rd, 2010
LOL, Sockpuppet Pointer, Judge Joker is a long time sandbox ghost that I met I think some time in 2008, we considered hiring him at BNT, but thats about all the association I’ve had with him. I think people that actually know me, and know him, would all chuckle at this sort of association.
You also outed yourself as KALOLZ. Only Kalel thinks IntLibber has that level of paranoia, and only Kalel cares enough about his spelling to actually capitalize the L. Give it up, your desperation is showing shiny.
Judge Joker
May 4th, 2010
Dear, anyone who thinks I’m a “KALOLZ” pseudonym.
I don’t have (USI) Unwarranted Self-Importance, so if you want me to out myself as you put it, you might be upset to realise it’s all ready out there on this site if you read the comments I posted.
So Lurk Moar!
Even googling Kalel Venkman will pull up results from my blog in some of the first results (UK version).
Other than that why not just ask me directly who I am? and how I can back up what I have said and believe? in stead of just having a quick lurk then posting your unwarranted Self-perspective.
IntLibber Brautigan
May 4th, 2010
Judge, he thinks you are me, not Kalel. He is evidently Kalel, getting ever more desperate.