Well we did it - There once was a girl named...
The Grrl / In Brief / Her Pictures / Her Roller Derby league September 2010
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
 
 
Wed, Nov. 23rd, 2005 11:12 am
Well we did it



My first day working for Livejournal was almost 3 years ago, and it started with setting up our 3rd cabinet in the Seattle facility with Brad. Next week Matthew and a guy will be removing all the leftover equipment still sitting there, and by December 1st it'll be emtpy.

(as a side note, I met Brad 5 years ago when he moved the couple of LJ servers in to the closet "noc" at Speakeasy)

When LJ was acquired in January last year I knew eventually we'd have to move the data center. Managing multiple centers in such distinct ways isn't efficient, and we weren't able to convince the Six Apart ops team to move their machines up to Seattle. It isn't often that we physically have to be there, but I personally have installed about 1/2 the equipment in that data center so maybe I've got emotional attachment. Yes, to computers.

But to be totally honest, I really really didn't want to have to manage moving 130 pieces of equipment to another state.

We stalled as much as we could, but finally the time to move came and over the last 3 weeks we've succeeded. All of Livejournal.com is now hosted out of a data center in San Francisco (365 Main).

I couldn't give enough praise and thanks to everyone on the LJ team, the Six Apart ops team, and the most uber respect to matthew, brad, crucially and xb95. I showed up Thursday in San Fran without a lot of sleep and worked straight through until last night when we finally got to have some sushi in celebration. Mark, Artur and Brad were in the office with me almost the whole time, each of us grabbing short breaks for sleep where we could. Matthew rocked managing the team of movers for the physical move, hopping planes and joining us by phone when he could.

And thanks to anyone who has had to come in contact with me in dealing with my crabbiness and one track mind.

The LJ team really do seem to have a unique way of being able to work together without killing each other. Its not all fun and games, but we get things done and when there are problems everyone is smart and dedicated enough to solve them. I'm thankful Management also seem to recognize this and allowed us to work the way we needed to get this done.

Personally I'm beating myself up about all the things that could have gone better, been better planned, how I want to fix things going forward. But I think in general, from a users perspective, the bumpiness over the last 5 days was manageable. Correct me if I'm wrong!

There is a fair amount to clean up, but for the first time in a month I should be able to go to sleep tonight without fighting fits of anxiety. And in my own bed, too!

In personal news, my brother Mark is getting married on Saturday. I'm flying to Phoenix Friday morning and coming back Tuesday. I haven't seen the family since New Years, so I'm really looking forward to it. I'm meeting my nephew Ethan for the first time. I've been putting my personal life on hold for the last month, and I can't wait to get back to "normal."

Current Mood: undefined
Current Music: matthew sweet - sick of myself

18CommentReplyShare

beatnikside
beatnikside
beatnik sidearm
Wed, Nov. 23rd, 2005 07:51 pm (UTC)

Many thanks for bustin' ass on behalf of us slackers. I've had a paid account since 1999 and it's been worth every penny. And congrats to your brother!


ReplyThread
ljkrissy
ljkrissy
Krissy
Wed, Nov. 23rd, 2005 07:52 pm (UTC)

Congratulations! Have y'all been rewarded with any time off yet??


ReplyThread
lisa
lisa
Lisa Phillips
Wed, Nov. 23rd, 2005 07:58 pm (UTC)

There's still too much work to take time off, but I think we can take a day for Thanksgiving and a longer break is to come in the next couple of months...


ReplyThread Parent
endquote
endquote
Josh Santangelo
Wed, Nov. 23rd, 2005 07:54 pm (UTC)

Congrats on getting 'er done. From a user-side the blips weren't that noticable. I'm glad you'll be able to chill out a bit for a while now.


ReplyThread
und1sk0
und1sk0
chris
Wed, Nov. 23rd, 2005 08:18 pm (UTC)

damn, you were in town? :( i would have bought you several beers for helping us out the other week, but i'm in the midwest for thanksgiving.

chris@revolt.com


ReplyThread
onah
onah
Ana
Wed, Nov. 23rd, 2005 08:33 pm (UTC)

though I certainly would have done it better... ;) I can't imagine how you all pulled this off so seamlessly. way to go.


ReplyThread
llarian
llarian
The cat that walked by himself
Wed, Nov. 23rd, 2005 08:35 pm (UTC)

As somebody who actually knows the kind of effort required to do a move like that as seamlessly as you guys did, I'm in awe. There were glitches and certainly things that got missed, but the sheer size of the undertaking is amazing.

Congrats on getting that done as well as you guys did. Livejournal is known in the tech community for having some amusingly unorthodox solutions to problems, but the results speak for themselves.


ReplyThread
vespa59
vespa59
Right you are, Ken
Wed, Nov. 23rd, 2005 09:01 pm (UTC)

Awesome job, homie. We barely noticed a thing!


ReplyThread
ainetl
ainetl
ann tracy
Wed, Nov. 23rd, 2005 09:10 pm (UTC)

thanks for all your work lisa. i can't imagine my life with LJ. have a great trip to Phx and have fun at the wedding.


ReplyThread
jetgirl23
jetgirl23
It Thing Hard-On #23
Wed, Nov. 23rd, 2005 09:15 pm (UTC)

You so rock my world. :) Can't wait till you come back.


ReplyThread
jamies
jamies
Jamie
Wed, Nov. 23rd, 2005 09:32 pm (UTC)

congrats. and I didn't notice a damn thing... nice job.


ReplyThread
rahaeli
rahaeli
Hello Kitty, Destroyer of Worlds
Wed, Nov. 23rd, 2005 09:37 pm (UTC)

What I think is most amazing is that we managed to do an 8 hour conference call, on no sleep whatsoever, and nobody wound up screaming at each other for more than about three seconds of sheer exhausted snapping.

It's kind of cheesy, but I think that's pretty much a sign of how to do teamwork.


ReplyThread
moody_warthog
moody_warthog
Ilya_P
Wed, Nov. 23rd, 2005 09:54 pm (UTC)

Thanks for everything done for us from Rrrrussia. LJ means a lot for us.
Appreciate your&team work. Hope you'll get through the rest of it finding less and less bumps on the road.


ReplyThread
coffeechica
coffeechica
Carrie: she tampered in God's domain
Wed, Nov. 23rd, 2005 10:21 pm (UTC)

You are my hero.


ReplyThread
chris
chris
mechanical found ghost
Wed, Nov. 23rd, 2005 10:46 pm (UTC)

the blogosphere is in your debt


ReplyThread
dannster
dannster
dannster
Thu, Nov. 24th, 2005 01:31 am (UTC)

Didn't notice anything.
Well, I'm not on Cartman anymore and that makes me sad.... but other than that...all Aces!


ReplyThread
jamesd
James
Thu, Nov. 24th, 2005 02:22 am (UTC)

Congratulations on the successful move! Not perfect but surviving is success and all of you managed to do that without too much pain for the end users.


ReplyThread
kamakhai
kamakhai
so extravagant my cataract
Tue, Dec. 6th, 2005 05:27 am (UTC)

just wanted to introduce myself. i've enjoyed your comments on lauren's journal for some time.

also, i wanted to express my disappointment that you've apparently decided not to move yourself along with the servers to SF. we need roller girls too, you know.


ReplyThread