Where do you draw the line?

Anyone who works in a job that requires an on call status should have stoped and asked that question there at some point in their career. I have been asking it for the last few weeks myself.

There are a million list of x number of types of y people in the world and both of those are pretty arbatraty. I know www.cracked.com has plenty of those types of list and they can be very entertaining to read. But when it comes to work how many types of people are there really? From the bosses perspective there are only two, those that get the job done and those who don't. Over the years I have always worked to be in the first of those two, and never let myself be called the other.

But are those yours, mine, or anyone eleses only choices? Can I be the guy who bust my but at work, but still needs more time to finish a project? Or do I end up being the whiner who just makes excuses? Can I be the over worked Network Eng. who is doing the best he can filling the role of Server Admin? Or am I seen as the over paid guy who can't prioritize his task list?

I am a person who really hates to not get a job done. I know that at times it may not be perfect, it may not be 'right', but it will work and it will get the job done. I have an irrational need to figure things out, and I love new tech stuff to play with. I can't stand to spend money and then see new stuff sitting around not being used. I think it is this mental phobia that causes me to stay late at work and to work on my days off.

So when do you know it is time to say enough? When do you stop and say I am not going to work another 12 or 14 hour day? On decide I'm not coming in on my day off?

Now, I know that when a school is down I have to respond. I have to take care of our customers they are the reason I have a job and we have the budget to buy the stuff I want to play with. So I am not looking at those times as an issue or as times that should be limited.

I'm looking at the other issues like my new Wireless gear. Should I stay late to get everything configured and installed so that I can deploy the new APs. Well, it turns out there are a dozen other little changes that have to be made to make the new gear work. They are all little changes, and it really doesn't seem that they will take all that long. I think you know where this is going though. One change leads to another and that change ties into another project that if I plan it right can be taken care of with a single set of changes and a single outage window instead of two. The I am looking at building up so much more work that then seems to build on itself.

What about the other projects? SPAM filter, VMware, Active Directory, Web Filter, Business Plans, and I don't know how many other major projects! And as anyone knows, no matter what the vendor/sales weasle claims, the easy install/conversion is never easy, and it is never quick. Out of all the projects I have worked through I have never had one with out issues. If I ever do have a product that works 100% perfect with out any calls to tech support I may just have do it over just to make sure that it really was right.

Now, back to the problem at hand. When should I stop and put everything down and go home knowing that it doesn't have to be done tonight? I don't like leaving projects undone or partly done. I think it is mostly because the next day I spend way too much time getting back to where I was and what was done the night before.

The other problem is that when I look at the hours I put in and think about taking time off there is always a reason not too. There is always something that has to be done. How do you pick when to cut and run?

I know this is an odd post, and it rambles a bit, but it is something all IT folks need to consider.

Power outages suck even when you have everything you should to stay up.

Had a blast Monday. There was a transformer that went out and as the electric company rerouted power a second transformer blew up. I mean really blew up, sparks fire the whole deal.

Now, anytime there is a power outage IT is effected. Good news is here we have a room size UPS and a Generator. Bad news, when that transformer blew the surge was big enough that the 125A breaker on the UPS was tripped. Generator was running, but the power couldn’t get there. Now this really sucks as all of my servers and network gear is down. Good news, it took less than 5 minutes from the trip to getting power restored to the room. All total our customers lost around 10 minutes of network access while the 7606 booted up.

Now for the bad news, we lost two servers. Oddly enough it was two of the newest servers we have, HP DL380 G6 servers. I was amazed that some of our older gear didn’t die. Now HP is 99% of the time great to get repairs from, but Monday afternoon they sucked. First, we have 24/7/4 hour on all of our servers. The ticket for both servers was entered just before 4 PM, system boards showed up at 8:38 PM. Now, 30 minutes over is not bad, but there was no tech to install them. I called in and was talking to someone live by 8:45 PM, and the fist thing I am told is it will be 4 hours to get a tech on site to install the boards. I just about blew my top.

Now, I’ve been doing this for a long time and I know what you do and don’t do when this happens. First, you tell them this is a network down emergency, production outage, or some other combination of key words that help the script reader know that this is a real issue. Next you ask to be escalated to a supervisor. Don’t take no for an answer, but don’t be rude or cuss because that will give them an out at some companies to disconnect the call.

Now, back to the issue at hand, no tech. Well, I was at their mercy so I sat and waited. Guy showed up at 11:30 PM. Nice guy, but he didn’t seem to have very much experience with the HP servers. Once the boards were replaced one of the servers still wouldn’t boot. Tech called in to the tech support line, they wanted me to initialize the drives, I asked for the phone and told the script reader on the phone that we needed to skip ahead and look at the error message at boot. The Smart Array controller showed a config error, and that it was locked. Turned out we needed more parts needed after all. He went directly to the warehouse for me and brought the part back as he really didn’t want to sit and wait for it.

All told it was just after 3 AM when both servers were back up and running. At this point I thought it was time to go home and get some sleep, boy was I wrong. The gate was locked and I couldn’t find the remote to open it, I was trapped. Now, I could have pouted and whined, but instead I took the time to apply patches to several servers and perform some other house keeping that needed to be done. All told I managed to stay busy till 8 AM, then I went home happy that I didn’t need to worry about returning to work as I had already put in 8 hours for Tuesday.

Of course the calls and emails to the Blackberry managed to get me out of bed by noon. Oh well, that is what they pay me for and at least I have a job.