Strap in friends, because this one is a wild ride.
I had stepped into the role of team lead of our IS dept with zero training on our HP mainframe system (early 90s).
The previous team lead wasn’t very well liked and was basically punted out unceremoniously.
While I was still getting up to speed, we had an upgrade on the schedule to have three new hard drives added to the system.
These were SCSI drives back then and required a bunch of pre-wiring and configuration before they could be used. Our contact engineer came out the day before installation to do all that work in preparation of coming back the next morning to get the drives online and integrated into the system.
Back at that time, drives came installed on little metal sleds that fit into the bays.
The CE came back the next day, shut down the system, did the final installations and powered back up. … Nothing.
Two of the drives would mount but one wouldn’t. Did some checking on wiring and tried again. Still nothing. Pull the drive sleds out and just reseat them in different positions on the bus. Now the one drive that originally didn’t mount did and the other two didn’t. What the hell… Check the configs again, reboot again and, success. Everything finally came up as planned.
We had configured the new drives to be a part of the main system volume, so data began migrating to the new devices right away. Because there was so much trouble getting things working, the CE hung around just to make sure everything stayed up and running.
About an hour later, the system came crashing down hard. The CE says, “Do you smell something burning?” Never a good phrase.
We pull the new drives out and then completely apart. One drive, the first one that wouldn’t mount, had been installed on the sled a bit too low. Low enough for metal to metal contact, which shorted out the SCSI bus, bringing the system to its knees.
Fixed that little problem, plug everything back in and … nothing. The drives all mounted fine, but access to the data was completely fucked,
Whatever… Just scratch the drives and reload from backup, you say.
That would work…if there were backups. Come to find out that the previous lead hadn’t been making backups in about six months and no one knew. I was still so green at the time that I wasn’t even aware how backups on this machine worked, let alone make any.
So we have no working system, no good data and no backups. Time to hop a train to Mexico.
We take the three new drives out of the system and reboot, crossing all fingers that we might get lucky. The OS actually booted, but that was it. The data was hopelessly gone.
The CE then started working the phone, calling every next-level support contact he had. After a few hours of pulling drives, changing settings, whimpering, plugging in drives, asking various deities for favors, we couldn’t do any more.
The final possibility was to plug everything back in and let the support team dial in via the emergency 2400 baud support modem.
For the next 18 hours or so, HP support engineers used debug tools to access the data on the new drives and basically recreate it on the original drives.
Once they finished, they asked to make a set of backup tapes. This backup took about 12 hours to run. (Three times longer than normal as I found out later.)
Then we had to scratch the drives and do a reload. This was almost the scariest part because up until that time, there was still blind hope. Wiping the drives meant that we were about to lose everything.
We scratched the drives, reloaded from the backup and then rebooted.
Success! Absolute fucking success. The engineers had restored the data perfectly. We could even find the record that happened to be in mid-write when the system went down. Tears were shed and backs were slapped. We then declared the entire HP support team to be literal gods.
40+ hours were spent in total fixing this problem and much beer was consumed afterwards.
I spent another five years in that position and we never had another serious incident. And you can be damn sure we had a rock solid backup rotation.
(Well, there actually was another problem involving a nightly backup and an inconveniently placed, and accidentally pressed, E-stop button, but that story isn’t nearly as exciting.)
No kidding. Where I’m working now, it takes an HP CE over a week just to bring out a new hot swappable drive after we jump through a number of request hoops.
I rocked a Samsung Alias 2 for 4 years before I got an iPhone 5. The e-ink keyboard was awesome how it changed when you flipped the screen open to portrait or landscape.
Would love to know what you guys think and your reasoning for it. I’m starting to see a lot of Apple Watches/Fitbits/Galaxy Watches in my area compared to more traditional timepieces.
I wore a standard wristwatch all the time until about 25 years ago. I began noticing that I was conscious of feeling something on my wrist and I couldn’t stand it.
I then carried a nice pocket-watch around for a while until I got my first cell phone.
This right here is the reason I call for permanent DST.
I’m at latitude 42N and having less daylight time in the evenings during the warm months would be awful.
I was with a group years ago where one guy did this at a Pizza Hut. He put in about $10 and played Happy Birthday repeatedly. But it wasn’t just some normal version of the song. Instead it was some crazy, jazzed up version with multiple singers, firecrackers, etc.
Just incredibly obnoxious.
After about the 8th play through, the manager unplugged the jukebox. The guy who put in the money started an argument with the manager about how now he’s lost his money. After some back and forth, the manager gave up and refunded him $5 … but also made the mistake of plugging the jukebox back in.
Well you can guess what happened next.
We got loudly kicked out after that.
Two weeks?? Longest vacation I’ve been on in the past 20+ years was eight days, with the eighth day being unintentional due to an issue with the airlines.
Having your own router/access point can’t be stressed enough.
And, you don’t even need their modem. Sure it’s an additional outlay of cash, but buying your own modem gets you a nice upgrade and no worries about someone connecting to the Xfinity access point that’s bundled in their equipment.
No, and I don’t miss it at all.
I’ve used BT headphones even when my last two devices had jacks and wouldn’t want to go back.
Prior to that, I can’t count the number of times I would snag the wires on something or get them tangled in some way.
Now I use my BT bone conduction headphones everywhere.
It could be in large red flashing neon and people will still ignore instructions like this.
Even if the trash dumpster is a mere 50 feet away.
People just don’t care. Once it’s out of their hands, no matter where it ends up, it’s someone else’s problem.
One complex where I used to live, our trash dumpster was a full on compactor, very nicely concealed behind a small hill. Signs leading up would say, “No cardboard!” Signs plastered all over the machine, “No cardboard!” Yet, at least once, if not twice a week, the thing would be clogged because people would toss in boxes by the truckload. Not even broken down.
Meanwhile, just around the corner, a specific dumpster meant for cardboard only that would take all of 10 more seconds to get to.
Wow, that is done really well. How long did it take to finally notice it?
My favorite camouflaged cache was a hollowed out tree limb that had been attached to a metal stake which was placed in a forest around other sticks and leaves. I only found it because I accidently kicked it.
Ugh, I once made the mistake of not checking what I was ordering at a bar.
Sour pumpkin beer. I know that many people like sours, and I like some pumpkins, but together, I’m out.
I think of all the times I did that working on my cars years ago.
It was just something you did and no one ever even blinked. Old oil, gas, brake fluid, etc, right down the storm drain.
The federal judge presiding over Donald Trump’s criminal election interference case shut down a bid by the former president to set his trial more than two-and-a-half years away....
I just finished this book the other day.
I found it about 400 pages too long, but the aspect of the humans being at least on par with the jellies was refreshing.
Former President Donald Trump was indicted for an unprecedented third time on August 1, adding another set of serious federal charges to the mounting legal issues he faces.
I’ve put together a collage of some books from last months What are you Reading? post. It’s mostly random, but the more discussion something gets the more it stands out to me. Going forward I’m going to make a new post every month to talk about what people are reading....
For instance I know i can navigate to lemmy.ml/c/[email protected] to see the “daystrominstitute” community at startrek.website, but is there a way where I can see all (or even just the most popular) communites at startrek.website, while still logged into my lemmy.ml instance (so i can just hit the subscribe...
Yes, you could just go to startrek.website/communities for the list of communities on that particular instance.
However, subscribing to a community on an instance where you do not have an account takes a few extra steps.
When looking at the list of communities on another instance where you do not have an account, clicking the Subscribe link from the list will result in a popup stating that you’re not logged in and if you click on the community name, a Subscribe button will not be shown.
However, most, if not all, communities will have instructions in the sidebar for how to subscribe from your main instance without the need for creating an account on the instance where that community resides.
To do this using the example of the startrek.website instance, if you wish to subscribe to the daystrominstitute community, you would enter “!daystrominstitute” into the search field on your main Lemmy instance where you are logged in.
Generally the first item in the results will be a link back to that community, which will look like:
“Daystrom [email protected] - 448 subscribers”
Click that link and you will see the content of that community again, but now notice that a Subscribe button will be shown in the sidebar.
Clicking Subscribe may result in a note that says “Subscribe pending”, which I believe is due to some syncing that needs to occur between instances. Just give it a few minutes and then that community will appear in your list of subscribed communities.
You can now read, post and reply to that community just as if you had an account on that instance.
Water flowing under rule (lemmy.world)
Lemmy, what are some of your "oh shit" work stories?
What was the last dumb phone you had before your first smartphone? (lemmy.world)
I rocked a Samsung Alias 2 for 4 years before I got an iPhone 5. The e-ink keyboard was awesome how it changed when you flipped the screen open to portrait or landscape.
Do you prefer to wear a smartwatch or a regular watch?
Would love to know what you guys think and your reasoning for it. I’m starting to see a lot of Apple Watches/Fitbits/Galaxy Watches in my area compared to more traditional timepieces.
What's the best question to ask someone who you don't know very well?
First night at his new forever home (lemmy.world)
Three years old and the most gentle, affectionate little one I’ve ever met. Love him like mad already. Still haven’t got a name, though
Map: The states in favor of daylight saving time all year long (www.nbcnews.com)
In the past six years, 19 states have made efforts to move to year-round daylight saving time. So what’s in the way?
Judge Tells Ivanka She Can Probably Afford a Babysitter (newrepublic.com)
Ivanka Trump lost her appeal and is officially set to testify in the fraud trial against the Trump Organization.
An Emotional Fish - Grey Matter [1990] (www.youtube.com)
They Eat Their Own - Like A Drug [1991] [Explicit] (youtu.be)
Detroit Breakdown (Live) (www.youtube.com)
Milk (pawb.social)
Philly thief steals a shop-vac filled with hundreds of giant hornets (www.inquirer.com)
Look at them going about their lives, unaware of what's to come. (startrek.website)
We've clearly created a utopia. (startrek.website)
They Live - 1988 - Starring Roddy Piper, Keith David and Meg Foster. Written and directed by John Carpenter. (lemmy.world)
cross-posted from: kbin.social/m/13thFloor/t/457507...
Sounds like a plan (sopuli.xyz)
Google DRM when? (lemmy.ca)
Also, fuck xfinity and their BS. Can’t even change wifi password without downloading an app.
Journey - Of a Lifetime (1975) (www.youtube.com)
[POLL] Do you use/require a headphone jack on your phone? (strawpoll.com)
Alternatively, if your current phone doesn’t have a headphone jack, do you wish it did?
All of Japan's Toyota Assembly Plants Shut Down for a Day Because Their Server Ran Out of Disk Space (www.reuters.com)
Read the dumpster (i.imgur.com)
Oh no… (sh.itjust.works)
Found this very subtle fake bolt cache today (sh.itjust.works)
https://sh.itjust.works/pictrs/image/2be32aaf-9e6a-42f9-8dcc-9a86217220b3.webp...
I'd saw off my leg for my grocery store to start carrying something besides shitty IPA's and Budweiser (lemmy.ml)
I wouldn't have survived in their time (startrek.website)
Reject modernity. Embrace tradition. (feddit.cl)
edit: Don’t do this. Embrace modernity and don’t pollute the soil.
Judge rejects Trump bid to push federal election trial to 2026 (www.cnbc.com)
The federal judge presiding over Donald Trump’s criminal election interference case shut down a bid by the former president to set his trial more than two-and-a-half years away....
Any sci-fi with aliens where humans are not the less advanced race?
What the title says, I’m tired of the trope where humans are the least advanced in the universe....
Why are barns in the United States often red? (i.imgur.com)
Currently developing an Android app to sync subscriptions and saved posts between accounts (sh.itjust.works)
It’s open source and I’d like some feedbacks, as you can see I such at designing mobile apps....
Trump was just indicted for trying to steal the 2020 election (www.vox.com)
Former President Donald Trump was indicted for an unprecedented third time on August 1, adding another set of serious federal charges to the mounting legal issues he faces.
What are you Reading? (August 2023) (lemmy.world)
I’ve put together a collage of some books from last months What are you Reading? post. It’s mostly random, but the more discussion something gets the more it stands out to me. Going forward I’m going to make a new post every month to talk about what people are reading....
Who the hell cuts their pizza into fifths? (lemmy.world)
Brought to you by Digiorno
Opinions: What is a movie you genuinely like, that is rated below 60% on rotten tomatoes? (xkcd.com)
Inspired by the linked XKCD. Using 60% instead of 50% because that’s an easy filter to apply on rottentomatoes....
Can I view a list of the communities of a lemmy instance while logged into my own?
For instance I know i can navigate to lemmy.ml/c/[email protected] to see the “daystrominstitute” community at startrek.website, but is there a way where I can see all (or even just the most popular) communites at startrek.website, while still logged into my lemmy.ml instance (so i can just hit the subscribe...
We went to the county fair yesterday- a pig peed on my leg.
This mildly infuriated me and I felt the need to share my mild infuriation....