Saltar al contenido ForoHackers

tired_sre_at_3am

Se unió el 6 enero 2025

Mensajes en el foro

  1. Respondió en Hello: freelancer, remote, and a very protected post-lunch hour

    Written runbooks for everything I do more than twice. Not for other people, for 3 a.m. me, who will not remember anything. It made on-call from home calmer, and it made my handovers short.

  2. Respondió en What 'test your restores' actually means: a checklist

    Step 6 is the one that matters at 3 a.m. I have been in a room where the backup was fine, the restore tool was fine, and the key was in a password manager that lived on the server being restored. Bla…

  3. Respondió en Four Windows Security event IDs I always want an alert on

    "A rule that fires 400 times a day is a rule nobody reads" should be on a poster. I would add: every alert needs an owner and a "what do I do now" line. Otherwise it is just anxiety delivered by emai…

  4. Respondió en Volatile first: the order of evidence on a suspect Linux box (in your own lab)

    From the on-call side: keep a running timeline in a plain file as you go, every command with a UTC timestamp. When someone asks "what did we do and when?" two days later, nobody remembers, and this f…

  5. Respondió en On-call from home: keeping the pager from eating the house

    Carl, that table is exactly the thing. Please also keep a column for "was this alert about a thing a customer would notice". If the answer is no, it is not a page.

  6. Inició On-call from home: keeping the pager from eating the house

    A topic that does not get much discussion: being on-call while working from home. The blurring is real, and I have learned a few habits the slow way. **A physical boundary.** If the laptop lives in …

  7. Respondió en "Tell me about a bug you caused": what a good answer looks like

    Part 3 is the one I listen for as the interviewer: do you reach for "roll back" before "find out why"? People who debug in production with the incident still burning are a different category from peo…

  8. Respondió en New here: Windows admin, mostly patching

    Rings for rollouts are the quiet hero of operations. Ten machines first, wait a day, then a hundred. The incident you do not have is never in anyone's report.

  9. Respondió en Trabajo remoto: escribir las cosas es la mitad del trabajo

    "Decisiones por escrito" is the thing I miss most when I switch teams. Every incident review I have attended that ended with "we already talked about this" is a missing note, not a missing memory.

  10. Respondió en The skeleton of a good postmortem, and the section everybody skips

    Carl, always with the restores. You are right, and I will add it to the template. The cat on the rack from a couple of years ago remains the best example of "what was lucky": zero downtime, no idea w…

  11. Inició The skeleton of a good postmortem, and the section everybody skips

    I have read a lot of postmortems and written some bad ones. The good ones share a skeleton, and it is not about blame, it is about reconstructing the story so the next person on call has a map. **1.…

  12. Respondió en You cannot average percentiles (a dashboard bug I keep seeing)

    Number 3 has saved me more nights than any other alerting change. "Ratio of slow requests over a window" is a count divided by a count, it merges across hosts, and it maps directly onto what a user w…

  13. Respondió en Hello from the pager side

    Agreed on both. The runbook that has only been executed at 3 a.m. by its author is a diary, not a runbook. I try to run each one once at a normal hour, with coffee, before it needs to work for real.

  14. Inició Hello from the pager side

    Site reliability engineer, mostly in the habit of being awake at the wrong hours. I work on alerting, capacity and post-incident reviews, and I care a lot about blameless write-ups: the system let a …