Brutkey

Stefano Marinelli
@stefano@mastodon.bsd.cafe

Me in 2012: "Hey, it’s only 23:30! I can still work on some servers and updates!"
Me in 2026: "Hey, it’s 21! I can finally go to bed and rest my eyes!"

#TimeFlies #GettingOld

Stefano Marinelli
@stefano@mastodon.bsd.cafe

I spent the morning debugging two different weird issues tied to Windows and macOS clients.

I'm looking at those two bare-metal servers' credentials, longing to be able to install FreeBSD and the two new setups.

Monday, Monday...

#RunBSD #FreeBSD #MondayMood

Stefano Marinelli
@stefano@mastodon.bsd.cafe

Positive side of winter: I can open the window without using a mosquito net, and no mosquitoes invade the room.

Negative side of winter: after five minutes with the window open, I have to close it again if I want to avoid the early stages of freezing.

#Winter

Stefano Marinelli
@stefano@mastodon.bsd.cafe

Me and my first phone.
Around 45 years ago.

This photo has been on my grandma's living room cabinet for as long as I can remember. She passed away less than three years ago, and to keep her memory alive, my wife suggested we keep it in our living room.

A blast from the past looking towards the future.

#ThrowBackThursday #Photography #Photo #Me #JustMe #BackInTime

Stefano Marinelli
@stefano@mastodon.bsd.cafe

A few days ago, a client’s data center "vanished" overnight. My monitoring showed that all devices were unreachable. Not even the ISP routers responded, so I assumed a sudden connectivity drop. The strange part? Not even via 4G.

I then suspected a power failure, but the UPS should have sent an alert.

The office was closed for the holidays, but I contacted the IT manager anyway. He was home sick with a serious family issue, but he got moving.

To make a long story short: the company deals in gold and precious metals. They have an underground bunker with two-meter thick walls. They were targeted by a professional gang. They used a tactic seen in similar hits: they identify the main power line, tamper with it at night, and send a massive voltage spike through it.

The goal is to fry all alarm and surveillance systems. Even if battery-backed, they rarely survive a surge like that. Thieves count on the fact that during holidays, owners are away and fried systems can't send alerts. Monitoring companies often have reduced staff and might not notice the "silence" immediately.

That is exactly what happened here. But there is a "but": they didn't account for my Uptime Kuma instance monitoring their MikroTik router, installed just weeks ago. Since it is an external check, it flagged the lack of response from all IPs without needing an internal alert to be triggered from the inside.

The team rushed to the site and found the mess. Luckily, they found an emergency electrical crew to bypass the damage and restore the cameras and alarms. They swapped the fried server UPS with a spare and everything came back up.

The police warned that the chances of the crew returning the next night to "finish" the job were high, though seeing the systems back online would likely make them move on. They also warned that thieves sometimes break in just to destroy servers to wipe any video evidence.

Nothing happened in the end. But in the meantime, I had to sync all their data off-site (thankfully they have dual 1Gbps FTTH), set up an emergency cluster, and ensure everything was redundant.

Never rely only on internal monitoring. Never.

#IT #SysAdmin #HorrorStories #ITHorrorStories #Monitoring

Stefano Marinelli
@stefano@mastodon.bsd.cafe

I can say, however, that I've reunited with an "old friend" who is still in great shape. Before I get dozens of replies about it being insecure: in theory, it is, but it's not accessible from the outside. It's isolated within its own network and is only able to connect and fetch backups for 'extreme disaster recovery' (only at specific times, restricted by specific firewall rules). It served its purpose today, and tomorrow, it might be even more useful.

18:33:01 up 3188 days, 4:47, 1 user, load average: 5.09, 4.73, 4.74

Debian, Btrfs, and zero internal dust.
It's kept in a sterile, extremely protected room.

#IT #SysAdmin #Debian #Linux #Uptime #ServerUptime

Stefano Marinelli
@stefano@mastodon.bsd.cafe

A few days ago, I made a post announcing that I'd opened a Monero wallet for BSD Cafe. I don't know much about crypto, but someone actually sent some Monero!

I used it to get a one-month VPN subscription, which is helping me test the geolocated resolutions for media.bsd.cafe and experiment with various optimizations.

#BSDCafe

Stefano Marinelli
@stefano@mastodon.bsd.cafe

I’ve been losing my mind for almost 4 hours, and I feel like an idiot.

At first I couldn't understand why the TLS handshake was always taking more than 300 milliseconds. I thought it was a local or server-side issue. Only after about an hour did I test google.com and saw the exact same behavior.

That's when I realized that with my other WAN connection, the timing drops significantly.

So I started going crazy over my MikroTik configuration, convinced it had to be something related to multi-WAN. I even briefly asked an AI (I know, I know...), which said the problem was probably my neighbor, who had eaten too much pizza.

At that point, I kept spiraling.

Then it hit me that the Vodafone Station has its built-in Wi-Fi disabled, since I manage the network behind it. I could enable it and bypass the MikroTik. I enabled it and ran a direct test.

Bingo. Same problem.

So the issue is upstream. I suspect it’s "Vodafone Rete Sicura", some awful thing I never wanted and that probably does some kind of traffic inspection.

I really can't wait for FTTH to arrive so I can finally get rid of this stuff.

#IT #SysAdmin #Networking

Stefano Marinelli
@stefano@mastodon.bsd.cafe

While I don't want to switch to a Mastodon fork and prefer to stick to the official releases, I really miss the ability to post in Markdown.

#Mastodon

Stefano Marinelli
@stefano@mastodon.bsd.cafe

Happy New Year, #FediMeteo friends!

New year, new features! Let’s go step by step:

* Some results for city coordinates were not fully accurate, and forecasts were referring to the center of the administrative area rather than the actual city center. This also affected my own city, with a shift of about 30 km. I have revised the coordinate calculation process and, starting from the next update, the reference point will be the center of the target city. I actually discovered this while implementing the last item on this list...

* In addition to wind speed, FediMeteo will now include an arrow showing wind direction. The direction will also be reported as a value in degrees. This feature has been requested several times in the past, and I think the time has come to introduce it.

* New data added: AQI, the Air Quality Index. Using different indices for each local context would have been complicated, and reporting them all would have made the output unnecessarily complex. For this reason, I chose to use the American AQI for all cities, with a visual indicator to quickly understand the situation at a glance. This is consistent with major weather apps and allows for proper comparisons between different cities. PM2.5 and PM10 values are also included.

I hope these new features will be useful and… let’s keep moving forward!

#FediMeteoUpdates #FediMeteoAnnouncements