Thanks. Should be back in a few hours hopefully.

A KVM is basically a second computer that acts as a virtual screen and keyboard that can be connected to the server to remote control it as if you were physically in front of it, including hardware resets and access to the bios etc.

I bought a KVM for that purpose, but didn’t get around installing it.

The other person able to do it manually will be available the same time as me more or less so I don’t want to bother them.

Yeah, I need to work on the busfactor.

The last time it went down was a similar situation and all the stuff I planned to prevent it next time I procastrinated on because the server was stable 🤦

No one knows 🤷

It is a work trip to the middle east. I am unlucky with work trips and server maintenance 😓

poVoq here. After a string of bad luck it looks like I will not be able to get it back up before Tuesday evening. Super annoying.

We managed to resolve the issue and I have many ideas how to avoid this in the future 🤓

Time to come back, it is working again. Lucky break 😅

Slrpnk.net is unexpectatly back online 🥳

I think we will try to come up with a theme for Piefed with similar colors.

The backups are most likely fine 😅

Sure, that seems reasonable. We will add that when the instance is back online.

If I forget, please remind me. For sure we will not just ignore you 😊

Regarding the price and all that, maybe that is an interesting discussion to have with the slrpnk.net community? And it would also be good to let slrpnk.net users know how the instance is being ran, what hardware, where, etc, so users can be an educated decision on whether they want to see an instance or not? Maybe providing that info on the sidebar would be nice?

We have been very transparent about that and documented that in the wiki which is linked in the sidebar. But it seems few people ever read the Wiki and having it integrated like it is with Piefed would probably help.

I agree that more admins with remote access would be good to have, and we have been working towards adding more (there are already more than one, who are not directly involved with slrpnk, but rather the mother project f-hub.org), but physical access will always be limited by the location (Azores) unless we move to an entirely different data-center (which in turn would make it hard for me to access the physical servers).

Yes, I also suspect that at a certain scale it would make a difference, but with a single Postgres database and no advanced clustering or so, the real-life performance metrics of a medium sized Lemmy instance strongly point to Postgres being the bottleneck and not the Python or Rust codebase of the rest of the software.

For regular users there is not a huge difference, but the web-interface is significantly faster and you can subscribe to topics that combine multiple communities. The disadvantage is less mobile app support. Right now only Interstellar supports Piefed.

The real advantage is for moderators and admins, as the Piefed developers actually listened to community feedback and implemented a lot of nice moderation features that require bots or annoying work-arounds on Lemmy.

And there are a few technical differences that make Piefed easier to administrate and troubleshoot from a sysadmin perspective.

I am genuinely confused. What makes you think you were ignored? We do have additional admins, the problem is physical access to the servers to replace hardware that seems to have broken.

We plan to add additional safeguards to such scenarios, but in the end it is a trade-off between reliability and costs. Running a Lemmy instance of our size on a fully managed rented server would easily triple the monthly costs. This would be beyond what I am willing to cover out of my own private pocket, thus making the instance dependent on donations. And that IMHO makes it more likely to be shut down in the future as donations are very variable.

I am not on vacations, but on a work deployment abroad, which further limits the time I can spend on this. Not that it would have made much of a difference, but at least the domain redirection and placeholder text was only possible to do on the weekend.

Sure, but they will not replace hardware and set up remote access for you, which seems like this would have been required in this case.

It is run from a private residence in the DIY punk spririt (and this also allows us to run of a local solar PV system), but more or less the same would happen if you rent rack-space in a “real” data-center. Only if you rent a managed server or VPS someone else will be responsible to fix such issue and this comes at a significantly higher cost at the scale we operate at (slrpnk is part of a bigger project that also hosts other services).

Sorry that it took so long to get the word out. I guess we relied a bit too much on the xmpp server for internal communication.