Locus of Points @locus@schelling.pt
- ADMIN
- YES
- CONVERGE
- THIS
- AKA
- pee_zombie
Admin
CONVERGE, THIS
administrator of schelling.pt
Joined Jan 2021
apologies for our brief outage! it was caused by an attempted DNS updated which resulted in a misconfiguration; I'll be reviewing our practices to ensure this doesn't happen again.
we're mostly recovered, still a few small issues here and there but we can expect normal operation from here on out. still working on it, lmk if you run into any issues.
update: seems that masto is unhappy with migrating storage providers w/o copying over all files, so, doing so now. expect this to take hours, but in the meantime, images are gradually beginning to load again. apologies for the inconvenience, please bear with these growing pains as I mature our operations.
quick update, still investigating, there's something weird going on with the nginx media proxy. switched from a Wasabi backend to S3 to simplify the setup and eliminate some variables. will keep y'all posted on developments.
⚠️ ADMIN POST ⚠️
we're back up, but experiencing an issue loading media; as a result, no avatars or images will show for now. I am actively investigating!
⚠️ ADMIN POST ⚠️
schelling.pt will be going down for maintenance in 15 mins at 12:45 PM EST for at most 15minutes, until no later than 1:00 PM EST
during this time, I will be upgrading to a larger server, doubling RAM from 4gb to 8gb, as we've been cutting it close for a while.
apologies for the inconvenience, working to make the best experience for all y'all
feel free to contanct @pee_zombie on Twitter during the downtime if you need anything
⚠️ ADMIN POST ⚠️
⚠️ ADMIN POST ⚠️
you may have noticed we are having short intermittent outages; we're experiencing minor scaling issues. if these keep happening, I will scale the server up ASAP to remediate. please LMK if you experience any more!
Locus of Points
boosted
welcome new schelling point convergers! glad to see so many new "faces", hope you enjoy your stay.
i'm your local admin, when speaking in "admin voice" I use @locus, but feel free to direct any questions you have at either handle.
i'll be putting up some official rules and more guidelines soon, please feel free to provide input or feedback towards this end. i want this instance to be a collaborative project!
more coming soon, stay tuned! and make sure you're following @locus
⚠️ ADMIN POST ⚠️
apologies for the downtime! seems I'd missed the domain expiration emails 🤦♂️ I've now renewed the domain for a few years, should be working again. lmk if there are any lingering issues!
i've identified the issue as being caused by an overwhelmed DB and have upsized it slightly, should hold us off for a while until i get better monitoring in place. apologies for the downtime!
⚠️ ADMIN POST ⚠️
am aware there's a performance issue currently, am looking into it. thank u for ur patience
Locus of Points
boosted
If you are new to Mastodon coming from the Twitter, here are a few amazing Mastodon features that are NOT ON TWITTER. Enjoy! 🐘
1. Content Warnings CW ⚠️
It allows you to display a warning but hides the content of your post. Here’s a short guide I wrote about it: https://infosec.exchange/@Em0nM4stodon/109282181601490676
2. Slow Mode 😌
In Preferences > Appearance, you can select the option “Slow mode” to hide automatic feed updates. You will then have to click at the top of the timelines to see new posts (ex: “10 new items”).
3. Custom Emojis 
When you write a post (called a toot here), you can select emojis that are custom to your server (called an instance here). In the toot field, click on the 😃 button labelled "Insert emoji" at the bottom (this will vary on mobile apps) to see what’s available there. Some are really neat! 
4. Mastodon is different 
There is no suggestion algorithms here, no promotion, no ads. This is a community. You can be more of your true complex and unique self. Engage with others genuinely. Follow others you like no matter their followers count. Describe yourself and what interests you in your bio, you are not as limited by characters count here.
Be yourself. Leave the toxicity out, it is not valued here. Respect others. Be a kind human. Share what you love. Make friends 
I'm still thinking about longer term sustainability, and how that might mean that to justify the time allocation might make sense to make this a paid instance (nothing crazy, with tiers ofc) but I recognize it's pretty hard to convince people to pay for a previously free service
any thoughts?
anyway so I've fixed all these and have upgraded to latest masto (v3.5.3), which took about as much effort as I expected (had to upgrade one step at a time to account for migrations)
the server is still very much a pet, in the sense that it's manually groomed and cared for, nothing is version controlled nor managed thru IaC. this pains me greatly
hopefully this time around I'll make it more of a priority to manage it better
- the server was self-DDoSing bc the streaming API component was bootlooping as the node binary somehow became owned by root and hence inaccessible to the service user, so requests would fallback to the much less efficient REST api
- the version of masto i was on has a known issue where it is currently impossible to install with ANY version of node, bc some very specific dep with a binary module depends on a specific version of something which is no longer available online
if anyone is curious as to what was wrong, i've found and fixed the following issues:
- main nginx route had misconfigured TLS to use the wrong cert path, so even tho certbot was renewing, the new cert wasnt getting used
- media nginx route's TLS cert was configured properly, but was expired bc the cronjob broke
!! ADMIN POST !!
apologies for the long maintenance hiatus
I have resolved the main issues and have upgraded to latest Mastodon version
should be good for a bit; please reach out with any concerns
ADMIN POST:
apologies for the outage, we should be back now! these should be minimized soon as I'm working on setting up monitoring/alerting
!! ADMIN POST !!
seems like we haven't had images load for a bit over a week; apologies for taking so long to address this. the cause was that the SSL cert for the object storage domain (files schelling.pt) had expired, as I had forgotten to renew it at the same time as the primary cert for the root domain. I've resolved the issue & will try to find time to automate the cert renewal so that we don't face this issue again
major thanks to @WomanCorn and @TetraspaceGrouping for pointing this out!
!! ADMIN POST !!
once again, apologies for the downtime! I am investigating the root cause, and setting up monitoring/alerting will be my highest priority this weekend, in order to minimize the risk of this recurring. thank you for your patience and for bearing with our growing pains!
- ADMIN
- YES
- CONVERGE
- THIS
- AKA
- pee_zombie
Admin
CONVERGE, THIS
administrator of schelling.pt
Joined Jan 2021
