With the sluggishness of the site these last few days you can sometimes get slowness or infinitely loading icon when hitting submit. If you aren’t sure if your post submitted, refresh the original page in a different tab before hitting submit again to see if it went through. I see double and triple posts in almost every comment section.
Is there a way to load balance these lemmy instances across multiple backend servers?
Yes, and I think they are.
Backend sizes likely need to be larger and more abundant. Depending on hosting provider this could be mean lighting your wallet on fire every month till you either get enough donations to make up for it, or you give up and shut it down.
This problem will only get worse as Lemmy increases in popularity from “extremely niche, extremely low volume”/trivial volume (~50-100k users) to anything other than that (1mill - 100mill users or further). And gets more difficult since economies of scale may be difficult to apply here due to the hosting model of Lemmy, meaning cost/user is likely to stay quite high when normally it would be drastically reduced the more users you have.
I’m gonna guess individual instances are going to be forced to break down into instances that host only a couple communities each as time goes on. This would be the pragmatic choice since it’s very likely that individual communities can grow large enough to tank Lemmy instance performance for all others on that instance. Although that presents non-trivial UX concerns.
Though, again, many smaller scale instances means infrastructure costs may stay prohibitively high as a result of over provisioning or a lack of value from aggressive caching, read replicas…etc 🤔
Edit: I wonder if an instance can host individual communities on their own hardware. Or if there is some abstraction that lets users browse potentially hundreds/thousands of instances easily as if they where all communities?
I agree that having many smaller instances which serve more distinct communities is by far the most pragmatic choice. The idea of communities distinct from instances is nice on paper, but it gives instance managers the impression a single instance can serve a wide-range of interests without incurring proportional overhead on the resources.
Lemmy is the closest thing we have to the evolution of reddit. So far it’s just a matter of handling the backend which was nearly always done by a well-resourced organization instead of individual volunteers. I think instances will naturally become more abundant, and specific to their purposes, and users will be able to distribute their accounts accordingly.