What we know for sure from 15 years of SO moderation is this:
Up/down-voting as a means of moderation does not work.
Up-votes tend to indicate that a post has been around a lot, that it frequently pops up in links/search results, or the it is about "curious oddity" topics. Not necessarily that it is good. Up-votes means that most "canonical" posts get stuck with a lot of crappy answers that we can neither improve nor get rid of.
There is no "library of knowledge" here. There's rather "oh there's some good books in shelf Z, cramped in between the cheap detective stories, the sensational tabloid newspapers, the very old and incorrect books and various unrelated fiction. Read them all and you might figure out which are the good ones".
Down-votes create a whole lot of needless drama and conflict. Public shaming as a means to achieve site moderation does kind of work, but basing the whole site on it wasn't a good idea. Because no positive reaction comes out of it, no will to learn and improve. Humans respond poorly to critique given in public no matter how valid it is. As I keep saying, the very basics of management is to give praise loudly in public, but to give critique discreetly in private. Not understanding such very basics of human behavior is probably what you get when tech nerds with lots of technical knowledge and no HR experience build a site. Not a lot of empathy or awareness of how humans behave.
Reputation as means to indicate moderator suitability does not work.
There is zero correlation between technical domain expertise and moderator suitability. Everyone knows this and everyone keeps pointing it out, yet the system is still there. We have lots of high rep users who are clearly not suitable for moderation, where there's users who are both suitable and willing to do more user moderation tasks, but are held back since they lack privileges. (Personally I have some 200k+ rep and 39 out of 40 of the "moderator criteria" used for mod elections, but I don't think I would make a good moderator at all.)
Due to decreased site activity, reputation will get even harder to come by, making the model even more flawed.
Having users with zero interest/suitability for moderation doing moderation does not work.
These kind of users just want low quality content gone. They don't care how, they don't care about other users, they don't care about the long term, they just "down vote, close vote and move on". We even encourage that. But this builds a bad culture over time, where there is a constant site scoop creep driven by the most pedantic user moderators, never by the most lax ones. Overall on SE, site scoops tend to get increasingly narrow, but rarely ever wider. Eventually you end up with sites that a very large part of all content gets closed and the few self-absorbed, pedantic users that remain even think that's a good thing.
To keep pretending that the above flawed systems aren't flawed is to keep losing users. There is no easy fix to any of it - as much as everyone keeps trying, we are still stuck with the bad site culture and it is very hard to change. Bad as in lots of strong believers in the above flawed systems. (Just watch as this answer is no doubt getting down-voted into meta oblivion.)
I have no idea how to fix the site and judging by the previous 20+ bumbling experiments that have been carried out in attempts to fix it, no one else here does either. Creating a new, better site and start fresh is probably the only sensible thing to do.