Modernizing curation: A proposal for The Workshop and The Archive

I commend you for the efforts to bring more questions to Stack Overflow, but let me highlight some problems.

Promoting questions to The Archive

We don't know which questions are worth preserving; we only know which ones are not worth preserving anymore.

All questions that can be answered should be publicly visible so that they get a chance at becoming valuable. Only after some time can we judge whether the question turned out to be a useful contribution or misleading. We can cast downvotes and upvotes from the time it is posted, but even heavily downvoted questions can result in helpful answers. And it is answers that we are optimising for, not questions. If a terrible question received good answers, we can rewrite the question ourselves to match the answers. But a good question will only become useful once it receives a good answer. It may take years before a good answer appears. Or maybe it has taken curators years to observe that a bad question attracted good answers and needs to be improved. Either way, it's only after the question has been publicly exposed for a long time that we can truly assess its usefulness.

Threaded replies are a necessary evil

Stack Overflow isn't the only place where I find answers after searching on Google. From time to time, I get results on discussion forums and I hate those. Why? Because they are designed for solving the immediate problem of the asker, not for presenting quick solutions for future researchers.

Threaded replies are good at getting to the solution, but they are an absolute time-waster when it comes to reading them. And selecting the best/accepted/working/preferred solution isn't making it a lot better. Because the solution took multiple messages to get to, it isn't contained in a single post, but rather the whole conversation thread is relevant. The reader must go through all the posts to fully understand how they arrived at the solution that worked and why.

Stack Overflow excels because it is designed such that the answers are written in a self-contained way and aimed at everyone with the same problem. You only need to read one post to understand what the proposed solution is. The next post contains another alternative self-contained solution. Posts are ordered by how useful everyone thinks they are, not by time or by what the asker thinks. It's quick and easy to find solutions.

We don't allow working on solutions in the Staging Ground because we don't want to lose valuable answers. If curators answered questions in the Staging Ground, a possibly good solution could get buried in a threaded response without getting posted as an actual answer. And what you are proposing is exactly that!

Threaded replies only work for temporary comments when trying to clarify the question and should be deleted once it is ready to receive answers.

The premise of this question strikes me as incoherent in several ways.

Before I outline my claims, I'll offer my counter-theory: engagement is a stupid goal. The goal should be that people should find what they want (and click on an ad or two in the process). If their question is already answered, there's no need for them to engage. Technology doesn't change that fast. The site has successfully built a body of knowledge that answers a vast number of questions, with no further engagement.

On to my specific arguments.

The question claims that the questions we are closing frequently contain the nuclei of good questions. I very much doubt this. Take away the duplicates, the homework, and the completely off-topic questions, and you have already 'dammed the flow to a trickle.'

The question presumes that the experiment has successfully diverted questions that would otherwise be closed, and thus the correlation of reduced closing is in fact causation. Reading the front page, I still see a torrent (to stick to the damp metaphors) of close-worthy sewage from people who are not getting the message. Heck, I continue to see first-time questions that need to be in staging ground just showing up. If closes are down, perhaps it's because more closers have gotten fed up and departed.

Finally: this proposal seems to me to essentially describe what I see happening in staging ground, where a discussion attempts to lead the questioner to state a coherent question, or solves their problem in the comments. Why not invest in that instead of launching on a hyper-complex alternative?

Postscript: To make my views clear: If you-all want to try to make some money by adding a conversational help structure in parallel with Q&A, I'm 100% neutral. As I see it, 99% of the challenge is going to be building the experience -- and the hardest 99% of the 99% is going to be building the expert community to provide the value and moderate the process. The existing process and community does not transfer / translate. The comments on this question are one barometer into the mood of the existing community, and it isn't encouraging.

The opinion experiment revealed that users aren't looking for subjective debates; rather, they most often seek detailed explanations.

The community told you this before you went through with the experiment, with a feature set, that made it impossible to submit our opinion or moderate the questions that were asked. You ignored our feedback.

To capture this value, we propose a radical shift: stop closing questions and introduce a new curation model—The Workshop for collaboration and The Archive for curation. Sit down, grab a drink, and start reading.

You are going to ignore our feedback. I am positive my feedback will be ignored, but this sounds like a horrible idea.

After running the opinion-based question experiment for just about two months, we know it's still raining upstream, and we have water sitting in a reservoir just behind the dam.

The experiment failed because you implemented a half-baked feature, without listening to the feedback you received, and essentially tried to relaunch the entire Stack Exchange Q&A model overnight. We told you not being able to issue downvotes was a bad idea. We indicated being unable to close the questions was a bad idea. We told you the entire concept of asking a question, seeking strangers' opinions on a matter, was a bad idea. You ignored all the feedback.

In short, yes. Though this should not surprise anyone, given that they can’t be closed and there is no real, defined set of rules. Anyone poking around on SEDE can see that question closures are continuing to decline.

You allowed users to ask questions that cannot be moderated by the community or the elected community moderators, proceeded to not moderate the content yourself, and then point out question closures decreased? Of course they decreased – you created a situation that would only result in the results you were expecting.

As of this writing, there are 1600+ opinion-based questions, 78% of which have at least one reply. In terms of engagement, this is a clear signal of demand. By our count about 85 replies were about telling the asker that their questions should have been asked as traditional Q&A (which is less than 2% of overall replies). This however isn't an easy thing to track, so there may be a few unaccounted for.

So is there an actual demand or did users not understand what they were doing, ended up asking a question seeking an opinion, when they actually wanted to ask a question seeking an actual solution? In my personal experience, every question I looked at, was improperly submitted seeking my opinion.

By damming these up for years, we have incidentally been blocking meaningful attempts by people trying to gain deeper understandings and have “why” conversations that help them grow. “Opinion-based” often just meant “nuance required.”

It isn't hard to ask a question seeking a deeper understanding of a topic, I have done it countless times, and I have answered hundreds upon thousands of those types of questions. It just takes effort on the individual asking the question.

Expert Endorsements (a working title), a unique vote reserved for some community members, would be a validation of the question’s future value from the established curators on Stack Overflow. It would be the opposite of a close flag. Getting one means you have created content that is the cream of the crop; it means your question can be found easily and that it should be surfaced to future searchers and learners, because it's good.

Quora has something similar, I don't use Quora, because the users with "expert endorsements" charge for their endorsements (i.e answers). I want no part in a community where this could become a reality. The predicted outcome of "free votes" being issued was seen, users voting for each other, and getting them suspended but the behavior is difficult to track so lots of users are getting away with that behavior.

We would probably want to move towards a threaded discussion format to support The Workshop.

So you want to turn Stack Exchange communities into Reddit subreddits? I have been a member of forums in the past, they never last; the only community I have ever seen last more than 5 years, is Stack Overflow and that's because it's NOT a place for discussions. If threaded discussions become a reality. I won't be part of the community. I will delete my profile.

I have purposefully been extremely harsh in my answer. I wasn't as harsh in my past feedback to the other failed horrible experiments, and those experiments still went live, with the predicated outcomes the community shared. So I have no doubt this new experiment will be implemented despite the community hating it.

Getting rid of traditional Q&A is going to alienate most of the community, and this new system takes much more buy-in from the very users you're alienating.

Everyone currently responsible for the curation of this site, from moderators to reviewers to regular users, has been doing so under the traditional Q&A format for decades now. Users ask questions, others answer them, and everyone else on the site contributes by voting, flagging, and editing.

Replacing the Q&A format that StackOverflow was built on with messier discussions that will need way more time and effort to curate, moderate, and then eventually elevate will drive away the curators that have been running your site all these years.

It's already clear that the company doesn't care what we think when running these side experiments but please, please stop trying to ruin the traditional Q&A format. If you kill it the site dies. Full stop.

I've picked and chosen some points I wanted to address here. The short of it, however, is that there's a lot of I disagree with here, much that I have concerns over, and little I agree with. Some of those thoughts of mine are below:

The opinion experiment revealed that users aren't looking for subjective debates; rather, they most often seek detailed explanations. To capture this value, we propose a radical shift: stop closing questions and introduce a new curation model

Yes, it has been known for a long time that users value detailed explanations. I've mentioned many times, including in a call with yourself, that the answers I find the most helpful are the ones where the question is well asked, so I can confirm it's the same issue I am encountering, and where the answer(s) explain their solutions, so I can learn from that solution and adapt it to my own needs. Not closing questions doesn't help this; questions are closed so that they can be improved before they receive answers. Answers to unclear or vague questions result in guessed or vague answers; those are not going to contain detailed explanations because they inherently can't or the answer is going to be too wide in scope.

After running the opinion-based question experiment for just about two months, we know it's still raining upstream, and we have water sitting in a reservoir just behind the dam.

You're not wrong here, thank you for confirming this, but actions speak louder than words.

Are closures for “opinion-based” reasons going down and are we seeing more “opinion” questions?

In short, yes. Though this should not surprise anyone, given that they can’t be closed and there is no real, defined set of rules. Anyone poking around on SEDE can see that question closures are continuing to decline.

I'll be blunt, but I'm not sure what you're trying to state here. Are you talking about the number of "traditional" based questions being closed as opinion based is going down, or that the Opinion-Based questions aren't getting closed? The latter cannot be closed, and the former should (hopefully) be posted as an opinion-based question, so of course the number is going down; you've forced it. This is, with respect, some what of a nonsense statistics; it tells you/us nothing.

Yes. Opinion-based questions are getting about three times more replies than regular questions (answers + comments). As of this writing, there are 1600+ opinion-based questions, 78% of which have at least one reply. ... By our count about 85 replies were about telling the asker that their questions should have been asked as traditional Q&A (which is less than 2% of overall replies). This however isn't an easy thing to track, so there may be a few unaccounted for.

As a mod, many of the responses I've seen have been to tell users they've posted things in the wrong place. With no curation on these types of questions, this makes it impossible to know if the question received responses because it was good or because people are trying to tell the user that they should have posted else where.

Not to mention that out of the 4 types (we're still missing "How To?"), 3 go to the opinion-based questions, so it's a little unsurprising to see more volume when 75% of paths take you to the new question type. It's not a level playing field. Without curation, or comments, I can't take the above at anything else but somewhat meaningless KPIs, as it wasn't a fair comparison.

Ironically, we found that the “Troubleshooting” type we introduced as an option during the experiment was almost entirely categorizable under our new implementation category: users trying to get bugs fixed.

Admittedly, I'm, not sure what else you would expect to see from a troubleshooting problem; the category strongly suggests that fixing bugs is what it is aimed at. Like mentioned, many times, the missing "How To" category would cover the majority of other "implementation" questions, but I've frequently seen those end up as "Advice" and "Best Practice" questions, which they simply aren't.

The Workshop would be the single entry point for all questions and utilize a threaded reply structure with improved notifications rather than the strict Q&A format. This would allow for messier collaboration.

So how do questions get "out" of the Workshop? If this is just the Staging Ground why not just use the Staging Ground? How does it differ? Threaded responses for answers are AWFUL. I hate when I end up Reddit, or similar sites, and I have to pick through 15 replies between an OP and answerer to build a complete answer from those comments. It's time consuming and I rarely get the result I want. Answers, as they exist now, give me a complete answer in one space; no back and forth between the user with additional statements like "But what if it's a Saturday and the wind is blowing in the other direction, while my cat is locked outside in the snow?"

Expert Endorsements (a working title), a unique vote reserved for some community members, would be a validation of the question’s future value from the established curators on Stack Overflow. It would be the opposite of a close flag.

Not to seem silly, but we already have an opposite to a close vote; the reopen vote.

Getting one means you have created content that is the cream of the crop; it means your question can be found easily and that it should be surfaced to future searchers and learners, because it's good.

That already exists in terms of votes; content with a higher amount of votes has been evaluated by the community (not a single "expert") as good. Why not trust all those people, rather than 1 person?

I can also see avenues for abuse of such a feature. Mods, would need tooling to identify that. How it could be abused is likely a better discussion for another time.

how do we visually protect The Archive when content is promoted from The Workshop? If a discussion page in The Workshop looks like a standard forum thread, we risk losing the prestige and discoverability of the best answer. We cannot let the harvest be buried by the chaff.

Simply put; the question should never have been answered in "The Workshop". If it was, it was answered in the wrong place, just like if someone answered a question in the staging ground in the comments; those comments are "lost" when it's migrated. Answering questions in a threaded discussion environment is not what people using Stack Overflow to find answers want to see, because it doesn't enable them to find the answer easily; it ends up spaced out of many comments, and loses a significant amount of value. With the threaded style, there's no drive for the author of the many "answers" to consolidate their first "answer" into a complete answer. What you have aren't answers any more, they're long-form comments.

1. How does a discussion in The Workshop become an artifact in The Archive? Is this a physical migration (moving the Question and its best Answer to a new page) or a visual transformation (signaling high quality within the existing thread)? What are the mechanics of promoting a conversational chain in The Workshop into a standalone, Endorsed Artifact in The Archive? For example, if a “messy” Workshop question receives a brilliant answer, does the question need to be pruned before it enters The Archive, or can the answer stand on its own?

As the answers shouldn't be in the Workshop, there's no need to migrate anything. The "Archive" (which isn't a good name), is where the answers are, not the discussion area. What defines the "best" answer as well, votes? That can easily be gamed; FGITW anyone? The answer with the most votes is not always the "best"; especially if one is older than the other and things have moved on. You're doing more harm by leaving those answers (that, again, should never have been in the "staging" environment) behind.

Who should have the power to ‘Endorse’ a question or answer? Should it be reserved for Gold Badge holders, elected moderators, or a new tier of subject matter experts?

The community; that's what we already do. As both a gold badge holder and an elected moderator, I don't want to be a sole arbitrator of what is "good" for a post. It ends up with clear conflicts of interest. For example, why would I say someone else's answer is better than mine? I, personally, would almost certainly be happy to, however, there are plenty of users out there that don't use their gold badge for "good", or at all, and will repeatedly answer the same duplicate over and over; those people could easily abuse their badge to promote their answer(s) over everyone else.

As mods, as we have stated to the community many times, we are not SMEs in all domains. Relying on a mod who in an SME in a subject to mark an answer as "good" is also not a task I want as a mod; it's not what I wanted to achieve when I put my nomination in.

On a discussion page in The Workshop, when a curator gives an Expert Endorsement, how should it look? If the rest of the page utilizes a threaded discussion format, does the Endorsed Answer get "pinned" to the top? Does it get a gold frame? Should the classic answer styling be reserved exclusively for Endorsed Content to signal its status as an artifact on The Archive? And crucially, how do we allow users to toggle between the “messy” collaborative view and the clean “curated” view?

Answers shouldn't be part of the discussion; they should be an answer. Leave the comments as threaded, and the answers as answers. If you want an answer endorsed, we have a similar feature already: bounties. Why does it need to look any different?

I thought about this some more, and I came to the conclusion that this would be a huge nail in the coffin for Stack Overflow. The whole proposal actually sounds very offensive to the community that built Stack Overflow.

You are basing the proposal on faulty premises. The experiment showed that there is more "water" because it was designed to brainwash us into thinking this. You tricked users into using the new feature even for questions that weren't opinion-based (which, to me, felt like it was most of them), you didn't provide required moderation tools, and you didn't allow the questions to be downvoted or closed. The whole experiment was designed to get you the results you wanted.

You treat question closure as something negative, which means the end of the question. And even if it is, it isn't meant to be that way. Closed questions are waiting for an edit from the asker! We want them to be reopened. If not enough questions get reopened, then do something to address this problem!

The fact that 50% of the newly posted questions get closed isn't our fault (at least I hope not). It's your fault! You're not trying to help new users succeed on the site. You're not providing them help in asking questions, in understanding the site, and in searching for existing duplicates. You made some good improvements, such as the staging ground, but you refuse to utilise it more. You tried to improve the asking wizard, but you ignored our feedback. No wonder so many new questions are unsuitable.

Stack Overflow is dying because you are killing it!

Here are some ideas you could try instead:

1. Push more questions into Staging Ground. Make it more discoverable, too. Maybe show an indication when questions in followed tags are pending in SG.
2. Implement some helpful QOL improvements to the question asking process. I don't mind if you use LLMs to scan the question and identify potential issues with it before it gets posted. You can tell the user when they need to remove noise, fix grammar, write a better title, etc.
3. Reduce the question ban time. We don't need to be so harsh anymore. We can reduce it to a week or a month ban. Questions from such users should definitely land in Staging Ground.
4. Make reopening more frictionless. Perhaps notify close voters when the question they closed gets reopen votes. Highlight questions with reopen votes in the question list. Bump the question when it receives a reopen vote.
5. For moderators, implement tools that would enable us to investigate closures en masse. We have no easy way to find which questions were closed for which reason. It's really difficult to investigate invalid close votes.

The Workshop would be the single entry point for all questions and utilize a threaded reply structure with improved notifications rather than the strict Q&A format. This would allow for messier collaboration. Incomplete or unclear questions could be discussed back and forth without penalty. We want to keep the conversation open while the water soaks in. The purpose is to see what value these posts might hold for the future.

I seriously don't see any value in this. If you want useful artifacts it needs to be clear what is a question and what are answers. Everything else are comments.

With such threaded responses and going back and forth in separate replies, you will not be able to get any useful long term artifact. Nobody will go and extract that into something useful and all participants who contributed something meaningful would have to do that.

If you want people to be able to ask poor questions and have them answered, the format is not the problem. The closing process is what prevents people from writing answers and sometimes such questions are resolved through comments. Not much different from the current process. Starting with a mess will never yield good results. On top of that you will seriously drive away the remaining experts who are still hanging around and answering here.

If you are worried about amount of asked questions then maybe you can give users with question ban more opportunity to ask a new question, maybe you can allow them to ask one question an month instead of every six months. If that goes well, you can maybe even reduce that further. But whatever you do, don't change the Q/A format.

Also the quality of responses will go down, because it is a different thing when you are writing a proper answer and different when you are just chatting with someone.

If there was anything to learn from opinion based questions it was that the format was wrong. Replies don't work. It looks like you have some activity but most of it were just comments which would commonly be deleted as no longer needed on the Q/A.

Please, don't do this. You will just waste some more time on such endeavors and you will get nowhere. If you want to broaden the scope of questions and allow some recommendations and opinion based topics, then please make those in the same format as regular Q/A. If it works for sites like Software Recommendations it will work here, too. And you will be able eventually sort out existing useful Q/A pairs which were not on topic in regular Q/A.

Incomplete or unclear questions could be discussed back and forth without penalty.

As somebody who primarily focuses on answering questions, this sounds unfun to me.

The original differentiator SO had was that its users tend to ask fairly high-quality questions. This in turn made answering questions interesting: you don't have to burn time hand-holding users who need help articulating their problem or wade through lower-effort and uninteresting-to-answer questions.

I've seen this dynamic play out first-hand in other online forums/subreddits -- there's no element of curation, so answerers tend to eventually burn out and disengage due to the repetition.

So while I do think there's merit to making the question-asking experience smoother via staging ground/workshop/etc, I also think it'll be important to maintain a strong element of curation or filtration to help answerers have a positive experience.

The revelation came when we analyzed the question users submitted using the “Opinion-based” option provided during the experiment. These were not just random solicitations for comments or opinions, which was a concern for both the company and the community, but a majority appeared to be questions asking for explanations.

In contrast, I do like the idea of making a push to encourage "explanation/why" type questions.

Part of the reason why I stopped participating on StackOverflow a few years ago was because as I became more experienced, I eventually got bored answering the more narrow "how do I do X" or "implementation" type questions that are common on SO. Having questions that require broader/more holistic expertise or require navigating non-trivial tradeoffs does sound much more interesting to me.

That said, I'm not sure you need this whole workshop/archive concept to support such questions -- frankly, it feels like overkill. (Or perhaps more charitably, orthogonal).

Instead, it seems to me you could accomplish a similar result more simply and quickly by:

1. Tweaking the Stackoverflow tour, "How to ask", and other such pages make it clear "explanation" type questions are allowed and encouraged.
2. Tweaking staging ground to provide templates for such questions.
3. Setting up new tags and such to make it easier for interested answerers to find such questions.
4. Working with meta to develop best practices and guidelines on how to answer and moderate such questions. For example, I imagine it would be a good idea to still expect the question-asker to do some initial research, perhaps to try defining tradeoffs/the design-space, etc... And conversely, the moderation policies should be updated to distinguish "explanation" questions from subjective/opinion-based ones.

As noted, these questions are not really subjective. So StackOverflow's pre-existing Q&A framework should still do a decent job of accommodating these questions. The most comprehensive/thoughtful/accurate answers will be voted to the top; users will accept the explanation that clicked best with them.

If you legalise drugs, fewer people will be reported for drug use, fewer arrests will be made, reported drug use will go up, and demand will seem to increase.

You relaxed the rules and got increased engagement.

If engagement is your primary metric... Well, good job.
I can think of a few more changes to increase (short-term) engagement.

You're talking about the volume of questions we close as if it's a problem. To me, it's a result of a focus on quality. If you loosen that focus, of course you're going to close fewer questions.

One line that really drew my attention:

The median reputation for users asking these opinion questions was 10x higher than those asking troubleshooting questions (113 vs 11), and their accounts were significantly older (median age of ~ 7 years vs ~ 2.5 years).

113 rep is not significant. Especially not on accounts that have had about 7 years to collect as much. That means that the account received less than two upvotes per year.

To call these users "experienced developers (who) are hungry for these conversations" seems disingenuous. Experience doesn't come with years; it comes with action.

To me, these users don't even qualify as active members of the site.

Ironically, we found that the “Troubleshooting” type we introduced as an option during the experiment was almost entirely categorizable under our new implementation category: users trying to get bugs fixed.

I'd separate "my code that is supposed to do X is broken, how do I fix it?" from "I want to do X- how do I do it?". The latter form is usually more useful to future readers.

In the current curation system, curators close questions as opinion-based because they don’t fit our strict “one right answer” format. In reality, a lot of these are questions of conceptual understanding, based in facts, reasoned through collaboratively, and grounded in truth. By damming these up for years, we have incidentally been blocking meaningful attempts by people trying to gain deeper understandings and have “why” conversations that help them grow. “Opinion-based” often just meant “nuance required.”

There's a definition of "constructive subjective" questions that is blessed for regular Q&A:

Constructive subjective questions:

• inspire answers that explain “why” and “how”
• tend to have long, not short, answers
• have a constructive, fair, and impartial tone
• invite sharing experiences over opinions
• insist that opinion be backed up with facts and references
• are more than just mindless social fun

I've raised this a couple times and don't feel like this is getting acknowledged in the announcements about this experiment. That makes me feel confused and frustrated.

The proposal: The Workshop & The Archive

At a high level, I'm in favour of this,
(assuming it's not implying that closure doesn't exist in the "Archive" space).

I've been thinking about the idea of intentionally putting a "library/knowledgebase" space and a "helpdesk" space side by side. I know that "helpdesk" is a bit of a trigger-word to us, but I understand that to be because of the frustration and challenges that come from trying to achieve the former when people approach it only as the latter.

I think having two spaces side by side- one geared towards interactive help (like a helpdesk), and one geared towards being a library where information can be searched and referenced (long-term, reusable value)- can be an arrangement where the spaces complement each other: a helpdesk can give a concrete picture of what issues people are having today (which could motivate adding new items to the library, for future reuse), and a library/knowledgebase (ideally) provides material that can be referenced in a helpdesk, so helpers there don't have to repeat themselves at length about particular building blocks of knowledge.

I also think there's value in allowing people to choose the experience that they want. If someone wants to seek help and doesn't want to do the work of creating a post that meets Archive standards, they can choose an experience where that won't be expected of them. If someone wants to help those people and don't necessarily care to contribute to an Archive (and there are people like this), they can choose that experience. If someone just wants to contribute to the Archive, they can choose that experience. If someone wants to contribute to both, and to bridge between them, they can choose that experience.

I'd expect this to result in less tension between these different groups.

Also, speaking for myself personally, it would be a huge relief to me to be able to choose (on a case-by-case basis) an experience where I get review of a question post I draft from people with more expertise from me. There are so many domains I don't have expertise in, and there are often questions I want to ask, but have no idea what details to include, or how even to pose the question well, because I have little to no idea how things in that domain work (Ex. Linux / Ubuntu things- especially when it comes to layers which separate concerns). In those cases, I really want to have a space that is built for me to get help writing up a good question post. Maybe that can just be meta, but it would seem awkward to me to see that as the happy path for this. I wish Staging Ground existed on other sites, and I'm glad to see that you're still thinking along these lines, or leaning into those ideas.

I'm not a fan of the word "archive" though. It carries a connotation of a place where things aren't going to be referenced often... which is the whole point of a library or knowledgebase. I'd call it a library or knowledgebase.

I have some specific thoughts on mechanics for a helpdesk space, such as how to rate limit things, no web indexing (it's the library's goal to be searchable), and automatic item deletion after a year or so.

The Workshop would be the single entry point for all questions

There aren't privilege barriers today to posting Q&A, and I don't see a reason to add them. If one wants to get review of a new Archive entry they'd like to propose, then sure, they can use the Workspace for that, but they shouldn't be forced to.

If one person thinks an Archive Q&A isn't useful, they can just downvote. It it's out of site scope, they can just close-vote.

The Workshop would be intended to catch almost every appropriately on-topic, answerable question on Stack Overflow.

So... how does the "opinion-based content" experiment play into the definition of "on-topic and answerable"? What are the standards for subjective Q&A in the Workshop and the Archive? I'd be pleased to hear if/that the guidelines for "constructive subjective Q&A" will stay in place for the Archive.

Expert Endorsements (a working title), a unique vote reserved for some community members, would be a validation of the question’s future value from the established curators on Stack Overflow. It would be the opposite of a close flag. Getting one means you have created content that is the cream of the crop; it means your question can be found easily and that it should be surfaced to future searchers and learners, because it's good.

Instead of voting to close, you’d vote to elevate. The system wouldn’t be built to hide or remove; it would be optimized to highlight what's worth looking at. Curators would not need to classify and categorize every single incoming question, only pull out the most useful posts and cultivate them for the harvest. [...]

2. The curator: Who should have the power to ‘Endorse’ a question or answer? Should it be reserved for Gold Badge holders, elected moderators, or a new tier of subject matter experts?

I feel like this is overcomplicating process / adding an unnecessary barrier. If someone (anyone) sees a pearl in the sand, and they think they can bring it out, I think they should just do it, and that they shouldn't have to have a special role granted to them to be able to. They can (/should just be able to) just write up Q&A for the Archive and post (and then ideally add a link to the new Archive item(s) in the motivating Workspace item's comments).

This presents a critical challenge: how do we visually protect The Archive when content is promoted from The Workshop? If a discussion page in The Workshop looks like a standard forum thread, we risk losing the prestige and discoverability of the best answer.

1. The mechanics: How does a discussion in The Workshop become an artifact in The Archive?

As I proposed above, just let the act of creating Q&A in the Archive work as it does with posting Q&A today. Writing good Q&A is work, and I say let that work be rewarded to whoever does it. If someone is really concerned that some solution they provide in the Workspace will be written up by someone else in an answer post to a new Archive Q&A (and that someone else will get the rep/credit for it), then... they can just be the person who writes up that new Archive Q&A, and use it to help in the Workspace, instead of just participating in the Workspace.

This tells us that experienced developers are hungry for these conversations.

No

Or at least not necessarily. My interpretation is that there's simply a correlation between:

• People who carefully select a fitting category in the "ask question" dialog
• People who earn more reputation on Stackoverflow

Or vice versa: if you're too lazy to select anything but the default in the UI, you're less likely to have high reputation.

In general, I like that you want to ease the conflict between curators and askers and that you're bringing in new ideas.

However, my personal advice would be start slowly. Make small steps. Implement a few medium sized features that are met with enthusiasm by askers and curators. Seeing your recent track record, this current vision is too heavy, it may easily break your back.

I;d love if we made duplicates fun or at least enjoyable. People love to quote themselves, so here's what I said a year ago:

Finding a duplicate for someone -> should feel like getting your answer accepted. You found the correct solution.
Having your question flagged/closed as duplicate -> should feel like getting a perfect instant answer. It most often is.

How about making another web-site to try such revolutionary water ideas?

what it means to run out of water.

On the other side, in russian language "too much water" (in the book) phrase is understood as "too much of fluff/filler" (too wordy). This is exactly what we do not want StackOverflow to be: instead of a concise Q&A with only information to the point you are proposing to allow all kind of stuff to stay.

The statistic:

Opinion-based questions are getting about three times more replies than regular questions

tells us, that if you mark certain topic as a place without the rules, then the users are more kin to post there "something" (not necessarily a good content).

I personally don't visit advice-topics, who am I to advice? But I'll gladly answer a specific question if I know the answer and there is none yet given. I am doing it not just because I can or for reputation (after certain point reputation doesn't matter anymore), but because I think about future visitors. Few times I got a quick help myself from my own posts!

I was amazed how good StackOverflow is (was?) at giving a concise answer to a specific problem. You know web-site which has lots of answers, but I never visit it directly? It's reddit. Don't become reddit please, there is already one.

With the rise of AI the users are going to visit web-sites (all web-sites) less often. The "googling" era is obviously at decline. Will attempts to "just get some users" and to "show more ads" as of recent make a significant difference to income is a question. But the new content will very clearly suffer. Your choice.

I really like that this proposal seems to come more early in the development process compared to other such changes lately and it seems to explain more than the average initiative. On the other hand I really don't like that the solution seems to be to put so much extra burden on curators, demolish the Q&A model over Reddit style threaded replies and in general doesn't give that much emphasis on content quality.

It seems to me to be something like: hey let's be like Reddit and additionally we can also be like Wikipedia in a two level system.

But I believe that:

• Reddit like threaded replies are not an efficient way for gathering knowledge
• Distillation of the knowledge from such conversations is a non-trivial, extremely time-consuming act
• While it's maybe optimal in the sense of (no downvotes, no close votes, I get my half-baked question answered and improved and elevated for free) no friction experience for the asker, it's a far worse experience for the curators and answerers. It would quickly overload the experts.

This would only make sense if the shortage is really only on the side of question askers and not also on the side of answerers or curators.

Why can't the elevation vote not simply be traditional upvotes? Why can't the workshop not simply be the staging ground for everyone (possibly extended)?

You basically ask us to do everything twice: like please give everyone a quick answer to whatever they write, and then polish it and answer it again, but even better. Be a relentless live debugger/help desk and a book writer. But who actually has time for that? I may be wrong, but I think the one cardinal problem is simply that we don't have the amount of free labor needed to make this paradise a reality. (AI has, we haven't. That's simply how it is.)

I like Shog9's wisdom from the past, but I don't like his analogies. Analogies always leave important parts of reality out and can easily be changed into anything else. Just replace water with garbage and the picture might become more clear.

I personally would have done this instead: I would have decoupled questions from answers and focused on finding and linking existing content to new questions (maybe without much voting). My premise would be that most new questions are duplicates, some aren't and curation/expert time is limited while being friendly to everyone doesn't cost a thing. And I would have also concentrated on improving existing content. But this path you explicitly do not seem to follow.

It seems that the answer to AI chatbots seems to be that you can write anything and human experts will give you tons of immediate help. I'm really not sure we can beat AI at its own game, especially not without high quality content.

We will surely learn from this big experiment if Reddit like threaded replies are useful for knowledge generation. Unfortunately that is your weakest point. You don't measure quality/knowledge generation. You only measure engagement. This engagement might just be meaningless for the purpose of the library creation. It's absolutely unclear and my impression was that people are not getting help with the opinion questions (and there wasn't a single positive example in the Q&A asking for positive or negative examples). That means if, if you try that (and I think you will), you will have no chance of seeing if it goes wrong. You go into it blind.

In the end we might just only collect a pile of garbage (questions didn't get closed for fun in the past). But it's worthwhile to find out. New usage of SO is so low now, one can as well take more extreme measures and bet the whole house, there isn't much to lose.

Also, the step size of your experimental updates is much too large. Instead of for example "just" abolish close votes and downvotes you also want to change the UI and the roles of people. The probability that this altogether is positive is very small and would require you to rely on luck to get everything right. If you fail you will not know what was the cause. Believe it or not, but you learn more from doing smaller steps at a time.

P.S.: I slept over it and now I think that the vision is basically a human super chatbot knowledge machine. A chatbot that can answer every question without any problems or reservations, extracts knowledge on the go and is powered by an endless supply of free human labor. Problem is that it still competes with AI chatbots, they might still be faster and cheaper and this endeavor might be futile. Additionally it's not clear if the knowledge base really needs the chatbot output to distil knowledge. Instead it might be more efficient to completely ignore that and start generating knowledge by other means, whatever they are.

I might be missing something, but I do not see what problem this is meant to solve that cannot already be solved through less extreme measures. Anyone can already endorse useful questions by upvoting them, we do not really need a separate "Archive" for them. The "Workshop" already exists in the form of closed questions. They can be submitted for review and reopened when fit. The workflow could be made friendlier, sure. But a separate zone for all new questions as I understand this just seems like unnecessary complexity and friction.

Incomplete or unclear questions could be discussed back and forth without penalty. We want to keep the conversation open while the water soaks in. The purpose is to see what value these posts might hold for the future.

I do see the value in revising incomplete/unclear questions without penalty. The current workflow is kind of needlessly hostile. For example, a closed question is pretty much a dead end unless you are lucky enough to have enough people cast reopen votes, which is hard for low-visibility questions. And getting a question reviewed after the 9-day deletion window is very hard without moderator assistance.

As such, questions should only enter a "Workshop"-like state once closed, then they can be revised without penalty until reopened. Revise the workflow for only questions deemed as needing improvement, not all questions.

In the current curation system, curators close questions as opinion-based because they don’t fit our strict "one right answer" format. In reality, a lot of these are questions of conceptual understanding, based in facts, reasoned through collaboratively, and grounded in truth. By damming these up for years, we have incidentally been blocking meaningful attempts by people trying to gain deeper understandings and have "why" conversations that help them grow. "Opinion-based" often just meant "nuance required."

Then we should revisit the wording and usage of "Opinion-based." This reason used to be known as "Primarily opinion-based," and before that, "Not a real question" or "Not constructive" were used to roughly mean too opinion-based. The help center does not suggest all opinion-based (subjective) questions are off-topic so long as they are constructive:

Some subjective questions are allowed, but "subjective" does not mean "anything goes". All subjective questions are expected to be constructive. What does that mean? Constructive subjective questions: inspire answers that explain "why" and "how" [...]

The "Opinion-based" close reason could be revised to exclude constructive questions, even if they are subjective. In some ways I feel that the old close reasons might have done a better job at this.

But we do not need to "remove closing" just to address these issues.

What we know for sure from 15 years of SO moderation is this:

• Up/down-voting as a means of moderation does not work.

  Up-votes tend to indicate that a post has been around a lot, that it frequently pops up in links/search results, or the it is about "curious oddity" topics. Not necessarily that it is good. Up-votes means that most "canonical" posts get stuck with a lot of crappy answers that we can neither improve nor get rid of.

  There is no "library of knowledge" here. There's rather "oh there's some good books in shelf Z, cramped in between the cheap detective stories, the sensational tabloid newspapers, the very old and incorrect books and various unrelated fiction. Read them all and you might figure out which are the good ones".

  Down-votes create a whole lot of needless drama and conflict. Public shaming as a means to achieve site moderation does kind of work, but basing the whole site on it wasn't a good idea. Because no positive reaction comes out of it, no will to learn and improve. Humans respond poorly to critique given in public no matter how valid it is. As I keep saying, the very basics of management is to give praise loudly in public, but to give critique discreetly in private. Not understanding such very basics of human behavior is probably what you get when tech nerds with lots of technical knowledge and no HR experience build a site. Not a lot of empathy or awareness of how humans behave.

• Reputation as means to indicate moderator suitability does not work.

  There is zero correlation between technical domain expertise and moderator suitability. Everyone knows this and everyone keeps pointing it out, yet the system is still there. We have lots of high rep users who are clearly not suitable for moderation, where there's users who are both suitable and willing to do more user moderation tasks, but are held back since they lack privileges. (Personally I have some 200k+ rep and 39 out of 40 of the "moderator criteria" used for mod elections, but I don't think I would make a good moderator at all.)

  Due to decreased site activity, reputation will get even harder to come by, making the model even more flawed.

• Having users with zero interest/suitability for moderation doing moderation does not work.

  These kind of users just want low quality content gone. They don't care how, they don't care about other users, they don't care about the long term, they just "down vote, close vote and move on". We even encourage that. But this builds a bad culture over time, where there is a constant site scope creep driven by the most pedantic user moderators, never by the most lax ones. Overall on SE, site scopes tend to get increasingly narrow, but rarely ever wider. Eventually you end up with sites that a very large part of all content gets closed and the few self-absorbed, pedantic users that remain even think that's a good thing.

To keep pretending that the above flawed systems aren't flawed is to keep losing users. There is no easy fix to any of it - as much as everyone keeps trying, we are still stuck with the bad site culture and it is very hard to change. Bad as in lots of strong believers in the above flawed systems. (Just watch as this answer is no doubt getting down-voted into meta oblivion.)

I have no idea how to fix the site and judging by the previous 20+ bumbling experiments that have been carried out in attempts to fix it, no one else here does either. Creating a new, better site and start fresh is probably the only sensible thing to do.

I'd be willing to prototype this on Generative AI Stack Exchange: I was already planning to experiment with this kind of thing on that site. It's a good test-bed, because the ideas you've described fit the kinds of questions that get asked there.

I don't think this is something you should prototype on Stack Overflow, because the ideas you've described don't fit the kinds of questions that get asked here.

Overall thoughts: Ambitious but maybe promising. The experiment's 16% doesn't impress me without a much more nuanced breakdown. But for the Workshop/Archive: the community puts so many hours of work into trying to curate, so I'm happy to hear about anything which might improve the process. A couple concerns:

• how much work this might be to bring to fruition...
• encouraging a lot of separate Workshop discussions might fragment the knowledge

No link-only answers

Except for links to our Q&A/Archive/SE network, right?

#1/#3: Definitely need a highly readable Q&A page for the Archive. Stack Overflow has always been decent at this, with room for improvement. The recent/upcoming changes (comment UI, experimental questions, native ads) have made the pages worse—less readable—not better.

I think my preference would be an Archive which is a separate page, with most of the Workshop chit chat gone. If we have useful answers for people, let's let people come and get them. They could then optionally click to view the Workshop page, where they might learn more nuance or history of the discussion.

I don't like the idea of there being two different places, because there can be only one default list of results on the question page, on the home page, on any page, and whatever isn't the default will be ignored. We've seen this with collectives, with discussions, with articles, with chat, if it isn't at the place people go to to find content, they aren't going to interact with it.

I'm also 100% against creating a separate interface to support these "opinion based" discussions that we all know aren't all opinion based, because it leads to confusion. From people not understanding how to use a given interface, to people having the wrong expectations when they use it, or not knowing which one is the correct one to use, no amount of ai generated "guidance" on the right side is going to solve this. We need one, clean, interface that serves the needs of the community with clear expectations of what we expect from askers. We have Q&A, it has worked well for us all these years, I see no reason to go from it to discussions. It is more than capable of serving users looking for more of a discussion with the new(ish) threaded comments, and if a user feels they have an authoritative answer to provide they can. It's effectively your "workshop" vs "archive" idea built into one step. The only thing holding back this interface from serving users the way this site was intended to operate is the tools we have for curation.

The friction we have at the moment that causes all of this toxicity around asking and answering is due to the tools we have for indicating the quality/usefulness of questions/answers, how we deal with questions that need some additional information to be adequate potential additions to the knowledgebase, and how those tools have resulted in community created guidelines that go against the natural way people get and receive help (such as having a discussion in comments to flesh out what the user is looking for!) Q&A itself isn't the problem here, Q&A isn't inherently a solution that can't support the questions that aren't quite ready for a complete answer yet. We simply don't allow that collaboration to occur in a way that is useful and fair to the person looking for help.

What I propose as an alternative solution is to change the effects that close votes have on questions. If a question "Needs details or clarity," the closer who is voting for that reason gets to choose a sub reason (controlled by the community mods for each community) to indicate what details or clarity is missing. Is it missing a code sample? A screenshot of something? an expected outcome? Make it clear to everyone who comes by what it is this question is missing so that it can be fixed. However, the second part of this change is this vote should no longer prevent the question from being answered. If the question gets an adequate answer and people find the answer useful, mission accomplished. Stop getting stuck on this "but it didn't have an MRE!" madness if someone had the expertise to solve the problem without it. The goal should be leading the asker to an answer, not finding the right close reason bin to dump it into. If the question is just the umpteenth question about how to print text on the page with java it's not going to interfere with actual useful content. We don't need to get in the way of them getting help, tune the low quality deletion system to dispose of the question due to it's lack of value. The elevation of useful content should happen organically, rather than being controlled by curators.

As far as downvotes go... I don't see a good solution to them. I don't want to see them go, but i don't see this network ever gaining back it's popularity with the way voting works today. It's not about how much rep you get from votes... it's about the negative feeling of knowing someone "didn't like" what you created. I don't think we should get rid of voting or force voting to provide a reason or any of that, rather, maybe upvotes and downvotes shouldn't be combined into a visible score and shouldn't be so prominently displayed. Maybe downvotes shouldn't cause a rep loss or cause an immediate risk of receiving a 6 month "question ban" that most users never climb out of. Maybe I can be convinced we simply don't need them on questions once the new close system more or less covers all the reason's we'd cast a downvote on a question.

Put another way, I'd rather we build this, "knowledgebase," we are trying to build/maintain here such that the users who come here looking for help can use it as the helpdesk they expect it to be without it getting in the way of what we are trying to accomplish. Ask Questions, Get Answers, No Distractions. The useful content can ideally be identified by how often people visit it and hopefully upvote it, rather than putting it on curators to sift through the mess and pick/choose what goes in. Unfortunately, I don't know how feasible this is due to ai summaries and solutions like AI Assist that don't give any incentive to visit and pay into the feedback loop. Most importantly, however, we can't continue to just push everyone away and keep people from helping each other over whether or not a given question fits in the knowledgebase and expect to continue to have a relevant knowledgebase for years to come. Something has to give.

Do it the other way around. Send closed questions to a Reddit-like workshop, where they can die without polluting the main site

Okay, another answer, this time, even though this might not be something that I would take part in personally, I try to make it happen. Here is your new business model:

It seems clear to me that you want to become like Reddit only with a bit higher quality. Fair enough. If that was indeed the goal, how would one do it?

• Askers can ask anything, repliers reply to that in any possible way, voters vote on it - all like in Reddit
• Do not ask people to dig through the mud and search for diamonds. You won't find enough people anyway. The only thing that can do that without burning out is AI. Use AI to write meaningful excerpts of question and answer for questions with a score above a threshold. Train the AI to take votes into account. How you do that exactly? It's your business secret. If you do it well, you'll create value. If not, then not. That's your big task. The new core of the business is knowing how to automatically extract knowledge from a water/mud/garbage mixture.
• Publish these excerpts to the second tier after some time. Make them editable there for trusted users. Add links between the original discussions and the edited summaries.
• Very important: every question gets only a single summary which summarizes the question and the answer, even if there are five different solutions outlined in five different replies. The expectation would be that all sufficiently upvoted solutions are included in the summary in the order of their votes.
• Visitors can them consume the summaries, which hopefully have a quality high enough to compete with LLM results (i.e. they may not be tailored but they explain more and are right more often).
• Optionally add a duplicate finder system, where related questions / summaries are linked as early as possible, without limiting the ability to comment on anything endlessly.
• State that you want to become a high-quality Reddit alternative.
• No AI content creation by users, only by you, only in the one intermediate summarizing step.

Simplified user roles:

• askers ask questions
• answerers reply to questions
• all visitors vote on content (up and down)
• AI sumarizes vote weighted content
• experts (distinguished answerers) edit AI created summaries, maybe if they want to can even bypass the AI part, but let's see if this route is frequented
• visitors read summaries or ask their own question
• no close, deletion only of spam, instead only votes and visitor counts

Anything else, especially when it puts too much burden on curators, stay away from it. More AI, also stay away from it. One crucial AI application that you must do well and one only. You don't want to fail. Make it simple (For example are question types really needed?).

And would I like it personally? Not that it matters because you surely can find a different audience elsewhere. I really like that it's more oriented towards improving content, more collaborative. And I like summarizing multiple answers. This concentrates knowledge much more. But I don't like conversations. My time is too precious to take part in them. Still, if empirically it's shown that this is the only way to get humans to reveal their problems .. what use is a system that nobody uses. I definitely don't believe in that all questions have been asked already, so making a system where people ask them can't be too bad. But I myself would not volunteer for it probably.

The alternative is that everyone uses only chatbots in the future and amount of publicly available knowledge will shrink. Maybe generating new knowledge will actually become harder not easier.