undefined | Better HN

0 pointsshagie2y ago0 comments

The difficulty that SO has had trouble with is the "then the site suffers." How do you measure that? They want to run some A/B test that allows the corresponding measurement to show that things are better with a change.

However, it feels that the only way that they've really accepted measuring it from a sales / marketing view (as that's what brings in the revenue) is the "engagement" metric. People signing up, asking questions, and accessing the site.

Better moderation tools which would result in fewer but higher quality questions on the site shows up in that measurement of engagement as "worse".

---

The part that you encountered is that "if the question isn't answerable, it should be closed." That in turn feeds other parts of the system. Users are more likely to update their questions if they are closed rather than if they're left open. Other people who answer questions (but rarely engage in fixing up questions) are less likely to click on questions that are closed. People that routinely ask poor questions that get closed start getting automated warnings about their question quality before they ask a question and end up with a question ban if that behavior persists. A closed question without answers or edits to improve it get automatically deleted after 30 days.

Without going in and commenting on a question and then spending time with the person ("why don't you just answer it if you think you know the answer rather than commenting? If you don't like it just don't read it." is something I've seen many times) closing the question is a way to suggest improvements to the question without exposing yourself to users who not infrequently then pursue a... negative engagement with the person trying to help them ask a better question.

I can see about digging more (it's been a long time since I went looking for it) but somewhere on one of the meta sites was a post about the different interactions and the "engagement" metric for new users asking a first question.

The best way to not have them ask a second question is to completely ignore their question - no votes, no comments, no answers. Closing a question results in more people asking a second question that is positively received than having no interaction.

(late edit - did the digging - https://meta.stackexchange.com/questions/216683/what-happens... )

0 comments

interroboink2y ago

I may be going in circles here, but when you say:

> The best way to not have them ask a second question is to completely ignore their question...

Leaving aside the issues of interpreting that data[1], and taking your conclusion at face value:

I get the sense that you imply this is a bad thing. But is it?

I agree that if the ultimate goal is "boost engagement metrics," then it's a bad thing. I suppose I just don't agree with that being the ultimate goal. And I sure wouldn't mind if other people in the community de-prioritized that goal, too. My opinion here applies to much about the modern internet landscape, to be fair (:

[1] eg: was their choice to ask a second question caused by a particular interaction? Or maybe users that ask second questions are more likely to ask a good first question, or other explanations and confounding factors? By another reading of it, we could say users who got their first question closed were 2x as likely to leave permanently than those who had no interaction (and this applies to over 2x as many users, so is even bigger in absolute terms). It's rather muddy.

shagieOP2y ago

The "boost engagement metrics" is a bit of tea leaf reading from posts from SO employees.

https://meta.stackoverflow.com/questions/423143/

> Experiment goals and success criteria

> Given that this is our first attempt to display additional content within the main content area, we’re interested in learning how users will engage with it and whether this will provide any incremental value when trying to find relevant content to get closer to their just-in-time needs.

> Our null hypothesis is that engagement with related questions remains the same in both experiment groups. We will determine whether the variant is a winner if there is a statistical significant lift to clicks on related questions, thus allowing us to reject the null hypothesis.

https://meta.stackoverflow.com/questions/422972/collectives-...

> We learned that there is a post-join increase in user engagement. For users who joined a collective, their activity in the collective’s tags increased by about 30% afterwards, compared to before joining. This was extremely encouraging from early on, validating that there is merit in creating a focused space.

> ...

> We’ve managed to put a beta product through its paces and arrive at something that does (some of) what we’d hoped – notably, increasing engagement in the subject area and teaching us about what works and what needs more iteration. This all happened without negatively impacting the core community in a big way, though certainly, we’d hoped the positive impacts would be more apparent to the community. Now, as we look ahead to bringing Collectives into a new stage and additional method of implementation, we’ve got a product that’s been iterated on and improved upon without disrupting what works well.

https://meta.stackoverflow.com/questions/422973/collectives-...

> A collective does not depend on sponsorship to continue existing. Sponsors may come and go, as with other sponsorship instances on Stack Overflow and Stack Exchange sites. Like those other scenarios, the health of the content and the engagement of the established community will determine the direction of the collective.

https://meta.stackoverflow.com/questions/422975/

> Growth rate and engagement levels

> To aim for the best possible outcome with this initial set of collectives, it was important to focus on subject areas that are currently “on the rise,” with increasing amounts of questions, answers, and traffic across the community. We also looked for high (or steadily increasing) levels of contributor engagement, since that is essential for maintaining content quality. And we looked for spaces with established best practices that allow subject matter experts to emerge. This would help ensure that the collective can remain relevant in the long term.

---

If there's one consistent thing that SO employees seem to be basing the success of a particular "did this idea work" it is engagement.

It's not a community goal for engagement - it's a corporate goal. And thus the active answering / curating / moderating community has very little ability to influence SO corporate metrics other than through engagement (or lack of it).

Engagement is being used as the proxy for all other metrics - potentially subdivided (new user engagement, established user engagement, etc...).

Going back to your question, I believe that SO is best served by having good content that is easily discoverable and accessible. That involves actively curating posts which also means removing questions (and answers) that make the existing good content harder to discover (having 100k posts about null pointers makes it harder to find the post that people have put more effort into having it answer the question).

I also believe that in today's internet culture, anything that is "negative" is considered to be rude or hurtful in some circles. Saying "this question isn't a good fit for the Q&A format that Stack Overflow provides, if you are seeking a discussion about {topic} you may find asking about it on reddit /r/AskProgramming to fit the style of communication that you want to have or the handheld guidance that you need" - gets back a "why do you close this question? Why not ignore it and let someone else answer it" and is seen by some as rude and thus perpetuates the "SO is rude" meme.

I believe that SO is much better suited to a Q&A format and optimizing for that means making asking questions harder (and removing questions easier) which means either getting better buy-in from people who are unfamiliar with that model ( https://stackoverflow.com/tour remains rather controversial with some of the user base about opinions and discussions) or dealing with the repercussions of "SO is mean"... and that's where it kind of runs into difficulty.

j / k navigate · click thread line to collapse