Using real member accounts to post the fake AI content is so much worse.
I can’t even imagine the thought process that led to thinking it was a good idea to generate fake posts but assign them to real users. If they were going to fake posts, why not fake the users, too? They turned one bad decision into a much worse decision.
It must be sad to see your proud little corner of the internet decline in popularity, but this move took whatever positive historical vibes it had and lit them on fire.
> Using real member accounts to post the fake AI content is so much worse.
Yeah, that part seems not just ethically bankrupt, but also really really stupid and counterproductive?
It all made a bit more sense to me when I learned that the guy who did all this is not, in fact, a physicist like I had originally assumed, but rather an "SEO lead at Shopify". [1]
> the thought process that led to thinking it was a good idea
Seemingly that can be found here: physicsforums [dot] com/threads/official-response-to-allegations-of-ai-misuse-in-pf.1078532/
> During the past several months, to counter declining engagement, I experimented with generative AI, intending to add value to "thin threads", especially those with 0 replies.
> For experiment #3, I used real accounts that I considered to be decade-old spammers as best I could tell, but I can't be 100% certain. I categorically regret and reject such a shameful and unethical decision.
(that website also modifies your clipboard when you copy stuff, big fuck you to the site owner, seemingly lacking any sort of guiding compass, good riddance to them)
But what's the end game envisioned here? Turning the website into an LLM proxy and hoping enough suckers look at your ads instead of those from other sloperators?
Boosting impressions to maintain "passive income" is one possibility.
I’ve seen niche forums in the past insert affiliate links, so when somebody clicks on that HP graphing calculator they go straight to Amazon and the site operator gets a 2% commission. This was the approach of a website that plagiarized Obscure Sorrows and was discussed here on HN (https://news.ycombinator.com/item?id=48611823).
And then there’s tinfoil hat territory: massive influence operations that span multiple forums and Wikipedia and random blogs to support specific points of view about scientific claims, preferred technology approaches tied to commercial platforms, and even historical claims such as who or what developed a scientific concept.
Like other zombified sites, it exists to collect ad revenue from Google. It's amazing how despite 25+ years of technological change, that Google's decades-old "click ad" model is stronger than ever.
"Fake posting activity" is a weak framing. It is real activity. The risk in allowing AI-generated content is that users will be able to sniff out low-quality content produced by AI that they don't find engaging, but if they don't like it they can always leave. If users love it then AI-generated content is not bad at all.
Here’s the problem - it’s the same problem in many areas where AI is put to (ab)use:
I already have my own AI accounts. Several of them, really. Some of them paid. For work and for personal. If I want any kind of take from AI, which is in fact a very frequent occurrence, I can just prompt for one. For free even! And with my choice of model based on how important or complex the topic. There is zero value-add for me when some product decides to squirt an extra helping of AI somewhere it wasn’t needed. Summary of a long piece of text (while still giving access to the unabridged text)? Probably fine. An AI answer to a question someone else posted on a forum? Or injected above my web search results (looking right at you, Google)? No thanks!
I'm not sure you have explained the problem. You have explained why PhysicsForum might not appeal to you, but it doesn't have to appeal to you. It is not a problem if it doesn't appeal to you. It never appealed to most people.
Xp84 made a clear description of the real problem— i.e the one everyone talks about and which the OP is about too.
You’re basically saying that if “you have a problem with something, then it’s your problem” - well shit - that’s exactly the meaning of having a problem with sometimes and you’re literally restating it saying it’s in fact not a problem— presumably because you want to be contrarian on the internet, or why is that not a problem?
The thing is that, by common definition, a problem ceases to be a problem when there is a solution. To not remove problems from the pool once they are solved would leave everything to be a problem forevermore, which isn't useful. I appreciate your desire to want to play a game of semantics and pet definitions so you can get your pent up angst out, but it is uninteresting. To restore good faith, how about you introduce an actual problem into the discussion?
I can easily follow xp84's comments and random3's comment. I see no games with semantics, just a straightforward complaint about dishonest use of AI slop. Which the site owner acknowledges.
Given your understanding and emotional disconnection, perhaps you can rationally explain their position?
There is complaint, like someone complaining that they don't like your new haircut, but it is ultimately not their haircut to worry about. How is your apparent poor choice in style a problem to the onlooker?
The "fake" aspect is the fact that it's presented as human activity. The risk is therefore that users identify low-quality content and then realize that the forum has no qualms about misusing/abusing existing accounts and misleading new ones.
Are you excited to generate an account and communicate with experts such as the ones in the article today?
Why not? AI needs an interface and there isn't one interface to rule them all, as evidenced by Photoshop still raking in money hand over fist despite ChatGPT having full photo editing capabilities. UI and UX matter. The appeal of online forums has always been that you don't have to ask; the UI puts topics in front of you. The AI labs haven't really offered anything like that, so it still rests on third-parties to provide that UI.
A good product is a good product. The implementation details don't matter.
It's ironic because the article (and many comments in the previous submission) are about how one shouldn't mess with user data to show fake engagement!
As the percentage of LLM-generated stuff in the world increases relative to natural stuff, I don't see how it's possible to continue to train new models indefinitely. At some point, won't LLMs be trained on a corpus that is majority LLM-generated data, with everything that entails both good and bad? Seems to me that singularity is just going to kick off a self-reinforcing spiral... Bigger minds than me have to have considered this, is this a legitimate concern?
I wonder if someone has redone the experiment of retraining LLMs on LLM output that was conducted a few years back that showed a pretty drastic degradation in quality. I imagine it's hard to perform on the closed source frontier models and the ballooning cost of training doesn't help.
In theory as the models improve their ability to ape human writing the effect should diminish, though iterative reingestion of their own output is probably still detrimental long term.
Maybe with time we'll accomplish the same thing that's happened with pre 45 steel and the instruments will need less of it. I guess in this analogy it'd be better quality filters that can detect and remove LLM-spam.
So, how did all these hacked accounts or backdated posts happen? The article never says that even though the author talks to the site founder and admin.
Jesus. I'm risking identifying myself here, but it has happened before. I was one of the earlier moderators of PhysicsForums back in 2003 and one of the younger ones. I think myself and one another was an actual student, whereas the others were all professors or retired. I believe my last login was 2013 and last real activity well before that. I no longer have access to the email used and couldn't recover the account even if I wanted to.
I only talked to Greg remotely and never met him. This is a huge betrayal of trust. Older moderators have died since then. This is a very longstanding community. Even if we didn't stick around forever, many of us became real-world friends. Adding autogenerated summaries is one thing, but inserting comments in a real person's name that they never made, when you have no knowledge of what became of that person, who may even now be dead, is beyond the pale. You're rewriting history. The fact that this site still exists at all is extremely gratifying. It's a huge part of what has made me who I am today. It's a big part of why I don't really pontificate all that much, don't have a blog. I read what I wrote there nearly a quarter century ago, and it's not all bunk. A lot of what I said holds up. Plenty of it was raw factual data never in dispute. But when it came to politics and economics and forecasting of the future, I said things so wrong and thing I so strongly disagree with now that having that record teaches me how flimsy and full of shit so many of our opinions are, no matter how smart we are and no matter how strongly we hold those opinions. I've 180'd things I was completely certain of during the first Bush administration. Heck, you can add voting for Bush to that list. There is no way in hell I would do that today.
History matters, Greg. It can both humble and enlighten us. But only if it's true.
It’s especially disappointing to see this done in such an unnecessarily dishonest way. If the project truly wanted to add value, I can imagine interesting AI uses that don’t lie or trick people. You could post a correctly-dated AI Answer (and prompt it to use the web search tool and cite sources) to posts that apparently never got one, and credit the exact model used, which would help future readers decide how confident they might be (a GPT-3 answer vs a GPT-6 Astra answer, for instance, are very different).
Heck, you could even engagement-farm if you have to, by taking science stories in the news and say, generating a poll to try to spur discussion.
But what they actually chose to do is super gross, helps nobody, hurts the site’s reputation, and even devalues the site tremendously as possible training data for AI. (Which, some AI detractors might actually be happy about, but I think isn’t really a good thing.)
Many years ago I did a search for my name and turned up a LOT of things I did not actually write. This wasn't LLMs, this was scrapers doing a poor job of scarfing up old content--comparing what I saw vs my recall of the discussions there was a lot of stuff that lost the quoting, words were attributed to me that were actually quotes I was responding to.
Also this is a good example of the trend that has already been ongoing on HN since 2022. I think one other long-lasting HNer is working on a similar analysis of HN at the moment.
HN essentially became mainstream after the New Yorker profile, Reddit's degradation due to the mod blackout and API changes, and Lemmy flopping.
I've gotten my HN back to the pre-2022 consensus but it takes severe filtering, with around 200 posts being hidden a day. And it's those posts that drive the majority of discourse on the platform now.
The fact that throwaways like you are so normalized is a huge reason.
HN has also gotten divorced from tech and water-cooler conversations in much of the tech industry, and a lot of that seems to be because HN no longer has the same plurality of Bay/NYC/Seattle/Austin techies that it used to, and the userbase is way older than much of tech now (I was always younger than average on HN, but the amount of 1980s to early 2000s references shows that the platform has frozen in time demographically).
Ignore Reddit - HN has functionally become Slashdot (the fact that I even know what slashdot is highlights how the age demographics are so skewed on HN).
Edit: can't reply
> 2022 is when tech funding dried up...
Tech funding didn't dry up then. It just coencided with the time period that most vintages that originated in early 2010s were closing out and those that originated in 2016-18 were hitting their 5 year mark.
This meant a lot of dry-powder didn't get deployed until 2023-24.
> stands to reason that we ran out of hacker topics...
They are still being posted at the same rate based on my filtering. The difference is a massive deluge in non-"traditional HN" related content being posted, and the younger generation of techies and founders (those below 35) not using HN.
2022 is when tech funding dried up, so it stands to reason that we ran out of hacker topics. The odd person still doing something cool in their basement for fun does not supply a sufficient stream of curiosities to keep the feed full. That required the unique time (ZIRP, etc.) when people were paid handsomely to explore cool things day in, day out.
HN was a product of its time.
--
> They are still being posted at the same rate based on my filtering.
"I rewrote X in Rust" isn't a hacker topic. Being able to draw some kind of connection to the tech isn't the same thing.
That's a good point. Fwiw, I use throwaways because I don't want certain types of users to know anything about me. I wouldn't have felt this way, say, 5 years back.
I built my own system using both sentiment polarity analysis, hard-coded logic, and some semi-active tuning, but I'd rather not divulge specifics. IMO anyone can spin up their own similar system.
> I don't want certain types of users to know anything about me. I wouldn't have felt this way, say, 5 years back...
Frankly, most people DGAF about random HNers and (a general or meta-comment targeted at the reader, not you OP) if someone needs to use throwaways to post negatively polarized statements on HN, then they shouldn't even be on HN in the first place.
> if someone needs to use throwaways to post negatively polarized statements on HN
If it were only read by other HN users who understand that a user account is not a human then it wouldn't matter, but the world is not so simple. There are people, who do not understand that forum accounts are not humans, who are working hard to associate internet activity with humans and will use it against them.
Of course, we already know how to identify individuals by their writing style, so a throwaway account probably isn't the screen the OP hopes it is. Still, it is the best source of making it as difficult as possible we seemingly have.
Politics is discussed significantly more on HN after 2022 than before. I'll wait on that HNer to finish their analysis and post it on here, but they've also noticed the same trend.
My impression for a while was that there were nasty political threads that devolved into flagging and name-calling. I have seen things about politics (elections, corruption, contentious lawsuits, etc.), but I haven’t seen the accompanying flame wars.
There was also an experimental period where they’d banned all political articles, and then they reversed it.
The "Dead Internet Theory" is too sad and too easy. I'd propose the "Dead Society Theory", the history of the Internet from the late 2000s to the late 2010s left people socially scare, paranoid, angry and hopeless about the possibility of communicating online.
Which is to say the Internet can be "dead" without being filled with undetectable or detectable fake posts, not that fake posts help things.
IMHO, the theory is wrong. There is tons of real, human interactions online. I see it on reddit constantly. The bot activity is easily ignored or algorithmically suppressed most of the time.
If the dead internet theory were true or real, Google and Facebook would not generate so much revenue. Advertisers would notice that they are not getting any conversions and stop spending.
Using real member accounts to post the fake AI content is so much worse.
I can’t even imagine the thought process that led to thinking it was a good idea to generate fake posts but assign them to real users. If they were going to fake posts, why not fake the users, too? They turned one bad decision into a much worse decision.
It must be sad to see your proud little corner of the internet decline in popularity, but this move took whatever positive historical vibes it had and lit them on fire.
Yeah, that part seems not just ethically bankrupt, but also really really stupid and counterproductive?
It all made a bit more sense to me when I learned that the guy who did all this is not, in fact, a physicist like I had originally assumed, but rather an "SEO lead at Shopify". [1]
[1] https://www.mkedmc.org/people/greg-bernhardt
Seemingly that can be found here: physicsforums [dot] com/threads/official-response-to-allegations-of-ai-misuse-in-pf.1078532/
> During the past several months, to counter declining engagement, I experimented with generative AI, intending to add value to "thin threads", especially those with 0 replies.
> For experiment #3, I used real accounts that I considered to be decade-old spammers as best I could tell, but I can't be 100% certain. I categorically regret and reject such a shameful and unethical decision.
(that website also modifies your clipboard when you copy stuff, big fuck you to the site owner, seemingly lacking any sort of guiding compass, good riddance to them)
I’ve seen niche forums in the past insert affiliate links, so when somebody clicks on that HP graphing calculator they go straight to Amazon and the site operator gets a 2% commission. This was the approach of a website that plagiarized Obscure Sorrows and was discussed here on HN (https://news.ycombinator.com/item?id=48611823).
And then there’s tinfoil hat territory: massive influence operations that span multiple forums and Wikipedia and random blogs to support specific points of view about scientific claims, preferred technology approaches tied to commercial platforms, and even historical claims such as who or what developed a scientific concept.
I already have my own AI accounts. Several of them, really. Some of them paid. For work and for personal. If I want any kind of take from AI, which is in fact a very frequent occurrence, I can just prompt for one. For free even! And with my choice of model based on how important or complex the topic. There is zero value-add for me when some product decides to squirt an extra helping of AI somewhere it wasn’t needed. Summary of a long piece of text (while still giving access to the unabridged text)? Probably fine. An AI answer to a question someone else posted on a forum? Or injected above my web search results (looking right at you, Google)? No thanks!
You’re basically saying that if “you have a problem with something, then it’s your problem” - well shit - that’s exactly the meaning of having a problem with sometimes and you’re literally restating it saying it’s in fact not a problem— presumably because you want to be contrarian on the internet, or why is that not a problem?
There is complaint, like someone complaining that they don't like your new haircut, but it is ultimately not their haircut to worry about. How is your apparent poor choice in style a problem to the onlooker?
Are you excited to generate an account and communicate with experts such as the ones in the article today?
A good product is a good product. The implementation details don't matter.
https://news.ycombinator.com/item?id=42816284 - Jan 24, 2025 | 154 comments
It's ironic because the article (and many comments in the previous submission) are about how one shouldn't mess with user data to show fake engagement!
For millennia we created documents supporting the geocentric model, all wrong! How'd we break out of that rut?
I don't know if AI are currently equipped to deal with this as humans did, but maybe they can be?
In theory as the models improve their ability to ape human writing the effect should diminish, though iterative reingestion of their own output is probably still detrimental long term.
I only talked to Greg remotely and never met him. This is a huge betrayal of trust. Older moderators have died since then. This is a very longstanding community. Even if we didn't stick around forever, many of us became real-world friends. Adding autogenerated summaries is one thing, but inserting comments in a real person's name that they never made, when you have no knowledge of what became of that person, who may even now be dead, is beyond the pale. You're rewriting history. The fact that this site still exists at all is extremely gratifying. It's a huge part of what has made me who I am today. It's a big part of why I don't really pontificate all that much, don't have a blog. I read what I wrote there nearly a quarter century ago, and it's not all bunk. A lot of what I said holds up. Plenty of it was raw factual data never in dispute. But when it came to politics and economics and forecasting of the future, I said things so wrong and thing I so strongly disagree with now that having that record teaches me how flimsy and full of shit so many of our opinions are, no matter how smart we are and no matter how strongly we hold those opinions. I've 180'd things I was completely certain of during the first Bush administration. Heck, you can add voting for Bush to that list. There is no way in hell I would do that today.
History matters, Greg. It can both humble and enlighten us. But only if it's true.
Heck, you could even engagement-farm if you have to, by taking science stories in the news and say, generating a poll to try to spur discussion.
But what they actually chose to do is super gross, helps nobody, hurts the site’s reputation, and even devalues the site tremendously as possible training data for AI. (Which, some AI detractors might actually be happy about, but I think isn’t really a good thing.)
Many years ago I did a search for my name and turned up a LOT of things I did not actually write. This wasn't LLMs, this was scrapers doing a poor job of scarfing up old content--comparing what I saw vs my recall of the discussions there was a lot of stuff that lost the quoting, words were attributed to me that were actually quotes I was responding to.
Also this is a good example of the trend that has already been ongoing on HN since 2022. I think one other long-lasting HNer is working on a similar analysis of HN at the moment.
Edit: can't reply
> Appreciate some more details...
It's their story to tell. Not mine.
I've been reading HN since 2013 and the quality has gone down, kind of like eternal September.
HN essentially became mainstream after the New Yorker profile, Reddit's degradation due to the mod blackout and API changes, and Lemmy flopping.
I've gotten my HN back to the pre-2022 consensus but it takes severe filtering, with around 200 posts being hidden a day. And it's those posts that drive the majority of discourse on the platform now.
The fact that throwaways like you are so normalized is a huge reason.
HN has also gotten divorced from tech and water-cooler conversations in much of the tech industry, and a lot of that seems to be because HN no longer has the same plurality of Bay/NYC/Seattle/Austin techies that it used to, and the userbase is way older than much of tech now (I was always younger than average on HN, but the amount of 1980s to early 2000s references shows that the platform has frozen in time demographically).
Ignore Reddit - HN has functionally become Slashdot (the fact that I even know what slashdot is highlights how the age demographics are so skewed on HN).
Edit: can't reply
> 2022 is when tech funding dried up...
Tech funding didn't dry up then. It just coencided with the time period that most vintages that originated in early 2010s were closing out and those that originated in 2016-18 were hitting their 5 year mark.
This meant a lot of dry-powder didn't get deployed until 2023-24.
> stands to reason that we ran out of hacker topics...
They are still being posted at the same rate based on my filtering. The difference is a massive deluge in non-"traditional HN" related content being posted, and the younger generation of techies and founders (those below 35) not using HN.
HN was a product of its time.
--
> They are still being posted at the same rate based on my filtering.
"I rewrote X in Rust" isn't a hacker topic. Being able to draw some kind of connection to the tech isn't the same thing.
What tools do you use for filtering if I may ask?
> I don't want certain types of users to know anything about me. I wouldn't have felt this way, say, 5 years back...
Frankly, most people DGAF about random HNers and (a general or meta-comment targeted at the reader, not you OP) if someone needs to use throwaways to post negatively polarized statements on HN, then they shouldn't even be on HN in the first place.
It is never most people you need to worry about.
> if someone needs to use throwaways to post negatively polarized statements on HN
If it were only read by other HN users who understand that a user account is not a human then it wouldn't matter, but the world is not so simple. There are people, who do not understand that forum accounts are not humans, who are working hard to associate internet activity with humans and will use it against them.
Of course, we already know how to identify individuals by their writing style, so a throwaway account probably isn't the screen the OP hopes it is. Still, it is the best source of making it as difficult as possible we seemingly have.
Appreciate some more details.
There was also an experimental period where they’d banned all political articles, and then they reversed it.
Which is to say the Internet can be "dead" without being filled with undetectable or detectable fake posts, not that fake posts help things.
If the dead internet theory were true or real, Google and Facebook would not generate so much revenue. Advertisers would notice that they are not getting any conversions and stop spending.