Very interesting. The author claims to have proved that markets can be informationally efficient or competitive, but not both. The implications for policy and regulation are significant.
The fact that free markets don't exist, and that supply and demand is not a natural law that implies efficient markets has never stopped people acting like both are true and stuffing fingers in their ears.
But, both free markets and supply/demand are useful enough concepts to talk loosely about processes to understand the interest that I'll enjoy digging into this.
They're not just useful concepts tho, they're how every business operates, and the concepts cover the vast majority of situations.
The behavioral economics/Freakonomics thing was like "Hey, here's this thing that might if you squint real hard fall outside of efficient market theory" and then for a decade people took that to mean that that the base concepts were worthless, which was a severe overcorrection from people that didn't understand economics.
Knowledge accumulation seems to have cyclical patterns where general observations are condensed into "laws", and common wisdom is to treat these as unassailable truths. As exceptions and contradictions are found, the popular wisdom is to entirely dismiss the overwhelming universal conformity with the earlier principles because it sounds smart to common people. A lot of normal people see the basic problems with most methodological maxims but are led to believe that the maxim is truth. It takes a podcaster with a "study" to then "debunk" the "myth" they were yesterday defending as unimpeachable. People are generally not well informed enough to know anything and the podcaster class is the worst imaginable vector for knowledge dissemination or popularization. Just clickbait garbage monkeys with foreign money and vicious antisocial tendencies.
The vast majority of real world businesses create prices based on cost plus systems, not on supply/demand, which are generally impossible to measure directly. Another good chunk of smaller businesses simply copy the prices of larger businesses, at least in b2c markets. Sure, if their goods are not selling, they might reduce price, but they might try other tactics as well (marketing, targeted discounts, etc).
these "other tactics" are just attempts at increasing demand. Cost plus is how many business set prices but that is entirely separate from what the market decides. If cost plus is not an attractive price, it won't sell, because the demand is not there.
There are many ways that free and competitive markets can fail other than behavioural economics! Monopoly, informational asymmetry, externalities… All of these are plausibly pervasive.
Economics is one of the scientific sore spots for liberal ideology. If you are liberal and mostly swim in liberal circles, you probably believe that economics is mostly bunk pseudoscience. Akin to conservatives and climate change
Well, for better or worse, markets are becoming increasingly dominated by "non-biological utility maximizers" - mostly hft bots, but now also llm-based reasoning agents.
In a P=NP world, it still takes only one uninformed participant to make the market inefficient. I don't think the implication is bidirectionally true unless you assume every single player is rational, infinitely smart, and has access to the same set of information.
The title is wrong but you’re also wrong. Read the abstract of the paper. Here’s the relevant section:
> Combined with Maymin (2011), who proved that market efficiency requires P = NP, this yields a fundamental impossibility: markets can be informationally efficient or competitive, but not both.
Seems that HN's auto-headline rewriting in this case has made a critical error :)
>Artificial intelligence, by expanding firms' computational capabilities, is pushing markets from the competitive regime toward the collusive regime, explaining the empirical emergence of algorithmic collusion without explicit coordination.
I have to dig more into the paper but I don't see how this follows, except in the most straightforward way. Basically, if everyone uses the same methods to derive price, of course there will be "collusion", or in other words, everyone will have the same price. But this doesn't seem like a result of compute per se, but simply better communication networks and information flows. You could have gotten the same result in medieval England by having everyone post their selling prices on the town square board.
Again, I haven't dug into the paper yet, but it seems like what really matters for firms is "compute"/$ (if the "compute" is an LLM or an assistant that has to go walk the 10 minutes down to the square makes little difference)
Edit: Isn't another implication of this, that increased compute -> collusion imply that increased compute -> communism becomes feasible?
I think this goes to my point above though, the primary problem preventing fully automated luxury communism isn't compute per se, but actually observing the information flows to make it possible. Capitalism famously solves this information problem through the pricing mechanism. So in effect, he's arguing that extra compute makes information gathering more efficient, and at the limit you get perfect information. Which, yeah, I guess so. Assuming everything can be perfectly measured, even theoretically.
Yeah, the most obvious recent example of this is RealPage’s YieldStar product. It advised property managers on what they should set their rental rates to, and allegedly established a cartel in which RealPage’s customers coordinated in pricing their units.
YieldStar was technically an “AI” product, but I don’t really think the computational abilities were what enabled the collusion. RealPage’s employees (according to the DoJ[0]) would actively monitor whether companies were following their pricing recommendations and call up companies that defected. And the software itself used dark patterns to make it easier to simply follow the YieldStar pricing suggestions, rather than set a lower rental rate and be more competitive. The algorithmic pricing I think did allow people to launder their own judgement and simple “trust the process” in a way that in the past would have required knowing complicity with the cartel, but I don’t think it required substantial compute capacity.
(This isn’t a comment on the paper by the way, which I glanced at but did not have the background knowledge to fully comprehend)
I’m not sure the use of a common algorithm was the most damming part of that. They also pooled otherwise proprietary information and penalized landlords who failed to follow the “recommendations”
You could imagine the exact same scheme without the use of a computer.
I think the common algorithm / the computer were the fig leaf, not the enabler, yes. The point is that they tried to launder obvious cartel practices as a simple computer recommendation system.
The paper seems to be based on an invalid assumption. From the abstract:
> If P != NP, the collusion detection problem is computationally infeasible for markets satisfying a natural instance-hardness condition on their demand structure, rendering punishment threats non-credible and collusion unstable.
...and then from the paper:
> Stigler (1964) famously argued that the “chief difficulty” of collusion is detecting “secret price-cutting.”
The thing is that Stigler's insight is far from proven, and indeed, the primary difficulty in collusion is often not the detection of defection. Firms know they're being undercut all the time. The problem is that very often, there is nothing they can do about it. Markets are specifically structured as firm-to-firm transactions, where competing firms have no leverage over what your firm can do or what sort of transactions you can conduct, and as long as this condition holds it doesn't matter if you know that a competitor is fucking you over, you can't do anything about it.
I'd argue that the increase in collusion and anticompetitive behavior lately is because these conditions increasingly don't hold. When you intersperse another party in the transaction, eg. a regulatory agency, permitting body, or exclusive distribution deal, you introduce a leverage point for incumbents to punish competitors who choose to undercut them.
Why can’t my firm react if we find out we’re being undercut by a competitor? Or are you saying that we “know” only in a theoretical, “we can’t prove we’re not being undercut” sort of way, but without “proof” we can’t take action?
You absolutely can react, but in general, in a functioning competitive market, you can not alter your competitors' actions.
Examples of the former: cutting prices yourself; increasing product quality; differentiating yourself; spending more on advertising to get the word out about your product.
Examples of the latter: crafting exclusive deals with your distributors to prevent your competitors from getting shelf space; politically influencing regulatory bodies to declare your competitors' existence illegal; making direct agreements with the leadership of opposing firms to not drop prices or hike wages; assassinating, extorting, or kidnapping rival business leaders.
Basically it comes down to "control yourself, because you cannot control others". In a functioning market, you have no control over what rival firms. Your only legal reaction to competition is to improve your own offering until it is the best it can be. In pathological markets where the assumption is (as in the paper) that you can punish rivals for not colluding, you actively make your competitor's offering worse.
These things should not be submitted here unless there is meaningful editorial/peer commentary about it being significant or correct. There are dozens of bogus attempts to prove and disprove this
I don't have the mathematical chops to really analyze this on my own. Is this a bigger deal than the fact that real world markets already violate all the theoretical assumptions (e.g. unimpeded access to new entrants, perfect information, etc.etc), and so, in practice are never perfectly competitive or efficient?
Defi does a pretty good job regarding unimpeded access in comparison to the more traditional venues. This isn’t just about getting money into the system, but also what instruments you have access to.
The term ‘perfect information’ is a bit of a mirage, and has been shown to be impossible in physics (uncertainty principle).
What really matters is information advantage: Does your inexact expected value function consistently beat others’ calculations in the market. Here, the true value - value really is just a word and is dependent on people - is irrelevant.
This feels like it's pointing in the same direction Hayek was already pointing 80 years ago. This paper says that even if we had enough computing power to solve these problems, Hayek's argument was that we still wouldnt have all the information needed to do those calculations in the first place. That information is spread across millions of people, constantly changing, and often only known locally. So even if the computation became easy, getting all the inputs would still be impossible to achieve
But you can just give computers to each of those millions of people and propagate the information, instead of having less reliable humans doing it? That's kind of what's already happening.
> markets can be informationally efficient or competitive, but not both
Really interesting conclusion, but I can't help but feel this is overly reductive, as stated. Surely market efficiency is a sliding scale and so is market competitiveness.
Okay, so a perfectly competitive market cannot also be a perfectly efficient market. Interesting! But I'm confused about how this may work when efficiency and competitiveness are a sliding scale. Should we think of this as one axis (with a spectrum from efficiency to competitiveness) or as two separate axes that just happen to have an exclusive relationship between their extremes?
There's a long history of proving results like this.
NP-Completeness is the norm, not the exception. Any system that's complex enough is almost surely NP-Complete. For similar reasons, Turing Machine Equivalence is also the norm, not the exception.
These results are interesting but not unexpected. A more interesting question is under what conditions is the problem difficult to find solutions for. Many NP-Complete instance ensembles turn out to effectively have polynomial time solutions (3-SAT w/ uniform clause variable choice, Hamilton Cycles in Erdos-Renyi random graphs), so proving NP-Completeness is not a death knell for approximation.
Right. Maybe a better and more humble title is "Identifying General Instances of Market Collusion is NP-complete". Not as headline grabbing, but more in line with the actual result.
Not quite sure what you're suggesting here; perhaps it's satire?
If P!=NP then it is arbitrarily smaller, for the same reason that e^x > Cx^N for any constants C and N, as long as x grows big enough. There is no epsilon in that can overcome that, no matter how big you make it, because x will eventually dominate the equation.
There are a lot of cases where pragmatically x remains small enough that it doesn't matter, and a P algorithm will give you an answer more quickly. (For the same reason I only ever write bubble sorts: I would only write my own at all if I knew that the list would never be bigger than 10. Even then it's only when using the library is too much trouble for some reason.)
But we care about P and NP when the number can potentially be very, very large.
No, it's not satire. The difficulty of finding the optimal solution says nothing about what it takes to come within 99.999% of optimal with 99.999% probability.
So I'm not talking about the number of steps needed to prove optimality with a correct P algorithm versus an exponential one.
I'm only talking about how this applies to the efficient market hypothesis.
The argument structure is interesting, and reminds me a lot of Solomonoff Induction, but constrained into NP by the assumptions. I'm not sure the front half is enough to support the back half of the paper arguing that the current LLM craze means firms are actually running collusion detection algorithms, even unintentionally.
When everyone uses AI to study the same indicators and figures out how the prices move with those indicators they all start investing at the same time and the prices move together. AI silently gets everyone on the same page.
next stage - a single AI which computes the "correct" price that everyone else agrees on, and instantly reprice all financial instruments without trading them, thus not paying transaction costs
I hate to make the worn out AI to RNG comparison, but this kind of simultaneous "collusion" is really like assuming that everyone is using the same RNG seed to make their calls.
I don’t think anyone in the business thinks that the markets are 100% efficient, just that they are sufficiently efficient that beating them is a genuinely hard job requiring heavy, expensive analysis.
> just that they are sufficiently efficient that beating them is a genuinely hard job requiring heavy, expensive analysis
No expensive analysis is needed. Beating them just requires (1) having disproportionate capital to others, and (2) having disproportionate control (and therefore know in advance) whatever the markets are pricing.
It helps that there are enough believers (in the religious/cult sense) that markets are 100% efficient, that they will deny that any of that is happening - even while they lose their shirts to those who are doing the market manipulation.
> Artificial intelligence, by expanding firms' computational capabilities, is pushing markets from the competitive regime toward the collusive regime, explaining the empirical emergence of algorithmic collusion without explicit coordination.
Wow... this is quite fascinating. It has been theorized for a little while that widespread AI could form accidental trusts due to optimization around one-another. This seems to be taking it a step further and arguing that if P!=NP, then markets are certain to trend towards collusive.
This is interesting. Adam Smith said "People of the same trade seldom meet together, even for merriment and diversion, but the conversation ends in a conspiracy against the public, or in some contrivance to raise prices."
The annoying part is that, as the same Adam Smith says, regulating industries would end up enforcing such assemblies, reinforcing the problem... after all, industries can share information via the market itself...
And proposed solutions end up being controversial: employees ownership, open source, paying taxes over stocks ownership... or just hoping that colluders will be broken by a randomly ocurring incumbent...
It's a vast generalization of the cost to verify a problem/solution.
- P means polynomial
- NP means nondeterministic polynomial
Roughly polynomial (P) represents the upper bound of the cost to verify, and the polynomial characterization says that given a problem with a certain input (e.g. the input can be the number of training examples in ML training set or the number of constraints/conditions in a general problem— e.g. all the places you want to visit on your next trip, given many "wants" in a group)
When the cost is polynomial relative to the input size it means it can only be finitely larger than the input - that's a characteristic of the polynomial which is just a finite sum of powers of x (the input).
When the cost is nondeterministic polynomial, one way to think about it in what is called a nondeterministic Turing machine. The nondeterministic part refers to the "states" that the machines can transition to from any current state. When the transition can happen to more than one state, we say it's non-deterministic— and can imagine it's determined by some probability.
The general assumption is that polynomial (P) is easier than nondeterministic polnomial (NP). This isn't necessarily the case as there can be arbitrarily large finite numbers (making P solutions intractible)
The P vs NP problem is one of the main open problems and generally considered a crank magnet and general confusion. For a good (likely the best) resource see https://scottaaronson.blog/
A problem in P can be solved in polynomial time - the computation required grows relatively slowly as the input size increases. Like sorting a list of numbers.
A problem in NP requires exponential time or greater, but a proposed solution can be verified quickly. For example, checking a completed Sudoku puzzle.
It is believed but unproven that all problems in NP are NOT in P.
A problem in NP can have a (positive) solution verified in polynomial time. That's it. Requiring more than polynomial time to solve isn't part of the definition, and in fact it's an open question whether any problems in NP require more than polynomial time to solve.
Every single problem in P is in NP. What is believed but unproven is that some problems in NP are not in P.
I have never really understood this, from the time it was first introduced to me in my undergraduate education nearly 40 years ago. It seems to boil down to saying "all unsolvable problems are not in the set of solvable problems" which seems like a tautology. I don't get why "P != NP" is considered so profound.
I feel like I have not yet found the proper explanation. Or I'm just too dense to get it.
Not sure you're wanting an explanation, but it comes down more to equivalent algorithms than rigid categories. For example there is a P algo to sort a list of numbers, but not to solve a sudoku (NP). However there is a polynomial algo to check sudoku (spaces ^ 2 if you check every space against every other space for rule violation).
However, the reason all NP algos are part of the same category is because you can solve any problem in NP by switching the problem into another problem in the same category and solving that. For example, you can turn sudoku into a graph coloring problem, which is also NP. You can turn sorting (P) into something like balancing a tree, which is also P.
The major question is "is there any algorithm that would allow us to change some NP problems into P problems, solve it, then use it for the original problem". E.g. could we take graph coloring and turn it into sorting a list of numbers?
So basically, if there is any way to bridge the two, then it might mean every NP problem is actually solvable by a P algorithm, under some transformation. This would be immense because it would completely change the way we solve those algorithms and greatly reduce compute costs.
While this seems far-fetched, realize that there are some problems that seem extremely expensive if done the naiive way, but are actually solvable in P. For example, you _could_ write an exponential sorting algo (try every element in every position), but clever people found a way to make it efficient (P). So its possible we just need the right algo to completely change the landscape of computing.
However, as you say, its almost self-evidently true that P != NP, but has never been proven so (to do so, we need to prove that no such algorithm can exist). But clearly, solving an exponentially complex problem using a O(log n) algo would be remarkable.
To take a concrete example, currently the best algos to exhaustively check a board game like chess or go are exponential (NP). Its easy to verify the winner, but its exponential to enumerate every possible move (e.g. 80^turns states). If we found a polynomial way to solve this (even by converting to something simplified), then it would mean we could exhaustively search chess polynomial to the number of moves (e.g. turns^100). This changes it from "cannot be done in the lifespan of universe" to "its possible with a powerful computer in measurable time". We already use heuristics and estimates to explore the exponential space in efficient time, so if we had a polynomial algo chess, markets, optimization, and other NP problems would be extremely efficient to solve.
In short, P means Polynomial time (i.e. markets can solve computation problems efficiently) and NP means Non-Deterministic Polynomial time (i.e. markets can verify solutions of computation problems efficiently but solutions are found by luck).
If P != NP, it means luck CANNOT be engineered and markets are competitive.
in CS we define a complexity class as a set of problems that have the same growth characteristic. that is for a problem size N, how long does it take in the worst case to find the solution for that problem.
one such class is the Polynomial class, or P, where the time to solution is some fixed exponent of N (like N^2, or 3).
the next big step is NP, which require a polynomial number of nondeterministic steps, whose solution can only be verified in polynomial time. usually solutions to NP problems are exponential in cost with respect to N (like 2^N), but thats not part of their definition.
problems in NP are generally identified by mapping them into a well known problem known to be in NP, where the mapping has to occur in polynomial time.
its an open question as to whether NP as a class can actually be solved in P time, but most people doubt that that is really the case.
If markets were perfectly efficient, entrepreneurship would not exist. An entrepreneur is, at this level, someone who looks for an arbitrage opportunity in correcting a market inefficiency, usually of the form "there is a market for X, X could be provided, but X is not currently provided."
That seems to stretch the meaning of market inefficiency. Is the lack of unlimited free energy an inefficiency in a market? Because an entrepreneur who achieves that is going to do pretty well. I’d say that would be creating value not optimizing market efficiency.
Yeah, and arbitrage. Arbitrage is exploiting the difference in prices of the same asset between two markets. Arbitrage is also risk-free or darn close to it. Entrepreneurship is anything but. Arbitrage is not "gee this product doesn't exist, I'll start a company and invent it and manufacture it and sell it" ...
In perfectly efficient markets, arbitrage gets paid exactly as much as it costs to discover and execute the arbitrage. Entrepreneurs would still exist, they would just be ambivalent between finding new arbitrage opportunities and seeking market-rate employment for their skills in finding arbitrage.
Seems like t is a very critical variable then. For example, you could imagine a particular market is "perfectly" efficient at the moment (however you want to define the boundaries of a particular market), and there is no opportunity. But then a completely unrelated company or university makes a fundemental advancement in materials science that fundamentally changes the landscape. An exogenous shock in other words.
In a certain sense I guess this is why every anti-trust suit fundamentally comes down to defining the market bubble more than anything else.
Ok so the market is perfect... but I exist and have labor to provide. Am I not naturally an "inefficiency" by not participating in the market? Therefore by participating alone, I have a new wrinkle to work from?
Keeping in mind the mistake in the HN title (should be "P != NP"), the interesting part of the abstract is this:
> Combined with Maymin (2011), who proved that market efficiency requires P = NP, this yields a fundamental impossibility: markets can be informationally efficient or competitive, but not both.
The idea that markets are an optimizing algorithm is kinda old already, and well established.
Both papers seem to be jokes about it, based on complete caricatures of competitiveness and efficiency. It's kinda like a recent paper that was posted here proving "general intelligence" impossible while ignoring that humans exist.
Except Maymin 2011 fails to even establish that his narrow definition of "markets are efficient" (specifically, finding a profitable technical analysis) is actually in NP.
The argument is not that you get better prices, it’s that you get accurate prices.
First, this definition has always been circular: what’s the most accurate price? The one the market comes up with. More market, more accuracy!
Second, there is never any reconciliation of the costs society is saddled with in order to chase arbitrarily more accurate prices, the most obvious of which is the massive quantity of fat skimmed off by the financial services sector.
Third, as an index investor, I more or less couldn’t care less. This hyperfixation on accuracy only really matters to people who are actively trading, which is already a fool’s game.
> First, this definition has always been circular: what’s the most accurate price? The one the market comes up with. More market, more accuracy!
Market makers and HFT don't determine price: price is usually purely determined by the net inflows and outflows as decided by humans. MMs just smooth it out over time so everyone gets good pricing at the time and in the size they want it.
> Second, there is never any reconciliation of the costs society is saddled with in order to chase arbitrarily more accurate prices
By definition market makers are earning a fraction of the price improvement they provide, ergo the costs to society have to be less that the benefits for better pricing for the companies to stay in business!
> Third, as an index investor, I more or less couldn’t care less
As an index investor you should absolutely care! How do you think you are able to buy into the fund at a reasonable price? And then how do you think the fund is able to rebalance without transaction costs destroying performance long-term?
"Third, I propose computational antitrust: the principle that market complexity itself is a competitive safeguard, and that regulators should consider computational difficulty as a design parameter."
This "should" is doing a lot of work here. The paper is mainly about a game-theoretic model allegedly corresponding to real markets, but establishing what regulators ought to do requires far more rationale than mere math. It requires a bridge from "is" to "ought." It reminds me of Hume's warning about this kind of non-sequitur:
"In every system of morality, which I have hitherto met with, I have always remarked, that the author proceeds for some time in the ordinary ways of reasoning, ... ; when all of a sudden I am surprised to find, that instead of the usual copulations of propositions, is, and is not, I meet with no proposition that is not connected with an ought, or an ought not. This change is imperceptible; but is however, of the last consequence."
> the collusion detection problem is computationally infeasible for markets satisfying a natural instance-hardness condition on their demand structure, rendering punishment threats non-credible and collusion unstable.
And yet we’ve clearly observed stable price fixing cartels. Maybe the word “unstable” means too much or the game theory model used doesn’t describe the real world accurately. When theory is contradicted by the evidence, it would be wise to consider the theory is flawed.
Game theory here is applied to two fundamental market theorems. It’s a way to analyze the validity of those assumptions, rather than to build a new model. Empirical evidence to the contrary is expected given mutually inconsistent premises, which is what the author’s results predict. The author has simply used game theory math to disprove economist math.
Nuclear has been in maintenance mode for so long that there are doubts about if anyone could right now detonate one without shitting their pants on account if it would even go off.
I suspect this is a standard mathematical “it is computationally impossible to do this in the general case despite it being entirely feasible in many cases”.
The author looks credible:
https://philipmaymin.com/about-philip
Thank you for sharing this on HN.
--
To the mods: The title needs to be edited to replace the equal sign with not-equal.
But, both free markets and supply/demand are useful enough concepts to talk loosely about processes to understand the interest that I'll enjoy digging into this.
The behavioral economics/Freakonomics thing was like "Hey, here's this thing that might if you squint real hard fall outside of efficient market theory" and then for a decade people took that to mean that that the base concepts were worthless, which was a severe overcorrection from people that didn't understand economics.
Most filters are to avoid sensational titles, AFAIK.
BASIC[1] came out in 1964, and Pascal[2] came out in 1970.
---
[1] https://en.wikipedia.org/wiki/Dartmouth_BASIC
[2] https://en.wikipedia.org/wiki/Pascal_(programming_language)
Markets are efficient if and only if P = NP https://arxiv.org/abs/1002.2284
:)
> Combined with Maymin (2011), who proved that market efficiency requires P = NP, this yields a fundamental impossibility: markets can be informationally efficient or competitive, but not both.
"Markets are competitive if and only if P != NP"
Seems that HN's auto-headline rewriting in this case has made a critical error :)
>Artificial intelligence, by expanding firms' computational capabilities, is pushing markets from the competitive regime toward the collusive regime, explaining the empirical emergence of algorithmic collusion without explicit coordination.
I have to dig more into the paper but I don't see how this follows, except in the most straightforward way. Basically, if everyone uses the same methods to derive price, of course there will be "collusion", or in other words, everyone will have the same price. But this doesn't seem like a result of compute per se, but simply better communication networks and information flows. You could have gotten the same result in medieval England by having everyone post their selling prices on the town square board.
Again, I haven't dug into the paper yet, but it seems like what really matters for firms is "compute"/$ (if the "compute" is an LLM or an assistant that has to go walk the 10 minutes down to the square makes little difference)
Edit: Isn't another implication of this, that increased compute -> collusion imply that increased compute -> communism becomes feasible?
I think this goes to my point above though, the primary problem preventing fully automated luxury communism isn't compute per se, but actually observing the information flows to make it possible. Capitalism famously solves this information problem through the pricing mechanism. So in effect, he's arguing that extra compute makes information gathering more efficient, and at the limit you get perfect information. Which, yeah, I guess so. Assuming everything can be perfectly measured, even theoretically.
YieldStar was technically an “AI” product, but I don’t really think the computational abilities were what enabled the collusion. RealPage’s employees (according to the DoJ[0]) would actively monitor whether companies were following their pricing recommendations and call up companies that defected. And the software itself used dark patterns to make it easier to simply follow the YieldStar pricing suggestions, rather than set a lower rental rate and be more competitive. The algorithmic pricing I think did allow people to launder their own judgement and simple “trust the process” in a way that in the past would have required knowing complicity with the cartel, but I don’t think it required substantial compute capacity.
(This isn’t a comment on the paper by the way, which I glanced at but did not have the background knowledge to fully comprehend)
[0] See the section labeled “RealPage Uses Multiple Mechanisms To Increase Compliance With Price Recommendations” https://www.federalregister.gov/documents/2026/01/21/2026-01...
You could imagine the exact same scheme without the use of a computer.
“Markets are competitive if and only if P ≠ NP”
It’s 2026, people, you don't have to use crude ASCII approximations of mathematical symbols any more.
> If P != NP, the collusion detection problem is computationally infeasible for markets satisfying a natural instance-hardness condition on their demand structure, rendering punishment threats non-credible and collusion unstable.
...and then from the paper:
> Stigler (1964) famously argued that the “chief difficulty” of collusion is detecting “secret price-cutting.”
The thing is that Stigler's insight is far from proven, and indeed, the primary difficulty in collusion is often not the detection of defection. Firms know they're being undercut all the time. The problem is that very often, there is nothing they can do about it. Markets are specifically structured as firm-to-firm transactions, where competing firms have no leverage over what your firm can do or what sort of transactions you can conduct, and as long as this condition holds it doesn't matter if you know that a competitor is fucking you over, you can't do anything about it.
I'd argue that the increase in collusion and anticompetitive behavior lately is because these conditions increasingly don't hold. When you intersperse another party in the transaction, eg. a regulatory agency, permitting body, or exclusive distribution deal, you introduce a leverage point for incumbents to punish competitors who choose to undercut them.
Examples of the former: cutting prices yourself; increasing product quality; differentiating yourself; spending more on advertising to get the word out about your product.
Examples of the latter: crafting exclusive deals with your distributors to prevent your competitors from getting shelf space; politically influencing regulatory bodies to declare your competitors' existence illegal; making direct agreements with the leadership of opposing firms to not drop prices or hike wages; assassinating, extorting, or kidnapping rival business leaders.
Basically it comes down to "control yourself, because you cannot control others". In a functioning market, you have no control over what rival firms. Your only legal reaction to competition is to improve your own offering until it is the best it can be. In pathological markets where the assumption is (as in the paper) that you can punish rivals for not colluding, you actively make your competitor's offering worse.
The term ‘perfect information’ is a bit of a mirage, and has been shown to be impossible in physics (uncertainty principle).
What really matters is information advantage: Does your inexact expected value function consistently beat others’ calculations in the market. Here, the true value - value really is just a word and is dependent on people - is irrelevant.
Really interesting conclusion, but I can't help but feel this is overly reductive, as stated. Surely market efficiency is a sliding scale and so is market competitiveness.
Okay, so a perfectly competitive market cannot also be a perfectly efficient market. Interesting! But I'm confused about how this may work when efficiency and competitiveness are a sliding scale. Should we think of this as one axis (with a spectrum from efficiency to competitiveness) or as two separate axes that just happen to have an exclusive relationship between their extremes?
NP-Completeness is the norm, not the exception. Any system that's complex enough is almost surely NP-Complete. For similar reasons, Turing Machine Equivalence is also the norm, not the exception.
These results are interesting but not unexpected. A more interesting question is under what conditions is the problem difficult to find solutions for. Many NP-Complete instance ensembles turn out to effectively have polynomial time solutions (3-SAT w/ uniform clause variable choice, Hamilton Cycles in Erdos-Renyi random graphs), so proving NP-Completeness is not a death knell for approximation.
NP problems gets solved with heuristics every day.
You don't know that!
You do realise that you'll never work in this town again? \s :)
If P!=NP then it is arbitrarily smaller, for the same reason that e^x > Cx^N for any constants C and N, as long as x grows big enough. There is no epsilon in that can overcome that, no matter how big you make it, because x will eventually dominate the equation.
There are a lot of cases where pragmatically x remains small enough that it doesn't matter, and a P algorithm will give you an answer more quickly. (For the same reason I only ever write bubble sorts: I would only write my own at all if I knew that the list would never be bigger than 10. Even then it's only when using the library is too much trouble for some reason.)
But we care about P and NP when the number can potentially be very, very large.
So I'm not talking about the number of steps needed to prove optimality with a correct P algorithm versus an exponential one.
I'm only talking about how this applies to the efficient market hypothesis.
No expensive analysis is needed. Beating them just requires (1) having disproportionate capital to others, and (2) having disproportionate control (and therefore know in advance) whatever the markets are pricing.
It helps that there are enough believers (in the religious/cult sense) that markets are 100% efficient, that they will deny that any of that is happening - even while they lose their shirts to those who are doing the market manipulation.
Now seriously, I wonder if AI collusion/use in investments would add to the market inefficiency and create opportunities for observing investors.
Wow... this is quite fascinating. It has been theorized for a little while that widespread AI could form accidental trusts due to optimization around one-another. This seems to be taking it a step further and arguing that if P!=NP, then markets are certain to trend towards collusive.
The annoying part is that, as the same Adam Smith says, regulating industries would end up enforcing such assemblies, reinforcing the problem... after all, industries can share information via the market itself...
And proposed solutions end up being controversial: employees ownership, open source, paying taxes over stocks ownership... or just hoping that colluders will be broken by a randomly ocurring incumbent...
- P means polynomial - NP means nondeterministic polynomial
Roughly polynomial (P) represents the upper bound of the cost to verify, and the polynomial characterization says that given a problem with a certain input (e.g. the input can be the number of training examples in ML training set or the number of constraints/conditions in a general problem— e.g. all the places you want to visit on your next trip, given many "wants" in a group)
When the cost is polynomial relative to the input size it means it can only be finitely larger than the input - that's a characteristic of the polynomial which is just a finite sum of powers of x (the input).
When the cost is nondeterministic polynomial, one way to think about it in what is called a nondeterministic Turing machine. The nondeterministic part refers to the "states" that the machines can transition to from any current state. When the transition can happen to more than one state, we say it's non-deterministic— and can imagine it's determined by some probability.
The general assumption is that polynomial (P) is easier than nondeterministic polnomial (NP). This isn't necessarily the case as there can be arbitrarily large finite numbers (making P solutions intractible)
The P vs NP problem is one of the main open problems and generally considered a crank magnet and general confusion. For a good (likely the best) resource see https://scottaaronson.blog/
A problem in P can be solved in polynomial time - the computation required grows relatively slowly as the input size increases. Like sorting a list of numbers.
A problem in NP requires exponential time or greater, but a proposed solution can be verified quickly. For example, checking a completed Sudoku puzzle.
It is believed but unproven that all problems in NP are NOT in P.
Every single problem in P is in NP. What is believed but unproven is that some problems in NP are not in P.
I feel like I have not yet found the proper explanation. Or I'm just too dense to get it.
However, the reason all NP algos are part of the same category is because you can solve any problem in NP by switching the problem into another problem in the same category and solving that. For example, you can turn sudoku into a graph coloring problem, which is also NP. You can turn sorting (P) into something like balancing a tree, which is also P.
The major question is "is there any algorithm that would allow us to change some NP problems into P problems, solve it, then use it for the original problem". E.g. could we take graph coloring and turn it into sorting a list of numbers?
So basically, if there is any way to bridge the two, then it might mean every NP problem is actually solvable by a P algorithm, under some transformation. This would be immense because it would completely change the way we solve those algorithms and greatly reduce compute costs.
While this seems far-fetched, realize that there are some problems that seem extremely expensive if done the naiive way, but are actually solvable in P. For example, you _could_ write an exponential sorting algo (try every element in every position), but clever people found a way to make it efficient (P). So its possible we just need the right algo to completely change the landscape of computing.
However, as you say, its almost self-evidently true that P != NP, but has never been proven so (to do so, we need to prove that no such algorithm can exist). But clearly, solving an exponentially complex problem using a O(log n) algo would be remarkable.
To take a concrete example, currently the best algos to exhaustively check a board game like chess or go are exponential (NP). Its easy to verify the winner, but its exponential to enumerate every possible move (e.g. 80^turns states). If we found a polynomial way to solve this (even by converting to something simplified), then it would mean we could exhaustively search chess polynomial to the number of moves (e.g. turns^100). This changes it from "cannot be done in the lifespan of universe" to "its possible with a powerful computer in measurable time". We already use heuristics and estimates to explore the exponential space in efficient time, so if we had a polynomial algo chess, markets, optimization, and other NP problems would be extremely efficient to solve.
This video explains in detail: https://www.youtube.com/watch?v=YX40hbAHx3s
In short, P means Polynomial time (i.e. markets can solve computation problems efficiently) and NP means Non-Deterministic Polynomial time (i.e. markets can verify solutions of computation problems efficiently but solutions are found by luck).
If P != NP, it means luck CANNOT be engineered and markets are competitive.
P complexity class
https://en.wikipedia.org/wiki/P_(complexity)
NP complexity class
https://en.wikipedia.org/wiki/NP_(complexity)
P vs NP question
https://en.wikipedia.org/wiki/P_versus_NP_problem
one such class is the Polynomial class, or P, where the time to solution is some fixed exponent of N (like N^2, or 3).
the next big step is NP, which require a polynomial number of nondeterministic steps, whose solution can only be verified in polynomial time. usually solutions to NP problems are exponential in cost with respect to N (like 2^N), but thats not part of their definition.
problems in NP are generally identified by mapping them into a well known problem known to be in NP, where the mapping has to occur in polynomial time.
its an open question as to whether NP as a class can actually be solved in P time, but most people doubt that that is really the case.
In a certain sense I guess this is why every anti-trust suit fundamentally comes down to defining the market bubble more than anything else.
> Combined with Maymin (2011), who proved that market efficiency requires P = NP, this yields a fundamental impossibility: markets can be informationally efficient or competitive, but not both.
(Note that Maymin is the author of both papers.)
Both papers seem to be jokes about it, based on complete caricatures of competitiveness and efficiency. It's kinda like a recent paper that was posted here proving "general intelligence" impossible while ignoring that humans exist.
Want to buy/sell a stock?
Humans need to manually submit in the system.
First, this definition has always been circular: what’s the most accurate price? The one the market comes up with. More market, more accuracy!
Second, there is never any reconciliation of the costs society is saddled with in order to chase arbitrarily more accurate prices, the most obvious of which is the massive quantity of fat skimmed off by the financial services sector.
Third, as an index investor, I more or less couldn’t care less. This hyperfixation on accuracy only really matters to people who are actively trading, which is already a fool’s game.
Market makers and HFT don't determine price: price is usually purely determined by the net inflows and outflows as decided by humans. MMs just smooth it out over time so everyone gets good pricing at the time and in the size they want it.
> Second, there is never any reconciliation of the costs society is saddled with in order to chase arbitrarily more accurate prices
By definition market makers are earning a fraction of the price improvement they provide, ergo the costs to society have to be less that the benefits for better pricing for the companies to stay in business!
> Third, as an index investor, I more or less couldn’t care less
As an index investor you should absolutely care! How do you think you are able to buy into the fund at a reasonable price? And then how do you think the fund is able to rebalance without transaction costs destroying performance long-term?
This "should" is doing a lot of work here. The paper is mainly about a game-theoretic model allegedly corresponding to real markets, but establishing what regulators ought to do requires far more rationale than mere math. It requires a bridge from "is" to "ought." It reminds me of Hume's warning about this kind of non-sequitur:
"In every system of morality, which I have hitherto met with, I have always remarked, that the author proceeds for some time in the ordinary ways of reasoning, ... ; when all of a sudden I am surprised to find, that instead of the usual copulations of propositions, is, and is not, I meet with no proposition that is not connected with an ought, or an ought not. This change is imperceptible; but is however, of the last consequence."
And yet we’ve clearly observed stable price fixing cartels. Maybe the word “unstable” means too much or the game theory model used doesn’t describe the real world accurately. When theory is contradicted by the evidence, it would be wise to consider the theory is flawed.
Nuclear has been in maintenance mode for so long that there are doubts about if anyone could right now detonate one without shitting their pants on account if it would even go off.