a16z Discusses the Future of On-Chain Markets: Why Predictability Is the Key Foundation?

Authored by: Pranav Garimidi, Joachim Neu, and Max Resnick, a16z crypto

Compiled by: Glendon, Techub News

Blockchain can now confidently claim it possesses the capabilities needed to compete with existing financial infrastructure. Current production systems can handle tens of thousands of transactions per second, with potential for orders of magnitude improvement in the future.

However, beyond raw throughput, financial applications also require predictability. When a transaction is sent—whether it is a trade, auction bid, or options exercise—the normal operation of a financial system relies on a reliable guarantee of when the transaction will be included on-chain. If transactions face unpredictable delays (whether from malicious attacks or random events), many applications will become unusable. To make on-chain financial applications competitive, blockchains must provide short-term inclusion guarantees: that is, once a valid transaction is submitted to the network, there is a promise it will be included as quickly as possible.

For example, consider on-chain order books. An efficient order book requires market makers to provide liquidity through continuous order placements. The key issue faced by market makers is to minimize the bid-ask spread while avoiding adverse selection due to quotes diverging from overall market levels. To do this, market makers must continually update their orders to reflect changes in the external world. For instance, if a statement from the Federal Reserve causes asset prices to spike, market makers need to respond immediately and update their orders to the new price. In this case, if the transaction for updating the market maker’s order does not get included on-chain immediately, arbitrageurs will execute their trades at outdated prices, resulting in losses for the market makers. Consequently, market makers are forced to widen the spread to reduce their risk exposure to such events, thus diminishing the competitiveness of on-chain trading venues.

Predictable transaction inclusion mechanisms provide market makers with strong assurances, enabling them to respond quickly to off-chain events and maintain efficient operation of the on-chain market.

Current State versus Future Demand

Currently, existing blockchains only provide finality guarantees that take effect within a few seconds. While these guarantees are sufficient for applications like payments, they are too weak for the multitude of financial applications that require real-time responsiveness. Taking order books as an example: for market makers, if arbitrageurs can submit their trades to the blockchain before them, the guarantee of “being included within the next few seconds” is meaningless. In the absence of strong inclusion guarantees, market makers can only respond to increased adverse selection risk by widening the spread, thus offering users worse prices. This, in turn, makes on-chain trading less attractive compared to other venues that can provide stronger guarantees.

To enable blockchain to truly realize its vision as modern capital market infrastructure, developers must address these issues, allowing high-value applications like order books to thrive.

How difficult is it to achieve predictability?

Enhancing existing blockchains’ transaction inclusion guarantees to support financial-grade applications is a challenging task. Currently, some protocols rely on a single node (the “leader”) to determine the order of transaction inclusion at any given time. While this simplifies the engineering implementation of high-performance chains, it also introduces potential economic bottlenecks, allowing the leader to extract value. Typically, during a window of time when a node acts as a leader, it has complete control over which transactions are included in the block.

For any blockchain handling financial activities, the leader holds a privileged position. If this single leader decides not to include a transaction, the only remedy is to wait for the next leader willing to include that transaction. In an unpermissioned network, the leader has the incentive to profit from this, often referred to as MEV (maximal extractable value). The impact of MEV extends beyond actions like “sandwiching AMM trades.” Even if the leader can only delay inclusion of a transaction by a few milliseconds, it can still extract substantial profits and reduce the efficiency of underlying applications. If the order book only prioritizes certain traders’ transactions, then all other traders will be at a disadvantage. In the worst-case scenario, the leader’s behavior can even alienate traders, causing them to leave the platform altogether.

Imagine a scenario where interest rates rise and the price of ETH drops by 5% immediately. All market makers on the order book will rush to cancel their orders and place new ones at the new price. Meanwhile, all arbitrageurs will submit orders to sell ETH at the prices of the old orders. If that order book operates on a protocol controlled by a single leader, that leader will have enormous power. The leader could directly prevent all market makers from canceling their orders, allowing arbitrageurs to realize significant profits. Alternatively, the leader could choose not to directly block cancellation but delay it until the arbitrageurs have completed their trades. The leader could even insert their own arbitrage trades to exploit the price differences completely.

The conflict of two ideal demands: the necessity of censorship resistance and information hiding

Faced with these advantages, it becomes unprofitable for market makers to participate; they risk being exploited whenever prices change. The crux of the problem lies in the two excessive privileges held by market leaders: 1) the leader can censor others’ transactions; 2) the leader can view others’ transactions and submit their own in response. Either of these issues could have catastrophic consequences.

Example

We can illustrate this problem with the following example. Consider an auction with two bidders, Alice and Bob, where Bob happens to be the “leader” of the block containing the auction. (There are only two bidders for illustrative purposes; the same reasoning applies regardless of the number of bidders.)

The auction accepts bids from the start to the end of the block production, suppose from time t=0 to t=1. Alice submits a bid bA at time tA, and Bob submits a bid bB at time tB > tA. Since Bob is the leader of the block, he can always ensure he bids last. Alice and Bob also have a continuously updating source of asset price information to read (e.g., the mid-price from a centralized exchange). At time t, suppose that price is pt. We assume that at time t, both expect the asset price at the end of the auction (t=1) to always be pt. That is, at any given moment, Alice and Bob expect the asset price at the end of the auction to equal the price they currently see. The auction rules are simple: the bidder with the higher bid wins the auction and pays their bid.

The necessity of censorship resistance

Now consider what happens when Bob can leverage his advantage as a leader. If Bob can block Alice’s bid, the auction will obviously fail. Bob only needs to submit an arbitrarily small amount and ensure he wins the auction, as there are no other bids. This will result in the auction closing with negligible revenue.

The need for information hiding

A more complex scenario arises when Bob cannot directly block Alice’s bid, but can still see Alice’s bid before placing his own. In this case, Bob has a simple strategy: when he bids, he simply checks if ptB is greater than bA. If it is, Bob submits a bid slightly higher than bA; if not, Bob does not place a bid at all. By adopting this strategy, Bob causes Alice to suffer from adverse selection. Alice will only win when price movements make her bid ultimately higher than the expected value of the asset. Whenever Alice wins the auction, she expects to lose money, making it better for her not to participate. With all competitive bidders gone, Bob only needs to submit an arbitrarily small amount again to win, resulting in the auction effectively generating zero revenue.

The key conclusion here is that the duration of the auction does not matter. As long as Bob can censor Alice’s bid or see Alice’s bid before placing his own, the auction is doomed to fail.

The principles in this example apply equally to any high-frequency asset trading scenario, whether spot trading, perpetual contracts, or derivatives exchanges: if there is a leader like Bob with enormous power, they can cause the market to collapse entirely. To ensure that on-chain products serving these use cases can continue to operate, they must never grant leaders this power.

How do these issues manifest in today’s practice?

The above description paints a grim picture for on-chain transactions in any unpermissioned single-leader protocol. However, why do decentralized exchanges (DEXs) on many single-leader protocols still see healthy trading volumes?

In reality, a combination of two forces can offset the issues mentioned above:

Leaders do not fully exploit their economic power because they often have a significant vested interest in the success of the underlying ecosystem;

Applications have constructed corresponding evasion mechanisms to avoid being affected by these issues.

While these two factors have allowed DeFi to continue to thrive, they are insufficient for on-chain markets to truly compete with off-chain markets in the long run.

To become a leader on a blockchain with significant economic activity, one must hold a large amount of staked assets. Therefore, leaders either hold a substantial amount of their own stake or have enough reputation to attract other token holders to delegate their staking. In either case, large node operators are typically well-reputed entities. Beyond reputation, this stake also means operators have an economic incentive to ensure their relevant chains perform well. For this reason, we have not generally seen leaders fully exploit their market power as described above—but that does not mean these issues do not exist.

Firstly, relying on node operators’ goodwill through social pressure and long-term incentives is not a solid foundation for the future of finance. As the scale of on-chain financial activities increases, the potential profits for leaders also rise accordingly. The larger this potential profit, the harder it becomes to constrain leaders’ behavior socially, causing them to act against their own short-term interests.

Secondly, the degree to which leaders exploit their market power ranges from benign to market-destroying. Node operators can unilaterally push to leverage their power for higher profits. When some operators test the boundaries of acceptable behavior, others quickly follow suit. A single node’s actions may seem insignificant, but when all nodes make changes, the impact becomes evident.

The best example of this phenomenon might be the “time game”: when leaders try to announce blocks as late as possible while ensuring the blocks are valid for the protocol, they can earn higher rewards. This can lead to extended block generation times, and if leaders are too aggressive, certain blocks may even be skipped. Despite the profitability of these strategies being well-known, leaders choose not to engage in them for the sake of maintaining good order on the chain.

However, this social balance is extremely fragile. Once a node operator begins to exploit these strategies for higher rewards without facing any penalties, other operators will quickly follow suit. The time game is just one example of how leaders can increase profits without fully exploiting their market power. Leaders can take many other actions to enhance their own gains, often at the expense of applications’ interests. Viewed in isolation, these actions may be effective for applications, but ultimately the scale will reach a critical point where on-chain costs will exceed benefits.

Another factor that enables DeFi to operate is that applications move important logic off-chain, only posting results on-chain. For instance, any protocol requiring rapid auctions will execute off-chain. These applications typically run their necessary mechanisms on a set of permissioned nodes to avoid conflicts with malicious leaders.

For example, UniswapX runs its Dutch auctions off-chain to complete transactions on the Ethereum mainnet, and Cowswap similarly operates its batch auctions off-chain. While this is feasible for applications, it places the underlying architecture and the value propositions built on-chain in a precarious position. If the execution logic of applications resides off-chain, the underlying architecture can only be used for settlement. One of DeFi’s most powerful selling points is composability. When all executions occur off-chain, these applications essentially exist in isolated environments. Relying on off-chain execution also adds new assumptions to the trust model of these applications. In addition to relying on the underlying chain to remain active, these off-chain infrastructures must also operate correctly for the applications to function.

How to achieve predictability

To address these issues, protocols need to satisfy two properties: consistent transaction inclusion and ordering rules, and privacy of transactions before confirmation.

Primary Requirement: Censorship Resistance

We summarize the first property as short-term censorship resistance. If a protocol can guarantee that any transaction reaching a valid node will be included in the next possible block, then that protocol has short-term censorship resistance.

More accurately, we assume the protocol runs on a fixed clock cycle, where each block is generated at fixed intervals, such as one block every 100 milliseconds. We want to guarantee that if a transaction arrives at a valid node at t=250 milliseconds, it will be included in the block generated at t=300 milliseconds. Adversaries should not be able to selectively include only certain transactions they hear while ignoring others. The crux of this definition is that users and applications should have an extremely reliable way to receive transactions at any point in time. There should be no situation where a single node drops a transaction due to malicious behavior or simple operational failure, resulting in the transaction not being completed.

While this definition requires guaranteeing that all transactions reaching valid nodes will be included, the overhead of actually achieving this might be too high. The key is that the protocol must be robust enough that the entry points for on-chain transactions can operate in an extremely predictable and understandable manner. An unpermissioned single-leader protocol clearly does not meet this characteristic, as if the single leader is in a “Byzantine state” at any moment, transactions cannot be included on-chain in any other way. However, even a set of four nodes that can guarantee transaction inclusion in each time slot greatly increases the number of options for users and applications to land transactions. To ensure applications can operate reliably, sacrificing some performance is worthwhile. Finding the right balance between robustness and performance still requires much work, but the guarantees provided by existing protocols are far from sufficient.

Given that the protocol can guarantee inclusion and ensure data integrity, the sorting mechanism is somewhat acquired for free. The protocol is free to use any deterministic sorting rule to ensure consistency in ordering. The simplest solution is to sort by priority fees or allow applications to flexibly order transactions that interact with their state. The best way to sort transactions remains an active area of research, but in any case, the sorting rule only makes sense when there are transactions to be sorted.

Requirement Two: Hiding

After achieving short-term censorship resistance, the next most important property that the protocol needs to provide is a form of privacy we call “hiding.”

Hiding: No party, other than the node to which the transaction is submitted, can know any information about the transaction before it is confirmed by the protocol.

Protocols with hiding may allow nodes to see all transactions submitted to them in plaintext but require that the rest of the protocol keeps the contents of the transactions completely hidden until consensus is achieved and the order in the final log is determined. For example, the protocol might use time-lock encryption to keep the content of the entire block hidden until a certain deadline; or the protocol might use threshold encryption, such that the block is decrypted only after unanimous confirmation of its irreversibility by a committee.

This means that a node could tamper with any transaction information submitted to it, but the rest of the protocol does not know the specific details of the transaction contents before consensus is reached. By the time the transaction information is revealed to other nodes in the network, the transaction has already been sorted and confirmed, so no other nodes can execute it in advance. To make this definition meaningful, it implies that multiple nodes can process the same transaction within any given timeframe.

We have opted to forgo stronger schemes (i.e., where only users can know transaction information before confirmation, such as encrypted memory pools) because the protocol needs some steps to filter out spam transactions. If the contents of transactions are completely hidden from the entire network, then the network cannot distinguish between spam transactions and valid transactions. The only solution is to leak some unhidden metadata in the transaction, such as the address of the payer that will incur fees regardless of whether the transaction is valid. However, this metadata may leak enough information for adversaries to take advantage. Therefore, we prefer to let a single node fully possess transaction details while preventing other nodes in the network from obtaining any information about them. But this means that to make this feature useful, users must have at least one honest node as an entry point to successfully submit transactions in every transaction time slot.

Conclusion

A protocol that combines short-term censorship resistance and privacy protection provides an ideal foundation for building financial applications. Returning to our earlier example of hosting auctions on-chain, these two characteristics directly address the issues that Bob could cause leading to market collapse. Bob can neither censor Alice’s bid nor leverage Alice’s bid to influence his own, which precisely resolves Problems 1 and 2 from our previous example.

With resistance to short-term content censorship, any operation submitting a transaction (whether a trade or an auction bid) can be assured of immediate inclusion. Market makers can alter their orders; bidders can swiftly place bids; liquidation operations can be completed efficiently. Users can be confident that any actions they take will be executed immediately. This, in turn, will enable the next generation of low-latency real-world financial applications to be built entirely on blockchain. To allow blockchain to truly compete with existing financial infrastructure, even surpassing its performance, we need to address far more than throughput issues.