Distinguishing MEV from Expected Noise

tl;dr

Skip Protocol updated their MEV dashboard to show the $ value of MEV that is likely extracted by each validator, as opposed to the orderbook discrepancy that can come from MEV or network noise as in the initial version.
Previously, the dashboard showed cumulative orderbook discrepancy and highlighted the dishonest behavior of the validator run by the research team at dYdX Trading Inc. (”dYdX Trading”), which had 4x higher orderbook discrepancies than the rest of the validator set. However, factors like transaction gossip delays and various validator setups can lead to orderbook discrepancies. This means that even when validators are honest, the dashboard showed nonzero values that increased slowly over time for all validators, and viewers should interpret the data with care.
In this version of the dashboard, Skip’s nodes compare their observations of the block proposer’s orderbook discrepancies against the orderbook discrepancies of each other in order to determine whether the block proposer’s value is unexpectedly high and therefore likely indicates MEV activity. Only if more than 2/3 of Skip’s nodes conclude that the block proposer has extracted MEV, will their median orderbook discrepancy be added to the cumulative view.
dYdX Trading is continuing to explore further mechanisms and protocol level designs to deter MEV extraction.

Introduction

MEV (Maximal Extractable Value) extraction by validators poses a significant threat to the integrity of a fair and transparent trading environment. Validators extracting MEV hurt users by executing orders at suboptimal prices and ignoring cancellations for profit. In order to discourage such dishonest behavior, we partnered with Skip Protocol to create a public dashboard that highlights how much MEV each validator may be extracting.

The dashboard compares the trades proposed by the validators and the honest nodes run by Skip Protocol, which do not propose trades but still are part of the network, to calculate a metric termed orderbook discrepancy. Orderbook discrepancy represents how different are two sets of proposed trades, and a high discrepancy from the honest nodes’ trades can indicate MEV extraction. Using this data, potential delegators can make informed decisions and the relevant governance community can punish bad actors if necessary.

Skip Protocol’s Dashboard Exposes the Bad Validator

During the public testnet phase, the dashboard effectively caught the “bad validator” set up by the dYdX Trading research team. Whenever it proposed a block, the dydx-research validator used a common strategy known as “sandwiching” and filled orders at their limit price even if better prices were available to collect the difference for profit. This dishonest behavior is evident in the screenshot below, which shows the cumulative orderbook discrepancy of the dydx-research validator to be 60% of their proposed volume in the past 1000 blocks, which is 3 times that of the second highest validator.

However, this earlier version of the dashboard displayed orderbook discrepancies that can come from regular network latencies as well as MEV. For instance, suppose there are 2 market buy orders that are placed around the same time from two separate front ends. Call them A and B. Skip’s nodes close to these front ends may see A before B, and match them with other orders on the book C and D, respectively. On the other hand, a validator in New Zealand could see B before A since the orders may need to be travel longer distance through more nodes in the network. So this validator may match A with D and B with C, leading to a nonzero orderbook discrepancy measured by Skip’s nodes even if it is not extracting MEV. Although Skip’s nodes are distributed globally to ensure a fair coverage, it is important to avoid results that cast a shadow on honest validators.

More precise detection of MEV

The new version of the dashboard displays the potential dollar value amount of MEV extracted by summing the orderbook discrepancies only for the blocks when they exceed an acceptable level. This design makes it easier to identify the bad actors against noise. Specifically, the research validator's cumulative value increases, while others’ values remain close to zero in the new dashboard, in contrast with the previous version that showed everyone's orderbook discrepancy values growing due to network noise, but at rates lower than the research validator.

In addition to calculating orderbook discrepancies, Skip’s nodes determine whether the block proposer has extracted MEV by comparing the observed value to the expected level. The individual node concludes that the block proposer is dishonest if the block proposer’s matched trades results in an orderbook discrepancy that is more than 2 standard deviations above the average discrepancy. Only if more than 2/3 of the honest nodes independently conclude that the block proposer has extracted MEV in this block, the median of observed value is added to the cumulative view.

To calculate the average and standard deviations of the discrepancies, the individual node compares its matched trades against the block proposer’s matched trades as well as what all other nodes would have proposed as their matched trades. Consider the example from above: even though some of Skip’s nodes that saw A before B would match A with C and B with D, Skip’s other nodes closer to New Zealand may have matched A with D and B with C like the validator. Therefore, the nodes will see orderbook discrepancies amongst themselves and the New Zealand validator’s orderbook discrepancy will not be considered as MEV.

In other words, if an order needs to travel large geographical distance to reach the rest of the network, the nodes and block proposers will see them with similar latency, which leads to higher average, standard deviation, and therefore the threshold of discrepancy to be deemed as MEV. This ensures that the block proposer’s orderbook discrepancy will not stand out solely due to an order with a relatively high latency to reach the block proposer. Based on this design, if the block proposer’s matched trades lead to unusually high orderbook discrepancies compared to other nodes’ matched trades, this discrepancy is likely due to dishonest behavior rather than network noise. With the cumulative view only counting values that are deemed to be MEV by a super majority of Skip’s nodes, this iteration of the dashboard detects dishonest behavior more precisely.

What’s next?

We hope that this version of the dashboard sufficiently highlights any dishonest behavior with respect to MEV. dYdX Trading is committed to promoting a fair and transparent trading environment: we plan to continue experimenting with a “bad validator” on the public testnet to understand the effectiveness of the dashboard and refine the approach as necessary. In addition to the current approach, protocol-level mechanisms to prevent MEV extraction leveraging technologies like Vote Extensions and Trusted Execution Environment are being actively researched. Stay tuned for more updates on our progress and continued initiatives against MEV extraction.

Appendix

See calculation details here.

Legitimacy and Disclaimer

Crypto-assets can be highly volatile and trading crypto-assets involves risk of loss, particularly when using leverage. Investment into crypto-assets may not be regulated and may not be adequate for retail investors. Do your own research and due diligence before engaging in any activity involving crypto-assets.

dYdX is a decentralised, disintermediated and permissionless protocol, and is not available in the U.S. or to U.S. persons as well as in other restricted jurisdictions. The dYdX Foundation does not operate or participate in the operation of any component of the dYdX Chain’s infrastructure.

The dYdX Foundation’s purpose is to support the current implementation and any future implementations of the dYdX protocol and to foster community-driven growth in the dYdX ecosystem.

The dYdX Chain software is open-source software to be used or implemented by any party in accordance with the applicable license. At no time should the dYdX Chain and/or its software or related components be deemed to be a product or service provided or made available in any way by the dYdX Foundation. Interactions with the dYdX Chain software or any implementation thereof are permissionless and disintermediated, subject to the terms of the applicable licenses and code. Users who interact with the dYdX Chain software (or any implementations thereof) will not be interacting with the dYdX Foundation in any way whatsoever. The dYdX Foundation does not make any representations, warranties or covenants in connection with the dYdX Chain software (or any implementations and/or components thereof), including (without limitation) with regard to their technical properties or performance, as well as their actual or potential usefulness or suitability for any particular purpose, and users agree to rely on the dYdX Chain software (or any implementations and/or components thereof) “AS IS, WHERE IS”.

Nothing in this post should be used or considered as legal, financial, tax, or any other advice, nor as an instruction or invitation to act by anyone. Users should conduct their own research and due diligence before making any decisions. The dYdX Foundation may alter or update any information in this post in the future at its sole discretion and assumes no obligation to publicly disclose any such change. This post is solely based on the information available to the dYdX Foundation at the time it was published and should only be read and taken into consideration at the time it was published and on the basis of the circumstances that surrounded it. The dYdX Foundation makes no guarantees of future performance and is under no obligation to undertake any of the activities contemplated herein.