If the strategy is truly good, why not make money yourself? Three papers reveal the harsh truth behind selling indicators.

In the world of cryptocurrency trading, many people often believe in specific “trading indicators.” However, numerous studies point out that most strategies claiming to have stable backtested profits are not proven effective by the market but are simply survivors selected from a larger pool. It’s like doing a hundred past year’s exam questions and scoring full marks; it doesn’t mean you’ll perform equally well on this year’s exam. This is the “overfitting” trap of trading strategies. A more practical question is, if a strategy truly performs so well, why not leverage it yourself instead of selling or publicly sharing it?

After all, truly effective strategies are often limited by capacity; as capital increases, the advantage is quickly eroded by trading behavior and market reactions.

Developers of trading indicators often only showcase their best results to raise funds

A paper published by the American Mathematical Society highlights the bias in backtesting, revealing that under traditional backtesting frameworks, some technical strategies can indeed generate significant positive returns on historical data. This is why technical analysis has been favored by markets for years. However, the authors further point out that such results often ignore a critical issue: data-snooping bias.

When researchers test hundreds or even thousands of trading rules simultaneously, statistically, a few will perform exceptionally well—even if the market is entirely random. Judging the effectiveness of technical analysis based solely on these post-hoc winners is akin to mistaking luck for skill.

After correcting for bias, the advantage of technical strategies shrinks significantly

To address this issue, the study employs more rigorous statistical testing methods, adjusting for the bias caused by multiple testing. The results show that once corrected, strategies that initially appeared to have significant excess returns lose almost all statistical significance. In other words, these trading rules are unlikely to reproduce their past performance in out-of-sample environments, indicating they do not truly capture sustainable market structures.

Including transaction costs makes actual returns even more pessimistic

The study also considers transaction costs. Since technical trading strategies often involve high turnover, once fees, slippage, and market impact costs are included, even strategies with marginal positive returns tend to turn negative. The authors note that this finding is highly relevant for practical trading, as most published backtest results tend to underestimate the frictions present in real trading environments.

The conclusion does not entirely dismiss technical analysis but suggests its role is better suited for risk management, trend recognition, or behavioral insights rather than as a standalone profit source. In today’s highly competitive and information-efficient markets, relying solely on historical prices and volume signals makes it difficult to sustain a trading advantage.

Backtesting errors in trading indicators: like using past exam questions to ace the college entrance exam

A paper titled “The Probability of Backtest Overfitting” points out that the perfect backtest performance you see is highly likely a product of data overfitting. In quantitative finance, backtesting is the standard tool for evaluating strategy risk and return. However, with increased computing power, researchers can now easily test billions of strategy combinations on the same historical data.

The authors compare this to: “If you interrogate the data long enough, it will eventually confess.” As researchers continuously tweak parameters (like moving average lengths, entry thresholds, etc.) until the performance looks perfect, they are often just fitting past market noise rather than capturing signals for the future. It’s like doing past exam questions repeatedly until you can score 100 on last year’s test—this doesn’t guarantee you’ll perform equally well this year because it’s a different thing.

To solve this, the team introduces a key metric: the Probability of Backtest Overfitting (PBO). PBO measures the chance that the best-performing strategy in backtest will underperform in the future. A high PBO indicates the strategy is likely overfitted to the data; a low PBO suggests robustness.

In an experiment with a strategy boasting a Sharpe ratio of 1.27—very attractive to average investors—the PBO was found to be 55%. Despite all in-sample backtests showing positive returns, over 53% of out-of-sample tests resulted in losses. This demonstrates that even high-Sharpe strategies can be the product of overfitting.

Empirical study of the Indian stock market: RSI, MACD struggle to consistently beat the market

After discussing backtesting and statistical issues, let’s look at real-world research. An 18-year empirical study of the Indian stock market shows that popular technical analysis tools generally struggle to help traders generate consistent excess returns. Even during some bear markets with short-term advantages, risk-adjusted performance remains insufficient to prove long-term profitability.

This study, authored by S. Muruganandan from Sri Dharmasthala Manjunatheshwara College of Business, published in the Colombo Business Journal, used data from the Bombay Stock Exchange (BSE) Sensex from February 2000 to May 2018. Covering multiple bull, bear, and sideways markets, it examined the actual profitability of two common indicators: Relative Strength Index (RSI) and Moving Average Convergence Divergence (MACD).

RSI performs poorly, unable to create stable advantages in any market cycle

Results show that RSI trading strategies, whether buying or selling signals, did not significantly outperform the unconditional average return of doing nothing over the entire sample period. Even before deducting transaction costs, performance was already inefficient.

Further analysis across different market cycles reveals that during most bull markets, RSI frequently issued sell signals but failed to effectively follow through on trends. During bear or sideways markets, buy signals increased but often led to early entries and poor returns. The study notes that RSI’s structure makes it prone to contrarian trades in trending markets, which can hurt performance. From a risk-adjusted perspective, most RSI strategies had negative Sharpe ratios, indicating risk was not compensated by returns.

MACD performs slightly better, with short-term gains in bear markets

Compared to RSI, MACD shows somewhat better performance but still lacks stability. The study finds that MACD buy signals do not significantly outperform the market across all cycles. However, sell signals during bear markets do produce statistically significant positive returns, outperforming the unconditional average.

This suggests that during market declines, MACD can help traders avoid some losses or profit from shorting. Yet, when risk is considered, even these sell signals have low Sharpe ratios, indicating the returns do not sufficiently compensate for the volatility involved. In other words, MACD can be useful in certain scenarios but falls short of being a reliable long-term profit tool.

The study concludes that in the Indian stock market, under the weak-form efficiency hypothesis, historical prices are already reflected in current prices, making it difficult for technical indicators to generate abnormal returns over the long term. Even in emerging markets with less perfect information, the advantage of technical analysis diminishes over time. The authors emphasize that including transaction costs, slippage, and capital costs would further worsen the practical performance of these strategies.

If strategies are truly good, why not make money yourself? Three papers reveal the harsh truth behind selling indicators

Originally published in Chain News ABMedia.