We tried to build a signal engine. Here's what we found.
Before SunUp launched, we spent weeks trying to build the thing most crypto apps claim to have: a model that reliably picks profitable entries and exits. We tested it the way a quant desk would – and we're publishing the results, including the failures, because what we learned is the reason SunUp works the way it does.
How we tested
- 45-coin universe, deliberately including coins that have since fallen out of the top 50 – testing only today's winners flatters any strategy (survivorship bias).
- Five years of exchange data covering two full market cycles: the 2021 top, the 2022 crash, the 2023 recovery, and the 2024–26 bull and bear.
- Walk-forward evaluation – every decision uses only information available at the time. No peeking.
- Realistic transaction costs (0.5% per round trip) – the single most common thing backtests "forget".
- Out-of-sample guards and significance thresholds, pre-registered before each run – a strategy had to work on data it was never tuned on, in every market regime separately.
What we tested – and what the evidence said
Technical-analysis signals (trend, momentum, RSI, MACD, Bollinger, structure)
Every weighting we tried was negative after costs out of sample. Tuning the weights found configurations that looked brilliant in-sample and failed the moment they met new data.
Faster or slower candles (1-hour, 4-hour, daily, weekly)
Faster candles multiplied trading costs on signals that had no edge to pay for them; slower candles fired too rarely to prove anything and still lost. Sampling frequency was never the problem – the information simply isn't in price history at these horizons.
Market-regime filters (bitcoin trend, breadth, relative strength, sentiment, hidden Markov models)
"Only trade in favourable conditions" sounds sensible. Every version we built – including a latent-state model fitted with strict no-look-ahead filtering – failed to improve results out of sample.
Portfolio strategies (momentum rotation, trend overlays, mean reversion)
Across five years, no systematic strategy we tested beat simply holding bitcoin – most lost money outright while the benchmark gained.
Derivatives data (perpetual funding rates)
A promising lead on a handful of coins vanished when tested across the full 45-coin universe with proper statistics – a pattern we saw repeatedly, and the reason small-sample "proof" means so little.
The small-sample illusion
We ran the same engine on 200 randomly chosen 10-coin universes. More than half looked profitable purely by luck – on a system we know has no edge. A backtest on a small basket of coins can "prove" almost anything.
Volatility ranges
One thing crypto data genuinely supports: volatility is forecastable even when direction isn't. The "typical range this week" shown on every SunUp reading comes from a model whose 80% bands contained the real outcome 80% of the time in five years of out-of-sample testing.
Describing conditions
The engine is good at what a weather report is good at: saying what is happening – trend, momentum, volatility, nearby support and ceilings – clearly and consistently. That's what a SunUp reading is.
What this means for SunUp
We could have kept the buy/sell buttons and marketed the in-sample numbers. Instead the findings are built into the product:
- Readings, not tips. UPTREND, BREAKDOWN or NO SIGNAL describe technical conditions. They don't predict prices, and we don't pretend otherwise.
- A public track record. Strong readings are tracked automatically and the outcomes – wins and losses alike – are shown in the app. No cherry-picking; the record is the record.
- Honest ranges instead of price targets. Calibrated "typical range" bands, because that's what the data actually supports.
- Nothing is traded, ever. SunUp never touches your money and never will.
Five questions to ask any crypto signal product
If an app or group claims their signals make money, their evidence should survive these – ours is above, failures included:
- Were transaction costs included? (They erase most claimed edges on their own.)
- Was it tested out of sample – on data the model was never tuned on?
- How many coins, over how many market cycles? Ten coins over one bull run proves nothing – see above.
- Does the universe include coins that died, or only today's survivors?
- Did it beat just holding bitcoin? Most things don't.