Long-time list member Fabrizio A. sent me this question (name used with permission):
Fabrizio A.
Trade-Ideas backtest results present some challenges because TI only keeps the first occurrence of each alert per symbol per day. That means the Strategy Cruncher can’t evaluate whether a better trade might have occurred later in the day. So basically, it only works properly if the alert is natively designed to trigger just once per day. In other cases, I use the optimizer for other types of filters, but I need to try different time windows by running multiple backtests
Is there a better way to approach this?
Dave:
Fabrizio is describing the backtesting puzzle I posted about several days ago.
How do you optimize a strategy that’s based on an alert that can happen multiple times a day (like a new high)?
When you apply a filter, oftentimes you’re simply “postponing” the trade in a symbol that you believe should be filtered out since a symbol can print many new high alerts throughout the day.
(Contrast the new high alert with an alert like an opening range breakout, which can only happen by definition once per day.)
So, how do you optimize a strategy based on an alert that can happen multiple times?
There are two main ways to approach this problem.
- Optimize on columns whose values don’t change during the day
- Select a handful of combos to run backtests on
Let’s take #1:
When you restrict your columns to ones that don’t change during the trading day (ATR, Gap%, etc), you simplify the problem significantly since you eliminate the “peek-a-boo” trade issue above.
That is, when you filter on columns that don’t change, even if a new high alert appears throughout the day it will still be filtered out.
This is one of the main reasons I prioritize columns whose values don’t change during the day.
When this isn’t possible (maybe none of the columns are predictive enough to use), then it’s perfectly fine to run multiple backtests to try to find the best starting point.
I wouldn’t obsess too much about finding the “perfect” window of time for your strategy, though.
There’s likely not a “holy grail” time window. Just use your experience about the trading day to choose 3-4 windows of time that make sense and run backtests for each of them.
I’d be looking at the results to compare:
- The number of trades (more = better)
- Proximity to the open (more movement, in general)
- The metrics/equity curve of each starting point backtest
Great question, Fabrizio – thanks for sharing with the group!
-Dave
