Studio Strategies, Part 2: Run a Backtest You Can Learn From

A backtest is a rehearsal with recorded data

Part 1 gave us a specific BTC 15m strategy: buy a model-estimated price gap with IOC, avoid the final two minutes, and hold filled shares to settlement. A backtest runs that definition through recorded inputs and simulates the resulting orders. It answers “What would this model have done under these replay assumptions?” It does not report live fills or future returns.

Start with the unchanged Part 1 draft. Write down the universe, thresholds, sizing, and the question you want answered. For the first run, ask whether the graph creates understandable trades with usable data. Finding a profitable parameter comes later, if at all.

1. Run the backtest in Studio

Finish all three graphs and fix the validation issues Studio shows. Find Backtest 24h in the editor toolbar; a narrow screen may show only 24h and the play icon. Open the adjacent Settings control before your first run.

Choose Follow strategy for Execution direction and Standard venue fees for Fees. Then click Backtest 24h. The report opens while the run is preparing or replaying. Wait for completion or read the reported failure before changing the strategy.

The Studio control uses a fixed recent 24-hour window ending at the last completed hour. It is not a date-range picker. Inspect the reported window when comparing runs: pressing the button after the hour changes can include a different market period. Administrative replay tools have separate controls; this tutorial does not assume access to them.

2. Read more than the big P&L number

The report’s simulated P&L is a starting point. Read it together with fills, markets, the equity curve, Max drawdown, Maker / taker, Fees, and Recorded data. Replay time tells you how long computation took, not how long the strategy traded.

A fill is an execution, not necessarily a complete round-trip trade or an independent bet. Partial fills and many trades in the same market can inflate the count. A positive result dominated by one outcome deserves a different interpretation from gains spread across unrelated periods.

• P&L: how much the simulated account gained or lost under this run’s accounting and costs.
• Max drawdown: the largest fall from a previous equity peak during the replay. A positive finish can hide a painful path.
• Maker / taker: whether executions came from resting orders or immediate liquidity-taking orders. This should fit the strategy you meant to build.
• Recorded data: evidence that inputs were replayed. A large row count alone does not establish complete, timely coverage of every required feed.

3. Work through a small result by hand

Here is invented arithmetic, not a Quant Fun backtest. Suppose you buy 10 shares at $0.60 in each of ten separate markets and hold each to binary settlement. Six win and four lose. Purchases cost $60 in total; winning payouts total $60. Gross profit is $0 despite a 60% win rate. After any fees, the result is negative.

Now suppose a simulated account rises from $100 to $112, falls to $103, and finishes at $108. Final gain is $8, but the peak-to-trough drawdown is $9. Neither “+$8” nor the final balance describes that decline by itself. Keep profit, cost, and drawdown in the same conversation.

Costs matter especially when the modeled advantage is small. Polymarket fees depend on the market’s parameters and execution price. Use the standard-fee report for your baseline; do not assume all markets are free or count an unverified rebate as profit.

Polymarket: fee structure and market-specific parameters

4. If there are no fills, investigate in order

Zero fills can mean the signal never qualified. It can also mean missing history, unavailable model inputs, an order below venue minimums, insufficient collateral, or an IOC that missed its permitted price. Lowering every threshold will not repair missing data.

Read “Why there were no fills” when the report provides it. Use its unmet conditions to locate the first useful investigation. A failed replay is a failed experiment to diagnose, not evidence that the strategy made zero profit.

• Check the universe: did the selected assets and horizons have relevant recorded markets?
• Check required feeds and warm-up: a 5m model needs usable prior history; a volume baseline needs recorded trades.
• Check the conditions: were time remaining, price range, or minimum edge excluding every opportunity?
• Check execution and size: did orders meet minimums, have collateral, and encounter tradable liquidity at the allowed price?

5. Understand why a signal is not a fill

Imagine the graph sees an ask of 0.60 and approves a purchase. Before the simulated order arrives, the available ask moves higher. An IOC limited to the permitted price can miss or fill only the remaining quantity there. A chart touching 0.60 does not prove your whole order could have traded at 0.60.

A resting maker order has another problem: orders ahead of it may consume the available selling. Replay uses recorded market information and execution assumptions to model these effects. It cannot reconstruct information that was never recorded. Missing or stale signals should stop decisions rather than be replaced with convenient values.

Keep the timing in perspective. Live feeds can update frequently, but Studio’s shared strategy evaluation uses a heartbeat, normally one second; weather uses a minute cadence. A short lookback does not make a strategy react to every tick with zero delay.

6. Change one thing and keep a research notebook

Save the baseline before editing it. For example, compare an edge threshold of 0.05 with 0.07 while leaving BTC, 15m, 5m volatility history, entry time, order type, and sizing unchanged. Your hypothesis is that a larger gap may improve trade quality while reducing opportunities. Fewer trades alone do not prove improvement.

Record a row for each experiment: version name, exact replay window, changed setting, fee mode, fills, P&L, drawdown, and one observation. A spreadsheet or a note is enough. If the time window changed between runs, say so; you changed both the rule and the data.

The following are example notebook entries, not measured results. Version A: edge 0.05, 24 fills, +$3 P&L, $7 drawdown. Version B: edge 0.07, 8 fills, +$2 P&L, $2 drawdown. B made less money and experienced a smaller decline in this imagined sample. Whether that tradeoff helps requires more data and comparable exposure.

7. Use research modes for a specific question

No fees — research mode can help isolate sensitivity to the fee model. It does not remove the spread or make orders fill for free. Inverse — flip buy and sell is a different execution scenario, not a guaranteed mathematical negative of the original P&L: inventory, fills, settlement, and costs can differ.

Neither inverse nor no-fee results can serve as publication proof. For the eligible prediction-market strategy in this lesson, Studio requires a completed Follow strategy run with Standard venue fees for the current definition before publication. “Use this backtest” closes the completed standard report. Editing the strategy logic means you need a matching run again.

8. Give the idea a test it has not already seen

If you try enough thresholds on the same day, one can look good by chance. Choosing that winner and calling the same day a test is overfitting: you have tailored the rule to the sample. Decide your next settings before seeing a fresh period, then keep them fixed through that evaluation.

Because the standard Studio replay is a rolling 24-hour window, hourly reruns overlap heavily. They are not independent test days. Keep dated results and evaluate later, non-overlapping recorded periods as they become available. If you need controlled older windows or deeper execution diagnostics, those require separate research or administrative tooling.

Before considering live use, you should understand the loss cases, required data, position size, and exit behavior. Publishing a strategy and enabling funded execution are separate steps. Part 3 develops more advanced research examples while keeping that same habit: state the reason, build the rule, and test what could go wrong.