1. Executive summary

Crypto futures strategy validation project

Summary report of all experiments — evidence, reasoning and methodology

Experiments E-01 ~ E-24 · decisions D-001 ~ D-032 · 10,342+ recorded statistical trials

July 16, 2026

Period covered: 2026-07-07 ~ 2026-07-15 (9 days) · data: 4 years of Binance USDT-M futures

1. Executive summary

This document is the complete record of a crypto futures automated trading strategy validation project carried out over 9 days, from July 7 to 15, 2026. It covers 24 experiments (E-01~E-24), 32 design decisions (D-001~D-032), and more than 10,342 statistical trials traceable through the ledger and experiment records (5,481 ledger rows from the automated-search era + retroactive inclusion of 4,861 documented manual-era searches — a documented lower bound; the counting basis is in section 3.5).

Final record: 10,342+ statistical trials, 0 admitted to live trading. 6 classes of manually designed strategies, 4,752 automated-search combinations, 468 cointegrated-pair runs, and some 200 runs of 5-minute high-frequency and limit-order (maker) execution research all failed their preregistered pass lines.

Reading this number as failure is misreading this document. From the start, the project's objective function was not profit but "a low-deviation validation system + a credible track record". By that standard, the results are as follows.

The validation machine blocked the deployment of a strategy with a losing profile. The sealed test gate filtered out in advance the deployment of a strategy that lost -7.45% in that period's backtest (E-06). P&L had it actually been deployed is counterfactual and cannot be established, but this is direct evidence that the gate worked as designed.

Every verdict was made by preregistered rules. The discipline of fixing pass lines before running and not modifying them after seeing results worked at least 4 times in 9 days, at points where the temptation of post-hoc rationalization was real.

"0 admitted" is not a universal statement. The correct interpretation is the limited conclusion that "within the registered search space, cost structure and power limits, there was no deployable candidate". That the test filter's power itself is low (detection of ~22% even for a genuine annual Sharpe of 2.0) was also quantified and recorded alongside.

The body is organized as (chapter 2) project philosophy, (chapter 3) methodology infrastructure, (chapter 4) the record of all experiments, (chapter 5) a casebook of measured bugs, (chapter 6) overall insights, (chapter 7) future roadmap, and (appendix) lists of decisions and experiments. The [Basics] boxes along the way are included so that readers without a financial engineering background can understand each section and continue studying on their own.

| [Basics] If this project were drug development, the structural analogy running through this whole document is the drug development pipeline. A pharmaceutical company synthesizes thousands of candidate compounds (strategy candidates), screens them in cell experiments (train-period backtests), confirms them in animal experiments (val-period comparison), and runs human clinical trials only once, under a preregistered protocol (the sealed test period). Just as resubmitting a drug that failed clinical trials "with only the dose changed" is forbidden, this project forbids reviving parameters of dead strategies. The final approval rate of drug candidates is one in thousands. The pharmaceutical industry keeps going anyway because the system that eliminates candidates is itself the asset — the cost of releasing an ineffective drug to market is overwhelmingly larger than the cost of maintaining the pipeline. Trading is the same: the loss from deploying a fake strategy on a real account is larger than the cost of maintaining the validation system. This project's "some 10,000 trials, 0 admitted" does not mean 0 passed clinical trials; it means there was not yet a candidate worth sending to clinical trials. | | --- |

2. Project philosophy — what is being optimized

2.1 Objective function: not profit but a validation system

The first line of Master Plan v1.0 (2026-07-07) defines the objective function as: "not maximizing profit but completing 'a low-deviation validation system + a track record'." Deviation here means the difference between the return a backtest predicted and the return realized in live trading. However high a backtest's return, if it does not reproduce live, that backtest is fiction. Conversely, a low-deviation system has value even without a profitable strategy, as a machine that "can deploy a strategy as is once one exists".

Initial capital is about 1,000,000 KRW (≈730 USDT). At this scale, aiming to maximize profit is mathematically meaningless (premise 1 below); instead, the design idea is to use the real advantage of small capital — that mistakes are cheap — to build the validation system.

2.2 The 3 corrected premises

Before starting, the plan explicitly corrected 3 common misconceptions. The 9 days of experiments that followed were in effect a demonstration of these premises.

Small capital ≠ high expected return. The only advantage of small capital is that slippage is small, so strategy returns are realized close to theoretical values. The "high return potential" felt with small capital is mostly increased variance mistaken for increased expected return.

Price movement ≠ account return. Even if BTC moves ±5% a day, what a strategy captures is range × capture rate. Leverage does not raise the capture rate — it only scales both gains and losses by the same multiple.

Parameter sweep ≠ diversification. RSI 14 and RSI 21 are structurally highly correlated and give no diversification. Real diversification arises only between strategies with different market mechanisms (mean reversion vs trend vs market-neutral). This premise is why strategy candidates are counted afterwards by 'class (mechanism)'.

| [Basics] Sharpe ratio, MDD — the common language of this document. Sharpe ratio = mean return ÷ standard deviation of returns. It measures not "the size of returns" but "the quality of returns relative to fluctuation". A 10% annual return earned quietly is Sharpe 2.0 (excellent); earned on a roller coaster it is Sharpe 0.3 (indistinguishable from noise). Conventionally, 1.0 or above annualized is notable, and below 0.5 is not worth deploying. This project's strategy discard line is also OOS Sharpe 0.5. MDD (Maximum Drawdown) = the largest decline from an account peak. Even with the same return, MDD -8% and -40% are completely different strategies. Recovering from -40% needs +67% (asymmetry), and people break before that. This project's account-level kill criterion is MDD -15%. The priority of verdict metrics is Sharpe, MDD, probability of ruin and deviation > return. Return sitting at the very end of the verdict list is this project's identity. | | --- |

2.3 Kill Criteria — stop rules fixed in numbers

The kill criteria finalized on the first day (final version v1.0) are as follows, with a clause forbidding amendment while in a loss. Amendment is possible only when all positions are closed + there is a written rationale + a revision history entry.

The reason this table exists is psychological. In loss periods, people drift toward "refining the backtest further instead of dropping the strategy" — the trading version of the pattern of expanding the plan when execution stalls. Fixing the numbers in advance makes it possible not to give the decision to oneself in that moment.

| [Basics] Preregistration — why fix pass lines before seeing results. Toss 1,024 coins and one of them comes up heads 10 times in a row. Calling that coin "the heads coin" is the error of post-hoc selection. The trading version: test 100 parameters and a few will look good by chance. If, after seeing results, you draw the line "pass if Sharpe 1.2 or above", that line was in fact drawn to let the ones that were lucky pass. Preregistration is the only known way to block this: fix pass lines, decision rules and search ranges in documents before running, and do not modify them after seeing results. It is the same reason medical clinical trials made protocol preregistration mandatory from the late 1990s — after mandatory registration, the share of 'effective' conclusions fell sharply. The effects did not vanish; effects that had never existed were being created by post-hoc selection. Moments in this project when preregistration actually worked: the sealed test opening pass line (D-008 → E-06 fail verdict), the carry scan rule (D-012 automatic verdict), the rotation retrial pass line (D-015 → D-016 rejection), and all gates of the harness, pairs and 5-minute families (D-017~D-030). Each was a point where the temptation "it would pass if we loosened the bar a little" was real. | | --- |

3. Methodology infrastructure — devices that structurally block optimism bias

The default state of a backtest is 'optimism'. A backtester built without any safeguards almost always produces better numbers than reality. This chapter records the blocking device the project installed for each path through which that optimism seeps in. All experiment results must be read on top of these devices.

3.1 Data — 4 years, 3 regimes, automated quality checks

4 years (2022-07~2026-07) of Binance USDT-M futures BTCUSDT were collected at 4 resolutions: 1 minute, 5 minutes, 1 hour and 1 day (2.11 million 1-minute bars). In addition: spot 1h, a 13-symbol universe at 1h, funding for 22 symbols. Right after collection, automatic quality checks examine missing data (timestamp continuity), outliers (±30% — flagged, not removed), OHLC consistency and duplicates. Across all resolutions over 4 years there were 0 gaps, 0 duplicates and 0 OHLC violations (E-01). The only known gap is 1 spot candle at 2023-03-24 13:00 — an exchange matching-engine halt; the rule is to keep it missing without interpolation.

The period is 4 years for regime coverage. The train period (2022-07~2024-11) alone includes 3 regimes: a bear market (a decline of around -50%), recovery and a bull market. A strategy built on single-regime data becomes a fossil of that regime.

| [Basics] Candles (OHLCV) and data resolution. One candle summarizes the open, high, low, close and volume over a fixed time interval. 4 years of 1-hour bars is about 35 thousand candles; 1-minute bars are 2.11 million. Candles are summaries, so information is lost. Within the same 1-hour bar you cannot know whether the high or the low came first — this ambiguity matters when a stop and a take-profit fall in the same candle, and this project handles it with the conservative rule 'the worse side comes first'. Going beyond this limit needs trade-level data (AggTrade), which is actually used in the later experiments (E-22~24). | | --- |

3.2 Sealed 3-way split — train / val / test

4 years of 1-hour bars were split 60/20/20 in time order: train 21,128 bars (22-07~24-11), val 7,043 bars (24-11~25-09), test 7,043 bars (25-09~26-07). Each file is sealed with a sha256 hash (E-01), and at the code level the test partition does not open without an explicit consent flag. The same date boundaries are enforced on other resolutions and symbols.

The test was opened only 1 time, on 2026-07-09 (E-06, the final volatility breakout verdict), and then declared consumed — reuse in any search is forbidden. Thereafter, the final gate for new strategies was replaced by the D-010 rule: a double gate of (a) incoming data that keep accumulating after the sealing date (structurally uncontaminable future data) + (b) paper-trading deviation.

| [Basics] Overfitting and in-sample/out-of-sample — mock exams and the college entrance exam. Overfitting is a state where a strategy has memorized not the rules of the market but accidental patterns in past data. It is like a student who memorized the answer bank getting full marks on mock exams (= the data used for learning, in-sample) and collapsing on the real entrance exam (= data never seen before, out-of-sample). train is the workbook (solve freely and keep a mistake notebook), val is the mock exam (check whether the study method works), and test is the entrance exam (once in a lifetime; you cannot retake it after seeing the score). If you solve the exam paper in advance and change your study direction, that exam score is void — so the test is sealed with a hash and treated as 'consumed' the moment it is opened. The moment this distinction proved its practical value in this project: the volatility breakout strategy looked promising with a best train Sharpe of 1.39 and walk-forward 1.20, but collapsed to -1.42 on the sealed test (E-06). Without the test, this strategy would have become a deployment candidate on the strength of a wf Sharpe of 1.2, carrying a loss profile of -7.5% in that period's backtest (whether it would have been realized is counterfactual and not asserted). | | --- |

3.3 Execution model — devices blocking optimism bias

The execution model of the in-house backtest engine (D-001, 300-line cap) enforces the following rules.

No look-ahead: the signal is decided at candle i's close and the fill happens at candle i+1's open. Seeing the close and buying at that close is using future information.

Full costs: taker fee 0.05% + slippage 0.05% (fixed in the adverse direction). Round-trip trading cost about 0.2%. Funding is charged to held positions at funding times.

Exchange constraints: entries below the minimum order amount (100 USDT) or quantity step are skipped — 'the theoretically buyable 0.3 dollars' worth' does not exist.

Conservative stop fills: if a gap goes through the stop price, the fill is computed at the (worse) open, not the stop price.

Bankruptcy freeze: if capital falls to 0 or below, everything thereafter is frozen at 0. Leaving negative capital contaminates all statistics (see the bug cases in chapter 5).

| [Basics] Fees, slippage, maker/taker — why costs become king. A taker is a market order that fills immediately (the side that 'takes' liquidity from the order book); a maker is a limit order placed at a desired price to wait (the side that 'makes' liquidity). Exchanges give makers cheaper fees — on Binance futures around 0.05% for takers and around 0.02% for makers. Slippage is the difference between the price at which an order was placed and the price at which it actually filled. Market orders fill by eating through the order book, so they always slip slightly against you. Does 0.2% round trip look small? Trading 2 times a day is 500 times a year, and costs alone are 100% a year. This is the direct cause of death of every 5-minute strategy tested in this project — the signal earned 0.02~0.19% per trade while costs were 0.2% per trade. Costs are certain and signals uncertain. This asymmetry is the most frequently repeated cause of death in this whole document. | | --- |

| [Basics] Funding rates — the string tying futures prices to spot. Perpetual futures have no expiry, so a mechanism called the funding rate keeps the futures price from drifting away from spot. Every 8 hours (4 hours for some symbols), if the futures price is above spot, long holders pay short holders, and vice versa. In bull markets, demand for leveraged longs keeps funding positive — collecting only the funding without directional risk via 'buy spot + sell futures' is the funding carry strategy (strategy 3 in section 4.1). BTC's 4-year average funding was 7.94% annualized, positive 88.8% of the time. The problem is that this return is on position notional, not total capital, and after entry and exit costs there are periods when what remains cannot compete with the risk-free rate — that is the regime since 2025. | | --- |

3.4 The validation funnel — the same gates every strategy passes

The validation order applied identically to every strategy: (1) train grid search (recording the number of combinations searched is mandatory) → (2) val comparison (whether the best train candidates' performance collapses) → (3) walk-forward (rolling re-optimization — a rehearsal of real operation) → (4) final gate (the sealed test 1 time / after it is consumed, D-010 incoming data + paper). Failing at any stage means not going further down.

| [Basics] Walk-forward — repeated reproduction without knowing the future. Backtesting all 4 years with fixed parameters amounts to applying 'the optimum learned 4 years later' to the past of 4 years ago. Walk-forward corrects this: for example, choose parameters on months 1~12 of data and trade month 13. Then choose again on months 2~13 and trade month 14. Sliding the window and repeating gives a scorecard of decisions made 'with only the information up to that point' at every moment — this is the real out-of-sample return curve. As a side effect, it also defines the deployment model: this project treats passing walk-forward as validation of the deployment method 're-optimization 1 time per month' (D-005). The live bot must also re-optimize at the same interval for validation and real operation to be consistent. | | --- |

| [Basics] Monte Carlo — making ten thousand histories out of one. The MDD a backtest shows is the result of 'the single sequence that actually happened'. Had the same trades come in a different order, the drawdown would have differed. Monte Carlo (block bootstrap) randomly rearranges blocks of returns to create thousands of virtual histories and reads probabilities such as P(MDD ≤ -15%) from that distribution. The sizing verdict (E-05) was made with this tool: comparing 6 options on the same signals with only sizing changed, option F (25% of capital × leverage 2 + volatility target) had the lowest kill-breach probability at 12.6%. Caution: rearrangement only shuffles observed returns, so 'a crash that never happened' cannot be reflected — the estimate is an optimistic lower bound. | | --- |

3.5 Multiple-testing ledger and DSR — honesty about the number of attempts

The peak of this project's methodology is the single ledger (harness_ledger.csv). Rule: every strategy attempt, regardless of tool, counts as complete only once it reaches the ledger. Backtest results not in the ledger are not trusted.

Definition of the trial hierarchy (D-032) — 'how many times was it tried' cannot be answered without a definition. This project distinguishes run (1 execution of a script) / parameter combination (configuration) / walk-forward internal fit (1 re-search per window) / statistical trial (the unit counted in the multiple-testing denominator — every combination or fit for which a performance number was observed once). As of 2026-07-16 the total trial count is 10,342: 5,481 rows recorded directly in the ledger (harness 4,852 + pairs 498 + experiment registry mirror 131) plus 4,861 documented manual-era searches before the ledger system (7/14) included retroactively (RSI wf 2,592, breakout wf 1,152, maker counterfactual replay 446 and others, 14 batches). This is a documented lower bound; if unrecorded trials existed, the real N is larger and the correction correspondingly less strict — it is stated that using a lower bound errs anti-conservative.

Why the number of attempts matters so much. Repeating manual variants like "let's also try a Bollinger touch version" is uncounted multiple testing — the probability that the 10th variant passes by chance accumulates, and without records it cannot be corrected. The ledger forces this accumulation to be counted, and DSR (box below) raises the pass line by that count.

| [Basics] Multiple testing and DSR — what do you see if you toss 2,000 coins? Test 2,000 strategies and even with pure randomness (0 genuine skill), 1st place posts a plausible score. This project's measurement: searching 2,000 combinations on 21,000 bars of data, 1st place reaches an annualized Sharpe of about 2.2 from noise alone (a mathematical certainty — pre-computed with the E[max] approximation and confirmed at 3.65 on real data). That is, 'a Sharpe of 2 came out' is no evidence in itself in the context of 2,000 attempts. DSR (Deflated Sharpe Ratio, Bailey & López de Prado 2014) corrects for this: it subtracts 'the expected maximum of N attempts' from the observed Sharpe and asks whether what remains is statistically significant. The larger N, the higher the pass line — so the ledger counting attempts is DSR's denominator. The pass line itself was also calibrated before running (D-017): 4,000 experiments on synthetic null data (0 genuine skill) set the whole funnel's false-positive rate at 4.5%, and it was registered alongside that, as the price, power (the probability of catching a genuine edge) is only ~22% at an annual Sharpe of 2.0. Knowing and stating in numbers how many genuine edges the filter misses — this is the basis for interpreting '0 survivors' correctly. | | --- |

As an auxiliary rule, "too good / too clean numbers are bug signals". Before a verdict, 'is it physically possible + is the activity level sensible' must be checked. The 5 bugs this rule actually caught are collected in chapter 5.

4. Record of all experiments

This chapter records all experiments from E-01 to E-24 in 4 phases: (4.1) 6 classes of manually designed strategies, (4.2) the automated search harness, (4.3) cointegrated pairs, (4.4) high-frequency and limit-order execution research. Each experiment is described in the structure 'why this hypothesis (evidence, reasoning) → how it was validated (method) → what came out (result) → how it was judged → what was learned'.

4.1 Phase 1 — 6 classes of manually designed strategies (E-02~E-13, 7/8~7/13)

Strategy 1: RSI mean reversion (E-02, E-03) — discarded

Hypothesis and rationale: the oldest and most heavily mined hypothesis, that when price reaches a short-term oversold condition (RSI low + lower Bollinger band) it returns to the mean. It was chosen first not because an edge was expected but as a standard sample to test the whole pipeline end to end (data → signal → engine → validation).

Method: train grid of 81 combinations → val comparison → walk-forward over 32 windows (2,592 total searches including per-window re-searches, all recorded).

Result: the defaults gave -8.9% (Sharpe -0.33). Even the best of the grid had a train Sharpe of only 0.258. Walk-forward OOS Sharpe 0.177 — below the discard line (0.5). 12 of 32 windows had 0 trades at all.

Verdict and lesson: automatically discarded by the kill criteria. That some combinations looked good on val (Sharpe 0.97) is interpreted as luck on a sample of 17 trades — the lesson that a Sharpe from few trades is noise, not a number, was institutionalized as the 'minimum trade count' gate (eligible condition) in every later experiment.

| [Basics] Mean reversion vs trend following — two opposite grammars for reading the market. Mean reversion bets that "it moved too far, so it will come back" (the rubber-band analogy). Trend following bets that "it has started moving, so it will keep going" (the inertia analogy). They take opposite positions on the same price movement, so the two families die in different regimes — mean reversion bleeds in runaway trends, trend following in sideways markets. RSI is an indicator summarizing the ratio of recent gains to losses on a 0~100 scale (low means oversold), and Bollinger Bands are a band of moving average ± 2× standard deviation (a lower-band touch = a statistically rare decline). What matters: since these indicators were published in the 1970s~80s, they have been a commons mined by the whole world for decades. The prior probability that an edge remains in simple combinations of public indicators is low, and this project's experiments end up confirming that repeatedly. | | --- |

Strategy 2: volatility breakout (E-04, E-05, E-06) — failed the sealed test

Hypothesis and rationale: the Donchian channel family hypothesis that breaking the highest price of the last N periods signals the start of a trend. Its return mechanism is the opposite of strategy 1 (mean reversion), so it was selected as a 'real diversification' candidate. An ATR-based stop was attached to cut trend following's long-tail losses.

Method: grid of 36 combinations → val comparison → walk-forward (1,152 total searches) → 6 sizing A/B options + Monte Carlo → the sealed test opened 1 time (pass line preregistered in D-008: Sharpe ≥0.5, MDD >-15%, trades ≥8).

Result: best train 1.39. On val comparison the fixed parameters were wiped out (-0.57/-1.16/-0.41), but walk-forward (monthly re-optimization) gave OOS Sharpe 1.198, +93.7% — this gap exposed a protocol gap, 'fixed deployment vs adaptive deployment are different validations', leading to D-005 (officially adopting adaptive deployment). In the sizing experiment (E-05), option F (25% × 2x + volatility target) was adopted: Sharpe was much the same across all 6 options at 1.19~1.25, while MDD ranged from -8.5% to -19.8% — the lesson that sizing does not change the shape of the return curve, only its scale.

Opening the sealed test (E-06): Sharpe -1.417, -7.45%, fail. A drop from walk-forward 1.20 → test -1.42. Candidate causes are a regime change (trend following bleeding sideways) or the materialization of grid selection bias — reanalysis of the test is forbidden, so no definitive diagnosis was made; they were recorded only as candidates.

Lesson: this failure is the project's biggest achievement. Without the gate, this strategy would have become a deployment candidate on the strength of a wf Sharpe of 1.2, carrying a loss profile of -7.45% in that period's backtest. It is recorded not as a 'demonstration of blocking' a realized loss but as direct evidence that the gate worked as designed. At the same time the test was consumed, and the final gate for new strategies was thereafter replaced by D-010 (incoming data + paper).

Strategy 3: funding carry (E-07, E-08, E-09) — BTC on hold, universe expansion rejected

Hypothesis and rationale: a market-neutral strategy collecting funding regardless of market direction (spot long + futures short). Measured over 4 years, funding was positive 88.8% of the time at 7.94% annualized — the mechanism itself is real.

Method: a feasibility scan before building a simulator (D-011) — compute the win rate at which cumulative funding by holding period exceeds the round-trip cost (0.5%). 3 days 0.2%, 7 days 3.1%, 14 days 12%, 30 days 58%. Timing (short-turnover) variants were rejected before even building a simulator — costs structurally crush them. Only the long-hold variant proceeded with a dedicated simulator (including a basis, fee and funding breakdown report).

Result: train 3.41% a year (Sharpe 1.73, MDD -2.8%), val 1.24% a year. The profile (high Sharpe, very low drawdown) is textbook-good, but the absolute return falls short even of a risk-free alternative. The 22-symbol expansion scan (E-09) failed the preregistered rule (altcoin median > BTC+3%p) and was rejected automatically.

Verdict and lesson: BTC carry is not rejected but on hold — a retrial condition (30-day rolling annualized 8%+ sustained) is registered and the simulator preserved, so the retrial cost ≈ 0. The key finding is that funding carry is a single factor where symbol diversification does not help: every symbol 11~16% in 2024, every symbol 2~6% in 2025 — the 22 symbols effectively move on one switch, 'bull-market leverage demand'. Picking top symbols (WIF 19.9%) after the fact is cherry-picking, and the preregistered rule blocked it.

Strategy 5: liquidation cascade proxy (E-10) — rejected

Hypothesis and rationale: chain liquidations (cascades) of leveraged positions push prices excessively regardless of fundamentals, followed immediately by a rebound — a rare hypothesis with a clear mechanism. However, with no real liquidation data, it was tested through a proxy of 1-minute bar fingerprints (volume spike + sharp drop + recovery wick).

Result: all 12 train combinations had negative Sharpe (-0.72~-2.75). Not overfitting but death at the hypothesis level — even a consistent ordering was confirmed, where looser conditions are worse (more noise trading).

Verdict and lesson: rejected, with the cause of death autopsied precisely: by the time the 'recovery wick' is confirmed, much of the rebound is already spent — the residual return from entering on the next candle < the round-trip cost. If a real edge exists, it is on the side of collecting limit fills during the crash, which cannot be expressed in a candle backtest. So a real-time liquidation recorder (record_liquidations.py) was started running continuously, and a revisit is allowed only as a separate hypothesis after 3+ months of accumulation (D-013). A case where a rejection produced an infrastructure investment.

Strategy 6: cross-sectional momentum rotation (E-11, E-12, E-13) — rejected

Hypothesis and rationale: hold the top K of 13 symbols by recent relative return and rotate periodically — porting to crypto the cross-sectional momentum factor documented longest in equity and commodity markets.

Method and journey: validating this strategy had the most twists in the project. In the 1st run, a simulation bug (negative capital left in bankrupt legs → an impossible MDD of -108%) was found and fixed. In the 1st walk-forward, another bug (warm-up period > validation slice → 0 trades in every window) was found and fixed. In the 2nd wf, OOS Sharpe 0.581 barely passed the preregistered line (0.5).

Decisive validation: it passed, but with 3 warnings attached — MDD -41% (2.7x the kill), all of the return from a single window (the 2024-11 altcoin surge, w22), and a borderline pass after 384 searches. Sizing measured at capital 730 (E-13) swung -9.7%/+71%/+46% — the sizing sensitivity itself is a sign of no edge. In the retrial at capital 2,000 with minimum-order threshold interference removed (4 pass lines preregistered, 1 run, D-015), Sharpe passed, but excluding the 1 best window the 3-year sum was -1.14% — failing the return-concentration clause and rejected automatically (D-016).

Lesson: a strategy where 'all 3 years of return came from one month' is a lottery, not statistics. That this verdict was made without consuming the sealed test demonstrated the effectiveness of the D-010 gate design. It was also measured that capital of 730 USDT is itself a physical constraint on K-diversified strategies (per-leg minimum orders not met) — this finding feeds directly into the later TSMOM design (chapter 7).

| [Basics] Factors, beta and the 'return concentration' check — the turkey's 3 years. Beta (β) is the component that rises with the market as a whole (not free but the price of market risk); alpha (α) is the component earned independently of the market (genuine skill). In bull-market backtests, a long strategy produces plausible returns from beta alone even with 0 alpha — the phenomenon this project calls 'the shadow of beta'. The return concentration check (summing after excluding the best window/month) is a cheap tool to catch it. If all 3 years of return came from one month, the substance of that strategy is 'the event of that month', not a repeatable rule. A turkey fed every day has perfect statistics until the day before Thanksgiving — the problem is the one day not in the sample. Conversely, a strategy whose returns cluster in one month is 'a turkey farm that lived 3 years on one Thanksgiving day', and nothing guarantees that day comes again. | | --- |

4.2 Phase 2 — the automated search harness (E-14, 7/14)

The logic of the switch: as manual strategies failed 6 classes in a row, the temptation to 'try one more variant' grew. But uncounted manual variants are the most dangerous multiple testing of all. So instead of giving up search, search was mechanized and the statistics made honest: a harness (Phase 6) that generates 4,752 combinations from a block grammar of 7 entries × 6 exits × 4 filters × 3 directions, forces every attempt into the ledger, and corrects for multiple testing with DSR.

Validating the filter first: step 1 of implementing the harness was not a strategy but self-validation of the filter — if the filter is broken, the harness becomes a lie factory. In 400 experiments on synthetic null data (0 genuine skill), DSR had 0 false positives, whereas uncorrected statistics would have let 87.5% pass falsely. This process also found a defect in the original design: the plan's 'p<0.05' gate had 0% false positives but also effectively 0 power (1% detection at an annual Sharpe of 2.0) — 'a machine that rejects everything'. Before running on real data, it was redesigned as a compound funnel (DSR >0.5 → val annual Sharpe ≥1.4 & degradation ≤0.6) and calibrated and registered at 4.5% false positives / power (annual Sharpe 1.5/2.0/2.5 → 5/22/54%) (D-017). Registering even the filter's limits in numbers is the methodological core of this experiment.

Result: all 4,752 combinations run, 0 errors. Of 4,279 passing the trade-count gate, 0 passed the DSR gate. Median eligible annual Sharpe -1.08 (costs eat almost everything), best +1.73 — an overwhelming failure, not reaching even half of the null noise ceiling (E[max] 3.65).

Autopsy findings: the top 10 were all long-only + trend-type entries — interpreted as the beta shadow of BTC's upward drift in the train period. The mean of short/both combinations was -1.3 or below. Verdict: on single-symbol BTC 1-hour bars, the search space of the public-indicator block grammar has no edge exceeding costs. The 6 manual classes and the 4,752 automatic combinations independently reached the same conclusion — mutually confirmed.

4.3 Phase 3 — cointegrated pair trading (E-15, 7/14)

Hypothesis and rationale: the only market-neutral mechanism not yet touched. If two assets are in a long-term equilibrium relationship (cointegration), temporary widening of the spread reverts — one of the few hypotheses where why it makes money can be explained in one sentence. All 78 pair combinations of the 13 symbols.

Method: implemented the Engle-Granger cointegration test in-house (synthetic validation: 100% detection of cointegrated pairs, 6% false detection of independent pairs ≈ nominal 5%) and built a two-leg simulator that enters and exits on the spread z-score only in periods passing the rolling test. The search space (78 pairs × 6 parameters = 468 runs) and pass line were registered before running (D-018), and the attempts were added to the same ledger as the harness.

Result: all 468 runs, 0 errors, 0 survivors. The decisive sign was not Sharpe but absence of activity — a median of 8 trades over 2.4 years, 97% with fewer than 30. The design-time warning (labelled [inference]) that cointegration in crypto pairs is short-lived and unstable was confirmed by measurement. Even the best annual Sharpe among combinations with activity was +0.61 — 1/6 of the null ceiling.

| [Basics] Cointegration — the drunk owner and the dog. The classic analogy for cointegration: a drunk owner and an unleashed dog walk home together. Each path is erratic (both random walks), but the distance between them stays within a range — because the dog follows its owner. If two asset prices have such a relationship, one can bet on the 'distance (spread)' narrowing when it widens. Not needing to know the direction is the appeal of this strategy. The z-score is 'how many standard deviations the current distance is from normal'. For example, enter at z=2 and exit at z=0.5. The Engle-Granger test determines whether that leash is real (whether the spread is statistically stable). An analogical way to put why this failed in crypto: crypto's dogs often break their leashes. The relationship between two coins is reshaped within weeks by sector fads, listings and narratives, and periods with a leash are short. The safeguard of trading only in periods that pass the test resulted in 'hardly trading at all' — an honest failure where the safeguard working led to rejection. | | --- |

4.4 Phase 4 — high-frequency and limit-order execution research (E-16~E-24, 7/15, GPT session)

The logic of the switch: up to this point, the cause of death was overwhelmingly 'costs > signal'. That leaves two questions — (1) what if signal frequency is raised to enlarge the sample? (2) what if costs themselves are lowered with maker (limit) fees? This phase is the systematic exhaustion of those two questions. It was carried out by a collaborating AI (tag [gpt 5.6 sol]), and discipline was kept by introducing a separate experiment registry (SQLite, registering code commits, data hashes, parameters and verdict reasons before running).

Answer to the frequency question (E-16, E-19, E-20, E-21)

5-minute shock reversal, 16 combinations (E-16): activity of 2 per day was achieved exactly, but the best combination was -6.75% even with no costs — the signal itself was negative, not a fee problem. The directionality gate, 36 runs (E-19), and daily trend × 5-minute entry, 6 families, 72 runs (E-20/21), showed the same pattern: up to +0.008~+0.190% per signal before costs could be produced, but the market-order round-trip cost (~0.2%) was always larger. Extending to a 24-hour hold improved gross to +0.19%, but after costs + funding it was -0.03% — it got right up to the wall of costs but did not get over it.

Answer to the cost question (E-17, E-18, E-22~E-24)

Counterfactual audit (E-17): 446 of all earlier attempts were replayed under the counterfactual 'apply only maker fees at the same fill prices' — 36 newly turned profitable, and 5 survived exploratorily on val comparison. However, this is an optimistic upper bound assuming limit orders always fill, so it was used not as grounds for revival but only as confirmation of 'the value of building a real limit-order engine' (D-021).

Real limit-order path (E-18): 96 combinations modelling non-fills, TTL and queue position — wiped out. The autopsy produced this phase's biggest finding: 31.5% of stops occurred inside the entry candle. Limit orders placed at extreme prices fill with high probability not at 'a good price' but 'only when bad information arrives'.

Replay with official trade data (E-22~24): the remaining candidate (RSI 24h) was replayed with Binance's official AggTrade (the record of all trades, checksum-verified). A 24-signal pilot → expanded to 200 signals turned positive at +0.063%/signal in train. The post-hoc finding that only longs were good in train (+0.258% vs shorts -0.359%) was not promoted on the same data but carried forward fixed to 69 independent val signals — and collapsed to -0.191%/signal. Not a fill-assumption problem; the directional return itself did not reproduce. Family rejected (D-030).

| [Basics] Adverse selection of limit orders — why 'buying cheap' is not free. A limit order is placed at a desired price to wait. Fees are cheaper and there is no slippage — so why does not everyone use only limit orders? Because of adverse selection. An analogy: suppose you place an order to buy a used car at 5% below market. When does that order fill? Usually at the moment it has become known that the car has a problem and everyone wants to sell. 'It came to my price' often means 'a reason appeared for the price to fall further'. E-18's measurement is exactly this: fills of buy limits placed under a falling knife were correlated with the arrival of bad information (31.5% stopped within the entry candle), and the entry quality lost was larger than the fees saved. Maker fee savings are traded for the information cost of fills, and whether that trade pays can be validated only with trade-level data — that is why E-22~24 went all the way down to official AggTrade. | | --- |

| [Basics] AggTrade and the fill models F0/F1/F2 — the ladder of backtest resolution. AggTrade is the trade-level record an exchange publishes (aggregates of who bought how many at what price and when). If a candle is a summary of 1 hour, AggTrade is every event within that hour. Limit-order backtests become honest only with this resolution — because 'did the market pass through my order price, and how big was the queue ahead of me' cannot be known from candles. The ladder of fill models: F0 (fills as soon as price touches — optimistic), F1 (fills only if price trades through — accounting for the queue), F2 (a conservative queue assumption). Running all three together shows how sensitive results are to the choice of model. In E-24, F1 and F2 agreed in both being negative, so confidence in the rejection is high. Markout is where the price is relative to my fill price 1, 5, 30 and 60 seconds after the fill — the standard measure of fill quality. All being negative means 'at every moment of filling, the market was moving against me', direct evidence of adverse selection. | | --- |

Summary of phase 4: 9 experiments rejected in a row. But the value of the questions this phase closed is large — the hypothesis that "maker will save it", which without validation would have left lingering regret forever, was closed through a 3-step ladder of counterfactual upper bound → fill model → official trade data. However, the exact scope of what was closed is 'static extreme-price limit entries do not yield a validated strategy family'. Dynamic execution hypotheses such as cancelling quotes after detecting toxic order flow, post-only based on trade intensity, and entering after confirming quote recovery remain unvalidated — though, as an area of intense infrastructure competition, they are kept at low priority. The by-products — an event execution engine, AggTrade replay infrastructure and 269 signals of validation data — are reused as is for the deviation validation of Phase 7 (paper trading).

5. Casebook of measured bugs — "too-good numbers are bug signals"

5 bugs actually caught in the validation code over 9 days. What they share: all were caught because results were oddly good or oddly clean. Most backtest bugs err in the optimistic direction (pessimistic bugs only kill strategies and cause no deployment accidents), so the habit of 'first asking whether it is physically possible before a verdict' is the last line of defence.

The purpose of this casebook is not self-reproach but a checklist: when building a new simulator, first check with synthetic data (1) bankruptcy handling, (2) the warm-up/slice relationship, (3) degenerate inputs (constant series, a sample of 1), (4) whether totals are within physical limits. This is why 'negative controls (nothing passes on null data)' and 'positive controls (a planted fake edge does pass)' became the standard acceptance procedure for new tools in this project.

6. Overall insights

6.1 Costs are king — cause no. 1 on every death certificate

Compressing the cause of death across some 10,000 trials into one: the signal is uncertain, but the costs are certain. A median annual Sharpe of -1.08 for 1-hour indicator combinations, and expectancy of +0.008~+0.19% per signal for the 5-minute family vs a round-trip cost of 0.2% — the same wall everywhere. Among the detours lowering costs with maker orders, the 'static limit' path was blocked by another cost, adverse selection. Implication: future candidates must either enlarge the signal (long horizon, low turnover) or have a different cost structure altogether (execution research based on measurement).

6.2 The shadow of beta — the illusion of bull-market backtests

All-long at the top of the harness, single-month concentration in rotation, the long bias that was good only in 5-minute train — all variations of the same phenomenon. On rising-market data, long strategies collect market returns without skill. The defensive tools are the return concentration check (summing after excluding the best period), regime breakdown and standalone validation of the short side, and these have been institutionalized as preregistered auxiliary verdicts for the next experiment (TSMOM).

6.3 False positives and power — the honest price tag of the filter

Multiple-testing correction is not free. The price of tightening to 4.5% false positives is missing 78% of genuine annual Sharpes of 2.0. So '0 survivors' is not proof that "the market has no edge" but the limited statement that "with this volume of data and these costs, there is no edge one can be confident in". A filter used without knowing this limit becomes superstition, not discipline — calibrating and registering the filter's power is part of the methodology.

6.4 What the process caught — cases demonstrating the discipline

Preregistration blocked post-hoc rationalization: the test opening verdict (E-06), the automatic carry scan verdict (E-09), the rotation retrial (E-13), and carrying the AggTrade long bias forward to an independent val (E-23→24). The last one in particular worked at 'the moment of greatest temptation' (+0.26% found in train).

The minimum trade-count gate blocked noise Sharpes: a val Sharpe of 0.97 on a sample of 17 trades (E-03) and pairs with 8 trades (E-15) were automatically excluded from verdicts.

An external audit (2026-07-16) pointed out 2 cases of overstatement (the -7.45% counterfactual, the scope of the maker closure) and ambiguity in the trial hierarchy — all accepted and corrected in D-032. A record that can be audited is itself evidence of discipline.

Negative and positive controls secured trust in the tools: DSR (0/400 false positives, versus uncorrected), the cointegration test (6% false detection), and the harness and pair simulators all confirmed 0 passes on null data before real data were fed in.

6.5 Accounting for realized value

Spent: 9 days of time and all strategy candidates. Gained: (1) demonstration that the sealed gate works — blocking deployment of a strategy at -7.45% in backtest (counterfactual, so not booked as an avoided real loss), (2) reusable infrastructure — 3 validation engines, 2 dedicated simulators, an event execution core, AggTrade replay, a ledger/registry and 108 tests, (3) closed questions — a quantitative record of why public-indicator combinations, crypto cointegration, 5-minute high frequency and the maker detour do not work under this cost structure, (4) the track record itself — the fact that all 10,342 trials sit within a chain of preregistration, recording and verdicts. Measured against the success criteria of Master Plan 0.2 (not profit), the project is proceeding as planned.

7. Future roadmap (as of 2026-07-16)

Priority 1 — forward validation of the fixed E-20/21 signals at measured costs (preregistered in D-033): the only candidate group in the current record that got 'within 1cm of the cost wall' (24h hold gross +0.19%/signal). The external audit pointed out, validly, that the cause of death may be not the signal but the cost assumption (fixed 10bp round-trip slippage) — the measured spread + impact of small BTC orders may be around 1~3bp round trip. Signals and parameters fully fixed, only incoming data used, BBO, spread and delayed prices recorded at signal times, 3 paths shadowed simultaneously (market / marketable limit / post-only), a sample of ≥200 signals or 8 weeks, and a verdict of a block-bootstrap 95% lower bound on net expectancy > 0 after autocorrelation correction. This is not reviving old data but the exact intended use of the D-002 incoming data.

Priority 2 — multi-asset time-series momentum (TSMOM) on daily bars: the last remaining classic mechanism. Distinction from what was rejected — rotation is cross-sectional (relative rank), TSMOM is the absolute sign per symbol; earlier experiments were single-symbol or ultra-short holds, while TSMOM is 13 symbols × multi-day holds with low turnover, so the cost share is fundamentally different. Design upgrades (reflecting the external audit): a point-in-time universe (reflecting listing and delisting dates — applying the current 13 symbols retroactively is survivorship bias), inverse volatility weighting, per-symbol and per-sector weight caps, separate validation of long/short and long/flat, reporting before and after removing BTC/ETH beta together, strengthening return concentration by removing the top 3 events rather than the best month, and separate validation capital (precedent of D-005/D-015).

Time-dependent — accumulating liquidation data: the cascade hypothesis based on real liquidations (the D-013 revisit condition) presupposes 3+ months of recording. The decision to introduce a VPS starts this clock — the only item where every delay pushes back the point at which validation becomes possible.

Priority 3 — funding × residual momentum composite factor: a new mechanism using funding not as a source of returns but as an indicator of overheated positioning — short on high positive funding + negative residual momentum, long on the opposite, beta-neutral, multi-day holds. Its mechanism differs from the rejected carry (a single bull-market factor), so it is a subject for new registration.

Priority 4 — liquidation cascade conditional branches: redefine E-10's single hypothesis of 'rebound after a crash' as continuation/reversal branches — continuation on a liquidation spike + sharp OI drop with no quote recovery; reversal on exhaustion of aggressive selling + bid recovery. The verdict criterion is not a period (3 months) but 100~200 large events. Needs real liquidation, OI and L2 data — recorder accumulation is a prerequisite.

Monitoring — funding carry retrial: automatic retrial when the 30-day rolling annualized rate recovers to 8%+. With the simulator preserved, the retrial cost ≈ 0.

Linked to Phase 7 — execution research: measure and calibrate the fill models (F1/F2) with shadow orders and BBO/L2 collection. E-21's 'right before the cost wall' candidate group is preserved as a reference group to be re-evaluated at measured costs.

Appendix A. List of experiments (E-01~E-24)

Appendix B. List of decisions (D-001~D-032)

| [Basics] To keep studying with this document. Concept path: Sharpe ratio and MDD (section 2.2) → in-sample/out-of-sample (3.2) → cost structure (3.3) → walk-forward and Monte Carlo (3.4) → multiple testing and DSR (3.5) → beta and return concentration (4.1 strategy 6) → cointegration (4.3) → adverse selection and fill models (4.4). This order overlaps with the standard curriculum of beginner → intermediate quant methodology. Recommended keywords/literature: overfitting and backtests — Bailey & López de Prado's 'The Deflated Sharpe Ratio' (2014) and the backtesting chapters of López de Prado, 'Advances in Financial Machine Learning' (2018); multiple testing — the White's Reality Check family; time-series momentum — Moskowitz, Ooi & Pedersen 'Time Series Momentum' (2012); cointegration — Engle & Granger (1987); market microstructure and adverse selection — the Glosten-Milgrom model, search 'adverse selection limit order'. Autocorrelation correction of Sharpe — Lo (2002) 'The Statistics of Sharpe Ratios'; probability of backtest overfitting — Bailey et al. 'The Probability of Backtest Overfitting' (CSCV/PBO). Hands-on reproduction: this project's repository preserves the code, settings, ledgers and verdict files of every experiment in a runnable state. The reproduction path for any experiment E-xx is the corresponding entry in experiment-log.md → the matching config/script → the verdict file in reports. | | --- |

— End of document. 2026-07-16, experiment period 2026-07-07~15. Original records: docs/experiment-log.md, docs/decisions.md, reports/harness_ledger.csv.