[gpt 5.6 sol] Experiment audit and reasoning record

[gpt 5.6 sol] Experiment audit and reasoning record

Annotation tag: gpt 5.6 sol This name is an experiment-author tag specified by the user; it is not technical proof or an identifier of a runtime model version. Written 2026-07-15.

1. Audit conclusion

So far the system has not "failed to generate strategies"; it produced many signals but removed most of them at the gates of cost, lack of activity, independent validation, return concentration and multiple testing. The maker counterfactual audit of 2026-07-15 screened all 4,752 existing harness combinations for cost recoverability and exactly replayed 145 registered-grid + 301 harness-selected = 446 under four cost scenarios.

• Training-positive even with existing taker costs: 209
• Newly recovered with maker entry / taker exit: 36
• Recovered only under the optimistic upper bound of maker on both entry and exit: 19
• Negative expectancy even with no costs: 86
• Remaining not recovered: 96
• The 36 maker-recovered compared against the existing val: 5 exploratory survivors

This result confirms the possibility of limit-order strategies but is not evidence of real fills. It is a cost counterfactual that keeps every order filled at the existing next-bar open, so the next step is a limit/OCO execution engine.

2. Summary of experiments to date

3. Reasoning methods used

3.1 Causal decomposition of signal and cost

The same signals were replayed in the order baseline cost → maker entry/taker exit → maker-both upper bound → no cost. If negative even without costs, it was classified as a structural failure of the signal itself, not a fee problem. The 5m shock reversal is the representative case.

3.2 Activity-first judgment

Even with a high Sharpe, no judgment was made if trades were too few. For the pair strategies, before changing costs, 97% had fewer than 30 trades, so they were excluded from the maker audit priority. Conversely, for the 5m strategies, activity of 2 per day was confirmed first and then economics were judged.

3.3 Chronological independent validation

After selecting cost-sensitive candidates in train, the same rules were applied to the existing val. Parameters were not changed after seeing results. Of the 36 maker-recovered, only 5 simultaneously met Sharpe ≥0.5, PF >1, positive return and degradation ≤60% on val.

3.4 Path concentration and sensitivity

Beyond total return, removing the best month, changing sizing and changing the minimum-order threshold were examined together. Rotation flipped between -9.7%~+71% depending on sizing and was negative excluding the best month, so it was judged to have no edge.

3.5 Separating multiple testing from power

The harness's DSR strongly suppresses false positives but is a setting that would also miss most genuine annual Sharpes of 1.5. So "0 admitted" is not a universal statement that the market has no edge, but a limited conclusion that, under the registered grammar, costs and gates, there is no deployable candidate.

3.6 Physical plausibility checks

Too-good numbers, negative equity, slices shorter than the warm-up, and accidental regime filters caused by minimum orders were treated as signs of bugs or model contamination. Before statistical verdicts, fill counts and capital paths were checked for whether they were actually possible.

3.7 Distinguishing mechanism from parameters

Merely adjusting the numbers of a dead strategy is not a new hypothesis. Maker entry is treated not as a simple fee cut but as a new execution mechanism: "fills selectively without chasing price". So cost-counterfactual survivors are kept only as candidates for real limit/OCO validation, not as an official revival.

4. The 5 maker val exploratory survivors

Frequency is 0.18~0.27 per day over 293 val days. The several-per-day target cannot be reached by using these candidates as they are. A 5m/15m entry grammar must be registered separately on a real limit-order engine.

5. Next validation design

1. Add an execution engine supporting pending limit orders, TTL, ATR entry offsets and OCO TP/SL.
2. Run three fill scenarios at once: touch / 1-tick trade-through / conservative fill haircut.
3. If both TP and SL are touched within one bar, SL first; gap stops apply at the next open.
4. Use the 5 above as a reference group and check whether performance holds with real limit fills.
5. Separately preregister a 5m/15m trend-breakout-then-pullback limit strategy and test a 0.5~3 per day target.

6. Additional reasoning from the real limit-order path

What survived the maker counterfactual differed from what survives with real pending limits. D-022's RSI/BB extreme orders had a fill rate that was too high, 84~91%, even with a 20-minute TTL and a regime filter, and 31.5% of the best combination's trades were stopped out on the entry bar. This means the limit did not select a "good price" but selected bars where a further drop or rise occurred.

So the next limit-order hypothesis must change the order.

1. Use an extreme touch only as an observation state (armed), not as an order signal.
2. Recovery is confirmed when the close returns inside the BB or RSI crosses back over its threshold.
3. After confirmed recovery, place the limit at the first pullback, so no order is exposed to a price that is still moving.
4. Keep TTL, non-fills, OCO and trade-through stress unchanged.

This is not a numeric tweak of D-023 but a separate mechanism with a different order of information.

Related raw results:

• reports/maker_counterfactual_audit.csv
• reports/maker_counterfactual_summary.json
• reports/maker_counterfactual_val.csv