Backtest study
We backtested our own scoring method. Here's what didn't work.
We published the July 2026 results of a backtest on current members plus selected delisted companies. This historical snapshot has not been rerun after the calculation fixes identified in the September audit.
In the saved July 2026 result, no single factor cleared the usual significance bar (|t| ≥ 2) after Newey–West correction for overlapping windows. The sector composite's 12-month rank information coefficient was -0.027 (t = -1.00, 16 quarters): a small negative point estimate. These are historical figures awaiting a rerun after the audit fixes.
01The test
What we tested
We ran our own scoring method through a backtest and are publishing the result, the data, and the method, including the part that found nothing.
The universe is a sample of the S&P 500: 170 current members on the survivors panel, and 188 names once a delisted supplement is added. The saved report lists two dropped tickers (IPG, HES), without recording their individual exclusion reasons. We score companies at 20 quarterly dates and measure forward excess return against SPY at six and twelve months.
The method under test is our sector composite (four axes weighted 30/30/20/20) plus 24 individual factor signals: the F-Score, ROIC, margins, leverage and liquidity, growth, momentum, earnings yield, and earnings-surprise and PEAD measures. Six insider Form-4 signals had no usable history over the window and are marked as missing. The forward outlook axis is neutralized here: it would need analyst consensus for a past quarter, which we do not store.
02Method
How we measured it
For each factor we rank every ticker in the cross-section and take the Spearman rank correlation with forward excess return, the rank information coefficient (IC). A positive IC means higher-ranked names did better; zero means the ranking carried no information.
Overlapping forward windows make the quarterly IC estimates autocorrelated, which can overstate a naive t-statistic. The run uses Newey–West standard errors (Bartlett kernel, lag = ceil(months / 3) − 1) to account for this dependence; the overlapping windows remain in the sample. Separately, each company is excluded from its own peer pool. The sector composite and its axes are set out on the methodology page.
03Results
Results
The tables below give the rank IC, its Newey–West t-statistic, and the quarter count for each factor, at six and twelve months. The all-names panel comes first; the survivors-only panel is folded below it. The sector composite and F-Score rows are set apart.
| Factor | Mean IC | t (NW) | Quarters |
|---|---|---|---|
| Sector composite | -0.002 | -0.06 | 18 |
| F-Score | 0.011 | 0.32 | 18 |
| ROIC | 0.008 | 0.17 | 18 |
| Operating margin | -0.010 | -0.37 | 18 |
| FCF margin | -0.013 | -0.43 | 18 |
| Gross margin | -0.023 | -0.72 | 18 |
| Net margin | -0.050 | -1.61 | 18 |
| Debt / EBITDA | 0.004 | 0.09 | 18 |
| Net debt / EBITDA | -0.010 | -0.18 | 18 |
| Current ratio | 0.064 | 0.95 | 18 |
| Revenue growth (YoY) | 0.032 | 0.79 | 18 |
| EPS growth (YoY) | -0.016 | -0.33 | 18 |
| Momentum (12−1) | 0.035 | 0.79 | 18 |
| Earnings yield | 0.020 | 0.65 | 18 |
| FCF yield | 0.029 | 0.79 | 18 |
| Earnings surprise (SUE, 8q) | 0.003 | 0.11 | 18 |
| SUE drift | 0.012 | 0.65 | 18 |
| PEAD (CAR ±3d) | 0.003 | 0.12 | 18 |
| PEAD (CAR ±1d) | -0.002 | -0.08 | 18 |
| Factor | Mean IC | t (NW) | Quarters |
|---|---|---|---|
| Sector composite | -0.027 | -1.00 | 16 |
| F-Score | -0.021 | -0.51 | 16 |
| ROIC | 0.041 | 0.56 | 16 |
| Operating margin | -0.001 | -0.03 | 16 |
| FCF margin | -0.009 | -0.19 | 16 |
| Gross margin | -0.025 | -0.87 | 16 |
| Net margin | -0.054 | -1.40 | 16 |
| Debt / EBITDA | -0.018 | -0.27 | 16 |
| Net debt / EBITDA | -0.041 | -0.62 | 16 |
| Current ratio | 0.095 | 1.13 | 16 |
| Revenue growth (YoY) | 0.045 | 0.87 | 16 |
| EPS growth (YoY) | -0.029 | -0.56 | 16 |
| Momentum (12−1) | -0.012 | -0.27 | 16 |
| Earnings yield | 0.016 | 0.51 | 16 |
| FCF yield | 0.009 | 0.20 | 16 |
| Earnings surprise (SUE, 8q) | -0.003 | -0.09 | 16 |
| SUE drift | 0.023 | 0.68 | 16 |
| PEAD (CAR ±3d) | -0.019 | -0.72 | 16 |
| PEAD (CAR ±1d) | -0.028 | -1.01 | 16 |
Six insider Form-4 factors are left out of the tables: they had no usable history over the window.
Survivors-only panel (170 current members)
| Factor | Mean IC | t (NW) | Quarters |
|---|---|---|---|
| Sector composite | -0.005 | -0.20 | 18 |
| F-Score | 0.003 | 0.10 | 18 |
| ROIC | -0.002 | -0.05 | 18 |
| Operating margin | -0.029 | -1.05 | 18 |
| FCF margin | -0.027 | -0.76 | 18 |
| Gross margin | -0.043 | -1.23 | 18 |
| Net margin | -0.055 | -1.73 | 18 |
| Debt / EBITDA | 0.004 | 0.09 | 18 |
| Net debt / EBITDA | -0.021 | -0.36 | 18 |
| Current ratio | 0.059 | 0.80 | 18 |
| Revenue growth (YoY) | 0.032 | 0.83 | 18 |
| EPS growth (YoY) | -0.014 | -0.28 | 18 |
| Momentum (12−1) | 0.020 | 0.37 | 18 |
| Earnings yield | 0.022 | 0.66 | 18 |
| FCF yield | 0.023 | 0.67 | 18 |
| Earnings surprise (SUE, 8q) | -0.005 | -0.16 | 18 |
| SUE drift | 0.006 | 0.24 | 18 |
| PEAD (CAR ±3d) | -0.002 | -0.09 | 18 |
| PEAD (CAR ±1d) | -0.008 | -0.36 | 18 |
| Factor | Mean IC | t (NW) | Quarters |
|---|---|---|---|
| Sector composite | -0.029 | -1.06 | 16 |
| F-Score | -0.026 | -0.63 | 16 |
| ROIC | 0.026 | 0.38 | 16 |
| Operating margin | -0.029 | -1.00 | 16 |
| FCF margin | -0.027 | -0.57 | 16 |
| Gross margin | -0.048 | -1.32 | 16 |
| Net margin | -0.063 | -1.53 | 16 |
| Debt / EBITDA | -0.017 | -0.25 | 16 |
| Net debt / EBITDA | -0.054 | -0.81 | 16 |
| Current ratio | 0.093 | 1.02 | 16 |
| Revenue growth (YoY) | 0.052 | 1.00 | 16 |
| EPS growth (YoY) | -0.022 | -0.42 | 16 |
| Momentum (12−1) | -0.032 | -0.58 | 16 |
| Earnings yield | 0.017 | 0.52 | 16 |
| FCF yield | 0.001 | 0.03 | 16 |
| Earnings surprise (SUE, 8q) | -0.011 | -0.26 | 16 |
| SUE drift | 0.016 | 0.35 | 16 |
| PEAD (CAR ±3d) | -0.022 | -0.85 | 16 |
| PEAD (CAR ±1d) | -0.029 | -1.07 | 16 |
04Limits
What this does not mean
A null result is easy to over-read in either direction. A few things this study does not establish.
The research filter differs from today's “Passes the quality and price filter” rule
The candidate-cohort statistics in the JSON use a research filter with F-Score and absolute profitability and leverage checks. Today's “Passes the quality and price filter” rule uses sector-relative quality and growth checks instead. The historical calculation also uses annual figures where today's method can use trailing figures. This study does not validate the current “Passes the quality and price filter” rule's returns.
Partial survivorship correction
The all-names panel adds a delisted supplement, but the EDGAR filing wall means not every delisted name can be recovered. Some survivorship bias remains, so treat the correction as partial.
One sample, one short period
This is a single S&P 500 sample over 20 quarterly dates, and the twelve-month horizon rests on fewer of them still. That is a short record and one regime; the confidence intervals are correspondingly wide.
IC is not a portfolio return
A rank IC measures ranking information, not the return of a tradable strategy. It ignores turnover, capacity, and the transaction costs a real portfolio pays (we assume 20 basis points a side).
Not a refutation of the F-Score literature
Piotroski's 2000 F-Score, and later replications, generally find an effect, often concentrated in small, cheap, or distressed names. This test is our own sector composite, out of that period and on a large-cap sample: a different instrument, and no overturning of that work. And the sector composite is not the F-Score; the F-Score is one of its inputs.
05Consequence
What it means for this site
This is why the site's verdict is written as a classification, not a forecast. The sector composite sorts a company against its sector by a fixed rule; the backtest is the reason we attach no return promise to that sort.
We will claim predictive power only once a backtest shows a durable, out-of-period edge. So far it has not.
We classify the company against its sector by a fixed rule. This is not a price forecast or investment advice.
06Data
Data and reproducibility
These files contain the saved aggregate results for every factor, horizon, and panel, including the empty insider rows. The JSON has curated metadata, and the CSV is the flat factor-IC table. They do not include the original inputs or company-quarter observations needed to reproduce the calculations. The July 2026 figures have been preserved and have not been recalculated after the September audit fixes.
Derived from SEC EDGAR filings (public domain) and our own computed statistics; released to the public domain under CC0.
2026-07-10