Backtest study

We backtested our own scoring method. Here's what didn't work.

We published the July 2026 results of a backtest on current members plus selected delisted companies. This historical snapshot has not been rerun after the calculation fixes identified in the September audit.

In the saved July 2026 result, no single factor cleared the usual significance bar (|t| ≥ 2) after Newey–West correction for overlapping windows. The sector composite's 12-month rank information coefficient was -0.027 (t = -1.00, 16 quarters): a small negative point estimate. These are historical figures awaiting a rerun after the audit fixes.

01The test

What we tested

We ran our own scoring method through a backtest and are publishing the result, the data, and the method, including the part that found nothing.

The universe is a sample of the S&P 500: 170 current members on the survivors panel, and 188 names once a delisted supplement is added. The saved report lists two dropped tickers (IPG, HES), without recording their individual exclusion reasons. We score companies at 20 quarterly dates and measure forward excess return against SPY at six and twelve months.

The method under test is our sector composite (four axes weighted 30/30/20/20) plus 24 individual factor signals: the F-Score, ROIC, margins, leverage and liquidity, growth, momentum, earnings yield, and earnings-surprise and PEAD measures. Six insider Form-4 signals had no usable history over the window and are marked as missing. The forward outlook axis is neutralized here: it would need analyst consensus for a past quarter, which we do not store.

02Method

How we measured it

For each factor we rank every ticker in the cross-section and take the Spearman rank correlation with forward excess return, the rank information coefficient (IC). A positive IC means higher-ranked names did better; zero means the ranking carried no information.

Overlapping forward windows make the quarterly IC estimates autocorrelated, which can overstate a naive t-statistic. The run uses Newey–West standard errors (Bartlett kernel, lag = ceil(months / 3) − 1) to account for this dependence; the overlapping windows remain in the sample. Separately, each company is excluded from its own peer pool. The sector composite and its axes are set out on the methodology page.

How we calculate

03Results

Results

The tables below give the rank IC, its Newey–West t-statistic, and the quarter count for each factor, at six and twelve months. The all-names panel comes first; the survivors-only panel is folded below it. The sector composite and F-Score rows are set apart.

6m · All names (188)
FactorMean ICt (NW)Quarters
Sector composite-0.002-0.0618
F-Score0.0110.3218
ROIC0.0080.1718
Operating margin-0.010-0.3718
FCF margin-0.013-0.4318
Gross margin-0.023-0.7218
Net margin-0.050-1.6118
Debt / EBITDA0.0040.0918
Net debt / EBITDA-0.010-0.1818
Current ratio0.0640.9518
Revenue growth (YoY)0.0320.7918
EPS growth (YoY)-0.016-0.3318
Momentum (12−1)0.0350.7918
Earnings yield0.0200.6518
FCF yield0.0290.7918
Earnings surprise (SUE, 8q)0.0030.1118
SUE drift0.0120.6518
PEAD (CAR ±3d)0.0030.1218
PEAD (CAR ±1d)-0.002-0.0818
12m · All names (188)
FactorMean ICt (NW)Quarters
Sector composite-0.027-1.0016
F-Score-0.021-0.5116
ROIC0.0410.5616
Operating margin-0.001-0.0316
FCF margin-0.009-0.1916
Gross margin-0.025-0.8716
Net margin-0.054-1.4016
Debt / EBITDA-0.018-0.2716
Net debt / EBITDA-0.041-0.6216
Current ratio0.0951.1316
Revenue growth (YoY)0.0450.8716
EPS growth (YoY)-0.029-0.5616
Momentum (12−1)-0.012-0.2716
Earnings yield0.0160.5116
FCF yield0.0090.2016
Earnings surprise (SUE, 8q)-0.003-0.0916
SUE drift0.0230.6816
PEAD (CAR ±3d)-0.019-0.7216
PEAD (CAR ±1d)-0.028-1.0116

Six insider Form-4 factors are left out of the tables: they had no usable history over the window.

Survivors-only panel (170 current members)
6m · Survivors only (170)
FactorMean ICt (NW)Quarters
Sector composite-0.005-0.2018
F-Score0.0030.1018
ROIC-0.002-0.0518
Operating margin-0.029-1.0518
FCF margin-0.027-0.7618
Gross margin-0.043-1.2318
Net margin-0.055-1.7318
Debt / EBITDA0.0040.0918
Net debt / EBITDA-0.021-0.3618
Current ratio0.0590.8018
Revenue growth (YoY)0.0320.8318
EPS growth (YoY)-0.014-0.2818
Momentum (12−1)0.0200.3718
Earnings yield0.0220.6618
FCF yield0.0230.6718
Earnings surprise (SUE, 8q)-0.005-0.1618
SUE drift0.0060.2418
PEAD (CAR ±3d)-0.002-0.0918
PEAD (CAR ±1d)-0.008-0.3618
12m · Survivors only (170)
FactorMean ICt (NW)Quarters
Sector composite-0.029-1.0616
F-Score-0.026-0.6316
ROIC0.0260.3816
Operating margin-0.029-1.0016
FCF margin-0.027-0.5716
Gross margin-0.048-1.3216
Net margin-0.063-1.5316
Debt / EBITDA-0.017-0.2516
Net debt / EBITDA-0.054-0.8116
Current ratio0.0931.0216
Revenue growth (YoY)0.0521.0016
EPS growth (YoY)-0.022-0.4216
Momentum (12−1)-0.032-0.5816
Earnings yield0.0170.5216
FCF yield0.0010.0316
Earnings surprise (SUE, 8q)-0.011-0.2616
SUE drift0.0160.3516
PEAD (CAR ±3d)-0.022-0.8516
PEAD (CAR ±1d)-0.029-1.0716

04Limits

What this does not mean

A null result is easy to over-read in either direction. A few things this study does not establish.

  • The research filter differs from today's “Passes the quality and price filter” rule

    The candidate-cohort statistics in the JSON use a research filter with F-Score and absolute profitability and leverage checks. Today's “Passes the quality and price filter” rule uses sector-relative quality and growth checks instead. The historical calculation also uses annual figures where today's method can use trailing figures. This study does not validate the current “Passes the quality and price filter” rule's returns.

  • Partial survivorship correction

    The all-names panel adds a delisted supplement, but the EDGAR filing wall means not every delisted name can be recovered. Some survivorship bias remains, so treat the correction as partial.

  • One sample, one short period

    This is a single S&P 500 sample over 20 quarterly dates, and the twelve-month horizon rests on fewer of them still. That is a short record and one regime; the confidence intervals are correspondingly wide.

  • IC is not a portfolio return

    A rank IC measures ranking information, not the return of a tradable strategy. It ignores turnover, capacity, and the transaction costs a real portfolio pays (we assume 20 basis points a side).

  • Not a refutation of the F-Score literature

    Piotroski's 2000 F-Score, and later replications, generally find an effect, often concentrated in small, cheap, or distressed names. This test is our own sector composite, out of that period and on a large-cap sample: a different instrument, and no overturning of that work. And the sector composite is not the F-Score; the F-Score is one of its inputs.

05Consequence

What it means for this site

This is why the site's verdict is written as a classification, not a forecast. The sector composite sorts a company against its sector by a fixed rule; the backtest is the reason we attach no return promise to that sort.

We will claim predictive power only once a backtest shows a durable, out-of-period edge. So far it has not.

We classify the company against its sector by a fixed rule. This is not a price forecast or investment advice.

The method's tracked recordHow we calculate

06Data

Data and reproducibility

These files contain the saved aggregate results for every factor, horizon, and panel, including the empty insider rows. The JSON has curated metadata, and the CSV is the flat factor-IC table. They do not include the original inputs or company-quarter observations needed to reproduce the calculations. The July 2026 figures have been preserved and have not been recalculated after the September audit fixes.

Derived from SEC EDGAR filings (public domain) and our own computed statistics; released to the public domain under CC0.

2026-07-10