polyphys.analyze.stats#
polyphys.analyze.stats#
The polyphys.analyze.stats module estimates the uncertainty of a
time average over correlated molecular-dynamics frames with the blocking
method of Flyvbjerg and Petersen.
Consecutive frames of a trajectory are not independent. The textbook standard error of the mean, \(\sigma / \sqrt{n}\), where \(n\) is the number of frames and \(\sigma\) the standard deviation of the observable over them, counts every frame as a fresh observation and so understates the true error, often by a large factor. Blocking averages neighboring frames in pairs, repeatedly, and recomputes the standard error at each level. Once the blocks are longer than the correlation time, the lag over which the observable’s autocorrelation decays, the block means are effectively independent and the estimate stops growing: that plateau is the error bar to report. The ratio of the plateau variance of the mean to the naive one, the statistical inefficiency \(s\), says how many frames one independent sample is worth; the true standard error is \(\sigma \sqrt{s / n}\).
The module distinguishes two kinds of average that look alike in a table:
a time average over the frames of one run (one
whole), whose frames are correlated and whose error needs this module;an ensemble average over independent replicates (one
ensemble), whose members are independent runs and whose error is the naive standard error of the replicate means, aspolyphys.analyze.measurer.sem()computes it.
Notation#
- \(x_t\)
The value of one observable at frame \(t = 1, \dots, n\).
- \(n\)
The number of frames,
n_frames.- \(\sigma^2\)
The variance of \(x_t\) over the frames, with
ddof=1.- \(k\)
The blocking level, \(k = 0, 1, 2, \dots\).
- \(B\)
The block size in frames at level \(k\), \(B = 2^k\).
- \(m\)
The number of blocks at that level, \(\lfloor n / B \rfloor\).
- \(\mathrm{SE}(B)\)
The standard error of the mean estimated from the \(m\) block means; \(\mathrm{SE}(1) = \sigma / \sqrt{n}\) is the naive estimate.
- \(s(B)\)
The statistical inefficiency estimated at block size \(B\): the variance of the block means times \(B\), divided by \(\sigma^2\). When \(n\) is a power of two, \(s(B) = (\mathrm{SE}(B) / \mathrm{SE}(1))^2\).
- \(s\)
The statistical inefficiency, the plateau of \(s(B)\) at block sizes beyond the correlation time.
- \(\rho(t)\)
The normalized autocorrelation function of \(x_t\) at a lag of \(t\) frames, with \(\rho(0) = 1\); for a stationary series \(s = \sum_{t=-\infty}^{\infty} \rho(t)\).
- \(\tau_\mathrm{int}\)
The integrated autocorrelation time in Wolff’s (2004) convention, \(\tau_\mathrm{int} = s / 2\), in frames.
Contents#
The full blocking table:
block_average() returns a BlockingResult, one row per blocking
level, so the caller can read the plateau directly, as Flyvbjerg and Petersen
do.
Choosing the plateau:
select_plateau() applies one of two rules to a table, chosen with the
method keyword; there is no default, because the rule is an estimator
choice.
'lee'The block size satisfies \(B^3 > 2 n (\mathrm{SE}(B) / \mathrm{SE}(1))^4\), scanning from the largest block downward, as Lee et al. (2011), Eq. (14), and the
pyblockpackage implement it. Wolff (2004), Eq. (47), derives the same balance between the systematic and the statistical error of the error estimate, with a constant four times smaller in \(B^3\), because his correlation time is \(\tau_\mathrm{int} = s / 2\) where Lee et al. use \(s\).'sem_err'The first level whose standard error lies within one error bar of each of the next two levels. A convenience heuristic without a published source; it tends to pick the start of the plateau and so to err on the low side.
Under either rule, select_plateau() warns when the series fails the
acceptance criterion of Lee et al. (2011): the block size the 'lee' rule
selects must be below \(n / 50\), which amounts to \(n / s > 500\),
more than about 500 independent samples.
Convenience wrappers:
sem_blocked() and statistical_inefficiency() run both steps and
return the selected row’s value.
Conventions#
Frames run along axis 0 of the input, shape
(n_frames, ...). Every trailing axis is an independent observable: a per-frame histogram(n_frames, n_bins)gives one error bar per bin.Block sizes are powers of two,
1, 2, 4, .... Levelkhas block size2**k; when a level has an odd number of blocks, the last one is dropped before the next pairing, as Allen and Tildesley’s example code does.Variances use
ddof=1. The error of a standard error, \(\mathrm{SE} / \sqrt{2 (m - 1)}\) for \(m\) blocks, and the error of the inefficiency, \(s \sqrt{2 / (m - 1)}\), are the leading-order Gaussian estimates (Lee et al. 2011, Eq. (15)).Results are in the units of the input; the statistical inefficiency is dimensionless, in frames per independent sample.
The input must be stationary: discard the equilibration transient before calling. A drift or a rare switch between metastable states shows up as a standard error that keeps rising with block size, and the selectors then report that no plateau was found rather than pick a level.
References
H. Flyvbjerg and H. G. Petersen, “Error estimates on averages of correlated data”, J. Chem. Phys. 91, 461-466 (1989). https://doi.org/10.1063/1.457480
U. Wolff (ALPHA Collaboration), “Monte Carlo errors with less errors”, Comput. Phys. Commun. 156, 143-153 (2004); erratum 176, 383 (2007). https://doi.org/10.1016/S0010-4655(03)00467-3
R. M. Lee, G. J. Conduit, N. Nemec, P. López Ríos, and N. D. Drummond, “Strategies for improving the efficiency of quantum Monte Carlo calculations”, Phys. Rev. E 83, 066706 (2011); erratum 85, 029908 (2012). https://doi.org/10.1103/PhysRevE.83.066706
M. P. Allen and D. J. Tildesley, Computer Simulation of Liquids, 2nd ed. (Oxford University Press, 2017); example code
error_calc.pyat Allen-Tildesley/examplesJ. S. Spencer,
pyblock:pyblock.blocking.find_optimal_block. jsspencer/pyblock
Added in version 0.5.0.
Module Attributes
The plateau-selection rules |
Functions
|
Compute the Flyvbjerg-Petersen blocking table of a time series. |
|
Pick the blocking level that estimates the plateau, by a named rule. |
|
Standard error of a time average over correlated frames. |
|
Estimate the frames per independent sample of a time series, its statistical inefficiency, from its blocking table. |
Classes
|
The blocking table of one time series, one row per blocking level. |
- class polyphys.analyze.stats.BlockingResult(n_frames, block_size, n_blocks, mean, var, sem, sem_err, statistical_inefficiency, inefficiency_err)[source]#
Bases:
objectThe blocking table of one time series, one row per blocking level.
Level
kaverages the frames in contiguous blocks of2**kframes and treats the block means as the sample. Every per-level array has the level along axis 0; the trailing axes are those of one frame of the input, one entry per observable.- Variables:
n_frames (
int) – Number of frames in the input, the sample size at level 0.block_size (
numpy.ndarray) – Frames per block at each level,2**k; shape(n_levels,),int64.n_blocks (
numpy.ndarray) – Blocks at each level,n_frames // block_size; shape(n_levels,),int64.mean (
numpy.ndarray) – Mean of the block means, in the units of the input; shape(n_levels, *observable_shape),float64. It changes across levels only when a trailing frame is dropped at an odd level.var (
numpy.ndarray) – Variance of the block means withddof=1, in squared input units.sem (
numpy.ndarray) – Standard error of the mean at each level,sqrt(var / n_blocks), in the units of the input. Level 0 is the naive estimate.sem_err (
numpy.ndarray) – Leading-order error ofsem,sem / sqrt(2 (n_blocks - 1)).statistical_inefficiency (
numpy.ndarray) –block_size * var / var[0], dimensionless: frames per independent sample, as estimated at that level. Level 0 is exactly 1.inefficiency_err (
numpy.ndarray) – Leading-order error of the inefficiency,statistical_inefficiency * sqrt(2 / (n_blocks - 1)). It neglects the error ofvar[0].
- Parameters:
n_frames (int)
block_size (NDArray[int64])
n_blocks (NDArray[int64])
mean (NDArray[float64])
var (NDArray[float64])
sem (NDArray[float64])
sem_err (NDArray[float64])
statistical_inefficiency (NDArray[float64])
inefficiency_err (NDArray[float64])
Examples
>>> import numpy as np >>> from polyphys.analyze.stats import block_average >>> table = block_average(np.repeat([1.0, 3.0, 2.0, 4.0], 4)) >>> table.n_levels, table.observable_shape (3, ()) >>> table.block_size array([1, 2, 4])
Added in version 0.5.0.
- block_size: numpy.typing.NDArray[numpy.int64]#
- inefficiency_err: numpy.typing.NDArray[numpy.float64]#
- mean: numpy.typing.NDArray[numpy.float64]#
- n_blocks: numpy.typing.NDArray[numpy.int64]#
- property observable_shape: tuple[int, ...]#
Shape of one frame of the input,
()for a scalar series.
- sem: numpy.typing.NDArray[numpy.float64]#
- sem_err: numpy.typing.NDArray[numpy.float64]#
- statistical_inefficiency: numpy.typing.NDArray[numpy.float64]#
- var: numpy.typing.NDArray[numpy.float64]#
- polyphys.analyze.stats.PlateauMethod#
The plateau-selection rules
select_plateau()accepts.alias of
Literal[‘lee’, ‘sem_err’]
- polyphys.analyze.stats.block_average(x, min_blocks=4)[source]#
Compute the Flyvbjerg-Petersen blocking table of a time series.
Starting from the frames themselves (block size 1), the series is replaced by the means of consecutive pairs, and the mean, variance, standard error, and statistical inefficiency are recorded at each level, until the next halving would leave fewer than min_blocks blocks. The full table is option (a) of the plateau choice: the caller reads where the standard error stops growing.
- Parameters:
x (
numpy.typing.ArrayLike) – Time series with frames along axis 0, shape(n_frames, ...). Every trailing axis is an independent observable. Any numeric dtype; the computation is infloat64and x is not modified.min_blocks (
int) – Smallest number of blocks kept; at least 2. The last level has at least this many blocks. With 4 blocks the error of the standard error is already 41% of its value, so levels this deep are mostly a guard against reading noise as a plateau.
- Returns:
The table; see its attributes for shapes and units.
- Return type:
- Raises:
ValueError – If x is a scalar, holds NaN or inf, has fewer than min_blocks frames, or has an observable that is constant across frames (its inefficiency would be 0/0); or if min_blocks is below 2.
Notes
For a series \(x_t\) of \(n\) frames, level \(k = 0, 1, \dots\) has block size \(B_k = 2^k\) frames, \(m_k = \lfloor n / B_k \rfloor\) blocks, and block means \(\bar{x}_j^{(k)}\), the average of \(x_t\) over block \(j = 1, \dots, m_k\). With \(\langle \bar{x}^{(k)} \rangle\) the mean of the block means, the table records
\[\mathrm{var}_k = \frac{1}{m_k - 1} \sum_{j=1}^{m_k} \left(\bar{x}_j^{(k)} - \langle \bar{x}^{(k)} \rangle\right)^2, \quad \mathrm{SE}_k = \sqrt{\mathrm{var}_k / m_k}, \quad s_k = B_k \, \mathrm{var}_k / \mathrm{var}_0 ,\]the variance of the block means, the standard error of the mean, and the statistical inefficiency at level \(k\); \(\mathrm{var}_0\) is the variance of the frames themselves. As \(B_k\) grows past the correlation time, \(\mathrm{SE}_k\) reaches a plateau at the true standard error of the time average and \(s_k\) at the statistical inefficiency \(s = \sum_{t=-\infty}^{\infty} \rho(t)\), where \(\rho(t)\) is the normalized autocorrelation function of \(x_t\) at a lag of \(t\) frames. This is twice the integrated autocorrelation time \(\tau_\mathrm{int}\) of Wolff (2004), Eq. (23) and (25).
When a level has an odd number of blocks, the trailing block is left out of the next pairing, so the frames used shrink by at most one block per level and
meancan drift slightly between levels.With \(n_\mathrm{obs}\) the number of observables, the product of the trailing dimensions of x (1 for a 1-D series), time complexity is \(O(n \, n_\mathrm{obs})\), since the levels touch \(n + n/2 + n/4 + \dots < 2n\) frames in all. Space is one
float64copy of the input, halved in place, plus the table of \(O(\log_2 n \, n_\mathrm{obs})\) entries.References
H. Flyvbjerg and H. G. Petersen, J. Chem. Phys. 91, 461 (1989), https://doi.org/10.1063/1.457480. Implementation after M. P. Allen and D. J. Tildesley, Computer Simulation of Liquids, 2nd ed. (2017), example code
error_calc.py.Examples
A signal of \(n = 16\) frames that holds each value for \(L = 4\) frames has an exact table: for every block size \(B \le L\) the block means are the held values themselves, so the inefficiency is \(s(B) = B (n - 1) / (n - B)\), here 1, 30/14 and 5.
>>> import numpy as np >>> from polyphys.analyze.stats import block_average >>> x = np.repeat([1.0, 3.0, 2.0, 4.0], 4) >>> table = block_average(x, min_blocks=4) >>> table.block_size, table.n_blocks (array([1, 2, 4]), array([16, 8, 4])) >>> table.statistical_inefficiency.round(4) array([1. , 2.1429, 5. ]) >>> table.sem.round(4) array([0.2887, 0.4226, 0.6455])
A two-column input gives one table column per observable:
>>> table = block_average(np.column_stack([x, 2.0 * x])) >>> table.observable_shape, table.sem.shape ((2,), (3, 2)) >>> np.allclose(table.sem[:, 1], 2.0 * table.sem[:, 0]) True
Added in version 0.5.0.
- polyphys.analyze.stats.select_plateau(result, *, method)[source]#
Pick the blocking level that estimates the plateau, by a named rule.
- Parameters:
result (
polyphys.analyze.stats.BlockingResult) – A table fromblock_average().method (
typing.Literal['lee','sem_err']) –The selection rule; required, because it is an estimator choice.
'lee'The largest-block-downward scan of Lee et al. (2011): a level qualifies when its block size \(B\), in frames, satisfies \(B^3 > 2 n (\mathrm{SE}(B) / \mathrm{SE}(1))^4\), where \(n\) is the number of frames, \(\mathrm{SE}(B)\) the level’s standard error, and \(\mathrm{SE}(1)\) the naive one at \(B = 1\). The selected level is the smallest block size of the qualifying run that includes the largest block. The rule balances the systematic underestimate of the error at small blocks against its growing statistical error at large ones.
'sem_err'The first level whose standard error is within one
sem_errof each of the next two levels, that is, where the table stops rising. A convenience heuristic, not a published rule.
- Returns:
The selected level index into the table’s axis 0: an
intfor a scalar series, else anint64array of shaperesult.observable_shape.-1marks an observable for which no level qualifies.- Return type:
- Raises:
ValueError – If method is not one of the rules above.
- Warns:
RuntimeWarning – When no level qualifies for some observable, which means the series is too short for its correlation time or is not stationary. Also, under either method, when an observable has a selected level but the block size the
'lee'rule selects is not belown_frames / 50, or that rule finds no level: Lee et al. (2011) then consider the estimate unreliable. The selected level is still returned.
Notes
With \(n_\mathrm{levels}\) the number of rows of the table and \(n_\mathrm{obs}\) the number of observables, both rules are \(O(n_\mathrm{levels} \, n_\mathrm{obs})\) in time and space.
Lee et al. write their Eq. (14) as \(B_\mathrm{opt} = (2 n n_\mathrm{corr}^2)^{1/3}\), the optimal block size for \(n\) frames, with \(n_\mathrm{corr}\) the statistical inefficiency \(s\), the plateau of \((\mathrm{SE}(B) / \mathrm{SE}(1))^2\); hence the fourth power in the rule above. They cite Wolff (2004), whose Eq. (47) reads \(B = \tau (2N/\tau)^{1/3}\) for \(N\) frames, with \(\tau\) the integrated autocorrelation time, which is \(s / 2\). The two prescriptions therefore differ by a factor of 4 in \(B^3\); neither constant is exact, as Wolff’s derivation is an order-of-magnitude balance. This function implements Lee’s form, as
pyblockdoes.Lee et al. accept the analysis only when \(n / B_\mathrm{opt} > 50\), which with Eq. (14) reads \(n / s > 500\): the series must hold more than about 500 independent samples. That is a property of the series, not of the rule, so this function tests it at the
'lee'level under either method. The'sem_err'rule can stop at a level with more than 50 blocks on a series that fails the test, where the table levels off before the blocks outgrow the correlation time.References
R. M. Lee et al., Phys. Rev. E 83, 066706 (2011), Sec. IV, Eq. (13)-(15). U. Wolff, Comput. Phys. Commun. 156, 143 (2004), Eq. (46)-(48).
Examples
A signal that holds each value for 8 frames has a statistical inefficiency \(s \approx 8\), so on the plateau \((\mathrm{SE}(B) / \mathrm{SE}(1))^4 = s^2 \approx 8^2\). With \(n = 16384\) frames the Lee rule then needs \(B^3 > 2 \cdot 16384 \cdot 8^2\), that is \(B > 128\), and it selects the first power of two above 128:
>>> import numpy as np >>> from polyphys.analyze.stats import block_average, select_plateau >>> rng = np.random.default_rng(7) >>> x = np.repeat(rng.standard_normal(2048), 8) >>> table = block_average(x) >>> level = select_plateau(table, method='lee') >>> int(table.block_size[level]) 256
The
'sem_err'rule stops where the standard error first flattens, which for this signal is at or just past the plateau length:>>> level = select_plateau(table, method='sem_err') >>> bool(table.block_size[level] >= 8) True
Holding each of 64 values for 256 frames gives \(s \approx 256\), so the series holds only 64 independent samples. The
'sem_err'rule stops at the period, where 64 blocks remain, but the Lee block size leaves fewer than 50, so both rules warn:>>> import warnings >>> rng = np.random.default_rng(7) >>> x = np.repeat(rng.standard_normal(64), 256) >>> table = block_average(x) >>> with warnings.catch_warnings(record=True) as caught: ... warnings.simplefilter('always') ... level = select_plateau(table, method='sem_err') >>> int(table.block_size[level]), int(table.n_blocks[level]) (256, 64) >>> [type(w.message).__name__ for w in caught] ['RuntimeWarning']
Added in version 0.5.0.
- polyphys.analyze.stats.sem_blocked(x, *, method, min_blocks=4)[source]#
Standard error of a time average over correlated frames.
Runs
block_average()andselect_plateau()and returns the standard error at the selected level. This is the error bar to report for the mean of x over its frames.- Parameters:
x (
numpy.typing.ArrayLike) – Time series with frames along axis 0, shape(n_frames, ...).method (
typing.Literal['lee','sem_err']) – The plateau rule; seeselect_plateau().min_blocks (
int) – Passed toblock_average().
- Returns:
The blocked standard error, in the units of x: a
floatfor a scalar series, else afloat64array of shapex.shape[1:]. NaN where no plateau was found.- Return type:
- Raises:
ValueError – As
block_average()andselect_plateau().- Warns:
RuntimeWarning – As
select_plateau().
Notes
For independent frames the result agrees with
polyphys.analyze.measurer.sem()within its own error bar; for correlated frames it is larger by \(\sqrt{s}\), where \(s\) is the statistical inefficiency, the number of frames that make one independent sample. With \(n\) the number of frames and \(n_\mathrm{obs}\) the number of observables (the product of the trailing dimensions of x), time is \(O(n \, n_\mathrm{obs})\) and space onefloat64copy of x.Examples
Each value of this signal is held for 8 frames, so \(s \approx 8\) and the blocked error is about \(\sqrt{8} \approx 2.8\) times the naive one:
>>> import numpy as np >>> from polyphys.analyze.stats import sem_blocked >>> from polyphys.analyze.measurer import sem >>> rng = np.random.default_rng(7) >>> x = np.repeat(rng.standard_normal(2048), 8) >>> ratio = sem_blocked(x, method='lee') / sem(x) >>> bool(2.3 < ratio < 3.4) True
Added in version 0.5.0.
- polyphys.analyze.stats.statistical_inefficiency(x, *, method, min_blocks=4)[source]#
Estimate the frames per independent sample of a time series, its statistical inefficiency, from its blocking table.
Runs
block_average()andselect_plateau()and returns the statistical inefficiency at the selected level: the ratio of the plateau variance of the mean to the naive one, son_frames / sis the effective number of independent frames.- Parameters:
x (
numpy.typing.ArrayLike) – Time series with frames along axis 0, shape(n_frames, ...).method (
typing.Literal['lee','sem_err']) – The plateau rule; seeselect_plateau().min_blocks (
int) – Passed toblock_average().
- Returns:
Dimensionless, in frames; 1 for independent frames, below 1 for anticorrelated ones. A
floatfor a scalar series, else afloat64array of shapex.shape[1:]. NaN where no plateau was found.- Return type:
- Raises:
ValueError – As
block_average()andselect_plateau().- Warns:
RuntimeWarning – As
select_plateau().
Notes
For a stationary series \(x_t\) whose normalized autocorrelation function at a lag of \(t\) frames is \(\rho(t)\), with \(\rho(0) = 1\), the inefficiency is \(s = \sum_{t=-\infty}^{\infty} \rho(t) = 2 \tau_\mathrm{int}\), where \(\tau_\mathrm{int}\) is the integrated autocorrelation time (Wolff 2004, Eq. (23) and (25)). For an AR(1) process, \(x_t = \phi x_{t-1} + \epsilon_t\) with coefficient \(|\phi| < 1\) and independent noise \(\epsilon_t\), \(\rho(t) = \phi^{|t|}\) and \(s = (1 + \phi) / (1 - \phi)\). With \(n\) the number of frames and \(n_\mathrm{obs}\) the number of observables (the product of the trailing dimensions of x), time is \(O(n \, n_\mathrm{obs})\) and space one
float64copy of x.Examples
Each value of this signal is held for 8 frames, so \(s \approx 8\):
>>> import numpy as np >>> from polyphys.analyze.stats import statistical_inefficiency >>> rng = np.random.default_rng(7) >>> x = np.repeat(rng.standard_normal(2048), 8) >>> s = statistical_inefficiency(x, method='lee') >>> bool(5.0 < s < 11.0) True
Added in version 0.5.0.