polyphys.analyze.stats#

polyphys.analyze.stats#

The polyphys.analyze.stats module estimates the uncertainty of a time average over correlated molecular-dynamics frames with the blocking method of Flyvbjerg and Petersen.

Consecutive frames of a trajectory are not independent. The textbook standard error of the mean, \(\sigma / \sqrt{n}\), where \(n\) is the number of frames and \(\sigma\) the standard deviation of the observable over them, counts every frame as a fresh observation and so understates the true error, often by a large factor. Blocking averages neighboring frames in pairs, repeatedly, and recomputes the standard error at each level. Once the blocks are longer than the correlation time, the lag over which the observable’s autocorrelation decays, the block means are effectively independent and the estimate stops growing: that plateau is the error bar to report. The ratio of the plateau variance of the mean to the naive one, the statistical inefficiency \(s\), says how many frames one independent sample is worth; the true standard error is \(\sigma \sqrt{s / n}\).

The module distinguishes two kinds of average that look alike in a table:

  • a time average over the frames of one run (one whole), whose frames are correlated and whose error needs this module;

  • an ensemble average over independent replicates (one ensemble), whose members are independent runs and whose error is the naive standard error of the replicate means, as polyphys.analyze.measurer.sem() computes it.

Notation#

\(x_t\)

The value of one observable at frame \(t = 1, \dots, n\).

\(n\)

The number of frames, n_frames.

\(\sigma^2\)

The variance of \(x_t\) over the frames, with ddof=1.

\(k\)

The blocking level, \(k = 0, 1, 2, \dots\).

\(B\)

The block size in frames at level \(k\), \(B = 2^k\).

\(m\)

The number of blocks at that level, \(\lfloor n / B \rfloor\).

\(\mathrm{SE}(B)\)

The standard error of the mean estimated from the \(m\) block means; \(\mathrm{SE}(1) = \sigma / \sqrt{n}\) is the naive estimate.

\(s(B)\)

The statistical inefficiency estimated at block size \(B\): the variance of the block means times \(B\), divided by \(\sigma^2\). When \(n\) is a power of two, \(s(B) = (\mathrm{SE}(B) / \mathrm{SE}(1))^2\).

\(s\)

The statistical inefficiency, the plateau of \(s(B)\) at block sizes beyond the correlation time.

\(\rho(t)\)

The normalized autocorrelation function of \(x_t\) at a lag of \(t\) frames, with \(\rho(0) = 1\); for a stationary series \(s = \sum_{t=-\infty}^{\infty} \rho(t)\).

\(\tau_\mathrm{int}\)

The integrated autocorrelation time in Wolff’s (2004) convention, \(\tau_\mathrm{int} = s / 2\), in frames.

Contents#

The full blocking table: block_average() returns a BlockingResult, one row per blocking level, so the caller can read the plateau directly, as Flyvbjerg and Petersen do.

Choosing the plateau: select_plateau() applies one of two rules to a table, chosen with the method keyword; there is no default, because the rule is an estimator choice.

'lee'

The block size satisfies \(B^3 > 2 n (\mathrm{SE}(B) / \mathrm{SE}(1))^4\), scanning from the largest block downward, as Lee et al. (2011), Eq. (14), and the pyblock package implement it. Wolff (2004), Eq. (47), derives the same balance between the systematic and the statistical error of the error estimate, with a constant four times smaller in \(B^3\), because his correlation time is \(\tau_\mathrm{int} = s / 2\) where Lee et al. use \(s\).

'sem_err'

The first level whose standard error lies within one error bar of each of the next two levels. A convenience heuristic without a published source; it tends to pick the start of the plateau and so to err on the low side.

Under either rule, select_plateau() warns when the series fails the acceptance criterion of Lee et al. (2011): the block size the 'lee' rule selects must be below \(n / 50\), which amounts to \(n / s > 500\), more than about 500 independent samples.

Convenience wrappers: sem_blocked() and statistical_inefficiency() run both steps and return the selected row’s value.

Conventions#

  • Frames run along axis 0 of the input, shape (n_frames, ...). Every trailing axis is an independent observable: a per-frame histogram (n_frames, n_bins) gives one error bar per bin.

  • Block sizes are powers of two, 1, 2, 4, .... Level k has block size 2**k; when a level has an odd number of blocks, the last one is dropped before the next pairing, as Allen and Tildesley’s example code does.

  • Variances use ddof=1. The error of a standard error, \(\mathrm{SE} / \sqrt{2 (m - 1)}\) for \(m\) blocks, and the error of the inefficiency, \(s \sqrt{2 / (m - 1)}\), are the leading-order Gaussian estimates (Lee et al. 2011, Eq. (15)).

  • Results are in the units of the input; the statistical inefficiency is dimensionless, in frames per independent sample.

  • The input must be stationary: discard the equilibration transient before calling. A drift or a rare switch between metastable states shows up as a standard error that keeps rising with block size, and the selectors then report that no plateau was found rather than pick a level.

References

  • H. Flyvbjerg and H. G. Petersen, “Error estimates on averages of correlated data”, J. Chem. Phys. 91, 461-466 (1989). https://doi.org/10.1063/1.457480

  • U. Wolff (ALPHA Collaboration), “Monte Carlo errors with less errors”, Comput. Phys. Commun. 156, 143-153 (2004); erratum 176, 383 (2007). https://doi.org/10.1016/S0010-4655(03)00467-3

  • R. M. Lee, G. J. Conduit, N. Nemec, P. López Ríos, and N. D. Drummond, “Strategies for improving the efficiency of quantum Monte Carlo calculations”, Phys. Rev. E 83, 066706 (2011); erratum 85, 029908 (2012). https://doi.org/10.1103/PhysRevE.83.066706

  • M. P. Allen and D. J. Tildesley, Computer Simulation of Liquids, 2nd ed. (Oxford University Press, 2017); example code error_calc.py at Allen-Tildesley/examples

  • J. S. Spencer, pyblock: pyblock.blocking.find_optimal_block. jsspencer/pyblock

Added in version 0.5.0.

Module Attributes

PlateauMethod

The plateau-selection rules select_plateau() accepts.

Functions

block_average(x[, min_blocks])

Compute the Flyvbjerg-Petersen blocking table of a time series.

select_plateau(result, *, method)

Pick the blocking level that estimates the plateau, by a named rule.

sem_blocked(x, *, method[, min_blocks])

Standard error of a time average over correlated frames.

statistical_inefficiency(x, *, method[, ...])

Estimate the frames per independent sample of a time series, its statistical inefficiency, from its blocking table.

Classes

BlockingResult(n_frames, block_size, ...)

The blocking table of one time series, one row per blocking level.

class polyphys.analyze.stats.BlockingResult(n_frames, block_size, n_blocks, mean, var, sem, sem_err, statistical_inefficiency, inefficiency_err)[source]#

Bases: object

The blocking table of one time series, one row per blocking level.

Level k averages the frames in contiguous blocks of 2**k frames and treats the block means as the sample. Every per-level array has the level along axis 0; the trailing axes are those of one frame of the input, one entry per observable.

Variables:
  • n_frames (int) – Number of frames in the input, the sample size at level 0.

  • block_size (numpy.ndarray) – Frames per block at each level, 2**k; shape (n_levels,), int64.

  • n_blocks (numpy.ndarray) – Blocks at each level, n_frames // block_size; shape (n_levels,), int64.

  • mean (numpy.ndarray) – Mean of the block means, in the units of the input; shape (n_levels, *observable_shape), float64. It changes across levels only when a trailing frame is dropped at an odd level.

  • var (numpy.ndarray) – Variance of the block means with ddof=1, in squared input units.

  • sem (numpy.ndarray) – Standard error of the mean at each level, sqrt(var / n_blocks), in the units of the input. Level 0 is the naive estimate.

  • sem_err (numpy.ndarray) – Leading-order error of sem, sem / sqrt(2 (n_blocks - 1)).

  • statistical_inefficiency (numpy.ndarray) – block_size * var / var[0], dimensionless: frames per independent sample, as estimated at that level. Level 0 is exactly 1.

  • inefficiency_err (numpy.ndarray) – Leading-order error of the inefficiency, statistical_inefficiency * sqrt(2 / (n_blocks - 1)). It neglects the error of var[0].

Parameters:
  • n_frames (int)

  • block_size (NDArray[int64])

  • n_blocks (NDArray[int64])

  • mean (NDArray[float64])

  • var (NDArray[float64])

  • sem (NDArray[float64])

  • sem_err (NDArray[float64])

  • statistical_inefficiency (NDArray[float64])

  • inefficiency_err (NDArray[float64])

Examples

>>> import numpy as np
>>> from polyphys.analyze.stats import block_average
>>> table = block_average(np.repeat([1.0, 3.0, 2.0, 4.0], 4))
>>> table.n_levels, table.observable_shape
(3, ())
>>> table.block_size
array([1, 2, 4])

Added in version 0.5.0.

block_size: numpy.typing.NDArray[numpy.int64]#
inefficiency_err: numpy.typing.NDArray[numpy.float64]#
mean: numpy.typing.NDArray[numpy.float64]#
n_blocks: numpy.typing.NDArray[numpy.int64]#
n_frames: int#
property n_levels: int#

Number of blocking levels, the length of axis 0.

property observable_shape: tuple[int, ...]#

Shape of one frame of the input, () for a scalar series.

sem: numpy.typing.NDArray[numpy.float64]#
sem_err: numpy.typing.NDArray[numpy.float64]#
statistical_inefficiency: numpy.typing.NDArray[numpy.float64]#
var: numpy.typing.NDArray[numpy.float64]#
polyphys.analyze.stats.PlateauMethod#

The plateau-selection rules select_plateau() accepts.

alias of Literal[‘lee’, ‘sem_err’]

polyphys.analyze.stats.block_average(x, min_blocks=4)[source]#

Compute the Flyvbjerg-Petersen blocking table of a time series.

Starting from the frames themselves (block size 1), the series is replaced by the means of consecutive pairs, and the mean, variance, standard error, and statistical inefficiency are recorded at each level, until the next halving would leave fewer than min_blocks blocks. The full table is option (a) of the plateau choice: the caller reads where the standard error stops growing.

Parameters:
  • x (numpy.typing.ArrayLike) – Time series with frames along axis 0, shape (n_frames, ...). Every trailing axis is an independent observable. Any numeric dtype; the computation is in float64 and x is not modified.

  • min_blocks (int) – Smallest number of blocks kept; at least 2. The last level has at least this many blocks. With 4 blocks the error of the standard error is already 41% of its value, so levels this deep are mostly a guard against reading noise as a plateau.

Returns:

The table; see its attributes for shapes and units.

Return type:

polyphys.analyze.stats.BlockingResult

Raises:

ValueError – If x is a scalar, holds NaN or inf, has fewer than min_blocks frames, or has an observable that is constant across frames (its inefficiency would be 0/0); or if min_blocks is below 2.

Notes

For a series \(x_t\) of \(n\) frames, level \(k = 0, 1, \dots\) has block size \(B_k = 2^k\) frames, \(m_k = \lfloor n / B_k \rfloor\) blocks, and block means \(\bar{x}_j^{(k)}\), the average of \(x_t\) over block \(j = 1, \dots, m_k\). With \(\langle \bar{x}^{(k)} \rangle\) the mean of the block means, the table records

\[\mathrm{var}_k = \frac{1}{m_k - 1} \sum_{j=1}^{m_k} \left(\bar{x}_j^{(k)} - \langle \bar{x}^{(k)} \rangle\right)^2, \quad \mathrm{SE}_k = \sqrt{\mathrm{var}_k / m_k}, \quad s_k = B_k \, \mathrm{var}_k / \mathrm{var}_0 ,\]

the variance of the block means, the standard error of the mean, and the statistical inefficiency at level \(k\); \(\mathrm{var}_0\) is the variance of the frames themselves. As \(B_k\) grows past the correlation time, \(\mathrm{SE}_k\) reaches a plateau at the true standard error of the time average and \(s_k\) at the statistical inefficiency \(s = \sum_{t=-\infty}^{\infty} \rho(t)\), where \(\rho(t)\) is the normalized autocorrelation function of \(x_t\) at a lag of \(t\) frames. This is twice the integrated autocorrelation time \(\tau_\mathrm{int}\) of Wolff (2004), Eq. (23) and (25).

When a level has an odd number of blocks, the trailing block is left out of the next pairing, so the frames used shrink by at most one block per level and mean can drift slightly between levels.

With \(n_\mathrm{obs}\) the number of observables, the product of the trailing dimensions of x (1 for a 1-D series), time complexity is \(O(n \, n_\mathrm{obs})\), since the levels touch \(n + n/2 + n/4 + \dots < 2n\) frames in all. Space is one float64 copy of the input, halved in place, plus the table of \(O(\log_2 n \, n_\mathrm{obs})\) entries.

References

H. Flyvbjerg and H. G. Petersen, J. Chem. Phys. 91, 461 (1989), https://doi.org/10.1063/1.457480. Implementation after M. P. Allen and D. J. Tildesley, Computer Simulation of Liquids, 2nd ed. (2017), example code error_calc.py.

Examples

A signal of \(n = 16\) frames that holds each value for \(L = 4\) frames has an exact table: for every block size \(B \le L\) the block means are the held values themselves, so the inefficiency is \(s(B) = B (n - 1) / (n - B)\), here 1, 30/14 and 5.

>>> import numpy as np
>>> from polyphys.analyze.stats import block_average
>>> x = np.repeat([1.0, 3.0, 2.0, 4.0], 4)
>>> table = block_average(x, min_blocks=4)
>>> table.block_size, table.n_blocks
(array([1, 2, 4]), array([16,  8,  4]))
>>> table.statistical_inefficiency.round(4)
array([1.    , 2.1429, 5.    ])
>>> table.sem.round(4)
array([0.2887, 0.4226, 0.6455])

A two-column input gives one table column per observable:

>>> table = block_average(np.column_stack([x, 2.0 * x]))
>>> table.observable_shape, table.sem.shape
((2,), (3, 2))
>>> np.allclose(table.sem[:, 1], 2.0 * table.sem[:, 0])
True

Added in version 0.5.0.

polyphys.analyze.stats.select_plateau(result, *, method)[source]#

Pick the blocking level that estimates the plateau, by a named rule.

Parameters:
  • result (polyphys.analyze.stats.BlockingResult) – A table from block_average().

  • method (typing.Literal['lee', 'sem_err']) –

    The selection rule; required, because it is an estimator choice.

    'lee'

    The largest-block-downward scan of Lee et al. (2011): a level qualifies when its block size \(B\), in frames, satisfies \(B^3 > 2 n (\mathrm{SE}(B) / \mathrm{SE}(1))^4\), where \(n\) is the number of frames, \(\mathrm{SE}(B)\) the level’s standard error, and \(\mathrm{SE}(1)\) the naive one at \(B = 1\). The selected level is the smallest block size of the qualifying run that includes the largest block. The rule balances the systematic underestimate of the error at small blocks against its growing statistical error at large ones.

    'sem_err'

    The first level whose standard error is within one sem_err of each of the next two levels, that is, where the table stops rising. A convenience heuristic, not a published rule.

Returns:

The selected level index into the table’s axis 0: an int for a scalar series, else an int64 array of shape result.observable_shape. -1 marks an observable for which no level qualifies.

Return type:

int | numpy.typing.NDArray[numpy.int_]

Raises:

ValueError – If method is not one of the rules above.

Warns:

RuntimeWarning – When no level qualifies for some observable, which means the series is too short for its correlation time or is not stationary. Also, under either method, when an observable has a selected level but the block size the 'lee' rule selects is not below n_frames / 50, or that rule finds no level: Lee et al. (2011) then consider the estimate unreliable. The selected level is still returned.

Notes

With \(n_\mathrm{levels}\) the number of rows of the table and \(n_\mathrm{obs}\) the number of observables, both rules are \(O(n_\mathrm{levels} \, n_\mathrm{obs})\) in time and space.

Lee et al. write their Eq. (14) as \(B_\mathrm{opt} = (2 n n_\mathrm{corr}^2)^{1/3}\), the optimal block size for \(n\) frames, with \(n_\mathrm{corr}\) the statistical inefficiency \(s\), the plateau of \((\mathrm{SE}(B) / \mathrm{SE}(1))^2\); hence the fourth power in the rule above. They cite Wolff (2004), whose Eq. (47) reads \(B = \tau (2N/\tau)^{1/3}\) for \(N\) frames, with \(\tau\) the integrated autocorrelation time, which is \(s / 2\). The two prescriptions therefore differ by a factor of 4 in \(B^3\); neither constant is exact, as Wolff’s derivation is an order-of-magnitude balance. This function implements Lee’s form, as pyblock does.

Lee et al. accept the analysis only when \(n / B_\mathrm{opt} > 50\), which with Eq. (14) reads \(n / s > 500\): the series must hold more than about 500 independent samples. That is a property of the series, not of the rule, so this function tests it at the 'lee' level under either method. The 'sem_err' rule can stop at a level with more than 50 blocks on a series that fails the test, where the table levels off before the blocks outgrow the correlation time.

References

R. M. Lee et al., Phys. Rev. E 83, 066706 (2011), Sec. IV, Eq. (13)-(15). U. Wolff, Comput. Phys. Commun. 156, 143 (2004), Eq. (46)-(48).

Examples

A signal that holds each value for 8 frames has a statistical inefficiency \(s \approx 8\), so on the plateau \((\mathrm{SE}(B) / \mathrm{SE}(1))^4 = s^2 \approx 8^2\). With \(n = 16384\) frames the Lee rule then needs \(B^3 > 2 \cdot 16384 \cdot 8^2\), that is \(B > 128\), and it selects the first power of two above 128:

>>> import numpy as np
>>> from polyphys.analyze.stats import block_average, select_plateau
>>> rng = np.random.default_rng(7)
>>> x = np.repeat(rng.standard_normal(2048), 8)
>>> table = block_average(x)
>>> level = select_plateau(table, method='lee')
>>> int(table.block_size[level])
256

The 'sem_err' rule stops where the standard error first flattens, which for this signal is at or just past the plateau length:

>>> level = select_plateau(table, method='sem_err')
>>> bool(table.block_size[level] >= 8)
True

Holding each of 64 values for 256 frames gives \(s \approx 256\), so the series holds only 64 independent samples. The 'sem_err' rule stops at the period, where 64 blocks remain, but the Lee block size leaves fewer than 50, so both rules warn:

>>> import warnings
>>> rng = np.random.default_rng(7)
>>> x = np.repeat(rng.standard_normal(64), 256)
>>> table = block_average(x)
>>> with warnings.catch_warnings(record=True) as caught:
...     warnings.simplefilter('always')
...     level = select_plateau(table, method='sem_err')
>>> int(table.block_size[level]), int(table.n_blocks[level])
(256, 64)
>>> [type(w.message).__name__ for w in caught]
['RuntimeWarning']

Added in version 0.5.0.

polyphys.analyze.stats.sem_blocked(x, *, method, min_blocks=4)[source]#

Standard error of a time average over correlated frames.

Runs block_average() and select_plateau() and returns the standard error at the selected level. This is the error bar to report for the mean of x over its frames.

Parameters:
Returns:

The blocked standard error, in the units of x: a float for a scalar series, else a float64 array of shape x.shape[1:]. NaN where no plateau was found.

Return type:

float | numpy.typing.NDArray[numpy.double]

Raises:

ValueError – As block_average() and select_plateau().

Warns:

RuntimeWarning – As select_plateau().

Notes

For independent frames the result agrees with polyphys.analyze.measurer.sem() within its own error bar; for correlated frames it is larger by \(\sqrt{s}\), where \(s\) is the statistical inefficiency, the number of frames that make one independent sample. With \(n\) the number of frames and \(n_\mathrm{obs}\) the number of observables (the product of the trailing dimensions of x), time is \(O(n \, n_\mathrm{obs})\) and space one float64 copy of x.

Examples

Each value of this signal is held for 8 frames, so \(s \approx 8\) and the blocked error is about \(\sqrt{8} \approx 2.8\) times the naive one:

>>> import numpy as np
>>> from polyphys.analyze.stats import sem_blocked
>>> from polyphys.analyze.measurer import sem
>>> rng = np.random.default_rng(7)
>>> x = np.repeat(rng.standard_normal(2048), 8)
>>> ratio = sem_blocked(x, method='lee') / sem(x)
>>> bool(2.3 < ratio < 3.4)
True

Added in version 0.5.0.

polyphys.analyze.stats.statistical_inefficiency(x, *, method, min_blocks=4)[source]#

Estimate the frames per independent sample of a time series, its statistical inefficiency, from its blocking table.

Runs block_average() and select_plateau() and returns the statistical inefficiency at the selected level: the ratio of the plateau variance of the mean to the naive one, so n_frames / s is the effective number of independent frames.

Parameters:
Returns:

Dimensionless, in frames; 1 for independent frames, below 1 for anticorrelated ones. A float for a scalar series, else a float64 array of shape x.shape[1:]. NaN where no plateau was found.

Return type:

float | numpy.typing.NDArray[numpy.double]

Raises:

ValueError – As block_average() and select_plateau().

Warns:

RuntimeWarning – As select_plateau().

Notes

For a stationary series \(x_t\) whose normalized autocorrelation function at a lag of \(t\) frames is \(\rho(t)\), with \(\rho(0) = 1\), the inefficiency is \(s = \sum_{t=-\infty}^{\infty} \rho(t) = 2 \tau_\mathrm{int}\), where \(\tau_\mathrm{int}\) is the integrated autocorrelation time (Wolff 2004, Eq. (23) and (25)). For an AR(1) process, \(x_t = \phi x_{t-1} + \epsilon_t\) with coefficient \(|\phi| < 1\) and independent noise \(\epsilon_t\), \(\rho(t) = \phi^{|t|}\) and \(s = (1 + \phi) / (1 - \phi)\). With \(n\) the number of frames and \(n_\mathrm{obs}\) the number of observables (the product of the trailing dimensions of x), time is \(O(n \, n_\mathrm{obs})\) and space one float64 copy of x.

Examples

Each value of this signal is held for 8 frames, so \(s \approx 8\):

>>> import numpy as np
>>> from polyphys.analyze.stats import statistical_inefficiency
>>> rng = np.random.default_rng(7)
>>> x = np.repeat(rng.standard_normal(2048), 8)
>>> s = statistical_inefficiency(x, method='lee')
>>> bool(5.0 < s < 11.0)
True

Added in version 0.5.0.