polyphys.analyze.pipeline#
polyphys.analyze.pipeline#
The polyphys.analyze.pipeline module reduces the probe artifacts of one
space down the lineage, segment to whole to ensemble to
ensemble average, for every property that a study’s analysis table
declares. It holds no physics of its own: it selects the files of each
property, calls polyphys.manage.organizer to build the wholes, the
ensembles, and the ensemble averages, and lets a post-hook derive further
artifacts from the wholes, such as a time average, an autocorrelation
function, a density profile, or the pair-separation histograms of the foci.
A study describes its analysis as data: one
AnalysisTable per particle group, listing a
PropertySpec per probe artifact; both records are in
polyphys.spec. The tables of the maintainer’s studies live beside each
study’s ProjectSpec in the companion repository
polyphys-lab, so a data correction or a special case is code in the
study’s folder, never an option of this module.
Notation#
- property
The name of one probe artifact without its lineage name and group, such as
gyrTMonorrHistCrd: a name (gyrT,rHist) followed by a species (Mon,Crd). Its files end in-<property>.npy(a whole) or.<property>.npy(a segment).- relation
How the segments of a whole combine, as
whole_from_segments()defines it:'tseries'concatenates along the frames,'histogram'sums, and'bin_edge'keeps one copy of identical edges.- whole type
The shape of one whole, as
ensemble()defines it:'vector'(a 1-D array),'matrix'(an N-D array, stacked along a new first axis),'bin_edge'(edges shared by the wholes), or'dataframe'(a table whose columns are elements, such as the pairs of foci). The ensemble average of a vector is a DataFrame of-mean,-var, and-semcolumns; of a matrix, one array per statistic; of edges, the edges; of a DataFrame, a DataFrame with those three statistics per column.- post-hook
A callable
(wholes, context) -> tuple[Derived, ...]applied to a property’s wholes before the ensembles are built. It returns every artifact to reduce, so a hook can pass the wholes through under the property’s own name, add derived artifacts, or replace the wholes with a reduction of them.- lineage of a table
'segment'when the probe phase wrote the group’s artifacts in segments,'whole'when it wrote one file per whole. A property of the study and the group, not of this module.- \(r_k\)
The autocorrelation at lag \(k\) frames, from
acf().- \(\bar P(R)\)
The probability density of the separation vector of a pair of foci, in one over the length unit cubed, from
pair_separation_density().
Contents#
Records:
Derived is one artifact a hook returns, and HookContext
what a hook may read. The table records, PropertySpec
and AnalysisTable, and their builders are in
polyphys.spec.
Hooks:
TimeAverage averages a time series over its frames;
Autocorrelation adds the autocorrelation function of a series;
LocalDensities adds the number density and volume fraction profiles
of a histogram through an injected DensityKernel;
FociPairDistances turns the distance-matrix time series of the foci
into their pair histograms, separation densities, and separation time series.
half_contour() is the separation bound of a ring at full bond extension.
Drivers:
reduce_space() runs every property of a table over one space;
space_tseries_from_spec() and space_hists_from_spec() stack the
ensemble averages of one property into the space table with the spec’s
columns and rounding step;
reduce_stamps() combines the stamps of a space;
equilibrium_wholes() reduces each whole’s time series to its
equilibrium value with a blocked error, beside the whole’s stamps.
Conventions#
A property whose files are not found raises
FileNotFoundErrorbefore any directory is made or any property reduced; nothing is skipped in silence.No output overwrites another:
AnalysisTablerejects a repeated name among the outputs its hooks declare, andreduce_space()raisesValueErrorwhen a hook that declares none returns a name already produced.The wholes of one ensemble must hold identical bin edges, or
ValueErrornames the ensemble and the two wholes; the organizer takes the first whole’s edges without comparing them.Outputs go to
<root>/analysis/<prefix>-<group>-<stage>for the stageswholeSim(thesegmentlineage only),ens, andensAvg, asmake_database()builds them, with the namessave_artifact()gives.The ensemble of
'dataframe'wholes is their long form, every whole’s rows with awholeand arowcolumn, so no whole is lost before the ensemble average; the average is taken row by row across the wholes.The error of a whole’s time average is the blocked standard error of
sem_blocked(), since the frames of one run are correlated; across the wholes of an ensemble, independent runs, the organizer’s plain standard error stays.A density profile’s ensemble average also carries
<property>-norm-sem, the standard error over the wholes of each whole’s profile divided by its own value in the bin that the sum rule’s default normalizer names (PROFILE_NORMALIZERS): the SEM across profiles normalized separately for each whole. Normalization is nonlinear, so it is not generally an uncertainty estimate for the normalized ensemble mean, and neither is the ensemble average’s-sem. SeeDerived.Progress is logged at INFO level; no warning is raised for a renamed artifact, because every output name is declared.
Complexities are stated in
n_files(the files of one property),n_frames,n_bins,n_wholes, andn_pairs.
Dependencies#
Python standard library: dataclasses, glob, logging, os, re.
numpy, pandas.
polyphys.analyze.clusters: the foci pair kernels.polyphys.manage.organizer: the lineage reducers.polyphys.spec:ProjectSpec, the table records, andPerWhole.
References
K. Kremer and G. S. Grest, “Dynamics of entangled linear polymer melts: A molecular-dynamics simulation”, J. Chem. Phys. 92, 5057 (1990), for the FENE bond and its maximum extension \(R_0 = 1.5\,\sigma\). https://doi.org/10.1063/1.458541
Examples
A table that reads as data: the gyration radius time series with its autocorrelation, the radial histogram of the crowders with their density profiles, and the planar histograms with their edges:
>>> from polyphys.analyze.pipeline import Autocorrelation, LocalDensities
>>> from polyphys.spec import (
... AnalysisTable, edge, histogram, histogram_2d, tseries)
>>> table = AnalysisTable(
... group='all', lineage='segment', stamps=False, properties=(
... tseries('gyrT', 'Mon', post_hook=Autocorrelation(nlags=100)),
... histogram('r', 'Crd', post_hook=LocalDensities()),
... histogram_2d('xy', 'Crd'),
... edge('x'),
... ))
>>> [prop.prop for prop in table.properties]
['gyrTMon', 'rHistCrd', 'xyHistCrd', 'xEdge']
Added in version 0.6.0.
Changed in version 0.9.0: AnalysisTable,
PropertySpec, PostHook,
and the builders tseries(),
histogram(), histogram_2d(),
edge(), and matrix_tseries()
moved to polyphys.spec; importing them from here warned with
DeprecationWarning.
Changed in version 1.0.0: Those names are no longer served from here; import them from
polyphys.spec.
Functions
|
The equilibrium value of each whole's time series, with its error, beside the whole's stamps. |
|
The largest separation two monomers of a ring can have. |
|
Reduce every property of a table over one space of probe artifacts. |
|
Combine the stamps of one space into whole and ensemble stamps. |
|
Stack the ensemble-averaged histograms of one property over a space, with the columns and the rounding step the study's spec declares. |
|
Stack the ensemble-averaged time series of one property over a space, with the columns and the rounding step the study's spec declares. |
Classes
|
Pass a time series through and add its autocorrelation; a post-hook. |
|
The volume-fraction kernel that |
|
One artifact a post-hook returns, ready for the ensemble stages. |
|
Reduce the distance-matrix time series of the foci; a post-hook. |
|
What a post-hook may read besides the wholes. |
Pass a histogram through and add its density profiles; a post-hook. |
|
|
Average a time series over its frames; a post-hook. |
- class polyphys.analyze.pipeline.Autocorrelation(nlags)[source]#
Bases:
objectPass a time series through and add its autocorrelation; a post-hook.
Notes
Returns the series under its own name and \(r_k\) for \(k = 0, \dots,\) nlags under
<property>-acf, both'vector'wholes. No confidence band is stored: Bartlett’s band is a function of the stored \(r_k\), the number of frames, and the level, sobartlett_half_width()rebuilds it at any level when it is drawn. Time is \(O(n_\mathrm{frames} \log n_\mathrm{frames})\) per whole.Examples
The alternating series has \(r_1 = -7/8\) and \(r_2 = 6/8\):
>>> import numpy as np >>> from polyphys.spec import tseries >>> hook = Autocorrelation(nlags=2) >>> context = HookContext(tseries('gyrT', 'Mon')) >>> series, acf_out = hook({'w1': np.tile([1.0, -1.0], 4)}, context) >>> series.name, acf_out.name ('gyrTMon', 'gyrTMon-acf') >>> np.round(acf_out.wholes['w1'], 6) array([ 1. , -0.875, 0.75 ])
Added in version 0.6.0.
- class polyphys.analyze.pipeline.DensityKernel(*args, **kwargs)[source]#
Bases:
typing.ProtocolThe volume-fraction kernel that
LocalDensitiescalls.A kernel turns the histogram of one species in one whole into its local number density and local volume fraction, bin by bin, from the geometry the whole’s parser instance describes. The study’s driver supplies one, so this module imports no density model.
Added in version 0.6.0.
- __call__(counts, edges, info, direction, diameter)[source]#
- Parameters:
counts (
numpy.ndarray) – The histogram of one whole, shape(n_bins,): the number of particle centers in each bin, summed over the frames.edges (
numpy.ndarray) – Its bin edges, shape(n_bins + 1,), in the length unit.info (
polyphys.manage.parser.ParserBase) – The whole’s parser instance, which holds the box sizes. It holds no number of frames, so a kernel that divides counts by the frames gets them from the study’s driver.direction (
str) – The direction of the histogram, such as'r'or'z'.diameter (
float) – The diameter of the species, in the length unit.
- Returns:
The number density and the volume fraction, each of shape
(n_bins,).- Return type:
- class polyphys.analyze.pipeline.Derived(name, wholes, whole_type, edges=None, normalizer=None)[source]#
Bases:
objectOne artifact a post-hook returns, ready for the ensemble stages.
- Variables:
name (
str) – The output property, which names the files; for example'rRhoCrd'or'gyrTMon-acf'.wholes (
dictofstrtonumpy.ndarrayorpandas.DataFrame) – The artifact of each whole, by whole name.whole_type (
{'vector', 'matrix', 'bin_edge', 'dataframe'}) – The shape of each whole; see the module notation.edges (
Mappingofstrtonumpy.ndarray, optional) – The bin edges of each whole, for a'vector'histogram, so its ensembles carry abin_centercolumn.normalizer (
{'first', 'last', 'max'}, optional) – For a'vector'profile, the reference bin of a per-whole normalization: the first bin, the last bin, or the bin where the ensemble average is largest. Each whole is divided by its own value in that bin, and the standard error over the wholes of these separately normalized profiles is written beside the ensemble average as<name>-norm-sem; it is not generally an uncertainty estimate for the normalized ensemble mean. The ensemble average itself is not normalized;polyphys.analyze.sumrule.space_profiles()does that and carries the column through.
- Raises:
ValueError – If whole_type is unknown, if edges or normalizer is given for a whole type other than
'vector', or if normalizer is not one of its three values.- Parameters:
Notes
The wholes of an ensemble are independent runs, so their spread is a valid error of any function of the profile, the normalized one included, with no covariance between a bin and its normalizer to track. The ensemble picks the bin so that every whole is normalized at the same place: a per-whole
'max'would bias each ratio towards one before the spread is taken. The error is zero in the reference bin by construction. The scale of a profile, such as the1 / aora^2of the sum rule, cancels in the ratio, so the column holds for the scaled profile too.Examples
>>> import numpy as np >>> out = Derived('gyrMon', {'w1': np.array([1.0])}, 'vector') >>> out.name, out.whole_type ('gyrMon', 'vector')
Added in version 0.6.0.
Changed in version 0.7.0: Added normalizer.
- edges: collections.abc.Mapping[str, numpy.ndarray] | None = None#
- wholes: dict[str, numpy.ndarray] | dict[str, pandas.DataFrame]#
- class polyphys.analyze.pipeline.FociPairDistances(bin_size, r_max, n_monomers)[source]#
Bases:
objectReduce the distance-matrix time series of the foci; a post-hook.
On
distMatT<species>, whose whole is the(n_frames, n_foci, n_foci)matrix of center-to-center distances between the foci with the foci’s backbone indices on the diagonal of frame 0, it writes three'dataframe'wholes with one column per pair, taggedloc<i>loc<j>genDist<d>withi < jthe backbone indices anddtheir genomic distance in bonds:pairDistHist<species>: the histogram of each pair’s separation on one grid shared by every pair and whole, with abin_centercolumn;pairDistRdf<species>: \(\bar P(R)\), the probability density of each pair’s separation vector on that grid, withbin_center;pairDistT<species>: each pair’s separation over the frames.
- Variables:
bin_size (
floatorcallable()) – The bin width \(\Delta r\), in the length unit, or a function of the whole’s parser instance that gives it.r_max (
floatorcallable()) – The upper end of the grid, or a function of the parser instance that gives it, such ashalf_contour()of the ring; the grid runs to the first multiple of bin_size at or above it.n_monomers (
intorcallable()) – The number of monomers on the backbone, or a function of the parser instance that gives it.
- Raises:
ValueError – When built: if a constant bin_size or r_max is not a finite positive number, or a constant n_monomers is not a positive integer; a Python or NumPy number, never a bool, and for n_monomers never a float, even
10.0. When called: if the context has no spec (the topology comes from it); if a whole is not a(n_frames, n_foci, n_foci)array; if a callable quantity gives a value that breaks the same rule, which names the quantity and the whole; or if a separation lies above the grid, which names the whole and the separation, since a dropped frame would break the normalization.- Parameters:
bin_size (float | Callable[[ParserBase], float])
r_max (float | Callable[[ParserBase], float])
n_monomers (int | Callable[[ParserBase], int])
Notes
Every pair has one separation per frame, so each histogram column sums to
n_framesand \(\sum_k \bar P_k \Delta V_k = 1\) exactly. The research code set the grid from each whole’s own largest separation, so wholes of one ensemble had different edges; a bound from the system gives one grid. Time is \(O(n_\mathrm{pairs} n_\mathrm{frames} \log n_\mathrm{bins})\) and space \(O(n_\mathrm{pairs} (n_\mathrm{frames} + n_\mathrm{bins}))\) per whole.Examples
Three foci at backbone indices 0, 4, and 8 of a ring of 10 monomers, over two frames, on a grid of unit bins to 3:
>>> import numpy as np >>> from polyphys.tests.synthetic import RingCub >>> from polyphys.spec import ProjectSpec, matrix_tseries >>> spec = ProjectSpec( ... parser=RingCub, group='bug', divisor=0.025, ... crowder_attr='phi_bulk_c', ... physical_attrs=('space',), species_sizes={}, ... tseries_properties=()) >>> frame0 = np.array([[0.0, 1.5, 2.5], [1.5, 4.0, 0.5], [2.5, 0.5, 8.0]]) >>> frame1 = np.array([[0.0, 1.5, 2.9], [1.5, 0.0, 2.5], [2.9, 2.5, 0.0]]) >>> hook = FociPairDistances(bin_size=1.0, r_max=3.0, n_monomers=10) >>> context = HookContext(matrix_tseries('distMatT', 'Foci'), spec=spec) >>> hist, density, series = hook({'w1': np.stack([frame0, frame1])}, ... context) >>> list(hist.wholes['w1'].columns) ['loc0loc4genDist4', 'loc0loc8genDist2', 'loc4loc8genDist4', 'bin_center'] >>> hist.wholes['w1']['loc0loc8genDist2'].to_list() [0, 0, 2] >>> series.wholes['w1']['loc4loc8genDist4'].to_list() [0.5, 2.5]
A constant that breaks the rule raises when the hook is built:
>>> FociPairDistances(bin_size=1.0, r_max=3.0, n_monomers=10.5) Traceback (most recent call last): ... ValueError: FociPairDistances: 'n_monomers' must be a positive integer ...
Added in version 0.6.0.
Changed in version 1.0.0: The constants are checked when the hook is built, and n_monomers, a constant or what a callable gives, must be a positive integer, where a float such as 10.5 was truncated to 10.
- n_monomers: int | collections.abc.Callable[[polyphys.manage.parser.ParserBase], int]#
- outputs(prop)[source]#
The pair histograms, separation densities, and separation time series of the species of prop.
Examples
>>> from polyphys.spec import matrix_tseries >>> hook = FociPairDistances(bin_size=1.0, r_max=3.0, n_monomers=10) >>> hook.outputs(matrix_tseries('distMatT', 'Foci')) ('pairDistHistFoci', 'pairDistRdfFoci', 'pairDistTFoci')
Added in version 0.9.0.
- Return type:
- Parameters:
prop (PropertySpec)
- class polyphys.analyze.pipeline.HookContext(prop, spec=None, group='bug', infos=dict(), edges=None, density=None)[source]#
Bases:
objectWhat a post-hook may read besides the wholes.
Added in version 0.6.0.
- Variables:
prop (
PropertySpec) – The property the wholes belong to.spec (
ProjectSpec, optional) – The study’s spec: its parser, topology, and species sizes.group (
str) – The particle group.infos (
MappingofstrtoParserBase) – The parser instance of each whole, by whole name, which holds the whole’s attributes such as sizes and counts.edges (
Mappingofstrtonumpy.ndarray, optional) – The bin edges of each whole, for a property with an edge file.density (
DensityKernel, optional) – The injected volume-fraction kernel.
- Parameters:
prop (PropertySpec)
spec (ProjectSpec | None)
group (str)
infos (Mapping[str, ParserBase])
density (DensityKernel | None)
- density: polyphys.analyze.pipeline.DensityKernel | None = None#
- edges: collections.abc.Mapping[str, numpy.ndarray] | None = None#
- spec: polyphys.spec.ProjectSpec | None = None#
- class polyphys.analyze.pipeline.LocalDensities[source]#
Bases:
objectPass a histogram through and add its density profiles; a post-hook.
On
<direction>Hist<species>it adds<direction>Rho<species>, the local number density, and<direction>Phi<species>, the local volume fraction, from theDensityKernelgiven toreduce_space(), with the species’ diameter read from the attribute the study’sspecies_sizesnames.- Raises:
ValueError – When called: if the property is not a
histogram(), if the context has no kernel, no spec, or no edges, or if the species is not in the spec’s species_sizes.
Notes
The three outputs share the histogram’s edges, so their ensembles carry the same
bin_centercolumn. The two profiles get the normalizer ofPROFILE_NORMALIZERSfor the parser’s geometry and the histogram’s direction, when the pair has one, so their ensemble averages carry<property>-norm-sem, the SEM across profiles normalized separately for each whole (seeDerived). Time is one kernel call per whole.Examples
With a stub kernel that returns the counts and a constant, on one whole of the
ChainCylstudy:>>> import numpy as np >>> from polyphys.tests.synthetic import ChainCyl >>> from polyphys.spec import ProjectSpec, histogram >>> spec = ProjectSpec( ... parser=ChainCyl, group='bug', divisor=0.025, ... crowder_attr='phi_bulk_c', ... physical_attrs=('space',), species_sizes={'Crd': 'dcrowd'}, ... tseries_properties=()) >>> whole = ('N20r5.5lz25.0ac1.0nc100' ... 'dt0.005bdump1000adump5000ens1') >>> def kernel(counts, edges, info, direction, size): ... return counts, np.full(counts.shape, size) >>> context = HookContext( ... histogram('r', 'Crd'), spec=spec, group='all', ... infos={whole: ChainCyl(whole, 'whole', 'all')}, ... edges={whole: np.array([0.0, 1.0, 2.0])}, density=kernel) >>> outputs = LocalDensities()({whole: np.array([3.0, 5.0])}, context) >>> [out.name for out in outputs] ['rHistCrd', 'rRhoCrd', 'rPhiCrd'] >>> outputs[2].wholes[whole] array([1., 1.]) >>> outputs[2].normalizer 'first'
Added in version 0.6.0.
Changed in version 0.7.0: The profiles carry the sum rule’s default normalizer.
- outputs(prop)[source]#
The histogram, passed through, and its
<direction>Rhoand<direction>Phiprofiles.- Raises:
ValueError – If prop is not a
histogram()property.- Return type:
- Parameters:
prop (PropertySpec)
Examples
>>> from polyphys.spec import histogram >>> LocalDensities().outputs(histogram('z', 'Crd')) ('zHistCrd', 'zRhoCrd', 'zPhiCrd')
Added in version 0.9.0.
- class polyphys.analyze.pipeline.TimeAverage(output, drop_first_column=False)[source]#
Bases:
objectAverage a time series over its frames; a post-hook.
- Variables:
output (
str) – The name of the average, without the species, which is appended:TimeAverage('clustersHist')on'clustersHistTFoci'writes'clustersHistFoci'.drop_first_column (
bool, defaultFalse) – Drop column 0 of each frame before averaging. The size histograms ofcount_clusters()carry the count of empty clusters, always 0, in column 0.
- Parameters:
Notes
The frames are axis 0; the average has the shape of one frame, so a
(n_frames, n_sizes)histogram time series becomes ann_sizesvector, reduced as a'vector'whole. Time and space are \(O(n_\mathrm{frames} n_\mathrm{sizes})\) per whole.Examples
>>> import numpy as np >>> from polyphys.spec import matrix_tseries >>> hook = TimeAverage('clustersHist', drop_first_column=True) >>> context = HookContext(matrix_tseries('clustersHistT', 'Foci')) >>> wholes = {'w1': np.array([[0, 1, 2], [0, 3, 4], [0, 5, 6]])} >>> (out,) = hook(wholes, context) >>> out.name, out.wholes['w1'] ('clustersHistFoci', array([3., 4.]))
Added in version 0.6.0.
- outputs(prop)[source]#
The name of the average: output then the species of prop.
Examples
>>> from polyphys.spec import matrix_tseries >>> TimeAverage('clustersHist').outputs( ... matrix_tseries('clustersHistT', 'Foci')) ('clustersHistFoci',)
Added in version 0.9.0.
- Return type:
- Parameters:
prop (PropertySpec)
- polyphys.analyze.pipeline.equilibrium_wholes(ens_database, properties, whole_stamps, *, method='lee', topology=None)[source]#
The equilibrium value of each whole’s time series, with its error, beside the whole’s stamps.
For each property, every ensemble file of the
ensstage is read once, and each of its columns, one whole’s time series, is reduced to its time average, its variance, and the blocked standard error of the average. The results of all the properties are merged with whole_stamps onwhole, one row per whole: the table the research code’sequilibrium_wholesbuilt for theallInOnephase.- Parameters:
ens_database (
str|os.PathLike[str]) – Theensdirectory of one space and group,<root>/analysis/<prefix>-<group>-ens, whose<ensemble_long>-<group>-<property>.csvfilesreduce_space()wrote.properties (
collections.abc.Mapping[str,str]) – The time-series property of each ensemble file, mapped to its output name:{'gyrTMon': 'gyrMon'}reads the-gyrTMon.csvfiles and writesgyrMon-mean,gyrMon-var, andgyrMon-sem.whole_stamps (
pandas.DataFrame) – The stamps of the space’s wholes, one row per whole, with awholecolumn: the first result ofreduce_stamps().method (
typing.Literal['lee','sem_err']) – The plateau rule ofsem_blocked();'lee'is the published rule.topology (
str|None) – Appended to each whole’s name as.<topology>before the merge, for stamps that the probe phase keyed by the whole’s file name, such as...ens1.ring; the ensemble files name a whole without it.
- Returns:
whole_stamps followed by, for each property in the order of properties, the columns
<output>-mean,<output>-var, and<output>-sem, with the rows in the order of whole_stamps.- Return type:
- Raises:
FileNotFoundError – If ens_database holds no CSV file, or a property matches none.
ValueError – If whole_stamps has no
wholecolumn or lists a whole twice; if two properties share an output name; if the wholes of a property’s files differ from those of the stamps, which names the wholes on each side; or asblock_average()raises for a series that is too short or holds NaN.
- Warns:
RuntimeWarning – As
select_plateau(): when a whole’s series is too short for its correlation time, its-semis NaN or unreliable.
Notes
-meanis the time average over the frames,-varthe sample variance of the frames (ddof = 1), and-semthe standard error of the time average by block averaging, which accounts for the correlation between consecutive frames of one run (Flyvbjerg and Petersen, 1989; the plateau rule of Lee et al., 2011; both inpolyphys.analyze.stats). A constant series has a-semof 0. Blocking applies within a whole only: the wholes of an ensemble are independent runs, soensemble_avg()keeps the plain standard error across them. The research code took-semas the per-frame standard error, which the blocked one exceeds by about \(\sqrt{s}\) for a statistical inefficiency \(s\), and its-varas the population variance.With \(n_\mathrm{frames}\) the frames of the longest series and \(n_\mathrm{wholes}\) the wholes of the space, time is \(O(n_\mathrm{wholes}\, n_\mathrm{frames})\) per property, and space is one ensemble file at a time plus a
float64copy of it.Examples
Two wholes of one ensemble whose series alternate between two values, so their time averages are exact and the blocked error vanishes at a block size of two frames:
>>> import tempfile >>> import numpy as np >>> import pandas as pd >>> stem = 'N20r5.5lz25.0ac1.0nc100dt0.005bdump1000adump5000' >>> wholes = [f'{stem}ens1', f'{stem}ens2'] >>> root = tempfile.mkdtemp() >>> pd.DataFrame({wholes[0]: np.tile([0.0, 2.0], 2048), ... wholes[1]: np.tile([1.0, 3.0], 2048)} ... ).to_csv(f'{root}/{stem}-bug-gyrTMon.csv', index=False) >>> stamps = pd.DataFrame({'whole': wholes, 'n_frames': [4096, 4096]}) >>> table = equilibrium_wholes(root, {'gyrTMon': 'gyrMon'}, stamps) >>> table.drop(columns='whole').round(3) n_frames gyrMon-mean gyrMon-var gyrMon-sem 0 4096 1.0 1.0 0.0 1 4096 2.0 1.0 0.0
Added in version 0.6.0.
- polyphys.analyze.pipeline.half_contour(n_monomers, bond_max)[source]#
The largest separation two monomers of a ring can have.
- Parameters:
- Returns:
(n_monomers // 2) * bond_max: the two monomers are at mostn_monomers // 2bonds apart along the ring, and a path of bonds is longest when straight and fully extended.- Return type:
- Raises:
ValueError – If n_monomers is below 2 or bond_max is not positive.
Notes
A bound, not a typical value: it lets a pair-distance histogram cover every separation the chain allows, so that its counts conserve the number of frames and a separation above it marks a broken bond or an unwrapping error. Pass the largest bond’s \(R_0\) when the bonds differ; the bound is then loose by the ratio of the largest to the typical bond, which costs bins, not correctness.
Examples
A ring of 10 beads with bonds that stretch to 1.5:
>>> half_contour(10, 1.5) 7.5
Added in version 0.6.0.
- polyphys.analyze.pipeline.reduce_space(input_database, spec, table, *, hierarchy=None, density=None, save=True)[source]#
Reduce every property of a table over one space of probe artifacts.
- Parameters:
input_database (
str|os.PathLike[str]) – The probe directory of the space,<root>/probes/<prefix>-<group>-probe; it may hold a glob, as the research code’s drivers pass it.spec (
polyphys.spec.ProjectSpec) – The study’s spec: its parser, topology, and species sizes.table (
polyphys.spec.AnalysisTable) – The group’s analysis table.hierarchy (
str|None) – The glob, relative to input_database, that lists the artifact files; the default is<spec.hierarchy>/<spec.hierarchy>, the research code’sdir/filelayout.density (
polyphys.analyze.pipeline.DensityKernel|None) – The volume-fraction kernel forLocalDensities.save (
bool) – Write the stages’ files under<root>/analysis/;Falsereturns the ensemble averages without writing anything.
- Returns:
The ensemble averages by output property, then by ensemble name: a DataFrame for a
'vector'or'dataframe'output, a dict of arrays for a'matrix', and the edges for a'bin_edge'.- Return type:
dict[str,dict[str,numpy.ndarray|pandas.DataFrame]]- Raises:
FileNotFoundError – If the glob matches no file, or a property or its edge property matches no file; checked for every property before any directory is made.
ValueError – If the wholes of one ensemble hold different bin edges; if a histogram’s edge files do not cover its wholes; if a hook returns an output name the table already produced; as the organizer and the hooks raise.
Notes
For each property in turn: its files are selected by their last name token, wholes are built (for the
'segment'lineage withwhole_from_segments(), written towholeSim; else withwhole_from_file()), the edges are loaded and checked, the post-hook is applied, and every returned artifact goes throughensemble()andensemble_avg(), or their long-form counterparts for'dataframe'wholes. With table.stamps, the stamps are combined first byreduce_stamps(). One property’s wholes are in memory at a time, so peak memory is the largest property’sn_wholeswholes plus its ensembles; time is \(O(n_\mathrm{files}\, n_\mathrm{frames})\) per property plus the hook’s.Examples
Two replicates of one state point of the
ChainCylstudy, written as wholes and reduced without saving:>>> import tempfile >>> import numpy as np >>> from polyphys.tests.synthetic import ChainCyl >>> from polyphys.spec import AnalysisTable, ProjectSpec, tseries >>> spec = ProjectSpec( ... parser=ChainCyl, group='bug', divisor=0.025, ... crowder_attr='phi_bulk_c', ... physical_attrs=('space',), species_sizes={'Mon': 'dmon'}, ... tseries_properties=('gyrTMon',)) >>> stem = 'N20r5.5lz25.0ac1.0nc100dt0.005bdump1000adump5000' >>> root = tempfile.mkdtemp() >>> np.save(f'{root}/{stem}ens1-bug-gyrTMon.npy', np.array([1.0, 2.0])) >>> np.save(f'{root}/{stem}ens2-bug-gyrTMon.npy', np.array([3.0, 6.0])) >>> table = AnalysisTable('bug', 'whole', False, (tseries('gyrT', 'Mon'),)) >>> averages = reduce_space(root, spec, table, hierarchy='*', save=False) >>> averages['gyrTMon'][stem] gyrTMon-mean gyrTMon-var gyrTMon-sem 0 2.0 2.0 1.0 1 4.0 8.0 2.0
Added in version 0.6.0.
- polyphys.analyze.pipeline.reduce_stamps(stamp_files, lineage, save_to=(None, None, None))[source]#
Combine the stamps of one space into whole and ensemble stamps.
- Parameters:
stamp_files (
collections.abc.Sequence[tuple[str]]) – The stamp CSV files, each path in a 1-tuple, assort_filenames()returns them.lineage (
typing.Literal['segment','whole']) – The lineage of the files.save_to (
tuple[str|os.PathLike[str] |None,str|os.PathLike[str] |None,str|os.PathLike[str] |None]) – ThewholeSim,ens, andensAvgdirectories;Noneskips a stage’s file. For'segment'stamps the combined segment stamps go to the first, the whole stamps to the second, and the ensemble stamps to the third; for'whole'stamps the whole stamps go to the second and the ensemble stamps to the third.
- Return type:
- Returns:
whole_stamps (
pandas.DataFrame) – One row per whole.ensemble_stamps (
pandas.DataFrame) – One row per ensemble.
- Raises:
ValueError – As
children_stamps()andparents_stamps().
Notes
Segment stamps combine with
children_stamps(), then roll up once withparents_stamps()into whole stamps; whole stamps combine directly. Either way the whole stamps roll up once more into ensemble stamps. Time and space are \(O(n_\mathrm{files}\, n_\mathrm{columns})\).Examples
whole_stamps, ensemble_stamps = reduce_stamps( [(path,) for path in sorted(stamp_paths)], 'whole')
Added in version 0.6.0.
- polyphys.analyze.pipeline.space_hists_from_spec(input_database, spec, prop, *, group=None, bin_center=None, normalize=False, round_to=3, save=False)[source]#
Stack the ensemble-averaged histograms of one property over a space, with the columns and the rounding step the study’s spec declares.
The spec supplies every study choice of
space_hists(), asspace_tseries_from_spec()describes; no study default is taken from the library.- Parameters:
input_database (
str|os.PathLike[str]) – TheensAvgdirectory of the space, which holds the files<ensemble_long>-<group>-<prop>-ensAvg.csv.spec (
polyphys.spec.ProjectSpec) – The study’s spec.prop (
str) – The property, such as'rHistCrd'.group (
str|None) – The particle group of the files; the default is the polymer group, spec.group. A histogram of the crowders is in the'all'group.bin_center (
numpy.ndarray|None) – The bin centers, shape(n_bins,); otherwise each file must hold abin_centercolumn.normalize (
bool) – Add<prop>-norm, each file’s<prop>-meandivided by its sum.round_to (
int) – The number of decimal places of the rounded \(\phi_c\).save (
bool) – Also write the table to<space>-<group>-<prop>-space.csvin theanalysisphase’sspacedirectory next to input_database.
- Returns:
The table of
space_hists(): each file’s columns, one column per physical attribute, a spec.crowder_attr column, andphi_bulk_c_round.- Return type:
- Warns:
UserWarning – If normalize is true and a histogram sums to zero, as
space_hists()warns.
Notes
The cost is that of
space_hists(): \(O(n_\mathrm{rows}\, n_\mathrm{cols})\) time and memory.Examples
>>> import tempfile >>> import numpy as np >>> import pandas as pd >>> from pathlib import Path >>> from polyphys.tests.synthetic import ChainCyl >>> from polyphys.spec import ProjectSpec >>> spec = ProjectSpec( ... parser=ChainCyl, group='bug', divisor=0.1, ... crowder_attr='phi_bulk_c', physical_attrs=('nmon',), ... species_sizes={'Mon': 'dmon'}, tseries_properties=()) >>> database = Path(tempfile.mkdtemp()) / 'analysis' / 'N20-all-ensAvg' >>> database.mkdir(parents=True) >>> state = 'N20r5.5lz25.0ac1.0nc100dt0.005bdump1000adump5000' >>> pd.DataFrame({'rHistMon-mean': [1.0, 3.0]}).to_csv( ... database / f'{state}-all-rHistMon-ensAvg.csv', index=False) >>> space = space_hists_from_spec( ... database, spec, 'rHistMon', group='all', ... bin_center=np.array([0.5, 1.5]), normalize=True) >>> space[['bin_center', 'rHistMon-norm', 'phi_bulk_c_round']] bin_center rHistMon-norm phi_bulk_c_round 0 0.5 0.25 0.0 1 1.5 0.75 0.0
Added in version 0.9.0.
- polyphys.analyze.pipeline.space_tseries_from_spec(input_database, spec, prop, *, group=None, round_to=3, save=False)[source]#
Stack the ensemble-averaged time series of one property over a space, with the columns and the rounding step the study’s spec declares.
The spec supplies every study choice of
space_tseries(): the parser, theensemble_longglob prefix (spec.hierarchy), the attributes that become columns (spec.physical_attrs), the rounding step \(\Delta\) of the bulk volume fraction of crowders \(\phi_c\) (spec.divisor), and the parser attribute that holds \(\phi_c\) (spec.crowder_attr). No study default is taken from the library.- Parameters:
input_database (
str|os.PathLike[str]) – TheensAvgdirectory of the space, which holds the files<ensemble_long>-<group>-<prop>-ensAvg.csv.spec (
polyphys.spec.ProjectSpec) – The study’s spec.prop (
str) – The property, such as'gyrTMon'.group (
str|None) – The particle group of the files; the default is the polymer group, spec.group.round_to (
int) – The number of decimal places of the rounded \(\phi_c\).save (
bool) – Also write the table to<space>-<group>-<prop>-space.csvin theanalysisphase’sspacedirectory next to input_database.
- Returns:
The table of
space_tseries(): each file’s columns,t_index,time, one column per physical attribute, a spec.crowder_attr column, andphi_bulk_c_round.- Return type:
Notes
The cost is that of
space_tseries(): \(O(n_\mathrm{rows}\, n_\mathrm{cols})\) time and memory for the rows and columns of the result.Examples
The spec’s step of 0.1 rounds \(\phi_c \approx 0.0123\) of this state point to 0.0:
>>> import tempfile >>> import pandas as pd >>> from pathlib import Path >>> from polyphys.tests.synthetic import ChainCyl >>> from polyphys.spec import ProjectSpec >>> spec = ProjectSpec( ... parser=ChainCyl, group='bug', divisor=0.1, ... crowder_attr='phi_bulk_c', physical_attrs=('nmon',), ... species_sizes={'Mon': 'dmon'}, tseries_properties=('gyrTMon',)) >>> database = Path(tempfile.mkdtemp()) / 'analysis' / 'N20-bug-ensAvg' >>> database.mkdir(parents=True) >>> state = 'N20r5.5lz25.0ac1.0nc100dt0.005bdump1000adump5000' >>> pd.DataFrame({'gyrTMon-mean': [1.0, 2.0]}).to_csv( ... database / f'{state}-bug-gyrTMon-ensAvg.csv', index=False) >>> space = space_tseries_from_spec(database, spec, 'gyrTMon') >>> space[['time', 'gyrTMon-mean', 'nmon', 'phi_bulk_c_round']] time gyrTMon-mean nmon phi_bulk_c_round 0 0.0 1.0 20 0.0 1 5.0 2.0 20 0.0
Added in version 0.9.0.