Studies: adding a study#
This page shows how to add a study to PolyPhys: the parser class that reads its file names, the specs that describe its probe and its analysis, the folder that holds them, and the checks that run on them. It also shows how to read the tables that the research code before PolyPhys wrote.
Notation#
- study
One research project: one simulation model, swept over its parameters, such as
HnsCub, a ring polymer with H-NS proteins among crowders in a periodic cube.- keyword
A short label in a file name followed by its value, such as
N200for a chain of 200 monomers orens1for the first replicate.- parser class
A subclass of
ParserBasethat declares a study’s keywords and reads a name into typed attributes and lineage names.- spec
A record of a study’s choices: the
ProjectSpecof its analysis, theProbeSpecof its probe, and oneAnalysisTableper particle group.- species
A kind of particle, labeled as in the property names:
Monfor the monomers,Crdfor the crowders,Hnsfor the H-NS cores.- \(a_i\)
The diameter of the particles of species \(i\), in the length unit of the simulation.
The Lineage page defines the lineage levels and the groups.
The steps#
Adding a study changes nothing in polyphys/, which ships no study’s
parser: a study declares its parser in its own folder, as each of the
maintainer’s published studies does in
polyphys-lab.
Declare the study’s parser class, as in “Declaring the parser” below.
Make a folder for the study as a Python package. Its
spec.pybuilds the study’sProjectSpec(asSPEC), itsProbeSpec(asPROBE), and a tuple of itsAnalysisTablerecords, one per particle group (asANALYSIS); its__init__.pyexports the three. The Probe page, “A study’s probe as data”, and the Pipeline page, “A study’s analysis as data”, describe the last two.Build the specs. Each attribute a
ProjectSpecnames is checked against the parser when the spec is built, so a typo raises whenspec.pyis imported, not when the analysis runs.Test the study: check its specs against its parser and against the values its analysis used before, as polyphys-lab’s
tests/test_studies.pydoes for the maintainer’s studies.Run the study’s data through the phases, as the Workflow page describes.
The maintainer’s published studies are in the companion repository
polyphys-lab, one folder per study, each installed as
polyphys_lab.<study>. PolyPhys does not depend on it; a study elsewhere
uses the same layout in its own repository. In PolyPhys 1.0.0, importing
one of these parsers from polyphys.manage.parser raises an ImportError
that names projects/<study>/parser.py in the PolyPhys repository; that
folder is now projects/<study>/python/ in polyphys-lab. The Design decisions
page records why each study is a folder outside the package.
Declaring the parser#
A parser class is a declaration: it lists the study’s filename keywords, the
order in which each lineage writes them, any fixed values, the particle
species whose bulk number density and volume fraction to compute, and the
attribute that holds each group’s dump period (here fixed in _defaults).
ParserBase does the parsing. Each keyword gets a KeywordSpec naming the
attribute that stores its value and the value’s type; a float keyword may
also convert the value as scale * value + offset, here lcube = 2 l from
the half-side l. The conversion is exact decimal arithmetic on the value’s
text, rounded once to a float, so the cylinder size D = 2 r - 1 of r4.1
is 7.2 and prints as D7.2 in the short names. A bulk volume fraction is
nominal, the species’ bead volume over the volume its centers can reach, and
can exceed one for a species nearly as wide as the box; a species with no
room for its centers raises ValueError naming the state point:
>>> from polyphys.manage.parser import KeywordSpec, ParserBase, Species
>>> class ChainCub(ParserBase):
... _geometry = 'cubic'
... _topology = 'linear'
... _groups = ['bug', 'all']
... _dump_keywords = {'bug': 'bdump', 'all': 'adump'}
... _keywords = {
... 'N': KeywordSpec('nmon', int),
... 'ac': KeywordSpec('dcrowd', float),
... 'nc': KeywordSpec('ncrowd', int),
... 'l': KeywordSpec('lcube', float, scale=2.0),
... 'ens': KeywordSpec('ensemble_id', int),
... 'j': KeywordSpec('segment_id', int),
... }
... _patterns = {
... 'segment': ('N', 'ac', 'nc', 'l', 'ens', 'j'),
... 'whole': ('N', 'ac', 'nc', 'l', 'ens'),
... 'ensemble_long': ('N', 'ac', 'nc', 'l'),
... 'ensemble': ('N', 'ac', 'nc'),
... 'space': ('N', 'ac'),
... }
... _defaults = {'dmon': 1.0, 'bdump': 1000, 'adump': 5000}
... _species = (Species('m', 'nmon', 'dmon'),
... Species('c', 'ncrowd', 'dcrowd'))
... _project_attributes = {
... 'segment': ['dmon', 'rho_bulk_m', 'rho_bulk_c'],
... 'whole': ['dmon', 'rho_bulk_m', 'rho_bulk_c'],
... 'ensemble_long': ['dmon', 'rho_bulk_m', 'rho_bulk_c'],
... 'ensemble': ['dmon'],
... 'space': ['dmon'],
... }
>>> artifact = ChainCub('N100ac2.0nc50l10.0ens1.bug', 'whole', 'bug')
>>> artifact.lcube, artifact.ensemble, artifact.space
(20.0, 'N100ac2nc50', 'N100ac2')
>>> artifact.dump_keyword, artifact.bdump
('bdump', 1000)
The declaration is checked when the class is created: a pattern keyword
without a KeywordSpec, an attribute that appears twice in one pattern, a
species whose size or number no long-form name sets, or a group without a
dump keyword raises TypeError. A
name that lacks a keyword of its lineage’s pattern, or whose value is not a
number, raises ValueError naming the keyword, the lineage, and the name.
A value that is a function of parsed values but fits none of the tables,
such as a complementary percentage pct_two = 100 - pct_one of two
crowder species, goes in an override of
_derived_attributes, which runs at every lineage after the bulk values.
Declaring the ProjectSpec#
A parser says how a study’s names read; a ProjectSpec from
polyphys.spec says how its analysis reads them. It holds what the parser
does not; the geometry, topology, and glob patterns come from the parser.
parserThe study’s
ParserBasesubclass.groupThe particle group of the polymer, such as
'bug'.divisorThe step to which the bulk volume fraction of crowders is rounded in the
phi_bulk_c_roundcolumn; a study choice with no default.crowder_attrThe parser attribute that holds that volume fraction, such as
phi_bulk_c; a study choice with no default, keyword-only.physical_attrsThe parser attributes that become columns of the
spacetables.species_sizesEach density-profile species label mapped to the parser attribute that holds that species’ diameter.
tseries_propertiesThe study’s time series, such as
'gyrTMon'.
The spec for the cubic parser ChainCub declared above:
>>> from polyphys.spec import ProjectSpec
>>> chain = ProjectSpec(
... parser=ChainCub, group='bug', divisor=0.025,
... crowder_attr='phi_bulk_c',
... physical_attrs=('space', 'ensemble_long', 'nmon', 'dcrowd'),
... species_sizes={'Mon': 'dmon', 'Crd': 'dcrowd'},
... tseries_properties=('gyrTMon',))
>>> chain.geometry, chain.directions
('cubic', ('r',))
>>> chain.space_pattern, chain.space_hierarchy, chain.hierarchy
('N*ac*', 'N*', 'N*')
space_pattern matches the study’s space names, and space_hierarchy
and hierarchy are the glob prefixes of its space and ensemble_long
names, which the stage functions take.
Each attribute a spec names is checked against the attributes the parser sets
at ensemble_long, so a typo or a missing declaration raises ValueError when
the spec is built. The space stage reads its fields from the spec, as in
space_tseries_from_spec(database, chain, 'gyrTMon') from
polyphys.analyze.pipeline.
A study that probes its trajectories with polyphys.probe builds a
ProbeSpec beside its ProjectSpec: the MDAnalysis selection of each
species, the label of the chain, the chain-size observables to measure,
whether the chain is unwrapped for its asphericity, and the settings of its
spatial histograms, pair distances, and binders; see Probe.
The study’s drivers and figure code will join its specs in its folder.
Reading tables the research code wrote#
PolyPhys names a bulk quantity <quantity>_bulk_<species>; the research code
it replaces named it <quantity>_<species>_bulk, and wrote every parser
attribute as a column of its *-stamps.csv files, from where the names carry
into its ensemble-average and space tables. BULK_NAME_MAP in
polyphys.manage.organizer maps each old name to its new one, and
rename_bulk_columns applies the map to a table read from such a file,
leaving every other column unchanged:
Research code |
PolyPhys |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
>>> import pandas as pd
>>> from polyphys.manage.organizer import rename_bulk_columns
>>> old = pd.DataFrame({'nmon': [20], 'phi_c_bulk': [0.1], 'n_frames': [100]})
>>> list(rename_bulk_columns(old).columns)
['nmon', 'phi_bulk_c', 'n_frames']
The parser classes carry no alias for an old name: a name reaches a file only
when a parser lists it in attributes, and an alias there would give every
new table both columns.