Design decisions#
This page records design decisions that shape PolyPhys, with the evidence behind each and what it gives up.
manage builds on no workflow framework#
Decided on 2026-10-31 (amirhs1/PolyPhys#19).
Decision. polyphys.manage stays a small internal layer that turns the
names of existing simulation artifacts into lineage and feeds
polyphys.analyze. It does not build on signac or
another data-space framework, and it is not developed as a standalone data
manager. A study is added by declaring a parser class and a ProjectSpec
(see Studies).
The question. signac manages a data space of simulations indexed by
state points, the key-value records that describe each job [1].
The PolyPhys lineage segment → whole → ensemble_long → ensemble → space is
structurally a specialization of such a data space, so a thin signac adapter
was the alternative.
Evidence. From the signac 2.4.1 documentation and source:
A signac project stores its data in a
workspacedirectory with one subdirectory per job, named by the job id, the hash of the job’s state point. A state point is unique, so replicas need a replica key.Project.import_from(origin, schema)brings an existing data space into a project by walking the directories underoriginand mapping each to a state point, through a schema string such asa_{a:int}/b_{b:int}or a function of the path. It copies the data into the workspace by default; moving instead is possible, and the documentation warns that a failed import can then leave an inconsistent state.Linked views give the job directories human-readable names without copying.
PolyPhys starts from a different place. Its inputs are LAMMPS output that
was never organized by signac, and the physics is in the filenames: one
ensemble_long directory holds the files of many wholes, segments, and
properties, and the analysis stages (ens, ensAvg, space) are
directories of aggregates rather than state points. Under signac each whole
would become a job in a hashed directory, the lineage names would survive
only in linked views, and trajectories of tens of megabytes to gigabytes,
kept on the clusters that produced them, would be copied or moved into the
workspace.
Given up. signac’s state-point queries, linked views, job documents, and the workflow layer signac-flow. PolyPhys keeps its own small equivalent of a state point: the attributes a parser reads from a name.
Gained. No new dependency, and the lineage, its names, and the files stay exactly where the simulations wrote them.
Not done. The comparison rests on signac’s documentation and source; no prototype mapped a real space onto signac jobs. Revisit this decision if PolyPhys starts to create simulation workspaces itself rather than read existing ones.
Each study is a folder outside the package#
Decided on 2026-10-31 and 2026-11-02 (amirhs1/PolyPhys#76).
Decision. The package holds mechanisms and no study. polyphys.spec
(polyphys.manage.projects before 0.9.0) defines ProjectSpec, the record of what a study’s analysis needs beyond
its parser, and checks it against the parser; PolyPhys holds no table of
studies. Each study is one folder, projects/<study>/, which
pyproject.toml excludes from the installed package and CI lints,
type-checks, and tests with it. Today the folder holds the study’s
ProjectSpec, its ProbeSpec, and its AnalysisTable records, built
beside one another; its drivers and figure code follow. A correction to one
study’s data, such as merging two neighboring crowder volume fractions, is code in
that study’s folder with its reason, never an option of the library. The
published studies’ folders are in this repository; unpublished studies use
the same layout in the research code (see “Where PolyPhys comes from” on
the front page).
Why. A study should run from its own repository without editing a table inside the installed package (amirhs1/PolyPhys#18). A rounding step, a list of time series, a selection string, or which attributes become columns are choices of one study, not facts of the library; and the research code carried the same study table in three copies, one per stage.
The parsers. Until 1.0.0 the parser classes of nine studies, such as
HnsCub and TransFociCub, shipped in polyphys.manage.parser. The
maintainer decided on 2026-11-02 that they move to their studies’ folders,
and on 2026-11-07 how (amirhs1/PolyPhys#197): the parsers of the four
published studies are in projects/<study>/parser.py, those of the
unpublished studies are in the research code, and importing any of
the nine from polyphys.manage.parser raises ImportError that says where
it went, since a module in the installed package cannot import from
projects/. On 2026-11-09 the maintainer took the five unpublished
studies off that list and off the documentation, so their names are not
public; importing one now fails as for any name the module never
defined. The package’s own tests and examples use made-up parsers in
polyphys.tests.synthetic.
The companion repository. On 2026-11-08 the maintainer decided that
PolyPhys stands alone: the package and its tests do not refer to the
studies. In 1.1.0 the projects/ folder moved to the companion repository
polyphys-lab (amirhs1/PolyPhys#275), where each study is a
folder installed as polyphys_lab.<study>, with the tests that run its
specs on real frames. The lab depends on a released PolyPhys, never the
reverse.