Skip to content

Bench_Scenarios

Bench Scenarios

Bench scenarios are controlled environments for developing and validating interference detection. Each bench models a different coupling channel type — from simple linear additive channels to deeply nonlinear cascaded systems. Together they span the complexity space that real-world multi-optimizer deployments encounter.

The benches live in examples/bench/ in the godon repository. Each has its own directory with docker-compose configuration, target definitions, and breeder configs.

Microgrid

A pair of coupled microgrid simulators sharing a power bus. Optimizer A adjusts load distribution on one microgrid; the coupling factor determines how much of A's load change leaks into B's grid.

Property Value
Directory examples/bench/scenario-microgrid (workflow: bench-scenario-microgrid.yml)
Channel type Linear additive
Coupling mechanism Shared power bus — neighbor load adds directly to objectives
Detection status Working reliably across all coupling strengths (0.0-0.9)
Trials needed 100-300
Configurable COUPLING_FACTOR environment variable (0.0 to 0.9)

The watermark signal passes through the coupling channel unchanged — same frequency content, proportional amplitude. FFT-based spectral detection with permutation testing reliably detects interference even at weak coupling (0.1). Zero false positives at coupling=0.0.

Microgrid 6-Breeder

An extended microgrid bench with six coupled simulators, each controlled by an independent optimizer. All six share a power bus with configurable coupling. The scenario tests interference detection at scale — 30 possible pairwise interference channels with overlapping watermark signals.

Property Value
Directory examples/bench/scenario-microgrid-6breeder (workflow: bench-scenario-microgrid-6breeder.yml)
Channel type Linear additive
Coupling mechanism Shared power bus — neighbor loads add directly to objectives, all pairs coupled
Detection status Validated — FFT + Rayleigh detection on 20 of 30 pairwise tests with high specificity
Trials produced 450+ per breeder (5/6 breeders active)
Configurable coupling_factor workflow input (default: 0.9)
Watermark slots 6 unique slots using prime period pairs (17/19, 23/29, 31/37, 41/43, 47/53, 67/71)

Each breeder receives a unique watermark slot — a pair of non-overlapping prime periods injected as multi-frequency composite perturbations. The observer performs pairwise detection across all ordered pairs using FFT spectral analysis and Rayleigh phase coherence testing.

Results at coupling_factor=0.9: 5/6 breeders produced 450+ complete trials. Detection correctly identifies coupled pairs and does not fire on uncoupled pairs (e.g., breeder 2 -> breeder 3 non-detection at p=0.34). Zero false positives across all pairwise tests.

Greenhouse

A pair of coupled greenhouse simulators sharing outside climate conditions. Optimizer A's waste heat, CO2 exhaust, and humidity drift affect the outside environment that Optimizer B's greenhouse is exposed to. The coupling signal then passes through zone thermal inertia, multiplicative growth models with dead zones, crop phase transitions with sensitivity jumps, irreversible damage thresholds, and sensor noise before reaching B's observed objectives.

Property Value
Directory examples/bench/scenario-4 (workflow: bench-scenario-4.yml, simulator image: godon-bench-greenhouse)
Channel type Deeply nonlinear cascaded
Coupling mechanism Waste heat, CO2 exhaust, humidity drift through thermal inertia, multiplicative growth, phase-dependent thresholds
Detection status No reliable method yet — research phase
Signal path stages Coupling delta → thermal inertia (×0.01) → zone physics → multiplicative growth (dead zones) → phase-dependent sensitivity (1x→3x) → irreversible damage → sensor noise → receiver exploration variance
SNR at objectives ~0.002 (coupling signal 500× weaker than receiver's own exploration noise)

The watermark signal is physically present in the system but heavily distorted by the time it reaches the observed objectives. All tested detection methods (FFT, mutual information, transfer entropy, Granger causality, cross-correlation, convergent cross mapping) have not produced reliable results at typical trial budgets (200-300 trials).

Promising research directions:

  • Higher trial counts — thousands of trials may provide enough statistical power for methods like mutual information or distance correlation to detect the weak signal
  • Intermediate state detection — measuring interference at raw sensor readings (zone temperatures, CO2 levels) before the growth model, where the signal is less distorted
  • Dedicated analysis phases — holding the receiver optimizer still while the sender probes aggressively, eliminating the 500× exploration noise

Detection Coverage

Scenario Channel Type Status
Microgrid Linear additive Reliable (2 breeders, coupling 0.0-0.9)
Microgrid 6-Breeder Linear additive Validated at scale (6 breeders, 20 pairwise tests)
Greenhouse Deeply nonlinear cascaded Research phase

As new benches are added, they should target specific gaps in this coverage — for example, a nonlinear channel with measurable intermediate state, or a non-stationary channel with phase transitions.

Adding a New Bench

Each bench scenario follows a standard structure:

examples/bench/scenario-<name>/
├── docker-compose.yml          # Simulator containers with coupling config
├── targets/
│   ├── <target>-1.yaml         # Target definition for simulator 1
│   └── <target>-2.yaml         # Target definition for simulator 2
└── breeders/
    ├── breeder-1.yml            # Breeder config for optimizer 1
    └── breeder-2.yml            # Breeder config for optimizer 2

The simulator should expose an HTTP API compatible with godon's target interface. The coupling mechanism should be configurable via environment variables. See the microgrid bench as a reference implementation.

Further Reading