Add Gamma SBAS production workflow

2026-05-21 04:53:39 +08:00
parent 9000feeee8
commit cc22b9ac2d
46 changed files with 11386 additions and 332 deletions
@@ -0,0 +1,439 @@
+# SBAS-InSAR Production Pipeline Design
+
+Date: 2026-05-19
+
+## Decision
+
+SBAS-InSAR production becomes an independent production workflow and page. It must not depend on the existing coarse time-series pairing layer as its production authority.
+
+The old time-series pairing code may remain temporarily for compatibility and candidate discovery, but the new SBAS-InSAR page and backend API should bypass it by default. Deletion should happen only after the new workflow can create, run, publish, and browse Gamma SBAS/IPTA products end to end.
+
+The legacy ISCE2/MintPy time-series production chain is disabled by default. The `timeseries-production` backend code and old catalog pages may remain as compatibility code, but they are no longer exposed as production-management subpages. The active SBAS production route is `/api/sbas-insar-production` and the active UI view is `sbas_insar_production`.
+
+## Scope
+
+Initial production target:
+
+- Sensor: LT1 SLC
+- Engine: Gamma
+- Workflow: Gamma DIFF + IPTA `mb` + `ts_rate`
+- Processor code: `gamma_ipta_sbas`
+- Default display product: LOS velocity toward radar positive
+
+Out of scope for the first implementation slice:
+
+- custom SBAS inversion in application code
+- direct reuse of the current PS/time-series pair graph as final Gamma `itab`
+- full automatic stack approval without a Gamma baseline and quality audit
+- cross-satellite LT1A/LT1B stack mixing
+
+## Product Contract
+
+Every successful SBAS-InSAR production run should publish:
+
+- `product_summary.json`
+- `stack_manifest.json`
+- `gamma_command_manifest.json`
+- `pair_network.json`
+- `quality_summary.json`
+- `los_rate_toward_mm_per_year.tif`
+- `los_rate_toward_mm_per_year.geo_preview.png`
+- `los_rate_toward_mm_per_year.rdc_preview.bmp`
+- `los_rate_away_mm_per_year.tif`
+- `los_rate_away_mm_per_year.rdc_preview.bmp`
+- `los_sigma_mm_per_year.tif`
+- `los_sigma_mm_per_year.geo_preview.png`
+- `los_sigma_mm_per_year.rdc_preview.bmp`
+- `ts_rate_rad_per_year.tif`
+- `sigma_rate_rad_per_year.tif`
+- monitoring-point time-series `png/csv/json`
+- raw logs for each Gamma stage
+
+The current trial product remains the reference implementation:
+
+```text
+backend/runtime/gamma_ipta_trials/lt1b_r114_e1312_n438_20240516_20251002
+```
+
+Preview rule:
+
+- UI default map previews must be rendered from geocoded EPSG:4326 GeoTIFFs.
+- RDC/RMLI BMP browse images are processing QA artifacts only.
+- Product names should make coordinate state explicit: `geo_preview` for map previews and `rdc_preview` for radar-geometry previews.
+
+## LOS Sign Convention
+
+Gamma `ts_rate` outputs phase rate in `rad/year`. The system must store both sign conventions explicitly:
+
+```text
+los_rate_away_mm_per_year = phase_rate * wavelength / (4*pi) * 1000
+los_rate_toward_mm_per_year = -phase_rate * wavelength / (4*pi) * 1000
+```
+
+Default UI display:
+
+```text
+LOS toward radar positive
+```
+
+This matches Gamma `dispmap` default `sflg=0`: motion away from radar is negative, motion toward radar is positive.
+
+## Page Design
+
+Add a separate production view:
+
+```text
+Production Management
+  - D-InSAR Runs
+  - SBAS-InSAR Production
+  - D-InSAR Products
+```
+
+The SBAS page is operational, not a marketing landing page. First screen should show:
+
+- runtime capability: Gamma install, WSL distro, workflow support
+- available SBAS stacks or trial runs
+- selected run summary
+- LOS velocity geocoded preview
+- product file list
+- monitoring-point curve
+- quality metrics
+- stage checklist
+
+The current old "time-series run" and "time-series products" views are hidden from the production workspace. Legacy route aliases such as `ps_production` and `ps_products` should redirect to the SBAS-InSAR production view rather than opening the old ISCE2/MintPy workflow.
+
+## Backend API
+
+Initial read-only API:
+
+```text
+GET /api/sbas-insar-production/capabilities
+GET /api/sbas-insar-production/trial-runs
+GET /api/sbas-insar-production/trial-runs/{trial_id}
+GET /api/sbas-insar-production/trial-runs/{trial_id}/artifacts/{relative_path}
+```
+
+Stack discovery and hard-constraint audit API:
+
+```text
+POST /api/sbas-insar-production/stacks/discover
+POST /api/sbas-insar-production/stacks/{stack_id}/audit
+```
+
+`stacks/discover` scans LT1 source roots directly and groups scenes by:
+
+- platform, for example `LT1A` or `LT1B`
+- satellite mode, for example `MONO`
+- receiving station
+- relative orbit
+- orbit direction
+- imaging mode
+- polarization
+- center bucket, for example `E131.2_N43.8`
+
+It also checks LT1 precise orbit TXT availability against `PYINT_ORBIT_POOL_TXT` / `ORBIT_POOL_ENVI`.
+
+`stacks/{stack_id}/audit` writes a reproducible manifest under:
+
+```text
+backend/runtime/sbas_insar_production/stack_manifests/{stack_id}/
+```
+
+The manifest status is `READY_FOR_GAMMA_BASELINE_AUDIT` only after hard grouping, minimum scene count, precise orbit availability, and an initial adjacent temporal network are satisfied. Gamma `base_calc` remains the next required audit before final `itab` approval.
+
+Writable production planning API:
+
+```text
+POST /api/sbas-insar-production/stacks/{stack_id}/runs
+GET /api/sbas-insar-production/runs
+GET /api/sbas-insar-production/runs/{run_id}
+GET /api/sbas-insar-production/runs/{run_id}/artifacts/{relative_path}
+```
+
+The current `runs` submission is a dry-run planning submission. It writes:
+
+```text
+backend/runtime/sbas_insar_production/runs/{run_id}/run_manifest.json
+backend/runtime/sbas_insar_production/runs/{run_id}/stack_manifest.json
+backend/runtime/sbas_insar_production/runs/{run_id}/pair_network.json
+backend/runtime/sbas_insar_production/runs/{run_id}/gamma_command_manifest.json
+backend/runtime/sbas_insar_production/runs/{run_id}/monitor_points.json
+```
+
+The created run status is:
+
+```text
+PLANNED_GAMMA_BASELINE_AUDIT
+```
+
+This is intentionally not a Gamma execution trigger yet. The next runnable slice should add:
+
+```text
+POST /api/sbas-insar-production/runs/{run_id}/baseline-audit
+POST /api/sbas-insar-production/runs/{run_id}/itab-decision
+POST /api/sbas-insar-production/runs/{run_id}/coregistration
+POST /api/sbas-insar-production/runs/{run_id}/coregistration/jobs
+POST /api/sbas-insar-production/runs/{run_id}/monitor-points
+POST /api/sbas-insar-production/runs/{run_id}/retry-stage
+```
+
+`baseline-audit` currently supports:
+
+- script-only mode: generate/reparse `scripts/01_baseline_audit.sh` and existing outputs
+- execution mode: run Gamma `par_LT1_SLC`, `LT1_precision_orbit.py`, `multi_look`, and `base_calc`
+- output parsing: write `baseline_audit_summary.json` and `pair_network_baseline_audit.json`
+
+`itab-decision` is the current production gate:
+
+- `approve`: copies Gamma `work/gamma/diff/itab_adjacent` to `work/gamma/diff/itab_approved`, writes `itab_decision.json`, moves the run to `ITAB_APPROVED`, and makes `coregistration` the next stage
+- `reject`: writes `itab_decision.json`, moves the run to `ITAB_REJECTED`, and blocks further Gamma stages until the pair network is revised
+
+`coregistration` supports script generation. It writes:
+
+```text
+backend/runtime/sbas_insar_production/runs/{run_id}/scripts/02_coreg_common_ref.sh
+backend/runtime/sbas_insar_production/runs/{run_id}/coregistration_plan.json
+```
+
+The generated script consumes `work/gamma/diff/itab_approved` as the approval gate, uses the stack reference date as common geometry, and prepares Gamma `SLC_coreg.py` calls for every non-reference date.
+
+`coregistration/jobs` submits the generated script to the existing `SystemTask` + `SystemJob` background queue as job type `SBAS_COREGISTRATION`. The job runs Gamma `SLC_coreg.py`, writes `coregistration_summary.json`, updates the run manifest to `COREGISTRATION_READY` or `COREGISTRATION_FAILED`, and advances the next stage to `rdc_dem` only when all expected RSLC/RMLI outputs and common tab files exist.
+
+## Backend Services
+
+First slice:
+
+```text
+sbas_insar_production_service.py
+  - discover local Gamma IPTA trial summaries
+  - normalize products and artifact URLs
+  - expose sign convention and product metadata
+  - serve safe artifacts from trial roots
+```
+
+Second slice:
+
+```text
+gamma_ipta_stack_planner.py
+  - hard group LT1 scenes by platform, relative orbit, direction, mode, polarization, center bucket
+  - require precise orbit availability
+  - emit stack_manifest.json
+
+gamma_ipta_pair_planner.py
+  - build initial temporal network
+  - run Gamma baseline audit
+  - emit pair_network.json and Gamma itab
+
+gamma_ipta_job_runner.py
+  - execute official Gamma commands stage by stage
+  - write command manifests and logs
+
+sbas_insar_product_publisher.py
+  - publish GeoTIFF/BMP/CSV/JSON products
+  - register products into unified result catalog
+```
+
+## Future Database Model
+
+Use unified pipeline tables rather than adding many one-off SBAS tables:
+
+```text
+pipeline_runs
+pipeline_stages
+pipeline_products
+pipeline_quality_metrics
+pipeline_logs
+```
+
+Minimum fields for `pipeline_runs`:
+
+- `run_id`
+- `workflow_code = sbas_insar`
+- `processor_code = gamma_ipta_sbas`
+- `engine_code = gamma`
+- `status`
+- `stack_manifest_path`
+- `work_root`
+- `publish_root`
+- `created_by`
+- `created_at`
+- `started_at`
+- `ended_at`
+- `summary_json`
+
+For the first slice, use filesystem discovery only. Do not add migrations until the run submission workflow is ready.
+
+## Gamma Stage Contract
+
+The managed runner should preserve the successful trial chain:
+
+1. `par_LT1_SLC`
+2. `LT1_precision_orbit.py`
+3. `multi_look`
+4. `base_calc`
+5. `SLC_coreg.py`
+6. `gc_map1` / `geocode` / `gc_map_fine`
+7. `phase_sim_orb`
+8. `SLC_diff_intf`
+9. `adf`
+10. `mcf`
+11. `mb`
+12. `ts_rate`
+13. `geocode_back`
+14. `data2geotiff`
+15. LOS sign conversion and preview generation
+16. monitoring-point time-series extraction
+
+The application is an orchestrator. Gamma remains the processing authority.
+
+## Migration Plan
+
+Phase 1: read-only SBAS production page
+
+- add design document
+- add backend API for existing Gamma trial discovery
+- add page entry and product preview
+- keep old time-series page available as legacy
+
+Phase 2: managed run submission
+
+- add stack discovery and audit endpoints
+- add planned-run submission endpoint
+- write production run manifest, command manifest, and monitor-point config
+- add Gamma runner skeleton
+- queue job with stage updates
+
+Phase 3: unified pipeline management
+
+- add generic pipeline tables
+- move Gamma SBAS run records into pipeline tables
+- register products through the unified product catalog
+
+Phase 4: remove old SBAS/time-series pairing dependency
+
+- hide old SBAS entry completely
+- keep any reusable discovery functions as internal utilities
+- delete obsolete UI and API routes after dependency audit
+
+## Acceptance Criteria For Phase 1
+
+- SBAS-InSAR production appears as its own production workspace view.
+- Existing Gamma IPTA trial can be listed from the backend API.
+- The selected trial shows LOS velocity preview, sigma/GeoTIFF products, monitor-point curve, and quality summary.
+- Artifact serving is constrained to the trial root.
+- No existing D-InSAR, flood, or legacy time-series routes are broken.
+
+## Implementation Progress On 2026-05-19
+
+Implemented:
+
+- read-only SBAS-InSAR production page
+- trial product browser for the local Gamma IPTA validation run
+- artifact API constrained to published trial outputs
+- LT1 filesystem stack discovery independent of the old time-series pairing layer
+- hard grouping by platform, satellite mode, receiving station, relative orbit, orbit direction, imaging mode, polarization, and center bucket
+- precise orbit TXT availability check against the configured Gamma/PyINT orbit pool
+- stack manifest and initial adjacent pair-network JSON generation
+- geocoded web previews generated from `los_rate_toward_mm_per_year.tif` and `los_sigma_mm_per_year.tif`
+- planned SBAS production run creation from a READY stack manifest
+- filesystem production run browser and artifact download API
+- Gamma stage plan manifest with execution disabled until baseline audit runner is attached
+- monitoring-point config contract with explicit placeholder status for `auto_low_sigma_high_rate`
+- Gamma baseline audit script generation and output parser
+- baseline audit result display in the SBAS production page
+- itab approval/rejection API and page controls
+- common-reference coregistration script generation and page summary
+- queued `SBAS_COREGISTRATION` background job submission through the existing task/job queue
+- coregistration execution summary parser and manifest status update to `COREGISTRATION_READY` / `COREGISTRATION_FAILED`
+
+Local verification:
+
+- scanned `1500` LT1 scene directories from the local data pool
+- found READY candidates with all required precise orbit TXT files
+- generated one manifest at:
+
+```text
+backend/runtime/sbas_insar_production/stack_manifests/sbas_2e6301f64a10/20260519T122146Z_stack_manifest.json
+```
+
+- created one dry-run production plan at:
+
+```text
+backend/runtime/sbas_insar_production/runs/sbas_ab96afabead5/run_manifest.json
+```
+
+The dry-run production plan uses the local LT1B relOrbit `114` stack around `E129.2_N44.1`, with `7` scenes and `6` initial adjacent temporal pairs.
+
+- baseline audit script:
+
+```text
+backend/runtime/sbas_insar_production/runs/sbas_ab96afabead5/scripts/01_baseline_audit.sh
+```
+
+- baseline audit summary:
+
+```text
+backend/runtime/sbas_insar_production/runs/sbas_ab96afabead5/baseline_audit_summary.json
+```
+
+The baseline audit ran Gamma 20240627 `par_LT1_SLC`, `LT1_precision_orbit.py`, `multi_look`, and `base_calc` against all 7 LT1B scenes. It completed with status `BASELINE_AUDIT_READY` after the outer terminal command timed out, because the WSL process continued to completion in the background.
+
+Gamma `base_calc` adjacent-network result:
+
+- all-pair count: `21`
+- adjacent-pair count: `6`
+- max absolute perpendicular baseline: `731.9957 m`
+- max temporal gap: `224 days`
+
+The current adjacent network is connected, but several Bperp values are large enough that a human baseline/quality review is still required before using this `itab` for the full SBAS inversion.
+
+The current run has been approved for the next controlled trial step:
+
+```text
+status = ITAB_APPROVED
+next_stage = coregistration
+approved itab = backend/runtime/sbas_insar_production/runs/sbas_ab96afabead5/work/gamma/diff/itab_approved
+decision record = backend/runtime/sbas_insar_production/runs/sbas_ab96afabead5/itab_decision.json
+```
+
+The common-reference co-registration script has been generated:
+
+```text
+backend/runtime/sbas_insar_production/runs/sbas_ab96afabead5/scripts/02_coreg_common_ref.sh
+backend/runtime/sbas_insar_production/runs/sbas_ab96afabead5/coregistration_plan.json
+```
+
+Current status before executing the queued job:
+
+```text
+status = COREGISTRATION_SCRIPT_READY
+next_stage = execute_coregistration
+common reference date = 20241007
+```
+
+The actual `SLC_coreg.py` execution is now wired as a background job endpoint and page action, but has not been production-tested in this pass. It consumes `itab_approved`, not the pre-audit or unapproved pair plan.
+
+Expected post-job outputs:
+
+```text
+backend/runtime/sbas_insar_production/runs/sbas_ab96afabead5/coregistration_summary.json
+backend/runtime/sbas_insar_production/runs/sbas_ab96afabead5/work/gamma/common_20241007/SLC_tab
+backend/runtime/sbas_insar_production/runs/sbas_ab96afabead5/work/gamma/common_20241007/RMLI_tab
+backend/runtime/sbas_insar_production/runs/sbas_ab96afabead5/work/gamma/common_20241007/rslc/*.rslc
+backend/runtime/sbas_insar_production/runs/sbas_ab96afabead5/work/gamma/common_20241007/rmli/*.mli
+```
+
+After the job succeeds:
+
+```text
+status = COREGISTRATION_READY
+next_stage = rdc_dem
+```
+
+Open product-display decisions:
+
+- the current Gamma `*.bmp` previews are RDC processing-geometry products; keep them visible only as QA artifacts
+- the first UI map preview should use `los_rate_toward_mm_per_year.geo_preview.png`
+- the first sigma preview should use `los_sigma_mm_per_year.geo_preview.png`
+- the current monitoring-point curve is a single automatic sample point, not a monitoring network
+- production monitoring curves need user-selected lon/lat points, imported monitoring points, or a quality-filtered automatic sampler before they can be treated as formal outputs