Add SARscape Go wrapper workflow
This commit is contained in:
@@ -0,0 +1,16 @@
|
|||||||
|
* text=auto
|
||||||
|
|
||||||
|
.gitattributes text eol=lf
|
||||||
|
.gitignore text eol=lf
|
||||||
|
*.md text eol=lf
|
||||||
|
*.py text eol=lf
|
||||||
|
*.go text eol=lf
|
||||||
|
*.mod text eol=lf
|
||||||
|
*.toml text eol=lf
|
||||||
|
*.yml text eol=lf
|
||||||
|
*.idl text eol=lf
|
||||||
|
*.pro text eol=lf
|
||||||
|
*.cmd text eol=crlf
|
||||||
|
|
||||||
|
*.exe binary
|
||||||
|
*.sav binary
|
||||||
+18
@@ -7,3 +7,21 @@ __pycache__/
|
|||||||
/.tmp_tests/
|
/.tmp_tests/
|
||||||
logs/
|
logs/
|
||||||
GMTED2010.jp2
|
GMTED2010.jp2
|
||||||
|
|
||||||
|
# Local wrapper state
|
||||||
|
gf3wrapper.json
|
||||||
|
.gf3_extract/
|
||||||
|
.gf3_runtime/
|
||||||
|
|
||||||
|
# Production data and generated outputs
|
||||||
|
/data/
|
||||||
|
/outputs/
|
||||||
|
*.tar.gz
|
||||||
|
*.tgz
|
||||||
|
*.zip
|
||||||
|
*.tif
|
||||||
|
*.tiff
|
||||||
|
*.img
|
||||||
|
*.dat
|
||||||
|
*.sml
|
||||||
|
*.hdr
|
||||||
|
|||||||
@@ -1,17 +1,50 @@
|
|||||||
# GF3 L1A to L2 Pipeline
|
# GF3 L1A to L2 Pipeline
|
||||||
|
|
||||||
This repository contains a Python package and CLI for converting GF3 L1A
|
This repository contains two GF-3 L1A to L2 processing paths:
|
||||||
products to L2 GeoTIFF outputs.
|
|
||||||
|
|
||||||
The heavy work is still handled by GDAL, Rasterio, and NumPy. The package adds
|
- **SARscape Go wrapper**: Windows-oriented production wrapper around ENVI/IDL Runtime and SARscape. This is the recommended path when SARscape is available.
|
||||||
pipeline structure around the original script: explicit configuration, safe path
|
- **Python pipeline**: GDAL/Rasterio/NumPy implementation kept as a pure Python package and CLI.
|
||||||
handling, logging, restart-friendly skipping, per-scene error isolation, and
|
|
||||||
block-based radiometric calibration.
|
|
||||||
|
|
||||||
## Install
|
The SARscape wrapper currently has the more complete operational workflow for GF-3 scene import, multilook/filter/geocode processing, batch directory discovery, saved defaults, and progress reporting.
|
||||||
|
|
||||||
Use an environment that already has a working GDAL/OSGeo Python binding. On
|
## Repository Layout
|
||||||
Windows, Conda/Miniforge is usually the least fragile route.
|
|
||||||
|
```text
|
||||||
|
dist/windows/ Prebuilt Windows executable
|
||||||
|
docs/ Operational documentation
|
||||||
|
src/gf3_l1a2l2/ Python package
|
||||||
|
tests/ Python unit tests and small fixtures
|
||||||
|
tools/gf3_sarscape_wrapper/ Go wrapper source and embedded SAV asset
|
||||||
|
tools/gf3_sarscape_wrapper/idl/ IDL/SARscape source scripts
|
||||||
|
```
|
||||||
|
|
||||||
|
Large GF-3 archives, extracted scenes, SARscape outputs, and local runtime files are intentionally not tracked in Git.
|
||||||
|
|
||||||
|
## Recommended: SARscape Go Wrapper
|
||||||
|
|
||||||
|
Use the prebuilt executable:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
.\dist\windows\gf3wrapper.exe `
|
||||||
|
-input "D:\GF3\L1A_BATCH" `
|
||||||
|
-output "E:\GF3\L2_SARscape" `
|
||||||
|
-dem "D:\DEM\COPDEM_GLO30_China_4326_DEM" `
|
||||||
|
-pol "HH,HV"
|
||||||
|
```
|
||||||
|
|
||||||
|
Supported input forms:
|
||||||
|
|
||||||
|
- A single `*.tar.gz` GF-3 L1A package.
|
||||||
|
- A single `*.meta.xml` scene metadata file.
|
||||||
|
- A directory containing multiple `*.tar.gz` or `*.meta.xml` scenes.
|
||||||
|
|
||||||
|
The wrapper writes one output subdirectory per scene. On successful runs it saves defaults next to the executable in `gf3wrapper.json`, so double-click interactive mode can reuse the previous input/output/DEM/polarization settings.
|
||||||
|
|
||||||
|
More details: [docs/sarscape_go_wrapper.md](docs/sarscape_go_wrapper.md)
|
||||||
|
|
||||||
|
## Python Pipeline
|
||||||
|
|
||||||
|
Use an environment that already has working GDAL/OSGeo Python bindings. On Windows, Conda/Miniforge is usually the least fragile route.
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
conda env create -f environment.yml
|
conda env create -f environment.yml
|
||||||
@@ -19,7 +52,7 @@ conda activate gf3-l1a2l2
|
|||||||
pip install -e . --no-deps
|
pip install -e . --no-deps
|
||||||
```
|
```
|
||||||
|
|
||||||
## Run
|
Run:
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
gf3-l1a2l2 run --input D:\GF3\L1A_BATCH --output E:\GF3\L2_Image --dem .\GMTED2010.jp2
|
gf3-l1a2l2 run --input D:\GF3\L1A_BATCH --output E:\GF3\L2_Image --dem .\GMTED2010.jp2
|
||||||
@@ -38,22 +71,29 @@ gf3-l1a2l2 run `
|
|||||||
--dst-srs EPSG:4326
|
--dst-srs EPSG:4326
|
||||||
```
|
```
|
||||||
|
|
||||||
By default, output files are written under one subdirectory per discovered
|
Legacy wrappers remain available:
|
||||||
scene. Use `--flat-output` to write all outputs directly into the output
|
|
||||||
directory.
|
|
||||||
|
|
||||||
The legacy files are now compatibility wrappers:
|
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
python gf3_L1A_To_L2.py run --input D:\GF3\L1A_BATCH --output E:\GF3\L2_Image --dem .\GMTED2010.jp2
|
python gf3_L1A_To_L2.py run --input D:\GF3\L1A_BATCH --output E:\GF3\L2_Image --dem .\GMTED2010.jp2
|
||||||
python Decompression.py D:\GF3\L1A_BATCH
|
python Decompression.py D:\GF3\L1A_BATCH
|
||||||
```
|
```
|
||||||
|
|
||||||
## Notes
|
## Development Checks
|
||||||
|
|
||||||
- The CLI extracts `.tar.gz` and `.zip` products before processing unless
|
Python:
|
||||||
`--skip-decompress` is used.
|
|
||||||
- Existing L2 outputs are skipped unless `--overwrite` is used.
|
```powershell
|
||||||
- Logs are written to `<output>\logs`.
|
python -m pytest
|
||||||
- Without representative GF3 products, verification is limited to parser unit
|
```
|
||||||
tests and Python syntax/import checks.
|
|
||||||
|
Go wrapper:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
cd .\tools\gf3_sarscape_wrapper
|
||||||
|
$env:GOTELEMETRY = "off"
|
||||||
|
go build -o ..\..\dist\windows\gf3wrapper.exe .
|
||||||
|
```
|
||||||
|
|
||||||
|
## Data Policy
|
||||||
|
|
||||||
|
Do not commit production GF-3 archives or generated products. Keep large data on NAS/storage and document the path in run notes. Only small sanitized fixtures should be added under `tests/fixtures`.
|
||||||
|
|||||||
Vendored
+17
@@ -0,0 +1,17 @@
|
|||||||
|
# Windows Distribution
|
||||||
|
|
||||||
|
`gf3wrapper.exe` is the prebuilt SARscape Go wrapper.
|
||||||
|
|
||||||
|
Run from the repository root:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
.\dist\windows\gf3wrapper.exe `
|
||||||
|
-input "D:\GF3\L1A_BATCH" `
|
||||||
|
-output "E:\GF3\L2_SARscape" `
|
||||||
|
-dem "D:\DEM\COPDEM_GLO30_China_4326_DEM" `
|
||||||
|
-pol "HH,HV"
|
||||||
|
```
|
||||||
|
|
||||||
|
The executable requires ENVI/IDL Runtime and SARscape on the target machine.
|
||||||
|
|
||||||
|
After successful runs, `gf3wrapper.json` is created in this directory to store defaults. That file is local runtime state and is ignored by Git.
|
||||||
Vendored
BIN
Binary file not shown.
@@ -0,0 +1,45 @@
|
|||||||
|
# Repository Layout
|
||||||
|
|
||||||
|
```text
|
||||||
|
.
|
||||||
|
|-- dist/
|
||||||
|
| `-- windows/
|
||||||
|
| `-- gf3wrapper.exe
|
||||||
|
|-- docs/
|
||||||
|
| |-- repository_layout.md
|
||||||
|
| `-- sarscape_go_wrapper.md
|
||||||
|
|-- src/
|
||||||
|
| `-- gf3_l1a2l2/
|
||||||
|
|-- tests/
|
||||||
|
| `-- fixtures/
|
||||||
|
|-- tools/
|
||||||
|
| `-- gf3_sarscape_wrapper/
|
||||||
|
| |-- assets/
|
||||||
|
| |-- idl/
|
||||||
|
| |-- go.mod
|
||||||
|
| `-- main.go
|
||||||
|
|-- README.md
|
||||||
|
|-- environment.yml
|
||||||
|
`-- pyproject.toml
|
||||||
|
```
|
||||||
|
|
||||||
|
## What Belongs In Git
|
||||||
|
|
||||||
|
- Python package source.
|
||||||
|
- Go wrapper source.
|
||||||
|
- IDL/SARscape source scripts.
|
||||||
|
- Small embedded runtime assets needed to build the wrapper.
|
||||||
|
- Small sanitized tests and fixtures.
|
||||||
|
- Documentation and reproducible build instructions.
|
||||||
|
- Small convenience binaries when the team explicitly wants a ready-to-run Windows artifact.
|
||||||
|
|
||||||
|
## What Does Not Belong In Git
|
||||||
|
|
||||||
|
- Raw GF-3 production archives.
|
||||||
|
- Extracted scenes.
|
||||||
|
- SARscape output directories.
|
||||||
|
- Local wrapper config files such as `gf3wrapper.json`.
|
||||||
|
- Runtime folders such as `.gf3_extract` and `.gf3_runtime`.
|
||||||
|
- DEM files and other large environment-specific dependencies.
|
||||||
|
|
||||||
|
Use NAS or object storage for production data and record exact paths in run logs or issue notes.
|
||||||
@@ -0,0 +1,112 @@
|
|||||||
|
# SARscape Go Wrapper
|
||||||
|
|
||||||
|
`gf3wrapper.exe` is a Windows command-line wrapper for the SARscape/IDL GF-3 processing workflow. It embeds `gf3_sarscape_cli.sav`, starts `idlrt.exe`, and passes scene metadata, output directory, DEM, and polarization arguments to SARscape.
|
||||||
|
|
||||||
|
This path is intended for operational runs where ENVI/IDL Runtime and SARscape are installed and licensed.
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
- Windows host.
|
||||||
|
- ENVI/IDL Runtime, default path:
|
||||||
|
`C:\Program Files\Harris\ENVI56\IDL88\bin\bin.x86_64\idlrt.exe`
|
||||||
|
- SARscape available to the IDL Runtime environment.
|
||||||
|
- DEM file, for example:
|
||||||
|
`D:\DEM\COPDEM_GLO30_China_4326_DEM`
|
||||||
|
- GF-3 L1A input as `*.tar.gz`, `*.meta.xml`, or a directory containing multiple scenes.
|
||||||
|
|
||||||
|
The IDL Runtime path can be overridden with `-idlrt` or the `IDLRT_PATH` environment variable.
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
Use the prebuilt executable:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
.\dist\windows\gf3wrapper.exe `
|
||||||
|
-input "D:\GF3\L1A_BATCH" `
|
||||||
|
-output "E:\GF3\L2_SARscape" `
|
||||||
|
-dem "D:\DEM\COPDEM_GLO30_China_4326_DEM" `
|
||||||
|
-pol "HH,HV"
|
||||||
|
```
|
||||||
|
|
||||||
|
Double-click mode is also supported. When launched without arguments, the wrapper prompts for input, output, DEM, polarizations, and IDL Runtime path. Press Enter to reuse saved defaults.
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
The wrapper supports:
|
||||||
|
|
||||||
|
- A single GF-3 archive: `GF3_....tar.gz`
|
||||||
|
- A single extracted scene metadata file: `GF3_....meta.xml`
|
||||||
|
- A directory containing many archives and/or extracted scenes
|
||||||
|
|
||||||
|
Directory mode recursively scans for `*.tar.gz`, `*.tgz`, and `*.meta.xml`. If both archive and metadata exist for the same scene name, the archive is preferred. Internal `.gf3_extract` folders are skipped.
|
||||||
|
|
||||||
|
## Outputs
|
||||||
|
|
||||||
|
Each scene is processed into its own output directory:
|
||||||
|
|
||||||
|
```text
|
||||||
|
E:\GF3\L2_SARscape\
|
||||||
|
GF3_SCENE_NAME\
|
||||||
|
.gf3_extract\ Extracted archive, when input was .tar.gz
|
||||||
|
*_slc.sml
|
||||||
|
*_ml.sml
|
||||||
|
*_filt.sml
|
||||||
|
*_geo.sml
|
||||||
|
```
|
||||||
|
|
||||||
|
The embedded SAV is materialized under:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<output>\.gf3_runtime\gf3_sarscape_cli.sav
|
||||||
|
```
|
||||||
|
|
||||||
|
## Saved Defaults
|
||||||
|
|
||||||
|
After a successful run, the wrapper writes `gf3wrapper.json` next to the executable:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"idlrt_path": "C:\\Program Files\\Harris\\ENVI56\\IDL88\\bin\\bin.x86_64\\idlrt.exe",
|
||||||
|
"dem_file": "D:\\DEM\\COPDEM_GLO30_China_4326_DEM",
|
||||||
|
"polarizations": "HH,HV",
|
||||||
|
"last_input": "D:\\GF3\\L1A_BATCH",
|
||||||
|
"last_output": "E:\\GF3\\L2_SARscape",
|
||||||
|
"keep_extracted": true
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Command-line flags override saved defaults. If `-input`, `-output`, `-dem`, or `-pol` are omitted, saved values are reused when available.
|
||||||
|
|
||||||
|
## Build
|
||||||
|
|
||||||
|
From the repository root:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
cd .\tools\gf3_sarscape_wrapper
|
||||||
|
$env:GOTELEMETRY = "off"
|
||||||
|
go build -o ..\..\dist\windows\gf3wrapper.exe .
|
||||||
|
```
|
||||||
|
|
||||||
|
If Go cannot write to the default user cache directory, set local cache paths:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
$env:GOCACHE = "C:\Temp\go-build-cache"
|
||||||
|
$env:APPDATA = "C:\Temp\go-appdata"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Source Files
|
||||||
|
|
||||||
|
```text
|
||||||
|
tools/gf3_sarscape_wrapper/main.go
|
||||||
|
tools/gf3_sarscape_wrapper/assets/gf3_sarscape_cli.sav
|
||||||
|
tools/gf3_sarscape_wrapper/idl/gf3_sarscape_cli.pro
|
||||||
|
tools/gf3_sarscape_wrapper/idl/gf3_sarscape_pipeline.pro
|
||||||
|
```
|
||||||
|
|
||||||
|
`assets/gf3_sarscape_cli.sav` is embedded into the Go executable at build time.
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- Keep production archives and generated products outside Git.
|
||||||
|
- Keep only small sanitized fixtures in `tests/fixtures`.
|
||||||
|
- The prebuilt executable in `dist/windows` is a convenience artifact for Windows users. Rebuild it after changing `main.go` or the embedded `.sav`.
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
# GF-3 SARscape Go Wrapper
|
||||||
|
|
||||||
|
This wrapper embeds `assets/gf3_sarscape_cli.sav` and calls the local ENVI/IDL Runtime:
|
||||||
|
|
||||||
|
```text
|
||||||
|
idlrt.exe gf3_sarscape_cli.sav -args input_meta_xml out_dir dem_file HH,HV
|
||||||
|
```
|
||||||
|
|
||||||
|
It is the operational Windows path for SARscape-based GF-3 processing.
|
||||||
|
|
||||||
|
## Build
|
||||||
|
|
||||||
|
From this directory:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
$env:GOTELEMETRY = "off"
|
||||||
|
go build -o ..\..\dist\windows\gf3wrapper.exe .
|
||||||
|
```
|
||||||
|
|
||||||
|
## Run
|
||||||
|
|
||||||
|
Single archive:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
..\..\dist\windows\gf3wrapper.exe `
|
||||||
|
-input "D:\GF3\GF3_SCENE.tar.gz" `
|
||||||
|
-output "E:\GF3\L2_SARscape" `
|
||||||
|
-dem "D:\DEM\COPDEM_GLO30_China_4326_DEM" `
|
||||||
|
-pol "HH,HV"
|
||||||
|
```
|
||||||
|
|
||||||
|
Batch directory:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
..\..\dist\windows\gf3wrapper.exe `
|
||||||
|
-input "D:\GF3\L1A_BATCH" `
|
||||||
|
-output "E:\GF3\L2_SARscape"
|
||||||
|
```
|
||||||
|
|
||||||
|
If `gf3wrapper.json` exists next to the executable, omitted `-input`, `-output`, `-dem`, and `-pol` values are loaded from that file.
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- The server still needs ENVI, IDL Runtime, and SARscape installed and licensed.
|
||||||
|
- `.tar.gz` inputs are extracted under each scene output directory in `.gf3_extract`.
|
||||||
|
- The embedded SAV is extracted to `<output>\.gf3_runtime\gf3_sarscape_cli.sav`.
|
||||||
|
- More operational details are in `docs/sarscape_go_wrapper.md`.
|
||||||
Binary file not shown.
@@ -0,0 +1,3 @@
|
|||||||
|
module gf3wrapper
|
||||||
|
|
||||||
|
go 1.22
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
.compile 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\gf3_sarscape_pipeline.pro'
|
||||||
|
.compile 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\gf3_sarscape_cli.pro'
|
||||||
|
resolve_all, skip_routines='envi', /continue_on_error
|
||||||
|
save, /routines, filename='C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\gf3_sarscape_cli.sav'
|
||||||
|
exit
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
.f
|
||||||
|
.compile 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\gf3_sarscape_pipeline.pro'
|
||||||
|
.compile 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\gf3_sarscape_cli.pro'
|
||||||
|
save, /routines, filename='C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\gf3_sarscape_cli.sav'
|
||||||
|
exit
|
||||||
@@ -0,0 +1,79 @@
|
|||||||
|
function gf3_cli_split_csv, value
|
||||||
|
compile_opt idl2
|
||||||
|
|
||||||
|
text = strtrim(value, 2)
|
||||||
|
if text eq '' then return, ['HH', 'HV']
|
||||||
|
|
||||||
|
parts = ['']
|
||||||
|
first = 1
|
||||||
|
while 1 do begin
|
||||||
|
pos = strpos(text, ',')
|
||||||
|
if pos lt 0 then begin
|
||||||
|
token = strtrim(text, 2)
|
||||||
|
if first then parts = [token] else parts = [parts, token]
|
||||||
|
break
|
||||||
|
endif
|
||||||
|
token = strtrim(strmid(text, 0, pos), 2)
|
||||||
|
if first then begin
|
||||||
|
parts = [token]
|
||||||
|
first = 0
|
||||||
|
endif else begin
|
||||||
|
parts = [parts, token]
|
||||||
|
endelse
|
||||||
|
text = strtrim(strmid(text, pos + 1), 2)
|
||||||
|
endwhile
|
||||||
|
|
||||||
|
return, parts
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_sarscape_cli
|
||||||
|
compile_opt idl2
|
||||||
|
|
||||||
|
args = command_line_args(count=argc)
|
||||||
|
if argc lt 3 then begin
|
||||||
|
print, 'Usage: idlrt.exe gf3_sarscape_cli.sav -args input_meta_xml out_dir dem_file [polarizations]'
|
||||||
|
print, 'Example polarizations: HH,HV or HH or HV'
|
||||||
|
return
|
||||||
|
endif
|
||||||
|
|
||||||
|
input_meta_xml = args[0]
|
||||||
|
out_dir = args[1]
|
||||||
|
dem_file = args[2]
|
||||||
|
file_mkdir, out_dir
|
||||||
|
|
||||||
|
log_file = out_dir + '\gf3_sarscape_cli.log'
|
||||||
|
openw, lun, log_file, /get_lun
|
||||||
|
catch, err
|
||||||
|
if err ne 0 then begin
|
||||||
|
printf, lun, 'ERROR: ', !error_state.msg
|
||||||
|
free_lun, lun
|
||||||
|
catch, /cancel
|
||||||
|
return
|
||||||
|
endif
|
||||||
|
|
||||||
|
printf, lun, 'CLI start'
|
||||||
|
printf, lun, 'input_meta_xml=', input_meta_xml
|
||||||
|
printf, lun, 'out_dir=', out_dir
|
||||||
|
printf, lun, 'dem_file=', dem_file
|
||||||
|
|
||||||
|
if argc ge 4 then begin
|
||||||
|
polarizations = gf3_cli_split_csv(args[3])
|
||||||
|
endif else begin
|
||||||
|
polarizations = ['HH', 'HV']
|
||||||
|
endelse
|
||||||
|
|
||||||
|
printf, lun, 'polarizations=', strjoin(polarizations, ',')
|
||||||
|
|
||||||
|
e = envi(/headless)
|
||||||
|
printf, lun, 'ENVI headless initialized'
|
||||||
|
|
||||||
|
gf3_sarscape_run, $
|
||||||
|
input_meta_xml=input_meta_xml, $
|
||||||
|
out_dir=out_dir, $
|
||||||
|
dem_file=dem_file, $
|
||||||
|
polarizations=polarizations
|
||||||
|
|
||||||
|
printf, lun, 'CLI end'
|
||||||
|
free_lun, lun
|
||||||
|
end
|
||||||
@@ -0,0 +1,506 @@
|
|||||||
|
function gf3_normalize_errmsg, value
|
||||||
|
compile_opt idl2
|
||||||
|
|
||||||
|
if n_elements(value) eq 0 then return, ''
|
||||||
|
|
||||||
|
sz = size(value, /tname)
|
||||||
|
|
||||||
|
if sz eq 'STRING' then begin
|
||||||
|
if n_elements(value) eq 1 then return, value
|
||||||
|
return, strjoin(value, ' ')
|
||||||
|
endif
|
||||||
|
|
||||||
|
if sz eq 'BYTE' then begin
|
||||||
|
if n_elements(value) eq 0 then return, ''
|
||||||
|
return, string(value)
|
||||||
|
endif
|
||||||
|
|
||||||
|
return, string(value)
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
function gf3_float_str, value
|
||||||
|
compile_opt idl2
|
||||||
|
return, string(double(value), format='(F0.1)')
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
function gf3_base_root, polarization
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
|
||||||
|
pol = strupcase(strtrim(polarization, 2))
|
||||||
|
case pol of
|
||||||
|
'HH': return, cfg.imported_hh_root
|
||||||
|
'HV': return, cfg.imported_hv_root
|
||||||
|
else: message, 'Unsupported polarization: ' + pol
|
||||||
|
endcase
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
function gf3_stage_root, polarization, stage_name
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
|
||||||
|
pol = strlowcase(strtrim(polarization, 2))
|
||||||
|
prefix = cfg.output_prefix
|
||||||
|
if prefix eq '' then prefix = 'gf3'
|
||||||
|
|
||||||
|
case strlowcase(strtrim(stage_name, 2)) of
|
||||||
|
'multilook': return, cfg.out_dir + '\' + prefix + '_' + pol + '_ml'
|
||||||
|
'filter': return, cfg.out_dir + '\' + prefix + '_' + pol + '_filt'
|
||||||
|
'geocode': return, cfg.out_dir + '\' + prefix + '_' + pol + '_geo'
|
||||||
|
else: message, 'Unsupported stage: ' + stage_name
|
||||||
|
endcase
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
function gf3_scene_token, meta_xml
|
||||||
|
compile_opt idl2
|
||||||
|
|
||||||
|
base = file_basename(meta_xml)
|
||||||
|
pos = strpos(strlowcase(base), '.meta.xml')
|
||||||
|
if pos ge 0 then base = strmid(base, 0, pos)
|
||||||
|
return, base
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
function gf3_imported_root_for, polarization
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
|
||||||
|
pol = strupcase(strtrim(polarization, 2))
|
||||||
|
pattern = cfg.out_dir + '\*_' + pol + '_slc.sml'
|
||||||
|
matches = file_search(pattern, count=count)
|
||||||
|
if count eq 0 then return, ''
|
||||||
|
|
||||||
|
newest = matches[0]
|
||||||
|
newest_time = (file_info(newest)).mtime
|
||||||
|
for i = 1, count - 1 do begin
|
||||||
|
info = file_info(matches[i])
|
||||||
|
if info.mtime gt newest_time then begin
|
||||||
|
newest = matches[i]
|
||||||
|
newest_time = info.mtime
|
||||||
|
endif
|
||||||
|
endfor
|
||||||
|
|
||||||
|
return, strmid(newest, 0, strlen(newest) - 4)
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_refresh_import_roots
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
|
||||||
|
hh = gf3_imported_root_for('HH')
|
||||||
|
hv = gf3_imported_root_for('HV')
|
||||||
|
|
||||||
|
if hh ne '' then cfg.imported_hh_root = hh
|
||||||
|
if hv ne '' then cfg.imported_hv_root = hv
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_run_module, module_name, set_params_proc, errmsg=errmsg, working_directory=working_directory
|
||||||
|
compile_opt idl2
|
||||||
|
|
||||||
|
errmsg = ''
|
||||||
|
ob = obj_new('SARscapeBatch', Module=module_name)
|
||||||
|
if ~obj_valid(ob) then begin
|
||||||
|
errmsg = 'Create object failed: ' + module_name
|
||||||
|
print, errmsg
|
||||||
|
return
|
||||||
|
endif
|
||||||
|
|
||||||
|
ob->SetParam, 'GENERAL_PARAMETERS_CMD.AVAILABLE_MEMORY_SIZE_GB', '8.0000000'
|
||||||
|
ob->SetParam, 'GENERAL_PARAMETERS_CMD.LOAD_IMAGES', 'OK'
|
||||||
|
ob->SetParam, 'GENERAL_PARAMETERS_CMD.DELETE_TEMPORARY_FILES', 'OK'
|
||||||
|
ob->SetParam, 'GENERAL_PARAMETERS_CMD.MAKE_TIFF', 'OK'
|
||||||
|
ob->SetParam, 'GENERAL_PARAMETERS_CMD.QUICK_LOOK_FORMAT', 'ql_tiff'
|
||||||
|
ob->SetParam, 'GENERAL_PARAMETERS_CMD.SARSCAPE_TRACE_LEVEL', '3.0000000'
|
||||||
|
ob->SetParam, 'GENERAL_PARAMETERS_CMD.RENAME_THE_FILE_USING_PARAMETERS_FLAG', 'OK'
|
||||||
|
ob->SetParam, 'GENERAL_PARAMETERS_CMD.INSERT_GEO_POINTS_FLAG', 'NotOK'
|
||||||
|
ob->SetParam, 'GENERAL_PARAMETERS_CMD.VERBOSE_TRACE', 'OK'
|
||||||
|
ob->SetParam, 'GENERAL_PARAMETERS_CMD.VERBOSE_STEP', 'OK'
|
||||||
|
ob->SetParam, 'GENERAL_PARAMETERS_CMD.VERBOSE_BAR', 'OK'
|
||||||
|
ob->SetParam, 'GENERAL_PARAMETERS_CMD.APPLY_CALIBRATION_CONSTANT_FLAG', 'OK'
|
||||||
|
ob->SetParam, 'GENERAL_PARAMETERS_CMD.FILL_DUMMY_DURING_IMPORT', 'OK'
|
||||||
|
|
||||||
|
call_procedure, set_params_proc, ob
|
||||||
|
|
||||||
|
ok = ob->VerifyParams(Silent=0)
|
||||||
|
if ~ok then begin
|
||||||
|
errmsg = 'VerifyParams failed: ' + module_name
|
||||||
|
print, errmsg
|
||||||
|
return
|
||||||
|
endif
|
||||||
|
|
||||||
|
if n_elements(working_directory) eq 0 then begin
|
||||||
|
ok = ob->Execute()
|
||||||
|
endif else begin
|
||||||
|
exec_errmsg = ''
|
||||||
|
ok = ob->Execute(WORKING_DIRECTORY=working_directory, ERRMSG=exec_errmsg)
|
||||||
|
errmsg = gf3_normalize_errmsg(exec_errmsg)
|
||||||
|
endelse
|
||||||
|
|
||||||
|
if ok then begin
|
||||||
|
print, 'Success: ' + module_name
|
||||||
|
endif else begin
|
||||||
|
if errmsg eq '' then begin
|
||||||
|
err_code = ''
|
||||||
|
err_text = get_SARscape_error_string('OK', ERROR_CODE=err_code)
|
||||||
|
errmsg = 'Execution failed [' + module_name + '] EC [' + err_code + ']: ' + err_text
|
||||||
|
endif
|
||||||
|
print, errmsg
|
||||||
|
endelse
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_pipeline_set_import_params, ob
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
|
||||||
|
ob->SetParam, 'MAIN_BASIC_IMPORT_GAOFEN3_CMD_CMD.SARSCAPEENVIRONMENT', 'IDL_ENVI_ENV'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_IMPORT_GAOFEN3_CMD_CMD.INPUT_FILE_LIST', [cfg.input_meta_xml]
|
||||||
|
ob->SetParam, 'MAIN_BASIC_IMPORT_GAOFEN3_CMD_CMD.OUTPUT_FILE_LIST', [cfg.import_root]
|
||||||
|
ob->SetParam, 'MAIN_BASIC_IMPORT_GAOFEN3_CMD_CMD.CROSS_COPOLARIZATION_FLAG', cfg.import_polarization_mode
|
||||||
|
ob->SetParam, 'MAIN_BASIC_IMPORT_GAOFEN3_CMD_CMD.DOPP_ESTIMATION_FROM_RASTER_FLAG', 'OK'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_IMPORT_GAOFEN3_CMD_CMD.KEEP_ALL_LINES_FLAG', 'NotOK'
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_pipeline_set_multilook_params, ob
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
common gf3_pipeline_state, state
|
||||||
|
|
||||||
|
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.SARSCAPEENVIRONMENT', 'IDL_ENVI_ENV'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.INPUT_FILE_LIST', [state.current_input_root]
|
||||||
|
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.OUTPUT_FILE_LIST', [state.current_output_root]
|
||||||
|
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.MULTILOOK_METHOD', cfg.multilook_method
|
||||||
|
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.RANGE_MULTILOOK', gf3_float_str(cfg.range_looks)
|
||||||
|
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.AZIMUTH_MULTILOOK', gf3_float_str(cfg.azimuth_looks)
|
||||||
|
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.GRID_SIZE_FOR_SUGGESTED_LOOKS', gf3_float_str(cfg.grid_size_for_suggested_looks)
|
||||||
|
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.FILL_DUMMY_FLAG', 'OK'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.FILL_DUMMY_METHOD', 'min_image_value'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.ROWS_WINDOW_NUMBER', '3.0000000'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.COLS_WINDOW_NUMBER', '3.0000000'
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_pipeline_set_filter_params, ob
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
common gf3_pipeline_state, state
|
||||||
|
|
||||||
|
ob->SetParam, 'MAIN_BASIC_DESPECKLE_CONVENTIONAL_CMD.SARSCAPEENVIRONMENT', 'IDL_ENVI_ENV'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_DESPECKLE_CONVENTIONAL_CMD.INPUT_FILE_LIST', [state.current_input_root]
|
||||||
|
ob->SetParam, 'MAIN_BASIC_DESPECKLE_CONVENTIONAL_CMD.OUTPUT_FILE_LIST', [state.current_output_root]
|
||||||
|
ob->SetParam, 'MAIN_BASIC_DESPECKLE_CONVENTIONAL_CMD.FILT_TYPE', cfg.filter_type
|
||||||
|
ob->SetParam, 'MAIN_BASIC_DESPECKLE_CONVENTIONAL_CMD.EQUIVALENT_LOOKS', '-1.0000000'
|
||||||
|
ob->SetParam, 'PARAMETERS_GENERIC_FILTERS.WIN_SIZE', gf3_float_str(cfg.filter_window)
|
||||||
|
ob->SetParam, 'PARAMETERS_GENERIC_FILTERS.ROWS_WINDOW_NUMBER', gf3_float_str(cfg.filter_window)
|
||||||
|
ob->SetParam, 'PARAMETERS_GENERIC_FILTERS.COLS_WINDOW_NUMBER', gf3_float_str(cfg.filter_window)
|
||||||
|
ob->SetParam, 'PARAMETERS_GENERIC_FILTERS.WIN_MODE_SIZE', gf3_float_str(cfg.filter_window)
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_pipeline_set_geocode_params, ob
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
common gf3_pipeline_state, state
|
||||||
|
|
||||||
|
ob->SetParam, 'OUT_CARTOGRAPHIC_SYSTEM.OCS_CARTOGRAPHIC_SYSTEM_SPECIFICATION', ''
|
||||||
|
ob->SetParam, 'OUT_CARTOGRAPHIC_SYSTEM.OCS_STATE', cfg.ocs_state
|
||||||
|
ob->SetParam, 'OUT_CARTOGRAPHIC_SYSTEM.OCS_HEMISPHERE', cfg.ocs_hemisphere
|
||||||
|
ob->SetParam, 'OUT_CARTOGRAPHIC_SYSTEM.OCS_PROJECTION', cfg.ocs_projection
|
||||||
|
ob->SetParam, 'OUT_CARTOGRAPHIC_SYSTEM.OCS_ELLIPSOID', 'WGS84'
|
||||||
|
ob->SetParam, 'OUT_CARTOGRAPHIC_SYSTEM.OCS_REFERENCE_HEIGHT', '0.0'
|
||||||
|
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.SARSCAPEENVIRONMENT', 'IDL_ENVI_ENV'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.INPUT_FILE_LIST', [state.current_input_root]
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.OUTPUT_FILE_LIST', [state.current_output_root]
|
||||||
|
if cfg.dem_file ne '' then ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.DEM_FILE_NAME', cfg.dem_file
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_GRID_SIZE_X', gf3_float_str(cfg.geocode_grid_x)
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_GRID_SIZE_Y', gf3_float_str(cfg.geocode_grid_y)
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_RESAMPLING_TYPE', cfg.geocode_resampling
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.CALIBRATION_FLAG', cfg.calibration_flag
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEO_SCATTERING_AREA_METHOD', 'sine_area_estimation'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.TRUE_AREA_EXPONENT', '1.0000000'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.MAX_VALUE_IN_CALIBRATION', '100.00000'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.RAD_NORMALIZATION_FLAG', cfg.rad_normalization_flag
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_RAD_NORM_DEG', '2.0000000'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_RAD_NORM_ANG', '-1.0000000'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_SIGMA_FLAG', cfg.output_sigma_flag
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_GAMMA_FLAG', cfg.output_gamma_flag
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_BETA_FLAG', cfg.output_beta_flag
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_COREG_IMAGES_FLAG', 'NotOK'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GENERATE_LIA_FLAG', 'NotOK'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.OUTPUT_TYPE', cfg.output_type
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.DUMMY_REMOVAL_FLAG', 'NotOK'
|
||||||
|
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_BLOCK_SIZE', '30000.000'
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_init_default_cfg
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
|
||||||
|
cfg = { $
|
||||||
|
input_meta_xml: 'C:\Users\Administrator\Desktop\GF3_Test\GF3_scene_20260513\GF3_SYC_FSI_051370_E124.6_N51.7_20260513_L1A_HHHV_L10007355955.meta.xml', $
|
||||||
|
work_dir: 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\work', $
|
||||||
|
temp_dir: 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\temp', $
|
||||||
|
out_dir: 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\out', $
|
||||||
|
output_prefix: '', $
|
||||||
|
dem_file: 'D:\DEM\COPDEM_GLO30_China_4326_DEM', $
|
||||||
|
import_root: 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\out\gf3_import', $
|
||||||
|
import_polarization_mode: 'ALL_POL', $
|
||||||
|
imported_hh_root: 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\out\gaofer3_20260513_214340051_D_HH_slc', $
|
||||||
|
imported_hv_root: 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\out\gaofer3_20260513_214340051_D_HV_slc', $
|
||||||
|
multilook_method: 'time_domain', $
|
||||||
|
range_looks: 2.0, $
|
||||||
|
azimuth_looks: 2.0, $
|
||||||
|
grid_size_for_suggested_looks: 15.0, $
|
||||||
|
filter_type: 'lee', $
|
||||||
|
filter_window: 5.0, $
|
||||||
|
geocode_grid_x: 15.0, $
|
||||||
|
geocode_grid_y: 15.0, $
|
||||||
|
geocode_resampling: '4th_order_cc', $
|
||||||
|
calibration_flag: 'OK', $
|
||||||
|
rad_normalization_flag: 'OK', $
|
||||||
|
output_sigma_flag: 'OK', $
|
||||||
|
output_gamma_flag: 'NotOK', $
|
||||||
|
output_beta_flag: 'NotOK', $
|
||||||
|
output_type: 'output_type_linear', $
|
||||||
|
ocs_state: 'GEO-GLOBAL', $
|
||||||
|
ocs_hemisphere: 'NONE', $
|
||||||
|
ocs_projection: 'GEO' $
|
||||||
|
}
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_configure, input_meta_xml=input_meta_xml, out_dir=out_dir, dem_file=dem_file, $
|
||||||
|
work_dir=work_dir, temp_dir=temp_dir, range_looks=range_looks, $
|
||||||
|
azimuth_looks=azimuth_looks, filter_type=filter_type, filter_window=filter_window, $
|
||||||
|
geocode_grid_x=geocode_grid_x, geocode_grid_y=geocode_grid_y, $
|
||||||
|
imported_hh_root=imported_hh_root, imported_hv_root=imported_hv_root
|
||||||
|
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
|
||||||
|
gf3_init_default_cfg
|
||||||
|
|
||||||
|
if n_elements(input_meta_xml) gt 0 then cfg.input_meta_xml = input_meta_xml
|
||||||
|
if n_elements(out_dir) gt 0 then cfg.out_dir = out_dir
|
||||||
|
if n_elements(dem_file) gt 0 then cfg.dem_file = dem_file
|
||||||
|
if n_elements(work_dir) gt 0 then cfg.work_dir = work_dir
|
||||||
|
if n_elements(temp_dir) gt 0 then cfg.temp_dir = temp_dir
|
||||||
|
if n_elements(range_looks) gt 0 then cfg.range_looks = double(range_looks)
|
||||||
|
if n_elements(azimuth_looks) gt 0 then cfg.azimuth_looks = double(azimuth_looks)
|
||||||
|
if n_elements(filter_type) gt 0 then cfg.filter_type = filter_type
|
||||||
|
if n_elements(filter_window) gt 0 then cfg.filter_window = double(filter_window)
|
||||||
|
if n_elements(geocode_grid_x) gt 0 then cfg.geocode_grid_x = double(geocode_grid_x)
|
||||||
|
if n_elements(geocode_grid_y) gt 0 then cfg.geocode_grid_y = double(geocode_grid_y)
|
||||||
|
if n_elements(imported_hh_root) gt 0 then cfg.imported_hh_root = imported_hh_root
|
||||||
|
if n_elements(imported_hv_root) gt 0 then cfg.imported_hv_root = imported_hv_root
|
||||||
|
|
||||||
|
cfg.import_root = cfg.out_dir + '\gf3_import'
|
||||||
|
cfg.output_prefix = gf3_scene_token(cfg.input_meta_xml)
|
||||||
|
|
||||||
|
if n_elements(work_dir) eq 0 then cfg.work_dir = cfg.out_dir + '\work'
|
||||||
|
if n_elements(temp_dir) eq 0 then cfg.temp_dir = cfg.out_dir + '\temp'
|
||||||
|
|
||||||
|
if n_elements(imported_hh_root) eq 0 then cfg.imported_hh_root = ''
|
||||||
|
if n_elements(imported_hv_root) eq 0 then cfg.imported_hv_root = ''
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_prepare_dirs
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
|
||||||
|
file_mkdir, cfg.work_dir
|
||||||
|
file_mkdir, cfg.temp_dir
|
||||||
|
file_mkdir, cfg.out_dir
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_start_batch
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
SARscape_Batch_Init, Temp_Directory=cfg.temp_dir
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_stop_batch
|
||||||
|
compile_opt idl2
|
||||||
|
SARscape_Batch_Exit
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_run_import, errmsg=errmsg
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
|
||||||
|
errmsg = ''
|
||||||
|
gf3_run_module, 'ImportGaofen3', 'gf3_pipeline_set_import_params', errmsg=errmsg, working_directory=cfg.work_dir
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_run_multilook, polarization, errmsg=errmsg
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
common gf3_pipeline_state, state
|
||||||
|
|
||||||
|
state = {current_input_root: gf3_base_root(polarization), current_output_root: gf3_stage_root(polarization, 'multilook')}
|
||||||
|
errmsg = ''
|
||||||
|
gf3_run_module, 'BaseMultilooking', 'gf3_pipeline_set_multilook_params', errmsg=errmsg, working_directory=cfg.work_dir
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_run_filter, polarization, errmsg=errmsg
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
common gf3_pipeline_state, state
|
||||||
|
|
||||||
|
state = {current_input_root: gf3_stage_root(polarization, 'multilook'), current_output_root: gf3_stage_root(polarization, 'filter')}
|
||||||
|
errmsg = ''
|
||||||
|
gf3_run_module, 'DespeckleConventionalSingle', 'gf3_pipeline_set_filter_params', errmsg=errmsg, working_directory=cfg.work_dir
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_run_geocode, polarization, errmsg=errmsg
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
common gf3_pipeline_state, state
|
||||||
|
|
||||||
|
state = {current_input_root: gf3_stage_root(polarization, 'filter'), current_output_root: gf3_stage_root(polarization, 'geocode')}
|
||||||
|
errmsg = ''
|
||||||
|
gf3_run_module, 'BasicGeocoding', 'gf3_pipeline_set_geocode_params', errmsg=errmsg, working_directory=cfg.work_dir
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_run_single_pol_pipeline, polarization, do_import=do_import, errmsg=errmsg
|
||||||
|
compile_opt idl2
|
||||||
|
|
||||||
|
errmsg = ''
|
||||||
|
if keyword_set(do_import) then begin
|
||||||
|
gf3_run_import, errmsg=errmsg
|
||||||
|
if errmsg ne '' then return
|
||||||
|
gf3_refresh_import_roots
|
||||||
|
endif
|
||||||
|
|
||||||
|
gf3_run_multilook, polarization, errmsg=errmsg
|
||||||
|
if errmsg ne '' then return
|
||||||
|
|
||||||
|
gf3_run_filter, polarization, errmsg=errmsg
|
||||||
|
if errmsg ne '' then return
|
||||||
|
|
||||||
|
gf3_run_geocode, polarization, errmsg=errmsg
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
function gf3_should_run_pol, polarizations, polarization
|
||||||
|
compile_opt idl2
|
||||||
|
|
||||||
|
if n_elements(polarizations) eq 0 then return, 1
|
||||||
|
|
||||||
|
wanted = strupcase(strtrim(polarizations, 2))
|
||||||
|
pol = strupcase(strtrim(polarization, 2))
|
||||||
|
|
||||||
|
for i = 0, n_elements(wanted) - 1 do begin
|
||||||
|
if wanted[i] eq 'ALL' then return, 1
|
||||||
|
if wanted[i] eq pol then return, 1
|
||||||
|
endfor
|
||||||
|
|
||||||
|
return, 0
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_sarscape_run, input_meta_xml=input_meta_xml, out_dir=out_dir, dem_file=dem_file, $
|
||||||
|
polarizations=polarizations, skip_import=skip_import, work_dir=work_dir, temp_dir=temp_dir, $
|
||||||
|
range_looks=range_looks, azimuth_looks=azimuth_looks, filter_type=filter_type, $
|
||||||
|
filter_window=filter_window, geocode_grid_x=geocode_grid_x, geocode_grid_y=geocode_grid_y
|
||||||
|
|
||||||
|
compile_opt idl2
|
||||||
|
|
||||||
|
gf3_configure, input_meta_xml=input_meta_xml, out_dir=out_dir, dem_file=dem_file, $
|
||||||
|
work_dir=work_dir, temp_dir=temp_dir, range_looks=range_looks, $
|
||||||
|
azimuth_looks=azimuth_looks, filter_type=filter_type, filter_window=filter_window, $
|
||||||
|
geocode_grid_x=geocode_grid_x, geocode_grid_y=geocode_grid_y
|
||||||
|
|
||||||
|
gf3_prepare_dirs
|
||||||
|
gf3_start_batch
|
||||||
|
|
||||||
|
errmsg = ''
|
||||||
|
|
||||||
|
if ~keyword_set(skip_import) then begin
|
||||||
|
gf3_run_import, errmsg=errmsg
|
||||||
|
if errmsg eq '' then gf3_refresh_import_roots
|
||||||
|
endif else begin
|
||||||
|
gf3_refresh_import_roots
|
||||||
|
endelse
|
||||||
|
|
||||||
|
if errmsg eq '' && gf3_should_run_pol(polarizations, 'HH') then gf3_run_single_pol_pipeline, 'HH', errmsg=errmsg
|
||||||
|
if errmsg eq '' && gf3_should_run_pol(polarizations, 'HV') then gf3_run_single_pol_pipeline, 'HV', errmsg=errmsg
|
||||||
|
|
||||||
|
gf3_stop_batch
|
||||||
|
if errmsg ne '' then print, 'Pipeline stopped: ' + errmsg
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_sarscape_pipeline
|
||||||
|
compile_opt idl2
|
||||||
|
common gf3_pipeline_cfg, cfg
|
||||||
|
|
||||||
|
gf3_init_default_cfg
|
||||||
|
gf3_prepare_dirs
|
||||||
|
gf3_start_batch
|
||||||
|
|
||||||
|
errmsg = ''
|
||||||
|
gf3_run_import, errmsg=errmsg
|
||||||
|
if errmsg eq '' then gf3_refresh_import_roots
|
||||||
|
if errmsg eq '' then gf3_run_single_pol_pipeline, 'HH', errmsg=errmsg
|
||||||
|
if errmsg eq '' then gf3_run_single_pol_pipeline, 'HV', errmsg=errmsg
|
||||||
|
|
||||||
|
gf3_stop_batch
|
||||||
|
if errmsg ne '' then print, 'Pipeline stopped: ' + errmsg
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_sarscape_import_only
|
||||||
|
compile_opt idl2
|
||||||
|
|
||||||
|
gf3_init_default_cfg
|
||||||
|
gf3_prepare_dirs
|
||||||
|
gf3_start_batch
|
||||||
|
errmsg = ''
|
||||||
|
gf3_run_import, errmsg=errmsg
|
||||||
|
gf3_stop_batch
|
||||||
|
if errmsg ne '' then print, 'Import stopped: ' + errmsg
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_sarscape_hh_only
|
||||||
|
compile_opt idl2
|
||||||
|
|
||||||
|
gf3_init_default_cfg
|
||||||
|
gf3_prepare_dirs
|
||||||
|
gf3_start_batch
|
||||||
|
errmsg = ''
|
||||||
|
gf3_run_single_pol_pipeline, 'HH', do_import=1, errmsg=errmsg
|
||||||
|
gf3_stop_batch
|
||||||
|
if errmsg ne '' then print, 'HH pipeline stopped: ' + errmsg
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
pro gf3_sarscape_hv_only
|
||||||
|
compile_opt idl2
|
||||||
|
|
||||||
|
gf3_init_default_cfg
|
||||||
|
gf3_prepare_dirs
|
||||||
|
gf3_start_batch
|
||||||
|
errmsg = ''
|
||||||
|
gf3_run_single_pol_pipeline, 'HV', do_import=1, errmsg=errmsg
|
||||||
|
gf3_stop_batch
|
||||||
|
if errmsg ne '' then print, 'HV pipeline stopped: ' + errmsg
|
||||||
|
end
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
@echo off
|
||||||
|
set IDLRT=C:\Program Files\Harris\ENVI56\IDL88\bin\bin.x86_64\idlrt.exe
|
||||||
|
set SAV=C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\gf3_sarscape_cli.sav
|
||||||
|
set META=C:\Users\Administrator\Desktop\GF3_Test\GF3_scene_20260513\GF3_SYC_FSI_051370_E124.6_N51.7_20260513_L1A_HHHV_L10007355955.meta.xml
|
||||||
|
set OUT=C:\Users\Administrator\Desktop\GF3_Test\GF3_cli_test_out_20260513
|
||||||
|
set DEM=D:\DEM\COPDEM_GLO30_China_4326_DEM
|
||||||
|
|
||||||
|
"%IDLRT%" "%SAV%" -args "%META%" "%OUT%" "%DEM%" "HH,HV"
|
||||||
@@ -0,0 +1,594 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"archive/tar"
|
||||||
|
"bufio"
|
||||||
|
"compress/gzip"
|
||||||
|
"embed"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"io/fs"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
//go:embed assets/gf3_sarscape_cli.sav
|
||||||
|
var embedded embed.FS
|
||||||
|
|
||||||
|
type config struct {
|
||||||
|
configPath string `json:"-"`
|
||||||
|
input string `json:"input"`
|
||||||
|
output string `json:"output"`
|
||||||
|
dem string `json:"dem"`
|
||||||
|
polarizations string `json:"polarizations"`
|
||||||
|
idlrt string `json:"idlrt"`
|
||||||
|
keepExtracted bool `json:"keep_extracted"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type scene struct {
|
||||||
|
metaPath string
|
||||||
|
archivePath string
|
||||||
|
name string
|
||||||
|
}
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
cfg := parseFlags()
|
||||||
|
if len(os.Args) == 1 {
|
||||||
|
var err error
|
||||||
|
cfg, err = promptConfig(cfg)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Printf("ERROR: %v\n", err)
|
||||||
|
waitForEnter()
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := run(cfg); err != nil {
|
||||||
|
fmt.Printf("ERROR: %v\n", err)
|
||||||
|
if len(os.Args) == 1 {
|
||||||
|
waitForEnter()
|
||||||
|
}
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
if err := saveConfig(cfg.configPath, cfg); err != nil {
|
||||||
|
fmt.Printf("WARNING: save config: %v\n", err)
|
||||||
|
}
|
||||||
|
if len(os.Args) == 1 {
|
||||||
|
fmt.Println("Done.")
|
||||||
|
waitForEnter()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseFlags() config {
|
||||||
|
cfg := defaultConfig()
|
||||||
|
configPath := preScanConfigPath()
|
||||||
|
cfg.configPath = configPath
|
||||||
|
_ = loadConfig(configPath, &cfg)
|
||||||
|
|
||||||
|
flag.StringVar(&cfg.configPath, "config", configPath, "Config file path")
|
||||||
|
flag.StringVar(&cfg.input, "input", cfg.input, "GF-3 input: .meta.xml, .tar.gz, or directory")
|
||||||
|
flag.StringVar(&cfg.output, "output", cfg.output, "Output directory")
|
||||||
|
flag.StringVar(&cfg.dem, "dem", cfg.dem, "SARscape DEM file")
|
||||||
|
flag.StringVar(&cfg.polarizations, "pol", cfg.polarizations, "Polarizations: HH, HV, or HH,HV")
|
||||||
|
flag.StringVar(&cfg.idlrt, "idlrt", cfg.idlrt, "Path to idlrt.exe")
|
||||||
|
flag.BoolVar(&cfg.keepExtracted, "keep-extracted", cfg.keepExtracted, "Keep extracted archives under output/.gf3_extract")
|
||||||
|
flag.Parse()
|
||||||
|
return cfg
|
||||||
|
}
|
||||||
|
|
||||||
|
func defaultConfig() config {
|
||||||
|
return config{
|
||||||
|
dem: `D:\DEM\COPDEM_GLO30_China_4326_DEM`,
|
||||||
|
polarizations: "HH,HV",
|
||||||
|
idlrt: defaultIDLRT(),
|
||||||
|
keepExtracted: true,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func preScanConfigPath() string {
|
||||||
|
defaultPath := defaultConfigPath()
|
||||||
|
for i := 1; i < len(os.Args)-1; i++ {
|
||||||
|
if os.Args[i] == "-config" {
|
||||||
|
return os.Args[i+1]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return defaultPath
|
||||||
|
}
|
||||||
|
|
||||||
|
func defaultConfigPath() string {
|
||||||
|
exe, err := os.Executable()
|
||||||
|
if err != nil {
|
||||||
|
return "gf3wrapper.json"
|
||||||
|
}
|
||||||
|
return filepath.Join(filepath.Dir(exe), "gf3wrapper.json")
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadConfig(path string, cfg *config) error {
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var stored struct {
|
||||||
|
IDLRTPath string `json:"idlrt_path"`
|
||||||
|
DemFile string `json:"dem_file"`
|
||||||
|
Polarizations string `json:"polarizations"`
|
||||||
|
LastInput string `json:"last_input"`
|
||||||
|
LastOutput string `json:"last_output"`
|
||||||
|
IDLRT string `json:"idlrt"`
|
||||||
|
Dem string `json:"dem"`
|
||||||
|
Input string `json:"input"`
|
||||||
|
Output string `json:"output"`
|
||||||
|
KeepExtracted *bool `json:"keep_extracted"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(data, &stored); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if stored.IDLRTPath != "" {
|
||||||
|
cfg.idlrt = stored.IDLRTPath
|
||||||
|
} else if stored.IDLRT != "" {
|
||||||
|
cfg.idlrt = stored.IDLRT
|
||||||
|
}
|
||||||
|
if stored.DemFile != "" {
|
||||||
|
cfg.dem = stored.DemFile
|
||||||
|
} else if stored.Dem != "" {
|
||||||
|
cfg.dem = stored.Dem
|
||||||
|
}
|
||||||
|
if stored.Polarizations != "" {
|
||||||
|
cfg.polarizations = stored.Polarizations
|
||||||
|
}
|
||||||
|
if stored.LastInput != "" {
|
||||||
|
cfg.input = stored.LastInput
|
||||||
|
} else if stored.Input != "" {
|
||||||
|
cfg.input = stored.Input
|
||||||
|
}
|
||||||
|
if stored.LastOutput != "" {
|
||||||
|
cfg.output = stored.LastOutput
|
||||||
|
} else if stored.Output != "" {
|
||||||
|
cfg.output = stored.Output
|
||||||
|
}
|
||||||
|
if stored.KeepExtracted != nil {
|
||||||
|
cfg.keepExtracted = *stored.KeepExtracted
|
||||||
|
}
|
||||||
|
cfg.configPath = path
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func saveConfig(path string, cfg config) error {
|
||||||
|
if path == "" {
|
||||||
|
path = defaultConfigPath()
|
||||||
|
}
|
||||||
|
stored := struct {
|
||||||
|
IDLRTPath string `json:"idlrt_path"`
|
||||||
|
DemFile string `json:"dem_file"`
|
||||||
|
Polarizations string `json:"polarizations"`
|
||||||
|
LastInput string `json:"last_input"`
|
||||||
|
LastOutput string `json:"last_output"`
|
||||||
|
KeepExtracted bool `json:"keep_extracted"`
|
||||||
|
}{
|
||||||
|
IDLRTPath: cfg.idlrt,
|
||||||
|
DemFile: cfg.dem,
|
||||||
|
Polarizations: cfg.polarizations,
|
||||||
|
LastInput: cfg.input,
|
||||||
|
LastOutput: cfg.output,
|
||||||
|
KeepExtracted: cfg.keepExtracted,
|
||||||
|
}
|
||||||
|
data, err := json.MarshalIndent(stored, "", " ")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return os.WriteFile(path, data, 0o644)
|
||||||
|
}
|
||||||
|
|
||||||
|
func promptConfig(defaults config) (config, error) {
|
||||||
|
reader := bufio.NewReader(os.Stdin)
|
||||||
|
cfg := defaults
|
||||||
|
|
||||||
|
fmt.Println("GF-3 SARscape Wrapper")
|
||||||
|
fmt.Println()
|
||||||
|
|
||||||
|
cfg.input = prompt(reader, "Input .tar.gz / .meta.xml / folder", cfg.input)
|
||||||
|
cfg.output = prompt(reader, "Output folder", cfg.output)
|
||||||
|
cfg.dem = prompt(reader, "DEM file", cfg.dem)
|
||||||
|
cfg.polarizations = prompt(reader, "Polarizations", cfg.polarizations)
|
||||||
|
cfg.idlrt = prompt(reader, "IDL Runtime", cfg.idlrt)
|
||||||
|
|
||||||
|
if cfg.input == "" || cfg.output == "" || cfg.dem == "" {
|
||||||
|
return cfg, errors.New("input, output, and DEM are required")
|
||||||
|
}
|
||||||
|
return cfg, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func prompt(reader *bufio.Reader, label, defaultValue string) string {
|
||||||
|
if defaultValue != "" {
|
||||||
|
fmt.Printf("%s [%s]: ", label, defaultValue)
|
||||||
|
} else {
|
||||||
|
fmt.Printf("%s: ", label)
|
||||||
|
}
|
||||||
|
text, _ := reader.ReadString('\n')
|
||||||
|
text = strings.TrimSpace(text)
|
||||||
|
if text == "" {
|
||||||
|
return defaultValue
|
||||||
|
}
|
||||||
|
return text
|
||||||
|
}
|
||||||
|
|
||||||
|
func waitForEnter() {
|
||||||
|
fmt.Println()
|
||||||
|
fmt.Print("Press Enter to exit...")
|
||||||
|
_, _ = bufio.NewReader(os.Stdin).ReadString('\n')
|
||||||
|
}
|
||||||
|
|
||||||
|
func run(cfg config) error {
|
||||||
|
if cfg.input == "" || cfg.output == "" || cfg.dem == "" {
|
||||||
|
return errors.New("required flags: -input, -output, -dem")
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(cfg.idlrt); err != nil {
|
||||||
|
return fmt.Errorf("idlrt not found: %s: %w", cfg.idlrt, err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(cfg.dem); err != nil {
|
||||||
|
return fmt.Errorf("DEM not found: %s: %w", cfg.dem, err)
|
||||||
|
}
|
||||||
|
if err := os.MkdirAll(cfg.output, 0o755); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
savPath, err := materializeSAV(cfg.output)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
scenes, err := discoverScenes(cfg)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if len(scenes) == 0 {
|
||||||
|
return fmt.Errorf("no *.meta.xml scenes found in %s", cfg.input)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, sc := range scenes {
|
||||||
|
sceneOut := filepath.Join(cfg.output, sc.name)
|
||||||
|
if err := os.MkdirAll(sceneOut, 0o755); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if sc.archivePath != "" {
|
||||||
|
extractDir := filepath.Join(sceneOut, ".gf3_extract")
|
||||||
|
fmt.Printf("Extracting %s\n", sc.archivePath)
|
||||||
|
if err := extractTarGz(sc.archivePath, extractDir); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
extractedScenes, err := findMetaXML(extractDir)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if len(extractedScenes) == 0 {
|
||||||
|
return fmt.Errorf("no *.meta.xml found in archive: %s", sc.archivePath)
|
||||||
|
}
|
||||||
|
sc.metaPath = extractedScenes[0].metaPath
|
||||||
|
sc.name = sceneName(sc.metaPath)
|
||||||
|
}
|
||||||
|
if err := runScene(cfg, savPath, sc, sceneOut); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func materializeSAV(outputDir string) (string, error) {
|
||||||
|
data, err := embedded.ReadFile("assets/gf3_sarscape_cli.sav")
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
runtimeDir := filepath.Join(outputDir, ".gf3_runtime")
|
||||||
|
if err := os.MkdirAll(runtimeDir, 0o755); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
savPath := filepath.Join(runtimeDir, "gf3_sarscape_cli.sav")
|
||||||
|
if err := os.WriteFile(savPath, data, 0o644); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return savPath, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func discoverScenes(cfg config) ([]scene, error) {
|
||||||
|
info, err := os.Stat(cfg.input)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if info.IsDir() {
|
||||||
|
return discoverScenesInDir(cfg.input, cfg.output)
|
||||||
|
}
|
||||||
|
lower := strings.ToLower(cfg.input)
|
||||||
|
if strings.HasSuffix(lower, ".meta.xml") {
|
||||||
|
return []scene{{metaPath: cfg.input, name: sceneName(cfg.input)}}, nil
|
||||||
|
}
|
||||||
|
if strings.HasSuffix(lower, ".tar.gz") || strings.HasSuffix(lower, ".tgz") {
|
||||||
|
return []scene{{archivePath: cfg.input, name: safeName(trimArchiveExt(filepath.Base(cfg.input)))}}, nil
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("unsupported input type: %s", cfg.input)
|
||||||
|
}
|
||||||
|
|
||||||
|
func discoverScenesInDir(inputDir, outputDir string) ([]scene, error) {
|
||||||
|
_ = outputDir
|
||||||
|
scenesByName := make(map[string]scene)
|
||||||
|
|
||||||
|
err := filepath.WalkDir(inputDir, func(path string, d fs.DirEntry, err error) error {
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if d.IsDir() {
|
||||||
|
if d.Name() == ".gf3_extract" {
|
||||||
|
return filepath.SkipDir
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
lower := strings.ToLower(d.Name())
|
||||||
|
switch {
|
||||||
|
case strings.HasSuffix(lower, ".meta.xml"):
|
||||||
|
sc := scene{metaPath: path, name: sceneName(path)}
|
||||||
|
key := strings.ToLower(sc.name)
|
||||||
|
if _, ok := scenesByName[key]; !ok {
|
||||||
|
scenesByName[key] = sc
|
||||||
|
}
|
||||||
|
case strings.HasSuffix(lower, ".tar.gz") || strings.HasSuffix(lower, ".tgz"):
|
||||||
|
sc := scene{archivePath: path, name: safeName(trimArchiveExt(filepath.Base(path)))}
|
||||||
|
scenesByName[strings.ToLower(sc.name)] = sc
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
scenes := make([]scene, 0, len(scenesByName))
|
||||||
|
for _, sc := range scenesByName {
|
||||||
|
scenes = append(scenes, sc)
|
||||||
|
}
|
||||||
|
sort.Slice(scenes, func(i, j int) bool {
|
||||||
|
return strings.ToLower(scenes[i].name) < strings.ToLower(scenes[j].name)
|
||||||
|
})
|
||||||
|
return scenes, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func findMetaXML(root string) ([]scene, error) {
|
||||||
|
var scenes []scene
|
||||||
|
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if d.IsDir() {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if strings.HasSuffix(strings.ToLower(d.Name()), ".meta.xml") {
|
||||||
|
scenes = append(scenes, scene{metaPath: path, name: sceneName(path)})
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
return scenes, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func uniqueScenes(scenes []scene) []scene {
|
||||||
|
seen := make(map[string]bool, len(scenes))
|
||||||
|
out := make([]scene, 0, len(scenes))
|
||||||
|
for _, sc := range scenes {
|
||||||
|
key := filepath.Clean(sc.metaPath)
|
||||||
|
if seen[key] {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
seen[key] = true
|
||||||
|
out = append(out, sc)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func runScene(cfg config, savPath string, sc scene, sceneOut string) error {
|
||||||
|
fmt.Printf("Processing %s\n", sc.metaPath)
|
||||||
|
fmt.Printf("Output %s\n", sceneOut)
|
||||||
|
|
||||||
|
cmd := exec.Command(cfg.idlrt, savPath, "-args", sc.metaPath, sceneOut, cfg.dem, cfg.polarizations)
|
||||||
|
cmd.Stdout = os.Stdout
|
||||||
|
cmd.Stderr = os.Stderr
|
||||||
|
cmd.Dir = sceneOut
|
||||||
|
|
||||||
|
if err := cmd.Start(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
done := make(chan error, 1)
|
||||||
|
go func() {
|
||||||
|
done <- cmd.Wait()
|
||||||
|
}()
|
||||||
|
|
||||||
|
ticker := time.NewTicker(2 * time.Second)
|
||||||
|
defer ticker.Stop()
|
||||||
|
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case err := <-done:
|
||||||
|
fmt.Printf("\r%-100s\r", "")
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("scene failed %s: %w", sc.metaPath, err)
|
||||||
|
}
|
||||||
|
fmt.Println("Completed", sc.name)
|
||||||
|
return nil
|
||||||
|
case <-ticker.C:
|
||||||
|
fmt.Printf("\r%s", progressLine(sceneOut, sc.name, cfg.polarizations))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func progressLine(sceneOut, sceneNameValue, polarizations string) string {
|
||||||
|
stage, done, total := progressStage(sceneOut, sceneNameValue, polarizations)
|
||||||
|
width := 28
|
||||||
|
filled := 0
|
||||||
|
if total > 0 {
|
||||||
|
filled = done * width / total
|
||||||
|
}
|
||||||
|
if filled > width {
|
||||||
|
filled = width
|
||||||
|
}
|
||||||
|
bar := strings.Repeat("#", filled) + strings.Repeat("-", width-filled)
|
||||||
|
return fmt.Sprintf("[%s] %2d/%2d %s", bar, done, total, stage)
|
||||||
|
}
|
||||||
|
|
||||||
|
func progressStage(sceneOut, sceneNameValue, polarizations string) (string, int, int) {
|
||||||
|
pols := requestedPolarizations(polarizations)
|
||||||
|
checks := make([]struct {
|
||||||
|
label string
|
||||||
|
path string
|
||||||
|
}, 0, len(pols)*4)
|
||||||
|
|
||||||
|
for _, pol := range pols {
|
||||||
|
checks = append(checks, struct {
|
||||||
|
label string
|
||||||
|
path string
|
||||||
|
}{"import " + pol, filepath.Join(sceneOut, "*_"+pol+"_slc.sml")})
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, pol := range pols {
|
||||||
|
lower := strings.ToLower(pol)
|
||||||
|
prefix := sceneNameValue + "_" + lower
|
||||||
|
checks = append(checks,
|
||||||
|
struct {
|
||||||
|
label string
|
||||||
|
path string
|
||||||
|
}{"multilook " + pol, filepath.Join(sceneOut, prefix+"_ml.sml")},
|
||||||
|
struct {
|
||||||
|
label string
|
||||||
|
path string
|
||||||
|
}{"filter " + pol, filepath.Join(sceneOut, prefix+"_filt.sml")},
|
||||||
|
struct {
|
||||||
|
label string
|
||||||
|
path string
|
||||||
|
}{"geocode " + pol, filepath.Join(sceneOut, prefix+"_geo.sml")},
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
done := 0
|
||||||
|
for _, check := range checks {
|
||||||
|
if matched(check.path) {
|
||||||
|
done++
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
return check.label, done, len(checks)
|
||||||
|
}
|
||||||
|
return "finishing", done, len(checks)
|
||||||
|
}
|
||||||
|
|
||||||
|
func requestedPolarizations(value string) []string {
|
||||||
|
parts := strings.Split(value, ",")
|
||||||
|
out := make([]string, 0, len(parts))
|
||||||
|
for _, part := range parts {
|
||||||
|
pol := strings.ToUpper(strings.TrimSpace(part))
|
||||||
|
if pol == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if pol == "ALL" {
|
||||||
|
return []string{"HH", "HV"}
|
||||||
|
}
|
||||||
|
out = append(out, pol)
|
||||||
|
}
|
||||||
|
if len(out) == 0 {
|
||||||
|
return []string{"HH", "HV"}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func matched(pattern string) bool {
|
||||||
|
matches, err := filepath.Glob(pattern)
|
||||||
|
if err != nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return len(matches) > 0
|
||||||
|
}
|
||||||
|
|
||||||
|
func extractTarGz(src, dst string) error {
|
||||||
|
if err := os.MkdirAll(dst, 0o755); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
f, err := os.Open(src)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
|
||||||
|
gz, err := gzip.NewReader(f)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer gz.Close()
|
||||||
|
|
||||||
|
tr := tar.NewReader(gz)
|
||||||
|
for {
|
||||||
|
hdr, err := tr.Next()
|
||||||
|
if errors.Is(err, io.EOF) {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
target := filepath.Join(dst, filepath.Clean(hdr.Name))
|
||||||
|
if !strings.HasPrefix(target, filepath.Clean(dst)+string(os.PathSeparator)) && filepath.Clean(target) != filepath.Clean(dst) {
|
||||||
|
return fmt.Errorf("archive entry escapes destination: %s", hdr.Name)
|
||||||
|
}
|
||||||
|
switch hdr.Typeflag {
|
||||||
|
case tar.TypeDir:
|
||||||
|
if err := os.MkdirAll(target, 0o755); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
case tar.TypeReg:
|
||||||
|
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
out, err := os.OpenFile(target, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o644)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
_, copyErr := io.Copy(out, tr)
|
||||||
|
closeErr := out.Close()
|
||||||
|
if copyErr != nil {
|
||||||
|
return copyErr
|
||||||
|
}
|
||||||
|
if closeErr != nil {
|
||||||
|
return closeErr
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func sceneName(metaPath string) string {
|
||||||
|
base := filepath.Base(metaPath)
|
||||||
|
base = strings.TrimSuffix(base, ".meta.xml")
|
||||||
|
base = strings.TrimSuffix(base, ".META.XML")
|
||||||
|
return safeName(base)
|
||||||
|
}
|
||||||
|
|
||||||
|
func safeName(s string) string {
|
||||||
|
replacer := strings.NewReplacer(" ", "_", ":", "_", "/", "_", "\\", "_")
|
||||||
|
return replacer.Replace(s)
|
||||||
|
}
|
||||||
|
|
||||||
|
func trimArchiveExt(name string) string {
|
||||||
|
lower := strings.ToLower(name)
|
||||||
|
switch {
|
||||||
|
case strings.HasSuffix(lower, ".tar.gz"):
|
||||||
|
return name[:len(name)-7]
|
||||||
|
case strings.HasSuffix(lower, ".tgz"):
|
||||||
|
return name[:len(name)-4]
|
||||||
|
default:
|
||||||
|
return strings.TrimSuffix(name, filepath.Ext(name))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func defaultIDLRT() string {
|
||||||
|
if v := os.Getenv("IDLRT_PATH"); v != "" {
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
return `C:\Program Files\Harris\ENVI56\IDL88\bin\bin.x86_64\idlrt.exe`
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user