Add SARscape Go wrapper workflow

This commit is contained in:
GF3 Pipeline Bot
2026-05-29 19:52:32 +08:00
parent 46474274c2
commit 5c24c0e64c
16 changed files with 1517 additions and 22 deletions
+16
View File
@@ -0,0 +1,16 @@
* text=auto
.gitattributes text eol=lf
.gitignore text eol=lf
*.md text eol=lf
*.py text eol=lf
*.go text eol=lf
*.mod text eol=lf
*.toml text eol=lf
*.yml text eol=lf
*.idl text eol=lf
*.pro text eol=lf
*.cmd text eol=crlf
*.exe binary
*.sav binary
+18
View File
@@ -7,3 +7,21 @@ __pycache__/
/.tmp_tests/ /.tmp_tests/
logs/ logs/
GMTED2010.jp2 GMTED2010.jp2
# Local wrapper state
gf3wrapper.json
.gf3_extract/
.gf3_runtime/
# Production data and generated outputs
/data/
/outputs/
*.tar.gz
*.tgz
*.zip
*.tif
*.tiff
*.img
*.dat
*.sml
*.hdr
+62 -22
View File
@@ -1,17 +1,50 @@
# GF3 L1A to L2 Pipeline # GF3 L1A to L2 Pipeline
This repository contains a Python package and CLI for converting GF3 L1A This repository contains two GF-3 L1A to L2 processing paths:
products to L2 GeoTIFF outputs.
The heavy work is still handled by GDAL, Rasterio, and NumPy. The package adds - **SARscape Go wrapper**: Windows-oriented production wrapper around ENVI/IDL Runtime and SARscape. This is the recommended path when SARscape is available.
pipeline structure around the original script: explicit configuration, safe path - **Python pipeline**: GDAL/Rasterio/NumPy implementation kept as a pure Python package and CLI.
handling, logging, restart-friendly skipping, per-scene error isolation, and
block-based radiometric calibration.
## Install The SARscape wrapper currently has the more complete operational workflow for GF-3 scene import, multilook/filter/geocode processing, batch directory discovery, saved defaults, and progress reporting.
Use an environment that already has a working GDAL/OSGeo Python binding. On ## Repository Layout
Windows, Conda/Miniforge is usually the least fragile route.
```text
dist/windows/ Prebuilt Windows executable
docs/ Operational documentation
src/gf3_l1a2l2/ Python package
tests/ Python unit tests and small fixtures
tools/gf3_sarscape_wrapper/ Go wrapper source and embedded SAV asset
tools/gf3_sarscape_wrapper/idl/ IDL/SARscape source scripts
```
Large GF-3 archives, extracted scenes, SARscape outputs, and local runtime files are intentionally not tracked in Git.
## Recommended: SARscape Go Wrapper
Use the prebuilt executable:
```powershell
.\dist\windows\gf3wrapper.exe `
-input "D:\GF3\L1A_BATCH" `
-output "E:\GF3\L2_SARscape" `
-dem "D:\DEM\COPDEM_GLO30_China_4326_DEM" `
-pol "HH,HV"
```
Supported input forms:
- A single `*.tar.gz` GF-3 L1A package.
- A single `*.meta.xml` scene metadata file.
- A directory containing multiple `*.tar.gz` or `*.meta.xml` scenes.
The wrapper writes one output subdirectory per scene. On successful runs it saves defaults next to the executable in `gf3wrapper.json`, so double-click interactive mode can reuse the previous input/output/DEM/polarization settings.
More details: [docs/sarscape_go_wrapper.md](docs/sarscape_go_wrapper.md)
## Python Pipeline
Use an environment that already has working GDAL/OSGeo Python bindings. On Windows, Conda/Miniforge is usually the least fragile route.
```powershell ```powershell
conda env create -f environment.yml conda env create -f environment.yml
@@ -19,7 +52,7 @@ conda activate gf3-l1a2l2
pip install -e . --no-deps pip install -e . --no-deps
``` ```
## Run Run:
```powershell ```powershell
gf3-l1a2l2 run --input D:\GF3\L1A_BATCH --output E:\GF3\L2_Image --dem .\GMTED2010.jp2 gf3-l1a2l2 run --input D:\GF3\L1A_BATCH --output E:\GF3\L2_Image --dem .\GMTED2010.jp2
@@ -38,22 +71,29 @@ gf3-l1a2l2 run `
--dst-srs EPSG:4326 --dst-srs EPSG:4326
``` ```
By default, output files are written under one subdirectory per discovered Legacy wrappers remain available:
scene. Use `--flat-output` to write all outputs directly into the output
directory.
The legacy files are now compatibility wrappers:
```powershell ```powershell
python gf3_L1A_To_L2.py run --input D:\GF3\L1A_BATCH --output E:\GF3\L2_Image --dem .\GMTED2010.jp2 python gf3_L1A_To_L2.py run --input D:\GF3\L1A_BATCH --output E:\GF3\L2_Image --dem .\GMTED2010.jp2
python Decompression.py D:\GF3\L1A_BATCH python Decompression.py D:\GF3\L1A_BATCH
``` ```
## Notes ## Development Checks
- The CLI extracts `.tar.gz` and `.zip` products before processing unless Python:
`--skip-decompress` is used.
- Existing L2 outputs are skipped unless `--overwrite` is used. ```powershell
- Logs are written to `<output>\logs`. python -m pytest
- Without representative GF3 products, verification is limited to parser unit ```
tests and Python syntax/import checks.
Go wrapper:
```powershell
cd .\tools\gf3_sarscape_wrapper
$env:GOTELEMETRY = "off"
go build -o ..\..\dist\windows\gf3wrapper.exe .
```
## Data Policy
Do not commit production GF-3 archives or generated products. Keep large data on NAS/storage and document the path in run notes. Only small sanitized fixtures should be added under `tests/fixtures`.
+17
View File
@@ -0,0 +1,17 @@
# Windows Distribution
`gf3wrapper.exe` is the prebuilt SARscape Go wrapper.
Run from the repository root:
```powershell
.\dist\windows\gf3wrapper.exe `
-input "D:\GF3\L1A_BATCH" `
-output "E:\GF3\L2_SARscape" `
-dem "D:\DEM\COPDEM_GLO30_China_4326_DEM" `
-pol "HH,HV"
```
The executable requires ENVI/IDL Runtime and SARscape on the target machine.
After successful runs, `gf3wrapper.json` is created in this directory to store defaults. That file is local runtime state and is ignored by Git.
BIN
View File
Binary file not shown.
+45
View File
@@ -0,0 +1,45 @@
# Repository Layout
```text
.
|-- dist/
| `-- windows/
| `-- gf3wrapper.exe
|-- docs/
| |-- repository_layout.md
| `-- sarscape_go_wrapper.md
|-- src/
| `-- gf3_l1a2l2/
|-- tests/
| `-- fixtures/
|-- tools/
| `-- gf3_sarscape_wrapper/
| |-- assets/
| |-- idl/
| |-- go.mod
| `-- main.go
|-- README.md
|-- environment.yml
`-- pyproject.toml
```
## What Belongs In Git
- Python package source.
- Go wrapper source.
- IDL/SARscape source scripts.
- Small embedded runtime assets needed to build the wrapper.
- Small sanitized tests and fixtures.
- Documentation and reproducible build instructions.
- Small convenience binaries when the team explicitly wants a ready-to-run Windows artifact.
## What Does Not Belong In Git
- Raw GF-3 production archives.
- Extracted scenes.
- SARscape output directories.
- Local wrapper config files such as `gf3wrapper.json`.
- Runtime folders such as `.gf3_extract` and `.gf3_runtime`.
- DEM files and other large environment-specific dependencies.
Use NAS or object storage for production data and record exact paths in run logs or issue notes.
+112
View File
@@ -0,0 +1,112 @@
# SARscape Go Wrapper
`gf3wrapper.exe` is a Windows command-line wrapper for the SARscape/IDL GF-3 processing workflow. It embeds `gf3_sarscape_cli.sav`, starts `idlrt.exe`, and passes scene metadata, output directory, DEM, and polarization arguments to SARscape.
This path is intended for operational runs where ENVI/IDL Runtime and SARscape are installed and licensed.
## Requirements
- Windows host.
- ENVI/IDL Runtime, default path:
`C:\Program Files\Harris\ENVI56\IDL88\bin\bin.x86_64\idlrt.exe`
- SARscape available to the IDL Runtime environment.
- DEM file, for example:
`D:\DEM\COPDEM_GLO30_China_4326_DEM`
- GF-3 L1A input as `*.tar.gz`, `*.meta.xml`, or a directory containing multiple scenes.
The IDL Runtime path can be overridden with `-idlrt` or the `IDLRT_PATH` environment variable.
## Quick Start
Use the prebuilt executable:
```powershell
.\dist\windows\gf3wrapper.exe `
-input "D:\GF3\L1A_BATCH" `
-output "E:\GF3\L2_SARscape" `
-dem "D:\DEM\COPDEM_GLO30_China_4326_DEM" `
-pol "HH,HV"
```
Double-click mode is also supported. When launched without arguments, the wrapper prompts for input, output, DEM, polarizations, and IDL Runtime path. Press Enter to reuse saved defaults.
## Inputs
The wrapper supports:
- A single GF-3 archive: `GF3_....tar.gz`
- A single extracted scene metadata file: `GF3_....meta.xml`
- A directory containing many archives and/or extracted scenes
Directory mode recursively scans for `*.tar.gz`, `*.tgz`, and `*.meta.xml`. If both archive and metadata exist for the same scene name, the archive is preferred. Internal `.gf3_extract` folders are skipped.
## Outputs
Each scene is processed into its own output directory:
```text
E:\GF3\L2_SARscape\
GF3_SCENE_NAME\
.gf3_extract\ Extracted archive, when input was .tar.gz
*_slc.sml
*_ml.sml
*_filt.sml
*_geo.sml
```
The embedded SAV is materialized under:
```text
<output>\.gf3_runtime\gf3_sarscape_cli.sav
```
## Saved Defaults
After a successful run, the wrapper writes `gf3wrapper.json` next to the executable:
```json
{
"idlrt_path": "C:\\Program Files\\Harris\\ENVI56\\IDL88\\bin\\bin.x86_64\\idlrt.exe",
"dem_file": "D:\\DEM\\COPDEM_GLO30_China_4326_DEM",
"polarizations": "HH,HV",
"last_input": "D:\\GF3\\L1A_BATCH",
"last_output": "E:\\GF3\\L2_SARscape",
"keep_extracted": true
}
```
Command-line flags override saved defaults. If `-input`, `-output`, `-dem`, or `-pol` are omitted, saved values are reused when available.
## Build
From the repository root:
```powershell
cd .\tools\gf3_sarscape_wrapper
$env:GOTELEMETRY = "off"
go build -o ..\..\dist\windows\gf3wrapper.exe .
```
If Go cannot write to the default user cache directory, set local cache paths:
```powershell
$env:GOCACHE = "C:\Temp\go-build-cache"
$env:APPDATA = "C:\Temp\go-appdata"
```
## Source Files
```text
tools/gf3_sarscape_wrapper/main.go
tools/gf3_sarscape_wrapper/assets/gf3_sarscape_cli.sav
tools/gf3_sarscape_wrapper/idl/gf3_sarscape_cli.pro
tools/gf3_sarscape_wrapper/idl/gf3_sarscape_pipeline.pro
```
`assets/gf3_sarscape_cli.sav` is embedded into the Go executable at build time.
## Notes
- Keep production archives and generated products outside Git.
- Keep only small sanitized fixtures in `tests/fixtures`.
- The prebuilt executable in `dist/windows` is a convenience artifact for Windows users. Rebuild it after changing `main.go` or the embedded `.sav`.
+47
View File
@@ -0,0 +1,47 @@
# GF-3 SARscape Go Wrapper
This wrapper embeds `assets/gf3_sarscape_cli.sav` and calls the local ENVI/IDL Runtime:
```text
idlrt.exe gf3_sarscape_cli.sav -args input_meta_xml out_dir dem_file HH,HV
```
It is the operational Windows path for SARscape-based GF-3 processing.
## Build
From this directory:
```powershell
$env:GOTELEMETRY = "off"
go build -o ..\..\dist\windows\gf3wrapper.exe .
```
## Run
Single archive:
```powershell
..\..\dist\windows\gf3wrapper.exe `
-input "D:\GF3\GF3_SCENE.tar.gz" `
-output "E:\GF3\L2_SARscape" `
-dem "D:\DEM\COPDEM_GLO30_China_4326_DEM" `
-pol "HH,HV"
```
Batch directory:
```powershell
..\..\dist\windows\gf3wrapper.exe `
-input "D:\GF3\L1A_BATCH" `
-output "E:\GF3\L2_SARscape"
```
If `gf3wrapper.json` exists next to the executable, omitted `-input`, `-output`, `-dem`, and `-pol` values are loaded from that file.
## Notes
- The server still needs ENVI, IDL Runtime, and SARscape installed and licensed.
- `.tar.gz` inputs are extracted under each scene output directory in `.gf3_extract`.
- The embedded SAV is extracted to `<output>\.gf3_runtime\gf3_sarscape_cli.sav`.
- More operational details are in `docs/sarscape_go_wrapper.md`.
+3
View File
@@ -0,0 +1,3 @@
module gf3wrapper
go 1.22
@@ -0,0 +1,5 @@
.compile 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\gf3_sarscape_pipeline.pro'
.compile 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\gf3_sarscape_cli.pro'
resolve_all, skip_routines='envi', /continue_on_error
save, /routines, filename='C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\gf3_sarscape_cli.sav'
exit
@@ -0,0 +1,5 @@
.f
.compile 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\gf3_sarscape_pipeline.pro'
.compile 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\gf3_sarscape_cli.pro'
save, /routines, filename='C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\gf3_sarscape_cli.sav'
exit
@@ -0,0 +1,79 @@
function gf3_cli_split_csv, value
compile_opt idl2
text = strtrim(value, 2)
if text eq '' then return, ['HH', 'HV']
parts = ['']
first = 1
while 1 do begin
pos = strpos(text, ',')
if pos lt 0 then begin
token = strtrim(text, 2)
if first then parts = [token] else parts = [parts, token]
break
endif
token = strtrim(strmid(text, 0, pos), 2)
if first then begin
parts = [token]
first = 0
endif else begin
parts = [parts, token]
endelse
text = strtrim(strmid(text, pos + 1), 2)
endwhile
return, parts
end
pro gf3_sarscape_cli
compile_opt idl2
args = command_line_args(count=argc)
if argc lt 3 then begin
print, 'Usage: idlrt.exe gf3_sarscape_cli.sav -args input_meta_xml out_dir dem_file [polarizations]'
print, 'Example polarizations: HH,HV or HH or HV'
return
endif
input_meta_xml = args[0]
out_dir = args[1]
dem_file = args[2]
file_mkdir, out_dir
log_file = out_dir + '\gf3_sarscape_cli.log'
openw, lun, log_file, /get_lun
catch, err
if err ne 0 then begin
printf, lun, 'ERROR: ', !error_state.msg
free_lun, lun
catch, /cancel
return
endif
printf, lun, 'CLI start'
printf, lun, 'input_meta_xml=', input_meta_xml
printf, lun, 'out_dir=', out_dir
printf, lun, 'dem_file=', dem_file
if argc ge 4 then begin
polarizations = gf3_cli_split_csv(args[3])
endif else begin
polarizations = ['HH', 'HV']
endelse
printf, lun, 'polarizations=', strjoin(polarizations, ',')
e = envi(/headless)
printf, lun, 'ENVI headless initialized'
gf3_sarscape_run, $
input_meta_xml=input_meta_xml, $
out_dir=out_dir, $
dem_file=dem_file, $
polarizations=polarizations
printf, lun, 'CLI end'
free_lun, lun
end
@@ -0,0 +1,506 @@
function gf3_normalize_errmsg, value
compile_opt idl2
if n_elements(value) eq 0 then return, ''
sz = size(value, /tname)
if sz eq 'STRING' then begin
if n_elements(value) eq 1 then return, value
return, strjoin(value, ' ')
endif
if sz eq 'BYTE' then begin
if n_elements(value) eq 0 then return, ''
return, string(value)
endif
return, string(value)
end
function gf3_float_str, value
compile_opt idl2
return, string(double(value), format='(F0.1)')
end
function gf3_base_root, polarization
compile_opt idl2
common gf3_pipeline_cfg, cfg
pol = strupcase(strtrim(polarization, 2))
case pol of
'HH': return, cfg.imported_hh_root
'HV': return, cfg.imported_hv_root
else: message, 'Unsupported polarization: ' + pol
endcase
end
function gf3_stage_root, polarization, stage_name
compile_opt idl2
common gf3_pipeline_cfg, cfg
pol = strlowcase(strtrim(polarization, 2))
prefix = cfg.output_prefix
if prefix eq '' then prefix = 'gf3'
case strlowcase(strtrim(stage_name, 2)) of
'multilook': return, cfg.out_dir + '\' + prefix + '_' + pol + '_ml'
'filter': return, cfg.out_dir + '\' + prefix + '_' + pol + '_filt'
'geocode': return, cfg.out_dir + '\' + prefix + '_' + pol + '_geo'
else: message, 'Unsupported stage: ' + stage_name
endcase
end
function gf3_scene_token, meta_xml
compile_opt idl2
base = file_basename(meta_xml)
pos = strpos(strlowcase(base), '.meta.xml')
if pos ge 0 then base = strmid(base, 0, pos)
return, base
end
function gf3_imported_root_for, polarization
compile_opt idl2
common gf3_pipeline_cfg, cfg
pol = strupcase(strtrim(polarization, 2))
pattern = cfg.out_dir + '\*_' + pol + '_slc.sml'
matches = file_search(pattern, count=count)
if count eq 0 then return, ''
newest = matches[0]
newest_time = (file_info(newest)).mtime
for i = 1, count - 1 do begin
info = file_info(matches[i])
if info.mtime gt newest_time then begin
newest = matches[i]
newest_time = info.mtime
endif
endfor
return, strmid(newest, 0, strlen(newest) - 4)
end
pro gf3_refresh_import_roots
compile_opt idl2
common gf3_pipeline_cfg, cfg
hh = gf3_imported_root_for('HH')
hv = gf3_imported_root_for('HV')
if hh ne '' then cfg.imported_hh_root = hh
if hv ne '' then cfg.imported_hv_root = hv
end
pro gf3_run_module, module_name, set_params_proc, errmsg=errmsg, working_directory=working_directory
compile_opt idl2
errmsg = ''
ob = obj_new('SARscapeBatch', Module=module_name)
if ~obj_valid(ob) then begin
errmsg = 'Create object failed: ' + module_name
print, errmsg
return
endif
ob->SetParam, 'GENERAL_PARAMETERS_CMD.AVAILABLE_MEMORY_SIZE_GB', '8.0000000'
ob->SetParam, 'GENERAL_PARAMETERS_CMD.LOAD_IMAGES', 'OK'
ob->SetParam, 'GENERAL_PARAMETERS_CMD.DELETE_TEMPORARY_FILES', 'OK'
ob->SetParam, 'GENERAL_PARAMETERS_CMD.MAKE_TIFF', 'OK'
ob->SetParam, 'GENERAL_PARAMETERS_CMD.QUICK_LOOK_FORMAT', 'ql_tiff'
ob->SetParam, 'GENERAL_PARAMETERS_CMD.SARSCAPE_TRACE_LEVEL', '3.0000000'
ob->SetParam, 'GENERAL_PARAMETERS_CMD.RENAME_THE_FILE_USING_PARAMETERS_FLAG', 'OK'
ob->SetParam, 'GENERAL_PARAMETERS_CMD.INSERT_GEO_POINTS_FLAG', 'NotOK'
ob->SetParam, 'GENERAL_PARAMETERS_CMD.VERBOSE_TRACE', 'OK'
ob->SetParam, 'GENERAL_PARAMETERS_CMD.VERBOSE_STEP', 'OK'
ob->SetParam, 'GENERAL_PARAMETERS_CMD.VERBOSE_BAR', 'OK'
ob->SetParam, 'GENERAL_PARAMETERS_CMD.APPLY_CALIBRATION_CONSTANT_FLAG', 'OK'
ob->SetParam, 'GENERAL_PARAMETERS_CMD.FILL_DUMMY_DURING_IMPORT', 'OK'
call_procedure, set_params_proc, ob
ok = ob->VerifyParams(Silent=0)
if ~ok then begin
errmsg = 'VerifyParams failed: ' + module_name
print, errmsg
return
endif
if n_elements(working_directory) eq 0 then begin
ok = ob->Execute()
endif else begin
exec_errmsg = ''
ok = ob->Execute(WORKING_DIRECTORY=working_directory, ERRMSG=exec_errmsg)
errmsg = gf3_normalize_errmsg(exec_errmsg)
endelse
if ok then begin
print, 'Success: ' + module_name
endif else begin
if errmsg eq '' then begin
err_code = ''
err_text = get_SARscape_error_string('OK', ERROR_CODE=err_code)
errmsg = 'Execution failed [' + module_name + '] EC [' + err_code + ']: ' + err_text
endif
print, errmsg
endelse
end
pro gf3_pipeline_set_import_params, ob
compile_opt idl2
common gf3_pipeline_cfg, cfg
ob->SetParam, 'MAIN_BASIC_IMPORT_GAOFEN3_CMD_CMD.SARSCAPEENVIRONMENT', 'IDL_ENVI_ENV'
ob->SetParam, 'MAIN_BASIC_IMPORT_GAOFEN3_CMD_CMD.INPUT_FILE_LIST', [cfg.input_meta_xml]
ob->SetParam, 'MAIN_BASIC_IMPORT_GAOFEN3_CMD_CMD.OUTPUT_FILE_LIST', [cfg.import_root]
ob->SetParam, 'MAIN_BASIC_IMPORT_GAOFEN3_CMD_CMD.CROSS_COPOLARIZATION_FLAG', cfg.import_polarization_mode
ob->SetParam, 'MAIN_BASIC_IMPORT_GAOFEN3_CMD_CMD.DOPP_ESTIMATION_FROM_RASTER_FLAG', 'OK'
ob->SetParam, 'MAIN_BASIC_IMPORT_GAOFEN3_CMD_CMD.KEEP_ALL_LINES_FLAG', 'NotOK'
end
pro gf3_pipeline_set_multilook_params, ob
compile_opt idl2
common gf3_pipeline_cfg, cfg
common gf3_pipeline_state, state
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.SARSCAPEENVIRONMENT', 'IDL_ENVI_ENV'
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.INPUT_FILE_LIST', [state.current_input_root]
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.OUTPUT_FILE_LIST', [state.current_output_root]
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.MULTILOOK_METHOD', cfg.multilook_method
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.RANGE_MULTILOOK', gf3_float_str(cfg.range_looks)
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.AZIMUTH_MULTILOOK', gf3_float_str(cfg.azimuth_looks)
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.GRID_SIZE_FOR_SUGGESTED_LOOKS', gf3_float_str(cfg.grid_size_for_suggested_looks)
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.FILL_DUMMY_FLAG', 'OK'
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.FILL_DUMMY_METHOD', 'min_image_value'
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.ROWS_WINDOW_NUMBER', '3.0000000'
ob->SetParam, 'MAIN_BASIC_MULTILOOKING.COLS_WINDOW_NUMBER', '3.0000000'
end
pro gf3_pipeline_set_filter_params, ob
compile_opt idl2
common gf3_pipeline_cfg, cfg
common gf3_pipeline_state, state
ob->SetParam, 'MAIN_BASIC_DESPECKLE_CONVENTIONAL_CMD.SARSCAPEENVIRONMENT', 'IDL_ENVI_ENV'
ob->SetParam, 'MAIN_BASIC_DESPECKLE_CONVENTIONAL_CMD.INPUT_FILE_LIST', [state.current_input_root]
ob->SetParam, 'MAIN_BASIC_DESPECKLE_CONVENTIONAL_CMD.OUTPUT_FILE_LIST', [state.current_output_root]
ob->SetParam, 'MAIN_BASIC_DESPECKLE_CONVENTIONAL_CMD.FILT_TYPE', cfg.filter_type
ob->SetParam, 'MAIN_BASIC_DESPECKLE_CONVENTIONAL_CMD.EQUIVALENT_LOOKS', '-1.0000000'
ob->SetParam, 'PARAMETERS_GENERIC_FILTERS.WIN_SIZE', gf3_float_str(cfg.filter_window)
ob->SetParam, 'PARAMETERS_GENERIC_FILTERS.ROWS_WINDOW_NUMBER', gf3_float_str(cfg.filter_window)
ob->SetParam, 'PARAMETERS_GENERIC_FILTERS.COLS_WINDOW_NUMBER', gf3_float_str(cfg.filter_window)
ob->SetParam, 'PARAMETERS_GENERIC_FILTERS.WIN_MODE_SIZE', gf3_float_str(cfg.filter_window)
end
pro gf3_pipeline_set_geocode_params, ob
compile_opt idl2
common gf3_pipeline_cfg, cfg
common gf3_pipeline_state, state
ob->SetParam, 'OUT_CARTOGRAPHIC_SYSTEM.OCS_CARTOGRAPHIC_SYSTEM_SPECIFICATION', ''
ob->SetParam, 'OUT_CARTOGRAPHIC_SYSTEM.OCS_STATE', cfg.ocs_state
ob->SetParam, 'OUT_CARTOGRAPHIC_SYSTEM.OCS_HEMISPHERE', cfg.ocs_hemisphere
ob->SetParam, 'OUT_CARTOGRAPHIC_SYSTEM.OCS_PROJECTION', cfg.ocs_projection
ob->SetParam, 'OUT_CARTOGRAPHIC_SYSTEM.OCS_ELLIPSOID', 'WGS84'
ob->SetParam, 'OUT_CARTOGRAPHIC_SYSTEM.OCS_REFERENCE_HEIGHT', '0.0'
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.SARSCAPEENVIRONMENT', 'IDL_ENVI_ENV'
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.INPUT_FILE_LIST', [state.current_input_root]
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.OUTPUT_FILE_LIST', [state.current_output_root]
if cfg.dem_file ne '' then ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.DEM_FILE_NAME', cfg.dem_file
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_GRID_SIZE_X', gf3_float_str(cfg.geocode_grid_x)
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_GRID_SIZE_Y', gf3_float_str(cfg.geocode_grid_y)
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_RESAMPLING_TYPE', cfg.geocode_resampling
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.CALIBRATION_FLAG', cfg.calibration_flag
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEO_SCATTERING_AREA_METHOD', 'sine_area_estimation'
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.TRUE_AREA_EXPONENT', '1.0000000'
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.MAX_VALUE_IN_CALIBRATION', '100.00000'
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.RAD_NORMALIZATION_FLAG', cfg.rad_normalization_flag
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_RAD_NORM_DEG', '2.0000000'
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_RAD_NORM_ANG', '-1.0000000'
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_SIGMA_FLAG', cfg.output_sigma_flag
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_GAMMA_FLAG', cfg.output_gamma_flag
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_BETA_FLAG', cfg.output_beta_flag
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_COREG_IMAGES_FLAG', 'NotOK'
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GENERATE_LIA_FLAG', 'NotOK'
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.OUTPUT_TYPE', cfg.output_type
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.DUMMY_REMOVAL_FLAG', 'NotOK'
ob->SetParam, 'MAIN_BASIC_CALIBRATION_AND_GEO_CMD.GEOCODE_BLOCK_SIZE', '30000.000'
end
pro gf3_init_default_cfg
compile_opt idl2
common gf3_pipeline_cfg, cfg
cfg = { $
input_meta_xml: 'C:\Users\Administrator\Desktop\GF3_Test\GF3_scene_20260513\GF3_SYC_FSI_051370_E124.6_N51.7_20260513_L1A_HHHV_L10007355955.meta.xml', $
work_dir: 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\work', $
temp_dir: 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\temp', $
out_dir: 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\out', $
output_prefix: '', $
dem_file: 'D:\DEM\COPDEM_GLO30_China_4326_DEM', $
import_root: 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\out\gf3_import', $
import_polarization_mode: 'ALL_POL', $
imported_hh_root: 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\out\gaofer3_20260513_214340051_D_HH_slc', $
imported_hv_root: 'C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\out\gaofer3_20260513_214340051_D_HV_slc', $
multilook_method: 'time_domain', $
range_looks: 2.0, $
azimuth_looks: 2.0, $
grid_size_for_suggested_looks: 15.0, $
filter_type: 'lee', $
filter_window: 5.0, $
geocode_grid_x: 15.0, $
geocode_grid_y: 15.0, $
geocode_resampling: '4th_order_cc', $
calibration_flag: 'OK', $
rad_normalization_flag: 'OK', $
output_sigma_flag: 'OK', $
output_gamma_flag: 'NotOK', $
output_beta_flag: 'NotOK', $
output_type: 'output_type_linear', $
ocs_state: 'GEO-GLOBAL', $
ocs_hemisphere: 'NONE', $
ocs_projection: 'GEO' $
}
end
pro gf3_configure, input_meta_xml=input_meta_xml, out_dir=out_dir, dem_file=dem_file, $
work_dir=work_dir, temp_dir=temp_dir, range_looks=range_looks, $
azimuth_looks=azimuth_looks, filter_type=filter_type, filter_window=filter_window, $
geocode_grid_x=geocode_grid_x, geocode_grid_y=geocode_grid_y, $
imported_hh_root=imported_hh_root, imported_hv_root=imported_hv_root
compile_opt idl2
common gf3_pipeline_cfg, cfg
gf3_init_default_cfg
if n_elements(input_meta_xml) gt 0 then cfg.input_meta_xml = input_meta_xml
if n_elements(out_dir) gt 0 then cfg.out_dir = out_dir
if n_elements(dem_file) gt 0 then cfg.dem_file = dem_file
if n_elements(work_dir) gt 0 then cfg.work_dir = work_dir
if n_elements(temp_dir) gt 0 then cfg.temp_dir = temp_dir
if n_elements(range_looks) gt 0 then cfg.range_looks = double(range_looks)
if n_elements(azimuth_looks) gt 0 then cfg.azimuth_looks = double(azimuth_looks)
if n_elements(filter_type) gt 0 then cfg.filter_type = filter_type
if n_elements(filter_window) gt 0 then cfg.filter_window = double(filter_window)
if n_elements(geocode_grid_x) gt 0 then cfg.geocode_grid_x = double(geocode_grid_x)
if n_elements(geocode_grid_y) gt 0 then cfg.geocode_grid_y = double(geocode_grid_y)
if n_elements(imported_hh_root) gt 0 then cfg.imported_hh_root = imported_hh_root
if n_elements(imported_hv_root) gt 0 then cfg.imported_hv_root = imported_hv_root
cfg.import_root = cfg.out_dir + '\gf3_import'
cfg.output_prefix = gf3_scene_token(cfg.input_meta_xml)
if n_elements(work_dir) eq 0 then cfg.work_dir = cfg.out_dir + '\work'
if n_elements(temp_dir) eq 0 then cfg.temp_dir = cfg.out_dir + '\temp'
if n_elements(imported_hh_root) eq 0 then cfg.imported_hh_root = ''
if n_elements(imported_hv_root) eq 0 then cfg.imported_hv_root = ''
end
pro gf3_prepare_dirs
compile_opt idl2
common gf3_pipeline_cfg, cfg
file_mkdir, cfg.work_dir
file_mkdir, cfg.temp_dir
file_mkdir, cfg.out_dir
end
pro gf3_start_batch
compile_opt idl2
common gf3_pipeline_cfg, cfg
SARscape_Batch_Init, Temp_Directory=cfg.temp_dir
end
pro gf3_stop_batch
compile_opt idl2
SARscape_Batch_Exit
end
pro gf3_run_import, errmsg=errmsg
compile_opt idl2
common gf3_pipeline_cfg, cfg
errmsg = ''
gf3_run_module, 'ImportGaofen3', 'gf3_pipeline_set_import_params', errmsg=errmsg, working_directory=cfg.work_dir
end
pro gf3_run_multilook, polarization, errmsg=errmsg
compile_opt idl2
common gf3_pipeline_cfg, cfg
common gf3_pipeline_state, state
state = {current_input_root: gf3_base_root(polarization), current_output_root: gf3_stage_root(polarization, 'multilook')}
errmsg = ''
gf3_run_module, 'BaseMultilooking', 'gf3_pipeline_set_multilook_params', errmsg=errmsg, working_directory=cfg.work_dir
end
pro gf3_run_filter, polarization, errmsg=errmsg
compile_opt idl2
common gf3_pipeline_cfg, cfg
common gf3_pipeline_state, state
state = {current_input_root: gf3_stage_root(polarization, 'multilook'), current_output_root: gf3_stage_root(polarization, 'filter')}
errmsg = ''
gf3_run_module, 'DespeckleConventionalSingle', 'gf3_pipeline_set_filter_params', errmsg=errmsg, working_directory=cfg.work_dir
end
pro gf3_run_geocode, polarization, errmsg=errmsg
compile_opt idl2
common gf3_pipeline_cfg, cfg
common gf3_pipeline_state, state
state = {current_input_root: gf3_stage_root(polarization, 'filter'), current_output_root: gf3_stage_root(polarization, 'geocode')}
errmsg = ''
gf3_run_module, 'BasicGeocoding', 'gf3_pipeline_set_geocode_params', errmsg=errmsg, working_directory=cfg.work_dir
end
pro gf3_run_single_pol_pipeline, polarization, do_import=do_import, errmsg=errmsg
compile_opt idl2
errmsg = ''
if keyword_set(do_import) then begin
gf3_run_import, errmsg=errmsg
if errmsg ne '' then return
gf3_refresh_import_roots
endif
gf3_run_multilook, polarization, errmsg=errmsg
if errmsg ne '' then return
gf3_run_filter, polarization, errmsg=errmsg
if errmsg ne '' then return
gf3_run_geocode, polarization, errmsg=errmsg
end
function gf3_should_run_pol, polarizations, polarization
compile_opt idl2
if n_elements(polarizations) eq 0 then return, 1
wanted = strupcase(strtrim(polarizations, 2))
pol = strupcase(strtrim(polarization, 2))
for i = 0, n_elements(wanted) - 1 do begin
if wanted[i] eq 'ALL' then return, 1
if wanted[i] eq pol then return, 1
endfor
return, 0
end
pro gf3_sarscape_run, input_meta_xml=input_meta_xml, out_dir=out_dir, dem_file=dem_file, $
polarizations=polarizations, skip_import=skip_import, work_dir=work_dir, temp_dir=temp_dir, $
range_looks=range_looks, azimuth_looks=azimuth_looks, filter_type=filter_type, $
filter_window=filter_window, geocode_grid_x=geocode_grid_x, geocode_grid_y=geocode_grid_y
compile_opt idl2
gf3_configure, input_meta_xml=input_meta_xml, out_dir=out_dir, dem_file=dem_file, $
work_dir=work_dir, temp_dir=temp_dir, range_looks=range_looks, $
azimuth_looks=azimuth_looks, filter_type=filter_type, filter_window=filter_window, $
geocode_grid_x=geocode_grid_x, geocode_grid_y=geocode_grid_y
gf3_prepare_dirs
gf3_start_batch
errmsg = ''
if ~keyword_set(skip_import) then begin
gf3_run_import, errmsg=errmsg
if errmsg eq '' then gf3_refresh_import_roots
endif else begin
gf3_refresh_import_roots
endelse
if errmsg eq '' && gf3_should_run_pol(polarizations, 'HH') then gf3_run_single_pol_pipeline, 'HH', errmsg=errmsg
if errmsg eq '' && gf3_should_run_pol(polarizations, 'HV') then gf3_run_single_pol_pipeline, 'HV', errmsg=errmsg
gf3_stop_batch
if errmsg ne '' then print, 'Pipeline stopped: ' + errmsg
end
pro gf3_sarscape_pipeline
compile_opt idl2
common gf3_pipeline_cfg, cfg
gf3_init_default_cfg
gf3_prepare_dirs
gf3_start_batch
errmsg = ''
gf3_run_import, errmsg=errmsg
if errmsg eq '' then gf3_refresh_import_roots
if errmsg eq '' then gf3_run_single_pol_pipeline, 'HH', errmsg=errmsg
if errmsg eq '' then gf3_run_single_pol_pipeline, 'HV', errmsg=errmsg
gf3_stop_batch
if errmsg ne '' then print, 'Pipeline stopped: ' + errmsg
end
pro gf3_sarscape_import_only
compile_opt idl2
gf3_init_default_cfg
gf3_prepare_dirs
gf3_start_batch
errmsg = ''
gf3_run_import, errmsg=errmsg
gf3_stop_batch
if errmsg ne '' then print, 'Import stopped: ' + errmsg
end
pro gf3_sarscape_hh_only
compile_opt idl2
gf3_init_default_cfg
gf3_prepare_dirs
gf3_start_batch
errmsg = ''
gf3_run_single_pol_pipeline, 'HH', do_import=1, errmsg=errmsg
gf3_stop_batch
if errmsg ne '' then print, 'HH pipeline stopped: ' + errmsg
end
pro gf3_sarscape_hv_only
compile_opt idl2
gf3_init_default_cfg
gf3_prepare_dirs
gf3_start_batch
errmsg = ''
gf3_run_single_pol_pipeline, 'HV', do_import=1, errmsg=errmsg
gf3_stop_batch
if errmsg ne '' then print, 'HV pipeline stopped: ' + errmsg
end
@@ -0,0 +1,8 @@
@echo off
set IDLRT=C:\Program Files\Harris\ENVI56\IDL88\bin\bin.x86_64\idlrt.exe
set SAV=C:\Users\Administrator\Desktop\GF3_Test\GF3_ENVI_IDL\gf3_sarscape_cli.sav
set META=C:\Users\Administrator\Desktop\GF3_Test\GF3_scene_20260513\GF3_SYC_FSI_051370_E124.6_N51.7_20260513_L1A_HHHV_L10007355955.meta.xml
set OUT=C:\Users\Administrator\Desktop\GF3_Test\GF3_cli_test_out_20260513
set DEM=D:\DEM\COPDEM_GLO30_China_4326_DEM
"%IDLRT%" "%SAV%" -args "%META%" "%OUT%" "%DEM%" "HH,HV"
+594
View File
@@ -0,0 +1,594 @@
package main
import (
"archive/tar"
"bufio"
"compress/gzip"
"embed"
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"io/fs"
"os"
"os/exec"
"path/filepath"
"sort"
"strings"
"time"
)
//go:embed assets/gf3_sarscape_cli.sav
var embedded embed.FS
type config struct {
configPath string `json:"-"`
input string `json:"input"`
output string `json:"output"`
dem string `json:"dem"`
polarizations string `json:"polarizations"`
idlrt string `json:"idlrt"`
keepExtracted bool `json:"keep_extracted"`
}
type scene struct {
metaPath string
archivePath string
name string
}
func main() {
cfg := parseFlags()
if len(os.Args) == 1 {
var err error
cfg, err = promptConfig(cfg)
if err != nil {
fmt.Printf("ERROR: %v\n", err)
waitForEnter()
os.Exit(1)
}
}
if err := run(cfg); err != nil {
fmt.Printf("ERROR: %v\n", err)
if len(os.Args) == 1 {
waitForEnter()
}
os.Exit(1)
}
if err := saveConfig(cfg.configPath, cfg); err != nil {
fmt.Printf("WARNING: save config: %v\n", err)
}
if len(os.Args) == 1 {
fmt.Println("Done.")
waitForEnter()
}
}
func parseFlags() config {
cfg := defaultConfig()
configPath := preScanConfigPath()
cfg.configPath = configPath
_ = loadConfig(configPath, &cfg)
flag.StringVar(&cfg.configPath, "config", configPath, "Config file path")
flag.StringVar(&cfg.input, "input", cfg.input, "GF-3 input: .meta.xml, .tar.gz, or directory")
flag.StringVar(&cfg.output, "output", cfg.output, "Output directory")
flag.StringVar(&cfg.dem, "dem", cfg.dem, "SARscape DEM file")
flag.StringVar(&cfg.polarizations, "pol", cfg.polarizations, "Polarizations: HH, HV, or HH,HV")
flag.StringVar(&cfg.idlrt, "idlrt", cfg.idlrt, "Path to idlrt.exe")
flag.BoolVar(&cfg.keepExtracted, "keep-extracted", cfg.keepExtracted, "Keep extracted archives under output/.gf3_extract")
flag.Parse()
return cfg
}
func defaultConfig() config {
return config{
dem: `D:\DEM\COPDEM_GLO30_China_4326_DEM`,
polarizations: "HH,HV",
idlrt: defaultIDLRT(),
keepExtracted: true,
}
}
func preScanConfigPath() string {
defaultPath := defaultConfigPath()
for i := 1; i < len(os.Args)-1; i++ {
if os.Args[i] == "-config" {
return os.Args[i+1]
}
}
return defaultPath
}
func defaultConfigPath() string {
exe, err := os.Executable()
if err != nil {
return "gf3wrapper.json"
}
return filepath.Join(filepath.Dir(exe), "gf3wrapper.json")
}
func loadConfig(path string, cfg *config) error {
data, err := os.ReadFile(path)
if err != nil {
return err
}
var stored struct {
IDLRTPath string `json:"idlrt_path"`
DemFile string `json:"dem_file"`
Polarizations string `json:"polarizations"`
LastInput string `json:"last_input"`
LastOutput string `json:"last_output"`
IDLRT string `json:"idlrt"`
Dem string `json:"dem"`
Input string `json:"input"`
Output string `json:"output"`
KeepExtracted *bool `json:"keep_extracted"`
}
if err := json.Unmarshal(data, &stored); err != nil {
return err
}
if stored.IDLRTPath != "" {
cfg.idlrt = stored.IDLRTPath
} else if stored.IDLRT != "" {
cfg.idlrt = stored.IDLRT
}
if stored.DemFile != "" {
cfg.dem = stored.DemFile
} else if stored.Dem != "" {
cfg.dem = stored.Dem
}
if stored.Polarizations != "" {
cfg.polarizations = stored.Polarizations
}
if stored.LastInput != "" {
cfg.input = stored.LastInput
} else if stored.Input != "" {
cfg.input = stored.Input
}
if stored.LastOutput != "" {
cfg.output = stored.LastOutput
} else if stored.Output != "" {
cfg.output = stored.Output
}
if stored.KeepExtracted != nil {
cfg.keepExtracted = *stored.KeepExtracted
}
cfg.configPath = path
return nil
}
func saveConfig(path string, cfg config) error {
if path == "" {
path = defaultConfigPath()
}
stored := struct {
IDLRTPath string `json:"idlrt_path"`
DemFile string `json:"dem_file"`
Polarizations string `json:"polarizations"`
LastInput string `json:"last_input"`
LastOutput string `json:"last_output"`
KeepExtracted bool `json:"keep_extracted"`
}{
IDLRTPath: cfg.idlrt,
DemFile: cfg.dem,
Polarizations: cfg.polarizations,
LastInput: cfg.input,
LastOutput: cfg.output,
KeepExtracted: cfg.keepExtracted,
}
data, err := json.MarshalIndent(stored, "", " ")
if err != nil {
return err
}
return os.WriteFile(path, data, 0o644)
}
func promptConfig(defaults config) (config, error) {
reader := bufio.NewReader(os.Stdin)
cfg := defaults
fmt.Println("GF-3 SARscape Wrapper")
fmt.Println()
cfg.input = prompt(reader, "Input .tar.gz / .meta.xml / folder", cfg.input)
cfg.output = prompt(reader, "Output folder", cfg.output)
cfg.dem = prompt(reader, "DEM file", cfg.dem)
cfg.polarizations = prompt(reader, "Polarizations", cfg.polarizations)
cfg.idlrt = prompt(reader, "IDL Runtime", cfg.idlrt)
if cfg.input == "" || cfg.output == "" || cfg.dem == "" {
return cfg, errors.New("input, output, and DEM are required")
}
return cfg, nil
}
func prompt(reader *bufio.Reader, label, defaultValue string) string {
if defaultValue != "" {
fmt.Printf("%s [%s]: ", label, defaultValue)
} else {
fmt.Printf("%s: ", label)
}
text, _ := reader.ReadString('\n')
text = strings.TrimSpace(text)
if text == "" {
return defaultValue
}
return text
}
func waitForEnter() {
fmt.Println()
fmt.Print("Press Enter to exit...")
_, _ = bufio.NewReader(os.Stdin).ReadString('\n')
}
func run(cfg config) error {
if cfg.input == "" || cfg.output == "" || cfg.dem == "" {
return errors.New("required flags: -input, -output, -dem")
}
if _, err := os.Stat(cfg.idlrt); err != nil {
return fmt.Errorf("idlrt not found: %s: %w", cfg.idlrt, err)
}
if _, err := os.Stat(cfg.dem); err != nil {
return fmt.Errorf("DEM not found: %s: %w", cfg.dem, err)
}
if err := os.MkdirAll(cfg.output, 0o755); err != nil {
return err
}
savPath, err := materializeSAV(cfg.output)
if err != nil {
return err
}
scenes, err := discoverScenes(cfg)
if err != nil {
return err
}
if len(scenes) == 0 {
return fmt.Errorf("no *.meta.xml scenes found in %s", cfg.input)
}
for _, sc := range scenes {
sceneOut := filepath.Join(cfg.output, sc.name)
if err := os.MkdirAll(sceneOut, 0o755); err != nil {
return err
}
if sc.archivePath != "" {
extractDir := filepath.Join(sceneOut, ".gf3_extract")
fmt.Printf("Extracting %s\n", sc.archivePath)
if err := extractTarGz(sc.archivePath, extractDir); err != nil {
return err
}
extractedScenes, err := findMetaXML(extractDir)
if err != nil {
return err
}
if len(extractedScenes) == 0 {
return fmt.Errorf("no *.meta.xml found in archive: %s", sc.archivePath)
}
sc.metaPath = extractedScenes[0].metaPath
sc.name = sceneName(sc.metaPath)
}
if err := runScene(cfg, savPath, sc, sceneOut); err != nil {
return err
}
}
return nil
}
func materializeSAV(outputDir string) (string, error) {
data, err := embedded.ReadFile("assets/gf3_sarscape_cli.sav")
if err != nil {
return "", err
}
runtimeDir := filepath.Join(outputDir, ".gf3_runtime")
if err := os.MkdirAll(runtimeDir, 0o755); err != nil {
return "", err
}
savPath := filepath.Join(runtimeDir, "gf3_sarscape_cli.sav")
if err := os.WriteFile(savPath, data, 0o644); err != nil {
return "", err
}
return savPath, nil
}
func discoverScenes(cfg config) ([]scene, error) {
info, err := os.Stat(cfg.input)
if err != nil {
return nil, err
}
if info.IsDir() {
return discoverScenesInDir(cfg.input, cfg.output)
}
lower := strings.ToLower(cfg.input)
if strings.HasSuffix(lower, ".meta.xml") {
return []scene{{metaPath: cfg.input, name: sceneName(cfg.input)}}, nil
}
if strings.HasSuffix(lower, ".tar.gz") || strings.HasSuffix(lower, ".tgz") {
return []scene{{archivePath: cfg.input, name: safeName(trimArchiveExt(filepath.Base(cfg.input)))}}, nil
}
return nil, fmt.Errorf("unsupported input type: %s", cfg.input)
}
func discoverScenesInDir(inputDir, outputDir string) ([]scene, error) {
_ = outputDir
scenesByName := make(map[string]scene)
err := filepath.WalkDir(inputDir, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
if d.Name() == ".gf3_extract" {
return filepath.SkipDir
}
return nil
}
lower := strings.ToLower(d.Name())
switch {
case strings.HasSuffix(lower, ".meta.xml"):
sc := scene{metaPath: path, name: sceneName(path)}
key := strings.ToLower(sc.name)
if _, ok := scenesByName[key]; !ok {
scenesByName[key] = sc
}
case strings.HasSuffix(lower, ".tar.gz") || strings.HasSuffix(lower, ".tgz"):
sc := scene{archivePath: path, name: safeName(trimArchiveExt(filepath.Base(path)))}
scenesByName[strings.ToLower(sc.name)] = sc
}
return nil
})
if err != nil {
return nil, err
}
scenes := make([]scene, 0, len(scenesByName))
for _, sc := range scenesByName {
scenes = append(scenes, sc)
}
sort.Slice(scenes, func(i, j int) bool {
return strings.ToLower(scenes[i].name) < strings.ToLower(scenes[j].name)
})
return scenes, nil
}
func findMetaXML(root string) ([]scene, error) {
var scenes []scene
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
return nil
}
if strings.HasSuffix(strings.ToLower(d.Name()), ".meta.xml") {
scenes = append(scenes, scene{metaPath: path, name: sceneName(path)})
}
return nil
})
return scenes, err
}
func uniqueScenes(scenes []scene) []scene {
seen := make(map[string]bool, len(scenes))
out := make([]scene, 0, len(scenes))
for _, sc := range scenes {
key := filepath.Clean(sc.metaPath)
if seen[key] {
continue
}
seen[key] = true
out = append(out, sc)
}
return out
}
func runScene(cfg config, savPath string, sc scene, sceneOut string) error {
fmt.Printf("Processing %s\n", sc.metaPath)
fmt.Printf("Output %s\n", sceneOut)
cmd := exec.Command(cfg.idlrt, savPath, "-args", sc.metaPath, sceneOut, cfg.dem, cfg.polarizations)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.Dir = sceneOut
if err := cmd.Start(); err != nil {
return err
}
done := make(chan error, 1)
go func() {
done <- cmd.Wait()
}()
ticker := time.NewTicker(2 * time.Second)
defer ticker.Stop()
for {
select {
case err := <-done:
fmt.Printf("\r%-100s\r", "")
if err != nil {
return fmt.Errorf("scene failed %s: %w", sc.metaPath, err)
}
fmt.Println("Completed", sc.name)
return nil
case <-ticker.C:
fmt.Printf("\r%s", progressLine(sceneOut, sc.name, cfg.polarizations))
}
}
}
func progressLine(sceneOut, sceneNameValue, polarizations string) string {
stage, done, total := progressStage(sceneOut, sceneNameValue, polarizations)
width := 28
filled := 0
if total > 0 {
filled = done * width / total
}
if filled > width {
filled = width
}
bar := strings.Repeat("#", filled) + strings.Repeat("-", width-filled)
return fmt.Sprintf("[%s] %2d/%2d %s", bar, done, total, stage)
}
func progressStage(sceneOut, sceneNameValue, polarizations string) (string, int, int) {
pols := requestedPolarizations(polarizations)
checks := make([]struct {
label string
path string
}, 0, len(pols)*4)
for _, pol := range pols {
checks = append(checks, struct {
label string
path string
}{"import " + pol, filepath.Join(sceneOut, "*_"+pol+"_slc.sml")})
}
for _, pol := range pols {
lower := strings.ToLower(pol)
prefix := sceneNameValue + "_" + lower
checks = append(checks,
struct {
label string
path string
}{"multilook " + pol, filepath.Join(sceneOut, prefix+"_ml.sml")},
struct {
label string
path string
}{"filter " + pol, filepath.Join(sceneOut, prefix+"_filt.sml")},
struct {
label string
path string
}{"geocode " + pol, filepath.Join(sceneOut, prefix+"_geo.sml")},
)
}
done := 0
for _, check := range checks {
if matched(check.path) {
done++
continue
}
return check.label, done, len(checks)
}
return "finishing", done, len(checks)
}
func requestedPolarizations(value string) []string {
parts := strings.Split(value, ",")
out := make([]string, 0, len(parts))
for _, part := range parts {
pol := strings.ToUpper(strings.TrimSpace(part))
if pol == "" {
continue
}
if pol == "ALL" {
return []string{"HH", "HV"}
}
out = append(out, pol)
}
if len(out) == 0 {
return []string{"HH", "HV"}
}
return out
}
func matched(pattern string) bool {
matches, err := filepath.Glob(pattern)
if err != nil {
return false
}
return len(matches) > 0
}
func extractTarGz(src, dst string) error {
if err := os.MkdirAll(dst, 0o755); err != nil {
return err
}
f, err := os.Open(src)
if err != nil {
return err
}
defer f.Close()
gz, err := gzip.NewReader(f)
if err != nil {
return err
}
defer gz.Close()
tr := tar.NewReader(gz)
for {
hdr, err := tr.Next()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
return err
}
target := filepath.Join(dst, filepath.Clean(hdr.Name))
if !strings.HasPrefix(target, filepath.Clean(dst)+string(os.PathSeparator)) && filepath.Clean(target) != filepath.Clean(dst) {
return fmt.Errorf("archive entry escapes destination: %s", hdr.Name)
}
switch hdr.Typeflag {
case tar.TypeDir:
if err := os.MkdirAll(target, 0o755); err != nil {
return err
}
case tar.TypeReg:
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
return err
}
out, err := os.OpenFile(target, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o644)
if err != nil {
return err
}
_, copyErr := io.Copy(out, tr)
closeErr := out.Close()
if copyErr != nil {
return copyErr
}
if closeErr != nil {
return closeErr
}
}
}
return nil
}
func sceneName(metaPath string) string {
base := filepath.Base(metaPath)
base = strings.TrimSuffix(base, ".meta.xml")
base = strings.TrimSuffix(base, ".META.XML")
return safeName(base)
}
func safeName(s string) string {
replacer := strings.NewReplacer(" ", "_", ":", "_", "/", "_", "\\", "_")
return replacer.Replace(s)
}
func trimArchiveExt(name string) string {
lower := strings.ToLower(name)
switch {
case strings.HasSuffix(lower, ".tar.gz"):
return name[:len(name)-7]
case strings.HasSuffix(lower, ".tgz"):
return name[:len(name)-4]
default:
return strings.TrimSuffix(name, filepath.Ext(name))
}
}
func defaultIDLRT() string {
if v := os.Getenv("IDLRT_PATH"); v != "" {
return v
}
return `C:\Program Files\Harris\ENVI56\IDL88\bin\bin.x86_64\idlrt.exe`
}