Initial D-InSAR bundle restore tool

This commit is contained in:
Harmon
2026-05-11 10:43:04 +08:00
commit e595231e04
8 changed files with 2082 additions and 0 deletions
+5
View File
@@ -0,0 +1,5 @@
*.exe
*.zip
release/
.manual_check_*/
tmp_manual_check/
@@ -0,0 +1,142 @@
# D-InSAR 去重源数据包反向还原工具任务书
日期:2026-05-11
## 背景
本系统新增“去重源数据包”分发模式。该模式不直接生成每个干涉对的 `Task_*` 目录,而是只分发唯一源影像、唯一精密轨道文件和配对关系文件,减少外部分发时的重复复制量。
反向还原工具由任务接收方本地运行,将去重源数据包还原为传统 D-InSAR `Task_*` 目录结构。
## 输入目录结构
```text
BundleRoot/
data/
scene_<hash>_<source_name>/
...
orbit/
orbit_<hash>_<orbit_name>.txt
...
pairs.json
manifest.json
```
`orbit/` 可能不存在,或 `pairs.json` 内某些配对的轨道字段为空。
## 输出目录结构
```text
OutputRoot/
Task_YYYYMMDD_YYYYMMDD/
master/
<master source product content>
slave/
<slave source product content>
orbit/
<master/slave orbit files, if present>
.dinsar_pair.json
```
输出目录名称优先使用 `pairs.json` 内的 `task_alias`,若为空则使用 `task_name`,再为空则使用 `pair_id`
## pairs.json 关键字段
```json
{
"schema": "dinsar_source_bundle_pairs.v1",
"exported_at": "2026-05-11T00:00:00Z",
"pairs": [
{
"pair_id": "pair_0001",
"task_name": "Task_20250101_20250113",
"task_alias": "Task_20250101_20250113",
"master_data": "data/scene_xxx_master",
"slave_data": "data/scene_yyy_slave",
"master_orbit": "orbit/orbit_xxx.txt",
"slave_orbit": "orbit/orbit_yyy.txt",
"master_imaging_date": "20250101",
"slave_imaging_date": "20250113",
"time_baseline_days": 12
}
]
}
```
## 还原规则
1. 读取 `pairs.json`
2. 对每个 pair 创建目标 `Task` 目录。
3.`master_data` 指向的数据复制到 `Task/master/`
4.`slave_data` 指向的数据复制到 `Task/slave/`
5.`master_orbit` / `slave_orbit` 存在,将轨道文件复制到 `Task/orbit/`
6. 生成 `.dinsar_pair.json`,至少保留:
- `pair_id`
- `task_name`
- `task_alias`
- `master_data`
- `slave_data`
- `master_orbit`
- `slave_orbit`
- `master_imaging_date`
- `slave_imaging_date`
- `time_baseline_days`
- `restored_at`
7. 每个 Task 应采用临时目录还原,全部成功后再重命名为最终目录,避免半成品。
## 覆盖策略
工具应提供参数:
- `--skip-existing`:默认开启。若目标 `Task/master``Task/slave` 均存在且非空,则跳过。
- `--overwrite`:删除并重建已存在的目标 Task。
- `--limit N`:最多还原 N 个 pair,便于分批执行。
- `--dry-run`:只打印计划,不复制。
`--skip-existing``--overwrite` 同时出现时应报错。
## 校验要求
启动前:
- 检查 `pairs.json` 是否存在且可解析。
- 检查 `data/` 是否存在。
- 检查每个 pair 的 `master_data` / `slave_data` 是否存在。
- 轨道缺失不应阻断还原,但要记录 warning。
还原后:
- `Task/master/` 非空。
- `Task/slave/` 非空。
- `.dinsar_pair.json` 存在。
## 日志与报告
工具结束后输出 `restore_report.json`
```json
{
"started_at": "...",
"finished_at": "...",
"input_root": "...",
"output_root": "...",
"total_pairs": 20,
"restored": 18,
"skipped": 2,
"failed": 0,
"warnings": []
}
```
同时建议输出人类可读日志 `restore.log`
## 建议实现
建议使用 Python 3.10+
- `argparse` 处理命令行参数。
- `pathlib.Path` 处理路径。
- `shutil.copytree(..., dirs_exist_ok=True)` / `shutil.copy2()` 处理复制。
- Windows 下注意长路径和权限异常。
该工具不需要连接本系统数据库,也不需要调用本系统 API。
+243
View File
@@ -0,0 +1,243 @@
# D-InSAR 去重源数据包恢复工具
这个工具用于把“去重源数据包”恢复成传统 D-InSAR `Task_*` 目录结构。
适合接收方在本地运行:选择一个 `BundleRoot` 输入目录,工具会按 `pairs.json` 中的配对关系,在输出目录中恢复每个 `Task_*`
## 产物
当前提供两个 Windows 可执行文件:
- `dinsar-restore.exe`:图形界面版,推荐普通用户双击使用。
- `dinsar-restore-console.exe`:命令行版,适合脚本、批处理或自动化调用。
Go 编译后的 exe 是单文件程序,不需要安装 Python,也不需要额外运行环境。
## 输入目录格式
输入目录应类似:
```text
BundleRoot/
data/
scene_<hash>_<source_name>/
...
orbit/
orbit_<hash>_<orbit_name>.txt
...
pairs.json
manifest.json
```
其中:
- `pairs.json` 必须存在。
- `data/` 必须存在。
- `orbit/` 可以不存在;轨道文件缺失只会记录 warning,不会阻断恢复。
## 输出目录格式
恢复后输出目录类似:
```text
OutputRoot/
Task_YYYYMMDD_YYYYMMDD/
master/
slave/
orbit/
.dinsar_pair.json
restore_report.json
restore.log
```
Task 目录名优先使用:
1. `task_alias`
2. `task_name`
3. `pair_id`
## 图形界面用法
双击运行:
```text
dinsar-restore.exe
```
操作流程:
1. 点击输入目录的“选择...”按钮,选择 `BundleRoot`
2. 建议先点击“数据自检”。
3. 点击输出目录的“选择...”按钮,选择 `OutputRoot`
4. 根据需要设置 `Dry run``覆盖已有 Task``Limit`
5. 点击“开始恢复”。
6. 在运行日志区域查看进度、warning 和错误。
### 数据自检
“数据自检”只检查输入数据,不复制文件,也不会生成 `Task_*`
自检内容包括:
- `pairs.json` 是否存在、是否能解析。
- `data/` 是否存在。
- 每个 pair 的 Task 目录名是否合法。
- `master_data` 是否存在、是否为非空目录。
- `slave_data` 是否存在、是否为非空目录。
- 是否存在重复 Task 目录名。
- `master_orbit` / `slave_orbit` 是否存在。
注意:轨道文件缺失只记为 warning,不算失败。
### 分批恢复
如果数据量很大,可以用 `Limit` 分批执行。
例如共有 1000 个 Task
1. 第一次设置 `Limit=200`,恢复 200 个。
2. 第二次仍设置 `Limit=200`,工具会跳过已完成的 200 个,再恢复新的 200 个。
3. 重复执行,直到全部完成。
已完成 Task 的判断标准是:
- `Task/master/` 存在且非空。
- `Task/slave/` 存在且非空。
已跳过的 Task 不会占用 `Limit` 数量。
### 覆盖已有 Task
默认情况下,已经完整恢复的 Task 会被跳过。
如果勾选“覆盖已有 Task”,工具会删除并重建已有 Task 目录。这个选项适合重新生成结果,但使用前应确认输出目录中没有需要保留的手工文件。
### Dry run
勾选 `Dry run` 后,工具只打印计划,不复制文件。
可以用它先确认将要恢复哪些 Task。
## 命令行用法
命令行版文件:
```powershell
.\dinsar-restore-console.exe
```
数据自检:
```powershell
.\dinsar-restore-console.exe --input D:\BundleRoot --check-only
```
正式恢复:
```powershell
.\dinsar-restore-console.exe --input D:\BundleRoot --output D:\OutputRoot
```
Dry run
```powershell
.\dinsar-restore-console.exe --input D:\BundleRoot --output D:\OutputRoot --dry-run
```
覆盖已有 Task
```powershell
.\dinsar-restore-console.exe --input D:\BundleRoot --output D:\OutputRoot --overwrite
```
分批恢复,每次最多恢复 200 个新 Task:
```powershell
.\dinsar-restore-console.exe --input D:\BundleRoot --output D:\OutputRoot --limit 200
```
## 参数说明
| 参数 | 说明 |
| --- | --- |
| `--input` | 输入目录,即 `BundleRoot`。 |
| `--output` | 输出目录,即 `OutputRoot``--check-only` 时可以不填。 |
| `--check-only` | 只做数据自检,不恢复文件。 |
| `--dry-run` | 只打印恢复计划,不复制文件。 |
| `--limit N` | 本次最多恢复 N 个新 Task。已跳过的 Task 不计入数量。 |
| `--overwrite` | 删除并重建已有 Task。 |
| `--skip-existing` | 跳过已完成 Task,默认开启。通常不需要手动设置。 |
`--overwrite` 和显式设置的 `--skip-existing` 不能同时使用。
## 输出文件
正式恢复后,输出目录会生成:
- `restore.log`:人类可读日志。
- `restore_report.json`:机器可读恢复报告。
- 每个 `Task_*` 内的 `.dinsar_pair.json`:该 Task 对应的 pair 元信息。
`restore_report.json` 示例:
```json
{
"started_at": "2026-05-11T00:00:00Z",
"finished_at": "2026-05-11T00:10:00Z",
"input_root": "D:\\BundleRoot",
"output_root": "D:\\OutputRoot",
"total_pairs": 1000,
"restored": 200,
"skipped": 0,
"failed": 0,
"warnings": []
}
```
## 重新编译
需要 Go 1.22 或更高版本。
图形界面版:
```powershell
go build -ldflags "-H=windowsgui" -o dinsar-restore.exe .
```
命令行版:
```powershell
go build -o dinsar-restore-console.exe .
```
运行测试:
```powershell
go test ./...
```
## 发布建议
建议 GitHub Release 附带以下文件:
- `dinsar-restore.exe`
- `dinsar-restore-console.exe`
- `README.md`
Release 标题示例:
```text
D-InSAR Restore Tool v0.1.0
```
Release 说明可写:
```text
Initial release.
- Add Windows GUI restore tool.
- Add console restore tool.
- Add data self-check before restore.
- Support batch continuation with Limit.
- Generate restore_report.json and restore.log.
```
+3
View File
@@ -0,0 +1,3 @@
module dinsar-restore
go 1.22
+9
View File
@@ -0,0 +1,9 @@
//go:build !windows
package main
import "errors"
func runGUI() error {
return errors.New("GUI mode is only supported on Windows")
}
+591
View File
@@ -0,0 +1,591 @@
//go:build windows
package main
import (
"fmt"
"io"
"runtime"
"strconv"
"strings"
"sync"
"syscall"
"unsafe"
)
const (
windowTitle = "D-InSAR 数据包恢复工具"
wmDestroy = 0x0002
wmCommand = 0x0111
wmClose = 0x0010
wmSetFont = 0x0030
bnClicked = 0
wsOverlappedWindow = 0x00CF0000
wsVisible = 0x10000000
wsChild = 0x40000000
wsTabStop = 0x00010000
wsBorder = 0x00800000
wsVScroll = 0x00200000
wsHScroll = 0x00100000
esLeft = 0x0000
esMultiline = 0x0004
esAutoVScroll = 0x0040
esAutoHScroll = 0x0080
esReadOnly = 0x0800
bsPushButton = 0x00000000
bsAutoCheckBox = 0x00000003
swShow = 5
cwUseDefault = 0x80000000
gwlUserData = -21
idInputEdit = 1001
idInputBrowse = 1002
idOutputEdit = 1003
idOutputBrowse = 1004
idDryRun = 1005
idOverwrite = 1006
idLimitEdit = 1007
idStart = 1008
idLogEdit = 1009
idCheck = 1010
bmGetCheck = 0x00F0
bstChecked = 1
maxPathBuffer = 4096
)
var (
user32 = syscall.NewLazyDLL("user32.dll")
kernel32 = syscall.NewLazyDLL("kernel32.dll")
gdi32 = syscall.NewLazyDLL("gdi32.dll")
shell32 = syscall.NewLazyDLL("shell32.dll")
ole32 = syscall.NewLazyDLL("ole32.dll")
procRegisterClassExW = user32.NewProc("RegisterClassExW")
procCreateWindowExW = user32.NewProc("CreateWindowExW")
procDefWindowProcW = user32.NewProc("DefWindowProcW")
procDestroyWindow = user32.NewProc("DestroyWindow")
procDispatchMessageW = user32.NewProc("DispatchMessageW")
procGetMessageW = user32.NewProc("GetMessageW")
procPostQuitMessage = user32.NewProc("PostQuitMessage")
procSendMessageW = user32.NewProc("SendMessageW")
procShowWindow = user32.NewProc("ShowWindow")
procUpdateWindow = user32.NewProc("UpdateWindow")
procSetWindowTextW = user32.NewProc("SetWindowTextW")
procGetWindowTextW = user32.NewProc("GetWindowTextW")
procGetWindowTextLenW = user32.NewProc("GetWindowTextLengthW")
procEnableWindow = user32.NewProc("EnableWindow")
procSetWindowLongPtrW = user32.NewProc("SetWindowLongPtrW")
procGetWindowLongPtrW = user32.NewProc("GetWindowLongPtrW")
procMessageBoxW = user32.NewProc("MessageBoxW")
procGetModuleHandleW = kernel32.NewProc("GetModuleHandleW")
procCreateFontW = gdi32.NewProc("CreateFontW")
procSHBrowseForFolderW = shell32.NewProc("SHBrowseForFolderW")
procSHGetPathFromIDListW = shell32.NewProc("SHGetPathFromIDListW")
procCoTaskMemFree = ole32.NewProc("CoTaskMemFree")
procOleInitialize = ole32.NewProc("OleInitialize")
procOleUninitialize = ole32.NewProc("OleUninitialize")
)
type point struct {
x int32
y int32
}
type msg struct {
hwnd uintptr
message uint32
wParam uintptr
lParam uintptr
time uint32
pt point
}
type wndClassEx struct {
cbSize uint32
style uint32
lpfnWndProc uintptr
cbClsExtra int32
cbWndExtra int32
hInstance uintptr
hIcon uintptr
hCursor uintptr
hbrBackground uintptr
lpszMenuName *uint16
lpszClassName *uint16
hIconSm uintptr
}
type browseInfo struct {
hwndOwner uintptr
pidlRoot uintptr
pszDisplayName *uint16
lpszTitle *uint16
ulFlags uint32
lpfn uintptr
lParam uintptr
iImage int32
}
type guiApp struct {
hwnd uintptr
font uintptr
inputEdit uintptr
outputEdit uintptr
dryRun uintptr
overwrite uintptr
limitEdit uintptr
startButton uintptr
checkButton uintptr
logEdit uintptr
mu sync.Mutex
logText string
running bool
}
func runGUI() error {
runtime.LockOSThread()
defer runtime.UnlockOSThread()
procOleInitialize.Call(0)
defer procOleUninitialize.Call()
instance, _, _ := procGetModuleHandleW.Call(0)
className, _ := syscall.UTF16PtrFromString("DinsarRestoreWindow")
title, _ := syscall.UTF16PtrFromString(windowTitle)
wc := wndClassEx{
cbSize: uint32(unsafe.Sizeof(wndClassEx{})),
lpfnWndProc: syscall.NewCallback(windowProc),
hInstance: instance,
hbrBackground: 5 + 1,
lpszClassName: className,
}
if atom, _, err := procRegisterClassExW.Call(uintptr(unsafe.Pointer(&wc))); atom == 0 {
return fmt.Errorf("RegisterClassExW failed: %v", err)
}
app := &guiApp{}
hwnd, _, err := procCreateWindowExW.Call(
0,
uintptr(unsafe.Pointer(className)),
uintptr(unsafe.Pointer(title)),
wsOverlappedWindow|wsVisible,
cwUseDefault,
cwUseDefault,
1040,
720,
0,
0,
instance,
uintptr(unsafe.Pointer(app)),
)
if hwnd == 0 {
return fmt.Errorf("CreateWindowExW failed: %v", err)
}
app.hwnd = hwnd
procShowWindow.Call(hwnd, swShow)
procUpdateWindow.Call(hwnd)
var m msg
for {
ret, _, _ := procGetMessageW.Call(uintptr(unsafe.Pointer(&m)), 0, 0, 0)
if int32(ret) <= 0 {
break
}
procDispatchMessageW.Call(uintptr(unsafe.Pointer(&m)))
}
return nil
}
func windowProc(hwnd uintptr, msg uint32, wParam uintptr, lParam uintptr) uintptr {
app := appFromWindow(hwnd)
switch msg {
case wmClose:
if app != nil && app.isRunning() {
messageBox(hwnd, "任务正在运行,请等待结束。", windowTitle)
return 0
}
procDestroyWindow.Call(hwnd)
return 0
case wmDestroy:
procPostQuitMessage.Call(0)
return 0
case wmCommand:
id := int(wParam & 0xffff)
notify := int((wParam >> 16) & 0xffff)
if notify == bnClicked && app != nil {
switch id {
case idInputBrowse:
if selected := chooseDirectory(hwnd, "选择 BundleRoot 输入目录"); selected != "" {
setWindowText(app.inputEdit, selected)
}
case idOutputBrowse:
if selected := chooseDirectory(hwnd, "选择 OutputRoot 输出目录"); selected != "" {
setWindowText(app.outputEdit, selected)
}
case idStart:
app.startRestore()
case idCheck:
app.startCheck()
}
}
return 0
case 0x0081: // WM_NCCREATE
cs := (*createStruct)(unsafe.Pointer(lParam))
procSetWindowLongPtrW.Call(hwnd, winIndex(gwlUserData), cs.createParams)
return 1
case 0x0001: // WM_CREATE
app = appFromWindow(hwnd)
if app != nil {
app.hwnd = hwnd
app.createControls()
}
return 0
}
ret, _, _ := procDefWindowProcW.Call(hwnd, uintptr(msg), wParam, lParam)
return ret
}
type createStruct struct {
createParams uintptr
instance uintptr
menu uintptr
parent uintptr
cy int32
cx int32
y int32
x int32
style int32
name *uint16
class *uint16
exStyle uint32
}
func appFromWindow(hwnd uintptr) *guiApp {
if hwnd == 0 {
return nil
}
ptr, _, _ := procGetWindowLongPtrW.Call(hwnd, winIndex(gwlUserData))
if ptr == 0 {
return nil
}
return (*guiApp)(unsafe.Pointer(ptr))
}
func (a *guiApp) createControls() {
a.font = createFont("Microsoft YaHei UI", 20)
createLabel(a.hwnd, "输入目录 BundleRoot", 24, 28, 210, 30, a.font)
a.inputEdit = createEdit(a.hwnd, "", idInputEdit, 240, 24, 640, 34, false, a.font)
createButton(a.hwnd, "选择...", idInputBrowse, 895, 24, 100, 34, a.font)
createLabel(a.hwnd, "输出目录 OutputRoot", 24, 78, 210, 30, a.font)
a.outputEdit = createEdit(a.hwnd, "", idOutputEdit, 240, 74, 640, 34, false, a.font)
createButton(a.hwnd, "选择...", idOutputBrowse, 895, 74, 100, 34, a.font)
a.dryRun = createCheckBox(a.hwnd, "Dry run 只打印计划", idDryRun, 240, 128, 230, 32, a.font)
a.overwrite = createCheckBox(a.hwnd, "覆盖已有 Task", idOverwrite, 500, 128, 190, 32, a.font)
createLabel(a.hwnd, "Limit", 720, 130, 60, 30, a.font)
a.limitEdit = createEdit(a.hwnd, "", idLimitEdit, 780, 126, 100, 34, false, a.font)
a.checkButton = createButton(a.hwnd, "数据自检", idCheck, 760, 174, 105, 38, a.font)
a.startButton = createButton(a.hwnd, "开始恢复", idStart, 890, 174, 105, 38, a.font)
createLabel(a.hwnd, "运行日志", 24, 226, 140, 30, a.font)
a.logEdit = createEdit(a.hwnd, "", idLogEdit, 24, 262, 970, 365, true, a.font)
a.appendLog("请选择输入目录后可先数据自检;选择输出目录后可开始恢复。\r\n")
}
func (a *guiApp) startRestore() {
if a.isRunning() {
return
}
input := strings.TrimSpace(getWindowText(a.inputEdit))
output := strings.TrimSpace(getWindowText(a.outputEdit))
if input == "" {
messageBox(a.hwnd, "请选择输入目录 BundleRoot。", windowTitle)
return
}
if output == "" {
messageBox(a.hwnd, "请选择输出目录 OutputRoot。", windowTitle)
return
}
limit, ok := a.parseLimit()
if !ok {
return
}
cfg := config{
inputRoot: input,
outputRoot: output,
skipExisting: true,
overwrite: isChecked(a.overwrite),
limit: limit,
dryRun: isChecked(a.dryRun),
logWriter: guiLogWriter{app: a},
}
if cfg.overwrite {
cfg.skipExisting = false
}
a.setRunning(true)
a.setLog("")
a.appendLog("开始执行...\r\n")
go func() {
result, err := run(cfg)
if err != nil {
a.appendLog("错误: " + err.Error() + "\r\n")
messageBox(a.hwnd, "执行失败:\r\n"+err.Error(), windowTitle)
} else {
a.appendLog(fmt.Sprintf("完成: restored=%d skipped=%d failed=%d warnings=%d\r\n",
result.report.Restored,
result.report.Skipped,
result.report.Failed,
len(result.report.Warnings),
))
messageBox(a.hwnd, "执行完成。", windowTitle)
}
a.setRunning(false)
}()
}
func (a *guiApp) startCheck() {
if a.isRunning() {
return
}
input := strings.TrimSpace(getWindowText(a.inputEdit))
if input == "" {
messageBox(a.hwnd, "请选择输入目录 BundleRoot。", windowTitle)
return
}
limit, ok := a.parseLimit()
if !ok {
return
}
cfg := config{
inputRoot: input,
limit: limit,
checkOnly: true,
logWriter: guiLogWriter{app: a},
}
a.setRunning(true)
a.setLog("")
a.appendLog("开始数据自检...\r\n")
go func() {
result, err := runCheck(cfg)
if err != nil {
a.appendLog("错误: " + err.Error() + "\r\n")
messageBox(a.hwnd, "自检失败:\r\n"+err.Error(), windowTitle)
} else if result.report.Failed > 0 {
a.appendLog(fmt.Sprintf("自检完成: total=%d checked=%d valid=%d failed=%d warnings=%d\r\n",
result.report.TotalPairs,
result.report.Checked,
result.report.Valid,
result.report.Failed,
len(result.report.Warnings),
))
messageBox(a.hwnd, fmt.Sprintf("自检完成,但发现 %d 个失败项。请查看日志。", result.report.Failed), windowTitle)
} else {
a.appendLog(fmt.Sprintf("自检通过: total=%d checked=%d valid=%d warnings=%d\r\n",
result.report.TotalPairs,
result.report.Checked,
result.report.Valid,
len(result.report.Warnings),
))
messageBox(a.hwnd, "自检通过。", windowTitle)
}
a.setRunning(false)
}()
}
func (a *guiApp) parseLimit() (int, bool) {
limitText := strings.TrimSpace(getWindowText(a.limitEdit))
if limitText == "" {
return 0, true
}
parsed, err := strconv.Atoi(limitText)
if err != nil || parsed < 0 {
messageBox(a.hwnd, "Limit 必须是大于等于 0 的整数。", windowTitle)
return 0, false
}
return parsed, true
}
func (a *guiApp) isRunning() bool {
a.mu.Lock()
defer a.mu.Unlock()
return a.running
}
func (a *guiApp) setRunning(running bool) {
a.mu.Lock()
a.running = running
a.mu.Unlock()
enable := uintptr(1)
if running {
enable = 0
}
procEnableWindow.Call(a.startButton, enable)
procEnableWindow.Call(a.checkButton, enable)
procEnableWindow.Call(a.inputEdit, enable)
procEnableWindow.Call(a.outputEdit, enable)
procEnableWindow.Call(a.dryRun, enable)
procEnableWindow.Call(a.overwrite, enable)
procEnableWindow.Call(a.limitEdit, enable)
}
func (a *guiApp) setLog(text string) {
a.mu.Lock()
a.logText = text
a.mu.Unlock()
setWindowText(a.logEdit, text)
}
func (a *guiApp) appendLog(text string) {
a.mu.Lock()
a.logText += text
fullText := a.logText
a.mu.Unlock()
setWindowText(a.logEdit, fullText)
}
type guiLogWriter struct {
app *guiApp
}
func (w guiLogWriter) Write(p []byte) (int, error) {
if w.app != nil {
w.app.appendLog(strings.ReplaceAll(string(p), "\n", "\r\n"))
}
return len(p), nil
}
func createLabel(parent uintptr, text string, x, y, width, height int32, font uintptr) uintptr {
return createControl("STATIC", text, wsChild|wsVisible, 0, parent, 0, x, y, width, height, font)
}
func createEdit(parent uintptr, text string, id int, x, y, width, height int32, multiline bool, font uintptr) uintptr {
style := uint32(wsChild | wsVisible | wsBorder | wsTabStop | esLeft | esAutoHScroll)
if multiline {
style = wsChild | wsVisible | wsBorder | wsVScroll | wsHScroll | esLeft | esMultiline | esAutoVScroll | esAutoHScroll | esReadOnly
}
return createControl("EDIT", text, style, 0, parent, id, x, y, width, height, font)
}
func createButton(parent uintptr, text string, id int, x, y, width, height int32, font uintptr) uintptr {
return createControl("BUTTON", text, wsChild|wsVisible|wsTabStop|bsPushButton, 0, parent, id, x, y, width, height, font)
}
func createCheckBox(parent uintptr, text string, id int, x, y, width, height int32, font uintptr) uintptr {
return createControl("BUTTON", text, wsChild|wsVisible|wsTabStop|bsAutoCheckBox, 0, parent, id, x, y, width, height, font)
}
func createControl(className string, text string, style uint32, exStyle uint32, parent uintptr, id int, x, y, width, height int32, font uintptr) uintptr {
classPtr, _ := syscall.UTF16PtrFromString(className)
textPtr, _ := syscall.UTF16PtrFromString(text)
hwnd, _, _ := procCreateWindowExW.Call(
uintptr(exStyle),
uintptr(unsafe.Pointer(classPtr)),
uintptr(unsafe.Pointer(textPtr)),
uintptr(style),
uintptr(x),
uintptr(y),
uintptr(width),
uintptr(height),
parent,
uintptr(id),
0,
0,
)
if hwnd != 0 && font != 0 {
procSendMessageW.Call(hwnd, wmSetFont, font, 1)
}
return hwnd
}
func createFont(name string, size int32) uintptr {
namePtr, _ := syscall.UTF16PtrFromString(name)
font, _, _ := procCreateFontW.Call(
uintptr(-size),
0,
0,
0,
400,
0,
0,
0,
1,
0,
0,
0,
0,
uintptr(unsafe.Pointer(namePtr)),
)
return font
}
func setWindowText(hwnd uintptr, text string) {
ptr, _ := syscall.UTF16PtrFromString(text)
procSetWindowTextW.Call(hwnd, uintptr(unsafe.Pointer(ptr)))
}
func getWindowText(hwnd uintptr) string {
length, _, _ := procGetWindowTextLenW.Call(hwnd)
buffer := make([]uint16, int(length)+1)
procGetWindowTextW.Call(hwnd, uintptr(unsafe.Pointer(&buffer[0])), uintptr(len(buffer)))
return syscall.UTF16ToString(buffer)
}
func isChecked(hwnd uintptr) bool {
ret, _, _ := procSendMessageW.Call(hwnd, bmGetCheck, 0, 0)
return ret == bstChecked
}
func messageBox(hwnd uintptr, text string, title string) {
textPtr, _ := syscall.UTF16PtrFromString(text)
titlePtr, _ := syscall.UTF16PtrFromString(title)
procMessageBoxW.Call(hwnd, uintptr(unsafe.Pointer(textPtr)), uintptr(unsafe.Pointer(titlePtr)), 0)
}
func chooseDirectory(hwnd uintptr, title string) string {
titlePtr, _ := syscall.UTF16PtrFromString(title)
displayName := make([]uint16, maxPathBuffer)
bi := browseInfo{
hwndOwner: hwnd,
pszDisplayName: &displayName[0],
lpszTitle: titlePtr,
ulFlags: 0x00000001 | 0x00000010 | 0x00000040,
}
pidl, _, _ := procSHBrowseForFolderW.Call(uintptr(unsafe.Pointer(&bi)))
if pidl == 0 {
return ""
}
defer procCoTaskMemFree.Call(pidl)
pathBuffer := make([]uint16, maxPathBuffer)
ret, _, _ := procSHGetPathFromIDListW.Call(pidl, uintptr(unsafe.Pointer(&pathBuffer[0])))
if ret == 0 {
return ""
}
return syscall.UTF16ToString(pathBuffer)
}
func winIndex(value int32) uintptr {
return uintptr(value)
}
var _ io.Writer = guiLogWriter{}
+847
View File
@@ -0,0 +1,847 @@
package main
import (
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"log"
"os"
"path/filepath"
"runtime"
"strings"
"time"
)
const (
pairsFileName = "pairs.json"
reportFileName = "restore_report.json"
logFileName = "restore.log"
pairMetaName = ".dinsar_pair.json"
tempSuffix = ".tmp"
)
type config struct {
inputRoot string
outputRoot string
skipExisting bool
overwrite bool
limit int
dryRun bool
checkOnly bool
logWriter io.Writer
}
type pairsDocument struct {
Schema string `json:"schema"`
ExportedAt string `json:"exported_at"`
Pairs []pair `json:"pairs"`
}
type pair struct {
PairID string `json:"pair_id"`
TaskName string `json:"task_name"`
TaskAlias string `json:"task_alias"`
MasterData string `json:"master_data"`
SlaveData string `json:"slave_data"`
MasterOrbit string `json:"master_orbit"`
SlaveOrbit string `json:"slave_orbit"`
MasterImagingDate string `json:"master_imaging_date"`
SlaveImagingDate string `json:"slave_imaging_date"`
TimeBaselineDays int `json:"time_baseline_days"`
}
type pairMetadata struct {
PairID string `json:"pair_id"`
TaskName string `json:"task_name"`
TaskAlias string `json:"task_alias"`
MasterData string `json:"master_data"`
SlaveData string `json:"slave_data"`
MasterOrbit string `json:"master_orbit"`
SlaveOrbit string `json:"slave_orbit"`
MasterImagingDate string `json:"master_imaging_date"`
SlaveImagingDate string `json:"slave_imaging_date"`
TimeBaselineDays int `json:"time_baseline_days"`
RestoredAt string `json:"restored_at"`
}
type restoreReport struct {
StartedAt string `json:"started_at"`
FinishedAt string `json:"finished_at"`
InputRoot string `json:"input_root"`
OutputRoot string `json:"output_root"`
TotalPairs int `json:"total_pairs"`
Restored int `json:"restored"`
Skipped int `json:"skipped"`
Failed int `json:"failed"`
Warnings []string `json:"warnings"`
}
type restoreResult struct {
report restoreReport
}
type checkReport struct {
StartedAt string `json:"started_at"`
FinishedAt string `json:"finished_at"`
InputRoot string `json:"input_root"`
TotalPairs int `json:"total_pairs"`
Checked int `json:"checked"`
Valid int `json:"valid"`
Failed int `json:"failed"`
Warnings []string `json:"warnings"`
Errors []string `json:"errors"`
}
type checkResult struct {
report checkReport
}
func main() {
if len(os.Args) == 1 && runtime.GOOS == "windows" {
if err := runGUI(); err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
return
}
cfg, err := parseFlags(os.Args[1:])
if err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(2)
}
if cfg.checkOnly {
result, err := runCheck(cfg)
if err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
fmt.Printf("check: total=%d checked=%d valid=%d failed=%d warnings=%d\n",
result.report.TotalPairs,
result.report.Checked,
result.report.Valid,
result.report.Failed,
len(result.report.Warnings),
)
if result.report.Failed > 0 {
os.Exit(1)
}
return
}
result, err := run(cfg)
if err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
fmt.Printf("done: restored=%d skipped=%d failed=%d warnings=%d\n",
result.report.Restored,
result.report.Skipped,
result.report.Failed,
len(result.report.Warnings),
)
}
func parseFlags(args []string) (config, error) {
var cfg config
fs := flag.NewFlagSet("dinsar-restore", flag.ContinueOnError)
fs.SetOutput(io.Discard)
fs.StringVar(&cfg.inputRoot, "input", "", "source bundle root")
fs.StringVar(&cfg.outputRoot, "output", "", "restored output root")
fs.BoolVar(&cfg.skipExisting, "skip-existing", true, "skip existing completed tasks")
fs.BoolVar(&cfg.overwrite, "overwrite", false, "delete and rebuild existing task directories")
fs.IntVar(&cfg.limit, "limit", 0, "maximum number of pairs to restore")
fs.BoolVar(&cfg.dryRun, "dry-run", false, "print plan without copying files")
fs.BoolVar(&cfg.checkOnly, "check-only", false, "validate the source bundle without restoring")
if err := fs.Parse(args); err != nil {
return cfg, err
}
explicitFlags := map[string]bool{}
fs.Visit(func(f *flag.Flag) {
explicitFlags[f.Name] = true
})
if cfg.inputRoot == "" {
return cfg, errors.New("missing --input")
}
if cfg.outputRoot == "" && !cfg.checkOnly {
return cfg, errors.New("missing --output")
}
if cfg.overwrite && explicitFlags["skip-existing"] {
return cfg, errors.New("--skip-existing and --overwrite cannot be used together")
}
if cfg.overwrite {
cfg.skipExisting = false
}
if cfg.limit < 0 {
return cfg, errors.New("--limit must be zero or greater")
}
return cfg, nil
}
func run(cfg config) (restoreResult, error) {
started := time.Now().UTC()
inputRoot, err := filepath.Abs(cfg.inputRoot)
if err != nil {
return restoreResult{}, fmt.Errorf("resolve input root: %w", err)
}
outputRoot, err := filepath.Abs(cfg.outputRoot)
if err != nil {
return restoreResult{}, fmt.Errorf("resolve output root: %w", err)
}
report := restoreReport{
StartedAt: started.Format(time.RFC3339),
InputRoot: inputRoot,
OutputRoot: outputRoot,
Warnings: []string{},
}
pairsDoc, err := loadAndValidateInput(inputRoot)
if err != nil {
return restoreResult{}, err
}
report.TotalPairs = len(pairsDoc.Pairs)
if !cfg.dryRun {
if err := os.MkdirAll(outputRoot, 0755); err != nil {
return restoreResult{}, fmt.Errorf("create output root: %w", err)
}
}
logOutput := defaultLogWriter(cfg.logWriter)
logger, logFile, err := newLogger(outputRoot, cfg.dryRun, logOutput)
if err != nil {
return restoreResult{}, err
}
if logFile != nil {
defer logFile.Close()
}
plannedPairs := pairsDoc.Pairs
logger.Printf("started input=%s output=%s total_pairs=%d planned_pairs=%d dry_run=%t",
inputRoot, outputRoot, report.TotalPairs, len(plannedPairs), cfg.dryRun)
limitCount := 0
for i, p := range plannedPairs {
if cfg.limit > 0 && limitCount >= cfg.limit {
logger.Printf("limit reached processed=%d limit=%d", limitCount, cfg.limit)
break
}
issues := checkPair(inputRoot, p, false)
if len(issues.errors) > 0 {
report.Failed++
for _, item := range issues.errors {
logger.Printf("failed pair_index=%d pair_id=%s error=%s", i, p.PairID, item)
}
continue
}
warnings, err := restorePair(cfg, inputRoot, outputRoot, p, logger)
report.Warnings = append(report.Warnings, warnings...)
if err != nil {
if errors.Is(err, errSkipped) {
report.Skipped++
continue
}
report.Failed++
logger.Printf("failed pair_index=%d pair_id=%s error=%v", i, p.PairID, err)
continue
}
limitCount++
if !cfg.dryRun {
report.Restored++
}
}
report.FinishedAt = time.Now().UTC().Format(time.RFC3339)
logger.Printf("finished restored=%d skipped=%d failed=%d warnings=%d",
report.Restored, report.Skipped, report.Failed, len(report.Warnings))
if !cfg.dryRun {
if err := writeJSON(filepath.Join(outputRoot, reportFileName), report); err != nil {
return restoreResult{}, fmt.Errorf("write report: %w", err)
}
} else {
if err := json.NewEncoder(logOutput).Encode(report); err != nil {
return restoreResult{}, fmt.Errorf("print dry-run report: %w", err)
}
}
return restoreResult{report: report}, nil
}
func runCheck(cfg config) (checkResult, error) {
started := time.Now().UTC()
inputRoot, err := filepath.Abs(cfg.inputRoot)
if err != nil {
return checkResult{}, fmt.Errorf("resolve input root: %w", err)
}
report := checkReport{
StartedAt: started.Format(time.RFC3339),
InputRoot: inputRoot,
Warnings: []string{},
Errors: []string{},
}
logOutput := defaultLogWriter(cfg.logWriter)
logger := log.New(logOutput, "", log.LstdFlags)
logger.Printf("check started input=%s", inputRoot)
pairsDoc, err := loadAndValidateInput(inputRoot)
if err != nil {
report.Failed = 1
report.Errors = append(report.Errors, err.Error())
report.FinishedAt = time.Now().UTC().Format(time.RFC3339)
printCheckReport(logOutput, report)
return checkResult{report: report}, nil
}
report.TotalPairs = len(pairsDoc.Pairs)
plannedPairs := pairsDoc.Pairs
if cfg.limit > 0 && cfg.limit < len(plannedPairs) {
plannedPairs = plannedPairs[:cfg.limit]
}
seenTasks := map[string]int{}
for i, p := range plannedPairs {
report.Checked++
issues := checkPair(inputRoot, p, true)
taskName, taskNameErr := taskDirectoryName(p)
if taskNameErr == nil {
if firstIndex, exists := seenTasks[taskName]; exists {
issues.errors = append(issues.errors, fmt.Sprintf("duplicate task directory %s also used by pair index %d", taskName, firstIndex))
} else {
seenTasks[taskName] = i
}
}
if len(issues.warnings) > 0 {
report.Warnings = append(report.Warnings, issues.warnings...)
for _, warning := range issues.warnings {
logger.Printf("warning pair_index=%d pair_id=%s %s", i, p.PairID, warning)
}
}
if len(issues.errors) > 0 {
report.Failed++
for _, item := range issues.errors {
message := fmt.Sprintf("pair_index=%d pair_id=%s %s", i, p.PairID, item)
report.Errors = append(report.Errors, message)
logger.Printf("error %s", message)
}
continue
}
report.Valid++
logger.Printf("valid pair_index=%d pair_id=%s task=%s", i, p.PairID, taskName)
}
report.FinishedAt = time.Now().UTC().Format(time.RFC3339)
logger.Printf("check finished total=%d checked=%d valid=%d failed=%d warnings=%d",
report.TotalPairs, report.Checked, report.Valid, report.Failed, len(report.Warnings))
printCheckReport(logOutput, report)
return checkResult{report: report}, nil
}
type pairCheckIssues struct {
warnings []string
errors []string
}
func checkPair(inputRoot string, p pair, includeOrbitWarnings bool) pairCheckIssues {
var issues pairCheckIssues
if _, err := taskDirectoryName(p); err != nil {
issues.errors = append(issues.errors, err.Error())
}
if err := validatePairPaths(inputRoot, p); err != nil {
issues.errors = append(issues.errors, err.Error())
return issues
}
masterSource, _ := safeJoin(inputRoot, p.MasterData)
slaveSource, _ := safeJoin(inputRoot, p.SlaveData)
if err := requireDir(masterSource, "master_data"); err != nil {
issues.errors = append(issues.errors, err.Error())
} else if nonEmpty, err := dirNonEmpty(masterSource); err != nil {
issues.errors = append(issues.errors, fmt.Sprintf("check master_data contents: %v", err))
} else if !nonEmpty {
issues.errors = append(issues.errors, fmt.Sprintf("master_data directory is empty: %s", p.MasterData))
}
if err := requireDir(slaveSource, "slave_data"); err != nil {
issues.errors = append(issues.errors, err.Error())
} else if nonEmpty, err := dirNonEmpty(slaveSource); err != nil {
issues.errors = append(issues.errors, fmt.Sprintf("check slave_data contents: %v", err))
} else if !nonEmpty {
issues.errors = append(issues.errors, fmt.Sprintf("slave_data directory is empty: %s", p.SlaveData))
}
if includeOrbitWarnings {
issues.warnings = append(issues.warnings, checkOrbitWarnings(inputRoot, p)...)
}
return issues
}
func checkOrbitWarnings(inputRoot string, p pair) []string {
taskName, err := taskDirectoryName(p)
if err != nil {
taskName = pairLabel(p)
}
return orbitWarnings(inputRoot, p, taskName)
}
func printCheckReport(writer io.Writer, report checkReport) {
if writer == nil {
return
}
encoder := json.NewEncoder(writer)
encoder.SetIndent("", " ")
_ = encoder.Encode(report)
}
func loadAndValidateInput(inputRoot string) (pairsDocument, error) {
if stat, err := os.Stat(inputRoot); err != nil {
return pairsDocument{}, fmt.Errorf("input root is not accessible: %w", err)
} else if !stat.IsDir() {
return pairsDocument{}, fmt.Errorf("input root is not a directory: %s", inputRoot)
}
dataDir := filepath.Join(inputRoot, "data")
if stat, err := os.Stat(dataDir); err != nil {
return pairsDocument{}, fmt.Errorf("data directory is not accessible: %w", err)
} else if !stat.IsDir() {
return pairsDocument{}, fmt.Errorf("data path is not a directory: %s", dataDir)
}
pairsPath := filepath.Join(inputRoot, pairsFileName)
file, err := os.Open(pairsPath)
if err != nil {
return pairsDocument{}, fmt.Errorf("open pairs.json: %w", err)
}
defer file.Close()
var doc pairsDocument
decoder := json.NewDecoder(file)
decoder.DisallowUnknownFields()
if err := decoder.Decode(&doc); err != nil {
return pairsDocument{}, fmt.Errorf("parse pairs.json: %w", err)
}
if len(doc.Pairs) == 0 {
return pairsDocument{}, errors.New("pairs.json contains no pairs")
}
return doc, nil
}
func validatePairPaths(inputRoot string, p pair) error {
if strings.TrimSpace(p.MasterData) == "" {
return fmt.Errorf("pair %s missing master_data", pairLabel(p))
}
if strings.TrimSpace(p.SlaveData) == "" {
return fmt.Errorf("pair %s missing slave_data", pairLabel(p))
}
if _, err := safeJoin(inputRoot, p.MasterData); err != nil {
return fmt.Errorf("pair %s invalid master_data: %w", pairLabel(p), err)
}
if _, err := safeJoin(inputRoot, p.SlaveData); err != nil {
return fmt.Errorf("pair %s invalid slave_data: %w", pairLabel(p), err)
}
if strings.TrimSpace(p.MasterOrbit) != "" {
if _, err := safeJoin(inputRoot, p.MasterOrbit); err != nil {
return fmt.Errorf("pair %s invalid master_orbit: %w", pairLabel(p), err)
}
}
if strings.TrimSpace(p.SlaveOrbit) != "" {
if _, err := safeJoin(inputRoot, p.SlaveOrbit); err != nil {
return fmt.Errorf("pair %s invalid slave_orbit: %w", pairLabel(p), err)
}
}
return nil
}
var errSkipped = errors.New("skipped")
func restorePair(cfg config, inputRoot string, outputRoot string, p pair, logger *log.Logger) ([]string, error) {
taskDirName, err := taskDirectoryName(p)
if err != nil {
return nil, err
}
taskDir := filepath.Join(outputRoot, taskDirName)
if !isSubpath(outputRoot, taskDir) {
return nil, fmt.Errorf("task directory escapes output root: %s", taskDirName)
}
tempDir := taskDir + tempSuffix
masterSource, _ := safeJoin(inputRoot, p.MasterData)
slaveSource, _ := safeJoin(inputRoot, p.SlaveData)
if err := requireDir(masterSource, "master_data"); err != nil {
return nil, err
}
if err := requireDir(slaveSource, "slave_data"); err != nil {
return nil, err
}
existingComplete, err := taskComplete(taskDir)
if err != nil {
return nil, err
}
if existingComplete && cfg.skipExisting {
logger.Printf("skip existing task=%s", taskDirName)
return nil, errSkipped
}
if cfg.dryRun {
logger.Printf("dry-run restore task=%s master=%s slave=%s", taskDirName, p.MasterData, p.SlaveData)
return orbitWarnings(inputRoot, p, taskDirName), nil
}
if pathExists(taskDir) && !cfg.overwrite {
return nil, fmt.Errorf("task already exists and is not complete; use --overwrite or remove it: %s", taskDir)
}
if cfg.overwrite {
if err := os.RemoveAll(taskDir); err != nil {
return nil, fmt.Errorf("remove existing task %s: %w", taskDir, err)
}
}
if pathExists(tempDir) {
if err := os.RemoveAll(tempDir); err != nil {
return nil, fmt.Errorf("remove stale temp directory %s: %w", tempDir, err)
}
}
logger.Printf("restore task=%s", taskDirName)
if err := os.MkdirAll(tempDir, 0755); err != nil {
return nil, fmt.Errorf("create temp task directory: %w", err)
}
cleanupTemp := true
defer func() {
if cleanupTemp {
_ = os.RemoveAll(tempDir)
}
}()
if err := copyDirContents(masterSource, filepath.Join(tempDir, "master")); err != nil {
return nil, fmt.Errorf("copy master data: %w", err)
}
if err := copyDirContents(slaveSource, filepath.Join(tempDir, "slave")); err != nil {
return nil, fmt.Errorf("copy slave data: %w", err)
}
warnings := copyOrbitFiles(inputRoot, tempDir, taskDirName, p, logger)
meta := pairMetadata{
PairID: p.PairID,
TaskName: p.TaskName,
TaskAlias: p.TaskAlias,
MasterData: p.MasterData,
SlaveData: p.SlaveData,
MasterOrbit: p.MasterOrbit,
SlaveOrbit: p.SlaveOrbit,
MasterImagingDate: p.MasterImagingDate,
SlaveImagingDate: p.SlaveImagingDate,
TimeBaselineDays: p.TimeBaselineDays,
RestoredAt: time.Now().UTC().Format(time.RFC3339),
}
if err := writeJSON(filepath.Join(tempDir, pairMetaName), meta); err != nil {
return warnings, fmt.Errorf("write pair metadata: %w", err)
}
if err := validateRestoredTask(tempDir); err != nil {
return warnings, err
}
if err := os.Rename(tempDir, taskDir); err != nil {
return warnings, fmt.Errorf("publish restored task: %w", err)
}
cleanupTemp = false
return warnings, nil
}
func taskDirectoryName(p pair) (string, error) {
for _, candidate := range []string{p.TaskAlias, p.TaskName, p.PairID} {
name := strings.TrimSpace(candidate)
if name == "" {
continue
}
if name != filepath.Base(name) || strings.ContainsAny(name, `/\:`) {
return "", fmt.Errorf("invalid task directory name: %q", name)
}
return name, nil
}
return "", errors.New("pair has no task_alias, task_name, or pair_id")
}
func copyOrbitFiles(inputRoot string, tempDir string, taskDirName string, p pair, logger *log.Logger) []string {
var warnings []string
for _, entry := range []struct {
label string
value string
}{
{label: "master_orbit", value: p.MasterOrbit},
{label: "slave_orbit", value: p.SlaveOrbit},
} {
if strings.TrimSpace(entry.value) == "" {
continue
}
source, err := safeJoin(inputRoot, entry.value)
if err != nil {
warning := fmt.Sprintf("task %s %s invalid: %v", taskDirName, entry.label, err)
warnings = append(warnings, warning)
logger.Printf("warning %s", warning)
continue
}
stat, err := os.Stat(source)
if err != nil {
warning := fmt.Sprintf("task %s %s missing: %s", taskDirName, entry.label, entry.value)
warnings = append(warnings, warning)
logger.Printf("warning %s", warning)
continue
}
if stat.IsDir() {
warning := fmt.Sprintf("task %s %s is a directory, expected file: %s", taskDirName, entry.label, entry.value)
warnings = append(warnings, warning)
logger.Printf("warning %s", warning)
continue
}
orbitDir := filepath.Join(tempDir, "orbit")
if err := os.MkdirAll(orbitDir, 0755); err != nil {
warning := fmt.Sprintf("task %s cannot create orbit directory: %v", taskDirName, err)
warnings = append(warnings, warning)
logger.Printf("warning %s", warning)
continue
}
if err := copyFile(source, filepath.Join(orbitDir, filepath.Base(source)), stat.Mode()); err != nil {
warning := fmt.Sprintf("task %s cannot copy %s %s: %v", taskDirName, entry.label, entry.value, err)
warnings = append(warnings, warning)
logger.Printf("warning %s", warning)
continue
}
}
return warnings
}
func orbitWarnings(inputRoot string, p pair, taskDirName string) []string {
var warnings []string
for _, entry := range []struct {
label string
value string
}{
{label: "master_orbit", value: p.MasterOrbit},
{label: "slave_orbit", value: p.SlaveOrbit},
} {
if strings.TrimSpace(entry.value) == "" {
continue
}
source, err := safeJoin(inputRoot, entry.value)
if err != nil {
warnings = append(warnings, fmt.Sprintf("task %s %s invalid: %v", taskDirName, entry.label, err))
continue
}
if stat, err := os.Stat(source); err != nil {
warnings = append(warnings, fmt.Sprintf("task %s %s missing: %s", taskDirName, entry.label, entry.value))
} else if stat.IsDir() {
warnings = append(warnings, fmt.Sprintf("task %s %s is a directory, expected file: %s", taskDirName, entry.label, entry.value))
}
}
return warnings
}
func copyDirContents(sourceDir string, destDir string) error {
sourceEntries, err := os.ReadDir(sourceDir)
if err != nil {
return err
}
if err := os.MkdirAll(destDir, 0755); err != nil {
return err
}
for _, entry := range sourceEntries {
sourcePath := filepath.Join(sourceDir, entry.Name())
destPath := filepath.Join(destDir, entry.Name())
info, err := entry.Info()
if err != nil {
return err
}
mode := info.Mode()
switch {
case mode&os.ModeSymlink != 0:
return fmt.Errorf("symlinks are not supported: %s", sourcePath)
case info.IsDir():
if err := copyDirContents(sourcePath, destPath); err != nil {
return err
}
case mode.IsRegular():
if err := copyFile(sourcePath, destPath, mode); err != nil {
return err
}
default:
return fmt.Errorf("unsupported file type: %s", sourcePath)
}
}
return nil
}
func copyFile(sourcePath string, destPath string, mode os.FileMode) error {
if err := os.MkdirAll(filepath.Dir(destPath), 0755); err != nil {
return err
}
source, err := os.Open(sourcePath)
if err != nil {
return err
}
defer source.Close()
dest, err := os.OpenFile(destPath, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, mode)
if err != nil {
return err
}
if _, err := io.Copy(dest, source); err != nil {
_ = dest.Close()
return err
}
return dest.Close()
}
func taskComplete(taskDir string) (bool, error) {
masterNonEmpty, err := dirNonEmpty(filepath.Join(taskDir, "master"))
if err != nil {
return false, err
}
slaveNonEmpty, err := dirNonEmpty(filepath.Join(taskDir, "slave"))
if err != nil {
return false, err
}
return masterNonEmpty && slaveNonEmpty, nil
}
func validateRestoredTask(taskDir string) error {
masterNonEmpty, err := dirNonEmpty(filepath.Join(taskDir, "master"))
if err != nil {
return fmt.Errorf("validate master directory: %w", err)
}
if !masterNonEmpty {
return errors.New("restored Task/master is empty")
}
slaveNonEmpty, err := dirNonEmpty(filepath.Join(taskDir, "slave"))
if err != nil {
return fmt.Errorf("validate slave directory: %w", err)
}
if !slaveNonEmpty {
return errors.New("restored Task/slave is empty")
}
if _, err := os.Stat(filepath.Join(taskDir, pairMetaName)); err != nil {
return fmt.Errorf("validate pair metadata: %w", err)
}
return nil
}
func dirNonEmpty(path string) (bool, error) {
entries, err := os.ReadDir(path)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return false, nil
}
return false, err
}
return len(entries) > 0, nil
}
func requireDir(path string, label string) error {
stat, err := os.Stat(path)
if err != nil {
return fmt.Errorf("%s is not accessible: %w", label, err)
}
if !stat.IsDir() {
return fmt.Errorf("%s is not a directory: %s", label, path)
}
return nil
}
func writeJSON(path string, value any) error {
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
return err
}
file, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0644)
if err != nil {
return err
}
defer file.Close()
encoder := json.NewEncoder(file)
encoder.SetIndent("", " ")
return encoder.Encode(value)
}
func newLogger(outputRoot string, dryRun bool, output io.Writer) (*log.Logger, *os.File, error) {
if output == nil {
output = io.Discard
}
if dryRun {
return log.New(output, "", log.LstdFlags), nil, nil
}
if err := os.MkdirAll(outputRoot, 0755); err != nil {
return nil, nil, err
}
logFile, err := os.OpenFile(filepath.Join(outputRoot, logFileName), os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0644)
if err != nil {
return nil, nil, err
}
writer := io.MultiWriter(output, logFile)
return log.New(writer, "", log.LstdFlags), logFile, nil
}
func defaultLogWriter(writer io.Writer) io.Writer {
if writer != nil {
return writer
}
return os.Stdout
}
func safeJoin(root string, relative string) (string, error) {
if filepath.IsAbs(relative) {
return "", fmt.Errorf("absolute path is not allowed: %s", relative)
}
cleanRelative := filepath.Clean(relative)
if cleanRelative == "." || strings.HasPrefix(cleanRelative, ".."+string(filepath.Separator)) || cleanRelative == ".." {
return "", fmt.Errorf("path escapes root: %s", relative)
}
joined := filepath.Join(root, cleanRelative)
if !isSubpath(root, joined) {
return "", fmt.Errorf("path escapes root: %s", relative)
}
return joined, nil
}
func isSubpath(root string, path string) bool {
absRoot, err := filepath.Abs(root)
if err != nil {
return false
}
absPath, err := filepath.Abs(path)
if err != nil {
return false
}
rel, err := filepath.Rel(absRoot, absPath)
if err != nil {
return false
}
return rel == "." || (!strings.HasPrefix(rel, ".."+string(filepath.Separator)) && rel != "..")
}
func pathExists(path string) bool {
_, err := os.Stat(path)
return err == nil
}
func pairLabel(p pair) string {
if strings.TrimSpace(p.PairID) != "" {
return p.PairID
}
if strings.TrimSpace(p.TaskName) != "" {
return p.TaskName
}
if strings.TrimSpace(p.TaskAlias) != "" {
return p.TaskAlias
}
return "<unknown>"
}
+242
View File
@@ -0,0 +1,242 @@
package main
import (
"encoding/json"
"io"
"os"
"path/filepath"
"testing"
)
func TestRunRestoresPair(t *testing.T) {
root := t.TempDir()
input := filepath.Join(root, "bundle")
output := filepath.Join(root, "out")
writeTestFile(t, filepath.Join(input, "data", "scene_master", "m.txt"), "master")
writeTestFile(t, filepath.Join(input, "data", "scene_slave", "s.txt"), "slave")
writeTestFile(t, filepath.Join(input, "orbit", "master.txt"), "master orbit")
doc := pairsDocument{
Schema: "dinsar_source_bundle_pairs.v1",
Pairs: []pair{{
PairID: "pair_0001",
TaskName: "Task_20250101_20250113",
TaskAlias: "Task_20250101_20250113",
MasterData: "data/scene_master",
SlaveData: "data/scene_slave",
MasterOrbit: "orbit/master.txt",
SlaveOrbit: "orbit/missing.txt",
MasterImagingDate: "20250101",
SlaveImagingDate: "20250113",
TimeBaselineDays: 12,
}},
}
writeTestJSON(t, filepath.Join(input, pairsFileName), doc)
result, err := run(config{inputRoot: input, outputRoot: output, skipExisting: true})
if err != nil {
t.Fatalf("run failed: %v", err)
}
if result.report.Restored != 1 {
t.Fatalf("restored = %d, want 1", result.report.Restored)
}
if len(result.report.Warnings) != 1 {
t.Fatalf("warnings = %d, want 1", len(result.report.Warnings))
}
for _, path := range []string{
filepath.Join(output, "Task_20250101_20250113", "master", "m.txt"),
filepath.Join(output, "Task_20250101_20250113", "slave", "s.txt"),
filepath.Join(output, "Task_20250101_20250113", "orbit", "master.txt"),
filepath.Join(output, "Task_20250101_20250113", pairMetaName),
filepath.Join(output, reportFileName),
filepath.Join(output, logFileName),
} {
if _, err := os.Stat(path); err != nil {
t.Fatalf("expected %s to exist: %v", path, err)
}
}
}
func TestRunSkipsExistingCompleteTask(t *testing.T) {
root := t.TempDir()
input := filepath.Join(root, "bundle")
output := filepath.Join(root, "out")
task := filepath.Join(output, "Task_20250101_20250113")
writeTestFile(t, filepath.Join(input, "data", "scene_master", "m.txt"), "master")
writeTestFile(t, filepath.Join(input, "data", "scene_slave", "s.txt"), "slave")
writeTestFile(t, filepath.Join(task, "master", "old.txt"), "old master")
writeTestFile(t, filepath.Join(task, "slave", "old.txt"), "old slave")
writeTestJSON(t, filepath.Join(input, pairsFileName), pairsDocument{
Pairs: []pair{{
PairID: "pair_0001",
TaskName: "Task_20250101_20250113",
MasterData: "data/scene_master",
SlaveData: "data/scene_slave",
}},
})
result, err := run(config{inputRoot: input, outputRoot: output, skipExisting: true})
if err != nil {
t.Fatalf("run failed: %v", err)
}
if result.report.Skipped != 1 {
t.Fatalf("skipped = %d, want 1", result.report.Skipped)
}
if result.report.Restored != 0 {
t.Fatalf("restored = %d, want 0", result.report.Restored)
}
}
func TestRunLimitContinuesAfterExistingTasks(t *testing.T) {
root := t.TempDir()
input := filepath.Join(root, "bundle")
output := filepath.Join(root, "out")
var pairs []pair
for i := 1; i <= 4; i++ {
taskName := "Task_20250101_202501" + string(rune('0'+i))
masterDir := filepath.Join(input, "data", "scene_master_"+string(rune('0'+i)))
slaveDir := filepath.Join(input, "data", "scene_slave_"+string(rune('0'+i)))
writeTestFile(t, filepath.Join(masterDir, "m.txt"), "master")
writeTestFile(t, filepath.Join(slaveDir, "s.txt"), "slave")
pairs = append(pairs, pair{
PairID: "pair_000" + string(rune('0'+i)),
TaskName: taskName,
MasterData: filepath.ToSlash(filepath.Join("data", "scene_master_"+string(rune('0'+i)))),
SlaveData: filepath.ToSlash(filepath.Join("data", "scene_slave_"+string(rune('0'+i)))),
})
}
writeTestJSON(t, filepath.Join(input, pairsFileName), pairsDocument{Pairs: pairs})
first, err := run(config{inputRoot: input, outputRoot: output, skipExisting: true, limit: 2, logWriter: io.Discard})
if err != nil {
t.Fatalf("first run failed: %v", err)
}
if first.report.Restored != 2 {
t.Fatalf("first restored = %d, want 2", first.report.Restored)
}
second, err := run(config{inputRoot: input, outputRoot: output, skipExisting: true, limit: 2, logWriter: io.Discard})
if err != nil {
t.Fatalf("second run failed: %v", err)
}
if second.report.Restored != 2 {
t.Fatalf("second restored = %d, want 2", second.report.Restored)
}
if second.report.Skipped != 2 {
t.Fatalf("second skipped = %d, want 2", second.report.Skipped)
}
for _, p := range pairs {
if _, err := os.Stat(filepath.Join(output, p.TaskName, pairMetaName)); err != nil {
t.Fatalf("expected %s metadata to exist: %v", p.TaskName, err)
}
}
}
func TestRunCheckReportsMissingSlaveData(t *testing.T) {
root := t.TempDir()
input := filepath.Join(root, "bundle")
writeTestFile(t, filepath.Join(input, "data", "scene_master", "m.txt"), "master")
writeTestJSON(t, filepath.Join(input, pairsFileName), pairsDocument{
Pairs: []pair{{
PairID: "pair_0001",
TaskName: "Task_20250101_20250113",
MasterData: "data/scene_master",
SlaveData: "data/scene_missing",
}},
})
result, err := runCheck(config{inputRoot: input, logWriter: io.Discard})
if err != nil {
t.Fatalf("runCheck failed: %v", err)
}
if result.report.Failed != 1 {
t.Fatalf("failed = %d, want 1", result.report.Failed)
}
if result.report.Valid != 0 {
t.Fatalf("valid = %d, want 0", result.report.Valid)
}
}
func TestRunCheckTreatsMissingOrbitAsWarning(t *testing.T) {
root := t.TempDir()
input := filepath.Join(root, "bundle")
writeTestFile(t, filepath.Join(input, "data", "scene_master", "m.txt"), "master")
writeTestFile(t, filepath.Join(input, "data", "scene_slave", "s.txt"), "slave")
writeTestJSON(t, filepath.Join(input, pairsFileName), pairsDocument{
Pairs: []pair{{
PairID: "pair_0001",
TaskName: "Task_20250101_20250113",
MasterData: "data/scene_master",
SlaveData: "data/scene_slave",
MasterOrbit: "orbit/missing.txt",
}},
})
result, err := runCheck(config{inputRoot: input, logWriter: io.Discard})
if err != nil {
t.Fatalf("runCheck failed: %v", err)
}
if result.report.Failed != 0 {
t.Fatalf("failed = %d, want 0", result.report.Failed)
}
if result.report.Valid != 1 {
t.Fatalf("valid = %d, want 1", result.report.Valid)
}
if len(result.report.Warnings) != 1 {
t.Fatalf("warnings = %d, want 1", len(result.report.Warnings))
}
}
func TestParseFlagsRejectsSkipExistingWithOverwrite(t *testing.T) {
_, err := parseFlags([]string{"--input", "in", "--output", "out", "--skip-existing", "--overwrite"})
if err == nil {
t.Fatal("expected error")
}
}
func TestParseFlagsAllowsOverwriteByItself(t *testing.T) {
cfg, err := parseFlags([]string{"--input", "in", "--output", "out", "--overwrite"})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if cfg.skipExisting {
t.Fatal("skipExisting = true, want false")
}
}
func TestSafeJoinRejectsEscapingPath(t *testing.T) {
if _, err := safeJoin(t.TempDir(), "../outside"); err == nil {
t.Fatal("expected escaping path error")
}
}
func writeTestFile(t *testing.T, path string, content string) {
t.Helper()
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte(content), 0644); err != nil {
t.Fatal(err)
}
}
func writeTestJSON(t *testing.T, path string, value any) {
t.Helper()
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
t.Fatal(err)
}
bytes, err := json.MarshalIndent(value, "", " ")
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, bytes, 0644); err != nil {
t.Fatal(err)
}
}