Initial D-InSAR bundle restore tool
This commit is contained in:
Harmon
@@ -0,0 +1,5 @@
|
||||
*.exe
|
||||
*.zip
|
||||
release/
|
||||
.manual_check_*/
|
||||
tmp_manual_check/
|
||||
@@ -0,0 +1,142 @@
|
||||
# D-InSAR 去重源数据包反向还原工具任务书
|
||||
|
||||
日期：2026-05-11
|
||||
|
||||
## 背景
|
||||
|
||||
本系统新增“去重源数据包”分发模式。该模式不直接生成每个干涉对的 `Task_*` 目录，而是只分发唯一源影像、唯一精密轨道文件和配对关系文件，减少外部分发时的重复复制量。
|
||||
|
||||
反向还原工具由任务接收方本地运行，将去重源数据包还原为传统 D-InSAR `Task_*` 目录结构。
|
||||
|
||||
## 输入目录结构
|
||||
|
||||
```text
|
||||
BundleRoot/
|
||||
data/
|
||||
scene_<hash>_<source_name>/
|
||||
...
|
||||
orbit/
|
||||
orbit_<hash>_<orbit_name>.txt
|
||||
...
|
||||
pairs.json
|
||||
manifest.json
|
||||
```
|
||||
|
||||
`orbit/` 可能不存在，或 `pairs.json` 内某些配对的轨道字段为空。
|
||||
|
||||
## 输出目录结构
|
||||
|
||||
```text
|
||||
OutputRoot/
|
||||
Task_YYYYMMDD_YYYYMMDD/
|
||||
master/
|
||||
<master source product content>
|
||||
slave/
|
||||
<slave source product content>
|
||||
orbit/
|
||||
<master/slave orbit files, if present>
|
||||
.dinsar_pair.json
|
||||
```
|
||||
|
||||
输出目录名称优先使用 `pairs.json` 内的 `task_alias`，若为空则使用 `task_name`，再为空则使用 `pair_id`。
|
||||
|
||||
## pairs.json 关键字段
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "dinsar_source_bundle_pairs.v1",
|
||||
"exported_at": "2026-05-11T00:00:00Z",
|
||||
"pairs": [
|
||||
{
|
||||
"pair_id": "pair_0001",
|
||||
"task_name": "Task_20250101_20250113",
|
||||
"task_alias": "Task_20250101_20250113",
|
||||
"master_data": "data/scene_xxx_master",
|
||||
"slave_data": "data/scene_yyy_slave",
|
||||
"master_orbit": "orbit/orbit_xxx.txt",
|
||||
"slave_orbit": "orbit/orbit_yyy.txt",
|
||||
"master_imaging_date": "20250101",
|
||||
"slave_imaging_date": "20250113",
|
||||
"time_baseline_days": 12
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## 还原规则
|
||||
|
||||
1. 读取 `pairs.json`。
|
||||
2. 对每个 pair 创建目标 `Task` 目录。
|
||||
3. 将 `master_data` 指向的数据复制到 `Task/master/`。
|
||||
4. 将 `slave_data` 指向的数据复制到 `Task/slave/`。
|
||||
5. 如 `master_orbit` / `slave_orbit` 存在，将轨道文件复制到 `Task/orbit/`。
|
||||
6. 生成 `.dinsar_pair.json`，至少保留：
|
||||
- `pair_id`
|
||||
- `task_name`
|
||||
- `task_alias`
|
||||
- `master_data`
|
||||
- `slave_data`
|
||||
- `master_orbit`
|
||||
- `slave_orbit`
|
||||
- `master_imaging_date`
|
||||
- `slave_imaging_date`
|
||||
- `time_baseline_days`
|
||||
- `restored_at`
|
||||
7. 每个 Task 应采用临时目录还原，全部成功后再重命名为最终目录，避免半成品。
|
||||
|
||||
## 覆盖策略
|
||||
|
||||
工具应提供参数：
|
||||
|
||||
- `--skip-existing`：默认开启。若目标 `Task/master` 和 `Task/slave` 均存在且非空，则跳过。
|
||||
- `--overwrite`：删除并重建已存在的目标 Task。
|
||||
- `--limit N`：最多还原 N 个 pair，便于分批执行。
|
||||
- `--dry-run`：只打印计划，不复制。
|
||||
|
||||
`--skip-existing` 与 `--overwrite` 同时出现时应报错。
|
||||
|
||||
## 校验要求
|
||||
|
||||
启动前：
|
||||
|
||||
- 检查 `pairs.json` 是否存在且可解析。
|
||||
- 检查 `data/` 是否存在。
|
||||
- 检查每个 pair 的 `master_data` / `slave_data` 是否存在。
|
||||
- 轨道缺失不应阻断还原，但要记录 warning。
|
||||
|
||||
还原后：
|
||||
|
||||
- `Task/master/` 非空。
|
||||
- `Task/slave/` 非空。
|
||||
- `.dinsar_pair.json` 存在。
|
||||
|
||||
## 日志与报告
|
||||
|
||||
工具结束后输出 `restore_report.json`：
|
||||
|
||||
```json
|
||||
{
|
||||
"started_at": "...",
|
||||
"finished_at": "...",
|
||||
"input_root": "...",
|
||||
"output_root": "...",
|
||||
"total_pairs": 20,
|
||||
"restored": 18,
|
||||
"skipped": 2,
|
||||
"failed": 0,
|
||||
"warnings": []
|
||||
}
|
||||
```
|
||||
|
||||
同时建议输出人类可读日志 `restore.log`。
|
||||
|
||||
## 建议实现
|
||||
|
||||
建议使用 Python 3.10+：
|
||||
|
||||
- `argparse` 处理命令行参数。
|
||||
- `pathlib.Path` 处理路径。
|
||||
- `shutil.copytree(..., dirs_exist_ok=True)` / `shutil.copy2()` 处理复制。
|
||||
- Windows 下注意长路径和权限异常。
|
||||
|
||||
该工具不需要连接本系统数据库，也不需要调用本系统 API。
|
||||
@@ -0,0 +1,243 @@
|
||||
# D-InSAR 去重源数据包恢复工具
|
||||
|
||||
这个工具用于把“去重源数据包”恢复成传统 D-InSAR `Task_*` 目录结构。
|
||||
|
||||
适合接收方在本地运行：选择一个 `BundleRoot` 输入目录，工具会按 `pairs.json` 中的配对关系，在输出目录中恢复每个 `Task_*`。
|
||||
|
||||
## 产物
|
||||
|
||||
当前提供两个 Windows 可执行文件：
|
||||
|
||||
- `dinsar-restore.exe`：图形界面版，推荐普通用户双击使用。
|
||||
- `dinsar-restore-console.exe`：命令行版，适合脚本、批处理或自动化调用。
|
||||
|
||||
Go 编译后的 exe 是单文件程序，不需要安装 Python，也不需要额外运行环境。
|
||||
|
||||
## 输入目录格式
|
||||
|
||||
输入目录应类似：
|
||||
|
||||
```text
|
||||
BundleRoot/
|
||||
data/
|
||||
scene_<hash>_<source_name>/
|
||||
...
|
||||
orbit/
|
||||
orbit_<hash>_<orbit_name>.txt
|
||||
...
|
||||
pairs.json
|
||||
manifest.json
|
||||
```
|
||||
|
||||
其中：
|
||||
|
||||
- `pairs.json` 必须存在。
|
||||
- `data/` 必须存在。
|
||||
- `orbit/` 可以不存在；轨道文件缺失只会记录 warning，不会阻断恢复。
|
||||
|
||||
## 输出目录格式
|
||||
|
||||
恢复后输出目录类似：
|
||||
|
||||
```text
|
||||
OutputRoot/
|
||||
Task_YYYYMMDD_YYYYMMDD/
|
||||
master/
|
||||
slave/
|
||||
orbit/
|
||||
.dinsar_pair.json
|
||||
restore_report.json
|
||||
restore.log
|
||||
```
|
||||
|
||||
Task 目录名优先使用：
|
||||
|
||||
1. `task_alias`
|
||||
2. `task_name`
|
||||
3. `pair_id`
|
||||
|
||||
## 图形界面用法
|
||||
|
||||
双击运行：
|
||||
|
||||
```text
|
||||
dinsar-restore.exe
|
||||
```
|
||||
|
||||
操作流程：
|
||||
|
||||
1. 点击输入目录的“选择...”按钮，选择 `BundleRoot`。
|
||||
2. 建议先点击“数据自检”。
|
||||
3. 点击输出目录的“选择...”按钮，选择 `OutputRoot`。
|
||||
4. 根据需要设置 `Dry run`、`覆盖已有 Task`、`Limit`。
|
||||
5. 点击“开始恢复”。
|
||||
6. 在运行日志区域查看进度、warning 和错误。
|
||||
|
||||
### 数据自检
|
||||
|
||||
“数据自检”只检查输入数据，不复制文件，也不会生成 `Task_*`。
|
||||
|
||||
自检内容包括：
|
||||
|
||||
- `pairs.json` 是否存在、是否能解析。
|
||||
- `data/` 是否存在。
|
||||
- 每个 pair 的 Task 目录名是否合法。
|
||||
- `master_data` 是否存在、是否为非空目录。
|
||||
- `slave_data` 是否存在、是否为非空目录。
|
||||
- 是否存在重复 Task 目录名。
|
||||
- `master_orbit` / `slave_orbit` 是否存在。
|
||||
|
||||
注意：轨道文件缺失只记为 warning，不算失败。
|
||||
|
||||
### 分批恢复
|
||||
|
||||
如果数据量很大，可以用 `Limit` 分批执行。
|
||||
|
||||
例如共有 1000 个 Task：
|
||||
|
||||
1. 第一次设置 `Limit=200`，恢复 200 个。
|
||||
2. 第二次仍设置 `Limit=200`，工具会跳过已完成的 200 个，再恢复新的 200 个。
|
||||
3. 重复执行，直到全部完成。
|
||||
|
||||
已完成 Task 的判断标准是：
|
||||
|
||||
- `Task/master/` 存在且非空。
|
||||
- `Task/slave/` 存在且非空。
|
||||
|
||||
已跳过的 Task 不会占用 `Limit` 数量。
|
||||
|
||||
### 覆盖已有 Task
|
||||
|
||||
默认情况下，已经完整恢复的 Task 会被跳过。
|
||||
|
||||
如果勾选“覆盖已有 Task”，工具会删除并重建已有 Task 目录。这个选项适合重新生成结果，但使用前应确认输出目录中没有需要保留的手工文件。
|
||||
|
||||
### Dry run
|
||||
|
||||
勾选 `Dry run` 后，工具只打印计划，不复制文件。
|
||||
|
||||
可以用它先确认将要恢复哪些 Task。
|
||||
|
||||
## 命令行用法
|
||||
|
||||
命令行版文件：
|
||||
|
||||
```powershell
|
||||
.\dinsar-restore-console.exe
|
||||
```
|
||||
|
||||
数据自检：
|
||||
|
||||
```powershell
|
||||
.\dinsar-restore-console.exe --input D:\BundleRoot --check-only
|
||||
```
|
||||
|
||||
正式恢复：
|
||||
|
||||
```powershell
|
||||
.\dinsar-restore-console.exe --input D:\BundleRoot --output D:\OutputRoot
|
||||
```
|
||||
|
||||
Dry run：
|
||||
|
||||
```powershell
|
||||
.\dinsar-restore-console.exe --input D:\BundleRoot --output D:\OutputRoot --dry-run
|
||||
```
|
||||
|
||||
覆盖已有 Task：
|
||||
|
||||
```powershell
|
||||
.\dinsar-restore-console.exe --input D:\BundleRoot --output D:\OutputRoot --overwrite
|
||||
```
|
||||
|
||||
分批恢复，每次最多恢复 200 个新 Task：
|
||||
|
||||
```powershell
|
||||
.\dinsar-restore-console.exe --input D:\BundleRoot --output D:\OutputRoot --limit 200
|
||||
```
|
||||
|
||||
## 参数说明
|
||||
|
||||
| 参数 | 说明 |
|
||||
| --- | --- |
|
||||
| `--input` | 输入目录，即 `BundleRoot`。 |
|
||||
| `--output` | 输出目录，即 `OutputRoot`。`--check-only` 时可以不填。 |
|
||||
| `--check-only` | 只做数据自检，不恢复文件。 |
|
||||
| `--dry-run` | 只打印恢复计划，不复制文件。 |
|
||||
| `--limit N` | 本次最多恢复 N 个新 Task。已跳过的 Task 不计入数量。 |
|
||||
| `--overwrite` | 删除并重建已有 Task。 |
|
||||
| `--skip-existing` | 跳过已完成 Task，默认开启。通常不需要手动设置。 |
|
||||
|
||||
`--overwrite` 和显式设置的 `--skip-existing` 不能同时使用。
|
||||
|
||||
## 输出文件
|
||||
|
||||
正式恢复后，输出目录会生成：
|
||||
|
||||
- `restore.log`：人类可读日志。
|
||||
- `restore_report.json`：机器可读恢复报告。
|
||||
- 每个 `Task_*` 内的 `.dinsar_pair.json`：该 Task 对应的 pair 元信息。
|
||||
|
||||
`restore_report.json` 示例：
|
||||
|
||||
```json
|
||||
{
|
||||
"started_at": "2026-05-11T00:00:00Z",
|
||||
"finished_at": "2026-05-11T00:10:00Z",
|
||||
"input_root": "D:\\BundleRoot",
|
||||
"output_root": "D:\\OutputRoot",
|
||||
"total_pairs": 1000,
|
||||
"restored": 200,
|
||||
"skipped": 0,
|
||||
"failed": 0,
|
||||
"warnings": []
|
||||
}
|
||||
```
|
||||
|
||||
## 重新编译
|
||||
|
||||
需要 Go 1.22 或更高版本。
|
||||
|
||||
图形界面版：
|
||||
|
||||
```powershell
|
||||
go build -ldflags "-H=windowsgui" -o dinsar-restore.exe .
|
||||
```
|
||||
|
||||
命令行版：
|
||||
|
||||
```powershell
|
||||
go build -o dinsar-restore-console.exe .
|
||||
```
|
||||
|
||||
运行测试：
|
||||
|
||||
```powershell
|
||||
go test ./...
|
||||
```
|
||||
|
||||
## 发布建议
|
||||
|
||||
建议 GitHub Release 附带以下文件：
|
||||
|
||||
- `dinsar-restore.exe`
|
||||
- `dinsar-restore-console.exe`
|
||||
- `README.md`
|
||||
|
||||
Release 标题示例：
|
||||
|
||||
```text
|
||||
D-InSAR Restore Tool v0.1.0
|
||||
```
|
||||
|
||||
Release 说明可写：
|
||||
|
||||
```text
|
||||
Initial release.
|
||||
|
||||
- Add Windows GUI restore tool.
|
||||
- Add console restore tool.
|
||||
- Add data self-check before restore.
|
||||
- Support batch continuation with Limit.
|
||||
- Generate restore_report.json and restore.log.
|
||||
```
|
||||
@@ -0,0 +1,9 @@
|
||||
//go:build !windows
|
||||
|
||||
package main
|
||||
|
||||
import "errors"
|
||||
|
||||
func runGUI() error {
|
||||
return errors.New("GUI mode is only supported on Windows")
|
||||
}
|
||||
+591
@@ -0,0 +1,591 @@
|
||||
//go:build windows
|
||||
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"runtime"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"syscall"
|
||||
"unsafe"
|
||||
)
|
||||
|
||||
const (
|
||||
windowTitle = "D-InSAR 数据包恢复工具"
|
||||
|
||||
wmDestroy = 0x0002
|
||||
wmCommand = 0x0111
|
||||
wmClose = 0x0010
|
||||
wmSetFont = 0x0030
|
||||
|
||||
bnClicked = 0
|
||||
|
||||
wsOverlappedWindow = 0x00CF0000
|
||||
wsVisible = 0x10000000
|
||||
wsChild = 0x40000000
|
||||
wsTabStop = 0x00010000
|
||||
wsBorder = 0x00800000
|
||||
wsVScroll = 0x00200000
|
||||
wsHScroll = 0x00100000
|
||||
esLeft = 0x0000
|
||||
esMultiline = 0x0004
|
||||
esAutoVScroll = 0x0040
|
||||
esAutoHScroll = 0x0080
|
||||
esReadOnly = 0x0800
|
||||
bsPushButton = 0x00000000
|
||||
bsAutoCheckBox = 0x00000003
|
||||
|
||||
swShow = 5
|
||||
cwUseDefault = 0x80000000
|
||||
|
||||
gwlUserData = -21
|
||||
|
||||
idInputEdit = 1001
|
||||
idInputBrowse = 1002
|
||||
idOutputEdit = 1003
|
||||
idOutputBrowse = 1004
|
||||
idDryRun = 1005
|
||||
idOverwrite = 1006
|
||||
idLimitEdit = 1007
|
||||
idStart = 1008
|
||||
idLogEdit = 1009
|
||||
idCheck = 1010
|
||||
|
||||
bmGetCheck = 0x00F0
|
||||
bstChecked = 1
|
||||
|
||||
maxPathBuffer = 4096
|
||||
)
|
||||
|
||||
var (
|
||||
user32 = syscall.NewLazyDLL("user32.dll")
|
||||
kernel32 = syscall.NewLazyDLL("kernel32.dll")
|
||||
gdi32 = syscall.NewLazyDLL("gdi32.dll")
|
||||
shell32 = syscall.NewLazyDLL("shell32.dll")
|
||||
ole32 = syscall.NewLazyDLL("ole32.dll")
|
||||
|
||||
procRegisterClassExW = user32.NewProc("RegisterClassExW")
|
||||
procCreateWindowExW = user32.NewProc("CreateWindowExW")
|
||||
procDefWindowProcW = user32.NewProc("DefWindowProcW")
|
||||
procDestroyWindow = user32.NewProc("DestroyWindow")
|
||||
procDispatchMessageW = user32.NewProc("DispatchMessageW")
|
||||
procGetMessageW = user32.NewProc("GetMessageW")
|
||||
procPostQuitMessage = user32.NewProc("PostQuitMessage")
|
||||
procSendMessageW = user32.NewProc("SendMessageW")
|
||||
procShowWindow = user32.NewProc("ShowWindow")
|
||||
procUpdateWindow = user32.NewProc("UpdateWindow")
|
||||
procSetWindowTextW = user32.NewProc("SetWindowTextW")
|
||||
procGetWindowTextW = user32.NewProc("GetWindowTextW")
|
||||
procGetWindowTextLenW = user32.NewProc("GetWindowTextLengthW")
|
||||
procEnableWindow = user32.NewProc("EnableWindow")
|
||||
procSetWindowLongPtrW = user32.NewProc("SetWindowLongPtrW")
|
||||
procGetWindowLongPtrW = user32.NewProc("GetWindowLongPtrW")
|
||||
procMessageBoxW = user32.NewProc("MessageBoxW")
|
||||
|
||||
procGetModuleHandleW = kernel32.NewProc("GetModuleHandleW")
|
||||
|
||||
procCreateFontW = gdi32.NewProc("CreateFontW")
|
||||
|
||||
procSHBrowseForFolderW = shell32.NewProc("SHBrowseForFolderW")
|
||||
procSHGetPathFromIDListW = shell32.NewProc("SHGetPathFromIDListW")
|
||||
procCoTaskMemFree = ole32.NewProc("CoTaskMemFree")
|
||||
procOleInitialize = ole32.NewProc("OleInitialize")
|
||||
procOleUninitialize = ole32.NewProc("OleUninitialize")
|
||||
)
|
||||
|
||||
type point struct {
|
||||
x int32
|
||||
y int32
|
||||
}
|
||||
|
||||
type msg struct {
|
||||
hwnd uintptr
|
||||
message uint32
|
||||
wParam uintptr
|
||||
lParam uintptr
|
||||
time uint32
|
||||
pt point
|
||||
}
|
||||
|
||||
type wndClassEx struct {
|
||||
cbSize uint32
|
||||
style uint32
|
||||
lpfnWndProc uintptr
|
||||
cbClsExtra int32
|
||||
cbWndExtra int32
|
||||
hInstance uintptr
|
||||
hIcon uintptr
|
||||
hCursor uintptr
|
||||
hbrBackground uintptr
|
||||
lpszMenuName *uint16
|
||||
lpszClassName *uint16
|
||||
hIconSm uintptr
|
||||
}
|
||||
|
||||
type browseInfo struct {
|
||||
hwndOwner uintptr
|
||||
pidlRoot uintptr
|
||||
pszDisplayName *uint16
|
||||
lpszTitle *uint16
|
||||
ulFlags uint32
|
||||
lpfn uintptr
|
||||
lParam uintptr
|
||||
iImage int32
|
||||
}
|
||||
|
||||
type guiApp struct {
|
||||
hwnd uintptr
|
||||
font uintptr
|
||||
inputEdit uintptr
|
||||
outputEdit uintptr
|
||||
dryRun uintptr
|
||||
overwrite uintptr
|
||||
limitEdit uintptr
|
||||
startButton uintptr
|
||||
checkButton uintptr
|
||||
logEdit uintptr
|
||||
mu sync.Mutex
|
||||
logText string
|
||||
running bool
|
||||
}
|
||||
|
||||
func runGUI() error {
|
||||
runtime.LockOSThread()
|
||||
defer runtime.UnlockOSThread()
|
||||
procOleInitialize.Call(0)
|
||||
defer procOleUninitialize.Call()
|
||||
|
||||
instance, _, _ := procGetModuleHandleW.Call(0)
|
||||
className, _ := syscall.UTF16PtrFromString("DinsarRestoreWindow")
|
||||
title, _ := syscall.UTF16PtrFromString(windowTitle)
|
||||
|
||||
wc := wndClassEx{
|
||||
cbSize: uint32(unsafe.Sizeof(wndClassEx{})),
|
||||
lpfnWndProc: syscall.NewCallback(windowProc),
|
||||
hInstance: instance,
|
||||
hbrBackground: 5 + 1,
|
||||
lpszClassName: className,
|
||||
}
|
||||
if atom, _, err := procRegisterClassExW.Call(uintptr(unsafe.Pointer(&wc))); atom == 0 {
|
||||
return fmt.Errorf("RegisterClassExW failed: %v", err)
|
||||
}
|
||||
|
||||
app := &guiApp{}
|
||||
hwnd, _, err := procCreateWindowExW.Call(
|
||||
0,
|
||||
uintptr(unsafe.Pointer(className)),
|
||||
uintptr(unsafe.Pointer(title)),
|
||||
wsOverlappedWindow|wsVisible,
|
||||
cwUseDefault,
|
||||
cwUseDefault,
|
||||
1040,
|
||||
720,
|
||||
0,
|
||||
0,
|
||||
instance,
|
||||
uintptr(unsafe.Pointer(app)),
|
||||
)
|
||||
if hwnd == 0 {
|
||||
return fmt.Errorf("CreateWindowExW failed: %v", err)
|
||||
}
|
||||
app.hwnd = hwnd
|
||||
|
||||
procShowWindow.Call(hwnd, swShow)
|
||||
procUpdateWindow.Call(hwnd)
|
||||
|
||||
var m msg
|
||||
for {
|
||||
ret, _, _ := procGetMessageW.Call(uintptr(unsafe.Pointer(&m)), 0, 0, 0)
|
||||
if int32(ret) <= 0 {
|
||||
break
|
||||
}
|
||||
procDispatchMessageW.Call(uintptr(unsafe.Pointer(&m)))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func windowProc(hwnd uintptr, msg uint32, wParam uintptr, lParam uintptr) uintptr {
|
||||
app := appFromWindow(hwnd)
|
||||
switch msg {
|
||||
case wmClose:
|
||||
if app != nil && app.isRunning() {
|
||||
messageBox(hwnd, "任务正在运行，请等待结束。", windowTitle)
|
||||
return 0
|
||||
}
|
||||
procDestroyWindow.Call(hwnd)
|
||||
return 0
|
||||
case wmDestroy:
|
||||
procPostQuitMessage.Call(0)
|
||||
return 0
|
||||
case wmCommand:
|
||||
id := int(wParam & 0xffff)
|
||||
notify := int((wParam >> 16) & 0xffff)
|
||||
if notify == bnClicked && app != nil {
|
||||
switch id {
|
||||
case idInputBrowse:
|
||||
if selected := chooseDirectory(hwnd, "选择 BundleRoot 输入目录"); selected != "" {
|
||||
setWindowText(app.inputEdit, selected)
|
||||
}
|
||||
case idOutputBrowse:
|
||||
if selected := chooseDirectory(hwnd, "选择 OutputRoot 输出目录"); selected != "" {
|
||||
setWindowText(app.outputEdit, selected)
|
||||
}
|
||||
case idStart:
|
||||
app.startRestore()
|
||||
case idCheck:
|
||||
app.startCheck()
|
||||
}
|
||||
}
|
||||
return 0
|
||||
case 0x0081: // WM_NCCREATE
|
||||
cs := (*createStruct)(unsafe.Pointer(lParam))
|
||||
procSetWindowLongPtrW.Call(hwnd, winIndex(gwlUserData), cs.createParams)
|
||||
return 1
|
||||
case 0x0001: // WM_CREATE
|
||||
app = appFromWindow(hwnd)
|
||||
if app != nil {
|
||||
app.hwnd = hwnd
|
||||
app.createControls()
|
||||
}
|
||||
return 0
|
||||
}
|
||||
ret, _, _ := procDefWindowProcW.Call(hwnd, uintptr(msg), wParam, lParam)
|
||||
return ret
|
||||
}
|
||||
|
||||
type createStruct struct {
|
||||
createParams uintptr
|
||||
instance uintptr
|
||||
menu uintptr
|
||||
parent uintptr
|
||||
cy int32
|
||||
cx int32
|
||||
y int32
|
||||
x int32
|
||||
style int32
|
||||
name *uint16
|
||||
class *uint16
|
||||
exStyle uint32
|
||||
}
|
||||
|
||||
func appFromWindow(hwnd uintptr) *guiApp {
|
||||
if hwnd == 0 {
|
||||
return nil
|
||||
}
|
||||
ptr, _, _ := procGetWindowLongPtrW.Call(hwnd, winIndex(gwlUserData))
|
||||
if ptr == 0 {
|
||||
return nil
|
||||
}
|
||||
return (*guiApp)(unsafe.Pointer(ptr))
|
||||
}
|
||||
|
||||
func (a *guiApp) createControls() {
|
||||
a.font = createFont("Microsoft YaHei UI", 20)
|
||||
|
||||
createLabel(a.hwnd, "输入目录 BundleRoot", 24, 28, 210, 30, a.font)
|
||||
a.inputEdit = createEdit(a.hwnd, "", idInputEdit, 240, 24, 640, 34, false, a.font)
|
||||
createButton(a.hwnd, "选择...", idInputBrowse, 895, 24, 100, 34, a.font)
|
||||
|
||||
createLabel(a.hwnd, "输出目录 OutputRoot", 24, 78, 210, 30, a.font)
|
||||
a.outputEdit = createEdit(a.hwnd, "", idOutputEdit, 240, 74, 640, 34, false, a.font)
|
||||
createButton(a.hwnd, "选择...", idOutputBrowse, 895, 74, 100, 34, a.font)
|
||||
|
||||
a.dryRun = createCheckBox(a.hwnd, "Dry run 只打印计划", idDryRun, 240, 128, 230, 32, a.font)
|
||||
a.overwrite = createCheckBox(a.hwnd, "覆盖已有 Task", idOverwrite, 500, 128, 190, 32, a.font)
|
||||
createLabel(a.hwnd, "Limit", 720, 130, 60, 30, a.font)
|
||||
a.limitEdit = createEdit(a.hwnd, "", idLimitEdit, 780, 126, 100, 34, false, a.font)
|
||||
|
||||
a.checkButton = createButton(a.hwnd, "数据自检", idCheck, 760, 174, 105, 38, a.font)
|
||||
a.startButton = createButton(a.hwnd, "开始恢复", idStart, 890, 174, 105, 38, a.font)
|
||||
createLabel(a.hwnd, "运行日志", 24, 226, 140, 30, a.font)
|
||||
a.logEdit = createEdit(a.hwnd, "", idLogEdit, 24, 262, 970, 365, true, a.font)
|
||||
|
||||
a.appendLog("请选择输入目录后可先数据自检；选择输出目录后可开始恢复。\r\n")
|
||||
}
|
||||
|
||||
func (a *guiApp) startRestore() {
|
||||
if a.isRunning() {
|
||||
return
|
||||
}
|
||||
|
||||
input := strings.TrimSpace(getWindowText(a.inputEdit))
|
||||
output := strings.TrimSpace(getWindowText(a.outputEdit))
|
||||
if input == "" {
|
||||
messageBox(a.hwnd, "请选择输入目录 BundleRoot。", windowTitle)
|
||||
return
|
||||
}
|
||||
if output == "" {
|
||||
messageBox(a.hwnd, "请选择输出目录 OutputRoot。", windowTitle)
|
||||
return
|
||||
}
|
||||
limit, ok := a.parseLimit()
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
|
||||
cfg := config{
|
||||
inputRoot: input,
|
||||
outputRoot: output,
|
||||
skipExisting: true,
|
||||
overwrite: isChecked(a.overwrite),
|
||||
limit: limit,
|
||||
dryRun: isChecked(a.dryRun),
|
||||
logWriter: guiLogWriter{app: a},
|
||||
}
|
||||
if cfg.overwrite {
|
||||
cfg.skipExisting = false
|
||||
}
|
||||
|
||||
a.setRunning(true)
|
||||
a.setLog("")
|
||||
a.appendLog("开始执行...\r\n")
|
||||
go func() {
|
||||
result, err := run(cfg)
|
||||
if err != nil {
|
||||
a.appendLog("错误: " + err.Error() + "\r\n")
|
||||
messageBox(a.hwnd, "执行失败：\r\n"+err.Error(), windowTitle)
|
||||
} else {
|
||||
a.appendLog(fmt.Sprintf("完成: restored=%d skipped=%d failed=%d warnings=%d\r\n",
|
||||
result.report.Restored,
|
||||
result.report.Skipped,
|
||||
result.report.Failed,
|
||||
len(result.report.Warnings),
|
||||
))
|
||||
messageBox(a.hwnd, "执行完成。", windowTitle)
|
||||
}
|
||||
a.setRunning(false)
|
||||
}()
|
||||
}
|
||||
|
||||
func (a *guiApp) startCheck() {
|
||||
if a.isRunning() {
|
||||
return
|
||||
}
|
||||
|
||||
input := strings.TrimSpace(getWindowText(a.inputEdit))
|
||||
if input == "" {
|
||||
messageBox(a.hwnd, "请选择输入目录 BundleRoot。", windowTitle)
|
||||
return
|
||||
}
|
||||
limit, ok := a.parseLimit()
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
|
||||
cfg := config{
|
||||
inputRoot: input,
|
||||
limit: limit,
|
||||
checkOnly: true,
|
||||
logWriter: guiLogWriter{app: a},
|
||||
}
|
||||
|
||||
a.setRunning(true)
|
||||
a.setLog("")
|
||||
a.appendLog("开始数据自检...\r\n")
|
||||
go func() {
|
||||
result, err := runCheck(cfg)
|
||||
if err != nil {
|
||||
a.appendLog("错误: " + err.Error() + "\r\n")
|
||||
messageBox(a.hwnd, "自检失败：\r\n"+err.Error(), windowTitle)
|
||||
} else if result.report.Failed > 0 {
|
||||
a.appendLog(fmt.Sprintf("自检完成: total=%d checked=%d valid=%d failed=%d warnings=%d\r\n",
|
||||
result.report.TotalPairs,
|
||||
result.report.Checked,
|
||||
result.report.Valid,
|
||||
result.report.Failed,
|
||||
len(result.report.Warnings),
|
||||
))
|
||||
messageBox(a.hwnd, fmt.Sprintf("自检完成，但发现 %d 个失败项。请查看日志。", result.report.Failed), windowTitle)
|
||||
} else {
|
||||
a.appendLog(fmt.Sprintf("自检通过: total=%d checked=%d valid=%d warnings=%d\r\n",
|
||||
result.report.TotalPairs,
|
||||
result.report.Checked,
|
||||
result.report.Valid,
|
||||
len(result.report.Warnings),
|
||||
))
|
||||
messageBox(a.hwnd, "自检通过。", windowTitle)
|
||||
}
|
||||
a.setRunning(false)
|
||||
}()
|
||||
}
|
||||
|
||||
func (a *guiApp) parseLimit() (int, bool) {
|
||||
limitText := strings.TrimSpace(getWindowText(a.limitEdit))
|
||||
if limitText == "" {
|
||||
return 0, true
|
||||
}
|
||||
parsed, err := strconv.Atoi(limitText)
|
||||
if err != nil || parsed < 0 {
|
||||
messageBox(a.hwnd, "Limit 必须是大于等于 0 的整数。", windowTitle)
|
||||
return 0, false
|
||||
}
|
||||
return parsed, true
|
||||
}
|
||||
|
||||
func (a *guiApp) isRunning() bool {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
return a.running
|
||||
}
|
||||
|
||||
func (a *guiApp) setRunning(running bool) {
|
||||
a.mu.Lock()
|
||||
a.running = running
|
||||
a.mu.Unlock()
|
||||
enable := uintptr(1)
|
||||
if running {
|
||||
enable = 0
|
||||
}
|
||||
procEnableWindow.Call(a.startButton, enable)
|
||||
procEnableWindow.Call(a.checkButton, enable)
|
||||
procEnableWindow.Call(a.inputEdit, enable)
|
||||
procEnableWindow.Call(a.outputEdit, enable)
|
||||
procEnableWindow.Call(a.dryRun, enable)
|
||||
procEnableWindow.Call(a.overwrite, enable)
|
||||
procEnableWindow.Call(a.limitEdit, enable)
|
||||
}
|
||||
|
||||
func (a *guiApp) setLog(text string) {
|
||||
a.mu.Lock()
|
||||
a.logText = text
|
||||
a.mu.Unlock()
|
||||
setWindowText(a.logEdit, text)
|
||||
}
|
||||
|
||||
func (a *guiApp) appendLog(text string) {
|
||||
a.mu.Lock()
|
||||
a.logText += text
|
||||
fullText := a.logText
|
||||
a.mu.Unlock()
|
||||
setWindowText(a.logEdit, fullText)
|
||||
}
|
||||
|
||||
type guiLogWriter struct {
|
||||
app *guiApp
|
||||
}
|
||||
|
||||
func (w guiLogWriter) Write(p []byte) (int, error) {
|
||||
if w.app != nil {
|
||||
w.app.appendLog(strings.ReplaceAll(string(p), "\n", "\r\n"))
|
||||
}
|
||||
return len(p), nil
|
||||
}
|
||||
|
||||
func createLabel(parent uintptr, text string, x, y, width, height int32, font uintptr) uintptr {
|
||||
return createControl("STATIC", text, wsChild|wsVisible, 0, parent, 0, x, y, width, height, font)
|
||||
}
|
||||
|
||||
func createEdit(parent uintptr, text string, id int, x, y, width, height int32, multiline bool, font uintptr) uintptr {
|
||||
style := uint32(wsChild | wsVisible | wsBorder | wsTabStop | esLeft | esAutoHScroll)
|
||||
if multiline {
|
||||
style = wsChild | wsVisible | wsBorder | wsVScroll | wsHScroll | esLeft | esMultiline | esAutoVScroll | esAutoHScroll | esReadOnly
|
||||
}
|
||||
return createControl("EDIT", text, style, 0, parent, id, x, y, width, height, font)
|
||||
}
|
||||
|
||||
func createButton(parent uintptr, text string, id int, x, y, width, height int32, font uintptr) uintptr {
|
||||
return createControl("BUTTON", text, wsChild|wsVisible|wsTabStop|bsPushButton, 0, parent, id, x, y, width, height, font)
|
||||
}
|
||||
|
||||
func createCheckBox(parent uintptr, text string, id int, x, y, width, height int32, font uintptr) uintptr {
|
||||
return createControl("BUTTON", text, wsChild|wsVisible|wsTabStop|bsAutoCheckBox, 0, parent, id, x, y, width, height, font)
|
||||
}
|
||||
|
||||
func createControl(className string, text string, style uint32, exStyle uint32, parent uintptr, id int, x, y, width, height int32, font uintptr) uintptr {
|
||||
classPtr, _ := syscall.UTF16PtrFromString(className)
|
||||
textPtr, _ := syscall.UTF16PtrFromString(text)
|
||||
hwnd, _, _ := procCreateWindowExW.Call(
|
||||
uintptr(exStyle),
|
||||
uintptr(unsafe.Pointer(classPtr)),
|
||||
uintptr(unsafe.Pointer(textPtr)),
|
||||
uintptr(style),
|
||||
uintptr(x),
|
||||
uintptr(y),
|
||||
uintptr(width),
|
||||
uintptr(height),
|
||||
parent,
|
||||
uintptr(id),
|
||||
0,
|
||||
0,
|
||||
)
|
||||
if hwnd != 0 && font != 0 {
|
||||
procSendMessageW.Call(hwnd, wmSetFont, font, 1)
|
||||
}
|
||||
return hwnd
|
||||
}
|
||||
|
||||
func createFont(name string, size int32) uintptr {
|
||||
namePtr, _ := syscall.UTF16PtrFromString(name)
|
||||
font, _, _ := procCreateFontW.Call(
|
||||
uintptr(-size),
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
400,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
1,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
0,
|
||||
uintptr(unsafe.Pointer(namePtr)),
|
||||
)
|
||||
return font
|
||||
}
|
||||
|
||||
func setWindowText(hwnd uintptr, text string) {
|
||||
ptr, _ := syscall.UTF16PtrFromString(text)
|
||||
procSetWindowTextW.Call(hwnd, uintptr(unsafe.Pointer(ptr)))
|
||||
}
|
||||
|
||||
func getWindowText(hwnd uintptr) string {
|
||||
length, _, _ := procGetWindowTextLenW.Call(hwnd)
|
||||
buffer := make([]uint16, int(length)+1)
|
||||
procGetWindowTextW.Call(hwnd, uintptr(unsafe.Pointer(&buffer[0])), uintptr(len(buffer)))
|
||||
return syscall.UTF16ToString(buffer)
|
||||
}
|
||||
|
||||
func isChecked(hwnd uintptr) bool {
|
||||
ret, _, _ := procSendMessageW.Call(hwnd, bmGetCheck, 0, 0)
|
||||
return ret == bstChecked
|
||||
}
|
||||
|
||||
func messageBox(hwnd uintptr, text string, title string) {
|
||||
textPtr, _ := syscall.UTF16PtrFromString(text)
|
||||
titlePtr, _ := syscall.UTF16PtrFromString(title)
|
||||
procMessageBoxW.Call(hwnd, uintptr(unsafe.Pointer(textPtr)), uintptr(unsafe.Pointer(titlePtr)), 0)
|
||||
}
|
||||
|
||||
func chooseDirectory(hwnd uintptr, title string) string {
|
||||
titlePtr, _ := syscall.UTF16PtrFromString(title)
|
||||
displayName := make([]uint16, maxPathBuffer)
|
||||
bi := browseInfo{
|
||||
hwndOwner: hwnd,
|
||||
pszDisplayName: &displayName[0],
|
||||
lpszTitle: titlePtr,
|
||||
ulFlags: 0x00000001 | 0x00000010 | 0x00000040,
|
||||
}
|
||||
pidl, _, _ := procSHBrowseForFolderW.Call(uintptr(unsafe.Pointer(&bi)))
|
||||
if pidl == 0 {
|
||||
return ""
|
||||
}
|
||||
defer procCoTaskMemFree.Call(pidl)
|
||||
|
||||
pathBuffer := make([]uint16, maxPathBuffer)
|
||||
ret, _, _ := procSHGetPathFromIDListW.Call(pidl, uintptr(unsafe.Pointer(&pathBuffer[0])))
|
||||
if ret == 0 {
|
||||
return ""
|
||||
}
|
||||
return syscall.UTF16ToString(pathBuffer)
|
||||
}
|
||||
|
||||
func winIndex(value int32) uintptr {
|
||||
return uintptr(value)
|
||||
}
|
||||
|
||||
var _ io.Writer = guiLogWriter{}
|
||||
@@ -0,0 +1,847 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
const (
|
||||
pairsFileName = "pairs.json"
|
||||
reportFileName = "restore_report.json"
|
||||
logFileName = "restore.log"
|
||||
pairMetaName = ".dinsar_pair.json"
|
||||
tempSuffix = ".tmp"
|
||||
)
|
||||
|
||||
type config struct {
|
||||
inputRoot string
|
||||
outputRoot string
|
||||
skipExisting bool
|
||||
overwrite bool
|
||||
limit int
|
||||
dryRun bool
|
||||
checkOnly bool
|
||||
logWriter io.Writer
|
||||
}
|
||||
|
||||
type pairsDocument struct {
|
||||
Schema string `json:"schema"`
|
||||
ExportedAt string `json:"exported_at"`
|
||||
Pairs []pair `json:"pairs"`
|
||||
}
|
||||
|
||||
type pair struct {
|
||||
PairID string `json:"pair_id"`
|
||||
TaskName string `json:"task_name"`
|
||||
TaskAlias string `json:"task_alias"`
|
||||
MasterData string `json:"master_data"`
|
||||
SlaveData string `json:"slave_data"`
|
||||
MasterOrbit string `json:"master_orbit"`
|
||||
SlaveOrbit string `json:"slave_orbit"`
|
||||
MasterImagingDate string `json:"master_imaging_date"`
|
||||
SlaveImagingDate string `json:"slave_imaging_date"`
|
||||
TimeBaselineDays int `json:"time_baseline_days"`
|
||||
}
|
||||
|
||||
type pairMetadata struct {
|
||||
PairID string `json:"pair_id"`
|
||||
TaskName string `json:"task_name"`
|
||||
TaskAlias string `json:"task_alias"`
|
||||
MasterData string `json:"master_data"`
|
||||
SlaveData string `json:"slave_data"`
|
||||
MasterOrbit string `json:"master_orbit"`
|
||||
SlaveOrbit string `json:"slave_orbit"`
|
||||
MasterImagingDate string `json:"master_imaging_date"`
|
||||
SlaveImagingDate string `json:"slave_imaging_date"`
|
||||
TimeBaselineDays int `json:"time_baseline_days"`
|
||||
RestoredAt string `json:"restored_at"`
|
||||
}
|
||||
|
||||
type restoreReport struct {
|
||||
StartedAt string `json:"started_at"`
|
||||
FinishedAt string `json:"finished_at"`
|
||||
InputRoot string `json:"input_root"`
|
||||
OutputRoot string `json:"output_root"`
|
||||
TotalPairs int `json:"total_pairs"`
|
||||
Restored int `json:"restored"`
|
||||
Skipped int `json:"skipped"`
|
||||
Failed int `json:"failed"`
|
||||
Warnings []string `json:"warnings"`
|
||||
}
|
||||
|
||||
type restoreResult struct {
|
||||
report restoreReport
|
||||
}
|
||||
|
||||
type checkReport struct {
|
||||
StartedAt string `json:"started_at"`
|
||||
FinishedAt string `json:"finished_at"`
|
||||
InputRoot string `json:"input_root"`
|
||||
TotalPairs int `json:"total_pairs"`
|
||||
Checked int `json:"checked"`
|
||||
Valid int `json:"valid"`
|
||||
Failed int `json:"failed"`
|
||||
Warnings []string `json:"warnings"`
|
||||
Errors []string `json:"errors"`
|
||||
}
|
||||
|
||||
type checkResult struct {
|
||||
report checkReport
|
||||
}
|
||||
|
||||
func main() {
|
||||
if len(os.Args) == 1 && runtime.GOOS == "windows" {
|
||||
if err := runGUI(); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "error:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
cfg, err := parseFlags(os.Args[1:])
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, "error:", err)
|
||||
os.Exit(2)
|
||||
}
|
||||
|
||||
if cfg.checkOnly {
|
||||
result, err := runCheck(cfg)
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, "error:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Printf("check: total=%d checked=%d valid=%d failed=%d warnings=%d\n",
|
||||
result.report.TotalPairs,
|
||||
result.report.Checked,
|
||||
result.report.Valid,
|
||||
result.report.Failed,
|
||||
len(result.report.Warnings),
|
||||
)
|
||||
if result.report.Failed > 0 {
|
||||
os.Exit(1)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
result, err := run(cfg)
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, "error:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
fmt.Printf("done: restored=%d skipped=%d failed=%d warnings=%d\n",
|
||||
result.report.Restored,
|
||||
result.report.Skipped,
|
||||
result.report.Failed,
|
||||
len(result.report.Warnings),
|
||||
)
|
||||
}
|
||||
|
||||
func parseFlags(args []string) (config, error) {
|
||||
var cfg config
|
||||
fs := flag.NewFlagSet("dinsar-restore", flag.ContinueOnError)
|
||||
fs.SetOutput(io.Discard)
|
||||
fs.StringVar(&cfg.inputRoot, "input", "", "source bundle root")
|
||||
fs.StringVar(&cfg.outputRoot, "output", "", "restored output root")
|
||||
fs.BoolVar(&cfg.skipExisting, "skip-existing", true, "skip existing completed tasks")
|
||||
fs.BoolVar(&cfg.overwrite, "overwrite", false, "delete and rebuild existing task directories")
|
||||
fs.IntVar(&cfg.limit, "limit", 0, "maximum number of pairs to restore")
|
||||
fs.BoolVar(&cfg.dryRun, "dry-run", false, "print plan without copying files")
|
||||
fs.BoolVar(&cfg.checkOnly, "check-only", false, "validate the source bundle without restoring")
|
||||
|
||||
if err := fs.Parse(args); err != nil {
|
||||
return cfg, err
|
||||
}
|
||||
explicitFlags := map[string]bool{}
|
||||
fs.Visit(func(f *flag.Flag) {
|
||||
explicitFlags[f.Name] = true
|
||||
})
|
||||
if cfg.inputRoot == "" {
|
||||
return cfg, errors.New("missing --input")
|
||||
}
|
||||
if cfg.outputRoot == "" && !cfg.checkOnly {
|
||||
return cfg, errors.New("missing --output")
|
||||
}
|
||||
if cfg.overwrite && explicitFlags["skip-existing"] {
|
||||
return cfg, errors.New("--skip-existing and --overwrite cannot be used together")
|
||||
}
|
||||
if cfg.overwrite {
|
||||
cfg.skipExisting = false
|
||||
}
|
||||
if cfg.limit < 0 {
|
||||
return cfg, errors.New("--limit must be zero or greater")
|
||||
}
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
func run(cfg config) (restoreResult, error) {
|
||||
started := time.Now().UTC()
|
||||
|
||||
inputRoot, err := filepath.Abs(cfg.inputRoot)
|
||||
if err != nil {
|
||||
return restoreResult{}, fmt.Errorf("resolve input root: %w", err)
|
||||
}
|
||||
outputRoot, err := filepath.Abs(cfg.outputRoot)
|
||||
if err != nil {
|
||||
return restoreResult{}, fmt.Errorf("resolve output root: %w", err)
|
||||
}
|
||||
|
||||
report := restoreReport{
|
||||
StartedAt: started.Format(time.RFC3339),
|
||||
InputRoot: inputRoot,
|
||||
OutputRoot: outputRoot,
|
||||
Warnings: []string{},
|
||||
}
|
||||
|
||||
pairsDoc, err := loadAndValidateInput(inputRoot)
|
||||
if err != nil {
|
||||
return restoreResult{}, err
|
||||
}
|
||||
report.TotalPairs = len(pairsDoc.Pairs)
|
||||
|
||||
if !cfg.dryRun {
|
||||
if err := os.MkdirAll(outputRoot, 0755); err != nil {
|
||||
return restoreResult{}, fmt.Errorf("create output root: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
logOutput := defaultLogWriter(cfg.logWriter)
|
||||
logger, logFile, err := newLogger(outputRoot, cfg.dryRun, logOutput)
|
||||
if err != nil {
|
||||
return restoreResult{}, err
|
||||
}
|
||||
if logFile != nil {
|
||||
defer logFile.Close()
|
||||
}
|
||||
|
||||
plannedPairs := pairsDoc.Pairs
|
||||
|
||||
logger.Printf("started input=%s output=%s total_pairs=%d planned_pairs=%d dry_run=%t",
|
||||
inputRoot, outputRoot, report.TotalPairs, len(plannedPairs), cfg.dryRun)
|
||||
|
||||
limitCount := 0
|
||||
for i, p := range plannedPairs {
|
||||
if cfg.limit > 0 && limitCount >= cfg.limit {
|
||||
logger.Printf("limit reached processed=%d limit=%d", limitCount, cfg.limit)
|
||||
break
|
||||
}
|
||||
issues := checkPair(inputRoot, p, false)
|
||||
if len(issues.errors) > 0 {
|
||||
report.Failed++
|
||||
for _, item := range issues.errors {
|
||||
logger.Printf("failed pair_index=%d pair_id=%s error=%s", i, p.PairID, item)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
warnings, err := restorePair(cfg, inputRoot, outputRoot, p, logger)
|
||||
report.Warnings = append(report.Warnings, warnings...)
|
||||
if err != nil {
|
||||
if errors.Is(err, errSkipped) {
|
||||
report.Skipped++
|
||||
continue
|
||||
}
|
||||
report.Failed++
|
||||
logger.Printf("failed pair_index=%d pair_id=%s error=%v", i, p.PairID, err)
|
||||
continue
|
||||
}
|
||||
limitCount++
|
||||
if !cfg.dryRun {
|
||||
report.Restored++
|
||||
}
|
||||
}
|
||||
|
||||
report.FinishedAt = time.Now().UTC().Format(time.RFC3339)
|
||||
logger.Printf("finished restored=%d skipped=%d failed=%d warnings=%d",
|
||||
report.Restored, report.Skipped, report.Failed, len(report.Warnings))
|
||||
|
||||
if !cfg.dryRun {
|
||||
if err := writeJSON(filepath.Join(outputRoot, reportFileName), report); err != nil {
|
||||
return restoreResult{}, fmt.Errorf("write report: %w", err)
|
||||
}
|
||||
} else {
|
||||
if err := json.NewEncoder(logOutput).Encode(report); err != nil {
|
||||
return restoreResult{}, fmt.Errorf("print dry-run report: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
return restoreResult{report: report}, nil
|
||||
}
|
||||
|
||||
func runCheck(cfg config) (checkResult, error) {
|
||||
started := time.Now().UTC()
|
||||
inputRoot, err := filepath.Abs(cfg.inputRoot)
|
||||
if err != nil {
|
||||
return checkResult{}, fmt.Errorf("resolve input root: %w", err)
|
||||
}
|
||||
|
||||
report := checkReport{
|
||||
StartedAt: started.Format(time.RFC3339),
|
||||
InputRoot: inputRoot,
|
||||
Warnings: []string{},
|
||||
Errors: []string{},
|
||||
}
|
||||
|
||||
logOutput := defaultLogWriter(cfg.logWriter)
|
||||
logger := log.New(logOutput, "", log.LstdFlags)
|
||||
logger.Printf("check started input=%s", inputRoot)
|
||||
|
||||
pairsDoc, err := loadAndValidateInput(inputRoot)
|
||||
if err != nil {
|
||||
report.Failed = 1
|
||||
report.Errors = append(report.Errors, err.Error())
|
||||
report.FinishedAt = time.Now().UTC().Format(time.RFC3339)
|
||||
printCheckReport(logOutput, report)
|
||||
return checkResult{report: report}, nil
|
||||
}
|
||||
report.TotalPairs = len(pairsDoc.Pairs)
|
||||
|
||||
plannedPairs := pairsDoc.Pairs
|
||||
if cfg.limit > 0 && cfg.limit < len(plannedPairs) {
|
||||
plannedPairs = plannedPairs[:cfg.limit]
|
||||
}
|
||||
|
||||
seenTasks := map[string]int{}
|
||||
for i, p := range plannedPairs {
|
||||
report.Checked++
|
||||
issues := checkPair(inputRoot, p, true)
|
||||
taskName, taskNameErr := taskDirectoryName(p)
|
||||
if taskNameErr == nil {
|
||||
if firstIndex, exists := seenTasks[taskName]; exists {
|
||||
issues.errors = append(issues.errors, fmt.Sprintf("duplicate task directory %s also used by pair index %d", taskName, firstIndex))
|
||||
} else {
|
||||
seenTasks[taskName] = i
|
||||
}
|
||||
}
|
||||
|
||||
if len(issues.warnings) > 0 {
|
||||
report.Warnings = append(report.Warnings, issues.warnings...)
|
||||
for _, warning := range issues.warnings {
|
||||
logger.Printf("warning pair_index=%d pair_id=%s %s", i, p.PairID, warning)
|
||||
}
|
||||
}
|
||||
if len(issues.errors) > 0 {
|
||||
report.Failed++
|
||||
for _, item := range issues.errors {
|
||||
message := fmt.Sprintf("pair_index=%d pair_id=%s %s", i, p.PairID, item)
|
||||
report.Errors = append(report.Errors, message)
|
||||
logger.Printf("error %s", message)
|
||||
}
|
||||
continue
|
||||
}
|
||||
report.Valid++
|
||||
logger.Printf("valid pair_index=%d pair_id=%s task=%s", i, p.PairID, taskName)
|
||||
}
|
||||
|
||||
report.FinishedAt = time.Now().UTC().Format(time.RFC3339)
|
||||
logger.Printf("check finished total=%d checked=%d valid=%d failed=%d warnings=%d",
|
||||
report.TotalPairs, report.Checked, report.Valid, report.Failed, len(report.Warnings))
|
||||
printCheckReport(logOutput, report)
|
||||
return checkResult{report: report}, nil
|
||||
}
|
||||
|
||||
type pairCheckIssues struct {
|
||||
warnings []string
|
||||
errors []string
|
||||
}
|
||||
|
||||
func checkPair(inputRoot string, p pair, includeOrbitWarnings bool) pairCheckIssues {
|
||||
var issues pairCheckIssues
|
||||
if _, err := taskDirectoryName(p); err != nil {
|
||||
issues.errors = append(issues.errors, err.Error())
|
||||
}
|
||||
if err := validatePairPaths(inputRoot, p); err != nil {
|
||||
issues.errors = append(issues.errors, err.Error())
|
||||
return issues
|
||||
}
|
||||
|
||||
masterSource, _ := safeJoin(inputRoot, p.MasterData)
|
||||
slaveSource, _ := safeJoin(inputRoot, p.SlaveData)
|
||||
if err := requireDir(masterSource, "master_data"); err != nil {
|
||||
issues.errors = append(issues.errors, err.Error())
|
||||
} else if nonEmpty, err := dirNonEmpty(masterSource); err != nil {
|
||||
issues.errors = append(issues.errors, fmt.Sprintf("check master_data contents: %v", err))
|
||||
} else if !nonEmpty {
|
||||
issues.errors = append(issues.errors, fmt.Sprintf("master_data directory is empty: %s", p.MasterData))
|
||||
}
|
||||
if err := requireDir(slaveSource, "slave_data"); err != nil {
|
||||
issues.errors = append(issues.errors, err.Error())
|
||||
} else if nonEmpty, err := dirNonEmpty(slaveSource); err != nil {
|
||||
issues.errors = append(issues.errors, fmt.Sprintf("check slave_data contents: %v", err))
|
||||
} else if !nonEmpty {
|
||||
issues.errors = append(issues.errors, fmt.Sprintf("slave_data directory is empty: %s", p.SlaveData))
|
||||
}
|
||||
if includeOrbitWarnings {
|
||||
issues.warnings = append(issues.warnings, checkOrbitWarnings(inputRoot, p)...)
|
||||
}
|
||||
return issues
|
||||
}
|
||||
|
||||
func checkOrbitWarnings(inputRoot string, p pair) []string {
|
||||
taskName, err := taskDirectoryName(p)
|
||||
if err != nil {
|
||||
taskName = pairLabel(p)
|
||||
}
|
||||
return orbitWarnings(inputRoot, p, taskName)
|
||||
}
|
||||
|
||||
func printCheckReport(writer io.Writer, report checkReport) {
|
||||
if writer == nil {
|
||||
return
|
||||
}
|
||||
encoder := json.NewEncoder(writer)
|
||||
encoder.SetIndent("", " ")
|
||||
_ = encoder.Encode(report)
|
||||
}
|
||||
|
||||
func loadAndValidateInput(inputRoot string) (pairsDocument, error) {
|
||||
if stat, err := os.Stat(inputRoot); err != nil {
|
||||
return pairsDocument{}, fmt.Errorf("input root is not accessible: %w", err)
|
||||
} else if !stat.IsDir() {
|
||||
return pairsDocument{}, fmt.Errorf("input root is not a directory: %s", inputRoot)
|
||||
}
|
||||
|
||||
dataDir := filepath.Join(inputRoot, "data")
|
||||
if stat, err := os.Stat(dataDir); err != nil {
|
||||
return pairsDocument{}, fmt.Errorf("data directory is not accessible: %w", err)
|
||||
} else if !stat.IsDir() {
|
||||
return pairsDocument{}, fmt.Errorf("data path is not a directory: %s", dataDir)
|
||||
}
|
||||
|
||||
pairsPath := filepath.Join(inputRoot, pairsFileName)
|
||||
file, err := os.Open(pairsPath)
|
||||
if err != nil {
|
||||
return pairsDocument{}, fmt.Errorf("open pairs.json: %w", err)
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
var doc pairsDocument
|
||||
decoder := json.NewDecoder(file)
|
||||
decoder.DisallowUnknownFields()
|
||||
if err := decoder.Decode(&doc); err != nil {
|
||||
return pairsDocument{}, fmt.Errorf("parse pairs.json: %w", err)
|
||||
}
|
||||
if len(doc.Pairs) == 0 {
|
||||
return pairsDocument{}, errors.New("pairs.json contains no pairs")
|
||||
}
|
||||
return doc, nil
|
||||
}
|
||||
|
||||
func validatePairPaths(inputRoot string, p pair) error {
|
||||
if strings.TrimSpace(p.MasterData) == "" {
|
||||
return fmt.Errorf("pair %s missing master_data", pairLabel(p))
|
||||
}
|
||||
if strings.TrimSpace(p.SlaveData) == "" {
|
||||
return fmt.Errorf("pair %s missing slave_data", pairLabel(p))
|
||||
}
|
||||
if _, err := safeJoin(inputRoot, p.MasterData); err != nil {
|
||||
return fmt.Errorf("pair %s invalid master_data: %w", pairLabel(p), err)
|
||||
}
|
||||
if _, err := safeJoin(inputRoot, p.SlaveData); err != nil {
|
||||
return fmt.Errorf("pair %s invalid slave_data: %w", pairLabel(p), err)
|
||||
}
|
||||
if strings.TrimSpace(p.MasterOrbit) != "" {
|
||||
if _, err := safeJoin(inputRoot, p.MasterOrbit); err != nil {
|
||||
return fmt.Errorf("pair %s invalid master_orbit: %w", pairLabel(p), err)
|
||||
}
|
||||
}
|
||||
if strings.TrimSpace(p.SlaveOrbit) != "" {
|
||||
if _, err := safeJoin(inputRoot, p.SlaveOrbit); err != nil {
|
||||
return fmt.Errorf("pair %s invalid slave_orbit: %w", pairLabel(p), err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
var errSkipped = errors.New("skipped")
|
||||
|
||||
func restorePair(cfg config, inputRoot string, outputRoot string, p pair, logger *log.Logger) ([]string, error) {
|
||||
taskDirName, err := taskDirectoryName(p)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
taskDir := filepath.Join(outputRoot, taskDirName)
|
||||
if !isSubpath(outputRoot, taskDir) {
|
||||
return nil, fmt.Errorf("task directory escapes output root: %s", taskDirName)
|
||||
}
|
||||
tempDir := taskDir + tempSuffix
|
||||
|
||||
masterSource, _ := safeJoin(inputRoot, p.MasterData)
|
||||
slaveSource, _ := safeJoin(inputRoot, p.SlaveData)
|
||||
|
||||
if err := requireDir(masterSource, "master_data"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := requireDir(slaveSource, "slave_data"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
existingComplete, err := taskComplete(taskDir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if existingComplete && cfg.skipExisting {
|
||||
logger.Printf("skip existing task=%s", taskDirName)
|
||||
return nil, errSkipped
|
||||
}
|
||||
if cfg.dryRun {
|
||||
logger.Printf("dry-run restore task=%s master=%s slave=%s", taskDirName, p.MasterData, p.SlaveData)
|
||||
return orbitWarnings(inputRoot, p, taskDirName), nil
|
||||
}
|
||||
if pathExists(taskDir) && !cfg.overwrite {
|
||||
return nil, fmt.Errorf("task already exists and is not complete; use --overwrite or remove it: %s", taskDir)
|
||||
}
|
||||
if cfg.overwrite {
|
||||
if err := os.RemoveAll(taskDir); err != nil {
|
||||
return nil, fmt.Errorf("remove existing task %s: %w", taskDir, err)
|
||||
}
|
||||
}
|
||||
if pathExists(tempDir) {
|
||||
if err := os.RemoveAll(tempDir); err != nil {
|
||||
return nil, fmt.Errorf("remove stale temp directory %s: %w", tempDir, err)
|
||||
}
|
||||
}
|
||||
|
||||
logger.Printf("restore task=%s", taskDirName)
|
||||
if err := os.MkdirAll(tempDir, 0755); err != nil {
|
||||
return nil, fmt.Errorf("create temp task directory: %w", err)
|
||||
}
|
||||
|
||||
cleanupTemp := true
|
||||
defer func() {
|
||||
if cleanupTemp {
|
||||
_ = os.RemoveAll(tempDir)
|
||||
}
|
||||
}()
|
||||
|
||||
if err := copyDirContents(masterSource, filepath.Join(tempDir, "master")); err != nil {
|
||||
return nil, fmt.Errorf("copy master data: %w", err)
|
||||
}
|
||||
if err := copyDirContents(slaveSource, filepath.Join(tempDir, "slave")); err != nil {
|
||||
return nil, fmt.Errorf("copy slave data: %w", err)
|
||||
}
|
||||
|
||||
warnings := copyOrbitFiles(inputRoot, tempDir, taskDirName, p, logger)
|
||||
|
||||
meta := pairMetadata{
|
||||
PairID: p.PairID,
|
||||
TaskName: p.TaskName,
|
||||
TaskAlias: p.TaskAlias,
|
||||
MasterData: p.MasterData,
|
||||
SlaveData: p.SlaveData,
|
||||
MasterOrbit: p.MasterOrbit,
|
||||
SlaveOrbit: p.SlaveOrbit,
|
||||
MasterImagingDate: p.MasterImagingDate,
|
||||
SlaveImagingDate: p.SlaveImagingDate,
|
||||
TimeBaselineDays: p.TimeBaselineDays,
|
||||
RestoredAt: time.Now().UTC().Format(time.RFC3339),
|
||||
}
|
||||
if err := writeJSON(filepath.Join(tempDir, pairMetaName), meta); err != nil {
|
||||
return warnings, fmt.Errorf("write pair metadata: %w", err)
|
||||
}
|
||||
if err := validateRestoredTask(tempDir); err != nil {
|
||||
return warnings, err
|
||||
}
|
||||
if err := os.Rename(tempDir, taskDir); err != nil {
|
||||
return warnings, fmt.Errorf("publish restored task: %w", err)
|
||||
}
|
||||
cleanupTemp = false
|
||||
return warnings, nil
|
||||
}
|
||||
|
||||
func taskDirectoryName(p pair) (string, error) {
|
||||
for _, candidate := range []string{p.TaskAlias, p.TaskName, p.PairID} {
|
||||
name := strings.TrimSpace(candidate)
|
||||
if name == "" {
|
||||
continue
|
||||
}
|
||||
if name != filepath.Base(name) || strings.ContainsAny(name, `/\:`) {
|
||||
return "", fmt.Errorf("invalid task directory name: %q", name)
|
||||
}
|
||||
return name, nil
|
||||
}
|
||||
return "", errors.New("pair has no task_alias, task_name, or pair_id")
|
||||
}
|
||||
|
||||
func copyOrbitFiles(inputRoot string, tempDir string, taskDirName string, p pair, logger *log.Logger) []string {
|
||||
var warnings []string
|
||||
for _, entry := range []struct {
|
||||
label string
|
||||
value string
|
||||
}{
|
||||
{label: "master_orbit", value: p.MasterOrbit},
|
||||
{label: "slave_orbit", value: p.SlaveOrbit},
|
||||
} {
|
||||
if strings.TrimSpace(entry.value) == "" {
|
||||
continue
|
||||
}
|
||||
source, err := safeJoin(inputRoot, entry.value)
|
||||
if err != nil {
|
||||
warning := fmt.Sprintf("task %s %s invalid: %v", taskDirName, entry.label, err)
|
||||
warnings = append(warnings, warning)
|
||||
logger.Printf("warning %s", warning)
|
||||
continue
|
||||
}
|
||||
stat, err := os.Stat(source)
|
||||
if err != nil {
|
||||
warning := fmt.Sprintf("task %s %s missing: %s", taskDirName, entry.label, entry.value)
|
||||
warnings = append(warnings, warning)
|
||||
logger.Printf("warning %s", warning)
|
||||
continue
|
||||
}
|
||||
if stat.IsDir() {
|
||||
warning := fmt.Sprintf("task %s %s is a directory, expected file: %s", taskDirName, entry.label, entry.value)
|
||||
warnings = append(warnings, warning)
|
||||
logger.Printf("warning %s", warning)
|
||||
continue
|
||||
}
|
||||
orbitDir := filepath.Join(tempDir, "orbit")
|
||||
if err := os.MkdirAll(orbitDir, 0755); err != nil {
|
||||
warning := fmt.Sprintf("task %s cannot create orbit directory: %v", taskDirName, err)
|
||||
warnings = append(warnings, warning)
|
||||
logger.Printf("warning %s", warning)
|
||||
continue
|
||||
}
|
||||
if err := copyFile(source, filepath.Join(orbitDir, filepath.Base(source)), stat.Mode()); err != nil {
|
||||
warning := fmt.Sprintf("task %s cannot copy %s %s: %v", taskDirName, entry.label, entry.value, err)
|
||||
warnings = append(warnings, warning)
|
||||
logger.Printf("warning %s", warning)
|
||||
continue
|
||||
}
|
||||
}
|
||||
return warnings
|
||||
}
|
||||
|
||||
func orbitWarnings(inputRoot string, p pair, taskDirName string) []string {
|
||||
var warnings []string
|
||||
for _, entry := range []struct {
|
||||
label string
|
||||
value string
|
||||
}{
|
||||
{label: "master_orbit", value: p.MasterOrbit},
|
||||
{label: "slave_orbit", value: p.SlaveOrbit},
|
||||
} {
|
||||
if strings.TrimSpace(entry.value) == "" {
|
||||
continue
|
||||
}
|
||||
source, err := safeJoin(inputRoot, entry.value)
|
||||
if err != nil {
|
||||
warnings = append(warnings, fmt.Sprintf("task %s %s invalid: %v", taskDirName, entry.label, err))
|
||||
continue
|
||||
}
|
||||
if stat, err := os.Stat(source); err != nil {
|
||||
warnings = append(warnings, fmt.Sprintf("task %s %s missing: %s", taskDirName, entry.label, entry.value))
|
||||
} else if stat.IsDir() {
|
||||
warnings = append(warnings, fmt.Sprintf("task %s %s is a directory, expected file: %s", taskDirName, entry.label, entry.value))
|
||||
}
|
||||
}
|
||||
return warnings
|
||||
}
|
||||
|
||||
func copyDirContents(sourceDir string, destDir string) error {
|
||||
sourceEntries, err := os.ReadDir(sourceDir)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.MkdirAll(destDir, 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, entry := range sourceEntries {
|
||||
sourcePath := filepath.Join(sourceDir, entry.Name())
|
||||
destPath := filepath.Join(destDir, entry.Name())
|
||||
info, err := entry.Info()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
mode := info.Mode()
|
||||
switch {
|
||||
case mode&os.ModeSymlink != 0:
|
||||
return fmt.Errorf("symlinks are not supported: %s", sourcePath)
|
||||
case info.IsDir():
|
||||
if err := copyDirContents(sourcePath, destPath); err != nil {
|
||||
return err
|
||||
}
|
||||
case mode.IsRegular():
|
||||
if err := copyFile(sourcePath, destPath, mode); err != nil {
|
||||
return err
|
||||
}
|
||||
default:
|
||||
return fmt.Errorf("unsupported file type: %s", sourcePath)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func copyFile(sourcePath string, destPath string, mode os.FileMode) error {
|
||||
if err := os.MkdirAll(filepath.Dir(destPath), 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
source, err := os.Open(sourcePath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer source.Close()
|
||||
|
||||
dest, err := os.OpenFile(destPath, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, mode)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := io.Copy(dest, source); err != nil {
|
||||
_ = dest.Close()
|
||||
return err
|
||||
}
|
||||
return dest.Close()
|
||||
}
|
||||
|
||||
func taskComplete(taskDir string) (bool, error) {
|
||||
masterNonEmpty, err := dirNonEmpty(filepath.Join(taskDir, "master"))
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
slaveNonEmpty, err := dirNonEmpty(filepath.Join(taskDir, "slave"))
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return masterNonEmpty && slaveNonEmpty, nil
|
||||
}
|
||||
|
||||
func validateRestoredTask(taskDir string) error {
|
||||
masterNonEmpty, err := dirNonEmpty(filepath.Join(taskDir, "master"))
|
||||
if err != nil {
|
||||
return fmt.Errorf("validate master directory: %w", err)
|
||||
}
|
||||
if !masterNonEmpty {
|
||||
return errors.New("restored Task/master is empty")
|
||||
}
|
||||
slaveNonEmpty, err := dirNonEmpty(filepath.Join(taskDir, "slave"))
|
||||
if err != nil {
|
||||
return fmt.Errorf("validate slave directory: %w", err)
|
||||
}
|
||||
if !slaveNonEmpty {
|
||||
return errors.New("restored Task/slave is empty")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(taskDir, pairMetaName)); err != nil {
|
||||
return fmt.Errorf("validate pair metadata: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func dirNonEmpty(path string) (bool, error) {
|
||||
entries, err := os.ReadDir(path)
|
||||
if err != nil {
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return false, nil
|
||||
}
|
||||
return false, err
|
||||
}
|
||||
return len(entries) > 0, nil
|
||||
}
|
||||
|
||||
func requireDir(path string, label string) error {
|
||||
stat, err := os.Stat(path)
|
||||
if err != nil {
|
||||
return fmt.Errorf("%s is not accessible: %w", label, err)
|
||||
}
|
||||
if !stat.IsDir() {
|
||||
return fmt.Errorf("%s is not a directory: %s", label, path)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func writeJSON(path string, value any) error {
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
file, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0644)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
encoder := json.NewEncoder(file)
|
||||
encoder.SetIndent("", " ")
|
||||
return encoder.Encode(value)
|
||||
}
|
||||
|
||||
func newLogger(outputRoot string, dryRun bool, output io.Writer) (*log.Logger, *os.File, error) {
|
||||
if output == nil {
|
||||
output = io.Discard
|
||||
}
|
||||
if dryRun {
|
||||
return log.New(output, "", log.LstdFlags), nil, nil
|
||||
}
|
||||
if err := os.MkdirAll(outputRoot, 0755); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
logFile, err := os.OpenFile(filepath.Join(outputRoot, logFileName), os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0644)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
writer := io.MultiWriter(output, logFile)
|
||||
return log.New(writer, "", log.LstdFlags), logFile, nil
|
||||
}
|
||||
|
||||
func defaultLogWriter(writer io.Writer) io.Writer {
|
||||
if writer != nil {
|
||||
return writer
|
||||
}
|
||||
return os.Stdout
|
||||
}
|
||||
|
||||
func safeJoin(root string, relative string) (string, error) {
|
||||
if filepath.IsAbs(relative) {
|
||||
return "", fmt.Errorf("absolute path is not allowed: %s", relative)
|
||||
}
|
||||
cleanRelative := filepath.Clean(relative)
|
||||
if cleanRelative == "." || strings.HasPrefix(cleanRelative, ".."+string(filepath.Separator)) || cleanRelative == ".." {
|
||||
return "", fmt.Errorf("path escapes root: %s", relative)
|
||||
}
|
||||
joined := filepath.Join(root, cleanRelative)
|
||||
if !isSubpath(root, joined) {
|
||||
return "", fmt.Errorf("path escapes root: %s", relative)
|
||||
}
|
||||
return joined, nil
|
||||
}
|
||||
|
||||
func isSubpath(root string, path string) bool {
|
||||
absRoot, err := filepath.Abs(root)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
absPath, err := filepath.Abs(path)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
rel, err := filepath.Rel(absRoot, absPath)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
return rel == "." || (!strings.HasPrefix(rel, ".."+string(filepath.Separator)) && rel != "..")
|
||||
}
|
||||
|
||||
func pathExists(path string) bool {
|
||||
_, err := os.Stat(path)
|
||||
return err == nil
|
||||
}
|
||||
|
||||
func pairLabel(p pair) string {
|
||||
if strings.TrimSpace(p.PairID) != "" {
|
||||
return p.PairID
|
||||
}
|
||||
if strings.TrimSpace(p.TaskName) != "" {
|
||||
return p.TaskName
|
||||
}
|
||||
if strings.TrimSpace(p.TaskAlias) != "" {
|
||||
return p.TaskAlias
|
||||
}
|
||||
return "<unknown>"
|
||||
}
|
||||
+242
@@ -0,0 +1,242 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRunRestoresPair(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
input := filepath.Join(root, "bundle")
|
||||
output := filepath.Join(root, "out")
|
||||
|
||||
writeTestFile(t, filepath.Join(input, "data", "scene_master", "m.txt"), "master")
|
||||
writeTestFile(t, filepath.Join(input, "data", "scene_slave", "s.txt"), "slave")
|
||||
writeTestFile(t, filepath.Join(input, "orbit", "master.txt"), "master orbit")
|
||||
|
||||
doc := pairsDocument{
|
||||
Schema: "dinsar_source_bundle_pairs.v1",
|
||||
Pairs: []pair{{
|
||||
PairID: "pair_0001",
|
||||
TaskName: "Task_20250101_20250113",
|
||||
TaskAlias: "Task_20250101_20250113",
|
||||
MasterData: "data/scene_master",
|
||||
SlaveData: "data/scene_slave",
|
||||
MasterOrbit: "orbit/master.txt",
|
||||
SlaveOrbit: "orbit/missing.txt",
|
||||
MasterImagingDate: "20250101",
|
||||
SlaveImagingDate: "20250113",
|
||||
TimeBaselineDays: 12,
|
||||
}},
|
||||
}
|
||||
writeTestJSON(t, filepath.Join(input, pairsFileName), doc)
|
||||
|
||||
result, err := run(config{inputRoot: input, outputRoot: output, skipExisting: true})
|
||||
if err != nil {
|
||||
t.Fatalf("run failed: %v", err)
|
||||
}
|
||||
if result.report.Restored != 1 {
|
||||
t.Fatalf("restored = %d, want 1", result.report.Restored)
|
||||
}
|
||||
if len(result.report.Warnings) != 1 {
|
||||
t.Fatalf("warnings = %d, want 1", len(result.report.Warnings))
|
||||
}
|
||||
|
||||
for _, path := range []string{
|
||||
filepath.Join(output, "Task_20250101_20250113", "master", "m.txt"),
|
||||
filepath.Join(output, "Task_20250101_20250113", "slave", "s.txt"),
|
||||
filepath.Join(output, "Task_20250101_20250113", "orbit", "master.txt"),
|
||||
filepath.Join(output, "Task_20250101_20250113", pairMetaName),
|
||||
filepath.Join(output, reportFileName),
|
||||
filepath.Join(output, logFileName),
|
||||
} {
|
||||
if _, err := os.Stat(path); err != nil {
|
||||
t.Fatalf("expected %s to exist: %v", path, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunSkipsExistingCompleteTask(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
input := filepath.Join(root, "bundle")
|
||||
output := filepath.Join(root, "out")
|
||||
task := filepath.Join(output, "Task_20250101_20250113")
|
||||
|
||||
writeTestFile(t, filepath.Join(input, "data", "scene_master", "m.txt"), "master")
|
||||
writeTestFile(t, filepath.Join(input, "data", "scene_slave", "s.txt"), "slave")
|
||||
writeTestFile(t, filepath.Join(task, "master", "old.txt"), "old master")
|
||||
writeTestFile(t, filepath.Join(task, "slave", "old.txt"), "old slave")
|
||||
|
||||
writeTestJSON(t, filepath.Join(input, pairsFileName), pairsDocument{
|
||||
Pairs: []pair{{
|
||||
PairID: "pair_0001",
|
||||
TaskName: "Task_20250101_20250113",
|
||||
MasterData: "data/scene_master",
|
||||
SlaveData: "data/scene_slave",
|
||||
}},
|
||||
})
|
||||
|
||||
result, err := run(config{inputRoot: input, outputRoot: output, skipExisting: true})
|
||||
if err != nil {
|
||||
t.Fatalf("run failed: %v", err)
|
||||
}
|
||||
if result.report.Skipped != 1 {
|
||||
t.Fatalf("skipped = %d, want 1", result.report.Skipped)
|
||||
}
|
||||
if result.report.Restored != 0 {
|
||||
t.Fatalf("restored = %d, want 0", result.report.Restored)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunLimitContinuesAfterExistingTasks(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
input := filepath.Join(root, "bundle")
|
||||
output := filepath.Join(root, "out")
|
||||
|
||||
var pairs []pair
|
||||
for i := 1; i <= 4; i++ {
|
||||
taskName := "Task_20250101_202501" + string(rune('0'+i))
|
||||
masterDir := filepath.Join(input, "data", "scene_master_"+string(rune('0'+i)))
|
||||
slaveDir := filepath.Join(input, "data", "scene_slave_"+string(rune('0'+i)))
|
||||
writeTestFile(t, filepath.Join(masterDir, "m.txt"), "master")
|
||||
writeTestFile(t, filepath.Join(slaveDir, "s.txt"), "slave")
|
||||
pairs = append(pairs, pair{
|
||||
PairID: "pair_000" + string(rune('0'+i)),
|
||||
TaskName: taskName,
|
||||
MasterData: filepath.ToSlash(filepath.Join("data", "scene_master_"+string(rune('0'+i)))),
|
||||
SlaveData: filepath.ToSlash(filepath.Join("data", "scene_slave_"+string(rune('0'+i)))),
|
||||
})
|
||||
}
|
||||
writeTestJSON(t, filepath.Join(input, pairsFileName), pairsDocument{Pairs: pairs})
|
||||
|
||||
first, err := run(config{inputRoot: input, outputRoot: output, skipExisting: true, limit: 2, logWriter: io.Discard})
|
||||
if err != nil {
|
||||
t.Fatalf("first run failed: %v", err)
|
||||
}
|
||||
if first.report.Restored != 2 {
|
||||
t.Fatalf("first restored = %d, want 2", first.report.Restored)
|
||||
}
|
||||
|
||||
second, err := run(config{inputRoot: input, outputRoot: output, skipExisting: true, limit: 2, logWriter: io.Discard})
|
||||
if err != nil {
|
||||
t.Fatalf("second run failed: %v", err)
|
||||
}
|
||||
if second.report.Restored != 2 {
|
||||
t.Fatalf("second restored = %d, want 2", second.report.Restored)
|
||||
}
|
||||
if second.report.Skipped != 2 {
|
||||
t.Fatalf("second skipped = %d, want 2", second.report.Skipped)
|
||||
}
|
||||
for _, p := range pairs {
|
||||
if _, err := os.Stat(filepath.Join(output, p.TaskName, pairMetaName)); err != nil {
|
||||
t.Fatalf("expected %s metadata to exist: %v", p.TaskName, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunCheckReportsMissingSlaveData(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
input := filepath.Join(root, "bundle")
|
||||
|
||||
writeTestFile(t, filepath.Join(input, "data", "scene_master", "m.txt"), "master")
|
||||
writeTestJSON(t, filepath.Join(input, pairsFileName), pairsDocument{
|
||||
Pairs: []pair{{
|
||||
PairID: "pair_0001",
|
||||
TaskName: "Task_20250101_20250113",
|
||||
MasterData: "data/scene_master",
|
||||
SlaveData: "data/scene_missing",
|
||||
}},
|
||||
})
|
||||
|
||||
result, err := runCheck(config{inputRoot: input, logWriter: io.Discard})
|
||||
if err != nil {
|
||||
t.Fatalf("runCheck failed: %v", err)
|
||||
}
|
||||
if result.report.Failed != 1 {
|
||||
t.Fatalf("failed = %d, want 1", result.report.Failed)
|
||||
}
|
||||
if result.report.Valid != 0 {
|
||||
t.Fatalf("valid = %d, want 0", result.report.Valid)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunCheckTreatsMissingOrbitAsWarning(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
input := filepath.Join(root, "bundle")
|
||||
|
||||
writeTestFile(t, filepath.Join(input, "data", "scene_master", "m.txt"), "master")
|
||||
writeTestFile(t, filepath.Join(input, "data", "scene_slave", "s.txt"), "slave")
|
||||
writeTestJSON(t, filepath.Join(input, pairsFileName), pairsDocument{
|
||||
Pairs: []pair{{
|
||||
PairID: "pair_0001",
|
||||
TaskName: "Task_20250101_20250113",
|
||||
MasterData: "data/scene_master",
|
||||
SlaveData: "data/scene_slave",
|
||||
MasterOrbit: "orbit/missing.txt",
|
||||
}},
|
||||
})
|
||||
|
||||
result, err := runCheck(config{inputRoot: input, logWriter: io.Discard})
|
||||
if err != nil {
|
||||
t.Fatalf("runCheck failed: %v", err)
|
||||
}
|
||||
if result.report.Failed != 0 {
|
||||
t.Fatalf("failed = %d, want 0", result.report.Failed)
|
||||
}
|
||||
if result.report.Valid != 1 {
|
||||
t.Fatalf("valid = %d, want 1", result.report.Valid)
|
||||
}
|
||||
if len(result.report.Warnings) != 1 {
|
||||
t.Fatalf("warnings = %d, want 1", len(result.report.Warnings))
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseFlagsRejectsSkipExistingWithOverwrite(t *testing.T) {
|
||||
_, err := parseFlags([]string{"--input", "in", "--output", "out", "--skip-existing", "--overwrite"})
|
||||
if err == nil {
|
||||
t.Fatal("expected error")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseFlagsAllowsOverwriteByItself(t *testing.T) {
|
||||
cfg, err := parseFlags([]string{"--input", "in", "--output", "out", "--overwrite"})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if cfg.skipExisting {
|
||||
t.Fatal("skipExisting = true, want false")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSafeJoinRejectsEscapingPath(t *testing.T) {
|
||||
if _, err := safeJoin(t.TempDir(), "../outside"); err == nil {
|
||||
t.Fatal("expected escaping path error")
|
||||
}
|
||||
}
|
||||
|
||||
func writeTestFile(t *testing.T, path string, content string) {
|
||||
t.Helper()
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(path, []byte(content), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func writeTestJSON(t *testing.T, path string, value any) {
|
||||
t.Helper()
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
bytes, err := json.MarshalIndent(value, "", " ")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(path, bytes, 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user