Harmon/map-asset-gateway
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial import of map-asset-gateway

Browse Source
This commit is contained in:
Harmon
2026-04-14 13:10:10 +08:00
commit 1893ad3857
30 changed files with 6676 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+32
View File
@@ -0,0 +1,32 @@
# Local machine configuration
config/service.env
# Runtime data and generated assets
data/
logs/
temp/
.tmp/
# Go local caches
.gocache/
.gomodcache/
.gotmp/
.gopath/
# Python cache
__pycache__/
*.pyc
*.pyo
# Build and test artifacts
*.exe
*.test
*.out
*.err
*.log
*.pid
coverage.out
# OS / editor noise
.DS_Store
Thumbs.db
README.md
+399
View File
@@ -0,0 +1,399 @@
# map-asset-gateway
`map-asset-gateway` 是一个面向现有底图瓦片和矢量文件的目录管理与分发服务。
它不负责生成瓦片，不负责 GIS 生产流程，也不负责旧版遥感结果处理流水线。它只做这几件事：
- 扫描已有的底图瓦片目录
- 扫描已有的 GeoJSON / Shapefile 文件
- 将元数据写入 SQLite
- 为底图和矢量资源签发访问 token
- 按 token 提供 catalog、瓦片访问和矢量下载
- 将底图或矢量描述推送给下游系统
## 技术栈
- Go
- SQLite
- Go 标准库 `net/http`
## 适用场景
适合已经有离线瓦片目录、GeoJSON、SHP 数据，希望补一层统一管理、授权和分发接口的场景。
不适合下面这些需求：
- 在线切片或重建瓦片
- 栅格处理、预览图生成、图例生成
- PostgreSQL / PostGIS 任务流水线
- Python worker 驱动的结果管理系统
## 运行架构
服务启动时会按下面的顺序运行：
1. 读取 `config/service.env` 或环境变量
2. 打开 SQLite 并自动建表
3. 注册默认扫描源
4. 如果 `AUTO_SCAN_ON_START=true`，自动扫描底图和矢量目录
5. 启动 HTTP API 和管理页
这是一个单进程服务，没有额外 worker，没有外部数据库依赖。
## 目录结构
```text
.
├─ services/api-go/ Go 服务源码
├─ config/ 环境变量配置
├─ scripts/ 启动脚本
├─ docs/ 接入文档
├─ data/ 本地数据目录，不建议提交到 Git
├─ logs/ 本地运行日志，不建议提交到 Git
└─ temp/ 本地临时文件，不建议提交到 Git
```
说明：
- `services/api-go` 是当前项目主体。
- 历史遗留 demo 已从仓库中移除，避免和当前服务边界混淆。
- `data/`、`logs/`、`temp/`、`.gocache/`、`.gomodcache/`、`.gotmp/`、`.gopath/` 都属于本地运行或构建产物。
## 快速启动
### 前置条件
- Go 1.26 或兼容版本
- 可写的本地目录用于放 SQLite、扫描根目录和 Go 缓存
- Windows 环境下可直接使用仓库自带批处理脚本
### 配置
复制示例配置：
```text
config/service.env.example -> config/service.env
```
关键配置项：
| 变量 | 说明 |
| --- | --- |
| `API_HOST` | 监听地址 |
| `API_PORT` | 监听端口 |
| `API_BASE_URL` | API 对外基地址 |
| `TILE_PUBLIC_BASE_URL` | 底图下载地址前缀 |
| `VECTOR_PUBLIC_BASE_URL` | 矢量下载地址前缀 |
| `SQLITE_PATH` | SQLite 文件路径 |
| `BASEMAP_SCAN_ROOT` | 底图扫描根目录 |
| `VECTOR_SCAN_ROOT` | 矢量扫描根目录 |
| `AUTO_SCAN_ON_START` | 启动时是否自动扫描 |
默认示例见 [config/service.env.example](./config/service.env.example)。
### 启动
Windows 下推荐：
```powershell
.\start-all.bat
```
或者直接运行 API：
```powershell
.\scripts\run-api.bat
```
停止：
```powershell
.\stop-all.bat
```
启动后可访问：
- 管理页: `http://127.0.0.1:8910/admin/`
- 健康检查: `http://127.0.0.1:8910/health`
- 就绪检查: `http://127.0.0.1:8910/ready`
## 数据组织与扫描规则
### 底图目录
服务支持两种底图目录形式。
### 1. 带 `manifest.json` 的目录
```text
data/tiles/gaode-image/
manifest.json
tiles/
0/0/0.png
1/0/0.png
```
示例 `manifest.json`：
```json
{
"code": "gaode-image",
"name": "Gaode Image",
"type": "xyz",
"version": "2026-04",
"status": "ready",
"tile_format": "png",
"tile_scheme": "xyz",
"min_zoom": 0,
"max_zoom": 18,
"root_path": "tiles",
"is_default": true
}
```
### 2. 直接是 `z/x/y.ext` 的裸目录
```text
data/tiles/google-image/
0/0/0.webp
1/0/0.webp
```
这类目录会自动导入为：
- `code = 目录名归一化后的值`
- `version = current`
- `type = xyz`
说明：
- `google-image`、`google_image`、`google.image` 最终都会被归一化成类似 `google-image` 的内部 code。
- 默认版本路由 `/tiles/{code}/{z}/{x}/{y}.ext` 会解析到该底图当前默认版本。
### 矢量目录
`VECTOR_SCAN_ROOT` 会被递归扫描。
示例：
```text
data/geojson/admin-boundary.geojson
data/geojson/roads.zip
data/geojson/landuse/parcel.shp
data/geojson/landuse/parcel.dbf
data/geojson/landuse/parcel.shx
```
扫描规则：
- `.geojson` 和 `.json` 会被识别为 `geojson`
- `.zip` 会被识别为 `shpzip`
- `.shp` 会被识别为 `shapefile`
- `shapefile` 下载时会按同名 sidecar 文件动态打包成 zip
## 访问控制
服务使用 token 做资源访问控制。
一个 token 可以授权：
- 一个或多个底图
- 一个或多个矢量资源
支持的 token 传递方式：
- `Authorization: Bearer <token>`
- `X-Access-Token: <token>`
- 查询参数 `token=`
- 查询参数 `access_token=`
## 典型使用流程
### 1. 启动服务并扫描目录
如果开启了 `AUTO_SCAN_ON_START=true`，服务会在启动时自动扫描。
也可以通过管理接口手动触发：
- `POST /api/v1/admin/scan-sources/{code}/run`
- `POST /api/v1/admin/vectors/rescan`
### 2. 创建 token
示例：
```json
POST /api/v1/admin/tokens
{
"name": "insar-v2",
"basemap_codes": ["google-image"],
"vector_codes": ["全国行政区.geojson", "landuse/parcel.shp"]
}
```
### 3. 访问 catalog 或资源
```text
GET /api/v1/catalog/basemaps?token=bmp_xxx
GET /api/v1/catalog/vectors?token=bmp_xxx
GET /tiles/google-image/10/843/388.webp?token=bmp_xxx
GET /tiles/google-image/current/10/843/388.webp?token=bmp_xxx
GET /geojson/全国行政区.geojson?token=bmp_xxx
GET /shp/landuse/parcel.shp?token=bmp_xxx
GET /vectors/landuse-parcel/download?token=bmp_xxx
```
说明：
- `/tiles/{code}/{z}/{x}/{y}.ext` 走默认版本兼容路由
- `/tiles/{code}/{version}/{z}/{x}/{y}.ext` 显式指定版本
- `/geojson/{...}` 支持按原始相对路径、文件名或内部 code 解析
- `/shp/{...}` 会把 `.shp + .dbf + .shx ...` 打成 zip 返回
### 4. 推送给下游系统
先注册目标系统：
- `GET /api/v1/admin/targets`
- `POST /api/v1/admin/targets`
再触发推送：
- `POST /api/v1/admin/pushes`
- `POST /api/v1/admin/vector-pushes`
推送内容是底图版本或矢量资源的描述信息，不是资源文件本体上传。
## API 概览
### 系统接口
- `GET /health`
- `GET /ready`
- `GET /api/v1/system`
### 管理接口
- `GET /admin/`
- `GET /api/v1/admin/dashboard`
- `GET /api/v1/admin/basemaps`
- `GET /api/v1/admin/vectors`
- `GET /api/v1/admin/scan-sources`
- `POST /api/v1/admin/scan-sources`
- `POST /api/v1/admin/scan-sources/{code}/run`
- `POST /api/v1/admin/vectors/rescan`
- `POST /api/v1/admin/basemaps/{code}/default-version`
- `GET /api/v1/admin/targets`
- `POST /api/v1/admin/targets`
- `GET /api/v1/admin/tokens`
- `POST /api/v1/admin/tokens`
- `POST /api/v1/admin/tokens/{id}/disable`
- `GET /api/v1/admin/push-records`
- `POST /api/v1/admin/pushes`
- `GET /api/v1/admin/vector-push-records`
- `POST /api/v1/admin/vector-pushes`
### Catalog 与下载接口
- `GET /api/v1/catalog/basemaps`
- `GET /api/v1/catalog/basemaps/{code}`
- `GET /api/v1/catalog/vectors`
- `GET /api/v1/catalog/vectors/{code}`
- `GET /tiles/{code}/{z}/{x}/{y}.ext`
- `GET /tiles/{code}/{version}/{z}/{x}/{y}.ext`
- `GET /geojson/{relative/path/to/file.geojson}`
- `GET /shp/{relative/path/to/file.shp}`
- `GET /vectors/{code}/download`
## 数据库表
- `basemaps`
- `basemap_versions`
- `scan_sources`
- `scan_runs`
- `target_systems`
- `service_tokens`
- `token_grants`
- `push_records`
- `vector_assets`
- `vector_token_grants`
- `vector_push_records`
## 提交到 Git 前的约定
这个仓库里有很多目录本质上是本地运行数据或缓存，不建议提交到版本库。
默认不建议提交：
- `data/`
- `logs/`
- `temp/`
- `.tmp/`
- `.gocache/`
- `.gomodcache/`
- `.gotmp/`
- `.gopath/`
- `config/service.env`
原因：
- `data/` 里通常是真实底图、真实 GeoJSON/SHP、以及本地 SQLite 数据库
- `logs/` 和 `temp/` 是运行期产物
- `.gocache/`、`.gomodcache/`、`.gotmp/`、`.gopath/` 是本地 Go 构建缓存
- `config/service.env` 往往包含机器本地路径
如果需要给新同事一个可运行的仓库，建议提交：
- 源码
- `config/service.env.example`
- 文档
- 一个最小化的目录约定说明
而不是直接提交大体积瓦片、数据库和本地缓存。
## 开发与测试
在仓库内使用本地 Go 缓存目录跑测试：
```powershell
$env:GOCACHE='C:\path\to\map-asset-gateway\.gocache'
$env:GOMODCACHE='C:\path\to\map-asset-gateway\.gomodcache'
$env:GOTMPDIR='C:\path\to\map-asset-gateway\.gotmp'
$env:GOPATH='C:\path\to\map-asset-gateway\.gopath'
$env:GOPROXY='https://goproxy.cn,direct'
$env:GOSUMDB='off'
go test ./...
```
当前测试主要覆盖：
- 底图扫描
- 矢量扫描
- token 鉴权
- 默认版本瓦片路由
- GeoJSON / SHP 兼容访问路径
## 相关文档
- [config/service.env.example](./config/service.env.example)
- [docs/INSAR_MANAGEMENT_SYSTEM_V2.md](./docs/INSAR_MANAGEMENT_SYSTEM_V2.md)
## 项目边界
再次强调，这个项目当前的边界是：
- 扫描
- 建目录
- 发 token
- 鉴权
- 下载
- 推送资源描述
不包括：
- Python worker
- PostgreSQL
- PostGIS
- 在线切片
- 栅格处理任务
- 结果资产管理流水线
config/service.env.example
+25
View File
@@ -0,0 +1,25 @@
# Copy this file to config/service.env and adjust the machine-specific paths.
APP_ENV=dev
SERVICE_NAME=map-asset-gateway
SERVICE_VERSION=0.2.0
SERVICE_COMMIT=local
API_HOST=127.0.0.1
API_PORT=8910
API_BASE_URL=http://127.0.0.1:8910
TILE_PUBLIC_BASE_URL=http://127.0.0.1:8910
VECTOR_PUBLIC_BASE_URL=http://127.0.0.1:8910
SQLITE_PATH=C:\path\to\map-asset-gateway\data\map-asset-gateway.db
BASEMAP_SCAN_ROOT=C:\path\to\map-asset-gateway\data\tiles
VECTOR_SCAN_ROOT=C:\path\to\map-asset-gateway\data\geojson
AUTO_SCAN_ON_START=true
GOCACHE=C:\path\to\map-asset-gateway\.gocache
GOMODCACHE=C:\path\to\map-asset-gateway\.gomodcache
GOTMPDIR=C:\path\to\map-asset-gateway\.gotmp
GOPATH=C:\path\to\map-asset-gateway\.gopath
GOPROXY=https://goproxy.cn,direct
GOSUMDB=off
scripts/load-env.bat
+18
View File
@@ -0,0 +1,18 @@
@echo off
set "ENV_FILE=%~1"
if "%ENV_FILE%"=="" set "ENV_FILE=%~dp0..\config\service.env"
if not exist "%ENV_FILE%" exit /b 0
for /f "usebackq eol=# tokens=1,* delims==" %%A in ("%ENV_FILE%") do (
if not "%%~A"=="" call :set_if_missing "%%~A" "%%~B"
)
exit /b 0
:set_if_missing
call set "__current=%%%~1%%"
if defined __current goto :clear_and_exit
set "%~1=%~2"
:clear_and_exit
set "__current="
exit /b 0
scripts/run-api.bat
+41
View File
@@ -0,0 +1,41 @@
@echo off
setlocal
chcp 65001 >nul 2>&1
title Map Asset Gateway
set "ROOT=%~dp0..\\"
call "%ROOT%scripts\load-env.bat" "%ROOT%config\service.env"
if "%GOCACHE%"=="" set "GOCACHE=%ROOT%.gocache"
if "%GOMODCACHE%"=="" set "GOMODCACHE=%ROOT%.gomodcache"
if "%GOTMPDIR%"=="" set "GOTMPDIR=%ROOT%.gotmp"
if "%GOPATH%"=="" set "GOPATH=%ROOT%.gopath"
if "%GOPROXY%"=="" set "GOPROXY=https://goproxy.cn,direct"
if "%GOSUMDB%"=="" set "GOSUMDB=off"
if "%API_HOST%"=="" set "API_HOST=127.0.0.1"
if "%API_PORT%"=="" set "API_PORT=8910"
if "%API_BASE_URL%"=="" set "API_BASE_URL=http://127.0.0.1:%API_PORT%"
if "%TILE_PUBLIC_BASE_URL%"=="" set "TILE_PUBLIC_BASE_URL=%API_BASE_URL%"
if "%VECTOR_PUBLIC_BASE_URL%"=="" set "VECTOR_PUBLIC_BASE_URL=%API_BASE_URL%"
if "%SQLITE_PATH%"=="" set "SQLITE_PATH=%ROOT%data\map-asset-gateway.db"
if "%BASEMAP_SCAN_ROOT%"=="" set "BASEMAP_SCAN_ROOT=%ROOT%data\tiles"
if "%VECTOR_SCAN_ROOT%"=="" set "VECTOR_SCAN_ROOT=%ROOT%data\geojson"
echo ========================================
echo Map Asset Gateway
echo ========================================
echo.
echo Env file: %ROOT%config\service.env
echo Listen: %API_BASE_URL%
echo Tile base: %TILE_PUBLIC_BASE_URL%
echo Vector base: %VECTOR_PUBLIC_BASE_URL%
echo SQLite: %SQLITE_PATH%
echo Scan root: %BASEMAP_SCAN_ROOT%
echo Vector root: %VECTOR_SCAN_ROOT%
echo.
pushd "%ROOT%services\api-go"
go run .\cmd\api
popd
if "%NO_PAUSE%"=="" pause
services/api-go/cmd/api/main.go
+124
View File
@@ -0,0 +1,124 @@
package main
import (
"context"
"log"
"net/http"
"os"
"path/filepath"
"strconv"
"strings"
"time"
"map-asset-gateway/api-go/internal/basemap"
"map-asset-gateway/api-go/internal/basemapapi"
"map-asset-gateway/api-go/internal/config"
)
func main() {
cfg, err := config.Load()
if err != nil {
log.Fatalf("load config: %v", err)
}
store, err := basemap.OpenStore(basemap.StoreConfig{
SQLitePath: cfg.SQLitePath,
ScanRoot: cfg.BasemapScanRoot,
VectorScanRoot: cfg.VectorScanRoot,
APIBaseURL: cfg.APIBaseURL,
TileBaseURL: cfg.TilePublicBaseURL,
VectorBaseURL: cfg.VectorPublicBaseURL,
AutoScanOnStart: cfg.AutoScanOnStart,
})
if err != nil {
log.Fatalf("open sqlite store: %v", err)
}
defer func() {
if err := store.Close(); err != nil {
log.Printf("close store: %v", err)
}
}()
systemInfo := basemap.SystemInfo{
ServiceName: cfg.ServiceName,
ServiceVersion: cfg.ServiceVersion,
Commit: cfg.Commit,
Env: cfg.Env,
APIBaseURL: cfg.APIBaseURL,
TileBaseURL: cfg.TilePublicBaseURL,
VectorBaseURL: cfg.VectorPublicBaseURL,
SQLitePath: cfg.SQLitePath,
ScanRoot: cfg.BasemapScanRoot,
VectorScanRoot: cfg.VectorScanRoot,
StartedAt: time.Now().UTC(),
}
if cfg.AutoScanOnStart && strings.TrimSpace(cfg.BasemapScanRoot) != "" {
if _, err := store.RunScan(context.Background(), "local-basemaps"); err != nil {
log.Printf("startup scan skipped: %v", err)
} else {
log.Printf("startup scan completed for %s", cfg.BasemapScanRoot)
}
}
if cfg.AutoScanOnStart && strings.TrimSpace(cfg.VectorScanRoot) != "" {
if _, err := store.RunVectorScan(context.Background()); err != nil {
log.Printf("startup vector scan skipped: %v", err)
} else {
log.Printf("startup vector scan completed for %s", cfg.VectorScanRoot)
}
}
server, err := basemapapi.New(store, systemInfo)
if err != nil {
log.Fatalf("init api server: %v", err)
}
httpServer := &http.Server{
Addr: cfg.ListenAddr(),
Handler: server.Handler(),
ReadHeaderTimeout: 5 * time.Second,
}
cleanupPID, err := writePIDFile(strings.TrimSpace(os.Getenv("RS_API_PID_FILE")))
if err != nil {
log.Fatalf("write pid file: %v", err)
}
if cleanupPID != nil {
defer cleanupPID()
}
log.Printf("map asset gateway listening on http://%s", cfg.ListenAddr())
log.Printf("api base url: %s", cfg.APIBaseURL)
log.Printf("tile public base url: %s", cfg.TilePublicBaseURL)
log.Printf("vector public base url: %s", cfg.VectorPublicBaseURL)
log.Printf("sqlite path: %s", cfg.SQLitePath)
log.Printf("scan root: %s", cfg.BasemapScanRoot)
log.Printf("vector scan root: %s", cfg.VectorScanRoot)
if err := httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
log.Fatalf("listen and serve: %v", err)
}
}
func writePIDFile(path string) (func(), error) {
if path == "" {
return nil, nil
}
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return nil, err
}
pidText := strconv.Itoa(os.Getpid())
if err := os.WriteFile(path, []byte(pidText+"\n"), 0o644); err != nil {
return nil, err
}
cleanup := func() {
content, err := os.ReadFile(path)
if err != nil {
return
}
if strings.TrimSpace(string(content)) == pidText {
_ = os.Remove(path)
}
}
return cleanup, nil
}
services/api-go/go.mod
+21
View File
@@ -0,0 +1,21 @@
module map-asset-gateway/api-go
go 1.26
require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b // indirect
golang.org/x/sys v0.36.0 // indirect
modernc.org/libc v1.66.10 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
)
require (
golang.org/x/sync v0.17.0 // indirect
modernc.org/sqlite v1.39.1
)
services/api-go/go.sum
+49
View File
@@ -0,0 +1,49 @@
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b h1:M2rDM6z3Fhozi9O7NWsxAkg/yqS/lQJ6PmkyIV3YP+o=
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b/go.mod h1:3//PLf8L/X+8b4vuAfHzxeRUl04Adcb341+IGKfnqS8=
golang.org/x/mod v0.27.0 h1:kb+q2PyFnEADO2IEF935ehFUXlWiNjJWtRNgBLSfbxQ=
golang.org/x/mod v0.27.0/go.mod h1:rWI627Fq0DEoudcK+MBkNkCe0EetEaDSwJJkCcjpazc=
golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.36.0 h1:KVRy2GtZBrk1cBYA7MKu5bEZFxQk4NIDV6RLVcC8o0k=
golang.org/x/sys v0.36.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/tools v0.36.0 h1:kWS0uv/zsvHEle1LbV5LE8QujrxB3wfQyxHfhOk0Qkg=
golang.org/x/tools v0.36.0/go.mod h1:WBDiHKJK8YgLHlcQPYQzNCkUxUypCaa5ZegCVutKm+s=
modernc.org/cc/v4 v4.26.5 h1:xM3bX7Mve6G8K8b+T11ReenJOT+BmVqQj0FY5T4+5Y4=
modernc.org/cc/v4 v4.26.5/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=
modernc.org/ccgo/v4 v4.28.1 h1:wPKYn5EC/mYTqBO373jKjvX2n+3+aK7+sICCv4Fjy1A=
modernc.org/ccgo/v4 v4.28.1/go.mod h1:uD+4RnfrVgE6ec9NGguUNdhqzNIeeomeXf6CL0GTE5Q=
modernc.org/fileutil v1.3.40 h1:ZGMswMNc9JOCrcrakF1HrvmergNLAmxOPjizirpfqBA=
modernc.org/fileutil v1.3.40/go.mod h1:HxmghZSZVAz/LXcMNwZPA/DRrQZEVP9VX0V4LQGQFOc=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.66.10 h1:yZkb3YeLx4oynyR+iUsXsybsX4Ubx7MQlSYEw4yj59A=
modernc.org/libc v1.66.10/go.mod h1:8vGSEwvoUoltr4dlywvHqjtAqHBaw0j1jI7iFBTAr2I=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.1.4 h1:2kNGMRiUjrp4LcaPuLY2PzUfqM/w9N23quVwhKt5Qm8=
modernc.org/opt v0.1.4/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.39.1 h1:H+/wGFzuSCIEVCvXYVHX5RQglwhMOvtHSv+VtidL2r4=
modernc.org/sqlite v1.39.1/go.mod h1:9fjQZ0mB1LLP0GYrp39oOJXx/I2sxEnZtzCmEQIKvGE=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
services/api-go/internal/basemap/auth.go
+535
View File
@@ -0,0 +1,535 @@
package basemap
import (
"context"
"crypto/sha256"
"database/sql"
"encoding/hex"
"errors"
"fmt"
"mime"
"os"
"path/filepath"
"sort"
"strings"
"map-asset-gateway/api-go/internal/uid"
)
var ErrBasemapVersionNotFound = errors.New("basemap version not found")
func (s *Store) CreateToken(ctx context.Context, input CreateTokenInput) (CreatedToken, error) {
name := strings.TrimSpace(input.Name)
if name == "" {
return CreatedToken{}, errors.New("token name is required")
}
if len(input.BasemapCodes) == 0 {
if len(input.VectorCodes) == 0 {
return CreatedToken{}, errors.New("at least one basemap code or vector code is required")
}
}
resolvedVectorAssets := map[string]VectorAsset{}
for _, ref := range input.VectorCodes {
ref = strings.TrimSpace(ref)
if ref == "" {
continue
}
asset, err := s.GetVectorAssetByRef(ctx, ref)
if err != nil {
return CreatedToken{}, err
}
resolvedVectorAssets[asset.Code] = asset
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return CreatedToken{}, err
}
defer tx.Rollback()
token, tokenHash, err := s.generateTokenSecret()
if err != nil {
return CreatedToken{}, err
}
now := nowUTC()
tokenID := uid.New()
prefix := token
if len(prefix) > 16 {
prefix = prefix[:16]
}
_, err = tx.ExecContext(ctx, `
INSERT INTO service_tokens (id, name, token_hash, token_prefix, status, expires_at, last_used_at, created_at)
VALUES (?, ?, ?, ?, 'active', ?, NULL, ?)
`, tokenID, name, tokenHash, prefix, nullableTime(input.ExpiresAt), toRFC3339(now))
if err != nil {
return CreatedToken{}, err
}
normalizedCodes := map[string]struct{}{}
for _, code := range input.BasemapCodes {
normalized := normalizeBasemapCode(code)
if normalized != "" {
normalizedCodes[normalized] = struct{}{}
}
}
if len(normalizedCodes) == 0 {
if len(resolvedVectorAssets) == 0 {
return CreatedToken{}, errors.New("no valid basemap codes provided")
}
}
for _, code := range sortedKeys(normalizedCodes) {
var basemapID string
err := tx.QueryRowContext(ctx, `SELECT id FROM basemaps WHERE code = ?`, code).Scan(&basemapID)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return CreatedToken{}, fmt.Errorf("basemap %q not found", code)
}
return CreatedToken{}, err
}
grantID := uid.New()
if _, err := tx.ExecContext(ctx, `
INSERT INTO token_grants (id, token_id, basemap_id, basemap_version_id, permission, created_at)
VALUES (?, ?, ?, NULL, ?, ?)
`, grantID, tokenID, basemapID, readPermission, toRFC3339(now)); err != nil {
return CreatedToken{}, err
}
}
vectorCodes := make([]string, 0, len(resolvedVectorAssets))
for code := range resolvedVectorAssets {
vectorCodes = append(vectorCodes, code)
}
sort.Strings(vectorCodes)
for _, code := range vectorCodes {
asset := resolvedVectorAssets[code]
grantID := uid.New()
if _, err := tx.ExecContext(ctx, `
INSERT INTO vector_token_grants (id, token_id, vector_asset_id, permission, created_at)
VALUES (?, ?, ?, ?, ?)
`, grantID, tokenID, asset.ID, readPermission, toRFC3339(now)); err != nil {
return CreatedToken{}, err
}
}
if err := tx.Commit(); err != nil {
return CreatedToken{}, err
}
meta, err := s.getTokenByID(ctx, tokenID)
if err != nil {
return CreatedToken{}, err
}
return CreatedToken{
Meta: meta,
Token: token,
}, nil
}
func (s *Store) ListTokens(ctx context.Context) ([]ServiceToken, error) {
rows, err := s.db.QueryContext(ctx, `
SELECT id, name, token_hash, token_prefix, status, expires_at, last_used_at, created_at
FROM service_tokens
ORDER BY created_at DESC
`)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ServiceToken
for rows.Next() {
item, _, err := scanServiceToken(rows)
if err != nil {
return nil, err
}
items = append(items, item)
}
if err := rows.Err(); err != nil {
return nil, err
}
grantsByToken, err := s.loadAllTokenGrants(ctx)
if err != nil {
return nil, err
}
vectorGrantsByToken, err := s.loadAllVectorTokenGrants(ctx)
if err != nil {
return nil, err
}
for i := range items {
items[i].Grants = grantsByToken[items[i].ID]
items[i].VectorGrants = vectorGrantsByToken[items[i].ID]
}
return items, nil
}
func (s *Store) loadAllTokenGrants(ctx context.Context) (map[string][]TokenGrant, error) {
rows, err := s.db.QueryContext(ctx, `
SELECT
g.id,
g.token_id,
g.basemap_id,
b.code,
g.basemap_version_id,
v.version,
g.permission,
g.created_at
FROM token_grants g
JOIN basemaps b ON b.id = g.basemap_id
LEFT JOIN basemap_versions v ON v.id = g.basemap_version_id
ORDER BY g.created_at DESC
`)
if err != nil {
return nil, err
}
defer rows.Close()
result := map[string][]TokenGrant{}
for rows.Next() {
item, err := scanTokenGrant(rows)
if err != nil {
return nil, err
}
result[item.TokenID] = append(result[item.TokenID], item)
}
return result, rows.Err()
}
func (s *Store) loadAllVectorTokenGrants(ctx context.Context) (map[string][]VectorTokenGrant, error) {
rows, err := s.db.QueryContext(ctx, `
SELECT
g.id,
g.token_id,
g.vector_asset_id,
v.code,
g.permission,
g.created_at
FROM vector_token_grants g
JOIN vector_assets v ON v.id = g.vector_asset_id
ORDER BY g.created_at DESC
`)
if err != nil {
return nil, err
}
defer rows.Close()
result := map[string][]VectorTokenGrant{}
for rows.Next() {
item, err := scanVectorTokenGrant(rows)
if err != nil {
return nil, err
}
result[item.TokenID] = append(result[item.TokenID], item)
}
return result, rows.Err()
}
func (s *Store) DisableToken(ctx context.Context, tokenID string) error {
result, err := s.db.ExecContext(ctx, `UPDATE service_tokens SET status = 'disabled' WHERE id = ?`, strings.TrimSpace(tokenID))
if err != nil {
return err
}
count, err := result.RowsAffected()
if err != nil {
return err
}
if count == 0 {
return fmt.Errorf("token %q not found", tokenID)
}
return nil
}
func (s *Store) getTokenByID(ctx context.Context, tokenID string) (ServiceToken, error) {
row := s.db.QueryRowContext(ctx, `
SELECT id, name, token_hash, token_prefix, status, expires_at, last_used_at, created_at
FROM service_tokens
WHERE id = ?
`, tokenID)
item, _, err := scanServiceToken(row)
if err != nil {
return ServiceToken{}, err
}
grants, err := s.loadAllTokenGrants(ctx)
if err != nil {
return ServiceToken{}, err
}
item.Grants = grants[item.ID]
vectorGrants, err := s.loadAllVectorTokenGrants(ctx)
if err != nil {
return ServiceToken{}, err
}
item.VectorGrants = vectorGrants[item.ID]
return item, nil
}
func (s *Store) AuthorizeToken(ctx context.Context, rawToken string) (TokenAuth, error) {
rawToken = strings.TrimSpace(rawToken)
if rawToken == "" {
return TokenAuth{}, errors.New("missing token")
}
sum := sha256.Sum256([]byte(rawToken))
tokenHash := hex.EncodeToString(sum[:])
row := s.db.QueryRowContext(ctx, `
SELECT id, name, token_hash, token_prefix, status, expires_at, last_used_at, created_at
FROM service_tokens
WHERE token_hash = ?
`, tokenHash)
item, _, err := scanServiceToken(row)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return TokenAuth{}, errors.New("invalid token")
}
return TokenAuth{}, err
}
if item.Status != "active" {
return TokenAuth{}, errors.New("token is disabled")
}
if item.ExpiresAt != nil && item.ExpiresAt.Before(nowUTC()) {
return TokenAuth{}, errors.New("token is expired")
}
grantsByToken, err := s.loadAllTokenGrants(ctx)
if err != nil {
return TokenAuth{}, err
}
item.Grants = grantsByToken[item.ID]
vectorGrantsByToken, err := s.loadAllVectorTokenGrants(ctx)
if err != nil {
return TokenAuth{}, err
}
item.VectorGrants = vectorGrantsByToken[item.ID]
permissions := map[string]map[string]VersionScope{}
for _, grant := range item.Grants {
group, ok := permissions[grant.Permission]
if !ok {
group = map[string]VersionScope{}
permissions[grant.Permission] = group
}
scope := group[grant.BasemapCode]
if grant.BasemapVersion == nil {
scope.AllVersions = true
scope.Versions = nil
} else if !scope.AllVersions {
if scope.Versions == nil {
scope.Versions = map[string]struct{}{}
}
scope.Versions[*grant.BasemapVersion] = struct{}{}
}
group[grant.BasemapCode] = scope
}
vectorPermissions := map[string]map[string]struct{}{}
for _, grant := range item.VectorGrants {
group, ok := vectorPermissions[grant.Permission]
if !ok {
group = map[string]struct{}{}
vectorPermissions[grant.Permission] = group
}
group[grant.VectorCode] = struct{}{}
}
_, _ = s.db.ExecContext(ctx, `UPDATE service_tokens SET last_used_at = ? WHERE id = ?`, toRFC3339(nowUTC()), item.ID)
return TokenAuth{
Token: item,
Permissions: permissions,
VectorPermissions: vectorPermissions,
}, nil
}
func (auth TokenAuth) CanReadBasemap(code string, version string) bool {
group, ok := auth.Permissions[readPermission]
if !ok {
return false
}
scope, ok := group[normalizeBasemapCode(code)]
if !ok {
return false
}
if scope.AllVersions {
return true
}
if version == "" {
return len(scope.Versions) > 0
}
_, ok = scope.Versions[strings.TrimSpace(version)]
return ok
}
func (s *Store) FilterCatalog(auth TokenAuth, basemaps []Basemap) []Basemap {
filtered := make([]Basemap, 0, len(basemaps))
for _, basemap := range basemaps {
if !auth.CanReadBasemap(basemap.Code, "") {
continue
}
copyItem := basemap
copyItem.Versions = nil
copyItem.Default = nil
for _, version := range basemap.Versions {
if auth.CanReadBasemap(basemap.Code, version.Version) {
copyItem.Versions = append(copyItem.Versions, version)
if version.IsDefault {
copyVersion := version
copyItem.Default = &copyVersion
}
}
}
if len(copyItem.Versions) == 0 {
continue
}
if copyItem.Default == nil {
copyVersion := copyItem.Versions[0]
copyItem.Default = &copyVersion
}
filtered = append(filtered, copyItem)
}
return filtered
}
func (s *Store) ResolveTile(ctx context.Context, auth TokenAuth, basemapCode, version, tilePath string) (TileDescriptor, error) {
basemapCode = normalizeBasemapCode(basemapCode)
version = strings.TrimSpace(version)
tilePath = filepath.Clean(strings.TrimPrefix(strings.TrimSpace(tilePath), "/"))
if tilePath == "." || tilePath == "" {
return TileDescriptor{}, errors.New("tile path is required")
}
if !auth.CanReadBasemap(basemapCode, version) {
return TileDescriptor{}, errors.New("token has no access to this basemap")
}
row := s.db.QueryRowContext(ctx, `
SELECT
v.id,
v.basemap_id,
b.code,
v.version,
v.status,
v.is_default,
v.manifest_path,
v.tile_root_path,
v.url_template,
v.tile_format,
v.tile_scheme,
v.min_zoom,
v.max_zoom,
v.bbox_json,
v.attribution,
v.metadata_json,
v.created_at,
v.updated_at
FROM basemap_versions v
JOIN basemaps b ON b.id = v.basemap_id
WHERE b.code = ? AND v.version = ?
`, basemapCode, version)
versionRow, err := scanBasemapVersionRow(row)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return TileDescriptor{}, fmt.Errorf("%w: %s/%s", ErrBasemapVersionNotFound, basemapCode, version)
}
return TileDescriptor{}, err
}
versionItem := decodeBasemapVersion(versionRow)
basemap, err := s.GetBasemapByCode(ctx, basemapCode)
if err != nil {
return TileDescriptor{}, err
}
filePath, err := safeJoin(versionItem.TileRootPath, tilePath)
if err != nil {
return TileDescriptor{}, err
}
info, err := os.Stat(filePath)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return TileDescriptor{}, fmt.Errorf("tile %s not found", tilePath)
}
return TileDescriptor{}, err
}
if info.IsDir() {
return TileDescriptor{}, errors.New("tile path is a directory")
}
contentType := mime.TypeByExtension(filepath.Ext(filePath))
if contentType == "" {
contentType = "application/octet-stream"
}
return TileDescriptor{
FilePath: filePath,
ContentType: contentType,
Basemap: basemap,
Version: versionItem,
Token: auth.Token,
RelativePath: tilePath,
}, nil
}
func (s *Store) ResolveDefaultTile(ctx context.Context, auth TokenAuth, basemapCode, tilePath string) (TileDescriptor, error) {
basemapCode = normalizeBasemapCode(basemapCode)
if basemapCode == "" {
return TileDescriptor{}, errors.New("basemap code is required")
}
if !auth.CanReadBasemap(basemapCode, "") {
return TileDescriptor{}, errors.New("token has no access to this basemap")
}
version, err := s.GetDefaultBasemapVersion(ctx, basemapCode)
if err != nil {
return TileDescriptor{}, err
}
if !auth.CanReadBasemap(basemapCode, version.Version) {
return TileDescriptor{}, errors.New("token has no access to the default basemap version")
}
return s.ResolveTile(ctx, auth, basemapCode, version.Version, tilePath)
}
func (s *Store) GetDefaultBasemapVersion(ctx context.Context, basemapCode string) (BasemapVersion, error) {
basemapCode = normalizeBasemapCode(basemapCode)
row := s.db.QueryRowContext(ctx, `
SELECT
v.id,
v.basemap_id,
b.code,
v.version,
v.status,
v.is_default,
v.manifest_path,
v.tile_root_path,
v.url_template,
v.tile_format,
v.tile_scheme,
v.min_zoom,
v.max_zoom,
v.bbox_json,
v.attribution,
v.metadata_json,
v.created_at,
v.updated_at
FROM basemap_versions v
JOIN basemaps b ON b.id = v.basemap_id
WHERE b.code = ?
ORDER BY v.is_default DESC, v.version ASC
LIMIT 1
`, basemapCode)
versionRow, err := scanBasemapVersionRow(row)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return BasemapVersion{}, fmt.Errorf("basemap %q has no versions", basemapCode)
}
return BasemapVersion{}, err
}
return decodeBasemapVersion(versionRow), nil
}
func safeJoin(root, relative string) (string, error) {
root = filepath.Clean(root)
if root == "." || root == "" {
return "", errors.New("invalid tile root")
}
target := filepath.Clean(filepath.Join(root, relative))
rel, err := filepath.Rel(root, target)
if err != nil {
return "", err
}
if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return "", errors.New("invalid tile path")
}
return target, nil
}
services/api-go/internal/basemap/catalog.go
+302
View File
@@ -0,0 +1,302 @@
package basemap
import (
"context"
"database/sql"
"errors"
"fmt"
"path/filepath"
"strings"
"map-asset-gateway/api-go/internal/uid"
)
func (s *Store) EnsureScanSource(ctx context.Context, input CreateScanSourceInput) (ScanSource, error) {
code := normalizeBasemapCode(input.Code)
if code == "" {
return ScanSource{}, errors.New("scan source code is required")
}
name := strings.TrimSpace(input.Name)
if name == "" {
name = code
}
rootPath := strings.TrimSpace(input.RootPath)
if rootPath == "" {
return ScanSource{}, errors.New("scan source root path is required")
}
absRoot, err := filepath.Abs(rootPath)
if err != nil {
return ScanSource{}, fmt.Errorf("resolve root path: %w", err)
}
manifestName := strings.TrimSpace(input.ManifestName)
if manifestName == "" {
manifestName = defaultManifestName
}
now := nowUTC()
id := uid.Deterministic("scan-source", code)
_, err = s.db.ExecContext(ctx, `
INSERT INTO scan_sources (
id, code, name, root_path, manifest_name, enabled, metadata_json, created_at, updated_at
) VALUES (?, ?, ?, ?, ?, 1, '', ?, ?)
ON CONFLICT(code) DO UPDATE SET
name = excluded.name,
root_path = excluded.root_path,
manifest_name = excluded.manifest_name,
enabled = 1,
updated_at = excluded.updated_at
`, id, code, name, absRoot, manifestName, toRFC3339(now), toRFC3339(now))
if err != nil {
return ScanSource{}, fmt.Errorf("upsert scan source: %w", err)
}
return s.GetScanSourceByCode(ctx, code)
}
func (s *Store) GetScanSourceByCode(ctx context.Context, code string) (ScanSource, error) {
row := s.db.QueryRowContext(ctx, `
SELECT id, code, name, root_path, manifest_name, enabled, metadata_json, created_at, updated_at
FROM scan_sources
WHERE code = ?
`, normalizeBasemapCode(code))
item, err := scanScanSource(row)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return ScanSource{}, fmt.Errorf("scan source %q not found", code)
}
return ScanSource{}, err
}
return item, nil
}
func (s *Store) ListScanSources(ctx context.Context) ([]ScanSource, error) {
rows, err := s.db.QueryContext(ctx, `
SELECT id, code, name, root_path, manifest_name, enabled, metadata_json, created_at, updated_at
FROM scan_sources
ORDER BY code
`)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ScanSource
for rows.Next() {
item, err := scanScanSource(rows)
if err != nil {
return nil, err
}
items = append(items, item)
}
return items, rows.Err()
}
func (s *Store) ListBasemaps(ctx context.Context) ([]Basemap, error) {
rows, err := s.db.QueryContext(ctx, `
SELECT id, code, name, type, status, description, created_at, updated_at
FROM basemaps
ORDER BY code
`)
if err != nil {
return nil, err
}
defer rows.Close()
var items []Basemap
indexByID := map[string]int{}
for rows.Next() {
item, err := scanBasemap(rows)
if err != nil {
return nil, err
}
indexByID[item.ID] = len(items)
items = append(items, item)
}
if err := rows.Err(); err != nil {
return nil, err
}
versionRows, err := s.db.QueryContext(ctx, `
SELECT
v.id,
v.basemap_id,
b.code,
v.version,
v.status,
v.is_default,
v.manifest_path,
v.tile_root_path,
v.url_template,
v.tile_format,
v.tile_scheme,
v.min_zoom,
v.max_zoom,
v.bbox_json,
v.attribution,
v.metadata_json,
v.created_at,
v.updated_at
FROM basemap_versions v
JOIN basemaps b ON b.id = v.basemap_id
ORDER BY b.code, v.version
`)
if err != nil {
return nil, err
}
defer versionRows.Close()
for versionRows.Next() {
row, err := scanBasemapVersionRow(versionRows)
if err != nil {
return nil, err
}
index, ok := indexByID[row.BasemapID]
if !ok {
continue
}
version := decodeBasemapVersion(row)
items[index].Versions = append(items[index].Versions, version)
if version.IsDefault {
copyValue := version
items[index].Default = &copyValue
}
}
if err := versionRows.Err(); err != nil {
return nil, err
}
return items, nil
}
func (s *Store) GetBasemapByCode(ctx context.Context, code string) (Basemap, error) {
items, err := s.ListBasemaps(ctx)
if err != nil {
return Basemap{}, err
}
normalized := normalizeBasemapCode(code)
for _, item := range items {
if item.Code == normalized {
return item, nil
}
}
return Basemap{}, fmt.Errorf("basemap %q not found", code)
}
func (s *Store) SetDefaultVersion(ctx context.Context, basemapCode, version string) error {
basemapCode = normalizeBasemapCode(basemapCode)
version = strings.TrimSpace(version)
if basemapCode == "" || version == "" {
return errors.New("basemap code and version are required")
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer tx.Rollback()
var basemapID string
if err := tx.QueryRowContext(ctx, `SELECT id FROM basemaps WHERE code = ?`, basemapCode).Scan(&basemapID); err != nil {
if errors.Is(err, sql.ErrNoRows) {
return fmt.Errorf("basemap %q not found", basemapCode)
}
return err
}
result, err := tx.ExecContext(ctx, `
UPDATE basemap_versions
SET is_default = CASE WHEN version = ? THEN 1 ELSE 0 END, updated_at = ?
WHERE basemap_id = ?
`, version, toRFC3339(nowUTC()), basemapID)
if err != nil {
return err
}
count, err := result.RowsAffected()
if err != nil {
return err
}
if count == 0 {
return fmt.Errorf("version %q not found for basemap %q", version, basemapCode)
}
if _, err := tx.ExecContext(ctx, `UPDATE basemaps SET updated_at = ? WHERE id = ?`, toRFC3339(nowUTC()), basemapID); err != nil {
return err
}
return tx.Commit()
}
func (s *Store) ListScanRuns(ctx context.Context, limit int) ([]ScanRun, error) {
if limit <= 0 {
limit = 20
}
rows, err := s.db.QueryContext(ctx, `
SELECT
r.id,
r.scan_source_id,
s.code,
r.status,
r.scanned_count,
r.added_count,
r.updated_count,
r.removed_count,
r.summary_json,
r.started_at,
r.finished_at
FROM scan_runs r
JOIN scan_sources s ON s.id = r.scan_source_id
ORDER BY r.started_at DESC
LIMIT ?
`, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var items []ScanRun
for rows.Next() {
item, err := scanScanRun(rows)
if err != nil {
return nil, err
}
items = append(items, item)
}
return items, rows.Err()
}
func (s *Store) ListPushRecords(ctx context.Context, limit int) ([]PushRecord, error) {
if limit <= 0 {
limit = 20
}
rows, err := s.db.QueryContext(ctx, `
SELECT
p.id,
p.target_system_id,
t.code,
p.basemap_version_id,
b.code,
v.version,
p.status,
p.request_json,
p.response_status,
p.response_body,
p.error_message,
p.pushed_at,
p.finished_at
FROM push_records p
JOIN target_systems t ON t.id = p.target_system_id
JOIN basemap_versions v ON v.id = p.basemap_version_id
JOIN basemaps b ON b.id = v.basemap_id
ORDER BY p.pushed_at DESC
LIMIT ?
`, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var items []PushRecord
for rows.Next() {
item, err := scanPushRecord(rows)
if err != nil {
return nil, err
}
items = append(items, item)
}
return items, rows.Err()
}
services/api-go/internal/basemap/compat_test.go
+106
View File
@@ -0,0 +1,106 @@
package basemap
import (
"context"
"os"
"path/filepath"
"testing"
)
func TestInsarCompatibilityHelpers(t *testing.T) {
ctx := context.Background()
root := t.TempDir()
tileRoot := filepath.Join(root, "tiles")
vectorRoot := filepath.Join(root, "vectors")
sqlitePath := filepath.Join(root, "basemap.db")
if err := os.MkdirAll(filepath.Join(tileRoot, "google_image", "0", "0"), 0o755); err != nil {
t.Fatalf("mkdir tile root: %v", err)
}
if err := os.WriteFile(filepath.Join(tileRoot, "google_image", "0", "0", "0.webp"), []byte("tile"), 0o644); err != nil {
t.Fatalf("write tile: %v", err)
}
if err := os.MkdirAll(filepath.Join(vectorRoot, "landuse"), 0o755); err != nil {
t.Fatalf("mkdir vector root: %v", err)
}
if err := os.WriteFile(filepath.Join(vectorRoot, "全国行政区.geojson"), []byte(`{"type":"FeatureCollection","features":[]}`), 0o644); err != nil {
t.Fatalf("write geojson: %v", err)
}
for _, name := range []string{"parcel.shp", "parcel.dbf", "parcel.shx"} {
if err := os.WriteFile(filepath.Join(vectorRoot, "landuse", name), []byte(name), 0o644); err != nil {
t.Fatalf("write shapefile part %s: %v", name, err)
}
}
store, err := OpenStore(StoreConfig{
SQLitePath: sqlitePath,
ScanRoot: tileRoot,
VectorScanRoot: vectorRoot,
APIBaseURL: "http://127.0.0.1:8910",
TileBaseURL: "http://127.0.0.1:8910",
VectorBaseURL: "http://127.0.0.1:8910",
})
if err != nil {
t.Fatalf("open store: %v", err)
}
defer func() {
if err := store.Close(); err != nil {
t.Fatalf("close store: %v", err)
}
}()
if _, err := store.RunScan(ctx, defaultScanSourceCode); err != nil {
t.Fatalf("run basemap scan: %v", err)
}
if _, err := store.RunVectorScan(ctx); err != nil {
t.Fatalf("run vector scan: %v", err)
}
created, err := store.CreateToken(ctx, CreateTokenInput{
Name: "insar-v2",
BasemapCodes: []string{"google_image"},
VectorCodes: []string{"全国行政区.geojson", "landuse/parcel.shp"},
})
if err != nil {
t.Fatalf("create token: %v", err)
}
auth, err := store.AuthorizeToken(ctx, created.Token)
if err != nil {
t.Fatalf("authorize token: %v", err)
}
tile, err := store.ResolveDefaultTile(ctx, auth, "google_image", "0/0/0.webp")
if err != nil {
t.Fatalf("resolve default tile: %v", err)
}
if tile.Version.Version != "current" {
t.Fatalf("unexpected default version: %s", tile.Version.Version)
}
geojson, err := store.ResolveVectorDownloadByRef(ctx, auth, "全国行政区.geojson", "geojson")
if err != nil {
t.Fatalf("resolve geojson by path: %v", err)
}
if geojson.Asset.Format != "geojson" {
t.Fatalf("unexpected geojson format: %s", geojson.Asset.Format)
}
if filepath.Base(geojson.FilePath) != "全国行政区.geojson" {
t.Fatalf("unexpected geojson file: %s", geojson.FilePath)
}
shp, err := store.ResolveVectorDownloadByRef(ctx, auth, "landuse/parcel.shp", "shapefile")
if err != nil {
t.Fatalf("resolve shapefile by path: %v", err)
}
if !shp.Zipped {
t.Fatal("expected shapefile download to be zipped")
}
if shp.FileName != "parcel.zip" {
t.Fatalf("unexpected shapefile name: %s", shp.FileName)
}
if len(shp.PackageFiles) != 3 {
t.Fatalf("unexpected shapefile package size: %d", len(shp.PackageFiles))
}
}
services/api-go/internal/basemap/dashboard.go
+51
View File
@@ -0,0 +1,51 @@
package basemap
import "context"
func (s *Store) Dashboard(ctx context.Context, info SystemInfo, flash string, createdToken string) (DashboardData, error) {
basemaps, err := s.ListBasemaps(ctx)
if err != nil {
return DashboardData{}, err
}
vectorAssets, err := s.ListVectorAssets(ctx)
if err != nil {
return DashboardData{}, err
}
scanSources, err := s.ListScanSources(ctx)
if err != nil {
return DashboardData{}, err
}
scanRuns, err := s.ListScanRuns(ctx, 10)
if err != nil {
return DashboardData{}, err
}
targetSystems, err := s.ListTargetSystems(ctx)
if err != nil {
return DashboardData{}, err
}
tokens, err := s.ListTokens(ctx)
if err != nil {
return DashboardData{}, err
}
pushRecords, err := s.ListPushRecords(ctx, 10)
if err != nil {
return DashboardData{}, err
}
vectorPushRecords, err := s.ListVectorPushRecords(ctx, 10)
if err != nil {
return DashboardData{}, err
}
return DashboardData{
SystemInfo: info,
Basemaps: basemaps,
VectorAssets: vectorAssets,
ScanSources: scanSources,
ScanRuns: scanRuns,
TargetSystems: targetSystems,
Tokens: tokens,
PushRecords: pushRecords,
VectorPushRecords: vectorPushRecords,
Flash: flash,
CreatedToken: createdToken,
}, nil
}
services/api-go/internal/basemap/helpers.go
+8
View File
@@ -0,0 +1,8 @@
package basemap
func defaultText(value, fallback string) string {
if value != "" {
return value
}
return fallback
}
services/api-go/internal/basemap/manifest.go
+93
View File
@@ -0,0 +1,93 @@
package basemap
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
)
func LoadManifest(path string) (Manifest, error) {
content, err := os.ReadFile(path)
if err != nil {
return Manifest{}, err
}
var item Manifest
if err := json.Unmarshal(content, &item); err != nil {
return Manifest{}, fmt.Errorf("decode manifest %s: %w", path, err)
}
item.Code = normalizeCode(item.Code)
item.Name = strings.TrimSpace(item.Name)
item.Type = defaultText(strings.TrimSpace(item.Type), "xyz")
item.Description = strings.TrimSpace(item.Description)
item.Version = strings.TrimSpace(item.Version)
item.Status = defaultText(strings.TrimSpace(item.Status), "ready")
item.TileFormat = normalizeTileFormat(item.TileFormat)
item.TileScheme = defaultText(strings.TrimSpace(item.TileScheme), "xyz")
item.Attribution = strings.TrimSpace(item.Attribution)
item.RootPath = strings.TrimSpace(item.RootPath)
if item.RootPath == "" {
item.RootPath = "tiles"
}
if item.MaxZoom < item.MinZoom {
return Manifest{}, fmt.Errorf("manifest %s has invalid zoom range", path)
}
if len(item.BBox) != 0 && len(item.BBox) != 4 {
return Manifest{}, fmt.Errorf("manifest %s has invalid bbox", path)
}
if item.Code == "" {
return Manifest{}, fmt.Errorf("manifest %s missing code", path)
}
if item.Name == "" {
item.Name = item.Code
}
if item.Version == "" {
item.Version = filepath.Base(filepath.Dir(path))
}
if item.Metadata == nil {
item.Metadata = map[string]any{}
}
return item, nil
}
func normalizeCode(value string) string {
value = strings.TrimSpace(strings.ToLower(value))
if value == "" {
return ""
}
var builder strings.Builder
lastDash := false
for _, r := range value {
switch {
case r >= 'a' && r <= 'z':
builder.WriteRune(r)
lastDash = false
case r >= '0' && r <= '9':
builder.WriteRune(r)
lastDash = false
case r == '_' || r == '-' || r == '.' || r == ' ':
if builder.Len() > 0 && !lastDash {
builder.WriteByte('-')
lastDash = true
}
}
}
return strings.Trim(builder.String(), "-")
}
func normalizeTileFormat(value string) string {
value = strings.TrimSpace(strings.ToLower(value))
switch value {
case "jpg", "jpeg":
return "jpg"
case "png", "webp", "pbf":
return value
case "":
return "png"
default:
return value
}
}
services/api-go/internal/basemap/models.go
+235
View File
@@ -0,0 +1,235 @@
package basemap
import "time"
type SystemInfo struct {
ServiceName string `json:"service_name"`
ServiceVersion string `json:"service_version"`
Commit string `json:"commit"`
Env string `json:"env"`
APIBaseURL string `json:"api_base_url"`
TileBaseURL string `json:"tile_base_url"`
VectorBaseURL string `json:"vector_base_url"`
SQLitePath string `json:"sqlite_path"`
ScanRoot string `json:"scan_root"`
VectorScanRoot string `json:"vector_scan_root"`
StartedAt time.Time `json:"started_at"`
}
type Basemap struct {
ID string `json:"id"`
Code string `json:"code"`
Name string `json:"name"`
Type string `json:"type"`
Status string `json:"status"`
Description string `json:"description,omitempty"`
Default *BasemapVersion `json:"default_version,omitempty"`
Versions []BasemapVersion `json:"versions,omitempty"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
type BasemapVersion struct {
ID string `json:"id"`
BasemapID string `json:"basemap_id"`
BasemapCode string `json:"basemap_code"`
Version string `json:"version"`
Status string `json:"status"`
IsDefault bool `json:"is_default"`
ManifestPath string `json:"manifest_path"`
TileRootPath string `json:"tile_root_path"`
URLTemplate string `json:"url_template"`
TileFormat string `json:"tile_format"`
TileScheme string `json:"tile_scheme"`
MinZoom int `json:"min_zoom"`
MaxZoom int `json:"max_zoom"`
BBox []float64 `json:"bbox,omitempty"`
Attribution string `json:"attribution,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
type ScanSource struct {
ID string `json:"id"`
Code string `json:"code"`
Name string `json:"name"`
RootPath string `json:"root_path"`
ManifestName string `json:"manifest_name"`
Enabled bool `json:"enabled"`
Metadata map[string]any `json:"metadata,omitempty"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
type ScanRun struct {
ID string `json:"id"`
ScanSourceID string `json:"scan_source_id"`
SourceCode string `json:"source_code"`
Status string `json:"status"`
ScannedCount int `json:"scanned_count"`
AddedCount int `json:"added_count"`
UpdatedCount int `json:"updated_count"`
RemovedCount int `json:"removed_count"`
Summary map[string]any `json:"summary,omitempty"`
StartedAt time.Time `json:"started_at"`
FinishedAt time.Time `json:"finished_at"`
}
type TargetSystem struct {
ID string `json:"id"`
Code string `json:"code"`
Name string `json:"name"`
CallbackURL string `json:"callback_url"`
CallbackMethod string `json:"callback_method"`
CallbackHeaders map[string]string `json:"callback_headers,omitempty"`
Enabled bool `json:"enabled"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
type ServiceToken struct {
ID string `json:"id"`
Name string `json:"name"`
TokenPrefix string `json:"token_prefix"`
Status string `json:"status"`
ExpiresAt *time.Time `json:"expires_at,omitempty"`
LastUsedAt *time.Time `json:"last_used_at,omitempty"`
CreatedAt time.Time `json:"created_at"`
Grants []TokenGrant `json:"grants,omitempty"`
VectorGrants []VectorTokenGrant `json:"vector_grants,omitempty"`
}
type TokenGrant struct {
ID string `json:"id"`
TokenID string `json:"token_id"`
BasemapID string `json:"basemap_id"`
BasemapCode string `json:"basemap_code"`
BasemapVersionID *string `json:"basemap_version_id,omitempty"`
BasemapVersion *string `json:"basemap_version,omitempty"`
Permission string `json:"permission"`
CreatedAt time.Time `json:"created_at"`
}
type PushRecord struct {
ID string `json:"id"`
TargetSystemID string `json:"target_system_id"`
TargetSystemCode string `json:"target_system_code"`
BasemapVersionID string `json:"basemap_version_id"`
BasemapCode string `json:"basemap_code"`
BasemapVersion string `json:"basemap_version"`
Status string `json:"status"`
Request map[string]any `json:"request,omitempty"`
ResponseStatus *int `json:"response_status,omitempty"`
ResponseBody string `json:"response_body,omitempty"`
ErrorMessage string `json:"error_message,omitempty"`
PushedAt time.Time `json:"pushed_at"`
FinishedAt *time.Time `json:"finished_at,omitempty"`
}
type VectorAsset struct {
ID string `json:"id"`
Code string `json:"code"`
Name string `json:"name"`
Format string `json:"format"`
Status string `json:"status"`
Description string `json:"description,omitempty"`
FilePath string `json:"file_path"`
FileName string `json:"file_name"`
FileSize int64 `json:"file_size"`
Checksum string `json:"checksum"`
DownloadURL string `json:"download_url"`
Metadata map[string]any `json:"metadata,omitempty"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
type VectorTokenGrant struct {
ID string `json:"id"`
TokenID string `json:"token_id"`
VectorID string `json:"vector_id"`
VectorCode string `json:"vector_code"`
Permission string `json:"permission"`
CreatedAt time.Time `json:"created_at"`
}
type VectorPushRecord struct {
ID string `json:"id"`
TargetSystemID string `json:"target_system_id"`
TargetSystemCode string `json:"target_system_code"`
VectorAssetID string `json:"vector_asset_id"`
VectorCode string `json:"vector_code"`
Status string `json:"status"`
Request map[string]any `json:"request,omitempty"`
ResponseStatus *int `json:"response_status,omitempty"`
ResponseBody string `json:"response_body,omitempty"`
ErrorMessage string `json:"error_message,omitempty"`
PushedAt time.Time `json:"pushed_at"`
FinishedAt *time.Time `json:"finished_at,omitempty"`
}
type Manifest struct {
Code string `json:"code"`
Name string `json:"name"`
Type string `json:"type"`
Description string `json:"description"`
Version string `json:"version"`
Status string `json:"status"`
TileFormat string `json:"tile_format"`
TileScheme string `json:"tile_scheme"`
MinZoom int `json:"min_zoom"`
MaxZoom int `json:"max_zoom"`
BBox []float64 `json:"bbox"`
Attribution string `json:"attribution"`
RootPath string `json:"root_path"`
IsDefault bool `json:"is_default"`
Metadata map[string]any `json:"metadata"`
}
type CreateScanSourceInput struct {
Code string `json:"code"`
Name string `json:"name"`
RootPath string `json:"root_path"`
ManifestName string `json:"manifest_name"`
}
type CreateTargetSystemInput struct {
Code string `json:"code"`
Name string `json:"name"`
CallbackURL string `json:"callback_url"`
CallbackMethod string `json:"callback_method"`
CallbackHeaders map[string]string `json:"callback_headers"`
}
type CreateTokenInput struct {
Name string
BasemapCodes []string
VectorCodes []string
ExpiresAt *time.Time
}
type TokenAuth struct {
Token ServiceToken
// permission -> basemap_code -> version scope
Permissions map[string]map[string]VersionScope
VectorPermissions map[string]map[string]struct{}
}
type VersionScope struct {
AllVersions bool
Versions map[string]struct{}
}
type DashboardData struct {
SystemInfo SystemInfo
Basemaps []Basemap
VectorAssets []VectorAsset
ScanSources []ScanSource
ScanRuns []ScanRun
TargetSystems []TargetSystem
Tokens []ServiceToken
PushRecords []PushRecord
VectorPushRecords []VectorPushRecord
Flash string
CreatedToken string
}
services/api-go/internal/basemap/push.go
+327
View File
@@ -0,0 +1,327 @@
package basemap
import (
"bytes"
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
"map-asset-gateway/api-go/internal/uid"
)
func (s *Store) CreateTargetSystem(ctx context.Context, input CreateTargetSystemInput) (TargetSystem, error) {
code := normalizeBasemapCode(input.Code)
if code == "" {
return TargetSystem{}, errors.New("target system code is required")
}
name := strings.TrimSpace(input.Name)
if name == "" {
name = code
}
callbackURL := strings.TrimSpace(input.CallbackURL)
if callbackURL == "" {
return TargetSystem{}, errors.New("callback url is required")
}
method := strings.ToUpper(strings.TrimSpace(input.CallbackMethod))
if method == "" {
method = http.MethodPost
}
if input.CallbackHeaders == nil {
input.CallbackHeaders = map[string]string{}
}
now := nowUTC()
id := uid.Deterministic("target-system", code)
_, err := s.db.ExecContext(ctx, `
INSERT INTO target_systems (
id, code, name, callback_url, callback_method, callback_headers_json, enabled, created_at, updated_at
) VALUES (?, ?, ?, ?, ?, ?, 1, ?, ?)
ON CONFLICT(code) DO UPDATE SET
name = excluded.name,
callback_url = excluded.callback_url,
callback_method = excluded.callback_method,
callback_headers_json = excluded.callback_headers_json,
enabled = 1,
updated_at = excluded.updated_at
`, id, code, name, callbackURL, method, writeJSON(input.CallbackHeaders), toRFC3339(now), toRFC3339(now))
if err != nil {
return TargetSystem{}, err
}
return s.GetTargetSystemByCode(ctx, code)
}
func (s *Store) GetTargetSystemByCode(ctx context.Context, code string) (TargetSystem, error) {
row := s.db.QueryRowContext(ctx, `
SELECT id, code, name, callback_url, callback_method, callback_headers_json, enabled, created_at, updated_at
FROM target_systems
WHERE code = ?
`, normalizeBasemapCode(code))
item, err := scanTargetSystem(row)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return TargetSystem{}, fmt.Errorf("target system %q not found", code)
}
return TargetSystem{}, err
}
return item, nil
}
func (s *Store) ListTargetSystems(ctx context.Context) ([]TargetSystem, error) {
rows, err := s.db.QueryContext(ctx, `
SELECT id, code, name, callback_url, callback_method, callback_headers_json, enabled, created_at, updated_at
FROM target_systems
ORDER BY code
`)
if err != nil {
return nil, err
}
defer rows.Close()
var items []TargetSystem
for rows.Next() {
item, err := scanTargetSystem(rows)
if err != nil {
return nil, err
}
items = append(items, item)
}
return items, rows.Err()
}
func (s *Store) PushBasemapVersion(ctx context.Context, targetCode, basemapCode, version string) (PushRecord, error) {
target, err := s.GetTargetSystemByCode(ctx, targetCode)
if err != nil {
return PushRecord{}, err
}
if !target.Enabled {
return PushRecord{}, fmt.Errorf("target system %q is disabled", target.Code)
}
basemapCode = normalizeBasemapCode(basemapCode)
version = strings.TrimSpace(version)
row := s.db.QueryRowContext(ctx, `
SELECT
v.id,
v.basemap_id,
b.code,
v.version,
v.status,
v.is_default,
v.manifest_path,
v.tile_root_path,
v.url_template,
v.tile_format,
v.tile_scheme,
v.min_zoom,
v.max_zoom,
v.bbox_json,
v.attribution,
v.metadata_json,
v.created_at,
v.updated_at
FROM basemap_versions v
JOIN basemaps b ON b.id = v.basemap_id
WHERE b.code = ? AND v.version = ?
`, basemapCode, version)
versionRow, err := scanBasemapVersionRow(row)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return PushRecord{}, fmt.Errorf("basemap version %s/%s not found", basemapCode, version)
}
return PushRecord{}, err
}
versionItem := decodeBasemapVersion(versionRow)
basemapItem, err := s.GetBasemapByCode(ctx, basemapCode)
if err != nil {
return PushRecord{}, err
}
payload := map[string]any{
"event": "basemap.published",
"service": strings.TrimSpace(s.cfg.APIBaseURL),
"basemap": basemapItem,
"version": versionItem,
"pushed_at": toRFC3339(nowUTC()),
"targetCode": target.Code,
}
body, err := json.Marshal(payload)
if err != nil {
return PushRecord{}, err
}
request, err := http.NewRequestWithContext(ctx, target.CallbackMethod, target.CallbackURL, bytes.NewReader(body))
if err != nil {
return PushRecord{}, err
}
request.Header.Set("Content-Type", "application/json")
for key, value := range target.CallbackHeaders {
request.Header.Set(key, value)
}
response, err := s.httpClient.Do(request)
record := PushRecord{
ID: uid.New(),
TargetSystemID: target.ID,
TargetSystemCode: target.Code,
BasemapVersionID: versionItem.ID,
BasemapCode: basemapItem.Code,
BasemapVersion: versionItem.Version,
Request: payload,
PushedAt: nowUTC(),
}
if err != nil {
record.Status = "failed"
record.ErrorMessage = err.Error()
} else {
defer response.Body.Close()
record.ResponseBody = limitedReadAll(response.Body, 256*1024)
status := response.StatusCode
record.ResponseStatus = &status
if status >= 200 && status < 300 {
record.Status = "succeeded"
} else {
record.Status = "failed"
record.ErrorMessage = fmt.Sprintf("target returned status %d", status)
}
}
finishedAt := nowUTC()
record.FinishedAt = &finishedAt
_, insertErr := s.db.ExecContext(ctx, `
INSERT INTO push_records (
id, target_system_id, basemap_version_id, status, request_json, response_status, response_body, error_message, pushed_at, finished_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`, record.ID, record.TargetSystemID, record.BasemapVersionID, record.Status, writeJSON(record.Request), record.ResponseStatus, record.ResponseBody, record.ErrorMessage, toRFC3339(record.PushedAt), nullableTime(record.FinishedAt))
if insertErr != nil {
return PushRecord{}, insertErr
}
if err != nil {
return record, err
}
if record.Status != "succeeded" {
return record, errors.New(record.ErrorMessage)
}
return record, nil
}
func (s *Store) ListVectorPushRecords(ctx context.Context, limit int) ([]VectorPushRecord, error) {
if limit <= 0 {
limit = 20
}
rows, err := s.db.QueryContext(ctx, `
SELECT
p.id,
p.target_system_id,
t.code,
p.vector_asset_id,
v.code,
p.status,
p.request_json,
p.response_status,
p.response_body,
p.error_message,
p.pushed_at,
p.finished_at
FROM vector_push_records p
JOIN target_systems t ON t.id = p.target_system_id
JOIN vector_assets v ON v.id = p.vector_asset_id
ORDER BY p.pushed_at DESC
LIMIT ?
`, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var items []VectorPushRecord
for rows.Next() {
item, err := scanVectorPushRecord(rows)
if err != nil {
return nil, err
}
items = append(items, item)
}
return items, rows.Err()
}
func (s *Store) PushVectorAsset(ctx context.Context, targetCode, vectorCode string) (VectorPushRecord, error) {
target, err := s.GetTargetSystemByCode(ctx, targetCode)
if err != nil {
return VectorPushRecord{}, err
}
if !target.Enabled {
return VectorPushRecord{}, fmt.Errorf("target system %q is disabled", target.Code)
}
vectorCode = normalizeBasemapCode(vectorCode)
asset, err := s.GetVectorAssetByCode(ctx, vectorCode)
if err != nil {
return VectorPushRecord{}, err
}
payload := map[string]any{
"event": "vector.published",
"service": strings.TrimSpace(s.cfg.APIBaseURL),
"vector": asset,
"pushed_at": toRFC3339(nowUTC()),
"targetCode": target.Code,
}
body, err := json.Marshal(payload)
if err != nil {
return VectorPushRecord{}, err
}
request, err := http.NewRequestWithContext(ctx, target.CallbackMethod, target.CallbackURL, bytes.NewReader(body))
if err != nil {
return VectorPushRecord{}, err
}
request.Header.Set("Content-Type", "application/json")
for key, value := range target.CallbackHeaders {
request.Header.Set(key, value)
}
response, err := s.httpClient.Do(request)
record := VectorPushRecord{
ID: uid.New(),
TargetSystemID: target.ID,
TargetSystemCode: target.Code,
VectorAssetID: asset.ID,
VectorCode: asset.Code,
Request: payload,
PushedAt: nowUTC(),
}
if err != nil {
record.Status = "failed"
record.ErrorMessage = err.Error()
} else {
defer response.Body.Close()
record.ResponseBody = limitedReadAll(response.Body, 256*1024)
status := response.StatusCode
record.ResponseStatus = &status
if status >= 200 && status < 300 {
record.Status = "succeeded"
} else {
record.Status = "failed"
record.ErrorMessage = fmt.Sprintf("target returned status %d", status)
}
}
finishedAt := nowUTC()
record.FinishedAt = &finishedAt
_, insertErr := s.db.ExecContext(ctx, `
INSERT INTO vector_push_records (
id, target_system_id, vector_asset_id, status, request_json, response_status, response_body, error_message, pushed_at, finished_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`, record.ID, record.TargetSystemID, record.VectorAssetID, record.Status, writeJSON(record.Request), record.ResponseStatus, record.ResponseBody, record.ErrorMessage, toRFC3339(record.PushedAt), nullableTime(record.FinishedAt))
if insertErr != nil {
return VectorPushRecord{}, insertErr
}
if err != nil {
return record, err
}
if record.Status != "succeeded" {
return record, errors.New(record.ErrorMessage)
}
return record, nil
}
services/api-go/internal/basemap/scan.go
+578
View File
@@ -0,0 +1,578 @@
package basemap
import (
"context"
"database/sql"
"errors"
"fmt"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"time"
"map-asset-gateway/api-go/internal/uid"
)
type scannedVersion struct {
BasemapCode string
Name string
Type string
Status string
Description string
Version string
Manifest string
TileRoot string
TileFormat string
TileScheme string
MinZoom int
MaxZoom int
BBox []float64
Attribution string
Metadata map[string]any
IsDefault bool
}
func (s *Store) RunScan(ctx context.Context, sourceCode string) (ScanRun, error) {
source, err := s.GetScanSourceByCode(ctx, sourceCode)
if err != nil {
return ScanRun{}, err
}
if !source.Enabled {
return ScanRun{}, fmt.Errorf("scan source %q is disabled", sourceCode)
}
startedAt := nowUTC()
runID := uid.New()
if _, err := s.db.ExecContext(ctx, `
INSERT INTO scan_runs (
id, scan_source_id, status, scanned_count, added_count, updated_count, removed_count, summary_json, started_at, finished_at
) VALUES (?, ?, 'running', 0, 0, 0, 0, '', ?, ?)
`, runID, source.ID, toRFC3339(startedAt), toRFC3339(startedAt)); err != nil {
return ScanRun{}, fmt.Errorf("create scan run: %w", err)
}
run, finalErr := s.runScan(ctx, runID, source, startedAt)
return run, finalErr
}
func (s *Store) runScan(ctx context.Context, runID string, source ScanSource, startedAt time.Time) (ScanRun, error) {
rootInfo, err := os.Stat(source.RootPath)
if err != nil {
return s.finishScanRun(ctx, runID, source, startedAt, "failed", nil, 0, 0, 0, 0, fmt.Errorf("stat scan root: %w", err))
}
if !rootInfo.IsDir() {
return s.finishScanRun(ctx, runID, source, startedAt, "failed", nil, 0, 0, 0, 0, fmt.Errorf("scan root %s is not a directory", source.RootPath))
}
manifestItems, coveredRoots, err := s.findManifestVersions(source)
if err != nil {
return s.finishScanRun(ctx, runID, source, startedAt, "failed", nil, 0, 0, 0, 0, err)
}
fallbackItems, err := s.findDirectTileRoots(source, coveredRoots)
if err != nil {
return s.finishScanRun(ctx, runID, source, startedAt, "failed", nil, 0, 0, 0, 0, err)
}
allItems := append(manifestItems, fallbackItems...)
sort.Slice(allItems, func(i, j int) bool {
if allItems[i].BasemapCode == allItems[j].BasemapCode {
return allItems[i].Version < allItems[j].Version
}
return allItems[i].BasemapCode < allItems[j].BasemapCode
})
addedCount, updatedCount, removedCount, err := s.persistScanResult(ctx, source, allItems)
if err != nil {
return s.finishScanRun(ctx, runID, source, startedAt, "failed", allItems, len(allItems), addedCount, updatedCount, removedCount, err)
}
return s.finishScanRun(ctx, runID, source, startedAt, "succeeded", allItems, len(allItems), addedCount, updatedCount, removedCount, nil)
}
func (s *Store) finishScanRun(
ctx context.Context,
runID string,
source ScanSource,
startedAt time.Time,
status string,
items []scannedVersion,
scannedCount int,
addedCount int,
updatedCount int,
removedCount int,
runErr error,
) (ScanRun, error) {
finishedAt := nowUTC()
summary := map[string]any{
"source_code": source.Code,
"scan_root": source.RootPath,
"scanned_count": scannedCount,
"added_count": addedCount,
"updated_count": updatedCount,
"removed_count": removedCount,
}
if runErr != nil {
summary["error"] = runErr.Error()
}
if len(items) > 0 {
codes := map[string]struct{}{}
for _, item := range items {
codes[item.BasemapCode] = struct{}{}
}
summary["basemap_codes"] = sortedKeys(codes)
}
_, updateErr := s.db.ExecContext(ctx, `
UPDATE scan_runs
SET status = ?, scanned_count = ?, added_count = ?, updated_count = ?, removed_count = ?, summary_json = ?, finished_at = ?
WHERE id = ?
`, status, scannedCount, addedCount, updatedCount, removedCount, writeJSON(summary), toRFC3339(finishedAt), runID)
if updateErr != nil {
if runErr != nil {
return ScanRun{}, fmt.Errorf("%v; update scan run: %w", runErr, updateErr)
}
return ScanRun{}, updateErr
}
run := ScanRun{
ID: runID,
ScanSourceID: source.ID,
SourceCode: source.Code,
Status: status,
ScannedCount: scannedCount,
AddedCount: addedCount,
UpdatedCount: updatedCount,
RemovedCount: removedCount,
Summary: summary,
StartedAt: startedAt,
FinishedAt: finishedAt,
}
if runErr != nil {
return run, runErr
}
return run, nil
}
func (s *Store) findManifestVersions(source ScanSource) ([]scannedVersion, map[string]struct{}, error) {
manifestName := strings.TrimSpace(source.ManifestName)
if manifestName == "" {
manifestName = defaultManifestName
}
var items []scannedVersion
coveredRoots := map[string]struct{}{}
err := filepath.WalkDir(source.RootPath, func(path string, entry os.DirEntry, walkErr error) error {
if walkErr != nil {
return walkErr
}
if entry.IsDir() {
return nil
}
if !strings.EqualFold(entry.Name(), manifestName) {
return nil
}
manifest, err := LoadManifest(path)
if err != nil {
return err
}
tileRoot, err := resolveTileRoot(filepath.Dir(path), manifest.RootPath)
if err != nil {
return err
}
format, minZoom, maxZoom, found, err := inspectTileRoot(tileRoot, manifest.TileFormat)
if err != nil {
return err
}
if found {
manifest.TileFormat = format
manifest.MinZoom = minZoom
manifest.MaxZoom = maxZoom
}
relPath, err := filepath.Rel(source.RootPath, tileRoot)
if err == nil {
first := strings.Split(filepath.ToSlash(relPath), "/")[0]
if first != "" && first != "." {
coveredRoots[first] = struct{}{}
}
}
items = append(items, scannedVersion{
BasemapCode: manifest.Code,
Name: defaultText(manifest.Name, manifest.Code),
Type: defaultText(manifest.Type, "xyz"),
Status: defaultText(manifest.Status, "ready"),
Description: manifest.Description,
Version: defaultText(manifest.Version, "current"),
Manifest: path,
TileRoot: tileRoot,
TileFormat: normalizeTileFormat(manifest.TileFormat),
TileScheme: defaultText(manifest.TileScheme, "xyz"),
MinZoom: manifest.MinZoom,
MaxZoom: manifest.MaxZoom,
BBox: manifest.BBox,
Attribution: manifest.Attribution,
Metadata: manifest.Metadata,
IsDefault: manifest.IsDefault,
})
return nil
})
if err != nil {
return nil, nil, fmt.Errorf("walk manifests: %w", err)
}
return items, coveredRoots, nil
}
func (s *Store) findDirectTileRoots(source ScanSource, skip map[string]struct{}) ([]scannedVersion, error) {
entries, err := os.ReadDir(source.RootPath)
if err != nil {
return nil, fmt.Errorf("read scan root: %w", err)
}
var items []scannedVersion
for _, entry := range entries {
if !entry.IsDir() {
continue
}
name := entry.Name()
if _, exists := skip[name]; exists {
continue
}
rootPath := filepath.Join(source.RootPath, name)
format, minZoom, maxZoom, found, err := inspectTileRoot(rootPath, "")
if err != nil {
return nil, err
}
if !found {
continue
}
code := normalizeBasemapCode(name)
if code == "" {
continue
}
items = append(items, scannedVersion{
BasemapCode: code,
Name: name,
Type: "xyz",
Status: "ready",
Version: "current",
TileRoot: rootPath,
TileFormat: format,
TileScheme: "xyz",
MinZoom: minZoom,
MaxZoom: maxZoom,
Metadata: map[string]any{
"discovered_from": "directory_scan",
},
IsDefault: true,
})
}
return items, nil
}
func inspectTileRoot(rootPath string, hintFormat string) (string, int, int, bool, error) {
entries, err := os.ReadDir(rootPath)
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return "", 0, 0, false, nil
}
return "", 0, 0, false, fmt.Errorf("read tile root %s: %w", rootPath, err)
}
var zooms []int
format := normalizeTileFormat(hintFormat)
for _, entry := range entries {
if !entry.IsDir() {
continue
}
zoom, err := strconv.Atoi(entry.Name())
if err != nil {
continue
}
zooms = append(zooms, zoom)
if format == "" || format == "png" {
detected, found, err := detectTileFormat(filepath.Join(rootPath, entry.Name()))
if err != nil {
return "", 0, 0, false, err
}
if found {
format = detected
}
}
}
if len(zooms) == 0 {
return "", 0, 0, false, nil
}
sort.Ints(zooms)
if format == "" {
format = "png"
}
return format, zooms[0], zooms[len(zooms)-1], true, nil
}
func detectTileFormat(zoomPath string) (string, bool, error) {
xEntries, err := os.ReadDir(zoomPath)
if err != nil {
return "", false, fmt.Errorf("read zoom path %s: %w", zoomPath, err)
}
for _, xEntry := range xEntries {
if !xEntry.IsDir() {
continue
}
tileEntries, err := os.ReadDir(filepath.Join(zoomPath, xEntry.Name()))
if err != nil {
return "", false, err
}
for _, tileEntry := range tileEntries {
if tileEntry.IsDir() {
continue
}
ext := strings.TrimPrefix(strings.ToLower(filepath.Ext(tileEntry.Name())), ".")
if ext == "" {
continue
}
return normalizeTileFormat(ext), true, nil
}
}
return "", false, nil
}
func resolveTileRoot(baseDir, relativePath string) (string, error) {
if strings.TrimSpace(relativePath) == "" {
relativePath = "tiles"
}
if filepath.IsAbs(relativePath) {
return filepath.Clean(relativePath), nil
}
resolved := filepath.Clean(filepath.Join(baseDir, relativePath))
return resolved, nil
}
func (s *Store) persistScanResult(ctx context.Context, source ScanSource, items []scannedVersion) (int, int, int, error) {
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return 0, 0, 0, err
}
defer tx.Rollback()
existing := map[string]struct{}{}
rows, err := tx.QueryContext(ctx, `
SELECT b.code, v.version
FROM basemap_versions v
JOIN basemaps b ON b.id = v.basemap_id
WHERE v.scan_source_id = ?
`, source.ID)
if err != nil {
return 0, 0, 0, err
}
for rows.Next() {
var code string
var version string
if err := rows.Scan(&code, &version); err != nil {
rows.Close()
return 0, 0, 0, err
}
existing[code+"::"+version] = struct{}{}
}
rows.Close()
if err := rows.Err(); err != nil {
return 0, 0, 0, err
}
now := nowUTC()
found := map[string]struct{}{}
touchedBasemaps := map[string]struct{}{}
addedCount := 0
updatedCount := 0
for _, item := range items {
key := item.BasemapCode + "::" + item.Version
found[key] = struct{}{}
if _, exists := existing[key]; exists {
updatedCount++
} else {
addedCount++
}
if err := s.upsertScannedVersionTx(ctx, tx, source, item, now); err != nil {
return 0, 0, 0, err
}
touchedBasemaps[item.BasemapCode] = struct{}{}
}
removedCount := 0
for key := range existing {
if _, ok := found[key]; ok {
continue
}
parts := strings.SplitN(key, "::", 2)
if len(parts) != 2 {
continue
}
if err := s.deleteVersionBySourceTx(ctx, tx, source.ID, parts[0], parts[1]); err != nil {
return 0, 0, 0, err
}
touchedBasemaps[parts[0]] = struct{}{}
removedCount++
}
for code := range touchedBasemaps {
if err := s.normalizeBasemapDefaultsTx(ctx, tx, code, now); err != nil {
return 0, 0, 0, err
}
}
if err := tx.Commit(); err != nil {
return 0, 0, 0, err
}
return addedCount, updatedCount, removedCount, nil
}
func (s *Store) upsertScannedVersionTx(ctx context.Context, tx *sql.Tx, source ScanSource, item scannedVersion, now time.Time) error {
basemapID := uid.Deterministic("basemap", item.BasemapCode)
versionID := uid.Deterministic("basemap-version", item.BasemapCode, item.Version)
_, err := tx.ExecContext(ctx, `
INSERT INTO basemaps (id, code, name, type, status, description, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(code) DO UPDATE SET
name = excluded.name,
type = excluded.type,
status = excluded.status,
description = excluded.description,
updated_at = excluded.updated_at
`, basemapID, item.BasemapCode, item.Name, item.Type, item.Status, item.Description, toRFC3339(now), toRFC3339(now))
if err != nil {
return err
}
isDefault := 0
if item.IsDefault {
isDefault = 1
}
_, err = tx.ExecContext(ctx, `
INSERT INTO basemap_versions (
id, basemap_id, scan_source_id, version, status, is_default, manifest_path, tile_root_path, url_template,
tile_format, tile_scheme, min_zoom, max_zoom, bbox_json, attribution, metadata_json, created_at, updated_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(basemap_id, version) DO UPDATE SET
scan_source_id = excluded.scan_source_id,
status = excluded.status,
is_default = CASE WHEN excluded.is_default = 1 THEN 1 ELSE basemap_versions.is_default END,
manifest_path = excluded.manifest_path,
tile_root_path = excluded.tile_root_path,
url_template = excluded.url_template,
tile_format = excluded.tile_format,
tile_scheme = excluded.tile_scheme,
min_zoom = excluded.min_zoom,
max_zoom = excluded.max_zoom,
bbox_json = excluded.bbox_json,
attribution = excluded.attribution,
metadata_json = excluded.metadata_json,
updated_at = excluded.updated_at
`, versionID, basemapID, source.ID, item.Version, item.Status, isDefault, item.Manifest, item.TileRoot, s.buildURLTemplate(item.BasemapCode, item.Version, item.TileFormat), item.TileFormat, item.TileScheme, item.MinZoom, item.MaxZoom, writeJSON(item.BBox), item.Attribution, writeJSON(item.Metadata), toRFC3339(now), toRFC3339(now))
if err != nil {
return err
}
if item.IsDefault {
_, err = tx.ExecContext(ctx, `
UPDATE basemap_versions
SET is_default = CASE WHEN version = ? THEN 1 ELSE 0 END, updated_at = ?
WHERE basemap_id = ?
`, item.Version, toRFC3339(now), basemapID)
if err != nil {
return err
}
}
return nil
}
func (s *Store) deleteVersionBySourceTx(ctx context.Context, tx *sql.Tx, sourceID, basemapCode, version string) error {
result, err := tx.ExecContext(ctx, `
DELETE FROM basemap_versions
WHERE scan_source_id = ? AND version = ? AND basemap_id = (SELECT id FROM basemaps WHERE code = ?)
`, sourceID, version, basemapCode)
if err != nil {
return err
}
if _, err := result.RowsAffected(); err != nil {
return err
}
_, err = tx.ExecContext(ctx, `
DELETE FROM basemaps
WHERE code = ? AND NOT EXISTS (
SELECT 1 FROM basemap_versions WHERE basemap_id = basemaps.id
)
`, basemapCode)
return err
}
func (s *Store) normalizeBasemapDefaultsTx(ctx context.Context, tx *sql.Tx, basemapCode string, now time.Time) error {
var basemapID string
err := tx.QueryRowContext(ctx, `SELECT id FROM basemaps WHERE code = ?`, basemapCode).Scan(&basemapID)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return nil
}
return err
}
rows, err := tx.QueryContext(ctx, `
SELECT version, is_default
FROM basemap_versions
WHERE basemap_id = ?
ORDER BY is_default DESC, version ASC
`, basemapID)
if err != nil {
return err
}
defer rows.Close()
type versionFlag struct {
Version string
IsDefault bool
}
var versions []versionFlag
for rows.Next() {
var item versionFlag
var isDefault int
if err := rows.Scan(&item.Version, &isDefault); err != nil {
return err
}
item.IsDefault = isDefault == 1
versions = append(versions, item)
}
if err := rows.Err(); err != nil {
return err
}
if len(versions) == 0 {
return nil
}
defaultVersion := versions[0].Version
for _, item := range versions {
if item.IsDefault {
defaultVersion = item.Version
break
}
}
if _, err := tx.ExecContext(ctx, `
UPDATE basemap_versions
SET is_default = CASE WHEN version = ? THEN 1 ELSE 0 END, updated_at = ?
WHERE basemap_id = ?
`, defaultVersion, toRFC3339(now), basemapID); err != nil {
return err
}
_, err = tx.ExecContext(ctx, `UPDATE basemaps SET updated_at = ? WHERE id = ?`, toRFC3339(now), basemapID)
return err
}
func (s *Store) buildURLTemplate(code, version, format string) string {
base := trimURL(s.cfg.TileBaseURL)
if base == "" {
base = trimURL(s.cfg.APIBaseURL)
}
format = normalizeTileFormat(format)
return fmt.Sprintf("%s/tiles/%s/%s/{z}/{x}/{y}.%s", base, code, version, format)
}
services/api-go/internal/basemap/store.go
+364
View File
@@ -0,0 +1,364 @@
package basemap
import (
"context"
"crypto/rand"
"crypto/sha256"
"database/sql"
"encoding/base32"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"sort"
"strings"
"time"
_ "modernc.org/sqlite"
)
const (
defaultScanSourceCode = "local-basemaps"
defaultManifestName = "manifest.json"
readPermission = "read"
defaultVectorSourceCode = "local-vectors"
)
type StoreConfig struct {
SQLitePath string
ScanRoot string
VectorScanRoot string
APIBaseURL string
TileBaseURL string
VectorBaseURL string
AutoScanOnStart bool
}
type Store struct {
db *sql.DB
cfg StoreConfig
httpClient *http.Client
}
type CreatedToken struct {
Meta ServiceToken `json:"meta"`
Token string `json:"token"`
}
type TileDescriptor struct {
FilePath string
ContentType string
Basemap Basemap
Version BasemapVersion
Token ServiceToken
RelativePath string
}
func OpenStore(cfg StoreConfig) (*Store, error) {
if strings.TrimSpace(cfg.SQLitePath) == "" {
return nil, errors.New("sqlite path is required")
}
if err := os.MkdirAll(filepath.Dir(cfg.SQLitePath), 0o755); err != nil {
return nil, fmt.Errorf("create sqlite dir: %w", err)
}
db, err := sql.Open("sqlite", cfg.SQLitePath)
if err != nil {
return nil, fmt.Errorf("open sqlite: %w", err)
}
db.SetConnMaxLifetime(10 * time.Minute)
db.SetMaxOpenConns(1)
db.SetMaxIdleConns(1)
store := &Store{
db: db,
cfg: cfg,
httpClient: &http.Client{
Timeout: 15 * time.Second,
},
}
if err := store.bootstrap(context.Background()); err != nil {
_ = db.Close()
return nil, err
}
if strings.TrimSpace(cfg.ScanRoot) != "" {
if _, err := store.EnsureScanSource(context.Background(), CreateScanSourceInput{
Code: defaultScanSourceCode,
Name: "Local Basemaps",
RootPath: cfg.ScanRoot,
ManifestName: defaultManifestName,
}); err != nil {
_ = db.Close()
return nil, err
}
}
if strings.TrimSpace(cfg.VectorScanRoot) != "" {
if err := os.MkdirAll(cfg.VectorScanRoot, 0o755); err != nil {
_ = db.Close()
return nil, fmt.Errorf("create vector scan root: %w", err)
}
}
return store, nil
}
func (s *Store) Close() error {
if s == nil || s.db == nil {
return nil
}
return s.db.Close()
}
func (s *Store) bootstrap(ctx context.Context) error {
statements := []string{
`PRAGMA foreign_keys = ON;`,
`CREATE TABLE IF NOT EXISTS basemaps (
id TEXT PRIMARY KEY,
code TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
type TEXT NOT NULL,
status TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);`,
`CREATE TABLE IF NOT EXISTS basemap_versions (
id TEXT PRIMARY KEY,
basemap_id TEXT NOT NULL,
scan_source_id TEXT,
version TEXT NOT NULL,
status TEXT NOT NULL,
is_default INTEGER NOT NULL DEFAULT 0,
manifest_path TEXT NOT NULL DEFAULT '',
tile_root_path TEXT NOT NULL,
url_template TEXT NOT NULL,
tile_format TEXT NOT NULL,
tile_scheme TEXT NOT NULL,
min_zoom INTEGER NOT NULL DEFAULT 0,
max_zoom INTEGER NOT NULL DEFAULT 0,
bbox_json TEXT NOT NULL DEFAULT '',
attribution TEXT NOT NULL DEFAULT '',
metadata_json TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
UNIQUE(basemap_id, version),
FOREIGN KEY (basemap_id) REFERENCES basemaps(id) ON DELETE CASCADE,
FOREIGN KEY (scan_source_id) REFERENCES scan_sources(id) ON DELETE SET NULL
);`,
`CREATE TABLE IF NOT EXISTS scan_sources (
id TEXT PRIMARY KEY,
code TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
root_path TEXT NOT NULL,
manifest_name TEXT NOT NULL,
enabled INTEGER NOT NULL DEFAULT 1,
metadata_json TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);`,
`CREATE TABLE IF NOT EXISTS scan_runs (
id TEXT PRIMARY KEY,
scan_source_id TEXT NOT NULL,
status TEXT NOT NULL,
scanned_count INTEGER NOT NULL DEFAULT 0,
added_count INTEGER NOT NULL DEFAULT 0,
updated_count INTEGER NOT NULL DEFAULT 0,
removed_count INTEGER NOT NULL DEFAULT 0,
summary_json TEXT NOT NULL DEFAULT '',
started_at TEXT NOT NULL,
finished_at TEXT NOT NULL,
FOREIGN KEY (scan_source_id) REFERENCES scan_sources(id) ON DELETE CASCADE
);`,
`CREATE TABLE IF NOT EXISTS target_systems (
id TEXT PRIMARY KEY,
code TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
callback_url TEXT NOT NULL,
callback_method TEXT NOT NULL,
callback_headers_json TEXT NOT NULL DEFAULT '',
enabled INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);`,
`CREATE TABLE IF NOT EXISTS service_tokens (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
token_hash TEXT NOT NULL UNIQUE,
token_prefix TEXT NOT NULL,
status TEXT NOT NULL,
expires_at TEXT,
last_used_at TEXT,
created_at TEXT NOT NULL
);`,
`CREATE TABLE IF NOT EXISTS token_grants (
id TEXT PRIMARY KEY,
token_id TEXT NOT NULL,
basemap_id TEXT NOT NULL,
basemap_version_id TEXT,
permission TEXT NOT NULL,
created_at TEXT NOT NULL,
FOREIGN KEY (token_id) REFERENCES service_tokens(id) ON DELETE CASCADE,
FOREIGN KEY (basemap_id) REFERENCES basemaps(id) ON DELETE CASCADE,
FOREIGN KEY (basemap_version_id) REFERENCES basemap_versions(id) ON DELETE CASCADE
);`,
`CREATE TABLE IF NOT EXISTS push_records (
id TEXT PRIMARY KEY,
target_system_id TEXT NOT NULL,
basemap_version_id TEXT NOT NULL,
status TEXT NOT NULL,
request_json TEXT NOT NULL DEFAULT '',
response_status INTEGER,
response_body TEXT NOT NULL DEFAULT '',
error_message TEXT NOT NULL DEFAULT '',
pushed_at TEXT NOT NULL,
finished_at TEXT,
FOREIGN KEY (target_system_id) REFERENCES target_systems(id) ON DELETE CASCADE,
FOREIGN KEY (basemap_version_id) REFERENCES basemap_versions(id) ON DELETE CASCADE
);`,
`CREATE TABLE IF NOT EXISTS vector_assets (
id TEXT PRIMARY KEY,
code TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
format TEXT NOT NULL,
status TEXT NOT NULL,
description TEXT NOT NULL DEFAULT '',
file_path TEXT NOT NULL,
file_name TEXT NOT NULL,
file_size INTEGER NOT NULL DEFAULT 0,
checksum TEXT NOT NULL DEFAULT '',
download_url TEXT NOT NULL,
metadata_json TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);`,
`CREATE TABLE IF NOT EXISTS vector_token_grants (
id TEXT PRIMARY KEY,
token_id TEXT NOT NULL,
vector_asset_id TEXT NOT NULL,
permission TEXT NOT NULL,
created_at TEXT NOT NULL,
FOREIGN KEY (token_id) REFERENCES service_tokens(id) ON DELETE CASCADE,
FOREIGN KEY (vector_asset_id) REFERENCES vector_assets(id) ON DELETE CASCADE
);`,
`CREATE TABLE IF NOT EXISTS vector_push_records (
id TEXT PRIMARY KEY,
target_system_id TEXT NOT NULL,
vector_asset_id TEXT NOT NULL,
status TEXT NOT NULL,
request_json TEXT NOT NULL DEFAULT '',
response_status INTEGER,
response_body TEXT NOT NULL DEFAULT '',
error_message TEXT NOT NULL DEFAULT '',
pushed_at TEXT NOT NULL,
finished_at TEXT,
FOREIGN KEY (target_system_id) REFERENCES target_systems(id) ON DELETE CASCADE,
FOREIGN KEY (vector_asset_id) REFERENCES vector_assets(id) ON DELETE CASCADE
);`,
`CREATE INDEX IF NOT EXISTS idx_versions_basemap ON basemap_versions(basemap_id);`,
`CREATE INDEX IF NOT EXISTS idx_versions_source ON basemap_versions(scan_source_id);`,
`CREATE INDEX IF NOT EXISTS idx_grants_token ON token_grants(token_id);`,
`CREATE INDEX IF NOT EXISTS idx_push_target ON push_records(target_system_id);`,
`CREATE INDEX IF NOT EXISTS idx_vector_grants_token ON vector_token_grants(token_id);`,
`CREATE INDEX IF NOT EXISTS idx_vector_push_target ON vector_push_records(target_system_id);`,
}
for _, statement := range statements {
if _, err := s.db.ExecContext(ctx, statement); err != nil {
return fmt.Errorf("bootstrap sqlite: %w", err)
}
}
return nil
}
func (s *Store) AutoScanEnabled() bool {
return s.cfg.AutoScanOnStart
}
func (s *Store) ScanRoot() string {
return s.cfg.ScanRoot
}
func (s *Store) VectorScanRoot() string {
return s.cfg.VectorScanRoot
}
func (s *Store) generateTokenSecret() (string, string, error) {
var raw [32]byte
if _, err := rand.Read(raw[:]); err != nil {
return "", "", err
}
encoder := base32.StdEncoding.WithPadding(base32.NoPadding)
secret := strings.ToLower(encoder.EncodeToString(raw[:]))
token := "bmp_" + secret
sum := sha256.Sum256([]byte(token))
return token, hex.EncodeToString(sum[:]), nil
}
func normalizeBasemapCode(value string) string {
return normalizeCode(value)
}
func nowUTC() time.Time {
return time.Now().UTC().Round(time.Second)
}
func toRFC3339(t time.Time) string {
return t.UTC().Format(time.RFC3339)
}
func nullableTime(t *time.Time) any {
if t == nil || t.IsZero() {
return nil
}
return toRFC3339(*t)
}
func scanNullString(value string) sql.NullString {
if strings.TrimSpace(value) == "" {
return sql.NullString{}
}
return sql.NullString{String: value, Valid: true}
}
func readMaybeJSON[T any](value string, target *T) error {
if strings.TrimSpace(value) == "" {
return nil
}
return json.Unmarshal([]byte(value), target)
}
func writeJSON(value any) string {
if value == nil {
return ""
}
encoded, err := json.Marshal(value)
if err != nil {
return ""
}
return string(encoded)
}
func sortedKeys(values map[string]struct{}) []string {
keys := make([]string, 0, len(values))
for key := range values {
keys = append(keys, key)
}
sort.Strings(keys)
return keys
}
func trimURL(value string) string {
return strings.TrimRight(strings.TrimSpace(value), "/")
}
func limitedReadAll(body io.Reader, limit int64) string {
content, err := io.ReadAll(io.LimitReader(body, limit))
if err != nil {
return ""
}
return string(content)
}
services/api-go/internal/basemap/store_rows.go
+344
View File
@@ -0,0 +1,344 @@
package basemap
import (
"database/sql"
"fmt"
"strings"
"time"
)
type basemapVersionRow struct {
ID string
BasemapID string
BasemapCode string
Version string
Status string
IsDefault bool
ManifestPath string
TileRootPath string
URLTemplate string
TileFormat string
TileScheme string
MinZoom int
MaxZoom int
BBoxJSON string
Attribution string
MetadataJSON string
CreatedAt string
UpdatedAt string
}
func parseTime(value string) time.Time {
if strings.TrimSpace(value) == "" {
return time.Time{}
}
parsed, err := time.Parse(time.RFC3339, value)
if err != nil {
return time.Time{}
}
return parsed
}
func scanBasemapVersionRow(scanner interface {
Scan(dest ...any) error
}) (basemapVersionRow, error) {
var row basemapVersionRow
var isDefault int
if err := scanner.Scan(
&row.ID,
&row.BasemapID,
&row.BasemapCode,
&row.Version,
&row.Status,
&isDefault,
&row.ManifestPath,
&row.TileRootPath,
&row.URLTemplate,
&row.TileFormat,
&row.TileScheme,
&row.MinZoom,
&row.MaxZoom,
&row.BBoxJSON,
&row.Attribution,
&row.MetadataJSON,
&row.CreatedAt,
&row.UpdatedAt,
); err != nil {
return basemapVersionRow{}, err
}
row.IsDefault = isDefault == 1
return row, nil
}
func decodeBasemapVersion(row basemapVersionRow) BasemapVersion {
item := BasemapVersion{
ID: row.ID,
BasemapID: row.BasemapID,
BasemapCode: row.BasemapCode,
Version: row.Version,
Status: row.Status,
IsDefault: row.IsDefault,
ManifestPath: row.ManifestPath,
TileRootPath: row.TileRootPath,
URLTemplate: row.URLTemplate,
TileFormat: row.TileFormat,
TileScheme: row.TileScheme,
MinZoom: row.MinZoom,
MaxZoom: row.MaxZoom,
Attribution: row.Attribution,
CreatedAt: parseTime(row.CreatedAt),
UpdatedAt: parseTime(row.UpdatedAt),
}
if strings.TrimSpace(row.BBoxJSON) != "" {
_ = readMaybeJSON(row.BBoxJSON, &item.BBox)
}
if strings.TrimSpace(row.MetadataJSON) != "" {
_ = readMaybeJSON(row.MetadataJSON, &item.Metadata)
}
if item.Metadata == nil {
item.Metadata = map[string]any{}
}
return item
}
func scanBasemap(scanner interface {
Scan(dest ...any) error
}) (Basemap, error) {
var item Basemap
var createdAt string
var updatedAt string
if err := scanner.Scan(
&item.ID,
&item.Code,
&item.Name,
&item.Type,
&item.Status,
&item.Description,
&createdAt,
&updatedAt,
); err != nil {
return Basemap{}, err
}
item.CreatedAt = parseTime(createdAt)
item.UpdatedAt = parseTime(updatedAt)
return item, nil
}
func scanScanSource(scanner interface {
Scan(dest ...any) error
}) (ScanSource, error) {
var item ScanSource
var enabled int
var metadataJSON string
var createdAt string
var updatedAt string
if err := scanner.Scan(
&item.ID,
&item.Code,
&item.Name,
&item.RootPath,
&item.ManifestName,
&enabled,
&metadataJSON,
&createdAt,
&updatedAt,
); err != nil {
return ScanSource{}, err
}
item.Enabled = enabled == 1
item.CreatedAt = parseTime(createdAt)
item.UpdatedAt = parseTime(updatedAt)
if metadataJSON != "" {
_ = readMaybeJSON(metadataJSON, &item.Metadata)
}
if item.Metadata == nil {
item.Metadata = map[string]any{}
}
return item, nil
}
func scanScanRun(scanner interface {
Scan(dest ...any) error
}) (ScanRun, error) {
var item ScanRun
var summaryJSON string
var startedAt string
var finishedAt string
if err := scanner.Scan(
&item.ID,
&item.ScanSourceID,
&item.SourceCode,
&item.Status,
&item.ScannedCount,
&item.AddedCount,
&item.UpdatedCount,
&item.RemovedCount,
&summaryJSON,
&startedAt,
&finishedAt,
); err != nil {
return ScanRun{}, err
}
item.StartedAt = parseTime(startedAt)
item.FinishedAt = parseTime(finishedAt)
if summaryJSON != "" {
_ = readMaybeJSON(summaryJSON, &item.Summary)
}
if item.Summary == nil {
item.Summary = map[string]any{}
}
return item, nil
}
func scanTargetSystem(scanner interface {
Scan(dest ...any) error
}) (TargetSystem, error) {
var item TargetSystem
var enabled int
var headersJSON string
var createdAt string
var updatedAt string
if err := scanner.Scan(
&item.ID,
&item.Code,
&item.Name,
&item.CallbackURL,
&item.CallbackMethod,
&headersJSON,
&enabled,
&createdAt,
&updatedAt,
); err != nil {
return TargetSystem{}, err
}
item.Enabled = enabled == 1
item.CreatedAt = parseTime(createdAt)
item.UpdatedAt = parseTime(updatedAt)
if headersJSON != "" {
_ = readMaybeJSON(headersJSON, &item.CallbackHeaders)
}
if item.CallbackHeaders == nil {
item.CallbackHeaders = map[string]string{}
}
return item, nil
}
func scanServiceToken(scanner interface {
Scan(dest ...any) error
}) (ServiceToken, string, error) {
var item ServiceToken
var tokenHash string
var expiresAt sql.NullString
var lastUsedAt sql.NullString
var createdAt string
if err := scanner.Scan(
&item.ID,
&item.Name,
&tokenHash,
&item.TokenPrefix,
&item.Status,
&expiresAt,
&lastUsedAt,
&createdAt,
); err != nil {
return ServiceToken{}, "", err
}
item.CreatedAt = parseTime(createdAt)
if expiresAt.Valid {
value := parseTime(expiresAt.String)
item.ExpiresAt = &value
}
if lastUsedAt.Valid {
value := parseTime(lastUsedAt.String)
item.LastUsedAt = &value
}
return item, tokenHash, nil
}
func scanTokenGrant(scanner interface {
Scan(dest ...any) error
}) (TokenGrant, error) {
var item TokenGrant
var versionID sql.NullString
var version sql.NullString
var createdAt string
if err := scanner.Scan(
&item.ID,
&item.TokenID,
&item.BasemapID,
&item.BasemapCode,
&versionID,
&version,
&item.Permission,
&createdAt,
); err != nil {
return TokenGrant{}, err
}
if versionID.Valid {
item.BasemapVersionID = &versionID.String
}
if version.Valid {
item.BasemapVersion = &version.String
}
item.CreatedAt = parseTime(createdAt)
return item, nil
}
func scanPushRecord(scanner interface {
Scan(dest ...any) error
}) (PushRecord, error) {
var item PushRecord
var requestJSON string
var responseStatus sql.NullInt64
var finishedAt sql.NullString
var pushedAt string
if err := scanner.Scan(
&item.ID,
&item.TargetSystemID,
&item.TargetSystemCode,
&item.BasemapVersionID,
&item.BasemapCode,
&item.BasemapVersion,
&item.Status,
&requestJSON,
&responseStatus,
&item.ResponseBody,
&item.ErrorMessage,
&pushedAt,
&finishedAt,
); err != nil {
return PushRecord{}, err
}
if requestJSON != "" {
_ = readMaybeJSON(requestJSON, &item.Request)
}
if item.Request == nil {
item.Request = map[string]any{}
}
if responseStatus.Valid {
value := int(responseStatus.Int64)
item.ResponseStatus = &value
}
item.PushedAt = parseTime(pushedAt)
if finishedAt.Valid {
value := parseTime(finishedAt.String)
item.FinishedAt = &value
}
return item, nil
}
func mustOneRow[T any](item T, err error) (T, error) {
if err != nil {
return item, err
}
return item, nil
}
func unexpectedRowsAffected(count int64, expected ...int64) error {
for _, value := range expected {
if count == value {
return nil
}
}
return fmt.Errorf("unexpected rows affected: %d", count)
}
services/api-go/internal/basemap/vector_rows.go
+102
View File
@@ -0,0 +1,102 @@
package basemap
import (
"database/sql"
)
func scanVectorAsset(scanner interface {
Scan(dest ...any) error
}) (VectorAsset, error) {
var item VectorAsset
var metadataJSON string
var createdAt string
var updatedAt string
if err := scanner.Scan(
&item.ID,
&item.Code,
&item.Name,
&item.Format,
&item.Status,
&item.Description,
&item.FilePath,
&item.FileName,
&item.FileSize,
&item.Checksum,
&item.DownloadURL,
&metadataJSON,
&createdAt,
&updatedAt,
); err != nil {
return VectorAsset{}, err
}
item.CreatedAt = parseTime(createdAt)
item.UpdatedAt = parseTime(updatedAt)
if metadataJSON != "" {
_ = readMaybeJSON(metadataJSON, &item.Metadata)
}
if item.Metadata == nil {
item.Metadata = map[string]any{}
}
return item, nil
}
func scanVectorTokenGrant(scanner interface {
Scan(dest ...any) error
}) (VectorTokenGrant, error) {
var item VectorTokenGrant
var createdAt string
if err := scanner.Scan(
&item.ID,
&item.TokenID,
&item.VectorID,
&item.VectorCode,
&item.Permission,
&createdAt,
); err != nil {
return VectorTokenGrant{}, err
}
item.CreatedAt = parseTime(createdAt)
return item, nil
}
func scanVectorPushRecord(scanner interface {
Scan(dest ...any) error
}) (VectorPushRecord, error) {
var item VectorPushRecord
var requestJSON string
var responseStatus sql.NullInt64
var finishedAt sql.NullString
var pushedAt string
if err := scanner.Scan(
&item.ID,
&item.TargetSystemID,
&item.TargetSystemCode,
&item.VectorAssetID,
&item.VectorCode,
&item.Status,
&requestJSON,
&responseStatus,
&item.ResponseBody,
&item.ErrorMessage,
&pushedAt,
&finishedAt,
); err != nil {
return VectorPushRecord{}, err
}
if requestJSON != "" {
_ = readMaybeJSON(requestJSON, &item.Request)
}
if item.Request == nil {
item.Request = map[string]any{}
}
if responseStatus.Valid {
value := int(responseStatus.Int64)
item.ResponseStatus = &value
}
item.PushedAt = parseTime(pushedAt)
if finishedAt.Valid {
value := parseTime(finishedAt.String)
item.FinishedAt = &value
}
return item, nil
}
services/api-go/internal/basemap/vectors.go
+555
View File
@@ -0,0 +1,555 @@
package basemap
import (
"archive/zip"
"context"
"crypto/sha256"
"database/sql"
"encoding/hex"
"errors"
"fmt"
"io"
"mime"
"net/http"
"os"
"path"
"path/filepath"
"sort"
"strings"
"time"
"map-asset-gateway/api-go/internal/uid"
)
type VectorDownloadDescriptor struct {
Asset VectorAsset
FilePath string
ContentType string
FileName string
Zipped bool
PackageFiles []string
}
func (s *Store) ListVectorAssets(ctx context.Context) ([]VectorAsset, error) {
rows, err := s.db.QueryContext(ctx, `
SELECT id, code, name, format, status, description, file_path, file_name, file_size, checksum, download_url, metadata_json, created_at, updated_at
FROM vector_assets
ORDER BY code
`)
if err != nil {
return nil, err
}
defer rows.Close()
var items []VectorAsset
for rows.Next() {
item, err := scanVectorAsset(rows)
if err != nil {
return nil, err
}
items = append(items, item)
}
return items, rows.Err()
}
func (s *Store) GetVectorAssetByCode(ctx context.Context, code string) (VectorAsset, error) {
row := s.db.QueryRowContext(ctx, `
SELECT id, code, name, format, status, description, file_path, file_name, file_size, checksum, download_url, metadata_json, created_at, updated_at
FROM vector_assets
WHERE code = ?
`, normalizeBasemapCode(code))
item, err := scanVectorAsset(row)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return VectorAsset{}, fmt.Errorf("vector asset %q not found", code)
}
return VectorAsset{}, err
}
return item, nil
}
func (s *Store) GetVectorAssetByRef(ctx context.Context, ref string) (VectorAsset, error) {
cleaned, err := cleanVectorAssetRef(ref)
if err != nil {
return VectorAsset{}, err
}
items, err := s.ListVectorAssets(ctx)
if err != nil {
return VectorAsset{}, err
}
for _, item := range items {
if vectorAssetMatchesRef(item, cleaned) {
return item, nil
}
}
return VectorAsset{}, fmt.Errorf("vector asset %q not found", ref)
}
func (s *Store) RunVectorScan(ctx context.Context) (map[string]any, error) {
root := strings.TrimSpace(s.cfg.VectorScanRoot)
if root == "" {
return nil, nil
}
if err := os.MkdirAll(root, 0o755); err != nil {
return nil, err
}
items, err := scanVectorRoot(root)
if err != nil {
return nil, err
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return nil, err
}
defer tx.Rollback()
rows, err := tx.QueryContext(ctx, `SELECT code FROM vector_assets`)
if err != nil {
return nil, err
}
existing := map[string]struct{}{}
for rows.Next() {
var code string
if err := rows.Scan(&code); err != nil {
rows.Close()
return nil, err
}
existing[code] = struct{}{}
}
rows.Close()
if err := rows.Err(); err != nil {
return nil, err
}
found := map[string]struct{}{}
added := 0
updated := 0
now := nowUTC()
for _, item := range items {
found[item.Code] = struct{}{}
if _, ok := existing[item.Code]; ok {
updated++
} else {
added++
}
if err := s.upsertVectorAssetTx(ctx, tx, item, now); err != nil {
return nil, err
}
}
removed := 0
for code := range existing {
if _, ok := found[code]; ok {
continue
}
if _, err := tx.ExecContext(ctx, `DELETE FROM vector_assets WHERE code = ?`, code); err != nil {
return nil, err
}
removed++
}
if err := tx.Commit(); err != nil {
return nil, err
}
return map[string]any{
"scan_root": root,
"scanned_count": len(items),
"added_count": added,
"updated_count": updated,
"removed_count": removed,
}, nil
}
type scannedVector struct {
Code string
Name string
Format string
Status string
Description string
FilePath string
FileName string
FileSize int64
Checksum string
Metadata map[string]any
}
func scanVectorRoot(root string) ([]scannedVector, error) {
shapefiles := map[string]string{}
var items []scannedVector
err := filepath.WalkDir(root, func(path string, entry os.DirEntry, walkErr error) error {
if walkErr != nil {
return walkErr
}
if entry.IsDir() {
return nil
}
ext := strings.ToLower(filepath.Ext(entry.Name()))
switch ext {
case ".geojson", ".json", ".zip":
item, err := buildScannedVector(root, path, ext)
if err != nil {
return err
}
items = append(items, item)
case ".shp":
base := strings.TrimSuffix(path, filepath.Ext(path))
shapefiles[base] = path
}
return nil
})
if err != nil {
return nil, err
}
for _, base := range sortedKeysStringMap(shapefiles) {
path := shapefiles[base]
item, err := buildScannedVector(root, path, ".shp")
if err != nil {
return nil, err
}
items = append(items, item)
}
sort.Slice(items, func(i, j int) bool {
return items[i].Code < items[j].Code
})
return items, nil
}
func sortedKeysStringMap(values map[string]string) []string {
keys := make([]string, 0, len(values))
for key := range values {
keys = append(keys, key)
}
sort.Strings(keys)
return keys
}
func buildScannedVector(root, path, ext string) (scannedVector, error) {
absPath, err := filepath.Abs(path)
if err != nil {
return scannedVector{}, err
}
info, err := os.Stat(absPath)
if err != nil {
return scannedVector{}, err
}
rel, err := filepath.Rel(root, absPath)
if err != nil {
return scannedVector{}, err
}
code := vectorCodeFromRel(rel, ext)
name := strings.TrimSuffix(filepath.Base(absPath), filepath.Ext(absPath))
format := vectorFormatFromExt(ext)
fileName := filepath.Base(absPath)
fileSize := info.Size()
checksum, err := hashVector(absPath, ext)
if err != nil {
return scannedVector{}, err
}
if format == "shapefile" {
fileName = name + ".zip"
}
return scannedVector{
Code: code,
Name: name,
Format: format,
Status: "ready",
FilePath: absPath,
FileName: fileName,
FileSize: fileSize,
Checksum: checksum,
Metadata: map[string]any{
"relative_path": filepath.ToSlash(rel),
},
}, nil
}
func vectorCodeFromRel(rel, ext string) string {
rel = filepath.ToSlash(rel)
rel = strings.TrimSuffix(rel, strings.ToLower(filepath.Ext(rel)))
rel = strings.TrimSuffix(rel, filepath.Ext(rel))
rel = strings.ReplaceAll(rel, "/", "-")
code := normalizeBasemapCode(rel)
if code != "" {
return code
}
sum := sha256.Sum256([]byte(rel))
return "vector-" + hex.EncodeToString(sum[:6])
}
func vectorFormatFromExt(ext string) string {
switch strings.ToLower(ext) {
case ".geojson", ".json":
return "geojson"
case ".shp":
return "shapefile"
case ".zip":
return "shpzip"
default:
return strings.TrimPrefix(strings.ToLower(ext), ".")
}
}
func hashVector(path, ext string) (string, error) {
sum := sha256.New()
if strings.EqualFold(ext, ".shp") {
for _, sibling := range shapefilePackageFiles(path) {
info, err := os.Stat(sibling)
if err != nil || info.IsDir() {
continue
}
if _, err := io.WriteString(sum, filepath.Base(sibling)); err != nil {
return "", err
}
if err := hashFileInto(sum, sibling); err != nil {
return "", err
}
}
return hex.EncodeToString(sum.Sum(nil)), nil
}
if err := hashFileInto(sum, path); err != nil {
return "", err
}
return hex.EncodeToString(sum.Sum(nil)), nil
}
func hashFileInto(writer io.Writer, path string) error {
file, err := os.Open(path)
if err != nil {
return err
}
defer file.Close()
_, err = io.Copy(writer, file)
return err
}
func shapefilePackageFiles(path string) []string {
base := strings.TrimSuffix(path, filepath.Ext(path))
candidates := []string{".shp", ".dbf", ".shx", ".prj", ".cpg", ".qix", ".sbn", ".sbx"}
files := make([]string, 0, len(candidates))
for _, ext := range candidates {
file := base + ext
if _, err := os.Stat(file); err == nil {
files = append(files, file)
}
}
sort.Strings(files)
return files
}
func (s *Store) upsertVectorAssetTx(ctx context.Context, tx *sql.Tx, item scannedVector, now time.Time) error {
id := uid.Deterministic("vector-asset", item.Code)
_, err := tx.ExecContext(ctx, `
INSERT INTO vector_assets (
id, code, name, format, status, description, file_path, file_name, file_size, checksum, download_url, metadata_json, created_at, updated_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(code) DO UPDATE SET
name = excluded.name,
format = excluded.format,
status = excluded.status,
description = excluded.description,
file_path = excluded.file_path,
file_name = excluded.file_name,
file_size = excluded.file_size,
checksum = excluded.checksum,
download_url = excluded.download_url,
metadata_json = excluded.metadata_json,
updated_at = excluded.updated_at
`, id, item.Code, item.Name, item.Format, item.Status, item.Description, item.FilePath, item.FileName, item.FileSize, item.Checksum, s.buildVectorDownloadURL(item.Code), writeJSON(item.Metadata), toRFC3339(now), toRFC3339(now))
return err
}
func (s *Store) buildVectorDownloadURL(code string) string {
base := trimURL(s.cfg.VectorBaseURL)
if base == "" {
base = trimURL(s.cfg.APIBaseURL)
}
return fmt.Sprintf("%s/vectors/%s/download", base, code)
}
func (auth TokenAuth) CanReadVector(code string) bool {
group, ok := auth.VectorPermissions[readPermission]
if !ok {
return false
}
_, ok = group[normalizeBasemapCode(code)]
return ok
}
func (s *Store) FilterVectors(auth TokenAuth, items []VectorAsset) []VectorAsset {
filtered := make([]VectorAsset, 0, len(items))
for _, item := range items {
if auth.CanReadVector(item.Code) {
filtered = append(filtered, item)
}
}
return filtered
}
func (s *Store) ResolveVectorDownload(ctx context.Context, auth TokenAuth, code string) (VectorDownloadDescriptor, error) {
code = normalizeBasemapCode(code)
if !auth.CanReadVector(code) {
return VectorDownloadDescriptor{}, errors.New("token has no access to this vector asset")
}
item, err := s.GetVectorAssetByCode(ctx, code)
if err != nil {
return VectorDownloadDescriptor{}, err
}
return buildVectorDownloadDescriptor(item)
}
func (s *Store) ResolveVectorDownloadByRef(ctx context.Context, auth TokenAuth, ref string, allowedFormats ...string) (VectorDownloadDescriptor, error) {
item, err := s.GetVectorAssetByRef(ctx, ref)
if err != nil {
return VectorDownloadDescriptor{}, err
}
if !auth.CanReadVector(item.Code) {
return VectorDownloadDescriptor{}, errors.New("token has no access to this vector asset")
}
if !vectorFormatAllowed(item.Format, allowedFormats) {
return VectorDownloadDescriptor{}, fmt.Errorf("vector asset %q is not available in requested format", ref)
}
return buildVectorDownloadDescriptor(item)
}
func buildVectorDownloadDescriptor(item VectorAsset) (VectorDownloadDescriptor, error) {
switch item.Format {
case "geojson":
contentType := mime.TypeByExtension(filepath.Ext(item.FilePath))
if contentType == "" {
contentType = "application/geo+json"
}
return VectorDownloadDescriptor{
Asset: item,
FilePath: item.FilePath,
FileName: item.FileName,
ContentType: contentType,
}, nil
case "shapefile":
files := shapefilePackageFiles(item.FilePath)
if len(files) == 0 {
return VectorDownloadDescriptor{}, errors.New("shapefile package is missing sidecar files")
}
return VectorDownloadDescriptor{
Asset: item,
FileName: item.FileName,
ContentType: "application/zip",
Zipped: true,
PackageFiles: files,
}, nil
case "shpzip":
return VectorDownloadDescriptor{
Asset: item,
FilePath: item.FilePath,
FileName: item.FileName,
ContentType: "application/zip",
}, nil
default:
contentType := mime.TypeByExtension(filepath.Ext(item.FilePath))
if contentType == "" {
contentType = "application/octet-stream"
}
return VectorDownloadDescriptor{
Asset: item,
FilePath: item.FilePath,
FileName: item.FileName,
ContentType: contentType,
}, nil
}
}
func cleanVectorAssetRef(value string) (string, error) {
value = strings.TrimSpace(strings.ReplaceAll(value, "\\", "/"))
value = strings.TrimPrefix(value, "/")
if value == "" {
return "", errors.New("vector asset path is required")
}
cleaned := path.Clean(value)
if cleaned == "." || cleaned == "/" || cleaned == ".." || strings.HasPrefix(cleaned, "../") {
return "", errors.New("invalid vector asset path")
}
return strings.TrimPrefix(cleaned, "./"), nil
}
func vectorAssetMatchesRef(item VectorAsset, ref string) bool {
for _, candidate := range vectorAssetRefs(item) {
if strings.EqualFold(candidate, ref) {
return true
}
}
return false
}
func vectorAssetRefs(item VectorAsset) []string {
values := []string{item.Code}
if relativePath := vectorRelativePath(item); relativePath != "" {
values = append(values, relativePath)
}
if item.FileName != "" {
values = append(values, strings.ReplaceAll(item.FileName, "\\", "/"))
}
return values
}
func vectorRelativePath(item VectorAsset) string {
if item.Metadata == nil {
return ""
}
value, ok := item.Metadata["relative_path"].(string)
if !ok {
return ""
}
cleaned, err := cleanVectorAssetRef(value)
if err != nil {
return ""
}
return cleaned
}
func vectorFormatAllowed(format string, allowed []string) bool {
if len(allowed) == 0 {
return true
}
for _, value := range allowed {
if strings.EqualFold(strings.TrimSpace(value), format) {
return true
}
}
return false
}
func WriteVectorZip(w http.ResponseWriter, descriptor VectorDownloadDescriptor) error {
archive := zip.NewWriter(w)
for _, path := range descriptor.PackageFiles {
file, err := os.Open(path)
if err != nil {
return err
}
info, err := file.Stat()
if err != nil {
file.Close()
return err
}
header, err := zip.FileInfoHeader(info)
if err != nil {
file.Close()
return err
}
header.Name = filepath.Base(path)
header.Method = zip.Deflate
writer, err := archive.CreateHeader(header)
if err != nil {
file.Close()
return err
}
if _, err := io.Copy(writer, file); err != nil {
file.Close()
return err
}
file.Close()
}
return archive.Close()
}
services/api-go/internal/basemapapi/admin_template.go
+1001
View File
File diff suppressed because it is too large Load Diff
services/api-go/internal/basemapapi/compat_test.go
+108
View File
@@ -0,0 +1,108 @@
package basemapapi
import (
"context"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"map-asset-gateway/api-go/internal/basemap"
)
func TestCompatibilityRoutes(t *testing.T) {
ctx := context.Background()
root := t.TempDir()
tileRoot := filepath.Join(root, "tiles")
vectorRoot := filepath.Join(root, "vectors")
sqlitePath := filepath.Join(root, "basemap.db")
if err := os.MkdirAll(filepath.Join(tileRoot, "google_image", "0", "0"), 0o755); err != nil {
t.Fatalf("mkdir tile root: %v", err)
}
if err := os.WriteFile(filepath.Join(tileRoot, "google_image", "0", "0", "0.webp"), []byte("tile"), 0o644); err != nil {
t.Fatalf("write tile: %v", err)
}
if err := os.MkdirAll(vectorRoot, 0o755); err != nil {
t.Fatalf("mkdir vector root: %v", err)
}
if err := os.WriteFile(filepath.Join(vectorRoot, "全国行政区.geojson"), []byte(`{"type":"FeatureCollection","features":[]}`), 0o644); err != nil {
t.Fatalf("write geojson: %v", err)
}
store, err := basemap.OpenStore(basemap.StoreConfig{
SQLitePath: sqlitePath,
ScanRoot: tileRoot,
VectorScanRoot: vectorRoot,
APIBaseURL: "http://127.0.0.1:8910",
TileBaseURL: "http://127.0.0.1:8910",
VectorBaseURL: "http://127.0.0.1:8910",
})
if err != nil {
t.Fatalf("open store: %v", err)
}
defer func() {
if err := store.Close(); err != nil {
t.Fatalf("close store: %v", err)
}
}()
if _, err := store.RunScan(ctx, "local-basemaps"); err != nil {
t.Fatalf("run scan: %v", err)
}
if _, err := store.RunVectorScan(ctx); err != nil {
t.Fatalf("run vector scan: %v", err)
}
token, err := store.CreateToken(ctx, basemap.CreateTokenInput{
Name: "compat",
BasemapCodes: []string{"google_image"},
VectorCodes: []string{"全国行政区.geojson"},
})
if err != nil {
t.Fatalf("create token: %v", err)
}
server, err := New(store, basemap.SystemInfo{
ServiceName: "map-asset-gateway",
ServiceVersion: "test",
APIBaseURL: "http://127.0.0.1:8910",
TileBaseURL: "http://127.0.0.1:8910",
VectorBaseURL: "http://127.0.0.1:8910",
SQLitePath: sqlitePath,
ScanRoot: tileRoot,
VectorScanRoot: vectorRoot,
})
if err != nil {
t.Fatalf("new server: %v", err)
}
httpServer := httptest.NewServer(server.Handler())
defer httpServer.Close()
response, err := http.Get(httpServer.URL + "/tiles/google_image/0/0/0.webp?token=" + token.Token)
if err != nil {
t.Fatalf("request default tile route: %v", err)
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
t.Fatalf("unexpected tile status: %d", response.StatusCode)
}
tileBody, err := io.ReadAll(response.Body)
if err != nil {
t.Fatalf("read tile body: %v", err)
}
if string(tileBody) != "tile" {
t.Fatalf("unexpected tile body: %q", string(tileBody))
}
response, err = http.Get(httpServer.URL + "/geojson/%E5%85%A8%E5%9B%BD%E8%A1%8C%E6%94%BF%E5%8C%BA.geojson?token=" + token.Token)
if err != nil {
t.Fatalf("request geojson route: %v", err)
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
t.Fatalf("unexpected geojson status: %d", response.StatusCode)
}
}
services/api-go/internal/basemapapi/server.go
+888
View File
@@ -0,0 +1,888 @@
package basemapapi
import (
"context"
"encoding/json"
"errors"
"fmt"
"html/template"
"net/http"
"net/url"
"strconv"
"strings"
"time"
"map-asset-gateway/api-go/internal/basemap"
)
type Server struct {
store *basemap.Store
system basemap.SystemInfo
startedAt time.Time
template *template.Template
mux *http.ServeMux
}
func New(store *basemap.Store, system basemap.SystemInfo) (*Server, error) {
tmpl, err := template.New("admin").Funcs(template.FuncMap{
"fmtTime": func(value time.Time) string {
if value.IsZero() {
return "-"
}
return value.Local().Format("2006-01-02 15:04:05")
},
"fmtTimePtr": func(value *time.Time) string {
if value == nil || value.IsZero() {
return "-"
}
return value.Local().Format("2006-01-02 15:04:05")
},
"defaultVersion": func(value *basemap.BasemapVersion) string {
if value == nil {
return "-"
}
return value.Version
},
"tokenGrants": func(grants []basemap.TokenGrant) string {
if len(grants) == 0 {
return "-"
}
parts := make([]string, 0, len(grants))
for _, grant := range grants {
if grant.BasemapVersion != nil {
parts = append(parts, grant.BasemapCode+":"+*grant.BasemapVersion)
} else {
parts = append(parts, grant.BasemapCode+":*")
}
}
return strings.Join(parts, ", ")
},
"vectorTokenGrants": func(grants []basemap.VectorTokenGrant) string {
if len(grants) == 0 {
return "-"
}
parts := make([]string, 0, len(grants))
for _, grant := range grants {
parts = append(parts, grant.VectorCode)
}
return strings.Join(parts, ", ")
},
"versionLabel": func(item basemap.BasemapVersion) string {
if item.IsDefault {
return item.Version + "（默认）"
}
return item.Version
},
"vectorPath": func(item basemap.VectorAsset) string {
if value, ok := item.Metadata["relative_path"].(string); ok && strings.TrimSpace(value) != "" {
return value
}
return item.FileName
},
"statusText": statusText,
"statusTone": statusTone,
"enabledText": func(enabled bool) string {
if enabled {
return "启用"
}
return "停用"
},
"enabledTone": func(enabled bool) string {
if enabled {
return "success"
}
return "danger"
},
"formatBytes": formatBytes,
"flashClass": func(value string) string {
if strings.HasPrefix(strings.TrimSpace(value), "操作失败：") {
return "danger"
}
return ""
},
}).Parse(adminTemplate)
if err != nil {
return nil, err
}
server := &Server{
store: store,
system: system,
startedAt: time.Now().UTC(),
template: tmpl,
mux: http.NewServeMux(),
}
server.routes()
return server, nil
}
func (s *Server) Handler() http.Handler {
return s.mux
}
func (s *Server) routes() {
s.mux.HandleFunc("GET /", func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/admin/", http.StatusFound)
})
s.mux.HandleFunc("GET /health", s.handleHealth)
s.mux.HandleFunc("GET /ready", s.handleReady)
s.mux.HandleFunc("GET /api/v1/system", s.handleSystem)
s.mux.HandleFunc("GET /api/v1/admin/dashboard", s.handleDashboardJSON)
s.mux.HandleFunc("GET /api/v1/admin/basemaps", s.handleBasemapsJSON)
s.mux.HandleFunc("GET /api/v1/admin/vectors", s.handleVectorsJSON)
s.mux.HandleFunc("GET /api/v1/admin/scan-sources", s.handleScanSourcesJSON)
s.mux.HandleFunc("POST /api/v1/admin/scan-sources", s.handleCreateScanSourceJSON)
s.mux.HandleFunc("POST /api/v1/admin/scan-sources/{code}/run", s.handleRunScanJSON)
s.mux.HandleFunc("POST /api/v1/admin/vectors/rescan", s.handleRunVectorScanJSON)
s.mux.HandleFunc("POST /api/v1/admin/basemaps/{code}/default-version", s.handleSetDefaultVersionJSON)
s.mux.HandleFunc("GET /api/v1/admin/targets", s.handleTargetsJSON)
s.mux.HandleFunc("POST /api/v1/admin/targets", s.handleCreateTargetJSON)
s.mux.HandleFunc("GET /api/v1/admin/tokens", s.handleTokensJSON)
s.mux.HandleFunc("POST /api/v1/admin/tokens", s.handleCreateTokenJSON)
s.mux.HandleFunc("POST /api/v1/admin/tokens/{id}/disable", s.handleDisableTokenJSON)
s.mux.HandleFunc("GET /api/v1/admin/push-records", s.handlePushRecordsJSON)
s.mux.HandleFunc("POST /api/v1/admin/pushes", s.handlePushJSON)
s.mux.HandleFunc("GET /api/v1/admin/vector-push-records", s.handleVectorPushRecordsJSON)
s.mux.HandleFunc("POST /api/v1/admin/vector-pushes", s.handleVectorPushJSON)
s.mux.HandleFunc("GET /api/v1/catalog/basemaps", s.handleCatalogList)
s.mux.HandleFunc("GET /api/v1/catalog/basemaps/{code}", s.handleCatalogGet)
s.mux.HandleFunc("GET /api/v1/catalog/vectors", s.handleVectorCatalogList)
s.mux.HandleFunc("GET /api/v1/catalog/vectors/{code}", s.handleVectorCatalogGet)
s.mux.HandleFunc("GET /tiles/{code}/{version}/{tilePath...}", s.handleTile)
s.mux.HandleFunc("GET /geojson/{assetPath...}", s.handleGeoJSONByPath)
s.mux.HandleFunc("GET /shp/{assetPath...}", s.handleShapeByPath)
s.mux.HandleFunc("GET /vectors/{code}/download", s.handleVectorDownload)
s.mux.HandleFunc("GET /admin/", s.handleAdminPage)
s.mux.HandleFunc("POST /admin/scan-sources", s.handleCreateScanSourceForm)
s.mux.HandleFunc("POST /admin/scan-sources/{code}/run", s.handleRunScanForm)
s.mux.HandleFunc("POST /admin/vectors/rescan", s.handleRunVectorScanForm)
s.mux.HandleFunc("POST /admin/basemaps/{code}/default", s.handleSetDefaultVersionForm)
s.mux.HandleFunc("POST /admin/targets", s.handleCreateTargetForm)
s.mux.HandleFunc("POST /admin/tokens", s.handleCreateTokenForm)
s.mux.HandleFunc("POST /admin/tokens/{id}/disable", s.handleDisableTokenForm)
s.mux.HandleFunc("POST /admin/pushes", s.handlePushForm)
s.mux.HandleFunc("POST /admin/vector-pushes", s.handleVectorPushForm)
}
func (s *Server) handleHealth(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]any{
"status": "ok",
"service": s.system.ServiceName,
"version": s.system.ServiceVersion,
})
}
func (s *Server) handleReady(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]any{
"status": "ready",
"started_at": s.startedAt,
"scan_root": s.system.ScanRoot,
"sqlite": s.system.SQLitePath,
})
}
func (s *Server) handleSystem(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, s.system)
}
func (s *Server) handleDashboardJSON(w http.ResponseWriter, r *http.Request) {
data, err := s.store.Dashboard(r.Context(), s.system, "", "")
if err != nil {
writeError(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, data)
}
func (s *Server) handleBasemapsJSON(w http.ResponseWriter, r *http.Request) {
items, err := s.store.ListBasemaps(r.Context())
if err != nil {
writeError(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, items)
}
func (s *Server) handleVectorsJSON(w http.ResponseWriter, r *http.Request) {
items, err := s.store.ListVectorAssets(r.Context())
if err != nil {
writeError(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, items)
}
func (s *Server) handleScanSourcesJSON(w http.ResponseWriter, r *http.Request) {
items, err := s.store.ListScanSources(r.Context())
if err != nil {
writeError(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, items)
}
func (s *Server) handleCreateScanSourceJSON(w http.ResponseWriter, r *http.Request) {
var input basemap.CreateScanSourceInput
if err := json.NewDecoder(r.Body).Decode(&input); err != nil {
writeError(w, http.StatusBadRequest, fmt.Errorf("decode json: %w", err))
return
}
item, err := s.store.EnsureScanSource(r.Context(), input)
if err != nil {
writeError(w, http.StatusBadRequest, err)
return
}
writeJSON(w, http.StatusCreated, item)
}
func (s *Server) handleRunScanJSON(w http.ResponseWriter, r *http.Request) {
run, err := s.store.RunScan(r.Context(), r.PathValue("code"))
if err != nil {
writeError(w, http.StatusBadRequest, err)
return
}
writeJSON(w, http.StatusOK, run)
}
func (s *Server) handleRunVectorScanJSON(w http.ResponseWriter, r *http.Request) {
result, err := s.store.RunVectorScan(r.Context())
if err != nil {
writeError(w, http.StatusBadRequest, err)
return
}
writeJSON(w, http.StatusOK, result)
}
func (s *Server) handleSetDefaultVersionJSON(w http.ResponseWriter, r *http.Request) {
var payload struct {
Version string `json:"version"`
}
if err := json.NewDecoder(r.Body).Decode(&payload); err != nil {
writeError(w, http.StatusBadRequest, err)
return
}
if err := s.store.SetDefaultVersion(r.Context(), r.PathValue("code"), payload.Version); err != nil {
writeError(w, http.StatusBadRequest, err)
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
func (s *Server) handleTargetsJSON(w http.ResponseWriter, r *http.Request) {
items, err := s.store.ListTargetSystems(r.Context())
if err != nil {
writeError(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, items)
}
func (s *Server) handleCreateTargetJSON(w http.ResponseWriter, r *http.Request) {
var input basemap.CreateTargetSystemInput
if err := json.NewDecoder(r.Body).Decode(&input); err != nil {
writeError(w, http.StatusBadRequest, err)
return
}
item, err := s.store.CreateTargetSystem(r.Context(), input)
if err != nil {
writeError(w, http.StatusBadRequest, err)
return
}
writeJSON(w, http.StatusCreated, item)
}
func (s *Server) handleTokensJSON(w http.ResponseWriter, r *http.Request) {
items, err := s.store.ListTokens(r.Context())
if err != nil {
writeError(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, items)
}
func (s *Server) handleCreateTokenJSON(w http.ResponseWriter, r *http.Request) {
input, err := decodeTokenInput(r)
if err != nil {
writeError(w, http.StatusBadRequest, err)
return
}
item, err := s.store.CreateToken(r.Context(), input)
if err != nil {
writeError(w, http.StatusBadRequest, err)
return
}
writeJSON(w, http.StatusCreated, item)
}
func (s *Server) handleDisableTokenJSON(w http.ResponseWriter, r *http.Request) {
if err := s.store.DisableToken(r.Context(), r.PathValue("id")); err != nil {
writeError(w, http.StatusBadRequest, err)
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
func (s *Server) handlePushRecordsJSON(w http.ResponseWriter, r *http.Request) {
items, err := s.store.ListPushRecords(r.Context(), 20)
if err != nil {
writeError(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, items)
}
func (s *Server) handleVectorPushRecordsJSON(w http.ResponseWriter, r *http.Request) {
items, err := s.store.ListVectorPushRecords(r.Context(), 20)
if err != nil {
writeError(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, items)
}
func (s *Server) handlePushJSON(w http.ResponseWriter, r *http.Request) {
var payload struct {
TargetCode string `json:"target_code"`
BasemapCode string `json:"basemap_code"`
Version string `json:"version"`
}
if err := json.NewDecoder(r.Body).Decode(&payload); err != nil {
writeError(w, http.StatusBadRequest, err)
return
}
item, err := s.store.PushBasemapVersion(r.Context(), payload.TargetCode, payload.BasemapCode, payload.Version)
if err != nil {
writeError(w, http.StatusBadRequest, err)
return
}
writeJSON(w, http.StatusCreated, item)
}
func (s *Server) handleVectorPushJSON(w http.ResponseWriter, r *http.Request) {
var payload struct {
TargetCode string `json:"target_code"`
VectorCode string `json:"vector_code"`
}
if err := json.NewDecoder(r.Body).Decode(&payload); err != nil {
writeError(w, http.StatusBadRequest, err)
return
}
item, err := s.store.PushVectorAsset(r.Context(), payload.TargetCode, payload.VectorCode)
if err != nil {
writeError(w, http.StatusBadRequest, err)
return
}
writeJSON(w, http.StatusCreated, item)
}
func (s *Server) handleCatalogList(w http.ResponseWriter, r *http.Request) {
auth, ok := s.requireToken(w, r)
if !ok {
return
}
items, err := s.store.ListBasemaps(r.Context())
if err != nil {
writeError(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, s.store.FilterCatalog(auth, items))
}
func (s *Server) handleCatalogGet(w http.ResponseWriter, r *http.Request) {
auth, ok := s.requireToken(w, r)
if !ok {
return
}
item, err := s.store.GetBasemapByCode(r.Context(), r.PathValue("code"))
if err != nil {
writeError(w, http.StatusNotFound, err)
return
}
filtered := s.store.FilterCatalog(auth, []basemap.Basemap{item})
if len(filtered) == 0 {
writeError(w, http.StatusForbidden, errors.New("token has no access to this basemap"))
return
}
writeJSON(w, http.StatusOK, filtered[0])
}
func (s *Server) handleVectorCatalogList(w http.ResponseWriter, r *http.Request) {
auth, ok := s.requireToken(w, r)
if !ok {
return
}
items, err := s.store.ListVectorAssets(r.Context())
if err != nil {
writeError(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, s.store.FilterVectors(auth, items))
}
func (s *Server) handleVectorCatalogGet(w http.ResponseWriter, r *http.Request) {
auth, ok := s.requireToken(w, r)
if !ok {
return
}
item, err := s.store.GetVectorAssetByCode(r.Context(), r.PathValue("code"))
if err != nil {
writeError(w, http.StatusNotFound, err)
return
}
filtered := s.store.FilterVectors(auth, []basemap.VectorAsset{item})
if len(filtered) == 0 {
writeError(w, http.StatusForbidden, errors.New("token has no access to this vector asset"))
return
}
writeJSON(w, http.StatusOK, filtered[0])
}
func (s *Server) handleTile(w http.ResponseWriter, r *http.Request) {
auth, ok := s.requireToken(w, r)
if !ok {
return
}
version := r.PathValue("version")
tilePath := r.PathValue("tilePath")
item, err := s.store.ResolveTile(r.Context(), auth, r.PathValue("code"), version, tilePath)
if err != nil && errors.Is(err, basemap.ErrBasemapVersionNotFound) && isTileZoom(version) {
item, err = s.store.ResolveDefaultTile(r.Context(), auth, r.PathValue("code"), version+"/"+tilePath)
}
if err != nil {
writeError(w, http.StatusNotFound, err)
return
}
w.Header().Set("Content-Type", item.ContentType)
http.ServeFile(w, r, item.FilePath)
}
func (s *Server) handleVectorDownload(w http.ResponseWriter, r *http.Request) {
auth, ok := s.requireToken(w, r)
if !ok {
return
}
item, err := s.store.ResolveVectorDownload(r.Context(), auth, r.PathValue("code"))
if err != nil {
writeError(w, http.StatusNotFound, err)
return
}
w.Header().Set("Content-Type", item.ContentType)
w.Header().Set("Content-Disposition", fmt.Sprintf("attachment; filename=%q", item.FileName))
if item.Zipped {
if err := basemap.WriteVectorZip(w, item); err != nil {
writeError(w, http.StatusInternalServerError, err)
}
return
}
http.ServeFile(w, r, item.FilePath)
}
func (s *Server) handleGeoJSONByPath(w http.ResponseWriter, r *http.Request) {
auth, ok := s.requireToken(w, r)
if !ok {
return
}
item, err := s.store.ResolveVectorDownloadByRef(r.Context(), auth, r.PathValue("assetPath"), "geojson")
if err != nil {
writeError(w, http.StatusNotFound, err)
return
}
w.Header().Set("Content-Type", item.ContentType)
http.ServeFile(w, r, item.FilePath)
}
func (s *Server) handleShapeByPath(w http.ResponseWriter, r *http.Request) {
auth, ok := s.requireToken(w, r)
if !ok {
return
}
item, err := s.store.ResolveVectorDownloadByRef(r.Context(), auth, r.PathValue("assetPath"), "shapefile", "shpzip")
if err != nil {
writeError(w, http.StatusNotFound, err)
return
}
w.Header().Set("Content-Type", item.ContentType)
w.Header().Set("Content-Disposition", fmt.Sprintf("attachment; filename=%q", item.FileName))
if item.Zipped {
if err := basemap.WriteVectorZip(w, item); err != nil {
writeError(w, http.StatusInternalServerError, err)
}
return
}
http.ServeFile(w, r, item.FilePath)
}
func (s *Server) handleAdminPage(w http.ResponseWriter, r *http.Request) {
data, err := s.store.Dashboard(r.Context(), s.system, r.URL.Query().Get("flash"), r.URL.Query().Get("token"))
if err != nil {
writeError(w, http.StatusInternalServerError, err)
return
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
if err := s.template.Execute(w, data); err != nil {
writeError(w, http.StatusInternalServerError, err)
}
}
func (s *Server) handleCreateScanSourceForm(w http.ResponseWriter, r *http.Request) {
if err := r.ParseForm(); err != nil {
s.redirectAdmin(w, r, "", "", err)
return
}
_, err := s.store.EnsureScanSource(r.Context(), basemap.CreateScanSourceInput{
Code: r.FormValue("code"),
Name: r.FormValue("name"),
RootPath: r.FormValue("root_path"),
ManifestName: r.FormValue("manifest_name"),
})
s.redirectAdmin(w, r, "扫描源已保存", "", err)
}
func (s *Server) handleRunScanForm(w http.ResponseWriter, r *http.Request) {
_, err := s.store.RunScan(r.Context(), r.PathValue("code"))
s.redirectAdmin(w, r, "底图扫描已完成", "", err)
}
func (s *Server) handleRunVectorScanForm(w http.ResponseWriter, r *http.Request) {
_, err := s.store.RunVectorScan(r.Context())
s.redirectAdmin(w, r, "矢量扫描已完成", "", err)
}
func (s *Server) handleSetDefaultVersionForm(w http.ResponseWriter, r *http.Request) {
if err := r.ParseForm(); err != nil {
s.redirectAdmin(w, r, "", "", err)
return
}
err := s.store.SetDefaultVersion(r.Context(), r.PathValue("code"), r.FormValue("version"))
s.redirectAdmin(w, r, "默认版本已更新", "", err)
}
func (s *Server) handleCreateTargetForm(w http.ResponseWriter, r *http.Request) {
if err := r.ParseForm(); err != nil {
s.redirectAdmin(w, r, "", "", err)
return
}
headers := map[string]string{}
for _, line := range splitLines(r.FormValue("headers")) {
key, value, ok := strings.Cut(line, ":")
if !ok {
continue
}
headers[strings.TrimSpace(key)] = strings.TrimSpace(value)
}
_, err := s.store.CreateTargetSystem(r.Context(), basemap.CreateTargetSystemInput{
Code: r.FormValue("code"),
Name: r.FormValue("name"),
CallbackURL: r.FormValue("callback_url"),
CallbackMethod: r.FormValue("callback_method"),
CallbackHeaders: headers,
})
s.redirectAdmin(w, r, "目标系统已保存", "", err)
}
func (s *Server) handleCreateTokenForm(w http.ResponseWriter, r *http.Request) {
input, err := decodeTokenInputFromForm(r)
if err != nil {
s.redirectAdmin(w, r, "", "", err)
return
}
item, err := s.store.CreateToken(r.Context(), input)
if err != nil {
s.redirectAdmin(w, r, "", "", err)
return
}
s.redirectAdmin(w, r, "访问令牌已创建", item.Token, nil)
}
func (s *Server) handleDisableTokenForm(w http.ResponseWriter, r *http.Request) {
err := s.store.DisableToken(r.Context(), r.PathValue("id"))
s.redirectAdmin(w, r, "访问令牌已停用", "", err)
}
func (s *Server) handlePushForm(w http.ResponseWriter, r *http.Request) {
if err := r.ParseForm(); err != nil {
s.redirectAdmin(w, r, "", "", err)
return
}
_, err := s.store.PushBasemapVersion(r.Context(), r.FormValue("target_code"), r.FormValue("basemap_code"), r.FormValue("version"))
s.redirectAdmin(w, r, "底图推送已完成", "", err)
}
func (s *Server) handleVectorPushForm(w http.ResponseWriter, r *http.Request) {
if err := r.ParseForm(); err != nil {
s.redirectAdmin(w, r, "", "", err)
return
}
_, err := s.store.PushVectorAsset(r.Context(), r.FormValue("target_code"), r.FormValue("vector_code"))
s.redirectAdmin(w, r, "矢量推送已完成", "", err)
}
func (s *Server) requireToken(w http.ResponseWriter, r *http.Request) (basemap.TokenAuth, bool) {
rawToken := extractToken(r)
auth, err := s.store.AuthorizeToken(r.Context(), rawToken)
if err != nil {
writeError(w, http.StatusUnauthorized, err)
return basemap.TokenAuth{}, false
}
return auth, true
}
func extractToken(r *http.Request) string {
if value := strings.TrimSpace(r.Header.Get("X-Access-Token")); value != "" {
return value
}
if header := strings.TrimSpace(r.Header.Get("Authorization")); header != "" {
if strings.HasPrefix(strings.ToLower(header), "bearer ") {
return strings.TrimSpace(header[7:])
}
}
for _, key := range []string{"token", "access_token"} {
if value := strings.TrimSpace(r.URL.Query().Get(key)); value != "" {
return value
}
}
return ""
}
func decodeTokenInput(r *http.Request) (basemap.CreateTokenInput, error) {
var payload struct {
Name string `json:"name"`
BasemapCodes []string `json:"basemap_codes"`
VectorCodes []string `json:"vector_codes"`
ExpiresAt string `json:"expires_at"`
}
if err := json.NewDecoder(r.Body).Decode(&payload); err != nil {
return basemap.CreateTokenInput{}, err
}
input := basemap.CreateTokenInput{
Name: payload.Name,
BasemapCodes: payload.BasemapCodes,
VectorCodes: payload.VectorCodes,
}
if strings.TrimSpace(payload.ExpiresAt) != "" {
parsed, err := time.Parse(time.RFC3339, payload.ExpiresAt)
if err != nil {
return basemap.CreateTokenInput{}, fmt.Errorf("parse expires_at: %w", err)
}
input.ExpiresAt = &parsed
}
return input, nil
}
func decodeTokenInputFromForm(r *http.Request) (basemap.CreateTokenInput, error) {
if err := r.ParseForm(); err != nil {
return basemap.CreateTokenInput{}, err
}
input := basemap.CreateTokenInput{
Name: r.FormValue("name"),
BasemapCodes: splitCSV(r.FormValue("basemap_codes")),
VectorCodes: splitCSV(r.FormValue("vector_codes")),
}
if value := strings.TrimSpace(r.FormValue("expires_at")); value != "" {
if minutes, err := strconv.Atoi(value); err == nil && minutes > 0 {
expiresAt := time.Now().UTC().Add(time.Duration(minutes) * time.Minute)
input.ExpiresAt = &expiresAt
}
}
return input, nil
}
func splitCSV(value string) []string {
fields := strings.FieldsFunc(value, func(r rune) bool {
return r == ',' || r == '\n' || r == '\r' || r == '\t' || r == ' '
})
result := make([]string, 0, len(fields))
for _, field := range fields {
field = strings.TrimSpace(field)
if field != "" {
result = append(result, field)
}
}
return result
}
func splitLines(value string) []string {
raw := strings.Split(strings.ReplaceAll(value, "\r\n", "\n"), "\n")
lines := make([]string, 0, len(raw))
for _, line := range raw {
line = strings.TrimSpace(line)
if line != "" {
lines = append(lines, line)
}
}
return lines
}
func isTileZoom(value string) bool {
if value == "" {
return false
}
for _, r := range value {
if r < '0' || r > '9' {
return false
}
}
return true
}
func statusText(value string) string {
switch strings.ToLower(strings.TrimSpace(value)) {
case "":
return "-"
case "active", "enabled":
return "启用"
case "disabled":
return "停用"
case "ready":
return "就绪"
case "succeeded":
return "成功"
case "failed":
return "失败"
case "pending":
return "待处理"
case "running":
return "运行中"
case "processing":
return "处理中"
case "current":
return "当前"
case "error":
return "异常"
default:
return value
}
}
func statusTone(value string) string {
switch strings.ToLower(strings.TrimSpace(value)) {
case "active", "enabled", "ready", "succeeded", "current":
return "success"
case "pending", "running", "processing":
return "warning"
case "disabled", "failed", "error", "expired", "invalid":
return "danger"
default:
return "neutral"
}
}
func formatBytes(size int64) string {
if size < 0 {
return "-"
}
units := []string{"B", "KB", "MB", "GB", "TB"}
value := float64(size)
unit := 0
for value >= 1024 && unit < len(units)-1 {
value /= 1024
unit++
}
if unit == 0 {
return fmt.Sprintf("%d %s", size, units[unit])
}
return fmt.Sprintf("%.1f %s", value, units[unit])
}
func localizeAdminError(err error) string {
if err == nil {
return ""
}
msg := strings.TrimSpace(err.Error())
quoted := quotedSegments(msg)
switch {
case msg == "scan source code is required":
return "扫描源编码不能为空"
case msg == "scan source root path is required":
return "扫描根目录不能为空"
case strings.HasPrefix(msg, "resolve root path:"):
return "扫描根目录解析失败：" + strings.TrimSpace(strings.TrimPrefix(msg, "resolve root path:"))
case len(quoted) == 1 && strings.HasPrefix(msg, `scan source "`) && strings.HasSuffix(msg, `" not found`):
return fmt.Sprintf("未找到扫描源：%s", quoted[0])
case len(quoted) == 1 && strings.HasPrefix(msg, `scan source "`) && strings.HasSuffix(msg, `" is disabled`):
return fmt.Sprintf("扫描源已停用：%s", quoted[0])
case strings.HasPrefix(msg, "read scan root:"):
return "读取扫描目录失败：" + strings.TrimSpace(strings.TrimPrefix(msg, "read scan root:"))
case strings.HasPrefix(msg, "scan root ") && strings.HasSuffix(msg, " is not a directory"):
return "配置的扫描根路径不是目录：" + strings.TrimSuffix(strings.TrimPrefix(msg, "scan root "), " is not a directory")
case msg == "target system code is required":
return "目标系统编码不能为空"
case msg == "callback url is required":
return "回调地址不能为空"
case len(quoted) == 1 && strings.HasPrefix(msg, `target system "`) && strings.HasSuffix(msg, `" not found`):
return fmt.Sprintf("未找到目标系统：%s", quoted[0])
case len(quoted) == 1 && strings.HasPrefix(msg, `target system "`) && strings.HasSuffix(msg, `" is disabled`):
return fmt.Sprintf("目标系统已停用：%s", quoted[0])
case msg == "token name is required":
return "令牌名称不能为空"
case msg == "at least one basemap code or vector code is required":
return "至少填写一个底图编码或矢量资源编码/路径"
case msg == "no valid basemap codes provided":
return "未提供有效的底图编码"
case len(quoted) == 1 && strings.HasPrefix(msg, `token "`) && strings.HasSuffix(msg, `" not found`):
return fmt.Sprintf("未找到访问令牌：%s", quoted[0])
case msg == "basemap code and version are required":
return "底图编码和版本不能为空"
case len(quoted) == 1 && strings.HasPrefix(msg, `basemap "`) && strings.HasSuffix(msg, `" not found`):
return fmt.Sprintf("未找到底图资源：%s", quoted[0])
case len(quoted) == 2 && strings.HasPrefix(msg, `version "`) && strings.Contains(msg, `" not found for basemap "`):
return fmt.Sprintf("底图资源 %s 未找到版本 %s", quoted[1], quoted[0])
case len(quoted) == 1 && strings.HasPrefix(msg, `vector asset "`) && strings.HasSuffix(msg, `" not found`):
return fmt.Sprintf("未找到矢量资源：%s", quoted[0])
case len(quoted) == 1 && strings.HasPrefix(msg, `vector asset "`) && strings.HasSuffix(msg, `" is not available in requested format`):
return fmt.Sprintf("矢量资源当前不可按所选格式提供：%s", quoted[0])
default:
return msg
}
}
func quotedSegments(value string) []string {
parts := strings.Split(value, `"`)
result := make([]string, 0, len(parts)/2)
for i := 1; i < len(parts); i += 2 {
result = append(result, parts[i])
}
return result
}
func (s *Server) redirectAdmin(w http.ResponseWriter, r *http.Request, flash string, token string, err error) {
if err != nil {
flash = "操作失败：" + localizeAdminError(err)
}
values := url.Values{}
if flash != "" {
values.Set("flash", flash)
}
if token != "" {
values.Set("token", token)
}
location := "/admin/"
if encoded := values.Encode(); encoded != "" {
location += "?" + encoded
}
http.Redirect(w, r, location, http.StatusSeeOther)
}
func writeJSON(w http.ResponseWriter, status int, value any) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.WriteHeader(status)
_ = json.NewEncoder(w).Encode(value)
}
func writeError(w http.ResponseWriter, status int, err error) {
writeJSON(w, status, map[string]any{
"error": err.Error(),
})
}
func runStartupScan(ctx context.Context, store *basemap.Store) error {
if !store.AutoScanEnabled() || strings.TrimSpace(store.ScanRoot()) == "" {
return nil
}
_, err := store.RunScan(ctx, "local-basemaps")
return err
}
services/api-go/internal/config/config.go
+108
View File
@@ -0,0 +1,108 @@
package config
import (
"fmt"
"os"
"path/filepath"
"strconv"
)
type Config struct {
Host string
Port int
APIBaseURL string
TilePublicBaseURL string
VectorPublicBaseURL string
SQLitePath string
BasemapScanRoot string
VectorScanRoot string
AutoScanOnStart bool
Env string
ServiceName string
ServiceVersion string
Commit string
}
func Load() (Config, error) {
if err := LoadEnvDefaults(); err != nil {
return Config{}, err
}
port, err := intEnv("API_PORT", 8910)
if err != nil {
return Config{}, fmt.Errorf("parse API_PORT: %w", err)
}
autoScanOnStart, err := boolEnv("AUTO_SCAN_ON_START", true)
if err != nil {
return Config{}, fmt.Errorf("parse AUTO_SCAN_ON_START: %w", err)
}
cfg := Config{
Host: stringEnv("API_HOST", "127.0.0.1"),
Port: port,
APIBaseURL: stringEnv("API_BASE_URL", ""),
TilePublicBaseURL: stringEnv("TILE_PUBLIC_BASE_URL", ""),
VectorPublicBaseURL: stringEnv("VECTOR_PUBLIC_BASE_URL", ""),
SQLitePath: stringEnv("SQLITE_PATH", filepath.Clean(filepath.Join("data", "map-asset-gateway.db"))),
BasemapScanRoot: stringEnv("BASEMAP_SCAN_ROOT", filepath.Clean(filepath.Join("data", "tiles"))),
VectorScanRoot: stringEnv("VECTOR_SCAN_ROOT", filepath.Clean(filepath.Join("data", "geojson"))),
AutoScanOnStart: autoScanOnStart,
Env: stringEnv("APP_ENV", "dev"),
ServiceName: stringEnv("SERVICE_NAME", "map-asset-gateway"),
ServiceVersion: stringEnv("SERVICE_VERSION", "0.2.0"),
Commit: stringEnv("SERVICE_COMMIT", "local"),
}
if cfg.APIBaseURL == "" {
cfg.APIBaseURL = deriveBaseURL(cfg.Host, cfg.Port)
}
if cfg.TilePublicBaseURL == "" {
cfg.TilePublicBaseURL = cfg.APIBaseURL
}
if cfg.VectorPublicBaseURL == "" {
cfg.VectorPublicBaseURL = cfg.APIBaseURL
}
return cfg, nil
}
func (c Config) ListenAddr() string {
return fmt.Sprintf("%s:%d", c.Host, c.Port)
}
func deriveBaseURL(host string, port int) string {
if host == "" || host == "0.0.0.0" {
host = "127.0.0.1"
}
return fmt.Sprintf("http://%s:%d", host, port)
}
func stringEnv(key, fallback string) string {
value, ok := os.LookupEnv(key)
if ok && value != "" {
return value
}
return fallback
}
func intEnv(key string, fallback int) (int, error) {
value, ok := os.LookupEnv(key)
if ok && value != "" {
parsed, err := strconv.Atoi(value)
if err != nil {
return 0, err
}
return parsed, nil
}
return fallback, nil
}
func boolEnv(key string, fallback bool) (bool, error) {
value, ok := os.LookupEnv(key)
if ok && value != "" {
parsed, err := strconv.ParseBool(value)
if err != nil {
return false, err
}
return parsed, nil
}
return fallback, nil
}
services/api-go/internal/config/envfile.go
+73
View File
@@ -0,0 +1,73 @@
package config
import (
"bufio"
"fmt"
"os"
"path/filepath"
"strings"
)
func LoadEnvDefaults() error {
if explicit := strings.TrimSpace(os.Getenv("APP_ENV_FILE")); explicit != "" {
if err := loadEnvFile(explicit); err != nil {
return fmt.Errorf("load APP_ENV_FILE %s: %w", explicit, err)
}
return nil
}
for _, candidate := range []string{
filepath.Join("config", "service.env"),
filepath.Join("..", "..", "config", "service.env"),
} {
if _, err := os.Stat(candidate); err == nil {
if err := loadEnvFile(candidate); err != nil {
return fmt.Errorf("load env file %s: %w", candidate, err)
}
return nil
}
}
return nil
}
func loadEnvFile(path string) error {
file, err := os.Open(path)
if err != nil {
return err
}
defer file.Close()
scanner := bufio.NewScanner(file)
lineNo := 0
for scanner.Scan() {
lineNo++
line := strings.TrimSpace(scanner.Text())
if line == "" || strings.HasPrefix(line, "#") {
continue
}
if strings.HasPrefix(line, "export ") {
line = strings.TrimSpace(strings.TrimPrefix(line, "export "))
}
key, value, found := strings.Cut(line, "=")
if !found {
return fmt.Errorf("line %d: missing '='", lineNo)
}
key = strings.TrimSpace(key)
value = strings.TrimSpace(value)
value = strings.Trim(value, `"'`)
if key == "" {
return fmt.Errorf("line %d: empty key", lineNo)
}
if _, exists := os.LookupEnv(key); exists {
continue
}
if err := os.Setenv(key, value); err != nil {
return fmt.Errorf("line %d: set %s: %w", lineNo, key, err)
}
}
if err := scanner.Err(); err != nil {
return err
}
return nil
}
services/api-go/internal/uid/deterministic.go
+25
View File
@@ -0,0 +1,25 @@
package uid
import (
"crypto/sha1"
"fmt"
"strings"
)
// Deterministic returns a stable UUID-like identifier derived from the input.
func Deterministic(parts ...string) string {
sum := sha1.Sum([]byte(strings.Join(parts, "::")))
b := sum[:16]
b[6] = (b[6] & 0x0f) | 0x50
b[8] = (b[8] & 0x3f) | 0x80
return fmt.Sprintf(
"%08x-%04x-%04x-%04x-%012x",
b[0:4],
b[4:6],
b[6:8],
b[8:10],
b[10:16],
)
}
services/api-go/internal/uid/uid.go
+25
View File
@@ -0,0 +1,25 @@
package uid
import (
"crypto/rand"
"fmt"
)
func New() string {
var b [16]byte
if _, err := rand.Read(b[:]); err != nil {
panic(err)
}
b[6] = (b[6] & 0x0f) | 0x40
b[8] = (b[8] & 0x3f) | 0x80
return fmt.Sprintf(
"%08x-%04x-%04x-%04x-%012x",
b[0:4],
b[4:6],
b[6:8],
b[8:10],
b[10:16],
)
}
start-all.bat
+38
View File
@@ -0,0 +1,38 @@
@echo off
setlocal
chcp 65001 >nul 2>&1
set "ROOT=%~dp0"
call "%ROOT%scripts\load-env.bat" "%ROOT%config\service.env"
set "LAUNCHER_DIR=%ROOT%temp\launcher"
if not exist "%LAUNCHER_DIR%" mkdir "%LAUNCHER_DIR%" >nul 2>&1
set "RS_API_PID_FILE=%LAUNCHER_DIR%\api.pid"
del /q "%RS_API_PID_FILE%" >nul 2>&1
if "%API_HOST%"=="" set "API_HOST=127.0.0.1"
if "%API_PORT%"=="" set "API_PORT=8910"
if "%API_BASE_URL%"=="" set "API_BASE_URL=http://127.0.0.1:%API_PORT%"
echo ========================================
echo Map Asset Gateway Launcher
echo ========================================
echo.
echo Env file: %ROOT%config\service.env
echo Starting Go service in a separate window...
echo.
powershell -NoProfile -ExecutionPolicy Bypass -Command "Start-Process -FilePath 'cmd.exe' -ArgumentList '/k','call ""%ROOT%scripts\run-api.bat""' -WorkingDirectory '%ROOT%' | Out-Null"
echo ========================================
echo Management Entry
echo ========================================
echo.
echo Admin Console : %API_BASE_URL%/admin/
echo API Health : %API_BASE_URL%/health
echo API Ready : %API_BASE_URL%/ready
echo Stop All : %ROOT%stop-all.bat
echo PID Files : %LAUNCHER_DIR%
echo.
if "%NO_PAUSE%"=="" pause
stop-all.bat
+101
View File
@@ -0,0 +1,101 @@
@echo off
setlocal
chcp 65001 >nul 2>&1
set "ROOT=%~dp0"
if exist "%ROOT%scripts\load-env.bat" call "%ROOT%scripts\load-env.bat" "%ROOT%config\service.env"
set "LAUNCHER_DIR=%ROOT%temp\launcher"
if "%API_PORT%"=="" set "API_PORT=8910"
echo ========================================
echo Map Asset Gateway Stopper
echo ========================================
echo.
call :stop_pid_file "Map Asset Gateway" "%LAUNCHER_DIR%\api.pid"
if not "%STOP_RESULT%"=="stopped" call :stop_port "Map Asset Gateway" "%API_PORT%"
call :stop_window_title "Map Asset Gateway"
echo.
echo Finished.
if "%NO_PAUSE%"=="" pause
exit /b 0
:stop_pid_file
set "STOP_RESULT=missing"
set "SERVICE_NAME=%~1"
set "PID_FILE=%~2"
if not exist "%PID_FILE%" (
echo [SKIP] %SERVICE_NAME% not running
set "SERVICE_NAME="
set "PID_FILE="
exit /b 0
)
set "TARGET_PID="
for /f "usebackq delims=" %%I in ("%PID_FILE%") do (
set "TARGET_PID=%%I"
goto :pid_loaded
)
:pid_loaded
if not defined TARGET_PID (
echo [SKIP] %SERVICE_NAME% not running
del /q "%PID_FILE%" >nul 2>&1
set "SERVICE_NAME="
set "PID_FILE="
set "TARGET_PID="
exit /b 0
)
call :kill_pid "%TARGET_PID%"
if not "%KILL_OK%"=="1" (
echo [SKIP] %SERVICE_NAME% not running
) else (
echo [OK] Stopped %SERVICE_NAME% PID=%TARGET_PID%
set "STOP_RESULT=stopped"
)
del /q "%PID_FILE%" >nul 2>&1
set "SERVICE_NAME="
set "PID_FILE="
set "TARGET_PID="
exit /b 0
:stop_port
set "SERVICE_NAME=%~1"
set "TARGET_PORT=%~2"
set "FOUND_PID="
for /f "tokens=5" %%I in ('netstat -ano -p TCP ^| findstr /R /C:":%TARGET_PORT% .*LISTENING"') do set "FOUND_PID=%%I"
if not defined FOUND_PID (
set "SERVICE_NAME="
set "TARGET_PORT="
exit /b 0
)
call :kill_pid "%FOUND_PID%"
if "%KILL_OK%"=="1" (
echo [OK] Stopped %SERVICE_NAME% by port %TARGET_PORT% PID=%FOUND_PID%
set "STOP_RESULT=stopped"
)
set "SERVICE_NAME="
set "TARGET_PORT="
set "FOUND_PID="
exit /b 0
:stop_window_title
taskkill /FI "WINDOWTITLE eq %~1*" /T /F >nul 2>&1
exit /b 0
:kill_pid
set "KILL_OK=0"
set "KILL_TARGET_PID=%~1"
if "%KILL_TARGET_PID%"=="" exit /b 0
taskkill /PID %KILL_TARGET_PID% /T /F >nul 2>&1
if errorlevel 1 (
powershell -NoProfile -ExecutionPolicy Bypass -Command "try { Stop-Process -Id %KILL_TARGET_PID% -Force -ErrorAction Stop; exit 0 } catch { exit 1 }" >nul 2>&1
if not errorlevel 1 set "KILL_OK=1"
) else (
set "KILL_OK=1"
)
set "KILL_TARGET_PID="
exit /b 0