Compare commits
2 Commits
bb717faac3
...
a2de8c5078
| Author | SHA1 | Date | |
|---|---|---|---|
| a2de8c5078 | |||
| e2e1beb6f7 |
159
AGENTS.md
159
AGENTS.md
@@ -2,85 +2,81 @@
|
||||
|
||||
Guidelines for AI coding agents working on BillAI - a microservices bill analysis system.
|
||||
|
||||
**Current Version:** 1.4.0 | Go: 1.24.0 | Node: 20+ | Python: 3.12+
|
||||
**Version:** See `CHANGELOG.md` for current version. Latest tag usually matches.
|
||||
|
||||
## Architecture
|
||||
|
||||
- `web/` - SvelteKit 5 + TailwindCSS 4 + TypeScript (Frontend, port 3000)
|
||||
- `server/` - Go 1.24 + Gin + MongoDB (API, port 8080)
|
||||
- `analyzer/` - Python 3.12 + FastAPI (Data cleaning, port 8001)
|
||||
|
||||
SvelteKit proxies `/api/*` requests to Go backend via `web/src/routes/api/[...path]/+server.ts`.
|
||||
**Proxy Caveat:** SvelteKit proxies `/api/*` to Go via `web/src/routes/api/[...path]/+server.ts`, but **only GET and POST are forwarded**. The Go backend intentionally uses `POST` for mutations (update, delete, manual create) to work around this. If you add PUT/PATCH/DELETE endpoints, you must also add them to the proxy.
|
||||
|
||||
## Build/Lint/Test Commands
|
||||
|
||||
### Frontend (web/)
|
||||
### Frontend (`web/`)
|
||||
```bash
|
||||
npm run dev # Start dev server
|
||||
npm run build # Production build
|
||||
npm run check # TypeScript check
|
||||
npm run lint # Prettier + ESLint
|
||||
npm run format # Format with Prettier
|
||||
npm run test # Run all tests (CI mode)
|
||||
npm run check # TypeScript + Svelte check
|
||||
npm run lint # Prettier --check + ESLint
|
||||
npm run format # Prettier --write
|
||||
npm run test # vitest --run (CI mode)
|
||||
npx vitest run src/xxx.spec.ts # Run single test file
|
||||
npx vitest run -t "pattern" # Run by name pattern
|
||||
```
|
||||
|
||||
### Backend (server/)
|
||||
**Note:** `web/` has a `yarn.lock` but scripts in `package.json` use `npm run`.
|
||||
|
||||
### Backend (`server/`)
|
||||
```bash
|
||||
go run . # Start server
|
||||
go build -o server . # Build binary
|
||||
go run . # Start server (reads server/config.yaml)
|
||||
go build -o server .
|
||||
go test ./... # Run all tests
|
||||
go test ./handler/... # Run handler tests
|
||||
go test -run TestName ./... # Run single test
|
||||
go test -v ./handler/... # Verbose output
|
||||
```
|
||||
|
||||
### Analyzer (analyzer/)
|
||||
### Analyzer (`analyzer/`)
|
||||
```bash
|
||||
python server.py # Start FastAPI
|
||||
uvicorn server:app --reload # Hot reload
|
||||
python server.py # Start FastAPI (has `if __name__ == "__main__"`)
|
||||
uvicorn server:app --reload # Requires cwd == analyzer/
|
||||
pytest # Run all tests
|
||||
pytest test_jd_cleaner.py # Single test file
|
||||
pytest -k "pattern" # Run by pattern
|
||||
```
|
||||
|
||||
### Docker
|
||||
```bash
|
||||
docker-compose up -d --build # Start/rebuild all services
|
||||
docker-compose logs -f server # Follow logs
|
||||
docker-compose down # Stop services
|
||||
docker compose up -d --build --remove-orphans # Start/rebuild all
|
||||
docker compose logs -f server # Follow logs
|
||||
docker compose down # Stop services
|
||||
```
|
||||
|
||||
## Code Style
|
||||
## Code Style & Conventions
|
||||
|
||||
### General
|
||||
- **Comments:** Chinese common for business logic; English for technical.
|
||||
- **Conventions:** Follow existing patterns. Check `package.json`/`go.mod`/`requirements.txt` before adding dependencies.
|
||||
### TypeScript/Svelte (`web/`)
|
||||
- **Formatting:** Prettier (tabs, single quotes, `trailingComma: none`, printWidth 100)
|
||||
- **Imports:** Use `$lib` alias. No relative imports for lib modules.
|
||||
- **Svelte 5:** Runes (`$state`, `$derived`, `$effect`, `$props`). Events: `onclick={fn}`.
|
||||
- **Types:** `export interface` for models. Frontend uses `camelCase`, API uses `snake_case`. Converters live in `$lib/models/bill.ts`.
|
||||
- **Auth:** Token stored in `localStorage` key `auth`. Always use `apiFetch()` from `$lib/api.ts` for authenticated requests.
|
||||
|
||||
### TypeScript/Svelte (web/)
|
||||
- **Formatting:** Prettier (tabs, single quotes, printWidth 100)
|
||||
- **Naming:** `PascalCase` for types/components, `camelCase` for variables
|
||||
- **Imports:** Use `$lib` alias, `$app/*` for SvelteKit builtins. No relative paths for lib modules.
|
||||
- **Svelte 5:** Use runes (`$state`, `$derived`, `$effect`, `$props`). Event: `onclick={fn}`.
|
||||
- **Types:** `export interface` for models. Frontend `camelCase`, API `snake_case`. Converters in `$lib/models/bill.ts`.
|
||||
- **Error Handling:** Check `response.ok`, throw `Error(\`HTTP ${status}\`)`. On 401: `auth.logout()` + redirect.
|
||||
- **Auth:** `createAuthStore()` in `$lib/stores/auth.ts`. Token in `localStorage` key `auth`. Use `apiFetch()` in `$lib/api.ts`.
|
||||
|
||||
### Go Backend (server/)
|
||||
### Go (`server/`)
|
||||
- **Module:** `billai-server` (import path). Use this in `go test` / `go build` when outside the directory.
|
||||
- **Layer:** `handler` → `service` → `adapter`/`repository` → `model`. No business logic in handlers.
|
||||
- **Struct tags:** JSON `snake_case`, `omitempty` optional. Pointer for optional patch fields. Sensitive: `json:"-"`.
|
||||
- **Error handling:** 500 for DB errors, 400 for bad requests, 404 not found. Wrap with `fmt.Errorf("context: %w", err)`.
|
||||
- **Response:** `Result bool`, `Message`, `Data *T`. Auth: `success bool`, `error`, `data`.
|
||||
- **Time:** Use custom `LocalTime` type (serializes as `2006-01-02 15:04:05`).
|
||||
- **Soft delete:** Never hard-delete. Filter `is_deleted: false` in queries.
|
||||
- **Struct tags:** JSON `snake_case`, `omitempty` for optional. Pointer types for optional patch fields. Sensitive fields: `json:"-"`.
|
||||
- **Response shapes:**
|
||||
- Business APIs: `result bool`, `message string`, `data *T`
|
||||
- Auth APIs: `success bool`, `error string`, `data *T` (and `code` for error types)
|
||||
- **Time:** Custom `LocalTime` type serializes as `"2006-01-02 15:04:05"`.
|
||||
- **Soft delete:** Never hard-delete. All queries filter `is_deleted: false`.
|
||||
|
||||
### Python Analyzer (analyzer/)
|
||||
- **Style:** PEP 8. `snake_case` variables, `UPPER_CASE` constants. Prefix private globals with `_`.
|
||||
- **Type hints:** Mandatory. Use `Optional[str]` or `str | None`.
|
||||
### Python (`analyzer/`)
|
||||
- **Style:** PEP 8. `snake_case` vars, `UPPER_CASE` constants. Prefix private globals with `_`.
|
||||
- **Type hints:** Mandatory. Prefer `str | None` or `Optional[str]`.
|
||||
- **Models:** `pydantic.BaseModel` for API schemas.
|
||||
- **Cleaners:** Extend `BaseCleaner(ABC)` from `cleaners/base.py`. Category rules in `config/category.yaml`.
|
||||
|
||||
## Key Patterns
|
||||
## Key Patterns & Quirks
|
||||
|
||||
### API Flow
|
||||
```
|
||||
@@ -89,35 +85,70 @@ Browser → SvelteKit proxy → Go (Gin) → handler → service → adapter →
|
||||
```
|
||||
|
||||
### Authentication
|
||||
- JWT (HS256). Token in `localStorage` key `auth`. Header: `Authorization: Bearer <token>`.
|
||||
- `middleware.AuthRequired()` wraps `/api/*` (except `/api/auth/*`).
|
||||
- 401 anywhere → `auth.logout()` + redirect `/login`.
|
||||
- JWT (HS256). Header: `Authorization: Bearer <token>`.
|
||||
- `middleware.AuthRequired()` guards authed routes. Public routes: `/api/auth/*`, `/api/changelog`, `/health`.
|
||||
- Frontend `apiFetch()` intercepts 401 → `auth.logout()` + redirect `/login`.
|
||||
|
||||
### File Processing
|
||||
Upload: ZIP/XLSX → Extract → Convert UTF-8 CSV → Detect bill type → Deduplicate → Clean → Save to MongoDB.
|
||||
### File Processing Pipeline
|
||||
Upload: ZIP/XLSX → Extract → Convert UTF-8 CSV → Detect bill type (alipay/wechat/jd) → Deduplicate against MongoDB → Clean via Python → Save cleaned data.
|
||||
|
||||
### Cross-Batch Refund Reconciliation
|
||||
When the original purchase and its refund are in different upload batches, within-batch logic can't match them. Two reconciliation paths handle this:
|
||||
|
||||
**Alipay** (`handler/upload.go` step 14): After cleaning, `cleaner.unresolved_refunds` (refund rows that found no matching expense in the same file) is returned by FastAPI and iterated in Go. `ReconcileRefund()` looks up the original expense by `transaction_id` or `merchant_order_no` and either soft-deletes (full refund) or deducts `amount` (partial). Alipay refund rows have their own distinct `transaction_id` (suffixed `_<refund_id>`), so they pass raw dedup and are naturally idempotent on re-upload.
|
||||
|
||||
**WeChat** (`handler/upload.go` step 15): WeChat re-exports the *same row* (same `transaction_id`) with an updated `当前状态` field (`已全额退款` or `已退款(¥X)`). `DeduplicateRawFile` detects these duplicate rows, extracts the refund info into `DeduplicateResult.WechatRefundUpdates`, and `ReconcileWechatRefund()` applies the update. WeChat's `¥X` is the **cumulative** total refunded, so `CleanedBill.original_amount` (set at first save) is used to compute `remaining = original_amount - X`.
|
||||
|
||||
### Adapter (Go ↔ Python)
|
||||
`adapter.Cleaner` interface: HTTP (`adapter/http`, default) or subprocess (`adapter/python`). Set via `ANALYZER_MODE` env var.
|
||||
`adapter.Cleaner` interface. Two modes:
|
||||
- `http` (default): calls FastAPI at `ANALYZER_URL`
|
||||
- `subprocess`: spawns `python analyzer/clean_bill.py`
|
||||
|
||||
Set via `ANALYZER_MODE` env var or `server/config.yaml` `analyzer.mode`.
|
||||
|
||||
### Config Precedence
|
||||
Go backend reads `server/config.yaml`, but Docker compose sets env vars (`ANALYZER_URL`, `MONGO_URI`, `JWT_SECRET`, etc.) that override it.
|
||||
|
||||
### SvelteKit Config Notes
|
||||
- `svelte.config.js` uses `adapter-node` for Docker SSR.
|
||||
- `csrf.trustedOrigins: ['*']` disables CSRF checks.
|
||||
- `onwarn` ignores all `a11y_*` warnings (chart components).
|
||||
|
||||
### Deployment
|
||||
- Gitea Actions self-hosted runner (`.gitea/workflows/deploy.yaml`), not GitHub.
|
||||
- `deploy.sh` is the manual deployment script (same logic as CI).
|
||||
|
||||
### Test Coverage
|
||||
Sparse. Existing tests:
|
||||
- `web/src/demo.spec.ts` / `page.svelte.spec.ts`
|
||||
- `server/service/changelog_test.go`
|
||||
- `analyzer/test_jd_cleaner.py`
|
||||
|
||||
## Important Files
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `web/src/lib/api.ts` | Central API client, auth injection |
|
||||
| `web/src/lib/stores/auth.ts` | Auth state, JWT handling |
|
||||
| `web/src/lib/models/bill.ts` | UIBill model + converters |
|
||||
| `server/main.go` | Entry point |
|
||||
| `server/handler/upload.go` | Full upload pipeline |
|
||||
| `server/handler/bills.go` | List/filter bills |
|
||||
| `server/model/bill.go` | Bill models, LocalTime type |
|
||||
| `server/adapter/adapter.go` | Cleaner interface |
|
||||
| `server/repository/mongo/repository.go` | MongoDB implementation |
|
||||
| `analyzer/server.py` | FastAPI entry |
|
||||
| `web/src/lib/api.ts` | Central API client, auth injection, all API functions |
|
||||
| `web/src/lib/stores/auth.ts` | Auth state, JWT handling, localStorage key `auth` |
|
||||
| `web/src/lib/models/bill.ts` | UIBill model + snake_case ↔ camelCase converters |
|
||||
| `web/src/routes/api/[...path]/+server.ts` | SvelteKit → Go proxy (GET/POST only) |
|
||||
| `server/main.go` | Entry point, wires adapter + repository + router |
|
||||
| `server/config.yaml` | Go backend config (Mongo, auth, paths, analyzer mode) |
|
||||
| `server/router/router.go` | Route table, auth group definitions |
|
||||
| `server/handler/upload.go` | Full upload pipeline handler |
|
||||
| `server/handler/bills.go` | List/filter/update/delete bills |
|
||||
| `server/model/bill.go` | Bill models, LocalTime type, BSON/JSON marshaling |
|
||||
| `server/adapter/adapter.go` | Cleaner interface definition |
|
||||
| `server/repository/mongo/repository.go` | MongoDB implementation, soft-delete queries |
|
||||
| `analyzer/server.py` | FastAPI entry, bill detection/clean endpoints |
|
||||
| `analyzer/cleaners/base.py` | BaseCleaner ABC |
|
||||
| `analyzer/category.py` | Category inference |
|
||||
| `analyzer/category.py` | Category inference engine |
|
||||
| `docker-compose.yaml` | Full stack orchestration |
|
||||
|
||||
## Agent Guidelines
|
||||
- **Before coding:** Search codebase to understand existing patterns and dependencies
|
||||
- **Dependencies:** Check `package.json`/`go.mod`/`requirements.txt` before adding new packages
|
||||
- **Tests:** Always run relevant test suite before committing changes
|
||||
- **Git commits:** Provide clear messages explaining the "why" of changes
|
||||
- **File references:** Use relative `file_path:line_number` format (e.g., `server/handler/changelog.go:12`) when mentioning code locations
|
||||
|
||||
- **Before coding:** Search codebase to understand existing patterns and dependencies.
|
||||
- **Dependencies:** Check `package.json`/`go.mod`/`requirements.txt` before adding new packages.
|
||||
- **Tests:** Run the relevant test suite before committing. If no tests exist for your change, verify manually.
|
||||
- **Git commits:** Provide clear messages explaining the "why" of changes.
|
||||
- **File references:** Use relative `file_path:line_number` format (e.g., `server/handler/changelog.go:12`) when mentioning code locations.
|
||||
|
||||
17
CHANGELOG.md
17
CHANGELOG.md
@@ -7,6 +7,23 @@
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### 新增
|
||||
- **微信跨批次退款核销** - 重复上传微信账单时,自动识别"当前状态"字段中的退款信息并核销原始支出记录
|
||||
- 支持「已全额退款」:原支出记录软删除
|
||||
- 支持「已退款(¥X)」:累计退款金额从原始金额(`original_amount`)中扣减,剩余金额更新到记录
|
||||
- 新增 `CleanedBill.original_amount` 字段,存储入库时的原始金额,保证多次部分退款累计计算正确
|
||||
- `ReconcileWechatRefund` 接口和 MongoDB 实现
|
||||
- **支付宝跨批次退款核销** - 上传含退款行的支付宝账单时,若原消费记录来自更早的批次,自动在数据库中核销
|
||||
- 退款行的「交易订单号」带后缀(`原订单号_退款编号`),天然幂等:重复上传时被原始数据去重拦截,不会重复核销
|
||||
- 全额退款软删除,部分退款扣减金额并追加备注
|
||||
- `ReconcileRefund` 接口和 MongoDB 实现
|
||||
|
||||
### 技术改进
|
||||
- `server/service/bill.go`:`DeduplicateRawFile` 在微信账单去重阶段检测已退款状态行,收集 `WechatRefundUpdates` 供后续核销
|
||||
- `server/service/bill.go`:`saveCleanedBillsFromCSV` / `saveCleanedBillsFromJSON` 入库时同步写入 `original_amount`
|
||||
- `analyzer/cleaners/alipay.py`:`_process_expenses` 收集同批次内未匹配的退款(`unresolved_refunds`),通过 FastAPI 响应透传至 Go
|
||||
- `analyzer/cleaners/base.py`:`BaseCleaner` 基类新增 `unresolved_refunds` 属性
|
||||
|
||||
## [1.4.0] - 2026-03-23
|
||||
|
||||
### 新增
|
||||
|
||||
5
CLAUDE.md
Normal file
5
CLAUDE.md
Normal file
@@ -0,0 +1,5 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
@AGENTS.md
|
||||
@@ -100,45 +100,54 @@ class AlipayCleaner(BaseCleaner):
|
||||
def _aggregate_refunds(self, refund_rows: list) -> dict:
|
||||
"""聚合退款金额"""
|
||||
order_refunds = {}
|
||||
|
||||
|
||||
for row in refund_rows:
|
||||
if len(row) >= 11:
|
||||
refund_order_no = row[9].strip()
|
||||
refund_merchant_no = row[10].strip()
|
||||
refund_amount = parse_amount(row[6])
|
||||
|
||||
|
||||
original_order = refund_order_no.split("_")[0] if "_" in refund_order_no else refund_order_no
|
||||
key = original_order if original_order else refund_merchant_no
|
||||
|
||||
|
||||
if key:
|
||||
if key not in order_refunds:
|
||||
order_refunds[key] = Decimal("0")
|
||||
order_refunds[key] += refund_amount
|
||||
order_refunds[key] = {
|
||||
"amount": Decimal("0"),
|
||||
"merchant_order_no": refund_merchant_no,
|
||||
"refund_order_no": refund_order_no,
|
||||
"time": row[0],
|
||||
"merchant": row[2],
|
||||
"description": row[4] if len(row) > 4 else "",
|
||||
}
|
||||
order_refunds[key]["amount"] += refund_amount
|
||||
print(f" 退款记录: {row[0]} | {row[2]} | {refund_amount}元")
|
||||
|
||||
|
||||
return order_refunds
|
||||
|
||||
def _process_expenses(self, expense_rows: list, order_refunds: dict) -> list:
|
||||
"""处理支出记录"""
|
||||
final_rows = []
|
||||
|
||||
matched_keys = set()
|
||||
|
||||
for row in expense_rows:
|
||||
if len(row) >= 12:
|
||||
order_no = row[9].strip()
|
||||
merchant_no = row[10].strip()
|
||||
expense_amount = parse_amount(row[6])
|
||||
|
||||
|
||||
# 查找对应的退款
|
||||
refund_amount = Decimal("0")
|
||||
matched_key = None
|
||||
|
||||
for key, amount in order_refunds.items():
|
||||
|
||||
for key, refund in order_refunds.items():
|
||||
if key and (order_no == key or merchant_no == key or order_no.startswith(key)):
|
||||
refund_amount = amount
|
||||
refund_amount = refund["amount"]
|
||||
matched_key = key
|
||||
break
|
||||
|
||||
|
||||
if matched_key:
|
||||
matched_keys.add(matched_key)
|
||||
if refund_amount >= expense_amount:
|
||||
# 全额退款,删除
|
||||
self.stats["fully_refunded"] += 1
|
||||
@@ -148,10 +157,10 @@ class AlipayCleaner(BaseCleaner):
|
||||
remaining = expense_amount - refund_amount
|
||||
new_row = row.copy()
|
||||
new_row[6] = format_amount(remaining)
|
||||
|
||||
|
||||
original_remark = new_row[11] if len(new_row) > 11 else ""
|
||||
new_row[11] = f"原金额{expense_amount}元,退款{refund_amount}元{';' + original_remark if original_remark else ''}"
|
||||
|
||||
|
||||
final_rows.append(new_row)
|
||||
self.stats["partially_refunded"] += 1
|
||||
print(f" 部分退款: {row[0]} | {row[2]} | 原{expense_amount}元 -> {format_amount(remaining)}元")
|
||||
@@ -163,7 +172,22 @@ class AlipayCleaner(BaseCleaner):
|
||||
self.stats["zero_amount"] = self.stats.get("zero_amount", 0) + 1
|
||||
else:
|
||||
final_rows.append(row)
|
||||
|
||||
|
||||
# 本批次内未匹配到对应支出的退款,交由调用方做跨批次核销
|
||||
self.unresolved_refunds = [
|
||||
{
|
||||
"order_no": key,
|
||||
"merchant_order_no": refund["merchant_order_no"],
|
||||
"refund_order_no": refund["refund_order_no"],
|
||||
"amount": float(format_amount(refund["amount"])),
|
||||
"time": refund["time"],
|
||||
"merchant": refund["merchant"],
|
||||
"description": refund["description"],
|
||||
}
|
||||
for key, refund in order_refunds.items()
|
||||
if key not in matched_keys
|
||||
]
|
||||
|
||||
return final_rows
|
||||
|
||||
def _is_platform_merchant(self, merchant: str) -> bool:
|
||||
|
||||
@@ -220,6 +220,9 @@ class BaseCleaner(ABC):
|
||||
"category_adjusted": 0,
|
||||
"final_count": 0,
|
||||
}
|
||||
|
||||
# 本次清理中未能在同批次内匹配到对应支出的退款(跨批次核销用)
|
||||
self.unresolved_refunds: list[dict] = []
|
||||
|
||||
def set_date_range(self, start_date: date | None, end_date: date | None):
|
||||
"""设置日期筛选范围"""
|
||||
|
||||
@@ -54,7 +54,23 @@ class WechatCleaner(BaseCleaner):
|
||||
|
||||
# 第三步:处理退款(包括转账退款)
|
||||
final_expense_rows, income_rows = self._process_refunds(expense_rows, income_rows)
|
||||
|
||||
|
||||
# 收集跨批次未匹配的 -退款 行(当前批次无对应支出记录,需 Go 侧跨批次核销)
|
||||
expense_merchants = {exp[2].strip() for exp in expense_rows}
|
||||
for refund_row in refund_rows:
|
||||
if refund_row[2].strip() not in expense_merchants:
|
||||
amount = float(parse_amount(refund_row[5]))
|
||||
if amount > 0:
|
||||
self.unresolved_refunds.append({
|
||||
"order_no": "",
|
||||
"merchant_order_no": refund_row[9].strip() if len(refund_row) > 9 else "",
|
||||
"refund_order_no": refund_row[8].strip() if len(refund_row) > 8 else "",
|
||||
"amount": amount,
|
||||
"time": refund_row[0],
|
||||
"merchant": refund_row[2],
|
||||
"description": refund_row[3] if len(refund_row) > 3 else "",
|
||||
})
|
||||
|
||||
print(f"\n处理结果:")
|
||||
print(f" 全额退款删除: {self.stats['fully_refunded']} 条")
|
||||
print(f" 部分退款调整: {self.stats['partially_refunded']} 条")
|
||||
|
||||
@@ -52,6 +52,7 @@ class CleanResponse(BaseModel):
|
||||
bill_type: str
|
||||
message: str
|
||||
output_path: Optional[str] = None
|
||||
unresolved_refunds: list[dict] = []
|
||||
|
||||
|
||||
class CategoryRequest(BaseModel):
|
||||
@@ -138,27 +139,27 @@ def do_clean(
|
||||
start: str = None,
|
||||
end: str = None,
|
||||
output_format: str = "csv"
|
||||
) -> tuple[bool, str, str]:
|
||||
) -> tuple[bool, str, str, list[dict]]:
|
||||
"""
|
||||
执行清洗逻辑
|
||||
|
||||
|
||||
Returns:
|
||||
(success, bill_type, message)
|
||||
(success, bill_type, message, unresolved_refunds)
|
||||
"""
|
||||
# 检查文件是否存在
|
||||
if not Path(input_path).exists():
|
||||
return False, "", f"文件不存在: {input_path}"
|
||||
|
||||
return False, "", f"文件不存在: {input_path}", []
|
||||
|
||||
# 检测账单类型
|
||||
if bill_type == "auto":
|
||||
detected_type = detect_bill_type(input_path)
|
||||
if detected_type is None:
|
||||
return False, "", "无法识别账单类型"
|
||||
return False, "", "无法识别账单类型", []
|
||||
bill_type = detected_type
|
||||
|
||||
|
||||
# 计算日期范围
|
||||
start_date, end_date = compute_date_range_from_values(year, month, start, end)
|
||||
|
||||
|
||||
# 创建对应的清理器
|
||||
try:
|
||||
if bill_type == "alipay":
|
||||
@@ -167,15 +168,15 @@ def do_clean(
|
||||
cleaner = JDCleaner(input_path, output_path, output_format)
|
||||
else:
|
||||
cleaner = WechatCleaner(input_path, output_path, output_format)
|
||||
|
||||
|
||||
cleaner.set_date_range(start_date, end_date)
|
||||
cleaner.clean()
|
||||
|
||||
|
||||
type_names = {"alipay": "支付宝", "wechat": "微信", "jd": "京东白条"}
|
||||
return True, bill_type, f"✅ {type_names.get(bill_type, bill_type)}账单清洗完成"
|
||||
|
||||
return True, bill_type, f"✅ {type_names.get(bill_type, bill_type)}账单清洗完成", cleaner.unresolved_refunds
|
||||
|
||||
except Exception as e:
|
||||
return False, bill_type, f"清洗失败: {str(e)}"
|
||||
return False, bill_type, f"清洗失败: {str(e)}", []
|
||||
|
||||
|
||||
# =============================================================================
|
||||
@@ -215,7 +216,7 @@ async def clean_bill(request: CleanRequest):
|
||||
|
||||
接收账单文件路径,执行清洗后输出到指定路径
|
||||
"""
|
||||
success, bill_type, message = do_clean(
|
||||
success, bill_type, message, unresolved_refunds = do_clean(
|
||||
input_path=request.input_path,
|
||||
output_path=request.output_path,
|
||||
bill_type=request.bill_type or "auto",
|
||||
@@ -225,15 +226,16 @@ async def clean_bill(request: CleanRequest):
|
||||
end=request.end,
|
||||
output_format=request.format or "csv"
|
||||
)
|
||||
|
||||
|
||||
if not success:
|
||||
raise HTTPException(status_code=400, detail=message)
|
||||
|
||||
|
||||
return CleanResponse(
|
||||
success=True,
|
||||
bill_type=bill_type,
|
||||
message=message,
|
||||
output_path=request.output_path
|
||||
output_path=request.output_path,
|
||||
unresolved_refunds=unresolved_refunds
|
||||
)
|
||||
|
||||
|
||||
@@ -264,7 +266,7 @@ async def clean_bill_upload(
|
||||
output_path = tmp_output.name
|
||||
|
||||
try:
|
||||
success, detected_type, message = do_clean(
|
||||
success, detected_type, message, unresolved_refunds = do_clean(
|
||||
input_path=input_path,
|
||||
output_path=output_path,
|
||||
bill_type=bill_type or "auto",
|
||||
@@ -274,15 +276,16 @@ async def clean_bill_upload(
|
||||
end=end,
|
||||
output_format=format or "csv"
|
||||
)
|
||||
|
||||
|
||||
if not success:
|
||||
raise HTTPException(status_code=400, detail=message)
|
||||
|
||||
|
||||
return CleanResponse(
|
||||
success=True,
|
||||
bill_type=detected_type,
|
||||
message=message,
|
||||
output_path=output_path
|
||||
output_path=output_path,
|
||||
unresolved_refunds=unresolved_refunds
|
||||
)
|
||||
|
||||
finally:
|
||||
|
||||
@@ -11,7 +11,8 @@ import sys
|
||||
sys.stdout.reconfigure(encoding='utf-8')
|
||||
|
||||
def test_jd_cleaner():
|
||||
zip_path = r'D:\Projects\BillAI\mock_data\京东交易流水(申请时间2026年01月26日13时29分47秒)(密码683263)_209.zip'
|
||||
base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
zip_path = os.path.join(base_dir, 'mock_data', '京东交易流水(申请时间2026年01月26日13时29分47秒)(密码683263)_209.zip')
|
||||
|
||||
with zipfile.ZipFile(zip_path, 'r') as zf:
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
|
||||
@@ -13,8 +13,20 @@ type CleanOptions struct {
|
||||
|
||||
// CleanResult 清洗结果
|
||||
type CleanResult struct {
|
||||
BillType string // 检测到的账单类型: alipay/wechat/jd
|
||||
Output string // 脚本输出信息
|
||||
BillType string // 检测到的账单类型: alipay/wechat/jd
|
||||
Output string // 脚本输出信息
|
||||
UnresolvedRefunds []UnresolvedRefund // 本次清洗未在同批次内匹配到对应支出的退款
|
||||
}
|
||||
|
||||
// UnresolvedRefund 本次清洗未在同批次内匹配到对应支出的退款
|
||||
type UnresolvedRefund struct {
|
||||
OrderNo string // 原订单号(去除退款后缀)
|
||||
MerchantOrderNo string // 商家订单号(备用匹配字段)
|
||||
RefundOrderNo string // 退款行自身的完整订单号(用于备注追溯)
|
||||
Amount float64 // 退款金额
|
||||
Time string // 退款时间
|
||||
Merchant string // 交易对方
|
||||
Description string // 商品说明
|
||||
}
|
||||
|
||||
// ConvertResult 格式转换结果
|
||||
|
||||
@@ -29,10 +29,22 @@ type CleanRequest struct {
|
||||
|
||||
// CleanResponse HTTP 清洗响应
|
||||
type CleanResponse struct {
|
||||
Success bool `json:"success"`
|
||||
BillType string `json:"bill_type"`
|
||||
Message string `json:"message"`
|
||||
OutputPath string `json:"output_path,omitempty"`
|
||||
Success bool `json:"success"`
|
||||
BillType string `json:"bill_type"`
|
||||
Message string `json:"message"`
|
||||
OutputPath string `json:"output_path,omitempty"`
|
||||
UnresolvedRefunds []UnresolvedRefund `json:"unresolved_refunds,omitempty"`
|
||||
}
|
||||
|
||||
// UnresolvedRefund 本次清洗未在同批次内匹配到对应支出的退款(与 Python 端 dict 字段对应)
|
||||
type UnresolvedRefund struct {
|
||||
OrderNo string `json:"order_no"`
|
||||
MerchantOrderNo string `json:"merchant_order_no"`
|
||||
RefundOrderNo string `json:"refund_order_no"`
|
||||
Amount float64 `json:"amount"`
|
||||
Time string `json:"time"`
|
||||
Merchant string `json:"merchant"`
|
||||
Description string `json:"description"`
|
||||
}
|
||||
|
||||
// ErrorResponse 错误响应
|
||||
@@ -149,9 +161,23 @@ func (c *Cleaner) Clean(inputPath, outputPath string, opts *adapter.CleanOptions
|
||||
}
|
||||
}
|
||||
|
||||
unresolvedRefunds := make([]adapter.UnresolvedRefund, 0, len(cleanResp.UnresolvedRefunds))
|
||||
for _, ur := range cleanResp.UnresolvedRefunds {
|
||||
unresolvedRefunds = append(unresolvedRefunds, adapter.UnresolvedRefund{
|
||||
OrderNo: ur.OrderNo,
|
||||
MerchantOrderNo: ur.MerchantOrderNo,
|
||||
RefundOrderNo: ur.RefundOrderNo,
|
||||
Amount: ur.Amount,
|
||||
Time: ur.Time,
|
||||
Merchant: ur.Merchant,
|
||||
Description: ur.Description,
|
||||
})
|
||||
}
|
||||
|
||||
return &adapter.CleanResult{
|
||||
BillType: cleanResp.BillType,
|
||||
Output: cleanResp.Message,
|
||||
BillType: cleanResp.BillType,
|
||||
Output: cleanResp.Message,
|
||||
UnresolvedRefunds: unresolvedRefunds,
|
||||
}, nil
|
||||
}
|
||||
|
||||
|
||||
@@ -98,7 +98,11 @@ func Login(c *gin.Context) {
|
||||
|
||||
secret := config.Global.JWTSecret
|
||||
if secret == "" {
|
||||
secret = "billai-default-secret"
|
||||
c.JSON(http.StatusInternalServerError, gin.H{
|
||||
"success": false,
|
||||
"error": "服务器 JWT 配置缺失",
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
|
||||
@@ -144,7 +148,12 @@ func ValidateToken(c *gin.Context) {
|
||||
|
||||
secret := config.Global.JWTSecret
|
||||
if secret == "" {
|
||||
secret = "billai-default-secret"
|
||||
c.JSON(http.StatusUnauthorized, gin.H{
|
||||
"success": false,
|
||||
"error": "服务器 JWT 配置缺失",
|
||||
"code": "TOKEN_INVALID",
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
token, err := jwt.ParseWithClaims(tokenString, &Claims{}, func(token *jwt.Token) (interface{}, error) {
|
||||
|
||||
@@ -2,7 +2,6 @@ package handler
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
@@ -154,18 +153,6 @@ func ListBills(c *gin.Context) {
|
||||
})
|
||||
}
|
||||
|
||||
// parsePageParam 解析分页参数
|
||||
func parsePageParam(s string, defaultVal int) int {
|
||||
if s == "" {
|
||||
return defaultVal
|
||||
}
|
||||
val, err := strconv.Atoi(s)
|
||||
if err != nil || val < 1 {
|
||||
return defaultVal
|
||||
}
|
||||
return val
|
||||
}
|
||||
|
||||
// MonthlyStatsResponse 月度统计响应
|
||||
type MonthlyStatsResponse struct {
|
||||
Result bool `json:"result"`
|
||||
@@ -202,6 +189,13 @@ func MonthlyStats(c *gin.Context) {
|
||||
// ReviewStats 获取待复核数据统计
|
||||
func ReviewStats(c *gin.Context) {
|
||||
repo := repository.GetRepository()
|
||||
if repo == nil {
|
||||
c.JSON(http.StatusServiceUnavailable, model.ReviewResponse{
|
||||
Result: false,
|
||||
Message: "数据库未连接",
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
// 从MongoDB查询所有需要复核的账单
|
||||
bills, err := repo.GetBillsNeedReview()
|
||||
|
||||
@@ -223,7 +223,7 @@ func Upload(c *gin.Context) {
|
||||
End: req.End,
|
||||
Format: req.Format,
|
||||
}
|
||||
_, cleanErr := service.RunCleanScript(processFilePath, outputPath, cleanOpts)
|
||||
cleanResult, cleanErr := service.RunCleanScript(processFilePath, outputPath, cleanOpts)
|
||||
if cleanErr != nil {
|
||||
service.CleanupExtractedFiles(extractedFiles)
|
||||
c.JSON(http.StatusInternalServerError, model.UploadResponse{
|
||||
@@ -255,22 +255,56 @@ func Upload(c *gin.Context) {
|
||||
}
|
||||
service.CleanupExtractedFiles(extractedFiles)
|
||||
|
||||
repo := repository.GetRepository()
|
||||
|
||||
// 13. 如果是京东账单,软删除其他来源中包含"京东-订单编号"的记录
|
||||
var jdRelatedDeleted int64
|
||||
if billType == "jd" {
|
||||
repo := repository.GetRepository()
|
||||
if repo != nil {
|
||||
deleted, err := repo.SoftDeleteJDRelatedBills()
|
||||
if err != nil {
|
||||
fmt.Printf("⚠️ 软删除京东关联记录失败: %v\n", err)
|
||||
} else if deleted > 0 {
|
||||
jdRelatedDeleted = deleted
|
||||
fmt.Printf("🗑️ 已软删除 %d 条其他来源中的京东关联记录\n", deleted)
|
||||
if billType == "jd" && repo != nil {
|
||||
deleted, err := repo.SoftDeleteJDRelatedBills()
|
||||
if err != nil {
|
||||
fmt.Printf("⚠️ 软删除京东关联记录失败: %v\n", err)
|
||||
} else if deleted > 0 {
|
||||
jdRelatedDeleted = deleted
|
||||
fmt.Printf("🗑️ 已软删除 %d 条其他来源中的京东关联记录\n", deleted)
|
||||
}
|
||||
}
|
||||
|
||||
// 14. 核销跨批次退款(支付宝:本次清洗中未在同批次内匹配到对应支出的退款)
|
||||
var reconciledCount int
|
||||
if repo != nil && cleanResult != nil {
|
||||
for _, ur := range cleanResult.UnresolvedRefunds {
|
||||
matched, rErr := repo.ReconcileRefund(billType, ur.OrderNo, ur.MerchantOrderNo, ur.Amount, ur.Time, ur.Merchant, ur.Description, ur.RefundOrderNo)
|
||||
if rErr != nil {
|
||||
fmt.Printf("⚠️ 退款核销失败: %v\n", rErr)
|
||||
continue
|
||||
}
|
||||
if matched {
|
||||
reconciledCount++
|
||||
fmt.Printf("💰 已核销退款: 订单%s, 金额%.2f元\n", ur.OrderNo, ur.Amount)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 14. 返回成功响应
|
||||
// 15. 核销跨批次微信退款(重复上传的行中携带的已退款状态)
|
||||
if repo != nil && len(dedupResult.WechatRefundUpdates) > 0 {
|
||||
for _, wu := range dedupResult.WechatRefundUpdates {
|
||||
matched, rErr := repo.ReconcileWechatRefund(wu.TransactionID, wu.FullRefund, wu.CumulativeRefundAmount)
|
||||
if rErr != nil {
|
||||
fmt.Printf("⚠️ 微信退款核销失败: %v\n", rErr)
|
||||
continue
|
||||
}
|
||||
if matched {
|
||||
reconciledCount++
|
||||
if wu.FullRefund {
|
||||
fmt.Printf("💰 已核销微信全额退款: 交易单号%s\n", wu.TransactionID)
|
||||
} else {
|
||||
fmt.Printf("💰 已核销微信部分退款: 交易单号%s, 累计退款%.2f元\n", wu.TransactionID, wu.CumulativeRefundAmount)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 16. 返回成功响应
|
||||
message := fmt.Sprintf("处理成功,新增 %d 条记录", cleanedCount)
|
||||
if dedupResult.DuplicateCount > 0 {
|
||||
message = fmt.Sprintf("处理成功,新增 %d 条,跳过 %d 条重复记录", cleanedCount, dedupResult.DuplicateCount)
|
||||
@@ -278,18 +312,22 @@ func Upload(c *gin.Context) {
|
||||
if jdRelatedDeleted > 0 {
|
||||
message = fmt.Sprintf("%s,标记删除 %d 条重复的京东订单", message, jdRelatedDeleted)
|
||||
}
|
||||
if reconciledCount > 0 {
|
||||
message = fmt.Sprintf("%s,核销退款 %d 条", message, reconciledCount)
|
||||
}
|
||||
|
||||
c.JSON(http.StatusOK, model.UploadResponse{
|
||||
Result: true,
|
||||
Message: message,
|
||||
Data: &model.UploadData{
|
||||
BillType: billType,
|
||||
FileURL: fmt.Sprintf("/download/%s", outputFileName),
|
||||
FileName: outputFileName,
|
||||
RawCount: rawCount,
|
||||
CleanedCount: cleanedCount,
|
||||
DuplicateCount: dedupResult.DuplicateCount,
|
||||
JDRelatedDeleted: jdRelatedDeleted,
|
||||
BillType: billType,
|
||||
FileURL: fmt.Sprintf("/download/%s", outputFileName),
|
||||
FileName: outputFileName,
|
||||
RawCount: rawCount,
|
||||
CleanedCount: cleanedCount,
|
||||
DuplicateCount: dedupResult.DuplicateCount,
|
||||
JDRelatedDeleted: jdRelatedDeleted,
|
||||
ReconciledRefundCount: reconciledCount,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
@@ -38,7 +38,13 @@ func AuthRequired() gin.HandlerFunc {
|
||||
|
||||
secret := config.Global.JWTSecret
|
||||
if secret == "" {
|
||||
secret = "billai-default-secret"
|
||||
c.JSON(http.StatusUnauthorized, gin.H{
|
||||
"success": false,
|
||||
"error": "服务器 JWT 配置缺失",
|
||||
"code": "TOKEN_INVALID",
|
||||
})
|
||||
c.Abort()
|
||||
return
|
||||
}
|
||||
|
||||
token, err := jwt.ParseWithClaims(tokenString, &Claims{}, func(token *jwt.Token) (interface{}, error) {
|
||||
|
||||
@@ -88,9 +88,10 @@ type CleanedBill struct {
|
||||
Category string `bson:"category" json:"category"` // 交易分类
|
||||
Merchant string `bson:"merchant" json:"merchant"` // 交易对方
|
||||
Description string `bson:"description" json:"description"` // 商品说明
|
||||
IncomeExpense string `bson:"income_expense" json:"income_expense"` // 收/支
|
||||
Amount float64 `bson:"amount" json:"amount"` // 金额
|
||||
PayMethod string `bson:"pay_method" json:"pay_method"` // 支付方式
|
||||
IncomeExpense string `bson:"income_expense" json:"income_expense"` // 收/支
|
||||
Amount float64 `bson:"amount" json:"amount"` // 金额
|
||||
OriginalAmount float64 `bson:"original_amount,omitempty" json:"original_amount,omitempty"` // 原始金额(入库时),用于微信跨批次退款核销
|
||||
PayMethod string `bson:"pay_method" json:"pay_method"` // 支付方式
|
||||
Status string `bson:"status" json:"status"` // 交易状态
|
||||
Remark string `bson:"remark" json:"remark"` // 备注
|
||||
ReviewLevel string `bson:"review_level" json:"review_level"` // 复核等级: HIGH/LOW/空
|
||||
|
||||
@@ -2,13 +2,14 @@ package model
|
||||
|
||||
// UploadData 上传响应数据
|
||||
type UploadData struct {
|
||||
BillType string `json:"bill_type,omitempty"` // alipay/wechat/jd
|
||||
FileURL string `json:"file_url,omitempty"` // 下载链接
|
||||
FileName string `json:"file_name,omitempty"` // 文件名
|
||||
RawCount int `json:"raw_count,omitempty"` // 存储到原始数据集合的记录数
|
||||
CleanedCount int `json:"cleaned_count,omitempty"` // 存储到清洗后数据集合的记录数
|
||||
DuplicateCount int `json:"duplicate_count,omitempty"` // 重复跳过的记录数
|
||||
JDRelatedDeleted int64 `json:"jd_related_deleted,omitempty"` // 软删除的京东关联记录数(其他来源中描述包含京东订单号的记录)
|
||||
BillType string `json:"bill_type,omitempty"` // alipay/wechat/jd
|
||||
FileURL string `json:"file_url,omitempty"` // 下载链接
|
||||
FileName string `json:"file_name,omitempty"` // 文件名
|
||||
RawCount int `json:"raw_count,omitempty"` // 存储到原始数据集合的记录数
|
||||
CleanedCount int `json:"cleaned_count,omitempty"` // 存储到清洗后数据集合的记录数
|
||||
DuplicateCount int `json:"duplicate_count,omitempty"` // 重复跳过的记录数
|
||||
JDRelatedDeleted int64 `json:"jd_related_deleted,omitempty"` // 软删除的京东关联记录数(其他来源中描述包含京东订单号的记录)
|
||||
ReconciledRefundCount int `json:"reconciled_refund_count,omitempty"` // 跨批次核销的退款记录数
|
||||
}
|
||||
|
||||
// UploadResponse 上传响应
|
||||
|
||||
@@ -4,6 +4,7 @@ package mongo
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"math"
|
||||
"time"
|
||||
|
||||
"go.mongodb.org/mongo-driver/bson"
|
||||
@@ -16,6 +17,9 @@ import (
|
||||
"billai-server/repository"
|
||||
)
|
||||
|
||||
// refundEpsilon 退款核销后剩余金额的容差阈值,小于该值视为已全额退款
|
||||
const refundEpsilon = 0.005
|
||||
|
||||
// Repository MongoDB 账单存储实现
|
||||
type Repository struct {
|
||||
client *mongo.Client
|
||||
@@ -498,6 +502,149 @@ func (r *Repository) SoftDeleteJDRelatedBills() (int64, error) {
|
||||
return result.ModifiedCount, nil
|
||||
}
|
||||
|
||||
// ReconcileRefund 将跨批次退款核销到已存储的清洗后账单
|
||||
// 按 bill_type + (transaction_id == orderNo 或 merchant_order_no == merchantOrderNo) 查找未删除记录
|
||||
// 全额退款(剩余金额 <= refundEpsilon)则软删除;部分退款则扣减 amount 并追加备注
|
||||
func (r *Repository) ReconcileRefund(billType, orderNo, merchantOrderNo string, refundAmount float64, refundTime, merchant, description, refundOrderNo string) (bool, error) {
|
||||
if r.cleanedCollection == nil {
|
||||
return false, fmt.Errorf("cleaned collection not initialized")
|
||||
}
|
||||
|
||||
if orderNo == "" && merchantOrderNo == "" {
|
||||
return false, nil
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
var or []bson.M
|
||||
if orderNo != "" {
|
||||
or = append(or, bson.M{"transaction_id": orderNo})
|
||||
}
|
||||
if merchantOrderNo != "" {
|
||||
or = append(or, bson.M{"merchant_order_no": merchantOrderNo})
|
||||
}
|
||||
filter := bson.M{
|
||||
"bill_type": billType,
|
||||
"is_deleted": bson.M{"$ne": true},
|
||||
"$or": or,
|
||||
}
|
||||
|
||||
var bill model.CleanedBill
|
||||
if err := r.cleanedCollection.FindOne(ctx, filter).Decode(&bill); err != nil {
|
||||
if err == mongo.ErrNoDocuments {
|
||||
return false, nil
|
||||
}
|
||||
return false, fmt.Errorf("查询待核销账单失败: %w", err)
|
||||
}
|
||||
|
||||
remaining := bill.Amount - refundAmount
|
||||
now := time.Now()
|
||||
|
||||
if remaining <= refundEpsilon {
|
||||
update := bson.M{"$set": bson.M{
|
||||
"is_deleted": true,
|
||||
"updated_at": now,
|
||||
"remark": fmt.Sprintf("[退款核销]全额退款%.2f元(退款单号%s);%s", refundAmount, refundOrderNo, bill.Remark),
|
||||
}}
|
||||
_, err := r.cleanedCollection.UpdateOne(ctx, bson.M{"_id": bill.ID}, update)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("核销退款失败: %w", err)
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
|
||||
remaining = math.Round(remaining*100) / 100
|
||||
update := bson.M{"$set": bson.M{
|
||||
"amount": remaining,
|
||||
"updated_at": now,
|
||||
"remark": fmt.Sprintf("原金额%.2f元,退款%.2f元(退款单号%s);%s", bill.Amount, refundAmount, refundOrderNo, bill.Remark),
|
||||
}}
|
||||
_, err := r.cleanedCollection.UpdateOne(ctx, bson.M{"_id": bill.ID}, update)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("核销退款失败: %w", err)
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
|
||||
// ReconcileWechatRefund 将跨批次微信退款核销到已存储的清洗后账单
|
||||
// 微信"当前状态"字段为累计退款金额,使用 original_amount(入库时原始金额)计算剩余
|
||||
func (r *Repository) ReconcileWechatRefund(transactionID string, fullRefund bool, cumulativeRefundAmount float64) (bool, error) {
|
||||
if r.cleanedCollection == nil {
|
||||
return false, fmt.Errorf("cleaned collection not initialized")
|
||||
}
|
||||
if transactionID == "" {
|
||||
return false, nil
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
filter := bson.M{
|
||||
"bill_type": "wechat",
|
||||
"transaction_id": transactionID,
|
||||
"is_deleted": bson.M{"$ne": true},
|
||||
}
|
||||
|
||||
var bill model.CleanedBill
|
||||
if err := r.cleanedCollection.FindOne(ctx, filter).Decode(&bill); err != nil {
|
||||
if err == mongo.ErrNoDocuments {
|
||||
return false, nil
|
||||
}
|
||||
return false, fmt.Errorf("查询待核销微信账单失败: %w", err)
|
||||
}
|
||||
|
||||
now := time.Now()
|
||||
|
||||
if fullRefund {
|
||||
update := bson.M{"$set": bson.M{
|
||||
"is_deleted": true,
|
||||
"updated_at": now,
|
||||
"remark": fmt.Sprintf("[退款核销]全额退款;%s", bill.Remark),
|
||||
}}
|
||||
_, err := r.cleanedCollection.UpdateOne(ctx, bson.M{"_id": bill.ID}, update)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("核销微信全额退款失败: %w", err)
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
|
||||
// 部分退款:用原始金额(original_amount)减去累计退款金额得到剩余
|
||||
originalAmount := bill.OriginalAmount
|
||||
if originalAmount <= 0 {
|
||||
originalAmount = bill.Amount // 兼容旧记录(无 original_amount 字段)
|
||||
}
|
||||
remaining := originalAmount - cumulativeRefundAmount
|
||||
|
||||
if remaining <= refundEpsilon {
|
||||
update := bson.M{"$set": bson.M{
|
||||
"is_deleted": true,
|
||||
"updated_at": now,
|
||||
"remark": fmt.Sprintf("[退款核销]全额退款%.2f元(原金额%.2f元);%s", cumulativeRefundAmount, originalAmount, bill.Remark),
|
||||
}}
|
||||
_, err := r.cleanedCollection.UpdateOne(ctx, bson.M{"_id": bill.ID}, update)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("核销微信退款失败: %w", err)
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
|
||||
remaining = math.Round(remaining*100) / 100
|
||||
update := bson.M{"$set": bson.M{
|
||||
"amount": remaining,
|
||||
"updated_at": now,
|
||||
"remark": fmt.Sprintf("原金额%.2f元,已退款%.2f元;%s", originalAmount, cumulativeRefundAmount, bill.Remark),
|
||||
}}
|
||||
_, err := r.cleanedCollection.UpdateOne(ctx, bson.M{"_id": bill.ID}, update)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("核销微信退款失败: %w", err)
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
|
||||
// 建议: 为提升查询性能,可为 bills_cleaned 添加索引
|
||||
// {transaction_id:1, bill_type:1} 和 {merchant_order_no:1, bill_type:1}
|
||||
|
||||
// GetClient 获取 MongoDB 客户端(用于兼容旧代码)
|
||||
func (r *Repository) GetClient() *mongo.Client {
|
||||
return r.client
|
||||
|
||||
@@ -56,4 +56,16 @@ type BillRepository interface {
|
||||
// 用于避免京东账单与其他来源(微信、支付宝)账单重复计算
|
||||
// 返回: 删除数量、错误
|
||||
SoftDeleteJDRelatedBills() (int64, error)
|
||||
|
||||
// ReconcileRefund 将跨批次退款核销到已存储的清洗后账单
|
||||
// 按 bill_type + (transaction_id == orderNo 或 merchant_order_no == merchantOrderNo) 查找未删除记录
|
||||
// 全额退款(剩余金额 <= 0.005)则软删除;部分退款则扣减 amount 并追加备注
|
||||
// 返回: 是否找到并核销了匹配记录、错误(未找到匹配记录不算错误,返回 matched=false)
|
||||
ReconcileRefund(billType, orderNo, merchantOrderNo string, refundAmount float64, refundTime, merchant, description, refundOrderNo string) (matched bool, err error)
|
||||
|
||||
// ReconcileWechatRefund 将跨批次微信退款核销到已存储的清洗后账单
|
||||
// 微信退款通过重复上传时"当前状态"字段携带退款信息来触发
|
||||
// fullRefund=true 时软删除原记录;否则用 original_amount - cumulativeRefundAmount 计算剩余金额
|
||||
// 返回: 是否找到并核销了匹配记录、错误
|
||||
ReconcileWechatRefund(transactionID string, fullRefund bool, cumulativeRefundAmount float64) (matched bool, err error)
|
||||
}
|
||||
|
||||
@@ -23,13 +23,21 @@ func getRepo() repository.BillRepository {
|
||||
return repository.GetRepository()
|
||||
}
|
||||
|
||||
// WechatRefundUpdate 微信重复行中携带的退款信息(用于跨批次退款核销)
|
||||
type WechatRefundUpdate struct {
|
||||
TransactionID string // 原消费行的交易单号
|
||||
FullRefund bool // 是否全额退款(已全额退款)
|
||||
CumulativeRefundAmount float64 // 累计退款金额(已退款(¥X)中的 X,与原始金额相减得剩余)
|
||||
}
|
||||
|
||||
// DeduplicateResult 去重结果
|
||||
type DeduplicateResult struct {
|
||||
OriginalCount int // 原始记录数
|
||||
DuplicateCount int // 重复记录数
|
||||
NewCount int // 新记录数
|
||||
DedupFilePath string // 去重后的文件路径(如果有去重则生成新文件)
|
||||
BillType string // 检测到的账单类型
|
||||
OriginalCount int // 原始记录数
|
||||
DuplicateCount int // 重复记录数
|
||||
NewCount int // 新记录数
|
||||
DedupFilePath string // 去重后的文件路径(如果有去重则生成新文件)
|
||||
BillType string // 检测到的账单类型
|
||||
WechatRefundUpdates []WechatRefundUpdate // 微信重复行中检测到的退款状态(用于跨批次核销)
|
||||
}
|
||||
|
||||
// DeduplicateRawFile 对原始文件进行去重检查,返回去重后的文件路径
|
||||
@@ -75,6 +83,17 @@ func DeduplicateRawFile(filePath, uploadBatch string) (*DeduplicateResult, error
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// 对于微信账单,找到"当前状态"列的索引,用于检测退款状态
|
||||
wechatStatusIdx := -1
|
||||
if billType == "wechat" {
|
||||
for i, col := range header {
|
||||
if col == "当前状态" {
|
||||
wechatStatusIdx = i
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 检查每行是否重复
|
||||
var newRows [][]string
|
||||
for _, row := range dataRows {
|
||||
@@ -101,6 +120,24 @@ func DeduplicateRawFile(filePath, uploadBatch string) (*DeduplicateResult, error
|
||||
newRows = append(newRows, row)
|
||||
} else {
|
||||
result.DuplicateCount++
|
||||
// 微信账单:检查重复行是否携带退款状态(跨批次退款核销)
|
||||
if wechatStatusIdx >= 0 && len(row) > wechatStatusIdx {
|
||||
status := strings.TrimSpace(row[wechatStatusIdx])
|
||||
if strings.Contains(status, "已全额退款") {
|
||||
result.WechatRefundUpdates = append(result.WechatRefundUpdates, WechatRefundUpdate{
|
||||
TransactionID: transactionID,
|
||||
FullRefund: true,
|
||||
})
|
||||
} else if strings.Contains(status, "已退款") {
|
||||
if amount := extractWechatRefundAmount(status); amount > 0 {
|
||||
result.WechatRefundUpdates = append(result.WechatRefundUpdates, WechatRefundUpdate{
|
||||
TransactionID: transactionID,
|
||||
FullRefund: false,
|
||||
CumulativeRefundAmount: amount,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -337,6 +374,7 @@ func saveCleanedBillsFromCSV(filePath, billType, sourceFile, uploadBatch string)
|
||||
bill.ReviewLevel = row[idx]
|
||||
}
|
||||
|
||||
bill.OriginalAmount = bill.Amount
|
||||
bills = append(bills, bill)
|
||||
}
|
||||
|
||||
@@ -431,6 +469,7 @@ func saveCleanedBillsFromJSON(filePath, billType, sourceFile, uploadBatch string
|
||||
bill.ReviewLevel = v
|
||||
}
|
||||
|
||||
bill.OriginalAmount = bill.Amount
|
||||
bills = append(bills, bill)
|
||||
}
|
||||
|
||||
@@ -512,3 +551,28 @@ func parseAmount(s string) float64 {
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// extractWechatRefundAmount 从微信"当前状态"字段中提取累计退款金额
|
||||
// 支持格式: "已退款(¥3.00)"、"已退款(¥3.00)"、"已退款¥3.00"
|
||||
func extractWechatRefundAmount(status string) float64 {
|
||||
start := strings.Index(status, "(")
|
||||
end := strings.LastIndex(status, ")")
|
||||
var inner string
|
||||
if start >= 0 && end > start {
|
||||
inner = status[start+1 : end]
|
||||
} else {
|
||||
idx := strings.Index(status, "已退款")
|
||||
if idx < 0 {
|
||||
return 0
|
||||
}
|
||||
inner = status[idx+len("已退款"):]
|
||||
}
|
||||
inner = strings.TrimPrefix(inner, "¥")
|
||||
inner = strings.TrimPrefix(inner, "¥")
|
||||
inner = strings.TrimSpace(inner)
|
||||
v, err := strconv.ParseFloat(inner, 64)
|
||||
if err != nil {
|
||||
return 0
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
@@ -263,7 +263,7 @@
|
||||
{:else}
|
||||
<div>
|
||||
<div class="text-center mb-6">
|
||||
<div class="text-3xl font-bold font-mono {record.incomeExpense === '收入' ? 'text-green-600 dark:text-green-400' : 'text-red-600 dark:text-red-400'}">
|
||||
<div class="text-3xl font-bold font-mono {record.incomeExpense === '收入' ? 'text-green-600 dark:text-green-400' : record.incomeExpense === '不计收支' ? 'text-muted-foreground' : 'text-red-600 dark:text-red-400'}">
|
||||
¥{record.amount.toFixed(2)}
|
||||
</div>
|
||||
<div class="text-sm text-muted-foreground mt-1">{record.incomeExpense || '支出'}金额</div>
|
||||
|
||||
@@ -199,6 +199,20 @@
|
||||
dayData.set(category, (dayData.get(category) || 0) + amount);
|
||||
});
|
||||
|
||||
// 填充缺失的日期(零支出日期)
|
||||
const now = new Date();
|
||||
const allDates: string[] = [];
|
||||
const cursor = new Date(cutoffDate);
|
||||
while (cursor <= now) {
|
||||
allDates.push(formatLocalDate(cursor));
|
||||
cursor.setDate(cursor.getDate() + 1);
|
||||
}
|
||||
allDates.forEach(dateStr => {
|
||||
if (!dailyMap.has(dateStr)) {
|
||||
dailyMap.set(dateStr, new Map());
|
||||
}
|
||||
});
|
||||
|
||||
const dayCount = dailyMap.size;
|
||||
|
||||
// 根据天数决定聚合粒度
|
||||
|
||||
Reference in New Issue
Block a user