feat: implement cross-batch Alipay refund reconciliation

When a refund row in an uploaded Alipay bill has no matching expense
row in the same batch (because the original purchase was uploaded in a
prior batch), the refund is now reconciled against the stored record in
bills_cleaned rather than being silently discarded.

Changes:
- analyzer/cleaners/base.py: add unresolved_refunds list to BaseCleaner
- analyzer/cleaners/alipay.py: _aggregate_refunds stores full refund
  metadata (dict); _process_expenses tracks matched keys and populates
  self.unresolved_refunds for unmatched refunds
- analyzer/server.py: thread unresolved_refunds through do_clean,
  CleanResponse, and both /clean endpoints
- server/adapter/adapter.go: add UnresolvedRefund type and field to CleanResult
- server/adapter/http/cleaner.go: deserialize unresolved_refunds from
  Python response and populate CleanResult
- server/repository/repository.go: add ReconcileRefund to BillRepository interface
- server/repository/mongo/repository.go: implement ReconcileRefund —
  full refund soft-deletes the bill, partial refund reduces amount and
  appends remark with original amount and refund order number
- server/handler/upload.go: capture clean result and call ReconcileRefund
  for each unresolved refund after saving cleaned bills
- server/model/response.go: add ReconciledRefundCount to UploadData

Also: add CLAUDE.md (@AGENTS.md), update AGENTS.md, fix DailyTrendChart
missing-date gap by filling zero-expense dates in daily map.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-06-16 19:29:47 +08:00
parent bb717faac3
commit e2e1beb6f7
12 changed files with 342 additions and 133 deletions

152
AGENTS.md
View File

@@ -2,85 +2,81 @@
Guidelines for AI coding agents working on BillAI - a microservices bill analysis system.
**Current Version:** 1.4.0 | Go: 1.24.0 | Node: 20+ | Python: 3.12+
**Version:** See `CHANGELOG.md` for current version. Latest tag usually matches.
## Architecture
- `web/` - SvelteKit 5 + TailwindCSS 4 + TypeScript (Frontend, port 3000)
- `server/` - Go 1.24 + Gin + MongoDB (API, port 8080)
- `analyzer/` - Python 3.12 + FastAPI (Data cleaning, port 8001)
SvelteKit proxies `/api/*` requests to Go backend via `web/src/routes/api/[...path]/+server.ts`.
**Proxy Caveat:** SvelteKit proxies `/api/*` to Go via `web/src/routes/api/[...path]/+server.ts`, but **only GET and POST are forwarded**. The Go backend intentionally uses `POST` for mutations (update, delete, manual create) to work around this. If you add PUT/PATCH/DELETE endpoints, you must also add them to the proxy.
## Build/Lint/Test Commands
### Frontend (web/)
### Frontend (`web/`)
```bash
npm run dev # Start dev server
npm run build # Production build
npm run check # TypeScript check
npm run lint # Prettier + ESLint
npm run format # Format with Prettier
npm run test # Run all tests (CI mode)
npm run check # TypeScript + Svelte check
npm run lint # Prettier --check + ESLint
npm run format # Prettier --write
npm run test # vitest --run (CI mode)
npx vitest run src/xxx.spec.ts # Run single test file
npx vitest run -t "pattern" # Run by name pattern
```
### Backend (server/)
**Note:** `web/` has a `yarn.lock` but scripts in `package.json` use `npm run`.
### Backend (`server/`)
```bash
go run . # Start server
go build -o server . # Build binary
go run . # Start server (reads server/config.yaml)
go build -o server .
go test ./... # Run all tests
go test ./handler/... # Run handler tests
go test -run TestName ./... # Run single test
go test -v ./handler/... # Verbose output
```
### Analyzer (analyzer/)
### Analyzer (`analyzer/`)
```bash
python server.py # Start FastAPI
uvicorn server:app --reload # Hot reload
python server.py # Start FastAPI (has `if __name__ == "__main__"`)
uvicorn server:app --reload # Requires cwd == analyzer/
pytest # Run all tests
pytest test_jd_cleaner.py # Single test file
pytest -k "pattern" # Run by pattern
```
### Docker
```bash
docker-compose up -d --build # Start/rebuild all services
docker-compose logs -f server # Follow logs
docker-compose down # Stop services
docker compose up -d --build --remove-orphans # Start/rebuild all
docker compose logs -f server # Follow logs
docker compose down # Stop services
```
## Code Style
## Code Style & Conventions
### General
- **Comments:** Chinese common for business logic; English for technical.
- **Conventions:** Follow existing patterns. Check `package.json`/`go.mod`/`requirements.txt` before adding dependencies.
### TypeScript/Svelte (`web/`)
- **Formatting:** Prettier (tabs, single quotes, `trailingComma: none`, printWidth 100)
- **Imports:** Use `$lib` alias. No relative imports for lib modules.
- **Svelte 5:** Runes (`$state`, `$derived`, `$effect`, `$props`). Events: `onclick={fn}`.
- **Types:** `export interface` for models. Frontend uses `camelCase`, API uses `snake_case`. Converters live in `$lib/models/bill.ts`.
- **Auth:** Token stored in `localStorage` key `auth`. Always use `apiFetch()` from `$lib/api.ts` for authenticated requests.
### TypeScript/Svelte (web/)
- **Formatting:** Prettier (tabs, single quotes, printWidth 100)
- **Naming:** `PascalCase` for types/components, `camelCase` for variables
- **Imports:** Use `$lib` alias, `$app/*` for SvelteKit builtins. No relative paths for lib modules.
- **Svelte 5:** Use runes (`$state`, `$derived`, `$effect`, `$props`). Event: `onclick={fn}`.
- **Types:** `export interface` for models. Frontend `camelCase`, API `snake_case`. Converters in `$lib/models/bill.ts`.
- **Error Handling:** Check `response.ok`, throw `Error(\`HTTP ${status}\`)`. On 401: `auth.logout()` + redirect.
- **Auth:** `createAuthStore()` in `$lib/stores/auth.ts`. Token in `localStorage` key `auth`. Use `apiFetch()` in `$lib/api.ts`.
### Go Backend (server/)
### Go (`server/`)
- **Module:** `billai-server` (import path). Use this in `go test` / `go build` when outside the directory.
- **Layer:** `handler``service``adapter`/`repository``model`. No business logic in handlers.
- **Struct tags:** JSON `snake_case`, `omitempty` optional. Pointer for optional patch fields. Sensitive: `json:"-"`.
- **Error handling:** 500 for DB errors, 400 for bad requests, 404 not found. Wrap with `fmt.Errorf("context: %w", err)`.
- **Response:** `Result bool`, `Message`, `Data *T`. Auth: `success bool`, `error`, `data`.
- **Time:** Use custom `LocalTime` type (serializes as `2006-01-02 15:04:05`).
- **Soft delete:** Never hard-delete. Filter `is_deleted: false` in queries.
- **Struct tags:** JSON `snake_case`, `omitempty` for optional. Pointer types for optional patch fields. Sensitive fields: `json:"-"`.
- **Response shapes:**
- Business APIs: `result bool`, `message string`, `data *T`
- Auth APIs: `success bool`, `error string`, `data *T` (and `code` for error types)
- **Time:** Custom `LocalTime` type serializes as `"2006-01-02 15:04:05"`.
- **Soft delete:** Never hard-delete. All queries filter `is_deleted: false`.
### Python Analyzer (analyzer/)
- **Style:** PEP 8. `snake_case` variables, `UPPER_CASE` constants. Prefix private globals with `_`.
- **Type hints:** Mandatory. Use `Optional[str]` or `str | None`.
### Python (`analyzer/`)
- **Style:** PEP 8. `snake_case` vars, `UPPER_CASE` constants. Prefix private globals with `_`.
- **Type hints:** Mandatory. Prefer `str | None` or `Optional[str]`.
- **Models:** `pydantic.BaseModel` for API schemas.
- **Cleaners:** Extend `BaseCleaner(ABC)` from `cleaners/base.py`. Category rules in `config/category.yaml`.
## Key Patterns
## Key Patterns & Quirks
### API Flow
```
@@ -89,35 +85,63 @@ Browser → SvelteKit proxy → Go (Gin) → handler → service → adapter →
```
### Authentication
- JWT (HS256). Token in `localStorage` key `auth`. Header: `Authorization: Bearer <token>`.
- `middleware.AuthRequired()` wraps `/api/*` (except `/api/auth/*`).
- 401 anywhere`auth.logout()` + redirect `/login`.
- JWT (HS256). Header: `Authorization: Bearer <token>`.
- `middleware.AuthRequired()` guards authed routes. Public routes: `/api/auth/*`, `/api/changelog`, `/health`.
- Frontend `apiFetch()` intercepts 401`auth.logout()` + redirect `/login`.
### File Processing
Upload: ZIP/XLSX → Extract → Convert UTF-8 CSV → Detect bill type → Deduplicate → Clean → Save to MongoDB.
### File Processing Pipeline
Upload: ZIP/XLSX → Extract → Convert UTF-8 CSV → Detect bill type (alipay/wechat/jd) → Deduplicate against MongoDB → Clean via Python → Save cleaned data.
### Adapter (Go ↔ Python)
`adapter.Cleaner` interface: HTTP (`adapter/http`, default) or subprocess (`adapter/python`). Set via `ANALYZER_MODE` env var.
`adapter.Cleaner` interface. Two modes:
- `http` (default): calls FastAPI at `ANALYZER_URL`
- `subprocess`: spawns `python analyzer/clean_bill.py`
Set via `ANALYZER_MODE` env var or `server/config.yaml` `analyzer.mode`.
### Config Precedence
Go backend reads `server/config.yaml`, but Docker compose sets env vars (`ANALYZER_URL`, `MONGO_URI`, `JWT_SECRET`, etc.) that override it.
### SvelteKit Config Notes
- `svelte.config.js` uses `adapter-node` for Docker SSR.
- `csrf.trustedOrigins: ['*']` disables CSRF checks.
- `onwarn` ignores all `a11y_*` warnings (chart components).
### Deployment
- Gitea Actions self-hosted runner (`.gitea/workflows/deploy.yaml`), not GitHub.
- `deploy.sh` is the manual deployment script (same logic as CI).
### Test Coverage
Sparse. Existing tests:
- `web/src/demo.spec.ts` / `page.svelte.spec.ts`
- `server/service/changelog_test.go`
- `analyzer/test_jd_cleaner.py`
## Important Files
| File | Role |
|---|---|
| `web/src/lib/api.ts` | Central API client, auth injection |
| `web/src/lib/stores/auth.ts` | Auth state, JWT handling |
| `web/src/lib/models/bill.ts` | UIBill model + converters |
| `server/main.go` | Entry point |
| `server/handler/upload.go` | Full upload pipeline |
| `server/handler/bills.go` | List/filter bills |
| `server/model/bill.go` | Bill models, LocalTime type |
| `server/adapter/adapter.go` | Cleaner interface |
| `server/repository/mongo/repository.go` | MongoDB implementation |
| `analyzer/server.py` | FastAPI entry |
| `web/src/lib/api.ts` | Central API client, auth injection, all API functions |
| `web/src/lib/stores/auth.ts` | Auth state, JWT handling, localStorage key `auth` |
| `web/src/lib/models/bill.ts` | UIBill model + snake_case ↔ camelCase converters |
| `web/src/routes/api/[...path]/+server.ts` | SvelteKit → Go proxy (GET/POST only) |
| `server/main.go` | Entry point, wires adapter + repository + router |
| `server/config.yaml` | Go backend config (Mongo, auth, paths, analyzer mode) |
| `server/router/router.go` | Route table, auth group definitions |
| `server/handler/upload.go` | Full upload pipeline handler |
| `server/handler/bills.go` | List/filter/update/delete bills |
| `server/model/bill.go` | Bill models, LocalTime type, BSON/JSON marshaling |
| `server/adapter/adapter.go` | Cleaner interface definition |
| `server/repository/mongo/repository.go` | MongoDB implementation, soft-delete queries |
| `analyzer/server.py` | FastAPI entry, bill detection/clean endpoints |
| `analyzer/cleaners/base.py` | BaseCleaner ABC |
| `analyzer/category.py` | Category inference |
| `analyzer/category.py` | Category inference engine |
| `docker-compose.yaml` | Full stack orchestration |
## Agent Guidelines
- **Before coding:** Search codebase to understand existing patterns and dependencies
- **Dependencies:** Check `package.json`/`go.mod`/`requirements.txt` before adding new packages
- **Tests:** Always run relevant test suite before committing changes
- **Git commits:** Provide clear messages explaining the "why" of changes
- **File references:** Use relative `file_path:line_number` format (e.g., `server/handler/changelog.go:12`) when mentioning code locations
- **Before coding:** Search codebase to understand existing patterns and dependencies.
- **Dependencies:** Check `package.json`/`go.mod`/`requirements.txt` before adding new packages.
- **Tests:** Run the relevant test suite before committing. If no tests exist for your change, verify manually.
- **Git commits:** Provide clear messages explaining the "why" of changes.
- **File references:** Use relative `file_path:line_number` format (e.g., `server/handler/changelog.go:12`) when mentioning code locations.