LocalDiplomacy/docs/LocalDiplomacy_PLAN.md

# LocalDiplomacy Architecture Plan

## Summary

LocalDiplomacy is a Mount & Blade II: Bannerlord singleplayer mod that keeps the roleplay and diplomacy feel of AIInfluence while moving AI orchestration out of the game process.

The game mod should be a thin, safe client. It captures compact world and dialogue context from Bannerlord, displays AI-driven conversations and events, validates proposed actions, and executes only whitelisted mutations. The external `LocalDiplomacy.Agent` service owns prompt construction, memory retrieval, KoboldCpp calls, tool execution, and audit logging.

Default stack:

- Bannerlord C# module: `LocalDiplomacy`
- External service: Python 3.11+ FastAPI app, `LocalDiplomacy.Agent`
- LLM backend: KoboldCpp OpenAI-compatible `/v1/chat/completions`, launched by the agent from the app working directory
- Local memory: Mem0 OSS with Qdrant
- Audit/source-of-truth event log: SQLite
- Tool calling: OpenAI-style tools, with mod-side validation before any Bannerlord mutation

## Goals

- Replace giant prompt text files with compact, structured game-state packets.
- Keep rich NPC dialogue, world events, and diplomatic statements.
- Allow AI to propose actions, but never let AI directly mutate the game.
- Make the AI layer swappable and debuggable outside Bannerlord.
- Store long-term character, kingdom, and campaign memories locally.
- Keep all player and campaign data on the user's machine by default.

## Non-Goals For v1

- Do not copy AIInfluence source, binary code, or assets.
- Do not implement intimate/child systems in the first pass.
- Do not implement TTS, STT, lip-sync, or generated voice in the first pass.
- Do not require cloud APIs.
- Do not send full save files, huge world dumps, or unbounded conversation logs to the model.

## Repository Layout

Target layout:

```text
LocalDiplomacy/
├── docs/
│   └── LocalDiplomacy_PLAN.md
├── src/
│   ├── LocalDiplomacy/
│   │   ├── LocalDiplomacy.csproj
│   │   ├── SubModule.cs
│   │   ├── Agent/
│   │   ├── Campaign/
│   │   ├── Contracts/
│   │   ├── Diplomacy/
│   │   ├── Settings/
│   │   └── UI/
│   └── LocalDiplomacy.Agent/
│       ├── pyproject.toml
│       ├── config.example.yaml
│       ├── localdiplomacy_agent/
│       │   ├── app.py
│       │   ├── config.py
│       │   ├── contracts.py
│       │   ├── koboldcpp_client.py
│       │   ├── memory.py
│       │   ├── prompts.py
│       │   ├── tools.py
│       │   └── event_log.py
│       └── tests/
└── module/
    └── LocalDiplomacy/
        ├── SubModule.xml
        ├── bin/
        ├── GUI/
        └── ModuleData/
```

## Bannerlord Module Design

The Bannerlord module should stay intentionally small and defensive.

Responsibilities:

- Load as a singleplayer module named `LocalDiplomacy`.
- Register campaign behavior for dialogue and world tick hooks.
- Collect relevant context from Bannerlord APIs.
- Serialize context into compact JSON requests.
- Send requests to the local agent.
- Render NPC replies, world events, and diplomatic statements.
- Validate proposed actions against current game state.
- Execute approved and valid actions.
- Report action results back to the agent.

Suggested dependencies:

- `Bannerlord.Harmony`
- `Bannerlord.UIExtenderEx`
- Native Bannerlord modules: `Native`, `SandBoxCore`, `Sandbox`, `StoryMode`, `CustomBattle`
- Optional later: MCM for settings UI

Core C# subsystems:

- `AgentClient`: HTTP client for the FastAPI service.
- `ContextBuilder`: creates compact request DTOs from Bannerlord state.
- `ActionValidator`: checks proposed actions against whitelists and live state.
- `ActionExecutor`: mutates Bannerlord only after validation.
- `DialogueBehavior`: hooks conversation entry points.
- `WorldTickBehavior`: sends periodic compact world diffs.
- `LocalDiplomacySettings`: service URL, timeout, debug flags, confirmation policy.
- `DebugLog`: mod-local logs for requests, responses, validation failures, and connectivity.

Feature coverage from `reference/aiinfulence_features.md` must be represented in the Bannerlord connector as validated action/context categories, not as in-game AI logic. The connector should expose live state and validate proposals for AI dialogue, NPC memory, trust/lie reactions, dynamic world events, diplomacy, romance/intimacy, settlement combat, death history, party tasks, trade/workshops/items, naval context, recruitment opportunities, visit history, non-combatant protection, and economic effects.

## External Agent Design

`LocalDiplomacy.Agent` is a local FastAPI app listening on `127.0.0.1:8766`.

Responsibilities:

- Accept compact game-state requests from the mod.
- Locate and launch `koboldcpp.exe` and `model.gguf` from the directory the app is run from.
- Monitor the KoboldCpp child process and wait for its API to become reachable before accepting AI requests.
- Retrieve relevant memories from Mem0/Qdrant.
- Read recent events from SQLite.
- Build bounded prompts/messages.
- Call KoboldCpp via OpenAI-compatible chat completions.
- Provide tool schemas to KoboldCpp.
- Execute local read/write-intention tools.
- Return final assistant text and action proposals to the mod.
- Persist memory writes and audit events.

Default command:

```powershell
uvicorn localdiplomacy_agent.app:app --host 127.0.0.1 --port 8766
```

Expected runtime files in the app working directory:

```text
LocalDiplomacy.Agent/
├── koboldcpp.exe
├── model.gguf
├── config.yaml
└── data/
```

On startup, the agent should:

1. Resolve the current working directory.
2. Verify `koboldcpp.exe` exists there.
3. Verify `model.gguf` exists there.
4. Start KoboldCpp as a child process if no compatible server is already reachable at the configured URL.
5. Launch with the configured port, context size, GPU flags, and `--jinjatools`.
6. Poll `/api/extra/version` or `/v1/models` until ready.
7. Surface a clear `/health` error if the executable, model, or API startup fails.

Default config:

```yaml
server:
  host: "127.0.0.1"
  port: 8766

koboldcpp:
  autostart: true
  executable_path: "./koboldcpp.exe"
  model_path: "./model.gguf"
  base_url: "http://127.0.0.1:5001"
  chat_path: "/v1/chat/completions"
  model: "local-model"
  port: 5001
  context_size: 8192
  extra_args:
    - "--jinjatools"
  timeout_seconds: 120
  tool_mode: "openai_tools"
  json_repair_retry: true

memory:
  provider: "mem0"
  vector_store: "qdrant"
  qdrant_host: "127.0.0.1"
  qdrant_port: 6333
  collection: "localdiplomacy_memories"
  embedder_provider: "ollama"
  embedder_model: "nomic-embed-text"
  llm_provider: "openai_compatible"

event_log:
  sqlite_path: "./data/localdiplomacy_events.sqlite3"

generation:
  temperature: 0.7
  max_tokens: 800
```

## KoboldCpp Integration

Use KoboldCpp latest stable. For packaged/local use, place `koboldcpp.exe` and the selected GGUF model named exactly `model.gguf` in the directory where `LocalDiplomacy.Agent` is run. The agent should launch KoboldCpp automatically using those files.

Default agent-launched command:

```powershell
.\koboldcpp.exe --model .\model.gguf --port 5001 --contextsize 8192 --jinjatools
```

Manual launch remains useful for debugging:

```powershell
koboldcpp.exe --model C:\Models\your-model.gguf --port 5001 --contextsize 8192 --jinjatools
```

Use `--jinja` or a custom chat template only when `model.gguf` requires it. Some modern instruct models are template-sensitive, so every model should pass a smoke test before gameplay.

Agent request shape:

```json
{
  "model": "local-model",
  "messages": [
    { "role": "system", "content": "LocalDiplomacy system instructions..." },
    { "role": "user", "content": "Compact scene packet..." }
  ],
  "tools": [],
  "tool_choice": "auto",
  "temperature": 0.7,
  "max_tokens": 800
}
```

Tool loop:

1. Agent sends messages plus tool schemas to KoboldCpp.
2. If the assistant returns `tool_calls`, the agent executes those tool handlers locally.
3. Agent appends `role: tool` messages with results.
4. Agent calls KoboldCpp again until the assistant returns final text or a configured loop limit is reached.
5. Agent returns the final response plus queued action proposals to the mod.

Fallback mode:

- If a model fails native tool calling, use a strict JSON response contract.
- Validate the JSON against Pydantic models.
- Attempt one repair prompt.
- If still invalid, return a graceful failure response and no actions.

## Public HTTP Interfaces

### `GET /health`

Returns service and dependency status.

Response:

```json
{
  "status": "ok",
  "agent_version": "0.1.0",
  "koboldcpp": "reachable",
  "memory": "reachable",
  "event_log": "ok"
}
```

### `POST /conversation/respond`

Main dialogue endpoint.

Request:

```json
{
  "campaign_id": "campaign-123",
  "save_id": "save-abc",
  "turn_id": "turn-001",
  "player_message": "What do you think of the war?",
  "player": {
    "id": "player",
    "name": "Aldric",
    "clan_id": "player_clan",
    "kingdom_id": "vlandia",
    "gold": 12000,
    "traits": {}
  },
  "npc": {
    "id": "lord_derthert",
    "name": "Derthert",
    "occupation": "lord",
    "clan_id": "dey_meroc",
    "kingdom_id": "vlandia",
    "traits": {},
    "relation_to_player": 12
  },
  "scene": {
    "location_id": "town_sargot",
    "conversation_state": "lord_dialogue",
    "player_is_prisoner": false,
    "npc_is_prisoner": false
  },
  "nearby_parties": [],
  "nearby_settlements": [],
  "kingdom_state": {},
  "recent_events": []
}
```

Response:

```json
{
  "turn_id": "turn-001",
  "assistant_text": "The war has teeth, but not yet a mind...",
  "mood": "wary",
  "tone": "formal",
  "visible_world_events": [],
  "game_actions": [],
  "memory_writes": [],
  "warnings": []
}
```

### `POST /world/tick`

Receives compact world diffs. The mod should call this on a low-frequency campaign tick, not every frame.

Request:

```json
{
  "campaign_id": "campaign-123",
  "save_id": "save-abc",
  "campaign_day": 481.5,
  "diff_id": "diff-001",
  "changed_kingdoms": [],
  "changed_settlements": [],
  "wars": [],
  "peace_deals": [],
  "battles": [],
  "notable_relation_changes": []
}
```

Response:

```json
{
  "diff_id": "diff-001",
  "accepted": true,
  "created_events": [],
  "warnings": []
}
```

### `POST /actions/result`

Reports whether the mod executed, rejected, or failed an action proposal.

Request:

```json
{
  "campaign_id": "campaign-123",
  "turn_id": "turn-001",
  "action_id": "action-001",
  "status": "executed",
  "reason": null,
  "result_summary": "Derthert proposed peace to Battania."
}
```

Response:

```json
{
  "accepted": true
}
```

### `GET /debug/status`

Returns runtime diagnostics for the debug UI.

Response:

```json
{
  "last_koboldcpp_latency_ms": 5420,
  "memory_count_estimate": 128,
  "queued_action_count": 0,
  "recent_errors": []
}
```

## Core Contracts

### `GameAction`

All AI action proposals use one normalized envelope:

```json
{
  "action_id": "action-001",
  "action_type": "propose_peace",
  "actor_id": "lord_derthert",
  "target_id": "kingdom_battania",
  "args": {
    "tribute_daily": 500
  },
  "confidence": 0.82,
  "reason": "Vlandia is losing two fronts and needs time.",
  "requires_player_confirmation": true
}
```

Validation rules:

- `action_type` must be in the whitelist.
- `actor_id` and `target_id` must resolve in current game state where required.
- Dangerous actions always require confirmation unless explicitly disabled in settings.
- The executor must reject stale proposals if game state changed materially.
- The AI's `reason` is for logs/UI only and must not override validation.

### Initial Action Whitelist

Dialogue:

- `attack`
- `surrender`
- `accept_surrender`
- `release`
- `propose_marriage`
- `accept_marriage`
- `reject_marriage`

Party:

- `follow_player`
- `go_to_settlement`
- `return_to_player`
- `attack_party`
- `patrol_settlement`
- `wait_near_settlement`
- `siege_settlement`
- `raid_village`

Diplomacy:

- `declare_war`
- `propose_peace`
- `accept_peace`
- `reject_peace`
- `propose_alliance`
- `accept_alliance`
- `reject_alliance`
- `break_alliance`
- `propose_trade_agreement`
- `accept_trade_agreement`
- `reject_trade_agreement`
- `end_trade_agreement`
- `demand_territory`
- `transfer_territory`
- `demand_tribute`
- `accept_tribute`
- `reject_tribute`
- `demand_reparations`
- `accept_reparations`
- `reject_reparations`

Recruitment:

- `hire_mercenary`
- `dismiss_mercenary`
- `offer_vassalage`
- `dismiss_vassal`
- `join_player_clan`
- `join_player_kingdom`
- `hire_mercenary_clan`
- `kick_from_clan`
- `dismiss_npc_mercenary`
- `release_npc_vassal`

## Tool Handlers

Tools exposed to the LLM should be narrow and descriptive.

Read-only tools:

- `get_npc_profile(character_id)`
- `get_relation_summary(source_id, target_id)`
- `get_nearby_parties(reference_id, limit)`
- `get_kingdom_status(kingdom_id)`
- `get_settlement_status(settlement_id)`
- `get_military_info(scope_id)`
- `get_naval_info(scope_id)`
- `get_trade_info(scope_id)`
- `get_romance_status(character_id, player_id)`
- `get_death_history(character_id)`
- `get_visit_history(character_id)`
- `get_recruitment_opportunities(scope_id)`
- `search_memory(query, campaign_id, character_id, kingdom_id, limit)`
- `list_recent_events(campaign_id, scope_id, limit)`

Write/intention tools:

- `remember_fact(scope, text, importance, tags)`
- `analyze_lie(speaker_id, listener_id, claim)`
- `propose_game_action(action_type, actor_id, target_id, args, confidence, reason)`
- `publish_diplomatic_statement(kingdom_id, speaker_id, statement_text, related_event_id)`
- `create_world_event(event_type, title, summary, involved_ids, importance, expires_after_days)`
- `update_world_event(event_id, summary, spread_to_ids, importance_delta)`
- `record_war_statistics(war_id, summary, stats)`
- `record_death_history(character_id, history_text, interaction_count)`

Important rule: no tool handler directly mutates Bannerlord. Mutation tools queue proposals and return proposal IDs. The mod decides whether execution is legal.

The initial action catalog should include every feature family in the AIInfluence reference: personality/backstory/speech generation, trust and lie detection, conversation memory, known secrets/info, dynamic event creation/spread/update/personalization, diplomatic statements/rounds/alliances/trade/territory/tribute/reparations/war fatigue/war stats/clan expulsion/pardons, romance/intimacy/marriage/degradation, AI-managed settlement combat, death history, party task chains, workshop sales, item exchanges, RP item creation, recruitment detection, settlement/party/visit tracking, non-combatant protection, naval context, and economic effects.

## Memory System

Use Mem0 OSS with Qdrant by default.

Memory scopes:

- `campaign_id`: isolates saves/campaigns.
- `character_id`: NPC-specific memory.
- `kingdom_id`: ruler and diplomacy memory.
- `player_id`: player-facing preferences and promises.

Memory categories:

- `personal_fact`
- `relationship_shift`
- `promise`
- `grudge`
- `secret`
- `diplomatic_history`
- `world_event_summary`
- `player_preference`

Retrieval policy:

- Retrieve by campaign plus the active NPC/kingdom.
- Include only the top relevant memories.
- Prefer atomic facts over full conversation excerpts.
- Include timestamps and confidence where available.
- Never rely on Mem0 as the authoritative game state; live state comes from the mod.

SQLite event ledger:

- Append every request summary, response summary, tool call, action proposal, action result, and memory write.
- Store enough metadata to debug behavior without storing huge raw prompts by default.
- Allow debug mode to store full prompts/responses for troubleshooting.

## Prompt Strategy

System prompt should define:

- The assistant role: an in-world Bannerlord NPC/diplomacy narrator.
- The hard boundary: never invent IDs, never claim actions happened unless tools/results confirm them.
- Tool usage rules: use read tools for missing relevant facts, use proposal tools for intended actions.
- Safety rules: major game actions require explicit agreement and valid targets.
- Style: grounded medieval political roleplay, concise enough for live gameplay.

Per-turn context should include:

- Current speaker and listener summaries.
- Live scene state.
- Relevant nearby parties/settlements.
- Current kingdom status if politically relevant.
- Retrieved memories.
- Recent events from SQLite.
- Player message.

Prompt budget policy:

- Keep the system message stable.
- Keep live state compact.
- Keep memories capped.
- Summarize old events.
- Do not paste giant rule files or full campaign state.

## UI Plan

v1 UI should mirror the useful feel of AIInfluence without copying assets:

- Dialogue popup/window for AI conversation.
- World events button and window.
- Diplomatic statements list.
- Debug status panel.
- Connection error banner when the external service is offline.

UI states:

- `Idle`
- `WaitingForAgent`
- `ShowingResponse`
- `ActionRequiresConfirmation`
- `AgentUnavailable`
- `ValidationFailed`

## Settings

Bannerlord settings:

- Agent URL, default `http://127.0.0.1:8766`
- Request timeout
- Enable debug logging
- Store full prompts in debug logs
- Auto-execute safe actions
- Always confirm diplomacy actions
- Always confirm hostile actions
- World tick interval

Agent settings:

- KoboldCpp autostart enabled/disabled
- KoboldCpp executable path, default `./koboldcpp.exe`
- KoboldCpp model path, default `./model.gguf`
- KoboldCpp base URL
- KoboldCpp port, context size, and extra launch args
- Model name
- Tool-call mode
- Temperature and max tokens
- Memory provider settings
- Qdrant connection
- SQLite path
- Prompt/debug logging level

## Implementation Milestones

### Milestone 1: Documentation and Contracts

- Add this architecture plan.
- Define JSON schemas/Pydantic models for all public contracts.
- Define matching C# DTOs.
- Add sample request/response fixtures.

### Milestone 2: External Agent Skeleton

- Create FastAPI app.
- Implement `/health`, `/debug/status`, and config loading.
- Implement KoboldCpp runtime file checks for `koboldcpp.exe` and `model.gguf`.
- Implement child-process launch/stop management for KoboldCpp.
- Add event log initialization.
- Add unit tests for config and contracts.

### Milestone 3: KoboldCpp Chat Loop

- Implement OpenAI-compatible client.
- Wait for autostarted KoboldCpp readiness before sending chat requests.
- Implement tool schema generation.
- Implement bounded tool loop.
- Add smoke test script for KoboldCpp connectivity and tool-call behavior.

### Milestone 4: Memory Layer

- Add Mem0/Qdrant integration.
- Add scoped memory writes and searches.
- Add fallback no-memory mode when Qdrant is offline.
- Test memory isolation by campaign and character.

### Milestone 5: Bannerlord Thin Client

- Create C# module skeleton and `SubModule.xml`.
- Add service health check.
- Add compact context builder.
- Add dialogue request/response path.
- Add graceful offline behavior.

### Milestone 6: Action Proposal and Validation

- Implement action proposal contract end-to-end.
- Add mod-side action whitelist.
- Validate IDs and game-state preconditions.
- Execute one safe action category first, then expand.

### Milestone 7: Diplomacy and Events

- Add diplomatic statement creation.
- Add world event ingestion and display.
- Add ruler dialogue actions.
- Add confirmation UI for dangerous actions.

## Test Plan

Agent unit tests:

- Contract validation rejects malformed requests.
- Tool schemas match handler signatures.
- `GameAction` normalization fills required defaults.
- Memory scope keys isolate campaigns and NPCs.
- SQLite event ledger appends expected records.

Agent integration tests:

- `/health` succeeds with dependencies mocked.
- KoboldCpp chat succeeds with a basic response.
- Tool loop handles one read-only tool call.
- Tool loop handles one action proposal.
- Invalid tool JSON triggers one repair attempt.
- Failed repair returns no actions.

Bannerlord smoke tests:

- Module loads with required dependencies.
- Settings load defaults.
- Agent unavailable state is shown gracefully.
- Dialogue UI sends a compact request.
- Response displays without executing actions.
- Invalid action proposal is rejected and logged.

Gameplay acceptance tests:

- NPC remembers a prior promise after save reload.
- Ruler creates a diplomatic statement from recent events.
- AI refuses or asks clarification when a target id is missing.
- Peace/war proposals require confirmation.
- Large campaign state remains compact and avoids giant generated text files.

## Rollout Strategy

Start with developer-only debug mode:

1. Put `koboldcpp.exe` and `model.gguf` beside the agent startup directory.
2. Run Qdrant manually or through Docker.
3. Run `LocalDiplomacy.Agent`; it should launch KoboldCpp automatically.
4. Launch Bannerlord with the mod.
5. Use debug panel to verify service health.
6. Test one dialogue response with actions disabled.
7. Enable action proposals but keep all execution behind confirmation.
8. Gradually enable safe auto-execution for low-risk actions.

## Risks And Mitigations

- KoboldCpp tool calling varies by model.
  - Mitigation: require a smoke test and keep JSON fallback mode.

- KoboldCpp executable/model may be missing from the working directory.
  - Mitigation: startup checks must produce explicit errors naming the missing file and expected path.

- Autostarted KoboldCpp may hang during model load.
  - Mitigation: readiness polling, startup timeout, process logs, and a manual-launch override.

- AI may hallucinate entities or action parameters.
  - Mitigation: IDs must come from context/tools and mod-side validation is mandatory.

- Memory retrieval may surface stale or wrong facts.
  - Mitigation: live game state always wins; memory is contextual, not authoritative.

- External service may be offline or slow.
  - Mitigation: short health checks, clear UI state, configurable timeouts.

- Bannerlord action APIs can have hidden constraints.
  - Mitigation: implement action categories incrementally and log validation failures.

## Research Sources

- KoboldCpp README and wiki: https://github.com/LostRuins/koboldcpp and https://github.com/LostRuins/koboldcpp/wiki
- KoboldCpp releases/tool-calling notes: https://github.com/LostRuins/koboldcpp/releases
- Mem0 OSS docs: https://github.com/mem0ai/mem0/blob/main/LLM.md
- Mem0/OpenMemory local MCP overview: https://mem0.ai/blog/how-to-make-your-clients-more-context-aware-with-openmemory-mcp
- Qdrant Mem0 integration: https://qdrant.tech/documentation/frameworks/mem0/
- Graphiti/Zep comparison option: https://www.getzep.com/product/open-source/