Enterprise-Knowledge-Base/REFACTORING_PLAN.md

# Wiki Ingest Language Refactoring - Implementation Plan

## Problem Statement

The wiki ingest pipeline contains hardcoded language mapping logic (lines 135-141 in `wiki_ingest.go`) that:
- Only supports 2 languages (Chinese and English)
- Duplicates logic already available in `types.LanguageLocaleName()`
- Uses inconsistent naming compared to the existing language middleware
- Cannot be easily extended for new languages

## Solution

Reuse the existing `types.LanguageLocaleName()` function from `/internal/types/context_helpers.go` which:
- Supports 9+ languages (Chinese, English, Korean, Japanese, Russian, French, German, Spanish, Portuguese)
- Is already tested and used throughout the codebase
- Provides consistent language name formatting for LLM prompts
- Handles unknown languages gracefully

## Implementation

### Change Location

**File:** `/internal/application/service/wiki_ingest.go`
**Method:** `ProcessWikiIngest()`
**Lines:** 135-141

### Before (Current Code)

```go
135	    // Determine language
136	    lang := "the same language as the source document"
137	    if kb.WikiConfig.WikiLanguage == "zh" {
138	        lang = "Chinese (中文)"
139	    } else if kb.WikiConfig.WikiLanguage == "en" {
140	        lang = "English"
141	    }
```

### After (Refactored Code)

```go
135	    // Determine language - reuse middleware infrastructure for consistent naming
136	    // Supports: Chinese (Simplified/Traditional), English, Korean, Japanese, Russian, French, German, Spanish, Portuguese
137	    lang := types.LanguageLocaleName(kb.WikiConfig.WikiLanguage)
```

### Code Diff

```diff
-   // Determine language
-   lang := "the same language as the source document"
-   if kb.WikiConfig.WikiLanguage == "zh" {
-       lang = "Chinese (中文)"
-   } else if kb.WikiConfig.WikiLanguage == "en" {
-       lang = "English"
-   }
+   // Determine language - reuse middleware infrastructure for consistent naming
+   // Supports: Chinese (Simplified/Traditional), English, Korean, Japanese, Russian, French, German, Spanish, Portuguese
+   lang := types.LanguageLocaleName(kb.WikiConfig.WikiLanguage)
```

### Verification

The types package is already imported at line 14:
```go
"github.com/Tencent/WeKnora/internal/types"
```

No additional imports needed.

## Impact Analysis

### Benefits

✅ **Code Reduction:** 6 lines → 3 lines (50% reduction)
✅ **Language Coverage:** 2 languages → 9+ languages
✅ **Consistency:** Aligns with middleware language naming conventions
✅ **Maintainability:** Centralized language mapping (single source of truth)
✅ **Extensibility:** Adding new languages only requires updating `LanguageLocaleName()`
✅ **Testing:** Reuses existing, tested function

### Supported Languages (After Refactoring)

| Code | Output |
|------|--------|
| `zh-CN`, `zh`, `zh-Hans` | Chinese (Simplified) |
| `zh-TW`, `zh-HK`, `zh-Hant` | Chinese (Traditional) |
| `en-US`, `en`, `en-GB` | English |
| `ko-KR`, `ko` | Korean |
| `ja-JP`, `ja` | Japanese |
| `ru-RU`, `ru` | Russian |
| `fr-FR`, `fr` | French |
| `de-DE`, `de` | German |
| `es-ES`, `es` | Spanish |
| `pt-BR`, `pt` | Portuguese |
| *unknown* | Returns the locale code as-is |

### Backward Compatibility

✅ **Fully Compatible**
- Existing KB configs with `WikiLanguage: "zh"` and `WikiLanguage: "en"` continue to work
- The function handles short codes: `"zh" → "Chinese (Simplified)"`, `"en" → "English"`
- No database schema changes required
- No migration needed

## Testing Recommendations

### Test Cases

```go
// Test 1: Existing short code support
lang := types.LanguageLocaleName("zh")     // Expected: "Chinese (Simplified)"
lang := types.LanguageLocaleName("en")     // Expected: "English"

// Test 2: Full locale code support
lang := types.LanguageLocaleName("zh-CN")  // Expected: "Chinese (Simplified)"
lang := types.LanguageLocaleName("en-US")  // Expected: "English"
lang := types.LanguageLocaleName("ko-KR")  // Expected: "Korean"

// Test 3: Unknown locale fallback
lang := types.LanguageLocaleName("xx-YY")  // Expected: "xx-YY"
lang := types.LanguageLocaleName("")       // Expected: ""
```

### Integration Testing

1. Create a wiki KB with `WikiLanguage: "zh"` and verify summary pages are generated in Chinese
2. Create a wiki KB with `WikiLanguage: "en"` and verify summary pages are generated in English
3. (Optional) Create a wiki KB with `WikiLanguage: "ko"` and verify Korean language support

## Future Enhancements

### Option 1: Full Locale Code Normalization (Recommended)

Store full locale codes in WikiConfig to align with middleware:

```go
// In WikiConfig struct
type WikiConfig struct {
    WikiLanguage string // Store "zh-CN" instead of "zh"
    // ...
}

// No conversion needed
lang := types.LanguageLocaleName(kb.WikiConfig.WikiLanguage)
```

### Option 2: Context-Aware Language Selection

Use context language as fallback:

```go
// Determine language: prefer KB config, fallback to context/env, then default
lang := kb.WikiConfig.WikiLanguage
if lang == "" {
    // Fallback to context language
    if ctxLang, ok := types.LanguageFromContext(ctx); ok {
        lang = ctxLang
    } else {
        // Fallback to env/default
        lang = types.DefaultLanguage()
    }
}
humanReadableLang := types.LanguageLocaleName(lang)
```

## Implementation Steps

1. **Step 1:** Open `/internal/application/service/wiki_ingest.go`
2. **Step 2:** Replace lines 135-141 with the refactored code shown above
3. **Step 3:** Save the file
4. **Step 4:** Run tests to verify:
   ```bash
   go test ./internal/application/service/...
   go test ./internal/types/...
   ```
5. **Step 5:** (Optional) Run the wiki ingest pipeline with test documents

## Rollback Plan

If issues arise:
```bash
git checkout HEAD -- internal/application/service/wiki_ingest.go
```

## Related Files

### Definition Location
- **File:** `/internal/types/context_helpers.go`
- **Function:** `LanguageLocaleName()` (lines 85-112)

### Other Usage Points (Informational)
- `/internal/types/context_helpers.go` - `LanguageNameFromContext()` function (line 82)

### Similar Patterns (For Future Refactoring)
- Search for other hardcoded language mappings in the codebase that could benefit from centralization

## Sign-Off

- **Requested By:** User (mentions "中间件中已经有language的判断逻辑")
- **Implementation Date:** [TBD]
- **Reviewer:** [TBD]
- **Approval:** [TBD]