Weknora/Enterprise-Knowledge-Base
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Update VLM configuration in knowledge base and API documentation

Browse Source 
- Replaced `vlm_model_id` with a structured `vlm_config` object in knowledge base configurations, including `enabled` flag and `model_id`.
- Updated API documentation to reflect changes in VLM configuration structure.
- Enhanced frontend interfaces to accommodate new VLM configuration format.
- Improved localization for new features related to knowledge graph and multimodal capabilities.
This commit is contained in:
wizardchen
2025-11-20 20:08:24 +08:00
parent cdb5937a66
commit c1f46df0dd
23 changed files with 1063 additions and 256 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/API.md
+10 -25
View File
@@ -362,12 +362,9 @@ curl --location 'http://localhost:8080/api/v1/knowledge-bases' \
"embedding_model_id": "dff7bc94-7885-4dd1-bfd5-bd96e4df2fc3",
"summary_model_id": "8aea788c-bb30-4898-809e-e40c14ffb48c",
"rerank_model_id": "b30171a1-787b-426e-a293-735cd5ac16c0",
"vlm_model_id": "f2083ad7-63e3-486d-a610-e6c56e58d72e",
"vlm_config": {
"model_name": "qwen2.5vl:3b",
"interface_type": "ollama",
"base_url": "",
"api_key": ""
"enabled": true,
"model_id": "f2083ad7-63e3-486d-a610-e6c56e58d72e"
},
"cos_config": {
"secret_id": "",
@@ -403,12 +400,9 @@ curl --location 'http://localhost:8080/api/v1/knowledge-bases' \
"embedding_model_id": "dff7bc94-7885-4dd1-bfd5-bd96e4df2fc3",
"summary_model_id": "8aea788c-bb30-4898-809e-e40c14ffb48c",
"rerank_model_id": "b30171a1-787b-426e-a293-735cd5ac16c0",
"vlm_model_id": "f2083ad7-63e3-486d-a610-e6c56e58d72e",
"vlm_config": {
"model_name": "qwen2.5vl:3b",
"base_url": "",
"api_key": "",
"interface_type": "ollama"
"enabled": true,
"model_id": "f2083ad7-63e3-486d-a610-e6c56e58d72e"
},
"cos_config": {
"secret_id": "",
@@ -466,12 +460,9 @@ curl --location 'http://localhost:8080/api/v1/knowledge-bases' \
"embedding_model_id": "dff7bc94-7885-4dd1-bfd5-bd96e4df2fc3",
"summary_model_id": "8aea788c-bb30-4898-809e-e40c14ffb48c",
"rerank_model_id": "b30171a1-787b-426e-a293-735cd5ac16c0",
"vlm_model_id": "f2083ad7-63e3-486d-a610-e6c56e58d72e",
"vlm_config": {
"model_name": "qwen2.5vl:3b",
"base_url": "http://host.docker.internal:11435/v1",
"api_key": "",
"interface_type": "ollama"
"enabled": true,
"model_id": "f2083ad7-63e3-486d-a610-e6c56e58d72e"
},
"cos_config": {
"secret_id": "",
@@ -529,12 +520,9 @@ curl --location 'http://localhost:8080/api/v1/knowledge-bases/kb-00000001' \
"embedding_model_id": "dff7bc94-7885-4dd1-bfd5-bd96e4df2fc3",
"summary_model_id": "8aea788c-bb30-4898-809e-e40c14ffb48c",
"rerank_model_id": "b30171a1-787b-426e-a293-735cd5ac16c0",
"vlm_model_id": "f2083ad7-63e3-486d-a610-e6c56e58d72e",
"vlm_config": {
"model_name": "qwen2.5vl:3b",
"base_url": "http://host.docker.internal:11435/v1",
"api_key": "",
"interface_type": "ollama"
"enabled": true,
"model_id": "f2083ad7-63e3-486d-a610-e6c56e58d72e"
},
"cos_config": {
"secret_id": "",
@@ -614,12 +602,9 @@ curl --location --request PUT 'http://localhost:8080/api/v1/knowledge-bases/b582
"embedding_model_id": "dff7bc94-7885-4dd1-bfd5-bd96e4df2fc3",
"summary_model_id": "8aea788c-bb30-4898-809e-e40c14ffb48c",
"rerank_model_id": "b30171a1-787b-426e-a293-735cd5ac16c0",
"vlm_model_id": "f2083ad7-63e3-486d-a610-e6c56e58d72e",
"vlm_config": {
"model_name": "qwen2.5vl:3b",
"base_url": "",
"api_key": "",
"interface_type": "ollama"
"enabled": true,
"model_id": "f2083ad7-63e3-486d-a610-e6c56e58d72e"
},
"cos_config": {
"secret_id": "",
docs/开启知识图谱功能.md
+85
View File
@@ -0,0 +1,85 @@
# 开启知识图谱功能指南
本文档介绍如何在 WeKnora 中启用并验证知识图谱（Neo4j）功能，帮助你完成从环境准备到前端配置的全流程。
## 前置条件
- 已完成 WeKnora 后端与前端的基础部署。
- 具备可用的 Docker/Docker Compose 运行环境。
- 本地或远端可访问的 Neo4j 服务（推荐使用项目自带的 Docker Compose）。
## 步骤一：配置环境变量
在项目根目录的 `.env` 文件中新增或修改以下变量：
```
NEO4J_ENABLE=true
NEO4J_URI=bolt://neo4j:7687
NEO4J_USERNAME=neo4j
NEO4J_PASSWORD=your_strong_password
# 可选：NEO4J_DATABASE=neo4j
```
说明：
- `NEO4J_ENABLE` 设置为 `true` 才会启用知识图谱相关逻辑。
- `NEO4J_URI` 中的 `neo4j` 为 docker-compose 服务名，如使用外部实例请替换为实际地址。
- 如果生产环境使用密钥管理，请确保密码通过安全方式注入。
## 步骤二：启动 Neo4j 服务
项目附带 Neo4j 组件，可直接用以下命令启动：
```bash
docker-compose --profile neo4j up -d
```
常见验证命令：
```bash
docker ps | grep neo4j
```
若需要自定义挂载或内存，可编辑 `docker-compose.yml` 中 `neo4j` 服务配置。
## 步骤三：重启 WeKnora 服务
为了让新的环境变量生效，重启后端与前端（示例仅供参考）：
```bash
make stop && make start
# 或者
docker compose up -d --build
```
确保后端日志中出现 `neo4j` 初始化成功的提示。
## 步骤四：在前端启用实体/关系抽取
1. 登录 WeKnora 前端管理页面。
2. 打开「知识库设置」或创建新的知识库。
3. 勾选「启用实体抽取」与「启用关系抽取」开关。
4. 根据界面提示补充所需的 LLM、回调或模型参数（若有）。
保存后，系统会在文档入库阶段自动触发实体与关系抽取任务。
## 步骤五：验证知识图谱
### 方式一：Neo4j 控制台
1. 访问 `http://localhost:7474`（或对应主机/端口）。
2. 使用 `.env` 中的账号密码登录。
3. 执行 `MATCH (n) RETURN n LIMIT 50;` 检查是否有新节点/关系。
### 方式二：WeKnora 界面
在知识库或对话页面中上传文档后，前端应展示图谱可视化入口；对话时系统会自动根据意图查询图谱并返回补充信息。
## 常见问题排查
- **无法连接 Neo4j**：确认网络可达、`NEO4J_URI` 与用户名密码正确，并检查 Neo4j 容器日志。
- **未生成节点**：确认知识库已开启实体/关系抽取，且上传的文档已完成解析；查看后端日志中是否有抽取任务异常。
- **查询无结果**：尝试在 Neo4j 控制台执行 `CALL db.schema.visualization;` 查看 schema 是否存在，必要时重新导入文档。
完成以上步骤后，知识图谱功能即成功启用，可结合 RAG 及 Agent 流程提升问答质量。
frontend/src/api/initialization/index.ts
+4 -1
View File
@@ -74,7 +74,10 @@ export interface DownloadTask {
export interface KBModelConfigRequest {
llmModelId: string
embeddingModelId: string
vllmModelId?: string
vlm_config?: {
enabled: boolean
model_id?: string
}
documentSplitting: {
chunkSize: number
chunkOverlap: number
frontend/src/api/knowledge-base/index.ts
+4 -2
View File
@@ -12,8 +12,10 @@ export function createKnowledgeBase(data: {
chunking_config?: any;
embedding_model_id?: string;
summary_model_id?: string;
vlm_model_id?: string;
vlm_config?: any;
vlm_config?: {
enabled: boolean;
model_id?: string;
};
cos_config?: any;
extract_config?: any;
faq_config?: { index_mode: string; question_index_mode?: string };
frontend/src/i18n/locales/en-US.ts
+48 -11
View File
@@ -698,16 +698,20 @@ export default {
uninitializedBanner: 'Some knowledge bases are not initialized. Configure model information in settings before adding documents.',
empty: {
title: 'No knowledge bases yet',
description: 'Click “Create Knowledge Base” in the top-right corner to add your first one.'
description: 'Click "Create Knowledge Base" in the top-right corner to add your first one.'
},
delete: {
confirmTitle: 'Delete Confirmation',
confirmMessage: 'Are you sure you want to delete the knowledge base “{name}”? This action cannot be undone.',
confirmMessage: 'Are you sure you want to delete the knowledge base "{name}"? This action cannot be undone.',
confirmButton: 'Delete'
},
messages: {
deleted: 'Knowledge base deleted',
deleteFailed: 'Failed to delete knowledge base'
},
features: {
knowledgeGraph: 'Knowledge Graph Enabled',
multimodal: 'Multimodal Enabled'
}
},
knowledgeEditor: {
@@ -868,8 +872,11 @@ export default {
separatorsDescription: 'Separators used when chunking documents',
separatorsPlaceholder: 'Select separators',
separators: {
doubleNewline: 'Double newline (\\n\\n)',
singleNewline: 'Single newline (\\n)',
doubleNewline: 'Double newline (\
\
)',
singleNewline: 'Single newline (\
)',
periodCn: 'Chinese period (。)',
exclamationCn: 'Exclamation mark (！)',
questionCn: 'Question mark (？)',
@@ -936,8 +943,29 @@ export default {
tagsLabel: 'Tags',
tagsDescription: 'Predefined entity tags (separate multiple tags with commas)',
tagsPlaceholder: 'Enter a tag and press Enter',
disabledWarning: 'Graph database is not enabled. Please enable the graph database first to use this feature.',
howToEnable: 'How to enable'
nodesLabel: 'Node Configuration',
nodesDescription: 'Define entity node types and attributes to extract',
addNode: 'Add Node',
nodeNameLabel: 'Node Name',
nodeNamePlaceholder: 'e.g., Person, Organization',
nodeChunksLabel: 'Associated Chunks',
nodeChunksPlaceholder: 'Enter chunk ID and press Enter',
nodeAttributesLabel: 'Node Attributes',
nodeAttributesPlaceholder: 'Enter attribute name and press Enter',
relationsLabel: 'Relation Configuration',
relationsDescription: 'Define relationship types between nodes',
addRelation: 'Add Relation',
relationNode1Label: 'Source Node',
relationNode1Placeholder: 'e.g., Person',
relationNode2Label: 'Target Node',
relationNode2Placeholder: 'e.g., Organization',
relationTypeLabel: 'Relation Type',
relationTypePlaceholder: 'e.g., WORKS_FOR, BELONGS_TO',
deleteNode: 'Delete Node',
deleteRelation: 'Delete Relation',
disabledWarning:
'Graph database is not enabled. Please follow the "Enable Knowledge Graph" guide before using this feature.',
howToEnable: 'View guide'
}
}
},
@@ -1014,7 +1042,16 @@ export default {
nullValuePlaceholder: '<NULL>',
documentTitleLabel: 'Document title:',
chunkCountLabel: 'Chunk count:',
chunkCountValue: 'At least {count}',
chunkCountValue: '{count} chunks',
documentDescriptionLabel: 'Description:',
documentStatusLabel: 'Status:',
documentSourceLabel: 'Source:',
documentFileLabel: 'File:',
documentMetadataLabel: 'Metadata',
documentInfoSummaryLabel: 'Document info',
documentInfoCount: '{count} of {requested} documents retrieved',
documentInfoErrors: 'Errors',
documentInfoEmpty: 'No document information available',
statusDescription: 'Status notes',
statusIndexed: 'Document is indexed and searchable',
statusSearchable: 'Search tools can locate document content',
@@ -1232,7 +1269,7 @@ export default {
dimensionDetected: 'Detection succeeded. Vector dimension: {value}',
dimensionFailed: 'Detection failed, please enter the dimension manually',
remoteDimensionDetected: 'Detected vector dimension: {value}',
dimensionHint: 'Model selected. Click “Detect Dimension” to fetch the vector dimension automatically.',
dimensionHint: 'Model selected. Click "Detect Dimension" to fetch the vector dimension automatically.',
setAsDefault: 'Set as default model',
loadModelListFailed: 'Failed to load model list',
listRefreshed: 'List refreshed',
@@ -1459,7 +1496,7 @@ export default {
},
fallbackPrompt: {
label: 'Fallback Prompt',
desc: 'Prompt used when fallback strategy is “model”'
desc: 'Prompt used when fallback strategy is "model"'
},
advanced: {
description: 'Configure query rewrite, fallback strategy and other advanced settings'
@@ -1559,7 +1596,7 @@ export default {
deleted: 'MCP service deleted',
deleteFailed: 'Failed to delete MCP service'
},
deleteConfirmBody: 'Delete MCP service “{name}”? This action cannot be undone.',
deleteConfirmBody: 'Delete MCP service "{name}"? This action cannot be undone.',
unnamed: 'Unnamed'
},
// New: Model Settings
@@ -1616,7 +1653,7 @@ export default {
description: 'Manage local Ollama service and view/download models',
status: {
label: 'Ollama Service Status',
desc: 'Automatically detect local Ollama service availability. If the service is down or the URL is incorrect, status will be “Unavailable”.',
desc: 'Automatically detect local Ollama service availability. If the service is down or the URL is incorrect, status will be "Unavailable".',
testing: 'Testing',
available: 'Available',
unavailable: 'Unavailable',
frontend/src/i18n/locales/ru-RU.ts
+14 -1
View File
@@ -795,6 +795,10 @@ export default {
messages: {
deleted: 'База знаний удалена',
deleteFailed: 'Не удалось удалить базу знаний'
},
features: {
knowledgeGraph: 'Граф знаний включен',
multimodal: 'Мультимодальность включена'
}
},
knowledgeEditor: {
@@ -1099,7 +1103,16 @@ export default {
nullValuePlaceholder: '<NULL>',
documentTitleLabel: 'Название документа:',
chunkCountLabel: 'Количество фрагментов:',
chunkCountValue: 'Не менее {count}',
chunkCountValue: '{count} фрагментов',
documentDescriptionLabel: 'Описание:',
documentStatusLabel: 'Статус:',
documentSourceLabel: 'Источник:',
documentFileLabel: 'Файл:',
documentMetadataLabel: 'Метаданные',
documentInfoSummaryLabel: 'Информация о документах',
documentInfoCount: '{count} из {requested} документов получено',
documentInfoErrors: 'Ошибки',
documentInfoEmpty: 'Нет данных о документах',
statusDescription: 'Информация о статусе',
statusIndexed: 'Документ проиндексирован и доступен для поиска',
statusSearchable: 'Можно искать содержимое документа с помощью инструментов',
frontend/src/i18n/locales/zh-CN.ts
+37 -3
View File
@@ -217,7 +217,16 @@ export default {
nullValuePlaceholder: "<NULL>",
documentTitleLabel: "文档标题:",
chunkCountLabel: "片段数量:",
chunkCountValue: "至少 {count} 个",
chunkCountValue: "{count} 个片段",
documentDescriptionLabel: "文档描述:",
documentStatusLabel: "处理状态:",
documentSourceLabel: "来源:",
documentFileLabel: "文件信息:",
documentMetadataLabel: "元数据",
documentInfoSummaryLabel: "文档信息",
documentInfoCount: "成功 {count} / 请求 {requested}",
documentInfoErrors: "错误详情",
documentInfoEmpty: "暂无文档信息",
statusDescription: "状态说明",
statusIndexed: "文档已索引并可搜索",
statusSearchable: "可使用搜索工具查找文档内容",
@@ -1021,6 +1030,10 @@ export default {
deleted: "已删除",
deleteFailed: "删除失败",
},
features: {
knowledgeGraph: "已启用知识图谱",
multimodal: "已启用多模态",
},
},
knowledgeEditor: {
titleCreate: "新建知识库",
@@ -1253,8 +1266,29 @@ export default {
tagsLabel: "标签",
tagsDescription: "预定义的实体标签（多个标签用逗号分隔）",
tagsPlaceholder: "输入标签后按回车",
disabledWarning: "图数据库未启用，无法使用知识图谱功能。请先启用图数据库后再使用此功能。",
howToEnable: "查看如何开启",
nodesLabel: "节点配置",
nodesDescription: "定义需要提取的实体节点类型及其属性",
addNode: "添加节点",
nodeNameLabel: "节点名称",
nodeNamePlaceholder: "如：Person、Organization",
nodeChunksLabel: "关联文档块",
nodeChunksPlaceholder: "输入 chunk ID 后按回车",
nodeAttributesLabel: "节点属性",
nodeAttributesPlaceholder: "输入属性名后按回车",
relationsLabel: "关系配置",
relationsDescription: "定义节点之间的关系类型",
addRelation: "添加关系",
relationNode1Label: "起始节点",
relationNode1Placeholder: "如：Person",
relationNode2Label: "目标节点",
relationNode2Placeholder: "如：Organization",
relationTypeLabel: "关系类型",
relationTypePlaceholder: "如：WORKS_FOR、BELONGS_TO",
deleteNode: "删除节点",
deleteRelation: "删除关系",
disabledWarning:
"图数据库未启用，无法使用知识图谱功能。请先根据《开启知识图谱功能指南》完成配置。",
howToEnable: "查看启用说明",
},
},
},
frontend/src/types/tool-results.ts
+20 -3
View File
@@ -92,11 +92,28 @@ export interface KnowledgeBaseListData {
}
// Document info data
export interface DocumentInfoData {
display_type: 'document_info';
export interface DocumentInfoDocument {
knowledge_id: string;
title: string;
chunk_count_min: number;
description?: string;
type?: string;
source?: string;
file_name?: string;
file_type?: string;
file_size?: number;
parse_status?: string;
chunk_count?: number;
metadata?: Record<string, any>;
type_icon?: string;
}
export interface DocumentInfoData {
display_type: 'document_info';
documents?: DocumentInfoDocument[];
total_docs: number;
requested: number;
errors?: string[];
title?: string;
}
// Graph query results data
frontend/src/views/chat/components/tool-results/DocumentInfo.vue
+240 -38
View File
@@ -1,73 +1,276 @@
<template>
<div class="document-info">
<div class="info-card">
<div class="info-section">
<div class="info-field">
<span class="field-label">{{ $t('chat.documentIdLabel') }}</span>
<span class="field-value"><code>{{ data.knowledge_id }}</code></span>
<div v-if="documents.length" class="documents-list">
<div
v-for="(doc, index) in documents"
:key="doc.knowledge_id || index"
class="result-card document-card"
>
<div class="result-header document-header">
<div class="result-title">
<span class="doc-index">#{{ index + 1 }}</span>
<span class="doc-title">{{ doc.title || $t('chat.notProvided') }}</span>
</div>
<div class="result-meta">
<span class="meta-chip" v-if="doc.chunk_count">
{{ $t('chat.chunkCountValue', { count: doc.chunk_count }) }}
</span>
</div>
</div>
<div class="info-field">
<span class="field-label">{{ $t('chat.documentTitleLabel') }}</span>
<span class="field-value">{{ data.title }}</span>
</div>
<div class="info-field">
<span class="field-label">{{ $t('chat.chunkCountLabel') }}</span>
<span class="field-value">{{ $t('chat.chunkCountValue', { count: data.chunk_count_min }) }}</span>
<div class="result-content expanded">
<div class="info-section">
<div class="info-field">
<span class="field-label">{{ $t('chat.documentIdLabel') }}</span>
<span class="field-value"><code>{{ doc.knowledge_id }}</code></span>
</div>
<div class="info-field" v-if="doc.description">
<span class="field-label">{{ $t('chat.documentDescriptionLabel') }}</span>
<span class="field-value">{{ doc.description }}</span>
</div>
<div class="info-field" v-if="doc.source || doc.type">
<span class="field-label">{{ $t('chat.documentSourceLabel') }}</span>
<span class="field-value">{{ formatSource(doc) }}</span>
</div>
<div class="info-field" v-if="doc.file_name || doc.file_type || doc.file_size">
<span class="field-label">{{ $t('chat.documentFileLabel') }}</span>
<span class="field-value">
<span v-if="doc.file_name">{{ doc.file_name }}</span>
<template v-if="doc.file_type">&nbsp;({{ doc.file_type }})</template>
<template v-if="doc.file_size">&nbsp;· {{ formatFileSize(doc.file_size) }}</template>
</span>
</div>
</div>
<div
v-if="doc.metadata && Object.keys(doc.metadata).length"
class="info-section metadata-section"
>
<div class="info-section-title">{{ $t('chat.documentMetadataLabel') }}</div>
<ul class="metadata-list">
<li
v-for="(value, key) in doc.metadata"
:key="`${doc.knowledge_id}-${key}`"
>
<span class="metadata-key">{{ key }}:</span>
<span class="metadata-value">{{ formatMetadataValue(value) }}</span>
</li>
</ul>
</div>
</div>
</div>
</div>
<div v-else class="empty-state">
{{ $t('chat.documentInfoEmpty') }}
</div>
</div>
</template>
<script setup lang="ts">
import { defineProps } from 'vue';
import type { DocumentInfoData } from '@/types/tool-results';
import { computed, defineProps } from 'vue';
import { useI18n } from 'vue-i18n';
import type { DocumentInfoData, DocumentInfoDocument } from '@/types/tool-results';
const props = defineProps<{
data: DocumentInfoData;
}>();
const { t } = useI18n();
const documents = computed(() => props.data?.documents ?? []);
const errors = computed(() => props.data?.errors?.filter(Boolean) ?? []);
const totalChunkCount = computed(() =>
documents.value.reduce((sum, doc) => sum + (doc.chunk_count || 0), 0),
);
const formatSource = (doc: DocumentInfoDocument) => {
if (doc.type && doc.source) {
return `${doc.type} · ${doc.source}`;
}
return doc.source || doc.type || t('chat.notProvided');
};
const formatFileSize = (size?: number) => {
if (!size || size <= 0) {
return t('chat.notProvided');
}
const units = ['B', 'KB', 'MB', 'GB'];
let value = size;
let unitIndex = 0;
while (value >= 1024 && unitIndex < units.length - 1) {
value /= 1024;
unitIndex += 1;
}
const fixed = value >= 10 || unitIndex === 0 ? 0 : 1;
return `${value.toFixed(fixed)} ${units[unitIndex]}`;
};
const formatMetadataValue = (value: unknown) => {
if (value === null || value === undefined) {
return t('chat.notProvided');
}
if (typeof value === 'object') {
try {
return JSON.stringify(value);
} catch {
return String(value);
}
}
return String(value);
};
</script>
<style lang="less" scoped>
@import './tool-results.less';
.document-info {
padding: 8px 0;
display: flex;
flex-direction: column;
gap: 12px;
}
.info-card {
background: @card-bg;
border: 1px solid @card-border;
border-radius: @card-radius;
padding: 12px;
.document-summary {
.summary-main {
display: flex;
justify-content: space-between;
align-items: center;
gap: 12px;
}
.summary-label {
font-size: 13px;
font-weight: 600;
color: #333;
}
.summary-value {
font-size: 13px;
color: #555;
}
.summary-meta {
margin-top: 8px;
display: flex;
flex-wrap: wrap;
gap: 6px;
}
.summary-errors {
margin-top: 10px;
padding-top: 10px;
border-top: 1px solid @card-border;
}
}
.status-section {
margin-top: 12px;
padding-top: 12px;
border-top: 1px solid @card-border;
}
.status-title {
.meta-chip {
font-size: 12px;
font-weight: 600;
color: #666;
text-transform: uppercase;
letter-spacing: 0.5px;
margin-bottom: 6px;
color: #555;
background: #ffffff;
border: 1px solid @card-border;
border-radius: 12px;
padding: 2px 10px;
line-height: 1.6;
white-space: nowrap;
}
.status-list {
.documents-list {
display: flex;
flex-direction: column;
gap: 10px;
}
.document-card {
margin: 0 10px 10px 10px;
.document-header {
align-items: center;
}
.doc-index {
font-weight: 600;
color: #07c05f;
}
.doc-title {
font-size: 14px;
font-weight: 500;
color: #222;
}
.status-pill {
font-size: 12px;
color: #07c05f;
border: 1px solid rgba(7, 192, 95, 0.5);
border-radius: 12px;
padding: 2px 10px;
line-height: 1.4;
}
}
.info-section {
margin-top: 0;
padding: 8px 0;
&:first-of-type {
padding-top: 4px;
}
}
.info-field {
display: flex;
gap: 12px;
margin-bottom: 6px;
font-size: 13px;
.field-label {
color: #8b8b8b;
min-width: 100px;
font-weight: 600;
}
.field-value {
flex: 1;
color: #333;
line-height: 1.5;
}
}
.metadata-section {
padding-top: 12px;
border-top: 1px dashed @card-border;
}
.metadata-list {
list-style: none;
margin: 6px 0 0;
padding: 0;
display: flex;
flex-direction: column;
gap: 4px;
li {
font-size: 12px;
color: #333;
}
.metadata-key {
font-weight: 600;
margin-right: 4px;
}
.metadata-value {
font-family: 'Monaco', 'Courier New', monospace;
}
}
.status-item {
font-size: 12px;
color: #00a870;
line-height: 1.5;
.empty-state {
font-size: 13px;
color: #666;
text-align: center;
padding: 16px;
border: 1px dashed @card-border;
border-radius: @card-radius;
background: #fff;
}
code {
@@ -78,4 +281,3 @@ code {
border-radius: 3px;
}
</style>
frontend/src/views/knowledge/KnowledgeBaseEditorModal.vue
+28 -11
View File
@@ -275,7 +275,17 @@ const initFormData = (type: 'document' | 'faq' = 'document') => {
nodeExtractConfig: {
enabled: false,
text: '',
tags: [] as string[]
tags: [] as string[],
nodes: [] as Array<{
name: string
chunks: string[]
attributes: string[]
}>,
relations: [] as Array<{
node1: string
node2: string
type: string
}>
}
}
}
@@ -331,9 +341,9 @@ const loadKBData = async () => {
separators: kb.chunking_config?.separators || ['\n\n', '\n', '。', '！', '？', ';', '；']
},
multimodalConfig: {
enabled: !!(kb.vlm_model_id || (kb.cos_config?.provider && kb.cos_config?.bucket_name)),
enabled: !!(kb.vlm_config?.enabled || (kb.cos_config?.provider && kb.cos_config?.bucket_name)),
storageType: (kb.cos_config?.provider || 'minio') as 'minio' | 'cos',
vllmModelId: kb.vlm_model_id || '',
vllmModelId: kb.vlm_config?.model_id || '',
minio: {
bucketName: kb.cos_config?.bucket_name || '',
useSSL: kb.cos_config?.use_ssl || false,
@@ -351,7 +361,9 @@ const loadKBData = async () => {
nodeExtractConfig: {
enabled: kb.extract_config?.enabled || false,
text: kb.extract_config?.text || '',
tags: kb.extract_config?.tags || []
tags: kb.extract_config?.tags || [],
nodes: kb.extract_config?.nodes || [],
relations: kb.extract_config?.relations || []
}
}
} catch (error) {
@@ -450,11 +462,14 @@ const buildSubmitData = () => {
}
// 添加多模态配置
data.vlm_config = {
enabled: formData.value.multimodalConfig.enabled,
model_id: formData.value.multimodalConfig.enabled
? (formData.value.multimodalConfig.vllmModelId || '')
: ''
}
if (formData.value.multimodalConfig.enabled) {
if (formData.value.multimodalConfig.vllmModelId) {
data.vlm_model_id = formData.value.multimodalConfig.vllmModelId
}
const storageType = formData.value.multimodalConfig.storageType
if (storageType === 'minio') {
data.cos_config = {
@@ -481,7 +496,9 @@ const buildSubmitData = () => {
data.extract_config = {
enabled: true,
text: formData.value.nodeExtractConfig.text,
tags: formData.value.nodeExtractConfig.tags
tags: formData.value.nodeExtractConfig.tags,
nodes: formData.value.nodeExtractConfig.nodes,
relations: formData.value.nodeExtractConfig.relations
}
}
@@ -540,14 +557,14 @@ const handleSubmit = async () => {
const config: KBModelConfigRequest = {
llmModelId: data.summary_model_id,
embeddingModelId: data.embedding_model_id,
vllmModelId: data.vlm_model_id || '',
vlm_config: data.vlm_config,
documentSplitting: {
chunkSize: data.chunking_config.chunk_size,
chunkOverlap: data.chunking_config.chunk_overlap,
separators: data.chunking_config.separators
},
multimodal: {
enabled: !!data.cos_config,
enabled: !!data.cos_config || !!data.vlm_config?.enabled,
storageType: data.cos_config?.provider || 'minio',
cos: data.cos_config?.provider === 'cos' ? {
secretId: data.cos_config.secret_id,
frontend/src/views/knowledge/KnowledgeBaseList.vue
+62 -3
View File
@@ -71,9 +71,23 @@
<!-- 卡片底部 -->
<div class="card-bottom">
<div class="type-badge" :class="{ 'document': (kb.type || 'document') === 'document', 'faq': kb.type === 'faq' }">
<t-icon :name="kb.type === 'faq' ? 'chat-bubble-help' : 'folder'" size="14px" />
<span>{{ kb.type === 'faq' ? $t('knowledgeEditor.basic.typeFAQ') : $t('knowledgeEditor.basic.typeDocument') }}</span>
<div class="bottom-left">
<div class="type-badge" :class="{ 'document': (kb.type || 'document') === 'document', 'faq': kb.type === 'faq' }">
<t-icon :name="kb.type === 'faq' ? 'chat-bubble-help' : 'folder'" size="14px" />
<span>{{ kb.type === 'faq' ? $t('knowledgeEditor.basic.typeFAQ') : $t('knowledgeEditor.basic.typeDocument') }}</span>
</div>
<div class="feature-badges">
<t-tooltip v-if="kb.extract_config?.enabled" :content="$t('knowledgeList.features.knowledgeGraph')" placement="top">
<div class="feature-badge kg">
<t-icon name="relation" size="14px" />
</div>
</t-tooltip>
<t-tooltip v-if="kb.vlm_config?.enabled || (kb.cos_config?.provider && kb.cos_config?.bucket_name)" :content="$t('knowledgeList.features.multimodal')" placement="top">
<div class="feature-badge multimodal">
<t-icon name="image" size="14px" />
</div>
</t-tooltip>
</div>
</div>
<span class="card-time">{{ kb.updated_at }}</span>
</div>
@@ -150,6 +164,9 @@ interface KB {
summary_model_id?: string;
type?: 'document' | 'faq';
showMore?: boolean;
vlm_config?: { enabled?: boolean; model_id?: string };
extract_config?: { enabled?: boolean };
cos_config?: { provider?: string; bucket_name?: string };
}
const kbs = ref<KB[]>([])
@@ -483,6 +500,12 @@ const handleKBEditorSuccess = (kbId: string) => {
margin-top: auto;
}
.bottom-left {
display: flex;
align-items: center;
gap: 8px;
}
.type-badge {
display: inline-flex;
align-items: center;
@@ -507,6 +530,42 @@ const handleKBEditorSuccess = (kbId: string) => {
}
}
.feature-badges {
display: flex;
align-items: center;
gap: 6px;
}
.feature-badge {
display: flex;
align-items: center;
justify-content: center;
width: 24px;
height: 24px;
border-radius: 4px;
cursor: pointer;
&.kg {
background: rgba(124, 77, 255, 0.1);
color: #7c4dff;
border: 1px solid rgba(124, 77, 255, 0.2);
&:hover {
background: rgba(124, 77, 255, 0.15);
}
}
&.multimodal {
background: rgba(255, 152, 0, 0.1);
color: #ff9800;
border: 1px solid rgba(255, 152, 0, 0.2);
&:hover {
background: rgba(255, 152, 0, 0.15);
}
}
}
.card-time {
color: #00000066;
font-family: "PingFang SC";
frontend/src/views/knowledge/settings/KBAdvancedSettings.vue
+248 -8
View File
@@ -219,11 +219,11 @@
<t-alert
v-if="!isGraphDatabaseEnabled"
theme="warning"
:message="$t('knowledgeEditor.advanced.graph.disabledWarning')"
style="margin-top: 8px;"
>
<template #operation>
<t-link theme="primary" @click="handleOpenSystemInfo">
<template #message>
<div>{{ $t('knowledgeEditor.advanced.graph.disabledWarning') }}</div>
<t-link class="graph-guide-link" theme="primary" @click="handleOpenGraphGuide">
{{ $t('knowledgeEditor.advanced.graph.howToEnable') }}
</t-link>
</template>
@@ -275,6 +275,132 @@
/>
</div>
</div>
<!-- Nodes Configuration -->
<div class="subsection-header">
<h4>{{ $t('knowledgeEditor.advanced.graph.nodesLabel') }}</h4>
<p class="subsection-desc">{{ $t('knowledgeEditor.advanced.graph.nodesDescription') }}</p>
</div>
<div v-for="(node, index) in localNodeExtract.nodes" :key="index" class="config-item">
<div class="config-item-header">
<span class="config-item-title">节点 #{{ index + 1 }}</span>
<t-button
theme="danger"
variant="text"
size="small"
@click="removeNode(index)"
>
{{ $t('knowledgeEditor.advanced.graph.deleteNode') }}
</t-button>
</div>
<div class="config-item-body">
<div class="config-field">
<label>{{ $t('knowledgeEditor.advanced.graph.nodeNameLabel') }}</label>
<t-input
v-model="node.name"
:placeholder="$t('knowledgeEditor.advanced.graph.nodeNamePlaceholder')"
@change="handleConfigChange"
/>
</div>
<div class="config-field">
<label>{{ $t('knowledgeEditor.advanced.graph.nodeChunksLabel') }}</label>
<t-tag-input
v-model="node.chunks"
:placeholder="$t('knowledgeEditor.advanced.graph.nodeChunksPlaceholder')"
@change="handleConfigChange"
/>
</div>
<div class="config-field">
<label>{{ $t('knowledgeEditor.advanced.graph.nodeAttributesLabel') }}</label>
<t-tag-input
v-model="node.attributes"
:placeholder="$t('knowledgeEditor.advanced.graph.nodeAttributesPlaceholder')"
@change="handleConfigChange"
/>
</div>
</div>
</div>
<t-button
theme="primary"
variant="outline"
@click="addNode"
style="margin-top: 12px;"
>
<template #icon>
<svg width="16" height="16" viewBox="0 0 16 16" fill="currentColor">
<path d="M8 2a1 1 0 011 1v4h4a1 1 0 110 2H9v4a1 1 0 11-2 0V9H3a1 1 0 110-2h4V3a1 1 0 011-1z"/>
</svg>
</template>
{{ $t('knowledgeEditor.advanced.graph.addNode') }}
</t-button>
<!-- Relations Configuration -->
<div class="subsection-header" style="margin-top: 24px;">
<h4>{{ $t('knowledgeEditor.advanced.graph.relationsLabel') }}</h4>
<p class="subsection-desc">{{ $t('knowledgeEditor.advanced.graph.relationsDescription') }}</p>
</div>
<div v-for="(relation, index) in localNodeExtract.relations" :key="index" class="config-item">
<div class="config-item-header">
<span class="config-item-title">关系 #{{ index + 1 }}</span>
<t-button
theme="danger"
variant="text"
size="small"
@click="removeRelation(index)"
>
{{ $t('knowledgeEditor.advanced.graph.deleteRelation') }}
</t-button>
</div>
<div class="config-item-body">
<div class="config-field">
<label>{{ $t('knowledgeEditor.advanced.graph.relationNode1Label') }}</label>
<t-input
v-model="relation.node1"
:placeholder="$t('knowledgeEditor.advanced.graph.relationNode1Placeholder')"
@change="handleConfigChange"
/>
</div>
<div class="config-field">
<label>{{ $t('knowledgeEditor.advanced.graph.relationNode2Label') }}</label>
<t-input
v-model="relation.node2"
:placeholder="$t('knowledgeEditor.advanced.graph.relationNode2Placeholder')"
@change="handleConfigChange"
/>
</div>
<div class="config-field">
<label>{{ $t('knowledgeEditor.advanced.graph.relationTypeLabel') }}</label>
<t-input
v-model="relation.type"
:placeholder="$t('knowledgeEditor.advanced.graph.relationTypePlaceholder')"
@change="handleConfigChange"
/>
</div>
</div>
</div>
<t-button
theme="primary"
variant="outline"
@click="addRelation"
style="margin-top: 12px;"
>
<template #icon>
<svg width="16" height="16" viewBox="0 0 16 16" fill="currentColor">
<path d="M8 2a1 1 0 011 1v4h4a1 1 0 110 2H9v4a1 1 0 11-2 0V9H3a1 1 0 110-2h4V3a1 1 0 011-1z"/>
</svg>
</template>
{{ $t('knowledgeEditor.advanced.graph.addRelation') }}
</t-button>
</div>
</div>
</div>
@@ -311,6 +437,16 @@ interface NodeExtractConfig {
enabled: boolean
text: string
tags: string[]
nodes: Array<{
name: string
chunks: string[]
attributes: string[]
}>
relations: Array<{
node1: string
node2: string
type: string
}>
}
interface Props {
@@ -327,7 +463,11 @@ const emit = defineEmits<{
}>()
const localMultimodal = ref<MultimodalConfig>({ ...props.multimodal })
const localNodeExtract = ref<NodeExtractConfig>({ ...props.nodeExtract })
const localNodeExtract = ref<NodeExtractConfig>({
...props.nodeExtract,
nodes: props.nodeExtract.nodes || [],
relations: props.nodeExtract.relations || []
})
const vllmSelectorRef = ref()
const isGraphDatabaseEnabled = ref(false)
@@ -391,7 +531,11 @@ watch(() => props.multimodal, (newVal) => {
}, { deep: true })
watch(() => props.nodeExtract, (newVal) => {
localNodeExtract.value = { ...newVal }
localNodeExtract.value = {
...newVal,
nodes: newVal.nodes || [],
relations: newVal.relations || []
}
}, { deep: true })
// Handle multimodal toggle
@@ -436,6 +580,10 @@ const handleAddModel = (subSection: string) => {
uiStore.openSettings('models', subSection)
}
const graphGuideUrl =
import.meta.env.VITE_KG_GUIDE_URL ||
'https://github.com/Tencent/WeKnora/blob/main/docs/%E5%BC%80%E5%90%AF%E7%9F%A5%E8%AF%86%E5%9B%BE%E8%B0%B1%E5%8A%9F%E8%83%BD.md'
// Handle knowledge graph toggle
const handleNodeExtractToggle = () => {
// Prevent enabling if graph database is not enabled
@@ -446,9 +594,9 @@ const handleNodeExtractToggle = () => {
emit('update:nodeExtract', localNodeExtract.value)
}
// Open system info page to show how to enable graph database
const handleOpenSystemInfo = () => {
uiStore.openSettings('system')
// Open guide documentation to show how to enable graph database
const handleOpenGraphGuide = () => {
window.open(graphGuideUrl, '_blank', 'noopener')
}
// Handle configuration change
@@ -457,6 +605,44 @@ const handleConfigChange = () => {
emit('update:nodeExtract', localNodeExtract.value)
}
// Add a new node
const addNode = () => {
if (!localNodeExtract.value.nodes) {
localNodeExtract.value.nodes = []
}
localNodeExtract.value.nodes.push({
name: '',
chunks: [],
attributes: []
})
handleConfigChange()
}
// Remove a node
const removeNode = (index: number) => {
localNodeExtract.value.nodes.splice(index, 1)
handleConfigChange()
}
// Add a new relation
const addRelation = () => {
if (!localNodeExtract.value.relations) {
localNodeExtract.value.relations = []
}
localNodeExtract.value.relations.push({
node1: '',
node2: '',
type: ''
})
handleConfigChange()
}
// Remove a relation
const removeRelation = (index: number) => {
localNodeExtract.value.relations.splice(index, 1)
handleConfigChange()
}
// The allModels prop keeps model options in sync
</script>
@@ -530,6 +716,11 @@ const handleConfigChange = () => {
align-items: center;
}
.graph-guide-link {
display: inline-block;
margin-top: 8px;
}
.subsection {
padding: 16px 20px;
margin: 12px 0 0 0;
@@ -570,5 +761,54 @@ const handleConfigChange = () => {
.storage-config {
margin-top: 8px;
}
.config-item {
background: #ffffff;
border: 1px solid #e5e7eb;
border-radius: 6px;
padding: 16px;
margin-bottom: 12px;
}
.config-item-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 16px;
padding-bottom: 12px;
border-bottom: 1px solid #f0f0f0;
}
.config-item-title {
font-size: 14px;
font-weight: 600;
color: #333333;
}
.config-item-body {
display: flex;
flex-direction: column;
gap: 12px;
}
.config-field {
display: flex;
flex-direction: column;
gap: 6px;
label {
font-size: 13px;
font-weight: 500;
color: #555555;
}
}
.subsection-desc {
font-size: 13px;
color: #666666;
margin: 4px 0 8px 8px;
line-height: 1.5;
}
</style>
internal/agent/tools/database_query.go
+2 -1
View File
@@ -36,7 +36,8 @@ func NewDatabaseQueryTool(db *gorm.DB, tenantID uint) *DatabaseQueryTool {
- name (VARCHAR): Knowledge base name
- description (TEXT): Description
- tenant_id (INTEGER): Owner tenant ID
- embedding_model_id, summary_model_id, rerank_model_id, vlm_model_id (VARCHAR): Model IDs
- embedding_model_id, summary_model_id, rerank_model_id (VARCHAR): Model IDs
- vlm_config (JSON): Includes VLM settings such as enabled flag and model_id
- created_at, updated_at (TIMESTAMP)
### knowledges (documents)
internal/agent/tools/get_document_info.go
+24 -21
View File
@@ -133,11 +133,19 @@ func (t *GetDocumentInfoTool) Execute(ctx context.Context, args map[string]inter
}
// Get chunk count
chunks, err := t.chunkService.GetRepository().ListChunksByKnowledgeID(ctx, t.tenantID, id)
chunkCount := 0
if err == nil {
chunkCount = len(chunks)
_, total, err := t.chunkService.GetRepository().ListPagedChunksByKnowledgeID(ctx, t.tenantID, id, &types.Pagination{
Page: 1,
PageSize: 1000,
}, []types.ChunkType{"text"}, "")
if err != nil {
mu.Lock()
results[id] = &docInfo{
err: fmt.Errorf("无法获取文档信息: %v", err),
}
mu.Unlock()
return
}
chunkCount := int(total)
mu.Lock()
results[id] = &docInfo{
@@ -220,26 +228,21 @@ func (t *GetDocumentInfoTool) Execute(ctx context.Context, args map[string]inter
output += "\n"
formattedDocs = append(formattedDocs, map[string]interface{}{
"knowledge_id": k.ID,
"title": k.Title,
"description": k.Description,
"type": k.Type,
"source": k.Source,
"file_name": k.FileName,
"file_type": k.FileType,
"file_size": k.FileSize,
"parse_status": k.ParseStatus,
"chunk_count_min": doc.chunkCount,
"metadata": k.GetMetadata(),
"type_icon": typeIcon,
"knowledge_id": k.ID,
"title": k.Title,
"description": k.Description,
"type": k.Type,
"source": k.Source,
"file_name": k.FileName,
"file_type": k.FileType,
"file_size": k.FileSize,
"parse_status": k.ParseStatus,
"chunk_count": doc.chunkCount,
"metadata": k.GetMetadata(),
"type_icon": typeIcon,
})
}
output += "=== Usage Tips ===\n"
output += "- Use knowledge_search to search document content\n"
output += "- Use list_knowledge_chunks to view context and related chunks\n"
output += "- Search results already contain full chunk content\n"
// Extract first document title for summary
var firstTitle string
if len(successDocs) > 0 && successDocs[0].knowledge != nil {
internal/application/service/knowledge.go
+105 -30
View File
@@ -156,9 +156,9 @@ func (s *knowledgeService) CreateKnowledgeFromFile(ctx context.Context,
}
// 检查VLM配置
if kb.VLMConfig.ModelName == "" || kb.VLMConfig.BaseURL == "" {
logger.Error(ctx, "VLM configuration incomplete for image multimodal processing")
return nil, werrors.NewBadRequestError("上传图片文件需要完整的VLM配置信息, 请前往系统设置页面进行补全")
if !kb.VLMConfig.Enabled || kb.VLMConfig.ModelID == "" {
logger.Error(ctx, "VLM model is not configured")
return nil, werrors.NewBadRequestError("上传图片文件需要设置VLM模型")
}
logger.Info(ctx, "Image multimodal configuration validation passed")
@@ -270,10 +270,7 @@ func (s *knowledgeService) CreateKnowledgeFromFile(ctx context.Context,
// Process document asynchronously
logger.Info(ctx, "Starting asynchronous document processing")
newCtx := logger.CloneContext(ctx)
if enableMultimodel == nil {
enableMultimodel = &kb.ChunkingConfig.EnableMultimodal
}
go s.processDocument(newCtx, kb, knowledge, file, *enableMultimodel)
go s.processDocument(newCtx, kb, knowledge, file, kb.VLMConfig.Enabled)
logger.Infof(ctx, "Knowledge from file created successfully, ID: %s", knowledge.ID)
return knowledge, nil
@@ -358,11 +355,8 @@ func (s *knowledgeService) CreateKnowledgeFromURL(ctx context.Context,
// Process URL asynchronously
logger.Info(ctx, "Starting asynchronous URL processing")
if enableMultimodel == nil {
enableMultimodel = &kb.ChunkingConfig.EnableMultimodal
}
newCtx := logger.CloneContext(ctx)
go s.processDocumentFromURL(newCtx, kb, knowledge, url, *enableMultimodel)
go s.processDocumentFromURL(newCtx, kb, knowledge, url, kb.VLMConfig.Enabled)
logger.Infof(ctx, "Knowledge from URL created successfully, ID: %s", knowledge.ID)
return knowledge, nil
@@ -856,6 +850,31 @@ func (s *knowledgeService) processDocument(ctx context.Context,
// Split file into chunks using document reader service
logger.GetLogger(ctx).Infof("processDocument split file content size: %d, file name: %s, file type: %s, separators: %v, enable multimodal: %v",
len(contentBytes), knowledge.FileName, knowledge.FileType, kb.ChunkingConfig.Separators, enableMultimodel)
var vlmConfig *proto.VLMConfig
if enableMultimodel {
vlmConfig, err = s.getVLMProtoConfig(ctx, kb)
if err != nil {
logger.GetLogger(ctx).WithField("knowledge_id", knowledge.ID).
WithField("error", err).Errorf("processDocument build VLM config failed")
knowledge.ParseStatus = "failed"
knowledge.ErrorMessage = err.Error()
knowledge.UpdatedAt = time.Now()
s.repo.UpdateKnowledge(ctx, knowledge)
span.RecordError(err)
return
}
if vlmConfig == nil {
logger.GetLogger(ctx).WithField("knowledge_id", knowledge.ID).
Error("processDocument enable multimodal but VLM config missing")
knowledge.ParseStatus = "failed"
knowledge.ErrorMessage = "VLM 配置缺失"
knowledge.UpdatedAt = time.Now()
s.repo.UpdateKnowledge(ctx, knowledge)
span.RecordError(errors.New("vlm config missing"))
return
}
}
resp, err := s.docReaderClient.ReadFromFile(ctx, &proto.ReadFromFileRequest{
FileContent: contentBytes,
FileName: knowledge.FileName,
@@ -874,12 +893,7 @@ func (s *knowledgeService) processDocument(ctx context.Context,
AppId: kb.StorageConfig.AppID,
PathPrefix: kb.StorageConfig.PathPrefix,
},
VlmConfig: &proto.VLMConfig{
ModelName: kb.VLMConfig.ModelName,
BaseUrl: kb.VLMConfig.BaseURL,
ApiKey: kb.VLMConfig.APIKey,
InterfaceType: kb.VLMConfig.InterfaceType,
},
VlmConfig: vlmConfig,
},
RequestId: ctx.Value(types.RequestIDContextKey).(string),
})
@@ -912,6 +926,30 @@ func (s *knowledgeService) processDocumentFromURL(ctx context.Context,
logger.GetLogger(ctx).Infof("processDocumentFromURL enableMultimodel: %v", enableMultimodel)
// Fetch and chunk content from URL
var vlmConfig *proto.VLMConfig
if enableMultimodel {
cfg, cfgErr := s.getVLMProtoConfig(ctx, kb)
if cfgErr != nil {
logger.GetLogger(ctx).WithField("knowledge_id", knowledge.ID).
WithField("error", cfgErr).Errorf("processDocumentFromURL build VLM config failed")
knowledge.ParseStatus = "failed"
knowledge.ErrorMessage = cfgErr.Error()
knowledge.UpdatedAt = time.Now()
s.repo.UpdateKnowledge(ctx, knowledge)
return
}
if cfg == nil {
logger.GetLogger(ctx).WithField("knowledge_id", knowledge.ID).
Error("processDocumentFromURL enable multimodal but VLM config missing")
knowledge.ParseStatus = "failed"
knowledge.ErrorMessage = "VLM 配置缺失"
knowledge.UpdatedAt = time.Now()
s.repo.UpdateKnowledge(ctx, knowledge)
return
}
vlmConfig = cfg
}
resp, err := s.docReaderClient.ReadFromURL(ctx, &proto.ReadFromURLRequest{
Url: url,
Title: knowledge.Title,
@@ -929,12 +967,7 @@ func (s *knowledgeService) processDocumentFromURL(ctx context.Context,
AppId: kb.StorageConfig.AppID,
PathPrefix: kb.StorageConfig.PathPrefix,
},
VlmConfig: &proto.VLMConfig{
ModelName: kb.VLMConfig.ModelName,
BaseUrl: kb.VLMConfig.BaseURL,
ApiKey: kb.VLMConfig.APIKey,
InterfaceType: kb.VLMConfig.InterfaceType,
},
VlmConfig: vlmConfig,
},
RequestId: ctx.Value(types.RequestIDContextKey).(string),
})
@@ -3065,11 +3098,35 @@ func (s *knowledgeService) triggerManualProcessing(ctx context.Context,
fileType := "md"
// 检查是否需要启用多模态（对于手动内容通常不需要，但保持一致性）
enableMultimodel := kb.ChunkingConfig.EnableMultimodal && kb.StorageConfig.Provider != ""
enableMultimodel := kb.VLMConfig.Enabled && kb.StorageConfig.Provider != ""
logger.GetLogger(ctx).Infof("triggerManualProcessing split manual content size: %d, file name: %s, file type: %s, separators: %v, enable multimodal: %v",
len(contentBytes), fileName, fileType, kb.ChunkingConfig.Separators, enableMultimodel)
var vlmConfig *proto.VLMConfig
if enableMultimodel {
cfg, cfgErr := s.getVLMProtoConfig(ctx, kb)
if cfgErr != nil {
logger.GetLogger(ctx).WithField("knowledge_id", knowledge.ID).
WithField("error", cfgErr).Errorf("triggerManualProcessing build VLM config failed")
knowledge.ParseStatus = "failed"
knowledge.ErrorMessage = cfgErr.Error()
knowledge.UpdatedAt = time.Now()
s.repo.UpdateKnowledge(ctx, knowledge)
return
}
if cfg == nil {
logger.GetLogger(ctx).WithField("knowledge_id", knowledge.ID).
Error("triggerManualProcessing enable multimodal but VLM config missing")
knowledge.ParseStatus = "failed"
knowledge.ErrorMessage = "VLM 配置缺失"
knowledge.UpdatedAt = time.Now()
s.repo.UpdateKnowledge(ctx, knowledge)
return
}
vlmConfig = cfg
}
// 调用 docreader 解析 markdown 内容
resp, err := s.docReaderClient.ReadFromFile(ctx, &proto.ReadFromFileRequest{
FileContent: contentBytes,
@@ -3089,12 +3146,7 @@ func (s *knowledgeService) triggerManualProcessing(ctx context.Context,
AppId: kb.StorageConfig.AppID,
PathPrefix: kb.StorageConfig.PathPrefix,
},
VlmConfig: &proto.VLMConfig{
ModelName: kb.VLMConfig.ModelName,
BaseUrl: kb.VLMConfig.BaseURL,
ApiKey: kb.VLMConfig.APIKey,
InterfaceType: kb.VLMConfig.InterfaceType,
},
VlmConfig: vlmConfig,
},
RequestId: ctx.Value(types.RequestIDContextKey).(string),
})
@@ -3173,6 +3225,29 @@ func (s *knowledgeService) cleanupKnowledgeResources(ctx context.Context, knowle
return cleanupErr
}
func (s *knowledgeService) getVLMProtoConfig(ctx context.Context, kb *types.KnowledgeBase) (*proto.VLMConfig, error) {
if kb == nil || !kb.VLMConfig.Enabled || kb.VLMConfig.ModelID == "" {
return nil, nil
}
model, err := s.modelService.GetModelByID(ctx, kb.VLMConfig.ModelID)
if err != nil {
return nil, err
}
interfaceType := model.Parameters.InterfaceType
if interfaceType == "" {
interfaceType = "openai"
}
return &proto.VLMConfig{
ModelName: model.Name,
BaseUrl: model.Parameters.BaseURL,
ApiKey: model.Parameters.APIKey,
InterfaceType: interfaceType,
}, nil
}
func IsImageType(fileType string) bool {
switch fileType {
case "jpg", "jpeg", "png", "gif", "webp", "bmp", "svg", "tiff":
internal/application/service/knowledgebase.go
+11 -11
View File
@@ -272,17 +272,17 @@ func (s *knowledgeBaseService) CopyKnowledgeBase(ctx context.Context,
targetKB = &types.KnowledgeBase{
ID: uuid.New().String(),
Name: sourceKB.Name,
Type: sourceKB.Type,
Description: sourceKB.Description,
TenantID: tenantID,
ChunkingConfig: sourceKB.ChunkingConfig,
ImageProcessingConfig: sourceKB.ImageProcessingConfig,
EmbeddingModelID: sourceKB.EmbeddingModelID,
SummaryModelID: sourceKB.SummaryModelID,
VLMModelID: sourceKB.VLMModelID,
StorageConfig: sourceKB.StorageConfig,
FAQConfig: faqConfig,
}
Type: sourceKB.Type,
Description: sourceKB.Description,
TenantID: tenantID,
ChunkingConfig: sourceKB.ChunkingConfig,
ImageProcessingConfig: sourceKB.ImageProcessingConfig,
EmbeddingModelID: sourceKB.EmbeddingModelID,
SummaryModelID: sourceKB.SummaryModelID,
VLMConfig: sourceKB.VLMConfig,
StorageConfig: sourceKB.StorageConfig,
FAQConfig: faqConfig,
}
targetKB.EnsureDefaults()
if err := s.repo.CreateKnowledgeBase(ctx, targetKB); err != nil {
return nil, nil, err
internal/handler/initialization.go
+58 -53
View File
@@ -85,9 +85,9 @@ func NewInitializationHandler(
// KBModelConfigRequest 知识库模型配置请求（简化版，只传模型ID）
type KBModelConfigRequest struct {
LLMModelID string `json:"llmModelId" binding:"required"`
EmbeddingModelID string `json:"embeddingModelId" binding:"required"`
VLLMModelID string `json:"vllmModelId"` // 可选
LLMModelID string `json:"llmModelId" binding:"required"`
EmbeddingModelID string `json:"embeddingModelId" binding:"required"`
VLMConfig *types.VLMConfig `json:"vlm_config"`
// 文档分块配置
DocumentSplitting struct {
@@ -249,17 +249,19 @@ func (h *InitializationHandler) UpdateKBConfig(c *gin.Context) {
kb.SummaryModelID = req.LLMModelID
kb.EmbeddingModelID = req.EmbeddingModelID
// 处理可选的VLLM模型
if req.VLLMModelID != "" {
vllmModel, err := h.modelService.GetModelByID(ctx, req.VLLMModelID)
// 处理多模态模型配置
kb.VLMConfig = types.VLMConfig{}
if req.VLMConfig != nil && req.Multimodal.Enabled && req.VLMConfig.ModelID != "" {
vllmModel, err := h.modelService.GetModelByID(ctx, req.VLMConfig.ModelID)
if err != nil || vllmModel == nil {
logger.Warn(ctx, "VLLM model not found", "modelId", req.VLLMModelID)
logger.Warn(ctx, "VLM model not found", "modelId", req.VLMConfig.ModelID)
} else {
// 只存储模型ID
kb.VLMModelID = req.VLLMModelID
kb.VLMConfig.Enabled = req.VLMConfig.Enabled
kb.VLMConfig.ModelID = req.VLMConfig.ModelID
}
} else {
kb.VLMModelID = ""
}
if !kb.VLMConfig.Enabled {
kb.VLMConfig.ModelID = ""
}
// 更新文档分块配置
@@ -432,13 +434,14 @@ func (h *InitializationHandler) InitializeByKB(c *gin.Context) {
// 处理模型创建/更新
modelsToProcess := []struct {
modelType types.ModelType
name string
source types.ModelSource
description string
baseURL string
apiKey string
dimension int
modelType types.ModelType
name string
source types.ModelSource
description string
baseURL string
apiKey string
dimension int
interfaceType string
}{
{
modelType: types.ModelTypeKnowledgeQA,
@@ -462,13 +465,14 @@ func (h *InitializationHandler) InitializeByKB(c *gin.Context) {
// 如果启用Rerank，添加Rerank模型
if req.Rerank.Enabled {
modelsToProcess = append(modelsToProcess, struct {
modelType types.ModelType
name string
source types.ModelSource
description string
baseURL string
apiKey string
dimension int
modelType types.ModelType
name string
source types.ModelSource
description string
baseURL string
apiKey string
dimension int
interfaceType string
}{
modelType: types.ModelTypeRerank,
name: req.Rerank.ModelName,
@@ -482,20 +486,22 @@ func (h *InitializationHandler) InitializeByKB(c *gin.Context) {
// 如果启用多模态，添加VLM模型
if req.Multimodal.Enabled && req.Multimodal.VLM != nil {
modelsToProcess = append(modelsToProcess, struct {
modelType types.ModelType
name string
source types.ModelSource
description string
baseURL string
apiKey string
dimension int
modelType types.ModelType
name string
source types.ModelSource
description string
baseURL string
apiKey string
dimension int
interfaceType string
}{
modelType: types.ModelTypeVLLM,
name: req.Multimodal.VLM.ModelName,
source: types.ModelSourceRemote,
description: "VLM Model",
baseURL: req.Multimodal.VLM.BaseURL,
apiKey: req.Multimodal.VLM.APIKey,
modelType: types.ModelTypeVLLM,
name: req.Multimodal.VLM.ModelName,
source: types.ModelSourceRemote,
description: "VLM Model",
baseURL: req.Multimodal.VLM.BaseURL,
apiKey: req.Multimodal.VLM.APIKey,
interfaceType: req.Multimodal.VLM.InterfaceType,
})
}
@@ -508,8 +514,9 @@ func (h *InitializationHandler) InitializeByKB(c *gin.Context) {
Source: modelInfo.source,
Description: modelInfo.description,
Parameters: types.ModelParameters{
BaseURL: modelInfo.baseURL,
APIKey: modelInfo.apiKey,
BaseURL: modelInfo.baseURL,
APIKey: modelInfo.apiKey,
InterfaceType: modelInfo.interfaceType,
},
IsDefault: false,
Status: types.ModelStatusActive,
@@ -529,7 +536,7 @@ func (h *InitializationHandler) InitializeByKB(c *gin.Context) {
case types.ModelTypeKnowledgeQA:
existingModelID = kb.SummaryModelID
case types.ModelTypeVLLM:
existingModelID = kb.VLMModelID
existingModelID = kb.VLMConfig.ModelID
}
// 如果知识库中已经有对应类型的模型ID，尝试获取并更新
@@ -603,13 +610,9 @@ func (h *InitializationHandler) InitializeByKB(c *gin.Context) {
// 更新多模态配置
if req.Multimodal.Enabled {
kb.ChunkingConfig.EnableMultimodal = true
kb.VLMModelID = vlmModelID
kb.VLMConfig = types.VLMConfig{
ModelName: req.Multimodal.VLM.ModelName,
BaseURL: req.Multimodal.VLM.BaseURL,
APIKey: req.Multimodal.VLM.APIKey,
InterfaceType: req.Multimodal.VLM.InterfaceType,
Enabled: req.Multimodal.Enabled,
ModelID: vlmModelID,
}
switch req.Multimodal.StorageType {
case "cos":
@@ -636,7 +639,6 @@ func (h *InitializationHandler) InitializeByKB(c *gin.Context) {
}
}
} else {
kb.VLMModelID = ""
kb.VLMConfig = types.VLMConfig{}
kb.StorageConfig = types.StorageConfig{}
}
@@ -1083,7 +1085,7 @@ func (h *InitializationHandler) GetCurrentConfigByKB(c *gin.Context) {
modelIDs := []string{
kb.EmbeddingModelID,
kb.SummaryModelID,
kb.VLMModelID,
kb.VLMConfig.ModelID,
}
for _, modelID := range modelIDs {
@@ -1170,15 +1172,18 @@ func (h *InitializationHandler) buildConfigResponse(ctx context.Context, models
}
multimodal := config["multimodal"].(map[string]interface{})
multimodal["vlm"] = map[string]interface{}{
"modelName": model.Name,
"baseUrl": baseURL,
"apiKey": apiKey,
"modelName": model.Name,
"baseUrl": baseURL,
"apiKey": apiKey,
"interfaceType": model.Parameters.InterfaceType,
"modelId": model.ID,
}
}
}
// 判断多模态是否启用：有VLM模型ID或有存储配置
hasMultimodal := (kb.VLMModelID != "" || kb.StorageConfig.SecretID != "" || kb.StorageConfig.BucketName != "")
hasMultimodal := ((kb.VLMConfig.Enabled && kb.VLMConfig.ModelID != "") ||
kb.StorageConfig.SecretID != "" || kb.StorageConfig.BucketName != "")
if config["multimodal"] == nil {
config["multimodal"] = map[string]interface{}{
"enabled": hasMultimodal,
internal/types/knowledgebase.go
+2 -16
View File
@@ -8,10 +8,6 @@ import (
"gorm.io/gorm"
)
const (
InitDefaultKnowledgeBaseID = "kb-00000001"
)
const (
KnowledgeBaseTypeDocument = "document"
KnowledgeBaseTypeFAQ = "faq"
@@ -59,8 +55,6 @@ type KnowledgeBase struct {
EmbeddingModelID string `yaml:"embedding_model_id" json:"embedding_model_id"`
// Summary model ID
SummaryModelID string `yaml:"summary_model_id" json:"summary_model_id"`
// VLM model ID
VLMModelID string `yaml:"vlm_model_id" json:"vlm_model_id"`
// VLM config
VLMConfig VLMConfig `yaml:"vlm_config" json:"vlm_config" gorm:"type:json"`
// Storage config
@@ -95,8 +89,6 @@ type ChunkingConfig struct {
ChunkOverlap int `yaml:"chunk_overlap" json:"chunk_overlap"`
// Separators
Separators []string `yaml:"separators" json:"separators"`
// Enable multimodal
EnableMultimodal bool `yaml:"enable_multimodal" json:"enable_multimodal"`
}
// COSConfig represents the COS configuration
@@ -174,14 +166,8 @@ func (c *ImageProcessingConfig) Scan(value interface{}) error {
// VLMConfig represents the VLM configuration
type VLMConfig struct {
// Model Name
ModelName string `yaml:"model_name" json:"model_name"`
// Base URL
BaseURL string `yaml:"base_url" json:"base_url"`
// API Key
APIKey string `yaml:"api_key" json:"api_key"`
// Interface Type: "ollama" or "openai"
InterfaceType string `yaml:"interface_type" json:"interface_type"`
Enabled bool `yaml:"enabled" json:"enabled"`
ModelID string `yaml:"model_id" json:"model_id"`
}
// Value implements the driver.Valuer interface, used to convert VLMConfig to database value
internal/types/model.go
+1
View File
@@ -44,6 +44,7 @@ type EmbeddingParameters struct {
type ModelParameters struct {
BaseURL string `yaml:"base_url" json:"base_url"`
APIKey string `yaml:"api_key" json:"api_key"`
InterfaceType string `yaml:"interface_type" json:"interface_type"`
EmbeddingParameters EmbeddingParameters `yaml:"embedding_parameters" json:"embedding_parameters"`
ParameterSize string `yaml:"parameter_size" json:"parameter_size"` // Ollama model parameter size (e.g., "7B", "13B", "70B")
}
migrations/mysql/00-init-db.sql
-1
View File
@@ -49,7 +49,6 @@ CREATE TABLE knowledge_bases (
embedding_model_id VARCHAR(64) NOT NULL,
summary_model_id VARCHAR(64) NOT NULL,
rerank_model_id VARCHAR(64) NOT NULL,
vlm_model_id VARCHAR(64) NOT NULL,
cos_config JSON NOT NULL,
vlm_config JSON NOT NULL,
extract_config JSON NULL,
migrations/mysql/03-cleanup-unreferenced-models.sql
+20 -6
View File
@@ -5,7 +5,7 @@
-- ============================================================================
-- This script removes models from the models table that meet ALL of these conditions:
-- 1. Not referenced by any knowledge_bases (embedding_model_id, summary_model_id, rerank_model_id, vlm_model_id)
-- 1. Not referenced by any knowledge_bases (embedding_model_id, summary_model_id, rerank_model_id, vlm_config.model_id)
-- 2. Not referenced by any knowledges (embedding_model_id)
-- 3. Not a default model (is_default = false)
-- 4. Not a system model (tenant_id != 0)
@@ -38,7 +38,11 @@ WHERE m.deleted_at IS NULL
UNION
SELECT DISTINCT rerank_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND rerank_model_id != ''
UNION
SELECT DISTINCT vlm_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND vlm_model_id != ''
SELECT DISTINCT JSON_UNQUOTE(JSON_EXTRACT(vlm_config, '$.model_id')) AS vlm_model_id
FROM knowledge_bases
WHERE deleted_at IS NULL
AND JSON_EXTRACT(vlm_config, '$.model_id') IS NOT NULL
AND JSON_UNQUOTE(JSON_EXTRACT(vlm_config, '$.model_id')) != ''
UNION
-- Models referenced by knowledges
SELECT DISTINCT embedding_model_id FROM knowledges WHERE deleted_at IS NULL AND embedding_model_id IS NOT NULL AND embedding_model_id != ''
@@ -61,7 +65,11 @@ ORDER BY m.created_at DESC;
-- UNION
-- SELECT DISTINCT rerank_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND rerank_model_id != ''
-- UNION
-- SELECT DISTINCT vlm_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND vlm_model_id != ''
-- SELECT DISTINCT JSON_UNQUOTE(JSON_EXTRACT(vlm_config, '$.model_id')) AS vlm_model_id
-- FROM knowledge_bases
-- WHERE deleted_at IS NULL
-- AND JSON_EXTRACT(vlm_config, '$.model_id') IS NOT NULL
-- AND JSON_UNQUOTE(JSON_EXTRACT(vlm_config, '$.model_id')) != ''
-- UNION
-- SELECT DISTINCT embedding_model_id FROM knowledges WHERE deleted_at IS NULL AND embedding_model_id IS NOT NULL AND embedding_model_id != ''
-- );
@@ -83,7 +91,11 @@ ORDER BY m.created_at DESC;
-- UNION
-- SELECT DISTINCT rerank_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND rerank_model_id != ''
-- UNION
-- SELECT DISTINCT vlm_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND vlm_model_id != ''
-- SELECT DISTINCT JSON_UNQUOTE(JSON_EXTRACT(vlm_config, '$.model_id')) AS vlm_model_id
-- FROM knowledge_bases
-- WHERE deleted_at IS NULL
-- AND JSON_EXTRACT(vlm_config, '$.model_id') IS NOT NULL
-- AND JSON_UNQUOTE(JSON_EXTRACT(vlm_config, '$.model_id')) != ''
-- UNION
-- SELECT DISTINCT embedding_model_id FROM knowledges WHERE deleted_at IS NULL AND embedding_model_id IS NOT NULL AND embedding_model_id != ''
-- );
@@ -122,9 +134,11 @@ WHERE deleted_at IS NULL AND rerank_model_id != ''
UNION ALL
SELECT
'vlm_model' as model_type,
COUNT(DISTINCT vlm_model_id) as referenced_count
COUNT(DISTINCT JSON_UNQUOTE(JSON_EXTRACT(vlm_config, '$.model_id'))) as referenced_count
FROM knowledge_bases
WHERE deleted_at IS NULL AND vlm_model_id != '';
WHERE deleted_at IS NULL
AND JSON_EXTRACT(vlm_config, '$.model_id') IS NOT NULL
AND JSON_UNQUOTE(JSON_EXTRACT(vlm_config, '$.model_id')) != '';
-- ============================================================================
-- Rollback Strategy
migrations/paradedb/00-init-db.sql
-1
View File
@@ -62,7 +62,6 @@ CREATE TABLE IF NOT EXISTS knowledge_bases (
embedding_model_id VARCHAR(64) NOT NULL,
summary_model_id VARCHAR(64) NOT NULL,
rerank_model_id VARCHAR(64) NOT NULL,
vlm_model_id VARCHAR(64) NOT NULL,
cos_config JSONB NOT NULL DEFAULT '{}',
vlm_config JSONB NOT NULL DEFAULT '{}',
extract_config JSONB NULL DEFAULT NULL,
migrations/paradedb/03-cleanup-unreferenced-models.sql
+40 -10
View File
@@ -5,7 +5,7 @@
-- ============================================================================
-- This script removes models from the models table that meet ALL of these conditions:
-- 1. Not referenced by any knowledge_bases (embedding_model_id, summary_model_id, rerank_model_id, vlm_model_id)
-- 1. Not referenced by any knowledge_bases (embedding_model_id, summary_model_id, rerank_model_id, vlm_config.model_id)
-- 2. Not referenced by any knowledges (embedding_model_id)
-- 3. Not a default model (is_default = false)
-- 4. Not a system model (tenant_id != 0)
@@ -37,7 +37,11 @@ WHERE m.deleted_at IS NULL
UNION
SELECT DISTINCT rerank_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND rerank_model_id != ''
UNION
SELECT DISTINCT vlm_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND vlm_model_id != ''
SELECT DISTINCT (vlm_config ->> 'model_id') AS vlm_model_id
FROM knowledge_bases
WHERE deleted_at IS NULL
AND vlm_config ->> 'model_id' IS NOT NULL
AND vlm_config ->> 'model_id' != ''
UNION
-- Models referenced by knowledges
SELECT DISTINCT embedding_model_id FROM knowledges WHERE deleted_at IS NULL AND embedding_model_id IS NOT NULL AND embedding_model_id != ''
@@ -60,7 +64,11 @@ ORDER BY m.created_at DESC;
-- UNION
-- SELECT DISTINCT rerank_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND rerank_model_id != ''
-- UNION
-- SELECT DISTINCT vlm_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND vlm_model_id != ''
-- SELECT DISTINCT (vlm_config ->> 'model_id') AS vlm_model_id
-- FROM knowledge_bases
-- WHERE deleted_at IS NULL
-- AND vlm_config ->> 'model_id' IS NOT NULL
-- AND vlm_config ->> 'model_id' != ''
-- UNION
-- SELECT DISTINCT embedding_model_id FROM knowledges WHERE deleted_at IS NULL AND embedding_model_id IS NOT NULL AND embedding_model_id != ''
-- );
@@ -82,7 +90,11 @@ ORDER BY m.created_at DESC;
-- UNION
-- SELECT DISTINCT rerank_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND rerank_model_id != ''
-- UNION
-- SELECT DISTINCT vlm_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND vlm_model_id != ''
-- SELECT DISTINCT (vlm_config ->> 'model_id') AS vlm_model_id
-- FROM knowledge_bases
-- WHERE deleted_at IS NULL
-- AND vlm_config ->> 'model_id' IS NOT NULL
-- AND vlm_config ->> 'model_id' != ''
-- UNION
-- SELECT DISTINCT embedding_model_id FROM knowledges WHERE deleted_at IS NULL AND embedding_model_id IS NOT NULL AND embedding_model_id != ''
-- );
@@ -100,7 +112,11 @@ WITH referenced_models AS (
UNION
SELECT rerank_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND rerank_model_id != ''
UNION
SELECT vlm_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND vlm_model_id != ''
SELECT vlm_config ->> 'model_id'
FROM knowledge_bases
WHERE deleted_at IS NULL
AND vlm_config ->> 'model_id' IS NOT NULL
AND vlm_config ->> 'model_id' != ''
UNION
SELECT embedding_model_id FROM knowledges WHERE deleted_at IS NULL AND embedding_model_id IS NOT NULL AND embedding_model_id != ''
)
@@ -127,7 +143,11 @@ WITH referenced_models AS (
UNION
SELECT rerank_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND rerank_model_id != ''
UNION
SELECT vlm_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND vlm_model_id != ''
SELECT vlm_config ->> 'model_id'
FROM knowledge_bases
WHERE deleted_at IS NULL
AND vlm_config ->> 'model_id' IS NOT NULL
AND vlm_config ->> 'model_id' != ''
UNION
SELECT embedding_model_id FROM knowledges WHERE deleted_at IS NULL AND embedding_model_id IS NOT NULL AND embedding_model_id != ''
)
@@ -145,7 +165,11 @@ WHERE m.deleted_at IS NULL
-- UNION
-- SELECT rerank_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND rerank_model_id != ''
-- UNION
-- SELECT vlm_model_id FROM knowledge_bases WHERE deleted_at IS NULL AND vlm_model_id != ''
-- SELECT vlm_config ->> 'model_id'
-- FROM knowledge_bases
-- WHERE deleted_at IS NULL
-- AND vlm_config ->> 'model_id' IS NOT NULL
-- AND vlm_config ->> 'model_id' != ''
-- UNION
-- SELECT embedding_model_id FROM knowledges WHERE deleted_at IS NULL AND embedding_model_id IS NOT NULL AND embedding_model_id != ''
-- )
@@ -189,9 +213,11 @@ WHERE deleted_at IS NULL AND rerank_model_id != ''
UNION ALL
SELECT
'vlm_model' as model_type,
COUNT(DISTINCT vlm_model_id) as referenced_count
COUNT(DISTINCT (vlm_config ->> 'model_id')) as referenced_count
FROM knowledge_bases
WHERE deleted_at IS NULL AND vlm_model_id != '';
WHERE deleted_at IS NULL
AND vlm_config ->> 'model_id' IS NOT NULL
AND vlm_config ->> 'model_id' != '';
-- List all referenced models with their reference count
WITH model_references AS (
@@ -201,7 +227,11 @@ WITH model_references AS (
UNION ALL
SELECT rerank_model_id, 'rerank' FROM knowledge_bases WHERE deleted_at IS NULL AND rerank_model_id != ''
UNION ALL
SELECT vlm_model_id, 'vlm' FROM knowledge_bases WHERE deleted_at IS NULL AND vlm_model_id != ''
SELECT vlm_config ->> 'model_id', 'vlm'
FROM knowledge_bases
WHERE deleted_at IS NULL
AND vlm_config ->> 'model_id' IS NOT NULL
AND vlm_config ->> 'model_id' != ''
UNION ALL
SELECT embedding_model_id, 'knowledge_embedding' FROM knowledges WHERE deleted_at IS NULL AND embedding_model_id IS NOT NULL AND embedding_model_id != ''
)