Qdrant 向量数据库完全指南:从入门到 Spring AI/LangChain4J 集成实践
前言
在人工智能和大语言模型(LLM)应用日益普及的今天,向量数据库成为了构建 AI 应用的关键基础设施。Qdrant 作为一款高性能的开源向量数据库,以其卓越的性能、易用性和丰富的功能特性,正在成为越来越多开发者的首选。
本文将详细介绍 Qdrant 的核心特性,并展示如何在 Spring Boot 项目中集成 Qdrant,以及如何配合 Spring AI 和 LangChain 等主流 AI 框架构建智能应用。
一、Qdrant 简介与核心特性
1.1 什么是 Qdrant?
Qdrant(读音:quadrant)是一个用 Rust 编写的开源向量相似度搜索引擎,专门用于存储、搜索和管理向量嵌入(Vector Embeddings)。它提供了高性能的向量搜索能力,支持过滤、负载均衡等功能,非常适合构建推荐系统、语义搜索、AI 助手等应用。
1.2 核心特性
✅ 高性能搜索
- 使用 HNSW(Hierarchical Navigable Small World)算法实现高效的近似最近邻搜索
- 支持实时索引更新,不影响搜索性能
- 单节点可处理数十亿级别的向量
✅ 丰富的过滤能力
- 支持在向量搜索时结合元数据进行过滤
- 提供类似 SQL 的过滤语法
- 支持复杂的布尔查询
✅ 易于部署和扩展
- 提供 Docker、Kubernetes 等多种部署方式
- 支持水平扩展和分片
- 提供 RESTful API 和 gRPC 接口
✅ 企业级特性
- 支持数据持久化
- 提供快照和备份功能
- 内置负载均衡和复制
- 支持访问控制和身份验证
二、Qdrant 快速开始
2.1 使用 Docker 启动 Qdrant
最简单的启动方式是使用 Docker:
# 启动 Qdrant 实例docker run -p6333:6333 -p6334:6334 \-v$(pwd)/qdrant_storage:/qdrant/storage \ qdrant/qdrant # 或者使用 docker-compose version: '3.8' services: qdrant: image: qdrant/qdrant ports: - "6333:6333" - "6334:6334" volumes: - ./qdrant_storage:/qdrant/storage 启动后:
- Web UI:http://localhost:6333/dashboard
- HTTP API:http://localhost:6333
- gRPC:localhost:6334
2.2 基本概念
- Collection(集合):向量的集合,类似于关系数据库的表
- Point(点):单个向量及其关联的 payload(元数据)
- Vector(向量):数值数组,表示数据的嵌入表示
- Payload(负载):与向量关联的元数据,可用于过滤
三、Spring Boot 集成 Qdrant
3.1 添加依赖
在 pom.xml 中添加 Qdrant Java 客户端依赖:
<dependencies><!-- Qdrant Java Client --><dependency><groupId>io.qdrant</groupId><artifactId>qdrant-java-client</artifactId><version>1.7.0</version></dependency><!-- Spring Boot Web (可选,用于构建 REST API) --><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency><!-- Lombok (简化代码) --><dependency><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><optional>true</optional></dependency></dependencies>3.2 配置 Qdrant 客户端
创建配置类来初始化 Qdrant 客户端:
packagecom.example.qdrant.config;importio.qdrant.client.QdrantClient;importio.qdrant.client.QdrantGrpcClient;importorg.springframework.beans.factory.annotation.Value;importorg.springframework.context.annotation.Bean;importorg.springframework.context.annotation.Configuration;@ConfigurationpublicclassQdrantConfig{@Value("${qdrant.host:localhost}")privateString host;@Value("${qdrant.port:6334}")privateint port;@Value("${qdrant.api-key:}")privateString apiKey;@BeanpublicQdrantClientqdrantClient(){QdrantGrpcClient.Builder builder =QdrantGrpcClient.newBuilder().host(host).port(port);if(!apiKey.isEmpty()){ builder.withApiKey(apiKey);}return builder.build();}}在 application.yml 中添加配置:
qdrant:host: localhost port:6334api-key:# 如果设置了 API Keyspring:application:name: qdrant-demo 3.3 创建 Collection 服务
packagecom.example.qdrant.service;importio.qdrant.client.QdrantClient;importio.qdrant.client.grpc.Collections;importio.qdrant.client.grpc.Points;importlombok.RequiredArgsConstructor;importlombok.extern.slf4j.Slf4j;importorg.springframework.stereotype.Service;importjava.util.List;@Slf4j@Service@RequiredArgsConstructorpublicclassQdrantCollectionService{privatefinalQdrantClient qdrantClient;/** * 创建集合 */publicvoidcreateCollection(String collectionName,int vectorSize)throwsException{Collections.VectorParams vectorParams =Collections.VectorParams.newBuilder().setSize(vectorSize).setDistance(Collections.Distance.Cosine).build(); qdrantClient.createCollectionAsync( collectionName,Collections.CreateCollection.newBuilder().setVectorsConfig(vectorParams).build()).get(); log.info("Collection '{}' created successfully", collectionName);}/** * 删除集合 */publicvoiddeleteCollection(String collectionName)throwsException{ qdrantClient.deleteCollectionAsync(collectionName).get(); log.info("Collection '{}' deleted successfully", collectionName);}/** * 检查集合是否存在 */publicbooleancollectionExists(String collectionName)throwsException{Collections.CollectionInfo info = qdrantClient.getCollectionInfoAsync(collectionName).get();return info !=null;}/** * 获取集合信息 */publicCollections.CollectionInfogetCollectionInfo(String collectionName)throwsException{return qdrantClient.getCollectionInfoAsync(collectionName).get();}}3.4 创建 Point 管理服务
packagecom.example.qdrant.service;importio.qdrant.client.QdrantClient;importio.qdrant.client.grpc.Points;importlombok.RequiredArgsConstructor;importlombok.extern.slf4j.Slf4j;importorg.springframework.stereotype.Service;importjava.util.List;importjava.util.Map;@Slf4j@Service@RequiredArgsConstructorpublicclassQdrantPointService{privatefinalQdrantClient qdrantClient;/** * 插入/更新向量点 */publicvoidupsertPoints(String collectionName,List<PointData> points)throwsException{List<Points.PointStruct> pointStructs = points.stream().map(this::convertToPointStruct).toList(); qdrantClient.upsertPointAsync( collectionName,Points.UpsertPoints.newBuilder().addAllPoints(pointStructs).build()).get(); log.info("Successfully upserted {} points to collection '{}'", points.size(), collectionName);}/** * 搜索向量 */publicList<SearchResult>search(String collectionName,List<Float> vector,int limit,Map<String,String> filter)throwsException{Points.SearchPoints.Builder searchBuilder =Points.SearchPoints.newBuilder().addAllVector(vector).setLimit(limit).withVectorSelector(Points.QueryVector.newBuilder().build());// 添加过滤条件if(filter !=null&&!filter.isEmpty()){Points.Filter filterBuilder =Points.Filter.newBuilder().addMust(Points.Condition.newBuilder().setField(Points.FieldCondition.newBuilder().setKey("category").setMatch(Points.Match.newBuilder().setTextValue(filter.get("category")).build()).build()).build()).build(); searchBuilder.setFilter(filterBuilder);}List<Points.RetrievedPoint> results = qdrantClient.searchPointAsync( collectionName, searchBuilder.build()).get();return results.stream().map(this::convertToSearchResult).toList();}/** * 删除点 */publicvoiddeletePoints(String collectionName,List<Long> ids)throwsException{Points.PointsSelector selector =Points.PointsSelector.newBuilder().setPointsSelector(Points.PointsIdsList.newBuilder().addAllIds(ids.stream().map(id ->Points.PointId.newBuilder().setNum(id).build()).toList()).build()).build(); qdrantClient.deletePointAsync(collectionName, selector).get(); log.info("Successfully deleted {} points from collection '{}'", ids.size(), collectionName);}privatePoints.PointStructconvertToPointStruct(PointData pointData){Points.PointStruct.Builder builder =Points.PointStruct.newBuilder().setId(Points.PointId.newBuilder().setNum(pointData.getId()).build()).addAllVector(pointData.getVector());// 添加 payloadif(pointData.getPayload()!=null){ pointData.getPayload().forEach((key, value)->{ builder.putPayload(key,Points.Value.newBuilder().setStringValue(value.toString()).build());});}return builder.build();}privateSearchResultconvertToSearchResult(Points.RetrievedPoint retrievedPoint){returnSearchResult.builder().id(retrievedPoint.getId().getNum()).score(retrievedPoint.getScore()).payload(retrievedPoint.getPayloadMap()).build();}}// DTO 类@Data@BuilderpublicclassPointData{privateLong id;privateList<Float> vector;privateMap<String,Object> payload;}@Data@BuilderpublicclassSearchResult{privateLong id;privatefloat score;privateMap<String,Points.Value> payload;}四、与 Spring AI 集成
Spring AI 是 Spring 生态系统中新兴的 AI 框架,提供了与各种 LLM 和向量数据库集成的统一接口。
4.1 添加依赖
<dependency><groupId>org.springframework.ai</groupId><artifactId>spring-ai-qdrant-spring-boot-starter</artifactId><version>1.0.0-M4</version></dependency><dependency><groupId>org.springframework.ai</groupId><artifactId>spring-ai-openai-spring-boot-starter</artifactId><version>1.0.0-M4</version></dependency>4.2 配置 Spring AI
spring:ai:# OpenAI 配置(用于生成嵌入)openai:api-key: ${OPENAI_API_KEY}# Qdrant 配置vectorstore:qdrant:host: localhost port:6334collection-name: documents initialize-schema:true4.3 创建 RAG 服务
packagecom.example.qdrant.service;importorg.springframework.ai.document.Document;importorg.springframework.ai.qdrant.QdrantVectorStore;importorg.springframework.ai.reader.TextReader;importorg.springframework.ai.transformer.splitter.TokenTextSplitter;importorg.springframework.ai.vectorstore.VectorStore;importorg.springframework.core.io.Resource;importorg.springframework.stereotype.Service;importjava.util.List;importjava.util.Map;@Service@RequiredArgsConstructorpublicclassRAGService{privatefinalVectorStore vectorStore;privatefinalQdrantVectorStore qdrantVectorStore;/** * 加载并存储文档 */publicvoidloadAndStoreDocuments(Resource resource)throwsException{// 读取文档TextReader textReader =newTextReader(resource);List<Document> documents = textReader.get();// 分割文档TokenTextSplitter splitter =newTokenTextSplitter();List<Document> splitDocuments = splitter.apply(documents);// 存储到向量数据库 vectorStore.add(splitDocuments); log.info("Stored {} document chunks in Qdrant", splitDocuments.size());}/** * 相似度搜索 */publicList<Document>similaritySearch(String query,int topK){return vectorStore.similaritySearch(org.springframework.ai.vectorstore.SearchRequest.query(query).withTopK(topK));}/** * 带过滤的相似度搜索 */publicList<Document>similaritySearchWithFilter(String query,int topK,String category){return vectorStore.similaritySearch(org.springframework.ai.vectorstore.SearchRequest.query(query).withTopK(topK).withFilterExpression("category == '"+ category +"'"));}/** * 删除文档 */publicvoiddeleteDocuments(List<String> ids){ vectorStore.delete(ids);}}4.4 创建聊天控制器
packagecom.example.qdrant.controller;importorg.springframework.ai.chat.messages.UserMessage;importorg.springframework.ai.chat.model.ChatResponse;importorg.springframework.ai.chat.prompt.Prompt;importorg.springframework.ai.openai.OpenAiChatModel;importorg.springframework.ai.vectorstore.VectorStore;importorg.springframework.ai.vectorstore.SearchRequest;importorg.springframework.web.bind.annotation.*;@RestController@RequestMapping("/api/chat")@RequiredArgsConstructorpublicclassChatController{privatefinalOpenAiChatModel chatModel;privatefinalVectorStore vectorStore;@PostMappingpublicStringchat(@RequestBodyChatRequest request){// 1. 检索相关文档List<Document> relevantDocs = vectorStore.similaritySearch(SearchRequest.query(request.getMessage()).withTopK(3));// 2. 构建增强提示词String context = relevantDocs.stream().map(Document::getContent).collect(Collectors.joining("\n\n"));String enhancedPrompt =String.format("Based on the following context:\n\n%s\n\nAnswer the question: %s", context, request.getMessage());// 3. 调用 LLMChatResponse response = chatModel.call(newPrompt(newUserMessage(enhancedPrompt)));return response.getResult().getOutput().getContent();}}五、与 LangChain4j 集成
LangChain4j 是 LangChain 的 Java 实现,提供了丰富的 AI 应用构建能力。
5.1 添加依赖
<dependencies><!-- LangChain4j Qdrant --><dependency><groupId>dev.langchain4j</groupId><artifactId>langchain4j-qdrant</artifactId><version>0.34.0</version></dependency><!-- LangChain4j OpenAI --><dependency><groupId>dev.langchain4j</groupId><artifactId>langchain4j-open-ai</artifactId><version>0.34.0</version></dependency></dependencies>5.2 配置 Qdrant Embedding Store
packagecom.example.qdrant.config;importdev.langchain4j.data.segment.TextSegment;importdev.langchain4j.model.openai.OpenAiEmbeddingModel;importdev.langchain4j.model.openai.OpenAiChatModel;importdev.langchain4j.store.embedding.qdrant.QdrantEmbeddingStore;importio.qdrant.client.QdrantClient;importorg.springframework.beans.factory.annotation.Value;importorg.springframework.context.annotation.Bean;importorg.springframework.context.annotation.Configuration;@ConfigurationpublicclassLangChainConfig{@Value("${langchain4j.openai.api-key}")privateString openAiApiKey;@Value("${qdrant.host:localhost}")privateString qdrantHost;@Value("${qdrant.port:6334}")privateint qdrantPort;@BeanpublicQdrantEmbeddingStoreqdrantEmbeddingStore(QdrantClient qdrantClient){returnQdrantEmbeddingStore.builder().host(qdrantHost).port(qdrantPort).collectionName("langchain_docs").build();}@BeanpublicOpenAiEmbeddingModelembeddingModel(){returnOpenAiEmbeddingModel.builder().apiKey(openAiApiKey).build();}@BeanpublicOpenAiChatModelchatModel(){returnOpenAiChatModel.builder().apiKey(openAiApiKey).build();}@BeanpublicConversationRetrieverconversationRetriever(QdrantEmbeddingStore embeddingStore,OpenAiEmbeddingModel embeddingModel){returnEmbeddingStoreRetriever.builder().embeddingStore(embeddingStore).embeddingModel(embeddingModel).maxResults(3).minScore(0.7).build();}}5.3 实现 RAG 服务
packagecom.example.qdrant.service;importdev.langchain4j.data.document.Document;importdev.langchain4j.data.document.DocumentSplitter;importdev.langchain4j.data.document.splitter.DocumentSplitters;importdev.langchain4j.data.segment.TextSegment;importdev.langchain4j.model.openai.OpenAiChatModel;importdev.langchain4j.model.openai.OpenAiEmbeddingModel;importdev.langchain4j.store.embedding.EmbeddingStore;importdev.langchain4j.store.embedding.EmbeddingStoreIngestor;importdev.langchain4j.store.embedding.qdrant.QdrantEmbeddingStore;importdev.langchain4j.service.AiServices;importlombok.RequiredArgsConstructor;importlombok.extern.slf4j.Slf4j;importorg.springframework.stereotype.Service;importjava.nio.file.Paths;importjava.util.List;@Slf4j@Service@RequiredArgsConstructorpublicclassLangChainRAGService{privatefinalQdrantEmbeddingStore embeddingStore;privatefinalOpenAiEmbeddingModel embeddingModel;privatefinalOpenAiChatModel chatModel;/** * 加载文档到向量存储 */publicvoidingestDocuments(String filePath)throwsException{// 创建文档摄入器EmbeddingStoreIngestor ingestor =EmbeddingStoreIngestor.builder().embeddingStore(embeddingStore).embeddingModel(embeddingModel).textSegmentSplitter(DocumentSplitters.recursive(300,30)).build();// 加载文档Document document =FileSystemDocumentLoader.loadDocument(Paths.get(filePath));// 摄入文档 ingestor.ingest(document); log.info("Document ingested successfully");}/** * RAG 聊天助手 */publicStringchat(String message){// 定义 AI 服务接口interfaceAssistant{Stringchat(String userMessage);}// 构建带检索的 AI 服务Assistant assistant =AiServices.builder(Assistant.class).chatLanguageModel(chatModel).retriever(EmbeddingStoreRetriever.from( embeddingStore, embeddingModel,3,0.7)).build();return assistant.chat(message);}/** * 语义搜索 */publicList<TextSegment>semanticSearch(String query,int topK){List<EmbeddingMatch<TextSegment>> matches = embeddingStore.findRelevant( embeddingModel.embed(query).content(), topK );return matches.stream().map(EmbeddingMatch::embedded).toList();}}六、实战案例:构建智能文档问答系统
6.1 系统架构
┌─────────────┐ │ 用户查询 │ └──────┬──────┘ │ ▼ ┌─────────────────────┐ │ REST API 层 │ ├─────────────────────┤ │ /api/chat │ │ /api/documents │ └──────┬──────────────┘ │ ▼ ┌─────────────────────┐ │ 业务服务层 │ ├─────────────────────┤ │ - RAGService │ │ - DocumentService │ └──────┬──────────────┘ │ ▼ ┌──────────────────────────┐ │ AI 框架层 │ ├──────────────────────────┤ │ - Spring AI / LangChain4j│ │ - Embedding Model │ │ - Chat Model │ └──────┬───────────────────┘ │ ▼ ┌──────────────────────────┐ │ Qdrant 向量数据库 │ ├──────────────────────────┤ │ - Collection: documents │ │ - Vector Search │ └──────────────────────────┘ 6.2 完整实现
packagecom.example.qdrant;@SpringBootApplication@EnableAiServices// LangChain4jpublicclassQdrantDemoApplication{publicstaticvoidmain(String[] args){SpringApplication.run(QdrantDemoApplication.class, args);}}@RestController@RequestMapping("/api")@RequiredArgsConstructorpublicclassDocumentController{privatefinalRAGService ragService;privatefinalLangChainRAGService langChainRAGService;/** * 上传文档 */@PostMapping("/documents/upload")publicResponseEntity<String>uploadDocument(@RequestParam("file")MultipartFile file){try{Path tempFile =Files.createTempFile("upload",".txt"); file.transferTo(tempFile);// 使用 Spring AI 方式// ragService.loadAndStoreDocuments(new FileSystemResource(tempFile));// 使用 LangChain4j 方式 langChainRAGService.ingestDocuments(tempFile.toString());returnResponseEntity.ok("Document uploaded and processed successfully");}catch(Exception e){ log.error("Error processing document", e);returnResponseEntity.status(500).body("Error processing document");}}/** * 问答接口 */@PostMapping("/chat")publicResponseEntity<ChatResponse>chat(@RequestBodyChatRequest request){String answer = langChainRAGService.chat(request.getMessage());returnResponseEntity.ok(newChatResponse(answer));}/** * 语义搜索 */@GetMapping("/search")publicResponseEntity<List<SearchResult>>search(@RequestParamString query,@RequestParam(defaultValue ="5")int topK){List<TextSegment> results = langChainRAGService.semanticSearch(query, topK);List<SearchResult> searchResults = results.stream().map(segment ->newSearchResult(segment.text(),null)).toList();returnResponseEntity.ok(searchResults);}}七、最佳实践与性能优化
7.1 向量维度选择
- text-embedding-ada-002 (OpenAI): 1536 维
- all-MiniLM-L6-v2: 384 维
- paraphrase-multilingual-MiniLM-L12-v2: 384 维(多语言)
建议:根据模型选择合适的维度,维度越高精度越高但存储和搜索成本也越高。
7.2 分片策略
// 创建分片集合Collections.CreateCollection createCollection =Collections.CreateCollection.newBuilder().setVectorsConfig(vectorParams).setShardNumber(4)// 4 个分片.setReplicationFactor(2)// 每个分片 2 个副本.build();7.3 索引参数调优
Collections.HnswConfigDiff hnswConfig =Collections.HnswConfigDiff.newBuilder().setM(16)// 每个节点连接数(范围 2-100).setEfConstruct(100)// 构建索引时的搜索深度.setFullScanThreshold(10000)// 触发全扫描的向量数量阈值.build();7.4 批量操作优化
// 批量插入时控制批次大小publicvoidbatchUpsert(String collectionName,List<PointData> allPoints)throwsException{int batchSize =100;List<List<PointData>> batches =Lists.partition(allPoints, batchSize);for(List<PointData> batch : batches){upsertPoints(collectionName, batch);Thread.sleep(100);// 避免 QPS 过高}}八、总结
Qdrant 作为一款现代化的向量数据库,具有以下优势:
- 高性能:基于 Rust 实现,性能出色
- 易集成:提供多语言客户端,与 Spring AI、LangChain 等框架集成良好
- 功能丰富:支持过滤、分片、复制等企业级特性
- 开源免费:完全开源,无供应商锁定
通过本文的介绍,你应该能够:
- 理解 Qdrant 的核心概念和特性
- 在 Spring Boot 项目中集成 Qdrant
- 配合 Spring AI 和 LangChain4j 构建 RAG 应用
- 掌握基本的性能优化技巧
接下来,建议你:
- 从简单的语义搜索开始实践
- 逐步构建完整的 RAG 应用
- 根据业务需求优化向量维度、索引参数等配置
参考资源
- Qdrant 官方文档: https://qdrant.tech/documentation/
- Spring AI 文档: https://docs.spring.io/spring-ai/reference/
- LangChain4j 文档: https://docs.langchain4j.dev/
- Qdrant Java Client: https://github.com/qdrant/qdrant-client-java