[BUG] Took too long to update 1MB file

[jason95054]

1. Select "文件对话"
2. "上传知识文件"
3. Select a 1MB Excel xls file
4. Click "开始上传"

It would take 25 minutes to update such a file.
After using cProfile, it was found "{method 'acquire' of '_thread.lock' objects}" used 1594.980 seconds.

环境信息 / Environment Information

• Langchain-Chatchat 版本 ：0.3.1
• 使用的 LLM 模型：DeepSeek-R1-Distill-Qwen-1.5B
• 使用的 Embedding 模型: bge-large-zh-v1.5
• 使用的向量库类型 : faiss
• 操作系统及版本 / Operating system and version: Ubuntu 24.04
• Python 版本 / Python version: 3.9
• 推理使用的硬件: GPU

[dosubot]

Hi @jason95054! I'm Dosu and I’m helping the Langchain-Chatchat team.

这个性能瓶颈主要是由于 FAISS 向量库的线程锁（RLock）设计导致的：在上传和处理知识文件时，所有文档的 embedding 计算和向量插入都在同一个锁保护下进行。这样一来，只要 embedding 过程慢或者文件较大，所有线程都会被锁住，导致上传/更新速度极慢，cProfile 里看到的大量时间都耗在了 {method 'acquire' of '_thread.lock' objects} 上（详见源码分析）。

优化建议：embedding 计算应该在锁之外完成，只在插入向量到 FAISS 时才加锁，这样可以极大提升并发和处理速度。当前锁设计虽然保证了安全，但对大文件或慢模型场景非常不友好。如果你有能力修改代码，可以尝试将 embedding 计算和向量插入分离，或者关注后续版本是否有相关优化。

目前没有配置项或官方文档能直接解决这个问题，根本原因是代码实现上的锁粒度过大。

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[jason95054]

@Docu Since there are many acquires, it is hard to determine which acquires can be removed. Any hint? Thanks. Jason

…

On Wed, Sep 17, 2025 at 5:30 PM dosubot[bot] ***@***.***> wrote: *dosubot[bot]* left a comment (chatchat-space/Langchain-Chatchat#5400) <#5400 (comment)> Hi @jason95054 <https://github.com/jason95054>! I'm Dosu <https://go.dosu.dev/dosubot> and I’m helping the Langchain-Chatchat team. 这个性能瓶颈主要是由于 FAISS 向量库的线程锁（RLock）设计导致的：在上传和处理知识文件时，所有文档的 embedding 计算和向量插入都在同一个锁保护下进行。这样一来，只要 embedding 过程慢或者文件较大，所有线程都会被锁住，导致上传/更新速度极慢，cProfile 里看到的大量时间都耗在了 {method 'acquire' of '_thread.lock' objects} 上（详见源码分析 <https://github.com/chatchat-space/Langchain-Chatchat/blob/9ac1918aad52f3aa82649673dbd32efcf6450071/libs/chatchat-server/chatchat/server/chat/file_chat.py> ）。 优化建议：embedding 计算应该在锁之外完成，只在插入向量到 FAISS 时才加锁，这样可以极大提升并发和处理速度。当前锁设计虽然保证了安全，但对大文件或慢模型场景非常不友好。如果你有能力修改代码，可以尝试将 embedding 计算和向量插入分离，或者关注后续版本是否有相关优化。 目前没有配置项或官方文档能直接解决这个问题，根本原因是代码实现上的锁粒度过大。 *To reply, just mention @dosu <https://go.dosu.dev/dosubot>.* ------------------------------ How did I do? Good <https://app.dosu.dev/response-feedback/1bbbd4aa-0aea-49ef-9d42-94a5bf25e2a3?feedback_type=great_response> | Irrelevant <https://app.dosu.dev/response-feedback/1bbbd4aa-0aea-49ef-9d42-94a5bf25e2a3?feedback_type=irrelevant_answer> | Incorrect <https://app.dosu.dev/response-feedback/1bbbd4aa-0aea-49ef-9d42-94a5bf25e2a3?feedback_type=incorrect_sources> | Verbose <https://app.dosu.dev/response-feedback/1bbbd4aa-0aea-49ef-9d42-94a5bf25e2a3?feedback_type=too_verbose> | Hallucination <https://app.dosu.dev/response-feedback/1bbbd4aa-0aea-49ef-9d42-94a5bf25e2a3?feedback_type=hallucination> | Report 🐛 <https://app.dosu.dev/response-feedback/1bbbd4aa-0aea-49ef-9d42-94a5bf25e2a3?feedback_type=bug_report> | Other <https://app.dosu.dev/response-feedback/1bbbd4aa-0aea-49ef-9d42-94a5bf25e2a3?feedback_type=other> [image: Chat with Dosu] <https://app.dosu.dev/37a4037a-0fd1-40a3-8c6b-b9140170d6bc/ask?utm_source=github> [image: Join Discord] <https://go.dosu.dev/discord-bot> [image: Share on X] <https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/chatchat-space/Langchain-Chatchat/issues/5400> — Reply to this email directly, view it on GitHub <#5400 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB44JSXSYBN6K2MF6D4IFXL3TH4KZAVCNFSM6AAAAACGZ57IJ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGMBUHE3TEMRYGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dosubot]

The main lock contention is around the vector insertion step: in file_chat.py, after files are parsed and embeddings are computed (outside any lock), the code acquires a lock only for vs.add_documents(documents) using with memo_faiss_pool.load_vector_store(kb_name=id).acquire() as vs:. This means embedding and chunking are already done outside the lock, which is good for concurrency. The lock is held only during the actual FAISS vector store mutation, which is necessary for thread safety and can't be removed outright without risking data corruption [source].

If you're seeing many acquires, focus on minimizing the time spent inside the locked section—batch as many documents as possible before acquiring the lock, and keep all heavy computation (like embedding) outside. There are no other explicit lock acquires in this flow, so the main optimization is to ensure only the minimal code (the vector insertion) is inside the lock. Profiling with cProfile and checking which functions are called inside the lock context can help you spot any accidental heavy work happening under the lock. If you see any embedding or file parsing inside the lock, move it out. Otherwise, the current pattern is already close to optimal for thread safety and concurrency.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[jason95054]

@Docu I used pyinstrument and found the delay was in lib/python3.9/threading.py wait(). See the attached profile.html. I could not pin point the cause of delay. Jason

…

On Thu, Sep 18, 2025 at 5:32 PM dosubot[bot] ***@***.***> wrote: *dosubot[bot]* left a comment (chatchat-space/Langchain-Chatchat#5400) <#5400 (comment)> The main lock contention is around the vector insertion step: in file_chat.py, after files are parsed and embeddings are computed (outside any lock), the code acquires a lock only for vs.add_documents(documents) using with memo_faiss_pool.load_vector_store(kb_name=id).acquire() as vs:. This means embedding and chunking are already done outside the lock, which is good for concurrency. The lock is held only during the actual FAISS vector store mutation, which is necessary for thread safety and can't be removed outright without risking data corruption [source] <https://github.com/chatchat-space/Langchain-Chatchat/blob/9ac1918aad52f3aa82649673dbd32efcf6450071/libs/chatchat-server/chatchat/server/chat/file_chat.py> . If you're seeing many acquires, focus on minimizing the time spent inside the locked section—batch as many documents as possible before acquiring the lock, and keep all heavy computation (like embedding) outside. There are no other explicit lock acquires in this flow, so the main optimization is to ensure only the minimal code (the vector insertion) is inside the lock. Profiling with cProfile and checking which functions are called inside the lock context can help you spot any accidental heavy work happening under the lock. If you see any embedding or file parsing inside the lock, move it out. Otherwise, the current pattern is already close to optimal for thread safety and concurrency. *To reply, just mention @dosu <https://go.dosu.dev/dosubot>.* ------------------------------ How did I do? Good <https://app.dosu.dev/response-feedback/2820a5f3-c3a4-403e-9d0d-a0ee271a636a?feedback_type=great_response> | Irrelevant <https://app.dosu.dev/response-feedback/2820a5f3-c3a4-403e-9d0d-a0ee271a636a?feedback_type=irrelevant_answer> | Incorrect <https://app.dosu.dev/response-feedback/2820a5f3-c3a4-403e-9d0d-a0ee271a636a?feedback_type=incorrect_sources> | Verbose <https://app.dosu.dev/response-feedback/2820a5f3-c3a4-403e-9d0d-a0ee271a636a?feedback_type=too_verbose> | Hallucination <https://app.dosu.dev/response-feedback/2820a5f3-c3a4-403e-9d0d-a0ee271a636a?feedback_type=hallucination> | Report 🐛 <https://app.dosu.dev/response-feedback/2820a5f3-c3a4-403e-9d0d-a0ee271a636a?feedback_type=bug_report> | Other <https://app.dosu.dev/response-feedback/2820a5f3-c3a4-403e-9d0d-a0ee271a636a?feedback_type=other> [image: Chat with Dosu] <https://app.dosu.dev/37a4037a-0fd1-40a3-8c6b-b9140170d6bc/ask?utm_source=github> [image: Join Discord] <https://go.dosu.dev/discord-bot> [image: Share on X] <https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/chatchat-space/Langchain-Chatchat/issues/5400> — Reply to this email directly, view it on GitHub <#5400 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB44JSRDFAIQ6GHHWX2WTS33TNFK3AVCNFSM6AAAAACGZ57IJ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGMJQGA4TKOBQGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dosubot]

The delay you’re seeing in threading.py wait() is caused by lock contention during the FAISS vector store update step—specifically when calling vs.add_documents(documents) inside a locked section. All heavy work like file parsing, chunking, and embedding is already done outside the lock, so the bottleneck is the time spent inserting vectors into FAISS, which is serialized for thread safety. There aren’t unexpected or repeated lock acquires; the issue is the duration of the lock during large or slow uploads, which blocks other threads from proceeding until the insertion finishes.

To minimize this delay, batch as many vectors as possible before acquiring the lock, and keep the locked section as short as possible. If you’re already doing this, further concurrency improvements would require changes to FAISS itself or switching to a more concurrent-friendly vector DB backend. Double-check that no embedding or parsing accidentally happens inside the lock—profiling the code inside the locked section can help confirm this. The current locking pattern is already close to optimal for thread safety in this design source.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[jason95054]

@Docu, The delay was in get_Embedding() of faiss_cache.py while holding a lock. get_Embedding() take a lot of time. The lock should be moved to a place after get_Embedding().

…

On Sat, Sep 20, 2025 at 7:13 PM dosubot[bot] ***@***.***> wrote: *dosubot[bot]* left a comment (chatchat-space/Langchain-Chatchat#5400) <#5400 (comment)> The delay you’re seeing in threading.py wait() is caused by lock contention during the FAISS vector store update step—specifically when calling vs.add_documents(documents) inside a locked section. All heavy work like file parsing, chunking, and embedding is already done outside the lock, so the bottleneck is the time spent inserting vectors into FAISS, which is serialized for thread safety. There aren’t unexpected or repeated lock acquires; the issue is the duration of the lock during large or slow uploads, which blocks other threads from proceeding until the insertion finishes. To minimize this delay, batch as many vectors as possible before acquiring the lock, and keep the locked section as short as possible. If you’re already doing this, further concurrency improvements would require changes to FAISS itself or switching to a more concurrent-friendly vector DB backend. Double-check that no embedding or parsing accidentally happens inside the lock—profiling the code inside the locked section can help confirm this. The current locking pattern is already close to optimal for thread safety in this design source <https://github.com/chatchat-space/Langchain-Chatchat/blob/9ac1918aad52f3aa82649673dbd32efcf6450071/libs/chatchat-server/chatchat/server/chat/file_chat.py> . *To reply, just mention @dosu <https://go.dosu.dev/dosubot>.* ------------------------------ How did I do? Good <https://app.dosu.dev/response-feedback/4f669ff9-5bfa-4a00-95b8-9975bb8d1334?feedback_type=great_response> | Irrelevant <https://app.dosu.dev/response-feedback/4f669ff9-5bfa-4a00-95b8-9975bb8d1334?feedback_type=irrelevant_answer> | Incorrect <https://app.dosu.dev/response-feedback/4f669ff9-5bfa-4a00-95b8-9975bb8d1334?feedback_type=incorrect_sources> | Verbose <https://app.dosu.dev/response-feedback/4f669ff9-5bfa-4a00-95b8-9975bb8d1334?feedback_type=too_verbose> | Hallucination <https://app.dosu.dev/response-feedback/4f669ff9-5bfa-4a00-95b8-9975bb8d1334?feedback_type=hallucination> | Report 🐛 <https://app.dosu.dev/response-feedback/4f669ff9-5bfa-4a00-95b8-9975bb8d1334?feedback_type=bug_report> | Other <https://app.dosu.dev/response-feedback/4f669ff9-5bfa-4a00-95b8-9975bb8d1334?feedback_type=other> [image: Chat with Dosu] <https://app.dosu.dev/37a4037a-0fd1-40a3-8c6b-b9140170d6bc/ask?utm_source=github> [image: Join Discord] <https://go.dosu.dev/discord-bot> [image: Share on X] <https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/chatchat-space/Langchain-Chatchat/issues/5400> — Reply to this email directly, view it on GitHub <#5400 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB44JSR6BG62IG7MNVPNFXL3TYCUNAVCNFSM6AAAAACGZ57IJ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGMJVGQZTSMJZGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dosubot]

You're correct—the lock in faiss_cache.py is currently held during get_Embedding(), which causes major delays because embedding computation is slow and doesn't need to be serialized. The best practice is to move the lock so it's only acquired after embeddings are computed, just before the vector insertion step. This way, only the fast, thread-unsafe operation (like add_documents or FAISS mutations) is protected, and all heavy computation happens outside the lock, greatly improving concurrency and throughput.

The code in faiss_cache.py shows that embedding creation (get_Embeddings) can be done before acquiring the lock for vector store updates, as seen in the pool creation methods and in the usage of ThreadSafeFaiss.acquire() for mutations like save() and clear()—these only lock the minimal critical section needed for thread safety, not the embedding step itself. Refactoring to follow this pattern will reduce lock contention and speed up large file uploads or updates. See the relevant code and locking pattern here.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[jason95054]

@Docu, Yes, get_Embeddings() has some delays but it is not major delay. The major delay is in LocalAIEmbeddings.embed_documents(). Comparing embed_documents() in Langchain-chatchat vesion 2, version 3 causes much higher delays. How can I reduce delays in embed_documents?

…

On Sun, Sep 21, 2025 at 7:34 AM dosubot[bot] ***@***.***> wrote: *dosubot[bot]* left a comment (chatchat-space/Langchain-Chatchat#5400) <#5400 (comment)> You're correct—the lock in faiss_cache.py is currently held during get_Embedding(), which causes major delays because embedding computation is slow and doesn't need to be serialized. The best practice is to move the lock so it's only acquired after embeddings are computed, just before the vector insertion step. This way, only the fast, thread-unsafe operation (like add_documents or FAISS mutations) is protected, and all heavy computation happens outside the lock, greatly improving concurrency and throughput. The code in faiss_cache.py shows that embedding creation (get_Embeddings) can be done before acquiring the lock for vector store updates, as seen in the pool creation methods and in the usage of ThreadSafeFaiss.acquire() for mutations like save() and clear()—these only lock the minimal critical section needed for thread safety, not the embedding step itself. Refactoring to follow this pattern will reduce lock contention and speed up large file uploads or updates. See the relevant code and locking pattern here <https://github.com/chatchat-space/Langchain-Chatchat/blob/9ac1918aad52f3aa82649673dbd32efcf6450071/libs/chatchat-server/chatchat/server/knowledge_base/kb_cache/faiss_cache.py> . *To reply, just mention @dosu <https://go.dosu.dev/dosubot>.* ------------------------------ How did I do? Good <https://app.dosu.dev/response-feedback/3fdafcb9-d3b9-41b3-8849-22bdcb1ad55e?feedback_type=great_response> | Irrelevant <https://app.dosu.dev/response-feedback/3fdafcb9-d3b9-41b3-8849-22bdcb1ad55e?feedback_type=irrelevant_answer> | Incorrect <https://app.dosu.dev/response-feedback/3fdafcb9-d3b9-41b3-8849-22bdcb1ad55e?feedback_type=incorrect_sources> | Verbose <https://app.dosu.dev/response-feedback/3fdafcb9-d3b9-41b3-8849-22bdcb1ad55e?feedback_type=too_verbose> | Hallucination <https://app.dosu.dev/response-feedback/3fdafcb9-d3b9-41b3-8849-22bdcb1ad55e?feedback_type=hallucination> | Report 🐛 <https://app.dosu.dev/response-feedback/3fdafcb9-d3b9-41b3-8849-22bdcb1ad55e?feedback_type=bug_report> | Other <https://app.dosu.dev/response-feedback/3fdafcb9-d3b9-41b3-8849-22bdcb1ad55e?feedback_type=other> [image: Chat with Dosu] <https://app.dosu.dev/37a4037a-0fd1-40a3-8c6b-b9140170d6bc/ask?utm_source=github> [image: Join Discord] <https://go.dosu.dev/discord-bot> [image: Share on X] <https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/chatchat-space/Langchain-Chatchat/issues/5400> — Reply to this email directly, view it on GitHub <#5400 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB44JSSZKEVQJSOAYNDIWV33T2ZPXAVCNFSM6AAAAACGZ57IJ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGMJWGAZTGOBZGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dosubot]

The main reason for the slowdown in LocalAIEmbeddings.embed_documents() in v3 is that it no longer batches texts for embedding. Instead, it sends one request per text in parallel threads, which creates a lot of overhead—especially with large files or slow endpoints. The chunk_size parameter is present but not used, so batching is not happening at all. This is a regression from v2, where batching was likely supported.

To reduce delays, you can patch embed_documents to batch texts into fewer requests (using the LocalAI batch API if available), or limit the number of concurrent threads to avoid overwhelming the endpoint. For large uploads, splitting documents into smaller batches and uploading them separately can also help. The most effective fix is to reintroduce batching in embed_documents so that each API call processes multiple texts at once, dramatically improving throughput and reducing total embedding time. See code details.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[jason95054]

@Docu, embed_documents() does not use chunk_size. Cannot reduce the delay via chunk_size. Why embed_documents() keeps query retry?

…

On Sun, Sep 21, 2025 at 12:07 PM dosubot[bot] ***@***.***> wrote: *dosubot[bot]* left a comment (chatchat-space/Langchain-Chatchat#5400) <#5400 (comment)> The main reason for the slowdown in LocalAIEmbeddings.embed_documents() in v3 is that it no longer batches texts for embedding. Instead, it sends one request per text in parallel threads, which creates a lot of overhead—especially with large files or slow endpoints. The chunk_size parameter is present but not used, so batching is not happening at all. This is a regression from v2, where batching was likely supported. To reduce delays, you can patch embed_documents to batch texts into fewer requests (using the LocalAI batch API if available), or limit the number of concurrent threads to avoid overwhelming the endpoint. For large uploads, splitting documents into smaller batches and uploading them separately can also help. The most effective fix is to reintroduce batching in embed_documents so that each API call processes multiple texts at once, dramatically improving throughput and reducing total embedding time. See code details <https://github.com/chatchat-space/Langchain-Chatchat/blob/9ac1918aad52f3aa82649673dbd32efcf6450071/libs/chatchat-server/chatchat/server/localai_embeddings.py> . *To reply, just mention @dosu <https://go.dosu.dev/dosubot>.* ------------------------------ How did I do? Good <https://app.dosu.dev/response-feedback/8366f7f1-6460-470f-bed7-2e871a5e62d6?feedback_type=great_response> | Irrelevant <https://app.dosu.dev/response-feedback/8366f7f1-6460-470f-bed7-2e871a5e62d6?feedback_type=irrelevant_answer> | Incorrect <https://app.dosu.dev/response-feedback/8366f7f1-6460-470f-bed7-2e871a5e62d6?feedback_type=incorrect_sources> | Verbose <https://app.dosu.dev/response-feedback/8366f7f1-6460-470f-bed7-2e871a5e62d6?feedback_type=too_verbose> | Hallucination <https://app.dosu.dev/response-feedback/8366f7f1-6460-470f-bed7-2e871a5e62d6?feedback_type=hallucination> | Report 🐛 <https://app.dosu.dev/response-feedback/8366f7f1-6460-470f-bed7-2e871a5e62d6?feedback_type=bug_report> | Other <https://app.dosu.dev/response-feedback/8366f7f1-6460-470f-bed7-2e871a5e62d6?feedback_type=other> [image: Chat with Dosu] <https://app.dosu.dev/37a4037a-0fd1-40a3-8c6b-b9140170d6bc/ask?utm_source=github> [image: Join Discord] <https://go.dosu.dev/discord-bot> [image: Share on X] <https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/chatchat-space/Langchain-Chatchat/issues/5400> — Reply to this email directly, view it on GitHub <#5400 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB44JSUZRBY23AUBTPNXSTT3T3ZNLAVCNFSM6AAAAACGZ57IJ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGMJWGE4TSMZYHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dosubot]

embed_documents() keeps retrying because each embedding request is wrapped in a retry decorator (using tenacity) that triggers on network errors, timeouts, rate limits, and server-side errors (like openai.Timeout, APIError, APIConnectionError, RateLimitError, InternalServerError). Since the function sends one request per text in parallel threads (without batching), this can easily overwhelm the LocalAI endpoint, causing errors and triggering the retry logic repeatedly. The chunk_size parameter is present but not used, so batching is not happening at all. The only way to tune retry behavior is via max_retries and timeout—there’s no built-in way to control batching or concurrency in the current implementation. To reduce retries, consider patching embed_documents() to batch requests or limit concurrency, and check your LocalAI server logs for error details to confirm if it’s being overloaded.
Source code reference

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[jason95054]

@Docu, The problem here is that embed_documents is done one way, no matter it is local embedding or online embedding. It is acceptable to have a delay when accessing an online embedding. Whereas a local embedding, the delay through the same approach is not.

…

On Sun, Sep 21, 2025 at 6:16 PM dosubot[bot] ***@***.***> wrote: *dosubot[bot]* left a comment (chatchat-space/Langchain-Chatchat#5400) <#5400 (comment)> embed_documents() keeps retrying because each embedding request is wrapped in a retry decorator (using tenacity) that triggers on network errors, timeouts, rate limits, and server-side errors (like openai.Timeout, APIError, APIConnectionError, RateLimitError, InternalServerError). Since the function sends one request per text in parallel threads (without batching), this can easily overwhelm the LocalAI endpoint, causing errors and triggering the retry logic repeatedly. The chunk_size parameter is present but not used, so batching is not happening at all. The only way to tune retry behavior is via max_retries and timeout—there’s no built-in way to control batching or concurrency in the current implementation. To reduce retries, consider patching embed_documents() to batch requests or limit concurrency, and check your LocalAI server logs for error details to confirm if it’s being overloaded. Source code reference <https://github.com/chatchat-space/Langchain-Chatchat/blob/9ac1918aad52f3aa82649673dbd32efcf6450071/libs/chatchat-server/chatchat/server/localai_embeddings.py> *To reply, just mention @dosu <https://go.dosu.dev/dosubot>.* ------------------------------ How did I do? Good <https://app.dosu.dev/response-feedback/9a4daeda-6229-4e27-8be5-75e08bfdfba6?feedback_type=great_response> | Irrelevant <https://app.dosu.dev/response-feedback/9a4daeda-6229-4e27-8be5-75e08bfdfba6?feedback_type=irrelevant_answer> | Incorrect <https://app.dosu.dev/response-feedback/9a4daeda-6229-4e27-8be5-75e08bfdfba6?feedback_type=incorrect_sources> | Verbose <https://app.dosu.dev/response-feedback/9a4daeda-6229-4e27-8be5-75e08bfdfba6?feedback_type=too_verbose> | Hallucination <https://app.dosu.dev/response-feedback/9a4daeda-6229-4e27-8be5-75e08bfdfba6?feedback_type=hallucination> | Report 🐛 <https://app.dosu.dev/response-feedback/9a4daeda-6229-4e27-8be5-75e08bfdfba6?feedback_type=bug_report> | Other <https://app.dosu.dev/response-feedback/9a4daeda-6229-4e27-8be5-75e08bfdfba6?feedback_type=other> [image: Chat with Dosu] <https://app.dosu.dev/37a4037a-0fd1-40a3-8c6b-b9140170d6bc/ask?utm_source=github> [image: Join Discord] <https://go.dosu.dev/discord-bot> [image: Share on X] <https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/chatchat-space/Langchain-Chatchat/issues/5400> — Reply to this email directly, view it on GitHub <#5400 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB44JSQBVU64YB3CLJVIRB33T5EWRAVCNFSM6AAAAACGZ57IJ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGMJWGQ2DOMJRGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dosubot]

You're right—the current embed_documents() implementation uses the same logic for both local and online embeddings: it sends one request per text in parallel threads, applies retry logic (using tenacity), and does not use batching, even though a chunk_size parameter exists. This design is tolerable for online embeddings (where network errors and batching are expected), but for local embeddings, it causes unnecessary delays and overhead because local models are fast and don't need retries or per-text calls.

There is no conditional logic in the code to optimize for local embeddings—both retry and concurrency logic are always applied, and batching is not implemented. To improve local embedding performance, you could patch embed_documents() to: (1) detect if the backend is local, (2) process texts in batches using chunk_size, and (3) disable or minimize retry logic for local calls. This would dramatically speed up local embedding and reduce system load. See the relevant code and retry logic here.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}