# Weave: High latency with LlamaIndex

**URL:** <https://community.wandb.ai/t/weave-high-latency-with-llamaindex/7468>\
**Category:** W&B Help\
**Created:** [August 12, 2024, 4:47pm UTC](https://community.wandb.ai/t/weave-high-latency-with-llamaindex/7468 "2024-08-12T16:47:02Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![tryrisotto](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/tryrisotto/32/2025_2.png) [@tryrisotto](https://community.wandb.ai/u/tryrisotto)\
**Post date:** [August 12, 2024, 4:47pm UTC](https://community.wandb.ai/t/weave-high-latency-with-llamaindex/7468/1 "2024-08-12T16:47:02Z")

</div>

Hey folks!

Just deployed a weave integration to production yesterday and saw it triple my llama-index query latency. ☹

My guess is that it is making calls to WandB synchronously during the query pipeline run, but I can’t tell for sure because it was basically a no-config setup (which was very nice).

Any suggestions for ways to improve this?

Thanks!  
Chris

---

<div class="post-metadata">

**Author:** ![artsiom](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/artsiom/32/1001_2.png) [@artsiom](https://community.wandb.ai/u/artsiom)\
**Post date:** [August 12, 2024, 8:24pm UTC](https://community.wandb.ai/t/weave-high-latency-with-llamaindex/7468/2 "2024-08-12T20:24:25Z")

</div>

Hi @tryrisotto~

Apologies you are seeing this behavior and thank you very much for writing in. Looking into this and we will get back to you as soon as we have any updates or follow up questions.

---

<div class="post-metadata">

**Author:** ![tryrisotto](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/tryrisotto/32/2025_2.png) [@tryrisotto](https://community.wandb.ai/u/tryrisotto)\
**Post date:** [August 13, 2024, 5:55pm UTC](https://community.wandb.ai/t/weave-high-latency-with-llamaindex/7468/3 "2024-08-13T17:55:01Z")

</div>

Digging into the weave source a bit, I’m curious whether the `finish_call` function is sync or async.

I traced the calls from the llamaindex integration here:

> <https://github.com/wandb/weave/blob/b08b65073b3646971b07fa18b4b8c46d7717eab4/weave/integrations/llamaindex/llamaindex.py#L78C24-L78C35>

And it appears there is a trace server that executes the HTTP request using the python `requests` library here (there are a few TraceServerInterface implementors, so I’m just guessing this is the culprit):

> <https://github.com/wandb/weave/blob/b08b65073b3646971b07fa18b4b8c46d7717eab4/weave/trace_server/remote_http_trace_server.py#L184>

This is definitely making a synchronous POST, but I’m not proficient enough in Python to know if this can be made async (from what I understand, it’s gotta be async all the way down in order for that to work correctly, and I do not currently use async/await or asyncio anywhere in my Python application, so not sure this would actually help).

---

<div class="post-metadata">

**Author:** ![artsiom](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/artsiom/32/1001_2.png) [@artsiom](https://community.wandb.ai/u/artsiom)\
**Post date:** [August 14, 2024, 4:34pm UTC](https://community.wandb.ai/t/weave-high-latency-with-llamaindex/7468/4 "2024-08-14T16:34:13Z")

</div>

HI @tryrisotto! Thank you very much for digging into this!

I have gone ahead and escalated this behavior to our Weave team. They have found a couple of things that couple possibly lead to this behavior such as each trace creates new versions of multiple objects for the Weave workflow, which could be causing the slowdown for you.

I will keep you posted as soon as I get any updates on my end as well.

Warmly,  
Artsiom

---

<div class="post-metadata">

**Author:** ![tryrisotto](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/tryrisotto/32/2025_2.png) [@tryrisotto](https://community.wandb.ai/u/tryrisotto)\
**Post date:** [August 14, 2024, 4:53pm UTC](https://community.wandb.ai/t/weave-high-latency-with-llamaindex/7468/5 "2024-08-14T16:53:35Z")

</div>

Amazing, thank you @artsiom ! Standing by…

---

<div class="post-metadata">

**Author:** ![tryrisotto](https://sea2.discourse-cdn.com/flex020/user_avatar/community.wandb.ai/tryrisotto/32/2025_2.png) [@tryrisotto](https://community.wandb.ai/u/tryrisotto)\
**Post date:** [January 9, 2025, 9:18pm UTC](https://community.wandb.ai/t/weave-high-latency-with-llamaindex/7468/6 "2025-01-09T21:18:18Z")

</div>

@artsiom It’s been a while! Do you know if this has been resolved? I’m curious what changes were made, if any?

---

<div class="post-metadata">

**Author:** ![mauricio\_shopsense](https://avatars.discourse-cdn.com/v4/letter/m/dec6dc/32.png) [@mauricio\_shopsense](https://community.wandb.ai/u/mauricio_shopsense)\
**Post date:** [January 16, 2025, 3:03pm UTC](https://community.wandb.ai/t/weave-high-latency-with-llamaindex/7468/7 "2025-01-16T15:03:15Z")

</div>

Did they find a solution ? this latency is the worst for the LLM agents, it takes so long.

---

<div class="post-metadata">

**Author:** ![jason-arkens17](https://avatars.discourse-cdn.com/v4/letter/j/f07891/32.png) [@jason-arkens17](https://community.wandb.ai/u/jason-arkens17)\
**Post date:** [May 14, 2025, 8:08pm UTC](https://community.wandb.ai/t/weave-high-latency-with-llamaindex/7468/8 "2025-05-14T20:08:07Z")

</div>

WB-20357 has been moved to Merged - This ticket will be closed for now. Thanks so much!