requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: /table/query. Reason

motzwanted · August 21, 2024, 6:08am

Hi,

I’m new to weave and I’m trying to run an evaluation pipeline with a custom weave.Model. But I run into a 500 Server Error. In this case, it seems related to an internal trace operation in the weave package, specifically during an attempt to query a table (/table/query endpoint). Here is the full traceback for context:

Traceback (most recent call last):
  File "/Users/amo/miniconda3/lib/python3.12/runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/miniconda3/lib/python3.12/runpy.py", line 88, in _run_code
    exec(code, run_globals)
  File "/Users/amo/.vscode/extensions/ms-python.debugpy-2024.10.0-darwin-arm64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/Users/amo/.vscode/extensions/ms-python.debugpy-2024.10.0-darwin-arm64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/Users/amo/.vscode/extensions/ms-python.debugpy-2024.10.0-darwin-arm64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/Users/amo/.vscode/extensions/ms-python.debugpy-2024.10.0-darwin-arm64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/.vscode/extensions/ms-python.debugpy-2024.10.0-darwin-arm64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/Users/amo/.vscode/extensions/ms-python.debugpy-2024.10.0-darwin-arm64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/Users/amo/Developer/code/patient-note-alignment/examples/weave_exp.py", line 176, in <module>
    print(asyncio.run(evaluation.evaluate(model)))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/nest_asyncio.py", line 30, in run
    return loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/nest_asyncio.py", line 98, in run_until_complete
    return f.result()
           ^^^^^^^^^^
  File "/Users/amo/miniconda3/lib/python3.12/asyncio/futures.py", line 203, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "/Users/amo/miniconda3/lib/python3.12/asyncio/tasks.py", line 314, in __step_run_and_handle_result
    result = coro.send(None)
             ^^^^^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/weave/trace/op.py", line 333, in wrapper
    res, _ = await _execute_call(wrapper, call, *args, **kwargs)  # type: ignore
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/weave/trace/op.py", line 213, in _call_async
    return handle_exception(e)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/weave/trace/op.py", line 211, in _call_async
    res = await func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/weave/flow/eval.py", line 300, in evaluate
    trial_rows = list(_rows) * self.trials
                 ^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/weave/trace/vals.py", line 313, in __iter__
    return iter(self.rows)
                ^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/weave/trace/vals.py", line 254, in rows
    self._rows = list(self._remote_iter())
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/weave/trace/vals.py", line 282, in _remote_iter
    response = self.server.table_query(
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/weave/trace_server/remote_http_trace_server.py", line 434, in table_query
    return self._generic_request(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/weave/trace_server/remote_http_trace_server.py", line 217, in _generic_request
    r = self._generic_request_executor(url, req)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/tenacity/__init__.py", line 336, in wrapped_f
    return copy(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/tenacity/__init__.py", line 475, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/tenacity/__init__.py", line 376, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/tenacity/__init__.py", line 398, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/miniconda3/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/miniconda3/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/tenacity/__init__.py", line 478, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/Users/amo/Library/Caches/pypoetry/virtualenvs/patient-note-alignment-fdNa5lcz-py3.12/lib/python3.12/site-packages/weave/trace_server/remote_http_trace_server.py", line 200, in _generic_request_executor
    raise requests.HTTPError(
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: /table/query. Reason:

frds · August 23, 2024, 11:46am

Hey @motzwanted – thanks for raising this, I am raising this with with the team currently working on weave at the moment at wandb. A couple of things that I think might be useful here initially; I wonder if you object to sharing

The version of weave that you are using with weave --version
A simple code snippet here of where you are using weave.model similar for example to here in our weave docs?

Look forward to hearing back

motzwanted · August 23, 2024, 12:01pm

Hi frds,

Thanks for your quick response.

I’m running with version 0.50.14, and this script is the pipeline I attempt to run:


import asyncio
import os
import pathlib
import datasets
import pydantic
from pydantic_settings import BaseSettings, SettingsConfigDict

import weave
from src.alignment.ops import HfAlignment
from src.alignment.metrics import AlignmentMetrics
from src.dataloader.loaders.nbme_notes import NbmeDatasetLoader
from src.dataloader.adapters.alignment import NmbeAdapter, SentenceSegmenter

os.environ["VLLM_API_BASE"] = "http://localhost:6538/v1"

weave.init("benchmark-alignment")

class Arguments(BaseSettings):
"""Args for the script."""

provider: str = "vllm" # "azure" | "vllm" | "mistral"
deployment: str = "meta-llama/Meta-Llama-3.1-70B-Instruct"
temperature: float = 0.1
token_limit: int = 128000

split: str = "test"
n_samples: int = 300
fewshots: str = "0"

num_workers: int = 12
batch_size: int = 1

response_wait_time: int = 0
use_cache: bool = True

model_config = SettingsConfigDict(cli_parse_args=True)

@pydantic.computed_field
def experiment_name(self) -&gt; str:
"""Get the experiment name."""
return f"{self.provider}-{self.deployment}-{self.prompt_file}-{self.n_samples}"

@pydantic.computed_field
def experiment_dir(self) -&gt; pathlib.Path:
"""Get the experiment directory."""
path_to_exp = self.save_path / str(self.experiment_name)
return path_to_exp

def _get_dataset(dset: datasets.Dataset | datasets.DatasetDict) -&gt; datasets.Dataset:
"""Get a `datasets.Dataset`."""
if isinstance(dset, datasets.Dataset):
return dset
return next(iter(dset.values()))

class AlignmentModel(weave.Model):
model_name: str
prompt_template: str
num_shots: int
temperature: float

@weave.op()
async def predict(self, sources: list, targets: list, predictions: list) -&gt; dict:
return {"predictions": predictions}

@weave.op
def evaluation_metrics(labels: list[list[int]], model_output: dict) -&gt; dict[str, float]:
"""Calculate the alignment metrics."""
predictions = model_output["predictions"]
return {
**AlignmentMetrics.recall_precision(labels, predictions),
**AlignmentMetrics.exact_match(labels, predictions),
**AlignmentMetrics.positive_ratio(labels, predictions),
**AlignmentMetrics.negative_ratio(labels, predictions),
}

def run(args: Arguments):
"""Run the script."""
data: datasets.Dataset = NbmeDatasetLoader().load_dataset(split="test", size=args.n_samples) # type: ignore

adapter = NmbeAdapter(segmenter=SentenceSegmenter())
data = data.map(
adapter,
num_proc=args.num_workers,
desc=f"Adapting dataset to `AlignmentModel` using `{NmbeAdapter.__name__}`.",
remove_columns=_get_dataset(data).column_names,
)

num_shots_ = [int(x) for x in args.fewshots.split(":")] if ":" in args.fewshots else [int(args.fewshots)]
for num_shots in num_shots_:
task_maker = HfAlignment(
provider=args.provider,
model_name=args.deployment,
token_limit=args.token_limit,
temperature=args.temperature,
raw_template=raw_template,
num_shots=num_shots,
source_key="sources",
target_key="targets",
fewshots_key="fewshots",
use_cache=args.use_cache,
cache_dir=str(pathlib.Path(f"~/.cache/llm_client/{args.deployment}").expanduser()),
wait_time=args.response_wait_time,
)
data = data.map(
task_maker,
num_proc=args.num_workers,
desc=f"Predicting {num_shots} shots alignments using `{HfAlignment.__name__}`.",
remove_columns=_get_dataset(data).column_names,
)
evaluation = weave.Evaluation(dataset=data.to_list(), scorers=[evaluation_metrics])
model = AlignmentModel(
model_name=args.deployment,
prompt_template=raw_template,
temperature=args.temperature,
num_shots=num_shots,
)
print(asyncio.run(evaluation.evaluate(model)))

if __name__ == "__main__":
args = Arguments()
run(args)

Corti

New York | Copenhagen | London

frds · August 30, 2024, 4:11pm

Hey @motzwanted ,

thanks for sharing this – I have passed this along to one of our engineers working on weave and will let you know when I hear back.

system · September 9, 2024, 3:54pm

Hey Andreas,

Wanted to update you that following an investigation with one of the weave engineers we have identified a bug here that has to do with data size. We have logged this as a bug report and will keep you updated regarding the progress of this.

Please do let us know if you further help on this or anything else wandb related and thanks again for bringing this to our attention.

Best,

Frida

system · September 9, 2024, 4:17pm

Hey Andreas,

Further to the above, we have a branch that addresses the issue that you are running into which can be installed with

pip install git+https://github.com/wandb/weave.git@temp/table_page_size_reduction

Please note that this is not merged and as such there may be some untested behaviour that is subject to change (this is not yet an official release and should be used at your own discretion/risk) however wanted to give you the hot of the press work here while this can be worked on

Best,

F

Topic		Replies	Views
500 Error W&B Help	2	9	August 28, 2024
Error handling W&B Help sweeps , wandb	6	116	June 27, 2024
Whoa, I found a bug W&B Help sweeps , wandb	5	564	December 30, 2023
Weave crashes when calling autolog() W&B Help wandb	2	97	April 30, 2024
Why does wandb not always save my errors to the logs? (the ones usually printed to stdout, stderr) W&B Help	2	428	June 24, 2024

requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: /table/query. Reason

Related Topics