Glob may not walk above the base path

W&B Help
1

I run a neural network and I want to use Wandb to track the metrics. Here is my code :

from wandb.keras import WandbCallback

wandb.init(project="my-project", name="my-run")
logs_dir = os.path.join(wandb.run.dir, "logs")
wandb.save(logs_dir)

*** many things ***

model.fit(
    train_X, train_Y,
    validation_data=(test_X, test_Y),
    epochs=20,
    batch_size=32,
    callbacks=[WandbCallback()]
)

However I get this error, coming from “callbacks=[WandbCallback()]” :

ValueError: Glob may not walk above the base path

Any ideas ?

2

Hi @thibault-labergerie , thank you for writing in. What version of wandb are you currently using?

Could you please provide the debug.log and debug-internal.log files associated with the run where you are running into this issue? These files should be located in the wandb folder relative to your working directory.

3

Hello, thanks for answering.

I am using wandb = 0.16.5 (and keras = 3.3.3).

Here is debug.log :

2024-05-21 17:08:32,038 INFO    MainThread:13408 [wandb_setup.py:_flush():76] Current SDK version is 0.16.5
2024-05-21 17:08:32,038 INFO    MainThread:13408 [wandb_setup.py:_flush():76] Configure stats pid to 13408
2024-05-21 17:08:32,038 INFO    MainThread:13408 [wandb_setup.py:_flush():76] Loading settings from C:\Users\Thibault\.config\wandb\settings
2024-05-21 17:08:32,038 INFO    MainThread:13408 [wandb_setup.py:_flush():76] Loading settings from C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\settings
2024-05-21 17:08:32,038 INFO    MainThread:13408 [wandb_setup.py:_flush():76] Loading settings from environment variables: {}
2024-05-21 17:08:32,038 INFO    MainThread:13408 [wandb_setup.py:_flush():76] Applying setup settings: {'_disable_service': False}
2024-05-21 17:08:32,039 INFO    MainThread:13408 [wandb_setup.py:_flush():76] Inferring run settings from compute environment: {'program_relpath': '..\\..\\..\\..\\Documents\\DeepReinforcement\\model_LSTM.py', 'program_abspath': 'C:\\Users\\Thibault\\Documents\\DeepReinforcement\\model_LSTM.py', 'program': 'C:\\Users\\Thibault\\Documents\\DeepReinforcement\\model_LSTM.py'}
2024-05-21 17:08:32,039 INFO    MainThread:13408 [wandb_setup.py:_flush():76] Applying login settings: {}
2024-05-21 17:08:32,039 INFO    MainThread:13408 [wandb_init.py:_log_setup():527] Logging user logs to C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\logs\debug.log
2024-05-21 17:08:32,039 INFO    MainThread:13408 [wandb_init.py:_log_setup():528] Logging internal logs to C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\logs\debug-internal.log
2024-05-21 17:08:32,039 INFO    MainThread:13408 [wandb_init.py:init():567] calling init triggers
2024-05-21 17:08:32,039 INFO    MainThread:13408 [wandb_init.py:init():574] wandb.init called with sweep_config: {}
config: {}
2024-05-21 17:08:32,039 INFO    MainThread:13408 [wandb_init.py:init():617] starting backend
2024-05-21 17:08:32,039 INFO    MainThread:13408 [wandb_init.py:init():621] setting up manager
2024-05-21 17:08:32,041 INFO    MainThread:13408 [backend.py:_multiprocessing_setup():105] multiprocessing start_methods=spawn, using: spawn
2024-05-21 17:08:32,042 INFO    MainThread:13408 [wandb_init.py:init():629] backend started and connected
2024-05-21 17:08:32,045 INFO    MainThread:13408 [wandb_init.py:init():721] updated telemetry
2024-05-21 17:08:32,046 INFO    MainThread:13408 [wandb_init.py:init():754] communicating run to backend with 90.0 second timeout
2024-05-21 17:08:32,381 INFO    MainThread:13408 [wandb_run.py:_on_init():2344] communicating current version
2024-05-21 17:08:32,436 INFO    MainThread:13408 [wandb_run.py:_on_init():2353] got version response upgrade_message: "wandb version 0.17.0 is available!  To upgrade, please run:\n $ pip install wandb --upgrade"

2024-05-21 17:08:32,436 INFO    MainThread:13408 [wandb_init.py:init():805] starting run threads in backend
2024-05-21 17:08:32,668 INFO    MainThread:13408 [wandb_run.py:_console_start():2323] atexit reg
2024-05-21 17:08:32,668 INFO    MainThread:13408 [wandb_run.py:_redirect():2178] redirect: wrap_raw
2024-05-21 17:08:32,668 INFO    MainThread:13408 [wandb_run.py:_redirect():2243] Wrapping output streams.
2024-05-21 17:08:32,668 INFO    MainThread:13408 [wandb_run.py:_redirect():2268] Redirects installed.
2024-05-21 17:08:32,668 INFO    MainThread:13408 [wandb_init.py:init():848] run started, returning control to user process
2024-05-21 17:08:40,598 WARNING MsgRouterThr:13408 [router.py:message_loop():77] message_loop has been closed

and debug-internal :

2024-05-21 17:08:32,042 INFO    StreamThr :13172 [internal.py:wandb_internal():86] W&B internal server running at pid: 13172, started at: 2024-05-21 17:08:32.041030
2024-05-21 17:08:32,043 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: status
2024-05-21 17:08:32,045 INFO    WriterThread:13172 [datastore.py:open_for_write():87] open: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\run-rk6q9dw5.wandb
2024-05-21 17:08:32,046 DEBUG   SenderThread:13172 [sender.py:send():379] send: header
2024-05-21 17:08:32,047 DEBUG   SenderThread:13172 [sender.py:send():379] send: run
2024-05-21 17:08:32,370 INFO    SenderThread:13172 [dir_watcher.py:__init__():211] watching files in: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files
2024-05-21 17:08:32,370 INFO    SenderThread:13172 [sender.py:_start_run_threads():1124] run started: rk6q9dw5 with start time 1716304112.042031
2024-05-21 17:08:32,381 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: check_version
2024-05-21 17:08:32,382 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: check_version
2024-05-21 17:08:32,441 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: run_start
2024-05-21 17:08:32,455 DEBUG   HandlerThread:13172 [system_info.py:__init__():26] System info init
2024-05-21 17:08:32,455 DEBUG   HandlerThread:13172 [system_info.py:__init__():41] System info init done
2024-05-21 17:08:32,455 INFO    HandlerThread:13172 [system_monitor.py:start():194] Starting system monitor
2024-05-21 17:08:32,456 INFO    SystemMonitor:13172 [system_monitor.py:_start():158] Starting system asset monitoring threads
2024-05-21 17:08:32,456 INFO    HandlerThread:13172 [system_monitor.py:probe():214] Collecting system info
2024-05-21 17:08:32,459 INFO    SystemMonitor:13172 [interfaces.py:start():190] Started cpu monitoring
2024-05-21 17:08:32,460 INFO    SystemMonitor:13172 [interfaces.py:start():190] Started disk monitoring
2024-05-21 17:08:32,461 INFO    SystemMonitor:13172 [interfaces.py:start():190] Started gpu monitoring
2024-05-21 17:08:32,466 INFO    SystemMonitor:13172 [interfaces.py:start():190] Started memory monitoring
2024-05-21 17:08:32,476 INFO    SystemMonitor:13172 [interfaces.py:start():190] Started network monitoring
2024-05-21 17:08:32,487 DEBUG   HandlerThread:13172 [system_info.py:probe():150] Probing system
2024-05-21 17:08:32,488 DEBUG   HandlerThread:13172 [gitlib.py:_init_repo():56] git repository is invalid
2024-05-21 17:08:32,489 DEBUG   HandlerThread:13172 [system_info.py:probe():198] Probing system done
2024-05-21 17:08:32,489 DEBUG   HandlerThread:13172 [system_monitor.py:probe():223] {'os': 'Windows-10-10.0.19041-SP0', 'python': '3.9.4', 'heartbeatAt': '2024-05-21T15:08:32.487137', 'startedAt': '2024-05-21T15:08:32.036568', 'docker': None, 'cuda': None, 'args': (), 'state': 'running', 'program': 'C:\\Users\\Thibault\\Documents\\DeepReinforcement\\model_LSTM.py', 'codePathLocal': '..\\..\\..\\..\\Documents\\DeepReinforcement\\model_LSTM.py', 'codePath': '..\\..\\..\\..\\Documents\\DeepReinforcement\\model_LSTM.py', 'host': 'DESKTOP-G6UPN3B', 'username': 'Thibault', 'executable': 'C:\\Users\\Thibault\\AppData\\Local\\Programs\\Python\\Python39\\python.exe', 'cpu_count': 6, 'cpu_count_logical': 12, 'cpu_freq': {'current': 3600.0, 'min': 0.0, 'max': 3600.0}, 'cpu_freq_per_core': [{'current': 3600.0, 'min': 0.0, 'max': 3600.0}], 'disk': {'/': {'total': 222.95411682128906, 'used': 196.4355239868164}}, 'gpu': 'NVIDIA GeForce GTX 1050 Ti', 'gpu_count': 1, 'gpu_devices': [{'name': 'NVIDIA GeForce GTX 1050 Ti', 'memory_total': 4294967296}], 'memory': {'total': 15.927452087402344}}
2024-05-21 17:08:32,489 INFO    HandlerThread:13172 [system_monitor.py:probe():224] Finished collecting system info
2024-05-21 17:08:32,489 INFO    HandlerThread:13172 [system_monitor.py:probe():227] Publishing system info
2024-05-21 17:08:32,489 INFO    HandlerThread:13172 [system_monitor.py:probe():229] Finished publishing system info
2024-05-21 17:08:32,493 ERROR   gpu       :13172 [interfaces.py:monitor():144] Failed to sample metric: Not Supported
2024-05-21 17:08:32,493 ERROR   gpu       :13172 [interfaces.py:monitor():144] Failed to sample metric: Not Supported
2024-05-21 17:08:32,498 DEBUG   SenderThread:13172 [sender.py:send():379] send: files
2024-05-21 17:08:32,498 INFO    SenderThread:13172 [sender.py:_save_file():1390] saving file wandb-metadata.json with policy now
2024-05-21 17:08:32,666 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: python_packages
2024-05-21 17:08:32,667 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: python_packages
2024-05-21 17:08:32,667 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: stop_status
2024-05-21 17:08:32,668 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: stop_status
2024-05-21 17:08:32,669 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-21 17:08:32,863 DEBUG   SenderThread:13172 [sender.py:send():379] send: telemetry
2024-05-21 17:08:33,011 INFO    wandb-upload_0:13172 [upload_job.py:push():131] Uploaded file C:\Users\Thibault\AppData\Local\Temp\tmpfax5xylhwandb\7h3tvy40-wandb-metadata.json
2024-05-21 17:08:33,375 INFO    Thread-16 :13172 [dir_watcher.py:_on_file_created():271] file/dir created: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\output.log
2024-05-21 17:08:33,375 INFO    Thread-16 :13172 [dir_watcher.py:_on_file_created():271] file/dir created: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\wandb-metadata.json
2024-05-21 17:08:33,375 INFO    Thread-16 :13172 [dir_watcher.py:_on_file_created():271] file/dir created: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\requirements.txt
2024-05-21 17:08:34,511 ERROR   gpu       :13172 [interfaces.py:monitor():144] Failed to sample metric: Not Supported
2024-05-21 17:08:34,511 ERROR   gpu       :13172 [interfaces.py:monitor():144] Failed to sample metric: Not Supported
2024-05-21 17:08:34,775 DEBUG   SenderThread:13172 [sender.py:send():379] send: telemetry
2024-05-21 17:08:34,779 DEBUG   SenderThread:13172 [sender.py:send():379] send: exit
2024-05-21 17:08:34,779 INFO    SenderThread:13172 [sender.py:send_exit():586] handling exit code: 1
2024-05-21 17:08:34,779 INFO    SenderThread:13172 [sender.py:send_exit():588] handling runtime: 2
2024-05-21 17:08:34,780 INFO    SenderThread:13172 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
2024-05-21 17:08:34,781 INFO    SenderThread:13172 [sender.py:send_exit():594] send defer
2024-05-21 17:08:34,781 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:34,781 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 0
2024-05-21 17:08:34,781 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:34,781 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 0
2024-05-21 17:08:34,781 INFO    SenderThread:13172 [sender.py:transition_state():614] send defer: 1
2024-05-21 17:08:34,781 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:34,781 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 1
2024-05-21 17:08:34,782 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:34,782 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 1
2024-05-21 17:08:34,782 INFO    SenderThread:13172 [sender.py:transition_state():614] send defer: 2
2024-05-21 17:08:34,782 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:34,782 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 2
2024-05-21 17:08:34,782 INFO    HandlerThread:13172 [system_monitor.py:finish():203] Stopping system monitor
2024-05-21 17:08:34,782 DEBUG   SystemMonitor:13172 [system_monitor.py:_start():172] Starting system metrics aggregation loop
2024-05-21 17:08:34,782 INFO    HandlerThread:13172 [interfaces.py:finish():202] Joined cpu monitor
2024-05-21 17:08:34,782 DEBUG   SystemMonitor:13172 [system_monitor.py:_start():179] Finished system metrics aggregation loop
2024-05-21 17:08:34,788 INFO    HandlerThread:13172 [interfaces.py:finish():202] Joined disk monitor
2024-05-21 17:08:34,788 DEBUG   SystemMonitor:13172 [system_monitor.py:_start():183] Publishing last batch of metrics
2024-05-21 17:08:34,812 ERROR   gpu       :13172 [interfaces.py:aggregate():161] Failed to serialize metric: division by zero
2024-05-21 17:08:34,812 INFO    HandlerThread:13172 [interfaces.py:finish():202] Joined gpu monitor
2024-05-21 17:08:34,812 INFO    HandlerThread:13172 [interfaces.py:finish():202] Joined memory monitor
2024-05-21 17:08:34,812 INFO    HandlerThread:13172 [interfaces.py:finish():202] Joined network monitor
2024-05-21 17:08:34,813 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:34,813 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 2
2024-05-21 17:08:34,813 INFO    SenderThread:13172 [sender.py:transition_state():614] send defer: 3
2024-05-21 17:08:34,813 DEBUG   SenderThread:13172 [sender.py:send():379] send: stats
2024-05-21 17:08:34,813 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:34,813 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 3
2024-05-21 17:08:34,813 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:34,813 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 3
2024-05-21 17:08:34,814 INFO    SenderThread:13172 [sender.py:transition_state():614] send defer: 4
2024-05-21 17:08:34,814 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:34,814 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 4
2024-05-21 17:08:34,814 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:34,814 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 4
2024-05-21 17:08:34,814 INFO    SenderThread:13172 [sender.py:transition_state():614] send defer: 5
2024-05-21 17:08:34,814 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:34,814 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 5
2024-05-21 17:08:34,814 DEBUG   SenderThread:13172 [sender.py:send():379] send: summary
2024-05-21 17:08:34,815 INFO    SenderThread:13172 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
2024-05-21 17:08:34,815 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:34,815 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 5
2024-05-21 17:08:34,815 INFO    SenderThread:13172 [sender.py:transition_state():614] send defer: 6
2024-05-21 17:08:34,815 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:34,815 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 6
2024-05-21 17:08:34,815 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:34,815 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 6
2024-05-21 17:08:34,818 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: status_report
2024-05-21 17:08:34,959 INFO    SenderThread:13172 [sender.py:transition_state():614] send defer: 7
2024-05-21 17:08:34,959 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:34,959 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 7
2024-05-21 17:08:34,960 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:34,960 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 7
2024-05-21 17:08:35,389 INFO    Thread-16 :13172 [dir_watcher.py:_on_file_modified():288] file/dir modified: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\config.yaml
2024-05-21 17:08:35,390 INFO    Thread-16 :13172 [dir_watcher.py:_on_file_modified():288] file/dir modified: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\output.log
2024-05-21 17:08:35,390 INFO    Thread-16 :13172 [dir_watcher.py:_on_file_created():271] file/dir created: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\wandb-summary.json
2024-05-21 17:08:35,785 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: poll_exit
2024-05-21 17:08:36,390 INFO    Thread-16 :13172 [dir_watcher.py:_on_file_modified():288] file/dir modified: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\output.log
2024-05-21 17:08:37,395 INFO    Thread-16 :13172 [dir_watcher.py:_on_file_modified():288] file/dir modified: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\output.log
2024-05-21 17:08:37,426 INFO    SenderThread:13172 [sender.py:transition_state():614] send defer: 8
2024-05-21 17:08:37,426 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: poll_exit
2024-05-21 17:08:37,426 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:37,426 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 8
2024-05-21 17:08:37,426 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:37,426 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 8
2024-05-21 17:08:37,427 INFO    SenderThread:13172 [job_builder.py:build():318] Attempting to build job artifact
2024-05-21 17:08:37,427 INFO    SenderThread:13172 [job_builder.py:_get_source_type():466] no source found
2024-05-21 17:08:37,427 INFO    SenderThread:13172 [sender.py:transition_state():614] send defer: 9
2024-05-21 17:08:37,427 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:37,427 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 9
2024-05-21 17:08:37,427 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:37,428 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 9
2024-05-21 17:08:37,428 INFO    SenderThread:13172 [dir_watcher.py:finish():358] shutting down directory watcher
2024-05-21 17:08:37,790 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: poll_exit
2024-05-21 17:08:38,410 INFO    SenderThread:13172 [dir_watcher.py:_on_file_modified():288] file/dir modified: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\output.log
2024-05-21 17:08:38,411 INFO    SenderThread:13172 [dir_watcher.py:finish():388] scan: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files
2024-05-21 17:08:38,411 INFO    SenderThread:13172 [dir_watcher.py:finish():402] scan save: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\config.yaml config.yaml
2024-05-21 17:08:38,411 INFO    SenderThread:13172 [dir_watcher.py:finish():402] scan save: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\output.log output.log
2024-05-21 17:08:38,413 INFO    SenderThread:13172 [dir_watcher.py:finish():402] scan save: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\requirements.txt requirements.txt
2024-05-21 17:08:38,415 INFO    SenderThread:13172 [dir_watcher.py:finish():402] scan save: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\wandb-metadata.json wandb-metadata.json
2024-05-21 17:08:38,416 INFO    SenderThread:13172 [dir_watcher.py:finish():402] scan save: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\wandb-summary.json wandb-summary.json
2024-05-21 17:08:38,418 INFO    SenderThread:13172 [sender.py:transition_state():614] send defer: 10
2024-05-21 17:08:38,418 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: poll_exit
2024-05-21 17:08:38,419 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:38,419 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 10
2024-05-21 17:08:38,421 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:38,421 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 10
2024-05-21 17:08:38,421 INFO    SenderThread:13172 [file_pusher.py:finish():172] shutting down file pusher
2024-05-21 17:08:38,768 INFO    wandb-upload_0:13172 [upload_job.py:push():131] Uploaded file C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\config.yaml
2024-05-21 17:08:38,803 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: poll_exit
2024-05-21 17:08:38,803 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: poll_exit
2024-05-21 17:08:38,824 INFO    wandb-upload_3:13172 [upload_job.py:push():131] Uploaded file C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\wandb-summary.json
2024-05-21 17:08:38,890 INFO    wandb-upload_1:13172 [upload_job.py:push():131] Uploaded file C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\output.log
2024-05-21 17:08:39,076 INFO    wandb-upload_2:13172 [upload_job.py:push():131] Uploaded file C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\files\requirements.txt
2024-05-21 17:08:39,287 INFO    Thread-15 :13172 [sender.py:transition_state():614] send defer: 11
2024-05-21 17:08:39,287 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:39,287 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 11
2024-05-21 17:08:39,288 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:39,288 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 11
2024-05-21 17:08:39,288 INFO    SenderThread:13172 [file_pusher.py:join():178] waiting for file pusher
2024-05-21 17:08:39,288 INFO    SenderThread:13172 [sender.py:transition_state():614] send defer: 12
2024-05-21 17:08:39,289 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:39,289 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 12
2024-05-21 17:08:39,289 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:39,289 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 12
2024-05-21 17:08:39,289 INFO    SenderThread:13172 [file_stream.py:finish():614] file stream finish called
2024-05-21 17:08:39,417 INFO    SenderThread:13172 [file_stream.py:finish():618] file stream finish is done
2024-05-21 17:08:39,417 INFO    SenderThread:13172 [sender.py:transition_state():614] send defer: 13
2024-05-21 17:08:39,418 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:39,418 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 13
2024-05-21 17:08:39,418 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:39,418 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 13
2024-05-21 17:08:39,418 INFO    SenderThread:13172 [sender.py:transition_state():614] send defer: 14
2024-05-21 17:08:39,418 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: defer
2024-05-21 17:08:39,419 DEBUG   SenderThread:13172 [sender.py:send():379] send: final
2024-05-21 17:08:39,419 INFO    HandlerThread:13172 [handler.py:handle_request_defer():172] handle defer: 14
2024-05-21 17:08:39,419 DEBUG   SenderThread:13172 [sender.py:send():379] send: footer
2024-05-21 17:08:39,419 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: defer
2024-05-21 17:08:39,419 INFO    SenderThread:13172 [sender.py:send_request_defer():610] handle sender defer: 14
2024-05-21 17:08:39,419 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: poll_exit
2024-05-21 17:08:39,420 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: poll_exit
2024-05-21 17:08:39,420 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: server_info
2024-05-21 17:08:39,420 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: poll_exit
2024-05-21 17:08:39,420 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: get_summary
2024-05-21 17:08:39,420 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: poll_exit
2024-05-21 17:08:39,421 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: sampled_history
2024-05-21 17:08:39,421 DEBUG   SenderThread:13172 [sender.py:send_request():406] send_request: server_info
2024-05-21 17:08:39,421 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: internal_messages
2024-05-21 17:08:39,545 INFO    MainThread:13172 [wandb_run.py:_footer_history_summary_info():3920] rendering history
2024-05-21 17:08:39,545 INFO    MainThread:13172 [wandb_run.py:_footer_history_summary_info():3952] rendering summary
2024-05-21 17:08:39,546 INFO    MainThread:13172 [wandb_run.py:_footer_sync_info():3879] logging synced files
2024-05-21 17:08:39,547 DEBUG   HandlerThread:13172 [handler.py:handle_request():146] handle_request: shutdown
2024-05-21 17:08:39,547 INFO    HandlerThread:13172 [handler.py:finish():866] shutting down handler
2024-05-21 17:08:40,430 INFO    WriterThread:13172 [datastore.py:close():296] close: C:\Users\Thibault\AppData\Local\atom\app-1.60.0\wandb\run-20240521_170832-rk6q9dw5\run-rk6q9dw5.wandb
2024-05-21 17:08:40,550 INFO    SenderThread:13172 [sender.py:finish():1546] shutting down sender
2024-05-21 17:08:40,550 INFO    SenderThread:13172 [file_pusher.py:finish():172] shutting down file pusher
2024-05-21 17:08:40,550 INFO    SenderThread:13172 [file_pusher.py:join():178] waiting for file pusher

Thanks for your help

4

Thank you so much for sending them over!

Could you try to update to latest Keras, wandb and try:

from wandb.integration.keras import WandbCallback

Sounds like we’ve had some changes on our end.

5

Hi there, I wanted to follow up on this request. Please let us know if we can be of further assistance or if your issue has been resolved.

6

Hello, unfortunately it does not work. Same problem mentionned here : callback - AttributeError: can't set attribute 'model' with WandbCallback in Kaggle (Keras 3.0.5, Wandb 0.16.4) - Stack Overflow

The solution is to use WandbMetricsLogger instead

1 Like
7

Thank you so much for the update! We will keep in mine that WandbMetricsLogger works instead. Would you like us to close this ticket out? How do you import WandbMetricsLogger?

8

Hi, since we have not heard back from you, we are going to close this request. If you would like to reopen the conversation, please let us know! Unfortunately, at the moment, we do not receive notifications if a thread reopens on Discourse. So, please feel free to create a new ticket regarding your concern if you’d like to continue the conversation.