2026-01-21 03:31:37,741 - distributed.worker - WARNING - Compute Failed Key: ('assign-2b4fe7e1e8495ba40104b3a03d635a65', 1) Function: subgraph_callable-1bbb92ff-7506-4aa6-ac47-eff1f0b0 args: ( source_id geometry 638 2794848621 POINT Z (447926.017 5900701.771 0.000)) kwargs: {} Exception: "GEOSException('IllegalArgumentException: point array must contain 0 or >1 elements\\n')"
2026-01-21 03:21:02,798 - distributed.worker - WARNING - Compute Failed Key: ('assign-1f48eafd0194490336db83dcf3900cc4', 1) Function: subgraph_callable-98ca5660-e3b5-457d-8ac6-3b5aa0f9 args: ( source_id geometry 297 2794848621 POINT Z (447926.017 5900701.771 0.000)) kwargs: {} Exception: "GEOSException('IllegalArgumentException: point array must contain 0 or >1 elements\\n')"
2026-01-19 15:02:53,837 - distributed.worker - WARNING - Compute Failed Key: shuffle-barrier-db5a10064d534ca37a757b32b7c9f12c Function: shuffle_barrier args: ('db5a10064d534ca37a757b32b7c9f12c', [6755, 6755, 6755, 6755, 6755, 6755, 6755, 6755, 6755, 6755, 6755, 6755, 6755, 6755, 6755, 6755]) kwargs: {} Exception: "RuntimeError('shuffle_barrier failed during shuffle db5a10064d534ca37a757b32b7c9f12c')"
2026-01-19 15:02:38,631 - distributed.worker - WARNING - Compute Failed Key: ('shuffle-transfer-dd0235e2b443afff522d4ae5c408aeda', 12) Function: shuffle_transfer args: ( schema_version detector_id ... heading_diff_cur_and_next _partitions 406 1 2794848467 ... -0.2 8 406 1 2794848514 ... 0.0 15 406 1 2794848443 ... 0.0 0 406 1 2794848509 ... 0.1 3 406 1 2794848489 ... 0.0 14 .. ... ... ... ... ... 439 1 2794848514 ... -0.7 15 439 1 2794848427 ... 0.0 2 439 1 2794848467 ... -0.5 8 439 1 2794848507 ... 0.0 3 439 1 2794848427 ... 0.0 2 [544 rows x 21 columns], 'dd0235e2b443afff522d4ae5c408aeda', 12, 16, '_par kwargs: {} Exception: "RuntimeError('shuffle_transfer failed during shuffle dd0235e2b443afff522d4ae5c408aeda')"
2026-01-19 12:31:04,998 - distributed.worker - WARNING - Compute Failed Key: ('hash-join-e286c972f21d9925673d10a728b4b70c', 12) Function: merge_unpack args: ('007491f1c91dcc736549d14c5ba9c6fb', 'fc064e7c7946a6aef7f5f25eeaec3b11', 12, 6511, 6513, 'inner', 'hashed_source_and_date', 'hashed_source_and_date', <distributed.protocol.serialize.Serialized object at 0x7f84fa1cd570>, ['_x', '_y']) kwargs: {} Exception: "RuntimeError('Worker tcp://172.21.25.37:44341 left during active shuffle 007491f1c91dcc736549d14c5ba9c6fb')"
2026-01-18 10:31:48,736 - distributed.worker - WARNING - Compute Failed Key: ('shuffle-transfer-ebc8095d3f53a53e1bc1d381943a9194', 1) Function: shuffle_transfer args: ( source_id timestamp ... date_diff_ms _partitions 0 2794848541 1768731589407 ... 16.0 13 1 2794848541 1768731590761 ... 13.0 13 2 2794848541 1768731597854 ... 381.0 13 3 2794848541 1768731601639 ... 23.0 13 4 2794848541 1768731604093 ... 20.0 13 ... ... ... ... ... ... 2070994 2794848614 1768732093637 ... 430.0 6 2070995 2794848614 1768732132140 ... 824.0 6 2070996 2794848614 1768732137865 ... 863.0 6 2070997 2794848614 1768732146551 ... 1005.0 6 2070998 2794848614 1768732186890 ... 658.0 6 [2070999 rows x 5 columns], 'ebc8095d3f53a53e1bc1d381943a9194', 1, 16, '_partitions') kwargs: {} Exception: "RuntimeError('shuffle_transfer failed during shuffle ebc8095d3f53a53e1bc1d381943a9194')"
2026-01-17 03:03:33,522 - distributed.worker - INFO - -------------------------------------------------
2026-01-17 03:03:33,522 - distributed.worker - INFO - Registered to: tcp://dask-scheduler:8786
2026-01-17 03:03:33,147 - distributed.worker - INFO - -------------------------------------------------
2026-01-17 03:03:33,146 - distributed.worker - INFO - Local Directory: /tmp/dask-worker-space/worker-07uie_gf
2026-01-17 03:03:33,146 - distributed.worker - INFO - Memory: 3.73 GiB
2026-01-17 03:03:33,146 - distributed.worker - INFO - Threads: 1
2026-01-17 03:03:33,146 - distributed.worker - INFO - -------------------------------------------------
2026-01-17 03:03:33,146 - distributed.worker - INFO - Waiting to connect to: tcp://dask-scheduler:8786
2026-01-17 03:03:33,146 - distributed.worker - INFO - dashboard at: 172.21.151.165:8790
2026-01-17 03:03:33,145 - distributed.worker - INFO - Listening to: tcp://172.21.151.165:40415
2026-01-17 03:03:33,145 - distributed.worker - INFO - Start worker at: tcp://172.21.151.165:40415