2025-12-06 02:20:15,320 - distributed.scheduler - INFO - Receive client connection: Client-19eed1c8-d24a-11f0-8001-f2cd832f225c

2025-12-06 02:10:13,753 - distributed.scheduler - INFO - Receive client connection: Client-b35f25d6-d248-11f0-8001-f2cd832f225c

2025-12-06 02:00:12,286 - distributed.scheduler - INFO - Receive client connection: Client-4cde8692-d247-11f0-8001-f2cd832f225c

2025-12-06 01:50:10,802 - distributed.scheduler - INFO - Receive client connection: Client-e65b6e8a-d245-11f0-8001-f2cd832f225c

2025-12-06 01:40:09,425 - distributed.scheduler - INFO - Receive client connection: Client-7fe7d0ba-d244-11f0-8001-f2cd832f225c

2025-12-06 01:30:08,022 - distributed.scheduler - INFO - Receive client connection: Client-1971801f-d243-11f0-8001-f2cd832f225c

2025-12-06 01:20:06,580 - distributed.scheduler - INFO - Receive client connection: Client-b2f4cc62-d241-11f0-8001-f2cd832f225c

2025-12-06 01:10:05,007 - distributed.scheduler - INFO - Receive client connection: Client-4c641de0-d240-11f0-8001-f2cd832f225c

2025-12-06 01:00:03,503 - distributed.scheduler - INFO - Receive client connection: Client-e5de4ea4-d23e-11f0-8001-f2cd832f225c

2025-12-06 00:50:02,025 - distributed.scheduler - INFO - Receive client connection: Client-7f5c3ccb-d23d-11f0-8001-f2cd832f225c

2025-12-06 00:40:00,741 - distributed.scheduler - INFO - Receive client connection: Client-18f6f64f-d23c-11f0-8001-f2cd832f225c

2025-12-06 00:30:59,449 - distributed.scheduler - INFO - Receive client connection: Client-d65551c2-d23a-11f0-8001-f2cd832f225c

2025-12-06 00:20:57,987 - distributed.scheduler - INFO - Receive client connection: Client-6fd4d81e-d239-11f0-8001-f2cd832f225c

2025-12-06 00:10:56,672 - distributed.scheduler - INFO - Receive client connection: Client-096ae5ed-d238-11f0-8001-f2cd832f225c

2025-12-06 00:00:55,176 - distributed.scheduler - INFO - Receive client connection: Client-a2e71b05-d236-11f0-8001-f2cd832f225c

2025-12-05 23:50:53,801 - distributed.scheduler - INFO - Receive client connection: Client-3c742abc-d235-11f0-8001-f2cd832f225c

2025-12-05 23:40:52,439 - distributed.scheduler - INFO - Receive client connection: Client-d6029972-d233-11f0-8001-f2cd832f225c

2025-12-05 23:30:50,933 - distributed.scheduler - INFO - Receive client connection: Client-6f7c92f4-d232-11f0-8001-f2cd832f225c

2025-12-05 23:20:49,283 - distributed.scheduler - INFO - Receive client connection: Client-08e0194b-d231-11f0-8001-f2cd832f225c

2025-12-05 23:10:47,899 - distributed.scheduler - INFO - Receive client connection: Client-a26b1b1b-d22f-11f0-8001-f2cd832f225c

2025-12-05 23:00:46,428 - distributed.scheduler - INFO - Receive client connection: Client-3bec4fe5-d22e-11f0-8001-f2cd832f225c

2025-12-05 22:50:44,932 - distributed.scheduler - INFO - Receive client connection: Client-d56651c2-d22c-11f0-8001-f2cd832f225c

2025-12-05 22:40:43,446 - distributed.scheduler - INFO - Receive client connection: Client-6ee308ad-d22b-11f0-8001-f2cd832f225c

2025-12-05 22:30:41,979 - distributed.scheduler - INFO - Receive client connection: Client-08617cbb-d22a-11f0-8001-f2cd832f225c

2025-12-05 22:20:40,524 - distributed.scheduler - INFO - Receive client connection: Client-a1e379d4-d228-11f0-8001-f2cd832f225c

2025-12-05 22:10:39,134 - distributed.scheduler - INFO - Receive client connection: Client-3b6e7ab3-d227-11f0-8001-f2cd832f225c

2025-12-05 22:00:37,519 - distributed.scheduler - INFO - Receive client connection: Client-d4d6a3d4-d225-11f0-8001-f2cd832f225c

2025-12-05 21:50:36,010 - distributed.scheduler - INFO - Receive client connection: Client-6e504a3d-d224-11f0-8001-f2cd832f225c

2025-12-05 21:40:34,481 - distributed.scheduler - INFO - Receive client connection: Client-07c606dd-d223-11f0-8001-f2cd832f225c

2025-12-05 21:30:33,105 - distributed.scheduler - INFO - Receive client connection: Client-a153cd0e-d221-11f0-8001-f2cd832f225c

2025-12-05 21:20:31,676 - distributed.scheduler - INFO - Receive client connection: Client-3ad97b7c-d220-11f0-8001-f2cd832f225c

2025-12-05 21:10:30,304 - distributed.scheduler - INFO - Receive client connection: Client-d466999f-d21e-11f0-8001-f2cd832f225c

2025-12-05 21:00:28,875 - distributed.scheduler - INFO - Receive client connection: Client-6deb6b4e-d21d-11f0-8001-f2cd832f225c

2025-12-05 20:50:27,500 - distributed.scheduler - INFO - Receive client connection: Client-07792a87-d21c-11f0-8001-f2cd832f225c

2025-12-05 20:40:26,137 - distributed.scheduler - INFO - Receive client connection: Client-a108d821-d21a-11f0-8001-f2cd832f225c

2025-12-05 20:30:24,724 - distributed.scheduler - INFO - Receive client connection: Client-3a8dce1f-d219-11f0-8001-f2cd832f225c

2025-12-05 20:20:23,277 - distributed.scheduler - INFO - Receive client connection: Client-d412be21-d217-11f0-8001-f2cd832f225c

2025-12-05 20:10:21,847 - distributed.scheduler - INFO - Receive client connection: Client-6d97e7d3-d216-11f0-8001-f2cd832f225c

2025-12-05 20:00:20,365 - distributed.scheduler - INFO - Receive client connection: Client-07147d43-d215-11f0-8001-f2cd832f225c

2025-12-05 19:50:18,925 - distributed.scheduler - INFO - Receive client connection: Client-a0983739-d213-11f0-8001-f2cd832f225c

2025-12-05 19:40:17,562 - distributed.scheduler - INFO - Receive client connection: Client-3a27c044-d212-11f0-8001-f2cd832f225c

2025-12-05 19:30:16,109 - distributed.scheduler - INFO - Receive client connection: Client-d3a93349-d210-11f0-8001-f2cd832f225c

2025-12-05 19:20:14,583 - distributed.scheduler - INFO - Receive client connection: Client-6d1ff394-d20f-11f0-8001-f2cd832f225c

2025-12-05 19:10:13,112 - distributed.scheduler - INFO - Receive client connection: Client-069f150f-d20e-11f0-8001-f2cd832f225c

2025-12-05 19:00:11,601 - distributed.scheduler - INFO - Receive client connection: Client-a0166227-d20c-11f0-8001-f2cd832f225c

2025-12-05 18:50:10,128 - distributed.scheduler - INFO - Receive client connection: Client-39959fd0-d20b-11f0-8001-f2cd832f225c

2025-12-05 18:40:08,709 - distributed.scheduler - INFO - Receive client connection: Client-d31c47ce-d209-11f0-8001-f2cd832f225c

2025-12-05 18:30:07,305 - distributed.scheduler - INFO - Receive client connection: Client-6ca62435-d208-11f0-8001-f2cd832f225c

2025-12-05 18:20:05,879 - distributed.scheduler - INFO - Receive client connection: Client-062bedcd-d207-11f0-8001-f2cd832f225c

2025-12-05 18:10:04,414 - distributed.scheduler - INFO - Receive client connection: Client-9faaba17-d205-11f0-8001-f2cd832f225c

2025-12-05 18:01:11,993 - distributed.scheduler - INFO - Close client connection: Client-39214de7-d204-11f0-8001-f2cd832f225c

2025-12-05 18:01:11,992 - distributed.scheduler - INFO - Remove client Client-39214de7-d204-11f0-8001-f2cd832f225c

2025-12-05 18:01:11,990 - distributed.scheduler - INFO - Remove client Client-39214de7-d204-11f0-8001-f2cd832f225c

2025-12-05 18:00:02,892 - distributed.scheduler - INFO - Receive client connection: Client-39214de7-d204-11f0-8001-f2cd832f225c

2025-12-05 17:50:39,881 - distributed.scheduler - INFO - Close client connection: Client-d2912b7a-d202-11f0-8001-f2cd832f225c

2025-12-05 17:50:39,879 - distributed.scheduler - INFO - Remove client Client-d2912b7a-d202-11f0-8001-f2cd832f225c

2025-12-05 17:50:39,877 - distributed.scheduler - INFO - Remove client Client-d2912b7a-d202-11f0-8001-f2cd832f225c

2025-12-05 17:50:01,327 - distributed.scheduler - INFO - Receive client connection: Client-d2912b7a-d202-11f0-8001-f2cd832f225c

2025-12-05 17:41:38,971 - distributed.scheduler - INFO - Close client connection: Client-8fe774a4-d201-11f0-8001-f2cd832f225c

2025-12-05 17:41:38,970 - distributed.scheduler - INFO - Remove client Client-8fe774a4-d201-11f0-8001-f2cd832f225c

2025-12-05 17:41:38,968 - distributed.scheduler - INFO - Remove client Client-8fe774a4-d201-11f0-8001-f2cd832f225c

2025-12-05 17:40:59,989 - distributed.scheduler - INFO - Receive client connection: Client-8fe774a4-d201-11f0-8001-f2cd832f225c

2025-12-05 17:32:10,126 - distributed.scheduler - INFO - Close client connection: Client-29675329-d200-11f0-8001-f2cd832f225c

2025-12-05 17:32:10,125 - distributed.scheduler - INFO - Remove client Client-29675329-d200-11f0-8001-f2cd832f225c

2025-12-05 17:32:10,123 - distributed.scheduler - INFO - Remove client Client-29675329-d200-11f0-8001-f2cd832f225c

2025-12-05 17:30:58,528 - distributed.scheduler - INFO - Receive client connection: Client-29675329-d200-11f0-8001-f2cd832f225c

2025-12-05 17:21:35,082 - distributed.scheduler - INFO - Close client connection: Client-c2ea9357-d1fe-11f0-8001-f2cd832f225c

2025-12-05 17:21:35,080 - distributed.scheduler - INFO - Remove client Client-c2ea9357-d1fe-11f0-8001-f2cd832f225c

2025-12-05 17:21:35,078 - distributed.scheduler - INFO - Remove client Client-c2ea9357-d1fe-11f0-8001-f2cd832f225c

2025-12-05 17:20:57,083 - distributed.scheduler - INFO - Receive client connection: Client-c2ea9357-d1fe-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,941 - distributed.scheduler - INFO - Close client connection: Client-5bf5ed54-d1f6-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,941 - distributed.scheduler - INFO - Close client connection: Client-f58247c0-d1f4-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,940 - distributed.scheduler - INFO - Close client connection: Client-c26fb4ec-d1f7-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,940 - distributed.scheduler - INFO - Close client connection: Client-28f0065e-d1f9-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,939 - distributed.scheduler - INFO - Close client connection: Client-8f714768-d1fa-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,939 - distributed.scheduler - INFO - Close client connection: Client-f5f23679-d1fb-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,937 - distributed.scheduler - INFO - Remove client Client-5bf5ed54-d1f6-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,936 - distributed.scheduler - INFO - Remove client Client-f58247c0-d1f4-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,935 - distributed.scheduler - INFO - Remove client Client-c26fb4ec-d1f7-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,935 - distributed.scheduler - INFO - Remove client Client-28f0065e-d1f9-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,934 - distributed.scheduler - INFO - Remove client Client-8f714768-d1fa-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,933 - distributed.scheduler - INFO - Remove client Client-f5f23679-d1fb-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,239 - distributed.scheduler - INFO - Close client connection: Client-5c700440-d1fd-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,237 - distributed.scheduler - INFO - Remove client Client-5c700440-d1fd-11f0-8001-f2cd832f225c

2025-12-05 17:11:36,235 - distributed.scheduler - INFO - Remove client Client-5c700440-d1fd-11f0-8001-f2cd832f225c

2025-12-05 17:10:55,640 - distributed.scheduler - INFO - Receive client connection: Client-5c700440-d1fd-11f0-8001-f2cd832f225c

2025-12-05 17:00:54,194 - distributed.scheduler - INFO - Receive client connection: Client-f5f23679-d1fb-11f0-8001-f2cd832f225c

2025-12-05 16:50:52,727 - distributed.scheduler - INFO - Receive client connection: Client-8f714768-d1fa-11f0-8001-f2cd832f225c

2025-12-05 16:40:51,253 - distributed.scheduler - INFO - Receive client connection: Client-28f0065e-d1f9-11f0-8001-f2cd832f225c

2025-12-05 16:30:49,784 - distributed.scheduler - INFO - Receive client connection: Client-c26fb4ec-d1f7-11f0-8001-f2cd832f225c

2025-12-05 16:20:48,366 - distributed.scheduler - INFO - Receive client connection: Client-5bf5ed54-d1f6-11f0-8001-f2cd832f225c

2025-12-05 16:10:47,000 - distributed.scheduler - INFO - Receive client connection: Client-f58247c0-d1f4-11f0-8001-f2cd832f225c

2025-12-05 16:02:03,278 - distributed.scheduler - INFO - Close client connection: Client-8f053793-d1f3-11f0-8001-f2cd832f225c

2025-12-05 16:02:03,277 - distributed.scheduler - INFO - Remove client Client-8f053793-d1f3-11f0-8001-f2cd832f225c

2025-12-05 16:02:03,275 - distributed.scheduler - INFO - Remove client Client-8f053793-d1f3-11f0-8001-f2cd832f225c

2025-12-05 16:00:45,551 - distributed.scheduler - INFO - Receive client connection: Client-8f053793-d1f3-11f0-8001-f2cd832f225c

2025-12-05 15:51:26,155 - distributed.scheduler - INFO - Close client connection: Client-28838462-d1f2-11f0-8001-f2cd832f225c

2025-12-05 15:51:26,154 - distributed.scheduler - INFO - Remove client Client-28838462-d1f2-11f0-8001-f2cd832f225c

2025-12-05 15:51:26,152 - distributed.scheduler - INFO - Remove client Client-28838462-d1f2-11f0-8001-f2cd832f225c

2025-12-05 15:50:44,078 - distributed.scheduler - INFO - Receive client connection: Client-28838462-d1f2-11f0-8001-f2cd832f225c

2025-12-05 15:41:25,734 - distributed.scheduler - INFO - Close client connection: Client-c1f68705-d1f0-11f0-8001-f2cd832f225c

2025-12-05 15:41:25,732 - distributed.scheduler - INFO - Remove client Client-c1f68705-d1f0-11f0-8001-f2cd832f225c

2025-12-05 15:41:25,730 - distributed.scheduler - INFO - Remove client Client-c1f68705-d1f0-11f0-8001-f2cd832f225c

2025-12-05 15:40:42,519 - distributed.scheduler - INFO - Receive client connection: Client-c1f68705-d1f0-11f0-8001-f2cd832f225c

2025-12-05 15:32:00,013 - distributed.scheduler - INFO - Close client connection: Client-5b757e1b-d1ef-11f0-8001-f2cd832f225c

2025-12-05 15:32:00,012 - distributed.scheduler - INFO - Remove client Client-5b757e1b-d1ef-11f0-8001-f2cd832f225c

2025-12-05 15:32:00,010 - distributed.scheduler - INFO - Remove client Client-5b757e1b-d1ef-11f0-8001-f2cd832f225c

2025-12-05 15:30:41,061 - distributed.scheduler - INFO - Receive client connection: Client-5b757e1b-d1ef-11f0-8001-f2cd832f225c

2025-12-05 15:21:23,997 - distributed.scheduler - INFO - Close client connection: Client-f4ff21d1-d1ed-11f0-8001-f2cd832f225c

2025-12-05 15:21:23,996 - distributed.scheduler - INFO - Remove client Client-f4ff21d1-d1ed-11f0-8001-f2cd832f225c

2025-12-05 15:21:23,995 - distributed.scheduler - INFO - Remove client Client-f4ff21d1-d1ed-11f0-8001-f2cd832f225c

2025-12-05 15:20:39,657 - distributed.scheduler - INFO - Receive client connection: Client-f4ff21d1-d1ed-11f0-8001-f2cd832f225c

2025-12-05 15:11:27,659 - distributed.scheduler - INFO - Close client connection: Client-8e7bd4a7-d1ec-11f0-8001-f2cd832f225c

2025-12-05 15:11:27,658 - distributed.scheduler - INFO - Remove client Client-8e7bd4a7-d1ec-11f0-8001-f2cd832f225c

2025-12-05 15:11:27,656 - distributed.scheduler - INFO - Remove client Client-8e7bd4a7-d1ec-11f0-8001-f2cd832f225c

2025-12-05 15:10:38,173 - distributed.scheduler - INFO - Receive client connection: Client-8e7bd4a7-d1ec-11f0-8001-f2cd832f225c

2025-12-05 15:01:50,655 - distributed.scheduler - INFO - Close client connection: Client-280aa9e2-d1eb-11f0-8001-f2cd832f225c

2025-12-05 15:01:50,653 - distributed.scheduler - INFO - Remove client Client-280aa9e2-d1eb-11f0-8001-f2cd832f225c

2025-12-05 15:01:50,651 - distributed.scheduler - INFO - Remove client Client-280aa9e2-d1eb-11f0-8001-f2cd832f225c

2025-12-05 15:00:36,812 - distributed.scheduler - INFO - Receive client connection: Client-280aa9e2-d1eb-11f0-8001-f2cd832f225c

2025-12-05 14:51:12,332 - distributed.scheduler - INFO - Close client connection: Client-c18747f4-d1e9-11f0-8001-f2cd832f225c

2025-12-05 14:51:12,331 - distributed.scheduler - INFO - Remove client Client-c18747f4-d1e9-11f0-8001-f2cd832f225c

2025-12-05 14:51:12,330 - distributed.scheduler - INFO - Remove client Client-c18747f4-d1e9-11f0-8001-f2cd832f225c

2025-12-05 14:50:35,318 - distributed.scheduler - INFO - Receive client connection: Client-c18747f4-d1e9-11f0-8001-f2cd832f225c

2025-12-05 14:41:08,258 - distributed.scheduler - INFO - Close client connection: Client-5b04c2e9-d1e8-11f0-8001-f2cd832f225c

2025-12-05 14:41:08,255 - distributed.scheduler - INFO - Remove client Client-5b04c2e9-d1e8-11f0-8001-f2cd832f225c

2025-12-05 14:41:08,254 - distributed.scheduler - INFO - Remove client Client-5b04c2e9-d1e8-11f0-8001-f2cd832f225c

2025-12-05 14:40:33,841 - distributed.scheduler - INFO - Receive client connection: Client-5b04c2e9-d1e8-11f0-8001-f2cd832f225c

2025-12-05 14:31:45,663 - distributed.scheduler - INFO - Close client connection: Client-f48aa1ef-d1e6-11f0-8001-f2cd832f225c

2025-12-05 14:31:45,662 - distributed.scheduler - INFO - Remove client Client-f48aa1ef-d1e6-11f0-8001-f2cd832f225c

2025-12-05 14:31:45,661 - distributed.scheduler - INFO - Remove client Client-f48aa1ef-d1e6-11f0-8001-f2cd832f225c

2025-12-05 14:30:32,420 - distributed.scheduler - INFO - Receive client connection: Client-f48aa1ef-d1e6-11f0-8001-f2cd832f225c

2025-12-05 14:21:06,482 - distributed.scheduler - INFO - Close client connection: Client-8e0927c0-d1e5-11f0-8001-f2cd832f225c

2025-12-05 14:21:06,481 - distributed.scheduler - INFO - Remove client Client-8e0927c0-d1e5-11f0-8001-f2cd832f225c

2025-12-05 14:21:06,480 - distributed.scheduler - INFO - Remove client Client-8e0927c0-d1e5-11f0-8001-f2cd832f225c

2025-12-05 14:20:30,947 - distributed.scheduler - INFO - Receive client connection: Client-8e0927c0-d1e5-11f0-8001-f2cd832f225c

2025-12-05 14:11:06,806 - distributed.scheduler - INFO - Close client connection: Client-2788561f-d1e4-11f0-8001-f2cd832f225c

2025-12-05 14:11:06,805 - distributed.scheduler - INFO - Remove client Client-2788561f-d1e4-11f0-8001-f2cd832f225c

2025-12-05 14:11:06,804 - distributed.scheduler - INFO - Remove client Client-2788561f-d1e4-11f0-8001-f2cd832f225c

2025-12-05 14:10:29,474 - distributed.scheduler - INFO - Receive client connection: Client-2788561f-d1e4-11f0-8001-f2cd832f225c

2025-12-05 14:01:37,767 - distributed.scheduler - INFO - Close client connection: Client-c10f5a8a-d1e2-11f0-8001-f2cd832f225c

2025-12-05 14:01:37,764 - distributed.scheduler - INFO - Remove client Client-c10f5a8a-d1e2-11f0-8001-f2cd832f225c

2025-12-05 14:01:37,763 - distributed.scheduler - INFO - Remove client Client-c10f5a8a-d1e2-11f0-8001-f2cd832f225c

2025-12-05 14:00:28,060 - distributed.scheduler - INFO - Receive client connection: Client-c10f5a8a-d1e2-11f0-8001-f2cd832f225c

2025-12-05 13:51:06,817 - distributed.scheduler - INFO - Close client connection: Client-5a9f8dd1-d1e1-11f0-8001-f2cd832f225c

2025-12-05 13:51:06,816 - distributed.scheduler - INFO - Remove client Client-5a9f8dd1-d1e1-11f0-8001-f2cd832f225c

2025-12-05 13:51:06,814 - distributed.scheduler - INFO - Remove client Client-5a9f8dd1-d1e1-11f0-8001-f2cd832f225c

2025-12-05 13:50:26,703 - distributed.scheduler - INFO - Receive client connection: Client-5a9f8dd1-d1e1-11f0-8001-f2cd832f225c

2025-12-05 13:41:06,799 - distributed.scheduler - INFO - Close client connection: Client-f41ef5e4-d1df-11f0-8001-f2cd832f225c

2025-12-05 13:41:06,799 - distributed.scheduler - INFO - Remove client Client-f41ef5e4-d1df-11f0-8001-f2cd832f225c

2025-12-05 13:41:06,798 - distributed.scheduler - INFO - Remove client Client-f41ef5e4-d1df-11f0-8001-f2cd832f225c

2025-12-05 13:40:25,230 - distributed.scheduler - INFO - Receive client connection: Client-f41ef5e4-d1df-11f0-8001-f2cd832f225c

2025-12-05 13:31:36,393 - distributed.scheduler - INFO - Close client connection: Client-2724af2a-d1dd-11f0-8001-f2cd832f225c

2025-12-05 13:31:36,391 - distributed.scheduler - INFO - Remove client Client-2724af2a-d1dd-11f0-8001-f2cd832f225c

2025-12-05 13:31:35,684 - distributed.scheduler - INFO - Close client connection: Client-8da63b95-d1de-11f0-8001-f2cd832f225c

2025-12-05 13:31:35,682 - distributed.scheduler - INFO - Remove client Client-8da63b95-d1de-11f0-8001-f2cd832f225c

2025-12-05 13:31:35,681 - distributed.scheduler - INFO - Remove client Client-8da63b95-d1de-11f0-8001-f2cd832f225c

2025-12-05 13:30:23,805 - distributed.scheduler - INFO - Receive client connection: Client-8da63b95-d1de-11f0-8001-f2cd832f225c

2025-12-05 13:20:22,349 - distributed.scheduler - INFO - Receive client connection: Client-2724af2a-d1dd-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,075 - distributed.scheduler - INFO - Close client connection: Client-8c3c0928-d1c9-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,074 - distributed.scheduler - INFO - Close client connection: Client-f2c2c896-d1ca-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,074 - distributed.scheduler - INFO - Close client connection: Client-594474de-d1cc-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,074 - distributed.scheduler - INFO - Close client connection: Client-bfc6796a-d1cd-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,074 - distributed.scheduler - INFO - Close client connection: Client-26433cba-d1cf-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,074 - distributed.scheduler - INFO - Close client connection: Client-f34257dc-d1d1-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,074 - distributed.scheduler - INFO - Close client connection: Client-8d26c1b5-d1d0-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,073 - distributed.scheduler - INFO - Close client connection: Client-59c43afb-d1d3-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,073 - distributed.scheduler - INFO - Close client connection: Client-c03c2f14-d1d4-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,073 - distributed.scheduler - INFO - Close client connection: Client-26b6f3e4-d1d6-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,072 - distributed.scheduler - INFO - Close client connection: Client-f3afa705-d1d8-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,072 - distributed.scheduler - INFO - Close client connection: Client-8d34e26d-d1d7-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,072 - distributed.scheduler - INFO - Close client connection: Client-5a2966a2-d1da-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,071 - distributed.scheduler - INFO - Remove client Client-8c3c0928-d1c9-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,070 - distributed.scheduler - INFO - Remove client Client-f2c2c896-d1ca-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,070 - distributed.scheduler - INFO - Remove client Client-594474de-d1cc-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,070 - distributed.scheduler - INFO - Remove client Client-bfc6796a-d1cd-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,069 - distributed.scheduler - INFO - Remove client Client-26433cba-d1cf-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,068 - distributed.scheduler - INFO - Remove client Client-f34257dc-d1d1-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,067 - distributed.scheduler - INFO - Remove client Client-8d26c1b5-d1d0-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,063 - distributed.scheduler - INFO - Remove client Client-59c43afb-d1d3-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,062 - distributed.scheduler - INFO - Remove client Client-c03c2f14-d1d4-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,062 - distributed.scheduler - INFO - Remove client Client-26b6f3e4-d1d6-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,061 - distributed.scheduler - INFO - Remove client Client-f3afa705-d1d8-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,060 - distributed.scheduler - INFO - Remove client Client-8d34e26d-d1d7-11f0-8001-f2cd832f225c

2025-12-05 13:10:59,059 - distributed.scheduler - INFO - Remove client Client-5a2966a2-d1da-11f0-8001-f2cd832f225c

2025-12-05 13:10:58,437 - distributed.scheduler - INFO - Close client connection: Client-c0af4a3c-d1db-11f0-8001-f2cd832f225c

2025-12-05 13:10:58,436 - distributed.scheduler - INFO - Remove client Client-c0af4a3c-d1db-11f0-8001-f2cd832f225c

2025-12-05 13:10:58,435 - distributed.scheduler - INFO - Remove client Client-c0af4a3c-d1db-11f0-8001-f2cd832f225c

2025-12-05 13:10:20,943 - distributed.scheduler - INFO - Receive client connection: Client-c0af4a3c-d1db-11f0-8001-f2cd832f225c

2025-12-05 13:00:19,435 - distributed.scheduler - INFO - Receive client connection: Client-5a2966a2-d1da-11f0-8001-f2cd832f225c

2025-12-05 12:50:18,019 - distributed.scheduler - INFO - Receive client connection: Client-f3afa705-d1d8-11f0-8001-f2cd832f225c

2025-12-05 12:40:16,585 - distributed.scheduler - INFO - Receive client connection: Client-8d34e26d-d1d7-11f0-8001-f2cd832f225c

2025-12-05 12:30:15,140 - distributed.scheduler - INFO - Receive client connection: Client-26b6f3e4-d1d6-11f0-8001-f2cd832f225c

2025-12-05 12:20:13,712 - distributed.scheduler - INFO - Receive client connection: Client-c03c2f14-d1d4-11f0-8001-f2cd832f225c

2025-12-05 12:10:12,299 - distributed.scheduler - INFO - Receive client connection: Client-59c43afb-d1d3-11f0-8001-f2cd832f225c

2025-12-05 12:00:10,829 - distributed.scheduler - INFO - Receive client connection: Client-f34257dc-d1d1-11f0-8001-f2cd832f225c

2025-12-05 11:50:10,017 - distributed.scheduler - INFO - Receive client connection: Client-8d26c1b5-d1d0-11f0-8001-f2cd832f225c

2025-12-05 11:40:07,907 - distributed.scheduler - INFO - Receive client connection: Client-26433cba-d1cf-11f0-8001-f2cd832f225c

2025-12-05 11:30:06,459 - distributed.scheduler - INFO - Receive client connection: Client-bfc6796a-d1cd-11f0-8001-f2cd832f225c

2025-12-05 11:20:04,987 - distributed.scheduler - INFO - Receive client connection: Client-594474de-d1cc-11f0-8001-f2cd832f225c

2025-12-05 11:10:03,514 - distributed.scheduler - INFO - Receive client connection: Client-f2c2c896-d1ca-11f0-8001-f2cd832f225c

2025-12-05 11:00:02,014 - distributed.scheduler - INFO - Receive client connection: Client-8c3c0928-d1c9-11f0-8001-f2cd832f225c

2025-12-05 10:50:37,907 - distributed.scheduler - INFO - Close client connection: Client-25b5799d-d1c8-11f0-8001-f2cd832f225c

2025-12-05 10:50:37,905 - distributed.scheduler - INFO - Remove client Client-25b5799d-d1c8-11f0-8001-f2cd832f225c

2025-12-05 10:50:37,904 - distributed.scheduler - INFO - Remove client Client-25b5799d-d1c8-11f0-8001-f2cd832f225c

2025-12-05 10:50:00,509 - distributed.scheduler - INFO - Receive client connection: Client-25b5799d-d1c8-11f0-8001-f2cd832f225c

2025-12-05 10:41:37,950 - distributed.scheduler - INFO - Close client connection: Client-e3130701-d1c6-11f0-8001-f2cd832f225c

2025-12-05 10:41:37,949 - distributed.scheduler - INFO - Remove client Client-e3130701-d1c6-11f0-8001-f2cd832f225c

2025-12-05 10:41:37,947 - distributed.scheduler - INFO - Remove client Client-e3130701-d1c6-11f0-8001-f2cd832f225c

2025-12-05 10:40:59,260 - distributed.scheduler - INFO - Receive client connection: Client-e3130701-d1c6-11f0-8001-f2cd832f225c

2025-12-05 10:32:06,092 - distributed.scheduler - INFO - Close client connection: Client-7c9cee1e-d1c5-11f0-8001-f2cd832f225c

2025-12-05 10:32:06,090 - distributed.scheduler - INFO - Remove client Client-7c9cee1e-d1c5-11f0-8001-f2cd832f225c

2025-12-05 10:32:06,088 - distributed.scheduler - INFO - Remove client Client-7c9cee1e-d1c5-11f0-8001-f2cd832f225c

2025-12-05 10:31:02,580 - distributed.scheduler - INFO - Receive client connection: Client-worker-7f6b4f3d-d1c5-11f0-88fe-de470ccc1e79

2025-12-05 10:30:57,817 - distributed.scheduler - INFO - Receive client connection: Client-7c9cee1e-d1c5-11f0-8001-f2cd832f225c

2025-12-05 10:21:37,197 - distributed.scheduler - INFO - Close client connection: Client-16398e24-d1c4-11f0-8001-f2cd832f225c

2025-12-05 10:21:37,195 - distributed.scheduler - INFO - Remove client Client-16398e24-d1c4-11f0-8001-f2cd832f225c

2025-12-05 10:21:37,193 - distributed.scheduler - INFO - Remove client Client-16398e24-d1c4-11f0-8001-f2cd832f225c

2025-12-05 10:20:56,543 - distributed.scheduler - INFO - Receive client connection: Client-16398e24-d1c4-11f0-8001-f2cd832f225c

2025-12-05 10:11:38,007 - distributed.scheduler - INFO - Close client connection: Client-afb8fc40-d1c2-11f0-8001-f2cd832f225c

2025-12-05 10:11:38,007 - distributed.scheduler - INFO - Remove client Client-afb8fc40-d1c2-11f0-8001-f2cd832f225c

2025-12-05 10:11:38,005 - distributed.scheduler - INFO - Remove client Client-afb8fc40-d1c2-11f0-8001-f2cd832f225c

2025-12-05 10:10:55,075 - distributed.scheduler - INFO - Receive client connection: Client-afb8fc40-d1c2-11f0-8001-f2cd832f225c

2025-12-05 10:02:11,895 - distributed.scheduler - INFO - Close client connection: Client-4951295d-d1c1-11f0-8001-f2cd832f225c

2025-12-05 10:02:11,894 - distributed.scheduler - INFO - Remove client Client-4951295d-d1c1-11f0-8001-f2cd832f225c

2025-12-05 10:02:11,892 - distributed.scheduler - INFO - Remove client Client-4951295d-d1c1-11f0-8001-f2cd832f225c

2025-12-05 10:00:53,764 - distributed.scheduler - INFO - Receive client connection: Client-4951295d-d1c1-11f0-8001-f2cd832f225c

2025-12-05 09:51:33,218 - distributed.scheduler - INFO - Close client connection: Client-e2dcea08-d1bf-11f0-8001-f2cd832f225c

2025-12-05 09:51:33,218 - distributed.scheduler - INFO - Remove client Client-e2dcea08-d1bf-11f0-8001-f2cd832f225c

2025-12-05 09:51:33,216 - distributed.scheduler - INFO - Remove client Client-e2dcea08-d1bf-11f0-8001-f2cd832f225c

2025-12-05 09:50:52,384 - distributed.scheduler - INFO - Receive client connection: Client-e2dcea08-d1bf-11f0-8001-f2cd832f225c

2025-12-05 09:41:30,124 - distributed.scheduler - INFO - Close client connection: Client-7c5b8425-d1be-11f0-8001-f2cd832f225c

2025-12-05 09:41:30,122 - distributed.scheduler - INFO - Remove client Client-7c5b8425-d1be-11f0-8001-f2cd832f225c

2025-12-05 09:41:30,120 - distributed.scheduler - INFO - Remove client Client-7c5b8425-d1be-11f0-8001-f2cd832f225c

2025-12-05 09:40:50,915 - distributed.scheduler - INFO - Receive client connection: Client-7c5b8425-d1be-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,423 - distributed.scheduler - INFO - Close client connection: Client-7ab1daac-d1a2-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,423 - distributed.scheduler - INFO - Close client connection: Client-e136d871-d1a3-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,422 - distributed.scheduler - INFO - Close client connection: Client-47b3b13b-d1a5-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,422 - distributed.scheduler - INFO - Close client connection: Client-af144ad9-d1a6-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,422 - distributed.scheduler - INFO - Close client connection: Client-1490dd69-d1a8-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,422 - distributed.scheduler - INFO - Close client connection: Client-7b09bef7-d1a9-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,422 - distributed.scheduler - INFO - Close client connection: Client-e194bfa3-d1aa-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,422 - distributed.scheduler - INFO - Close client connection: Client-480a9433-d1ac-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,422 - distributed.scheduler - INFO - Close client connection: Client-ae7fcd2b-d1ad-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,421 - distributed.scheduler - INFO - Close client connection: Client-1501317e-d1af-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,421 - distributed.scheduler - INFO - Close client connection: Client-7b7d08cc-d1b0-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,421 - distributed.scheduler - INFO - Close client connection: Client-e1f3c3f0-d1b1-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,421 - distributed.scheduler - INFO - Close client connection: Client-487888f8-d1b3-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,421 - distributed.scheduler - INFO - Close client connection: Client-af000bb8-d1b4-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,421 - distributed.scheduler - INFO - Close client connection: Client-157f580d-d1b6-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,421 - distributed.scheduler - INFO - Close client connection: Client-7bea28e7-d1b7-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,420 - distributed.scheduler - INFO - Close client connection: Client-e26c6d77-d1b8-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,420 - distributed.scheduler - INFO - Close client connection: Client-48e9fe70-d1ba-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,420 - distributed.scheduler - INFO - Close client connection: Client-af55d7d3-d1bb-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,417 - distributed.scheduler - INFO - Remove client Client-7ab1daac-d1a2-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,417 - distributed.scheduler - INFO - Remove client Client-e136d871-d1a3-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,416 - distributed.scheduler - INFO - Remove client Client-47b3b13b-d1a5-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,416 - distributed.scheduler - INFO - Remove client Client-af144ad9-d1a6-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,415 - distributed.scheduler - INFO - Remove client Client-1490dd69-d1a8-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,415 - distributed.scheduler - INFO - Remove client Client-7b09bef7-d1a9-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,414 - distributed.scheduler - INFO - Remove client Client-e194bfa3-d1aa-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,414 - distributed.scheduler - INFO - Remove client Client-480a9433-d1ac-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,413 - distributed.scheduler - INFO - Remove client Client-ae7fcd2b-d1ad-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,406 - distributed.scheduler - INFO - Remove client Client-1501317e-d1af-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,405 - distributed.scheduler - INFO - Remove client Client-7b7d08cc-d1b0-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,404 - distributed.scheduler - INFO - Remove client Client-e1f3c3f0-d1b1-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,403 - distributed.scheduler - INFO - Remove client Client-487888f8-d1b3-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,402 - distributed.scheduler - INFO - Remove client Client-af000bb8-d1b4-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,401 - distributed.scheduler - INFO - Remove client Client-157f580d-d1b6-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,400 - distributed.scheduler - INFO - Remove client Client-7bea28e7-d1b7-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,399 - distributed.scheduler - INFO - Remove client Client-e26c6d77-d1b8-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,398 - distributed.scheduler - INFO - Remove client Client-48e9fe70-d1ba-11f0-8001-f2cd832f225c

2025-12-05 09:32:05,397 - distributed.scheduler - INFO - Remove client Client-af55d7d3-d1bb-11f0-8001-f2cd832f225c

2025-12-05 09:32:04,741 - distributed.scheduler - INFO - Close client connection: Client-15dd252b-d1bd-11f0-8001-f2cd832f225c

2025-12-05 09:32:04,740 - distributed.scheduler - INFO - Remove client Client-15dd252b-d1bd-11f0-8001-f2cd832f225c

2025-12-05 09:32:04,738 - distributed.scheduler - INFO - Remove client Client-15dd252b-d1bd-11f0-8001-f2cd832f225c

2025-12-05 09:30:54,736 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('len-agg-9750a393bb813fb508b1f4393a28e50f', 0) NoneType: None

2025-12-05 09:30:54,736 - distributed.scheduler - ERROR - Couldn't gather keys {"('len-agg-9750a393bb813fb508b1f4393a28e50f', 0)": []} state: ['waiting'] workers: [] NoneType: None

2025-12-05 09:30:49,456 - distributed.scheduler - INFO - Receive client connection: Client-15dd252b-d1bd-11f0-8001-f2cd832f225c

2025-12-05 09:20:47,942 - distributed.scheduler - INFO - Receive client connection: Client-af55d7d3-d1bb-11f0-8001-f2cd832f225c

2025-12-05 09:10:46,611 - distributed.scheduler - INFO - Receive client connection: Client-48e9fe70-d1ba-11f0-8001-f2cd832f225c

2025-12-05 09:00:45,165 - distributed.scheduler - INFO - Receive client connection: Client-e26c6d77-d1b8-11f0-8001-f2cd832f225c

2025-12-05 08:50:43,686 - distributed.scheduler - INFO - Receive client connection: Client-7bea28e7-d1b7-11f0-8001-f2cd832f225c

2025-12-05 08:40:42,362 - distributed.scheduler - INFO - Receive client connection: Client-157f580d-d1b6-11f0-8001-f2cd832f225c

2025-12-05 08:30:46,132 - distributed.scheduler - INFO - Receive client connection: Client-worker-b219af1a-d1b4-11f0-880a-46fbd1744967

2025-12-05 08:30:45,923 - distributed.scheduler - INFO - Receive client connection: Client-worker-b1f9625a-d1b4-11f0-8bc1-f2a719d69604

2025-12-05 08:30:45,480 - distributed.scheduler - INFO - Receive client connection: Client-worker-b1aecfb1-d1b4-11f0-8956-8a1d5a425dc1

2025-12-05 08:30:44,854 - distributed.scheduler - INFO - Receive client connection: Client-worker-b15c4e91-d1b4-11f0-891f-b64bf7bee50e

2025-12-05 08:30:40,900 - distributed.scheduler - INFO - Receive client connection: Client-af000bb8-d1b4-11f0-8001-f2cd832f225c

2025-12-05 08:20:39,391 - distributed.scheduler - INFO - Receive client connection: Client-487888f8-d1b3-11f0-8001-f2cd832f225c

2025-12-05 08:10:37,898 - distributed.scheduler - INFO - Receive client connection: Client-e1f3c3f0-d1b1-11f0-8001-f2cd832f225c

2025-12-05 08:03:45,259 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.24:35179

2025-12-05 08:03:45,257 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.24:35179', status: running, memory: 0, processing: 0>

2025-12-05 08:03:44,992 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:42709

2025-12-05 08:03:44,991 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:42709', status: running, memory: 0, processing: 0>

2025-12-05 08:03:44,452 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:39699

2025-12-05 08:03:44,451 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:39699', status: running, memory: 0, processing: 0>

2025-12-05 08:03:44,232 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.24:33745', status: closed, memory: 0, processing: 0>

2025-12-05 08:03:44,034 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:36639', status: closed, memory: 0, processing: 0>

2025-12-05 08:03:43,393 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:45169

2025-12-05 08:03:43,390 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:45169', status: running, memory: 0, processing: 0>

2025-12-05 08:03:43,277 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:44533', status: closed, memory: 0, processing: 0>

2025-12-05 08:03:42,159 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:38605', status: closed, memory: 0, processing: 0>

2025-12-05 08:03:18,800 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.24:33745

2025-12-05 08:03:18,798 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.24:33745', status: closed, memory: 0, processing: 0>

2025-12-05 08:03:18,732 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:36639

2025-12-05 08:03:18,731 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:36639', status: closed, memory: 0, processing: 0>

2025-12-05 08:03:17,764 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:35459', status: closed, memory: 0, processing: 0>

2025-12-05 08:03:17,740 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.24:38793', status: closed, memory: 0, processing: 0>

2025-12-05 08:03:17,294 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:38605

2025-12-05 08:03:17,293 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:38605', status: closed, memory: 0, processing: 0>

2025-12-05 08:03:16,598 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:44533

2025-12-05 08:03:16,595 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:44533', status: closed, memory: 0, processing: 0>

2025-12-05 08:03:16,250 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:46857', status: closed, memory: 0, processing: 0>

2025-12-05 08:03:15,593 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:36249', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:53,298 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:46857

2025-12-05 08:02:53,297 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:46857', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:52,162 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:45691', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:51,944 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:35459

2025-12-05 08:02:51,943 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:35459', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:51,374 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.24:38793

2025-12-05 08:02:51,373 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.24:38793', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:51,020 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:36915', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:50,339 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.24:39997', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:49,620 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:36249

2025-12-05 08:02:49,619 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:36249', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:48,592 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:46203', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:25,324 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.24:39997

2025-12-05 08:02:25,323 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.24:39997', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:24,761 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:36915

2025-12-05 08:02:24,760 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:36915', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:24,278 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:45691

2025-12-05 08:02:24,276 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:45691', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:24,245 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.24:41507', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:23,805 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:40059', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:23,493 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:46203

2025-12-05 08:02:23,491 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:46203', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:23,051 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:37011', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:22,412 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:32879', status: closed, memory: 0, processing: 0>

2025-12-05 08:02:00,225 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:32879

2025-12-05 08:02:00,224 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:32879', status: closed, memory: 0, processing: 0>

2025-12-05 08:01:59,170 - distributed.scheduler - INFO - Close client connection: Client-worker-7e15a2f4-d1b0-11f0-8b43-f2a719d69604

2025-12-05 08:01:59,166 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:42193', status: closed, memory: 0, processing: 0>

2025-12-05 08:01:59,143 - distributed.scheduler - INFO - Remove client Client-worker-7e15a2f4-d1b0-11f0-8b43-f2a719d69604

2025-12-05 08:01:58,498 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:37011

2025-12-05 08:01:58,497 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:37011', status: closed, memory: 0, processing: 0>

2025-12-05 08:01:58,472 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:40059

2025-12-05 08:01:58,471 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:40059', status: closed, memory: 0, processing: 0>

2025-12-05 08:01:57,958 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.24:41507

2025-12-05 08:01:57,956 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.24:41507', status: closed, memory: 0, processing: 0>

2025-12-05 08:01:57,584 - distributed.scheduler - INFO - Close client connection: Client-worker-1045c119-d1a4-11f0-8867-de470ccc1e79

2025-12-05 08:01:57,579 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:41945', status: closed, memory: 0, processing: 0>

2025-12-05 08:01:57,578 - distributed.scheduler - INFO - Remove client Client-worker-1045c119-d1a4-11f0-8867-de470ccc1e79

2025-12-05 08:01:57,431 - distributed.scheduler - INFO - Close client connection: Client-worker-c5916bbb-d000-11f0-87cf-b64bf7bee50e

2025-12-05 08:01:57,428 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:46391', status: closed, memory: 0, processing: 0>

2025-12-05 08:01:57,424 - distributed.scheduler - INFO - Remove client Client-worker-c5916bbb-d000-11f0-87cf-b64bf7bee50e

2025-12-05 08:01:56,851 - distributed.scheduler - INFO - Close client connection: Client-worker-c6b8479d-d015-11f0-86c1-46fbd1744967

2025-12-05 08:01:56,843 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.24:45521', status: closed, memory: 0, processing: 0>

2025-12-05 08:01:56,841 - distributed.scheduler - INFO - Remove client Client-worker-c6b8479d-d015-11f0-86c1-46fbd1744967

2025-12-05 08:00:40,846 - distributed.scheduler - INFO - Receive client connection: Client-worker-7e15a2f4-d1b0-11f0-8b43-f2a719d69604

2025-12-05 08:00:36,496 - distributed.scheduler - INFO - Receive client connection: Client-7b7d08cc-d1b0-11f0-8001-f2cd832f225c

2025-12-05 07:50:35,057 - distributed.scheduler - INFO - Receive client connection: Client-1501317e-d1af-11f0-8001-f2cd832f225c

2025-12-05 07:40:33,585 - distributed.scheduler - INFO - Receive client connection: Client-ae7fcd2b-d1ad-11f0-8001-f2cd832f225c

2025-12-05 07:30:36,156 - distributed.scheduler - INFO - Receive client connection: Client-worker-4a6656e6-d1ac-11f0-878d-ee8ad16902f8

2025-12-05 07:30:36,073 - distributed.scheduler - INFO - Receive client connection: Client-worker-4a59bd56-d1ac-11f0-89a2-32a59f79d364

2025-12-05 07:30:32,191 - distributed.scheduler - INFO - Receive client connection: Client-480a9433-d1ac-11f0-8001-f2cd832f225c

2025-12-05 07:20:30,785 - distributed.scheduler - INFO - Receive client connection: Client-e194bfa3-d1aa-11f0-8001-f2cd832f225c

2025-12-05 07:10:29,260 - distributed.scheduler - INFO - Receive client connection: Client-7b09bef7-d1a9-11f0-8001-f2cd832f225c

2025-12-05 07:03:42,619 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:34931

2025-12-05 07:03:42,617 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:34931', status: running, memory: 0, processing: 0>

2025-12-05 07:03:41,542 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:34037', status: closed, memory: 0, processing: 0>

2025-12-05 07:03:41,125 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:39833

2025-12-05 07:03:41,124 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:39833', status: running, memory: 0, processing: 0>

2025-12-05 07:03:40,181 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:41751', status: closed, memory: 0, processing: 0>

2025-12-05 07:03:39,469 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:42193

2025-12-05 07:03:39,467 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:42193', status: closed, memory: 0, processing: 0>

2025-12-05 07:03:38,789 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:33081

2025-12-05 07:03:38,787 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:33081', status: running, memory: 0, processing: 0>

2025-12-05 07:03:38,498 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:34107', status: closed, memory: 0, processing: 0>

2025-12-05 07:03:37,826 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:39893', status: closed, memory: 0, processing: 0>

2025-12-05 07:03:15,524 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:34037

2025-12-05 07:03:15,523 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:34037', status: closed, memory: 0, processing: 0>

2025-12-05 07:03:14,435 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:35453', status: closed, memory: 0, processing: 0>

2025-12-05 07:03:13,726 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:34107

2025-12-05 07:03:13,725 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:34107', status: closed, memory: 0, processing: 0>

2025-12-05 07:03:12,614 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:39893

2025-12-05 07:03:12,610 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:39893', status: closed, memory: 0, processing: 0>

2025-12-05 07:03:12,491 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:40719', status: closed, memory: 0, processing: 0>

2025-12-05 07:03:11,731 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:44285', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:47,965 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:35453

2025-12-05 07:02:47,963 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:35453', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:47,480 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:41751

2025-12-05 07:02:47,479 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:41751', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:46,941 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:41863', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:46,669 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:43867', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:46,149 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:40719

2025-12-05 07:02:46,148 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:40719', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:45,391 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:44285

2025-12-05 07:02:45,389 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:44285', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:44,988 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:35235', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:44,425 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:41041', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:21,927 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:43867

2025-12-05 07:02:21,926 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:43867', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:21,000 - distributed.scheduler - INFO - Close client connection: Client-worker-c4e82bcf-d000-11f0-8826-8a1d5a425dc1

2025-12-05 07:02:20,993 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:38247', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:20,992 - distributed.scheduler - INFO - Remove client Client-worker-c4e82bcf-d000-11f0-8826-8a1d5a425dc1

2025-12-05 07:02:19,115 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:41863

2025-12-05 07:02:19,114 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:41863', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:18,136 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:39383', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:17,327 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:35235

2025-12-05 07:02:17,326 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:35235', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:16,274 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:33903', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:15,962 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:41041

2025-12-05 07:02:15,960 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:41041', status: closed, memory: 0, processing: 0>

2025-12-05 07:02:15,127 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:36527', status: closed, memory: 0, processing: 0>

2025-12-05 07:01:39,716 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:39383

2025-12-05 07:01:39,714 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:39383', status: closed, memory: 0, processing: 0>

2025-12-05 07:01:38,714 - distributed.scheduler - INFO - Close client connection: Client-worker-31aed6ae-ced7-11f0-87e3-32a59f79d364

2025-12-05 07:01:38,711 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:42075', status: closed, memory: 0, processing: 0>

2025-12-05 07:01:38,702 - distributed.scheduler - INFO - Remove client Client-worker-31aed6ae-ced7-11f0-87e3-32a59f79d364

2025-12-05 07:01:38,229 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:33903

2025-12-05 07:01:38,228 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:33903', status: closed, memory: 0, processing: 0>

2025-12-05 07:01:37,194 - distributed.scheduler - INFO - Close client connection: Client-worker-79ec714b-d100-11f0-8a67-f2a719d69604

2025-12-05 07:01:37,189 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:37951', status: closed, memory: 0, processing: 0>

2025-12-05 07:01:37,186 - distributed.scheduler - INFO - Remove client Client-worker-79ec714b-d100-11f0-8a67-f2a719d69604

2025-12-05 07:01:35,663 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:36527

2025-12-05 07:01:35,661 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:36527', status: closed, memory: 0, processing: 0>

2025-12-05 07:01:34,773 - distributed.scheduler - INFO - Close client connection: Client-worker-6bfb3f81-cf8b-11f0-8660-ee8ad16902f8

2025-12-05 07:01:34,764 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:44981', status: closed, memory: 0, processing: 0>

2025-12-05 07:01:34,743 - distributed.scheduler - INFO - Remove client Client-worker-6bfb3f81-cf8b-11f0-8660-ee8ad16902f8

2025-12-05 07:00:27,843 - distributed.scheduler - INFO - Receive client connection: Client-1490dd69-d1a8-11f0-8001-f2cd832f225c

2025-12-05 06:50:28,079 - distributed.scheduler - INFO - Receive client connection: Client-af144ad9-d1a6-11f0-8001-f2cd832f225c

2025-12-05 06:40:25,144 - distributed.scheduler - INFO - Receive client connection: Client-47b3b13b-d1a5-11f0-8001-f2cd832f225c

2025-12-05 06:31:42,664 - distributed.scheduler - INFO - Receive client connection: Client-worker-1045c119-d1a4-11f0-8867-de470ccc1e79

2025-12-05 06:31:34,601 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:37695 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.191:43191 after 30 s')

2025-12-05 06:31:34,372 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:43191 failed: OSError: Timed out trying to connect to tcp://172.21.12.191:43191 after 30 s

2025-12-05 06:31:34,153 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:43191 failed: OSError: Timed out trying to connect to tcp://172.21.12.191:43191 after 30 s

2025-12-05 06:31:33,867 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.191:43191 after 30 s')

2025-12-05 06:31:33,701 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], run-f2e207ff45ab5b13e8f3163e798c057d NoneType: None

2025-12-05 06:31:33,701 - distributed.scheduler - ERROR - Couldn't gather keys {'run-f2e207ff45ab5b13e8f3163e798c057d': []} state: ['processing'] workers: [] NoneType: None

2025-12-05 06:31:32,734 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:38247 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.191:43191 after 30 s')

2025-12-05 06:31:01,207 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:41945

2025-12-05 06:31:01,206 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:41945', status: closed, memory: 0, processing: 0>

2025-12-05 06:31:00,192 - distributed.scheduler - INFO - Close client connection: Client-worker-f157b015-d121-11f0-8803-de470ccc1e79

2025-12-05 06:31:00,186 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:43191', status: closed, memory: 0, processing: 0>

2025-12-05 06:31:00,184 - distributed.scheduler - INFO - Remove client Client-worker-f157b015-d121-11f0-8803-de470ccc1e79

2025-12-05 06:30:23,698 - distributed.scheduler - INFO - Receive client connection: Client-e136d871-d1a3-11f0-8001-f2cd832f225c

2025-12-05 06:20:22,217 - distributed.scheduler - INFO - Receive client connection: Client-7ab1daac-d1a2-11f0-8001-f2cd832f225c

2025-12-05 06:10:53,149 - distributed.scheduler - INFO - Close client connection: Client-14325078-d1a1-11f0-8001-f2cd832f225c

2025-12-05 06:10:53,148 - distributed.scheduler - INFO - Remove client Client-14325078-d1a1-11f0-8001-f2cd832f225c

2025-12-05 06:10:53,147 - distributed.scheduler - INFO - Remove client Client-14325078-d1a1-11f0-8001-f2cd832f225c

2025-12-05 06:10:20,761 - distributed.scheduler - INFO - Receive client connection: Client-14325078-d1a1-11f0-8001-f2cd832f225c

2025-12-05 06:01:19,713 - distributed.scheduler - INFO - Close client connection: Client-adb94ac1-d19f-11f0-8001-f2cd832f225c

2025-12-05 06:01:19,711 - distributed.scheduler - INFO - Remove client Client-adb94ac1-d19f-11f0-8001-f2cd832f225c

2025-12-05 06:01:19,711 - distributed.scheduler - INFO - Remove client Client-adb94ac1-d19f-11f0-8001-f2cd832f225c

2025-12-05 06:00:19,330 - distributed.scheduler - INFO - Receive client connection: Client-adb94ac1-d19f-11f0-8001-f2cd832f225c

2025-12-05 05:50:49,988 - distributed.scheduler - INFO - Close client connection: Client-473ebac0-d19e-11f0-8001-f2cd832f225c

2025-12-05 05:50:49,987 - distributed.scheduler - INFO - Remove client Client-473ebac0-d19e-11f0-8001-f2cd832f225c

2025-12-05 05:50:49,986 - distributed.scheduler - INFO - Remove client Client-473ebac0-d19e-11f0-8001-f2cd832f225c

2025-12-05 05:50:17,914 - distributed.scheduler - INFO - Receive client connection: Client-473ebac0-d19e-11f0-8001-f2cd832f225c

2025-12-05 05:40:49,019 - distributed.scheduler - INFO - Close client connection: Client-e0c1a706-d19c-11f0-8001-f2cd832f225c

2025-12-05 05:40:49,018 - distributed.scheduler - INFO - Remove client Client-e0c1a706-d19c-11f0-8001-f2cd832f225c

2025-12-05 05:40:49,017 - distributed.scheduler - INFO - Remove client Client-e0c1a706-d19c-11f0-8001-f2cd832f225c

2025-12-05 05:40:16,467 - distributed.scheduler - INFO - Receive client connection: Client-e0c1a706-d19c-11f0-8001-f2cd832f225c

2025-12-05 05:31:21,017 - distributed.scheduler - INFO - Close client connection: Client-7a45755f-d19b-11f0-8001-f2cd832f225c

2025-12-05 05:31:21,017 - distributed.scheduler - INFO - Remove client Client-7a45755f-d19b-11f0-8001-f2cd832f225c

2025-12-05 05:31:21,016 - distributed.scheduler - INFO - Remove client Client-7a45755f-d19b-11f0-8001-f2cd832f225c

2025-12-05 05:30:15,027 - distributed.scheduler - INFO - Receive client connection: Client-7a45755f-d19b-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,367 - distributed.scheduler - INFO - Close client connection: Client-36bdf61c-d18c-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,367 - distributed.scheduler - INFO - Close client connection: Client-79625c55-d18d-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,366 - distributed.scheduler - INFO - Close client connection: Client-dfe564dd-d18e-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,366 - distributed.scheduler - INFO - Close client connection: Client-4661e3d2-d190-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,366 - distributed.scheduler - INFO - Close client connection: Client-acdea3c6-d191-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,366 - distributed.scheduler - INFO - Close client connection: Client-135a22a8-d193-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,366 - distributed.scheduler - INFO - Close client connection: Client-79d8bc7f-d194-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,365 - distributed.scheduler - INFO - Close client connection: Client-e051da37-d195-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,365 - distributed.scheduler - INFO - Close client connection: Client-46d49070-d197-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,365 - distributed.scheduler - INFO - Close client connection: Client-ad504f41-d198-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,362 - distributed.scheduler - INFO - Remove client Client-36bdf61c-d18c-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,362 - distributed.scheduler - INFO - Remove client Client-79625c55-d18d-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,361 - distributed.scheduler - INFO - Remove client Client-dfe564dd-d18e-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,360 - distributed.scheduler - INFO - Remove client Client-4661e3d2-d190-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,359 - distributed.scheduler - INFO - Remove client Client-acdea3c6-d191-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,359 - distributed.scheduler - INFO - Remove client Client-135a22a8-d193-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,358 - distributed.scheduler - INFO - Remove client Client-79d8bc7f-d194-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,357 - distributed.scheduler - INFO - Remove client Client-e051da37-d195-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,356 - distributed.scheduler - INFO - Remove client Client-46d49070-d197-11f0-8001-f2cd832f225c

2025-12-05 05:20:46,355 - distributed.scheduler - INFO - Remove client Client-ad504f41-d198-11f0-8001-f2cd832f225c

2025-12-05 05:20:45,557 - distributed.scheduler - INFO - Close client connection: Client-13c5f518-d19a-11f0-8001-f2cd832f225c

2025-12-05 05:20:45,555 - distributed.scheduler - INFO - Remove client Client-13c5f518-d19a-11f0-8001-f2cd832f225c

2025-12-05 05:20:45,554 - distributed.scheduler - INFO - Remove client Client-13c5f518-d19a-11f0-8001-f2cd832f225c

2025-12-05 05:20:13,564 - distributed.scheduler - INFO - Receive client connection: Client-13c5f518-d19a-11f0-8001-f2cd832f225c

2025-12-05 05:10:12,161 - distributed.scheduler - INFO - Receive client connection: Client-ad504f41-d198-11f0-8001-f2cd832f225c

2025-12-05 05:00:10,726 - distributed.scheduler - INFO - Receive client connection: Client-46d49070-d197-11f0-8001-f2cd832f225c

2025-12-05 04:50:09,244 - distributed.scheduler - INFO - Receive client connection: Client-e051da37-d195-11f0-8001-f2cd832f225c

2025-12-05 04:40:07,831 - distributed.scheduler - INFO - Receive client connection: Client-79d8bc7f-d194-11f0-8001-f2cd832f225c

2025-12-05 04:30:06,374 - distributed.scheduler - INFO - Receive client connection: Client-135a22a8-d193-11f0-8001-f2cd832f225c

2025-12-05 04:20:04,942 - distributed.scheduler - INFO - Receive client connection: Client-acdea3c6-d191-11f0-8001-f2cd832f225c

2025-12-05 04:10:03,502 - distributed.scheduler - INFO - Receive client connection: Client-4661e3d2-d190-11f0-8001-f2cd832f225c

2025-12-05 04:00:02,057 - distributed.scheduler - INFO - Receive client connection: Client-dfe564dd-d18e-11f0-8001-f2cd832f225c

2025-12-05 03:50:00,576 - distributed.scheduler - INFO - Receive client connection: Client-79625c55-d18d-11f0-8001-f2cd832f225c

2025-12-05 03:40:59,274 - distributed.scheduler - INFO - Receive client connection: Client-36bdf61c-d18c-11f0-8001-f2cd832f225c

2025-12-05 03:32:16,740 - distributed.scheduler - INFO - Close client connection: Client-d03c1622-d18a-11f0-8001-f2cd832f225c

2025-12-05 03:32:16,738 - distributed.scheduler - INFO - Remove client Client-d03c1622-d18a-11f0-8001-f2cd832f225c

2025-12-05 03:32:16,737 - distributed.scheduler - INFO - Remove client Client-d03c1622-d18a-11f0-8001-f2cd832f225c

2025-12-05 03:30:57,808 - distributed.scheduler - INFO - Receive client connection: Client-d03c1622-d18a-11f0-8001-f2cd832f225c

2025-12-05 03:21:35,664 - distributed.scheduler - INFO - Close client connection: Client-69cbd58a-d189-11f0-8001-f2cd832f225c

2025-12-05 03:21:35,663 - distributed.scheduler - INFO - Remove client Client-69cbd58a-d189-11f0-8001-f2cd832f225c

2025-12-05 03:21:35,662 - distributed.scheduler - INFO - Remove client Client-69cbd58a-d189-11f0-8001-f2cd832f225c

2025-12-05 03:20:56,446 - distributed.scheduler - INFO - Receive client connection: Client-69cbd58a-d189-11f0-8001-f2cd832f225c

2025-12-05 03:11:34,820 - distributed.scheduler - INFO - Close client connection: Client-035b0a6f-d188-11f0-8001-f2cd832f225c

2025-12-05 03:11:34,819 - distributed.scheduler - INFO - Remove client Client-035b0a6f-d188-11f0-8001-f2cd832f225c

2025-12-05 03:11:34,817 - distributed.scheduler - INFO - Remove client Client-035b0a6f-d188-11f0-8001-f2cd832f225c

2025-12-05 03:10:55,075 - distributed.scheduler - INFO - Receive client connection: Client-035b0a6f-d188-11f0-8001-f2cd832f225c

2025-12-05 03:02:04,361 - distributed.scheduler - INFO - Close client connection: Client-9ceb0ea2-d186-11f0-8001-f2cd832f225c

2025-12-05 03:02:04,360 - distributed.scheduler - INFO - Remove client Client-9ceb0ea2-d186-11f0-8001-f2cd832f225c

2025-12-05 03:02:04,358 - distributed.scheduler - INFO - Remove client Client-9ceb0ea2-d186-11f0-8001-f2cd832f225c

2025-12-05 03:00:53,714 - distributed.scheduler - INFO - Receive client connection: Client-9ceb0ea2-d186-11f0-8001-f2cd832f225c

2025-12-05 02:51:30,863 - distributed.scheduler - INFO - Close client connection: Client-3674d848-d185-11f0-8001-f2cd832f225c

2025-12-05 02:51:30,862 - distributed.scheduler - INFO - Remove client Client-3674d848-d185-11f0-8001-f2cd832f225c

2025-12-05 02:51:30,860 - distributed.scheduler - INFO - Remove client Client-3674d848-d185-11f0-8001-f2cd832f225c

2025-12-05 02:50:52,326 - distributed.scheduler - INFO - Receive client connection: Client-3674d848-d185-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,887 - distributed.scheduler - INFO - Close client connection: Client-23856cbd-d142-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,887 - distributed.scheduler - INFO - Close client connection: Client-8a0adbb9-d143-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,887 - distributed.scheduler - INFO - Close client connection: Client-f0817048-d144-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,886 - distributed.scheduler - INFO - Close client connection: Client-56f7eeab-d146-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,886 - distributed.scheduler - INFO - Close client connection: Client-bd70695c-d147-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,886 - distributed.scheduler - INFO - Close client connection: Client-23ed5e95-d149-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,886 - distributed.scheduler - INFO - Close client connection: Client-8a752446-d14a-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,886 - distributed.scheduler - INFO - Close client connection: Client-f1385cd6-d14b-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,886 - distributed.scheduler - INFO - Close client connection: Client-57645046-d14d-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,886 - distributed.scheduler - INFO - Close client connection: Client-bdd9d341-d14e-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,886 - distributed.scheduler - INFO - Close client connection: Client-244bb7bd-d150-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,886 - distributed.scheduler - INFO - Close client connection: Client-8ad1461f-d151-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,886 - distributed.scheduler - INFO - Close client connection: Client-cd62428d-d152-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,886 - distributed.scheduler - INFO - Close client connection: Client-33e8b864-d154-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,886 - distributed.scheduler - INFO - Close client connection: Client-9a4bf305-d155-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,886 - distributed.scheduler - INFO - Close client connection: Client-00d3981c-d157-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,885 - distributed.scheduler - INFO - Close client connection: Client-6758c77e-d158-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,885 - distributed.scheduler - INFO - Close client connection: Client-cdd8bab4-d159-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,885 - distributed.scheduler - INFO - Close client connection: Client-344f34d2-d15b-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,885 - distributed.scheduler - INFO - Close client connection: Client-9ac1faf6-d15c-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,885 - distributed.scheduler - INFO - Close client connection: Client-01377021-d15e-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,885 - distributed.scheduler - INFO - Close client connection: Client-67b03f17-d15f-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,885 - distributed.scheduler - INFO - Close client connection: Client-ce21300d-d160-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,885 - distributed.scheduler - INFO - Close client connection: Client-34935732-d162-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,885 - distributed.scheduler - INFO - Close client connection: Client-9b132d7e-d163-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,885 - distributed.scheduler - INFO - Close client connection: Client-018fe9fd-d165-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,885 - distributed.scheduler - INFO - Close client connection: Client-68086463-d166-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,885 - distributed.scheduler - INFO - Close client connection: Client-ce81baae-d167-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,885 - distributed.scheduler - INFO - Close client connection: Client-34f8d3a1-d169-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,884 - distributed.scheduler - INFO - Close client connection: Client-9b78af86-d16a-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,884 - distributed.scheduler - INFO - Close client connection: Client-01e5a253-d16c-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,884 - distributed.scheduler - INFO - Close client connection: Client-6854b24f-d16d-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,884 - distributed.scheduler - INFO - Close client connection: Client-cec700db-d16e-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,884 - distributed.scheduler - INFO - Close client connection: Client-353d9d22-d170-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,884 - distributed.scheduler - INFO - Close client connection: Client-9bbe4653-d171-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,884 - distributed.scheduler - INFO - Close client connection: Client-023cc6f1-d173-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,883 - distributed.scheduler - INFO - Close client connection: Client-68b9484d-d174-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,883 - distributed.scheduler - INFO - Close client connection: Client-cf37274a-d175-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,883 - distributed.scheduler - INFO - Close client connection: Client-35b50d90-d177-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,883 - distributed.scheduler - INFO - Close client connection: Client-02ab2145-d17a-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,883 - distributed.scheduler - INFO - Close client connection: Client-9c330233-d178-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,883 - distributed.scheduler - INFO - Close client connection: Client-6926f1a4-d17b-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,883 - distributed.scheduler - INFO - Close client connection: Client-cf996719-d17c-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,882 - distributed.scheduler - INFO - Close client connection: Client-360de2a3-d17e-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,882 - distributed.scheduler - INFO - Close client connection: Client-0302987a-d181-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,882 - distributed.scheduler - INFO - Close client connection: Client-9c8f629e-d17f-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,880 - distributed.scheduler - INFO - Close client connection: Client-697aa8c8-d182-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,880 - distributed.scheduler - INFO - Remove client Client-23856cbd-d142-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,879 - distributed.scheduler - INFO - Remove client Client-8a0adbb9-d143-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,879 - distributed.scheduler - INFO - Remove client Client-f0817048-d144-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,879 - distributed.scheduler - INFO - Remove client Client-56f7eeab-d146-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,879 - distributed.scheduler - INFO - Remove client Client-bd70695c-d147-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,878 - distributed.scheduler - INFO - Remove client Client-23ed5e95-d149-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,878 - distributed.scheduler - INFO - Remove client Client-8a752446-d14a-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,878 - distributed.scheduler - INFO - Remove client Client-f1385cd6-d14b-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,877 - distributed.scheduler - INFO - Remove client Client-57645046-d14d-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,877 - distributed.scheduler - INFO - Remove client Client-bdd9d341-d14e-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,877 - distributed.scheduler - INFO - Remove client Client-244bb7bd-d150-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,876 - distributed.scheduler - INFO - Remove client Client-8ad1461f-d151-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,876 - distributed.scheduler - INFO - Remove client Client-cd62428d-d152-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,875 - distributed.scheduler - INFO - Remove client Client-33e8b864-d154-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,875 - distributed.scheduler - INFO - Remove client Client-9a4bf305-d155-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,875 - distributed.scheduler - INFO - Remove client Client-00d3981c-d157-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,874 - distributed.scheduler - INFO - Remove client Client-6758c77e-d158-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,874 - distributed.scheduler - INFO - Remove client Client-cdd8bab4-d159-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,874 - distributed.scheduler - INFO - Remove client Client-344f34d2-d15b-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,874 - distributed.scheduler - INFO - Remove client Client-9ac1faf6-d15c-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,873 - distributed.scheduler - INFO - Remove client Client-01377021-d15e-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,873 - distributed.scheduler - INFO - Remove client Client-67b03f17-d15f-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,873 - distributed.scheduler - INFO - Remove client Client-ce21300d-d160-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,872 - distributed.scheduler - INFO - Remove client Client-34935732-d162-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,872 - distributed.scheduler - INFO - Remove client Client-9b132d7e-d163-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,872 - distributed.scheduler - INFO - Remove client Client-018fe9fd-d165-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,871 - distributed.scheduler - INFO - Remove client Client-68086463-d166-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,871 - distributed.scheduler - INFO - Remove client Client-ce81baae-d167-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,871 - distributed.scheduler - INFO - Remove client Client-34f8d3a1-d169-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,870 - distributed.scheduler - INFO - Remove client Client-9b78af86-d16a-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,870 - distributed.scheduler - INFO - Remove client Client-01e5a253-d16c-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,869 - distributed.scheduler - INFO - Remove client Client-6854b24f-d16d-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,869 - distributed.scheduler - INFO - Remove client Client-cec700db-d16e-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,869 - distributed.scheduler - INFO - Remove client Client-353d9d22-d170-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,868 - distributed.scheduler - INFO - Remove client Client-9bbe4653-d171-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,868 - distributed.scheduler - INFO - Remove client Client-023cc6f1-d173-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,857 - distributed.scheduler - INFO - Remove client Client-68b9484d-d174-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,857 - distributed.scheduler - INFO - Remove client Client-cf37274a-d175-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,856 - distributed.scheduler - INFO - Remove client Client-35b50d90-d177-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,855 - distributed.scheduler - INFO - Remove client Client-02ab2145-d17a-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,855 - distributed.scheduler - INFO - Remove client Client-9c330233-d178-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,854 - distributed.scheduler - INFO - Remove client Client-6926f1a4-d17b-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,853 - distributed.scheduler - INFO - Remove client Client-cf996719-d17c-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,853 - distributed.scheduler - INFO - Remove client Client-360de2a3-d17e-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,852 - distributed.scheduler - INFO - Remove client Client-0302987a-d181-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,851 - distributed.scheduler - INFO - Remove client Client-9c8f629e-d17f-11f0-8001-f2cd832f225c

2025-12-05 02:41:30,846 - distributed.scheduler - INFO - Remove client Client-697aa8c8-d182-11f0-8001-f2cd832f225c

2025-12-05 02:41:29,912 - distributed.scheduler - INFO - Close client connection: Client-cfeec71b-d183-11f0-8001-f2cd832f225c

2025-12-05 02:41:29,911 - distributed.scheduler - INFO - Remove client Client-cfeec71b-d183-11f0-8001-f2cd832f225c

2025-12-05 02:41:29,910 - distributed.scheduler - INFO - Remove client Client-cfeec71b-d183-11f0-8001-f2cd832f225c

2025-12-05 02:40:50,812 - distributed.scheduler - INFO - Receive client connection: Client-cfeec71b-d183-11f0-8001-f2cd832f225c

2025-12-05 02:30:49,437 - distributed.scheduler - INFO - Receive client connection: Client-697aa8c8-d182-11f0-8001-f2cd832f225c

2025-12-05 02:20:48,013 - distributed.scheduler - INFO - Receive client connection: Client-0302987a-d181-11f0-8001-f2cd832f225c

2025-12-05 02:10:46,640 - distributed.scheduler - INFO - Receive client connection: Client-9c8f629e-d17f-11f0-8001-f2cd832f225c

2025-12-05 02:00:45,167 - distributed.scheduler - INFO - Receive client connection: Client-360de2a3-d17e-11f0-8001-f2cd832f225c

2025-12-05 01:50:43,783 - distributed.scheduler - INFO - Receive client connection: Client-cf996719-d17c-11f0-8001-f2cd832f225c

2025-12-05 01:40:42,400 - distributed.scheduler - INFO - Receive client connection: Client-6926f1a4-d17b-11f0-8001-f2cd832f225c

2025-12-05 01:30:40,963 - distributed.scheduler - INFO - Receive client connection: Client-02ab2145-d17a-11f0-8001-f2cd832f225c

2025-12-05 01:20:39,562 - distributed.scheduler - INFO - Receive client connection: Client-9c330233-d178-11f0-8001-f2cd832f225c

2025-12-05 01:10:38,102 - distributed.scheduler - INFO - Receive client connection: Client-35b50d90-d177-11f0-8001-f2cd832f225c

2025-12-05 01:00:36,661 - distributed.scheduler - INFO - Receive client connection: Client-cf37274a-d175-11f0-8001-f2cd832f225c

2025-12-05 00:50:35,210 - distributed.scheduler - INFO - Receive client connection: Client-68b9484d-d174-11f0-8001-f2cd832f225c

2025-12-05 00:40:33,761 - distributed.scheduler - INFO - Receive client connection: Client-023cc6f1-d173-11f0-8001-f2cd832f225c

2025-12-05 00:30:32,318 - distributed.scheduler - INFO - Receive client connection: Client-9bbe4653-d171-11f0-8001-f2cd832f225c

2025-12-05 00:20:30,850 - distributed.scheduler - INFO - Receive client connection: Client-353d9d22-d170-11f0-8001-f2cd832f225c

2025-12-05 00:10:29,445 - distributed.scheduler - INFO - Receive client connection: Client-cec700db-d16e-11f0-8001-f2cd832f225c

2025-12-05 00:00:28,068 - distributed.scheduler - INFO - Receive client connection: Client-6854b24f-d16d-11f0-8001-f2cd832f225c

2025-12-04 23:50:26,720 - distributed.scheduler - INFO - Receive client connection: Client-01e5a253-d16c-11f0-8001-f2cd832f225c

2025-12-04 23:40:25,381 - distributed.scheduler - INFO - Receive client connection: Client-9b78af86-d16a-11f0-8001-f2cd832f225c

2025-12-04 23:30:23,915 - distributed.scheduler - INFO - Receive client connection: Client-34f8d3a1-d169-11f0-8001-f2cd832f225c

2025-12-04 23:20:22,515 - distributed.scheduler - INFO - Receive client connection: Client-ce81baae-d167-11f0-8001-f2cd832f225c

2025-12-04 23:10:21,092 - distributed.scheduler - INFO - Receive client connection: Client-68086463-d166-11f0-8001-f2cd832f225c

2025-12-04 23:00:19,683 - distributed.scheduler - INFO - Receive client connection: Client-018fe9fd-d165-11f0-8001-f2cd832f225c

2025-12-04 22:50:18,234 - distributed.scheduler - INFO - Receive client connection: Client-9b132d7e-d163-11f0-8001-f2cd832f225c

2025-12-04 22:40:16,771 - distributed.scheduler - INFO - Receive client connection: Client-34935732-d162-11f0-8001-f2cd832f225c

2025-12-04 22:30:15,401 - distributed.scheduler - INFO - Receive client connection: Client-ce21300d-d160-11f0-8001-f2cd832f225c

2025-12-04 22:20:14,042 - distributed.scheduler - INFO - Receive client connection: Client-67b03f17-d15f-11f0-8001-f2cd832f225c

2025-12-04 22:10:12,622 - distributed.scheduler - INFO - Receive client connection: Client-01377021-d15e-11f0-8001-f2cd832f225c

2025-12-04 22:00:11,222 - distributed.scheduler - INFO - Receive client connection: Client-9ac1faf6-d15c-11f0-8001-f2cd832f225c

2025-12-04 21:50:09,854 - distributed.scheduler - INFO - Receive client connection: Client-344f34d2-d15b-11f0-8001-f2cd832f225c

2025-12-04 21:40:08,453 - distributed.scheduler - INFO - Receive client connection: Client-cdd8bab4-d159-11f0-8001-f2cd832f225c

2025-12-04 21:30:06,991 - distributed.scheduler - INFO - Receive client connection: Client-6758c77e-d158-11f0-8001-f2cd832f225c

2025-12-04 21:20:05,487 - distributed.scheduler - INFO - Receive client connection: Client-00d3981c-d157-11f0-8001-f2cd832f225c

2025-12-04 21:10:03,977 - distributed.scheduler - INFO - Receive client connection: Client-9a4bf305-d155-11f0-8001-f2cd832f225c

2025-12-04 21:00:02,703 - distributed.scheduler - INFO - Receive client connection: Client-33e8b864-d154-11f0-8001-f2cd832f225c

2025-12-04 20:50:01,196 - distributed.scheduler - INFO - Receive client connection: Client-cd62428d-d152-11f0-8001-f2cd832f225c

2025-12-04 20:41:00,021 - distributed.scheduler - INFO - Receive client connection: Client-8ad1461f-d151-11f0-8001-f2cd832f225c

2025-12-04 20:30:58,522 - distributed.scheduler - INFO - Receive client connection: Client-244bb7bd-d150-11f0-8001-f2cd832f225c

2025-12-04 20:20:57,154 - distributed.scheduler - INFO - Receive client connection: Client-bdd9d341-d14e-11f0-8001-f2cd832f225c

2025-12-04 20:10:55,760 - distributed.scheduler - INFO - Receive client connection: Client-57645046-d14d-11f0-8001-f2cd832f225c

2025-12-04 20:00:54,843 - distributed.scheduler - INFO - Receive client connection: Client-f1385cd6-d14b-11f0-8001-f2cd832f225c

2025-12-04 19:50:52,946 - distributed.scheduler - INFO - Receive client connection: Client-8a752446-d14a-11f0-8001-f2cd832f225c

2025-12-04 19:40:51,427 - distributed.scheduler - INFO - Receive client connection: Client-23ed5e95-d149-11f0-8001-f2cd832f225c

2025-12-04 19:30:49,985 - distributed.scheduler - INFO - Receive client connection: Client-bd70695c-d147-11f0-8001-f2cd832f225c

2025-12-04 19:20:48,575 - distributed.scheduler - INFO - Receive client connection: Client-56f7eeab-d146-11f0-8001-f2cd832f225c

2025-12-04 19:10:47,167 - distributed.scheduler - INFO - Receive client connection: Client-f0817048-d144-11f0-8001-f2cd832f225c

2025-12-04 19:00:45,765 - distributed.scheduler - INFO - Receive client connection: Client-8a0adbb9-d143-11f0-8001-f2cd832f225c

2025-12-04 18:50:44,277 - distributed.scheduler - INFO - Receive client connection: Client-23856cbd-d142-11f0-8001-f2cd832f225c

2025-12-04 18:41:20,236 - distributed.scheduler - INFO - Close client connection: Client-bd120af1-d140-11f0-8001-f2cd832f225c

2025-12-04 18:41:20,234 - distributed.scheduler - INFO - Remove client Client-bd120af1-d140-11f0-8001-f2cd832f225c

2025-12-04 18:41:20,232 - distributed.scheduler - INFO - Remove client Client-bd120af1-d140-11f0-8001-f2cd832f225c

2025-12-04 18:40:42,891 - distributed.scheduler - INFO - Receive client connection: Client-bd120af1-d140-11f0-8001-f2cd832f225c

2025-12-04 18:31:51,922 - distributed.scheduler - INFO - Close client connection: Client-5697c340-d13f-11f0-8001-f2cd832f225c

2025-12-04 18:31:51,921 - distributed.scheduler - INFO - Remove client Client-5697c340-d13f-11f0-8001-f2cd832f225c

2025-12-04 18:31:51,919 - distributed.scheduler - INFO - Remove client Client-5697c340-d13f-11f0-8001-f2cd832f225c

2025-12-04 18:30:41,466 - distributed.scheduler - INFO - Receive client connection: Client-5697c340-d13f-11f0-8001-f2cd832f225c

2025-12-04 18:21:21,544 - distributed.scheduler - INFO - Close client connection: Client-f01292c4-d13d-11f0-8001-f2cd832f225c

2025-12-04 18:21:21,543 - distributed.scheduler - INFO - Remove client Client-f01292c4-d13d-11f0-8001-f2cd832f225c

2025-12-04 18:21:21,541 - distributed.scheduler - INFO - Remove client Client-f01292c4-d13d-11f0-8001-f2cd832f225c

2025-12-04 18:20:39,965 - distributed.scheduler - INFO - Receive client connection: Client-f01292c4-d13d-11f0-8001-f2cd832f225c

2025-12-04 18:11:17,871 - distributed.scheduler - INFO - Close client connection: Client-89899325-d13c-11f0-8001-f2cd832f225c

2025-12-04 18:11:17,870 - distributed.scheduler - INFO - Remove client Client-89899325-d13c-11f0-8001-f2cd832f225c

2025-12-04 18:11:17,868 - distributed.scheduler - INFO - Remove client Client-89899325-d13c-11f0-8001-f2cd832f225c

2025-12-04 18:10:38,451 - distributed.scheduler - INFO - Receive client connection: Client-89899325-d13c-11f0-8001-f2cd832f225c

2025-12-04 18:01:54,217 - distributed.scheduler - INFO - Close client connection: Client-22ffbb60-d13b-11f0-8001-f2cd832f225c

2025-12-04 18:01:54,216 - distributed.scheduler - INFO - Remove client Client-22ffbb60-d13b-11f0-8001-f2cd832f225c

2025-12-04 18:01:54,215 - distributed.scheduler - INFO - Remove client Client-22ffbb60-d13b-11f0-8001-f2cd832f225c

2025-12-04 18:00:36,927 - distributed.scheduler - INFO - Receive client connection: Client-22ffbb60-d13b-11f0-8001-f2cd832f225c

2025-12-04 17:51:16,311 - distributed.scheduler - INFO - Close client connection: Client-5610f0e1-d138-11f0-8001-f2cd832f225c

2025-12-04 17:51:16,311 - distributed.scheduler - INFO - Remove client Client-5610f0e1-d138-11f0-8001-f2cd832f225c

2025-12-04 17:51:15,605 - distributed.scheduler - INFO - Close client connection: Client-bc839b26-d139-11f0-8001-f2cd832f225c

2025-12-04 17:51:15,604 - distributed.scheduler - INFO - Remove client Client-bc839b26-d139-11f0-8001-f2cd832f225c

2025-12-04 17:51:15,603 - distributed.scheduler - INFO - Remove client Client-bc839b26-d139-11f0-8001-f2cd832f225c

2025-12-04 17:50:35,476 - distributed.scheduler - INFO - Receive client connection: Client-bc839b26-d139-11f0-8001-f2cd832f225c

2025-12-04 17:40:34,104 - distributed.scheduler - INFO - Receive client connection: Client-5610f0e1-d138-11f0-8001-f2cd832f225c

2025-12-04 17:31:52,007 - distributed.scheduler - INFO - Close client connection: Client-ef9909d0-d136-11f0-8001-f2cd832f225c

2025-12-04 17:31:52,006 - distributed.scheduler - INFO - Remove client Client-ef9909d0-d136-11f0-8001-f2cd832f225c

2025-12-04 17:31:52,004 - distributed.scheduler - INFO - Remove client Client-ef9909d0-d136-11f0-8001-f2cd832f225c

2025-12-04 17:30:32,703 - distributed.scheduler - INFO - Receive client connection: Client-ef9909d0-d136-11f0-8001-f2cd832f225c

2025-12-04 17:21:13,180 - distributed.scheduler - INFO - Close client connection: Client-891cc1eb-d135-11f0-8001-f2cd832f225c

2025-12-04 17:21:13,179 - distributed.scheduler - INFO - Remove client Client-891cc1eb-d135-11f0-8001-f2cd832f225c

2025-12-04 17:21:13,177 - distributed.scheduler - INFO - Remove client Client-891cc1eb-d135-11f0-8001-f2cd832f225c

2025-12-04 17:20:31,262 - distributed.scheduler - INFO - Receive client connection: Client-891cc1eb-d135-11f0-8001-f2cd832f225c

2025-12-04 17:11:12,692 - distributed.scheduler - INFO - Close client connection: Client-22a423ac-d134-11f0-8001-f2cd832f225c

2025-12-04 17:11:12,689 - distributed.scheduler - INFO - Remove client Client-22a423ac-d134-11f0-8001-f2cd832f225c

2025-12-04 17:11:12,687 - distributed.scheduler - INFO - Remove client Client-22a423ac-d134-11f0-8001-f2cd832f225c

2025-12-04 17:10:29,847 - distributed.scheduler - INFO - Receive client connection: Client-22a423ac-d134-11f0-8001-f2cd832f225c

2025-12-04 17:01:42,242 - distributed.scheduler - INFO - Close client connection: Client-bc2318ac-d132-11f0-8001-f2cd832f225c

2025-12-04 17:01:42,241 - distributed.scheduler - INFO - Remove client Client-bc2318ac-d132-11f0-8001-f2cd832f225c

2025-12-04 17:01:42,239 - distributed.scheduler - INFO - Remove client Client-bc2318ac-d132-11f0-8001-f2cd832f225c

2025-12-04 17:00:32,500 - distributed.scheduler - INFO - Receive client connection: Client-worker-be9aebb6-d132-11f0-8b58-8253c573df33

2025-12-04 17:00:28,370 - distributed.scheduler - INFO - Receive client connection: Client-bc2318ac-d132-11f0-8001-f2cd832f225c

2025-12-04 16:51:10,763 - distributed.scheduler - INFO - Close client connection: Client-55ac3251-d131-11f0-8001-f2cd832f225c

2025-12-04 16:51:10,762 - distributed.scheduler - INFO - Remove client Client-55ac3251-d131-11f0-8001-f2cd832f225c

2025-12-04 16:51:10,760 - distributed.scheduler - INFO - Remove client Client-55ac3251-d131-11f0-8001-f2cd832f225c

2025-12-04 16:50:26,969 - distributed.scheduler - INFO - Receive client connection: Client-55ac3251-d131-11f0-8001-f2cd832f225c

2025-12-04 16:41:08,937 - distributed.scheduler - INFO - Close client connection: Client-ef385998-d12f-11f0-8001-f2cd832f225c

2025-12-04 16:41:08,936 - distributed.scheduler - INFO - Remove client Client-ef385998-d12f-11f0-8001-f2cd832f225c

2025-12-04 16:41:08,935 - distributed.scheduler - INFO - Remove client Client-ef385998-d12f-11f0-8001-f2cd832f225c

2025-12-04 16:40:25,589 - distributed.scheduler - INFO - Receive client connection: Client-ef385998-d12f-11f0-8001-f2cd832f225c

2025-12-04 16:31:36,276 - distributed.scheduler - INFO - Close client connection: Client-88c10c69-d12e-11f0-8001-f2cd832f225c

2025-12-04 16:31:36,275 - distributed.scheduler - INFO - Remove client Client-88c10c69-d12e-11f0-8001-f2cd832f225c

2025-12-04 16:31:36,273 - distributed.scheduler - INFO - Remove client Client-88c10c69-d12e-11f0-8001-f2cd832f225c

2025-12-04 16:30:24,186 - distributed.scheduler - INFO - Receive client connection: Client-88c10c69-d12e-11f0-8001-f2cd832f225c

2025-12-04 16:21:01,505 - distributed.scheduler - INFO - Close client connection: Client-223f519b-d12d-11f0-8001-f2cd832f225c

2025-12-04 16:21:01,503 - distributed.scheduler - INFO - Remove client Client-223f519b-d12d-11f0-8001-f2cd832f225c

2025-12-04 16:21:01,502 - distributed.scheduler - INFO - Remove client Client-223f519b-d12d-11f0-8001-f2cd832f225c

2025-12-04 16:20:22,708 - distributed.scheduler - INFO - Receive client connection: Client-223f519b-d12d-11f0-8001-f2cd832f225c

2025-12-04 16:11:05,643 - distributed.scheduler - INFO - Close client connection: Client-bbc2cd31-d12b-11f0-8001-f2cd832f225c

2025-12-04 16:11:05,643 - distributed.scheduler - INFO - Remove client Client-bbc2cd31-d12b-11f0-8001-f2cd832f225c

2025-12-04 16:11:05,643 - distributed.scheduler - INFO - Remove client Client-bbc2cd31-d12b-11f0-8001-f2cd832f225c

2025-12-04 16:10:21,271 - distributed.scheduler - INFO - Receive client connection: Client-bbc2cd31-d12b-11f0-8001-f2cd832f225c

2025-12-04 16:01:40,091 - distributed.scheduler - INFO - Close client connection: Client-55499cd2-d12a-11f0-8001-f2cd832f225c

2025-12-04 16:01:40,090 - distributed.scheduler - INFO - Remove client Client-55499cd2-d12a-11f0-8001-f2cd832f225c

2025-12-04 16:01:40,089 - distributed.scheduler - INFO - Remove client Client-55499cd2-d12a-11f0-8001-f2cd832f225c

2025-12-04 16:00:19,878 - distributed.scheduler - INFO - Receive client connection: Client-55499cd2-d12a-11f0-8001-f2cd832f225c

2025-12-04 15:51:03,852 - distributed.scheduler - INFO - Close client connection: Client-eed0eb1d-d128-11f0-8001-f2cd832f225c

2025-12-04 15:51:03,852 - distributed.scheduler - INFO - Remove client Client-eed0eb1d-d128-11f0-8001-f2cd832f225c

2025-12-04 15:51:03,851 - distributed.scheduler - INFO - Remove client Client-eed0eb1d-d128-11f0-8001-f2cd832f225c

2025-12-04 15:50:18,433 - distributed.scheduler - INFO - Receive client connection: Client-eed0eb1d-d128-11f0-8001-f2cd832f225c

2025-12-04 15:41:05,410 - distributed.scheduler - INFO - Close client connection: Client-885359cb-d127-11f0-8001-f2cd832f225c

2025-12-04 15:41:05,408 - distributed.scheduler - INFO - Remove client Client-885359cb-d127-11f0-8001-f2cd832f225c

2025-12-04 15:41:05,405 - distributed.scheduler - INFO - Remove client Client-885359cb-d127-11f0-8001-f2cd832f225c

2025-12-04 15:40:16,985 - distributed.scheduler - INFO - Receive client connection: Client-885359cb-d127-11f0-8001-f2cd832f225c

2025-12-04 15:31:33,738 - distributed.scheduler - INFO - Close client connection: Client-21de9921-d126-11f0-8001-f2cd832f225c

2025-12-04 15:31:33,737 - distributed.scheduler - INFO - Remove client Client-21de9921-d126-11f0-8001-f2cd832f225c

2025-12-04 15:31:33,735 - distributed.scheduler - INFO - Remove client Client-21de9921-d126-11f0-8001-f2cd832f225c

2025-12-04 15:30:15,594 - distributed.scheduler - INFO - Receive client connection: Client-21de9921-d126-11f0-8001-f2cd832f225c

2025-12-04 15:21:00,134 - distributed.scheduler - INFO - Close client connection: Client-bb5b99c6-d124-11f0-8001-f2cd832f225c

2025-12-04 15:21:00,133 - distributed.scheduler - INFO - Remove client Client-bb5b99c6-d124-11f0-8001-f2cd832f225c

2025-12-04 15:21:00,131 - distributed.scheduler - INFO - Remove client Client-bb5b99c6-d124-11f0-8001-f2cd832f225c

2025-12-04 15:20:14,110 - distributed.scheduler - INFO - Receive client connection: Client-bb5b99c6-d124-11f0-8001-f2cd832f225c

2025-12-04 15:10:53,966 - distributed.scheduler - INFO - Close client connection: Client-54de123c-d123-11f0-8001-f2cd832f225c

2025-12-04 15:10:53,965 - distributed.scheduler - INFO - Remove client Client-54de123c-d123-11f0-8001-f2cd832f225c

2025-12-04 15:10:53,963 - distributed.scheduler - INFO - Remove client Client-54de123c-d123-11f0-8001-f2cd832f225c

2025-12-04 15:10:12,662 - distributed.scheduler - INFO - Receive client connection: Client-54de123c-d123-11f0-8001-f2cd832f225c

2025-12-04 15:01:28,891 - distributed.scheduler - INFO - Close client connection: Client-ee64d452-d121-11f0-8001-f2cd832f225c

2025-12-04 15:01:28,889 - distributed.scheduler - INFO - Remove client Client-ee64d452-d121-11f0-8001-f2cd832f225c

2025-12-04 15:01:28,888 - distributed.scheduler - INFO - Remove client Client-ee64d452-d121-11f0-8001-f2cd832f225c

2025-12-04 15:00:16,237 - distributed.scheduler - INFO - Receive client connection: Client-worker-f157b015-d121-11f0-8803-de470ccc1e79

2025-12-04 15:00:11,244 - distributed.scheduler - INFO - Receive client connection: Client-ee64d452-d121-11f0-8001-f2cd832f225c

2025-12-04 14:50:54,202 - distributed.scheduler - INFO - Close client connection: Client-87eee761-d120-11f0-8001-f2cd832f225c

2025-12-04 14:50:54,201 - distributed.scheduler - INFO - Remove client Client-87eee761-d120-11f0-8001-f2cd832f225c

2025-12-04 14:50:54,199 - distributed.scheduler - INFO - Remove client Client-87eee761-d120-11f0-8001-f2cd832f225c

2025-12-04 14:50:09,848 - distributed.scheduler - INFO - Receive client connection: Client-87eee761-d120-11f0-8001-f2cd832f225c

2025-12-04 14:40:51,412 - distributed.scheduler - INFO - Close client connection: Client-216aa74a-d11f-11f0-8001-f2cd832f225c

2025-12-04 14:40:51,411 - distributed.scheduler - INFO - Remove client Client-216aa74a-d11f-11f0-8001-f2cd832f225c

2025-12-04 14:40:51,409 - distributed.scheduler - INFO - Remove client Client-216aa74a-d11f-11f0-8001-f2cd832f225c

2025-12-04 14:40:08,356 - distributed.scheduler - INFO - Receive client connection: Client-216aa74a-d11f-11f0-8001-f2cd832f225c

2025-12-04 14:31:24,366 - distributed.scheduler - INFO - Close client connection: Client-baf860bc-d11d-11f0-8001-f2cd832f225c

2025-12-04 14:31:24,366 - distributed.scheduler - INFO - Remove client Client-baf860bc-d11d-11f0-8001-f2cd832f225c

2025-12-04 14:31:24,364 - distributed.scheduler - INFO - Remove client Client-baf860bc-d11d-11f0-8001-f2cd832f225c

2025-12-04 14:30:06,988 - distributed.scheduler - INFO - Receive client connection: Client-baf860bc-d11d-11f0-8001-f2cd832f225c

2025-12-04 14:20:48,781 - distributed.scheduler - INFO - Close client connection: Client-54773d62-d11c-11f0-8001-f2cd832f225c

2025-12-04 14:20:48,781 - distributed.scheduler - INFO - Remove client Client-54773d62-d11c-11f0-8001-f2cd832f225c

2025-12-04 14:20:48,780 - distributed.scheduler - INFO - Remove client Client-54773d62-d11c-11f0-8001-f2cd832f225c

2025-12-04 14:20:05,512 - distributed.scheduler - INFO - Receive client connection: Client-54773d62-d11c-11f0-8001-f2cd832f225c

2025-12-04 14:10:46,892 - distributed.scheduler - INFO - Close client connection: Client-edf5614d-d11a-11f0-8001-f2cd832f225c

2025-12-04 14:10:46,891 - distributed.scheduler - INFO - Remove client Client-edf5614d-d11a-11f0-8001-f2cd832f225c

2025-12-04 14:10:46,890 - distributed.scheduler - INFO - Remove client Client-edf5614d-d11a-11f0-8001-f2cd832f225c

2025-12-04 14:10:04,036 - distributed.scheduler - INFO - Receive client connection: Client-edf5614d-d11a-11f0-8001-f2cd832f225c

2025-12-04 14:01:14,968 - distributed.scheduler - INFO - Close client connection: Client-878a855d-d119-11f0-8001-f2cd832f225c

2025-12-04 14:01:14,967 - distributed.scheduler - INFO - Remove client Client-878a855d-d119-11f0-8001-f2cd832f225c

2025-12-04 14:01:14,965 - distributed.scheduler - INFO - Remove client Client-878a855d-d119-11f0-8001-f2cd832f225c

2025-12-04 14:00:07,484 - distributed.scheduler - INFO - Receive client connection: Client-worker-8a608324-d119-11f0-88a8-4ec0916b0c7e

2025-12-04 14:00:02,712 - distributed.scheduler - INFO - Receive client connection: Client-878a855d-d119-11f0-8001-f2cd832f225c

2025-12-04 13:50:42,429 - distributed.scheduler - INFO - Close client connection: Client-21054d97-d118-11f0-8001-f2cd832f225c

2025-12-04 13:50:42,429 - distributed.scheduler - INFO - Remove client Client-21054d97-d118-11f0-8001-f2cd832f225c

2025-12-04 13:50:42,427 - distributed.scheduler - INFO - Remove client Client-21054d97-d118-11f0-8001-f2cd832f225c

2025-12-04 13:50:01,219 - distributed.scheduler - INFO - Receive client connection: Client-21054d97-d118-11f0-8001-f2cd832f225c

2025-12-04 13:41:40,628 - distributed.scheduler - INFO - Close client connection: Client-de5e607e-d116-11f0-8001-f2cd832f225c

2025-12-04 13:41:40,627 - distributed.scheduler - INFO - Remove client Client-de5e607e-d116-11f0-8001-f2cd832f225c

2025-12-04 13:41:40,625 - distributed.scheduler - INFO - Remove client Client-de5e607e-d116-11f0-8001-f2cd832f225c

2025-12-04 13:40:59,907 - distributed.scheduler - INFO - Receive client connection: Client-de5e607e-d116-11f0-8001-f2cd832f225c

2025-12-04 13:32:09,316 - distributed.scheduler - INFO - Close client connection: Client-77e823dc-d115-11f0-8001-f2cd832f225c

2025-12-04 13:32:09,316 - distributed.scheduler - INFO - Remove client Client-77e823dc-d115-11f0-8001-f2cd832f225c

2025-12-04 13:32:09,314 - distributed.scheduler - INFO - Remove client Client-77e823dc-d115-11f0-8001-f2cd832f225c

2025-12-04 13:31:04,046 - distributed.scheduler - INFO - Receive client connection: Client-worker-7b2b9220-d115-11f0-891b-32001d5d2425

2025-12-04 13:30:58,500 - distributed.scheduler - INFO - Receive client connection: Client-77e823dc-d115-11f0-8001-f2cd832f225c

2025-12-04 13:21:40,972 - distributed.scheduler - INFO - Close client connection: Client-1177a082-d114-11f0-8001-f2cd832f225c

2025-12-04 13:21:40,971 - distributed.scheduler - INFO - Remove client Client-1177a082-d114-11f0-8001-f2cd832f225c

2025-12-04 13:21:40,968 - distributed.scheduler - INFO - Remove client Client-1177a082-d114-11f0-8001-f2cd832f225c

2025-12-04 13:20:57,132 - distributed.scheduler - INFO - Receive client connection: Client-1177a082-d114-11f0-8001-f2cd832f225c

2025-12-04 13:11:39,114 - distributed.scheduler - INFO - Close client connection: Client-ab053dfa-d112-11f0-8001-f2cd832f225c

2025-12-04 13:11:39,112 - distributed.scheduler - INFO - Remove client Client-ab053dfa-d112-11f0-8001-f2cd832f225c

2025-12-04 13:11:39,111 - distributed.scheduler - INFO - Remove client Client-ab053dfa-d112-11f0-8001-f2cd832f225c

2025-12-04 13:10:55,764 - distributed.scheduler - INFO - Receive client connection: Client-ab053dfa-d112-11f0-8001-f2cd832f225c

2025-12-04 13:02:34,278 - distributed.scheduler - INFO - Close client connection: Client-448b656b-d111-11f0-8001-f2cd832f225c

2025-12-04 13:02:34,278 - distributed.scheduler - INFO - Remove client Client-448b656b-d111-11f0-8001-f2cd832f225c

2025-12-04 13:02:34,277 - distributed.scheduler - INFO - Remove client Client-448b656b-d111-11f0-8001-f2cd832f225c

2025-12-04 13:02:29,534 - distributed.scheduler - INFO - Receive client connection: Client-worker-7d45233e-d111-11f0-88e7-f23d8172cb06

2025-12-04 13:02:27,260 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:33109

2025-12-04 13:02:27,258 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:33109', status: running, memory: 0, processing: 0>

2025-12-04 13:02:26,177 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:44813', status: closed, memory: 0, processing: 0>

2025-12-04 13:02:24,684 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.102:40493

2025-12-04 13:02:24,684 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.102:40493', status: running, memory: 0, processing: 0>

2025-12-04 13:02:23,480 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.102:36113', status: closed, memory: 0, processing: 0>

2025-12-04 13:02:06,801 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:38179 failed: OSError: Timed out trying to connect to tcp://172.21.159.252:38179 after 30 s

2025-12-04 13:02:06,800 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:37263 failed: OSError: Timed out trying to connect to tcp://172.21.25.102:37263 after 30 s

2025-12-04 13:02:06,315 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:37695 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:38179 after 30 s')

2025-12-04 13:02:05,256 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:37263 after 30 s')

2025-12-04 13:02:05,196 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:45521 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:37263 after 30 s')

2025-12-04 13:02:05,126 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:37951 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:37263 after 30 s')

2025-12-04 13:01:35,918 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.102:36113

2025-12-04 13:01:35,916 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.102:36113', status: closed, memory: 0, processing: 0>

2025-12-04 13:01:35,365 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:44813

2025-12-04 13:01:35,364 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:44813', status: closed, memory: 0, processing: 0>

2025-12-04 13:01:34,947 - distributed.scheduler - INFO - Close client connection: Client-worker-47ebbe3d-d111-11f0-88a6-32001d5d2425

2025-12-04 13:01:34,938 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.102:37263', status: closed, memory: 0, processing: 0>

2025-12-04 13:01:34,937 - distributed.scheduler - INFO - Remove client Client-worker-47ebbe3d-d111-11f0-88a6-32001d5d2425

2025-12-04 13:01:34,395 - distributed.scheduler - INFO - Close client connection: Client-worker-da56ae8e-d058-11f0-8a87-8253c573df33

2025-12-04 13:01:34,375 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:38179', status: closed, memory: 0, processing: 0>

2025-12-04 13:01:34,374 - distributed.scheduler - INFO - Remove client Client-worker-da56ae8e-d058-11f0-8a87-8253c573df33

2025-12-04 13:01:00,887 - distributed.scheduler - INFO - Receive client connection: Client-worker-47ebbe3d-d111-11f0-88a6-32001d5d2425

2025-12-04 13:00:54,337 - distributed.scheduler - INFO - Receive client connection: Client-448b656b-d111-11f0-8001-f2cd832f225c

2025-12-04 12:51:34,409 - distributed.scheduler - INFO - Close client connection: Client-de0fd41f-d10f-11f0-8001-f2cd832f225c

2025-12-04 12:51:34,409 - distributed.scheduler - INFO - Remove client Client-de0fd41f-d10f-11f0-8001-f2cd832f225c

2025-12-04 12:51:34,407 - distributed.scheduler - INFO - Remove client Client-de0fd41f-d10f-11f0-8001-f2cd832f225c

2025-12-04 12:50:52,911 - distributed.scheduler - INFO - Receive client connection: Client-de0fd41f-d10f-11f0-8001-f2cd832f225c

2025-12-04 12:41:35,742 - distributed.scheduler - INFO - Close client connection: Client-778f3976-d10e-11f0-8001-f2cd832f225c

2025-12-04 12:41:35,740 - distributed.scheduler - INFO - Remove client Client-778f3976-d10e-11f0-8001-f2cd832f225c

2025-12-04 12:41:35,739 - distributed.scheduler - INFO - Remove client Client-778f3976-d10e-11f0-8001-f2cd832f225c

2025-12-04 12:40:51,442 - distributed.scheduler - INFO - Receive client connection: Client-778f3976-d10e-11f0-8001-f2cd832f225c

2025-12-04 12:37:40,241 - distributed.scheduler - INFO - Receive client connection: Client-05998ea4-d10e-11f0-8007-1eb5975970b7

2025-12-04 12:37:37,844 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.248:44925

2025-12-04 12:37:37,842 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.248:44925', status: running, memory: 0, processing: 0>

2025-12-04 12:37:37,062 - distributed.scheduler - INFO - Close client connection: Client-4f758106-c56c-11f0-8007-1eb5975970b7

2025-12-04 12:37:37,060 - distributed.scheduler - INFO - Remove client Client-4f758106-c56c-11f0-8007-1eb5975970b7

2025-12-04 12:37:37,058 - distributed.scheduler - INFO - Remove client Client-4f758106-c56c-11f0-8007-1eb5975970b7

2025-12-04 12:37:37,041 - distributed.scheduler - INFO - Close client connection: Client-worker-602870b4-d00d-11f0-87c3-4ec0916b0c7e

2025-12-04 12:37:37,036 - distributed.scheduler - INFO - Task run-aa94d0b861f7376f08eaeb58cb14f222 marked as failed because 3 workers died while trying to run it

2025-12-04 12:37:37,034 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.248:40659', status: closed, memory: 0, processing: 0>

2025-12-04 12:37:37,028 - distributed.scheduler - INFO - Remove client Client-worker-602870b4-d00d-11f0-87c3-4ec0916b0c7e

2025-12-04 12:36:09,497 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.233:39757

2025-12-04 12:36:09,494 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.233:39757', status: running, memory: 0, processing: 0>

2025-12-04 12:36:08,664 - distributed.scheduler - INFO - Close client connection: Client-worker-c02277d9-d03f-11f0-884a-f23d8172cb06

2025-12-04 12:36:08,653 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.233:41877', status: closed, memory: 0, processing: 0>

2025-12-04 12:36:08,651 - distributed.scheduler - INFO - Remove client Client-worker-c02277d9-d03f-11f0-884a-f23d8172cb06

2025-12-04 12:34:26,414 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:43191

2025-12-04 12:34:26,412 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:43191', status: closed, memory: 0, processing: 0>

2025-12-04 12:34:25,502 - distributed.scheduler - INFO - Close client connection: Client-worker-f931239e-d004-11f0-8733-de470ccc1e79

2025-12-04 12:34:25,496 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:40609', status: closed, memory: 0, processing: 0>

2025-12-04 12:34:25,492 - distributed.scheduler - INFO - Remove client Client-worker-f931239e-d004-11f0-8733-de470ccc1e79

2025-12-04 12:32:57,854 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.102:37263

2025-12-04 12:32:57,851 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.102:37263', status: closed, memory: 0, processing: 0>

2025-12-04 12:32:56,810 - distributed.scheduler - INFO - Close client connection: Client-worker-60cb4b5c-d022-11f0-87c6-32001d5d2425

2025-12-04 12:32:56,783 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.102:38881', status: closed, memory: 0, processing: 0>

2025-12-04 12:32:56,780 - distributed.scheduler - INFO - Remove client Client-worker-60cb4b5c-d022-11f0-87c6-32001d5d2425

2025-12-04 12:32:10,120 - distributed.scheduler - INFO - Close client connection: Client-1114164a-d10d-11f0-8001-f2cd832f225c

2025-12-04 12:32:10,119 - distributed.scheduler - INFO - Remove client Client-1114164a-d10d-11f0-8001-f2cd832f225c

2025-12-04 12:32:10,117 - distributed.scheduler - INFO - Remove client Client-1114164a-d10d-11f0-8001-f2cd832f225c

2025-12-04 12:30:50,009 - distributed.scheduler - INFO - Receive client connection: Client-1114164a-d10d-11f0-8001-f2cd832f225c

2025-12-04 12:21:28,494 - distributed.scheduler - INFO - Close client connection: Client-aaa20dc9-d10b-11f0-8001-f2cd832f225c

2025-12-04 12:21:28,494 - distributed.scheduler - INFO - Remove client Client-aaa20dc9-d10b-11f0-8001-f2cd832f225c

2025-12-04 12:21:28,492 - distributed.scheduler - INFO - Remove client Client-aaa20dc9-d10b-11f0-8001-f2cd832f225c

2025-12-04 12:20:48,639 - distributed.scheduler - INFO - Receive client connection: Client-aaa20dc9-d10b-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,333 - distributed.scheduler - INFO - Close client connection: Client-dbfd0158-d0ec-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,333 - distributed.scheduler - INFO - Close client connection: Client-42796ccb-d0ee-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,333 - distributed.scheduler - INFO - Close client connection: Client-a8fc9bfd-d0ef-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,333 - distributed.scheduler - INFO - Close client connection: Client-0f7cc9d4-d0f1-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,332 - distributed.scheduler - INFO - Close client connection: Client-75ff01d7-d0f2-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,332 - distributed.scheduler - INFO - Close client connection: Client-dc80b540-d0f3-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,332 - distributed.scheduler - INFO - Close client connection: Client-42f738d9-d0f5-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,332 - distributed.scheduler - INFO - Close client connection: Client-a960194a-d0f6-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,332 - distributed.scheduler - INFO - Close client connection: Client-0fcb8eba-d0f8-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,332 - distributed.scheduler - INFO - Close client connection: Client-764e0c8c-d0f9-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,332 - distributed.scheduler - INFO - Close client connection: Client-dcc0ae0f-d0fa-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,332 - distributed.scheduler - INFO - Close client connection: Client-433b4969-d0fc-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,332 - distributed.scheduler - INFO - Close client connection: Client-a9b01d1a-d0fd-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,331 - distributed.scheduler - INFO - Close client connection: Client-102b4be4-d0ff-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,331 - distributed.scheduler - INFO - Close client connection: Client-76a87dca-d100-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,331 - distributed.scheduler - INFO - Close client connection: Client-dd23e94b-d101-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,331 - distributed.scheduler - INFO - Close client connection: Client-43a81d8e-d103-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,330 - distributed.scheduler - INFO - Close client connection: Client-aa219430-d104-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,330 - distributed.scheduler - INFO - Close client connection: Client-10ef1b51-d106-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,330 - distributed.scheduler - INFO - Close client connection: Client-7712f381-d107-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,330 - distributed.scheduler - INFO - Close client connection: Client-dd99d022-d108-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,330 - distributed.scheduler - INFO - Remove client Client-dbfd0158-d0ec-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,330 - distributed.scheduler - INFO - Remove client Client-42796ccb-d0ee-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,329 - distributed.scheduler - INFO - Remove client Client-a8fc9bfd-d0ef-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,329 - distributed.scheduler - INFO - Remove client Client-0f7cc9d4-d0f1-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,327 - distributed.scheduler - INFO - Remove client Client-75ff01d7-d0f2-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,327 - distributed.scheduler - INFO - Remove client Client-dc80b540-d0f3-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,327 - distributed.scheduler - INFO - Remove client Client-42f738d9-d0f5-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,326 - distributed.scheduler - INFO - Remove client Client-a960194a-d0f6-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,326 - distributed.scheduler - INFO - Remove client Client-0fcb8eba-d0f8-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,326 - distributed.scheduler - INFO - Remove client Client-764e0c8c-d0f9-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,325 - distributed.scheduler - INFO - Remove client Client-dcc0ae0f-d0fa-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,325 - distributed.scheduler - INFO - Remove client Client-433b4969-d0fc-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,325 - distributed.scheduler - INFO - Remove client Client-a9b01d1a-d0fd-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,325 - distributed.scheduler - INFO - Remove client Client-102b4be4-d0ff-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,324 - distributed.scheduler - INFO - Remove client Client-76a87dca-d100-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,324 - distributed.scheduler - INFO - Remove client Client-dd23e94b-d101-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,324 - distributed.scheduler - INFO - Remove client Client-43a81d8e-d103-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,321 - distributed.scheduler - INFO - Remove client Client-aa219430-d104-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,321 - distributed.scheduler - INFO - Remove client Client-10ef1b51-d106-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,320 - distributed.scheduler - INFO - Remove client Client-7712f381-d107-11f0-8001-f2cd832f225c

2025-12-04 12:11:28,320 - distributed.scheduler - INFO - Remove client Client-dd99d022-d108-11f0-8001-f2cd832f225c

2025-12-04 12:11:27,729 - distributed.scheduler - INFO - Close client connection: Client-4428b401-d10a-11f0-8001-f2cd832f225c

2025-12-04 12:11:27,728 - distributed.scheduler - INFO - Remove client Client-4428b401-d10a-11f0-8001-f2cd832f225c

2025-12-04 12:11:27,726 - distributed.scheduler - INFO - Remove client Client-4428b401-d10a-11f0-8001-f2cd832f225c

2025-12-04 12:10:47,210 - distributed.scheduler - INFO - Receive client connection: Client-4428b401-d10a-11f0-8001-f2cd832f225c

2025-12-04 12:00:45,643 - distributed.scheduler - INFO - Receive client connection: Client-dd99d022-d108-11f0-8001-f2cd832f225c

2025-12-04 11:50:44,138 - distributed.scheduler - INFO - Receive client connection: Client-7712f381-d107-11f0-8001-f2cd832f225c

2025-12-04 11:40:43,283 - distributed.scheduler - INFO - Receive client connection: Client-10ef1b51-d106-11f0-8001-f2cd832f225c

2025-12-04 11:30:41,307 - distributed.scheduler - INFO - Receive client connection: Client-aa219430-d104-11f0-8001-f2cd832f225c

2025-12-04 11:20:39,892 - distributed.scheduler - INFO - Receive client connection: Client-43a81d8e-d103-11f0-8001-f2cd832f225c

2025-12-04 11:10:38,399 - distributed.scheduler - INFO - Receive client connection: Client-dd23e94b-d101-11f0-8001-f2cd832f225c

2025-12-04 11:00:42,469 - distributed.scheduler - INFO - Receive client connection: Client-worker-79ec714b-d100-11f0-8a67-f2a719d69604

2025-12-04 11:00:36,966 - distributed.scheduler - INFO - Receive client connection: Client-76a87dca-d100-11f0-8001-f2cd832f225c

2025-12-04 10:50:35,516 - distributed.scheduler - INFO - Receive client connection: Client-102b4be4-d0ff-11f0-8001-f2cd832f225c

2025-12-04 10:40:34,090 - distributed.scheduler - INFO - Receive client connection: Client-a9b01d1a-d0fd-11f0-8001-f2cd832f225c

2025-12-04 10:30:32,699 - distributed.scheduler - INFO - Receive client connection: Client-433b4969-d0fc-11f0-8001-f2cd832f225c

2025-12-04 10:20:31,269 - distributed.scheduler - INFO - Receive client connection: Client-dcc0ae0f-d0fa-11f0-8001-f2cd832f225c

2025-12-04 10:10:29,913 - distributed.scheduler - INFO - Receive client connection: Client-764e0c8c-d0f9-11f0-8001-f2cd832f225c

2025-12-04 10:00:32,772 - distributed.scheduler - INFO - Receive client connection: Client-worker-1264624b-d0f8-11f0-859a-e2572cf160eb

2025-12-04 10:00:28,417 - distributed.scheduler - INFO - Receive client connection: Client-0fcb8eba-d0f8-11f0-8001-f2cd832f225c

2025-12-04 09:50:27,092 - distributed.scheduler - INFO - Receive client connection: Client-a960194a-d0f6-11f0-8001-f2cd832f225c

2025-12-04 09:40:25,778 - distributed.scheduler - INFO - Receive client connection: Client-42f738d9-d0f5-11f0-8001-f2cd832f225c

2025-12-04 09:30:24,383 - distributed.scheduler - INFO - Receive client connection: Client-dc80b540-d0f3-11f0-8001-f2cd832f225c

2025-12-04 09:20:22,894 - distributed.scheduler - INFO - Receive client connection: Client-75ff01d7-d0f2-11f0-8001-f2cd832f225c

2025-12-04 09:10:21,423 - distributed.scheduler - INFO - Receive client connection: Client-0f7cc9d4-d0f1-11f0-8001-f2cd832f225c

2025-12-04 09:00:19,956 - distributed.scheduler - INFO - Receive client connection: Client-a8fc9bfd-d0ef-11f0-8001-f2cd832f225c

2025-12-04 08:50:18,473 - distributed.scheduler - INFO - Receive client connection: Client-42796ccb-d0ee-11f0-8001-f2cd832f225c

2025-12-04 08:40:17,048 - distributed.scheduler - INFO - Receive client connection: Client-dbfd0158-d0ec-11f0-8001-f2cd832f225c

2025-12-04 08:34:03,862 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:41729 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:41729 after 30 s

2025-12-04 08:34:03,861 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:46521 failed: OSError: Timed out trying to connect to tcp://172.21.12.150:46521 after 30 s

2025-12-04 08:34:03,824 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:40609 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:41729 after 30 s')

2025-12-04 08:34:03,787 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:46521 after 30 s')

2025-12-04 08:34:03,733 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:46391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:46521 after 30 s')

2025-12-04 08:34:03,731 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:38179 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:41729 after 30 s')

2025-12-04 08:34:03,730 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:37695 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:46521 after 30 s')

2025-12-04 08:34:03,730 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:38247 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:41729 after 30 s')

2025-12-04 08:34:03,729 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:38881 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:41729 after 30 s')

2025-12-04 08:34:03,729 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:40659 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:46521 after 30 s')

2025-12-04 08:34:03,728 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:46521 after 30 s')

2025-12-04 08:34:03,728 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:41877 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:46521 after 30 s')

2025-12-04 08:34:03,684 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:39713 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:41729 after 30 s')

2025-12-04 08:34:03,683 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:41729 after 30 s')

2025-12-04 08:34:03,682 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:45521 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:41729 after 30 s')

2025-12-04 08:34:03,673 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:46521 after 30 s')

2025-12-04 08:33:34,695 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:37951

2025-12-04 08:33:34,693 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:37951', status: closed, memory: 0, processing: 0>

2025-12-04 08:33:33,668 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:39969

2025-12-04 08:33:33,667 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:39969', status: running, memory: 0, processing: 0>

2025-12-04 08:33:33,571 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:41729', status: closed, memory: 0, processing: 0>

2025-12-04 08:33:32,756 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:46521', status: closed, memory: 0, processing: 0>

2025-12-04 08:33:21,681 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:37045 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:37045 after 30 s

2025-12-04 08:33:21,679 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:40741 failed: OSError: Timed out trying to connect to tcp://172.21.12.150:40741 after 30 s

2025-12-04 08:33:21,672 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:40609 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:37045 after 30 s')

2025-12-04 08:33:21,571 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:40741 after 30 s')

2025-12-04 08:33:21,570 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:38247 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:37045 after 30 s')

2025-12-04 08:33:21,569 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:41877 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:37045 after 30 s')

2025-12-04 08:33:21,568 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:40741 after 30 s')

2025-12-04 08:33:21,533 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:38179 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:37045 after 30 s')

2025-12-04 08:33:21,533 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:37045 after 30 s')

2025-12-04 08:33:21,532 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:37695 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:37045 after 30 s')

2025-12-04 08:33:21,532 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:39713 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:37045 after 30 s')

2025-12-04 08:33:21,495 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:45521 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:37045 after 30 s')

2025-12-04 08:33:21,494 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:38881 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:40741 after 30 s')

2025-12-04 08:33:21,478 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:37045 after 30 s')

2025-12-04 08:32:52,426 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:41729

2025-12-04 08:32:52,425 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:41729', status: closed, memory: 0, processing: 0>

2025-12-04 08:32:51,544 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:46521

2025-12-04 08:32:51,542 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:46521', status: closed, memory: 0, processing: 0>

2025-12-04 08:32:51,360 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:37045', status: closed, memory: 0, processing: 0>

2025-12-04 08:32:50,635 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:40741', status: closed, memory: 0, processing: 0>

2025-12-04 08:32:39,107 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:39299 failed: OSError: Timed out trying to connect to tcp://172.21.12.150:39299 after 30 s

2025-12-04 08:32:39,107 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:36017 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:36017 after 30 s

2025-12-04 08:32:39,098 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:40609 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36017 after 30 s')

2025-12-04 08:32:39,049 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:37695 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36017 after 30 s')

2025-12-04 08:32:39,049 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:39299 after 30 s')

2025-12-04 08:32:39,012 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:38247 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36017 after 30 s')

2025-12-04 08:32:39,011 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:39713 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36017 after 30 s')

2025-12-04 08:32:38,997 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36017 after 30 s')

2025-12-04 08:32:38,996 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:45521 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36017 after 30 s')

2025-12-04 08:32:38,995 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36017 after 30 s')

2025-12-04 08:32:38,994 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:46391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36017 after 30 s')

2025-12-04 08:32:38,969 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:38881 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36017 after 30 s')

2025-12-04 08:32:09,915 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:37045

2025-12-04 08:32:09,914 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:37045', status: closed, memory: 0, processing: 0>

2025-12-04 08:32:09,324 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:40741

2025-12-04 08:32:09,321 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:40741', status: closed, memory: 0, processing: 0>

2025-12-04 08:32:08,865 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:36017', status: closed, memory: 0, processing: 0>

2025-12-04 08:32:08,429 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:39299', status: closed, memory: 0, processing: 0>

2025-12-04 08:31:57,657 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:46725 failed: OSError: Timed out trying to connect to tcp://172.21.12.150:46725 after 30 s

2025-12-04 08:31:57,657 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:41965 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:41965 after 30 s

2025-12-04 08:31:57,611 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:46725 after 30 s')

2025-12-04 08:31:57,060 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:38179 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:46725 after 30 s')

2025-12-04 08:31:57,021 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:46725 after 30 s')

2025-12-04 08:31:57,010 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:41965 after 30 s')

2025-12-04 08:31:32,058 - distributed.scheduler - INFO - Close client connection: Client-75862d3b-d0eb-11f0-8001-f2cd832f225c

2025-12-04 08:31:32,056 - distributed.scheduler - INFO - Remove client Client-75862d3b-d0eb-11f0-8001-f2cd832f225c

2025-12-04 08:31:32,055 - distributed.scheduler - INFO - Remove client Client-75862d3b-d0eb-11f0-8001-f2cd832f225c

2025-12-04 08:31:27,699 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:39299

2025-12-04 08:31:27,698 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:39299', status: closed, memory: 0, processing: 0>

2025-12-04 08:31:26,765 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:46725', status: closed, memory: 0, processing: 0>

2025-12-04 08:31:24,880 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:36017

2025-12-04 08:31:24,878 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:36017', status: closed, memory: 0, processing: 0>

2025-12-04 08:31:23,566 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:41965', status: closed, memory: 0, processing: 0>

2025-12-04 08:31:04,744 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:46725

2025-12-04 08:31:04,744 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:46725', status: closed, memory: 0, processing: 0>

2025-12-04 08:31:03,792 - distributed.scheduler - INFO - Close client connection: Client-worker-b0f9caf8-cffc-11f0-84b3-e2572cf160eb

2025-12-04 08:31:03,787 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:38667', status: closed, memory: 0, processing: 0>

2025-12-04 08:31:03,785 - distributed.scheduler - INFO - Remove client Client-worker-b0f9caf8-cffc-11f0-84b3-e2572cf160eb

2025-12-04 08:30:59,983 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:41965

2025-12-04 08:30:59,981 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:41965', status: closed, memory: 0, processing: 0>

2025-12-04 08:30:58,873 - distributed.scheduler - INFO - Close client connection: Client-worker-0f03e30e-d05d-11f0-89d5-f2a719d69604

2025-12-04 08:30:58,861 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:40477', status: closed, memory: 0, processing: 0>

2025-12-04 08:30:58,860 - distributed.scheduler - INFO - Remove client Client-worker-0f03e30e-d05d-11f0-89d5-f2a719d69604

2025-12-04 08:30:15,633 - distributed.scheduler - INFO - Receive client connection: Client-75862d3b-d0eb-11f0-8001-f2cd832f225c

2025-12-04 08:20:53,964 - distributed.scheduler - INFO - Close client connection: Client-0efe3dbd-d0ea-11f0-8001-f2cd832f225c

2025-12-04 08:20:53,963 - distributed.scheduler - INFO - Remove client Client-0efe3dbd-d0ea-11f0-8001-f2cd832f225c

2025-12-04 08:20:53,962 - distributed.scheduler - INFO - Remove client Client-0efe3dbd-d0ea-11f0-8001-f2cd832f225c

2025-12-04 08:20:14,127 - distributed.scheduler - INFO - Receive client connection: Client-0efe3dbd-d0ea-11f0-8001-f2cd832f225c

2025-12-04 08:10:47,340 - distributed.scheduler - INFO - Close client connection: Client-a88de02c-d0e8-11f0-8001-f2cd832f225c

2025-12-04 08:10:47,340 - distributed.scheduler - INFO - Remove client Client-a88de02c-d0e8-11f0-8001-f2cd832f225c

2025-12-04 08:10:47,339 - distributed.scheduler - INFO - Remove client Client-a88de02c-d0e8-11f0-8001-f2cd832f225c

2025-12-04 08:10:12,761 - distributed.scheduler - INFO - Receive client connection: Client-a88de02c-d0e8-11f0-8001-f2cd832f225c

2025-12-04 08:01:21,092 - distributed.scheduler - INFO - Close client connection: Client-420792e8-d0e7-11f0-8001-f2cd832f225c

2025-12-04 08:01:21,090 - distributed.scheduler - INFO - Remove client Client-420792e8-d0e7-11f0-8001-f2cd832f225c

2025-12-04 08:01:21,089 - distributed.scheduler - INFO - Remove client Client-420792e8-d0e7-11f0-8001-f2cd832f225c

2025-12-04 08:00:11,258 - distributed.scheduler - INFO - Receive client connection: Client-420792e8-d0e7-11f0-8001-f2cd832f225c

2025-12-04 07:50:46,736 - distributed.scheduler - INFO - Close client connection: Client-db886d74-d0e5-11f0-8001-f2cd832f225c

2025-12-04 07:50:46,735 - distributed.scheduler - INFO - Remove client Client-db886d74-d0e5-11f0-8001-f2cd832f225c

2025-12-04 07:50:46,734 - distributed.scheduler - INFO - Remove client Client-db886d74-d0e5-11f0-8001-f2cd832f225c

2025-12-04 07:50:09,801 - distributed.scheduler - INFO - Receive client connection: Client-db886d74-d0e5-11f0-8001-f2cd832f225c

2025-12-04 07:40:42,910 - distributed.scheduler - INFO - Close client connection: Client-750e85ec-d0e4-11f0-8001-f2cd832f225c

2025-12-04 07:40:42,908 - distributed.scheduler - INFO - Remove client Client-750e85ec-d0e4-11f0-8001-f2cd832f225c

2025-12-04 07:40:42,907 - distributed.scheduler - INFO - Remove client Client-750e85ec-d0e4-11f0-8001-f2cd832f225c

2025-12-04 07:40:08,375 - distributed.scheduler - INFO - Receive client connection: Client-750e85ec-d0e4-11f0-8001-f2cd832f225c

2025-12-04 07:31:15,874 - distributed.scheduler - INFO - Close client connection: Client-0e93bbd8-d0e3-11f0-8001-f2cd832f225c

2025-12-04 07:31:15,873 - distributed.scheduler - INFO - Remove client Client-0e93bbd8-d0e3-11f0-8001-f2cd832f225c

2025-12-04 07:31:15,871 - distributed.scheduler - INFO - Remove client Client-0e93bbd8-d0e3-11f0-8001-f2cd832f225c

2025-12-04 07:30:06,950 - distributed.scheduler - INFO - Receive client connection: Client-0e93bbd8-d0e3-11f0-8001-f2cd832f225c

2025-12-04 07:20:43,503 - distributed.scheduler - INFO - Close client connection: Client-a8192f7f-d0e1-11f0-8001-f2cd832f225c

2025-12-04 07:20:43,501 - distributed.scheduler - INFO - Remove client Client-a8192f7f-d0e1-11f0-8001-f2cd832f225c

2025-12-04 07:20:43,500 - distributed.scheduler - INFO - Remove client Client-a8192f7f-d0e1-11f0-8001-f2cd832f225c

2025-12-04 07:20:05,523 - distributed.scheduler - INFO - Receive client connection: Client-a8192f7f-d0e1-11f0-8001-f2cd832f225c

2025-12-04 07:10:46,047 - distributed.scheduler - INFO - Close client connection: Client-419698e3-d0e0-11f0-8001-f2cd832f225c

2025-12-04 07:10:46,044 - distributed.scheduler - INFO - Remove client Client-419698e3-d0e0-11f0-8001-f2cd832f225c

2025-12-04 07:10:46,043 - distributed.scheduler - INFO - Remove client Client-419698e3-d0e0-11f0-8001-f2cd832f225c

2025-12-04 07:10:04,035 - distributed.scheduler - INFO - Receive client connection: Client-419698e3-d0e0-11f0-8001-f2cd832f225c

2025-12-04 07:01:16,762 - distributed.scheduler - INFO - Close client connection: Client-db1cc5d4-d0de-11f0-8001-f2cd832f225c

2025-12-04 07:01:16,761 - distributed.scheduler - INFO - Remove client Client-db1cc5d4-d0de-11f0-8001-f2cd832f225c

2025-12-04 07:01:16,759 - distributed.scheduler - INFO - Remove client Client-db1cc5d4-d0de-11f0-8001-f2cd832f225c

2025-12-04 07:00:02,609 - distributed.scheduler - INFO - Receive client connection: Client-db1cc5d4-d0de-11f0-8001-f2cd832f225c

2025-12-04 06:50:42,262 - distributed.scheduler - INFO - Close client connection: Client-74a8502d-d0dd-11f0-8001-f2cd832f225c

2025-12-04 06:50:42,261 - distributed.scheduler - INFO - Remove client Client-74a8502d-d0dd-11f0-8001-f2cd832f225c

2025-12-04 06:50:42,259 - distributed.scheduler - INFO - Remove client Client-74a8502d-d0dd-11f0-8001-f2cd832f225c

2025-12-04 06:50:01,229 - distributed.scheduler - INFO - Receive client connection: Client-74a8502d-d0dd-11f0-8001-f2cd832f225c

2025-12-04 06:41:40,266 - distributed.scheduler - INFO - Close client connection: Client-cb875822-d0da-11f0-8001-f2cd832f225c

2025-12-04 06:41:40,265 - distributed.scheduler - INFO - Remove client Client-cb875822-d0da-11f0-8001-f2cd832f225c

2025-12-04 06:41:39,411 - distributed.scheduler - INFO - Close client connection: Client-32073a02-d0dc-11f0-8001-f2cd832f225c

2025-12-04 06:41:39,410 - distributed.scheduler - INFO - Remove client Client-32073a02-d0dc-11f0-8001-f2cd832f225c

2025-12-04 06:41:39,408 - distributed.scheduler - INFO - Remove client Client-32073a02-d0dc-11f0-8001-f2cd832f225c

2025-12-04 06:40:59,945 - distributed.scheduler - INFO - Receive client connection: Client-32073a02-d0dc-11f0-8001-f2cd832f225c

2025-12-04 06:30:58,486 - distributed.scheduler - INFO - Receive client connection: Client-cb875822-d0da-11f0-8001-f2cd832f225c

2025-12-04 06:21:32,971 - distributed.scheduler - INFO - Close client connection: Client-650f74c9-d0d9-11f0-8001-f2cd832f225c

2025-12-04 06:21:32,970 - distributed.scheduler - INFO - Remove client Client-650f74c9-d0d9-11f0-8001-f2cd832f225c

2025-12-04 06:21:32,969 - distributed.scheduler - INFO - Remove client Client-650f74c9-d0d9-11f0-8001-f2cd832f225c

2025-12-04 06:20:57,084 - distributed.scheduler - INFO - Receive client connection: Client-650f74c9-d0d9-11f0-8001-f2cd832f225c

2025-12-04 06:11:30,766 - distributed.scheduler - INFO - Close client connection: Client-fe9c053d-d0d7-11f0-8001-f2cd832f225c

2025-12-04 06:11:30,763 - distributed.scheduler - INFO - Remove client Client-fe9c053d-d0d7-11f0-8001-f2cd832f225c

2025-12-04 06:11:30,762 - distributed.scheduler - INFO - Remove client Client-fe9c053d-d0d7-11f0-8001-f2cd832f225c

2025-12-04 06:10:55,691 - distributed.scheduler - INFO - Receive client connection: Client-fe9c053d-d0d7-11f0-8001-f2cd832f225c

2025-12-04 06:02:01,734 - distributed.scheduler - INFO - Close client connection: Client-9823e500-d0d6-11f0-8001-f2cd832f225c

2025-12-04 06:02:01,732 - distributed.scheduler - INFO - Remove client Client-9823e500-d0d6-11f0-8001-f2cd832f225c

2025-12-04 06:02:01,731 - distributed.scheduler - INFO - Remove client Client-9823e500-d0d6-11f0-8001-f2cd832f225c

2025-12-04 06:00:54,287 - distributed.scheduler - INFO - Receive client connection: Client-9823e500-d0d6-11f0-8001-f2cd832f225c

2025-12-04 05:51:27,112 - distributed.scheduler - INFO - Close client connection: Client-31ad2286-d0d5-11f0-8001-f2cd832f225c

2025-12-04 05:51:27,110 - distributed.scheduler - INFO - Remove client Client-31ad2286-d0d5-11f0-8001-f2cd832f225c

2025-12-04 05:51:27,109 - distributed.scheduler - INFO - Remove client Client-31ad2286-d0d5-11f0-8001-f2cd832f225c

2025-12-04 05:50:52,882 - distributed.scheduler - INFO - Receive client connection: Client-31ad2286-d0d5-11f0-8001-f2cd832f225c

2025-12-04 05:41:27,603 - distributed.scheduler - INFO - Close client connection: Client-cb269759-d0d3-11f0-8001-f2cd832f225c

2025-12-04 05:41:27,603 - distributed.scheduler - INFO - Remove client Client-cb269759-d0d3-11f0-8001-f2cd832f225c

2025-12-04 05:41:27,602 - distributed.scheduler - INFO - Remove client Client-cb269759-d0d3-11f0-8001-f2cd832f225c

2025-12-04 05:40:51,380 - distributed.scheduler - INFO - Receive client connection: Client-cb269759-d0d3-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,641 - distributed.scheduler - INFO - Close client connection: Client-b6aa1093-d082-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,641 - distributed.scheduler - INFO - Close client connection: Client-1d2f3345-d084-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,641 - distributed.scheduler - INFO - Close client connection: Client-83b1eccf-d085-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,641 - distributed.scheduler - INFO - Close client connection: Client-ea2e0e84-d086-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,641 - distributed.scheduler - INFO - Close client connection: Client-50aa1442-d088-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,641 - distributed.scheduler - INFO - Close client connection: Client-b724fc6d-d089-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,641 - distributed.scheduler - INFO - Close client connection: Client-1d9fe9c0-d08b-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,641 - distributed.scheduler - INFO - Close client connection: Client-8421010b-d08c-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,641 - distributed.scheduler - INFO - Close client connection: Client-51229c07-d08f-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,640 - distributed.scheduler - INFO - Close client connection: Client-eaa12f04-d08d-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,640 - distributed.scheduler - INFO - Close client connection: Client-b799bc0e-d090-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,640 - distributed.scheduler - INFO - Close client connection: Client-1e16d973-d092-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,640 - distributed.scheduler - INFO - Close client connection: Client-84992280-d093-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,640 - distributed.scheduler - INFO - Close client connection: Client-eb16e70b-d094-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,640 - distributed.scheduler - INFO - Close client connection: Client-518df461-d096-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,640 - distributed.scheduler - INFO - Close client connection: Client-b80f1f92-d097-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,640 - distributed.scheduler - INFO - Close client connection: Client-1e8be880-d099-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,640 - distributed.scheduler - INFO - Close client connection: Client-850c17c1-d09a-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,640 - distributed.scheduler - INFO - Close client connection: Client-eb7ea496-d09b-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,639 - distributed.scheduler - INFO - Close client connection: Client-51f9e64c-d09d-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,639 - distributed.scheduler - INFO - Close client connection: Client-b86f0461-d09e-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,639 - distributed.scheduler - INFO - Close client connection: Client-1ed78cd4-d0a0-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,639 - distributed.scheduler - INFO - Close client connection: Client-8554febf-d0a1-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,639 - distributed.scheduler - INFO - Close client connection: Client-c7f88eac-d0a2-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,639 - distributed.scheduler - INFO - Close client connection: Client-2e5fed8c-d0a4-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,639 - distributed.scheduler - INFO - Close client connection: Client-94cfce47-d0a5-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,639 - distributed.scheduler - INFO - Close client connection: Client-fb3ac825-d0a6-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,639 - distributed.scheduler - INFO - Close client connection: Client-61b8bc4a-d0a8-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,639 - distributed.scheduler - INFO - Close client connection: Client-c836b165-d0a9-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,639 - distributed.scheduler - INFO - Close client connection: Client-2eb99b28-d0ab-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,638 - distributed.scheduler - INFO - Close client connection: Client-951f3d55-d0ac-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,638 - distributed.scheduler - INFO - Close client connection: Client-fba7ff4b-d0ad-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,638 - distributed.scheduler - INFO - Close client connection: Client-6224ece2-d0af-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,638 - distributed.scheduler - INFO - Close client connection: Client-c8a1a64f-d0b0-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,638 - distributed.scheduler - INFO - Close client connection: Client-2f19f703-d0b2-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,638 - distributed.scheduler - INFO - Close client connection: Client-9595019a-d0b3-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,638 - distributed.scheduler - INFO - Close client connection: Client-fc04638a-d0b4-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,638 - distributed.scheduler - INFO - Close client connection: Client-6283666a-d0b6-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,638 - distributed.scheduler - INFO - Close client connection: Client-c903f3bf-d0b7-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,638 - distributed.scheduler - INFO - Close client connection: Client-2f801cf4-d0b9-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,637 - distributed.scheduler - INFO - Close client connection: Client-96003fec-d0ba-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,637 - distributed.scheduler - INFO - Close client connection: Client-fc79fec9-d0bb-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,637 - distributed.scheduler - INFO - Close client connection: Client-62e88a9d-d0bd-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,637 - distributed.scheduler - INFO - Close client connection: Client-c957bb43-d0be-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,637 - distributed.scheduler - INFO - Close client connection: Client-2fd39b2c-d0c0-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,637 - distributed.scheduler - INFO - Close client connection: Client-96510a7b-d0c1-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,637 - distributed.scheduler - INFO - Close client connection: Client-fcccad36-d0c2-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,635 - distributed.scheduler - INFO - Close client connection: Client-633eff3a-d0c4-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,635 - distributed.scheduler - INFO - Close client connection: Client-c9c39cc8-d0c5-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,635 - distributed.scheduler - INFO - Close client connection: Client-3089ee4f-d0c7-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,635 - distributed.scheduler - INFO - Close client connection: Client-fda7ae79-d0c9-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,635 - distributed.scheduler - INFO - Close client connection: Client-970b754d-d0c8-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,635 - distributed.scheduler - INFO - Close client connection: Client-642cc647-d0cb-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,635 - distributed.scheduler - INFO - Close client connection: Client-caa4f749-d0cc-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,634 - distributed.scheduler - INFO - Close client connection: Client-31229dd5-d0ce-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,634 - distributed.scheduler - INFO - Close client connection: Client-97c094c6-d0cf-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,634 - distributed.scheduler - INFO - Close client connection: Client-fe3383ec-d0d0-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,631 - distributed.scheduler - INFO - Remove client Client-b6aa1093-d082-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,631 - distributed.scheduler - INFO - Remove client Client-1d2f3345-d084-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,631 - distributed.scheduler - INFO - Remove client Client-83b1eccf-d085-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,631 - distributed.scheduler - INFO - Remove client Client-ea2e0e84-d086-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,630 - distributed.scheduler - INFO - Remove client Client-50aa1442-d088-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,630 - distributed.scheduler - INFO - Remove client Client-b724fc6d-d089-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,630 - distributed.scheduler - INFO - Remove client Client-1d9fe9c0-d08b-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,629 - distributed.scheduler - INFO - Remove client Client-8421010b-d08c-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,629 - distributed.scheduler - INFO - Remove client Client-51229c07-d08f-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,629 - distributed.scheduler - INFO - Remove client Client-eaa12f04-d08d-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,629 - distributed.scheduler - INFO - Remove client Client-b799bc0e-d090-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,628 - distributed.scheduler - INFO - Remove client Client-1e16d973-d092-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,628 - distributed.scheduler - INFO - Remove client Client-84992280-d093-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,628 - distributed.scheduler - INFO - Remove client Client-eb16e70b-d094-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,628 - distributed.scheduler - INFO - Remove client Client-518df461-d096-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,627 - distributed.scheduler - INFO - Remove client Client-b80f1f92-d097-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,627 - distributed.scheduler - INFO - Remove client Client-1e8be880-d099-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,627 - distributed.scheduler - INFO - Remove client Client-850c17c1-d09a-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,627 - distributed.scheduler - INFO - Remove client Client-eb7ea496-d09b-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,626 - distributed.scheduler - INFO - Remove client Client-51f9e64c-d09d-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,626 - distributed.scheduler - INFO - Remove client Client-b86f0461-d09e-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,626 - distributed.scheduler - INFO - Remove client Client-1ed78cd4-d0a0-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,625 - distributed.scheduler - INFO - Remove client Client-8554febf-d0a1-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,625 - distributed.scheduler - INFO - Remove client Client-c7f88eac-d0a2-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,625 - distributed.scheduler - INFO - Remove client Client-2e5fed8c-d0a4-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,625 - distributed.scheduler - INFO - Remove client Client-94cfce47-d0a5-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,624 - distributed.scheduler - INFO - Remove client Client-fb3ac825-d0a6-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,624 - distributed.scheduler - INFO - Remove client Client-61b8bc4a-d0a8-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,624 - distributed.scheduler - INFO - Remove client Client-c836b165-d0a9-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,623 - distributed.scheduler - INFO - Remove client Client-2eb99b28-d0ab-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,623 - distributed.scheduler - INFO - Remove client Client-951f3d55-d0ac-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,623 - distributed.scheduler - INFO - Remove client Client-fba7ff4b-d0ad-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,622 - distributed.scheduler - INFO - Remove client Client-6224ece2-d0af-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,622 - distributed.scheduler - INFO - Remove client Client-c8a1a64f-d0b0-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,622 - distributed.scheduler - INFO - Remove client Client-2f19f703-d0b2-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,621 - distributed.scheduler - INFO - Remove client Client-9595019a-d0b3-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,621 - distributed.scheduler - INFO - Remove client Client-fc04638a-d0b4-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,621 - distributed.scheduler - INFO - Remove client Client-6283666a-d0b6-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,621 - distributed.scheduler - INFO - Remove client Client-c903f3bf-d0b7-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,620 - distributed.scheduler - INFO - Remove client Client-2f801cf4-d0b9-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,620 - distributed.scheduler - INFO - Remove client Client-96003fec-d0ba-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,620 - distributed.scheduler - INFO - Remove client Client-fc79fec9-d0bb-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,619 - distributed.scheduler - INFO - Remove client Client-62e88a9d-d0bd-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,619 - distributed.scheduler - INFO - Remove client Client-c957bb43-d0be-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,618 - distributed.scheduler - INFO - Remove client Client-2fd39b2c-d0c0-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,618 - distributed.scheduler - INFO - Remove client Client-96510a7b-d0c1-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,617 - distributed.scheduler - INFO - Remove client Client-fcccad36-d0c2-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,603 - distributed.scheduler - INFO - Remove client Client-633eff3a-d0c4-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,602 - distributed.scheduler - INFO - Remove client Client-c9c39cc8-d0c5-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,601 - distributed.scheduler - INFO - Remove client Client-3089ee4f-d0c7-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,601 - distributed.scheduler - INFO - Remove client Client-fda7ae79-d0c9-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,600 - distributed.scheduler - INFO - Remove client Client-970b754d-d0c8-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,599 - distributed.scheduler - INFO - Remove client Client-642cc647-d0cb-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,599 - distributed.scheduler - INFO - Remove client Client-caa4f749-d0cc-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,597 - distributed.scheduler - INFO - Remove client Client-31229dd5-d0ce-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,596 - distributed.scheduler - INFO - Remove client Client-97c094c6-d0cf-11f0-8001-f2cd832f225c

2025-12-04 05:32:00,595 - distributed.scheduler - INFO - Remove client Client-fe3383ec-d0d0-11f0-8001-f2cd832f225c

2025-12-04 05:31:59,924 - distributed.scheduler - INFO - Close client connection: Client-64b6a56c-d0d2-11f0-8001-f2cd832f225c

2025-12-04 05:31:59,922 - distributed.scheduler - INFO - Remove client Client-64b6a56c-d0d2-11f0-8001-f2cd832f225c

2025-12-04 05:31:59,921 - distributed.scheduler - INFO - Remove client Client-64b6a56c-d0d2-11f0-8001-f2cd832f225c

2025-12-04 05:30:50,018 - distributed.scheduler - INFO - Receive client connection: Client-64b6a56c-d0d2-11f0-8001-f2cd832f225c

2025-12-04 05:20:48,529 - distributed.scheduler - INFO - Receive client connection: Client-fe3383ec-d0d0-11f0-8001-f2cd832f225c

2025-12-04 05:10:47,147 - distributed.scheduler - INFO - Receive client connection: Client-97c094c6-d0cf-11f0-8001-f2cd832f225c

2025-12-04 05:00:45,488 - distributed.scheduler - INFO - Receive client connection: Client-31229dd5-d0ce-11f0-8001-f2cd832f225c

2025-12-04 04:50:44,047 - distributed.scheduler - INFO - Receive client connection: Client-caa4f749-d0cc-11f0-8001-f2cd832f225c

2025-12-04 04:40:42,626 - distributed.scheduler - INFO - Receive client connection: Client-642cc647-d0cb-11f0-8001-f2cd832f225c

2025-12-04 04:30:41,132 - distributed.scheduler - INFO - Receive client connection: Client-fda7ae79-d0c9-11f0-8001-f2cd832f225c

2025-12-04 04:20:39,483 - distributed.scheduler - INFO - Receive client connection: Client-970b754d-d0c8-11f0-8001-f2cd832f225c

2025-12-04 04:10:38,011 - distributed.scheduler - INFO - Receive client connection: Client-3089ee4f-d0c7-11f0-8001-f2cd832f225c

2025-12-04 04:00:36,093 - distributed.scheduler - INFO - Receive client connection: Client-c9c39cc8-d0c5-11f0-8001-f2cd832f225c

2025-12-04 03:50:34,597 - distributed.scheduler - INFO - Receive client connection: Client-633eff3a-d0c4-11f0-8001-f2cd832f225c

2025-12-04 03:40:33,223 - distributed.scheduler - INFO - Receive client connection: Client-fcccad36-d0c2-11f0-8001-f2cd832f225c

2025-12-04 03:30:31,789 - distributed.scheduler - INFO - Receive client connection: Client-96510a7b-d0c1-11f0-8001-f2cd832f225c

2025-12-04 03:20:30,340 - distributed.scheduler - INFO - Receive client connection: Client-2fd39b2c-d0c0-11f0-8001-f2cd832f225c

2025-12-04 03:10:28,907 - distributed.scheduler - INFO - Receive client connection: Client-c957bb43-d0be-11f0-8001-f2cd832f225c

2025-12-04 03:00:27,548 - distributed.scheduler - INFO - Receive client connection: Client-62e88a9d-d0bd-11f0-8001-f2cd832f225c

2025-12-04 02:50:26,204 - distributed.scheduler - INFO - Receive client connection: Client-fc79fec9-d0bb-11f0-8001-f2cd832f225c

2025-12-04 02:40:24,778 - distributed.scheduler - INFO - Receive client connection: Client-96003fec-d0ba-11f0-8001-f2cd832f225c

2025-12-04 02:30:23,314 - distributed.scheduler - INFO - Receive client connection: Client-2f801cf4-d0b9-11f0-8001-f2cd832f225c

2025-12-04 02:20:21,877 - distributed.scheduler - INFO - Receive client connection: Client-c903f3bf-d0b7-11f0-8001-f2cd832f225c

2025-12-04 02:10:20,412 - distributed.scheduler - INFO - Receive client connection: Client-6283666a-d0b6-11f0-8001-f2cd832f225c

2025-12-04 02:00:18,953 - distributed.scheduler - INFO - Receive client connection: Client-fc04638a-d0b4-11f0-8001-f2cd832f225c

2025-12-04 01:50:17,599 - distributed.scheduler - INFO - Receive client connection: Client-9595019a-d0b3-11f0-8001-f2cd832f225c

2025-12-04 01:40:16,190 - distributed.scheduler - INFO - Receive client connection: Client-2f19f703-d0b2-11f0-8001-f2cd832f225c

2025-12-04 01:30:14,753 - distributed.scheduler - INFO - Receive client connection: Client-c8a1a64f-d0b0-11f0-8001-f2cd832f225c

2025-12-04 01:20:13,309 - distributed.scheduler - INFO - Receive client connection: Client-6224ece2-d0af-11f0-8001-f2cd832f225c

2025-12-04 01:10:11,871 - distributed.scheduler - INFO - Receive client connection: Client-fba7ff4b-d0ad-11f0-8001-f2cd832f225c

2025-12-04 01:00:10,350 - distributed.scheduler - INFO - Receive client connection: Client-951f3d55-d0ac-11f0-8001-f2cd832f225c

2025-12-04 00:50:09,059 - distributed.scheduler - INFO - Receive client connection: Client-2eb99b28-d0ab-11f0-8001-f2cd832f225c

2025-12-04 00:40:07,574 - distributed.scheduler - INFO - Receive client connection: Client-c836b165-d0a9-11f0-8001-f2cd832f225c

2025-12-04 00:30:06,125 - distributed.scheduler - INFO - Receive client connection: Client-61b8bc4a-d0a8-11f0-8001-f2cd832f225c

2025-12-04 00:20:04,713 - distributed.scheduler - INFO - Receive client connection: Client-fb3ac825-d0a6-11f0-8001-f2cd832f225c

2025-12-04 00:10:03,349 - distributed.scheduler - INFO - Receive client connection: Client-94cfce47-d0a5-11f0-8001-f2cd832f225c

2025-12-04 00:00:02,000 - distributed.scheduler - INFO - Receive client connection: Client-2e5fed8c-d0a4-11f0-8001-f2cd832f225c

2025-12-03 23:50:00,697 - distributed.scheduler - INFO - Receive client connection: Client-c7f88eac-d0a2-11f0-8001-f2cd832f225c

2025-12-03 23:40:59,394 - distributed.scheduler - INFO - Receive client connection: Client-8554febf-d0a1-11f0-8001-f2cd832f225c

2025-12-03 23:30:57,952 - distributed.scheduler - INFO - Receive client connection: Client-1ed78cd4-d0a0-11f0-8001-f2cd832f225c

2025-12-03 23:20:56,638 - distributed.scheduler - INFO - Receive client connection: Client-b86f0461-d09e-11f0-8001-f2cd832f225c

2025-12-03 23:10:55,249 - distributed.scheduler - INFO - Receive client connection: Client-51f9e64c-d09d-11f0-8001-f2cd832f225c

2025-12-03 23:00:53,809 - distributed.scheduler - INFO - Receive client connection: Client-eb7ea496-d09b-11f0-8001-f2cd832f225c

2025-12-03 22:50:52,433 - distributed.scheduler - INFO - Receive client connection: Client-850c17c1-d09a-11f0-8001-f2cd832f225c

2025-12-03 22:40:50,974 - distributed.scheduler - INFO - Receive client connection: Client-1e8be880-d099-11f0-8001-f2cd832f225c

2025-12-03 22:30:49,535 - distributed.scheduler - INFO - Receive client connection: Client-b80f1f92-d097-11f0-8001-f2cd832f225c

2025-12-03 22:20:48,064 - distributed.scheduler - INFO - Receive client connection: Client-518df461-d096-11f0-8001-f2cd832f225c

2025-12-03 22:10:46,658 - distributed.scheduler - INFO - Receive client connection: Client-eb16e70b-d094-11f0-8001-f2cd832f225c

2025-12-03 22:00:45,209 - distributed.scheduler - INFO - Receive client connection: Client-84992280-d093-11f0-8001-f2cd832f225c

2025-12-03 21:50:43,732 - distributed.scheduler - INFO - Receive client connection: Client-1e16d973-d092-11f0-8001-f2cd832f225c

2025-12-03 21:40:42,283 - distributed.scheduler - INFO - Receive client connection: Client-b799bc0e-d090-11f0-8001-f2cd832f225c

2025-12-03 21:30:40,882 - distributed.scheduler - INFO - Receive client connection: Client-51229c07-d08f-11f0-8001-f2cd832f225c

2025-12-03 21:20:39,407 - distributed.scheduler - INFO - Receive client connection: Client-eaa12f04-d08d-11f0-8001-f2cd832f225c

2025-12-03 21:10:37,946 - distributed.scheduler - INFO - Receive client connection: Client-8421010b-d08c-11f0-8001-f2cd832f225c

2025-12-03 21:00:36,470 - distributed.scheduler - INFO - Receive client connection: Client-1d9fe9c0-d08b-11f0-8001-f2cd832f225c

2025-12-03 20:50:35,047 - distributed.scheduler - INFO - Receive client connection: Client-b724fc6d-d089-11f0-8001-f2cd832f225c

2025-12-03 20:40:33,615 - distributed.scheduler - INFO - Receive client connection: Client-50aa1442-d088-11f0-8001-f2cd832f225c

2025-12-03 20:30:32,178 - distributed.scheduler - INFO - Receive client connection: Client-ea2e0e84-d086-11f0-8001-f2cd832f225c

2025-12-03 20:20:30,741 - distributed.scheduler - INFO - Receive client connection: Client-83b1eccf-d085-11f0-8001-f2cd832f225c

2025-12-03 20:10:29,258 - distributed.scheduler - INFO - Receive client connection: Client-1d2f3345-d084-11f0-8001-f2cd832f225c

2025-12-03 20:00:27,771 - distributed.scheduler - INFO - Receive client connection: Client-b6aa1093-d082-11f0-8001-f2cd832f225c

2025-12-03 19:50:58,787 - distributed.scheduler - INFO - Close client connection: Client-5033ac07-d081-11f0-8001-f2cd832f225c

2025-12-03 19:50:58,786 - distributed.scheduler - INFO - Remove client Client-5033ac07-d081-11f0-8001-f2cd832f225c

2025-12-03 19:50:58,784 - distributed.scheduler - INFO - Remove client Client-5033ac07-d081-11f0-8001-f2cd832f225c

2025-12-03 19:50:26,368 - distributed.scheduler - INFO - Receive client connection: Client-5033ac07-d081-11f0-8001-f2cd832f225c

2025-12-03 19:41:01,601 - distributed.scheduler - INFO - Close client connection: Client-e9b3d868-d07f-11f0-8001-f2cd832f225c

2025-12-03 19:41:01,600 - distributed.scheduler - INFO - Remove client Client-e9b3d868-d07f-11f0-8001-f2cd832f225c

2025-12-03 19:41:01,598 - distributed.scheduler - INFO - Remove client Client-e9b3d868-d07f-11f0-8001-f2cd832f225c

2025-12-03 19:40:24,907 - distributed.scheduler - INFO - Receive client connection: Client-e9b3d868-d07f-11f0-8001-f2cd832f225c

2025-12-03 19:31:34,396 - distributed.scheduler - INFO - Close client connection: Client-833e955c-d07e-11f0-8001-f2cd832f225c

2025-12-03 19:31:34,395 - distributed.scheduler - INFO - Remove client Client-833e955c-d07e-11f0-8001-f2cd832f225c

2025-12-03 19:31:34,393 - distributed.scheduler - INFO - Remove client Client-833e955c-d07e-11f0-8001-f2cd832f225c

2025-12-03 19:30:23,512 - distributed.scheduler - INFO - Receive client connection: Client-833e955c-d07e-11f0-8001-f2cd832f225c

2025-12-03 19:21:00,088 - distributed.scheduler - INFO - Close client connection: Client-1cc09c1f-d07d-11f0-8001-f2cd832f225c

2025-12-03 19:21:00,085 - distributed.scheduler - INFO - Remove client Client-1cc09c1f-d07d-11f0-8001-f2cd832f225c

2025-12-03 19:21:00,083 - distributed.scheduler - INFO - Remove client Client-1cc09c1f-d07d-11f0-8001-f2cd832f225c

2025-12-03 19:20:22,065 - distributed.scheduler - INFO - Receive client connection: Client-1cc09c1f-d07d-11f0-8001-f2cd832f225c

2025-12-03 19:10:56,564 - distributed.scheduler - INFO - Close client connection: Client-b650d59b-d07b-11f0-8001-f2cd832f225c

2025-12-03 19:10:56,563 - distributed.scheduler - INFO - Remove client Client-b650d59b-d07b-11f0-8001-f2cd832f225c

2025-12-03 19:10:56,562 - distributed.scheduler - INFO - Remove client Client-b650d59b-d07b-11f0-8001-f2cd832f225c

2025-12-03 19:10:20,711 - distributed.scheduler - INFO - Receive client connection: Client-b650d59b-d07b-11f0-8001-f2cd832f225c

2025-12-03 19:01:27,202 - distributed.scheduler - INFO - Close client connection: Client-1c6a6936-d076-11f0-8001-f2cd832f225c

2025-12-03 19:01:27,201 - distributed.scheduler - INFO - Close client connection: Client-82e2e10e-d077-11f0-8001-f2cd832f225c

2025-12-03 19:01:27,201 - distributed.scheduler - INFO - Close client connection: Client-e959d409-d078-11f0-8001-f2cd832f225c

2025-12-03 19:01:27,200 - distributed.scheduler - INFO - Remove client Client-1c6a6936-d076-11f0-8001-f2cd832f225c

2025-12-03 19:01:27,199 - distributed.scheduler - INFO - Remove client Client-82e2e10e-d077-11f0-8001-f2cd832f225c

2025-12-03 19:01:27,198 - distributed.scheduler - INFO - Remove client Client-e959d409-d078-11f0-8001-f2cd832f225c

2025-12-03 19:01:26,497 - distributed.scheduler - INFO - Close client connection: Client-4fd2e1b1-d07a-11f0-8001-f2cd832f225c

2025-12-03 19:01:26,495 - distributed.scheduler - INFO - Remove client Client-4fd2e1b1-d07a-11f0-8001-f2cd832f225c

2025-12-03 19:01:26,494 - distributed.scheduler - INFO - Remove client Client-4fd2e1b1-d07a-11f0-8001-f2cd832f225c

2025-12-03 19:00:19,248 - distributed.scheduler - INFO - Receive client connection: Client-4fd2e1b1-d07a-11f0-8001-f2cd832f225c

2025-12-03 18:50:17,824 - distributed.scheduler - INFO - Receive client connection: Client-e959d409-d078-11f0-8001-f2cd832f225c

2025-12-03 18:40:16,432 - distributed.scheduler - INFO - Receive client connection: Client-82e2e10e-d077-11f0-8001-f2cd832f225c

2025-12-03 18:30:15,025 - distributed.scheduler - INFO - Receive client connection: Client-1c6a6936-d076-11f0-8001-f2cd832f225c

2025-12-03 18:20:51,507 - distributed.scheduler - INFO - Close client connection: Client-b5e6a0df-d074-11f0-8001-f2cd832f225c

2025-12-03 18:20:51,505 - distributed.scheduler - INFO - Remove client Client-b5e6a0df-d074-11f0-8001-f2cd832f225c

2025-12-03 18:20:51,504 - distributed.scheduler - INFO - Remove client Client-b5e6a0df-d074-11f0-8001-f2cd832f225c

2025-12-03 18:20:13,537 - distributed.scheduler - INFO - Receive client connection: Client-b5e6a0df-d074-11f0-8001-f2cd832f225c

2025-12-03 18:10:50,720 - distributed.scheduler - INFO - Close client connection: Client-4f6a4881-d073-11f0-8001-f2cd832f225c

2025-12-03 18:10:50,719 - distributed.scheduler - INFO - Remove client Client-4f6a4881-d073-11f0-8001-f2cd832f225c

2025-12-03 18:10:50,717 - distributed.scheduler - INFO - Remove client Client-4f6a4881-d073-11f0-8001-f2cd832f225c

2025-12-03 18:10:12,116 - distributed.scheduler - INFO - Receive client connection: Client-4f6a4881-d073-11f0-8001-f2cd832f225c

2025-12-03 18:01:22,560 - distributed.scheduler - INFO - Close client connection: Client-e8ece154-d071-11f0-8001-f2cd832f225c

2025-12-03 18:01:22,559 - distributed.scheduler - INFO - Remove client Client-e8ece154-d071-11f0-8001-f2cd832f225c

2025-12-03 18:01:22,557 - distributed.scheduler - INFO - Remove client Client-e8ece154-d071-11f0-8001-f2cd832f225c

2025-12-03 18:00:10,669 - distributed.scheduler - INFO - Receive client connection: Client-e8ece154-d071-11f0-8001-f2cd832f225c

2025-12-03 17:50:51,922 - distributed.scheduler - INFO - Close client connection: Client-826fa30b-d070-11f0-8001-f2cd832f225c

2025-12-03 17:50:51,920 - distributed.scheduler - INFO - Remove client Client-826fa30b-d070-11f0-8001-f2cd832f225c

2025-12-03 17:50:51,919 - distributed.scheduler - INFO - Remove client Client-826fa30b-d070-11f0-8001-f2cd832f225c

2025-12-03 17:50:09,222 - distributed.scheduler - INFO - Receive client connection: Client-826fa30b-d070-11f0-8001-f2cd832f225c

2025-12-03 17:40:46,999 - distributed.scheduler - INFO - Close client connection: Client-1bf03565-d06f-11f0-8001-f2cd832f225c

2025-12-03 17:40:46,998 - distributed.scheduler - INFO - Remove client Client-1bf03565-d06f-11f0-8001-f2cd832f225c

2025-12-03 17:40:46,996 - distributed.scheduler - INFO - Remove client Client-1bf03565-d06f-11f0-8001-f2cd832f225c

2025-12-03 17:40:07,748 - distributed.scheduler - INFO - Receive client connection: Client-1bf03565-d06f-11f0-8001-f2cd832f225c

2025-12-03 17:31:18,319 - distributed.scheduler - INFO - Close client connection: Client-1b6a91e9-d068-11f0-8001-f2cd832f225c

2025-12-03 17:31:18,319 - distributed.scheduler - INFO - Close client connection: Client-81ece25a-d069-11f0-8001-f2cd832f225c

2025-12-03 17:31:18,319 - distributed.scheduler - INFO - Close client connection: Client-e86e3fd0-d06a-11f0-8001-f2cd832f225c

2025-12-03 17:31:18,319 - distributed.scheduler - INFO - Close client connection: Client-4eef2f4c-d06c-11f0-8001-f2cd832f225c

2025-12-03 17:31:18,318 - distributed.scheduler - INFO - Remove client Client-1b6a91e9-d068-11f0-8001-f2cd832f225c

2025-12-03 17:31:18,318 - distributed.scheduler - INFO - Remove client Client-81ece25a-d069-11f0-8001-f2cd832f225c

2025-12-03 17:31:18,318 - distributed.scheduler - INFO - Remove client Client-e86e3fd0-d06a-11f0-8001-f2cd832f225c

2025-12-03 17:31:18,317 - distributed.scheduler - INFO - Remove client Client-4eef2f4c-d06c-11f0-8001-f2cd832f225c

2025-12-03 17:31:17,651 - distributed.scheduler - INFO - Close client connection: Client-b56f5433-d06d-11f0-8001-f2cd832f225c

2025-12-03 17:31:17,650 - distributed.scheduler - INFO - Remove client Client-b56f5433-d06d-11f0-8001-f2cd832f225c

2025-12-03 17:31:17,648 - distributed.scheduler - INFO - Remove client Client-b56f5433-d06d-11f0-8001-f2cd832f225c

2025-12-03 17:30:06,270 - distributed.scheduler - INFO - Receive client connection: Client-b56f5433-d06d-11f0-8001-f2cd832f225c

2025-12-03 17:20:04,801 - distributed.scheduler - INFO - Receive client connection: Client-4eef2f4c-d06c-11f0-8001-f2cd832f225c

2025-12-03 17:10:03,331 - distributed.scheduler - INFO - Receive client connection: Client-e86e3fd0-d06a-11f0-8001-f2cd832f225c

2025-12-03 17:00:01,861 - distributed.scheduler - INFO - Receive client connection: Client-81ece25a-d069-11f0-8001-f2cd832f225c

2025-12-03 16:50:00,392 - distributed.scheduler - INFO - Receive client connection: Client-1b6a91e9-d068-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,422 - distributed.scheduler - INFO - Close client connection: Client-3dd6230c-d053-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,422 - distributed.scheduler - INFO - Close client connection: Client-a458384b-d054-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,422 - distributed.scheduler - INFO - Close client connection: Client-0adef9c7-d056-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,421 - distributed.scheduler - INFO - Close client connection: Client-715ebd6f-d057-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,421 - distributed.scheduler - INFO - Close client connection: Client-d7d7697b-d058-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,421 - distributed.scheduler - INFO - Close client connection: Client-3e54ed27-d05a-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,421 - distributed.scheduler - INFO - Close client connection: Client-0b748001-d05d-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,420 - distributed.scheduler - INFO - Close client connection: Client-a4e4d79e-d05b-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,420 - distributed.scheduler - INFO - Close client connection: Client-d87a6616-d05f-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,420 - distributed.scheduler - INFO - Close client connection: Client-72386490-d05e-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,420 - distributed.scheduler - INFO - Close client connection: Client-3ef7e786-d061-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,419 - distributed.scheduler - INFO - Close client connection: Client-a5682421-d062-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,419 - distributed.scheduler - INFO - Close client connection: Client-0be6c9b6-d064-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,419 - distributed.scheduler - INFO - Close client connection: Client-7256ad32-d065-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,417 - distributed.scheduler - INFO - Remove client Client-3dd6230c-d053-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,416 - distributed.scheduler - INFO - Remove client Client-a458384b-d054-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,416 - distributed.scheduler - INFO - Remove client Client-0adef9c7-d056-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,413 - distributed.scheduler - INFO - Remove client Client-715ebd6f-d057-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,412 - distributed.scheduler - INFO - Remove client Client-d7d7697b-d058-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,412 - distributed.scheduler - INFO - Remove client Client-3e54ed27-d05a-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,411 - distributed.scheduler - INFO - Remove client Client-0b748001-d05d-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,410 - distributed.scheduler - INFO - Remove client Client-a4e4d79e-d05b-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,409 - distributed.scheduler - INFO - Remove client Client-d87a6616-d05f-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,409 - distributed.scheduler - INFO - Remove client Client-72386490-d05e-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,409 - distributed.scheduler - INFO - Remove client Client-3ef7e786-d061-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,408 - distributed.scheduler - INFO - Remove client Client-a5682421-d062-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,408 - distributed.scheduler - INFO - Remove client Client-0be6c9b6-d064-11f0-8001-f2cd832f225c

2025-12-03 16:41:38,407 - distributed.scheduler - INFO - Remove client Client-7256ad32-d065-11f0-8001-f2cd832f225c

2025-12-03 16:41:37,906 - distributed.scheduler - INFO - Close client connection: Client-d8cefeea-d066-11f0-8001-f2cd832f225c

2025-12-03 16:41:37,904 - distributed.scheduler - INFO - Remove client Client-d8cefeea-d066-11f0-8001-f2cd832f225c

2025-12-03 16:41:37,902 - distributed.scheduler - INFO - Remove client Client-d8cefeea-d066-11f0-8001-f2cd832f225c

2025-12-03 16:40:59,137 - distributed.scheduler - INFO - Receive client connection: Client-d8cefeea-d066-11f0-8001-f2cd832f225c

2025-12-03 16:30:57,723 - distributed.scheduler - INFO - Receive client connection: Client-7256ad32-d065-11f0-8001-f2cd832f225c

2025-12-03 16:20:56,369 - distributed.scheduler - INFO - Receive client connection: Client-0be6c9b6-d064-11f0-8001-f2cd832f225c

2025-12-03 16:10:54,910 - distributed.scheduler - INFO - Receive client connection: Client-a5682421-d062-11f0-8001-f2cd832f225c

2025-12-03 16:00:53,553 - distributed.scheduler - INFO - Receive client connection: Client-3ef7e786-d061-11f0-8001-f2cd832f225c

2025-12-03 15:50:52,106 - distributed.scheduler - INFO - Receive client connection: Client-d87a6616-d05f-11f0-8001-f2cd832f225c

2025-12-03 15:40:51,052 - distributed.scheduler - INFO - Receive client connection: Client-72386490-d05e-11f0-8001-f2cd832f225c

2025-12-03 15:30:55,125 - distributed.scheduler - INFO - Receive client connection: Client-worker-0f03e30e-d05d-11f0-89d5-f2a719d69604

2025-12-03 15:30:49,142 - distributed.scheduler - INFO - Receive client connection: Client-0b748001-d05d-11f0-8001-f2cd832f225c

2025-12-03 15:20:47,577 - distributed.scheduler - INFO - Receive client connection: Client-a4e4d79e-d05b-11f0-8001-f2cd832f225c

2025-12-03 15:10:46,001 - distributed.scheduler - INFO - Receive client connection: Client-3e54ed27-d05a-11f0-8001-f2cd832f225c

2025-12-03 15:00:48,743 - distributed.scheduler - INFO - Receive client connection: Client-worker-da56ae8e-d058-11f0-8a87-8253c573df33

2025-12-03 15:00:44,561 - distributed.scheduler - INFO - Receive client connection: Client-d7d7697b-d058-11f0-8001-f2cd832f225c

2025-12-03 14:50:43,142 - distributed.scheduler - INFO - Receive client connection: Client-715ebd6f-d057-11f0-8001-f2cd832f225c

2025-12-03 14:40:41,683 - distributed.scheduler - INFO - Receive client connection: Client-0adef9c7-d056-11f0-8001-f2cd832f225c

2025-12-03 14:33:38,262 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:40477

2025-12-03 14:33:38,261 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:40477', status: closed, memory: 0, processing: 0>

2025-12-03 14:33:38,142 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:38179

2025-12-03 14:33:38,140 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:38179', status: closed, memory: 0, processing: 0>

2025-12-03 14:33:37,132 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:37665', status: closed, memory: 0, processing: 0>

2025-12-03 14:33:37,095 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:37885', status: closed, memory: 0, processing: 0>

2025-12-03 14:33:26,599 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:40475 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:40475 after 30 s

2025-12-03 14:33:26,599 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:38465 failed: OSError: Timed out trying to connect to tcp://172.21.159.252:38465 after 30 s

2025-12-03 14:33:26,581 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:38667 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40475 after 30 s')

2025-12-03 14:33:26,572 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40475 after 30 s')

2025-12-03 14:33:26,494 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:38247 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40475 after 30 s')

2025-12-03 14:33:26,492 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40475 after 30 s')

2025-12-03 14:33:26,492 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:39713 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40475 after 30 s')

2025-12-03 14:33:26,492 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40475 after 30 s')

2025-12-03 14:33:26,458 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:40609 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40475 after 30 s')

2025-12-03 14:33:26,449 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:45521 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40475 after 30 s')

2025-12-03 14:33:26,447 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:46391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40475 after 30 s')

2025-12-03 14:33:26,444 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:37695 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40475 after 30 s')

2025-12-03 14:33:26,441 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40475 after 30 s')

2025-12-03 14:33:26,431 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:38881 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40475 after 30 s')

2025-12-03 14:32:57,397 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:37665

2025-12-03 14:32:57,395 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:37665', status: closed, memory: 0, processing: 0>

2025-12-03 14:32:57,193 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:37885

2025-12-03 14:32:57,191 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:37885', status: closed, memory: 0, processing: 0>

2025-12-03 14:32:56,328 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:40475', status: closed, memory: 0, processing: 0>

2025-12-03 14:32:56,096 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:38465', status: closed, memory: 0, processing: 0>

2025-12-03 14:32:46,267 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:40475

2025-12-03 14:32:46,266 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:40475', status: closed, memory: 0, processing: 0>

2025-12-03 14:32:46,180 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:38465

2025-12-03 14:32:46,178 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:38465', status: closed, memory: 0, processing: 0>

2025-12-03 14:32:45,237 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:33613', status: closed, memory: 0, processing: 0>

2025-12-03 14:32:45,212 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:41723', status: closed, memory: 0, processing: 0>

2025-12-03 14:32:35,001 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:34543 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:34543 after 30 s

2025-12-03 14:32:35,001 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:43679 failed: OSError: Timed out trying to connect to tcp://172.21.159.252:43679 after 30 s

2025-12-03 14:32:34,671 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:34543 after 30 s')

2025-12-03 14:32:34,575 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:38881 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:34543 after 30 s')

2025-12-03 14:32:34,473 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:34543 after 30 s')

2025-12-03 14:32:05,462 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:33613

2025-12-03 14:32:05,461 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:33613', status: closed, memory: 0, processing: 0>

2025-12-03 14:32:04,917 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:41723

2025-12-03 14:32:04,915 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:41723', status: closed, memory: 0, processing: 0>

2025-12-03 14:32:04,233 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:34543', status: closed, memory: 0, processing: 0>

2025-12-03 14:32:04,141 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:43679', status: closed, memory: 0, processing: 0>

2025-12-03 14:31:28,981 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:34543

2025-12-03 14:31:28,980 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:34543', status: closed, memory: 0, processing: 0>

2025-12-03 14:31:27,944 - distributed.scheduler - INFO - Close client connection: Client-worker-04a1c29c-cf83-11f0-88c7-f2a719d69604

2025-12-03 14:31:27,943 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:34871', status: closed, memory: 0, processing: 0>

2025-12-03 14:31:27,941 - distributed.scheduler - INFO - Remove client Client-worker-04a1c29c-cf83-11f0-88c7-f2a719d69604

2025-12-03 14:31:20,536 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:43679

2025-12-03 14:31:20,535 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:43679', status: closed, memory: 0, processing: 0>

2025-12-03 14:31:19,702 - distributed.scheduler - INFO - Close client connection: Client-worker-c7340ade-d015-11f0-89b3-8253c573df33

2025-12-03 14:31:19,694 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:34943', status: closed, memory: 0, processing: 0>

2025-12-03 14:31:19,688 - distributed.scheduler - INFO - Remove client Client-worker-c7340ade-d015-11f0-89b3-8253c573df33

2025-12-03 14:30:40,180 - distributed.scheduler - INFO - Receive client connection: Client-a458384b-d054-11f0-8001-f2cd832f225c

2025-12-03 14:20:38,709 - distributed.scheduler - INFO - Receive client connection: Client-3dd6230c-d053-11f0-8001-f2cd832f225c

2025-12-03 14:11:14,113 - distributed.scheduler - INFO - Close client connection: Client-d74c2f9b-d051-11f0-8001-f2cd832f225c

2025-12-03 14:11:14,112 - distributed.scheduler - INFO - Remove client Client-d74c2f9b-d051-11f0-8001-f2cd832f225c

2025-12-03 14:11:14,111 - distributed.scheduler - INFO - Remove client Client-d74c2f9b-d051-11f0-8001-f2cd832f225c

2025-12-03 14:10:37,172 - distributed.scheduler - INFO - Receive client connection: Client-d74c2f9b-d051-11f0-8001-f2cd832f225c

2025-12-03 14:01:43,913 - distributed.scheduler - INFO - Close client connection: Client-70cc91ab-d050-11f0-8001-f2cd832f225c

2025-12-03 14:01:43,912 - distributed.scheduler - INFO - Remove client Client-70cc91ab-d050-11f0-8001-f2cd832f225c

2025-12-03 14:01:43,911 - distributed.scheduler - INFO - Remove client Client-70cc91ab-d050-11f0-8001-f2cd832f225c

2025-12-03 14:00:35,717 - distributed.scheduler - INFO - Receive client connection: Client-70cc91ab-d050-11f0-8001-f2cd832f225c

2025-12-03 13:51:08,584 - distributed.scheduler - INFO - Close client connection: Client-0a4f1020-d04f-11f0-8001-f2cd832f225c

2025-12-03 13:51:08,584 - distributed.scheduler - INFO - Remove client Client-0a4f1020-d04f-11f0-8001-f2cd832f225c

2025-12-03 13:51:08,583 - distributed.scheduler - INFO - Remove client Client-0a4f1020-d04f-11f0-8001-f2cd832f225c

2025-12-03 13:50:34,274 - distributed.scheduler - INFO - Receive client connection: Client-0a4f1020-d04f-11f0-8001-f2cd832f225c

2025-12-03 13:41:10,930 - distributed.scheduler - INFO - Close client connection: Client-a3d1cfbd-d04d-11f0-8001-f2cd832f225c

2025-12-03 13:41:10,929 - distributed.scheduler - INFO - Remove client Client-a3d1cfbd-d04d-11f0-8001-f2cd832f225c

2025-12-03 13:41:10,928 - distributed.scheduler - INFO - Remove client Client-a3d1cfbd-d04d-11f0-8001-f2cd832f225c

2025-12-03 13:40:32,825 - distributed.scheduler - INFO - Receive client connection: Client-a3d1cfbd-d04d-11f0-8001-f2cd832f225c

2025-12-03 13:31:44,618 - distributed.scheduler - INFO - Close client connection: Client-3d52daac-d04c-11f0-8001-f2cd832f225c

2025-12-03 13:31:44,617 - distributed.scheduler - INFO - Remove client Client-3d52daac-d04c-11f0-8001-f2cd832f225c

2025-12-03 13:31:44,616 - distributed.scheduler - INFO - Remove client Client-3d52daac-d04c-11f0-8001-f2cd832f225c

2025-12-03 13:30:31,370 - distributed.scheduler - INFO - Receive client connection: Client-3d52daac-d04c-11f0-8001-f2cd832f225c

2025-12-03 13:21:08,251 - distributed.scheduler - INFO - Close client connection: Client-d6d6fc16-d04a-11f0-8001-f2cd832f225c

2025-12-03 13:21:08,246 - distributed.scheduler - INFO - Remove client Client-d6d6fc16-d04a-11f0-8001-f2cd832f225c

2025-12-03 13:21:08,244 - distributed.scheduler - INFO - Remove client Client-d6d6fc16-d04a-11f0-8001-f2cd832f225c

2025-12-03 13:20:29,939 - distributed.scheduler - INFO - Receive client connection: Client-d6d6fc16-d04a-11f0-8001-f2cd832f225c

2025-12-03 13:11:08,536 - distributed.scheduler - INFO - Close client connection: Client-705d9682-d049-11f0-8001-f2cd832f225c

2025-12-03 13:11:08,535 - distributed.scheduler - INFO - Remove client Client-705d9682-d049-11f0-8001-f2cd832f225c

2025-12-03 13:11:08,534 - distributed.scheduler - INFO - Remove client Client-705d9682-d049-11f0-8001-f2cd832f225c

2025-12-03 13:10:28,511 - distributed.scheduler - INFO - Receive client connection: Client-705d9682-d049-11f0-8001-f2cd832f225c

2025-12-03 13:01:42,961 - distributed.scheduler - INFO - Close client connection: Client-09eb2985-d048-11f0-8001-f2cd832f225c

2025-12-03 13:01:42,957 - distributed.scheduler - INFO - Remove client Client-09eb2985-d048-11f0-8001-f2cd832f225c

2025-12-03 13:01:42,956 - distributed.scheduler - INFO - Remove client Client-09eb2985-d048-11f0-8001-f2cd832f225c

2025-12-03 13:00:27,140 - distributed.scheduler - INFO - Receive client connection: Client-09eb2985-d048-11f0-8001-f2cd832f225c

2025-12-03 12:51:10,889 - distributed.scheduler - INFO - Close client connection: Client-a377dcec-d046-11f0-8001-f2cd832f225c

2025-12-03 12:51:10,888 - distributed.scheduler - INFO - Remove client Client-a377dcec-d046-11f0-8001-f2cd832f225c

2025-12-03 12:51:10,886 - distributed.scheduler - INFO - Remove client Client-a377dcec-d046-11f0-8001-f2cd832f225c

2025-12-03 12:50:25,760 - distributed.scheduler - INFO - Receive client connection: Client-a377dcec-d046-11f0-8001-f2cd832f225c

2025-12-03 12:41:06,274 - distributed.scheduler - INFO - Close client connection: Client-3cfbd5b2-d045-11f0-8001-f2cd832f225c

2025-12-03 12:41:06,273 - distributed.scheduler - INFO - Remove client Client-3cfbd5b2-d045-11f0-8001-f2cd832f225c

2025-12-03 12:41:06,265 - distributed.scheduler - INFO - Remove client Client-3cfbd5b2-d045-11f0-8001-f2cd832f225c

2025-12-03 12:40:24,320 - distributed.scheduler - INFO - Receive client connection: Client-3cfbd5b2-d045-11f0-8001-f2cd832f225c

2025-12-03 12:31:33,046 - distributed.scheduler - INFO - Close client connection: Client-d67fc7e4-d043-11f0-8001-f2cd832f225c

2025-12-03 12:31:33,046 - distributed.scheduler - INFO - Remove client Client-d67fc7e4-d043-11f0-8001-f2cd832f225c

2025-12-03 12:31:33,044 - distributed.scheduler - INFO - Remove client Client-d67fc7e4-d043-11f0-8001-f2cd832f225c

2025-12-03 12:30:22,884 - distributed.scheduler - INFO - Receive client connection: Client-d67fc7e4-d043-11f0-8001-f2cd832f225c

2025-12-03 12:21:03,007 - distributed.scheduler - INFO - Close client connection: Client-6ff84621-d042-11f0-8001-f2cd832f225c

2025-12-03 12:21:03,005 - distributed.scheduler - INFO - Remove client Client-6ff84621-d042-11f0-8001-f2cd832f225c

2025-12-03 12:21:03,003 - distributed.scheduler - INFO - Remove client Client-6ff84621-d042-11f0-8001-f2cd832f225c

2025-12-03 12:20:21,372 - distributed.scheduler - INFO - Receive client connection: Client-6ff84621-d042-11f0-8001-f2cd832f225c

2025-12-03 12:11:00,747 - distributed.scheduler - INFO - Close client connection: Client-098143c1-d041-11f0-8001-f2cd832f225c

2025-12-03 12:11:00,745 - distributed.scheduler - INFO - Remove client Client-098143c1-d041-11f0-8001-f2cd832f225c

2025-12-03 12:11:00,743 - distributed.scheduler - INFO - Remove client Client-098143c1-d041-11f0-8001-f2cd832f225c

2025-12-03 12:10:19,966 - distributed.scheduler - INFO - Receive client connection: Client-098143c1-d041-11f0-8001-f2cd832f225c

2025-12-03 12:01:41,862 - distributed.scheduler - INFO - Close client connection: Client-a306afaa-d03f-11f0-8001-f2cd832f225c

2025-12-03 12:01:41,859 - distributed.scheduler - INFO - Remove client Client-a306afaa-d03f-11f0-8001-f2cd832f225c

2025-12-03 12:01:41,857 - distributed.scheduler - INFO - Remove client Client-a306afaa-d03f-11f0-8001-f2cd832f225c

2025-12-03 12:01:22,262 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: OSError: Timed out trying to connect to tcp://172.21.159.233:46261 after 30 s

2025-12-03 12:01:07,368 - distributed.scheduler - INFO - Receive client connection: Client-worker-c02277d9-d03f-11f0-884a-f23d8172cb06

2025-12-03 12:00:53,201 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.233:41877

2025-12-03 12:00:53,200 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.233:41877', status: closed, memory: 0, processing: 0>

2025-12-03 12:00:52,453 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:38881 failed: Exception: KeyError('9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,441 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:38881 failed: Exception: KeyError('accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,440 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:45521 failed: Exception: KeyError('9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,435 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:37695 failed: Exception: KeyError('9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,435 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:34943 failed: Exception: KeyError('9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,433 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:38667 failed: Exception: KeyError('9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,429 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:38667 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,429 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: Exception: KeyError('9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,427 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:46391 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,421 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:40659 failed: Exception: KeyError('9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,403 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:39713 failed: Exception: KeyError('9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,402 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: KeyError('9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,398 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:34943 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,397 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:37695 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,391 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,381 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,372 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,371 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:38247 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,370 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:40659 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,368 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle 9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,364 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:46391 failed: Exception: KeyError('9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,364 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:45521 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,363 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:38247 failed: Exception: KeyError('9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,352 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:39713 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,348 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:34871 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,343 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:40609 failed: Exception: KeyError('9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,342 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle 9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,338 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:34871 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle 9819c38570c066f4b0159139618f5ca9')

2025-12-03 12:00:52,331 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:40609 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,320 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: RuntimeError('Worker tcp://172.21.159.233:46261 left during active shuffle accfd2c78b98ab833598e5f444758bf0')

2025-12-03 12:00:52,293 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: OSError: Timed out during handshake while connecting to tcp://172.21.159.233:46261 after 30 s

2025-12-03 12:00:52,289 - distributed.scheduler - INFO - Close client connection: Client-worker-9812868e-cb6f-11f0-858d-f23d8172cb06

2025-12-03 12:00:52,280 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.233:46261', status: closed, memory: 0, processing: 0>

2025-12-03 12:00:52,276 - distributed.scheduler - INFO - Remove client Client-worker-9812868e-cb6f-11f0-858d-f23d8172cb06

2025-12-03 12:00:18,544 - distributed.scheduler - INFO - Receive client connection: Client-a306afaa-d03f-11f0-8001-f2cd832f225c

2025-12-03 11:51:05,766 - distributed.scheduler - INFO - Close client connection: Client-3c8111a4-d03e-11f0-8001-f2cd832f225c

2025-12-03 11:51:05,765 - distributed.scheduler - INFO - Remove client Client-3c8111a4-d03e-11f0-8001-f2cd832f225c

2025-12-03 11:51:05,764 - distributed.scheduler - INFO - Remove client Client-3c8111a4-d03e-11f0-8001-f2cd832f225c

2025-12-03 11:50:17,039 - distributed.scheduler - INFO - Receive client connection: Client-3c8111a4-d03e-11f0-8001-f2cd832f225c

2025-12-03 11:40:57,762 - distributed.scheduler - INFO - Close client connection: Client-d5fcded1-d03c-11f0-8001-f2cd832f225c

2025-12-03 11:40:57,760 - distributed.scheduler - INFO - Remove client Client-d5fcded1-d03c-11f0-8001-f2cd832f225c

2025-12-03 11:40:57,759 - distributed.scheduler - INFO - Remove client Client-d5fcded1-d03c-11f0-8001-f2cd832f225c

2025-12-03 11:40:15,551 - distributed.scheduler - INFO - Receive client connection: Client-d5fcded1-d03c-11f0-8001-f2cd832f225c

2025-12-03 11:31:30,950 - distributed.scheduler - INFO - Close client connection: Client-6f70ab7c-d03b-11f0-8001-f2cd832f225c

2025-12-03 11:31:30,949 - distributed.scheduler - INFO - Remove client Client-6f70ab7c-d03b-11f0-8001-f2cd832f225c

2025-12-03 11:31:30,948 - distributed.scheduler - INFO - Remove client Client-6f70ab7c-d03b-11f0-8001-f2cd832f225c

2025-12-03 11:30:14,011 - distributed.scheduler - INFO - Receive client connection: Client-6f70ab7c-d03b-11f0-8001-f2cd832f225c

2025-12-03 11:20:55,804 - distributed.scheduler - INFO - Close client connection: Client-08edbec6-d03a-11f0-8001-f2cd832f225c

2025-12-03 11:20:55,803 - distributed.scheduler - INFO - Remove client Client-08edbec6-d03a-11f0-8001-f2cd832f225c

2025-12-03 11:20:55,801 - distributed.scheduler - INFO - Remove client Client-08edbec6-d03a-11f0-8001-f2cd832f225c

2025-12-03 11:20:12,523 - distributed.scheduler - INFO - Receive client connection: Client-08edbec6-d03a-11f0-8001-f2cd832f225c

2025-12-03 11:10:52,687 - distributed.scheduler - INFO - Close client connection: Client-a2766bd8-d038-11f0-8001-f2cd832f225c

2025-12-03 11:10:52,685 - distributed.scheduler - INFO - Remove client Client-a2766bd8-d038-11f0-8001-f2cd832f225c

2025-12-03 11:10:52,683 - distributed.scheduler - INFO - Remove client Client-a2766bd8-d038-11f0-8001-f2cd832f225c

2025-12-03 11:10:11,125 - distributed.scheduler - INFO - Receive client connection: Client-a2766bd8-d038-11f0-8001-f2cd832f225c

2025-12-03 11:01:24,018 - distributed.scheduler - INFO - Close client connection: Client-3bf601d5-d037-11f0-8001-f2cd832f225c

2025-12-03 11:01:24,016 - distributed.scheduler - INFO - Remove client Client-3bf601d5-d037-11f0-8001-f2cd832f225c

2025-12-03 11:01:24,014 - distributed.scheduler - INFO - Remove client Client-3bf601d5-d037-11f0-8001-f2cd832f225c

2025-12-03 11:00:09,654 - distributed.scheduler - INFO - Receive client connection: Client-3bf601d5-d037-11f0-8001-f2cd832f225c

2025-12-03 10:50:51,042 - distributed.scheduler - INFO - Close client connection: Client-d56d95ef-d035-11f0-8001-f2cd832f225c

2025-12-03 10:50:51,041 - distributed.scheduler - INFO - Remove client Client-d56d95ef-d035-11f0-8001-f2cd832f225c

2025-12-03 10:50:51,039 - distributed.scheduler - INFO - Remove client Client-d56d95ef-d035-11f0-8001-f2cd832f225c

2025-12-03 10:50:08,124 - distributed.scheduler - INFO - Receive client connection: Client-d56d95ef-d035-11f0-8001-f2cd832f225c

2025-12-03 10:40:53,486 - distributed.scheduler - INFO - Close client connection: Client-6ee4f697-d034-11f0-8001-f2cd832f225c

2025-12-03 10:40:53,483 - distributed.scheduler - INFO - Remove client Client-6ee4f697-d034-11f0-8001-f2cd832f225c

2025-12-03 10:40:53,481 - distributed.scheduler - INFO - Remove client Client-6ee4f697-d034-11f0-8001-f2cd832f225c

2025-12-03 10:40:06,611 - distributed.scheduler - INFO - Receive client connection: Client-6ee4f697-d034-11f0-8001-f2cd832f225c

2025-12-03 10:31:20,221 - distributed.scheduler - INFO - Close client connection: Client-0860ea48-d033-11f0-8001-f2cd832f225c

2025-12-03 10:31:20,220 - distributed.scheduler - INFO - Remove client Client-0860ea48-d033-11f0-8001-f2cd832f225c

2025-12-03 10:31:20,218 - distributed.scheduler - INFO - Remove client Client-0860ea48-d033-11f0-8001-f2cd832f225c

2025-12-03 10:30:05,119 - distributed.scheduler - INFO - Receive client connection: Client-0860ea48-d033-11f0-8001-f2cd832f225c

2025-12-03 10:20:42,013 - distributed.scheduler - INFO - Close client connection: Client-a1e577e2-d031-11f0-8001-f2cd832f225c

2025-12-03 10:20:42,011 - distributed.scheduler - INFO - Remove client Client-a1e577e2-d031-11f0-8001-f2cd832f225c

2025-12-03 10:20:42,009 - distributed.scheduler - INFO - Remove client Client-a1e577e2-d031-11f0-8001-f2cd832f225c

2025-12-03 10:20:03,692 - distributed.scheduler - INFO - Receive client connection: Client-a1e577e2-d031-11f0-8001-f2cd832f225c

2025-12-03 10:10:37,803 - distributed.scheduler - INFO - Close client connection: Client-3b61f959-d030-11f0-8001-f2cd832f225c

2025-12-03 10:10:37,802 - distributed.scheduler - INFO - Remove client Client-3b61f959-d030-11f0-8001-f2cd832f225c

2025-12-03 10:10:37,801 - distributed.scheduler - INFO - Remove client Client-3b61f959-d030-11f0-8001-f2cd832f225c

2025-12-03 10:10:02,215 - distributed.scheduler - INFO - Receive client connection: Client-3b61f959-d030-11f0-8001-f2cd832f225c

2025-12-03 10:01:15,863 - distributed.scheduler - INFO - Close client connection: Client-d4e53f9f-d02e-11f0-8001-f2cd832f225c

2025-12-03 10:01:15,861 - distributed.scheduler - INFO - Remove client Client-d4e53f9f-d02e-11f0-8001-f2cd832f225c

2025-12-03 10:01:15,861 - distributed.scheduler - INFO - Remove client Client-d4e53f9f-d02e-11f0-8001-f2cd832f225c

2025-12-03 10:00:00,754 - distributed.scheduler - INFO - Receive client connection: Client-d4e53f9f-d02e-11f0-8001-f2cd832f225c

2025-12-03 09:51:34,149 - distributed.scheduler - INFO - Close client connection: Client-9233c61a-d02d-11f0-8001-f2cd832f225c

2025-12-03 09:51:34,148 - distributed.scheduler - INFO - Remove client Client-9233c61a-d02d-11f0-8001-f2cd832f225c

2025-12-03 09:51:34,147 - distributed.scheduler - INFO - Remove client Client-9233c61a-d02d-11f0-8001-f2cd832f225c

2025-12-03 09:50:59,379 - distributed.scheduler - INFO - Receive client connection: Client-9233c61a-d02d-11f0-8001-f2cd832f225c

2025-12-03 09:41:31,792 - distributed.scheduler - INFO - Close client connection: Client-2bbb33ea-d02c-11f0-8001-f2cd832f225c

2025-12-03 09:41:31,789 - distributed.scheduler - INFO - Remove client Client-2bbb33ea-d02c-11f0-8001-f2cd832f225c

2025-12-03 09:41:31,788 - distributed.scheduler - INFO - Remove client Client-2bbb33ea-d02c-11f0-8001-f2cd832f225c

2025-12-03 09:40:57,963 - distributed.scheduler - INFO - Receive client connection: Client-2bbb33ea-d02c-11f0-8001-f2cd832f225c

2025-12-03 09:32:04,897 - distributed.scheduler - INFO - Close client connection: Client-5ed8d550-d029-11f0-8001-f2cd832f225c

2025-12-03 09:32:04,896 - distributed.scheduler - INFO - Remove client Client-5ed8d550-d029-11f0-8001-f2cd832f225c

2025-12-03 09:32:04,016 - distributed.scheduler - INFO - Close client connection: Client-c552376f-d02a-11f0-8001-f2cd832f225c

2025-12-03 09:32:04,013 - distributed.scheduler - INFO - Remove client Client-c552376f-d02a-11f0-8001-f2cd832f225c

2025-12-03 09:32:04,012 - distributed.scheduler - INFO - Remove client Client-c552376f-d02a-11f0-8001-f2cd832f225c

2025-12-03 09:31:02,736 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('len-agg-7fa986b5dae63aa07b16d0b40a025318', 0) NoneType: None

2025-12-03 09:31:02,736 - distributed.scheduler - ERROR - Couldn't gather keys {"('len-agg-7fa986b5dae63aa07b16d0b40a025318', 0)": []} state: ['waiting'] workers: [] NoneType: None

2025-12-03 09:30:56,639 - distributed.scheduler - INFO - Receive client connection: Client-c552376f-d02a-11f0-8001-f2cd832f225c

2025-12-03 09:20:55,225 - distributed.scheduler - INFO - Receive client connection: Client-5ed8d550-d029-11f0-8001-f2cd832f225c

2025-12-03 09:11:33,415 - distributed.scheduler - INFO - Close client connection: Client-f85e989d-d027-11f0-8001-f2cd832f225c

2025-12-03 09:11:33,414 - distributed.scheduler - INFO - Remove client Client-f85e989d-d027-11f0-8001-f2cd832f225c

2025-12-03 09:11:33,414 - distributed.scheduler - INFO - Remove client Client-f85e989d-d027-11f0-8001-f2cd832f225c

2025-12-03 09:10:53,795 - distributed.scheduler - INFO - Receive client connection: Client-f85e989d-d027-11f0-8001-f2cd832f225c

2025-12-03 09:02:04,107 - distributed.scheduler - INFO - Close client connection: Client-91c9b02e-d026-11f0-8001-f2cd832f225c

2025-12-03 09:02:04,105 - distributed.scheduler - INFO - Remove client Client-91c9b02e-d026-11f0-8001-f2cd832f225c

2025-12-03 09:02:04,103 - distributed.scheduler - INFO - Remove client Client-91c9b02e-d026-11f0-8001-f2cd832f225c

2025-12-03 09:00:52,204 - distributed.scheduler - INFO - Receive client connection: Client-91c9b02e-d026-11f0-8001-f2cd832f225c

2025-12-03 08:51:28,352 - distributed.scheduler - INFO - Close client connection: Client-2b4b3fd1-d025-11f0-8001-f2cd832f225c

2025-12-03 08:51:28,350 - distributed.scheduler - INFO - Remove client Client-2b4b3fd1-d025-11f0-8001-f2cd832f225c

2025-12-03 08:51:28,349 - distributed.scheduler - INFO - Remove client Client-2b4b3fd1-d025-11f0-8001-f2cd832f225c

2025-12-03 08:50:50,739 - distributed.scheduler - INFO - Receive client connection: Client-2b4b3fd1-d025-11f0-8001-f2cd832f225c

2025-12-03 08:41:29,372 - distributed.scheduler - INFO - Close client connection: Client-c4c7a9c0-d023-11f0-8001-f2cd832f225c

2025-12-03 08:41:29,371 - distributed.scheduler - INFO - Remove client Client-c4c7a9c0-d023-11f0-8001-f2cd832f225c

2025-12-03 08:41:29,370 - distributed.scheduler - INFO - Remove client Client-c4c7a9c0-d023-11f0-8001-f2cd832f225c

2025-12-03 08:40:49,261 - distributed.scheduler - INFO - Receive client connection: Client-c4c7a9c0-d023-11f0-8001-f2cd832f225c

2025-12-03 08:31:59,638 - distributed.scheduler - INFO - Close client connection: Client-5e464d56-d022-11f0-8001-f2cd832f225c

2025-12-03 08:31:59,637 - distributed.scheduler - INFO - Remove client Client-5e464d56-d022-11f0-8001-f2cd832f225c

2025-12-03 08:31:59,635 - distributed.scheduler - INFO - Remove client Client-5e464d56-d022-11f0-8001-f2cd832f225c

2025-12-03 08:30:53,825 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('len-agg-62f4875dd3790e08bbff0e3bc5a04806', 0) NoneType: None

2025-12-03 08:30:53,825 - distributed.scheduler - ERROR - Couldn't gather keys {"('len-agg-62f4875dd3790e08bbff0e3bc5a04806', 0)": []} state: ['waiting'] workers: [] NoneType: None

2025-12-03 08:30:52,000 - distributed.scheduler - INFO - Receive client connection: Client-worker-60cb4b5c-d022-11f0-87c6-32001d5d2425

2025-12-03 08:30:47,791 - distributed.scheduler - INFO - Receive client connection: Client-5e464d56-d022-11f0-8001-f2cd832f225c

2025-12-03 08:21:27,836 - distributed.scheduler - INFO - Close client connection: Client-913a0f82-d01f-11f0-8001-f2cd832f225c

2025-12-03 08:21:27,832 - distributed.scheduler - INFO - Remove client Client-913a0f82-d01f-11f0-8001-f2cd832f225c

2025-12-03 08:21:27,006 - distributed.scheduler - INFO - Close client connection: Client-f7be5a8f-d020-11f0-8001-f2cd832f225c

2025-12-03 08:21:27,004 - distributed.scheduler - INFO - Remove client Client-f7be5a8f-d020-11f0-8001-f2cd832f225c

2025-12-03 08:21:27,003 - distributed.scheduler - INFO - Remove client Client-f7be5a8f-d020-11f0-8001-f2cd832f225c

2025-12-03 08:20:46,267 - distributed.scheduler - INFO - Receive client connection: Client-f7be5a8f-d020-11f0-8001-f2cd832f225c

2025-12-03 08:10:44,787 - distributed.scheduler - INFO - Receive client connection: Client-913a0f82-d01f-11f0-8001-f2cd832f225c

2025-12-03 08:01:57,446 - distributed.scheduler - INFO - Close client connection: Client-2ab097ab-d01e-11f0-8001-f2cd832f225c

2025-12-03 08:01:57,445 - distributed.scheduler - INFO - Remove client Client-2ab097ab-d01e-11f0-8001-f2cd832f225c

2025-12-03 08:01:57,444 - distributed.scheduler - INFO - Remove client Client-2ab097ab-d01e-11f0-8001-f2cd832f225c

2025-12-03 08:00:47,527 - distributed.scheduler - INFO - Receive client connection: Client-worker-2d3e3fd3-d01e-11f0-86e0-2a2d08e6be2e

2025-12-03 08:00:43,246 - distributed.scheduler - INFO - Receive client connection: Client-2ab097ab-d01e-11f0-8001-f2cd832f225c

2025-12-03 07:51:27,160 - distributed.scheduler - INFO - Close client connection: Client-c434b837-d01c-11f0-8001-f2cd832f225c

2025-12-03 07:51:27,158 - distributed.scheduler - INFO - Remove client Client-c434b837-d01c-11f0-8001-f2cd832f225c

2025-12-03 07:51:27,157 - distributed.scheduler - INFO - Remove client Client-c434b837-d01c-11f0-8001-f2cd832f225c

2025-12-03 07:50:41,820 - distributed.scheduler - INFO - Receive client connection: Client-c434b837-d01c-11f0-8001-f2cd832f225c

2025-12-03 07:41:23,080 - distributed.scheduler - INFO - Close client connection: Client-5dadfe0a-d01b-11f0-8001-f2cd832f225c

2025-12-03 07:41:23,079 - distributed.scheduler - INFO - Remove client Client-5dadfe0a-d01b-11f0-8001-f2cd832f225c

2025-12-03 07:41:23,078 - distributed.scheduler - INFO - Remove client Client-5dadfe0a-d01b-11f0-8001-f2cd832f225c

2025-12-03 07:40:40,310 - distributed.scheduler - INFO - Receive client connection: Client-5dadfe0a-d01b-11f0-8001-f2cd832f225c

2025-12-03 07:34:58,395 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:38821 failed: OSError: Timed out trying to connect to tcp://172.21.25.102:38821 after 30 s

2025-12-03 07:34:58,395 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:38667 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:38821 after 30 s')

2025-12-03 07:34:58,301 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:37695 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:38821 after 30 s')

2025-12-03 07:34:58,289 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:38821 after 30 s')

2025-12-03 07:34:58,289 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:39713 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:38821 after 30 s')

2025-12-03 07:34:58,287 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:34871 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:38821 after 30 s')

2025-12-03 07:34:58,282 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:45521 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:38821 after 30 s')

2025-12-03 07:34:58,281 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:38821 after 30 s')

2025-12-03 07:34:58,281 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:40659 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:38821 after 30 s')

2025-12-03 07:34:58,281 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:38821 after 30 s')

2025-12-03 07:34:58,259 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:38821 after 30 s')

2025-12-03 07:34:58,252 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:34943 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:38821 after 30 s')

2025-12-03 07:34:58,248 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:46391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:38821 after 30 s')

2025-12-03 07:34:58,243 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:38247 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:38821 after 30 s')

2025-12-03 07:34:29,081 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.102:38881

2025-12-03 07:34:29,079 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.102:38881', status: closed, memory: 0, processing: 0>

2025-12-03 07:34:28,145 - distributed.scheduler - INFO - Close client connection: Client-worker-766f0031-d01a-11f0-87b2-32001d5d2425

2025-12-03 07:34:28,123 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.102:38821', status: closed, memory: 0, processing: 0>

2025-12-03 07:34:28,122 - distributed.scheduler - INFO - Remove client Client-worker-766f0031-d01a-11f0-87b2-32001d5d2425

2025-12-03 07:34:12,332 - distributed.scheduler - INFO - Receive client connection: Client-worker-766f0031-d01a-11f0-87b2-32001d5d2425

2025-12-03 07:34:08,016 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:44961 failed: OSError: Timed out trying to connect to tcp://172.21.25.102:44961 after 30 s

2025-12-03 07:34:07,960 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:38667 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:44961 after 30 s')

2025-12-03 07:34:07,953 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:44961 after 30 s')

2025-12-03 07:34:07,952 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:44961 after 30 s')

2025-12-03 07:34:07,922 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:45521 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:44961 after 30 s')

2025-12-03 07:34:07,911 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:44961 after 30 s')

2025-12-03 07:34:07,908 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:44961 after 30 s')

2025-12-03 07:34:07,886 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:44961 after 30 s')

2025-12-03 07:33:38,748 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.102:38821

2025-12-03 07:33:38,746 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.102:38821', status: closed, memory: 0, processing: 0>

2025-12-03 07:33:37,785 - distributed.scheduler - INFO - Close client connection: Client-worker-52dfecac-d01a-11f0-879b-32001d5d2425

2025-12-03 07:33:37,772 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.102:44961', status: closed, memory: 0, processing: 0>

2025-12-03 07:33:37,772 - distributed.scheduler - INFO - Remove client Client-worker-52dfecac-d01a-11f0-879b-32001d5d2425

2025-12-03 07:33:12,680 - distributed.scheduler - INFO - Receive client connection: Client-worker-52dfecac-d01a-11f0-879b-32001d5d2425

2025-12-03 07:33:08,285 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:39713 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:36113 after 30 s')

2025-12-03 07:33:08,284 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:36113 failed: OSError: Timed out trying to connect to tcp://172.21.25.102:36113 after 30 s

2025-12-03 07:33:08,243 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:38667 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:36113 after 30 s')

2025-12-03 07:33:08,199 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:40609 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:36113 after 30 s')

2025-12-03 07:33:08,195 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:36113 after 30 s')

2025-12-03 07:33:08,190 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:34943 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:36113 after 30 s')

2025-12-03 07:33:08,177 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:40659 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:36113 after 30 s')

2025-12-03 07:32:39,078 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.102:44961

2025-12-03 07:32:39,076 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.102:44961', status: closed, memory: 0, processing: 0>

2025-12-03 07:32:38,041 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.102:36113', status: closed, memory: 0, processing: 0>

2025-12-03 07:32:30,981 - distributed.scheduler - INFO - Close client connection: Client-f7368b29-d019-11f0-8001-f2cd832f225c

2025-12-03 07:32:30,980 - distributed.scheduler - INFO - Remove client Client-f7368b29-d019-11f0-8001-f2cd832f225c

2025-12-03 07:32:30,979 - distributed.scheduler - INFO - Remove client Client-f7368b29-d019-11f0-8001-f2cd832f225c

2025-12-03 07:32:24,530 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.102:36113

2025-12-03 07:32:24,528 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.102:36113', status: closed, memory: 0, processing: 0>

2025-12-03 07:32:23,352 - distributed.scheduler - INFO - Close client connection: Client-worker-0f1e99a7-d01a-11f0-8746-32001d5d2425

2025-12-03 07:32:23,347 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.102:36683', status: closed, memory: 0, processing: 0>

2025-12-03 07:32:23,334 - distributed.scheduler - INFO - Remove client Client-worker-0f1e99a7-d01a-11f0-8746-32001d5d2425

2025-12-03 07:31:51,453 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:40569 failed: OSError: Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s

2025-12-03 07:31:51,442 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:40569 failed: OSError: Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s

2025-12-03 07:31:50,954 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s')

2025-12-03 07:31:50,574 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:34871 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s')

2025-12-03 07:31:50,433 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:36683 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s')

2025-12-03 07:31:50,194 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:40569 failed: OSError: Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s

2025-12-03 07:31:50,115 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:36683 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s')

2025-12-03 07:31:50,100 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:34943 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s')

2025-12-03 07:31:50,051 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:34871 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s')

2025-12-03 07:31:49,424 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:34943 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s')

2025-12-03 07:31:49,382 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:40659 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s')

2025-12-03 07:31:47,326 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:45521 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s')

2025-12-03 07:31:47,033 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:45521 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s')

2025-12-03 07:31:46,725 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s')

2025-12-03 07:31:46,370 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:40569 after 30 s')

2025-12-03 07:31:19,098 - distributed.scheduler - INFO - Receive client connection: Client-worker-0f1e99a7-d01a-11f0-8746-32001d5d2425

2025-12-03 07:31:19,045 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1236) NoneType: None

2025-12-03 07:31:19,045 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 62) NoneType: None

2025-12-03 07:31:19,045 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1814) NoneType: None

2025-12-03 07:31:19,045 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1244) NoneType: None

2025-12-03 07:31:19,045 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 138) NoneType: None

2025-12-03 07:31:19,045 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 51) NoneType: None

2025-12-03 07:31:19,045 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 586) NoneType: None

2025-12-03 07:31:19,045 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1427) NoneType: None

2025-12-03 07:31:19,045 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 141) NoneType: None

2025-12-03 07:31:19,045 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 2067) NoneType: None

2025-12-03 07:31:19,044 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 128) NoneType: None

2025-12-03 07:31:19,044 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1354) NoneType: None

2025-12-03 07:31:19,044 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 651) NoneType: None

2025-12-03 07:31:19,044 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 130) NoneType: None

2025-12-03 07:31:19,044 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 2078) NoneType: None

2025-12-03 07:31:19,044 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 2285) NoneType: None

2025-12-03 07:31:19,044 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1811) NoneType: None

2025-12-03 07:31:19,044 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 71) NoneType: None

2025-12-03 07:31:19,044 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 129) NoneType: None

2025-12-03 07:31:19,044 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1823) NoneType: None

2025-12-03 07:31:19,044 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1130) NoneType: None

2025-12-03 07:31:19,043 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 133) NoneType: None

2025-12-03 07:31:19,043 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1817) NoneType: None

2025-12-03 07:31:19,043 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 626) NoneType: None

2025-12-03 07:31:19,043 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 2043) NoneType: None

2025-12-03 07:31:19,043 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1234) NoneType: None

2025-12-03 07:31:19,043 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1819) NoneType: None

2025-12-03 07:31:19,043 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 337) NoneType: None

2025-12-03 07:31:19,043 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 136) NoneType: None

2025-12-03 07:31:19,043 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 636) NoneType: None

2025-12-03 07:31:19,043 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1816) NoneType: None

2025-12-03 07:31:19,043 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 425) NoneType: None

2025-12-03 07:31:19,042 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 593) NoneType: None

2025-12-03 07:31:19,042 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 142) NoneType: None

2025-12-03 07:31:19,042 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 140) NoneType: None

2025-12-03 07:31:19,042 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 343) NoneType: None

2025-12-03 07:31:19,042 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1971) NoneType: None

2025-12-03 07:31:19,042 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 137) NoneType: None

2025-12-03 07:31:19,042 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1495) NoneType: None

2025-12-03 07:31:19,042 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 25) NoneType: None

2025-12-03 07:31:19,042 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1809) NoneType: None

2025-12-03 07:31:19,042 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1243) NoneType: None

2025-12-03 07:31:19,042 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1213) NoneType: None

2025-12-03 07:31:19,041 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 134) NoneType: None

2025-12-03 07:31:19,041 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 132) NoneType: None

2025-12-03 07:31:19,041 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1810) NoneType: None

2025-12-03 07:31:19,041 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1931) NoneType: None

2025-12-03 07:31:19,041 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1340) NoneType: None

2025-12-03 07:31:19,041 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 666) NoneType: None

2025-12-03 07:31:19,041 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1287) NoneType: None

2025-12-03 07:31:19,041 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1722) NoneType: None

2025-12-03 07:31:19,041 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1938) NoneType: None

2025-12-03 07:31:19,040 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 256) NoneType: None

2025-12-03 07:31:19,040 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1921) NoneType: None

2025-12-03 07:31:19,040 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 454) NoneType: None

2025-12-03 07:31:19,040 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1399) NoneType: None

2025-12-03 07:31:19,040 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 789) NoneType: None

2025-12-03 07:31:19,040 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 143) NoneType: None

2025-12-03 07:31:19,039 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 135) NoneType: None

2025-12-03 07:31:19,039 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 1925) NoneType: None

2025-12-03 07:31:19,039 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 131) NoneType: None

2025-12-03 07:31:19,038 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-0755708212d6ed34bae97e12d6ba5b7a', 997) NoneType: None

2025-12-03 07:31:19,038 - distributed.scheduler - ERROR - Couldn't gather keys {"('query-0755708212d6ed34bae97e12d6ba5b7a', 997)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 131)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1925)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 135)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 143)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 789)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1399)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 454)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1921)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 256)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1938)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1722)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1287)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 666)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1340)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1931)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1810)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 132)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 134)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1213)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1243)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1809)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 25)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1495)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 137)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1971)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 343)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 140)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 142)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 593)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 425)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1816)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 636)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 136)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 337)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1819)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1234)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 2043)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 626)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1817)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 133)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1130)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1823)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 129)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 71)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1811)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 2285)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 2078)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 130)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 651)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1354)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 128)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 2067)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 141)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1427)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 586)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 51)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 138)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1244)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1814)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 62)": [], "('query-0755708212d6ed34bae97e12d6ba5b7a', 1236)": []} state: ['waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting'] workers: [] NoneType: None

2025-12-03 07:31:17,101 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.102:36683

2025-12-03 07:31:17,100 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.102:36683', status: closed, memory: 0, processing: 0>

2025-12-03 07:31:16,044 - distributed.scheduler - INFO - Close client connection: Client-worker-ea963dc8-cea0-11f0-85fe-32001d5d2425

2025-12-03 07:31:16,012 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.102:40569', status: closed, memory: 0, processing: 0>

2025-12-03 07:31:15,998 - distributed.scheduler - INFO - Remove client Client-worker-ea963dc8-cea0-11f0-85fe-32001d5d2425

2025-12-03 07:30:38,911 - distributed.scheduler - INFO - Receive client connection: Client-f7368b29-d019-11f0-8001-f2cd832f225c

2025-12-03 07:21:23,938 - distributed.scheduler - INFO - Close client connection: Client-90ae13e2-d018-11f0-8001-f2cd832f225c

2025-12-03 07:21:23,936 - distributed.scheduler - INFO - Remove client Client-90ae13e2-d018-11f0-8001-f2cd832f225c

2025-12-03 07:21:23,935 - distributed.scheduler - INFO - Remove client Client-90ae13e2-d018-11f0-8001-f2cd832f225c

2025-12-03 07:20:37,387 - distributed.scheduler - INFO - Receive client connection: Client-90ae13e2-d018-11f0-8001-f2cd832f225c

2025-12-03 07:11:19,268 - distributed.scheduler - INFO - Close client connection: Client-2a2aa0eb-d017-11f0-8001-f2cd832f225c

2025-12-03 07:11:19,267 - distributed.scheduler - INFO - Remove client Client-2a2aa0eb-d017-11f0-8001-f2cd832f225c

2025-12-03 07:11:19,266 - distributed.scheduler - INFO - Remove client Client-2a2aa0eb-d017-11f0-8001-f2cd832f225c

2025-12-03 07:10:35,899 - distributed.scheduler - INFO - Receive client connection: Client-2a2aa0eb-d017-11f0-8001-f2cd832f225c

2025-12-03 07:03:08,110 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:39713

2025-12-03 07:03:08,109 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:39713', status: running, memory: 0, processing: 0>

2025-12-03 07:03:07,219 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:39093', status: closed, memory: 0, processing: 0>

2025-12-03 07:02:55,793 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:39093

2025-12-03 07:02:55,791 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:39093', status: closed, memory: 0, processing: 0>

2025-12-03 07:02:54,825 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:46413', status: closed, memory: 0, processing: 0>

2025-12-03 07:02:42,422 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:36775 failed: OSError: Timed out trying to connect to tcp://172.21.12.199:36775 after 30 s

2025-12-03 07:02:42,408 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:38667 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:36775 after 30 s')

2025-12-03 07:02:42,383 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:38247 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:36775 after 30 s')

2025-12-03 07:02:42,329 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:36775 after 30 s')

2025-12-03 07:02:13,270 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:46413

2025-12-03 07:02:13,269 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:46413', status: closed, memory: 0, processing: 0>

2025-12-03 07:02:12,225 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:36775', status: closed, memory: 0, processing: 0>

2025-12-03 07:02:08,672 - distributed.scheduler - INFO - Close client connection: Client-c3ad09fd-d015-11f0-8001-f2cd832f225c

2025-12-03 07:02:08,672 - distributed.scheduler - INFO - Remove client Client-c3ad09fd-d015-11f0-8001-f2cd832f225c

2025-12-03 07:02:08,670 - distributed.scheduler - INFO - Remove client Client-c3ad09fd-d015-11f0-8001-f2cd832f225c

2025-12-03 07:01:55,556 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:36775

2025-12-03 07:01:55,554 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:36775', status: closed, memory: 0, processing: 0>

2025-12-03 07:01:54,507 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:41897', status: closed, memory: 0, processing: 0>

2025-12-03 07:01:17,528 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:41897

2025-12-03 07:01:17,527 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:41897', status: closed, memory: 0, processing: 0>

2025-12-03 07:01:17,240 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], run-9ab15cdb67e683d8a767621bf1be38e5 NoneType: None

2025-12-03 07:01:17,240 - distributed.scheduler - ERROR - Couldn't gather keys {'run-9ab15cdb67e683d8a767621bf1be38e5': []} state: ['processing'] workers: [] NoneType: None

2025-12-03 07:01:16,606 - distributed.scheduler - INFO - Close client connection: Client-worker-ec77a45a-ceb9-11f0-85d6-2a2d08e6be2e

2025-12-03 07:01:16,597 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:44619', status: closed, memory: 0, processing: 0>

2025-12-03 07:01:16,594 - distributed.scheduler - INFO - Remove client Client-worker-ec77a45a-ceb9-11f0-85d6-2a2d08e6be2e

2025-12-03 07:00:40,428 - distributed.scheduler - INFO - Receive client connection: Client-worker-c7340ade-d015-11f0-89b3-8253c573df33

2025-12-03 07:00:39,559 - distributed.scheduler - INFO - Receive client connection: Client-worker-c6b8479d-d015-11f0-86c1-46fbd1744967

2025-12-03 07:00:34,454 - distributed.scheduler - INFO - Receive client connection: Client-c3ad09fd-d015-11f0-8001-f2cd832f225c

2025-12-03 06:51:14,784 - distributed.scheduler - INFO - Close client connection: Client-5d358e48-d014-11f0-8001-f2cd832f225c

2025-12-03 06:51:14,783 - distributed.scheduler - INFO - Remove client Client-5d358e48-d014-11f0-8001-f2cd832f225c

2025-12-03 06:51:14,781 - distributed.scheduler - INFO - Remove client Client-5d358e48-d014-11f0-8001-f2cd832f225c

2025-12-03 06:50:33,049 - distributed.scheduler - INFO - Receive client connection: Client-5d358e48-d014-11f0-8001-f2cd832f225c

2025-12-03 06:41:14,902 - distributed.scheduler - INFO - Close client connection: Client-f6af5688-d012-11f0-8001-f2cd832f225c

2025-12-03 06:41:14,901 - distributed.scheduler - INFO - Remove client Client-f6af5688-d012-11f0-8001-f2cd832f225c

2025-12-03 06:41:14,900 - distributed.scheduler - INFO - Remove client Client-f6af5688-d012-11f0-8001-f2cd832f225c

2025-12-03 06:40:31,544 - distributed.scheduler - INFO - Receive client connection: Client-f6af5688-d012-11f0-8001-f2cd832f225c

2025-12-03 06:31:45,588 - distributed.scheduler - INFO - Close client connection: Client-9039f32e-d011-11f0-8001-f2cd832f225c

2025-12-03 06:31:45,587 - distributed.scheduler - INFO - Remove client Client-9039f32e-d011-11f0-8001-f2cd832f225c

2025-12-03 06:31:45,585 - distributed.scheduler - INFO - Remove client Client-9039f32e-d011-11f0-8001-f2cd832f225c

2025-12-03 06:30:30,154 - distributed.scheduler - INFO - Receive client connection: Client-9039f32e-d011-11f0-8001-f2cd832f225c

2025-12-03 06:21:12,306 - distributed.scheduler - INFO - Close client connection: Client-29c2a083-d010-11f0-8001-f2cd832f225c

2025-12-03 06:21:12,305 - distributed.scheduler - INFO - Remove client Client-29c2a083-d010-11f0-8001-f2cd832f225c

2025-12-03 06:21:12,303 - distributed.scheduler - INFO - Remove client Client-29c2a083-d010-11f0-8001-f2cd832f225c

2025-12-03 06:20:28,743 - distributed.scheduler - INFO - Receive client connection: Client-29c2a083-d010-11f0-8001-f2cd832f225c

2025-12-03 06:11:11,554 - distributed.scheduler - INFO - Close client connection: Client-c34fbfe3-d00e-11f0-8001-f2cd832f225c

2025-12-03 06:11:11,553 - distributed.scheduler - INFO - Remove client Client-c34fbfe3-d00e-11f0-8001-f2cd832f225c

2025-12-03 06:11:11,552 - distributed.scheduler - INFO - Remove client Client-c34fbfe3-d00e-11f0-8001-f2cd832f225c

2025-12-03 06:10:27,368 - distributed.scheduler - INFO - Receive client connection: Client-c34fbfe3-d00e-11f0-8001-f2cd832f225c

2025-12-03 06:01:42,318 - distributed.scheduler - INFO - Close client connection: Client-5ce52574-d00d-11f0-8001-f2cd832f225c

2025-12-03 06:01:42,316 - distributed.scheduler - INFO - Remove client Client-5ce52574-d00d-11f0-8001-f2cd832f225c

2025-12-03 06:01:42,314 - distributed.scheduler - INFO - Remove client Client-5ce52574-d00d-11f0-8001-f2cd832f225c

2025-12-03 06:00:32,009 - distributed.scheduler - INFO - Receive client connection: Client-worker-602870b4-d00d-11f0-87c3-4ec0916b0c7e

2025-12-03 06:00:26,046 - distributed.scheduler - INFO - Receive client connection: Client-5ce52574-d00d-11f0-8001-f2cd832f225c

2025-12-03 05:51:07,737 - distributed.scheduler - INFO - Close client connection: Client-f6607a2b-d00b-11f0-8001-f2cd832f225c

2025-12-03 05:51:07,736 - distributed.scheduler - INFO - Remove client Client-f6607a2b-d00b-11f0-8001-f2cd832f225c

2025-12-03 05:51:07,735 - distributed.scheduler - INFO - Remove client Client-f6607a2b-d00b-11f0-8001-f2cd832f225c

2025-12-03 05:50:24,544 - distributed.scheduler - INFO - Receive client connection: Client-f6607a2b-d00b-11f0-8001-f2cd832f225c

2025-12-03 05:41:08,413 - distributed.scheduler - INFO - Close client connection: Client-8fe3e09b-d00a-11f0-8001-f2cd832f225c

2025-12-03 05:41:08,413 - distributed.scheduler - INFO - Remove client Client-8fe3e09b-d00a-11f0-8001-f2cd832f225c

2025-12-03 05:41:08,411 - distributed.scheduler - INFO - Remove client Client-8fe3e09b-d00a-11f0-8001-f2cd832f225c

2025-12-03 05:40:23,110 - distributed.scheduler - INFO - Receive client connection: Client-8fe3e09b-d00a-11f0-8001-f2cd832f225c

2025-12-03 05:32:55,747 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:40743 failed: OSError: Timed out trying to connect to tcp://172.21.25.24:40743 after 30 s

2025-12-03 05:32:55,732 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:40609 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.24:40743 after 30 s')

2025-12-03 05:32:55,706 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44619 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.24:40743 after 30 s')

2025-12-03 05:32:55,697 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.24:40743 after 30 s')

2025-12-03 05:32:55,674 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:38667 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.24:40743 after 30 s')

2025-12-03 05:32:55,615 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.24:40743 after 30 s')

2025-12-03 05:32:55,593 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:37695 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.24:40743 after 30 s')

2025-12-03 05:32:55,588 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.24:40743 after 30 s')

2025-12-03 05:32:55,575 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:46391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.24:40743 after 30 s')

2025-12-03 05:32:55,572 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:40569 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.24:40743 after 30 s')

2025-12-03 05:32:40,711 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.248:40659

2025-12-03 05:32:40,710 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.248:40659', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:40,671 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:34943

2025-12-03 05:32:40,669 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:34943', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:39,862 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:40623', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:39,789 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.248:39905', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:39,624 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.24:45521

2025-12-03 05:32:39,622 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.24:45521', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:38,643 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.24:37771', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:27,755 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.248:39905

2025-12-03 05:32:27,753 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.248:39905', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:27,687 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:40623

2025-12-03 05:32:27,687 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:40623', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:27,555 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: KeyError('d26b6e3ecde3e0acc7af27bb1603df07')

2025-12-03 05:32:26,738 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:46678 remote=tcp://172.21.12.165:44981>: Stream is closed

2025-12-03 05:32:26,735 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.248:34257', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:26,674 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:36403', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:26,414 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.24:37771

2025-12-03 05:32:26,411 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.24:37771', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:25,433 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.24:40743', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:10,504 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.24:40743

2025-12-03 05:32:10,503 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.24:40743', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:10,146 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.248:34257

2025-12-03 05:32:10,144 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.248:34257', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:09,921 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:36403

2025-12-03 05:32:09,919 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:36403', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:09,425 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.24:42669', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:09,278 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.248:43411', status: closed, memory: 0, processing: 0>

2025-12-03 05:32:09,084 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:44969', status: closed, memory: 0, processing: 0>

2025-12-03 05:31:52,027 - distributed.scheduler - INFO - Close client connection: Client-285f467d-cffb-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,027 - distributed.scheduler - INFO - Close client connection: Client-8ee94f57-cffc-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,027 - distributed.scheduler - INFO - Close client connection: Client-f5674f05-cffd-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,027 - distributed.scheduler - INFO - Close client connection: Client-5be05e38-cfff-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,027 - distributed.scheduler - INFO - Close client connection: Client-28d56105-d002-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,027 - distributed.scheduler - INFO - Close client connection: Client-c258e80b-d000-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,027 - distributed.scheduler - INFO - Close client connection: Client-8f554480-d003-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,027 - distributed.scheduler - INFO - Close client connection: Client-f5e046fe-d004-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,027 - distributed.scheduler - INFO - Close client connection: Client-5c6c80bc-d006-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,026 - distributed.scheduler - INFO - Close client connection: Client-c2e42458-d007-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,024 - distributed.scheduler - INFO - Remove client Client-285f467d-cffb-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,024 - distributed.scheduler - INFO - Remove client Client-8ee94f57-cffc-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,023 - distributed.scheduler - INFO - Remove client Client-f5674f05-cffd-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,023 - distributed.scheduler - INFO - Remove client Client-5be05e38-cfff-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,023 - distributed.scheduler - INFO - Remove client Client-28d56105-d002-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,023 - distributed.scheduler - INFO - Remove client Client-c258e80b-d000-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,022 - distributed.scheduler - INFO - Remove client Client-8f554480-d003-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,022 - distributed.scheduler - INFO - Remove client Client-f5e046fe-d004-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,022 - distributed.scheduler - INFO - Remove client Client-5c6c80bc-d006-11f0-8001-f2cd832f225c

2025-12-03 05:31:52,021 - distributed.scheduler - INFO - Remove client Client-c2e42458-d007-11f0-8001-f2cd832f225c

2025-12-03 05:31:51,220 - distributed.scheduler - INFO - Close client connection: Client-2964a0e7-d009-11f0-8001-f2cd832f225c

2025-12-03 05:31:51,218 - distributed.scheduler - INFO - Remove client Client-2964a0e7-d009-11f0-8001-f2cd832f225c

2025-12-03 05:31:51,217 - distributed.scheduler - INFO - Remove client Client-2964a0e7-d009-11f0-8001-f2cd832f225c

2025-12-03 05:31:39,294 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.248:43411

2025-12-03 05:31:39,291 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.248:43411', status: closed, memory: 0, processing: 0>

2025-12-03 05:31:38,480 - distributed.scheduler - INFO - Close client connection: Client-worker-f871958e-d004-11f0-8745-4ec0916b0c7e

2025-12-03 05:31:38,471 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.248:35313', status: closed, memory: 0, processing: 0>

2025-12-03 05:31:38,469 - distributed.scheduler - INFO - Remove client Client-worker-f871958e-d004-11f0-8745-4ec0916b0c7e

2025-12-03 05:31:37,311 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:44969

2025-12-03 05:31:37,309 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:44969', status: closed, memory: 0, processing: 0>

2025-12-03 05:31:36,457 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:36091', status: closed, memory: 0, processing: 0>

2025-12-03 05:31:35,964 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.24:42669

2025-12-03 05:31:35,962 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.24:42669', status: closed, memory: 0, processing: 0>

2025-12-03 05:31:34,760 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.24:40191', status: closed, memory: 0, processing: 0>

2025-12-03 05:31:09,893 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.24:40191

2025-12-03 05:31:09,892 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.24:40191', status: closed, memory: 0, processing: 0>

2025-12-03 05:31:08,931 - distributed.scheduler - INFO - Close client connection: Client-worker-7b1cc80d-cae5-11f0-83f6-46fbd1744967

2025-12-03 05:31:08,930 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.24:34199', status: closed, memory: 0, processing: 0>

2025-12-03 05:31:08,927 - distributed.scheduler - INFO - Remove client Client-worker-7b1cc80d-cae5-11f0-83f6-46fbd1744967

2025-12-03 05:31:07,369 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:36091

2025-12-03 05:31:07,368 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:36091', status: closed, memory: 0, processing: 0>

2025-12-03 05:31:06,474 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:38069', status: closed, memory: 0, processing: 0>

2025-12-03 05:30:21,642 - distributed.scheduler - INFO - Receive client connection: Client-2964a0e7-d009-11f0-8001-f2cd832f225c

2025-12-03 05:20:20,177 - distributed.scheduler - INFO - Receive client connection: Client-c2e42458-d007-11f0-8001-f2cd832f225c

2025-12-03 05:10:18,772 - distributed.scheduler - INFO - Receive client connection: Client-5c6c80bc-d006-11f0-8001-f2cd832f225c

2025-12-03 05:00:22,863 - distributed.scheduler - INFO - Receive client connection: Client-worker-f931239e-d004-11f0-8733-de470ccc1e79

2025-12-03 05:00:22,675 - distributed.scheduler - INFO - Receive client connection: Client-worker-f918b828-d004-11f0-876f-ba070f366fac

2025-12-03 05:00:21,826 - distributed.scheduler - INFO - Receive client connection: Client-worker-f871958e-d004-11f0-8745-4ec0916b0c7e

2025-12-03 05:00:17,234 - distributed.scheduler - INFO - Receive client connection: Client-f5e046fe-d004-11f0-8001-f2cd832f225c

2025-12-03 04:50:15,689 - distributed.scheduler - INFO - Receive client connection: Client-8f554480-d003-11f0-8001-f2cd832f225c

2025-12-03 04:40:14,227 - distributed.scheduler - INFO - Receive client connection: Client-28d56105-d002-11f0-8001-f2cd832f225c

2025-12-03 04:30:18,282 - distributed.scheduler - INFO - Receive client connection: Client-worker-c5916bbb-d000-11f0-87cf-b64bf7bee50e

2025-12-03 04:30:17,078 - distributed.scheduler - INFO - Receive client connection: Client-worker-c4e82bcf-d000-11f0-8826-8a1d5a425dc1

2025-12-03 04:30:12,779 - distributed.scheduler - INFO - Receive client connection: Client-c258e80b-d000-11f0-8001-f2cd832f225c

2025-12-03 04:20:11,372 - distributed.scheduler - INFO - Receive client connection: Client-5be05e38-cfff-11f0-8001-f2cd832f225c

2025-12-03 04:10:09,951 - distributed.scheduler - INFO - Receive client connection: Client-f5674f05-cffd-11f0-8001-f2cd832f225c

2025-12-03 04:01:05,660 - distributed.scheduler - INFO - Receive client connection: Client-worker-b0f9caf8-cffc-11f0-84b3-e2572cf160eb

2025-12-03 04:00:56,989 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:40609

2025-12-03 04:00:56,987 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:40609', status: closed, memory: 0, processing: 0>

2025-12-03 04:00:56,855 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:38667

2025-12-03 04:00:56,853 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:38667', status: closed, memory: 0, processing: 0>

2025-12-03 04:00:56,045 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:39529', status: closed, memory: 0, processing: 0>

2025-12-03 04:00:55,958 - distributed.scheduler - INFO - Close client connection: Client-worker-f5796fcf-cf7e-11f0-8412-e2572cf160eb

2025-12-03 04:00:55,945 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:43201', status: closed, memory: 0, processing: 0>

2025-12-03 04:00:55,944 - distributed.scheduler - INFO - Remove client Client-worker-f5796fcf-cf7e-11f0-8412-e2572cf160eb

2025-12-03 04:00:54,982 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:38069

2025-12-03 04:00:54,982 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:38069', status: closed, memory: 0, processing: 0>

2025-12-03 04:00:54,478 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:37695

2025-12-03 04:00:54,478 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:37695', status: running, memory: 0, processing: 0>

2025-12-03 04:00:54,436 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.248:35313

2025-12-03 04:00:54,435 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.248:35313', status: closed, memory: 0, processing: 0>

2025-12-03 04:00:54,157 - distributed.scheduler - INFO - Close client connection: Client-worker-8dcd000f-cf61-11f0-889e-8253c573df33

2025-12-03 04:00:54,151 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:40391', status: closed, memory: 0, processing: 0>

2025-12-03 04:00:54,148 - distributed.scheduler - INFO - Remove client Client-worker-8dcd000f-cf61-11f0-889e-8253c573df33

2025-12-03 04:00:53,570 - distributed.scheduler - INFO - Close client connection: Client-worker-bbc7bf0e-cbb6-11f0-84ea-4ec0916b0c7e

2025-12-03 04:00:53,566 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.248:37517', status: closed, memory: 0, processing: 0>

2025-12-03 04:00:53,565 - distributed.scheduler - INFO - Remove client Client-worker-bbc7bf0e-cbb6-11f0-84ea-4ec0916b0c7e

2025-12-03 04:00:53,528 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:46011', status: closed, memory: 0, processing: 0>

2025-12-03 04:00:52,546 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:38247

2025-12-03 04:00:52,544 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:38247', status: closed, memory: 0, processing: 0>

2025-12-03 04:00:51,830 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:46391

2025-12-03 04:00:51,828 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:46391', status: closed, memory: 0, processing: 0>

2025-12-03 04:00:51,476 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:34679', status: closed, memory: 0, processing: 0>

2025-12-03 04:00:50,853 - distributed.scheduler - INFO - Close client connection: Client-worker-aa98e53e-cb88-11f0-8583-b64bf7bee50e

2025-12-03 04:00:50,849 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:45767', status: closed, memory: 0, processing: 0>

2025-12-03 04:00:50,847 - distributed.scheduler - INFO - Remove client Client-worker-aa98e53e-cb88-11f0-8583-b64bf7bee50e

2025-12-03 04:00:08,504 - distributed.scheduler - INFO - Receive client connection: Client-8ee94f57-cffc-11f0-8001-f2cd832f225c

2025-12-03 03:50:06,981 - distributed.scheduler - INFO - Receive client connection: Client-285f467d-cffb-11f0-8001-f2cd832f225c

2025-12-03 03:40:48,903 - distributed.scheduler - INFO - Close client connection: Client-c1e7069b-cff9-11f0-8001-f2cd832f225c

2025-12-03 03:40:48,902 - distributed.scheduler - INFO - Remove client Client-c1e7069b-cff9-11f0-8001-f2cd832f225c

2025-12-03 03:40:48,900 - distributed.scheduler - INFO - Remove client Client-c1e7069b-cff9-11f0-8001-f2cd832f225c

2025-12-03 03:40:05,572 - distributed.scheduler - INFO - Receive client connection: Client-c1e7069b-cff9-11f0-8001-f2cd832f225c

2025-12-03 03:33:30,856 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:39529

2025-12-03 03:33:30,855 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:39529', status: closed, memory: 0, processing: 0>

2025-12-03 03:33:30,016 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:39021', status: closed, memory: 0, processing: 0>

2025-12-03 03:33:29,622 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:34679

2025-12-03 03:33:29,621 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:34679', status: closed, memory: 0, processing: 0>

2025-12-03 03:33:28,559 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:46281', status: closed, memory: 0, processing: 0>

2025-12-03 03:33:05,651 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:39021

2025-12-03 03:33:05,650 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:39021', status: closed, memory: 0, processing: 0>

2025-12-03 03:33:04,584 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:32873', status: closed, memory: 0, processing: 0>

2025-12-03 03:33:04,250 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:46281

2025-12-03 03:33:04,247 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:46281', status: closed, memory: 0, processing: 0>

2025-12-03 03:33:03,264 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:38741', status: closed, memory: 0, processing: 0>

2025-12-03 03:32:39,225 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:32873

2025-12-03 03:32:39,224 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:32873', status: closed, memory: 0, processing: 0>

2025-12-03 03:32:38,173 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:34173', status: closed, memory: 0, processing: 0>

2025-12-03 03:32:38,008 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:38741

2025-12-03 03:32:38,007 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:38741', status: closed, memory: 0, processing: 0>

2025-12-03 03:32:37,079 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:39491', status: closed, memory: 0, processing: 0>

2025-12-03 03:32:11,175 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:34173

2025-12-03 03:32:11,174 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:34173', status: closed, memory: 0, processing: 0>

2025-12-03 03:32:10,335 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:39491

2025-12-03 03:32:10,333 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:39491', status: closed, memory: 0, processing: 0>

2025-12-03 03:32:10,287 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:40099', status: closed, memory: 0, processing: 0>

2025-12-03 03:32:09,367 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:44031', status: closed, memory: 0, processing: 0>

2025-12-03 03:31:43,835 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:44031

2025-12-03 03:31:43,835 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:44031', status: closed, memory: 0, processing: 0>

2025-12-03 03:31:42,966 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:38209', status: closed, memory: 0, processing: 0>

2025-12-03 03:31:40,174 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:40099

2025-12-03 03:31:40,172 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:40099', status: closed, memory: 0, processing: 0>

2025-12-03 03:31:39,281 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:43707', status: closed, memory: 0, processing: 0>

2025-12-03 03:31:34,780 - distributed.scheduler - INFO - Close client connection: Client-5c209ab2-cff8-11f0-8001-f2cd832f225c

2025-12-03 03:31:34,779 - distributed.scheduler - INFO - Remove client Client-5c209ab2-cff8-11f0-8001-f2cd832f225c

2025-12-03 03:31:34,777 - distributed.scheduler - INFO - Remove client Client-5c209ab2-cff8-11f0-8001-f2cd832f225c

2025-12-03 03:31:05,283 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:43707

2025-12-03 03:31:05,281 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:43707', status: closed, memory: 0, processing: 0>

2025-12-03 03:31:04,348 - distributed.scheduler - INFO - Close client connection: Client-worker-db850117-cb77-11f0-8479-de470ccc1e79

2025-12-03 03:31:04,340 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:44023', status: closed, memory: 0, processing: 0>

2025-12-03 03:31:04,339 - distributed.scheduler - INFO - Remove client Client-worker-db850117-cb77-11f0-8479-de470ccc1e79

2025-12-03 03:31:02,825 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:38209

2025-12-03 03:31:02,823 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:38209', status: closed, memory: 0, processing: 0>

2025-12-03 03:31:01,815 - distributed.scheduler - INFO - Close client connection: Client-worker-33d431f3-cd34-11f0-8623-8a1d5a425dc1

2025-12-03 03:31:01,811 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:42943', status: closed, memory: 0, processing: 0>

2025-12-03 03:31:01,809 - distributed.scheduler - INFO - Remove client Client-worker-33d431f3-cd34-11f0-8623-8a1d5a425dc1

2025-12-03 03:30:05,325 - distributed.scheduler - INFO - Receive client connection: Client-5c209ab2-cff8-11f0-8001-f2cd832f225c

2025-12-03 03:20:46,158 - distributed.scheduler - INFO - Close client connection: Client-f4ebc2ee-cff6-11f0-8001-f2cd832f225c

2025-12-03 03:20:46,157 - distributed.scheduler - INFO - Remove client Client-f4ebc2ee-cff6-11f0-8001-f2cd832f225c

2025-12-03 03:20:46,155 - distributed.scheduler - INFO - Remove client Client-f4ebc2ee-cff6-11f0-8001-f2cd832f225c

2025-12-03 03:20:02,675 - distributed.scheduler - INFO - Receive client connection: Client-f4ebc2ee-cff6-11f0-8001-f2cd832f225c

2025-12-03 03:10:42,961 - distributed.scheduler - INFO - Close client connection: Client-8e627e20-cff5-11f0-8001-f2cd832f225c

2025-12-03 03:10:42,957 - distributed.scheduler - INFO - Remove client Client-8e627e20-cff5-11f0-8001-f2cd832f225c

2025-12-03 03:10:42,955 - distributed.scheduler - INFO - Remove client Client-8e627e20-cff5-11f0-8001-f2cd832f225c

2025-12-03 03:10:01,154 - distributed.scheduler - INFO - Receive client connection: Client-8e627e20-cff5-11f0-8001-f2cd832f225c

2025-12-03 03:05:03,702 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:36085 failed: OSError: Timed out trying to connect to tcp://172.21.159.210:36085 after 30 s

2025-12-03 03:05:03,698 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:43201 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36085 after 30 s')

2025-12-03 03:05:03,628 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36085 after 30 s')

2025-12-03 03:05:03,627 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36085 after 30 s')

2025-12-03 03:05:03,627 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36085 after 30 s')

2025-12-03 03:05:03,601 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:42943 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36085 after 30 s')

2025-12-03 03:05:03,585 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36085 after 30 s')

2025-12-03 03:05:03,584 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:40569 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36085 after 30 s')

2025-12-03 03:05:03,581 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36085 after 30 s')

2025-12-03 03:05:03,577 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:34871 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36085 after 30 s')

2025-12-03 03:05:03,577 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36085 after 30 s')

2025-12-03 03:05:03,576 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:40391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36085 after 30 s')

2025-12-03 03:05:03,576 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36085 after 30 s')

2025-12-03 03:05:03,561 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36085 after 30 s')

2025-12-03 03:04:34,468 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:46011

2025-12-03 03:04:34,465 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:46011', status: closed, memory: 0, processing: 0>

2025-12-03 03:04:33,460 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:36085', status: closed, memory: 0, processing: 0>

2025-12-03 03:04:16,942 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:46543 failed: OSError: Timed out trying to connect to tcp://172.21.159.210:46543 after 30 s

2025-12-03 03:04:16,909 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:43201 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:46543 after 30 s')

2025-12-03 03:04:16,843 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:46543 after 30 s')

2025-12-03 03:04:16,840 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:46543 after 30 s')

2025-12-03 03:04:16,840 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:46543 after 30 s')

2025-12-03 03:04:16,839 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:46543 after 30 s')

2025-12-03 03:04:16,807 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:46543 after 30 s')

2025-12-03 03:04:16,792 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:40391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:46543 after 30 s')

2025-12-03 03:03:47,499 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:36085

2025-12-03 03:03:47,496 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:36085', status: closed, memory: 0, processing: 0>

2025-12-03 03:03:46,656 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:46543', status: closed, memory: 0, processing: 0>

2025-12-03 03:03:29,377 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:45907 failed: OSError: Timed out trying to connect to tcp://172.21.159.210:45907 after 30 s

2025-12-03 03:03:29,297 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:43201 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:45907 after 30 s')

2025-12-03 03:03:29,291 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:45907 after 30 s')

2025-12-03 03:03:29,290 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:45907 after 30 s')

2025-12-03 03:03:29,255 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44619 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:45907 after 30 s')

2025-12-03 03:03:29,234 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:34871 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:45907 after 30 s')

2025-12-03 03:03:29,200 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:42943 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:45907 after 30 s')

2025-12-03 03:03:29,199 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:45907 after 30 s')

2025-12-03 03:03:29,198 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:45907 after 30 s')

2025-12-03 03:03:29,197 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:45907 after 30 s')

2025-12-03 03:03:29,187 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:45907 after 30 s')

2025-12-03 03:02:59,893 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:46543

2025-12-03 03:02:59,891 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:46543', status: closed, memory: 0, processing: 0>

2025-12-03 03:02:59,058 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:45907', status: closed, memory: 0, processing: 0>

2025-12-03 03:02:44,671 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:40855 failed: OSError: Timed out trying to connect to tcp://172.21.159.210:40855 after 30 s

2025-12-03 03:02:44,242 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:42943 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:40855 after 30 s')

2025-12-03 03:02:44,144 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:43201 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:40855 after 30 s')

2025-12-03 03:02:44,132 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:44981 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:40855 after 30 s')

2025-12-03 03:02:44,058 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:40855 after 30 s')

2025-12-03 03:02:44,031 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44619 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:40855 after 30 s')

2025-12-03 03:02:44,024 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:40391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:40855 after 30 s')

2025-12-03 03:02:44,010 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:40855 after 30 s')

2025-12-03 03:02:43,998 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:40855 after 30 s')

2025-12-03 03:02:30,410 - distributed.scheduler - INFO - Close client connection: Client-4bb3cb25-cff4-11f0-8001-f2cd832f225c

2025-12-03 03:02:30,409 - distributed.scheduler - INFO - Remove client Client-4bb3cb25-cff4-11f0-8001-f2cd832f225c

2025-12-03 03:02:30,408 - distributed.scheduler - INFO - Remove client Client-4bb3cb25-cff4-11f0-8001-f2cd832f225c

2025-12-03 03:02:14,696 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:45907

2025-12-03 03:02:14,695 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:45907', status: closed, memory: 0, processing: 0>

2025-12-03 03:02:13,863 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:40855', status: closed, memory: 0, processing: 0>

2025-12-03 03:01:47,104 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:40855

2025-12-03 03:01:47,102 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:40855', status: closed, memory: 0, processing: 0>

2025-12-03 03:01:46,208 - distributed.scheduler - INFO - Close client connection: Client-worker-d2425638-cc0e-11f0-852f-ba070f366fac

2025-12-03 03:01:46,202 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:43399', status: closed, memory: 0, processing: 0>

2025-12-03 03:01:46,201 - distributed.scheduler - INFO - Remove client Client-worker-d2425638-cc0e-11f0-852f-ba070f366fac

2025-12-03 03:00:59,777 - distributed.scheduler - INFO - Receive client connection: Client-4bb3cb25-cff4-11f0-8001-f2cd832f225c

2025-12-03 02:51:40,926 - distributed.scheduler - INFO - Close client connection: Client-e540196e-cff2-11f0-8001-f2cd832f225c

2025-12-03 02:51:40,925 - distributed.scheduler - INFO - Remove client Client-e540196e-cff2-11f0-8001-f2cd832f225c

2025-12-03 02:51:40,923 - distributed.scheduler - INFO - Remove client Client-e540196e-cff2-11f0-8001-f2cd832f225c

2025-12-03 02:50:58,401 - distributed.scheduler - INFO - Receive client connection: Client-e540196e-cff2-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,722 - distributed.scheduler - INFO - Close client connection: Client-ae87b437-cfbd-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,722 - distributed.scheduler - INFO - Close client connection: Client-150804e7-cfbf-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,722 - distributed.scheduler - INFO - Close client connection: Client-7b87bb96-cfc0-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,722 - distributed.scheduler - INFO - Close client connection: Client-e204bace-cfc1-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,722 - distributed.scheduler - INFO - Close client connection: Client-4891b494-cfc3-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,722 - distributed.scheduler - INFO - Close client connection: Client-af10c310-cfc4-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,722 - distributed.scheduler - INFO - Close client connection: Client-157ffea9-cfc6-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,722 - distributed.scheduler - INFO - Close client connection: Client-7bfdc505-cfc7-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,721 - distributed.scheduler - INFO - Close client connection: Client-e27b02ef-cfc8-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,721 - distributed.scheduler - INFO - Close client connection: Client-48fc7319-cfca-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,721 - distributed.scheduler - INFO - Close client connection: Client-af84ba4f-cfcb-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,721 - distributed.scheduler - INFO - Close client connection: Client-15ffe3a5-cfcd-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,721 - distributed.scheduler - INFO - Close client connection: Client-7c7a6b99-cfce-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,721 - distributed.scheduler - INFO - Close client connection: Client-e2faf346-cfcf-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,721 - distributed.scheduler - INFO - Close client connection: Client-496d9a69-cfd1-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,721 - distributed.scheduler - INFO - Close client connection: Client-afe12232-cfd2-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,721 - distributed.scheduler - INFO - Close client connection: Client-16622137-cfd4-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,721 - distributed.scheduler - INFO - Close client connection: Client-7cd30143-cfd5-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,721 - distributed.scheduler - INFO - Close client connection: Client-e34bb159-cfd6-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,721 - distributed.scheduler - INFO - Close client connection: Client-49c67304-cfd8-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,720 - distributed.scheduler - INFO - Close client connection: Client-b0426478-cfd9-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,720 - distributed.scheduler - INFO - Close client connection: Client-16c0a3c7-cfdb-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,720 - distributed.scheduler - INFO - Close client connection: Client-7d404b6a-cfdc-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,720 - distributed.scheduler - INFO - Close client connection: Client-e3b7e6d6-cfdd-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,720 - distributed.scheduler - INFO - Close client connection: Client-4a27e5d8-cfdf-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,720 - distributed.scheduler - INFO - Close client connection: Client-b0a52d62-cfe0-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,720 - distributed.scheduler - INFO - Close client connection: Client-1742e03f-cfe2-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,720 - distributed.scheduler - INFO - Close client connection: Client-7dbbe837-cfe3-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,720 - distributed.scheduler - INFO - Close client connection: Client-e443680e-cfe4-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,719 - distributed.scheduler - INFO - Close client connection: Client-4ac3898a-cfe6-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,718 - distributed.scheduler - INFO - Close client connection: Client-b144d7a9-cfe7-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,718 - distributed.scheduler - INFO - Close client connection: Client-17c0b41f-cfe9-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,718 - distributed.scheduler - INFO - Close client connection: Client-7e38e378-cfea-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,718 - distributed.scheduler - INFO - Close client connection: Client-e4be21f9-cfeb-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,718 - distributed.scheduler - INFO - Close client connection: Client-4b643f87-cfed-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,718 - distributed.scheduler - INFO - Close client connection: Client-b1e75f81-cfee-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,717 - distributed.scheduler - INFO - Close client connection: Client-1865b30e-cff0-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,714 - distributed.scheduler - INFO - Remove client Client-ae87b437-cfbd-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,714 - distributed.scheduler - INFO - Remove client Client-150804e7-cfbf-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,714 - distributed.scheduler - INFO - Remove client Client-7b87bb96-cfc0-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,713 - distributed.scheduler - INFO - Remove client Client-e204bace-cfc1-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,713 - distributed.scheduler - INFO - Remove client Client-4891b494-cfc3-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,713 - distributed.scheduler - INFO - Remove client Client-af10c310-cfc4-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,712 - distributed.scheduler - INFO - Remove client Client-157ffea9-cfc6-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,712 - distributed.scheduler - INFO - Remove client Client-7bfdc505-cfc7-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,712 - distributed.scheduler - INFO - Remove client Client-e27b02ef-cfc8-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,712 - distributed.scheduler - INFO - Remove client Client-48fc7319-cfca-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,711 - distributed.scheduler - INFO - Remove client Client-af84ba4f-cfcb-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,711 - distributed.scheduler - INFO - Remove client Client-15ffe3a5-cfcd-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,711 - distributed.scheduler - INFO - Remove client Client-7c7a6b99-cfce-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,710 - distributed.scheduler - INFO - Remove client Client-e2faf346-cfcf-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,710 - distributed.scheduler - INFO - Remove client Client-496d9a69-cfd1-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,710 - distributed.scheduler - INFO - Remove client Client-afe12232-cfd2-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,709 - distributed.scheduler - INFO - Remove client Client-16622137-cfd4-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,709 - distributed.scheduler - INFO - Remove client Client-7cd30143-cfd5-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,709 - distributed.scheduler - INFO - Remove client Client-e34bb159-cfd6-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,708 - distributed.scheduler - INFO - Remove client Client-49c67304-cfd8-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,708 - distributed.scheduler - INFO - Remove client Client-b0426478-cfd9-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,708 - distributed.scheduler - INFO - Remove client Client-16c0a3c7-cfdb-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,707 - distributed.scheduler - INFO - Remove client Client-7d404b6a-cfdc-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,707 - distributed.scheduler - INFO - Remove client Client-e3b7e6d6-cfdd-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,706 - distributed.scheduler - INFO - Remove client Client-4a27e5d8-cfdf-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,706 - distributed.scheduler - INFO - Remove client Client-b0a52d62-cfe0-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,705 - distributed.scheduler - INFO - Remove client Client-1742e03f-cfe2-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,705 - distributed.scheduler - INFO - Remove client Client-7dbbe837-cfe3-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,704 - distributed.scheduler - INFO - Remove client Client-e443680e-cfe4-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,696 - distributed.scheduler - INFO - Remove client Client-4ac3898a-cfe6-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,695 - distributed.scheduler - INFO - Remove client Client-b144d7a9-cfe7-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,694 - distributed.scheduler - INFO - Remove client Client-17c0b41f-cfe9-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,694 - distributed.scheduler - INFO - Remove client Client-7e38e378-cfea-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,693 - distributed.scheduler - INFO - Remove client Client-e4be21f9-cfeb-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,692 - distributed.scheduler - INFO - Remove client Client-4b643f87-cfed-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,691 - distributed.scheduler - INFO - Remove client Client-b1e75f81-cfee-11f0-8001-f2cd832f225c

2025-12-03 02:41:36,691 - distributed.scheduler - INFO - Remove client Client-1865b30e-cff0-11f0-8001-f2cd832f225c

2025-12-03 02:41:35,920 - distributed.scheduler - INFO - Close client connection: Client-7ed2d3fb-cff1-11f0-8001-f2cd832f225c

2025-12-03 02:41:35,919 - distributed.scheduler - INFO - Remove client Client-7ed2d3fb-cff1-11f0-8001-f2cd832f225c

2025-12-03 02:41:35,917 - distributed.scheduler - INFO - Remove client Client-7ed2d3fb-cff1-11f0-8001-f2cd832f225c

2025-12-03 02:40:57,053 - distributed.scheduler - INFO - Receive client connection: Client-7ed2d3fb-cff1-11f0-8001-f2cd832f225c

2025-12-03 02:30:55,711 - distributed.scheduler - INFO - Receive client connection: Client-1865b30e-cff0-11f0-8001-f2cd832f225c

2025-12-03 02:20:54,265 - distributed.scheduler - INFO - Receive client connection: Client-b1e75f81-cfee-11f0-8001-f2cd832f225c

2025-12-03 02:10:52,768 - distributed.scheduler - INFO - Receive client connection: Client-4b643f87-cfed-11f0-8001-f2cd832f225c

2025-12-03 02:00:51,060 - distributed.scheduler - INFO - Receive client connection: Client-e4be21f9-cfeb-11f0-8001-f2cd832f225c

2025-12-03 01:50:49,565 - distributed.scheduler - INFO - Receive client connection: Client-7e38e378-cfea-11f0-8001-f2cd832f225c

2025-12-03 01:40:48,145 - distributed.scheduler - INFO - Receive client connection: Client-17c0b41f-cfe9-11f0-8001-f2cd832f225c

2025-12-03 01:30:46,715 - distributed.scheduler - INFO - Receive client connection: Client-b144d7a9-cfe7-11f0-8001-f2cd832f225c

2025-12-03 01:20:45,240 - distributed.scheduler - INFO - Receive client connection: Client-4ac3898a-cfe6-11f0-8001-f2cd832f225c

2025-12-03 01:10:43,780 - distributed.scheduler - INFO - Receive client connection: Client-e443680e-cfe4-11f0-8001-f2cd832f225c

2025-12-03 01:00:42,266 - distributed.scheduler - INFO - Receive client connection: Client-7dbbe837-cfe3-11f0-8001-f2cd832f225c

2025-12-03 00:50:40,850 - distributed.scheduler - INFO - Receive client connection: Client-1742e03f-cfe2-11f0-8001-f2cd832f225c

2025-12-03 00:40:39,236 - distributed.scheduler - INFO - Receive client connection: Client-b0a52d62-cfe0-11f0-8001-f2cd832f225c

2025-12-03 00:30:37,745 - distributed.scheduler - INFO - Receive client connection: Client-4a27e5d8-cfdf-11f0-8001-f2cd832f225c

2025-12-03 00:20:36,388 - distributed.scheduler - INFO - Receive client connection: Client-e3b7e6d6-cfdd-11f0-8001-f2cd832f225c

2025-12-03 00:10:34,978 - distributed.scheduler - INFO - Receive client connection: Client-7d404b6a-cfdc-11f0-8001-f2cd832f225c

2025-12-03 00:00:33,517 - distributed.scheduler - INFO - Receive client connection: Client-16c0a3c7-cfdb-11f0-8001-f2cd832f225c

2025-12-02 23:50:32,061 - distributed.scheduler - INFO - Receive client connection: Client-b0426478-cfd9-11f0-8001-f2cd832f225c

2025-12-02 23:40:30,679 - distributed.scheduler - INFO - Receive client connection: Client-49c67304-cfd8-11f0-8001-f2cd832f225c

2025-12-02 23:30:29,214 - distributed.scheduler - INFO - Receive client connection: Client-e34bb159-cfd6-11f0-8001-f2cd832f225c

2025-12-02 23:20:27,788 - distributed.scheduler - INFO - Receive client connection: Client-7cd30143-cfd5-11f0-8001-f2cd832f225c

2025-12-02 23:10:26,418 - distributed.scheduler - INFO - Receive client connection: Client-16622137-cfd4-11f0-8001-f2cd832f225c

2025-12-02 23:00:24,947 - distributed.scheduler - INFO - Receive client connection: Client-afe12232-cfd2-11f0-8001-f2cd832f225c

2025-12-02 22:50:23,567 - distributed.scheduler - INFO - Receive client connection: Client-496d9a69-cfd1-11f0-8001-f2cd832f225c

2025-12-02 22:40:22,194 - distributed.scheduler - INFO - Receive client connection: Client-e2faf346-cfcf-11f0-8001-f2cd832f225c

2025-12-02 22:30:20,729 - distributed.scheduler - INFO - Receive client connection: Client-7c7a6b99-cfce-11f0-8001-f2cd832f225c

2025-12-02 22:20:19,297 - distributed.scheduler - INFO - Receive client connection: Client-15ffe3a5-cfcd-11f0-8001-f2cd832f225c

2025-12-02 22:10:17,894 - distributed.scheduler - INFO - Receive client connection: Client-af84ba4f-cfcb-11f0-8001-f2cd832f225c

2025-12-02 22:00:16,357 - distributed.scheduler - INFO - Receive client connection: Client-48fc7319-cfca-11f0-8001-f2cd832f225c

2025-12-02 21:50:14,894 - distributed.scheduler - INFO - Receive client connection: Client-e27b02ef-cfc8-11f0-8001-f2cd832f225c

2025-12-02 21:40:13,435 - distributed.scheduler - INFO - Receive client connection: Client-7bfdc505-cfc7-11f0-8001-f2cd832f225c

2025-12-02 21:30:11,990 - distributed.scheduler - INFO - Receive client connection: Client-157ffea9-cfc6-11f0-8001-f2cd832f225c

2025-12-02 21:20:10,660 - distributed.scheduler - INFO - Receive client connection: Client-af10c310-cfc4-11f0-8001-f2cd832f225c

2025-12-02 21:10:09,173 - distributed.scheduler - INFO - Receive client connection: Client-4891b494-cfc3-11f0-8001-f2cd832f225c

2025-12-02 21:00:07,628 - distributed.scheduler - INFO - Receive client connection: Client-e204bace-cfc1-11f0-8001-f2cd832f225c

2025-12-02 20:50:06,185 - distributed.scheduler - INFO - Receive client connection: Client-7b87bb96-cfc0-11f0-8001-f2cd832f225c

2025-12-02 20:40:04,721 - distributed.scheduler - INFO - Receive client connection: Client-150804e7-cfbf-11f0-8001-f2cd832f225c

2025-12-02 20:30:03,261 - distributed.scheduler - INFO - Receive client connection: Client-ae87b437-cfbd-11f0-8001-f2cd832f225c

2025-12-02 20:20:33,508 - distributed.scheduler - INFO - Close client connection: Client-48064939-cfbc-11f0-8001-f2cd832f225c

2025-12-02 20:20:33,507 - distributed.scheduler - INFO - Remove client Client-48064939-cfbc-11f0-8001-f2cd832f225c

2025-12-02 20:20:33,505 - distributed.scheduler - INFO - Remove client Client-48064939-cfbc-11f0-8001-f2cd832f225c

2025-12-02 20:20:01,793 - distributed.scheduler - INFO - Receive client connection: Client-48064939-cfbc-11f0-8001-f2cd832f225c

2025-12-02 20:10:29,929 - distributed.scheduler - INFO - Close client connection: Client-e18bf6bf-cfba-11f0-8001-f2cd832f225c

2025-12-02 20:10:29,929 - distributed.scheduler - INFO - Remove client Client-e18bf6bf-cfba-11f0-8001-f2cd832f225c

2025-12-02 20:10:29,927 - distributed.scheduler - INFO - Remove client Client-e18bf6bf-cfba-11f0-8001-f2cd832f225c

2025-12-02 20:10:00,367 - distributed.scheduler - INFO - Receive client connection: Client-e18bf6bf-cfba-11f0-8001-f2cd832f225c

2025-12-02 20:02:01,328 - distributed.scheduler - INFO - Close client connection: Client-9ed910d3-cfb9-11f0-8001-f2cd832f225c

2025-12-02 20:02:01,327 - distributed.scheduler - INFO - Remove client Client-9ed910d3-cfb9-11f0-8001-f2cd832f225c

2025-12-02 20:02:01,325 - distributed.scheduler - INFO - Remove client Client-9ed910d3-cfb9-11f0-8001-f2cd832f225c

2025-12-02 20:00:58,972 - distributed.scheduler - INFO - Receive client connection: Client-9ed910d3-cfb9-11f0-8001-f2cd832f225c

2025-12-02 19:51:31,379 - distributed.scheduler - INFO - Close client connection: Client-3876c7ff-cfb8-11f0-8001-f2cd832f225c

2025-12-02 19:51:31,377 - distributed.scheduler - INFO - Remove client Client-3876c7ff-cfb8-11f0-8001-f2cd832f225c

2025-12-02 19:51:31,375 - distributed.scheduler - INFO - Remove client Client-3876c7ff-cfb8-11f0-8001-f2cd832f225c

2025-12-02 19:50:57,699 - distributed.scheduler - INFO - Receive client connection: Client-3876c7ff-cfb8-11f0-8001-f2cd832f225c

2025-12-02 19:41:39,889 - distributed.scheduler - INFO - Close client connection: Client-d20530cb-cfb6-11f0-8001-f2cd832f225c

2025-12-02 19:41:39,888 - distributed.scheduler - INFO - Remove client Client-d20530cb-cfb6-11f0-8001-f2cd832f225c

2025-12-02 19:41:39,886 - distributed.scheduler - INFO - Remove client Client-d20530cb-cfb6-11f0-8001-f2cd832f225c

2025-12-02 19:40:56,330 - distributed.scheduler - INFO - Receive client connection: Client-d20530cb-cfb6-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,378 - distributed.scheduler - INFO - Close client connection: Client-03fe630a-cfa6-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,377 - distributed.scheduler - INFO - Close client connection: Client-6a807c2c-cfa7-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,377 - distributed.scheduler - INFO - Close client connection: Client-d0fd2904-cfa8-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,377 - distributed.scheduler - INFO - Close client connection: Client-377e6730-cfaa-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,377 - distributed.scheduler - INFO - Close client connection: Client-9e0633da-cfab-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,376 - distributed.scheduler - INFO - Close client connection: Client-04858b0e-cfad-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,376 - distributed.scheduler - INFO - Close client connection: Client-6b02882a-cfae-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,376 - distributed.scheduler - INFO - Close client connection: Client-d174e15f-cfaf-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,375 - distributed.scheduler - INFO - Close client connection: Client-3806a0f5-cfb1-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,375 - distributed.scheduler - INFO - Close client connection: Client-050e10f4-cfb4-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,374 - distributed.scheduler - INFO - Close client connection: Client-9e8942db-cfb2-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,373 - distributed.scheduler - INFO - Remove client Client-03fe630a-cfa6-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,372 - distributed.scheduler - INFO - Remove client Client-6a807c2c-cfa7-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,371 - distributed.scheduler - INFO - Remove client Client-d0fd2904-cfa8-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,370 - distributed.scheduler - INFO - Remove client Client-377e6730-cfaa-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,370 - distributed.scheduler - INFO - Remove client Client-9e0633da-cfab-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,369 - distributed.scheduler - INFO - Remove client Client-04858b0e-cfad-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,368 - distributed.scheduler - INFO - Remove client Client-6b02882a-cfae-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,367 - distributed.scheduler - INFO - Remove client Client-d174e15f-cfaf-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,366 - distributed.scheduler - INFO - Remove client Client-3806a0f5-cfb1-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,361 - distributed.scheduler - INFO - Remove client Client-050e10f4-cfb4-11f0-8001-f2cd832f225c

2025-12-02 19:32:03,360 - distributed.scheduler - INFO - Remove client Client-9e8942db-cfb2-11f0-8001-f2cd832f225c

2025-12-02 19:32:02,797 - distributed.scheduler - INFO - Close client connection: Client-6b88eb27-cfb5-11f0-8001-f2cd832f225c

2025-12-02 19:32:02,795 - distributed.scheduler - INFO - Remove client Client-6b88eb27-cfb5-11f0-8001-f2cd832f225c

2025-12-02 19:32:02,793 - distributed.scheduler - INFO - Remove client Client-6b88eb27-cfb5-11f0-8001-f2cd832f225c

2025-12-02 19:30:54,879 - distributed.scheduler - INFO - Receive client connection: Client-6b88eb27-cfb5-11f0-8001-f2cd832f225c

2025-12-02 19:20:53,456 - distributed.scheduler - INFO - Receive client connection: Client-050e10f4-cfb4-11f0-8001-f2cd832f225c

2025-12-02 19:10:51,964 - distributed.scheduler - INFO - Receive client connection: Client-9e8942db-cfb2-11f0-8001-f2cd832f225c

2025-12-02 19:00:50,478 - distributed.scheduler - INFO - Receive client connection: Client-3806a0f5-cfb1-11f0-8001-f2cd832f225c

2025-12-02 18:50:48,903 - distributed.scheduler - INFO - Receive client connection: Client-d174e15f-cfaf-11f0-8001-f2cd832f225c

2025-12-02 18:40:47,519 - distributed.scheduler - INFO - Receive client connection: Client-6b02882a-cfae-11f0-8001-f2cd832f225c

2025-12-02 18:30:46,086 - distributed.scheduler - INFO - Receive client connection: Client-04858b0e-cfad-11f0-8001-f2cd832f225c

2025-12-02 18:20:44,624 - distributed.scheduler - INFO - Receive client connection: Client-9e0633da-cfab-11f0-8001-f2cd832f225c

2025-12-02 18:10:43,109 - distributed.scheduler - INFO - Receive client connection: Client-377e6730-cfaa-11f0-8001-f2cd832f225c

2025-12-02 18:00:41,633 - distributed.scheduler - INFO - Receive client connection: Client-d0fd2904-cfa8-11f0-8001-f2cd832f225c

2025-12-02 17:50:40,196 - distributed.scheduler - INFO - Receive client connection: Client-6a807c2c-cfa7-11f0-8001-f2cd832f225c

2025-12-02 17:40:38,730 - distributed.scheduler - INFO - Receive client connection: Client-03fe630a-cfa6-11f0-8001-f2cd832f225c

2025-12-02 17:31:46,822 - distributed.scheduler - INFO - Close client connection: Client-9d838b8c-cfa4-11f0-8001-f2cd832f225c

2025-12-02 17:31:46,821 - distributed.scheduler - INFO - Remove client Client-9d838b8c-cfa4-11f0-8001-f2cd832f225c

2025-12-02 17:31:46,819 - distributed.scheduler - INFO - Remove client Client-9d838b8c-cfa4-11f0-8001-f2cd832f225c

2025-12-02 17:30:37,299 - distributed.scheduler - INFO - Receive client connection: Client-9d838b8c-cfa4-11f0-8001-f2cd832f225c

2025-12-02 17:21:13,861 - distributed.scheduler - INFO - Close client connection: Client-370a9492-cfa3-11f0-8001-f2cd832f225c

2025-12-02 17:21:13,860 - distributed.scheduler - INFO - Remove client Client-370a9492-cfa3-11f0-8001-f2cd832f225c

2025-12-02 17:21:13,858 - distributed.scheduler - INFO - Remove client Client-370a9492-cfa3-11f0-8001-f2cd832f225c

2025-12-02 17:20:35,884 - distributed.scheduler - INFO - Receive client connection: Client-370a9492-cfa3-11f0-8001-f2cd832f225c

2025-12-02 17:11:13,760 - distributed.scheduler - INFO - Close client connection: Client-d0865a24-cfa1-11f0-8001-f2cd832f225c

2025-12-02 17:11:13,760 - distributed.scheduler - INFO - Remove client Client-d0865a24-cfa1-11f0-8001-f2cd832f225c

2025-12-02 17:11:13,759 - distributed.scheduler - INFO - Remove client Client-d0865a24-cfa1-11f0-8001-f2cd832f225c

2025-12-02 17:10:34,412 - distributed.scheduler - INFO - Receive client connection: Client-d0865a24-cfa1-11f0-8001-f2cd832f225c

2025-12-02 17:01:42,090 - distributed.scheduler - INFO - Close client connection: Client-6a08dea3-cfa0-11f0-8001-f2cd832f225c

2025-12-02 17:01:42,089 - distributed.scheduler - INFO - Remove client Client-6a08dea3-cfa0-11f0-8001-f2cd832f225c

2025-12-02 17:01:42,087 - distributed.scheduler - INFO - Remove client Client-6a08dea3-cfa0-11f0-8001-f2cd832f225c

2025-12-02 17:00:32,941 - distributed.scheduler - INFO - Receive client connection: Client-6a08dea3-cfa0-11f0-8001-f2cd832f225c

2025-12-02 16:51:10,145 - distributed.scheduler - INFO - Close client connection: Client-0390ed23-cf9f-11f0-8001-f2cd832f225c

2025-12-02 16:51:10,144 - distributed.scheduler - INFO - Remove client Client-0390ed23-cf9f-11f0-8001-f2cd832f225c

2025-12-02 16:51:10,142 - distributed.scheduler - INFO - Remove client Client-0390ed23-cf9f-11f0-8001-f2cd832f225c

2025-12-02 16:50:31,533 - distributed.scheduler - INFO - Receive client connection: Client-0390ed23-cf9f-11f0-8001-f2cd832f225c

2025-12-02 16:41:11,620 - distributed.scheduler - INFO - Close client connection: Client-d0181e8a-cf9a-11f0-8001-f2cd832f225c

2025-12-02 16:41:11,620 - distributed.scheduler - INFO - Close client connection: Client-69af6531-cf99-11f0-8001-f2cd832f225c

2025-12-02 16:41:11,620 - distributed.scheduler - INFO - Close client connection: Client-36914b49-cf9c-11f0-8001-f2cd832f225c

2025-12-02 16:41:11,620 - distributed.scheduler - INFO - Remove client Client-d0181e8a-cf9a-11f0-8001-f2cd832f225c

2025-12-02 16:41:11,619 - distributed.scheduler - INFO - Remove client Client-69af6531-cf99-11f0-8001-f2cd832f225c

2025-12-02 16:41:11,619 - distributed.scheduler - INFO - Remove client Client-36914b49-cf9c-11f0-8001-f2cd832f225c

2025-12-02 16:41:10,902 - distributed.scheduler - INFO - Close client connection: Client-9d1323f2-cf9d-11f0-8001-f2cd832f225c

2025-12-02 16:41:10,900 - distributed.scheduler - INFO - Remove client Client-9d1323f2-cf9d-11f0-8001-f2cd832f225c

2025-12-02 16:41:10,898 - distributed.scheduler - INFO - Remove client Client-9d1323f2-cf9d-11f0-8001-f2cd832f225c

2025-12-02 16:40:30,076 - distributed.scheduler - INFO - Receive client connection: Client-9d1323f2-cf9d-11f0-8001-f2cd832f225c

2025-12-02 16:30:28,601 - distributed.scheduler - INFO - Receive client connection: Client-36914b49-cf9c-11f0-8001-f2cd832f225c

2025-12-02 16:20:27,185 - distributed.scheduler - INFO - Receive client connection: Client-d0181e8a-cf9a-11f0-8001-f2cd832f225c

2025-12-02 16:10:25,884 - distributed.scheduler - INFO - Receive client connection: Client-69af6531-cf99-11f0-8001-f2cd832f225c

2025-12-02 16:01:49,533 - distributed.scheduler - INFO - Close client connection: Client-0336142a-cf98-11f0-8001-f2cd832f225c

2025-12-02 16:01:49,532 - distributed.scheduler - INFO - Remove client Client-0336142a-cf98-11f0-8001-f2cd832f225c

2025-12-02 16:01:49,531 - distributed.scheduler - INFO - Remove client Client-0336142a-cf98-11f0-8001-f2cd832f225c

2025-12-02 16:00:24,457 - distributed.scheduler - INFO - Receive client connection: Client-0336142a-cf98-11f0-8001-f2cd832f225c

2025-12-02 15:51:13,274 - distributed.scheduler - INFO - Close client connection: Client-9cb9354d-cf96-11f0-8001-f2cd832f225c

2025-12-02 15:51:13,273 - distributed.scheduler - INFO - Remove client Client-9cb9354d-cf96-11f0-8001-f2cd832f225c

2025-12-02 15:51:13,273 - distributed.scheduler - INFO - Remove client Client-9cb9354d-cf96-11f0-8001-f2cd832f225c

2025-12-02 15:50:23,023 - distributed.scheduler - INFO - Receive client connection: Client-9cb9354d-cf96-11f0-8001-f2cd832f225c

2025-12-02 15:41:16,968 - distributed.scheduler - INFO - Close client connection: Client-363c7467-cf95-11f0-8001-f2cd832f225c

2025-12-02 15:41:16,966 - distributed.scheduler - INFO - Remove client Client-363c7467-cf95-11f0-8001-f2cd832f225c

2025-12-02 15:41:16,964 - distributed.scheduler - INFO - Remove client Client-363c7467-cf95-11f0-8001-f2cd832f225c

2025-12-02 15:40:21,579 - distributed.scheduler - INFO - Receive client connection: Client-363c7467-cf95-11f0-8001-f2cd832f225c

2025-12-02 15:31:45,859 - distributed.scheduler - INFO - Close client connection: Client-cfc2d162-cf93-11f0-8001-f2cd832f225c

2025-12-02 15:31:45,858 - distributed.scheduler - INFO - Remove client Client-cfc2d162-cf93-11f0-8001-f2cd832f225c

2025-12-02 15:31:45,856 - distributed.scheduler - INFO - Remove client Client-cfc2d162-cf93-11f0-8001-f2cd832f225c

2025-12-02 15:30:20,157 - distributed.scheduler - INFO - Receive client connection: Client-cfc2d162-cf93-11f0-8001-f2cd832f225c

2025-12-02 15:21:08,588 - distributed.scheduler - INFO - Close client connection: Client-694b2a9d-cf92-11f0-8001-f2cd832f225c

2025-12-02 15:21:08,587 - distributed.scheduler - INFO - Remove client Client-694b2a9d-cf92-11f0-8001-f2cd832f225c

2025-12-02 15:21:08,585 - distributed.scheduler - INFO - Remove client Client-694b2a9d-cf92-11f0-8001-f2cd832f225c

2025-12-02 15:20:18,742 - distributed.scheduler - INFO - Receive client connection: Client-694b2a9d-cf92-11f0-8001-f2cd832f225c

2025-12-02 15:11:15,505 - distributed.scheduler - INFO - Close client connection: Client-9c571df9-cf8f-11f0-8001-f2cd832f225c

2025-12-02 15:11:15,504 - distributed.scheduler - INFO - Remove client Client-9c571df9-cf8f-11f0-8001-f2cd832f225c

2025-12-02 15:11:14,642 - distributed.scheduler - INFO - Close client connection: Client-02cb0531-cf91-11f0-8001-f2cd832f225c

2025-12-02 15:11:14,639 - distributed.scheduler - INFO - Remove client Client-02cb0531-cf91-11f0-8001-f2cd832f225c

2025-12-02 15:11:14,637 - distributed.scheduler - INFO - Remove client Client-02cb0531-cf91-11f0-8001-f2cd832f225c

2025-12-02 15:10:17,292 - distributed.scheduler - INFO - Receive client connection: Client-02cb0531-cf91-11f0-8001-f2cd832f225c

2025-12-02 15:00:15,899 - distributed.scheduler - INFO - Receive client connection: Client-9c571df9-cf8f-11f0-8001-f2cd832f225c

2025-12-02 14:51:09,332 - distributed.scheduler - INFO - Close client connection: Client-35d92fa9-cf8e-11f0-8001-f2cd832f225c

2025-12-02 14:51:09,332 - distributed.scheduler - INFO - Remove client Client-35d92fa9-cf8e-11f0-8001-f2cd832f225c

2025-12-02 14:51:09,331 - distributed.scheduler - INFO - Remove client Client-35d92fa9-cf8e-11f0-8001-f2cd832f225c

2025-12-02 14:50:14,441 - distributed.scheduler - INFO - Receive client connection: Client-35d92fa9-cf8e-11f0-8001-f2cd832f225c

2025-12-02 14:41:01,210 - distributed.scheduler - INFO - Close client connection: Client-cf598271-cf8c-11f0-8001-f2cd832f225c

2025-12-02 14:41:01,209 - distributed.scheduler - INFO - Remove client Client-cf598271-cf8c-11f0-8001-f2cd832f225c

2025-12-02 14:41:01,208 - distributed.scheduler - INFO - Remove client Client-cf598271-cf8c-11f0-8001-f2cd832f225c

2025-12-02 14:40:12,985 - distributed.scheduler - INFO - Receive client connection: Client-cf598271-cf8c-11f0-8001-f2cd832f225c

2025-12-02 14:31:38,952 - distributed.scheduler - INFO - Close client connection: Client-68dbe6f4-cf8b-11f0-8001-f2cd832f225c

2025-12-02 14:31:38,949 - distributed.scheduler - INFO - Remove client Client-68dbe6f4-cf8b-11f0-8001-f2cd832f225c

2025-12-02 14:31:38,948 - distributed.scheduler - INFO - Remove client Client-68dbe6f4-cf8b-11f0-8001-f2cd832f225c

2025-12-02 14:30:16,830 - distributed.scheduler - INFO - Receive client connection: Client-worker-6bfb3f81-cf8b-11f0-8660-ee8ad16902f8

2025-12-02 14:30:11,547 - distributed.scheduler - INFO - Receive client connection: Client-68dbe6f4-cf8b-11f0-8001-f2cd832f225c

2025-12-02 14:20:56,338 - distributed.scheduler - INFO - Close client connection: Client-025e4d7b-cf8a-11f0-8001-f2cd832f225c

2025-12-02 14:20:56,336 - distributed.scheduler - INFO - Remove client Client-025e4d7b-cf8a-11f0-8001-f2cd832f225c

2025-12-02 14:20:56,334 - distributed.scheduler - INFO - Remove client Client-025e4d7b-cf8a-11f0-8001-f2cd832f225c

2025-12-02 14:20:10,092 - distributed.scheduler - INFO - Receive client connection: Client-025e4d7b-cf8a-11f0-8001-f2cd832f225c

2025-12-02 14:10:54,993 - distributed.scheduler - INFO - Close client connection: Client-9be77946-cf88-11f0-8001-f2cd832f225c

2025-12-02 14:10:54,985 - distributed.scheduler - INFO - Remove client Client-9be77946-cf88-11f0-8001-f2cd832f225c

2025-12-02 14:10:54,984 - distributed.scheduler - INFO - Remove client Client-9be77946-cf88-11f0-8001-f2cd832f225c

2025-12-02 14:10:08,688 - distributed.scheduler - INFO - Receive client connection: Client-9be77946-cf88-11f0-8001-f2cd832f225c

2025-12-02 14:04:08,741 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:39537 after 30 s')

2025-12-02 14:04:08,361 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:39537 failed: OSError: Timed out trying to connect to tcp://172.21.12.165:39537 after 30 s

2025-12-02 14:04:08,352 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:39537 after 30 s')

2025-12-02 14:04:08,344 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:39537 after 30 s')

2025-12-02 14:04:08,336 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:39537 after 30 s')

2025-12-02 14:04:08,321 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:43399 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:39537 after 30 s')

2025-12-02 14:04:08,320 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:42943 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:39537 after 30 s')

2025-12-02 14:04:08,294 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:39537 after 30 s')

2025-12-02 14:04:08,278 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44619 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:39537 after 30 s')

2025-12-02 14:04:08,274 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:43201 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:39537 after 30 s')

2025-12-02 14:04:08,269 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:39537 after 30 s')

2025-12-02 14:04:08,228 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:39537 after 30 s')

2025-12-02 14:03:39,167 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:44981

2025-12-02 14:03:39,165 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:44981', status: closed, memory: 0, processing: 0>

2025-12-02 14:03:38,139 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:39537', status: closed, memory: 0, processing: 0>

2025-12-02 14:03:27,739 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37541 failed: OSError: Timed out trying to connect to tcp://172.21.12.165:37541 after 30 s

2025-12-02 14:03:27,704 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:42943 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37541 after 30 s')

2025-12-02 14:03:27,692 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37541 after 30 s')

2025-12-02 14:03:27,683 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37541 after 30 s')

2025-12-02 14:03:27,681 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37541 after 30 s')

2025-12-02 14:03:27,656 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37541 after 30 s')

2025-12-02 14:03:27,645 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37541 after 30 s')

2025-12-02 14:03:27,637 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37541 after 30 s')

2025-12-02 14:03:27,636 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37541 after 30 s')

2025-12-02 14:03:27,624 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:34871 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37541 after 30 s')

2025-12-02 14:02:58,531 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:39537

2025-12-02 14:02:58,530 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:39537', status: closed, memory: 0, processing: 0>

2025-12-02 14:02:57,532 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:37541', status: closed, memory: 0, processing: 0>

2025-12-02 14:02:45,593 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:38897 failed: OSError: Timed out trying to connect to tcp://172.21.12.165:38897 after 30 s

2025-12-02 14:02:45,554 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:43201 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:38897 after 30 s')

2025-12-02 14:02:15,263 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:37541

2025-12-02 14:02:15,262 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:37541', status: closed, memory: 0, processing: 0>

2025-12-02 14:02:14,253 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:38897', status: closed, memory: 0, processing: 0>

2025-12-02 14:01:46,938 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:45407 after 30 s')

2025-12-02 14:01:46,236 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:45407 failed: OSError: Timed out trying to connect to tcp://172.21.12.165:45407 after 30 s

2025-12-02 14:01:42,941 - distributed.scheduler - INFO - Close client connection: Client-cefe6829-cf85-11f0-8001-f2cd832f225c

2025-12-02 14:01:42,940 - distributed.scheduler - INFO - Remove client Client-cefe6829-cf85-11f0-8001-f2cd832f225c

2025-12-02 14:01:42,157 - distributed.scheduler - INFO - Close client connection: Client-357259fe-cf87-11f0-8001-f2cd832f225c

2025-12-02 14:01:42,157 - distributed.scheduler - INFO - Remove client Client-357259fe-cf87-11f0-8001-f2cd832f225c

2025-12-02 14:01:42,156 - distributed.scheduler - INFO - Remove client Client-357259fe-cf87-11f0-8001-f2cd832f225c

2025-12-02 14:01:16,410 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:38897

2025-12-02 14:01:16,409 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:38897', status: closed, memory: 0, processing: 0>

2025-12-02 14:01:15,356 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:45407', status: closed, memory: 0, processing: 0>

2025-12-02 14:00:47,930 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:45407

2025-12-02 14:00:47,928 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:45407', status: closed, memory: 0, processing: 0>

2025-12-02 14:00:46,983 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:40271', status: closed, memory: 0, processing: 0>

2025-12-02 14:00:07,293 - distributed.scheduler - INFO - Receive client connection: Client-357259fe-cf87-11f0-8001-f2cd832f225c

2025-12-02 13:50:05,917 - distributed.scheduler - INFO - Receive client connection: Client-cefe6829-cf85-11f0-8001-f2cd832f225c

2025-12-02 13:40:50,735 - distributed.scheduler - INFO - Close client connection: Client-68858c94-cf84-11f0-8001-f2cd832f225c

2025-12-02 13:40:50,725 - distributed.scheduler - INFO - Remove client Client-68858c94-cf84-11f0-8001-f2cd832f225c

2025-12-02 13:40:50,720 - distributed.scheduler - INFO - Remove client Client-68858c94-cf84-11f0-8001-f2cd832f225c

2025-12-02 13:40:04,497 - distributed.scheduler - INFO - Receive client connection: Client-68858c94-cf84-11f0-8001-f2cd832f225c

2025-12-02 13:31:18,176 - distributed.scheduler - INFO - Close client connection: Client-020ec36b-cf83-11f0-8001-f2cd832f225c

2025-12-02 13:31:18,175 - distributed.scheduler - INFO - Remove client Client-020ec36b-cf83-11f0-8001-f2cd832f225c

2025-12-02 13:31:18,173 - distributed.scheduler - INFO - Remove client Client-020ec36b-cf83-11f0-8001-f2cd832f225c

2025-12-02 13:30:07,458 - distributed.scheduler - INFO - Receive client connection: Client-worker-04a1c29c-cf83-11f0-88c7-f2a719d69604

2025-12-02 13:30:03,090 - distributed.scheduler - INFO - Receive client connection: Client-020ec36b-cf83-11f0-8001-f2cd832f225c

2025-12-02 13:20:46,536 - distributed.scheduler - INFO - Close client connection: Client-9ba0c899-cf81-11f0-8001-f2cd832f225c

2025-12-02 13:20:46,535 - distributed.scheduler - INFO - Remove client Client-9ba0c899-cf81-11f0-8001-f2cd832f225c

2025-12-02 13:20:46,535 - distributed.scheduler - INFO - Remove client Client-9ba0c899-cf81-11f0-8001-f2cd832f225c

2025-12-02 13:20:01,751 - distributed.scheduler - INFO - Receive client connection: Client-9ba0c899-cf81-11f0-8001-f2cd832f225c

2025-12-02 13:10:45,473 - distributed.scheduler - INFO - Close client connection: Client-352e3f62-cf80-11f0-8001-f2cd832f225c

2025-12-02 13:10:45,472 - distributed.scheduler - INFO - Remove client Client-352e3f62-cf80-11f0-8001-f2cd832f225c

2025-12-02 13:10:45,470 - distributed.scheduler - INFO - Remove client Client-352e3f62-cf80-11f0-8001-f2cd832f225c

2025-12-02 13:10:00,379 - distributed.scheduler - INFO - Receive client connection: Client-352e3f62-cf80-11f0-8001-f2cd832f225c

2025-12-02 13:02:09,857 - distributed.scheduler - INFO - Close client connection: Client-f28a3937-cf7e-11f0-8001-f2cd832f225c

2025-12-02 13:02:09,856 - distributed.scheduler - INFO - Remove client Client-f28a3937-cf7e-11f0-8001-f2cd832f225c

2025-12-02 13:02:09,854 - distributed.scheduler - INFO - Remove client Client-f28a3937-cf7e-11f0-8001-f2cd832f225c

2025-12-02 13:01:03,978 - distributed.scheduler - INFO - Receive client connection: Client-worker-f5796fcf-cf7e-11f0-8412-e2572cf160eb

2025-12-02 13:00:59,087 - distributed.scheduler - INFO - Receive client connection: Client-f28a3937-cf7e-11f0-8001-f2cd832f225c

2025-12-02 12:51:35,527 - distributed.scheduler - INFO - Close client connection: Client-8c1b70a0-cf7d-11f0-8001-f2cd832f225c

2025-12-02 12:51:35,527 - distributed.scheduler - INFO - Remove client Client-8c1b70a0-cf7d-11f0-8001-f2cd832f225c

2025-12-02 12:51:35,525 - distributed.scheduler - INFO - Remove client Client-8c1b70a0-cf7d-11f0-8001-f2cd832f225c

2025-12-02 12:50:57,728 - distributed.scheduler - INFO - Receive client connection: Client-8c1b70a0-cf7d-11f0-8001-f2cd832f225c

2025-12-02 12:41:43,551 - distributed.scheduler - INFO - Close client connection: Client-25b97a2a-cf7c-11f0-8001-f2cd832f225c

2025-12-02 12:41:43,551 - distributed.scheduler - INFO - Remove client Client-25b97a2a-cf7c-11f0-8001-f2cd832f225c

2025-12-02 12:41:43,550 - distributed.scheduler - INFO - Remove client Client-25b97a2a-cf7c-11f0-8001-f2cd832f225c

2025-12-02 12:40:56,451 - distributed.scheduler - INFO - Receive client connection: Client-25b97a2a-cf7c-11f0-8001-f2cd832f225c

2025-12-02 12:33:46,015 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:43201

2025-12-02 12:33:46,014 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:43201', status: closed, memory: 0, processing: 0>

2025-12-02 12:33:44,985 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:36153', status: closed, memory: 0, processing: 0>

2025-12-02 12:33:43,241 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:34871

2025-12-02 12:33:43,240 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:34871', status: closed, memory: 0, processing: 0>

2025-12-02 12:33:43,117 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:40271

2025-12-02 12:33:43,114 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:40271', status: closed, memory: 0, processing: 0>

2025-12-02 12:33:42,158 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:41397', status: closed, memory: 0, processing: 0>

2025-12-02 12:33:42,043 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:46515', status: closed, memory: 0, processing: 0>

2025-12-02 12:33:30,749 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:41655 failed: OSError: Timed out trying to connect to tcp://172.21.12.150:41655 after 30 s

2025-12-02 12:33:30,749 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:41445 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:41445 after 30 s

2025-12-02 12:33:30,748 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44619 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:41655 after 30 s')

2025-12-02 12:33:30,746 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:36783 failed: OSError: Timed out trying to connect to tcp://172.21.12.165:36783 after 30 s

2025-12-02 12:33:30,685 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:41655 after 30 s')

2025-12-02 12:33:30,677 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:36783 after 30 s')

2025-12-02 12:33:30,674 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:42943 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:41445 after 30 s')

2025-12-02 12:33:30,671 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:41655 after 30 s')

2025-12-02 12:33:30,650 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:40569 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:41445 after 30 s')

2025-12-02 12:33:01,802 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:46515

2025-12-02 12:33:01,800 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:46515', status: closed, memory: 0, processing: 0>

2025-12-02 12:33:01,506 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:36153

2025-12-02 12:33:01,504 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:36153', status: closed, memory: 0, processing: 0>

2025-12-02 12:33:01,431 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:41397

2025-12-02 12:33:01,429 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:41397', status: closed, memory: 0, processing: 0>

2025-12-02 12:33:00,536 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:41655', status: closed, memory: 0, processing: 0>

2025-12-02 12:33:00,532 - distributed.scheduler - INFO - Close client connection: Client-worker-fef3d47f-cf7a-11f0-88a4-f2a719d69604

2025-12-02 12:33:00,529 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:41445', status: closed, memory: 0, processing: 0>

2025-12-02 12:33:00,528 - distributed.scheduler - INFO - Remove client Client-worker-fef3d47f-cf7a-11f0-88a4-f2a719d69604

2025-12-02 12:33:00,474 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:36783', status: closed, memory: 0, processing: 0>

2025-12-02 12:32:41,903 - distributed.scheduler - INFO - Receive client connection: Client-worker-fef3d47f-cf7a-11f0-88a4-f2a719d69604

2025-12-02 12:32:39,226 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:41445

2025-12-02 12:32:39,225 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:41445', status: closed, memory: 0, processing: 0>

2025-12-02 12:32:38,836 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:41655

2025-12-02 12:32:38,834 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:41655', status: closed, memory: 0, processing: 0>

2025-12-02 12:32:38,334 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:35817', status: closed, memory: 0, processing: 0>

2025-12-02 12:32:37,953 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:36783

2025-12-02 12:32:37,952 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:36783', status: closed, memory: 0, processing: 0>

2025-12-02 12:32:37,887 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:42667', status: closed, memory: 0, processing: 0>

2025-12-02 12:32:37,026 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:37193', status: closed, memory: 0, processing: 0>

2025-12-02 12:32:18,552 - distributed.scheduler - INFO - Close client connection: Client-bf40db22-cf7a-11f0-8001-f2cd832f225c

2025-12-02 12:32:18,546 - distributed.scheduler - INFO - Remove client Client-bf40db22-cf7a-11f0-8001-f2cd832f225c

2025-12-02 12:32:18,545 - distributed.scheduler - INFO - Remove client Client-bf40db22-cf7a-11f0-8001-f2cd832f225c

2025-12-02 12:32:02,346 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37233 after 30 s')

2025-12-02 12:32:02,158 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:46147 after 30 s')

2025-12-02 12:32:01,838 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: OSError: Timed out trying to connect to tcp://172.21.12.165:37233 after 30 s

2025-12-02 12:32:01,838 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:46147 failed: OSError: Timed out trying to connect to tcp://172.21.12.150:46147 after 30 s

2025-12-02 12:32:01,837 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:44731 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:44731 after 30 s

2025-12-02 12:32:01,659 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:46147 after 30 s')

2025-12-02 12:31:32,044 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:37193

2025-12-02 12:31:32,042 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:37193', status: closed, memory: 0, processing: 0>

2025-12-02 12:31:31,551 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:35817

2025-12-02 12:31:31,549 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:35817', status: closed, memory: 0, processing: 0>

2025-12-02 12:31:31,079 - distributed.scheduler - INFO - Close client connection: Client-worker-65a2f2e1-cb6b-11f0-83d5-ee8ad16902f8

2025-12-02 12:31:31,064 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:37233', status: closed, memory: 0, processing: 0>

2025-12-02 12:31:31,063 - distributed.scheduler - INFO - Remove client Client-worker-65a2f2e1-cb6b-11f0-83d5-ee8ad16902f8

2025-12-02 12:31:31,058 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:42667

2025-12-02 12:31:31,057 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:42667', status: closed, memory: 0, processing: 0>

2025-12-02 12:31:30,588 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:44731 failed: OSError: Timed out during handshake while connecting to tcp://172.21.25.101:44731 after 30 s

2025-12-02 12:31:30,585 - distributed.scheduler - INFO - Close client connection: Client-worker-f4e578e5-cf69-11f0-8840-f2a719d69604

2025-12-02 12:31:30,582 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:44731', status: closed, memory: 0, processing: 0>

2025-12-02 12:31:30,560 - distributed.scheduler - INFO - Remove client Client-worker-f4e578e5-cf69-11f0-8840-f2a719d69604

2025-12-02 12:31:30,259 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: RuntimeError('Worker tcp://172.21.12.150:46147 left during active shuffle 730c1097d9ab95472438ec8b6897ebaa')

2025-12-02 12:31:30,148 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:46147 failed: OSError: Timed out during handshake while connecting to tcp://172.21.12.150:46147 after 30 s

2025-12-02 12:31:30,144 - distributed.scheduler - INFO - Close client connection: Client-worker-529dc737-c95b-11f0-82a8-e2572cf160eb

2025-12-02 12:31:30,133 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:46147', status: closed, memory: 0, processing: 0>

2025-12-02 12:31:30,128 - distributed.scheduler - INFO - Remove client Client-worker-529dc737-c95b-11f0-82a8-e2572cf160eb

2025-12-02 12:30:55,035 - distributed.scheduler - INFO - Receive client connection: Client-bf40db22-cf7a-11f0-8001-f2cd832f225c

2025-12-02 12:21:33,204 - distributed.scheduler - INFO - Close client connection: Client-58cc6d62-cf79-11f0-8001-f2cd832f225c

2025-12-02 12:21:33,202 - distributed.scheduler - INFO - Remove client Client-58cc6d62-cf79-11f0-8001-f2cd832f225c

2025-12-02 12:21:33,200 - distributed.scheduler - INFO - Remove client Client-58cc6d62-cf79-11f0-8001-f2cd832f225c

2025-12-02 12:20:53,655 - distributed.scheduler - INFO - Receive client connection: Client-58cc6d62-cf79-11f0-8001-f2cd832f225c

2025-12-02 12:11:30,890 - distributed.scheduler - INFO - Close client connection: Client-f24c1b3f-cf77-11f0-8001-f2cd832f225c

2025-12-02 12:11:30,889 - distributed.scheduler - INFO - Remove client Client-f24c1b3f-cf77-11f0-8001-f2cd832f225c

2025-12-02 12:11:30,888 - distributed.scheduler - INFO - Remove client Client-f24c1b3f-cf77-11f0-8001-f2cd832f225c

2025-12-02 12:10:52,197 - distributed.scheduler - INFO - Receive client connection: Client-f24c1b3f-cf77-11f0-8001-f2cd832f225c

2025-12-02 12:03:21,784 - distributed.scheduler - INFO - Close client connection: Client-8bcafbd8-cf76-11f0-8001-f2cd832f225c

2025-12-02 12:03:21,782 - distributed.scheduler - INFO - Remove client Client-8bcafbd8-cf76-11f0-8001-f2cd832f225c

2025-12-02 12:03:21,781 - distributed.scheduler - INFO - Remove client Client-8bcafbd8-cf76-11f0-8001-f2cd832f225c

2025-12-02 12:00:50,731 - distributed.scheduler - INFO - Receive client connection: Client-8bcafbd8-cf76-11f0-8001-f2cd832f225c

2025-12-02 11:51:29,310 - distributed.scheduler - INFO - Close client connection: Client-25485b17-cf75-11f0-8001-f2cd832f225c

2025-12-02 11:51:29,309 - distributed.scheduler - INFO - Remove client Client-25485b17-cf75-11f0-8001-f2cd832f225c

2025-12-02 11:51:29,307 - distributed.scheduler - INFO - Remove client Client-25485b17-cf75-11f0-8001-f2cd832f225c

2025-12-02 11:50:49,235 - distributed.scheduler - INFO - Receive client connection: Client-25485b17-cf75-11f0-8001-f2cd832f225c

2025-12-02 11:41:28,370 - distributed.scheduler - INFO - Close client connection: Client-bed65479-cf73-11f0-8001-f2cd832f225c

2025-12-02 11:41:28,370 - distributed.scheduler - INFO - Remove client Client-bed65479-cf73-11f0-8001-f2cd832f225c

2025-12-02 11:41:28,368 - distributed.scheduler - INFO - Remove client Client-bed65479-cf73-11f0-8001-f2cd832f225c

2025-12-02 11:40:47,906 - distributed.scheduler - INFO - Receive client connection: Client-bed65479-cf73-11f0-8001-f2cd832f225c

2025-12-02 11:31:59,469 - distributed.scheduler - INFO - Close client connection: Client-585f4963-cf72-11f0-8001-f2cd832f225c

2025-12-02 11:31:59,467 - distributed.scheduler - INFO - Remove client Client-585f4963-cf72-11f0-8001-f2cd832f225c

2025-12-02 11:31:59,465 - distributed.scheduler - INFO - Remove client Client-585f4963-cf72-11f0-8001-f2cd832f225c

2025-12-02 11:30:46,466 - distributed.scheduler - INFO - Receive client connection: Client-585f4963-cf72-11f0-8001-f2cd832f225c

2025-12-02 11:21:23,752 - distributed.scheduler - INFO - Close client connection: Client-57f6861d-cf6b-11f0-8001-f2cd832f225c

2025-12-02 11:21:23,752 - distributed.scheduler - INFO - Close client connection: Client-be75d549-cf6c-11f0-8001-f2cd832f225c

2025-12-02 11:21:23,751 - distributed.scheduler - INFO - Close client connection: Client-24eb5c0b-cf6e-11f0-8001-f2cd832f225c

2025-12-02 11:21:23,751 - distributed.scheduler - INFO - Close client connection: Client-8b5ab791-cf6f-11f0-8001-f2cd832f225c

2025-12-02 11:21:23,750 - distributed.scheduler - INFO - Remove client Client-57f6861d-cf6b-11f0-8001-f2cd832f225c

2025-12-02 11:21:23,749 - distributed.scheduler - INFO - Remove client Client-be75d549-cf6c-11f0-8001-f2cd832f225c

2025-12-02 11:21:23,747 - distributed.scheduler - INFO - Remove client Client-24eb5c0b-cf6e-11f0-8001-f2cd832f225c

2025-12-02 11:21:23,746 - distributed.scheduler - INFO - Remove client Client-8b5ab791-cf6f-11f0-8001-f2cd832f225c

2025-12-02 11:21:23,059 - distributed.scheduler - INFO - Close client connection: Client-f1e41141-cf70-11f0-8001-f2cd832f225c

2025-12-02 11:21:23,058 - distributed.scheduler - INFO - Remove client Client-f1e41141-cf70-11f0-8001-f2cd832f225c

2025-12-02 11:21:23,056 - distributed.scheduler - INFO - Remove client Client-f1e41141-cf70-11f0-8001-f2cd832f225c

2025-12-02 11:20:45,022 - distributed.scheduler - INFO - Receive client connection: Client-f1e41141-cf70-11f0-8001-f2cd832f225c

2025-12-02 11:10:43,495 - distributed.scheduler - INFO - Receive client connection: Client-8b5ab791-cf6f-11f0-8001-f2cd832f225c

2025-12-02 11:00:42,147 - distributed.scheduler - INFO - Receive client connection: Client-24eb5c0b-cf6e-11f0-8001-f2cd832f225c

2025-12-02 10:50:40,745 - distributed.scheduler - INFO - Receive client connection: Client-be75d549-cf6c-11f0-8001-f2cd832f225c

2025-12-02 10:40:39,293 - distributed.scheduler - INFO - Receive client connection: Client-57f6861d-cf6b-11f0-8001-f2cd832f225c

2025-12-02 10:31:49,432 - distributed.scheduler - INFO - Close client connection: Client-f1770661-cf69-11f0-8001-f2cd832f225c

2025-12-02 10:31:49,431 - distributed.scheduler - INFO - Remove client Client-f1770661-cf69-11f0-8001-f2cd832f225c

2025-12-02 10:31:49,429 - distributed.scheduler - INFO - Remove client Client-f1770661-cf69-11f0-8001-f2cd832f225c

2025-12-02 10:30:43,604 - distributed.scheduler - INFO - Receive client connection: Client-worker-f4e578e5-cf69-11f0-8840-f2a719d69604

2025-12-02 10:30:37,828 - distributed.scheduler - INFO - Receive client connection: Client-f1770661-cf69-11f0-8001-f2cd832f225c

2025-12-02 10:21:14,498 - distributed.scheduler - INFO - Close client connection: Client-8af91184-cf68-11f0-8001-f2cd832f225c

2025-12-02 10:21:14,497 - distributed.scheduler - INFO - Remove client Client-8af91184-cf68-11f0-8001-f2cd832f225c

2025-12-02 10:21:14,496 - distributed.scheduler - INFO - Remove client Client-8af91184-cf68-11f0-8001-f2cd832f225c

2025-12-02 10:20:36,388 - distributed.scheduler - INFO - Receive client connection: Client-8af91184-cf68-11f0-8001-f2cd832f225c

2025-12-02 10:11:12,467 - distributed.scheduler - INFO - Close client connection: Client-24759335-cf67-11f0-8001-f2cd832f225c

2025-12-02 10:11:12,465 - distributed.scheduler - INFO - Remove client Client-24759335-cf67-11f0-8001-f2cd832f225c

2025-12-02 10:11:12,463 - distributed.scheduler - INFO - Remove client Client-24759335-cf67-11f0-8001-f2cd832f225c

2025-12-02 10:10:34,912 - distributed.scheduler - INFO - Receive client connection: Client-24759335-cf67-11f0-8001-f2cd832f225c

2025-12-02 10:01:41,928 - distributed.scheduler - INFO - Close client connection: Client-bdf89cd4-cf65-11f0-8001-f2cd832f225c

2025-12-02 10:01:41,927 - distributed.scheduler - INFO - Remove client Client-bdf89cd4-cf65-11f0-8001-f2cd832f225c

2025-12-02 10:01:41,926 - distributed.scheduler - INFO - Remove client Client-bdf89cd4-cf65-11f0-8001-f2cd832f225c

2025-12-02 10:00:33,463 - distributed.scheduler - INFO - Receive client connection: Client-bdf89cd4-cf65-11f0-8001-f2cd832f225c

2025-12-02 09:51:09,393 - distributed.scheduler - INFO - Close client connection: Client-57868fd3-cf64-11f0-8001-f2cd832f225c

2025-12-02 09:51:09,392 - distributed.scheduler - INFO - Remove client Client-57868fd3-cf64-11f0-8001-f2cd832f225c

2025-12-02 09:51:09,390 - distributed.scheduler - INFO - Remove client Client-57868fd3-cf64-11f0-8001-f2cd832f225c

2025-12-02 09:50:32,088 - distributed.scheduler - INFO - Receive client connection: Client-57868fd3-cf64-11f0-8001-f2cd832f225c

2025-12-02 09:41:11,029 - distributed.scheduler - INFO - Close client connection: Client-f10aa3a6-cf62-11f0-8001-f2cd832f225c

2025-12-02 09:41:11,028 - distributed.scheduler - INFO - Remove client Client-f10aa3a6-cf62-11f0-8001-f2cd832f225c

2025-12-02 09:41:11,027 - distributed.scheduler - INFO - Remove client Client-f10aa3a6-cf62-11f0-8001-f2cd832f225c

2025-12-02 09:40:30,672 - distributed.scheduler - INFO - Receive client connection: Client-f10aa3a6-cf62-11f0-8001-f2cd832f225c

2025-12-02 09:31:40,463 - distributed.scheduler - INFO - Close client connection: Client-8a939314-cf61-11f0-8001-f2cd832f225c

2025-12-02 09:31:40,462 - distributed.scheduler - INFO - Remove client Client-8a939314-cf61-11f0-8001-f2cd832f225c

2025-12-02 09:31:40,460 - distributed.scheduler - INFO - Remove client Client-8a939314-cf61-11f0-8001-f2cd832f225c

2025-12-02 09:30:35,111 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('len-agg-f160d2eabf62bc04f46839381a45c4bb', 0) NoneType: None

2025-12-02 09:30:35,111 - distributed.scheduler - ERROR - Couldn't gather keys {"('len-agg-f160d2eabf62bc04f46839381a45c4bb', 0)": []} state: ['waiting'] workers: [] NoneType: None

2025-12-02 09:30:34,689 - distributed.scheduler - INFO - Receive client connection: Client-worker-8dcd000f-cf61-11f0-889e-8253c573df33

2025-12-02 09:30:29,238 - distributed.scheduler - INFO - Receive client connection: Client-8a939314-cf61-11f0-8001-f2cd832f225c

2025-12-02 09:21:03,339 - distributed.scheduler - INFO - Close client connection: Client-241086e0-cf60-11f0-8001-f2cd832f225c

2025-12-02 09:21:03,338 - distributed.scheduler - INFO - Remove client Client-241086e0-cf60-11f0-8001-f2cd832f225c

2025-12-02 09:21:03,336 - distributed.scheduler - INFO - Remove client Client-241086e0-cf60-11f0-8001-f2cd832f225c

2025-12-02 09:20:27,765 - distributed.scheduler - INFO - Receive client connection: Client-241086e0-cf60-11f0-8001-f2cd832f225c

2025-12-02 09:11:03,892 - distributed.scheduler - INFO - Close client connection: Client-bdaad2c7-cf5e-11f0-8001-f2cd832f225c

2025-12-02 09:11:03,891 - distributed.scheduler - INFO - Remove client Client-bdaad2c7-cf5e-11f0-8001-f2cd832f225c

2025-12-02 09:11:03,889 - distributed.scheduler - INFO - Remove client Client-bdaad2c7-cf5e-11f0-8001-f2cd832f225c

2025-12-02 09:10:26,477 - distributed.scheduler - INFO - Receive client connection: Client-bdaad2c7-cf5e-11f0-8001-f2cd832f225c

2025-12-02 09:02:27,702 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:40391

2025-12-02 09:02:27,701 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:40391', status: closed, memory: 0, processing: 0>

2025-12-02 09:02:26,652 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:38733', status: closed, memory: 0, processing: 0>

2025-12-02 09:02:26,170 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:44731

2025-12-02 09:02:26,167 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:44731', status: closed, memory: 0, processing: 0>

2025-12-02 09:02:25,030 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:42635', status: closed, memory: 0, processing: 0>

2025-12-02 09:02:16,520 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:38733

2025-12-02 09:02:16,518 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:38733', status: closed, memory: 0, processing: 0>

2025-12-02 09:02:15,740 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:33593', status: closed, memory: 0, processing: 0>

2025-12-02 09:02:14,021 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:42635

2025-12-02 09:02:14,019 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:42635', status: closed, memory: 0, processing: 0>

2025-12-02 09:02:13,030 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:38101', status: closed, memory: 0, processing: 0>

2025-12-02 09:02:03,833 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:33593

2025-12-02 09:02:03,832 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:33593', status: closed, memory: 0, processing: 0>

2025-12-02 09:02:03,033 - distributed.scheduler - INFO - Close client connection: Client-worker-2818a326-cf51-11f0-8837-8253c573df33

2025-12-02 09:02:03,026 - distributed.scheduler - INFO - Remove client Client-worker-2818a326-cf51-11f0-8837-8253c573df33

2025-12-02 09:02:03,024 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:38359', status: closed, memory: 0, processing: 0>

2025-12-02 09:02:02,175 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:38101

2025-12-02 09:02:02,173 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:38101', status: closed, memory: 0, processing: 0>

2025-12-02 09:02:01,142 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:33435', status: closed, memory: 0, processing: 0>

2025-12-02 09:01:51,999 - distributed.scheduler - INFO - Close client connection: Client-573427f8-cf5d-11f0-8001-f2cd832f225c

2025-12-02 09:01:51,997 - distributed.scheduler - INFO - Remove client Client-573427f8-cf5d-11f0-8001-f2cd832f225c

2025-12-02 09:01:51,996 - distributed.scheduler - INFO - Remove client Client-573427f8-cf5d-11f0-8001-f2cd832f225c

2025-12-02 09:01:33,105 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:33435

2025-12-02 09:01:33,103 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:33435', status: closed, memory: 0, processing: 0>

2025-12-02 09:01:31,958 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:40717', status: closed, memory: 0, processing: 0>

2025-12-02 09:00:58,797 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:40717

2025-12-02 09:00:58,796 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:40717', status: closed, memory: 0, processing: 0>

2025-12-02 09:00:58,729 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1717) NoneType: None

2025-12-02 09:00:58,729 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 944) NoneType: None

2025-12-02 09:00:58,729 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1534) NoneType: None

2025-12-02 09:00:58,729 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1295) NoneType: None

2025-12-02 09:00:58,729 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 334) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1885) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1557) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2109) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1197) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2144) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 159) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1843) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1343) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 829) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2221) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2123) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 211) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 245) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1610) NoneType: None

2025-12-02 09:00:58,728 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 446) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1070) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1440) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 706) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1264) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1765) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2270) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 605) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 861) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 998) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1636) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1284) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 888) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 432) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1669) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 974) NoneType: None

2025-12-02 09:00:58,727 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1821) NoneType: None

2025-12-02 09:00:58,726 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 158) NoneType: None

2025-12-02 09:00:58,726 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 993) NoneType: None

2025-12-02 09:00:58,726 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 595) NoneType: None

2025-12-02 09:00:58,726 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1405) NoneType: None

2025-12-02 09:00:58,726 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 478) NoneType: None

2025-12-02 09:00:58,726 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 468) NoneType: None

2025-12-02 09:00:58,726 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2113) NoneType: None

2025-12-02 09:00:58,726 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 900) NoneType: None

2025-12-02 09:00:58,726 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1501) NoneType: None

2025-12-02 09:00:58,726 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1336) NoneType: None

2025-12-02 09:00:58,726 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 429) NoneType: None

2025-12-02 09:00:58,725 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 365) NoneType: None

2025-12-02 09:00:58,725 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1598) NoneType: None

2025-12-02 09:00:58,725 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 142) NoneType: None

2025-12-02 09:00:58,725 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 486) NoneType: None

2025-12-02 09:00:58,725 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2229) NoneType: None

2025-12-02 09:00:58,725 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 336) NoneType: None

2025-12-02 09:00:58,725 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 109) NoneType: None

2025-12-02 09:00:58,725 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1614) NoneType: None

2025-12-02 09:00:58,725 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1232) NoneType: None

2025-12-02 09:00:58,725 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 397) NoneType: None

2025-12-02 09:00:58,724 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 805) NoneType: None

2025-12-02 09:00:58,724 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 30) NoneType: None

2025-12-02 09:00:58,724 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1781) NoneType: None

2025-12-02 09:00:58,724 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 640) NoneType: None

2025-12-02 09:00:58,724 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1792) NoneType: None

2025-12-02 09:00:58,724 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1429) NoneType: None

2025-12-02 09:00:58,724 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1028) NoneType: None

2025-12-02 09:00:58,724 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 302) NoneType: None

2025-12-02 09:00:58,724 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 557) NoneType: None

2025-12-02 09:00:58,724 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1797) NoneType: None

2025-12-02 09:00:58,723 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2069) NoneType: None

2025-12-02 09:00:58,723 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 21) NoneType: None

2025-12-02 09:00:58,723 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1485) NoneType: None

2025-12-02 09:00:58,723 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1728) NoneType: None

2025-12-02 09:00:58,723 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 655) NoneType: None

2025-12-02 09:00:58,723 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2080) NoneType: None

2025-12-02 09:00:58,723 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1934) NoneType: None

2025-12-02 09:00:58,723 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1587) NoneType: None

2025-12-02 09:00:58,723 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 61) NoneType: None

2025-12-02 09:00:58,723 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 515) NoneType: None

2025-12-02 09:00:58,723 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2063) NoneType: None

2025-12-02 09:00:58,722 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2064) NoneType: None

2025-12-02 09:00:58,722 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1053) NoneType: None

2025-12-02 09:00:58,722 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 931) NoneType: None

2025-12-02 09:00:58,722 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 800) NoneType: None

2025-12-02 09:00:58,722 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 662) NoneType: None

2025-12-02 09:00:58,722 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 947) NoneType: None

2025-12-02 09:00:58,722 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1607) NoneType: None

2025-12-02 09:00:58,722 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 69) NoneType: None

2025-12-02 09:00:58,721 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1157) NoneType: None

2025-12-02 09:00:58,721 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1064) NoneType: None

2025-12-02 09:00:58,721 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1149) NoneType: None

2025-12-02 09:00:58,721 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2021) NoneType: None

2025-12-02 09:00:58,721 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1550) NoneType: None

2025-12-02 09:00:58,721 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1117) NoneType: None

2025-12-02 09:00:58,721 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2285) NoneType: None

2025-12-02 09:00:58,721 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2141) NoneType: None

2025-12-02 09:00:58,721 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 541) NoneType: None

2025-12-02 09:00:58,721 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1874) NoneType: None

2025-12-02 09:00:58,720 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 896) NoneType: None

2025-12-02 09:00:58,720 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1365) NoneType: None

2025-12-02 09:00:58,720 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1585) NoneType: None

2025-12-02 09:00:58,720 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1581) NoneType: None

2025-12-02 09:00:58,720 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1214) NoneType: None

2025-12-02 09:00:58,720 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 793) NoneType: None

2025-12-02 09:00:58,720 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 319) NoneType: None

2025-12-02 09:00:58,720 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2206) NoneType: None

2025-12-02 09:00:58,720 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 917) NoneType: None

2025-12-02 09:00:58,719 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1541) NoneType: None

2025-12-02 09:00:58,719 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 514) NoneType: None

2025-12-02 09:00:58,719 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1086) NoneType: None

2025-12-02 09:00:58,719 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 406) NoneType: None

2025-12-02 09:00:58,719 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1251) NoneType: None

2025-12-02 09:00:58,719 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 437) NoneType: None

2025-12-02 09:00:58,719 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2255) NoneType: None

2025-12-02 09:00:58,719 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1998) NoneType: None

2025-12-02 09:00:58,719 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1696) NoneType: None

2025-12-02 09:00:58,718 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1646) NoneType: None

2025-12-02 09:00:58,718 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1686) NoneType: None

2025-12-02 09:00:58,718 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 693) NoneType: None

2025-12-02 09:00:58,718 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1982) NoneType: None

2025-12-02 09:00:58,718 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 238) NoneType: None

2025-12-02 09:00:58,718 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1390) NoneType: None

2025-12-02 09:00:58,718 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 789) NoneType: None

2025-12-02 09:00:58,718 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2244) NoneType: None

2025-12-02 09:00:58,718 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1510) NoneType: None

2025-12-02 09:00:58,718 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 989) NoneType: None

2025-12-02 09:00:58,717 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 850) NoneType: None

2025-12-02 09:00:58,717 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 930) NoneType: None

2025-12-02 09:00:58,717 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 897) NoneType: None

2025-12-02 09:00:58,717 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1757) NoneType: None

2025-12-02 09:00:58,717 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 374) NoneType: None

2025-12-02 09:00:58,717 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 654) NoneType: None

2025-12-02 09:00:58,717 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 911) NoneType: None

2025-12-02 09:00:58,717 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1237) NoneType: None

2025-12-02 09:00:58,717 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2149) NoneType: None

2025-12-02 09:00:58,716 - distributed.scheduler - ERROR - Couldn't gather keys {"('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2149)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1237)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 911)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 654)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 374)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1757)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 897)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 930)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 850)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 989)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1510)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2244)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 789)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1390)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 238)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1982)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 693)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1686)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1646)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1696)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1998)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2255)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 437)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1251)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 406)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1086)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 514)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1541)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 917)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2206)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 319)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 793)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1214)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1581)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1585)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1365)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 896)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1874)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 541)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2141)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2285)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1117)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1550)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2021)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1149)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1064)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1157)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 69)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1607)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 947)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 662)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 800)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 931)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1053)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2064)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2063)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 515)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 61)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1587)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1934)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2080)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 655)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1728)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1485)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 21)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2069)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1797)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 557)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 302)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1028)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1429)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1792)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 640)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1781)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 30)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 805)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 397)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1232)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1614)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 109)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 336)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2229)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 486)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 142)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1598)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 365)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 429)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1336)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1501)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 900)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2113)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 468)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 478)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1405)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 595)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 993)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 158)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1821)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 974)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1669)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 432)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 888)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1284)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1636)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 998)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 861)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 605)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2270)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1765)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1264)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 706)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1440)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1070)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 446)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1610)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 245)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 211)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2123)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2221)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 829)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1343)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1843)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 159)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2144)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1197)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 2109)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1557)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1885)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 334)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1295)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1534)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 944)": [], "('query-3aedb89f4f4f364f10a9b63d15c0a9ec', 1717)": []} state: ['processing', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'processing', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'memory', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'memory', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'memory', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'memory', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'processing', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'memory', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'waiting', 'processing'] workers: [] NoneType: None

2025-12-02 09:00:57,762 - distributed.scheduler - INFO - Close client connection: Client-worker-32cafad5-cd34-11f0-866e-f2a719d69604

2025-12-02 09:00:57,731 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:36229', status: closed, memory: 0, processing: 0>

2025-12-02 09:00:57,730 - distributed.scheduler - INFO - Remove client Client-worker-32cafad5-cd34-11f0-866e-f2a719d69604

2025-12-02 09:00:25,062 - distributed.scheduler - INFO - Receive client connection: Client-573427f8-cf5d-11f0-8001-f2cd832f225c

2025-12-02 08:50:59,455 - distributed.scheduler - INFO - Close client connection: Client-bd2c708f-cf57-11f0-8001-f2cd832f225c

2025-12-02 08:50:59,454 - distributed.scheduler - INFO - Close client connection: Client-56aaca1e-cf56-11f0-8001-f2cd832f225c

2025-12-02 08:50:59,454 - distributed.scheduler - INFO - Close client connection: Client-23afe28a-cf59-11f0-8001-f2cd832f225c

2025-12-02 08:50:59,453 - distributed.scheduler - INFO - Close client connection: Client-8a2faf5a-cf5a-11f0-8001-f2cd832f225c

2025-12-02 08:50:59,452 - distributed.scheduler - INFO - Remove client Client-bd2c708f-cf57-11f0-8001-f2cd832f225c

2025-12-02 08:50:59,451 - distributed.scheduler - INFO - Remove client Client-56aaca1e-cf56-11f0-8001-f2cd832f225c

2025-12-02 08:50:59,450 - distributed.scheduler - INFO - Remove client Client-23afe28a-cf59-11f0-8001-f2cd832f225c

2025-12-02 08:50:59,449 - distributed.scheduler - INFO - Remove client Client-8a2faf5a-cf5a-11f0-8001-f2cd832f225c

2025-12-02 08:50:58,832 - distributed.scheduler - INFO - Close client connection: Client-f0b55445-cf5b-11f0-8001-f2cd832f225c

2025-12-02 08:50:58,830 - distributed.scheduler - INFO - Remove client Client-f0b55445-cf5b-11f0-8001-f2cd832f225c

2025-12-02 08:50:58,829 - distributed.scheduler - INFO - Remove client Client-f0b55445-cf5b-11f0-8001-f2cd832f225c

2025-12-02 08:50:23,607 - distributed.scheduler - INFO - Receive client connection: Client-f0b55445-cf5b-11f0-8001-f2cd832f225c

2025-12-02 08:40:22,116 - distributed.scheduler - INFO - Receive client connection: Client-8a2faf5a-cf5a-11f0-8001-f2cd832f225c

2025-12-02 08:30:20,670 - distributed.scheduler - INFO - Receive client connection: Client-23afe28a-cf59-11f0-8001-f2cd832f225c

2025-12-02 08:20:19,157 - distributed.scheduler - INFO - Receive client connection: Client-bd2c708f-cf57-11f0-8001-f2cd832f225c

2025-12-02 08:10:17,698 - distributed.scheduler - INFO - Receive client connection: Client-56aaca1e-cf56-11f0-8001-f2cd832f225c

2025-12-02 08:01:33,545 - distributed.scheduler - INFO - Close client connection: Client-f024a60d-cf54-11f0-8001-f2cd832f225c

2025-12-02 08:01:33,544 - distributed.scheduler - INFO - Remove client Client-f024a60d-cf54-11f0-8001-f2cd832f225c

2025-12-02 08:01:33,542 - distributed.scheduler - INFO - Remove client Client-f024a60d-cf54-11f0-8001-f2cd832f225c

2025-12-02 08:00:16,190 - distributed.scheduler - INFO - Receive client connection: Client-f024a60d-cf54-11f0-8001-f2cd832f225c

2025-12-02 07:50:53,479 - distributed.scheduler - INFO - Close client connection: Client-899ae7c9-cf53-11f0-8001-f2cd832f225c

2025-12-02 07:50:53,477 - distributed.scheduler - INFO - Remove client Client-899ae7c9-cf53-11f0-8001-f2cd832f225c

2025-12-02 07:50:53,476 - distributed.scheduler - INFO - Remove client Client-899ae7c9-cf53-11f0-8001-f2cd832f225c

2025-12-02 07:50:14,696 - distributed.scheduler - INFO - Receive client connection: Client-899ae7c9-cf53-11f0-8001-f2cd832f225c

2025-12-02 07:40:54,879 - distributed.scheduler - INFO - Close client connection: Client-2316b2dc-cf52-11f0-8001-f2cd832f225c

2025-12-02 07:40:54,879 - distributed.scheduler - INFO - Remove client Client-2316b2dc-cf52-11f0-8001-f2cd832f225c

2025-12-02 07:40:54,878 - distributed.scheduler - INFO - Remove client Client-2316b2dc-cf52-11f0-8001-f2cd832f225c

2025-12-02 07:40:13,163 - distributed.scheduler - INFO - Receive client connection: Client-2316b2dc-cf52-11f0-8001-f2cd832f225c

2025-12-02 07:33:12,071 - distributed.scheduler - INFO - Receive client connection: Client-worker-2818a326-cf51-11f0-8837-8253c573df33

2025-12-02 07:33:07,690 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:32855 failed: OSError: Timed out trying to connect to tcp://172.21.159.252:32855 after 30 s

2025-12-02 07:33:07,630 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:32855 after 30 s')

2025-12-02 07:33:07,579 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:32855 after 30 s')

2025-12-02 07:33:07,576 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:32855 after 30 s')

2025-12-02 07:33:07,574 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:36229 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:32855 after 30 s')

2025-12-02 07:33:07,564 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:32855 after 30 s')

2025-12-02 07:32:38,251 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:38359

2025-12-02 07:32:38,249 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:38359', status: closed, memory: 0, processing: 0>

2025-12-02 07:32:37,435 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:32855', status: closed, memory: 0, processing: 0>

2025-12-02 07:32:09,405 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:43399 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:39395 after 30 s')

2025-12-02 07:32:09,387 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:39395 failed: OSError: Timed out trying to connect to tcp://172.21.159.252:39395 after 30 s

2025-12-02 07:32:09,304 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:39395 after 30 s')

2025-12-02 07:32:09,244 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:39395 after 30 s')

2025-12-02 07:32:09,235 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:39395 after 30 s')

2025-12-02 07:32:09,212 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:39395 after 30 s')

2025-12-02 07:32:09,199 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:42943 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:39395 after 30 s')

2025-12-02 07:32:09,198 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:39395 after 30 s')

2025-12-02 07:32:09,195 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:42075 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:39395 after 30 s')

2025-12-02 07:32:09,177 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:36229 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:39395 after 30 s')

2025-12-02 07:31:39,975 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:32855

2025-12-02 07:31:39,972 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:32855', status: closed, memory: 0, processing: 0>

2025-12-02 07:31:39,046 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:39395', status: closed, memory: 0, processing: 0>

2025-12-02 07:31:32,153 - distributed.scheduler - INFO - Close client connection: Client-bc9a3a05-cf50-11f0-8001-f2cd832f225c

2025-12-02 07:31:32,153 - distributed.scheduler - INFO - Remove client Client-bc9a3a05-cf50-11f0-8001-f2cd832f225c

2025-12-02 07:31:32,152 - distributed.scheduler - INFO - Remove client Client-bc9a3a05-cf50-11f0-8001-f2cd832f225c

2025-12-02 07:31:18,234 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41957 failed: OSError: Timed out trying to connect to tcp://172.21.159.252:41957 after 30 s

2025-12-02 07:31:18,010 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:43399 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:41957 after 30 s')

2025-12-02 07:31:17,856 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:41957 after 30 s')

2025-12-02 07:31:17,761 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:41957 after 30 s')

2025-12-02 07:30:47,989 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:39395

2025-12-02 07:30:47,987 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:39395', status: closed, memory: 0, processing: 0>

2025-12-02 07:30:47,169 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41957 failed: OSError: Timed out during handshake while connecting to tcp://172.21.159.252:41957 after 30 s

2025-12-02 07:30:47,169 - distributed.scheduler - INFO - Close client connection: Client-worker-a9807ff2-cd40-11f0-86e2-8253c573df33

2025-12-02 07:30:47,154 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:41957', status: closed, memory: 0, processing: 0>

2025-12-02 07:30:47,154 - distributed.scheduler - INFO - Remove client Client-worker-a9807ff2-cd40-11f0-86e2-8253c573df33

2025-12-02 07:30:11,728 - distributed.scheduler - INFO - Receive client connection: Client-bc9a3a05-cf50-11f0-8001-f2cd832f225c

2025-12-02 07:20:55,341 - distributed.scheduler - INFO - Close client connection: Client-5619e01b-cf4f-11f0-8001-f2cd832f225c

2025-12-02 07:20:55,340 - distributed.scheduler - INFO - Remove client Client-5619e01b-cf4f-11f0-8001-f2cd832f225c

2025-12-02 07:20:55,339 - distributed.scheduler - INFO - Remove client Client-5619e01b-cf4f-11f0-8001-f2cd832f225c

2025-12-02 07:20:10,264 - distributed.scheduler - INFO - Receive client connection: Client-5619e01b-cf4f-11f0-8001-f2cd832f225c

2025-12-02 07:10:49,693 - distributed.scheduler - INFO - Close client connection: Client-ef934467-cf4d-11f0-8001-f2cd832f225c

2025-12-02 07:10:49,691 - distributed.scheduler - INFO - Remove client Client-ef934467-cf4d-11f0-8001-f2cd832f225c

2025-12-02 07:10:49,689 - distributed.scheduler - INFO - Remove client Client-ef934467-cf4d-11f0-8001-f2cd832f225c

2025-12-02 07:10:08,761 - distributed.scheduler - INFO - Receive client connection: Client-ef934467-cf4d-11f0-8001-f2cd832f225c

2025-12-02 07:01:25,445 - distributed.scheduler - INFO - Close client connection: Client-890fcbeb-cf4c-11f0-8001-f2cd832f225c

2025-12-02 07:01:25,443 - distributed.scheduler - INFO - Remove client Client-890fcbeb-cf4c-11f0-8001-f2cd832f225c

2025-12-02 07:01:25,441 - distributed.scheduler - INFO - Remove client Client-890fcbeb-cf4c-11f0-8001-f2cd832f225c

2025-12-02 07:00:07,280 - distributed.scheduler - INFO - Receive client connection: Client-890fcbeb-cf4c-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,599 - distributed.scheduler - INFO - Close client connection: Client-11661d5e-cf32-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,599 - distributed.scheduler - INFO - Close client connection: Client-77d8c242-cf33-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,599 - distributed.scheduler - INFO - Close client connection: Client-de65f25d-cf34-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,599 - distributed.scheduler - INFO - Close client connection: Client-44e8faaa-cf36-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,598 - distributed.scheduler - INFO - Close client connection: Client-ab64b10f-cf37-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,598 - distributed.scheduler - INFO - Close client connection: Client-abf23516-cf3e-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,598 - distributed.scheduler - INFO - Close client connection: Client-11e2a2bb-cf39-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,598 - distributed.scheduler - INFO - Close client connection: Client-dee93f5a-cf3b-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,598 - distributed.scheduler - INFO - Close client connection: Client-78685a7a-cf3a-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,598 - distributed.scheduler - INFO - Close client connection: Client-456981d3-cf3d-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,597 - distributed.scheduler - INFO - Close client connection: Client-45d5b8ef-cf44-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,597 - distributed.scheduler - INFO - Close client connection: Client-126ae9f5-cf40-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,597 - distributed.scheduler - INFO - Close client connection: Client-78e7e6a3-cf41-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,597 - distributed.scheduler - INFO - Close client connection: Client-ac53e96e-cf45-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,597 - distributed.scheduler - INFO - Close client connection: Client-df5c26a9-cf42-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,597 - distributed.scheduler - INFO - Close client connection: Client-bc036874-cf49-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,596 - distributed.scheduler - INFO - Close client connection: Client-eefdc0d8-cf46-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,596 - distributed.scheduler - INFO - Close client connection: Client-558a305e-cf48-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,593 - distributed.scheduler - INFO - Remove client Client-11661d5e-cf32-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,593 - distributed.scheduler - INFO - Remove client Client-77d8c242-cf33-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,592 - distributed.scheduler - INFO - Remove client Client-de65f25d-cf34-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,592 - distributed.scheduler - INFO - Remove client Client-44e8faaa-cf36-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,591 - distributed.scheduler - INFO - Remove client Client-ab64b10f-cf37-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,590 - distributed.scheduler - INFO - Remove client Client-abf23516-cf3e-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,589 - distributed.scheduler - INFO - Remove client Client-11e2a2bb-cf39-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,589 - distributed.scheduler - INFO - Remove client Client-dee93f5a-cf3b-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,588 - distributed.scheduler - INFO - Remove client Client-78685a7a-cf3a-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,587 - distributed.scheduler - INFO - Remove client Client-456981d3-cf3d-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,586 - distributed.scheduler - INFO - Remove client Client-45d5b8ef-cf44-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,585 - distributed.scheduler - INFO - Remove client Client-126ae9f5-cf40-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,584 - distributed.scheduler - INFO - Remove client Client-78e7e6a3-cf41-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,584 - distributed.scheduler - INFO - Remove client Client-ac53e96e-cf45-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,583 - distributed.scheduler - INFO - Remove client Client-df5c26a9-cf42-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,581 - distributed.scheduler - INFO - Remove client Client-bc036874-cf49-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,580 - distributed.scheduler - INFO - Remove client Client-eefdc0d8-cf46-11f0-8001-f2cd832f225c

2025-12-02 06:50:45,580 - distributed.scheduler - INFO - Remove client Client-558a305e-cf48-11f0-8001-f2cd832f225c

2025-12-02 06:50:44,804 - distributed.scheduler - INFO - Close client connection: Client-2284c968-cf4b-11f0-8001-f2cd832f225c

2025-12-02 06:50:44,803 - distributed.scheduler - INFO - Remove client Client-2284c968-cf4b-11f0-8001-f2cd832f225c

2025-12-02 06:50:44,801 - distributed.scheduler - INFO - Remove client Client-2284c968-cf4b-11f0-8001-f2cd832f225c

2025-12-02 06:50:05,728 - distributed.scheduler - INFO - Receive client connection: Client-2284c968-cf4b-11f0-8001-f2cd832f225c

2025-12-02 06:40:04,273 - distributed.scheduler - INFO - Receive client connection: Client-bc036874-cf49-11f0-8001-f2cd832f225c

2025-12-02 06:30:02,838 - distributed.scheduler - INFO - Receive client connection: Client-558a305e-cf48-11f0-8001-f2cd832f225c

2025-12-02 06:20:01,298 - distributed.scheduler - INFO - Receive client connection: Client-eefdc0d8-cf46-11f0-8001-f2cd832f225c

2025-12-02 06:10:59,959 - distributed.scheduler - INFO - Receive client connection: Client-ac53e96e-cf45-11f0-8001-f2cd832f225c

2025-12-02 06:00:58,503 - distributed.scheduler - INFO - Receive client connection: Client-45d5b8ef-cf44-11f0-8001-f2cd832f225c

2025-12-02 05:50:57,079 - distributed.scheduler - INFO - Receive client connection: Client-df5c26a9-cf42-11f0-8001-f2cd832f225c

2025-12-02 05:40:55,700 - distributed.scheduler - INFO - Receive client connection: Client-78e7e6a3-cf41-11f0-8001-f2cd832f225c

2025-12-02 05:30:54,251 - distributed.scheduler - INFO - Receive client connection: Client-126ae9f5-cf40-11f0-8001-f2cd832f225c

2025-12-02 05:20:52,837 - distributed.scheduler - INFO - Receive client connection: Client-abf23516-cf3e-11f0-8001-f2cd832f225c

2025-12-02 05:10:51,312 - distributed.scheduler - INFO - Receive client connection: Client-456981d3-cf3d-11f0-8001-f2cd832f225c

2025-12-02 05:00:49,853 - distributed.scheduler - INFO - Receive client connection: Client-dee93f5a-cf3b-11f0-8001-f2cd832f225c

2025-12-02 04:50:48,380 - distributed.scheduler - INFO - Receive client connection: Client-78685a7a-cf3a-11f0-8001-f2cd832f225c

2025-12-02 04:40:46,885 - distributed.scheduler - INFO - Receive client connection: Client-11e2a2bb-cf39-11f0-8001-f2cd832f225c

2025-12-02 04:30:45,435 - distributed.scheduler - INFO - Receive client connection: Client-ab64b10f-cf37-11f0-8001-f2cd832f225c

2025-12-02 04:20:43,998 - distributed.scheduler - INFO - Receive client connection: Client-44e8faaa-cf36-11f0-8001-f2cd832f225c

2025-12-02 04:10:42,510 - distributed.scheduler - INFO - Receive client connection: Client-de65f25d-cf34-11f0-8001-f2cd832f225c

2025-12-02 04:00:40,961 - distributed.scheduler - INFO - Receive client connection: Client-77d8c242-cf33-11f0-8001-f2cd832f225c

2025-12-02 03:50:39,595 - distributed.scheduler - INFO - Receive client connection: Client-11661d5e-cf32-11f0-8001-f2cd832f225c

2025-12-02 03:41:17,309 - distributed.scheduler - INFO - Close client connection: Client-aae70af0-cf30-11f0-8001-f2cd832f225c

2025-12-02 03:41:17,308 - distributed.scheduler - INFO - Remove client Client-aae70af0-cf30-11f0-8001-f2cd832f225c

2025-12-02 03:41:17,306 - distributed.scheduler - INFO - Remove client Client-aae70af0-cf30-11f0-8001-f2cd832f225c

2025-12-02 03:40:38,139 - distributed.scheduler - INFO - Receive client connection: Client-aae70af0-cf30-11f0-8001-f2cd832f225c

2025-12-02 03:31:46,659 - distributed.scheduler - INFO - Close client connection: Client-4469bef6-cf2f-11f0-8001-f2cd832f225c

2025-12-02 03:31:46,658 - distributed.scheduler - INFO - Remove client Client-4469bef6-cf2f-11f0-8001-f2cd832f225c

2025-12-02 03:31:46,656 - distributed.scheduler - INFO - Remove client Client-4469bef6-cf2f-11f0-8001-f2cd832f225c

2025-12-02 03:30:36,697 - distributed.scheduler - INFO - Receive client connection: Client-4469bef6-cf2f-11f0-8001-f2cd832f225c

2025-12-02 03:21:11,052 - distributed.scheduler - INFO - Close client connection: Client-ddfd2056-cf2d-11f0-8001-f2cd832f225c

2025-12-02 03:21:11,049 - distributed.scheduler - INFO - Remove client Client-ddfd2056-cf2d-11f0-8001-f2cd832f225c

2025-12-02 03:21:11,047 - distributed.scheduler - INFO - Remove client Client-ddfd2056-cf2d-11f0-8001-f2cd832f225c

2025-12-02 03:20:35,362 - distributed.scheduler - INFO - Receive client connection: Client-ddfd2056-cf2d-11f0-8001-f2cd832f225c

2025-12-02 03:11:07,366 - distributed.scheduler - INFO - Close client connection: Client-777bca2e-cf2c-11f0-8001-f2cd832f225c

2025-12-02 03:11:07,365 - distributed.scheduler - INFO - Remove client Client-777bca2e-cf2c-11f0-8001-f2cd832f225c

2025-12-02 03:11:07,363 - distributed.scheduler - INFO - Remove client Client-777bca2e-cf2c-11f0-8001-f2cd832f225c

2025-12-02 03:10:33,885 - distributed.scheduler - INFO - Receive client connection: Client-777bca2e-cf2c-11f0-8001-f2cd832f225c

2025-12-02 03:01:40,122 - distributed.scheduler - INFO - Close client connection: Client-11033f52-cf2b-11f0-8001-f2cd832f225c

2025-12-02 03:01:40,122 - distributed.scheduler - INFO - Remove client Client-11033f52-cf2b-11f0-8001-f2cd832f225c

2025-12-02 03:01:40,120 - distributed.scheduler - INFO - Remove client Client-11033f52-cf2b-11f0-8001-f2cd832f225c

2025-12-02 03:00:32,474 - distributed.scheduler - INFO - Receive client connection: Client-11033f52-cf2b-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,548 - distributed.scheduler - INFO - Close client connection: Client-97c6a695-cef4-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,548 - distributed.scheduler - INFO - Close client connection: Client-fe409d8b-cef5-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,548 - distributed.scheduler - INFO - Close client connection: Client-64cab2dc-cef7-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,548 - distributed.scheduler - INFO - Close client connection: Client-cb4dcaf6-cef8-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,548 - distributed.scheduler - INFO - Close client connection: Client-31dab353-cefa-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,547 - distributed.scheduler - INFO - Close client connection: Client-984f97dc-cefb-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,547 - distributed.scheduler - INFO - Close client connection: Client-fed3ca2a-cefc-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,547 - distributed.scheduler - INFO - Close client connection: Client-65522bee-cefe-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,547 - distributed.scheduler - INFO - Close client connection: Client-cbc7dae8-ceff-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,547 - distributed.scheduler - INFO - Close client connection: Client-323d9439-cf01-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,547 - distributed.scheduler - INFO - Close client connection: Client-98c1c52c-cf02-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,547 - distributed.scheduler - INFO - Close client connection: Client-ff463908-cf03-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,547 - distributed.scheduler - INFO - Close client connection: Client-65c09a7d-cf05-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,547 - distributed.scheduler - INFO - Close client connection: Client-cc40b641-cf06-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,547 - distributed.scheduler - INFO - Close client connection: Client-32bedd3c-cf08-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,547 - distributed.scheduler - INFO - Close client connection: Client-99280d2e-cf09-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,546 - distributed.scheduler - INFO - Close client connection: Client-ffae33e5-cf0a-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,546 - distributed.scheduler - INFO - Close client connection: Client-424da4fb-cf0c-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,546 - distributed.scheduler - INFO - Close client connection: Client-a8ce7342-cf0d-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,546 - distributed.scheduler - INFO - Close client connection: Client-0f458ef2-cf0f-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,546 - distributed.scheduler - INFO - Close client connection: Client-75c4e9d8-cf10-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,546 - distributed.scheduler - INFO - Close client connection: Client-dc4247e2-cf11-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,546 - distributed.scheduler - INFO - Close client connection: Client-42c26373-cf13-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,546 - distributed.scheduler - INFO - Close client connection: Client-a9425b38-cf14-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,546 - distributed.scheduler - INFO - Close client connection: Client-0fbb805a-cf16-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,546 - distributed.scheduler - INFO - Close client connection: Client-7633730d-cf17-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,545 - distributed.scheduler - INFO - Close client connection: Client-dcb081b1-cf18-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,545 - distributed.scheduler - INFO - Close client connection: Client-43331aaa-cf1a-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,545 - distributed.scheduler - INFO - Close client connection: Client-a9b4bb25-cf1b-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,545 - distributed.scheduler - INFO - Close client connection: Client-1033c3aa-cf1d-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,545 - distributed.scheduler - INFO - Close client connection: Client-76b18998-cf1e-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,543 - distributed.scheduler - INFO - Close client connection: Client-dd2e0d05-cf1f-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,543 - distributed.scheduler - INFO - Close client connection: Client-43a895a4-cf21-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,543 - distributed.scheduler - INFO - Close client connection: Client-aa270d6b-cf22-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,543 - distributed.scheduler - INFO - Close client connection: Client-109c7ec1-cf24-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,543 - distributed.scheduler - INFO - Close client connection: Client-770fe9f6-cf25-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,543 - distributed.scheduler - INFO - Close client connection: Client-dd849c53-cf26-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,543 - distributed.scheduler - INFO - Close client connection: Client-4402eaa8-cf28-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,540 - distributed.scheduler - INFO - Remove client Client-97c6a695-cef4-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,540 - distributed.scheduler - INFO - Remove client Client-fe409d8b-cef5-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,540 - distributed.scheduler - INFO - Remove client Client-64cab2dc-cef7-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,539 - distributed.scheduler - INFO - Remove client Client-cb4dcaf6-cef8-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,539 - distributed.scheduler - INFO - Remove client Client-31dab353-cefa-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,539 - distributed.scheduler - INFO - Remove client Client-984f97dc-cefb-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,538 - distributed.scheduler - INFO - Remove client Client-fed3ca2a-cefc-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,538 - distributed.scheduler - INFO - Remove client Client-65522bee-cefe-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,538 - distributed.scheduler - INFO - Remove client Client-cbc7dae8-ceff-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,537 - distributed.scheduler - INFO - Remove client Client-323d9439-cf01-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,537 - distributed.scheduler - INFO - Remove client Client-98c1c52c-cf02-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,537 - distributed.scheduler - INFO - Remove client Client-ff463908-cf03-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,536 - distributed.scheduler - INFO - Remove client Client-65c09a7d-cf05-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,536 - distributed.scheduler - INFO - Remove client Client-cc40b641-cf06-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,536 - distributed.scheduler - INFO - Remove client Client-32bedd3c-cf08-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,535 - distributed.scheduler - INFO - Remove client Client-99280d2e-cf09-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,535 - distributed.scheduler - INFO - Remove client Client-ffae33e5-cf0a-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,535 - distributed.scheduler - INFO - Remove client Client-424da4fb-cf0c-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,534 - distributed.scheduler - INFO - Remove client Client-a8ce7342-cf0d-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,534 - distributed.scheduler - INFO - Remove client Client-0f458ef2-cf0f-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,534 - distributed.scheduler - INFO - Remove client Client-75c4e9d8-cf10-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,533 - distributed.scheduler - INFO - Remove client Client-dc4247e2-cf11-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,533 - distributed.scheduler - INFO - Remove client Client-42c26373-cf13-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,533 - distributed.scheduler - INFO - Remove client Client-a9425b38-cf14-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,532 - distributed.scheduler - INFO - Remove client Client-0fbb805a-cf16-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,532 - distributed.scheduler - INFO - Remove client Client-7633730d-cf17-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,532 - distributed.scheduler - INFO - Remove client Client-dcb081b1-cf18-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,531 - distributed.scheduler - INFO - Remove client Client-43331aaa-cf1a-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,531 - distributed.scheduler - INFO - Remove client Client-a9b4bb25-cf1b-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,530 - distributed.scheduler - INFO - Remove client Client-1033c3aa-cf1d-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,530 - distributed.scheduler - INFO - Remove client Client-76b18998-cf1e-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,520 - distributed.scheduler - INFO - Remove client Client-dd2e0d05-cf1f-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,519 - distributed.scheduler - INFO - Remove client Client-43a895a4-cf21-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,518 - distributed.scheduler - INFO - Remove client Client-aa270d6b-cf22-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,518 - distributed.scheduler - INFO - Remove client Client-109c7ec1-cf24-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,517 - distributed.scheduler - INFO - Remove client Client-770fe9f6-cf25-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,516 - distributed.scheduler - INFO - Remove client Client-dd849c53-cf26-11f0-8001-f2cd832f225c

2025-12-02 02:51:09,515 - distributed.scheduler - INFO - Remove client Client-4402eaa8-cf28-11f0-8001-f2cd832f225c

2025-12-02 02:51:08,707 - distributed.scheduler - INFO - Close client connection: Client-aa7d52c9-cf29-11f0-8001-f2cd832f225c

2025-12-02 02:51:08,705 - distributed.scheduler - INFO - Remove client Client-aa7d52c9-cf29-11f0-8001-f2cd832f225c

2025-12-02 02:51:08,703 - distributed.scheduler - INFO - Remove client Client-aa7d52c9-cf29-11f0-8001-f2cd832f225c

2025-12-02 02:50:30,966 - distributed.scheduler - INFO - Receive client connection: Client-aa7d52c9-cf29-11f0-8001-f2cd832f225c

2025-12-02 02:40:29,532 - distributed.scheduler - INFO - Receive client connection: Client-4402eaa8-cf28-11f0-8001-f2cd832f225c

2025-12-02 02:30:28,087 - distributed.scheduler - INFO - Receive client connection: Client-dd849c53-cf26-11f0-8001-f2cd832f225c

2025-12-02 02:20:26,702 - distributed.scheduler - INFO - Receive client connection: Client-770fe9f6-cf25-11f0-8001-f2cd832f225c

2025-12-02 02:10:25,313 - distributed.scheduler - INFO - Receive client connection: Client-109c7ec1-cf24-11f0-8001-f2cd832f225c

2025-12-02 02:00:23,914 - distributed.scheduler - INFO - Receive client connection: Client-aa270d6b-cf22-11f0-8001-f2cd832f225c

2025-12-02 01:50:22,470 - distributed.scheduler - INFO - Receive client connection: Client-43a895a4-cf21-11f0-8001-f2cd832f225c

2025-12-02 01:40:21,034 - distributed.scheduler - INFO - Receive client connection: Client-dd2e0d05-cf1f-11f0-8001-f2cd832f225c

2025-12-02 01:30:19,602 - distributed.scheduler - INFO - Receive client connection: Client-76b18998-cf1e-11f0-8001-f2cd832f225c

2025-12-02 01:20:18,149 - distributed.scheduler - INFO - Receive client connection: Client-1033c3aa-cf1d-11f0-8001-f2cd832f225c

2025-12-02 01:10:16,694 - distributed.scheduler - INFO - Receive client connection: Client-a9b4bb25-cf1b-11f0-8001-f2cd832f225c

2025-12-02 01:00:15,227 - distributed.scheduler - INFO - Receive client connection: Client-43331aaa-cf1a-11f0-8001-f2cd832f225c

2025-12-02 00:50:13,739 - distributed.scheduler - INFO - Receive client connection: Client-dcb081b1-cf18-11f0-8001-f2cd832f225c

2025-12-02 00:40:12,296 - distributed.scheduler - INFO - Receive client connection: Client-7633730d-cf17-11f0-8001-f2cd832f225c

2025-12-02 00:30:10,887 - distributed.scheduler - INFO - Receive client connection: Client-0fbb805a-cf16-11f0-8001-f2cd832f225c

2025-12-02 00:20:09,474 - distributed.scheduler - INFO - Receive client connection: Client-a9425b38-cf14-11f0-8001-f2cd832f225c

2025-12-02 00:10:08,000 - distributed.scheduler - INFO - Receive client connection: Client-42c26373-cf13-11f0-8001-f2cd832f225c

2025-12-02 00:00:06,538 - distributed.scheduler - INFO - Receive client connection: Client-dc4247e2-cf11-11f0-8001-f2cd832f225c

2025-12-01 23:50:05,096 - distributed.scheduler - INFO - Receive client connection: Client-75c4e9d8-cf10-11f0-8001-f2cd832f225c

2025-12-01 23:40:03,632 - distributed.scheduler - INFO - Receive client connection: Client-0f458ef2-cf0f-11f0-8001-f2cd832f225c

2025-12-01 23:30:02,235 - distributed.scheduler - INFO - Receive client connection: Client-a8ce7342-cf0d-11f0-8001-f2cd832f225c

2025-12-01 23:20:00,762 - distributed.scheduler - INFO - Receive client connection: Client-424da4fb-cf0c-11f0-8001-f2cd832f225c

2025-12-01 23:10:59,488 - distributed.scheduler - INFO - Receive client connection: Client-ffae33e5-cf0a-11f0-8001-f2cd832f225c

2025-12-01 23:00:57,992 - distributed.scheduler - INFO - Receive client connection: Client-99280d2e-cf09-11f0-8001-f2cd832f225c

2025-12-01 22:50:56,675 - distributed.scheduler - INFO - Receive client connection: Client-32bedd3c-cf08-11f0-8001-f2cd832f225c

2025-12-01 22:40:55,216 - distributed.scheduler - INFO - Receive client connection: Client-cc40b641-cf06-11f0-8001-f2cd832f225c

2025-12-01 22:30:53,754 - distributed.scheduler - INFO - Receive client connection: Client-65c09a7d-cf05-11f0-8001-f2cd832f225c

2025-12-01 22:20:52,331 - distributed.scheduler - INFO - Receive client connection: Client-ff463908-cf03-11f0-8001-f2cd832f225c

2025-12-01 22:10:50,837 - distributed.scheduler - INFO - Receive client connection: Client-98c1c52c-cf02-11f0-8001-f2cd832f225c

2025-12-01 22:00:49,346 - distributed.scheduler - INFO - Receive client connection: Client-323d9439-cf01-11f0-8001-f2cd832f225c

2025-12-01 21:50:47,956 - distributed.scheduler - INFO - Receive client connection: Client-cbc7dae8-ceff-11f0-8001-f2cd832f225c

2025-12-01 21:40:46,558 - distributed.scheduler - INFO - Receive client connection: Client-65522bee-cefe-11f0-8001-f2cd832f225c

2025-12-01 21:30:45,108 - distributed.scheduler - INFO - Receive client connection: Client-fed3ca2a-cefc-11f0-8001-f2cd832f225c

2025-12-01 21:20:43,615 - distributed.scheduler - INFO - Receive client connection: Client-984f97dc-cefb-11f0-8001-f2cd832f225c

2025-12-01 21:10:42,217 - distributed.scheduler - INFO - Receive client connection: Client-31dab353-cefa-11f0-8001-f2cd832f225c

2025-12-01 21:00:40,672 - distributed.scheduler - INFO - Receive client connection: Client-cb4dcaf6-cef8-11f0-8001-f2cd832f225c

2025-12-01 20:50:39,209 - distributed.scheduler - INFO - Receive client connection: Client-64cab2dc-cef7-11f0-8001-f2cd832f225c

2025-12-01 20:40:37,666 - distributed.scheduler - INFO - Receive client connection: Client-fe409d8b-cef5-11f0-8001-f2cd832f225c

2025-12-01 20:30:36,249 - distributed.scheduler - INFO - Receive client connection: Client-97c6a695-cef4-11f0-8001-f2cd832f225c

2025-12-01 20:21:08,350 - distributed.scheduler - INFO - Close client connection: Client-313ff919-cef3-11f0-8001-f2cd832f225c

2025-12-01 20:21:08,348 - distributed.scheduler - INFO - Remove client Client-313ff919-cef3-11f0-8001-f2cd832f225c

2025-12-01 20:21:08,346 - distributed.scheduler - INFO - Remove client Client-313ff919-cef3-11f0-8001-f2cd832f225c

2025-12-01 20:20:34,737 - distributed.scheduler - INFO - Receive client connection: Client-313ff919-cef3-11f0-8001-f2cd832f225c

2025-12-01 20:11:05,507 - distributed.scheduler - INFO - Close client connection: Client-cac406ff-cef1-11f0-8001-f2cd832f225c

2025-12-01 20:11:05,507 - distributed.scheduler - INFO - Remove client Client-cac406ff-cef1-11f0-8001-f2cd832f225c

2025-12-01 20:11:05,505 - distributed.scheduler - INFO - Remove client Client-cac406ff-cef1-11f0-8001-f2cd832f225c

2025-12-01 20:10:33,308 - distributed.scheduler - INFO - Receive client connection: Client-cac406ff-cef1-11f0-8001-f2cd832f225c

2025-12-01 20:01:38,681 - distributed.scheduler - INFO - Close client connection: Client-64351536-cef0-11f0-8001-f2cd832f225c

2025-12-01 20:01:38,680 - distributed.scheduler - INFO - Remove client Client-64351536-cef0-11f0-8001-f2cd832f225c

2025-12-01 20:01:38,678 - distributed.scheduler - INFO - Remove client Client-64351536-cef0-11f0-8001-f2cd832f225c

2025-12-01 20:00:31,738 - distributed.scheduler - INFO - Receive client connection: Client-64351536-cef0-11f0-8001-f2cd832f225c

2025-12-01 19:51:04,976 - distributed.scheduler - INFO - Close client connection: Client-fdb6f531-ceee-11f0-8001-f2cd832f225c

2025-12-01 19:51:04,975 - distributed.scheduler - INFO - Remove client Client-fdb6f531-ceee-11f0-8001-f2cd832f225c

2025-12-01 19:51:04,972 - distributed.scheduler - INFO - Remove client Client-fdb6f531-ceee-11f0-8001-f2cd832f225c

2025-12-01 19:50:30,292 - distributed.scheduler - INFO - Receive client connection: Client-fdb6f531-ceee-11f0-8001-f2cd832f225c

2025-12-01 19:41:04,348 - distributed.scheduler - INFO - Close client connection: Client-97345ecf-ceed-11f0-8001-f2cd832f225c

2025-12-01 19:41:04,346 - distributed.scheduler - INFO - Remove client Client-97345ecf-ceed-11f0-8001-f2cd832f225c

2025-12-01 19:41:04,344 - distributed.scheduler - INFO - Remove client Client-97345ecf-ceed-11f0-8001-f2cd832f225c

2025-12-01 19:40:28,813 - distributed.scheduler - INFO - Receive client connection: Client-97345ecf-ceed-11f0-8001-f2cd832f225c

2025-12-01 19:31:31,127 - distributed.scheduler - INFO - Close client connection: Client-30bd8def-ceec-11f0-8001-f2cd832f225c

2025-12-01 19:31:31,127 - distributed.scheduler - INFO - Remove client Client-30bd8def-ceec-11f0-8001-f2cd832f225c

2025-12-01 19:31:31,126 - distributed.scheduler - INFO - Remove client Client-30bd8def-ceec-11f0-8001-f2cd832f225c

2025-12-01 19:30:27,411 - distributed.scheduler - INFO - Receive client connection: Client-30bd8def-ceec-11f0-8001-f2cd832f225c

2025-12-01 19:20:59,639 - distributed.scheduler - INFO - Close client connection: Client-ca49c1ba-ceea-11f0-8001-f2cd832f225c

2025-12-01 19:20:59,638 - distributed.scheduler - INFO - Remove client Client-ca49c1ba-ceea-11f0-8001-f2cd832f225c

2025-12-01 19:20:59,636 - distributed.scheduler - INFO - Remove client Client-ca49c1ba-ceea-11f0-8001-f2cd832f225c

2025-12-01 19:20:26,032 - distributed.scheduler - INFO - Receive client connection: Client-ca49c1ba-ceea-11f0-8001-f2cd832f225c

2025-12-01 19:10:58,964 - distributed.scheduler - INFO - Close client connection: Client-63c4ca39-cee9-11f0-8001-f2cd832f225c

2025-12-01 19:10:58,963 - distributed.scheduler - INFO - Remove client Client-63c4ca39-cee9-11f0-8001-f2cd832f225c

2025-12-01 19:10:58,961 - distributed.scheduler - INFO - Remove client Client-63c4ca39-cee9-11f0-8001-f2cd832f225c

2025-12-01 19:10:24,526 - distributed.scheduler - INFO - Receive client connection: Client-63c4ca39-cee9-11f0-8001-f2cd832f225c

2025-12-01 19:01:27,989 - distributed.scheduler - INFO - Close client connection: Client-fc912c2d-cee0-11f0-8001-f2cd832f225c

2025-12-01 19:01:27,989 - distributed.scheduler - INFO - Close client connection: Client-63134267-cee2-11f0-8001-f2cd832f225c

2025-12-01 19:01:27,988 - distributed.scheduler - INFO - Close client connection: Client-c9abba4c-cee3-11f0-8001-f2cd832f225c

2025-12-01 19:01:27,988 - distributed.scheduler - INFO - Close client connection: Client-302c9cd4-cee5-11f0-8001-f2cd832f225c

2025-12-01 19:01:27,987 - distributed.scheduler - INFO - Close client connection: Client-96b0dc9c-cee6-11f0-8001-f2cd832f225c

2025-12-01 19:01:27,986 - distributed.scheduler - INFO - Remove client Client-fc912c2d-cee0-11f0-8001-f2cd832f225c

2025-12-01 19:01:27,985 - distributed.scheduler - INFO - Remove client Client-63134267-cee2-11f0-8001-f2cd832f225c

2025-12-01 19:01:27,984 - distributed.scheduler - INFO - Remove client Client-c9abba4c-cee3-11f0-8001-f2cd832f225c

2025-12-01 19:01:27,983 - distributed.scheduler - INFO - Remove client Client-302c9cd4-cee5-11f0-8001-f2cd832f225c

2025-12-01 19:01:27,980 - distributed.scheduler - INFO - Remove client Client-96b0dc9c-cee6-11f0-8001-f2cd832f225c

2025-12-01 19:01:27,192 - distributed.scheduler - INFO - Close client connection: Client-fd356512-cee7-11f0-8001-f2cd832f225c

2025-12-01 19:01:27,191 - distributed.scheduler - INFO - Remove client Client-fd356512-cee7-11f0-8001-f2cd832f225c

2025-12-01 19:01:27,189 - distributed.scheduler - INFO - Remove client Client-fd356512-cee7-11f0-8001-f2cd832f225c

2025-12-01 19:00:22,956 - distributed.scheduler - INFO - Receive client connection: Client-fd356512-cee7-11f0-8001-f2cd832f225c

2025-12-01 18:50:21,469 - distributed.scheduler - INFO - Receive client connection: Client-96b0dc9c-cee6-11f0-8001-f2cd832f225c

2025-12-01 18:40:19,974 - distributed.scheduler - INFO - Receive client connection: Client-302c9cd4-cee5-11f0-8001-f2cd832f225c

2025-12-01 18:30:18,504 - distributed.scheduler - INFO - Receive client connection: Client-c9abba4c-cee3-11f0-8001-f2cd832f225c

2025-12-01 18:20:16,877 - distributed.scheduler - INFO - Receive client connection: Client-63134267-cee2-11f0-8001-f2cd832f225c

2025-12-01 18:10:15,448 - distributed.scheduler - INFO - Receive client connection: Client-fc912c2d-cee0-11f0-8001-f2cd832f225c

2025-12-01 18:01:21,293 - distributed.scheduler - INFO - Close client connection: Client-95f73913-cedf-11f0-8001-f2cd832f225c

2025-12-01 18:01:21,291 - distributed.scheduler - INFO - Remove client Client-95f73913-cedf-11f0-8001-f2cd832f225c

2025-12-01 18:01:21,290 - distributed.scheduler - INFO - Remove client Client-95f73913-cedf-11f0-8001-f2cd832f225c

2025-12-01 18:00:13,781 - distributed.scheduler - INFO - Receive client connection: Client-95f73913-cedf-11f0-8001-f2cd832f225c

2025-12-01 17:50:47,868 - distributed.scheduler - INFO - Close client connection: Client-2f78716e-cede-11f0-8001-f2cd832f225c

2025-12-01 17:50:47,866 - distributed.scheduler - INFO - Remove client Client-2f78716e-cede-11f0-8001-f2cd832f225c

2025-12-01 17:50:47,864 - distributed.scheduler - INFO - Remove client Client-2f78716e-cede-11f0-8001-f2cd832f225c

2025-12-01 17:50:12,327 - distributed.scheduler - INFO - Receive client connection: Client-2f78716e-cede-11f0-8001-f2cd832f225c

2025-12-01 17:40:49,887 - distributed.scheduler - INFO - Close client connection: Client-c8f9ca22-cedc-11f0-8001-f2cd832f225c

2025-12-01 17:40:49,886 - distributed.scheduler - INFO - Remove client Client-c8f9ca22-cedc-11f0-8001-f2cd832f225c

2025-12-01 17:40:49,884 - distributed.scheduler - INFO - Remove client Client-c8f9ca22-cedc-11f0-8001-f2cd832f225c

2025-12-01 17:40:10,870 - distributed.scheduler - INFO - Receive client connection: Client-c8f9ca22-cedc-11f0-8001-f2cd832f225c

2025-12-01 17:31:21,254 - distributed.scheduler - INFO - Close client connection: Client-6278027d-cedb-11f0-8001-f2cd832f225c

2025-12-01 17:31:21,253 - distributed.scheduler - INFO - Remove client Client-6278027d-cedb-11f0-8001-f2cd832f225c

2025-12-01 17:31:21,252 - distributed.scheduler - INFO - Remove client Client-6278027d-cedb-11f0-8001-f2cd832f225c

2025-12-01 17:30:09,397 - distributed.scheduler - INFO - Receive client connection: Client-6278027d-cedb-11f0-8001-f2cd832f225c

2025-12-01 17:20:43,713 - distributed.scheduler - INFO - Close client connection: Client-fbf31b7a-ced9-11f0-8001-f2cd832f225c

2025-12-01 17:20:43,712 - distributed.scheduler - INFO - Remove client Client-fbf31b7a-ced9-11f0-8001-f2cd832f225c

2025-12-01 17:20:43,710 - distributed.scheduler - INFO - Remove client Client-fbf31b7a-ced9-11f0-8001-f2cd832f225c

2025-12-01 17:20:07,905 - distributed.scheduler - INFO - Receive client connection: Client-fbf31b7a-ced9-11f0-8001-f2cd832f225c

2025-12-01 17:10:43,497 - distributed.scheduler - INFO - Close client connection: Client-ebe0f7f8-cece-11f0-8001-f2cd832f225c

2025-12-01 17:10:43,497 - distributed.scheduler - INFO - Close client connection: Client-52476b7d-ced0-11f0-8001-f2cd832f225c

2025-12-01 17:10:43,497 - distributed.scheduler - INFO - Close client connection: Client-b8b77016-ced1-11f0-8001-f2cd832f225c

2025-12-01 17:10:43,496 - distributed.scheduler - INFO - Close client connection: Client-fb630733-ced2-11f0-8001-f2cd832f225c

2025-12-01 17:10:43,496 - distributed.scheduler - INFO - Close client connection: Client-61e63fba-ced4-11f0-8001-f2cd832f225c

2025-12-01 17:10:43,496 - distributed.scheduler - INFO - Close client connection: Client-c8645522-ced5-11f0-8001-f2cd832f225c

2025-12-01 17:10:43,495 - distributed.scheduler - INFO - Close client connection: Client-2ee4f4a6-ced7-11f0-8001-f2cd832f225c

2025-12-01 17:10:43,494 - distributed.scheduler - INFO - Remove client Client-ebe0f7f8-cece-11f0-8001-f2cd832f225c

2025-12-01 17:10:43,494 - distributed.scheduler - INFO - Remove client Client-52476b7d-ced0-11f0-8001-f2cd832f225c

2025-12-01 17:10:43,493 - distributed.scheduler - INFO - Remove client Client-b8b77016-ced1-11f0-8001-f2cd832f225c

2025-12-01 17:10:43,493 - distributed.scheduler - INFO - Remove client Client-fb630733-ced2-11f0-8001-f2cd832f225c

2025-12-01 17:10:43,492 - distributed.scheduler - INFO - Remove client Client-61e63fba-ced4-11f0-8001-f2cd832f225c

2025-12-01 17:10:43,491 - distributed.scheduler - INFO - Remove client Client-c8645522-ced5-11f0-8001-f2cd832f225c

2025-12-01 17:10:43,488 - distributed.scheduler - INFO - Remove client Client-2ee4f4a6-ced7-11f0-8001-f2cd832f225c

2025-12-01 17:10:42,842 - distributed.scheduler - INFO - Close client connection: Client-9573979e-ced8-11f0-8001-f2cd832f225c

2025-12-01 17:10:42,841 - distributed.scheduler - INFO - Remove client Client-9573979e-ced8-11f0-8001-f2cd832f225c

2025-12-01 17:10:42,839 - distributed.scheduler - INFO - Remove client Client-9573979e-ced8-11f0-8001-f2cd832f225c

2025-12-01 17:10:06,432 - distributed.scheduler - INFO - Receive client connection: Client-9573979e-ced8-11f0-8001-f2cd832f225c

2025-12-01 17:00:09,624 - distributed.scheduler - INFO - Receive client connection: Client-worker-31aed6ae-ced7-11f0-87e3-32a59f79d364

2025-12-01 17:00:04,896 - distributed.scheduler - INFO - Receive client connection: Client-2ee4f4a6-ced7-11f0-8001-f2cd832f225c

2025-12-01 16:50:03,403 - distributed.scheduler - INFO - Receive client connection: Client-c8645522-ced5-11f0-8001-f2cd832f225c

2025-12-01 16:40:01,957 - distributed.scheduler - INFO - Receive client connection: Client-61e63fba-ced4-11f0-8001-f2cd832f225c

2025-12-01 16:30:00,475 - distributed.scheduler - INFO - Receive client connection: Client-fb630733-ced2-11f0-8001-f2cd832f225c

2025-12-01 16:20:59,125 - distributed.scheduler - INFO - Receive client connection: Client-b8b77016-ced1-11f0-8001-f2cd832f225c

2025-12-01 16:10:57,758 - distributed.scheduler - INFO - Receive client connection: Client-52476b7d-ced0-11f0-8001-f2cd832f225c

2025-12-01 16:00:56,476 - distributed.scheduler - INFO - Receive client connection: Client-ebe0f7f8-cece-11f0-8001-f2cd832f225c

2025-12-01 15:51:34,127 - distributed.scheduler - INFO - Close client connection: Client-85626c6c-cecd-11f0-8001-f2cd832f225c

2025-12-01 15:51:34,124 - distributed.scheduler - INFO - Remove client Client-85626c6c-cecd-11f0-8001-f2cd832f225c

2025-12-01 15:51:34,123 - distributed.scheduler - INFO - Remove client Client-85626c6c-cecd-11f0-8001-f2cd832f225c

2025-12-01 15:50:55,014 - distributed.scheduler - INFO - Receive client connection: Client-85626c6c-cecd-11f0-8001-f2cd832f225c

2025-12-01 15:41:32,619 - distributed.scheduler - INFO - Close client connection: Client-1ee27cd0-cecc-11f0-8001-f2cd832f225c

2025-12-01 15:41:32,618 - distributed.scheduler - INFO - Remove client Client-1ee27cd0-cecc-11f0-8001-f2cd832f225c

2025-12-01 15:41:32,616 - distributed.scheduler - INFO - Remove client Client-1ee27cd0-cecc-11f0-8001-f2cd832f225c

2025-12-01 15:40:53,559 - distributed.scheduler - INFO - Receive client connection: Client-1ee27cd0-cecc-11f0-8001-f2cd832f225c

2025-12-01 15:32:03,136 - distributed.scheduler - INFO - Close client connection: Client-b865e037-ceca-11f0-8001-f2cd832f225c

2025-12-01 15:32:03,135 - distributed.scheduler - INFO - Remove client Client-b865e037-ceca-11f0-8001-f2cd832f225c

2025-12-01 15:32:03,133 - distributed.scheduler - INFO - Remove client Client-b865e037-ceca-11f0-8001-f2cd832f225c

2025-12-01 15:30:52,116 - distributed.scheduler - INFO - Receive client connection: Client-b865e037-ceca-11f0-8001-f2cd832f225c

2025-12-01 15:21:30,087 - distributed.scheduler - INFO - Close client connection: Client-51e1bfb6-cec9-11f0-8001-f2cd832f225c

2025-12-01 15:21:30,087 - distributed.scheduler - INFO - Remove client Client-51e1bfb6-cec9-11f0-8001-f2cd832f225c

2025-12-01 15:21:30,085 - distributed.scheduler - INFO - Remove client Client-51e1bfb6-cec9-11f0-8001-f2cd832f225c

2025-12-01 15:20:50,646 - distributed.scheduler - INFO - Receive client connection: Client-51e1bfb6-cec9-11f0-8001-f2cd832f225c

2025-12-01 15:11:30,736 - distributed.scheduler - INFO - Close client connection: Client-eb60ce21-cec7-11f0-8001-f2cd832f225c

2025-12-01 15:11:30,735 - distributed.scheduler - INFO - Remove client Client-eb60ce21-cec7-11f0-8001-f2cd832f225c

2025-12-01 15:11:30,733 - distributed.scheduler - INFO - Remove client Client-eb60ce21-cec7-11f0-8001-f2cd832f225c

2025-12-01 15:10:49,153 - distributed.scheduler - INFO - Receive client connection: Client-eb60ce21-cec7-11f0-8001-f2cd832f225c

2025-12-01 15:04:08,172 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:43605 after 30 s')

2025-12-01 15:04:08,160 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:43605 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:43605 after 30 s

2025-12-01 15:04:08,103 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:43605 after 30 s')

2025-12-01 15:04:08,093 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41957 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:43605 after 30 s')

2025-12-01 15:04:08,092 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:43605 after 30 s')

2025-12-01 15:04:08,090 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:43605 after 30 s')

2025-12-01 15:04:08,090 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:43605 after 30 s')

2025-12-01 15:04:08,060 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:36229 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:43605 after 30 s')

2025-12-01 15:04:08,056 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:40569 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:43605 after 30 s')

2025-12-01 15:04:08,044 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:43399 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:43605 after 30 s')

2025-12-01 15:03:38,954 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:42075

2025-12-01 15:03:38,952 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:42075', status: closed, memory: 0, processing: 0>

2025-12-01 15:03:37,926 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:43605', status: closed, memory: 0, processing: 0>

2025-12-01 15:03:27,894 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:43605

2025-12-01 15:03:27,891 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:43605', status: closed, memory: 0, processing: 0>

2025-12-01 15:03:26,828 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:39055', status: closed, memory: 0, processing: 0>

2025-12-01 15:03:16,231 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:39055

2025-12-01 15:03:16,229 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:39055', status: closed, memory: 0, processing: 0>

2025-12-01 15:03:15,229 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:33903', status: closed, memory: 0, processing: 0>

2025-12-01 15:03:04,165 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:45487 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:45487 after 30 s

2025-12-01 15:03:04,120 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:45487 after 30 s')

2025-12-01 15:03:04,059 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:45487 after 30 s')

2025-12-01 15:03:04,046 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:45487 after 30 s')

2025-12-01 15:03:04,041 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:45487 after 30 s')

2025-12-01 15:03:04,027 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41957 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:45487 after 30 s')

2025-12-01 15:03:04,026 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44619 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:45487 after 30 s')

2025-12-01 15:03:04,025 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:43399 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:45487 after 30 s')

2025-12-01 15:03:03,972 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:45487 after 30 s')

2025-12-01 15:02:34,849 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:33903

2025-12-01 15:02:34,847 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:33903', status: closed, memory: 0, processing: 0>

2025-12-01 15:02:33,859 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:45487', status: closed, memory: 0, processing: 0>

2025-12-01 15:02:19,396 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:37027 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:37027 after 30 s

2025-12-01 15:02:19,382 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:37027 after 30 s')

2025-12-01 15:02:18,606 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:37027 after 30 s')

2025-12-01 15:02:18,521 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:37027 after 30 s')

2025-12-01 15:02:18,471 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:37027 after 30 s')

2025-12-01 15:02:10,632 - distributed.scheduler - INFO - Close client connection: Client-84e5cfc5-cec6-11f0-8001-f2cd832f225c

2025-12-01 15:02:10,631 - distributed.scheduler - INFO - Remove client Client-84e5cfc5-cec6-11f0-8001-f2cd832f225c

2025-12-01 15:02:10,629 - distributed.scheduler - INFO - Remove client Client-84e5cfc5-cec6-11f0-8001-f2cd832f225c

2025-12-01 15:01:49,338 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:45487

2025-12-01 15:01:49,336 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:45487', status: closed, memory: 0, processing: 0>

2025-12-01 15:01:48,331 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:37027', status: closed, memory: 0, processing: 0>

2025-12-01 15:01:29,877 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:37027

2025-12-01 15:01:29,875 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:37027', status: closed, memory: 0, processing: 0>

2025-12-01 15:01:28,748 - distributed.scheduler - INFO - Close client connection: Client-worker-b088850e-cc38-11f0-85df-32a59f79d364

2025-12-01 15:01:28,741 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:34767', status: closed, memory: 0, processing: 0>

2025-12-01 15:01:28,711 - distributed.scheduler - INFO - Remove client Client-worker-b088850e-cc38-11f0-85df-32a59f79d364

2025-12-01 15:00:47,722 - distributed.scheduler - INFO - Receive client connection: Client-84e5cfc5-cec6-11f0-8001-f2cd832f225c

2025-12-01 14:51:25,524 - distributed.scheduler - INFO - Close client connection: Client-1e669514-cec5-11f0-8001-f2cd832f225c

2025-12-01 14:51:25,524 - distributed.scheduler - INFO - Remove client Client-1e669514-cec5-11f0-8001-f2cd832f225c

2025-12-01 14:51:25,522 - distributed.scheduler - INFO - Remove client Client-1e669514-cec5-11f0-8001-f2cd832f225c

2025-12-01 14:50:46,265 - distributed.scheduler - INFO - Receive client connection: Client-1e669514-cec5-11f0-8001-f2cd832f225c

2025-12-01 14:41:26,053 - distributed.scheduler - INFO - Close client connection: Client-ea609d25-ceb9-11f0-8001-f2cd832f225c

2025-12-01 14:41:26,053 - distributed.scheduler - INFO - Close client connection: Client-b76324c4-cebc-11f0-8001-f2cd832f225c

2025-12-01 14:41:26,052 - distributed.scheduler - INFO - Close client connection: Client-1de9b800-cebe-11f0-8001-f2cd832f225c

2025-12-01 14:41:26,052 - distributed.scheduler - INFO - Close client connection: Client-50e2d307-cebb-11f0-8001-f2cd832f225c

2025-12-01 14:41:26,051 - distributed.scheduler - INFO - Close client connection: Client-846cd4bb-cebf-11f0-8001-f2cd832f225c

2025-12-01 14:41:26,051 - distributed.scheduler - INFO - Close client connection: Client-eae96376-cec0-11f0-8001-f2cd832f225c

2025-12-01 14:41:26,051 - distributed.scheduler - INFO - Close client connection: Client-516ab186-cec2-11f0-8001-f2cd832f225c

2025-12-01 14:41:26,048 - distributed.scheduler - INFO - Remove client Client-ea609d25-ceb9-11f0-8001-f2cd832f225c

2025-12-01 14:41:26,047 - distributed.scheduler - INFO - Remove client Client-b76324c4-cebc-11f0-8001-f2cd832f225c

2025-12-01 14:41:26,044 - distributed.scheduler - INFO - Remove client Client-1de9b800-cebe-11f0-8001-f2cd832f225c

2025-12-01 14:41:26,043 - distributed.scheduler - INFO - Remove client Client-50e2d307-cebb-11f0-8001-f2cd832f225c

2025-12-01 14:41:26,043 - distributed.scheduler - INFO - Remove client Client-846cd4bb-cebf-11f0-8001-f2cd832f225c

2025-12-01 14:41:26,042 - distributed.scheduler - INFO - Remove client Client-eae96376-cec0-11f0-8001-f2cd832f225c

2025-12-01 14:41:26,040 - distributed.scheduler - INFO - Remove client Client-516ab186-cec2-11f0-8001-f2cd832f225c

2025-12-01 14:41:25,545 - distributed.scheduler - INFO - Close client connection: Client-b7e36fca-cec3-11f0-8001-f2cd832f225c

2025-12-01 14:41:25,544 - distributed.scheduler - INFO - Remove client Client-b7e36fca-cec3-11f0-8001-f2cd832f225c

2025-12-01 14:41:25,542 - distributed.scheduler - INFO - Remove client Client-b7e36fca-cec3-11f0-8001-f2cd832f225c

2025-12-01 14:40:44,777 - distributed.scheduler - INFO - Receive client connection: Client-b7e36fca-cec3-11f0-8001-f2cd832f225c

2025-12-01 14:30:43,359 - distributed.scheduler - INFO - Receive client connection: Client-516ab186-cec2-11f0-8001-f2cd832f225c

2025-12-01 14:20:41,889 - distributed.scheduler - INFO - Receive client connection: Client-eae96376-cec0-11f0-8001-f2cd832f225c

2025-12-01 14:10:40,446 - distributed.scheduler - INFO - Receive client connection: Client-846cd4bb-cebf-11f0-8001-f2cd832f225c

2025-12-01 14:00:38,960 - distributed.scheduler - INFO - Receive client connection: Client-1de9b800-cebe-11f0-8001-f2cd832f225c

2025-12-01 13:50:37,456 - distributed.scheduler - INFO - Receive client connection: Client-b76324c4-cebc-11f0-8001-f2cd832f225c

2025-12-01 13:40:35,995 - distributed.scheduler - INFO - Receive client connection: Client-50e2d307-cebb-11f0-8001-f2cd832f225c

2025-12-01 13:30:38,020 - distributed.scheduler - INFO - Receive client connection: Client-worker-ec77a45a-ceb9-11f0-85d6-2a2d08e6be2e

2025-12-01 13:30:34,527 - distributed.scheduler - INFO - Receive client connection: Client-ea609d25-ceb9-11f0-8001-f2cd832f225c

2025-12-01 13:21:08,079 - distributed.scheduler - INFO - Close client connection: Client-83efa99d-ceb8-11f0-8001-f2cd832f225c

2025-12-01 13:21:08,077 - distributed.scheduler - INFO - Remove client Client-83efa99d-ceb8-11f0-8001-f2cd832f225c

2025-12-01 13:21:08,075 - distributed.scheduler - INFO - Remove client Client-83efa99d-ceb8-11f0-8001-f2cd832f225c

2025-12-01 13:20:33,171 - distributed.scheduler - INFO - Receive client connection: Client-83efa99d-ceb8-11f0-8001-f2cd832f225c

2025-12-01 13:11:07,871 - distributed.scheduler - INFO - Close client connection: Client-1d6a9795-ceb7-11f0-8001-f2cd832f225c

2025-12-01 13:11:07,871 - distributed.scheduler - INFO - Remove client Client-1d6a9795-ceb7-11f0-8001-f2cd832f225c

2025-12-01 13:11:07,870 - distributed.scheduler - INFO - Remove client Client-1d6a9795-ceb7-11f0-8001-f2cd832f225c

2025-12-01 13:10:31,656 - distributed.scheduler - INFO - Receive client connection: Client-1d6a9795-ceb7-11f0-8001-f2cd832f225c

2025-12-01 13:01:37,114 - distributed.scheduler - INFO - Close client connection: Client-b6ef63e6-ceb5-11f0-8001-f2cd832f225c

2025-12-01 13:01:37,113 - distributed.scheduler - INFO - Remove client Client-b6ef63e6-ceb5-11f0-8001-f2cd832f225c

2025-12-01 13:01:37,111 - distributed.scheduler - INFO - Remove client Client-b6ef63e6-ceb5-11f0-8001-f2cd832f225c

2025-12-01 13:00:30,233 - distributed.scheduler - INFO - Receive client connection: Client-b6ef63e6-ceb5-11f0-8001-f2cd832f225c

2025-12-01 12:51:04,466 - distributed.scheduler - INFO - Close client connection: Client-50789b88-ceb4-11f0-8001-f2cd832f225c

2025-12-01 12:51:04,465 - distributed.scheduler - INFO - Remove client Client-50789b88-ceb4-11f0-8001-f2cd832f225c

2025-12-01 12:51:04,463 - distributed.scheduler - INFO - Remove client Client-50789b88-ceb4-11f0-8001-f2cd832f225c

2025-12-01 12:50:28,822 - distributed.scheduler - INFO - Receive client connection: Client-50789b88-ceb4-11f0-8001-f2cd832f225c

2025-12-01 12:41:04,819 - distributed.scheduler - INFO - Close client connection: Client-ea045147-ceb2-11f0-8001-f2cd832f225c

2025-12-01 12:41:04,818 - distributed.scheduler - INFO - Remove client Client-ea045147-ceb2-11f0-8001-f2cd832f225c

2025-12-01 12:41:04,817 - distributed.scheduler - INFO - Remove client Client-ea045147-ceb2-11f0-8001-f2cd832f225c

2025-12-01 12:40:27,445 - distributed.scheduler - INFO - Receive client connection: Client-ea045147-ceb2-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,518 - distributed.scheduler - INFO - Close client connection: Client-4fa7f883-cea6-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,518 - distributed.scheduler - INFO - Close client connection: Client-b6236e7f-cea7-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,517 - distributed.scheduler - INFO - Close client connection: Client-1ca01fc6-cea9-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,517 - distributed.scheduler - INFO - Close client connection: Client-83242441-ceaa-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,517 - distributed.scheduler - INFO - Close client connection: Client-e9a7fa61-ceab-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,516 - distributed.scheduler - INFO - Close client connection: Client-50240e55-cead-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,516 - distributed.scheduler - INFO - Close client connection: Client-b6a3eee1-ceae-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,515 - distributed.scheduler - INFO - Close client connection: Client-1d1d3822-ceb0-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,514 - distributed.scheduler - INFO - Remove client Client-4fa7f883-cea6-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,513 - distributed.scheduler - INFO - Remove client Client-b6236e7f-cea7-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,512 - distributed.scheduler - INFO - Remove client Client-1ca01fc6-cea9-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,511 - distributed.scheduler - INFO - Remove client Client-83242441-ceaa-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,510 - distributed.scheduler - INFO - Remove client Client-e9a7fa61-ceab-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,510 - distributed.scheduler - INFO - Remove client Client-50240e55-cead-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,506 - distributed.scheduler - INFO - Remove client Client-b6a3eee1-ceae-11f0-8001-f2cd832f225c

2025-12-01 12:31:38,505 - distributed.scheduler - INFO - Remove client Client-1d1d3822-ceb0-11f0-8001-f2cd832f225c

2025-12-01 12:31:37,812 - distributed.scheduler - INFO - Close client connection: Client-839502f0-ceb1-11f0-8001-f2cd832f225c

2025-12-01 12:31:37,811 - distributed.scheduler - INFO - Remove client Client-839502f0-ceb1-11f0-8001-f2cd832f225c

2025-12-01 12:31:37,809 - distributed.scheduler - INFO - Remove client Client-839502f0-ceb1-11f0-8001-f2cd832f225c

2025-12-01 12:30:26,079 - distributed.scheduler - INFO - Receive client connection: Client-839502f0-ceb1-11f0-8001-f2cd832f225c

2025-12-01 12:20:24,706 - distributed.scheduler - INFO - Receive client connection: Client-1d1d3822-ceb0-11f0-8001-f2cd832f225c

2025-12-01 12:10:23,250 - distributed.scheduler - INFO - Receive client connection: Client-b6a3eee1-ceae-11f0-8001-f2cd832f225c

2025-12-01 12:00:21,793 - distributed.scheduler - INFO - Receive client connection: Client-50240e55-cead-11f0-8001-f2cd832f225c

2025-12-01 11:50:20,346 - distributed.scheduler - INFO - Receive client connection: Client-e9a7fa61-ceab-11f0-8001-f2cd832f225c

2025-12-01 11:40:18,863 - distributed.scheduler - INFO - Receive client connection: Client-83242441-ceaa-11f0-8001-f2cd832f225c

2025-12-01 11:30:17,377 - distributed.scheduler - INFO - Receive client connection: Client-1ca01fc6-cea9-11f0-8001-f2cd832f225c

2025-12-01 11:20:15,927 - distributed.scheduler - INFO - Receive client connection: Client-b6236e7f-cea7-11f0-8001-f2cd832f225c

2025-12-01 11:10:14,509 - distributed.scheduler - INFO - Receive client connection: Client-4fa7f883-cea6-11f0-8001-f2cd832f225c

2025-12-01 11:01:22,486 - distributed.scheduler - INFO - Close client connection: Client-e91ef7ba-cea4-11f0-8001-f2cd832f225c

2025-12-01 11:01:22,486 - distributed.scheduler - INFO - Remove client Client-e91ef7ba-cea4-11f0-8001-f2cd832f225c

2025-12-01 11:01:22,484 - distributed.scheduler - INFO - Remove client Client-e91ef7ba-cea4-11f0-8001-f2cd832f225c

2025-12-01 11:00:12,978 - distributed.scheduler - INFO - Receive client connection: Client-e91ef7ba-cea4-11f0-8001-f2cd832f225c

2025-12-01 10:50:39,824 - distributed.scheduler - INFO - Close client connection: Client-82aa28c2-cea3-11f0-8001-f2cd832f225c

2025-12-01 10:50:39,823 - distributed.scheduler - INFO - Remove client Client-82aa28c2-cea3-11f0-8001-f2cd832f225c

2025-12-01 10:50:39,821 - distributed.scheduler - INFO - Remove client Client-82aa28c2-cea3-11f0-8001-f2cd832f225c

2025-12-01 10:50:11,593 - distributed.scheduler - INFO - Receive client connection: Client-82aa28c2-cea3-11f0-8001-f2cd832f225c

2025-12-01 10:40:47,149 - distributed.scheduler - INFO - Close client connection: Client-1c2d2eee-cea2-11f0-8001-f2cd832f225c

2025-12-01 10:40:47,149 - distributed.scheduler - INFO - Remove client Client-1c2d2eee-cea2-11f0-8001-f2cd832f225c

2025-12-01 10:40:47,148 - distributed.scheduler - INFO - Remove client Client-1c2d2eee-cea2-11f0-8001-f2cd832f225c

2025-12-01 10:40:10,152 - distributed.scheduler - INFO - Receive client connection: Client-1c2d2eee-cea2-11f0-8001-f2cd832f225c

2025-12-01 10:31:37,460 - distributed.scheduler - INFO - Receive client connection: Client-worker-ea963dc8-cea0-11f0-85fe-32001d5d2425

2025-12-01 10:31:36,435 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.102:40569

2025-12-01 10:31:36,434 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.102:40569', status: closed, memory: 0, processing: 0>

2025-12-01 10:31:35,573 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.102:38633', status: closed, memory: 0, processing: 0>

2025-12-01 10:31:34,103 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:44619

2025-12-01 10:31:34,101 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:44619', status: closed, memory: 0, processing: 0>

2025-12-01 10:31:33,157 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:38003', status: closed, memory: 0, processing: 0>

2025-12-01 10:31:29,734 - distributed.scheduler - INFO - Close client connection: Client-b5b258bc-cea0-11f0-8001-f2cd832f225c

2025-12-01 10:31:29,732 - distributed.scheduler - INFO - Remove client Client-b5b258bc-cea0-11f0-8001-f2cd832f225c

2025-12-01 10:31:29,731 - distributed.scheduler - INFO - Remove client Client-b5b258bc-cea0-11f0-8001-f2cd832f225c

2025-12-01 10:30:56,853 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.102:38633

2025-12-01 10:30:56,852 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.102:38633', status: closed, memory: 0, processing: 0>

2025-12-01 10:30:55,921 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:38003 failed: Exception: OSError('Timed out during handshake while connecting to tcp://172.21.25.102:41965 after 30 s')

2025-12-01 10:30:55,827 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41965 failed: OSError: Timed out during handshake while connecting to tcp://172.21.25.102:41965 after 30 s

2025-12-01 10:30:55,827 - distributed.scheduler - INFO - Close client connection: Client-worker-be204c78-cbb2-11f0-83fc-32001d5d2425

2025-12-01 10:30:55,826 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:44154 remote=tcp://172.21.25.24:34199>: Stream is closed

2025-12-01 10:30:55,826 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:54494 remote=tcp://172.21.159.233:46261>: Stream is closed

2025-12-01 10:30:55,826 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:38318 remote=tcp://172.21.12.165:37233>: Stream is closed

2025-12-01 10:30:55,823 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:38266 remote=tcp://172.21.159.248:37517>: Stream is closed

2025-12-01 10:30:55,823 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:36229 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:40036 remote=tcp://172.21.25.101:36229>: Stream is closed

2025-12-01 10:30:55,822 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:59640 remote=tcp://172.21.25.104:39025>: Stream is closed

2025-12-01 10:30:55,822 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41957 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:41194 remote=tcp://172.21.159.252:41957>: Stream is closed

2025-12-01 10:30:55,822 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:34767 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:33962 remote=tcp://172.21.25.97:34767>: Stream is closed

2025-12-01 10:30:55,822 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:59586 remote=tcp://172.21.25.6:45767>: Stream is closed

2025-12-01 10:30:55,822 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:43399 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:39266 remote=tcp://172.21.159.210:43399>: Stream is closed

2025-12-01 10:30:55,822 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:58472 remote=tcp://172.21.159.199:36327>: Stream is closed

2025-12-01 10:30:55,822 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:53346 remote=tcp://172.21.12.191:44023>: Stream is closed

2025-12-01 10:30:55,812 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.102:41965', status: closed, memory: 0, processing: 0>

2025-12-01 10:30:55,811 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41965 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:42172 remote=tcp://172.21.25.102:41965>: ConnectionResetError: [Errno 104] Connection reset by peer

2025-12-01 10:30:55,810 - distributed.scheduler - INFO - Remove client Client-worker-be204c78-cbb2-11f0-83fc-32001d5d2425

2025-12-01 10:30:43,721 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:38003

2025-12-01 10:30:43,720 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:38003', status: closed, memory: 0, processing: 0>

2025-12-01 10:30:42,768 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:40527 failed: OSError: Timed out during handshake while connecting to tcp://172.21.12.199:40527 after 30 s

2025-12-01 10:30:42,768 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:40527 failed: OSError: Timed out during handshake while connecting to tcp://172.21.12.199:40527 after 30 s

2025-12-01 10:30:42,767 - distributed.scheduler - INFO - Close client connection: Client-worker-a5379bec-caed-11f0-8411-2a2d08e6be2e

2025-12-01 10:30:42,763 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:40527', status: closed, memory: 0, processing: 0>

2025-12-01 10:30:42,761 - distributed.scheduler - INFO - Remove client Client-worker-a5379bec-caed-11f0-8411-2a2d08e6be2e

2025-12-01 10:30:08,723 - distributed.scheduler - INFO - Receive client connection: Client-b5b258bc-cea0-11f0-8001-f2cd832f225c

2025-12-01 10:20:42,994 - distributed.scheduler - INFO - Close client connection: Client-4f2c84b6-ce9f-11f0-8001-f2cd832f225c

2025-12-01 10:20:42,993 - distributed.scheduler - INFO - Remove client Client-4f2c84b6-ce9f-11f0-8001-f2cd832f225c

2025-12-01 10:20:42,991 - distributed.scheduler - INFO - Remove client Client-4f2c84b6-ce9f-11f0-8001-f2cd832f225c

2025-12-01 10:20:07,215 - distributed.scheduler - INFO - Receive client connection: Client-4f2c84b6-ce9f-11f0-8001-f2cd832f225c

2025-12-01 10:10:40,804 - distributed.scheduler - INFO - Close client connection: Client-e8ac8314-ce9d-11f0-8001-f2cd832f225c

2025-12-01 10:10:40,803 - distributed.scheduler - INFO - Remove client Client-e8ac8314-ce9d-11f0-8001-f2cd832f225c

2025-12-01 10:10:40,802 - distributed.scheduler - INFO - Remove client Client-e8ac8314-ce9d-11f0-8001-f2cd832f225c

2025-12-01 10:10:05,771 - distributed.scheduler - INFO - Receive client connection: Client-e8ac8314-ce9d-11f0-8001-f2cd832f225c

2025-12-01 10:01:11,505 - distributed.scheduler - INFO - Close client connection: Client-821621f7-ce9c-11f0-8001-f2cd832f225c

2025-12-01 10:01:11,504 - distributed.scheduler - INFO - Remove client Client-821621f7-ce9c-11f0-8001-f2cd832f225c

2025-12-01 10:01:11,502 - distributed.scheduler - INFO - Remove client Client-821621f7-ce9c-11f0-8001-f2cd832f225c

2025-12-01 10:00:04,148 - distributed.scheduler - INFO - Receive client connection: Client-821621f7-ce9c-11f0-8001-f2cd832f225c

2025-12-01 09:50:41,306 - distributed.scheduler - INFO - Close client connection: Client-1b937768-ce9b-11f0-8001-f2cd832f225c

2025-12-01 09:50:41,305 - distributed.scheduler - INFO - Remove client Client-1b937768-ce9b-11f0-8001-f2cd832f225c

2025-12-01 09:50:41,303 - distributed.scheduler - INFO - Remove client Client-1b937768-ce9b-11f0-8001-f2cd832f225c

2025-12-01 09:50:02,668 - distributed.scheduler - INFO - Receive client connection: Client-1b937768-ce9b-11f0-8001-f2cd832f225c

2025-12-01 09:40:38,334 - distributed.scheduler - INFO - Close client connection: Client-b510a0be-ce99-11f0-8001-f2cd832f225c

2025-12-01 09:40:38,333 - distributed.scheduler - INFO - Remove client Client-b510a0be-ce99-11f0-8001-f2cd832f225c

2025-12-01 09:40:38,331 - distributed.scheduler - INFO - Remove client Client-b510a0be-ce99-11f0-8001-f2cd832f225c

2025-12-01 09:40:01,188 - distributed.scheduler - INFO - Receive client connection: Client-b510a0be-ce99-11f0-8001-f2cd832f225c

2025-12-01 09:32:08,974 - distributed.scheduler - INFO - Close client connection: Client-726d166d-ce98-11f0-8001-f2cd832f225c

2025-12-01 09:32:08,972 - distributed.scheduler - INFO - Remove client Client-726d166d-ce98-11f0-8001-f2cd832f225c

2025-12-01 09:32:08,970 - distributed.scheduler - INFO - Remove client Client-726d166d-ce98-11f0-8001-f2cd832f225c

2025-12-01 09:30:59,902 - distributed.scheduler - INFO - Receive client connection: Client-726d166d-ce98-11f0-8001-f2cd832f225c

2025-12-01 09:21:37,859 - distributed.scheduler - INFO - Close client connection: Client-0bea66a6-ce97-11f0-8001-f2cd832f225c

2025-12-01 09:21:37,858 - distributed.scheduler - INFO - Remove client Client-0bea66a6-ce97-11f0-8001-f2cd832f225c

2025-12-01 09:21:37,857 - distributed.scheduler - INFO - Remove client Client-0bea66a6-ce97-11f0-8001-f2cd832f225c

2025-12-01 09:20:58,408 - distributed.scheduler - INFO - Receive client connection: Client-0bea66a6-ce97-11f0-8001-f2cd832f225c

2025-12-01 09:11:35,915 - distributed.scheduler - INFO - Close client connection: Client-a57bcc0a-ce95-11f0-8001-f2cd832f225c

2025-12-01 09:11:35,913 - distributed.scheduler - INFO - Remove client Client-a57bcc0a-ce95-11f0-8001-f2cd832f225c

2025-12-01 09:11:35,911 - distributed.scheduler - INFO - Remove client Client-a57bcc0a-ce95-11f0-8001-f2cd832f225c

2025-12-01 09:10:57,061 - distributed.scheduler - INFO - Receive client connection: Client-a57bcc0a-ce95-11f0-8001-f2cd832f225c

2025-12-01 09:02:11,017 - distributed.scheduler - INFO - Close client connection: Client-3ef60465-ce94-11f0-8001-f2cd832f225c

2025-12-01 09:02:11,016 - distributed.scheduler - INFO - Remove client Client-3ef60465-ce94-11f0-8001-f2cd832f225c

2025-12-01 09:02:11,014 - distributed.scheduler - INFO - Remove client Client-3ef60465-ce94-11f0-8001-f2cd832f225c

2025-12-01 09:00:55,553 - distributed.scheduler - INFO - Receive client connection: Client-3ef60465-ce94-11f0-8001-f2cd832f225c

2025-12-01 08:51:35,168 - distributed.scheduler - INFO - Close client connection: Client-a4e8c778-ce8e-11f0-8001-f2cd832f225c

2025-12-01 08:51:35,167 - distributed.scheduler - INFO - Close client connection: Client-0b70a382-ce90-11f0-8001-f2cd832f225c

2025-12-01 08:51:35,165 - distributed.scheduler - INFO - Close client connection: Client-71f33dc0-ce91-11f0-8001-f2cd832f225c

2025-12-01 08:51:35,164 - distributed.scheduler - INFO - Remove client Client-a4e8c778-ce8e-11f0-8001-f2cd832f225c

2025-12-01 08:51:35,163 - distributed.scheduler - INFO - Remove client Client-0b70a382-ce90-11f0-8001-f2cd832f225c

2025-12-01 08:51:35,161 - distributed.scheduler - INFO - Remove client Client-71f33dc0-ce91-11f0-8001-f2cd832f225c

2025-12-01 08:51:34,365 - distributed.scheduler - INFO - Close client connection: Client-d874d9bc-ce92-11f0-8001-f2cd832f225c

2025-12-01 08:51:34,362 - distributed.scheduler - INFO - Remove client Client-d874d9bc-ce92-11f0-8001-f2cd832f225c

2025-12-01 08:51:34,360 - distributed.scheduler - INFO - Remove client Client-d874d9bc-ce92-11f0-8001-f2cd832f225c

2025-12-01 08:50:54,078 - distributed.scheduler - INFO - Receive client connection: Client-d874d9bc-ce92-11f0-8001-f2cd832f225c

2025-12-01 08:40:52,600 - distributed.scheduler - INFO - Receive client connection: Client-71f33dc0-ce91-11f0-8001-f2cd832f225c

2025-12-01 08:30:51,122 - distributed.scheduler - INFO - Receive client connection: Client-0b70a382-ce90-11f0-8001-f2cd832f225c

2025-12-01 08:20:49,618 - distributed.scheduler - INFO - Receive client connection: Client-a4e8c778-ce8e-11f0-8001-f2cd832f225c

2025-12-01 08:11:22,651 - distributed.scheduler - INFO - Close client connection: Client-3e6aab10-ce8d-11f0-8001-f2cd832f225c

2025-12-01 08:11:22,650 - distributed.scheduler - INFO - Remove client Client-3e6aab10-ce8d-11f0-8001-f2cd832f225c

2025-12-01 08:11:22,648 - distributed.scheduler - INFO - Remove client Client-3e6aab10-ce8d-11f0-8001-f2cd832f225c

2025-12-01 08:10:48,172 - distributed.scheduler - INFO - Receive client connection: Client-3e6aab10-ce8d-11f0-8001-f2cd832f225c

2025-12-01 08:01:57,092 - distributed.scheduler - INFO - Close client connection: Client-d7d216f3-ce8b-11f0-8001-f2cd832f225c

2025-12-01 08:01:57,092 - distributed.scheduler - INFO - Remove client Client-d7d216f3-ce8b-11f0-8001-f2cd832f225c

2025-12-01 08:01:57,092 - distributed.scheduler - INFO - Remove client Client-d7d216f3-ce8b-11f0-8001-f2cd832f225c

2025-12-01 08:00:46,547 - distributed.scheduler - INFO - Receive client connection: Client-d7d216f3-ce8b-11f0-8001-f2cd832f225c

2025-12-01 07:51:23,228 - distributed.scheduler - INFO - Close client connection: Client-714c40e3-ce8a-11f0-8001-f2cd832f225c

2025-12-01 07:51:23,227 - distributed.scheduler - INFO - Remove client Client-714c40e3-ce8a-11f0-8001-f2cd832f225c

2025-12-01 07:51:23,225 - distributed.scheduler - INFO - Remove client Client-714c40e3-ce8a-11f0-8001-f2cd832f225c

2025-12-01 07:50:45,041 - distributed.scheduler - INFO - Receive client connection: Client-714c40e3-ce8a-11f0-8001-f2cd832f225c

2025-12-01 07:41:23,146 - distributed.scheduler - INFO - Close client connection: Client-0acbd02e-ce89-11f0-8001-f2cd832f225c

2025-12-01 07:41:23,144 - distributed.scheduler - INFO - Remove client Client-0acbd02e-ce89-11f0-8001-f2cd832f225c

2025-12-01 07:41:23,142 - distributed.scheduler - INFO - Remove client Client-0acbd02e-ce89-11f0-8001-f2cd832f225c

2025-12-01 07:40:43,577 - distributed.scheduler - INFO - Receive client connection: Client-0acbd02e-ce89-11f0-8001-f2cd832f225c

2025-12-01 07:31:56,219 - distributed.scheduler - INFO - Close client connection: Client-a45284ba-ce87-11f0-8001-f2cd832f225c

2025-12-01 07:31:56,218 - distributed.scheduler - INFO - Remove client Client-a45284ba-ce87-11f0-8001-f2cd832f225c

2025-12-01 07:31:56,217 - distributed.scheduler - INFO - Remove client Client-a45284ba-ce87-11f0-8001-f2cd832f225c

2025-12-01 07:30:42,155 - distributed.scheduler - INFO - Receive client connection: Client-a45284ba-ce87-11f0-8001-f2cd832f225c

2025-12-01 07:21:22,627 - distributed.scheduler - INFO - Close client connection: Client-3dd32017-ce86-11f0-8001-f2cd832f225c

2025-12-01 07:21:22,626 - distributed.scheduler - INFO - Remove client Client-3dd32017-ce86-11f0-8001-f2cd832f225c

2025-12-01 07:21:22,624 - distributed.scheduler - INFO - Remove client Client-3dd32017-ce86-11f0-8001-f2cd832f225c

2025-12-01 07:20:40,687 - distributed.scheduler - INFO - Receive client connection: Client-3dd32017-ce86-11f0-8001-f2cd832f225c

2025-12-01 07:11:21,900 - distributed.scheduler - INFO - Close client connection: Client-d7549249-ce84-11f0-8001-f2cd832f225c

2025-12-01 07:11:21,899 - distributed.scheduler - INFO - Remove client Client-d7549249-ce84-11f0-8001-f2cd832f225c

2025-12-01 07:11:21,897 - distributed.scheduler - INFO - Remove client Client-d7549249-ce84-11f0-8001-f2cd832f225c

2025-12-01 07:10:39,238 - distributed.scheduler - INFO - Receive client connection: Client-d7549249-ce84-11f0-8001-f2cd832f225c

2025-12-01 07:01:53,418 - distributed.scheduler - INFO - Close client connection: Client-70c4acee-ce83-11f0-8001-f2cd832f225c

2025-12-01 07:01:53,415 - distributed.scheduler - INFO - Remove client Client-70c4acee-ce83-11f0-8001-f2cd832f225c

2025-12-01 07:01:53,413 - distributed.scheduler - INFO - Remove client Client-70c4acee-ce83-11f0-8001-f2cd832f225c

2025-12-01 07:00:37,679 - distributed.scheduler - INFO - Receive client connection: Client-70c4acee-ce83-11f0-8001-f2cd832f225c

2025-12-01 06:51:19,204 - distributed.scheduler - INFO - Close client connection: Client-0a3ec976-ce82-11f0-8001-f2cd832f225c

2025-12-01 06:51:19,202 - distributed.scheduler - INFO - Remove client Client-0a3ec976-ce82-11f0-8001-f2cd832f225c

2025-12-01 06:51:19,201 - distributed.scheduler - INFO - Remove client Client-0a3ec976-ce82-11f0-8001-f2cd832f225c

2025-12-01 06:50:36,173 - distributed.scheduler - INFO - Receive client connection: Client-0a3ec976-ce82-11f0-8001-f2cd832f225c

2025-12-01 06:41:18,405 - distributed.scheduler - INFO - Close client connection: Client-a3bb548f-ce80-11f0-8001-f2cd832f225c

2025-12-01 06:41:18,403 - distributed.scheduler - INFO - Remove client Client-a3bb548f-ce80-11f0-8001-f2cd832f225c

2025-12-01 06:41:18,401 - distributed.scheduler - INFO - Remove client Client-a3bb548f-ce80-11f0-8001-f2cd832f225c

2025-12-01 06:40:34,707 - distributed.scheduler - INFO - Receive client connection: Client-a3bb548f-ce80-11f0-8001-f2cd832f225c

2025-12-01 06:31:52,926 - distributed.scheduler - INFO - Close client connection: Client-3d3bed21-ce7f-11f0-8001-f2cd832f225c

2025-12-01 06:31:52,925 - distributed.scheduler - INFO - Remove client Client-3d3bed21-ce7f-11f0-8001-f2cd832f225c

2025-12-01 06:31:52,923 - distributed.scheduler - INFO - Remove client Client-3d3bed21-ce7f-11f0-8001-f2cd832f225c

2025-12-01 06:30:33,225 - distributed.scheduler - INFO - Receive client connection: Client-3d3bed21-ce7f-11f0-8001-f2cd832f225c

2025-12-01 06:21:15,598 - distributed.scheduler - INFO - Close client connection: Client-d6bb91ee-ce7d-11f0-8001-f2cd832f225c

2025-12-01 06:21:15,597 - distributed.scheduler - INFO - Remove client Client-d6bb91ee-ce7d-11f0-8001-f2cd832f225c

2025-12-01 06:21:15,594 - distributed.scheduler - INFO - Remove client Client-d6bb91ee-ce7d-11f0-8001-f2cd832f225c

2025-12-01 06:20:31,763 - distributed.scheduler - INFO - Receive client connection: Client-d6bb91ee-ce7d-11f0-8001-f2cd832f225c

2025-12-01 06:11:13,036 - distributed.scheduler - INFO - Close client connection: Client-703d4050-ce7c-11f0-8001-f2cd832f225c

2025-12-01 06:11:13,035 - distributed.scheduler - INFO - Remove client Client-703d4050-ce7c-11f0-8001-f2cd832f225c

2025-12-01 06:11:13,033 - distributed.scheduler - INFO - Remove client Client-703d4050-ce7c-11f0-8001-f2cd832f225c

2025-12-01 06:10:30,311 - distributed.scheduler - INFO - Receive client connection: Client-703d4050-ce7c-11f0-8001-f2cd832f225c

2025-12-01 06:01:51,995 - distributed.scheduler - INFO - Close client connection: Client-09b45474-ce7b-11f0-8001-f2cd832f225c

2025-12-01 06:01:51,993 - distributed.scheduler - INFO - Remove client Client-09b45474-ce7b-11f0-8001-f2cd832f225c

2025-12-01 06:01:51,991 - distributed.scheduler - INFO - Remove client Client-09b45474-ce7b-11f0-8001-f2cd832f225c

2025-12-01 06:00:28,791 - distributed.scheduler - INFO - Receive client connection: Client-09b45474-ce7b-11f0-8001-f2cd832f225c

2025-12-01 05:51:08,073 - distributed.scheduler - INFO - Close client connection: Client-a33c70aa-ce79-11f0-8001-f2cd832f225c

2025-12-01 05:51:08,073 - distributed.scheduler - INFO - Remove client Client-a33c70aa-ce79-11f0-8001-f2cd832f225c

2025-12-01 05:51:08,072 - distributed.scheduler - INFO - Remove client Client-a33c70aa-ce79-11f0-8001-f2cd832f225c

2025-12-01 05:50:27,376 - distributed.scheduler - INFO - Receive client connection: Client-a33c70aa-ce79-11f0-8001-f2cd832f225c

2025-12-01 05:41:10,068 - distributed.scheduler - INFO - Close client connection: Client-3e1f97fb-ce78-11f0-8001-f2cd832f225c

2025-12-01 05:41:10,067 - distributed.scheduler - INFO - Remove client Client-3e1f97fb-ce78-11f0-8001-f2cd832f225c

2025-12-01 05:41:10,066 - distributed.scheduler - INFO - Remove client Client-3e1f97fb-ce78-11f0-8001-f2cd832f225c

2025-12-01 05:40:28,285 - distributed.scheduler - INFO - Receive client connection: Client-3e1f97fb-ce78-11f0-8001-f2cd832f225c

2025-12-01 05:31:40,899 - distributed.scheduler - INFO - Close client connection: Client-d64c0931-ce76-11f0-8001-f2cd832f225c

2025-12-01 05:31:40,898 - distributed.scheduler - INFO - Remove client Client-d64c0931-ce76-11f0-8001-f2cd832f225c

2025-12-01 05:31:40,896 - distributed.scheduler - INFO - Remove client Client-d64c0931-ce76-11f0-8001-f2cd832f225c

2025-12-01 05:30:24,551 - distributed.scheduler - INFO - Receive client connection: Client-d64c0931-ce76-11f0-8001-f2cd832f225c

2025-12-01 05:21:00,178 - distributed.scheduler - INFO - Close client connection: Client-6fcdb616-ce75-11f0-8001-f2cd832f225c

2025-12-01 05:21:00,177 - distributed.scheduler - INFO - Remove client Client-6fcdb616-ce75-11f0-8001-f2cd832f225c

2025-12-01 05:21:00,175 - distributed.scheduler - INFO - Remove client Client-6fcdb616-ce75-11f0-8001-f2cd832f225c

2025-12-01 05:20:23,101 - distributed.scheduler - INFO - Receive client connection: Client-6fcdb616-ce75-11f0-8001-f2cd832f225c

2025-12-01 05:10:59,500 - distributed.scheduler - INFO - Close client connection: Client-093a34fc-ce74-11f0-8001-f2cd832f225c

2025-12-01 05:10:59,497 - distributed.scheduler - INFO - Remove client Client-093a34fc-ce74-11f0-8001-f2cd832f225c

2025-12-01 05:10:59,496 - distributed.scheduler - INFO - Remove client Client-093a34fc-ce74-11f0-8001-f2cd832f225c

2025-12-01 05:10:21,513 - distributed.scheduler - INFO - Receive client connection: Client-093a34fc-ce74-11f0-8001-f2cd832f225c

2025-12-01 05:01:33,140 - distributed.scheduler - INFO - Close client connection: Client-a2ad75e7-ce72-11f0-8001-f2cd832f225c

2025-12-01 05:01:33,139 - distributed.scheduler - INFO - Remove client Client-a2ad75e7-ce72-11f0-8001-f2cd832f225c

2025-12-01 05:01:33,136 - distributed.scheduler - INFO - Remove client Client-a2ad75e7-ce72-11f0-8001-f2cd832f225c

2025-12-01 05:00:19,964 - distributed.scheduler - INFO - Receive client connection: Client-a2ad75e7-ce72-11f0-8001-f2cd832f225c

2025-12-01 04:50:50,396 - distributed.scheduler - INFO - Close client connection: Client-3c24798c-ce71-11f0-8001-f2cd832f225c

2025-12-01 04:50:50,395 - distributed.scheduler - INFO - Remove client Client-3c24798c-ce71-11f0-8001-f2cd832f225c

2025-12-01 04:50:50,393 - distributed.scheduler - INFO - Remove client Client-3c24798c-ce71-11f0-8001-f2cd832f225c

2025-12-01 04:50:18,442 - distributed.scheduler - INFO - Receive client connection: Client-3c24798c-ce71-11f0-8001-f2cd832f225c

2025-12-01 04:40:48,395 - distributed.scheduler - INFO - Close client connection: Client-d5a7896b-ce6f-11f0-8001-f2cd832f225c

2025-12-01 04:40:48,394 - distributed.scheduler - INFO - Remove client Client-d5a7896b-ce6f-11f0-8001-f2cd832f225c

2025-12-01 04:40:48,393 - distributed.scheduler - INFO - Remove client Client-d5a7896b-ce6f-11f0-8001-f2cd832f225c

2025-12-01 04:40:16,996 - distributed.scheduler - INFO - Receive client connection: Client-d5a7896b-ce6f-11f0-8001-f2cd832f225c

2025-12-01 04:31:21,220 - distributed.scheduler - INFO - Close client connection: Client-6f2a29a2-ce6e-11f0-8001-f2cd832f225c

2025-12-01 04:31:21,220 - distributed.scheduler - INFO - Remove client Client-6f2a29a2-ce6e-11f0-8001-f2cd832f225c

2025-12-01 04:31:21,219 - distributed.scheduler - INFO - Remove client Client-6f2a29a2-ce6e-11f0-8001-f2cd832f225c

2025-12-01 04:30:15,550 - distributed.scheduler - INFO - Receive client connection: Client-6f2a29a2-ce6e-11f0-8001-f2cd832f225c

2025-12-01 04:20:44,654 - distributed.scheduler - INFO - Close client connection: Client-08af7ade-ce6d-11f0-8001-f2cd832f225c

2025-12-01 04:20:44,653 - distributed.scheduler - INFO - Remove client Client-08af7ade-ce6d-11f0-8001-f2cd832f225c

2025-12-01 04:20:44,652 - distributed.scheduler - INFO - Remove client Client-08af7ade-ce6d-11f0-8001-f2cd832f225c

2025-12-01 04:20:14,122 - distributed.scheduler - INFO - Receive client connection: Client-08af7ade-ce6d-11f0-8001-f2cd832f225c

2025-12-01 04:10:42,486 - distributed.scheduler - INFO - Close client connection: Client-a22e34ab-ce6b-11f0-8001-f2cd832f225c

2025-12-01 04:10:42,485 - distributed.scheduler - INFO - Remove client Client-a22e34ab-ce6b-11f0-8001-f2cd832f225c

2025-12-01 04:10:42,484 - distributed.scheduler - INFO - Remove client Client-a22e34ab-ce6b-11f0-8001-f2cd832f225c

2025-12-01 04:10:12,652 - distributed.scheduler - INFO - Receive client connection: Client-a22e34ab-ce6b-11f0-8001-f2cd832f225c

2025-12-01 04:01:10,644 - distributed.scheduler - INFO - Close client connection: Client-081a4291-ce66-11f0-8001-f2cd832f225c

2025-12-01 04:01:10,643 - distributed.scheduler - INFO - Close client connection: Client-6e9a7d03-ce67-11f0-8001-f2cd832f225c

2025-12-01 04:01:10,643 - distributed.scheduler - INFO - Close client connection: Client-d522df7d-ce68-11f0-8001-f2cd832f225c

2025-12-01 04:01:10,641 - distributed.scheduler - INFO - Remove client Client-081a4291-ce66-11f0-8001-f2cd832f225c

2025-12-01 04:01:10,641 - distributed.scheduler - INFO - Remove client Client-6e9a7d03-ce67-11f0-8001-f2cd832f225c

2025-12-01 04:01:10,640 - distributed.scheduler - INFO - Remove client Client-d522df7d-ce68-11f0-8001-f2cd832f225c

2025-12-01 04:01:10,002 - distributed.scheduler - INFO - Close client connection: Client-3ba877d4-ce6a-11f0-8001-f2cd832f225c

2025-12-01 04:01:10,001 - distributed.scheduler - INFO - Remove client Client-3ba877d4-ce6a-11f0-8001-f2cd832f225c

2025-12-01 04:01:10,000 - distributed.scheduler - INFO - Remove client Client-3ba877d4-ce6a-11f0-8001-f2cd832f225c

2025-12-01 04:00:11,146 - distributed.scheduler - INFO - Receive client connection: Client-3ba877d4-ce6a-11f0-8001-f2cd832f225c

2025-12-01 03:50:09,645 - distributed.scheduler - INFO - Receive client connection: Client-d522df7d-ce68-11f0-8001-f2cd832f225c

2025-12-01 03:40:08,121 - distributed.scheduler - INFO - Receive client connection: Client-6e9a7d03-ce67-11f0-8001-f2cd832f225c

2025-12-01 03:30:06,671 - distributed.scheduler - INFO - Receive client connection: Client-081a4291-ce66-11f0-8001-f2cd832f225c

2025-12-01 03:20:42,904 - distributed.scheduler - INFO - Close client connection: Client-a1964826-ce64-11f0-8001-f2cd832f225c

2025-12-01 03:20:42,903 - distributed.scheduler - INFO - Remove client Client-a1964826-ce64-11f0-8001-f2cd832f225c

2025-12-01 03:20:42,901 - distributed.scheduler - INFO - Remove client Client-a1964826-ce64-11f0-8001-f2cd832f225c

2025-12-01 03:20:05,180 - distributed.scheduler - INFO - Receive client connection: Client-a1964826-ce64-11f0-8001-f2cd832f225c

2025-12-01 03:10:34,161 - distributed.scheduler - INFO - Close client connection: Client-3b17eac4-ce63-11f0-8001-f2cd832f225c

2025-12-01 03:10:34,160 - distributed.scheduler - INFO - Remove client Client-3b17eac4-ce63-11f0-8001-f2cd832f225c

2025-12-01 03:10:34,158 - distributed.scheduler - INFO - Remove client Client-3b17eac4-ce63-11f0-8001-f2cd832f225c

2025-12-01 03:10:03,726 - distributed.scheduler - INFO - Receive client connection: Client-3b17eac4-ce63-11f0-8001-f2cd832f225c

2025-12-01 03:01:01,327 - distributed.scheduler - INFO - Close client connection: Client-d48c3037-ce61-11f0-8001-f2cd832f225c

2025-12-01 03:01:01,326 - distributed.scheduler - INFO - Remove client Client-d48c3037-ce61-11f0-8001-f2cd832f225c

2025-12-01 03:01:01,324 - distributed.scheduler - INFO - Remove client Client-d48c3037-ce61-11f0-8001-f2cd832f225c

2025-12-01 03:00:02,186 - distributed.scheduler - INFO - Receive client connection: Client-d48c3037-ce61-11f0-8001-f2cd832f225c

2025-12-01 02:50:30,358 - distributed.scheduler - INFO - Close client connection: Client-6e0e4997-ce60-11f0-8001-f2cd832f225c

2025-12-01 02:50:30,357 - distributed.scheduler - INFO - Remove client Client-6e0e4997-ce60-11f0-8001-f2cd832f225c

2025-12-01 02:50:30,356 - distributed.scheduler - INFO - Remove client Client-6e0e4997-ce60-11f0-8001-f2cd832f225c

2025-12-01 02:50:00,737 - distributed.scheduler - INFO - Receive client connection: Client-6e0e4997-ce60-11f0-8001-f2cd832f225c

2025-12-01 02:41:27,847 - distributed.scheduler - INFO - Close client connection: Client-2b5fdd65-ce5f-11f0-8001-f2cd832f225c

2025-12-01 02:41:27,846 - distributed.scheduler - INFO - Remove client Client-2b5fdd65-ce5f-11f0-8001-f2cd832f225c

2025-12-01 02:41:27,844 - distributed.scheduler - INFO - Remove client Client-2b5fdd65-ce5f-11f0-8001-f2cd832f225c

2025-12-01 02:40:59,371 - distributed.scheduler - INFO - Receive client connection: Client-2b5fdd65-ce5f-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,804 - distributed.scheduler - INFO - Close client connection: Client-8e69fef1-ce2f-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,804 - distributed.scheduler - INFO - Close client connection: Client-f4f22c0e-ce30-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,804 - distributed.scheduler - INFO - Close client connection: Client-5b6ebc7a-ce32-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,804 - distributed.scheduler - INFO - Close client connection: Client-c1f023a9-ce33-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,804 - distributed.scheduler - INFO - Close client connection: Client-28780ba7-ce35-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,804 - distributed.scheduler - INFO - Close client connection: Client-8efaa7a9-ce36-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,803 - distributed.scheduler - INFO - Close client connection: Client-f572ae3b-ce37-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,803 - distributed.scheduler - INFO - Close client connection: Client-5bec5165-ce39-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,803 - distributed.scheduler - INFO - Close client connection: Client-c26bfe9c-ce3a-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,803 - distributed.scheduler - INFO - Close client connection: Client-28ecb445-ce3c-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,803 - distributed.scheduler - INFO - Close client connection: Client-8f6f8176-ce3d-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,802 - distributed.scheduler - INFO - Close client connection: Client-f5ee153a-ce3e-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,802 - distributed.scheduler - INFO - Close client connection: Client-5c5ef6ed-ce40-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,802 - distributed.scheduler - INFO - Close client connection: Client-c2d97fb5-ce41-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,802 - distributed.scheduler - INFO - Close client connection: Client-2956eb54-ce43-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,802 - distributed.scheduler - INFO - Close client connection: Client-8fd509ac-ce44-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,802 - distributed.scheduler - INFO - Close client connection: Client-f653b9ed-ce45-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,802 - distributed.scheduler - INFO - Close client connection: Client-5cc8ce5b-ce47-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,802 - distributed.scheduler - INFO - Close client connection: Client-c347d965-ce48-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,802 - distributed.scheduler - INFO - Close client connection: Client-29c59294-ce4a-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,801 - distributed.scheduler - INFO - Close client connection: Client-905034f7-ce4b-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,801 - distributed.scheduler - INFO - Close client connection: Client-f6d97abf-ce4c-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,801 - distributed.scheduler - INFO - Close client connection: Client-5d614529-ce4e-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,801 - distributed.scheduler - INFO - Close client connection: Client-c3e04802-ce4f-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,801 - distributed.scheduler - INFO - Close client connection: Client-2a61f247-ce51-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,801 - distributed.scheduler - INFO - Close client connection: Client-90e7f6ab-ce52-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,801 - distributed.scheduler - INFO - Close client connection: Client-f765012d-ce53-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,801 - distributed.scheduler - INFO - Close client connection: Client-5de336ad-ce55-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,799 - distributed.scheduler - INFO - Close client connection: Client-c4637c3c-ce56-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,799 - distributed.scheduler - INFO - Close client connection: Client-2ada6a2e-ce58-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,799 - distributed.scheduler - INFO - Close client connection: Client-91729aa2-ce59-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,799 - distributed.scheduler - INFO - Close client connection: Client-f7f84204-ce5a-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,799 - distributed.scheduler - INFO - Close client connection: Client-5e7463a2-ce5c-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,799 - distributed.scheduler - INFO - Remove client Client-8e69fef1-ce2f-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,798 - distributed.scheduler - INFO - Remove client Client-f4f22c0e-ce30-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,798 - distributed.scheduler - INFO - Remove client Client-5b6ebc7a-ce32-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,798 - distributed.scheduler - INFO - Remove client Client-c1f023a9-ce33-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,798 - distributed.scheduler - INFO - Remove client Client-28780ba7-ce35-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,797 - distributed.scheduler - INFO - Remove client Client-8efaa7a9-ce36-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,797 - distributed.scheduler - INFO - Remove client Client-f572ae3b-ce37-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,797 - distributed.scheduler - INFO - Remove client Client-5bec5165-ce39-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,796 - distributed.scheduler - INFO - Remove client Client-c26bfe9c-ce3a-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,796 - distributed.scheduler - INFO - Remove client Client-28ecb445-ce3c-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,796 - distributed.scheduler - INFO - Remove client Client-8f6f8176-ce3d-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,792 - distributed.scheduler - INFO - Remove client Client-f5ee153a-ce3e-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,791 - distributed.scheduler - INFO - Remove client Client-5c5ef6ed-ce40-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,791 - distributed.scheduler - INFO - Remove client Client-c2d97fb5-ce41-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,791 - distributed.scheduler - INFO - Remove client Client-2956eb54-ce43-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,790 - distributed.scheduler - INFO - Remove client Client-8fd509ac-ce44-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,790 - distributed.scheduler - INFO - Remove client Client-f653b9ed-ce45-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,790 - distributed.scheduler - INFO - Remove client Client-5cc8ce5b-ce47-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,790 - distributed.scheduler - INFO - Remove client Client-c347d965-ce48-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,789 - distributed.scheduler - INFO - Remove client Client-29c59294-ce4a-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,789 - distributed.scheduler - INFO - Remove client Client-905034f7-ce4b-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,789 - distributed.scheduler - INFO - Remove client Client-f6d97abf-ce4c-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,788 - distributed.scheduler - INFO - Remove client Client-5d614529-ce4e-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,788 - distributed.scheduler - INFO - Remove client Client-c3e04802-ce4f-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,788 - distributed.scheduler - INFO - Remove client Client-2a61f247-ce51-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,788 - distributed.scheduler - INFO - Remove client Client-90e7f6ab-ce52-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,787 - distributed.scheduler - INFO - Remove client Client-f765012d-ce53-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,787 - distributed.scheduler - INFO - Remove client Client-5de336ad-ce55-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,785 - distributed.scheduler - INFO - Remove client Client-c4637c3c-ce56-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,784 - distributed.scheduler - INFO - Remove client Client-2ada6a2e-ce58-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,784 - distributed.scheduler - INFO - Remove client Client-91729aa2-ce59-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,784 - distributed.scheduler - INFO - Remove client Client-f7f84204-ce5a-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,783 - distributed.scheduler - INFO - Remove client Client-5e7463a2-ce5c-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,189 - distributed.scheduler - INFO - Close client connection: Client-c4e0fd9c-ce5d-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,188 - distributed.scheduler - INFO - Remove client Client-c4e0fd9c-ce5d-11f0-8001-f2cd832f225c

2025-12-01 02:31:54,186 - distributed.scheduler - INFO - Remove client Client-c4e0fd9c-ce5d-11f0-8001-f2cd832f225c

2025-12-01 02:30:57,911 - distributed.scheduler - INFO - Receive client connection: Client-c4e0fd9c-ce5d-11f0-8001-f2cd832f225c

2025-12-01 02:20:56,569 - distributed.scheduler - INFO - Receive client connection: Client-5e7463a2-ce5c-11f0-8001-f2cd832f225c

2025-12-01 02:10:55,128 - distributed.scheduler - INFO - Receive client connection: Client-f7f84204-ce5a-11f0-8001-f2cd832f225c

2025-12-01 02:00:53,628 - distributed.scheduler - INFO - Receive client connection: Client-91729aa2-ce59-11f0-8001-f2cd832f225c

2025-12-01 01:50:52,012 - distributed.scheduler - INFO - Receive client connection: Client-2ada6a2e-ce58-11f0-8001-f2cd832f225c

2025-12-01 01:40:50,603 - distributed.scheduler - INFO - Receive client connection: Client-c4637c3c-ce56-11f0-8001-f2cd832f225c

2025-12-01 01:30:49,145 - distributed.scheduler - INFO - Receive client connection: Client-5de336ad-ce55-11f0-8001-f2cd832f225c

2025-12-01 01:20:47,690 - distributed.scheduler - INFO - Receive client connection: Client-f765012d-ce53-11f0-8001-f2cd832f225c

2025-12-01 01:10:46,243 - distributed.scheduler - INFO - Receive client connection: Client-90e7f6ab-ce52-11f0-8001-f2cd832f225c

2025-12-01 01:00:44,739 - distributed.scheduler - INFO - Receive client connection: Client-2a61f247-ce51-11f0-8001-f2cd832f225c

2025-12-01 00:50:43,269 - distributed.scheduler - INFO - Receive client connection: Client-c3e04802-ce4f-11f0-8001-f2cd832f225c

2025-12-01 00:40:41,813 - distributed.scheduler - INFO - Receive client connection: Client-5d614529-ce4e-11f0-8001-f2cd832f225c

2025-12-01 00:30:40,299 - distributed.scheduler - INFO - Receive client connection: Client-f6d97abf-ce4c-11f0-8001-f2cd832f225c

2025-12-01 00:20:38,773 - distributed.scheduler - INFO - Receive client connection: Client-905034f7-ce4b-11f0-8001-f2cd832f225c

2025-12-01 00:10:37,241 - distributed.scheduler - INFO - Receive client connection: Client-29c59294-ce4a-11f0-8001-f2cd832f225c

2025-12-01 00:00:35,790 - distributed.scheduler - INFO - Receive client connection: Client-c347d965-ce48-11f0-8001-f2cd832f225c

2025-11-30 23:50:34,335 - distributed.scheduler - INFO - Receive client connection: Client-5cc8ce5b-ce47-11f0-8001-f2cd832f225c

2025-11-30 23:40:32,942 - distributed.scheduler - INFO - Receive client connection: Client-f653b9ed-ce45-11f0-8001-f2cd832f225c

2025-11-30 23:30:31,491 - distributed.scheduler - INFO - Receive client connection: Client-8fd509ac-ce44-11f0-8001-f2cd832f225c

2025-11-30 23:20:30,037 - distributed.scheduler - INFO - Receive client connection: Client-2956eb54-ce43-11f0-8001-f2cd832f225c

2025-11-30 23:10:28,593 - distributed.scheduler - INFO - Receive client connection: Client-c2d97fb5-ce41-11f0-8001-f2cd832f225c

2025-11-30 23:00:27,164 - distributed.scheduler - INFO - Receive client connection: Client-5c5ef6ed-ce40-11f0-8001-f2cd832f225c

2025-11-30 22:50:25,796 - distributed.scheduler - INFO - Receive client connection: Client-f5ee153a-ce3e-11f0-8001-f2cd832f225c

2025-11-30 22:40:24,348 - distributed.scheduler - INFO - Receive client connection: Client-8f6f8176-ce3d-11f0-8001-f2cd832f225c

2025-11-30 22:30:22,869 - distributed.scheduler - INFO - Receive client connection: Client-28ecb445-ce3c-11f0-8001-f2cd832f225c

2025-11-30 22:20:21,401 - distributed.scheduler - INFO - Receive client connection: Client-c26bfe9c-ce3a-11f0-8001-f2cd832f225c

2025-11-30 22:10:19,937 - distributed.scheduler - INFO - Receive client connection: Client-5bec5165-ce39-11f0-8001-f2cd832f225c

2025-11-30 22:00:18,513 - distributed.scheduler - INFO - Receive client connection: Client-f572ae3b-ce37-11f0-8001-f2cd832f225c

2025-11-30 21:50:17,102 - distributed.scheduler - INFO - Receive client connection: Client-8efaa7a9-ce36-11f0-8001-f2cd832f225c

2025-11-30 21:40:15,626 - distributed.scheduler - INFO - Receive client connection: Client-28780ba7-ce35-11f0-8001-f2cd832f225c

2025-11-30 21:30:14,110 - distributed.scheduler - INFO - Receive client connection: Client-c1f023a9-ce33-11f0-8001-f2cd832f225c

2025-11-30 21:20:12,636 - distributed.scheduler - INFO - Receive client connection: Client-5b6ebc7a-ce32-11f0-8001-f2cd832f225c

2025-11-30 21:10:11,193 - distributed.scheduler - INFO - Receive client connection: Client-f4f22c0e-ce30-11f0-8001-f2cd832f225c

2025-11-30 21:00:09,683 - distributed.scheduler - INFO - Receive client connection: Client-8e69fef1-ce2f-11f0-8001-f2cd832f225c

2025-11-30 20:50:42,659 - distributed.scheduler - INFO - Close client connection: Client-27e24434-ce2e-11f0-8001-f2cd832f225c

2025-11-30 20:50:42,657 - distributed.scheduler - INFO - Remove client Client-27e24434-ce2e-11f0-8001-f2cd832f225c

2025-11-30 20:50:42,656 - distributed.scheduler - INFO - Remove client Client-27e24434-ce2e-11f0-8001-f2cd832f225c

2025-11-30 20:50:08,167 - distributed.scheduler - INFO - Receive client connection: Client-27e24434-ce2e-11f0-8001-f2cd832f225c

2025-11-30 20:40:43,730 - distributed.scheduler - INFO - Close client connection: Client-c16b7744-ce2c-11f0-8001-f2cd832f225c

2025-11-30 20:40:43,729 - distributed.scheduler - INFO - Remove client Client-c16b7744-ce2c-11f0-8001-f2cd832f225c

2025-11-30 20:40:43,727 - distributed.scheduler - INFO - Remove client Client-c16b7744-ce2c-11f0-8001-f2cd832f225c

2025-11-30 20:40:06,768 - distributed.scheduler - INFO - Receive client connection: Client-c16b7744-ce2c-11f0-8001-f2cd832f225c

2025-11-30 20:31:05,462 - distributed.scheduler - INFO - Close client connection: Client-5aeecfc9-ce2b-11f0-8001-f2cd832f225c

2025-11-30 20:31:05,462 - distributed.scheduler - INFO - Remove client Client-5aeecfc9-ce2b-11f0-8001-f2cd832f225c

2025-11-30 20:31:05,460 - distributed.scheduler - INFO - Remove client Client-5aeecfc9-ce2b-11f0-8001-f2cd832f225c

2025-11-30 20:30:05,323 - distributed.scheduler - INFO - Receive client connection: Client-5aeecfc9-ce2b-11f0-8001-f2cd832f225c

2025-11-30 20:20:38,107 - distributed.scheduler - INFO - Close client connection: Client-f46b492c-ce29-11f0-8001-f2cd832f225c

2025-11-30 20:20:38,106 - distributed.scheduler - INFO - Remove client Client-f46b492c-ce29-11f0-8001-f2cd832f225c

2025-11-30 20:20:38,105 - distributed.scheduler - INFO - Remove client Client-f46b492c-ce29-11f0-8001-f2cd832f225c

2025-11-30 20:20:03,853 - distributed.scheduler - INFO - Receive client connection: Client-f46b492c-ce29-11f0-8001-f2cd832f225c

2025-11-30 20:10:36,051 - distributed.scheduler - INFO - Close client connection: Client-8de9ac32-ce28-11f0-8001-f2cd832f225c

2025-11-30 20:10:36,050 - distributed.scheduler - INFO - Remove client Client-8de9ac32-ce28-11f0-8001-f2cd832f225c

2025-11-30 20:10:36,049 - distributed.scheduler - INFO - Remove client Client-8de9ac32-ce28-11f0-8001-f2cd832f225c

2025-11-30 20:10:02,368 - distributed.scheduler - INFO - Receive client connection: Client-8de9ac32-ce28-11f0-8001-f2cd832f225c

2025-11-30 20:01:06,728 - distributed.scheduler - INFO - Close client connection: Client-4ac45363-ce20-11f0-8001-f2cd832f225c

2025-11-30 20:01:06,728 - distributed.scheduler - INFO - Close client connection: Client-17c53eac-ce23-11f0-8001-f2cd832f225c

2025-11-30 20:01:06,727 - distributed.scheduler - INFO - Close client connection: Client-b14439eb-ce21-11f0-8001-f2cd832f225c

2025-11-30 20:01:06,727 - distributed.scheduler - INFO - Close client connection: Client-7e339f0b-ce24-11f0-8001-f2cd832f225c

2025-11-30 20:01:06,726 - distributed.scheduler - INFO - Close client connection: Client-e4b12b33-ce25-11f0-8001-f2cd832f225c

2025-11-30 20:01:06,725 - distributed.scheduler - INFO - Remove client Client-4ac45363-ce20-11f0-8001-f2cd832f225c

2025-11-30 20:01:06,724 - distributed.scheduler - INFO - Remove client Client-17c53eac-ce23-11f0-8001-f2cd832f225c

2025-11-30 20:01:06,723 - distributed.scheduler - INFO - Remove client Client-b14439eb-ce21-11f0-8001-f2cd832f225c

2025-11-30 20:01:06,722 - distributed.scheduler - INFO - Remove client Client-7e339f0b-ce24-11f0-8001-f2cd832f225c

2025-11-30 20:01:06,721 - distributed.scheduler - INFO - Remove client Client-e4b12b33-ce25-11f0-8001-f2cd832f225c

2025-11-30 20:01:05,925 - distributed.scheduler - INFO - Close client connection: Client-275cb855-ce27-11f0-8001-f2cd832f225c

2025-11-30 20:01:05,924 - distributed.scheduler - INFO - Remove client Client-275cb855-ce27-11f0-8001-f2cd832f225c

2025-11-30 20:01:05,923 - distributed.scheduler - INFO - Remove client Client-275cb855-ce27-11f0-8001-f2cd832f225c

2025-11-30 20:00:00,811 - distributed.scheduler - INFO - Receive client connection: Client-275cb855-ce27-11f0-8001-f2cd832f225c

2025-11-30 19:50:59,461 - distributed.scheduler - INFO - Receive client connection: Client-e4b12b33-ce25-11f0-8001-f2cd832f225c

2025-11-30 19:40:58,011 - distributed.scheduler - INFO - Receive client connection: Client-7e339f0b-ce24-11f0-8001-f2cd832f225c

2025-11-30 19:30:56,668 - distributed.scheduler - INFO - Receive client connection: Client-17c53eac-ce23-11f0-8001-f2cd832f225c

2025-11-30 19:20:55,191 - distributed.scheduler - INFO - Receive client connection: Client-b14439eb-ce21-11f0-8001-f2cd832f225c

2025-11-30 19:10:53,742 - distributed.scheduler - INFO - Receive client connection: Client-4ac45363-ce20-11f0-8001-f2cd832f225c

2025-11-30 19:01:56,941 - distributed.scheduler - INFO - Close client connection: Client-e42bf40a-ce1e-11f0-8001-f2cd832f225c

2025-11-30 19:01:56,940 - distributed.scheduler - INFO - Remove client Client-e42bf40a-ce1e-11f0-8001-f2cd832f225c

2025-11-30 19:01:56,939 - distributed.scheduler - INFO - Remove client Client-e42bf40a-ce1e-11f0-8001-f2cd832f225c

2025-11-30 19:00:52,121 - distributed.scheduler - INFO - Receive client connection: Client-e42bf40a-ce1e-11f0-8001-f2cd832f225c

2025-11-30 18:51:26,617 - distributed.scheduler - INFO - Close client connection: Client-7db1d4bb-ce1d-11f0-8001-f2cd832f225c

2025-11-30 18:51:26,615 - distributed.scheduler - INFO - Remove client Client-7db1d4bb-ce1d-11f0-8001-f2cd832f225c

2025-11-30 18:51:26,613 - distributed.scheduler - INFO - Remove client Client-7db1d4bb-ce1d-11f0-8001-f2cd832f225c

2025-11-30 18:50:50,696 - distributed.scheduler - INFO - Receive client connection: Client-7db1d4bb-ce1d-11f0-8001-f2cd832f225c

2025-11-30 18:41:27,212 - distributed.scheduler - INFO - Close client connection: Client-173646ce-ce1c-11f0-8001-f2cd832f225c

2025-11-30 18:41:27,212 - distributed.scheduler - INFO - Remove client Client-173646ce-ce1c-11f0-8001-f2cd832f225c

2025-11-30 18:41:27,210 - distributed.scheduler - INFO - Remove client Client-173646ce-ce1c-11f0-8001-f2cd832f225c

2025-11-30 18:40:49,262 - distributed.scheduler - INFO - Receive client connection: Client-173646ce-ce1c-11f0-8001-f2cd832f225c

2025-11-30 18:31:56,307 - distributed.scheduler - INFO - Close client connection: Client-b0bd7320-ce1a-11f0-8001-f2cd832f225c

2025-11-30 18:31:56,305 - distributed.scheduler - INFO - Remove client Client-b0bd7320-ce1a-11f0-8001-f2cd832f225c

2025-11-30 18:31:56,303 - distributed.scheduler - INFO - Remove client Client-b0bd7320-ce1a-11f0-8001-f2cd832f225c

2025-11-30 18:30:47,883 - distributed.scheduler - INFO - Receive client connection: Client-b0bd7320-ce1a-11f0-8001-f2cd832f225c

2025-11-30 18:21:22,401 - distributed.scheduler - INFO - Close client connection: Client-4a475e8b-ce19-11f0-8001-f2cd832f225c

2025-11-30 18:21:22,401 - distributed.scheduler - INFO - Remove client Client-4a475e8b-ce19-11f0-8001-f2cd832f225c

2025-11-30 18:21:22,399 - distributed.scheduler - INFO - Remove client Client-4a475e8b-ce19-11f0-8001-f2cd832f225c

2025-11-30 18:20:46,445 - distributed.scheduler - INFO - Receive client connection: Client-4a475e8b-ce19-11f0-8001-f2cd832f225c

2025-11-30 18:11:20,862 - distributed.scheduler - INFO - Close client connection: Client-e3be371d-ce17-11f0-8001-f2cd832f225c

2025-11-30 18:11:20,861 - distributed.scheduler - INFO - Remove client Client-e3be371d-ce17-11f0-8001-f2cd832f225c

2025-11-30 18:11:20,859 - distributed.scheduler - INFO - Remove client Client-e3be371d-ce17-11f0-8001-f2cd832f225c

2025-11-30 18:10:44,923 - distributed.scheduler - INFO - Receive client connection: Client-e3be371d-ce17-11f0-8001-f2cd832f225c

2025-11-30 18:01:51,586 - distributed.scheduler - INFO - Close client connection: Client-7d2ecf95-ce16-11f0-8001-f2cd832f225c

2025-11-30 18:01:51,585 - distributed.scheduler - INFO - Remove client Client-7d2ecf95-ce16-11f0-8001-f2cd832f225c

2025-11-30 18:01:51,583 - distributed.scheduler - INFO - Remove client Client-7d2ecf95-ce16-11f0-8001-f2cd832f225c

2025-11-30 18:00:43,361 - distributed.scheduler - INFO - Receive client connection: Client-7d2ecf95-ce16-11f0-8001-f2cd832f225c

2025-11-30 17:51:18,721 - distributed.scheduler - INFO - Close client connection: Client-16af7e5a-ce15-11f0-8001-f2cd832f225c

2025-11-30 17:51:18,720 - distributed.scheduler - INFO - Remove client Client-16af7e5a-ce15-11f0-8001-f2cd832f225c

2025-11-30 17:51:18,718 - distributed.scheduler - INFO - Remove client Client-16af7e5a-ce15-11f0-8001-f2cd832f225c

2025-11-30 17:50:41,904 - distributed.scheduler - INFO - Receive client connection: Client-16af7e5a-ce15-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,685 - distributed.scheduler - INFO - Close client connection: Client-15e50b33-ce07-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,684 - distributed.scheduler - INFO - Close client connection: Client-e2d4802d-ce09-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,684 - distributed.scheduler - INFO - Close client connection: Client-7c612984-ce08-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,684 - distributed.scheduler - INFO - Close client connection: Client-494fa616-ce0b-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,684 - distributed.scheduler - INFO - Close client connection: Client-afc45347-ce0c-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,683 - distributed.scheduler - INFO - Close client connection: Client-163df578-ce0e-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,683 - distributed.scheduler - INFO - Close client connection: Client-7cc30dc7-ce0f-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,683 - distributed.scheduler - INFO - Close client connection: Client-e3358495-ce10-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,683 - distributed.scheduler - INFO - Close client connection: Client-49b01d55-ce12-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,682 - distributed.scheduler - INFO - Remove client Client-15e50b33-ce07-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,681 - distributed.scheduler - INFO - Remove client Client-e2d4802d-ce09-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,681 - distributed.scheduler - INFO - Remove client Client-7c612984-ce08-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,678 - distributed.scheduler - INFO - Remove client Client-494fa616-ce0b-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,677 - distributed.scheduler - INFO - Remove client Client-afc45347-ce0c-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,676 - distributed.scheduler - INFO - Remove client Client-163df578-ce0e-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,675 - distributed.scheduler - INFO - Remove client Client-7cc30dc7-ce0f-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,674 - distributed.scheduler - INFO - Remove client Client-e3358495-ce10-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,673 - distributed.scheduler - INFO - Remove client Client-49b01d55-ce12-11f0-8001-f2cd832f225c

2025-11-30 17:41:15,000 - distributed.scheduler - INFO - Close client connection: Client-b0320741-ce13-11f0-8001-f2cd832f225c

2025-11-30 17:41:14,997 - distributed.scheduler - INFO - Remove client Client-b0320741-ce13-11f0-8001-f2cd832f225c

2025-11-30 17:41:14,995 - distributed.scheduler - INFO - Remove client Client-b0320741-ce13-11f0-8001-f2cd832f225c

2025-11-30 17:40:40,452 - distributed.scheduler - INFO - Receive client connection: Client-b0320741-ce13-11f0-8001-f2cd832f225c

2025-11-30 17:30:38,970 - distributed.scheduler - INFO - Receive client connection: Client-49b01d55-ce12-11f0-8001-f2cd832f225c

2025-11-30 17:20:37,544 - distributed.scheduler - INFO - Receive client connection: Client-e3358495-ce10-11f0-8001-f2cd832f225c

2025-11-30 17:10:36,164 - distributed.scheduler - INFO - Receive client connection: Client-7cc30dc7-ce0f-11f0-8001-f2cd832f225c

2025-11-30 17:00:34,707 - distributed.scheduler - INFO - Receive client connection: Client-163df578-ce0e-11f0-8001-f2cd832f225c

2025-11-30 16:50:33,250 - distributed.scheduler - INFO - Receive client connection: Client-afc45347-ce0c-11f0-8001-f2cd832f225c

2025-11-30 16:40:31,863 - distributed.scheduler - INFO - Receive client connection: Client-494fa616-ce0b-11f0-8001-f2cd832f225c

2025-11-30 16:30:30,433 - distributed.scheduler - INFO - Receive client connection: Client-e2d4802d-ce09-11f0-8001-f2cd832f225c

2025-11-30 16:20:29,048 - distributed.scheduler - INFO - Receive client connection: Client-7c612984-ce08-11f0-8001-f2cd832f225c

2025-11-30 16:10:27,620 - distributed.scheduler - INFO - Receive client connection: Client-15e50b33-ce07-11f0-8001-f2cd832f225c

2025-11-30 16:01:38,402 - distributed.scheduler - INFO - Close client connection: Client-af691450-ce05-11f0-8001-f2cd832f225c

2025-11-30 16:01:38,402 - distributed.scheduler - INFO - Remove client Client-af691450-ce05-11f0-8001-f2cd832f225c

2025-11-30 16:01:38,399 - distributed.scheduler - INFO - Remove client Client-af691450-ce05-11f0-8001-f2cd832f225c

2025-11-30 16:00:26,183 - distributed.scheduler - INFO - Receive client connection: Client-af691450-ce05-11f0-8001-f2cd832f225c

2025-11-30 15:51:04,382 - distributed.scheduler - INFO - Close client connection: Client-48ec6a4a-ce04-11f0-8001-f2cd832f225c

2025-11-30 15:51:04,380 - distributed.scheduler - INFO - Remove client Client-48ec6a4a-ce04-11f0-8001-f2cd832f225c

2025-11-30 15:51:04,378 - distributed.scheduler - INFO - Remove client Client-48ec6a4a-ce04-11f0-8001-f2cd832f225c

2025-11-30 15:50:24,741 - distributed.scheduler - INFO - Receive client connection: Client-48ec6a4a-ce04-11f0-8001-f2cd832f225c

2025-11-30 15:41:00,333 - distributed.scheduler - INFO - Close client connection: Client-e2666888-ce02-11f0-8001-f2cd832f225c

2025-11-30 15:41:00,331 - distributed.scheduler - INFO - Remove client Client-e2666888-ce02-11f0-8001-f2cd832f225c

2025-11-30 15:41:00,329 - distributed.scheduler - INFO - Remove client Client-e2666888-ce02-11f0-8001-f2cd832f225c

2025-11-30 15:40:23,234 - distributed.scheduler - INFO - Receive client connection: Client-e2666888-ce02-11f0-8001-f2cd832f225c

2025-11-30 15:31:35,170 - distributed.scheduler - INFO - Close client connection: Client-7be6afaf-ce01-11f0-8001-f2cd832f225c

2025-11-30 15:31:35,169 - distributed.scheduler - INFO - Remove client Client-7be6afaf-ce01-11f0-8001-f2cd832f225c

2025-11-30 15:31:35,167 - distributed.scheduler - INFO - Remove client Client-7be6afaf-ce01-11f0-8001-f2cd832f225c

2025-11-30 15:30:21,780 - distributed.scheduler - INFO - Receive client connection: Client-7be6afaf-ce01-11f0-8001-f2cd832f225c

2025-11-30 15:20:58,147 - distributed.scheduler - INFO - Close client connection: Client-15696e5b-ce00-11f0-8001-f2cd832f225c

2025-11-30 15:20:58,146 - distributed.scheduler - INFO - Remove client Client-15696e5b-ce00-11f0-8001-f2cd832f225c

2025-11-30 15:20:58,144 - distributed.scheduler - INFO - Remove client Client-15696e5b-ce00-11f0-8001-f2cd832f225c

2025-11-30 15:20:20,328 - distributed.scheduler - INFO - Receive client connection: Client-15696e5b-ce00-11f0-8001-f2cd832f225c

2025-11-30 15:10:57,509 - distributed.scheduler - INFO - Close client connection: Client-4871f185-cdfd-11f0-8001-f2cd832f225c

2025-11-30 15:10:57,508 - distributed.scheduler - INFO - Remove client Client-4871f185-cdfd-11f0-8001-f2cd832f225c

2025-11-30 15:10:56,966 - distributed.scheduler - INFO - Close client connection: Client-aeeef87b-cdfe-11f0-8001-f2cd832f225c

2025-11-30 15:10:56,965 - distributed.scheduler - INFO - Remove client Client-aeeef87b-cdfe-11f0-8001-f2cd832f225c

2025-11-30 15:10:56,964 - distributed.scheduler - INFO - Remove client Client-aeeef87b-cdfe-11f0-8001-f2cd832f225c

2025-11-30 15:10:18,898 - distributed.scheduler - INFO - Receive client connection: Client-aeeef87b-cdfe-11f0-8001-f2cd832f225c

2025-11-30 15:00:17,458 - distributed.scheduler - INFO - Receive client connection: Client-4871f185-cdfd-11f0-8001-f2cd832f225c

2025-11-30 14:50:53,328 - distributed.scheduler - INFO - Close client connection: Client-e1f0decf-cdfb-11f0-8001-f2cd832f225c

2025-11-30 14:50:53,327 - distributed.scheduler - INFO - Remove client Client-e1f0decf-cdfb-11f0-8001-f2cd832f225c

2025-11-30 14:50:53,325 - distributed.scheduler - INFO - Remove client Client-e1f0decf-cdfb-11f0-8001-f2cd832f225c

2025-11-30 14:50:15,991 - distributed.scheduler - INFO - Receive client connection: Client-e1f0decf-cdfb-11f0-8001-f2cd832f225c

2025-11-30 14:40:54,038 - distributed.scheduler - INFO - Close client connection: Client-7b62693a-cdfa-11f0-8001-f2cd832f225c

2025-11-30 14:40:54,037 - distributed.scheduler - INFO - Remove client Client-7b62693a-cdfa-11f0-8001-f2cd832f225c

2025-11-30 14:40:54,035 - distributed.scheduler - INFO - Remove client Client-7b62693a-cdfa-11f0-8001-f2cd832f225c

2025-11-30 14:40:14,431 - distributed.scheduler - INFO - Receive client connection: Client-7b62693a-cdfa-11f0-8001-f2cd832f225c

2025-11-30 14:31:22,864 - distributed.scheduler - INFO - Close client connection: Client-14cf6fe1-cdf9-11f0-8001-f2cd832f225c

2025-11-30 14:31:22,864 - distributed.scheduler - INFO - Remove client Client-14cf6fe1-cdf9-11f0-8001-f2cd832f225c

2025-11-30 14:31:22,863 - distributed.scheduler - INFO - Remove client Client-14cf6fe1-cdf9-11f0-8001-f2cd832f225c

2025-11-30 14:30:12,837 - distributed.scheduler - INFO - Receive client connection: Client-14cf6fe1-cdf9-11f0-8001-f2cd832f225c

2025-11-30 14:20:51,251 - distributed.scheduler - INFO - Close client connection: Client-ae4b8199-cdf7-11f0-8001-f2cd832f225c

2025-11-30 14:20:51,250 - distributed.scheduler - INFO - Remove client Client-ae4b8199-cdf7-11f0-8001-f2cd832f225c

2025-11-30 14:20:51,248 - distributed.scheduler - INFO - Remove client Client-ae4b8199-cdf7-11f0-8001-f2cd832f225c

2025-11-30 14:20:11,358 - distributed.scheduler - INFO - Receive client connection: Client-ae4b8199-cdf7-11f0-8001-f2cd832f225c

2025-11-30 14:10:49,703 - distributed.scheduler - INFO - Close client connection: Client-47cc6ac1-cdf6-11f0-8001-f2cd832f225c

2025-11-30 14:10:49,701 - distributed.scheduler - INFO - Remove client Client-47cc6ac1-cdf6-11f0-8001-f2cd832f225c

2025-11-30 14:10:49,699 - distributed.scheduler - INFO - Remove client Client-47cc6ac1-cdf6-11f0-8001-f2cd832f225c

2025-11-30 14:10:09,898 - distributed.scheduler - INFO - Receive client connection: Client-47cc6ac1-cdf6-11f0-8001-f2cd832f225c

2025-11-30 14:01:19,952 - distributed.scheduler - INFO - Close client connection: Client-e14a2685-cdf4-11f0-8001-f2cd832f225c

2025-11-30 14:01:19,952 - distributed.scheduler - INFO - Remove client Client-e14a2685-cdf4-11f0-8001-f2cd832f225c

2025-11-30 14:01:19,950 - distributed.scheduler - INFO - Remove client Client-e14a2685-cdf4-11f0-8001-f2cd832f225c

2025-11-30 14:00:08,421 - distributed.scheduler - INFO - Receive client connection: Client-e14a2685-cdf4-11f0-8001-f2cd832f225c

2025-11-30 13:50:44,792 - distributed.scheduler - INFO - Close client connection: Client-7acff4b2-cdf3-11f0-8001-f2cd832f225c

2025-11-30 13:50:44,791 - distributed.scheduler - INFO - Remove client Client-7acff4b2-cdf3-11f0-8001-f2cd832f225c

2025-11-30 13:50:44,789 - distributed.scheduler - INFO - Remove client Client-7acff4b2-cdf3-11f0-8001-f2cd832f225c

2025-11-30 13:50:06,997 - distributed.scheduler - INFO - Receive client connection: Client-7acff4b2-cdf3-11f0-8001-f2cd832f225c

2025-11-30 13:40:42,482 - distributed.scheduler - INFO - Close client connection: Client-144c0230-cdf2-11f0-8001-f2cd832f225c

2025-11-30 13:40:42,480 - distributed.scheduler - INFO - Remove client Client-144c0230-cdf2-11f0-8001-f2cd832f225c

2025-11-30 13:40:42,478 - distributed.scheduler - INFO - Remove client Client-144c0230-cdf2-11f0-8001-f2cd832f225c

2025-11-30 13:40:05,510 - distributed.scheduler - INFO - Receive client connection: Client-144c0230-cdf2-11f0-8001-f2cd832f225c

2025-11-30 13:31:11,940 - distributed.scheduler - INFO - Close client connection: Client-adca0257-cdf0-11f0-8001-f2cd832f225c

2025-11-30 13:31:11,939 - distributed.scheduler - INFO - Remove client Client-adca0257-cdf0-11f0-8001-f2cd832f225c

2025-11-30 13:31:11,937 - distributed.scheduler - INFO - Remove client Client-adca0257-cdf0-11f0-8001-f2cd832f225c

2025-11-30 13:30:04,029 - distributed.scheduler - INFO - Receive client connection: Client-adca0257-cdf0-11f0-8001-f2cd832f225c

2025-11-30 13:20:40,738 - distributed.scheduler - INFO - Close client connection: Client-474c1234-cdef-11f0-8001-f2cd832f225c

2025-11-30 13:20:40,737 - distributed.scheduler - INFO - Remove client Client-474c1234-cdef-11f0-8001-f2cd832f225c

2025-11-30 13:20:40,735 - distributed.scheduler - INFO - Remove client Client-474c1234-cdef-11f0-8001-f2cd832f225c

2025-11-30 13:20:02,584 - distributed.scheduler - INFO - Receive client connection: Client-474c1234-cdef-11f0-8001-f2cd832f225c

2025-11-30 13:10:59,957 - distributed.scheduler - INFO - Close client connection: Client-eba0d6d8-cded-11f0-8001-f2cd832f225c

2025-11-30 13:10:59,957 - distributed.scheduler - INFO - Remove client Client-eba0d6d8-cded-11f0-8001-f2cd832f225c

2025-11-30 13:10:59,956 - distributed.scheduler - INFO - Remove client Client-eba0d6d8-cded-11f0-8001-f2cd832f225c

2025-11-30 13:10:19,290 - distributed.scheduler - INFO - Receive client connection: Client-eba0d6d8-cded-11f0-8001-f2cd832f225c

2025-11-30 13:02:11,547 - distributed.scheduler - INFO - Close client connection: Client-9e33c1c5-cdec-11f0-8001-f2cd832f225c

2025-11-30 13:02:11,546 - distributed.scheduler - INFO - Remove client Client-9e33c1c5-cdec-11f0-8001-f2cd832f225c

2025-11-30 13:02:11,544 - distributed.scheduler - INFO - Remove client Client-9e33c1c5-cdec-11f0-8001-f2cd832f225c

2025-11-30 13:00:59,891 - distributed.scheduler - INFO - Receive client connection: Client-9e33c1c5-cdec-11f0-8001-f2cd832f225c

2025-11-30 12:51:35,811 - distributed.scheduler - INFO - Close client connection: Client-37a9bb2f-cdeb-11f0-8001-f2cd832f225c

2025-11-30 12:51:35,809 - distributed.scheduler - INFO - Remove client Client-37a9bb2f-cdeb-11f0-8001-f2cd832f225c

2025-11-30 12:51:35,807 - distributed.scheduler - INFO - Remove client Client-37a9bb2f-cdeb-11f0-8001-f2cd832f225c

2025-11-30 12:50:58,363 - distributed.scheduler - INFO - Receive client connection: Client-37a9bb2f-cdeb-11f0-8001-f2cd832f225c

2025-11-30 12:41:35,274 - distributed.scheduler - INFO - Close client connection: Client-d13984f8-cde9-11f0-8001-f2cd832f225c

2025-11-30 12:41:35,270 - distributed.scheduler - INFO - Remove client Client-d13984f8-cde9-11f0-8001-f2cd832f225c

2025-11-30 12:41:35,269 - distributed.scheduler - INFO - Remove client Client-d13984f8-cde9-11f0-8001-f2cd832f225c

2025-11-30 12:40:56,998 - distributed.scheduler - INFO - Receive client connection: Client-d13984f8-cde9-11f0-8001-f2cd832f225c

2025-11-30 12:32:05,525 - distributed.scheduler - INFO - Close client connection: Client-6ac486af-cde8-11f0-8001-f2cd832f225c

2025-11-30 12:32:05,524 - distributed.scheduler - INFO - Remove client Client-6ac486af-cde8-11f0-8001-f2cd832f225c

2025-11-30 12:32:05,522 - distributed.scheduler - INFO - Remove client Client-6ac486af-cde8-11f0-8001-f2cd832f225c

2025-11-30 12:30:55,615 - distributed.scheduler - INFO - Receive client connection: Client-6ac486af-cde8-11f0-8001-f2cd832f225c

2025-11-30 12:21:32,168 - distributed.scheduler - INFO - Close client connection: Client-03f015a0-cde0-11f0-8001-f2cd832f225c

2025-11-30 12:21:32,167 - distributed.scheduler - INFO - Close client connection: Client-370041d9-cddd-11f0-8001-f2cd832f225c

2025-11-30 12:21:32,167 - distributed.scheduler - INFO - Close client connection: Client-9d778651-cdde-11f0-8001-f2cd832f225c

2025-11-30 12:21:32,167 - distributed.scheduler - INFO - Close client connection: Client-6a6dc621-cde1-11f0-8001-f2cd832f225c

2025-11-30 12:21:32,166 - distributed.scheduler - INFO - Close client connection: Client-d0ed5ac4-cde2-11f0-8001-f2cd832f225c

2025-11-30 12:21:32,166 - distributed.scheduler - INFO - Close client connection: Client-37613034-cde4-11f0-8001-f2cd832f225c

2025-11-30 12:21:32,166 - distributed.scheduler - INFO - Close client connection: Client-9e432b74-cde5-11f0-8001-f2cd832f225c

2025-11-30 12:21:32,163 - distributed.scheduler - INFO - Remove client Client-03f015a0-cde0-11f0-8001-f2cd832f225c

2025-11-30 12:21:32,162 - distributed.scheduler - INFO - Remove client Client-370041d9-cddd-11f0-8001-f2cd832f225c

2025-11-30 12:21:32,161 - distributed.scheduler - INFO - Remove client Client-9d778651-cdde-11f0-8001-f2cd832f225c

2025-11-30 12:21:32,160 - distributed.scheduler - INFO - Remove client Client-6a6dc621-cde1-11f0-8001-f2cd832f225c

2025-11-30 12:21:32,160 - distributed.scheduler - INFO - Remove client Client-d0ed5ac4-cde2-11f0-8001-f2cd832f225c

2025-11-30 12:21:32,159 - distributed.scheduler - INFO - Remove client Client-37613034-cde4-11f0-8001-f2cd832f225c

2025-11-30 12:21:32,158 - distributed.scheduler - INFO - Remove client Client-9e432b74-cde5-11f0-8001-f2cd832f225c

2025-11-30 12:21:31,472 - distributed.scheduler - INFO - Close client connection: Client-044c3c59-cde7-11f0-8001-f2cd832f225c

2025-11-30 12:21:31,471 - distributed.scheduler - INFO - Remove client Client-044c3c59-cde7-11f0-8001-f2cd832f225c

2025-11-30 12:21:31,469 - distributed.scheduler - INFO - Remove client Client-044c3c59-cde7-11f0-8001-f2cd832f225c

2025-11-30 12:20:54,193 - distributed.scheduler - INFO - Receive client connection: Client-044c3c59-cde7-11f0-8001-f2cd832f225c

2025-11-30 12:10:53,503 - distributed.scheduler - INFO - Receive client connection: Client-9e432b74-cde5-11f0-8001-f2cd832f225c

2025-11-30 12:00:51,403 - distributed.scheduler - INFO - Receive client connection: Client-37613034-cde4-11f0-8001-f2cd832f225c

2025-11-30 11:50:50,020 - distributed.scheduler - INFO - Receive client connection: Client-d0ed5ac4-cde2-11f0-8001-f2cd832f225c

2025-11-30 11:40:48,556 - distributed.scheduler - INFO - Receive client connection: Client-6a6dc621-cde1-11f0-8001-f2cd832f225c

2025-11-30 11:30:47,114 - distributed.scheduler - INFO - Receive client connection: Client-03f015a0-cde0-11f0-8001-f2cd832f225c

2025-11-30 11:20:45,696 - distributed.scheduler - INFO - Receive client connection: Client-9d778651-cdde-11f0-8001-f2cd832f225c

2025-11-30 11:10:44,301 - distributed.scheduler - INFO - Receive client connection: Client-370041d9-cddd-11f0-8001-f2cd832f225c

2025-11-30 11:01:52,448 - distributed.scheduler - INFO - Close client connection: Client-d08ff9c2-cddb-11f0-8001-f2cd832f225c

2025-11-30 11:01:52,447 - distributed.scheduler - INFO - Remove client Client-d08ff9c2-cddb-11f0-8001-f2cd832f225c

2025-11-30 11:01:52,446 - distributed.scheduler - INFO - Remove client Client-d08ff9c2-cddb-11f0-8001-f2cd832f225c

2025-11-30 11:00:42,928 - distributed.scheduler - INFO - Receive client connection: Client-d08ff9c2-cddb-11f0-8001-f2cd832f225c

2025-11-30 10:51:21,190 - distributed.scheduler - INFO - Close client connection: Client-6a1d0b16-cdda-11f0-8001-f2cd832f225c

2025-11-30 10:51:21,189 - distributed.scheduler - INFO - Remove client Client-6a1d0b16-cdda-11f0-8001-f2cd832f225c

2025-11-30 10:51:21,187 - distributed.scheduler - INFO - Remove client Client-6a1d0b16-cdda-11f0-8001-f2cd832f225c

2025-11-30 10:50:41,563 - distributed.scheduler - INFO - Receive client connection: Client-6a1d0b16-cdda-11f0-8001-f2cd832f225c

2025-11-30 10:41:16,717 - distributed.scheduler - INFO - Close client connection: Client-039f1ecc-cdd9-11f0-8001-f2cd832f225c

2025-11-30 10:41:16,717 - distributed.scheduler - INFO - Remove client Client-039f1ecc-cdd9-11f0-8001-f2cd832f225c

2025-11-30 10:41:16,716 - distributed.scheduler - INFO - Remove client Client-039f1ecc-cdd9-11f0-8001-f2cd832f225c

2025-11-30 10:40:40,104 - distributed.scheduler - INFO - Receive client connection: Client-039f1ecc-cdd9-11f0-8001-f2cd832f225c

2025-11-30 10:31:47,063 - distributed.scheduler - INFO - Close client connection: Client-9d295db1-cdd7-11f0-8001-f2cd832f225c

2025-11-30 10:31:47,062 - distributed.scheduler - INFO - Remove client Client-9d295db1-cdd7-11f0-8001-f2cd832f225c

2025-11-30 10:31:47,060 - distributed.scheduler - INFO - Remove client Client-9d295db1-cdd7-11f0-8001-f2cd832f225c

2025-11-30 10:30:38,718 - distributed.scheduler - INFO - Receive client connection: Client-9d295db1-cdd7-11f0-8001-f2cd832f225c

2025-11-30 10:21:12,242 - distributed.scheduler - INFO - Close client connection: Client-36a0d08f-cdd6-11f0-8001-f2cd832f225c

2025-11-30 10:21:12,242 - distributed.scheduler - INFO - Remove client Client-36a0d08f-cdd6-11f0-8001-f2cd832f225c

2025-11-30 10:21:12,240 - distributed.scheduler - INFO - Remove client Client-36a0d08f-cdd6-11f0-8001-f2cd832f225c

2025-11-30 10:20:37,200 - distributed.scheduler - INFO - Receive client connection: Client-36a0d08f-cdd6-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,028 - distributed.scheduler - INFO - Close client connection: Client-f0a8fd87-cda3-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,028 - distributed.scheduler - INFO - Close client connection: Client-572c1bab-cda5-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,028 - distributed.scheduler - INFO - Close client connection: Client-bd982ede-cda6-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,028 - distributed.scheduler - INFO - Close client connection: Client-24109f73-cda8-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,028 - distributed.scheduler - INFO - Close client connection: Client-8a946125-cda9-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,028 - distributed.scheduler - INFO - Close client connection: Client-f11fb784-cdaa-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,028 - distributed.scheduler - INFO - Close client connection: Client-579ffdb9-cdac-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,028 - distributed.scheduler - INFO - Close client connection: Client-be196514-cdad-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,028 - distributed.scheduler - INFO - Close client connection: Client-2491adbd-cdaf-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,027 - distributed.scheduler - INFO - Close client connection: Client-8b050c1d-cdb0-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,027 - distributed.scheduler - INFO - Close client connection: Client-f17a15bf-cdb1-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,027 - distributed.scheduler - INFO - Close client connection: Client-9aad2b9c-cdb4-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,027 - distributed.scheduler - INFO - Close client connection: Client-40584e93-cdb3-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,027 - distributed.scheduler - INFO - Close client connection: Client-012325a1-cdb6-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,027 - distributed.scheduler - INFO - Close client connection: Client-67a079ec-cdb7-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,027 - distributed.scheduler - INFO - Close client connection: Client-ce2e165d-cdb8-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,027 - distributed.scheduler - INFO - Close client connection: Client-34b57c80-cdba-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,027 - distributed.scheduler - INFO - Close client connection: Client-9b48f095-cdbb-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,027 - distributed.scheduler - INFO - Close client connection: Client-ced05ede-cdbf-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,027 - distributed.scheduler - INFO - Close client connection: Client-01cbf194-cdbd-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,027 - distributed.scheduler - INFO - Close client connection: Client-68532869-cdbe-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,027 - distributed.scheduler - INFO - Close client connection: Client-35656665-cdc1-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,026 - distributed.scheduler - INFO - Close client connection: Client-9be3efc6-cdc2-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,026 - distributed.scheduler - INFO - Close client connection: Client-025b960b-cdc4-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,026 - distributed.scheduler - INFO - Close client connection: Client-68d318e4-cdc5-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,026 - distributed.scheduler - INFO - Close client connection: Client-cf525d94-cdc6-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,026 - distributed.scheduler - INFO - Close client connection: Client-9c4dc8c0-cdc9-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,026 - distributed.scheduler - INFO - Close client connection: Client-35d6b3d8-cdc8-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,026 - distributed.scheduler - INFO - Close client connection: Client-693cba1a-cdcc-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,026 - distributed.scheduler - INFO - Close client connection: Client-02c85771-cdcb-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,026 - distributed.scheduler - INFO - Close client connection: Client-032992a1-cdd2-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,026 - distributed.scheduler - INFO - Close client connection: Client-cfb043a7-cdcd-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,025 - distributed.scheduler - INFO - Close client connection: Client-363373e1-cdcf-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,025 - distributed.scheduler - INFO - Close client connection: Client-9cae0c17-cdd0-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,023 - distributed.scheduler - INFO - Close client connection: Client-699f0ed1-cdd3-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,020 - distributed.scheduler - INFO - Remove client Client-f0a8fd87-cda3-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,020 - distributed.scheduler - INFO - Remove client Client-572c1bab-cda5-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,020 - distributed.scheduler - INFO - Remove client Client-bd982ede-cda6-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,019 - distributed.scheduler - INFO - Remove client Client-24109f73-cda8-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,019 - distributed.scheduler - INFO - Remove client Client-8a946125-cda9-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,019 - distributed.scheduler - INFO - Remove client Client-f11fb784-cdaa-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,019 - distributed.scheduler - INFO - Remove client Client-579ffdb9-cdac-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,018 - distributed.scheduler - INFO - Remove client Client-be196514-cdad-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,018 - distributed.scheduler - INFO - Remove client Client-2491adbd-cdaf-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,018 - distributed.scheduler - INFO - Remove client Client-8b050c1d-cdb0-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,017 - distributed.scheduler - INFO - Remove client Client-f17a15bf-cdb1-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,017 - distributed.scheduler - INFO - Remove client Client-9aad2b9c-cdb4-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,017 - distributed.scheduler - INFO - Remove client Client-40584e93-cdb3-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,016 - distributed.scheduler - INFO - Remove client Client-012325a1-cdb6-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,016 - distributed.scheduler - INFO - Remove client Client-67a079ec-cdb7-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,016 - distributed.scheduler - INFO - Remove client Client-ce2e165d-cdb8-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,015 - distributed.scheduler - INFO - Remove client Client-34b57c80-cdba-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,015 - distributed.scheduler - INFO - Remove client Client-9b48f095-cdbb-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,015 - distributed.scheduler - INFO - Remove client Client-ced05ede-cdbf-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,014 - distributed.scheduler - INFO - Remove client Client-01cbf194-cdbd-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,014 - distributed.scheduler - INFO - Remove client Client-68532869-cdbe-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,013 - distributed.scheduler - INFO - Remove client Client-35656665-cdc1-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,013 - distributed.scheduler - INFO - Remove client Client-9be3efc6-cdc2-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,013 - distributed.scheduler - INFO - Remove client Client-025b960b-cdc4-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,012 - distributed.scheduler - INFO - Remove client Client-68d318e4-cdc5-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,012 - distributed.scheduler - INFO - Remove client Client-cf525d94-cdc6-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,011 - distributed.scheduler - INFO - Remove client Client-9c4dc8c0-cdc9-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,011 - distributed.scheduler - INFO - Remove client Client-35d6b3d8-cdc8-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,010 - distributed.scheduler - INFO - Remove client Client-693cba1a-cdcc-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,010 - distributed.scheduler - INFO - Remove client Client-02c85771-cdcb-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,010 - distributed.scheduler - INFO - Remove client Client-032992a1-cdd2-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,009 - distributed.scheduler - INFO - Remove client Client-cfb043a7-cdcd-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,009 - distributed.scheduler - INFO - Remove client Client-363373e1-cdcf-11f0-8001-f2cd832f225c

2025-11-30 10:11:14,008 - distributed.scheduler - INFO - Remove client Client-9cae0c17-cdd0-11f0-8001-f2cd832f225c

2025-11-30 10:11:13,995 - distributed.scheduler - INFO - Remove client Client-699f0ed1-cdd3-11f0-8001-f2cd832f225c

2025-11-30 10:11:13,143 - distributed.scheduler - INFO - Close client connection: Client-d01702d3-cdd4-11f0-8001-f2cd832f225c

2025-11-30 10:11:13,141 - distributed.scheduler - INFO - Remove client Client-d01702d3-cdd4-11f0-8001-f2cd832f225c

2025-11-30 10:11:13,141 - distributed.scheduler - INFO - Remove client Client-d01702d3-cdd4-11f0-8001-f2cd832f225c

2025-11-30 10:10:35,663 - distributed.scheduler - INFO - Receive client connection: Client-d01702d3-cdd4-11f0-8001-f2cd832f225c

2025-11-30 10:00:34,250 - distributed.scheduler - INFO - Receive client connection: Client-699f0ed1-cdd3-11f0-8001-f2cd832f225c

2025-11-30 09:50:32,861 - distributed.scheduler - INFO - Receive client connection: Client-032992a1-cdd2-11f0-8001-f2cd832f225c

2025-11-30 09:40:31,424 - distributed.scheduler - INFO - Receive client connection: Client-9cae0c17-cdd0-11f0-8001-f2cd832f225c

2025-11-30 09:30:29,999 - distributed.scheduler - INFO - Receive client connection: Client-363373e1-cdcf-11f0-8001-f2cd832f225c

2025-11-30 09:20:28,511 - distributed.scheduler - INFO - Receive client connection: Client-cfb043a7-cdcd-11f0-8001-f2cd832f225c

2025-11-30 09:10:27,131 - distributed.scheduler - INFO - Receive client connection: Client-693cba1a-cdcc-11f0-8001-f2cd832f225c

2025-11-30 09:00:25,749 - distributed.scheduler - INFO - Receive client connection: Client-02c85771-cdcb-11f0-8001-f2cd832f225c

2025-11-30 08:50:24,308 - distributed.scheduler - INFO - Receive client connection: Client-9c4dc8c0-cdc9-11f0-8001-f2cd832f225c

2025-11-30 08:40:22,906 - distributed.scheduler - INFO - Receive client connection: Client-35d6b3d8-cdc8-11f0-8001-f2cd832f225c

2025-11-30 08:30:21,422 - distributed.scheduler - INFO - Receive client connection: Client-cf525d94-cdc6-11f0-8001-f2cd832f225c

2025-11-30 08:20:19,963 - distributed.scheduler - INFO - Receive client connection: Client-68d318e4-cdc5-11f0-8001-f2cd832f225c

2025-11-30 08:10:18,549 - distributed.scheduler - INFO - Receive client connection: Client-025b960b-cdc4-11f0-8001-f2cd832f225c

2025-11-30 08:00:17,138 - distributed.scheduler - INFO - Receive client connection: Client-9be3efc6-cdc2-11f0-8001-f2cd832f225c

2025-11-30 07:50:15,689 - distributed.scheduler - INFO - Receive client connection: Client-35656665-cdc1-11f0-8001-f2cd832f225c

2025-11-30 07:40:14,087 - distributed.scheduler - INFO - Receive client connection: Client-ced05ede-cdbf-11f0-8001-f2cd832f225c

2025-11-30 07:30:12,645 - distributed.scheduler - INFO - Receive client connection: Client-68532869-cdbe-11f0-8001-f2cd832f225c

2025-11-30 07:20:11,135 - distributed.scheduler - INFO - Receive client connection: Client-01cbf194-cdbd-11f0-8001-f2cd832f225c

2025-11-30 07:10:09,646 - distributed.scheduler - INFO - Receive client connection: Client-9b48f095-cdbb-11f0-8001-f2cd832f225c

2025-11-30 07:00:08,056 - distributed.scheduler - INFO - Receive client connection: Client-34b57c80-cdba-11f0-8001-f2cd832f225c

2025-11-30 06:50:06,549 - distributed.scheduler - INFO - Receive client connection: Client-ce2e165d-cdb8-11f0-8001-f2cd832f225c

2025-11-30 06:40:04,993 - distributed.scheduler - INFO - Receive client connection: Client-67a079ec-cdb7-11f0-8001-f2cd832f225c

2025-11-30 06:30:03,547 - distributed.scheduler - INFO - Receive client connection: Client-012325a1-cdb6-11f0-8001-f2cd832f225c

2025-11-30 06:20:02,148 - distributed.scheduler - INFO - Receive client connection: Client-9aad2b9c-cdb4-11f0-8001-f2cd832f225c

2025-11-30 06:10:21,104 - distributed.scheduler - INFO - Receive client connection: Client-40584e93-cdb3-11f0-8001-f2cd832f225c

2025-11-30 06:00:59,290 - distributed.scheduler - INFO - Receive client connection: Client-f17a15bf-cdb1-11f0-8001-f2cd832f225c

2025-11-30 05:50:57,905 - distributed.scheduler - INFO - Receive client connection: Client-8b050c1d-cdb0-11f0-8001-f2cd832f225c

2025-11-30 05:40:56,517 - distributed.scheduler - INFO - Receive client connection: Client-2491adbd-cdaf-11f0-8001-f2cd832f225c

2025-11-30 05:30:55,096 - distributed.scheduler - INFO - Receive client connection: Client-be196514-cdad-11f0-8001-f2cd832f225c

2025-11-30 05:20:53,683 - distributed.scheduler - INFO - Receive client connection: Client-579ffdb9-cdac-11f0-8001-f2cd832f225c

2025-11-30 05:10:52,221 - distributed.scheduler - INFO - Receive client connection: Client-f11fb784-cdaa-11f0-8001-f2cd832f225c

2025-11-30 05:00:50,684 - distributed.scheduler - INFO - Receive client connection: Client-8a946125-cda9-11f0-8001-f2cd832f225c

2025-11-30 04:50:49,211 - distributed.scheduler - INFO - Receive client connection: Client-24109f73-cda8-11f0-8001-f2cd832f225c

2025-11-30 04:40:47,783 - distributed.scheduler - INFO - Receive client connection: Client-bd982ede-cda6-11f0-8001-f2cd832f225c

2025-11-30 04:30:46,449 - distributed.scheduler - INFO - Receive client connection: Client-572c1bab-cda5-11f0-8001-f2cd832f225c

2025-11-30 04:20:44,976 - distributed.scheduler - INFO - Receive client connection: Client-f0a8fd87-cda3-11f0-8001-f2cd832f225c

2025-11-30 04:11:18,283 - distributed.scheduler - INFO - Close client connection: Client-8a225afd-cda2-11f0-8001-f2cd832f225c

2025-11-30 04:11:18,282 - distributed.scheduler - INFO - Remove client Client-8a225afd-cda2-11f0-8001-f2cd832f225c

2025-11-30 04:11:18,280 - distributed.scheduler - INFO - Remove client Client-8a225afd-cda2-11f0-8001-f2cd832f225c

2025-11-30 04:10:43,466 - distributed.scheduler - INFO - Receive client connection: Client-8a225afd-cda2-11f0-8001-f2cd832f225c

2025-11-30 04:01:50,874 - distributed.scheduler - INFO - Close client connection: Client-23a67793-cda1-11f0-8001-f2cd832f225c

2025-11-30 04:01:50,873 - distributed.scheduler - INFO - Remove client Client-23a67793-cda1-11f0-8001-f2cd832f225c

2025-11-30 04:01:50,872 - distributed.scheduler - INFO - Remove client Client-23a67793-cda1-11f0-8001-f2cd832f225c

2025-11-30 04:00:42,033 - distributed.scheduler - INFO - Receive client connection: Client-23a67793-cda1-11f0-8001-f2cd832f225c

2025-11-30 03:51:20,000 - distributed.scheduler - INFO - Close client connection: Client-bd306461-cd9f-11f0-8001-f2cd832f225c

2025-11-30 03:51:19,999 - distributed.scheduler - INFO - Remove client Client-bd306461-cd9f-11f0-8001-f2cd832f225c

2025-11-30 03:51:19,997 - distributed.scheduler - INFO - Remove client Client-bd306461-cd9f-11f0-8001-f2cd832f225c

2025-11-30 03:50:40,663 - distributed.scheduler - INFO - Receive client connection: Client-bd306461-cd9f-11f0-8001-f2cd832f225c

2025-11-30 03:41:15,228 - distributed.scheduler - INFO - Close client connection: Client-56b5219a-cd9e-11f0-8001-f2cd832f225c

2025-11-30 03:41:15,228 - distributed.scheduler - INFO - Remove client Client-56b5219a-cd9e-11f0-8001-f2cd832f225c

2025-11-30 03:41:15,227 - distributed.scheduler - INFO - Remove client Client-56b5219a-cd9e-11f0-8001-f2cd832f225c

2025-11-30 03:40:39,231 - distributed.scheduler - INFO - Receive client connection: Client-56b5219a-cd9e-11f0-8001-f2cd832f225c

2025-11-30 03:31:45,548 - distributed.scheduler - INFO - Close client connection: Client-f03c5b11-cd9c-11f0-8001-f2cd832f225c

2025-11-30 03:31:45,547 - distributed.scheduler - INFO - Remove client Client-f03c5b11-cd9c-11f0-8001-f2cd832f225c

2025-11-30 03:31:45,545 - distributed.scheduler - INFO - Remove client Client-f03c5b11-cd9c-11f0-8001-f2cd832f225c

2025-11-30 03:30:37,787 - distributed.scheduler - INFO - Receive client connection: Client-f03c5b11-cd9c-11f0-8001-f2cd832f225c

2025-11-30 03:21:13,237 - distributed.scheduler - INFO - Close client connection: Client-89c6d0d9-cd9b-11f0-8001-f2cd832f225c

2025-11-30 03:21:13,236 - distributed.scheduler - INFO - Remove client Client-89c6d0d9-cd9b-11f0-8001-f2cd832f225c

2025-11-30 03:21:13,234 - distributed.scheduler - INFO - Remove client Client-89c6d0d9-cd9b-11f0-8001-f2cd832f225c

2025-11-30 03:20:36,392 - distributed.scheduler - INFO - Receive client connection: Client-89c6d0d9-cd9b-11f0-8001-f2cd832f225c

2025-11-30 03:11:09,543 - distributed.scheduler - INFO - Close client connection: Client-234b77e6-cd9a-11f0-8001-f2cd832f225c

2025-11-30 03:11:09,542 - distributed.scheduler - INFO - Remove client Client-234b77e6-cd9a-11f0-8001-f2cd832f225c

2025-11-30 03:11:09,540 - distributed.scheduler - INFO - Remove client Client-234b77e6-cd9a-11f0-8001-f2cd832f225c

2025-11-30 03:10:34,960 - distributed.scheduler - INFO - Receive client connection: Client-234b77e6-cd9a-11f0-8001-f2cd832f225c

2025-11-30 03:01:40,963 - distributed.scheduler - INFO - Close client connection: Client-bcb8eb58-cd98-11f0-8001-f2cd832f225c

2025-11-30 03:01:40,959 - distributed.scheduler - INFO - Remove client Client-bcb8eb58-cd98-11f0-8001-f2cd832f225c

2025-11-30 03:01:40,957 - distributed.scheduler - INFO - Remove client Client-bcb8eb58-cd98-11f0-8001-f2cd832f225c

2025-11-30 03:00:33,375 - distributed.scheduler - INFO - Receive client connection: Client-bcb8eb58-cd98-11f0-8001-f2cd832f225c

2025-11-30 02:51:11,422 - distributed.scheduler - INFO - Close client connection: Client-563fda33-cd97-11f0-8001-f2cd832f225c

2025-11-30 02:51:11,421 - distributed.scheduler - INFO - Remove client Client-563fda33-cd97-11f0-8001-f2cd832f225c

2025-11-30 02:51:11,419 - distributed.scheduler - INFO - Remove client Client-563fda33-cd97-11f0-8001-f2cd832f225c

2025-11-30 02:50:31,954 - distributed.scheduler - INFO - Receive client connection: Client-563fda33-cd97-11f0-8001-f2cd832f225c

2025-11-30 02:40:58,195 - distributed.scheduler - INFO - Close client connection: Client-efc7d7db-cd95-11f0-8001-f2cd832f225c

2025-11-30 02:40:58,191 - distributed.scheduler - INFO - Remove client Client-efc7d7db-cd95-11f0-8001-f2cd832f225c

2025-11-30 02:40:58,190 - distributed.scheduler - INFO - Remove client Client-efc7d7db-cd95-11f0-8001-f2cd832f225c

2025-11-30 02:40:30,556 - distributed.scheduler - INFO - Receive client connection: Client-efc7d7db-cd95-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,996 - distributed.scheduler - INFO - Close client connection: Client-db3e29f2-cd52-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,996 - distributed.scheduler - INFO - Close client connection: Client-41b4fb0e-cd54-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,996 - distributed.scheduler - INFO - Close client connection: Client-a833d144-cd55-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,996 - distributed.scheduler - INFO - Close client connection: Client-75364ee1-cd58-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,996 - distributed.scheduler - INFO - Close client connection: Client-0ebb39bf-cd57-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,996 - distributed.scheduler - INFO - Close client connection: Client-dbc74aeb-cd59-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,996 - distributed.scheduler - INFO - Close client connection: Client-4253fcc5-cd5b-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,996 - distributed.scheduler - INFO - Close client connection: Client-a8c53790-cd5c-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,996 - distributed.scheduler - INFO - Close client connection: Client-0f45cbf6-cd5e-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,996 - distributed.scheduler - INFO - Close client connection: Client-75dabf5c-cd5f-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,996 - distributed.scheduler - INFO - Close client connection: Client-42d66520-cd62-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,995 - distributed.scheduler - INFO - Close client connection: Client-dc57a063-cd60-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,995 - distributed.scheduler - INFO - Close client connection: Client-54d8aabd-cd82-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,995 - distributed.scheduler - INFO - Close client connection: Client-a95a44f3-cd63-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,995 - distributed.scheduler - INFO - Close client connection: Client-0fd8a86d-cd65-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,995 - distributed.scheduler - INFO - Close client connection: Client-7659dbbe-cd66-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,995 - distributed.scheduler - INFO - Close client connection: Client-dcdba269-cd67-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,995 - distributed.scheduler - INFO - Close client connection: Client-434a5eb9-cd69-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,995 - distributed.scheduler - INFO - Close client connection: Client-a9d1af5c-cd6a-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,995 - distributed.scheduler - INFO - Close client connection: Client-105188d6-cd6c-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,995 - distributed.scheduler - INFO - Close client connection: Client-76d306ab-cd6d-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,995 - distributed.scheduler - INFO - Close client connection: Client-dd437f3b-cd6e-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,994 - distributed.scheduler - INFO - Close client connection: Client-43c8abde-cd70-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,994 - distributed.scheduler - INFO - Close client connection: Client-aa44ec8d-cd71-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,994 - distributed.scheduler - INFO - Close client connection: Client-10ff0fba-cd73-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,994 - distributed.scheduler - INFO - Close client connection: Client-777d36b7-cd74-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,994 - distributed.scheduler - INFO - Close client connection: Client-de01a2c1-cd75-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,994 - distributed.scheduler - INFO - Close client connection: Client-44716044-cd77-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,994 - distributed.scheduler - INFO - Close client connection: Client-aaf544f2-cd78-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,994 - distributed.scheduler - INFO - Close client connection: Client-541f7f67-cd7b-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,994 - distributed.scheduler - INFO - Close client connection: Client-ed9c0615-cd79-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,994 - distributed.scheduler - INFO - Close client connection: Client-bac35a00-cd7c-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,994 - distributed.scheduler - INFO - Close client connection: Client-87cc258d-cd7f-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,994 - distributed.scheduler - INFO - Close client connection: Client-21492938-cd7e-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,993 - distributed.scheduler - INFO - Close client connection: Client-ee5ad784-cd80-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,993 - distributed.scheduler - INFO - Close client connection: Client-bb5ace6d-cd83-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,993 - distributed.scheduler - INFO - Close client connection: Client-eee03416-cd87-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,993 - distributed.scheduler - INFO - Close client connection: Client-21db2e3f-cd85-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,993 - distributed.scheduler - INFO - Close client connection: Client-8862a66b-cd86-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,993 - distributed.scheduler - INFO - Close client connection: Client-88e5a00d-cd8d-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,993 - distributed.scheduler - INFO - Close client connection: Client-556658d4-cd89-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,993 - distributed.scheduler - INFO - Close client connection: Client-bbea885c-cd8a-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,993 - distributed.scheduler - INFO - Close client connection: Client-ef5f9deb-cd8e-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,993 - distributed.scheduler - INFO - Close client connection: Client-226b5a2a-cd8c-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,993 - distributed.scheduler - INFO - Close client connection: Client-55dee2f8-cd90-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,992 - distributed.scheduler - INFO - Close client connection: Client-22cd4c60-cd93-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,991 - distributed.scheduler - INFO - Close client connection: Client-bc6307b7-cd91-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,987 - distributed.scheduler - INFO - Remove client Client-db3e29f2-cd52-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,987 - distributed.scheduler - INFO - Remove client Client-41b4fb0e-cd54-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,987 - distributed.scheduler - INFO - Remove client Client-a833d144-cd55-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,986 - distributed.scheduler - INFO - Remove client Client-75364ee1-cd58-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,986 - distributed.scheduler - INFO - Remove client Client-0ebb39bf-cd57-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,986 - distributed.scheduler - INFO - Remove client Client-dbc74aeb-cd59-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,985 - distributed.scheduler - INFO - Remove client Client-4253fcc5-cd5b-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,985 - distributed.scheduler - INFO - Remove client Client-a8c53790-cd5c-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,985 - distributed.scheduler - INFO - Remove client Client-0f45cbf6-cd5e-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,985 - distributed.scheduler - INFO - Remove client Client-75dabf5c-cd5f-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,984 - distributed.scheduler - INFO - Remove client Client-42d66520-cd62-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,984 - distributed.scheduler - INFO - Remove client Client-dc57a063-cd60-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,984 - distributed.scheduler - INFO - Remove client Client-54d8aabd-cd82-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,983 - distributed.scheduler - INFO - Remove client Client-a95a44f3-cd63-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,983 - distributed.scheduler - INFO - Remove client Client-0fd8a86d-cd65-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,983 - distributed.scheduler - INFO - Remove client Client-7659dbbe-cd66-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,983 - distributed.scheduler - INFO - Remove client Client-dcdba269-cd67-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,982 - distributed.scheduler - INFO - Remove client Client-434a5eb9-cd69-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,982 - distributed.scheduler - INFO - Remove client Client-a9d1af5c-cd6a-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,981 - distributed.scheduler - INFO - Remove client Client-105188d6-cd6c-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,981 - distributed.scheduler - INFO - Remove client Client-76d306ab-cd6d-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,980 - distributed.scheduler - INFO - Remove client Client-dd437f3b-cd6e-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,980 - distributed.scheduler - INFO - Remove client Client-43c8abde-cd70-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,980 - distributed.scheduler - INFO - Remove client Client-aa44ec8d-cd71-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,979 - distributed.scheduler - INFO - Remove client Client-10ff0fba-cd73-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,979 - distributed.scheduler - INFO - Remove client Client-777d36b7-cd74-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,979 - distributed.scheduler - INFO - Remove client Client-de01a2c1-cd75-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,978 - distributed.scheduler - INFO - Remove client Client-44716044-cd77-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,978 - distributed.scheduler - INFO - Remove client Client-aaf544f2-cd78-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,977 - distributed.scheduler - INFO - Remove client Client-541f7f67-cd7b-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,977 - distributed.scheduler - INFO - Remove client Client-ed9c0615-cd79-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,977 - distributed.scheduler - INFO - Remove client Client-bac35a00-cd7c-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,976 - distributed.scheduler - INFO - Remove client Client-87cc258d-cd7f-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,976 - distributed.scheduler - INFO - Remove client Client-21492938-cd7e-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,976 - distributed.scheduler - INFO - Remove client Client-ee5ad784-cd80-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,975 - distributed.scheduler - INFO - Remove client Client-bb5ace6d-cd83-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,975 - distributed.scheduler - INFO - Remove client Client-eee03416-cd87-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,974 - distributed.scheduler - INFO - Remove client Client-21db2e3f-cd85-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,974 - distributed.scheduler - INFO - Remove client Client-8862a66b-cd86-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,974 - distributed.scheduler - INFO - Remove client Client-88e5a00d-cd8d-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,973 - distributed.scheduler - INFO - Remove client Client-556658d4-cd89-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,973 - distributed.scheduler - INFO - Remove client Client-bbea885c-cd8a-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,972 - distributed.scheduler - INFO - Remove client Client-ef5f9deb-cd8e-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,972 - distributed.scheduler - INFO - Remove client Client-226b5a2a-cd8c-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,971 - distributed.scheduler - INFO - Remove client Client-55dee2f8-cd90-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,971 - distributed.scheduler - INFO - Remove client Client-22cd4c60-cd93-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,957 - distributed.scheduler - INFO - Remove client Client-bc6307b7-cd91-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,033 - distributed.scheduler - INFO - Close client connection: Client-8946c3a1-cd94-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,032 - distributed.scheduler - INFO - Remove client Client-8946c3a1-cd94-11f0-8001-f2cd832f225c

2025-11-30 02:31:20,032 - distributed.scheduler - INFO - Remove client Client-8946c3a1-cd94-11f0-8001-f2cd832f225c

2025-11-30 02:30:29,067 - distributed.scheduler - INFO - Receive client connection: Client-8946c3a1-cd94-11f0-8001-f2cd832f225c

2025-11-30 02:20:27,651 - distributed.scheduler - INFO - Receive client connection: Client-22cd4c60-cd93-11f0-8001-f2cd832f225c

2025-11-30 02:10:26,327 - distributed.scheduler - INFO - Receive client connection: Client-bc6307b7-cd91-11f0-8001-f2cd832f225c

2025-11-30 02:00:24,832 - distributed.scheduler - INFO - Receive client connection: Client-55dee2f8-cd90-11f0-8001-f2cd832f225c

2025-11-30 01:50:23,376 - distributed.scheduler - INFO - Receive client connection: Client-ef5f9deb-cd8e-11f0-8001-f2cd832f225c

2025-11-30 01:40:21,959 - distributed.scheduler - INFO - Receive client connection: Client-88e5a00d-cd8d-11f0-8001-f2cd832f225c

2025-11-30 01:30:20,538 - distributed.scheduler - INFO - Receive client connection: Client-226b5a2a-cd8c-11f0-8001-f2cd832f225c

2025-11-30 01:20:19,054 - distributed.scheduler - INFO - Receive client connection: Client-bbea885c-cd8a-11f0-8001-f2cd832f225c

2025-11-30 01:10:17,569 - distributed.scheduler - INFO - Receive client connection: Client-556658d4-cd89-11f0-8001-f2cd832f225c

2025-11-30 01:00:16,074 - distributed.scheduler - INFO - Receive client connection: Client-eee03416-cd87-11f0-8001-f2cd832f225c

2025-11-30 00:50:14,616 - distributed.scheduler - INFO - Receive client connection: Client-8862a66b-cd86-11f0-8001-f2cd832f225c

2025-11-30 00:40:13,106 - distributed.scheduler - INFO - Receive client connection: Client-21db2e3f-cd85-11f0-8001-f2cd832f225c

2025-11-30 00:30:11,635 - distributed.scheduler - INFO - Receive client connection: Client-bb5ace6d-cd83-11f0-8001-f2cd832f225c

2025-11-30 00:20:10,163 - distributed.scheduler - INFO - Receive client connection: Client-54d8aabd-cd82-11f0-8001-f2cd832f225c

2025-11-30 00:10:08,712 - distributed.scheduler - INFO - Receive client connection: Client-ee5ad784-cd80-11f0-8001-f2cd832f225c

2025-11-30 00:00:07,156 - distributed.scheduler - INFO - Receive client connection: Client-87cc258d-cd7f-11f0-8001-f2cd832f225c

2025-11-29 23:50:05,673 - distributed.scheduler - INFO - Receive client connection: Client-21492938-cd7e-11f0-8001-f2cd832f225c

2025-11-29 23:40:04,173 - distributed.scheduler - INFO - Receive client connection: Client-bac35a00-cd7c-11f0-8001-f2cd832f225c

2025-11-29 23:30:02,469 - distributed.scheduler - INFO - Receive client connection: Client-541f7f67-cd7b-11f0-8001-f2cd832f225c

2025-11-29 23:20:00,983 - distributed.scheduler - INFO - Receive client connection: Client-ed9c0615-cd79-11f0-8001-f2cd832f225c

2025-11-29 23:10:59,659 - distributed.scheduler - INFO - Receive client connection: Client-aaf544f2-cd78-11f0-8001-f2cd832f225c

2025-11-29 23:00:58,178 - distributed.scheduler - INFO - Receive client connection: Client-44716044-cd77-11f0-8001-f2cd832f225c

2025-11-29 22:50:56,828 - distributed.scheduler - INFO - Receive client connection: Client-de01a2c1-cd75-11f0-8001-f2cd832f225c

2025-11-29 22:40:55,322 - distributed.scheduler - INFO - Receive client connection: Client-777d36b7-cd74-11f0-8001-f2cd832f225c

2025-11-29 22:30:53,873 - distributed.scheduler - INFO - Receive client connection: Client-10ff0fba-cd73-11f0-8001-f2cd832f225c

2025-11-29 22:20:52,036 - distributed.scheduler - INFO - Receive client connection: Client-aa44ec8d-cd71-11f0-8001-f2cd832f225c

2025-11-29 22:10:50,592 - distributed.scheduler - INFO - Receive client connection: Client-43c8abde-cd70-11f0-8001-f2cd832f225c

2025-11-29 22:00:49,097 - distributed.scheduler - INFO - Receive client connection: Client-dd437f3b-cd6e-11f0-8001-f2cd832f225c

2025-11-29 21:50:47,731 - distributed.scheduler - INFO - Receive client connection: Client-76d306ab-cd6d-11f0-8001-f2cd832f225c

2025-11-29 21:40:46,270 - distributed.scheduler - INFO - Receive client connection: Client-105188d6-cd6c-11f0-8001-f2cd832f225c

2025-11-29 21:30:44,803 - distributed.scheduler - INFO - Receive client connection: Client-a9d1af5c-cd6a-11f0-8001-f2cd832f225c

2025-11-29 21:20:43,290 - distributed.scheduler - INFO - Receive client connection: Client-434a5eb9-cd69-11f0-8001-f2cd832f225c

2025-11-29 21:10:41,938 - distributed.scheduler - INFO - Receive client connection: Client-dcdba269-cd67-11f0-8001-f2cd832f225c

2025-11-29 21:00:40,466 - distributed.scheduler - INFO - Receive client connection: Client-7659dbbe-cd66-11f0-8001-f2cd832f225c

2025-11-29 20:50:38,998 - distributed.scheduler - INFO - Receive client connection: Client-0fd8a86d-cd65-11f0-8001-f2cd832f225c

2025-11-29 20:40:37,542 - distributed.scheduler - INFO - Receive client connection: Client-a95a44f3-cd63-11f0-8001-f2cd832f225c

2025-11-29 20:30:41,437 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('len-agg-6e2fa02138b574fca793373a51c7a6e2', 0) NoneType: None

2025-11-29 20:30:41,437 - distributed.scheduler - ERROR - Couldn't gather keys {"('len-agg-6e2fa02138b574fca793373a51c7a6e2', 0)": []} state: ['waiting'] workers: [] NoneType: None

2025-11-29 20:30:36,050 - distributed.scheduler - INFO - Receive client connection: Client-42d66520-cd62-11f0-8001-f2cd832f225c

2025-11-29 20:20:34,593 - distributed.scheduler - INFO - Receive client connection: Client-dc57a063-cd60-11f0-8001-f2cd832f225c

2025-11-29 20:10:33,165 - distributed.scheduler - INFO - Receive client connection: Client-75dabf5c-cd5f-11f0-8001-f2cd832f225c

2025-11-29 20:00:31,552 - distributed.scheduler - INFO - Receive client connection: Client-0f45cbf6-cd5e-11f0-8001-f2cd832f225c

2025-11-29 19:50:30,089 - distributed.scheduler - INFO - Receive client connection: Client-a8c53790-cd5c-11f0-8001-f2cd832f225c

2025-11-29 19:40:28,713 - distributed.scheduler - INFO - Receive client connection: Client-4253fcc5-cd5b-11f0-8001-f2cd832f225c

2025-11-29 19:30:27,175 - distributed.scheduler - INFO - Receive client connection: Client-dbc74aeb-cd59-11f0-8001-f2cd832f225c

2025-11-29 19:20:25,603 - distributed.scheduler - INFO - Receive client connection: Client-75364ee1-cd58-11f0-8001-f2cd832f225c

2025-11-29 19:10:24,167 - distributed.scheduler - INFO - Receive client connection: Client-0ebb39bf-cd57-11f0-8001-f2cd832f225c

2025-11-29 19:00:22,655 - distributed.scheduler - INFO - Receive client connection: Client-a833d144-cd55-11f0-8001-f2cd832f225c

2025-11-29 18:50:21,199 - distributed.scheduler - INFO - Receive client connection: Client-41b4fb0e-cd54-11f0-8001-f2cd832f225c

2025-11-29 18:40:19,812 - distributed.scheduler - INFO - Receive client connection: Client-db3e29f2-cd52-11f0-8001-f2cd832f225c

2025-11-29 18:31:25,350 - distributed.scheduler - INFO - Close client connection: Client-74c922c3-cd51-11f0-8001-f2cd832f225c

2025-11-29 18:31:25,349 - distributed.scheduler - INFO - Remove client Client-74c922c3-cd51-11f0-8001-f2cd832f225c

2025-11-29 18:31:25,347 - distributed.scheduler - INFO - Remove client Client-74c922c3-cd51-11f0-8001-f2cd832f225c

2025-11-29 18:30:18,410 - distributed.scheduler - INFO - Receive client connection: Client-74c922c3-cd51-11f0-8001-f2cd832f225c

2025-11-29 18:20:52,812 - distributed.scheduler - INFO - Close client connection: Client-0e4c3e7c-cd50-11f0-8001-f2cd832f225c

2025-11-29 18:20:52,811 - distributed.scheduler - INFO - Remove client Client-0e4c3e7c-cd50-11f0-8001-f2cd832f225c

2025-11-29 18:20:52,811 - distributed.scheduler - INFO - Remove client Client-0e4c3e7c-cd50-11f0-8001-f2cd832f225c

2025-11-29 18:20:16,971 - distributed.scheduler - INFO - Receive client connection: Client-0e4c3e7c-cd50-11f0-8001-f2cd832f225c

2025-11-29 18:10:51,467 - distributed.scheduler - INFO - Close client connection: Client-a7d646ea-cd4e-11f0-8001-f2cd832f225c

2025-11-29 18:10:51,464 - distributed.scheduler - INFO - Remove client Client-a7d646ea-cd4e-11f0-8001-f2cd832f225c

2025-11-29 18:10:51,462 - distributed.scheduler - INFO - Remove client Client-a7d646ea-cd4e-11f0-8001-f2cd832f225c

2025-11-29 18:10:15,575 - distributed.scheduler - INFO - Receive client connection: Client-a7d646ea-cd4e-11f0-8001-f2cd832f225c

2025-11-29 18:01:22,600 - distributed.scheduler - INFO - Close client connection: Client-4140f2ce-cd4d-11f0-8001-f2cd832f225c

2025-11-29 18:01:22,599 - distributed.scheduler - INFO - Remove client Client-4140f2ce-cd4d-11f0-8001-f2cd832f225c

2025-11-29 18:01:22,598 - distributed.scheduler - INFO - Remove client Client-4140f2ce-cd4d-11f0-8001-f2cd832f225c

2025-11-29 18:00:13,971 - distributed.scheduler - INFO - Receive client connection: Client-4140f2ce-cd4d-11f0-8001-f2cd832f225c

2025-11-29 17:50:50,442 - distributed.scheduler - INFO - Close client connection: Client-dac5f303-cd4b-11f0-8001-f2cd832f225c

2025-11-29 17:50:50,441 - distributed.scheduler - INFO - Remove client Client-dac5f303-cd4b-11f0-8001-f2cd832f225c

2025-11-29 17:50:50,439 - distributed.scheduler - INFO - Remove client Client-dac5f303-cd4b-11f0-8001-f2cd832f225c

2025-11-29 17:50:12,542 - distributed.scheduler - INFO - Receive client connection: Client-dac5f303-cd4b-11f0-8001-f2cd832f225c

2025-11-29 17:40:47,842 - distributed.scheduler - INFO - Close client connection: Client-74506e75-cd4a-11f0-8001-f2cd832f225c

2025-11-29 17:40:47,841 - distributed.scheduler - INFO - Remove client Client-74506e75-cd4a-11f0-8001-f2cd832f225c

2025-11-29 17:40:47,840 - distributed.scheduler - INFO - Remove client Client-74506e75-cd4a-11f0-8001-f2cd832f225c

2025-11-29 17:40:11,150 - distributed.scheduler - INFO - Receive client connection: Client-74506e75-cd4a-11f0-8001-f2cd832f225c

2025-11-29 17:31:16,238 - distributed.scheduler - INFO - Close client connection: Client-0dd26319-cd49-11f0-8001-f2cd832f225c

2025-11-29 17:31:16,237 - distributed.scheduler - INFO - Remove client Client-0dd26319-cd49-11f0-8001-f2cd832f225c

2025-11-29 17:31:16,235 - distributed.scheduler - INFO - Remove client Client-0dd26319-cd49-11f0-8001-f2cd832f225c

2025-11-29 17:30:09,701 - distributed.scheduler - INFO - Receive client connection: Client-0dd26319-cd49-11f0-8001-f2cd832f225c

2025-11-29 17:20:44,285 - distributed.scheduler - INFO - Close client connection: Client-da2ff0bc-cd44-11f0-8001-f2cd832f225c

2025-11-29 17:20:44,285 - distributed.scheduler - INFO - Close client connection: Client-40c34eab-cd46-11f0-8001-f2cd832f225c

2025-11-29 17:20:44,284 - distributed.scheduler - INFO - Remove client Client-da2ff0bc-cd44-11f0-8001-f2cd832f225c

2025-11-29 17:20:44,283 - distributed.scheduler - INFO - Remove client Client-40c34eab-cd46-11f0-8001-f2cd832f225c

2025-11-29 17:20:43,501 - distributed.scheduler - INFO - Close client connection: Client-a74d8ca7-cd47-11f0-8001-f2cd832f225c

2025-11-29 17:20:43,499 - distributed.scheduler - INFO - Remove client Client-a74d8ca7-cd47-11f0-8001-f2cd832f225c

2025-11-29 17:20:43,497 - distributed.scheduler - INFO - Remove client Client-a74d8ca7-cd47-11f0-8001-f2cd832f225c

2025-11-29 17:20:08,195 - distributed.scheduler - INFO - Receive client connection: Client-a74d8ca7-cd47-11f0-8001-f2cd832f225c

2025-11-29 17:10:06,662 - distributed.scheduler - INFO - Receive client connection: Client-40c34eab-cd46-11f0-8001-f2cd832f225c

2025-11-29 17:00:05,085 - distributed.scheduler - INFO - Receive client connection: Client-da2ff0bc-cd44-11f0-8001-f2cd832f225c

2025-11-29 16:50:39,268 - distributed.scheduler - INFO - Close client connection: Client-73b4d10e-cd43-11f0-8001-f2cd832f225c

2025-11-29 16:50:39,266 - distributed.scheduler - INFO - Remove client Client-73b4d10e-cd43-11f0-8001-f2cd832f225c

2025-11-29 16:50:39,265 - distributed.scheduler - INFO - Remove client Client-73b4d10e-cd43-11f0-8001-f2cd832f225c

2025-11-29 16:50:03,650 - distributed.scheduler - INFO - Receive client connection: Client-73b4d10e-cd43-11f0-8001-f2cd832f225c

2025-11-29 16:40:40,241 - distributed.scheduler - INFO - Close client connection: Client-0d3ac58a-cd42-11f0-8001-f2cd832f225c

2025-11-29 16:40:40,239 - distributed.scheduler - INFO - Remove client Client-0d3ac58a-cd42-11f0-8001-f2cd832f225c

2025-11-29 16:40:40,238 - distributed.scheduler - INFO - Remove client Client-0d3ac58a-cd42-11f0-8001-f2cd832f225c

2025-11-29 16:40:02,228 - distributed.scheduler - INFO - Receive client connection: Client-0d3ac58a-cd42-11f0-8001-f2cd832f225c

2025-11-29 16:31:03,856 - distributed.scheduler - INFO - Close client connection: Client-a6bd7e53-cd40-11f0-8001-f2cd832f225c

2025-11-29 16:31:03,851 - distributed.scheduler - INFO - Remove client Client-a6bd7e53-cd40-11f0-8001-f2cd832f225c

2025-11-29 16:31:03,849 - distributed.scheduler - INFO - Remove client Client-a6bd7e53-cd40-11f0-8001-f2cd832f225c

2025-11-29 16:30:06,288 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('len-agg-2958699b3f2aefa0adf52927ccbed12e', 0) NoneType: None

2025-11-29 16:30:06,287 - distributed.scheduler - ERROR - Couldn't gather keys {"('len-agg-2958699b3f2aefa0adf52927ccbed12e', 0)": []} state: ['waiting'] workers: [] NoneType: None

2025-11-29 16:30:05,456 - distributed.scheduler - INFO - Receive client connection: Client-worker-a9807ff2-cd40-11f0-86e2-8253c573df33

2025-11-29 16:30:00,781 - distributed.scheduler - INFO - Receive client connection: Client-a6bd7e53-cd40-11f0-8001-f2cd832f225c

2025-11-29 16:21:35,138 - distributed.scheduler - INFO - Close client connection: Client-6418dd9d-cd3f-11f0-8001-f2cd832f225c

2025-11-29 16:21:35,137 - distributed.scheduler - INFO - Remove client Client-6418dd9d-cd3f-11f0-8001-f2cd832f225c

2025-11-29 16:21:35,135 - distributed.scheduler - INFO - Remove client Client-6418dd9d-cd3f-11f0-8001-f2cd832f225c

2025-11-29 16:20:59,477 - distributed.scheduler - INFO - Receive client connection: Client-6418dd9d-cd3f-11f0-8001-f2cd832f225c

2025-11-29 16:11:32,981 - distributed.scheduler - INFO - Close client connection: Client-fd98ed91-cd3d-11f0-8001-f2cd832f225c

2025-11-29 16:11:32,980 - distributed.scheduler - INFO - Remove client Client-fd98ed91-cd3d-11f0-8001-f2cd832f225c

2025-11-29 16:11:32,978 - distributed.scheduler - INFO - Remove client Client-fd98ed91-cd3d-11f0-8001-f2cd832f225c

2025-11-29 16:10:58,012 - distributed.scheduler - INFO - Receive client connection: Client-fd98ed91-cd3d-11f0-8001-f2cd832f225c

2025-11-29 16:02:08,844 - distributed.scheduler - INFO - Close client connection: Client-9720f851-cd3c-11f0-8001-f2cd832f225c

2025-11-29 16:02:08,842 - distributed.scheduler - INFO - Remove client Client-9720f851-cd3c-11f0-8001-f2cd832f225c

2025-11-29 16:02:08,840 - distributed.scheduler - INFO - Remove client Client-9720f851-cd3c-11f0-8001-f2cd832f225c

2025-11-29 16:00:56,602 - distributed.scheduler - INFO - Receive client connection: Client-9720f851-cd3c-11f0-8001-f2cd832f225c

2025-11-29 15:51:32,250 - distributed.scheduler - INFO - Close client connection: Client-30a5a655-cd3b-11f0-8001-f2cd832f225c

2025-11-29 15:51:32,249 - distributed.scheduler - INFO - Remove client Client-30a5a655-cd3b-11f0-8001-f2cd832f225c

2025-11-29 15:51:32,247 - distributed.scheduler - INFO - Remove client Client-30a5a655-cd3b-11f0-8001-f2cd832f225c

2025-11-29 15:50:55,172 - distributed.scheduler - INFO - Receive client connection: Client-30a5a655-cd3b-11f0-8001-f2cd832f225c

2025-11-29 15:41:30,370 - distributed.scheduler - INFO - Close client connection: Client-ca31340b-cd39-11f0-8001-f2cd832f225c

2025-11-29 15:41:30,369 - distributed.scheduler - INFO - Remove client Client-ca31340b-cd39-11f0-8001-f2cd832f225c

2025-11-29 15:41:30,367 - distributed.scheduler - INFO - Remove client Client-ca31340b-cd39-11f0-8001-f2cd832f225c

2025-11-29 15:40:53,778 - distributed.scheduler - INFO - Receive client connection: Client-ca31340b-cd39-11f0-8001-f2cd832f225c

2025-11-29 15:32:03,662 - distributed.scheduler - INFO - Close client connection: Client-63b7bdd4-cd38-11f0-8001-f2cd832f225c

2025-11-29 15:32:03,661 - distributed.scheduler - INFO - Remove client Client-63b7bdd4-cd38-11f0-8001-f2cd832f225c

2025-11-29 15:32:03,660 - distributed.scheduler - INFO - Remove client Client-63b7bdd4-cd38-11f0-8001-f2cd832f225c

2025-11-29 15:30:52,365 - distributed.scheduler - INFO - Receive client connection: Client-63b7bdd4-cd38-11f0-8001-f2cd832f225c

2025-11-29 15:21:29,605 - distributed.scheduler - INFO - Close client connection: Client-fd3946d5-cd36-11f0-8001-f2cd832f225c

2025-11-29 15:21:29,604 - distributed.scheduler - INFO - Remove client Client-fd3946d5-cd36-11f0-8001-f2cd832f225c

2025-11-29 15:21:29,601 - distributed.scheduler - INFO - Remove client Client-fd3946d5-cd36-11f0-8001-f2cd832f225c

2025-11-29 15:20:50,902 - distributed.scheduler - INFO - Receive client connection: Client-fd3946d5-cd36-11f0-8001-f2cd832f225c

2025-11-29 15:11:28,114 - distributed.scheduler - INFO - Close client connection: Client-c99c6473-cd32-11f0-8001-f2cd832f225c

2025-11-29 15:11:28,113 - distributed.scheduler - INFO - Close client connection: Client-301faf3b-cd34-11f0-8001-f2cd832f225c

2025-11-29 15:11:28,113 - distributed.scheduler - INFO - Remove client Client-c99c6473-cd32-11f0-8001-f2cd832f225c

2025-11-29 15:11:28,112 - distributed.scheduler - INFO - Remove client Client-301faf3b-cd34-11f0-8001-f2cd832f225c

2025-11-29 15:11:27,350 - distributed.scheduler - INFO - Close client connection: Client-96bc7190-cd35-11f0-8001-f2cd832f225c

2025-11-29 15:11:27,348 - distributed.scheduler - INFO - Remove client Client-96bc7190-cd35-11f0-8001-f2cd832f225c

2025-11-29 15:11:27,346 - distributed.scheduler - INFO - Remove client Client-96bc7190-cd35-11f0-8001-f2cd832f225c

2025-11-29 15:10:49,466 - distributed.scheduler - INFO - Receive client connection: Client-96bc7190-cd35-11f0-8001-f2cd832f225c

2025-11-29 15:00:54,105 - distributed.scheduler - INFO - Receive client connection: Client-worker-33d431f3-cd34-11f0-8623-8a1d5a425dc1

2025-11-29 15:00:52,275 - distributed.scheduler - INFO - Receive client connection: Client-worker-32cafad5-cd34-11f0-866e-f2a719d69604

2025-11-29 15:00:47,802 - distributed.scheduler - INFO - Receive client connection: Client-301faf3b-cd34-11f0-8001-f2cd832f225c

2025-11-29 14:50:46,326 - distributed.scheduler - INFO - Receive client connection: Client-c99c6473-cd32-11f0-8001-f2cd832f225c

2025-11-29 14:41:24,670 - distributed.scheduler - INFO - Close client connection: Client-63107610-cd31-11f0-8001-f2cd832f225c

2025-11-29 14:41:24,669 - distributed.scheduler - INFO - Remove client Client-63107610-cd31-11f0-8001-f2cd832f225c

2025-11-29 14:41:24,668 - distributed.scheduler - INFO - Remove client Client-63107610-cd31-11f0-8001-f2cd832f225c

2025-11-29 14:40:44,788 - distributed.scheduler - INFO - Receive client connection: Client-63107610-cd31-11f0-8001-f2cd832f225c

2025-11-29 14:35:03,801 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:44823 failed: OSError: Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s

2025-11-29 14:35:03,787 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s')

2025-11-29 14:35:03,745 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s')

2025-11-29 14:35:03,707 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41957 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s')

2025-11-29 14:35:03,654 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:43399 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s')

2025-11-29 14:35:03,650 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s')

2025-11-29 14:35:03,649 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s')

2025-11-29 14:35:03,645 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:40527 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s')

2025-11-29 14:35:03,625 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s')

2025-11-29 14:35:03,623 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s')

2025-11-29 14:35:03,621 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s')

2025-11-29 14:35:03,620 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s')

2025-11-29 14:35:03,612 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:36229 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s')

2025-11-29 14:35:03,611 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41965 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s')

2025-11-29 14:35:03,599 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:34767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44823 after 30 s')

2025-11-29 14:34:34,440 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:42943

2025-11-29 14:34:34,437 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:42943', status: closed, memory: 0, processing: 0>

2025-11-29 14:34:33,485 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:44823', status: closed, memory: 0, processing: 0>

2025-11-29 14:34:19,776 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:43779 after 30 s')

2025-11-29 14:34:19,771 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:43779 failed: OSError: Timed out trying to connect to tcp://172.21.12.197:43779 after 30 s

2025-11-29 14:34:19,737 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:36229 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:43779 after 30 s')

2025-11-29 14:34:19,731 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:43779 after 30 s')

2025-11-29 14:34:19,645 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:43779 after 30 s')

2025-11-29 14:34:19,645 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:40527 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:43779 after 30 s')

2025-11-29 14:34:19,645 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:43779 after 30 s')

2025-11-29 14:34:19,644 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41965 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:43779 after 30 s')

2025-11-29 14:34:19,603 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:34767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:43779 after 30 s')

2025-11-29 14:33:50,460 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:44823

2025-11-29 14:33:50,458 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:44823', status: closed, memory: 0, processing: 0>

2025-11-29 14:33:49,485 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:43779', status: closed, memory: 0, processing: 0>

2025-11-29 14:33:35,601 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:40087 failed: OSError: Timed out trying to connect to tcp://172.21.12.197:40087 after 30 s

2025-11-29 14:33:35,513 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:40087 after 30 s')

2025-11-29 14:33:35,512 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:40087 after 30 s')

2025-11-29 14:33:35,501 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:40087 after 30 s')

2025-11-29 14:33:35,499 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:40087 after 30 s')

2025-11-29 14:33:35,496 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:40087 after 30 s')

2025-11-29 14:33:35,488 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:40087 after 30 s')

2025-11-29 14:33:35,485 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41965 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:40087 after 30 s')

2025-11-29 14:33:35,471 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:40527 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:40087 after 30 s')

2025-11-29 14:33:06,369 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:43779

2025-11-29 14:33:06,366 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:43779', status: closed, memory: 0, processing: 0>

2025-11-29 14:33:05,372 - distributed.scheduler - INFO - Close client connection: Client-worker-51dc242b-ccf5-11f0-858f-8a1d5a425dc1

2025-11-29 14:33:05,354 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:40087', status: closed, memory: 0, processing: 0>

2025-11-29 14:33:05,353 - distributed.scheduler - INFO - Remove client Client-worker-51dc242b-ccf5-11f0-858f-8a1d5a425dc1

2025-11-29 14:32:50,561 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41593 failed: OSError: Timed out trying to connect to tcp://172.21.159.252:41593 after 30 s

2025-11-29 14:32:50,540 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:41593 after 30 s')

2025-11-29 14:32:50,504 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:41593 after 30 s')

2025-11-29 14:32:50,428 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:40087 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:41593 after 30 s')

2025-11-29 14:32:50,411 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:41593 after 30 s')

2025-11-29 14:32:21,266 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:41957

2025-11-29 14:32:21,265 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:41957', status: closed, memory: 0, processing: 0>

2025-11-29 14:32:20,309 - distributed.scheduler - INFO - Close client connection: Client-worker-72151011-cae9-11f0-8592-8253c573df33

2025-11-29 14:32:20,296 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:41593', status: closed, memory: 0, processing: 0>

2025-11-29 14:32:20,295 - distributed.scheduler - INFO - Remove client Client-worker-72151011-cae9-11f0-8592-8253c573df33

2025-11-29 14:31:56,761 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:40579 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:40579 after 30 s

2025-11-29 14:31:56,739 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40579 after 30 s')

2025-11-29 14:31:56,653 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40579 after 30 s')

2025-11-29 14:31:56,653 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40579 after 30 s')

2025-11-29 14:31:56,625 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:40087 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40579 after 30 s')

2025-11-29 14:31:56,608 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40579 after 30 s')

2025-11-29 14:31:56,607 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40579 after 30 s')

2025-11-29 14:31:56,581 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41593 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40579 after 30 s')

2025-11-29 14:31:56,577 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40579 after 30 s')

2025-11-29 14:31:56,576 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:43399 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40579 after 30 s')

2025-11-29 14:31:56,567 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:34767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:40579 after 30 s')

2025-11-29 14:31:56,143 - distributed.scheduler - INFO - Close client connection: Client-fc8980b9-cd2f-11f0-8001-f2cd832f225c

2025-11-29 14:31:56,143 - distributed.scheduler - INFO - Remove client Client-fc8980b9-cd2f-11f0-8001-f2cd832f225c

2025-11-29 14:31:56,141 - distributed.scheduler - INFO - Remove client Client-fc8980b9-cd2f-11f0-8001-f2cd832f225c

2025-11-29 14:31:27,531 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:36229

2025-11-29 14:31:27,530 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:36229', status: closed, memory: 0, processing: 0>

2025-11-29 14:31:26,492 - distributed.scheduler - INFO - Close client connection: Client-worker-28ab5f9d-ca8d-11f0-84af-f2a719d69604

2025-11-29 14:31:26,479 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:40579', status: closed, memory: 0, processing: 0>

2025-11-29 14:31:26,476 - distributed.scheduler - INFO - Remove client Client-worker-28ab5f9d-ca8d-11f0-84af-f2a719d69604

2025-11-29 14:30:43,283 - distributed.scheduler - INFO - Receive client connection: Client-fc8980b9-cd2f-11f0-8001-f2cd832f225c

2025-11-29 14:21:20,473 - distributed.scheduler - INFO - Close client connection: Client-9605dc7a-cd2e-11f0-8001-f2cd832f225c

2025-11-29 14:21:20,472 - distributed.scheduler - INFO - Remove client Client-9605dc7a-cd2e-11f0-8001-f2cd832f225c

2025-11-29 14:21:20,470 - distributed.scheduler - INFO - Remove client Client-9605dc7a-cd2e-11f0-8001-f2cd832f225c

2025-11-29 14:20:41,795 - distributed.scheduler - INFO - Receive client connection: Client-9605dc7a-cd2e-11f0-8001-f2cd832f225c

2025-11-29 14:11:18,243 - distributed.scheduler - INFO - Close client connection: Client-2f7e83f2-cd2d-11f0-8001-f2cd832f225c

2025-11-29 14:11:18,241 - distributed.scheduler - INFO - Remove client Client-2f7e83f2-cd2d-11f0-8001-f2cd832f225c

2025-11-29 14:11:18,239 - distributed.scheduler - INFO - Remove client Client-2f7e83f2-cd2d-11f0-8001-f2cd832f225c

2025-11-29 14:10:40,281 - distributed.scheduler - INFO - Receive client connection: Client-2f7e83f2-cd2d-11f0-8001-f2cd832f225c

2025-11-29 14:01:49,830 - distributed.scheduler - INFO - Close client connection: Client-c8dde1e2-cd2b-11f0-8001-f2cd832f225c

2025-11-29 14:01:49,829 - distributed.scheduler - INFO - Remove client Client-c8dde1e2-cd2b-11f0-8001-f2cd832f225c

2025-11-29 14:01:49,827 - distributed.scheduler - INFO - Remove client Client-c8dde1e2-cd2b-11f0-8001-f2cd832f225c

2025-11-29 14:00:38,606 - distributed.scheduler - INFO - Receive client connection: Client-c8dde1e2-cd2b-11f0-8001-f2cd832f225c

2025-11-29 13:51:18,369 - distributed.scheduler - INFO - Close client connection: Client-62591f27-cd2a-11f0-8001-f2cd832f225c

2025-11-29 13:51:18,368 - distributed.scheduler - INFO - Remove client Client-62591f27-cd2a-11f0-8001-f2cd832f225c

2025-11-29 13:51:18,367 - distributed.scheduler - INFO - Remove client Client-62591f27-cd2a-11f0-8001-f2cd832f225c

2025-11-29 13:50:37,113 - distributed.scheduler - INFO - Receive client connection: Client-62591f27-cd2a-11f0-8001-f2cd832f225c

2025-11-29 13:41:12,185 - distributed.scheduler - INFO - Close client connection: Client-fbdac691-cd28-11f0-8001-f2cd832f225c

2025-11-29 13:41:12,183 - distributed.scheduler - INFO - Remove client Client-fbdac691-cd28-11f0-8001-f2cd832f225c

2025-11-29 13:41:12,182 - distributed.scheduler - INFO - Remove client Client-fbdac691-cd28-11f0-8001-f2cd832f225c

2025-11-29 13:40:35,657 - distributed.scheduler - INFO - Receive client connection: Client-fbdac691-cd28-11f0-8001-f2cd832f225c

2025-11-29 13:31:47,807 - distributed.scheduler - INFO - Close client connection: Client-9558f85d-cd27-11f0-8001-f2cd832f225c

2025-11-29 13:31:47,806 - distributed.scheduler - INFO - Remove client Client-9558f85d-cd27-11f0-8001-f2cd832f225c

2025-11-29 13:31:47,804 - distributed.scheduler - INFO - Remove client Client-9558f85d-cd27-11f0-8001-f2cd832f225c

2025-11-29 13:30:34,182 - distributed.scheduler - INFO - Receive client connection: Client-9558f85d-cd27-11f0-8001-f2cd832f225c

2025-11-29 13:21:10,302 - distributed.scheduler - INFO - Close client connection: Client-2ededd0a-cd26-11f0-8001-f2cd832f225c

2025-11-29 13:21:10,302 - distributed.scheduler - INFO - Remove client Client-2ededd0a-cd26-11f0-8001-f2cd832f225c

2025-11-29 13:21:10,301 - distributed.scheduler - INFO - Remove client Client-2ededd0a-cd26-11f0-8001-f2cd832f225c

2025-11-29 13:20:32,761 - distributed.scheduler - INFO - Receive client connection: Client-2ededd0a-cd26-11f0-8001-f2cd832f225c

2025-11-29 13:11:09,069 - distributed.scheduler - INFO - Close client connection: Client-c865165d-cd24-11f0-8001-f2cd832f225c

2025-11-29 13:11:09,068 - distributed.scheduler - INFO - Remove client Client-c865165d-cd24-11f0-8001-f2cd832f225c

2025-11-29 13:11:09,066 - distributed.scheduler - INFO - Remove client Client-c865165d-cd24-11f0-8001-f2cd832f225c

2025-11-29 13:10:31,334 - distributed.scheduler - INFO - Receive client connection: Client-c865165d-cd24-11f0-8001-f2cd832f225c

2025-11-29 13:01:39,456 - distributed.scheduler - INFO - Close client connection: Client-61c7bbb2-cd23-11f0-8001-f2cd832f225c

2025-11-29 13:01:39,456 - distributed.scheduler - INFO - Remove client Client-61c7bbb2-cd23-11f0-8001-f2cd832f225c

2025-11-29 13:01:39,454 - distributed.scheduler - INFO - Remove client Client-61c7bbb2-cd23-11f0-8001-f2cd832f225c

2025-11-29 13:00:29,680 - distributed.scheduler - INFO - Receive client connection: Client-61c7bbb2-cd23-11f0-8001-f2cd832f225c

2025-11-29 12:51:07,899 - distributed.scheduler - INFO - Close client connection: Client-fb476c1d-cd21-11f0-8001-f2cd832f225c

2025-11-29 12:51:07,898 - distributed.scheduler - INFO - Remove client Client-fb476c1d-cd21-11f0-8001-f2cd832f225c

2025-11-29 12:51:07,897 - distributed.scheduler - INFO - Remove client Client-fb476c1d-cd21-11f0-8001-f2cd832f225c

2025-11-29 12:50:28,216 - distributed.scheduler - INFO - Receive client connection: Client-fb476c1d-cd21-11f0-8001-f2cd832f225c

2025-11-29 12:41:05,668 - distributed.scheduler - INFO - Close client connection: Client-94d9c110-cd20-11f0-8001-f2cd832f225c

2025-11-29 12:41:05,666 - distributed.scheduler - INFO - Remove client Client-94d9c110-cd20-11f0-8001-f2cd832f225c

2025-11-29 12:41:05,665 - distributed.scheduler - INFO - Remove client Client-94d9c110-cd20-11f0-8001-f2cd832f225c

2025-11-29 12:40:26,873 - distributed.scheduler - INFO - Receive client connection: Client-94d9c110-cd20-11f0-8001-f2cd832f225c

2025-11-29 12:31:34,384 - distributed.scheduler - INFO - Close client connection: Client-2e6a1de1-cd1f-11f0-8001-f2cd832f225c

2025-11-29 12:31:34,382 - distributed.scheduler - INFO - Remove client Client-2e6a1de1-cd1f-11f0-8001-f2cd832f225c

2025-11-29 12:31:34,380 - distributed.scheduler - INFO - Remove client Client-2e6a1de1-cd1f-11f0-8001-f2cd832f225c

2025-11-29 12:30:25,517 - distributed.scheduler - INFO - Receive client connection: Client-2e6a1de1-cd1f-11f0-8001-f2cd832f225c

2025-11-29 12:21:02,052 - distributed.scheduler - INFO - Close client connection: Client-c7e8d4f1-cd1d-11f0-8001-f2cd832f225c

2025-11-29 12:21:02,049 - distributed.scheduler - INFO - Remove client Client-c7e8d4f1-cd1d-11f0-8001-f2cd832f225c

2025-11-29 12:21:02,048 - distributed.scheduler - INFO - Remove client Client-c7e8d4f1-cd1d-11f0-8001-f2cd832f225c

2025-11-29 12:20:24,042 - distributed.scheduler - INFO - Receive client connection: Client-c7e8d4f1-cd1d-11f0-8001-f2cd832f225c

2025-11-29 12:11:00,225 - distributed.scheduler - INFO - Close client connection: Client-616e4ac4-cd1c-11f0-8001-f2cd832f225c

2025-11-29 12:11:00,223 - distributed.scheduler - INFO - Remove client Client-616e4ac4-cd1c-11f0-8001-f2cd832f225c

2025-11-29 12:11:00,221 - distributed.scheduler - INFO - Remove client Client-616e4ac4-cd1c-11f0-8001-f2cd832f225c

2025-11-29 12:10:22,613 - distributed.scheduler - INFO - Receive client connection: Client-616e4ac4-cd1c-11f0-8001-f2cd832f225c

2025-11-29 12:01:32,970 - distributed.scheduler - INFO - Close client connection: Client-fad76386-cd1a-11f0-8001-f2cd832f225c

2025-11-29 12:01:32,968 - distributed.scheduler - INFO - Remove client Client-fad76386-cd1a-11f0-8001-f2cd832f225c

2025-11-29 12:01:32,966 - distributed.scheduler - INFO - Remove client Client-fad76386-cd1a-11f0-8001-f2cd832f225c

2025-11-29 12:00:20,998 - distributed.scheduler - INFO - Receive client connection: Client-fad76386-cd1a-11f0-8001-f2cd832f225c

2025-11-29 11:50:58,814 - distributed.scheduler - INFO - Close client connection: Client-945861d7-cd19-11f0-8001-f2cd832f225c

2025-11-29 11:50:58,812 - distributed.scheduler - INFO - Remove client Client-945861d7-cd19-11f0-8001-f2cd832f225c

2025-11-29 11:50:58,811 - distributed.scheduler - INFO - Remove client Client-945861d7-cd19-11f0-8001-f2cd832f225c

2025-11-29 11:50:19,545 - distributed.scheduler - INFO - Receive client connection: Client-945861d7-cd19-11f0-8001-f2cd832f225c

2025-11-29 11:40:58,845 - distributed.scheduler - INFO - Close client connection: Client-2ddc4bcc-cd18-11f0-8001-f2cd832f225c

2025-11-29 11:40:58,845 - distributed.scheduler - INFO - Remove client Client-2ddc4bcc-cd18-11f0-8001-f2cd832f225c

2025-11-29 11:40:58,843 - distributed.scheduler - INFO - Remove client Client-2ddc4bcc-cd18-11f0-8001-f2cd832f225c

2025-11-29 11:40:18,113 - distributed.scheduler - INFO - Receive client connection: Client-2ddc4bcc-cd18-11f0-8001-f2cd832f225c

2025-11-29 11:31:25,864 - distributed.scheduler - INFO - Close client connection: Client-c76b276c-cd16-11f0-8001-f2cd832f225c

2025-11-29 11:31:25,864 - distributed.scheduler - INFO - Remove client Client-c76b276c-cd16-11f0-8001-f2cd832f225c

2025-11-29 11:31:25,862 - distributed.scheduler - INFO - Remove client Client-c76b276c-cd16-11f0-8001-f2cd832f225c

2025-11-29 11:30:16,746 - distributed.scheduler - INFO - Receive client connection: Client-c76b276c-cd16-11f0-8001-f2cd832f225c

2025-11-29 11:20:52,601 - distributed.scheduler - INFO - Close client connection: Client-60eb2ab0-cd15-11f0-8001-f2cd832f225c

2025-11-29 11:20:52,600 - distributed.scheduler - INFO - Remove client Client-60eb2ab0-cd15-11f0-8001-f2cd832f225c

2025-11-29 11:20:52,598 - distributed.scheduler - INFO - Remove client Client-60eb2ab0-cd15-11f0-8001-f2cd832f225c

2025-11-29 11:20:15,279 - distributed.scheduler - INFO - Receive client connection: Client-60eb2ab0-cd15-11f0-8001-f2cd832f225c

2025-11-29 11:10:49,150 - distributed.scheduler - INFO - Close client connection: Client-fa6d2d90-cd13-11f0-8001-f2cd832f225c

2025-11-29 11:10:49,149 - distributed.scheduler - INFO - Remove client Client-fa6d2d90-cd13-11f0-8001-f2cd832f225c

2025-11-29 11:10:49,147 - distributed.scheduler - INFO - Remove client Client-fa6d2d90-cd13-11f0-8001-f2cd832f225c

2025-11-29 11:10:13,833 - distributed.scheduler - INFO - Receive client connection: Client-fa6d2d90-cd13-11f0-8001-f2cd832f225c

2025-11-29 11:01:23,719 - distributed.scheduler - INFO - Close client connection: Client-93ddb45f-cd12-11f0-8001-f2cd832f225c

2025-11-29 11:01:23,716 - distributed.scheduler - INFO - Remove client Client-93ddb45f-cd12-11f0-8001-f2cd832f225c

2025-11-29 11:01:23,714 - distributed.scheduler - INFO - Remove client Client-93ddb45f-cd12-11f0-8001-f2cd832f225c

2025-11-29 11:00:12,257 - distributed.scheduler - INFO - Receive client connection: Client-93ddb45f-cd12-11f0-8001-f2cd832f225c

2025-11-29 10:50:50,963 - distributed.scheduler - INFO - Close client connection: Client-2d5c7cd8-cd11-11f0-8001-f2cd832f225c

2025-11-29 10:50:50,959 - distributed.scheduler - INFO - Remove client Client-2d5c7cd8-cd11-11f0-8001-f2cd832f225c

2025-11-29 10:50:50,957 - distributed.scheduler - INFO - Remove client Client-2d5c7cd8-cd11-11f0-8001-f2cd832f225c

2025-11-29 10:50:10,796 - distributed.scheduler - INFO - Receive client connection: Client-2d5c7cd8-cd11-11f0-8001-f2cd832f225c

2025-11-29 10:40:46,818 - distributed.scheduler - INFO - Close client connection: Client-c6df1628-cd0f-11f0-8001-f2cd832f225c

2025-11-29 10:40:46,817 - distributed.scheduler - INFO - Remove client Client-c6df1628-cd0f-11f0-8001-f2cd832f225c

2025-11-29 10:40:46,815 - distributed.scheduler - INFO - Remove client Client-c6df1628-cd0f-11f0-8001-f2cd832f225c

2025-11-29 10:40:09,349 - distributed.scheduler - INFO - Receive client connection: Client-c6df1628-cd0f-11f0-8001-f2cd832f225c

2025-11-29 10:31:22,907 - distributed.scheduler - INFO - Close client connection: Client-605dbed2-cd0e-11f0-8001-f2cd832f225c

2025-11-29 10:31:22,906 - distributed.scheduler - INFO - Remove client Client-605dbed2-cd0e-11f0-8001-f2cd832f225c

2025-11-29 10:31:22,904 - distributed.scheduler - INFO - Remove client Client-605dbed2-cd0e-11f0-8001-f2cd832f225c

2025-11-29 10:30:07,916 - distributed.scheduler - INFO - Receive client connection: Client-605dbed2-cd0e-11f0-8001-f2cd832f225c

2025-11-29 10:20:46,834 - distributed.scheduler - INFO - Close client connection: Client-f9e62578-cd0c-11f0-8001-f2cd832f225c

2025-11-29 10:20:46,833 - distributed.scheduler - INFO - Remove client Client-f9e62578-cd0c-11f0-8001-f2cd832f225c

2025-11-29 10:20:46,831 - distributed.scheduler - INFO - Remove client Client-f9e62578-cd0c-11f0-8001-f2cd832f225c

2025-11-29 10:20:06,469 - distributed.scheduler - INFO - Receive client connection: Client-f9e62578-cd0c-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,023 - distributed.scheduler - INFO - Close client connection: Client-1c4e7d6e-ccf8-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,022 - distributed.scheduler - INFO - Close client connection: Client-82cb0269-ccf9-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,022 - distributed.scheduler - INFO - Close client connection: Client-e95d9fd2-ccfa-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,022 - distributed.scheduler - INFO - Close client connection: Client-4fd76b85-ccfc-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,022 - distributed.scheduler - INFO - Close client connection: Client-b651e4b2-ccfd-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,022 - distributed.scheduler - INFO - Close client connection: Client-1cd72809-ccff-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,021 - distributed.scheduler - INFO - Close client connection: Client-835556fc-cd00-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,021 - distributed.scheduler - INFO - Close client connection: Client-e9cd0ba4-cd01-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,021 - distributed.scheduler - INFO - Close client connection: Client-5055e45b-cd03-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,021 - distributed.scheduler - INFO - Close client connection: Client-b6c52825-cd04-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,020 - distributed.scheduler - INFO - Close client connection: Client-1d48fd94-cd06-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,020 - distributed.scheduler - INFO - Close client connection: Client-5fee463b-cd07-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,020 - distributed.scheduler - INFO - Close client connection: Client-c6659f54-cd08-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,020 - distributed.scheduler - INFO - Close client connection: Client-2cea3d95-cd0a-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,017 - distributed.scheduler - INFO - Remove client Client-1c4e7d6e-ccf8-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,016 - distributed.scheduler - INFO - Remove client Client-82cb0269-ccf9-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,015 - distributed.scheduler - INFO - Remove client Client-e95d9fd2-ccfa-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,014 - distributed.scheduler - INFO - Remove client Client-4fd76b85-ccfc-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,013 - distributed.scheduler - INFO - Remove client Client-b651e4b2-ccfd-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,013 - distributed.scheduler - INFO - Remove client Client-1cd72809-ccff-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,009 - distributed.scheduler - INFO - Remove client Client-835556fc-cd00-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,009 - distributed.scheduler - INFO - Remove client Client-e9cd0ba4-cd01-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,008 - distributed.scheduler - INFO - Remove client Client-5055e45b-cd03-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,007 - distributed.scheduler - INFO - Remove client Client-b6c52825-cd04-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,006 - distributed.scheduler - INFO - Remove client Client-1d48fd94-cd06-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,006 - distributed.scheduler - INFO - Remove client Client-5fee463b-cd07-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,005 - distributed.scheduler - INFO - Remove client Client-c6659f54-cd08-11f0-8001-f2cd832f225c

2025-11-29 10:10:44,004 - distributed.scheduler - INFO - Remove client Client-2cea3d95-cd0a-11f0-8001-f2cd832f225c

2025-11-29 10:10:43,192 - distributed.scheduler - INFO - Close client connection: Client-936d6358-cd0b-11f0-8001-f2cd832f225c

2025-11-29 10:10:43,191 - distributed.scheduler - INFO - Remove client Client-936d6358-cd0b-11f0-8001-f2cd832f225c

2025-11-29 10:10:43,189 - distributed.scheduler - INFO - Remove client Client-936d6358-cd0b-11f0-8001-f2cd832f225c

2025-11-29 10:10:05,043 - distributed.scheduler - INFO - Receive client connection: Client-936d6358-cd0b-11f0-8001-f2cd832f225c

2025-11-29 10:00:03,558 - distributed.scheduler - INFO - Receive client connection: Client-2cea3d95-cd0a-11f0-8001-f2cd832f225c

2025-11-29 09:50:02,067 - distributed.scheduler - INFO - Receive client connection: Client-c6659f54-cd08-11f0-8001-f2cd832f225c

2025-11-29 09:40:00,663 - distributed.scheduler - INFO - Receive client connection: Client-5fee463b-cd07-11f0-8001-f2cd832f225c

2025-11-29 09:30:59,351 - distributed.scheduler - INFO - Receive client connection: Client-1d48fd94-cd06-11f0-8001-f2cd832f225c

2025-11-29 09:20:57,866 - distributed.scheduler - INFO - Receive client connection: Client-b6c52825-cd04-11f0-8001-f2cd832f225c

2025-11-29 09:10:56,507 - distributed.scheduler - INFO - Receive client connection: Client-5055e45b-cd03-11f0-8001-f2cd832f225c

2025-11-29 09:00:54,984 - distributed.scheduler - INFO - Receive client connection: Client-e9cd0ba4-cd01-11f0-8001-f2cd832f225c

2025-11-29 08:50:53,579 - distributed.scheduler - INFO - Receive client connection: Client-835556fc-cd00-11f0-8001-f2cd832f225c

2025-11-29 08:40:52,132 - distributed.scheduler - INFO - Receive client connection: Client-1cd72809-ccff-11f0-8001-f2cd832f225c

2025-11-29 08:30:50,646 - distributed.scheduler - INFO - Receive client connection: Client-b651e4b2-ccfd-11f0-8001-f2cd832f225c

2025-11-29 08:20:49,214 - distributed.scheduler - INFO - Receive client connection: Client-4fd76b85-ccfc-11f0-8001-f2cd832f225c

2025-11-29 08:10:47,779 - distributed.scheduler - INFO - Receive client connection: Client-e95d9fd2-ccfa-11f0-8001-f2cd832f225c

2025-11-29 08:00:46,204 - distributed.scheduler - INFO - Receive client connection: Client-82cb0269-ccf9-11f0-8001-f2cd832f225c

2025-11-29 07:50:44,762 - distributed.scheduler - INFO - Receive client connection: Client-1c4e7d6e-ccf8-11f0-8001-f2cd832f225c

2025-11-29 07:41:21,096 - distributed.scheduler - INFO - Close client connection: Client-b5cea8e2-ccf6-11f0-8001-f2cd832f225c

2025-11-29 07:41:21,095 - distributed.scheduler - INFO - Remove client Client-b5cea8e2-ccf6-11f0-8001-f2cd832f225c

2025-11-29 07:41:21,092 - distributed.scheduler - INFO - Remove client Client-b5cea8e2-ccf6-11f0-8001-f2cd832f225c

2025-11-29 07:40:43,295 - distributed.scheduler - INFO - Receive client connection: Client-b5cea8e2-ccf6-11f0-8001-f2cd832f225c

2025-11-29 07:31:48,365 - distributed.scheduler - INFO - Close client connection: Client-4f5ae96e-ccf5-11f0-8001-f2cd832f225c

2025-11-29 07:31:48,362 - distributed.scheduler - INFO - Remove client Client-4f5ae96e-ccf5-11f0-8001-f2cd832f225c

2025-11-29 07:31:48,360 - distributed.scheduler - INFO - Remove client Client-4f5ae96e-ccf5-11f0-8001-f2cd832f225c

2025-11-29 07:30:46,113 - distributed.scheduler - INFO - Receive client connection: Client-worker-51dc242b-ccf5-11f0-858f-8a1d5a425dc1

2025-11-29 07:30:41,920 - distributed.scheduler - INFO - Receive client connection: Client-4f5ae96e-ccf5-11f0-8001-f2cd832f225c

2025-11-29 07:21:17,618 - distributed.scheduler - INFO - Close client connection: Client-e8e2dd14-ccf3-11f0-8001-f2cd832f225c

2025-11-29 07:21:17,617 - distributed.scheduler - INFO - Remove client Client-e8e2dd14-ccf3-11f0-8001-f2cd832f225c

2025-11-29 07:21:17,616 - distributed.scheduler - INFO - Remove client Client-e8e2dd14-ccf3-11f0-8001-f2cd832f225c

2025-11-29 07:20:40,508 - distributed.scheduler - INFO - Receive client connection: Client-e8e2dd14-ccf3-11f0-8001-f2cd832f225c

2025-11-29 07:11:16,448 - distributed.scheduler - INFO - Close client connection: Client-8263a678-ccf2-11f0-8001-f2cd832f225c

2025-11-29 07:11:16,447 - distributed.scheduler - INFO - Remove client Client-8263a678-ccf2-11f0-8001-f2cd832f225c

2025-11-29 07:11:16,445 - distributed.scheduler - INFO - Remove client Client-8263a678-ccf2-11f0-8001-f2cd832f225c

2025-11-29 07:10:39,039 - distributed.scheduler - INFO - Receive client connection: Client-8263a678-ccf2-11f0-8001-f2cd832f225c

2025-11-29 07:01:47,888 - distributed.scheduler - INFO - Close client connection: Client-1bd2f041-ccf1-11f0-8001-f2cd832f225c

2025-11-29 07:01:47,887 - distributed.scheduler - INFO - Remove client Client-1bd2f041-ccf1-11f0-8001-f2cd832f225c

2025-11-29 07:01:47,885 - distributed.scheduler - INFO - Remove client Client-1bd2f041-ccf1-11f0-8001-f2cd832f225c

2025-11-29 07:00:37,475 - distributed.scheduler - INFO - Receive client connection: Client-1bd2f041-ccf1-11f0-8001-f2cd832f225c

2025-11-29 06:51:16,260 - distributed.scheduler - INFO - Close client connection: Client-b55ca594-ccef-11f0-8001-f2cd832f225c

2025-11-29 06:51:16,254 - distributed.scheduler - INFO - Remove client Client-b55ca594-ccef-11f0-8001-f2cd832f225c

2025-11-29 06:51:16,252 - distributed.scheduler - INFO - Remove client Client-b55ca594-ccef-11f0-8001-f2cd832f225c

2025-11-29 06:50:36,073 - distributed.scheduler - INFO - Receive client connection: Client-b55ca594-ccef-11f0-8001-f2cd832f225c

2025-11-29 06:41:12,845 - distributed.scheduler - INFO - Close client connection: Client-4edcb92e-ccee-11f0-8001-f2cd832f225c

2025-11-29 06:41:12,844 - distributed.scheduler - INFO - Remove client Client-4edcb92e-ccee-11f0-8001-f2cd832f225c

2025-11-29 06:41:12,842 - distributed.scheduler - INFO - Remove client Client-4edcb92e-ccee-11f0-8001-f2cd832f225c

2025-11-29 06:40:34,616 - distributed.scheduler - INFO - Receive client connection: Client-4edcb92e-ccee-11f0-8001-f2cd832f225c

2025-11-29 06:31:44,029 - distributed.scheduler - INFO - Close client connection: Client-e85f6eee-ccec-11f0-8001-f2cd832f225c

2025-11-29 06:31:44,027 - distributed.scheduler - INFO - Remove client Client-e85f6eee-ccec-11f0-8001-f2cd832f225c

2025-11-29 06:31:44,023 - distributed.scheduler - INFO - Remove client Client-e85f6eee-ccec-11f0-8001-f2cd832f225c

2025-11-29 06:30:33,178 - distributed.scheduler - INFO - Receive client connection: Client-e85f6eee-ccec-11f0-8001-f2cd832f225c

2025-11-29 06:21:08,606 - distributed.scheduler - INFO - Close client connection: Client-81de2079-cceb-11f0-8001-f2cd832f225c

2025-11-29 06:21:08,604 - distributed.scheduler - INFO - Remove client Client-81de2079-cceb-11f0-8001-f2cd832f225c

2025-11-29 06:21:08,603 - distributed.scheduler - INFO - Remove client Client-81de2079-cceb-11f0-8001-f2cd832f225c

2025-11-29 06:20:31,698 - distributed.scheduler - INFO - Receive client connection: Client-81de2079-cceb-11f0-8001-f2cd832f225c

2025-11-29 06:11:08,991 - distributed.scheduler - INFO - Close client connection: Client-1b5e5b51-ccea-11f0-8001-f2cd832f225c

2025-11-29 06:11:08,989 - distributed.scheduler - INFO - Remove client Client-1b5e5b51-ccea-11f0-8001-f2cd832f225c

2025-11-29 06:11:08,987 - distributed.scheduler - INFO - Remove client Client-1b5e5b51-ccea-11f0-8001-f2cd832f225c

2025-11-29 06:10:30,234 - distributed.scheduler - INFO - Receive client connection: Client-1b5e5b51-ccea-11f0-8001-f2cd832f225c

2025-11-29 06:03:37,890 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:40087

2025-11-29 06:03:37,888 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:40087', status: closed, memory: 0, processing: 0>

2025-11-29 06:03:36,868 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:38387', status: closed, memory: 0, processing: 0>

2025-11-29 06:03:27,546 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:38387

2025-11-29 06:03:27,543 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:38387', status: closed, memory: 0, processing: 0>

2025-11-29 06:03:26,562 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:37319', status: closed, memory: 0, processing: 0>

2025-11-29 06:03:16,343 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:38739 failed: OSError: Timed out trying to connect to tcp://172.21.12.197:38739 after 30 s

2025-11-29 06:03:16,259 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:38739 after 30 s')

2025-11-29 06:03:16,240 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:38739 after 30 s')

2025-11-29 06:03:16,229 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:40527 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:38739 after 30 s')

2025-11-29 06:03:16,228 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:43399 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:38739 after 30 s')

2025-11-29 06:03:16,225 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:38739 after 30 s')

2025-11-29 06:03:16,195 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41965 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:38739 after 30 s')

2025-11-29 06:03:16,180 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:38739 after 30 s')

2025-11-29 06:03:16,179 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:34767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:38739 after 30 s')

2025-11-29 06:03:16,177 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:38739 after 30 s')

2025-11-29 06:03:16,165 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:40579 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:38739 after 30 s')

2025-11-29 06:02:47,097 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:37319

2025-11-29 06:02:47,094 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:37319', status: closed, memory: 0, processing: 0>

2025-11-29 06:02:46,078 - distributed.scheduler - INFO - Close client connection: Client-worker-f9a7076c-cce8-11f0-855a-8a1d5a425dc1

2025-11-29 06:02:46,059 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:38739', status: closed, memory: 0, processing: 0>

2025-11-29 06:02:46,058 - distributed.scheduler - INFO - Remove client Client-worker-f9a7076c-cce8-11f0-855a-8a1d5a425dc1

2025-11-29 06:02:24,156 - distributed.scheduler - INFO - Receive client connection: Client-worker-f9a7076c-cce8-11f0-855a-8a1d5a425dc1

2025-11-29 06:02:20,766 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:44247 failed: OSError: Timed out trying to connect to tcp://172.21.12.197:44247 after 30 s

2025-11-29 06:02:20,765 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44247 after 30 s')

2025-11-29 06:02:20,730 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44247 after 30 s')

2025-11-29 06:02:20,623 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:43399 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44247 after 30 s')

2025-11-29 06:02:20,622 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44247 after 30 s')

2025-11-29 06:02:20,622 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:34767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44247 after 30 s')

2025-11-29 06:02:20,611 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44247 after 30 s')

2025-11-29 06:02:20,559 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:44247 after 30 s')

2025-11-29 06:01:51,450 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:38739

2025-11-29 06:01:51,447 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:38739', status: closed, memory: 0, processing: 0>

2025-11-29 06:01:50,459 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:44247', status: closed, memory: 0, processing: 0>

2025-11-29 06:01:39,371 - distributed.scheduler - INFO - Close client connection: Client-4e741a29-cce7-11f0-8001-f2cd832f225c

2025-11-29 06:01:39,369 - distributed.scheduler - INFO - Remove client Client-4e741a29-cce7-11f0-8001-f2cd832f225c

2025-11-29 06:01:38,490 - distributed.scheduler - INFO - Close client connection: Client-b4e49119-cce8-11f0-8001-f2cd832f225c

2025-11-29 06:01:38,489 - distributed.scheduler - INFO - Remove client Client-b4e49119-cce8-11f0-8001-f2cd832f225c

2025-11-29 06:01:38,488 - distributed.scheduler - INFO - Remove client Client-b4e49119-cce8-11f0-8001-f2cd832f225c

2025-11-29 06:01:36,409 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:35449 after 30 s')

2025-11-29 06:01:36,362 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:35449 failed: OSError: Timed out trying to connect to tcp://172.21.12.197:35449 after 30 s

2025-11-29 06:01:36,355 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:35449 after 30 s')

2025-11-29 06:01:36,278 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:35449 after 30 s')

2025-11-29 06:01:36,271 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41593 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.197:35449 after 30 s')

2025-11-29 06:01:06,975 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:44247

2025-11-29 06:01:06,972 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:44247', status: closed, memory: 0, processing: 0>

2025-11-29 06:01:06,107 - distributed.scheduler - INFO - Close client connection: Client-worker-a4b3a3a4-caed-11f0-8484-8a1d5a425dc1

2025-11-29 06:01:06,089 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:35449', status: closed, memory: 0, processing: 0>

2025-11-29 06:01:06,088 - distributed.scheduler - INFO - Remove client Client-worker-a4b3a3a4-caed-11f0-8484-8a1d5a425dc1

2025-11-29 06:00:28,804 - distributed.scheduler - INFO - Receive client connection: Client-b4e49119-cce8-11f0-8001-f2cd832f225c

2025-11-29 05:50:27,451 - distributed.scheduler - INFO - Receive client connection: Client-4e741a29-cce7-11f0-8001-f2cd832f225c

2025-11-29 05:40:59,506 - distributed.scheduler - INFO - Close client connection: Client-e80196ea-cce5-11f0-8001-f2cd832f225c

2025-11-29 05:40:59,505 - distributed.scheduler - INFO - Remove client Client-e80196ea-cce5-11f0-8001-f2cd832f225c

2025-11-29 05:40:59,503 - distributed.scheduler - INFO - Remove client Client-e80196ea-cce5-11f0-8001-f2cd832f225c

2025-11-29 05:40:26,079 - distributed.scheduler - INFO - Receive client connection: Client-e80196ea-cce5-11f0-8001-f2cd832f225c

2025-11-29 05:31:33,068 - distributed.scheduler - INFO - Close client connection: Client-81887ad5-cce4-11f0-8001-f2cd832f225c

2025-11-29 05:31:33,067 - distributed.scheduler - INFO - Remove client Client-81887ad5-cce4-11f0-8001-f2cd832f225c

2025-11-29 05:31:33,065 - distributed.scheduler - INFO - Remove client Client-81887ad5-cce4-11f0-8001-f2cd832f225c

2025-11-29 05:30:24,693 - distributed.scheduler - INFO - Receive client connection: Client-81887ad5-cce4-11f0-8001-f2cd832f225c

2025-11-29 05:20:57,539 - distributed.scheduler - INFO - Close client connection: Client-1b0f3a1e-cce3-11f0-8001-f2cd832f225c

2025-11-29 05:20:57,538 - distributed.scheduler - INFO - Remove client Client-1b0f3a1e-cce3-11f0-8001-f2cd832f225c

2025-11-29 05:20:57,536 - distributed.scheduler - INFO - Remove client Client-1b0f3a1e-cce3-11f0-8001-f2cd832f225c

2025-11-29 05:20:23,237 - distributed.scheduler - INFO - Receive client connection: Client-1b0f3a1e-cce3-11f0-8001-f2cd832f225c

2025-11-29 05:10:56,692 - distributed.scheduler - INFO - Close client connection: Client-b48fd348-cce1-11f0-8001-f2cd832f225c

2025-11-29 05:10:56,691 - distributed.scheduler - INFO - Remove client Client-b48fd348-cce1-11f0-8001-f2cd832f225c

2025-11-29 05:10:56,689 - distributed.scheduler - INFO - Remove client Client-b48fd348-cce1-11f0-8001-f2cd832f225c

2025-11-29 05:10:21,780 - distributed.scheduler - INFO - Receive client connection: Client-b48fd348-cce1-11f0-8001-f2cd832f225c

2025-11-29 05:01:28,539 - distributed.scheduler - INFO - Close client connection: Client-4e0ca072-cce0-11f0-8001-f2cd832f225c

2025-11-29 05:01:28,538 - distributed.scheduler - INFO - Remove client Client-4e0ca072-cce0-11f0-8001-f2cd832f225c

2025-11-29 05:01:28,536 - distributed.scheduler - INFO - Remove client Client-4e0ca072-cce0-11f0-8001-f2cd832f225c

2025-11-29 05:00:20,300 - distributed.scheduler - INFO - Receive client connection: Client-4e0ca072-cce0-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,731 - distributed.scheduler - INFO - Close client connection: Client-808f5077-ccd6-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,731 - distributed.scheduler - INFO - Close client connection: Client-1a069cc1-ccd5-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,731 - distributed.scheduler - INFO - Close client connection: Client-e707b1f2-ccd7-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,731 - distributed.scheduler - INFO - Close client connection: Client-4d9974c8-ccd9-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,730 - distributed.scheduler - INFO - Close client connection: Client-b41e5482-ccda-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,730 - distributed.scheduler - INFO - Close client connection: Client-1a9cbd8d-ccdc-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,730 - distributed.scheduler - INFO - Close client connection: Client-81119f4e-ccdd-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,730 - distributed.scheduler - INFO - Remove client Client-808f5077-ccd6-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,730 - distributed.scheduler - INFO - Remove client Client-1a069cc1-ccd5-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,729 - distributed.scheduler - INFO - Remove client Client-e707b1f2-ccd7-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,729 - distributed.scheduler - INFO - Remove client Client-4d9974c8-ccd9-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,729 - distributed.scheduler - INFO - Remove client Client-b41e5482-ccda-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,728 - distributed.scheduler - INFO - Remove client Client-1a9cbd8d-ccdc-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,727 - distributed.scheduler - INFO - Remove client Client-81119f4e-ccdd-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,097 - distributed.scheduler - INFO - Close client connection: Client-e78dad4f-ccde-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,095 - distributed.scheduler - INFO - Remove client Client-e78dad4f-ccde-11f0-8001-f2cd832f225c

2025-11-29 04:50:54,093 - distributed.scheduler - INFO - Remove client Client-e78dad4f-ccde-11f0-8001-f2cd832f225c

2025-11-29 04:50:18,832 - distributed.scheduler - INFO - Receive client connection: Client-e78dad4f-ccde-11f0-8001-f2cd832f225c

2025-11-29 04:40:17,394 - distributed.scheduler - INFO - Receive client connection: Client-81119f4e-ccdd-11f0-8001-f2cd832f225c

2025-11-29 04:30:15,997 - distributed.scheduler - INFO - Receive client connection: Client-1a9cbd8d-ccdc-11f0-8001-f2cd832f225c

2025-11-29 04:20:14,544 - distributed.scheduler - INFO - Receive client connection: Client-b41e5482-ccda-11f0-8001-f2cd832f225c

2025-11-29 04:10:13,052 - distributed.scheduler - INFO - Receive client connection: Client-4d9974c8-ccd9-11f0-8001-f2cd832f225c

2025-11-29 04:00:11,477 - distributed.scheduler - INFO - Receive client connection: Client-e707b1f2-ccd7-11f0-8001-f2cd832f225c

2025-11-29 03:50:10,058 - distributed.scheduler - INFO - Receive client connection: Client-808f5077-ccd6-11f0-8001-f2cd832f225c

2025-11-29 03:40:08,554 - distributed.scheduler - INFO - Receive client connection: Client-1a069cc1-ccd5-11f0-8001-f2cd832f225c

2025-11-29 03:31:17,784 - distributed.scheduler - INFO - Close client connection: Client-b383ab65-ccd3-11f0-8001-f2cd832f225c

2025-11-29 03:31:17,783 - distributed.scheduler - INFO - Remove client Client-b383ab65-ccd3-11f0-8001-f2cd832f225c

2025-11-29 03:31:17,782 - distributed.scheduler - INFO - Remove client Client-b383ab65-ccd3-11f0-8001-f2cd832f225c

2025-11-29 03:30:07,064 - distributed.scheduler - INFO - Receive client connection: Client-b383ab65-ccd3-11f0-8001-f2cd832f225c

2025-11-29 03:20:33,365 - distributed.scheduler - INFO - Close client connection: Client-4d086f45-ccd2-11f0-8001-f2cd832f225c

2025-11-29 03:20:33,364 - distributed.scheduler - INFO - Remove client Client-4d086f45-ccd2-11f0-8001-f2cd832f225c

2025-11-29 03:20:33,363 - distributed.scheduler - INFO - Remove client Client-4d086f45-ccd2-11f0-8001-f2cd832f225c

2025-11-29 03:20:05,638 - distributed.scheduler - INFO - Receive client connection: Client-4d086f45-ccd2-11f0-8001-f2cd832f225c

2025-11-29 03:10:32,181 - distributed.scheduler - INFO - Close client connection: Client-e688be18-ccd0-11f0-8001-f2cd832f225c

2025-11-29 03:10:32,180 - distributed.scheduler - INFO - Remove client Client-e688be18-ccd0-11f0-8001-f2cd832f225c

2025-11-29 03:10:32,178 - distributed.scheduler - INFO - Remove client Client-e688be18-ccd0-11f0-8001-f2cd832f225c

2025-11-29 03:10:04,177 - distributed.scheduler - INFO - Receive client connection: Client-e688be18-ccd0-11f0-8001-f2cd832f225c

2025-11-29 03:00:57,664 - distributed.scheduler - INFO - Close client connection: Client-800be910-cccf-11f0-8001-f2cd832f225c

2025-11-29 03:00:57,662 - distributed.scheduler - INFO - Remove client Client-800be910-cccf-11f0-8001-f2cd832f225c

2025-11-29 03:00:57,661 - distributed.scheduler - INFO - Remove client Client-800be910-cccf-11f0-8001-f2cd832f225c

2025-11-29 03:00:02,734 - distributed.scheduler - INFO - Receive client connection: Client-800be910-cccf-11f0-8001-f2cd832f225c

2025-11-29 02:50:29,897 - distributed.scheduler - INFO - Close client connection: Client-1987c8a2-ccce-11f0-8001-f2cd832f225c

2025-11-29 02:50:29,896 - distributed.scheduler - INFO - Remove client Client-1987c8a2-ccce-11f0-8001-f2cd832f225c

2025-11-29 02:50:29,894 - distributed.scheduler - INFO - Remove client Client-1987c8a2-ccce-11f0-8001-f2cd832f225c

2025-11-29 02:50:01,234 - distributed.scheduler - INFO - Receive client connection: Client-1987c8a2-ccce-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,910 - distributed.scheduler - INFO - Close client connection: Client-d366361c-cc94-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,910 - distributed.scheduler - INFO - Close client connection: Client-39ec26cd-cc96-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,910 - distributed.scheduler - INFO - Close client connection: Client-a06c9bf3-cc97-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,910 - distributed.scheduler - INFO - Close client connection: Client-06f17d35-cc99-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,909 - distributed.scheduler - INFO - Close client connection: Client-6d71c5b4-cc9a-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,909 - distributed.scheduler - INFO - Close client connection: Client-d3f3a2d0-cc9b-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,909 - distributed.scheduler - INFO - Close client connection: Client-3a6c4408-cc9d-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,909 - distributed.scheduler - INFO - Close client connection: Client-0771850b-cca0-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,909 - distributed.scheduler - INFO - Close client connection: Client-a0f098b1-cc9e-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,909 - distributed.scheduler - INFO - Close client connection: Client-6dee1a8c-cca1-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,909 - distributed.scheduler - INFO - Close client connection: Client-d46d92dc-cca2-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,909 - distributed.scheduler - INFO - Close client connection: Client-3ae5f110-cca4-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,909 - distributed.scheduler - INFO - Close client connection: Client-a1551f85-cca5-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,909 - distributed.scheduler - INFO - Close client connection: Client-07d2a0b5-cca7-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,908 - distributed.scheduler - INFO - Close client connection: Client-6e5c80c5-cca8-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,908 - distributed.scheduler - INFO - Close client connection: Client-d4cc3583-cca9-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,908 - distributed.scheduler - INFO - Close client connection: Client-3b47727e-ccab-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,908 - distributed.scheduler - INFO - Close client connection: Client-a1bf3199-ccac-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,908 - distributed.scheduler - INFO - Close client connection: Client-0828fcc4-ccae-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,907 - distributed.scheduler - INFO - Close client connection: Client-6eaf3153-ccaf-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,907 - distributed.scheduler - INFO - Close client connection: Client-d530ef4e-ccb0-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,907 - distributed.scheduler - INFO - Close client connection: Client-3badd23d-ccb2-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,907 - distributed.scheduler - INFO - Close client connection: Client-a226971c-ccb3-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,907 - distributed.scheduler - INFO - Close client connection: Client-08a91832-ccb5-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,907 - distributed.scheduler - INFO - Close client connection: Client-6f2baba9-ccb6-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,907 - distributed.scheduler - INFO - Close client connection: Client-d5a0a69d-ccb7-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,907 - distributed.scheduler - INFO - Close client connection: Client-3c23c17c-ccb9-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,907 - distributed.scheduler - INFO - Close client connection: Client-a29a96b6-ccba-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,906 - distributed.scheduler - INFO - Close client connection: Client-092d47f9-ccbc-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,906 - distributed.scheduler - INFO - Close client connection: Client-6faa9d90-ccbd-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,906 - distributed.scheduler - INFO - Close client connection: Client-d6261c01-ccbe-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,906 - distributed.scheduler - INFO - Close client connection: Client-3c9fa9a3-ccc0-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,906 - distributed.scheduler - INFO - Close client connection: Client-a3142db5-ccc1-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,906 - distributed.scheduler - INFO - Close client connection: Client-098c3924-ccc3-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,906 - distributed.scheduler - INFO - Close client connection: Client-7008730b-ccc4-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,906 - distributed.scheduler - INFO - Close client connection: Client-d688bf8f-ccc5-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,905 - distributed.scheduler - INFO - Close client connection: Client-3d05c480-ccc7-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,905 - distributed.scheduler - INFO - Close client connection: Client-a3809e62-ccc8-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,905 - distributed.scheduler - INFO - Close client connection: Client-09f763a1-ccca-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,904 - distributed.scheduler - INFO - Close client connection: Client-7066f515-cccb-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,903 - distributed.scheduler - INFO - Remove client Client-d366361c-cc94-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,903 - distributed.scheduler - INFO - Remove client Client-39ec26cd-cc96-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,903 - distributed.scheduler - INFO - Remove client Client-a06c9bf3-cc97-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,902 - distributed.scheduler - INFO - Remove client Client-06f17d35-cc99-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,902 - distributed.scheduler - INFO - Remove client Client-6d71c5b4-cc9a-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,901 - distributed.scheduler - INFO - Remove client Client-d3f3a2d0-cc9b-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,901 - distributed.scheduler - INFO - Remove client Client-3a6c4408-cc9d-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,901 - distributed.scheduler - INFO - Remove client Client-0771850b-cca0-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,900 - distributed.scheduler - INFO - Remove client Client-a0f098b1-cc9e-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,900 - distributed.scheduler - INFO - Remove client Client-6dee1a8c-cca1-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,900 - distributed.scheduler - INFO - Remove client Client-d46d92dc-cca2-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,899 - distributed.scheduler - INFO - Remove client Client-3ae5f110-cca4-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,899 - distributed.scheduler - INFO - Remove client Client-a1551f85-cca5-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,899 - distributed.scheduler - INFO - Remove client Client-07d2a0b5-cca7-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,899 - distributed.scheduler - INFO - Remove client Client-6e5c80c5-cca8-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,898 - distributed.scheduler - INFO - Remove client Client-d4cc3583-cca9-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,898 - distributed.scheduler - INFO - Remove client Client-3b47727e-ccab-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,893 - distributed.scheduler - INFO - Remove client Client-a1bf3199-ccac-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,892 - distributed.scheduler - INFO - Remove client Client-0828fcc4-ccae-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,892 - distributed.scheduler - INFO - Remove client Client-6eaf3153-ccaf-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,891 - distributed.scheduler - INFO - Remove client Client-d530ef4e-ccb0-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,891 - distributed.scheduler - INFO - Remove client Client-3badd23d-ccb2-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,890 - distributed.scheduler - INFO - Remove client Client-a226971c-ccb3-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,890 - distributed.scheduler - INFO - Remove client Client-08a91832-ccb5-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,889 - distributed.scheduler - INFO - Remove client Client-6f2baba9-ccb6-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,889 - distributed.scheduler - INFO - Remove client Client-d5a0a69d-ccb7-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,889 - distributed.scheduler - INFO - Remove client Client-3c23c17c-ccb9-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,888 - distributed.scheduler - INFO - Remove client Client-a29a96b6-ccba-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,888 - distributed.scheduler - INFO - Remove client Client-092d47f9-ccbc-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,887 - distributed.scheduler - INFO - Remove client Client-6faa9d90-ccbd-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,887 - distributed.scheduler - INFO - Remove client Client-d6261c01-ccbe-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,886 - distributed.scheduler - INFO - Remove client Client-3c9fa9a3-ccc0-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,886 - distributed.scheduler - INFO - Remove client Client-a3142db5-ccc1-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,885 - distributed.scheduler - INFO - Remove client Client-098c3924-ccc3-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,885 - distributed.scheduler - INFO - Remove client Client-7008730b-ccc4-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,884 - distributed.scheduler - INFO - Remove client Client-d688bf8f-ccc5-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,884 - distributed.scheduler - INFO - Remove client Client-3d05c480-ccc7-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,883 - distributed.scheduler - INFO - Remove client Client-a3809e62-ccc8-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,882 - distributed.scheduler - INFO - Remove client Client-09f763a1-ccca-11f0-8001-f2cd832f225c

2025-11-29 02:41:29,872 - distributed.scheduler - INFO - Remove client Client-7066f515-cccb-11f0-8001-f2cd832f225c

2025-11-29 02:41:28,946 - distributed.scheduler - INFO - Close client connection: Client-d6e426bc-cccc-11f0-8001-f2cd832f225c

2025-11-29 02:41:28,945 - distributed.scheduler - INFO - Remove client Client-d6e426bc-cccc-11f0-8001-f2cd832f225c

2025-11-29 02:41:28,943 - distributed.scheduler - INFO - Remove client Client-d6e426bc-cccc-11f0-8001-f2cd832f225c

2025-11-29 02:40:59,937 - distributed.scheduler - INFO - Receive client connection: Client-d6e426bc-cccc-11f0-8001-f2cd832f225c

2025-11-29 02:30:58,493 - distributed.scheduler - INFO - Receive client connection: Client-7066f515-cccb-11f0-8001-f2cd832f225c

2025-11-29 02:20:57,138 - distributed.scheduler - INFO - Receive client connection: Client-09f763a1-ccca-11f0-8001-f2cd832f225c

2025-11-29 02:10:55,735 - distributed.scheduler - INFO - Receive client connection: Client-a3809e62-ccc8-11f0-8001-f2cd832f225c

2025-11-29 02:00:54,306 - distributed.scheduler - INFO - Receive client connection: Client-3d05c480-ccc7-11f0-8001-f2cd832f225c

2025-11-29 01:50:52,857 - distributed.scheduler - INFO - Receive client connection: Client-d688bf8f-ccc5-11f0-8001-f2cd832f225c

2025-11-29 01:40:51,392 - distributed.scheduler - INFO - Receive client connection: Client-7008730b-ccc4-11f0-8001-f2cd832f225c

2025-11-29 01:30:49,955 - distributed.scheduler - INFO - Receive client connection: Client-098c3924-ccc3-11f0-8001-f2cd832f225c

2025-11-29 01:20:48,543 - distributed.scheduler - INFO - Receive client connection: Client-a3142db5-ccc1-11f0-8001-f2cd832f225c

2025-11-29 01:10:47,152 - distributed.scheduler - INFO - Receive client connection: Client-3c9fa9a3-ccc0-11f0-8001-f2cd832f225c

2025-11-29 01:00:45,729 - distributed.scheduler - INFO - Receive client connection: Client-d6261c01-ccbe-11f0-8001-f2cd832f225c

2025-11-29 00:50:44,304 - distributed.scheduler - INFO - Receive client connection: Client-6faa9d90-ccbd-11f0-8001-f2cd832f225c

2025-11-29 00:40:42,853 - distributed.scheduler - INFO - Receive client connection: Client-092d47f9-ccbc-11f0-8001-f2cd832f225c

2025-11-29 00:30:41,274 - distributed.scheduler - INFO - Receive client connection: Client-a29a96b6-ccba-11f0-8001-f2cd832f225c

2025-11-29 00:20:39,905 - distributed.scheduler - INFO - Receive client connection: Client-3c23c17c-ccb9-11f0-8001-f2cd832f225c

2025-11-29 00:10:38,386 - distributed.scheduler - INFO - Receive client connection: Client-d5a0a69d-ccb7-11f0-8001-f2cd832f225c

2025-11-29 00:00:36,988 - distributed.scheduler - INFO - Receive client connection: Client-6f2baba9-ccb6-11f0-8001-f2cd832f225c

2025-11-28 23:50:35,514 - distributed.scheduler - INFO - Receive client connection: Client-08a91832-ccb5-11f0-8001-f2cd832f225c

2025-11-28 23:40:34,033 - distributed.scheduler - INFO - Receive client connection: Client-a226971c-ccb3-11f0-8001-f2cd832f225c

2025-11-28 23:30:32,619 - distributed.scheduler - INFO - Receive client connection: Client-3badd23d-ccb2-11f0-8001-f2cd832f225c

2025-11-28 23:20:31,173 - distributed.scheduler - INFO - Receive client connection: Client-d530ef4e-ccb0-11f0-8001-f2cd832f225c

2025-11-28 23:10:29,701 - distributed.scheduler - INFO - Receive client connection: Client-6eaf3153-ccaf-11f0-8001-f2cd832f225c

2025-11-28 23:00:28,199 - distributed.scheduler - INFO - Receive client connection: Client-0828fcc4-ccae-11f0-8001-f2cd832f225c

2025-11-28 22:50:26,882 - distributed.scheduler - INFO - Receive client connection: Client-a1bf3199-ccac-11f0-8001-f2cd832f225c

2025-11-28 22:40:25,470 - distributed.scheduler - INFO - Receive client connection: Client-3b47727e-ccab-11f0-8001-f2cd832f225c

2025-11-28 22:30:24,036 - distributed.scheduler - INFO - Receive client connection: Client-d4cc3583-cca9-11f0-8001-f2cd832f225c

2025-11-28 22:20:22,683 - distributed.scheduler - INFO - Receive client connection: Client-6e5c80c5-cca8-11f0-8001-f2cd832f225c

2025-11-28 22:10:21,153 - distributed.scheduler - INFO - Receive client connection: Client-07d2a0b5-cca7-11f0-8001-f2cd832f225c

2025-11-28 22:00:19,711 - distributed.scheduler - INFO - Receive client connection: Client-a1551f85-cca5-11f0-8001-f2cd832f225c

2025-11-28 21:50:18,352 - distributed.scheduler - INFO - Receive client connection: Client-3ae5f110-cca4-11f0-8001-f2cd832f225c

2025-11-28 21:40:16,940 - distributed.scheduler - INFO - Receive client connection: Client-d46d92dc-cca2-11f0-8001-f2cd832f225c

2025-11-28 21:30:15,482 - distributed.scheduler - INFO - Receive client connection: Client-6dee1a8c-cca1-11f0-8001-f2cd832f225c

2025-11-28 21:20:14,041 - distributed.scheduler - INFO - Receive client connection: Client-0771850b-cca0-11f0-8001-f2cd832f225c

2025-11-28 21:10:12,573 - distributed.scheduler - INFO - Receive client connection: Client-a0f098b1-cc9e-11f0-8001-f2cd832f225c

2025-11-28 21:00:11,079 - distributed.scheduler - INFO - Receive client connection: Client-3a6c4408-cc9d-11f0-8001-f2cd832f225c

2025-11-28 20:50:09,664 - distributed.scheduler - INFO - Receive client connection: Client-d3f3a2d0-cc9b-11f0-8001-f2cd832f225c

2025-11-28 20:40:08,191 - distributed.scheduler - INFO - Receive client connection: Client-6d71c5b4-cc9a-11f0-8001-f2cd832f225c

2025-11-28 20:30:06,724 - distributed.scheduler - INFO - Receive client connection: Client-06f17d35-cc99-11f0-8001-f2cd832f225c

2025-11-28 20:20:05,230 - distributed.scheduler - INFO - Receive client connection: Client-a06c9bf3-cc97-11f0-8001-f2cd832f225c

2025-11-28 20:10:03,761 - distributed.scheduler - INFO - Receive client connection: Client-39ec26cd-cc96-11f0-8001-f2cd832f225c

2025-11-28 20:00:02,270 - distributed.scheduler - INFO - Receive client connection: Client-d366361c-cc94-11f0-8001-f2cd832f225c

2025-11-28 19:50:34,585 - distributed.scheduler - INFO - Close client connection: Client-6ce8c3df-cc93-11f0-8001-f2cd832f225c

2025-11-28 19:50:34,584 - distributed.scheduler - INFO - Remove client Client-6ce8c3df-cc93-11f0-8001-f2cd832f225c

2025-11-28 19:50:34,582 - distributed.scheduler - INFO - Remove client Client-6ce8c3df-cc93-11f0-8001-f2cd832f225c

2025-11-28 19:50:00,824 - distributed.scheduler - INFO - Receive client connection: Client-6ce8c3df-cc93-11f0-8001-f2cd832f225c

2025-11-28 19:41:34,785 - distributed.scheduler - INFO - Close client connection: Client-2a419277-cc92-11f0-8001-f2cd832f225c

2025-11-28 19:41:34,784 - distributed.scheduler - INFO - Remove client Client-2a419277-cc92-11f0-8001-f2cd832f225c

2025-11-28 19:41:34,782 - distributed.scheduler - INFO - Remove client Client-2a419277-cc92-11f0-8001-f2cd832f225c

2025-11-28 19:40:59,503 - distributed.scheduler - INFO - Receive client connection: Client-2a419277-cc92-11f0-8001-f2cd832f225c

2025-11-28 19:32:02,273 - distributed.scheduler - INFO - Close client connection: Client-c3c53412-cc90-11f0-8001-f2cd832f225c

2025-11-28 19:32:02,272 - distributed.scheduler - INFO - Remove client Client-c3c53412-cc90-11f0-8001-f2cd832f225c

2025-11-28 19:32:02,271 - distributed.scheduler - INFO - Remove client Client-c3c53412-cc90-11f0-8001-f2cd832f225c

2025-11-28 19:30:58,063 - distributed.scheduler - INFO - Receive client connection: Client-c3c53412-cc90-11f0-8001-f2cd832f225c

2025-11-28 19:21:32,198 - distributed.scheduler - INFO - Close client connection: Client-5d57f533-cc8f-11f0-8001-f2cd832f225c

2025-11-28 19:21:32,197 - distributed.scheduler - INFO - Remove client Client-5d57f533-cc8f-11f0-8001-f2cd832f225c

2025-11-28 19:21:32,196 - distributed.scheduler - INFO - Remove client Client-5d57f533-cc8f-11f0-8001-f2cd832f225c

2025-11-28 19:20:56,722 - distributed.scheduler - INFO - Receive client connection: Client-5d57f533-cc8f-11f0-8001-f2cd832f225c

2025-11-28 19:11:29,015 - distributed.scheduler - INFO - Close client connection: Client-f6d55772-cc8d-11f0-8001-f2cd832f225c

2025-11-28 19:11:29,014 - distributed.scheduler - INFO - Remove client Client-f6d55772-cc8d-11f0-8001-f2cd832f225c

2025-11-28 19:11:29,013 - distributed.scheduler - INFO - Remove client Client-f6d55772-cc8d-11f0-8001-f2cd832f225c

2025-11-28 19:10:55,239 - distributed.scheduler - INFO - Receive client connection: Client-f6d55772-cc8d-11f0-8001-f2cd832f225c

2025-11-28 19:02:02,080 - distributed.scheduler - INFO - Close client connection: Client-905bf932-cc8c-11f0-8001-f2cd832f225c

2025-11-28 19:02:02,079 - distributed.scheduler - INFO - Remove client Client-905bf932-cc8c-11f0-8001-f2cd832f225c

2025-11-28 19:02:02,077 - distributed.scheduler - INFO - Remove client Client-905bf932-cc8c-11f0-8001-f2cd832f225c

2025-11-28 19:00:53,820 - distributed.scheduler - INFO - Receive client connection: Client-905bf932-cc8c-11f0-8001-f2cd832f225c

2025-11-28 18:51:27,262 - distributed.scheduler - INFO - Close client connection: Client-29e3d9f4-cc8b-11f0-8001-f2cd832f225c

2025-11-28 18:51:27,255 - distributed.scheduler - INFO - Remove client Client-29e3d9f4-cc8b-11f0-8001-f2cd832f225c

2025-11-28 18:51:27,253 - distributed.scheduler - INFO - Remove client Client-29e3d9f4-cc8b-11f0-8001-f2cd832f225c

2025-11-28 18:50:52,408 - distributed.scheduler - INFO - Receive client connection: Client-29e3d9f4-cc8b-11f0-8001-f2cd832f225c

2025-11-28 18:41:25,540 - distributed.scheduler - INFO - Close client connection: Client-8fe513fe-cc85-11f0-8001-f2cd832f225c

2025-11-28 18:41:25,540 - distributed.scheduler - INFO - Close client connection: Client-f66858ca-cc86-11f0-8001-f2cd832f225c

2025-11-28 18:41:25,540 - distributed.scheduler - INFO - Close client connection: Client-5ce1d076-cc88-11f0-8001-f2cd832f225c

2025-11-28 18:41:25,538 - distributed.scheduler - INFO - Remove client Client-8fe513fe-cc85-11f0-8001-f2cd832f225c

2025-11-28 18:41:25,537 - distributed.scheduler - INFO - Remove client Client-f66858ca-cc86-11f0-8001-f2cd832f225c

2025-11-28 18:41:25,537 - distributed.scheduler - INFO - Remove client Client-5ce1d076-cc88-11f0-8001-f2cd832f225c

2025-11-28 18:41:25,022 - distributed.scheduler - INFO - Close client connection: Client-c360ab40-cc89-11f0-8001-f2cd832f225c

2025-11-28 18:41:25,021 - distributed.scheduler - INFO - Remove client Client-c360ab40-cc89-11f0-8001-f2cd832f225c

2025-11-28 18:41:25,019 - distributed.scheduler - INFO - Remove client Client-c360ab40-cc89-11f0-8001-f2cd832f225c

2025-11-28 18:40:50,917 - distributed.scheduler - INFO - Receive client connection: Client-c360ab40-cc89-11f0-8001-f2cd832f225c

2025-11-28 18:30:49,461 - distributed.scheduler - INFO - Receive client connection: Client-5ce1d076-cc88-11f0-8001-f2cd832f225c

2025-11-28 18:20:48,042 - distributed.scheduler - INFO - Receive client connection: Client-f66858ca-cc86-11f0-8001-f2cd832f225c

2025-11-28 18:10:46,567 - distributed.scheduler - INFO - Receive client connection: Client-8fe513fe-cc85-11f0-8001-f2cd832f225c

2025-11-28 18:01:54,172 - distributed.scheduler - INFO - Close client connection: Client-295be4f4-cc84-11f0-8001-f2cd832f225c

2025-11-28 18:01:54,170 - distributed.scheduler - INFO - Remove client Client-295be4f4-cc84-11f0-8001-f2cd832f225c

2025-11-28 18:01:54,168 - distributed.scheduler - INFO - Remove client Client-295be4f4-cc84-11f0-8001-f2cd832f225c

2025-11-28 18:00:45,040 - distributed.scheduler - INFO - Receive client connection: Client-295be4f4-cc84-11f0-8001-f2cd832f225c

2025-11-28 17:51:18,409 - distributed.scheduler - INFO - Close client connection: Client-c2d38d73-cc82-11f0-8001-f2cd832f225c

2025-11-28 17:51:18,408 - distributed.scheduler - INFO - Remove client Client-c2d38d73-cc82-11f0-8001-f2cd832f225c

2025-11-28 17:51:18,406 - distributed.scheduler - INFO - Remove client Client-c2d38d73-cc82-11f0-8001-f2cd832f225c

2025-11-28 17:50:43,521 - distributed.scheduler - INFO - Receive client connection: Client-c2d38d73-cc82-11f0-8001-f2cd832f225c

2025-11-28 17:41:18,265 - distributed.scheduler - INFO - Close client connection: Client-5c579fc0-cc81-11f0-8001-f2cd832f225c

2025-11-28 17:41:18,263 - distributed.scheduler - INFO - Remove client Client-5c579fc0-cc81-11f0-8001-f2cd832f225c

2025-11-28 17:41:18,262 - distributed.scheduler - INFO - Remove client Client-5c579fc0-cc81-11f0-8001-f2cd832f225c

2025-11-28 17:40:42,130 - distributed.scheduler - INFO - Receive client connection: Client-5c579fc0-cc81-11f0-8001-f2cd832f225c

2025-11-28 17:31:50,676 - distributed.scheduler - INFO - Close client connection: Client-f5d49d5a-cc7f-11f0-8001-f2cd832f225c

2025-11-28 17:31:50,675 - distributed.scheduler - INFO - Remove client Client-f5d49d5a-cc7f-11f0-8001-f2cd832f225c

2025-11-28 17:31:50,673 - distributed.scheduler - INFO - Remove client Client-f5d49d5a-cc7f-11f0-8001-f2cd832f225c

2025-11-28 17:30:40,604 - distributed.scheduler - INFO - Receive client connection: Client-f5d49d5a-cc7f-11f0-8001-f2cd832f225c

2025-11-28 17:21:16,598 - distributed.scheduler - INFO - Close client connection: Client-8f5662c6-cc7e-11f0-8001-f2cd832f225c

2025-11-28 17:21:16,598 - distributed.scheduler - INFO - Remove client Client-8f5662c6-cc7e-11f0-8001-f2cd832f225c

2025-11-28 17:21:16,596 - distributed.scheduler - INFO - Remove client Client-8f5662c6-cc7e-11f0-8001-f2cd832f225c

2025-11-28 17:20:39,150 - distributed.scheduler - INFO - Receive client connection: Client-8f5662c6-cc7e-11f0-8001-f2cd832f225c

2025-11-28 17:11:19,819 - distributed.scheduler - INFO - Close client connection: Client-28dd6f15-cc7d-11f0-8001-f2cd832f225c

2025-11-28 17:11:19,819 - distributed.scheduler - INFO - Remove client Client-28dd6f15-cc7d-11f0-8001-f2cd832f225c

2025-11-28 17:11:19,818 - distributed.scheduler - INFO - Remove client Client-28dd6f15-cc7d-11f0-8001-f2cd832f225c

2025-11-28 17:10:37,736 - distributed.scheduler - INFO - Receive client connection: Client-28dd6f15-cc7d-11f0-8001-f2cd832f225c

2025-11-28 17:01:47,606 - distributed.scheduler - INFO - Close client connection: Client-c240fa11-cc7b-11f0-8001-f2cd832f225c

2025-11-28 17:01:47,604 - distributed.scheduler - INFO - Remove client Client-c240fa11-cc7b-11f0-8001-f2cd832f225c

2025-11-28 17:01:47,603 - distributed.scheduler - INFO - Remove client Client-c240fa11-cc7b-11f0-8001-f2cd832f225c

2025-11-28 17:00:36,087 - distributed.scheduler - INFO - Receive client connection: Client-c240fa11-cc7b-11f0-8001-f2cd832f225c

2025-11-28 16:51:16,447 - distributed.scheduler - INFO - Close client connection: Client-5bc076ec-cc7a-11f0-8001-f2cd832f225c

2025-11-28 16:51:16,446 - distributed.scheduler - INFO - Remove client Client-5bc076ec-cc7a-11f0-8001-f2cd832f225c

2025-11-28 16:51:16,444 - distributed.scheduler - INFO - Remove client Client-5bc076ec-cc7a-11f0-8001-f2cd832f225c

2025-11-28 16:50:34,622 - distributed.scheduler - INFO - Receive client connection: Client-5bc076ec-cc7a-11f0-8001-f2cd832f225c

2025-11-28 16:41:13,744 - distributed.scheduler - INFO - Close client connection: Client-f53ab236-cc78-11f0-8001-f2cd832f225c

2025-11-28 16:41:13,742 - distributed.scheduler - INFO - Remove client Client-f53ab236-cc78-11f0-8001-f2cd832f225c

2025-11-28 16:41:13,741 - distributed.scheduler - INFO - Remove client Client-f53ab236-cc78-11f0-8001-f2cd832f225c

2025-11-28 16:40:33,114 - distributed.scheduler - INFO - Receive client connection: Client-f53ab236-cc78-11f0-8001-f2cd832f225c

2025-11-28 16:31:46,234 - distributed.scheduler - INFO - Close client connection: Client-8ebab9b9-cc77-11f0-8001-f2cd832f225c

2025-11-28 16:31:46,233 - distributed.scheduler - INFO - Remove client Client-8ebab9b9-cc77-11f0-8001-f2cd832f225c

2025-11-28 16:31:46,231 - distributed.scheduler - INFO - Remove client Client-8ebab9b9-cc77-11f0-8001-f2cd832f225c

2025-11-28 16:30:31,658 - distributed.scheduler - INFO - Receive client connection: Client-8ebab9b9-cc77-11f0-8001-f2cd832f225c

2025-11-28 16:21:10,109 - distributed.scheduler - INFO - Close client connection: Client-2845393a-cc76-11f0-8001-f2cd832f225c

2025-11-28 16:21:10,107 - distributed.scheduler - INFO - Remove client Client-2845393a-cc76-11f0-8001-f2cd832f225c

2025-11-28 16:21:10,105 - distributed.scheduler - INFO - Remove client Client-2845393a-cc76-11f0-8001-f2cd832f225c

2025-11-28 16:20:30,261 - distributed.scheduler - INFO - Receive client connection: Client-2845393a-cc76-11f0-8001-f2cd832f225c

2025-11-28 16:11:08,621 - distributed.scheduler - INFO - Close client connection: Client-c1c783c9-cc74-11f0-8001-f2cd832f225c

2025-11-28 16:11:08,619 - distributed.scheduler - INFO - Remove client Client-c1c783c9-cc74-11f0-8001-f2cd832f225c

2025-11-28 16:11:08,617 - distributed.scheduler - INFO - Remove client Client-c1c783c9-cc74-11f0-8001-f2cd832f225c

2025-11-28 16:10:28,813 - distributed.scheduler - INFO - Receive client connection: Client-c1c783c9-cc74-11f0-8001-f2cd832f225c

2025-11-28 16:01:41,451 - distributed.scheduler - INFO - Close client connection: Client-5b45e5d0-cc73-11f0-8001-f2cd832f225c

2025-11-28 16:01:41,450 - distributed.scheduler - INFO - Remove client Client-5b45e5d0-cc73-11f0-8001-f2cd832f225c

2025-11-28 16:01:41,448 - distributed.scheduler - INFO - Remove client Client-5b45e5d0-cc73-11f0-8001-f2cd832f225c

2025-11-28 16:00:27,341 - distributed.scheduler - INFO - Receive client connection: Client-5b45e5d0-cc73-11f0-8001-f2cd832f225c

2025-11-28 15:51:09,544 - distributed.scheduler - INFO - Close client connection: Client-f4dd180f-cc71-11f0-8001-f2cd832f225c

2025-11-28 15:51:09,543 - distributed.scheduler - INFO - Remove client Client-f4dd180f-cc71-11f0-8001-f2cd832f225c

2025-11-28 15:51:09,541 - distributed.scheduler - INFO - Remove client Client-f4dd180f-cc71-11f0-8001-f2cd832f225c

2025-11-28 15:50:26,031 - distributed.scheduler - INFO - Receive client connection: Client-f4dd180f-cc71-11f0-8001-f2cd832f225c

2025-11-28 15:41:05,735 - distributed.scheduler - INFO - Close client connection: Client-8e618ead-cc70-11f0-8001-f2cd832f225c

2025-11-28 15:41:05,734 - distributed.scheduler - INFO - Remove client Client-8e618ead-cc70-11f0-8001-f2cd832f225c

2025-11-28 15:41:05,733 - distributed.scheduler - INFO - Remove client Client-8e618ead-cc70-11f0-8001-f2cd832f225c

2025-11-28 15:40:24,593 - distributed.scheduler - INFO - Receive client connection: Client-8e618ead-cc70-11f0-8001-f2cd832f225c

2025-11-28 15:31:41,119 - distributed.scheduler - INFO - Close client connection: Client-27e3d469-cc6f-11f0-8001-f2cd832f225c

2025-11-28 15:31:41,118 - distributed.scheduler - INFO - Remove client Client-27e3d469-cc6f-11f0-8001-f2cd832f225c

2025-11-28 15:31:41,116 - distributed.scheduler - INFO - Remove client Client-27e3d469-cc6f-11f0-8001-f2cd832f225c

2025-11-28 15:30:23,142 - distributed.scheduler - INFO - Receive client connection: Client-27e3d469-cc6f-11f0-8001-f2cd832f225c

2025-11-28 15:21:04,232 - distributed.scheduler - INFO - Close client connection: Client-c15b97af-cc6d-11f0-8001-f2cd832f225c

2025-11-28 15:21:04,231 - distributed.scheduler - INFO - Remove client Client-c15b97af-cc6d-11f0-8001-f2cd832f225c

2025-11-28 15:21:04,229 - distributed.scheduler - INFO - Remove client Client-c15b97af-cc6d-11f0-8001-f2cd832f225c

2025-11-28 15:20:21,630 - distributed.scheduler - INFO - Receive client connection: Client-c15b97af-cc6d-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,092 - distributed.scheduler - INFO - Close client connection: Client-47fa29ad-cc3e-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,092 - distributed.scheduler - INFO - Close client connection: Client-ae7a7f1d-cc3f-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,092 - distributed.scheduler - INFO - Close client connection: Client-1506dbcb-cc41-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,091 - distributed.scheduler - INFO - Close client connection: Client-7b9f36d5-cc42-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,091 - distributed.scheduler - INFO - Close client connection: Client-e21ae60a-cc43-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,091 - distributed.scheduler - INFO - Close client connection: Client-48a2a487-cc45-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,091 - distributed.scheduler - INFO - Close client connection: Client-af1b53dc-cc46-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,091 - distributed.scheduler - INFO - Close client connection: Client-159a2518-cc48-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,091 - distributed.scheduler - INFO - Close client connection: Client-7c0f311b-cc49-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,091 - distributed.scheduler - INFO - Close client connection: Client-e2a2d64a-cc4a-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,091 - distributed.scheduler - INFO - Close client connection: Client-49199543-cc4c-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,091 - distributed.scheduler - INFO - Close client connection: Client-af94c6b8-cc4d-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,090 - distributed.scheduler - INFO - Close client connection: Client-16175437-cc4f-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,090 - distributed.scheduler - INFO - Close client connection: Client-7c935fa4-cc50-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,090 - distributed.scheduler - INFO - Close client connection: Client-e314fdfa-cc51-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,090 - distributed.scheduler - INFO - Close client connection: Client-49a42c19-cc53-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,090 - distributed.scheduler - INFO - Close client connection: Client-b020db37-cc54-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,089 - distributed.scheduler - INFO - Close client connection: Client-16991a7a-cc56-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,089 - distributed.scheduler - INFO - Close client connection: Client-7d0d0d12-cc57-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,089 - distributed.scheduler - INFO - Close client connection: Client-e3873c31-cc58-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,089 - distributed.scheduler - INFO - Close client connection: Client-26259e94-cc5a-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,089 - distributed.scheduler - INFO - Close client connection: Client-8cb50f61-cc5b-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,089 - distributed.scheduler - INFO - Close client connection: Client-f32f0cc4-cc5c-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,089 - distributed.scheduler - INFO - Close client connection: Client-59a5fa44-cc5e-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,089 - distributed.scheduler - INFO - Close client connection: Client-c0236b33-cc5f-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,089 - distributed.scheduler - INFO - Close client connection: Client-26b04d3e-cc61-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,089 - distributed.scheduler - INFO - Close client connection: Client-8d28e104-cc62-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,088 - distributed.scheduler - INFO - Close client connection: Client-f3b20935-cc63-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,088 - distributed.scheduler - INFO - Close client connection: Client-5a369ef5-cc65-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,088 - distributed.scheduler - INFO - Close client connection: Client-c0ac7b98-cc66-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,088 - distributed.scheduler - INFO - Close client connection: Client-8da775d4-cc69-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,088 - distributed.scheduler - INFO - Close client connection: Client-27279622-cc68-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,087 - distributed.scheduler - INFO - Close client connection: Client-f42c4295-cc6a-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,086 - distributed.scheduler - INFO - Remove client Client-47fa29ad-cc3e-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,086 - distributed.scheduler - INFO - Remove client Client-ae7a7f1d-cc3f-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,086 - distributed.scheduler - INFO - Remove client Client-1506dbcb-cc41-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,085 - distributed.scheduler - INFO - Remove client Client-7b9f36d5-cc42-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,085 - distributed.scheduler - INFO - Remove client Client-e21ae60a-cc43-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,085 - distributed.scheduler - INFO - Remove client Client-48a2a487-cc45-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,084 - distributed.scheduler - INFO - Remove client Client-af1b53dc-cc46-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,084 - distributed.scheduler - INFO - Remove client Client-159a2518-cc48-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,083 - distributed.scheduler - INFO - Remove client Client-7c0f311b-cc49-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,083 - distributed.scheduler - INFO - Remove client Client-e2a2d64a-cc4a-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,083 - distributed.scheduler - INFO - Remove client Client-49199543-cc4c-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,082 - distributed.scheduler - INFO - Remove client Client-af94c6b8-cc4d-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,078 - distributed.scheduler - INFO - Remove client Client-16175437-cc4f-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,077 - distributed.scheduler - INFO - Remove client Client-7c935fa4-cc50-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,077 - distributed.scheduler - INFO - Remove client Client-e314fdfa-cc51-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,077 - distributed.scheduler - INFO - Remove client Client-49a42c19-cc53-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,076 - distributed.scheduler - INFO - Remove client Client-b020db37-cc54-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,076 - distributed.scheduler - INFO - Remove client Client-16991a7a-cc56-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,075 - distributed.scheduler - INFO - Remove client Client-7d0d0d12-cc57-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,075 - distributed.scheduler - INFO - Remove client Client-e3873c31-cc58-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,074 - distributed.scheduler - INFO - Remove client Client-26259e94-cc5a-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,073 - distributed.scheduler - INFO - Remove client Client-8cb50f61-cc5b-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,073 - distributed.scheduler - INFO - Remove client Client-f32f0cc4-cc5c-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,072 - distributed.scheduler - INFO - Remove client Client-59a5fa44-cc5e-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,072 - distributed.scheduler - INFO - Remove client Client-c0236b33-cc5f-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,071 - distributed.scheduler - INFO - Remove client Client-26b04d3e-cc61-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,071 - distributed.scheduler - INFO - Remove client Client-8d28e104-cc62-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,070 - distributed.scheduler - INFO - Remove client Client-f3b20935-cc63-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,070 - distributed.scheduler - INFO - Remove client Client-5a369ef5-cc65-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,069 - distributed.scheduler - INFO - Remove client Client-c0ac7b98-cc66-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,069 - distributed.scheduler - INFO - Remove client Client-8da775d4-cc69-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,068 - distributed.scheduler - INFO - Remove client Client-27279622-cc68-11f0-8001-f2cd832f225c

2025-11-28 15:11:01,058 - distributed.scheduler - INFO - Remove client Client-f42c4295-cc6a-11f0-8001-f2cd832f225c

2025-11-28 15:11:00,221 - distributed.scheduler - INFO - Close client connection: Client-5ada4a29-cc6c-11f0-8001-f2cd832f225c

2025-11-28 15:11:00,220 - distributed.scheduler - INFO - Remove client Client-5ada4a29-cc6c-11f0-8001-f2cd832f225c

2025-11-28 15:11:00,219 - distributed.scheduler - INFO - Remove client Client-5ada4a29-cc6c-11f0-8001-f2cd832f225c

2025-11-28 15:10:20,145 - distributed.scheduler - INFO - Receive client connection: Client-5ada4a29-cc6c-11f0-8001-f2cd832f225c

2025-11-28 15:00:18,389 - distributed.scheduler - INFO - Receive client connection: Client-f42c4295-cc6a-11f0-8001-f2cd832f225c

2025-11-28 14:50:16,886 - distributed.scheduler - INFO - Receive client connection: Client-8da775d4-cc69-11f0-8001-f2cd832f225c

2025-11-28 14:40:15,449 - distributed.scheduler - INFO - Receive client connection: Client-27279622-cc68-11f0-8001-f2cd832f225c

2025-11-28 14:30:13,999 - distributed.scheduler - INFO - Receive client connection: Client-c0ac7b98-cc66-11f0-8001-f2cd832f225c

2025-11-28 14:20:12,597 - distributed.scheduler - INFO - Receive client connection: Client-5a369ef5-cc65-11f0-8001-f2cd832f225c

2025-11-28 14:10:11,101 - distributed.scheduler - INFO - Receive client connection: Client-f3b20935-cc63-11f0-8001-f2cd832f225c

2025-11-28 14:00:09,583 - distributed.scheduler - INFO - Receive client connection: Client-8d28e104-cc62-11f0-8001-f2cd832f225c

2025-11-28 13:50:08,172 - distributed.scheduler - INFO - Receive client connection: Client-26b04d3e-cc61-11f0-8001-f2cd832f225c

2025-11-28 13:40:06,620 - distributed.scheduler - INFO - Receive client connection: Client-c0236b33-cc5f-11f0-8001-f2cd832f225c

2025-11-28 13:30:05,171 - distributed.scheduler - INFO - Receive client connection: Client-59a5fa44-cc5e-11f0-8001-f2cd832f225c

2025-11-28 13:20:03,775 - distributed.scheduler - INFO - Receive client connection: Client-f32f0cc4-cc5c-11f0-8001-f2cd832f225c

2025-11-28 13:10:02,347 - distributed.scheduler - INFO - Receive client connection: Client-8cb50f61-cc5b-11f0-8001-f2cd832f225c

2025-11-28 13:00:00,777 - distributed.scheduler - INFO - Receive client connection: Client-26259e94-cc5a-11f0-8001-f2cd832f225c

2025-11-28 12:50:59,515 - distributed.scheduler - INFO - Receive client connection: Client-e3873c31-cc58-11f0-8001-f2cd832f225c

2025-11-28 12:40:58,097 - distributed.scheduler - INFO - Receive client connection: Client-7d0d0d12-cc57-11f0-8001-f2cd832f225c

2025-11-28 12:30:56,709 - distributed.scheduler - INFO - Receive client connection: Client-16991a7a-cc56-11f0-8001-f2cd832f225c

2025-11-28 12:20:55,293 - distributed.scheduler - INFO - Receive client connection: Client-b020db37-cc54-11f0-8001-f2cd832f225c

2025-11-28 12:10:53,848 - distributed.scheduler - INFO - Receive client connection: Client-49a42c19-cc53-11f0-8001-f2cd832f225c

2025-11-28 12:00:52,288 - distributed.scheduler - INFO - Receive client connection: Client-e314fdfa-cc51-11f0-8001-f2cd832f225c

2025-11-28 11:50:50,816 - distributed.scheduler - INFO - Receive client connection: Client-7c935fa4-cc50-11f0-8001-f2cd832f225c

2025-11-28 11:40:49,381 - distributed.scheduler - INFO - Receive client connection: Client-16175437-cc4f-11f0-8001-f2cd832f225c

2025-11-28 11:30:47,898 - distributed.scheduler - INFO - Receive client connection: Client-af94c6b8-cc4d-11f0-8001-f2cd832f225c

2025-11-28 11:20:46,463 - distributed.scheduler - INFO - Receive client connection: Client-49199543-cc4c-11f0-8001-f2cd832f225c

2025-11-28 11:10:45,062 - distributed.scheduler - INFO - Receive client connection: Client-e2a2d64a-cc4a-11f0-8001-f2cd832f225c

2025-11-28 11:00:43,473 - distributed.scheduler - INFO - Receive client connection: Client-7c0f311b-cc49-11f0-8001-f2cd832f225c

2025-11-28 10:50:42,113 - distributed.scheduler - INFO - Receive client connection: Client-159a2518-cc48-11f0-8001-f2cd832f225c

2025-11-28 10:40:40,666 - distributed.scheduler - INFO - Receive client connection: Client-af1b53dc-cc46-11f0-8001-f2cd832f225c

2025-11-28 10:30:39,221 - distributed.scheduler - INFO - Receive client connection: Client-48a2a487-cc45-11f0-8001-f2cd832f225c

2025-11-28 10:20:37,698 - distributed.scheduler - INFO - Receive client connection: Client-e21ae60a-cc43-11f0-8001-f2cd832f225c

2025-11-28 10:10:36,260 - distributed.scheduler - INFO - Receive client connection: Client-7b9f36d5-cc42-11f0-8001-f2cd832f225c

2025-11-28 10:00:34,638 - distributed.scheduler - INFO - Receive client connection: Client-1506dbcb-cc41-11f0-8001-f2cd832f225c

2025-11-28 09:50:33,096 - distributed.scheduler - INFO - Receive client connection: Client-ae7a7f1d-cc3f-11f0-8001-f2cd832f225c

2025-11-28 09:40:31,641 - distributed.scheduler - INFO - Receive client connection: Client-47fa29ad-cc3e-11f0-8001-f2cd832f225c

2025-11-28 09:31:34,904 - distributed.scheduler - INFO - Close client connection: Client-e17dd53d-cc3c-11f0-8001-f2cd832f225c

2025-11-28 09:31:34,903 - distributed.scheduler - INFO - Remove client Client-e17dd53d-cc3c-11f0-8001-f2cd832f225c

2025-11-28 09:31:34,901 - distributed.scheduler - INFO - Remove client Client-e17dd53d-cc3c-11f0-8001-f2cd832f225c

2025-11-28 09:30:30,199 - distributed.scheduler - INFO - Receive client connection: Client-e17dd53d-cc3c-11f0-8001-f2cd832f225c

2025-11-28 09:21:01,178 - distributed.scheduler - INFO - Close client connection: Client-7b02acf3-cc3b-11f0-8001-f2cd832f225c

2025-11-28 09:21:01,177 - distributed.scheduler - INFO - Remove client Client-7b02acf3-cc3b-11f0-8001-f2cd832f225c

2025-11-28 09:21:01,175 - distributed.scheduler - INFO - Remove client Client-7b02acf3-cc3b-11f0-8001-f2cd832f225c

2025-11-28 09:20:28,768 - distributed.scheduler - INFO - Receive client connection: Client-7b02acf3-cc3b-11f0-8001-f2cd832f225c

2025-11-28 09:11:00,334 - distributed.scheduler - INFO - Close client connection: Client-14924849-cc3a-11f0-8001-f2cd832f225c

2025-11-28 09:11:00,333 - distributed.scheduler - INFO - Remove client Client-14924849-cc3a-11f0-8001-f2cd832f225c

2025-11-28 09:11:00,331 - distributed.scheduler - INFO - Remove client Client-14924849-cc3a-11f0-8001-f2cd832f225c

2025-11-28 09:10:27,407 - distributed.scheduler - INFO - Receive client connection: Client-14924849-cc3a-11f0-8001-f2cd832f225c

2025-11-28 09:01:29,383 - distributed.scheduler - INFO - Close client connection: Client-ae1d305d-cc38-11f0-8001-f2cd832f225c

2025-11-28 09:01:29,382 - distributed.scheduler - INFO - Remove client Client-ae1d305d-cc38-11f0-8001-f2cd832f225c

2025-11-28 09:01:29,380 - distributed.scheduler - INFO - Remove client Client-ae1d305d-cc38-11f0-8001-f2cd832f225c

2025-11-28 09:00:30,074 - distributed.scheduler - INFO - Receive client connection: Client-worker-b088850e-cc38-11f0-85df-32a59f79d364

2025-11-28 09:00:26,018 - distributed.scheduler - INFO - Receive client connection: Client-ae1d305d-cc38-11f0-8001-f2cd832f225c

2025-11-28 08:51:01,363 - distributed.scheduler - INFO - Close client connection: Client-47a12dba-cc37-11f0-8001-f2cd832f225c

2025-11-28 08:51:01,362 - distributed.scheduler - INFO - Remove client Client-47a12dba-cc37-11f0-8001-f2cd832f225c

2025-11-28 08:51:01,360 - distributed.scheduler - INFO - Remove client Client-47a12dba-cc37-11f0-8001-f2cd832f225c

2025-11-28 08:50:24,570 - distributed.scheduler - INFO - Receive client connection: Client-47a12dba-cc37-11f0-8001-f2cd832f225c

2025-11-28 08:41:00,595 - distributed.scheduler - INFO - Close client connection: Client-e12e62d3-cc35-11f0-8001-f2cd832f225c

2025-11-28 08:41:00,593 - distributed.scheduler - INFO - Remove client Client-e12e62d3-cc35-11f0-8001-f2cd832f225c

2025-11-28 08:41:00,592 - distributed.scheduler - INFO - Remove client Client-e12e62d3-cc35-11f0-8001-f2cd832f225c

2025-11-28 08:40:23,204 - distributed.scheduler - INFO - Receive client connection: Client-e12e62d3-cc35-11f0-8001-f2cd832f225c

2025-11-28 08:31:32,367 - distributed.scheduler - INFO - Close client connection: Client-7ab2137e-cc34-11f0-8001-f2cd832f225c

2025-11-28 08:31:32,365 - distributed.scheduler - INFO - Remove client Client-7ab2137e-cc34-11f0-8001-f2cd832f225c

2025-11-28 08:31:32,363 - distributed.scheduler - INFO - Remove client Client-7ab2137e-cc34-11f0-8001-f2cd832f225c

2025-11-28 08:30:21,765 - distributed.scheduler - INFO - Receive client connection: Client-7ab2137e-cc34-11f0-8001-f2cd832f225c

2025-11-28 08:20:57,387 - distributed.scheduler - INFO - Close client connection: Client-1433a09d-cc33-11f0-8001-f2cd832f225c

2025-11-28 08:20:57,385 - distributed.scheduler - INFO - Remove client Client-1433a09d-cc33-11f0-8001-f2cd832f225c

2025-11-28 08:20:57,384 - distributed.scheduler - INFO - Remove client Client-1433a09d-cc33-11f0-8001-f2cd832f225c

2025-11-28 08:20:20,313 - distributed.scheduler - INFO - Receive client connection: Client-1433a09d-cc33-11f0-8001-f2cd832f225c

2025-11-28 08:10:58,068 - distributed.scheduler - INFO - Close client connection: Client-adb4cca8-cc31-11f0-8001-f2cd832f225c

2025-11-28 08:10:58,067 - distributed.scheduler - INFO - Remove client Client-adb4cca8-cc31-11f0-8001-f2cd832f225c

2025-11-28 08:10:58,065 - distributed.scheduler - INFO - Remove client Client-adb4cca8-cc31-11f0-8001-f2cd832f225c

2025-11-28 08:10:18,862 - distributed.scheduler - INFO - Receive client connection: Client-adb4cca8-cc31-11f0-8001-f2cd832f225c

2025-11-28 08:01:23,870 - distributed.scheduler - INFO - Close client connection: Client-e0ac2bd5-cc2e-11f0-8001-f2cd832f225c

2025-11-28 08:01:23,870 - distributed.scheduler - INFO - Remove client Client-e0ac2bd5-cc2e-11f0-8001-f2cd832f225c

2025-11-28 08:01:22,874 - distributed.scheduler - INFO - Close client connection: Client-472b6e37-cc30-11f0-8001-f2cd832f225c

2025-11-28 08:01:22,873 - distributed.scheduler - INFO - Remove client Client-472b6e37-cc30-11f0-8001-f2cd832f225c

2025-11-28 08:01:22,871 - distributed.scheduler - INFO - Remove client Client-472b6e37-cc30-11f0-8001-f2cd832f225c

2025-11-28 08:00:17,325 - distributed.scheduler - INFO - Receive client connection: Client-472b6e37-cc30-11f0-8001-f2cd832f225c

2025-11-28 07:50:15,874 - distributed.scheduler - INFO - Receive client connection: Client-e0ac2bd5-cc2e-11f0-8001-f2cd832f225c

2025-11-28 07:40:51,863 - distributed.scheduler - INFO - Close client connection: Client-7a2e885a-cc2d-11f0-8001-f2cd832f225c

2025-11-28 07:40:51,863 - distributed.scheduler - INFO - Remove client Client-7a2e885a-cc2d-11f0-8001-f2cd832f225c

2025-11-28 07:40:51,861 - distributed.scheduler - INFO - Remove client Client-7a2e885a-cc2d-11f0-8001-f2cd832f225c

2025-11-28 07:40:14,423 - distributed.scheduler - INFO - Receive client connection: Client-7a2e885a-cc2d-11f0-8001-f2cd832f225c

2025-11-28 07:33:44,512 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:35499 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:35499 after 30 s

2025-11-28 07:33:44,496 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:35499 after 30 s')

2025-11-28 07:33:44,469 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:35499 after 30 s')

2025-11-28 07:33:44,453 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:35449 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:35499 after 30 s')

2025-11-28 07:33:44,353 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:35499 after 30 s')

2025-11-28 07:33:44,345 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:35499 after 30 s')

2025-11-28 07:33:44,331 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:35499 after 30 s')

2025-11-28 07:33:44,315 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41593 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:35499 after 30 s')

2025-11-28 07:33:44,314 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:43399 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:35499 after 30 s')

2025-11-28 07:33:44,276 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41965 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:35499 after 30 s')

2025-11-28 07:33:31,982 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:39155 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:39155 after 30 s

2025-11-28 07:33:31,924 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:39155 after 30 s')

2025-11-28 07:33:31,843 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:39155 after 30 s')

2025-11-28 07:33:31,841 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:40527 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:39155 after 30 s')

2025-11-28 07:33:31,830 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:39155 after 30 s')

2025-11-28 07:33:31,827 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41593 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:39155 after 30 s')

2025-11-28 07:33:31,817 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:39155 after 30 s')

2025-11-28 07:33:15,312 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:34767

2025-11-28 07:33:15,311 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:34767', status: closed, memory: 0, processing: 0>

2025-11-28 07:33:14,607 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: RuntimeError('Shuffle 129e8462c7b17b9736233ab83470f4a8 forgotten')

2025-11-28 07:33:14,578 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: KeyError('129e8462c7b17b9736233ab83470f4a8')

2025-11-28 07:33:14,189 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:35499', status: closed, memory: 0, processing: 0>

2025-11-28 07:33:02,871 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:35499

2025-11-28 07:33:02,870 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:35499', status: closed, memory: 0, processing: 0>

2025-11-28 07:33:02,275 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:35449 failed: Exception: RuntimeError('Shuffle 7cba66743f895696ef04e4fbbb073eda forgotten')

2025-11-28 07:33:02,273 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: RuntimeError('Shuffle 7cba66743f895696ef04e4fbbb073eda forgotten')

2025-11-28 07:33:02,209 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: RuntimeError('Shuffle 7cba66743f895696ef04e4fbbb073eda forgotten')

2025-11-28 07:33:02,190 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: RuntimeError('Shuffle 7cba66743f895696ef04e4fbbb073eda forgotten')

2025-11-28 07:33:02,166 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:40579 failed: Exception: RuntimeError('Shuffle 7cba66743f895696ef04e4fbbb073eda forgotten')

2025-11-28 07:33:01,701 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:39155', status: closed, memory: 0, processing: 0>

2025-11-28 07:32:52,359 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:34645 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:34645 after 30 s

2025-11-28 07:32:52,296 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:34645 after 30 s')

2025-11-28 07:32:52,219 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:35449 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:34645 after 30 s')

2025-11-28 07:32:52,187 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41593 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:34645 after 30 s')

2025-11-28 07:32:52,180 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:34645 after 30 s')

2025-11-28 07:32:52,179 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:40527 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:34645 after 30 s')

2025-11-28 07:32:52,177 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:40579 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:34645 after 30 s')

2025-11-28 07:32:52,161 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:34645 after 30 s')

2025-11-28 07:32:22,994 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:39155

2025-11-28 07:32:22,993 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:39155', status: closed, memory: 0, processing: 0>

2025-11-28 07:32:22,038 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:34645', status: closed, memory: 0, processing: 0>

2025-11-28 07:32:08,617 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:36785 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:36785 after 30 s

2025-11-28 07:32:08,567 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:36785 after 30 s')

2025-11-28 07:32:08,567 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:35449 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:36785 after 30 s')

2025-11-28 07:32:08,529 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:36785 after 30 s')

2025-11-28 07:32:08,518 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:36785 after 30 s')

2025-11-28 07:31:39,332 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:34645

2025-11-28 07:31:39,330 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:34645', status: closed, memory: 0, processing: 0>

2025-11-28 07:31:38,361 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:36785', status: closed, memory: 0, processing: 0>

2025-11-28 07:31:33,243 - distributed.scheduler - INFO - Close client connection: Client-13ad18a5-cc2c-11f0-8001-f2cd832f225c

2025-11-28 07:31:33,242 - distributed.scheduler - INFO - Remove client Client-13ad18a5-cc2c-11f0-8001-f2cd832f225c

2025-11-28 07:31:33,240 - distributed.scheduler - INFO - Remove client Client-13ad18a5-cc2c-11f0-8001-f2cd832f225c

2025-11-28 07:30:53,875 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], run-7cd591b608f7ee1fbb32b9a5ff887c2b NoneType: None

2025-11-28 07:30:53,874 - distributed.scheduler - ERROR - Couldn't gather keys {'run-7cd591b608f7ee1fbb32b9a5ff887c2b': []} state: ['processing'] workers: [] NoneType: None

2025-11-28 07:30:53,509 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:36785

2025-11-28 07:30:53,508 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:36785', status: closed, memory: 0, processing: 0>

2025-11-28 07:30:52,398 - distributed.scheduler - INFO - Close client connection: Client-worker-f0b93294-cbba-11f0-851a-32a59f79d364

2025-11-28 07:30:52,384 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:43181', status: closed, memory: 0, processing: 0>

2025-11-28 07:30:52,383 - distributed.scheduler - INFO - Remove client Client-worker-f0b93294-cbba-11f0-851a-32a59f79d364

2025-11-28 07:30:12,942 - distributed.scheduler - INFO - Receive client connection: Client-13ad18a5-cc2c-11f0-8001-f2cd832f225c

2025-11-28 07:20:51,549 - distributed.scheduler - INFO - Close client connection: Client-ad306918-cc2a-11f0-8001-f2cd832f225c

2025-11-28 07:20:51,547 - distributed.scheduler - INFO - Remove client Client-ad306918-cc2a-11f0-8001-f2cd832f225c

2025-11-28 07:20:51,545 - distributed.scheduler - INFO - Remove client Client-ad306918-cc2a-11f0-8001-f2cd832f225c

2025-11-28 07:20:11,514 - distributed.scheduler - INFO - Receive client connection: Client-ad306918-cc2a-11f0-8001-f2cd832f225c

2025-11-28 07:10:49,131 - distributed.scheduler - INFO - Close client connection: Client-46b27cf7-cc29-11f0-8001-f2cd832f225c

2025-11-28 07:10:49,130 - distributed.scheduler - INFO - Remove client Client-46b27cf7-cc29-11f0-8001-f2cd832f225c

2025-11-28 07:10:49,128 - distributed.scheduler - INFO - Remove client Client-46b27cf7-cc29-11f0-8001-f2cd832f225c

2025-11-28 07:10:10,063 - distributed.scheduler - INFO - Receive client connection: Client-46b27cf7-cc29-11f0-8001-f2cd832f225c

2025-11-28 07:01:22,669 - distributed.scheduler - INFO - Close client connection: Client-e01cc243-cc27-11f0-8001-f2cd832f225c

2025-11-28 07:01:22,667 - distributed.scheduler - INFO - Remove client Client-e01cc243-cc27-11f0-8001-f2cd832f225c

2025-11-28 07:01:22,665 - distributed.scheduler - INFO - Remove client Client-e01cc243-cc27-11f0-8001-f2cd832f225c

2025-11-28 07:00:08,450 - distributed.scheduler - INFO - Receive client connection: Client-e01cc243-cc27-11f0-8001-f2cd832f225c

2025-11-28 06:50:43,428 - distributed.scheduler - INFO - Close client connection: Client-799b2662-cc26-11f0-8001-f2cd832f225c

2025-11-28 06:50:43,426 - distributed.scheduler - INFO - Remove client Client-799b2662-cc26-11f0-8001-f2cd832f225c

2025-11-28 06:50:43,425 - distributed.scheduler - INFO - Remove client Client-799b2662-cc26-11f0-8001-f2cd832f225c

2025-11-28 06:50:06,984 - distributed.scheduler - INFO - Receive client connection: Client-799b2662-cc26-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,247 - distributed.scheduler - INFO - Close client connection: Client-9c20cc5f-cc11-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,247 - distributed.scheduler - INFO - Close client connection: Client-02950539-cc13-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,247 - distributed.scheduler - INFO - Close client connection: Client-35fe0b48-cc17-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,247 - distributed.scheduler - INFO - Close client connection: Client-690adb3d-cc14-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,246 - distributed.scheduler - INFO - Close client connection: Client-cf83fe05-cc15-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,246 - distributed.scheduler - INFO - Close client connection: Client-03144094-cc1a-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,246 - distributed.scheduler - INFO - Close client connection: Client-9c953fdf-cc18-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,246 - distributed.scheduler - INFO - Close client connection: Client-698de135-cc1b-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,245 - distributed.scheduler - INFO - Close client connection: Client-367d4672-cc1e-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,245 - distributed.scheduler - INFO - Close client connection: Client-d002b3f2-cc1c-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,245 - distributed.scheduler - INFO - Close client connection: Client-9cfa0f08-cc1f-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,244 - distributed.scheduler - INFO - Close client connection: Client-ebf6dd1d-cc20-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,244 - distributed.scheduler - INFO - Close client connection: Client-462947f4-cc22-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,244 - distributed.scheduler - INFO - Close client connection: Client-ac987742-cc23-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,242 - distributed.scheduler - INFO - Remove client Client-9c20cc5f-cc11-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,242 - distributed.scheduler - INFO - Remove client Client-02950539-cc13-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,241 - distributed.scheduler - INFO - Remove client Client-35fe0b48-cc17-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,241 - distributed.scheduler - INFO - Remove client Client-690adb3d-cc14-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,241 - distributed.scheduler - INFO - Remove client Client-cf83fe05-cc15-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,240 - distributed.scheduler - INFO - Remove client Client-03144094-cc1a-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,240 - distributed.scheduler - INFO - Remove client Client-9c953fdf-cc18-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,239 - distributed.scheduler - INFO - Remove client Client-698de135-cc1b-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,239 - distributed.scheduler - INFO - Remove client Client-367d4672-cc1e-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,238 - distributed.scheduler - INFO - Remove client Client-d002b3f2-cc1c-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,237 - distributed.scheduler - INFO - Remove client Client-9cfa0f08-cc1f-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,232 - distributed.scheduler - INFO - Remove client Client-ebf6dd1d-cc20-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,231 - distributed.scheduler - INFO - Remove client Client-462947f4-cc22-11f0-8001-f2cd832f225c

2025-11-28 06:40:44,230 - distributed.scheduler - INFO - Remove client Client-ac987742-cc23-11f0-8001-f2cd832f225c

2025-11-28 06:40:43,475 - distributed.scheduler - INFO - Close client connection: Client-13170d7d-cc25-11f0-8001-f2cd832f225c

2025-11-28 06:40:43,474 - distributed.scheduler - INFO - Remove client Client-13170d7d-cc25-11f0-8001-f2cd832f225c

2025-11-28 06:40:43,473 - distributed.scheduler - INFO - Remove client Client-13170d7d-cc25-11f0-8001-f2cd832f225c

2025-11-28 06:40:05,484 - distributed.scheduler - INFO - Receive client connection: Client-13170d7d-cc25-11f0-8001-f2cd832f225c

2025-11-28 06:30:04,028 - distributed.scheduler - INFO - Receive client connection: Client-ac987742-cc23-11f0-8001-f2cd832f225c

2025-11-28 06:20:02,672 - distributed.scheduler - INFO - Receive client connection: Client-462947f4-cc22-11f0-8001-f2cd832f225c

2025-11-28 06:10:21,849 - distributed.scheduler - INFO - Receive client connection: Client-ebf6dd1d-cc20-11f0-8001-f2cd832f225c

2025-11-28 06:00:59,834 - distributed.scheduler - INFO - Receive client connection: Client-9cfa0f08-cc1f-11f0-8001-f2cd832f225c

2025-11-28 05:50:58,391 - distributed.scheduler - INFO - Receive client connection: Client-367d4672-cc1e-11f0-8001-f2cd832f225c

2025-11-28 05:40:56,968 - distributed.scheduler - INFO - Receive client connection: Client-d002b3f2-cc1c-11f0-8001-f2cd832f225c

2025-11-28 05:30:55,580 - distributed.scheduler - INFO - Receive client connection: Client-698de135-cc1b-11f0-8001-f2cd832f225c

2025-11-28 05:20:54,149 - distributed.scheduler - INFO - Receive client connection: Client-03144094-cc1a-11f0-8001-f2cd832f225c

2025-11-28 05:10:52,697 - distributed.scheduler - INFO - Receive client connection: Client-9c953fdf-cc18-11f0-8001-f2cd832f225c

2025-11-28 05:00:51,080 - distributed.scheduler - INFO - Receive client connection: Client-35fe0b48-cc17-11f0-8001-f2cd832f225c

2025-11-28 04:50:49,663 - distributed.scheduler - INFO - Receive client connection: Client-cf83fe05-cc15-11f0-8001-f2cd832f225c

2025-11-28 04:40:48,243 - distributed.scheduler - INFO - Receive client connection: Client-690adb3d-cc14-11f0-8001-f2cd832f225c

2025-11-28 04:30:46,842 - distributed.scheduler - INFO - Receive client connection: Client-02950539-cc13-11f0-8001-f2cd832f225c

2025-11-28 04:20:45,471 - distributed.scheduler - INFO - Receive client connection: Client-9c20cc5f-cc11-11f0-8001-f2cd832f225c

2025-11-28 04:11:19,570 - distributed.scheduler - INFO - Close client connection: Client-35999a64-cc10-11f0-8001-f2cd832f225c

2025-11-28 04:11:19,569 - distributed.scheduler - INFO - Remove client Client-35999a64-cc10-11f0-8001-f2cd832f225c

2025-11-28 04:11:19,567 - distributed.scheduler - INFO - Remove client Client-35999a64-cc10-11f0-8001-f2cd832f225c

2025-11-28 04:10:43,955 - distributed.scheduler - INFO - Receive client connection: Client-35999a64-cc10-11f0-8001-f2cd832f225c

2025-11-28 04:01:47,356 - distributed.scheduler - INFO - Close client connection: Client-cf1ebf82-cc0e-11f0-8001-f2cd832f225c

2025-11-28 04:01:47,355 - distributed.scheduler - INFO - Remove client Client-cf1ebf82-cc0e-11f0-8001-f2cd832f225c

2025-11-28 04:01:47,353 - distributed.scheduler - INFO - Remove client Client-cf1ebf82-cc0e-11f0-8001-f2cd832f225c

2025-11-28 04:00:47,845 - distributed.scheduler - INFO - Receive client connection: Client-worker-d2425638-cc0e-11f0-852f-ba070f366fac

2025-11-28 04:00:42,526 - distributed.scheduler - INFO - Receive client connection: Client-cf1ebf82-cc0e-11f0-8001-f2cd832f225c

2025-11-28 03:51:17,808 - distributed.scheduler - INFO - Close client connection: Client-689dcaf5-cc0d-11f0-8001-f2cd832f225c

2025-11-28 03:51:17,807 - distributed.scheduler - INFO - Remove client Client-689dcaf5-cc0d-11f0-8001-f2cd832f225c

2025-11-28 03:51:17,805 - distributed.scheduler - INFO - Remove client Client-689dcaf5-cc0d-11f0-8001-f2cd832f225c

2025-11-28 03:50:41,060 - distributed.scheduler - INFO - Receive client connection: Client-689dcaf5-cc0d-11f0-8001-f2cd832f225c

2025-11-28 03:41:16,134 - distributed.scheduler - INFO - Close client connection: Client-0225134c-cc0c-11f0-8001-f2cd832f225c

2025-11-28 03:41:16,134 - distributed.scheduler - INFO - Remove client Client-0225134c-cc0c-11f0-8001-f2cd832f225c

2025-11-28 03:41:16,133 - distributed.scheduler - INFO - Remove client Client-0225134c-cc0c-11f0-8001-f2cd832f225c

2025-11-28 03:40:39,645 - distributed.scheduler - INFO - Receive client connection: Client-0225134c-cc0c-11f0-8001-f2cd832f225c

2025-11-28 03:34:13,979 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:33795 failed: OSError: Timed out trying to connect to tcp://172.21.159.210:33795 after 30 s

2025-11-28 03:34:13,931 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33795 after 30 s')

2025-11-28 03:34:13,925 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33795 after 30 s')

2025-11-28 03:34:13,925 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33795 after 30 s')

2025-11-28 03:34:13,923 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33795 after 30 s')

2025-11-28 03:34:13,922 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33795 after 30 s')

2025-11-28 03:34:13,920 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:43181 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33795 after 30 s')

2025-11-28 03:34:13,919 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33795 after 30 s')

2025-11-28 03:34:13,877 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33795 after 30 s')

2025-11-28 03:34:13,856 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:40527 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33795 after 30 s')

2025-11-28 03:33:44,762 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:43399

2025-11-28 03:33:44,760 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:43399', status: closed, memory: 0, processing: 0>

2025-11-28 03:33:43,763 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:33795', status: closed, memory: 0, processing: 0>

2025-11-28 03:33:32,005 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:33795

2025-11-28 03:33:32,003 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:33795', status: closed, memory: 0, processing: 0>

2025-11-28 03:33:31,074 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:45707', status: closed, memory: 0, processing: 0>

2025-11-28 03:33:20,061 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:39635 failed: OSError: Timed out trying to connect to tcp://172.21.159.210:39635 after 30 s

2025-11-28 03:33:20,041 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:35449 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:39635 after 30 s')

2025-11-28 03:33:20,001 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:39635 after 30 s')

2025-11-28 03:33:19,945 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:40527 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:39635 after 30 s')

2025-11-28 03:33:19,945 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:39635 after 30 s')

2025-11-28 03:33:19,943 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:39635 after 30 s')

2025-11-28 03:33:19,942 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:39635 after 30 s')

2025-11-28 03:33:19,942 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41593 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:39635 after 30 s')

2025-11-28 03:33:19,907 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:40579 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:39635 after 30 s')

2025-11-28 03:33:19,888 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41965 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:39635 after 30 s')

2025-11-28 03:32:50,651 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:45707

2025-11-28 03:32:50,648 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:45707', status: closed, memory: 0, processing: 0>

2025-11-28 03:32:49,772 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:39635', status: closed, memory: 0, processing: 0>

2025-11-28 03:32:38,182 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:42471 failed: OSError: Timed out trying to connect to tcp://172.21.159.210:42471 after 30 s

2025-11-28 03:32:38,172 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41965 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:42471 after 30 s')

2025-11-28 03:32:38,159 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:42471 after 30 s')

2025-11-28 03:32:37,857 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:42471 after 30 s')

2025-11-28 03:32:37,794 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:42471 after 30 s')

2025-11-28 03:32:37,740 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:42471 after 30 s')

2025-11-28 03:32:37,738 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:42471 after 30 s')

2025-11-28 03:32:37,738 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:46261 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:42471 after 30 s')

2025-11-28 03:32:37,737 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41593 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:42471 after 30 s')

2025-11-28 03:32:37,737 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37517 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:42471 after 30 s')

2025-11-28 03:32:37,736 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:40579 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:42471 after 30 s')

2025-11-28 03:32:37,703 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:35449 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:42471 after 30 s')

2025-11-28 03:32:37,685 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:42471 after 30 s')

2025-11-28 03:32:08,516 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:39635

2025-11-28 03:32:08,513 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:39635', status: closed, memory: 0, processing: 0>

2025-11-28 03:32:07,580 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:42471', status: closed, memory: 0, processing: 0>

2025-11-28 03:31:46,212 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:35563 failed: OSError: Timed out trying to connect to tcp://172.21.159.210:35563 after 30 s

2025-11-28 03:31:46,059 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], run-07316c3de4e05d5d79633c8546dd0fc6 NoneType: None

2025-11-28 03:31:46,059 - distributed.scheduler - ERROR - Couldn't gather keys {'run-07316c3de4e05d5d79633c8546dd0fc6': []} state: ['processing'] workers: [] NoneType: None

2025-11-28 03:31:46,038 - distributed.scheduler - INFO - Close client connection: Client-9ba61e12-cc0a-11f0-8001-f2cd832f225c

2025-11-28 03:31:46,037 - distributed.scheduler - INFO - Remove client Client-9ba61e12-cc0a-11f0-8001-f2cd832f225c

2025-11-28 03:31:46,037 - distributed.scheduler - INFO - Remove client Client-9ba61e12-cc0a-11f0-8001-f2cd832f225c

2025-11-28 03:31:46,035 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:35563 after 30 s')

2025-11-28 03:31:46,034 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:35563 after 30 s')

2025-11-28 03:31:46,033 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:40579 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:35563 after 30 s')

2025-11-28 03:31:45,973 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:35449 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:35563 after 30 s')

2025-11-28 03:31:45,941 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41965 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:35563 after 30 s')

2025-11-28 03:31:45,936 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:35563 after 30 s')

2025-11-28 03:31:45,915 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:43181 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:35563 after 30 s')

2025-11-28 03:31:16,743 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:42471

2025-11-28 03:31:16,742 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:42471', status: closed, memory: 0, processing: 0>

2025-11-28 03:31:15,786 - distributed.scheduler - INFO - Close client connection: Client-worker-6694b6a8-cb6b-11f0-8470-ba070f366fac

2025-11-28 03:31:15,755 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:35563', status: closed, memory: 0, processing: 0>

2025-11-28 03:31:15,754 - distributed.scheduler - INFO - Remove client Client-worker-6694b6a8-cb6b-11f0-8470-ba070f366fac

2025-11-28 03:30:38,187 - distributed.scheduler - INFO - Receive client connection: Client-9ba61e12-cc0a-11f0-8001-f2cd832f225c

2025-11-28 03:21:12,834 - distributed.scheduler - INFO - Close client connection: Client-351ee312-cc09-11f0-8001-f2cd832f225c

2025-11-28 03:21:12,832 - distributed.scheduler - INFO - Remove client Client-351ee312-cc09-11f0-8001-f2cd832f225c

2025-11-28 03:21:12,831 - distributed.scheduler - INFO - Remove client Client-351ee312-cc09-11f0-8001-f2cd832f225c

2025-11-28 03:20:36,668 - distributed.scheduler - INFO - Receive client connection: Client-351ee312-cc09-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,404 - distributed.scheduler - INFO - Close client connection: Client-2051d977-cbb8-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,404 - distributed.scheduler - INFO - Close client connection: Client-53cb6bd9-cbbc-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,404 - distributed.scheduler - INFO - Close client connection: Client-86d0e57c-cbb9-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,404 - distributed.scheduler - INFO - Close client connection: Client-ed514961-cbba-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,403 - distributed.scheduler - INFO - Close client connection: Client-ba3bf7e6-cbbd-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,403 - distributed.scheduler - INFO - Close client connection: Client-20c1a893-cbbf-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,403 - distributed.scheduler - INFO - Close client connection: Client-8748427c-cbc0-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,403 - distributed.scheduler - INFO - Close client connection: Client-edca6f17-cbc1-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,403 - distributed.scheduler - INFO - Close client connection: Client-543d08fe-cbc3-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,403 - distributed.scheduler - INFO - Close client connection: Client-babe0a85-cbc4-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,403 - distributed.scheduler - INFO - Close client connection: Client-2139676b-cbc6-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,403 - distributed.scheduler - INFO - Close client connection: Client-87acfdf2-cbc7-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,403 - distributed.scheduler - INFO - Close client connection: Client-ee3886bb-cbc8-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,403 - distributed.scheduler - INFO - Close client connection: Client-54bc68bd-cbca-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,402 - distributed.scheduler - INFO - Close client connection: Client-bb3bcefd-cbcb-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,402 - distributed.scheduler - INFO - Close client connection: Client-21b9dde0-cbcd-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,402 - distributed.scheduler - INFO - Close client connection: Client-88454974-cbce-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,402 - distributed.scheduler - INFO - Close client connection: Client-eecc647e-cbcf-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,402 - distributed.scheduler - INFO - Close client connection: Client-554f7c91-cbd1-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,402 - distributed.scheduler - INFO - Close client connection: Client-bbd13c7d-cbd2-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,402 - distributed.scheduler - INFO - Close client connection: Client-224dbd8a-cbd4-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,402 - distributed.scheduler - INFO - Close client connection: Client-88cf76e7-cbd5-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,402 - distributed.scheduler - INFO - Close client connection: Client-ef55841f-cbd6-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,402 - distributed.scheduler - INFO - Close client connection: Client-bc54dc1d-cbd9-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,401 - distributed.scheduler - INFO - Close client connection: Client-55e1242e-cbd8-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,401 - distributed.scheduler - INFO - Close client connection: Client-22d2b3da-cbdb-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,401 - distributed.scheduler - INFO - Close client connection: Client-89521232-cbdc-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,401 - distributed.scheduler - INFO - Close client connection: Client-efcdd4bc-cbdd-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,401 - distributed.scheduler - INFO - Close client connection: Client-56519b3b-cbdf-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,401 - distributed.scheduler - INFO - Close client connection: Client-bcc29cd7-cbe0-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,401 - distributed.scheduler - INFO - Close client connection: Client-233248a4-cbe2-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,401 - distributed.scheduler - INFO - Close client connection: Client-89a86e89-cbe3-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,401 - distributed.scheduler - INFO - Close client connection: Client-f01ea021-cbe4-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,401 - distributed.scheduler - INFO - Close client connection: Client-32c4acc7-cbe6-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,400 - distributed.scheduler - INFO - Close client connection: Client-9932a9c1-cbe7-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,400 - distributed.scheduler - INFO - Close client connection: Client-ffb9319e-cbe8-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,400 - distributed.scheduler - INFO - Close client connection: Client-66b1e676-cbea-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,400 - distributed.scheduler - INFO - Close client connection: Client-ccaba157-cbeb-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,400 - distributed.scheduler - INFO - Close client connection: Client-333455a7-cbed-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,400 - distributed.scheduler - INFO - Close client connection: Client-99b1723b-cbee-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,400 - distributed.scheduler - INFO - Close client connection: Client-0038a67c-cbf0-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,400 - distributed.scheduler - INFO - Close client connection: Client-66bdfb7c-cbf1-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,400 - distributed.scheduler - INFO - Close client connection: Client-cd371ae8-cbf2-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,400 - distributed.scheduler - INFO - Close client connection: Client-33b51421-cbf4-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,399 - distributed.scheduler - INFO - Close client connection: Client-9a33c97c-cbf5-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,399 - distributed.scheduler - INFO - Close client connection: Client-00bb9443-cbf7-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,399 - distributed.scheduler - INFO - Close client connection: Client-6745a965-cbf8-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,399 - distributed.scheduler - INFO - Close client connection: Client-34322636-cbfb-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,399 - distributed.scheduler - INFO - Close client connection: Client-cdb747a0-cbf9-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,399 - distributed.scheduler - INFO - Close client connection: Client-9ab4d242-cbfc-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,398 - distributed.scheduler - INFO - Close client connection: Client-012ed359-cbfe-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,398 - distributed.scheduler - INFO - Close client connection: Client-67ad8ea5-cbff-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,397 - distributed.scheduler - INFO - Close client connection: Client-ce1cf801-cc00-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,397 - distributed.scheduler - INFO - Close client connection: Client-3497d5b1-cc02-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,397 - distributed.scheduler - INFO - Close client connection: Client-9b1a6762-cc03-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,397 - distributed.scheduler - INFO - Close client connection: Client-681835a6-cc06-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,397 - distributed.scheduler - INFO - Close client connection: Client-019a0fd0-cc05-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,393 - distributed.scheduler - INFO - Remove client Client-2051d977-cbb8-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,393 - distributed.scheduler - INFO - Remove client Client-53cb6bd9-cbbc-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,393 - distributed.scheduler - INFO - Remove client Client-86d0e57c-cbb9-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,393 - distributed.scheduler - INFO - Remove client Client-ed514961-cbba-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,392 - distributed.scheduler - INFO - Remove client Client-ba3bf7e6-cbbd-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,392 - distributed.scheduler - INFO - Remove client Client-20c1a893-cbbf-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,392 - distributed.scheduler - INFO - Remove client Client-8748427c-cbc0-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,391 - distributed.scheduler - INFO - Remove client Client-edca6f17-cbc1-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,391 - distributed.scheduler - INFO - Remove client Client-543d08fe-cbc3-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,391 - distributed.scheduler - INFO - Remove client Client-babe0a85-cbc4-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,391 - distributed.scheduler - INFO - Remove client Client-2139676b-cbc6-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,390 - distributed.scheduler - INFO - Remove client Client-87acfdf2-cbc7-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,390 - distributed.scheduler - INFO - Remove client Client-ee3886bb-cbc8-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,390 - distributed.scheduler - INFO - Remove client Client-54bc68bd-cbca-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,389 - distributed.scheduler - INFO - Remove client Client-bb3bcefd-cbcb-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,389 - distributed.scheduler - INFO - Remove client Client-21b9dde0-cbcd-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,389 - distributed.scheduler - INFO - Remove client Client-88454974-cbce-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,389 - distributed.scheduler - INFO - Remove client Client-eecc647e-cbcf-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,388 - distributed.scheduler - INFO - Remove client Client-554f7c91-cbd1-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,388 - distributed.scheduler - INFO - Remove client Client-bbd13c7d-cbd2-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,388 - distributed.scheduler - INFO - Remove client Client-224dbd8a-cbd4-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,387 - distributed.scheduler - INFO - Remove client Client-88cf76e7-cbd5-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,387 - distributed.scheduler - INFO - Remove client Client-ef55841f-cbd6-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,387 - distributed.scheduler - INFO - Remove client Client-bc54dc1d-cbd9-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,387 - distributed.scheduler - INFO - Remove client Client-55e1242e-cbd8-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,386 - distributed.scheduler - INFO - Remove client Client-22d2b3da-cbdb-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,386 - distributed.scheduler - INFO - Remove client Client-89521232-cbdc-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,385 - distributed.scheduler - INFO - Remove client Client-efcdd4bc-cbdd-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,385 - distributed.scheduler - INFO - Remove client Client-56519b3b-cbdf-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,385 - distributed.scheduler - INFO - Remove client Client-bcc29cd7-cbe0-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,385 - distributed.scheduler - INFO - Remove client Client-233248a4-cbe2-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,384 - distributed.scheduler - INFO - Remove client Client-89a86e89-cbe3-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,384 - distributed.scheduler - INFO - Remove client Client-f01ea021-cbe4-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,384 - distributed.scheduler - INFO - Remove client Client-32c4acc7-cbe6-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,384 - distributed.scheduler - INFO - Remove client Client-9932a9c1-cbe7-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,383 - distributed.scheduler - INFO - Remove client Client-ffb9319e-cbe8-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,383 - distributed.scheduler - INFO - Remove client Client-66b1e676-cbea-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,383 - distributed.scheduler - INFO - Remove client Client-ccaba157-cbeb-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,382 - distributed.scheduler - INFO - Remove client Client-333455a7-cbed-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,382 - distributed.scheduler - INFO - Remove client Client-99b1723b-cbee-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,382 - distributed.scheduler - INFO - Remove client Client-0038a67c-cbf0-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,381 - distributed.scheduler - INFO - Remove client Client-66bdfb7c-cbf1-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,381 - distributed.scheduler - INFO - Remove client Client-cd371ae8-cbf2-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,380 - distributed.scheduler - INFO - Remove client Client-33b51421-cbf4-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,380 - distributed.scheduler - INFO - Remove client Client-9a33c97c-cbf5-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,380 - distributed.scheduler - INFO - Remove client Client-00bb9443-cbf7-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,379 - distributed.scheduler - INFO - Remove client Client-6745a965-cbf8-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,379 - distributed.scheduler - INFO - Remove client Client-34322636-cbfb-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,378 - distributed.scheduler - INFO - Remove client Client-cdb747a0-cbf9-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,378 - distributed.scheduler - INFO - Remove client Client-9ab4d242-cbfc-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,364 - distributed.scheduler - INFO - Remove client Client-012ed359-cbfe-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,363 - distributed.scheduler - INFO - Remove client Client-67ad8ea5-cbff-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,362 - distributed.scheduler - INFO - Remove client Client-ce1cf801-cc00-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,362 - distributed.scheduler - INFO - Remove client Client-3497d5b1-cc02-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,361 - distributed.scheduler - INFO - Remove client Client-9b1a6762-cc03-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,360 - distributed.scheduler - INFO - Remove client Client-681835a6-cc06-11f0-8001-f2cd832f225c

2025-11-28 03:11:10,359 - distributed.scheduler - INFO - Remove client Client-019a0fd0-cc05-11f0-8001-f2cd832f225c

2025-11-28 03:11:09,707 - distributed.scheduler - INFO - Close client connection: Client-ce9728b6-cc07-11f0-8001-f2cd832f225c

2025-11-28 03:11:09,705 - distributed.scheduler - INFO - Remove client Client-ce9728b6-cc07-11f0-8001-f2cd832f225c

2025-11-28 03:11:09,703 - distributed.scheduler - INFO - Remove client Client-ce9728b6-cc07-11f0-8001-f2cd832f225c

2025-11-28 03:10:35,157 - distributed.scheduler - INFO - Receive client connection: Client-ce9728b6-cc07-11f0-8001-f2cd832f225c

2025-11-28 03:00:33,699 - distributed.scheduler - INFO - Receive client connection: Client-681835a6-cc06-11f0-8001-f2cd832f225c

2025-11-28 02:50:32,249 - distributed.scheduler - INFO - Receive client connection: Client-019a0fd0-cc05-11f0-8001-f2cd832f225c

2025-11-28 02:40:30,787 - distributed.scheduler - INFO - Receive client connection: Client-9b1a6762-cc03-11f0-8001-f2cd832f225c

2025-11-28 02:30:29,311 - distributed.scheduler - INFO - Receive client connection: Client-3497d5b1-cc02-11f0-8001-f2cd832f225c

2025-11-28 02:20:27,912 - distributed.scheduler - INFO - Receive client connection: Client-ce1cf801-cc00-11f0-8001-f2cd832f225c

2025-11-28 02:10:26,526 - distributed.scheduler - INFO - Receive client connection: Client-67ad8ea5-cbff-11f0-8001-f2cd832f225c

2025-11-28 02:00:25,059 - distributed.scheduler - INFO - Receive client connection: Client-012ed359-cbfe-11f0-8001-f2cd832f225c

2025-11-28 01:50:23,653 - distributed.scheduler - INFO - Receive client connection: Client-9ab4d242-cbfc-11f0-8001-f2cd832f225c

2025-11-28 01:40:22,163 - distributed.scheduler - INFO - Receive client connection: Client-34322636-cbfb-11f0-8001-f2cd832f225c

2025-11-28 01:30:20,734 - distributed.scheduler - INFO - Receive client connection: Client-cdb747a0-cbf9-11f0-8001-f2cd832f225c

2025-11-28 01:20:19,368 - distributed.scheduler - INFO - Receive client connection: Client-6745a965-cbf8-11f0-8001-f2cd832f225c

2025-11-28 01:10:17,834 - distributed.scheduler - INFO - Receive client connection: Client-00bb9443-cbf7-11f0-8001-f2cd832f225c

2025-11-28 01:00:16,323 - distributed.scheduler - INFO - Receive client connection: Client-9a33c97c-cbf5-11f0-8001-f2cd832f225c

2025-11-28 00:50:14,906 - distributed.scheduler - INFO - Receive client connection: Client-33b51421-cbf4-11f0-8001-f2cd832f225c

2025-11-28 00:40:13,417 - distributed.scheduler - INFO - Receive client connection: Client-cd371ae8-cbf2-11f0-8001-f2cd832f225c

2025-11-28 00:30:11,998 - distributed.scheduler - INFO - Receive client connection: Client-66bdfb7c-cbf1-11f0-8001-f2cd832f225c

2025-11-28 00:20:10,504 - distributed.scheduler - INFO - Receive client connection: Client-0038a67c-cbf0-11f0-8001-f2cd832f225c

2025-11-28 00:10:08,988 - distributed.scheduler - INFO - Receive client connection: Client-99b1723b-cbee-11f0-8001-f2cd832f225c

2025-11-28 00:00:07,546 - distributed.scheduler - INFO - Receive client connection: Client-333455a7-cbed-11f0-8001-f2cd832f225c

2025-11-27 23:50:06,031 - distributed.scheduler - INFO - Receive client connection: Client-ccaba157-cbeb-11f0-8001-f2cd832f225c

2025-11-27 23:40:05,468 - distributed.scheduler - INFO - Receive client connection: Client-66b1e676-cbea-11f0-8001-f2cd832f225c

2025-11-27 23:30:03,188 - distributed.scheduler - INFO - Receive client connection: Client-ffb9319e-cbe8-11f0-8001-f2cd832f225c

2025-11-27 23:20:01,677 - distributed.scheduler - INFO - Receive client connection: Client-9932a9c1-cbe7-11f0-8001-f2cd832f225c

2025-11-27 23:10:00,338 - distributed.scheduler - INFO - Receive client connection: Client-32c4acc7-cbe6-11f0-8001-f2cd832f225c

2025-11-27 23:00:59,027 - distributed.scheduler - INFO - Receive client connection: Client-f01ea021-cbe4-11f0-8001-f2cd832f225c

2025-11-27 22:50:57,624 - distributed.scheduler - INFO - Receive client connection: Client-89a86e89-cbe3-11f0-8001-f2cd832f225c

2025-11-27 22:40:56,229 - distributed.scheduler - INFO - Receive client connection: Client-233248a4-cbe2-11f0-8001-f2cd832f225c

2025-11-27 22:30:54,861 - distributed.scheduler - INFO - Receive client connection: Client-bcc29cd7-cbe0-11f0-8001-f2cd832f225c

2025-11-27 22:20:53,507 - distributed.scheduler - INFO - Receive client connection: Client-56519b3b-cbdf-11f0-8001-f2cd832f225c

2025-11-27 22:10:52,019 - distributed.scheduler - INFO - Receive client connection: Client-efcdd4bc-cbdd-11f0-8001-f2cd832f225c

2025-11-27 22:00:50,582 - distributed.scheduler - INFO - Receive client connection: Client-89521232-cbdc-11f0-8001-f2cd832f225c

2025-11-27 21:50:49,120 - distributed.scheduler - INFO - Receive client connection: Client-22d2b3da-cbdb-11f0-8001-f2cd832f225c

2025-11-27 21:40:47,673 - distributed.scheduler - INFO - Receive client connection: Client-bc54dc1d-cbd9-11f0-8001-f2cd832f225c

2025-11-27 21:30:46,289 - distributed.scheduler - INFO - Receive client connection: Client-55e1242e-cbd8-11f0-8001-f2cd832f225c

2025-11-27 21:20:44,743 - distributed.scheduler - INFO - Receive client connection: Client-ef55841f-cbd6-11f0-8001-f2cd832f225c

2025-11-27 21:10:43,247 - distributed.scheduler - INFO - Receive client connection: Client-88cf76e7-cbd5-11f0-8001-f2cd832f225c

2025-11-27 21:00:41,774 - distributed.scheduler - INFO - Receive client connection: Client-224dbd8a-cbd4-11f0-8001-f2cd832f225c

2025-11-27 20:50:40,331 - distributed.scheduler - INFO - Receive client connection: Client-bbd13c7d-cbd2-11f0-8001-f2cd832f225c

2025-11-27 20:40:38,856 - distributed.scheduler - INFO - Receive client connection: Client-554f7c91-cbd1-11f0-8001-f2cd832f225c

2025-11-27 20:30:37,375 - distributed.scheduler - INFO - Receive client connection: Client-eecc647e-cbcf-11f0-8001-f2cd832f225c

2025-11-27 20:20:35,862 - distributed.scheduler - INFO - Receive client connection: Client-88454974-cbce-11f0-8001-f2cd832f225c

2025-11-27 20:10:34,327 - distributed.scheduler - INFO - Receive client connection: Client-21b9dde0-cbcd-11f0-8001-f2cd832f225c

2025-11-27 20:00:32,875 - distributed.scheduler - INFO - Receive client connection: Client-bb3bcefd-cbcb-11f0-8001-f2cd832f225c

2025-11-27 19:50:31,415 - distributed.scheduler - INFO - Receive client connection: Client-54bc68bd-cbca-11f0-8001-f2cd832f225c

2025-11-27 19:40:29,928 - distributed.scheduler - INFO - Receive client connection: Client-ee3886bb-cbc8-11f0-8001-f2cd832f225c

2025-11-27 19:30:28,390 - distributed.scheduler - INFO - Receive client connection: Client-87acfdf2-cbc7-11f0-8001-f2cd832f225c

2025-11-27 19:20:27,008 - distributed.scheduler - INFO - Receive client connection: Client-2139676b-cbc6-11f0-8001-f2cd832f225c

2025-11-27 19:10:25,574 - distributed.scheduler - INFO - Receive client connection: Client-babe0a85-cbc4-11f0-8001-f2cd832f225c

2025-11-27 19:00:24,099 - distributed.scheduler - INFO - Receive client connection: Client-543d08fe-cbc3-11f0-8001-f2cd832f225c

2025-11-27 18:50:22,729 - distributed.scheduler - INFO - Receive client connection: Client-edca6f17-cbc1-11f0-8001-f2cd832f225c

2025-11-27 18:40:21,252 - distributed.scheduler - INFO - Receive client connection: Client-8748427c-cbc0-11f0-8001-f2cd832f225c

2025-11-27 18:30:19,746 - distributed.scheduler - INFO - Receive client connection: Client-20c1a893-cbbf-11f0-8001-f2cd832f225c

2025-11-27 18:20:18,241 - distributed.scheduler - INFO - Receive client connection: Client-ba3bf7e6-cbbd-11f0-8001-f2cd832f225c

2025-11-27 18:10:16,879 - distributed.scheduler - INFO - Receive client connection: Client-53cb6bd9-cbbc-11f0-8001-f2cd832f225c

2025-11-27 18:00:21,189 - distributed.scheduler - INFO - Receive client connection: Client-worker-f0b93294-cbba-11f0-851a-32a59f79d364

2025-11-27 18:00:15,457 - distributed.scheduler - INFO - Receive client connection: Client-ed514961-cbba-11f0-8001-f2cd832f225c

2025-11-27 17:50:13,991 - distributed.scheduler - INFO - Receive client connection: Client-86d0e57c-cbb9-11f0-8001-f2cd832f225c

2025-11-27 17:40:12,543 - distributed.scheduler - INFO - Receive client connection: Client-2051d977-cbb8-11f0-8001-f2cd832f225c

2025-11-27 17:31:16,237 - distributed.scheduler - INFO - Close client connection: Client-b9d68e9b-cbb6-11f0-8001-f2cd832f225c

2025-11-27 17:31:16,236 - distributed.scheduler - INFO - Remove client Client-b9d68e9b-cbb6-11f0-8001-f2cd832f225c

2025-11-27 17:31:16,234 - distributed.scheduler - INFO - Remove client Client-b9d68e9b-cbb6-11f0-8001-f2cd832f225c

2025-11-27 17:30:14,356 - distributed.scheduler - INFO - Receive client connection: Client-worker-bbc7bf0e-cbb6-11f0-84ea-4ec0916b0c7e

2025-11-27 17:30:11,109 - distributed.scheduler - INFO - Receive client connection: Client-b9d68e9b-cbb6-11f0-8001-f2cd832f225c

2025-11-27 17:20:50,638 - distributed.scheduler - INFO - Close client connection: Client-535729cd-cbb5-11f0-8001-f2cd832f225c

2025-11-27 17:20:50,637 - distributed.scheduler - INFO - Remove client Client-535729cd-cbb5-11f0-8001-f2cd832f225c

2025-11-27 17:20:50,636 - distributed.scheduler - INFO - Remove client Client-535729cd-cbb5-11f0-8001-f2cd832f225c

2025-11-27 17:20:09,647 - distributed.scheduler - INFO - Receive client connection: Client-535729cd-cbb5-11f0-8001-f2cd832f225c

2025-11-27 17:10:50,065 - distributed.scheduler - INFO - Close client connection: Client-ecd69fb5-cbb3-11f0-8001-f2cd832f225c

2025-11-27 17:10:50,064 - distributed.scheduler - INFO - Remove client Client-ecd69fb5-cbb3-11f0-8001-f2cd832f225c

2025-11-27 17:10:50,062 - distributed.scheduler - INFO - Remove client Client-ecd69fb5-cbb3-11f0-8001-f2cd832f225c

2025-11-27 17:10:08,189 - distributed.scheduler - INFO - Receive client connection: Client-ecd69fb5-cbb3-11f0-8001-f2cd832f225c

2025-11-27 17:01:40,304 - distributed.scheduler - INFO - Receive client connection: Client-worker-be204c78-cbb2-11f0-83fc-32001d5d2425

2025-11-27 17:01:35,451 - distributed.scheduler - INFO - Close client connection: Client-864bcc75-cbb2-11f0-8001-f2cd832f225c

2025-11-27 17:01:35,450 - distributed.scheduler - INFO - Remove client Client-864bcc75-cbb2-11f0-8001-f2cd832f225c

2025-11-27 17:01:35,449 - distributed.scheduler - INFO - Remove client Client-864bcc75-cbb2-11f0-8001-f2cd832f225c

2025-11-27 17:01:15,669 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: OSError: Timed out trying to connect to tcp://172.21.25.102:41571 after 30 s

2025-11-27 17:01:15,669 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:45933 after 30 s')

2025-11-27 17:01:15,668 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:45933 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:45933 after 30 s

2025-11-27 17:01:15,668 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:38649 failed: OSError: Timed out trying to connect to tcp://172.21.159.248:38649 after 30 s

2025-11-27 17:01:15,659 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41593 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:45933 after 30 s')

2025-11-27 17:01:15,600 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:40527 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:38649 after 30 s')

2025-11-27 17:01:15,596 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:35449 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:41571 after 30 s')

2025-11-27 17:01:15,595 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37233 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:38649 after 30 s')

2025-11-27 17:01:15,564 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:45767 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:45933 after 30 s')

2025-11-27 17:01:15,541 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:35563 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:41571 after 30 s')

2025-11-27 17:01:15,538 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:44023 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:41571 after 30 s')

2025-11-27 17:01:15,529 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:40579 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.102:41571 after 30 s')

2025-11-27 17:00:46,411 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.102:41965

2025-11-27 17:00:46,410 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.102:41965', status: closed, memory: 0, processing: 0>

2025-11-27 17:00:45,350 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: OSError: Timed out during handshake while connecting to tcp://172.21.25.102:41571 after 30 s

2025-11-27 17:00:45,349 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:35449 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:58212 remote=tcp://172.21.12.197:35449>: Stream is closed

2025-11-27 17:00:45,349 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:40579 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:34252 remote=tcp://172.21.25.101:40579>: Stream is closed

2025-11-27 17:00:45,345 - distributed.scheduler - INFO - Close client connection: Client-worker-a31c6972-c571-11f0-8074-32001d5d2425

2025-11-27 17:00:45,332 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.102:41571', status: closed, memory: 0, processing: 0>

2025-11-27 17:00:45,330 - distributed.scheduler - INFO - Remove client Client-worker-a31c6972-c571-11f0-8074-32001d5d2425

2025-11-27 17:00:44,964 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:43181

2025-11-27 17:00:44,964 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:43181', status: closed, memory: 0, processing: 0>

2025-11-27 17:00:44,848 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.248:37517

2025-11-27 17:00:44,846 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.248:37517', status: closed, memory: 0, processing: 0>

2025-11-27 17:00:44,052 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:38649 failed: OSError: Timed out during handshake while connecting to tcp://172.21.159.248:38649 after 30 s

2025-11-27 17:00:44,050 - distributed.scheduler - INFO - Close client connection: Client-worker-c05b3df6-ca13-11f0-83fe-4ec0916b0c7e

2025-11-27 17:00:44,038 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.248:38649', status: closed, memory: 0, processing: 0>

2025-11-27 17:00:44,024 - distributed.scheduler - INFO - Remove client Client-worker-c05b3df6-ca13-11f0-83fe-4ec0916b0c7e

2025-11-27 17:00:43,721 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:45933 failed: OSError: Timed out during handshake while connecting to tcp://172.21.25.97:45933 after 30 s

2025-11-27 17:00:43,721 - distributed.scheduler - INFO - Close client connection: Client-worker-66780e90-cb6b-11f0-84ba-32a59f79d364

2025-11-27 17:00:43,686 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:45933', status: closed, memory: 0, processing: 0>

2025-11-27 17:00:43,685 - distributed.scheduler - INFO - Remove client Client-worker-66780e90-cb6b-11f0-84ba-32a59f79d364

2025-11-27 17:00:06,642 - distributed.scheduler - INFO - Receive client connection: Client-864bcc75-cbb2-11f0-8001-f2cd832f225c

2025-11-27 16:50:46,525 - distributed.scheduler - INFO - Close client connection: Client-1fd1ccdc-cbb1-11f0-8001-f2cd832f225c

2025-11-27 16:50:46,523 - distributed.scheduler - INFO - Remove client Client-1fd1ccdc-cbb1-11f0-8001-f2cd832f225c

2025-11-27 16:50:46,521 - distributed.scheduler - INFO - Remove client Client-1fd1ccdc-cbb1-11f0-8001-f2cd832f225c

2025-11-27 16:50:05,228 - distributed.scheduler - INFO - Receive client connection: Client-1fd1ccdc-cbb1-11f0-8001-f2cd832f225c

2025-11-27 16:40:47,715 - distributed.scheduler - INFO - Close client connection: Client-b94d5f51-cbaf-11f0-8001-f2cd832f225c

2025-11-27 16:40:47,713 - distributed.scheduler - INFO - Remove client Client-b94d5f51-cbaf-11f0-8001-f2cd832f225c

2025-11-27 16:40:47,712 - distributed.scheduler - INFO - Remove client Client-b94d5f51-cbaf-11f0-8001-f2cd832f225c

2025-11-27 16:40:03,731 - distributed.scheduler - INFO - Receive client connection: Client-b94d5f51-cbaf-11f0-8001-f2cd832f225c

2025-11-27 16:31:12,269 - distributed.scheduler - INFO - Close client connection: Client-52cf8535-cbae-11f0-8001-f2cd832f225c

2025-11-27 16:31:12,268 - distributed.scheduler - INFO - Remove client Client-52cf8535-cbae-11f0-8001-f2cd832f225c

2025-11-27 16:31:12,266 - distributed.scheduler - INFO - Remove client Client-52cf8535-cbae-11f0-8001-f2cd832f225c

2025-11-27 16:30:02,282 - distributed.scheduler - INFO - Receive client connection: Client-52cf8535-cbae-11f0-8001-f2cd832f225c

2025-11-27 16:20:40,377 - distributed.scheduler - INFO - Close client connection: Client-ec35e521-cbac-11f0-8001-f2cd832f225c

2025-11-27 16:20:40,376 - distributed.scheduler - INFO - Remove client Client-ec35e521-cbac-11f0-8001-f2cd832f225c

2025-11-27 16:20:40,374 - distributed.scheduler - INFO - Remove client Client-ec35e521-cbac-11f0-8001-f2cd832f225c

2025-11-27 16:20:00,665 - distributed.scheduler - INFO - Receive client connection: Client-ec35e521-cbac-11f0-8001-f2cd832f225c

2025-11-27 16:11:39,823 - distributed.scheduler - INFO - Close client connection: Client-a98e5a22-cbab-11f0-8001-f2cd832f225c

2025-11-27 16:11:39,821 - distributed.scheduler - INFO - Remove client Client-a98e5a22-cbab-11f0-8001-f2cd832f225c

2025-11-27 16:11:39,819 - distributed.scheduler - INFO - Remove client Client-a98e5a22-cbab-11f0-8001-f2cd832f225c

2025-11-27 16:10:59,333 - distributed.scheduler - INFO - Receive client connection: Client-a98e5a22-cbab-11f0-8001-f2cd832f225c

2025-11-27 16:02:12,224 - distributed.scheduler - INFO - Close client connection: Client-431d75e3-cbaa-11f0-8001-f2cd832f225c

2025-11-27 16:02:12,221 - distributed.scheduler - INFO - Remove client Client-431d75e3-cbaa-11f0-8001-f2cd832f225c

2025-11-27 16:02:12,219 - distributed.scheduler - INFO - Remove client Client-431d75e3-cbaa-11f0-8001-f2cd832f225c

2025-11-27 16:00:57,968 - distributed.scheduler - INFO - Receive client connection: Client-431d75e3-cbaa-11f0-8001-f2cd832f225c

2025-11-27 15:51:35,531 - distributed.scheduler - INFO - Close client connection: Client-dcace817-cba8-11f0-8001-f2cd832f225c

2025-11-27 15:51:35,530 - distributed.scheduler - INFO - Remove client Client-dcace817-cba8-11f0-8001-f2cd832f225c

2025-11-27 15:51:35,528 - distributed.scheduler - INFO - Remove client Client-dcace817-cba8-11f0-8001-f2cd832f225c

2025-11-27 15:50:56,606 - distributed.scheduler - INFO - Receive client connection: Client-dcace817-cba8-11f0-8001-f2cd832f225c

2025-11-27 15:41:34,938 - distributed.scheduler - INFO - Close client connection: Client-7635a3d7-cba7-11f0-8001-f2cd832f225c

2025-11-27 15:41:34,936 - distributed.scheduler - INFO - Remove client Client-7635a3d7-cba7-11f0-8001-f2cd832f225c

2025-11-27 15:41:34,935 - distributed.scheduler - INFO - Remove client Client-7635a3d7-cba7-11f0-8001-f2cd832f225c

2025-11-27 15:40:55,194 - distributed.scheduler - INFO - Receive client connection: Client-7635a3d7-cba7-11f0-8001-f2cd832f225c

2025-11-27 15:32:06,993 - distributed.scheduler - INFO - Close client connection: Client-0fb94816-cba6-11f0-8001-f2cd832f225c

2025-11-27 15:32:06,993 - distributed.scheduler - INFO - Remove client Client-0fb94816-cba6-11f0-8001-f2cd832f225c

2025-11-27 15:32:06,991 - distributed.scheduler - INFO - Remove client Client-0fb94816-cba6-11f0-8001-f2cd832f225c

2025-11-27 15:30:53,752 - distributed.scheduler - INFO - Receive client connection: Client-0fb94816-cba6-11f0-8001-f2cd832f225c

2025-11-27 15:21:32,230 - distributed.scheduler - INFO - Close client connection: Client-a93b76c3-cba4-11f0-8001-f2cd832f225c

2025-11-27 15:21:32,229 - distributed.scheduler - INFO - Remove client Client-a93b76c3-cba4-11f0-8001-f2cd832f225c

2025-11-27 15:21:32,227 - distributed.scheduler - INFO - Remove client Client-a93b76c3-cba4-11f0-8001-f2cd832f225c

2025-11-27 15:20:52,305 - distributed.scheduler - INFO - Receive client connection: Client-a93b76c3-cba4-11f0-8001-f2cd832f225c

2025-11-27 15:11:28,101 - distributed.scheduler - INFO - Close client connection: Client-42b52755-cba3-11f0-8001-f2cd832f225c

2025-11-27 15:11:28,099 - distributed.scheduler - INFO - Remove client Client-42b52755-cba3-11f0-8001-f2cd832f225c

2025-11-27 15:11:28,098 - distributed.scheduler - INFO - Remove client Client-42b52755-cba3-11f0-8001-f2cd832f225c

2025-11-27 15:10:50,807 - distributed.scheduler - INFO - Receive client connection: Client-42b52755-cba3-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,767 - distributed.scheduler - INFO - Close client connection: Client-41450f14-cb8e-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,767 - distributed.scheduler - INFO - Close client connection: Client-dad36e0a-cb8c-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,767 - distributed.scheduler - INFO - Close client connection: Client-a7bbe41e-cb8f-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,766 - distributed.scheduler - INFO - Close client connection: Client-0e3c869f-cb91-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,766 - distributed.scheduler - INFO - Close client connection: Client-74bc93c8-cb92-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,766 - distributed.scheduler - INFO - Close client connection: Client-db3f2d5a-cb93-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,766 - distributed.scheduler - INFO - Close client connection: Client-41c0d80b-cb95-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,766 - distributed.scheduler - INFO - Close client connection: Client-a8414c32-cb96-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,766 - distributed.scheduler - INFO - Close client connection: Client-0ec6b796-cb98-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,766 - distributed.scheduler - INFO - Close client connection: Client-754e10cc-cb99-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,766 - distributed.scheduler - INFO - Close client connection: Client-dbcabe1e-cb9a-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,766 - distributed.scheduler - INFO - Close client connection: Client-0f489832-cb9f-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,765 - distributed.scheduler - INFO - Close client connection: Client-42414306-cb9c-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,765 - distributed.scheduler - INFO - Close client connection: Client-75c2ea92-cba0-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,765 - distributed.scheduler - INFO - Close client connection: Client-a8c85270-cb9d-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,764 - distributed.scheduler - INFO - Remove client Client-41450f14-cb8e-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,764 - distributed.scheduler - INFO - Remove client Client-dad36e0a-cb8c-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,763 - distributed.scheduler - INFO - Remove client Client-a7bbe41e-cb8f-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,762 - distributed.scheduler - INFO - Remove client Client-0e3c869f-cb91-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,762 - distributed.scheduler - INFO - Remove client Client-74bc93c8-cb92-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,762 - distributed.scheduler - INFO - Remove client Client-db3f2d5a-cb93-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,761 - distributed.scheduler - INFO - Remove client Client-41c0d80b-cb95-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,761 - distributed.scheduler - INFO - Remove client Client-a8414c32-cb96-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,761 - distributed.scheduler - INFO - Remove client Client-0ec6b796-cb98-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,760 - distributed.scheduler - INFO - Remove client Client-754e10cc-cb99-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,760 - distributed.scheduler - INFO - Remove client Client-dbcabe1e-cb9a-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,760 - distributed.scheduler - INFO - Remove client Client-0f489832-cb9f-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,759 - distributed.scheduler - INFO - Remove client Client-42414306-cb9c-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,759 - distributed.scheduler - INFO - Remove client Client-75c2ea92-cba0-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,759 - distributed.scheduler - INFO - Remove client Client-a8c85270-cb9d-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,134 - distributed.scheduler - INFO - Close client connection: Client-dc34e06a-cba1-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,133 - distributed.scheduler - INFO - Remove client Client-dc34e06a-cba1-11f0-8001-f2cd832f225c

2025-11-27 15:01:59,132 - distributed.scheduler - INFO - Remove client Client-dc34e06a-cba1-11f0-8001-f2cd832f225c

2025-11-27 15:00:49,323 - distributed.scheduler - INFO - Receive client connection: Client-dc34e06a-cba1-11f0-8001-f2cd832f225c

2025-11-27 14:50:47,959 - distributed.scheduler - INFO - Receive client connection: Client-75c2ea92-cba0-11f0-8001-f2cd832f225c

2025-11-27 14:40:46,533 - distributed.scheduler - INFO - Receive client connection: Client-0f489832-cb9f-11f0-8001-f2cd832f225c

2025-11-27 14:30:45,066 - distributed.scheduler - INFO - Receive client connection: Client-a8c85270-cb9d-11f0-8001-f2cd832f225c

2025-11-27 14:20:43,556 - distributed.scheduler - INFO - Receive client connection: Client-42414306-cb9c-11f0-8001-f2cd832f225c

2025-11-27 14:10:42,156 - distributed.scheduler - INFO - Receive client connection: Client-dbcabe1e-cb9a-11f0-8001-f2cd832f225c

2025-11-27 14:00:40,721 - distributed.scheduler - INFO - Receive client connection: Client-754e10cc-cb99-11f0-8001-f2cd832f225c

2025-11-27 13:50:39,212 - distributed.scheduler - INFO - Receive client connection: Client-0ec6b796-cb98-11f0-8001-f2cd832f225c

2025-11-27 13:40:37,705 - distributed.scheduler - INFO - Receive client connection: Client-a8414c32-cb96-11f0-8001-f2cd832f225c

2025-11-27 13:30:36,242 - distributed.scheduler - INFO - Receive client connection: Client-41c0d80b-cb95-11f0-8001-f2cd832f225c

2025-11-27 13:20:34,771 - distributed.scheduler - INFO - Receive client connection: Client-db3f2d5a-cb93-11f0-8001-f2cd832f225c

2025-11-27 13:10:33,283 - distributed.scheduler - INFO - Receive client connection: Client-74bc93c8-cb92-11f0-8001-f2cd832f225c

2025-11-27 13:00:31,821 - distributed.scheduler - INFO - Receive client connection: Client-0e3c869f-cb91-11f0-8001-f2cd832f225c

2025-11-27 12:50:30,356 - distributed.scheduler - INFO - Receive client connection: Client-a7bbe41e-cb8f-11f0-8001-f2cd832f225c

2025-11-27 12:40:28,953 - distributed.scheduler - INFO - Receive client connection: Client-41450f14-cb8e-11f0-8001-f2cd832f225c

2025-11-27 12:30:27,589 - distributed.scheduler - INFO - Receive client connection: Client-dad36e0a-cb8c-11f0-8001-f2cd832f225c

2025-11-27 12:21:06,061 - distributed.scheduler - INFO - Close client connection: Client-7461ea33-cb8b-11f0-8001-f2cd832f225c

2025-11-27 12:21:06,060 - distributed.scheduler - INFO - Remove client Client-7461ea33-cb8b-11f0-8001-f2cd832f225c

2025-11-27 12:21:06,059 - distributed.scheduler - INFO - Remove client Client-7461ea33-cb8b-11f0-8001-f2cd832f225c

2025-11-27 12:20:26,224 - distributed.scheduler - INFO - Receive client connection: Client-7461ea33-cb8b-11f0-8001-f2cd832f225c

2025-11-27 12:11:02,690 - distributed.scheduler - INFO - Close client connection: Client-0de9cf83-cb8a-11f0-8001-f2cd832f225c

2025-11-27 12:11:02,689 - distributed.scheduler - INFO - Remove client Client-0de9cf83-cb8a-11f0-8001-f2cd832f225c

2025-11-27 12:11:02,688 - distributed.scheduler - INFO - Remove client Client-0de9cf83-cb8a-11f0-8001-f2cd832f225c

2025-11-27 12:10:24,836 - distributed.scheduler - INFO - Receive client connection: Client-0de9cf83-cb8a-11f0-8001-f2cd832f225c

2025-11-27 12:01:36,418 - distributed.scheduler - INFO - Close client connection: Client-a76ad6c9-cb88-11f0-8001-f2cd832f225c

2025-11-27 12:01:36,417 - distributed.scheduler - INFO - Remove client Client-a76ad6c9-cb88-11f0-8001-f2cd832f225c

2025-11-27 12:01:36,415 - distributed.scheduler - INFO - Remove client Client-a76ad6c9-cb88-11f0-8001-f2cd832f225c

2025-11-27 12:00:28,987 - distributed.scheduler - INFO - Receive client connection: Client-worker-aa98e53e-cb88-11f0-8583-b64bf7bee50e

2025-11-27 12:00:23,348 - distributed.scheduler - INFO - Receive client connection: Client-a76ad6c9-cb88-11f0-8001-f2cd832f225c

2025-11-27 11:50:57,346 - distributed.scheduler - INFO - Close client connection: Client-40f248ad-cb87-11f0-8001-f2cd832f225c

2025-11-27 11:50:57,345 - distributed.scheduler - INFO - Remove client Client-40f248ad-cb87-11f0-8001-f2cd832f225c

2025-11-27 11:50:57,342 - distributed.scheduler - INFO - Remove client Client-40f248ad-cb87-11f0-8001-f2cd832f225c

2025-11-27 11:50:21,940 - distributed.scheduler - INFO - Receive client connection: Client-40f248ad-cb87-11f0-8001-f2cd832f225c

2025-11-27 11:40:59,024 - distributed.scheduler - INFO - Close client connection: Client-da718cea-cb85-11f0-8001-f2cd832f225c

2025-11-27 11:40:59,023 - distributed.scheduler - INFO - Remove client Client-da718cea-cb85-11f0-8001-f2cd832f225c

2025-11-27 11:40:59,021 - distributed.scheduler - INFO - Remove client Client-da718cea-cb85-11f0-8001-f2cd832f225c

2025-11-27 11:40:20,469 - distributed.scheduler - INFO - Receive client connection: Client-da718cea-cb85-11f0-8001-f2cd832f225c

2025-11-27 11:32:01,455 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:45767

2025-11-27 11:32:01,453 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:45767', status: closed, memory: 0, processing: 0>

2025-11-27 11:32:00,395 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:41621', status: closed, memory: 0, processing: 0>

2025-11-27 11:31:50,052 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:41621

2025-11-27 11:31:50,050 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:41621', status: closed, memory: 0, processing: 0>

2025-11-27 11:31:49,101 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:35337', status: closed, memory: 0, processing: 0>

2025-11-27 11:31:38,042 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:35337

2025-11-27 11:31:38,041 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:35337', status: closed, memory: 0, processing: 0>

2025-11-27 11:31:36,998 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:36451', status: closed, memory: 0, processing: 0>

2025-11-27 11:31:34,306 - distributed.scheduler - INFO - Close client connection: Client-73f7f1e2-cb84-11f0-8001-f2cd832f225c

2025-11-27 11:31:34,305 - distributed.scheduler - INFO - Remove client Client-73f7f1e2-cb84-11f0-8001-f2cd832f225c

2025-11-27 11:31:34,304 - distributed.scheduler - INFO - Remove client Client-73f7f1e2-cb84-11f0-8001-f2cd832f225c

2025-11-27 11:31:21,324 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:36451

2025-11-27 11:31:21,322 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:36451', status: closed, memory: 0, processing: 0>

2025-11-27 11:31:20,203 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:35371', status: closed, memory: 0, processing: 0>

2025-11-27 11:30:55,825 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:35371

2025-11-27 11:30:55,824 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:35371', status: closed, memory: 0, processing: 0>

2025-11-27 11:30:54,677 - distributed.scheduler - INFO - Close client connection: Client-worker-7164ab57-cae9-11f0-84d7-b64bf7bee50e

2025-11-27 11:30:54,669 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:41969', status: closed, memory: 0, processing: 0>

2025-11-27 11:30:54,667 - distributed.scheduler - INFO - Remove client Client-worker-7164ab57-cae9-11f0-84d7-b64bf7bee50e

2025-11-27 11:30:19,047 - distributed.scheduler - INFO - Receive client connection: Client-73f7f1e2-cb84-11f0-8001-f2cd832f225c

2025-11-27 11:23:01,311 - distributed.scheduler - INFO - Close client connection: Client-5a373cf7-cb83-11f0-8001-f2cd832f225c

2025-11-27 11:23:01,310 - distributed.scheduler - INFO - Remove client Client-5a373cf7-cb83-11f0-8001-f2cd832f225c

2025-11-27 11:23:01,310 - distributed.scheduler - INFO - Remove client Client-5a373cf7-cb83-11f0-8001-f2cd832f225c

2025-11-27 11:22:26,346 - distributed.scheduler - INFO - Receive client connection: Client-5a373cf7-cb83-11f0-8001-f2cd832f225c

2025-11-27 10:00:09,310 - distributed.scheduler - INFO - Receive client connection: Client-worker-db850117-cb77-11f0-8479-de470ccc1e79

2025-11-27 09:01:00,184 - distributed.scheduler - INFO - Receive client connection: Client-worker-9812868e-cb6f-11f0-858d-f23d8172cb06

2025-11-27 08:30:59,939 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('len-agg-7abfa3451cb55bf90609c68cf18d7246', 0) NoneType: None

2025-11-27 08:30:59,939 - distributed.scheduler - ERROR - Couldn't gather keys {"('len-agg-7abfa3451cb55bf90609c68cf18d7246', 0)": []} state: ['waiting'] workers: [] NoneType: None

2025-11-27 08:30:59,160 - distributed.scheduler - INFO - Receive client connection: Client-worker-6694b6a8-cb6b-11f0-8470-ba070f366fac

2025-11-27 08:30:58,973 - distributed.scheduler - INFO - Receive client connection: Client-worker-66780e90-cb6b-11f0-84ba-32a59f79d364

2025-11-27 08:30:57,587 - distributed.scheduler - INFO - Receive client connection: Client-worker-65a2f2e1-cb6b-11f0-83d5-ee8ad16902f8

2025-11-27 08:02:53,903 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:45933

2025-11-27 08:02:53,901 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:45933', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:52,776 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:41113', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:49,160 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:44023

2025-11-27 08:02:49,158 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:44023', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:49,121 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:37233

2025-11-27 08:02:49,120 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:37233', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:48,964 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.233:46261

2025-11-27 08:02:48,962 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.233:46261', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:48,960 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:35563

2025-11-27 08:02:48,957 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:35563', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:48,169 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:33929', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:48,151 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:42133', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:48,131 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:40445', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:48,066 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.233:43111', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:38,011 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:41113

2025-11-27 08:02:38,010 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:41113', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:36,984 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:42881', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:35,497 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:33929

2025-11-27 08:02:35,496 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:33929', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:35,363 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:42133

2025-11-27 08:02:35,361 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:42133', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:35,340 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:40445

2025-11-27 08:02:35,340 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:40445', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:35,339 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.233:43111

2025-11-27 08:02:35,338 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.233:43111', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:34,575 - distributed.scheduler - INFO - Close client connection: Client-worker-6560110a-cb67-11f0-83b1-ee8ad16902f8

2025-11-27 08:02:34,571 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:42389', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:34,569 - distributed.scheduler - INFO - Remove client Client-worker-6560110a-cb67-11f0-83b1-ee8ad16902f8

2025-11-27 08:02:34,473 - distributed.scheduler - INFO - Close client connection: Client-worker-5d51675b-cb67-11f0-844e-de470ccc1e79

2025-11-27 08:02:34,469 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.233:37067', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:34,465 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:35333', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:34,465 - distributed.scheduler - INFO - Remove client Client-worker-5d51675b-cb67-11f0-844e-de470ccc1e79

2025-11-27 08:02:34,436 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:39541', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:19,142 - distributed.scheduler - INFO - Receive client connection: Client-worker-6560110a-cb67-11f0-83b1-ee8ad16902f8

2025-11-27 08:02:05,629 - distributed.scheduler - INFO - Receive client connection: Client-worker-5d51675b-cb67-11f0-844e-de470ccc1e79

2025-11-27 08:02:02,499 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:42881

2025-11-27 08:02:02,498 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:42881', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:01,481 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:36523', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:00,151 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:42389

2025-11-27 08:02:00,149 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:42389', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:00,104 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:39541

2025-11-27 08:02:00,103 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:39541', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:00,089 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.233:37067

2025-11-27 08:02:00,088 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.233:37067', status: closed, memory: 0, processing: 0>

2025-11-27 08:02:00,052 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:35333

2025-11-27 08:02:00,050 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:35333', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:59,248 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:33555', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:59,146 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:36933', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:59,113 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.233:39559', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:59,053 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:35737', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:25,136 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:36523

2025-11-27 08:01:25,136 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:36523', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:24,468 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:36933

2025-11-27 08:01:24,467 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:36933', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:24,348 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:33555

2025-11-27 08:01:24,347 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:33555', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:24,255 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:35737

2025-11-27 08:01:24,254 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:35737', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:24,137 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.233:39559

2025-11-27 08:01:24,125 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.233:39559', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:23,865 - distributed.scheduler - INFO - Close client connection: Client-worker-19cd08f0-ca89-11f0-83da-32a59f79d364

2025-11-27 08:01:23,860 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:40951', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:23,859 - distributed.scheduler - INFO - Remove client Client-worker-19cd08f0-ca89-11f0-83da-32a59f79d364

2025-11-27 08:01:23,414 - distributed.scheduler - INFO - Close client connection: Client-worker-240f5864-c9f2-11f0-834e-ba070f366fac

2025-11-27 08:01:23,414 - distributed.scheduler - INFO - Close client connection: Client-worker-c0160ca5-ca13-11f0-82a6-ee8ad16902f8

2025-11-27 08:01:23,404 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:41855', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:23,403 - distributed.scheduler - INFO - Remove client Client-worker-240f5864-c9f2-11f0-834e-ba070f366fac

2025-11-27 08:01:23,400 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:43927', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:23,378 - distributed.scheduler - INFO - Remove client Client-worker-c0160ca5-ca13-11f0-82a6-ee8ad16902f8

2025-11-27 08:01:23,187 - distributed.scheduler - INFO - Close client connection: Client-worker-3c2bc540-cad0-11f0-83a5-de470ccc1e79

2025-11-27 08:01:23,182 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:45615', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:23,181 - distributed.scheduler - INFO - Remove client Client-worker-3c2bc540-cad0-11f0-83a5-de470ccc1e79

2025-11-27 08:01:23,168 - distributed.scheduler - INFO - Close client connection: Client-worker-7bb54edd-cb4e-11f0-84de-f23d8172cb06

2025-11-27 08:01:23,149 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.233:37249', status: closed, memory: 0, processing: 0>

2025-11-27 08:01:23,148 - distributed.scheduler - INFO - Remove client Client-worker-7bb54edd-cb4e-11f0-84de-f23d8172cb06

2025-11-27 05:03:59,212 - distributed.scheduler - INFO - Receive client connection: Client-worker-7bb54edd-cb4e-11f0-84de-f23d8172cb06

2025-11-27 05:03:55,293 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:37245 failed: OSError: Timed out trying to connect to tcp://172.21.159.233:37245 after 30 s

2025-11-27 05:03:55,288 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:46147 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:37245 after 30 s')

2025-11-27 05:03:55,211 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:40527 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:37245 after 30 s')

2025-11-27 05:03:55,209 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41593 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:37245 after 30 s')

2025-11-27 05:03:55,209 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:45615 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:37245 after 30 s')

2025-11-27 05:03:55,179 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:35449 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:37245 after 30 s')

2025-11-27 05:03:55,176 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:37245 after 30 s')

2025-11-27 05:03:55,175 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:38649 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:37245 after 30 s')

2025-11-27 05:03:55,171 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:41855 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:37245 after 30 s')

2025-11-27 05:03:55,169 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:43927 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:37245 after 30 s')

2025-11-27 05:03:55,168 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:37245 after 30 s')

2025-11-27 05:03:55,136 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:41969 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:37245 after 30 s')

2025-11-27 05:03:55,113 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:37245 after 30 s')

2025-11-27 05:03:25,976 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.233:37249

2025-11-27 05:03:25,973 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.233:37249', status: closed, memory: 0, processing: 0>

2025-11-27 05:03:25,027 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.233:37245', status: closed, memory: 0, processing: 0>

2025-11-27 05:03:12,010 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:41185 failed: OSError: Timed out trying to connect to tcp://172.21.159.233:41185 after 30 s

2025-11-27 05:03:11,909 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:35449 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:41185 after 30 s')

2025-11-27 05:03:11,900 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:40951 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:41185 after 30 s')

2025-11-27 05:03:11,867 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:45615 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:41185 after 30 s')

2025-11-27 05:03:11,864 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:41969 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:41185 after 30 s')

2025-11-27 05:03:11,863 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:40527 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:41185 after 30 s')

2025-11-27 05:03:11,861 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:40579 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:41185 after 30 s')

2025-11-27 05:03:11,861 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:43927 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:41185 after 30 s')

2025-11-27 05:03:11,860 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:41185 after 30 s')

2025-11-27 05:03:11,860 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41593 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:41185 after 30 s')

2025-11-27 05:03:11,857 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:41185 after 30 s')

2025-11-27 05:03:11,829 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:41855 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:41185 after 30 s')

2025-11-27 05:03:11,824 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:41185 after 30 s')

2025-11-27 05:03:11,806 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:41185 after 30 s')

2025-11-27 05:02:42,606 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.233:37245

2025-11-27 05:02:42,603 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.233:37245', status: closed, memory: 0, processing: 0>

2025-11-27 05:02:41,700 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.233:41185', status: closed, memory: 0, processing: 0>

2025-11-27 05:02:16,580 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:39009 failed: OSError: Timed out trying to connect to tcp://172.21.159.233:39009 after 30 s

2025-11-27 05:02:16,580 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:46147 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:39009 after 30 s')

2025-11-27 05:02:16,533 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:40527 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:39009 after 30 s')

2025-11-27 05:02:16,514 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:45615 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:39009 after 30 s')

2025-11-27 05:02:16,511 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:36327 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:39009 after 30 s')

2025-11-27 05:02:16,508 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:43927 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:39009 after 30 s')

2025-11-27 05:02:16,507 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:35449 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:39009 after 30 s')

2025-11-27 05:02:16,507 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:41969 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:39009 after 30 s')

2025-11-27 05:02:16,472 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:41593 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:39009 after 30 s')

2025-11-27 05:02:16,461 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:39025 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:39009 after 30 s')

2025-11-27 05:02:16,459 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:40579 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:39009 after 30 s')

2025-11-27 05:02:16,458 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:39009 after 30 s')

2025-11-27 05:02:16,446 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:34199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:39009 after 30 s')

2025-11-27 05:01:47,260 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.233:41185

2025-11-27 05:01:47,258 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.233:41185', status: closed, memory: 0, processing: 0>

2025-11-27 05:01:46,337 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.233:39009', status: closed, memory: 0, processing: 0>

2025-11-27 05:01:30,728 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:42861 after 30 s')

2025-11-27 05:01:30,566 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:42861 failed: OSError: Timed out trying to connect to tcp://172.21.159.233:42861 after 30 s

2025-11-27 05:01:30,039 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:41969 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.233:42861 after 30 s')

2025-11-27 05:01:00,493 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.233:39009

2025-11-27 05:01:00,492 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.233:39009', status: closed, memory: 0, processing: 0>

2025-11-27 05:00:59,533 - distributed.scheduler - INFO - Close client connection: Client-worker-4d96d8ab-cae5-11f0-843e-f23d8172cb06

2025-11-27 05:00:59,480 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.233:42861', status: closed, memory: 0, processing: 0>

2025-11-27 05:00:59,479 - distributed.scheduler - INFO - Remove client Client-worker-4d96d8ab-cae5-11f0-843e-f23d8172cb06

2025-11-27 05:00:28,098 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('len-agg-7317820c0f317172a2bb4e626996c911', 0) NoneType: None

2025-11-27 05:00:28,098 - distributed.scheduler - ERROR - Couldn't gather keys {"('len-agg-7317820c0f317172a2bb4e626996c911', 0)": []} state: ['waiting'] workers: [] NoneType: None

2025-11-26 18:30:55,116 - distributed.scheduler - INFO - Receive client connection: Client-worker-0b6eca02-caf6-11f0-8424-7e5a6871f9da

2025-11-26 17:30:47,700 - distributed.scheduler - INFO - Receive client connection: Client-worker-a5379bec-caed-11f0-8411-2a2d08e6be2e

2025-11-26 17:30:46,823 - distributed.scheduler - INFO - Receive client connection: Client-worker-a4b3a3a4-caed-11f0-8484-8a1d5a425dc1

2025-11-26 17:00:43,940 - distributed.scheduler - INFO - Receive client connection: Client-worker-72151011-cae9-11f0-8592-8253c573df33

2025-11-26 17:00:43,713 - distributed.scheduler - INFO - Receive client connection: Client-worker-71f3c519-cae9-11f0-85b1-62b326af983b

2025-11-26 17:00:42,751 - distributed.scheduler - INFO - Receive client connection: Client-worker-7164ab57-cae9-11f0-84d7-b64bf7bee50e

2025-11-26 16:32:21,043 - distributed.scheduler - INFO - Receive client connection: Client-worker-7b1cc80d-cae5-11f0-83f6-46fbd1744967

2025-11-26 16:32:05,041 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:36327

2025-11-26 16:32:05,040 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:36327', status: running, memory: 0, processing: 0>

2025-11-26 16:32:04,905 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.104:39025

2025-11-26 16:32:04,904 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.104:39025', status: running, memory: 0, processing: 0>

2025-11-26 16:32:04,821 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:35449

2025-11-26 16:32:04,819 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:35449', status: closed, memory: 0, processing: 0>

2025-11-26 16:32:04,713 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:40527

2025-11-26 16:32:04,713 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:40527', status: closed, memory: 0, processing: 0>

2025-11-26 16:32:04,674 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:41593

2025-11-26 16:32:04,672 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:41593', status: closed, memory: 0, processing: 0>

2025-11-26 16:32:04,131 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:34403', status: closed, memory: 0, processing: 0>

2025-11-26 16:32:03,842 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:38443', status: closed, memory: 0, processing: 0>

2025-11-26 16:32:03,794 - distributed.scheduler - INFO - Close client connection: Client-worker-5ee36a2d-cae5-11f0-83d4-2a2d08e6be2e

2025-11-26 16:32:03,792 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.104:40007', status: closed, memory: 0, processing: 0>

2025-11-26 16:32:03,788 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:34785', status: closed, memory: 0, processing: 0>

2025-11-26 16:32:03,760 - distributed.scheduler - INFO - Remove client Client-worker-5ee36a2d-cae5-11f0-83d4-2a2d08e6be2e

2025-11-26 16:32:03,726 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:34195', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:33,695 - distributed.scheduler - INFO - Receive client connection: Client-worker-5ee36a2d-cae5-11f0-83d4-2a2d08e6be2e

2025-11-26 16:31:32,364 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:38443

2025-11-26 16:31:32,363 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:38443', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:31,580 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:34195

2025-11-26 16:31:31,579 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:34195', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:31,412 - distributed.scheduler - INFO - Close client connection: Client-worker-4fa4bb85-cae5-11f0-8414-8a1d5a425dc1

2025-11-26 16:31:31,409 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:34403

2025-11-26 16:31:31,409 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:34403', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:31,392 - distributed.scheduler - INFO - Task ('hash-join-ee1bc74ba336c9445f60d21152c4b937', 4) marked as failed because 3 workers died while trying to run it

2025-11-26 16:31:31,391 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:41237', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:31,281 - distributed.scheduler - INFO - Remove client Client-worker-4fa4bb85-cae5-11f0-8414-8a1d5a425dc1

2025-11-26 16:31:31,128 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.104:40007

2025-11-26 16:31:31,127 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.104:40007', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:30,945 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:34785

2025-11-26 16:31:30,944 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:34785', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:30,648 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:40117', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:30,424 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:43407', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:30,042 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:37563', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:29,979 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.104:34377', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:08,134 - distributed.scheduler - INFO - Receive client connection: Client-worker-4fa4bb85-cae5-11f0-8414-8a1d5a425dc1

2025-11-26 16:31:06,981 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:37563

2025-11-26 16:31:06,980 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:37563', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:06,010 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.104:34377

2025-11-26 16:31:06,009 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.104:34377', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:06,001 - distributed.scheduler - INFO - Close client connection: Client-worker-6bbc549b-c881-11f0-82b0-2a2d08e6be2e

2025-11-26 16:31:05,974 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:44675', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:05,861 - distributed.scheduler - INFO - Remove client Client-worker-6bbc549b-c881-11f0-82b0-2a2d08e6be2e

2025-11-26 16:31:05,784 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:43407

2025-11-26 16:31:05,784 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:43407', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:05,394 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:41237

2025-11-26 16:31:05,392 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:41237', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:05,055 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:40117

2025-11-26 16:31:05,054 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:40117', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:05,031 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.104:37735', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:04,897 - distributed.scheduler - INFO - Close client connection: Client-worker-3dcf5ea2-cae5-11f0-8505-62b326af983b

2025-11-26 16:31:04,882 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:41759', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:04,854 - distributed.scheduler - INFO - Remove client Client-worker-3dcf5ea2-cae5-11f0-8505-62b326af983b

2025-11-26 16:31:04,701 - distributed.scheduler - INFO - Receive client connection: Client-worker-4d96d8ab-cae5-11f0-843e-f23d8172cb06

2025-11-26 16:31:04,387 - distributed.scheduler - INFO - Close client connection: Client-worker-1ea134be-cae1-11f0-83d1-8a1d5a425dc1

2025-11-26 16:31:04,375 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:43201', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:04,363 - distributed.scheduler - INFO - Remove client Client-worker-1ea134be-cae1-11f0-83d1-8a1d5a425dc1

2025-11-26 16:31:04,106 - distributed.scheduler - INFO - Close client connection: Client-worker-36a05c5a-c868-11f0-83c0-8253c573df33

2025-11-26 16:31:04,080 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:33359', status: closed, memory: 0, processing: 0>

2025-11-26 16:31:04,076 - distributed.scheduler - INFO - Remove client Client-worker-36a05c5a-c868-11f0-83c0-8253c573df33

2025-11-26 16:30:38,238 - distributed.scheduler - INFO - Receive client connection: Client-worker-3dcf5ea2-cae5-11f0-8505-62b326af983b

2025-11-26 16:01:16,571 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-eb8d955cb13e92f9db219799f9eabc8a', 1253) NoneType: None

2025-11-26 16:01:16,571 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-eb8d955cb13e92f9db219799f9eabc8a', 2250) NoneType: None

2025-11-26 16:01:16,571 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('query-eb8d955cb13e92f9db219799f9eabc8a', 2046) NoneType: None

2025-11-26 16:01:16,570 - distributed.scheduler - ERROR - Couldn't gather keys {"('query-eb8d955cb13e92f9db219799f9eabc8a', 2046)": [], "('query-eb8d955cb13e92f9db219799f9eabc8a', 2250)": [], "('query-eb8d955cb13e92f9db219799f9eabc8a', 1253)": []} state: ['memory', 'processing', 'processing'] workers: [] NoneType: None

2025-11-26 16:01:08,980 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:41759

2025-11-26 16:01:08,979 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:41759', status: closed, memory: 0, processing: 0>

2025-11-26 16:01:08,518 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:41969

2025-11-26 16:01:08,517 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:41969', status: closed, memory: 0, processing: 0>

2025-11-26 16:01:08,477 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.233:42861

2025-11-26 16:01:08,476 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.233:42861', status: closed, memory: 0, processing: 0>

2025-11-26 16:01:08,202 - distributed.scheduler - INFO - Close client connection: Client-worker-6df9d1b6-c907-11f0-83c6-62b326af983b

2025-11-26 16:01:08,193 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:35905', status: closed, memory: 0, processing: 0>

2025-11-26 16:01:08,192 - distributed.scheduler - INFO - Remove client Client-worker-6df9d1b6-c907-11f0-83c6-62b326af983b

2025-11-26 16:01:08,025 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.104:37735

2025-11-26 16:01:08,024 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.104:37735', status: closed, memory: 0, processing: 0>

2025-11-26 16:01:07,954 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.24:34199

2025-11-26 16:01:07,953 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.24:34199', status: closed, memory: 0, processing: 0>

2025-11-26 16:01:07,921 - distributed.scheduler - INFO - Receive client connection: Client-worker-1ea134be-cae1-11f0-83d1-8a1d5a425dc1

2025-11-26 16:01:07,591 - distributed.scheduler - INFO - Close client connection: Client-worker-6dd9ce25-c907-11f0-830d-f23d8172cb06

2025-11-26 16:01:07,577 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.233:35101', status: closed, memory: 0, processing: 0>

2025-11-26 16:01:07,563 - distributed.scheduler - INFO - Remove client Client-worker-6dd9ce25-c907-11f0-830d-f23d8172cb06

2025-11-26 16:01:07,474 - distributed.scheduler - INFO - Close client connection: Client-worker-bec57e3c-c846-11f0-8348-b64bf7bee50e

2025-11-26 16:01:07,459 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:34191', status: closed, memory: 0, processing: 0>

2025-11-26 16:01:07,458 - distributed.scheduler - INFO - Remove client Client-worker-bec57e3c-c846-11f0-8348-b64bf7bee50e

2025-11-26 16:01:07,098 - distributed.scheduler - INFO - Close client connection: Client-worker-e4b8ad28-c9cc-11f0-82c1-7e5a6871f9da

2025-11-26 16:01:07,096 - distributed.scheduler - INFO - Remove client Client-worker-e4b8ad28-c9cc-11f0-82c1-7e5a6871f9da

2025-11-26 16:01:07,089 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.104:40113', status: closed, memory: 0, processing: 0>

2025-11-26 16:01:06,701 - distributed.scheduler - INFO - Close client connection: Client-worker-9a3a0ee6-c78e-11f0-8210-46fbd1744967

2025-11-26 16:01:06,672 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.24:43549', status: closed, memory: 0, processing: 0>

2025-11-26 16:01:06,670 - distributed.scheduler - INFO - Remove client Client-worker-9a3a0ee6-c78e-11f0-8210-46fbd1744967

2025-11-26 16:01:05,838 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.197:43201

2025-11-26 16:01:05,837 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.197:43201', status: closed, memory: 0, processing: 0>

2025-11-26 16:01:04,964 - distributed.scheduler - INFO - Close client connection: Client-worker-1ac80f26-c6eb-11f0-81b3-8a1d5a425dc1

2025-11-26 16:01:04,940 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.197:43129', status: closed, memory: 0, processing: 0>

2025-11-26 16:01:04,939 - distributed.scheduler - INFO - Remove client Client-worker-1ac80f26-c6eb-11f0-81b3-8a1d5a425dc1

2025-11-26 14:00:16,025 - distributed.scheduler - INFO - Receive client connection: Client-worker-3c2bc540-cad0-11f0-83a5-de470ccc1e79

2025-11-26 12:11:36,020 - distributed.scheduler - INFO - Close client connection: Client-f43fba6e-caa4-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,020 - distributed.scheduler - INFO - Close client connection: Client-5aba4bce-caa6-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,019 - distributed.scheduler - INFO - Close client connection: Client-c13a4475-caa7-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,019 - distributed.scheduler - INFO - Close client connection: Client-27b7eabc-caa9-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,019 - distributed.scheduler - INFO - Close client connection: Client-8e40e44d-caaa-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,019 - distributed.scheduler - INFO - Close client connection: Client-f4b86259-caab-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,019 - distributed.scheduler - INFO - Close client connection: Client-5b37a8ba-caad-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,019 - distributed.scheduler - INFO - Close client connection: Client-2834ca57-cab0-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,019 - distributed.scheduler - INFO - Close client connection: Client-c1b3f7f0-caae-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,018 - distributed.scheduler - INFO - Close client connection: Client-8eb0ab8a-cab1-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,018 - distributed.scheduler - INFO - Close client connection: Client-f52aefc4-cab2-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,018 - distributed.scheduler - INFO - Close client connection: Client-5bb43733-cab4-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,018 - distributed.scheduler - INFO - Close client connection: Client-c23b653c-cab5-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,016 - distributed.scheduler - INFO - Close client connection: Client-29a89e46-cab7-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,016 - distributed.scheduler - INFO - Close client connection: Client-8f30ea81-cab8-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,016 - distributed.scheduler - INFO - Close client connection: Client-f5ac0eb7-cab9-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,016 - distributed.scheduler - INFO - Close client connection: Client-5c2d2a2c-cabb-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,016 - distributed.scheduler - INFO - Close client connection: Client-c2a77529-cabc-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,015 - distributed.scheduler - INFO - Close client connection: Client-292a45d7-cabe-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,015 - distributed.scheduler - INFO - Close client connection: Client-8fa003e2-cabf-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,013 - distributed.scheduler - INFO - Remove client Client-f43fba6e-caa4-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,013 - distributed.scheduler - INFO - Remove client Client-5aba4bce-caa6-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,012 - distributed.scheduler - INFO - Remove client Client-c13a4475-caa7-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,012 - distributed.scheduler - INFO - Remove client Client-27b7eabc-caa9-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,012 - distributed.scheduler - INFO - Remove client Client-8e40e44d-caaa-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,011 - distributed.scheduler - INFO - Remove client Client-f4b86259-caab-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,011 - distributed.scheduler - INFO - Remove client Client-5b37a8ba-caad-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,010 - distributed.scheduler - INFO - Remove client Client-2834ca57-cab0-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,009 - distributed.scheduler - INFO - Remove client Client-c1b3f7f0-caae-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,008 - distributed.scheduler - INFO - Remove client Client-8eb0ab8a-cab1-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,008 - distributed.scheduler - INFO - Remove client Client-f52aefc4-cab2-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,007 - distributed.scheduler - INFO - Remove client Client-5bb43733-cab4-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,006 - distributed.scheduler - INFO - Remove client Client-c23b653c-cab5-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,001 - distributed.scheduler - INFO - Remove client Client-29a89e46-cab7-11f0-8001-f2cd832f225c

2025-11-26 12:11:36,000 - distributed.scheduler - INFO - Remove client Client-8f30ea81-cab8-11f0-8001-f2cd832f225c

2025-11-26 12:11:35,999 - distributed.scheduler - INFO - Remove client Client-f5ac0eb7-cab9-11f0-8001-f2cd832f225c

2025-11-26 12:11:35,998 - distributed.scheduler - INFO - Remove client Client-5c2d2a2c-cabb-11f0-8001-f2cd832f225c

2025-11-26 12:11:35,997 - distributed.scheduler - INFO - Remove client Client-c2a77529-cabc-11f0-8001-f2cd832f225c

2025-11-26 12:11:35,997 - distributed.scheduler - INFO - Remove client Client-292a45d7-cabe-11f0-8001-f2cd832f225c

2025-11-26 12:11:35,996 - distributed.scheduler - INFO - Remove client Client-8fa003e2-cabf-11f0-8001-f2cd832f225c

2025-11-26 12:11:35,286 - distributed.scheduler - INFO - Close client connection: Client-f618e010-cac0-11f0-8001-f2cd832f225c

2025-11-26 12:11:35,283 - distributed.scheduler - INFO - Remove client Client-f618e010-cac0-11f0-8001-f2cd832f225c

2025-11-26 12:11:35,282 - distributed.scheduler - INFO - Remove client Client-f618e010-cac0-11f0-8001-f2cd832f225c

2025-11-26 12:10:56,000 - distributed.scheduler - INFO - Receive client connection: Client-f618e010-cac0-11f0-8001-f2cd832f225c

2025-11-26 12:00:54,585 - distributed.scheduler - INFO - Receive client connection: Client-8fa003e2-cabf-11f0-8001-f2cd832f225c

2025-11-26 11:50:53,186 - distributed.scheduler - INFO - Receive client connection: Client-292a45d7-cabe-11f0-8001-f2cd832f225c

2025-11-26 11:40:51,712 - distributed.scheduler - INFO - Receive client connection: Client-c2a77529-cabc-11f0-8001-f2cd832f225c

2025-11-26 11:33:03,881 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:45615

2025-11-26 11:33:03,879 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:45615', status: closed, memory: 0, processing: 0>

2025-11-26 11:33:03,041 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:43341', status: closed, memory: 0, processing: 0>

2025-11-26 11:32:52,164 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:43341

2025-11-26 11:32:52,162 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:43341', status: closed, memory: 0, processing: 0>

2025-11-26 11:32:51,159 - distributed.scheduler - INFO - Close client connection: Client-worker-9bf2d989-cabb-11f0-8382-de470ccc1e79

2025-11-26 11:32:51,152 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:45477', status: closed, memory: 0, processing: 0>

2025-11-26 11:32:51,151 - distributed.scheduler - INFO - Remove client Client-worker-9bf2d989-cabb-11f0-8382-de470ccc1e79

2025-11-26 11:32:37,270 - distributed.scheduler - INFO - Receive client connection: Client-worker-9bf2d989-cabb-11f0-8382-de470ccc1e79

2025-11-26 11:32:34,052 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:43237 failed: OSError: Timed out trying to connect to tcp://172.21.12.191:43237 after 30 s

2025-11-26 11:32:33,913 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:43927 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.191:43237 after 30 s')

2025-11-26 11:32:33,322 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:40951 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.191:43237 after 30 s')

2025-11-26 11:32:33,238 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:41855 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.191:43237 after 30 s')

2025-11-26 11:32:03,350 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:45477

2025-11-26 11:32:03,349 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:45477', status: closed, memory: 0, processing: 0>

2025-11-26 11:32:02,342 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:43237', status: closed, memory: 0, processing: 0>

2025-11-26 11:31:31,563 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:43237

2025-11-26 11:31:31,561 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:43237', status: closed, memory: 0, processing: 0>

2025-11-26 11:31:30,528 - distributed.scheduler - INFO - Close client connection: Client-worker-6e0c1e62-c903-11f0-8258-de470ccc1e79

2025-11-26 11:31:30,521 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:33703', status: closed, memory: 0, processing: 0>

2025-11-26 11:31:30,520 - distributed.scheduler - INFO - Remove client Client-worker-6e0c1e62-c903-11f0-8258-de470ccc1e79

2025-11-26 11:30:50,282 - distributed.scheduler - INFO - Receive client connection: Client-5c2d2a2c-cabb-11f0-8001-f2cd832f225c

2025-11-26 11:20:48,816 - distributed.scheduler - INFO - Receive client connection: Client-f5ac0eb7-cab9-11f0-8001-f2cd832f225c

2025-11-26 11:10:47,377 - distributed.scheduler - INFO - Receive client connection: Client-8f30ea81-cab8-11f0-8001-f2cd832f225c

2025-11-26 11:00:47,538 - distributed.scheduler - INFO - Receive client connection: Client-29a89e46-cab7-11f0-8001-f2cd832f225c

2025-11-26 10:50:44,526 - distributed.scheduler - INFO - Receive client connection: Client-c23b653c-cab5-11f0-8001-f2cd832f225c

2025-11-26 10:40:43,006 - distributed.scheduler - INFO - Receive client connection: Client-5bb43733-cab4-11f0-8001-f2cd832f225c

2025-11-26 10:30:41,491 - distributed.scheduler - INFO - Receive client connection: Client-f52aefc4-cab2-11f0-8001-f2cd832f225c

2025-11-26 10:20:40,056 - distributed.scheduler - INFO - Receive client connection: Client-8eb0ab8a-cab1-11f0-8001-f2cd832f225c

2025-11-26 10:10:38,623 - distributed.scheduler - INFO - Receive client connection: Client-2834ca57-cab0-11f0-8001-f2cd832f225c

2025-11-26 10:00:37,160 - distributed.scheduler - INFO - Receive client connection: Client-c1b3f7f0-caae-11f0-8001-f2cd832f225c

2025-11-26 09:50:35,721 - distributed.scheduler - INFO - Receive client connection: Client-5b37a8ba-caad-11f0-8001-f2cd832f225c

2025-11-26 09:40:34,270 - distributed.scheduler - INFO - Receive client connection: Client-f4b86259-caab-11f0-8001-f2cd832f225c

2025-11-26 09:30:32,854 - distributed.scheduler - INFO - Receive client connection: Client-8e40e44d-caaa-11f0-8001-f2cd832f225c

2025-11-26 09:20:31,323 - distributed.scheduler - INFO - Receive client connection: Client-27b7eabc-caa9-11f0-8001-f2cd832f225c

2025-11-26 09:10:29,888 - distributed.scheduler - INFO - Receive client connection: Client-c13a4475-caa7-11f0-8001-f2cd832f225c

2025-11-26 09:00:28,424 - distributed.scheduler - INFO - Receive client connection: Client-5aba4bce-caa6-11f0-8001-f2cd832f225c

2025-11-26 08:50:27,004 - distributed.scheduler - INFO - Receive client connection: Client-f43fba6e-caa4-11f0-8001-f2cd832f225c

2025-11-26 08:41:05,358 - distributed.scheduler - INFO - Close client connection: Client-8dc6ddb3-caa3-11f0-8001-f2cd832f225c

2025-11-26 08:41:05,357 - distributed.scheduler - INFO - Remove client Client-8dc6ddb3-caa3-11f0-8001-f2cd832f225c

2025-11-26 08:41:05,355 - distributed.scheduler - INFO - Remove client Client-8dc6ddb3-caa3-11f0-8001-f2cd832f225c

2025-11-26 08:40:25,588 - distributed.scheduler - INFO - Receive client connection: Client-8dc6ddb3-caa3-11f0-8001-f2cd832f225c

2025-11-26 08:31:41,746 - distributed.scheduler - INFO - Close client connection: Client-2748e296-caa2-11f0-8001-f2cd832f225c

2025-11-26 08:31:41,745 - distributed.scheduler - INFO - Remove client Client-2748e296-caa2-11f0-8001-f2cd832f225c

2025-11-26 08:31:41,743 - distributed.scheduler - INFO - Remove client Client-2748e296-caa2-11f0-8001-f2cd832f225c

2025-11-26 08:30:24,133 - distributed.scheduler - INFO - Receive client connection: Client-2748e296-caa2-11f0-8001-f2cd832f225c

2025-11-26 08:21:03,269 - distributed.scheduler - INFO - Close client connection: Client-c0d4efef-caa0-11f0-8001-f2cd832f225c

2025-11-26 08:21:03,268 - distributed.scheduler - INFO - Remove client Client-c0d4efef-caa0-11f0-8001-f2cd832f225c

2025-11-26 08:21:03,266 - distributed.scheduler - INFO - Remove client Client-c0d4efef-caa0-11f0-8001-f2cd832f225c

2025-11-26 08:20:22,754 - distributed.scheduler - INFO - Receive client connection: Client-c0d4efef-caa0-11f0-8001-f2cd832f225c

2025-11-26 08:10:59,719 - distributed.scheduler - INFO - Close client connection: Client-5a5e59e7-ca9f-11f0-8001-f2cd832f225c

2025-11-26 08:10:59,719 - distributed.scheduler - INFO - Remove client Client-5a5e59e7-ca9f-11f0-8001-f2cd832f225c

2025-11-26 08:10:59,718 - distributed.scheduler - INFO - Remove client Client-5a5e59e7-ca9f-11f0-8001-f2cd832f225c

2025-11-26 08:10:21,354 - distributed.scheduler - INFO - Receive client connection: Client-5a5e59e7-ca9f-11f0-8001-f2cd832f225c

2025-11-26 08:01:34,961 - distributed.scheduler - INFO - Close client connection: Client-f3e739db-ca9d-11f0-8001-f2cd832f225c

2025-11-26 08:01:34,959 - distributed.scheduler - INFO - Remove client Client-f3e739db-ca9d-11f0-8001-f2cd832f225c

2025-11-26 08:01:34,957 - distributed.scheduler - INFO - Remove client Client-f3e739db-ca9d-11f0-8001-f2cd832f225c

2025-11-26 08:00:19,947 - distributed.scheduler - INFO - Receive client connection: Client-f3e739db-ca9d-11f0-8001-f2cd832f225c

2025-11-26 07:51:00,576 - distributed.scheduler - INFO - Close client connection: Client-26f535dd-ca9b-11f0-8001-f2cd832f225c

2025-11-26 07:51:00,576 - distributed.scheduler - INFO - Remove client Client-26f535dd-ca9b-11f0-8001-f2cd832f225c

2025-11-26 07:50:59,688 - distributed.scheduler - INFO - Close client connection: Client-8d7390e0-ca9c-11f0-8001-f2cd832f225c

2025-11-26 07:50:59,687 - distributed.scheduler - INFO - Remove client Client-8d7390e0-ca9c-11f0-8001-f2cd832f225c

2025-11-26 07:50:59,685 - distributed.scheduler - INFO - Remove client Client-8d7390e0-ca9c-11f0-8001-f2cd832f225c

2025-11-26 07:50:18,553 - distributed.scheduler - INFO - Receive client connection: Client-8d7390e0-ca9c-11f0-8001-f2cd832f225c

2025-11-26 07:40:17,107 - distributed.scheduler - INFO - Receive client connection: Client-26f535dd-ca9b-11f0-8001-f2cd832f225c

2025-11-26 07:31:28,462 - distributed.scheduler - INFO - Close client connection: Client-c078f049-ca99-11f0-8001-f2cd832f225c

2025-11-26 07:31:28,459 - distributed.scheduler - INFO - Remove client Client-c078f049-ca99-11f0-8001-f2cd832f225c

2025-11-26 07:31:28,457 - distributed.scheduler - INFO - Remove client Client-c078f049-ca99-11f0-8001-f2cd832f225c

2025-11-26 07:30:15,671 - distributed.scheduler - INFO - Receive client connection: Client-c078f049-ca99-11f0-8001-f2cd832f225c

2025-11-26 07:20:54,870 - distributed.scheduler - INFO - Close client connection: Client-5a02ab3b-ca98-11f0-8001-f2cd832f225c

2025-11-26 07:20:54,870 - distributed.scheduler - INFO - Remove client Client-5a02ab3b-ca98-11f0-8001-f2cd832f225c

2025-11-26 07:20:54,868 - distributed.scheduler - INFO - Remove client Client-5a02ab3b-ca98-11f0-8001-f2cd832f225c

2025-11-26 07:20:14,280 - distributed.scheduler - INFO - Receive client connection: Client-5a02ab3b-ca98-11f0-8001-f2cd832f225c

2025-11-26 07:10:54,140 - distributed.scheduler - INFO - Close client connection: Client-f386e127-ca96-11f0-8001-f2cd832f225c

2025-11-26 07:10:54,139 - distributed.scheduler - INFO - Remove client Client-f386e127-ca96-11f0-8001-f2cd832f225c

2025-11-26 07:10:54,137 - distributed.scheduler - INFO - Remove client Client-f386e127-ca96-11f0-8001-f2cd832f225c

2025-11-26 07:10:12,829 - distributed.scheduler - INFO - Receive client connection: Client-f386e127-ca96-11f0-8001-f2cd832f225c

2025-11-26 07:01:30,821 - distributed.scheduler - INFO - Close client connection: Client-8d09b06c-ca95-11f0-8001-f2cd832f225c

2025-11-26 07:01:30,821 - distributed.scheduler - INFO - Remove client Client-8d09b06c-ca95-11f0-8001-f2cd832f225c

2025-11-26 07:01:30,819 - distributed.scheduler - INFO - Remove client Client-8d09b06c-ca95-11f0-8001-f2cd832f225c

2025-11-26 07:00:11,397 - distributed.scheduler - INFO - Receive client connection: Client-8d09b06c-ca95-11f0-8001-f2cd832f225c

2025-11-26 06:50:53,058 - distributed.scheduler - INFO - Close client connection: Client-268e086c-ca94-11f0-8001-f2cd832f225c

2025-11-26 06:50:53,058 - distributed.scheduler - INFO - Remove client Client-268e086c-ca94-11f0-8001-f2cd832f225c

2025-11-26 06:50:53,057 - distributed.scheduler - INFO - Remove client Client-268e086c-ca94-11f0-8001-f2cd832f225c

2025-11-26 06:50:09,955 - distributed.scheduler - INFO - Receive client connection: Client-268e086c-ca94-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,185 - distributed.scheduler - INFO - Close client connection: Client-b005bdce-ca87-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,184 - distributed.scheduler - INFO - Close client connection: Client-166e634d-ca89-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,184 - distributed.scheduler - INFO - Close client connection: Client-7cf35a36-ca8a-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,184 - distributed.scheduler - INFO - Close client connection: Client-2619f4bb-ca8d-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,184 - distributed.scheduler - INFO - Close client connection: Client-bf92cd51-ca8b-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,183 - distributed.scheduler - INFO - Close client connection: Client-f311e222-ca8f-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,183 - distributed.scheduler - INFO - Close client connection: Client-8c9769a0-ca8e-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,182 - distributed.scheduler - INFO - Close client connection: Client-598f4bf7-ca91-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,179 - distributed.scheduler - INFO - Remove client Client-b005bdce-ca87-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,178 - distributed.scheduler - INFO - Remove client Client-166e634d-ca89-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,178 - distributed.scheduler - INFO - Remove client Client-7cf35a36-ca8a-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,177 - distributed.scheduler - INFO - Remove client Client-2619f4bb-ca8d-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,176 - distributed.scheduler - INFO - Remove client Client-bf92cd51-ca8b-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,176 - distributed.scheduler - INFO - Remove client Client-f311e222-ca8f-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,171 - distributed.scheduler - INFO - Remove client Client-8c9769a0-ca8e-11f0-8001-f2cd832f225c

2025-11-26 06:40:52,170 - distributed.scheduler - INFO - Remove client Client-598f4bf7-ca91-11f0-8001-f2cd832f225c

2025-11-26 06:40:51,370 - distributed.scheduler - INFO - Close client connection: Client-c013e211-ca92-11f0-8001-f2cd832f225c

2025-11-26 06:40:51,368 - distributed.scheduler - INFO - Remove client Client-c013e211-ca92-11f0-8001-f2cd832f225c

2025-11-26 06:40:51,366 - distributed.scheduler - INFO - Remove client Client-c013e211-ca92-11f0-8001-f2cd832f225c

2025-11-26 06:40:08,519 - distributed.scheduler - INFO - Receive client connection: Client-c013e211-ca92-11f0-8001-f2cd832f225c

2025-11-26 06:30:07,027 - distributed.scheduler - INFO - Receive client connection: Client-598f4bf7-ca91-11f0-8001-f2cd832f225c

2025-11-26 06:20:05,585 - distributed.scheduler - INFO - Receive client connection: Client-f311e222-ca8f-11f0-8001-f2cd832f225c

2025-11-26 06:10:04,161 - distributed.scheduler - INFO - Receive client connection: Client-8c9769a0-ca8e-11f0-8001-f2cd832f225c

2025-11-26 06:00:07,042 - distributed.scheduler - INFO - Receive client connection: Client-worker-28ab5f9d-ca8d-11f0-84af-f2a719d69604

2025-11-26 06:00:02,708 - distributed.scheduler - INFO - Receive client connection: Client-2619f4bb-ca8d-11f0-8001-f2cd832f225c

2025-11-26 05:50:01,203 - distributed.scheduler - INFO - Receive client connection: Client-bf92cd51-ca8b-11f0-8001-f2cd832f225c

2025-11-26 05:40:59,931 - distributed.scheduler - INFO - Receive client connection: Client-7cf35a36-ca8a-11f0-8001-f2cd832f225c

2025-11-26 05:31:04,379 - distributed.scheduler - INFO - Receive client connection: Client-worker-19cd08f0-ca89-11f0-83da-32a59f79d364

2025-11-26 05:30:58,434 - distributed.scheduler - INFO - Receive client connection: Client-166e634d-ca89-11f0-8001-f2cd832f225c

2025-11-26 05:20:57,132 - distributed.scheduler - INFO - Receive client connection: Client-b005bdce-ca87-11f0-8001-f2cd832f225c

2025-11-26 05:11:39,729 - distributed.scheduler - INFO - Close client connection: Client-498a6c71-ca86-11f0-8001-f2cd832f225c

2025-11-26 05:11:39,728 - distributed.scheduler - INFO - Remove client Client-498a6c71-ca86-11f0-8001-f2cd832f225c

2025-11-26 05:11:39,726 - distributed.scheduler - INFO - Remove client Client-498a6c71-ca86-11f0-8001-f2cd832f225c

2025-11-26 05:10:55,696 - distributed.scheduler - INFO - Receive client connection: Client-498a6c71-ca86-11f0-8001-f2cd832f225c

2025-11-26 05:02:09,917 - distributed.scheduler - INFO - Close client connection: Client-e30f2175-ca84-11f0-8001-f2cd832f225c

2025-11-26 05:02:09,916 - distributed.scheduler - INFO - Remove client Client-e30f2175-ca84-11f0-8001-f2cd832f225c

2025-11-26 05:02:09,915 - distributed.scheduler - INFO - Remove client Client-e30f2175-ca84-11f0-8001-f2cd832f225c

2025-11-26 05:00:54,272 - distributed.scheduler - INFO - Receive client connection: Client-e30f2175-ca84-11f0-8001-f2cd832f225c

2025-11-26 04:51:34,697 - distributed.scheduler - INFO - Close client connection: Client-7c9251bd-ca83-11f0-8001-f2cd832f225c

2025-11-26 04:51:34,696 - distributed.scheduler - INFO - Remove client Client-7c9251bd-ca83-11f0-8001-f2cd832f225c

2025-11-26 04:51:34,694 - distributed.scheduler - INFO - Remove client Client-7c9251bd-ca83-11f0-8001-f2cd832f225c

2025-11-26 04:50:52,832 - distributed.scheduler - INFO - Receive client connection: Client-7c9251bd-ca83-11f0-8001-f2cd832f225c

2025-11-26 04:41:34,874 - distributed.scheduler - INFO - Close client connection: Client-1614ebb3-ca82-11f0-8001-f2cd832f225c

2025-11-26 04:41:34,873 - distributed.scheduler - INFO - Remove client Client-1614ebb3-ca82-11f0-8001-f2cd832f225c

2025-11-26 04:41:34,871 - distributed.scheduler - INFO - Remove client Client-1614ebb3-ca82-11f0-8001-f2cd832f225c

2025-11-26 04:40:51,377 - distributed.scheduler - INFO - Receive client connection: Client-1614ebb3-ca82-11f0-8001-f2cd832f225c

2025-11-26 04:32:01,050 - distributed.scheduler - INFO - Close client connection: Client-af9f56f4-ca80-11f0-8001-f2cd832f225c

2025-11-26 04:32:01,049 - distributed.scheduler - INFO - Remove client Client-af9f56f4-ca80-11f0-8001-f2cd832f225c

2025-11-26 04:32:01,048 - distributed.scheduler - INFO - Remove client Client-af9f56f4-ca80-11f0-8001-f2cd832f225c

2025-11-26 04:30:49,983 - distributed.scheduler - INFO - Receive client connection: Client-af9f56f4-ca80-11f0-8001-f2cd832f225c

2025-11-26 04:21:31,775 - distributed.scheduler - INFO - Close client connection: Client-7c1fb391-ca7c-11f0-8001-f2cd832f225c

2025-11-26 04:21:31,775 - distributed.scheduler - INFO - Close client connection: Client-e29f4eb8-ca7d-11f0-8001-f2cd832f225c

2025-11-26 04:21:31,774 - distributed.scheduler - INFO - Remove client Client-7c1fb391-ca7c-11f0-8001-f2cd832f225c

2025-11-26 04:21:31,773 - distributed.scheduler - INFO - Remove client Client-e29f4eb8-ca7d-11f0-8001-f2cd832f225c

2025-11-26 04:21:31,020 - distributed.scheduler - INFO - Close client connection: Client-492065ec-ca7f-11f0-8001-f2cd832f225c

2025-11-26 04:21:31,018 - distributed.scheduler - INFO - Remove client Client-492065ec-ca7f-11f0-8001-f2cd832f225c

2025-11-26 04:21:31,017 - distributed.scheduler - INFO - Remove client Client-492065ec-ca7f-11f0-8001-f2cd832f225c

2025-11-26 04:20:48,519 - distributed.scheduler - INFO - Receive client connection: Client-492065ec-ca7f-11f0-8001-f2cd832f225c

2025-11-26 04:10:47,048 - distributed.scheduler - INFO - Receive client connection: Client-e29f4eb8-ca7d-11f0-8001-f2cd832f225c

2025-11-26 04:03:25,216 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:40579

2025-11-26 04:03:25,214 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:40579', status: closed, memory: 0, processing: 0>

2025-11-26 04:03:24,210 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:40951

2025-11-26 04:03:24,208 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:40951', status: closed, memory: 0, processing: 0>

2025-11-26 04:03:24,056 - distributed.scheduler - INFO - Close client connection: Client-worker-d274a710-ca7c-11f0-849c-f2a719d69604

2025-11-26 04:03:24,050 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:33477', status: closed, memory: 0, processing: 0>

2025-11-26 04:03:24,049 - distributed.scheduler - INFO - Remove client Client-worker-d274a710-ca7c-11f0-849c-f2a719d69604

2025-11-26 04:03:23,135 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:41925', status: closed, memory: 0, processing: 0>

2025-11-26 04:03:10,435 - distributed.scheduler - INFO - Receive client connection: Client-worker-d274a710-ca7c-11f0-849c-f2a719d69604

2025-11-26 04:03:06,508 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:41925

2025-11-26 04:03:06,507 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:41925', status: closed, memory: 0, processing: 0>

2025-11-26 04:03:05,919 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:33477

2025-11-26 04:03:05,917 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:33477', status: closed, memory: 0, processing: 0>

2025-11-26 04:03:05,477 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:43337', status: closed, memory: 0, processing: 0>

2025-11-26 04:03:04,937 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:33473', status: closed, memory: 0, processing: 0>

2025-11-26 04:02:53,764 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:44671 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:44671 after 30 s

2025-11-26 04:02:53,764 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:34049 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:34049 after 30 s

2025-11-26 04:02:53,761 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:43927 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:34049 after 30 s')

2025-11-26 04:02:53,719 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:46147 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:34049 after 30 s')

2025-11-26 04:02:53,647 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:41855 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:34049 after 30 s')

2025-11-26 04:02:53,632 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44675 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:44671 after 30 s')

2025-11-26 04:02:53,628 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:33703 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:34049 after 30 s')

2025-11-26 04:02:53,627 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:33359 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:34049 after 30 s')

2025-11-26 04:02:53,627 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:35905 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:44671 after 30 s')

2025-11-26 04:02:53,627 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:43129 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:44671 after 30 s')

2025-11-26 04:02:53,626 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:38649 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:34049 after 30 s')

2025-11-26 04:02:53,581 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:35101 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:44671 after 30 s')

2025-11-26 04:02:53,577 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:34191 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:44671 after 30 s')

2025-11-26 04:02:53,576 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:44671 after 30 s')

2025-11-26 04:02:53,572 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:40113 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:34049 after 30 s')

2025-11-26 04:02:53,553 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:43549 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:34049 after 30 s')

2025-11-26 04:02:24,617 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:33473

2025-11-26 04:02:24,616 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:33473', status: closed, memory: 0, processing: 0>

2025-11-26 04:02:24,309 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:43337

2025-11-26 04:02:24,307 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:43337', status: closed, memory: 0, processing: 0>

2025-11-26 04:02:23,437 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:34049', status: closed, memory: 0, processing: 0>

2025-11-26 04:02:23,435 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:34049 failed: CommClosedError: in <TCP (closed) Scheduler Broadcast local=tcp://172.21.12.167:57842 remote=tcp://172.21.25.101:34049>: Stream is closed

2025-11-26 04:02:23,237 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:44671', status: closed, memory: 0, processing: 0>

2025-11-26 04:02:05,893 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:43581 after 30 s

2025-11-26 04:02:05,885 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:40623 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:40623 after 30 s

2025-11-26 04:01:35,198 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:44671

2025-11-26 04:01:35,197 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:44671', status: closed, memory: 0, processing: 0>

2025-11-26 04:01:33,992 - distributed.scheduler - INFO - Close client connection: Client-worker-03ea302d-c864-11f0-8283-32a59f79d364

2025-11-26 04:01:33,972 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:40623', status: closed, memory: 0, processing: 0>

2025-11-26 04:01:33,966 - distributed.scheduler - INFO - Remove client Client-worker-03ea302d-c864-11f0-8283-32a59f79d364

2025-11-26 04:01:33,231 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:34049

2025-11-26 04:01:33,229 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:34049', status: closed, memory: 0, processing: 0>

2025-11-26 04:01:32,180 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:43581', status: closed, memory: 0, processing: 0>

2025-11-26 04:01:32,107 - distributed.scheduler - INFO - Close client connection: Client-worker-9ca411cf-c85b-11f0-833d-f2a719d69604

2025-11-26 04:01:32,100 - distributed.scheduler - INFO - Remove client Client-worker-9ca411cf-c85b-11f0-833d-f2a719d69604

2025-11-26 04:00:45,593 - distributed.scheduler - INFO - Receive client connection: Client-7c1fb391-ca7c-11f0-8001-f2cd832f225c

2025-11-26 03:51:35,656 - distributed.scheduler - INFO - Close client connection: Client-159e11a6-ca7b-11f0-8001-f2cd832f225c

2025-11-26 03:51:35,655 - distributed.scheduler - INFO - Remove client Client-159e11a6-ca7b-11f0-8001-f2cd832f225c

2025-11-26 03:51:35,654 - distributed.scheduler - INFO - Remove client Client-159e11a6-ca7b-11f0-8001-f2cd832f225c

2025-11-26 03:50:44,123 - distributed.scheduler - INFO - Receive client connection: Client-159e11a6-ca7b-11f0-8001-f2cd832f225c

2025-11-26 03:41:25,643 - distributed.scheduler - INFO - Close client connection: Client-af2571ce-ca79-11f0-8001-f2cd832f225c

2025-11-26 03:41:25,641 - distributed.scheduler - INFO - Remove client Client-af2571ce-ca79-11f0-8001-f2cd832f225c

2025-11-26 03:41:25,639 - distributed.scheduler - INFO - Remove client Client-af2571ce-ca79-11f0-8001-f2cd832f225c

2025-11-26 03:40:42,704 - distributed.scheduler - INFO - Receive client connection: Client-af2571ce-ca79-11f0-8001-f2cd832f225c

2025-11-26 03:31:55,678 - distributed.scheduler - INFO - Close client connection: Client-48b3a154-ca78-11f0-8001-f2cd832f225c

2025-11-26 03:31:55,676 - distributed.scheduler - INFO - Remove client Client-48b3a154-ca78-11f0-8001-f2cd832f225c

2025-11-26 03:31:55,674 - distributed.scheduler - INFO - Remove client Client-48b3a154-ca78-11f0-8001-f2cd832f225c

2025-11-26 03:30:41,346 - distributed.scheduler - INFO - Receive client connection: Client-48b3a154-ca78-11f0-8001-f2cd832f225c

2025-11-26 03:21:22,570 - distributed.scheduler - INFO - Close client connection: Client-e2327c5e-ca76-11f0-8001-f2cd832f225c

2025-11-26 03:21:22,569 - distributed.scheduler - INFO - Remove client Client-e2327c5e-ca76-11f0-8001-f2cd832f225c

2025-11-26 03:21:22,567 - distributed.scheduler - INFO - Remove client Client-e2327c5e-ca76-11f0-8001-f2cd832f225c

2025-11-26 03:20:39,862 - distributed.scheduler - INFO - Receive client connection: Client-e2327c5e-ca76-11f0-8001-f2cd832f225c

2025-11-26 03:11:21,865 - distributed.scheduler - INFO - Close client connection: Client-7babfbaf-ca75-11f0-8001-f2cd832f225c

2025-11-26 03:11:21,865 - distributed.scheduler - INFO - Remove client Client-7babfbaf-ca75-11f0-8001-f2cd832f225c

2025-11-26 03:11:21,863 - distributed.scheduler - INFO - Remove client Client-7babfbaf-ca75-11f0-8001-f2cd832f225c

2025-11-26 03:10:38,361 - distributed.scheduler - INFO - Receive client connection: Client-7babfbaf-ca75-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,520 - distributed.scheduler - INFO - Close client connection: Client-9bc2e27f-ca3d-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,520 - distributed.scheduler - INFO - Close client connection: Client-0247f151-ca3f-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,519 - distributed.scheduler - INFO - Close client connection: Client-35c9ff23-ca43-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,519 - distributed.scheduler - INFO - Close client connection: Client-68ca3d34-ca40-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,519 - distributed.scheduler - INFO - Close client connection: Client-cf5220f3-ca41-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,519 - distributed.scheduler - INFO - Close client connection: Client-9c4b525b-ca44-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,519 - distributed.scheduler - INFO - Close client connection: Client-02c77805-ca46-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,519 - distributed.scheduler - INFO - Close client connection: Client-69458f89-ca47-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,519 - distributed.scheduler - INFO - Close client connection: Client-cfc6d6fe-ca48-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,518 - distributed.scheduler - INFO - Close client connection: Client-36486488-ca4a-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,518 - distributed.scheduler - INFO - Close client connection: Client-9cc012e0-ca4b-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,518 - distributed.scheduler - INFO - Close client connection: Client-039427b1-ca4d-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,518 - distributed.scheduler - INFO - Close client connection: Client-6a03ee2c-ca4e-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,518 - distributed.scheduler - INFO - Close client connection: Client-13237bc1-ca51-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,518 - distributed.scheduler - INFO - Close client connection: Client-d07f0524-ca4f-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,518 - distributed.scheduler - INFO - Close client connection: Client-79a0cead-ca52-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,517 - distributed.scheduler - INFO - Close client connection: Client-e014195c-ca53-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,517 - distributed.scheduler - INFO - Close client connection: Client-468b2f4d-ca55-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,517 - distributed.scheduler - INFO - Close client connection: Client-ad13368f-ca56-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,517 - distributed.scheduler - INFO - Close client connection: Client-1390ebee-ca58-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,517 - distributed.scheduler - INFO - Close client connection: Client-7a0c8bdf-ca59-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,517 - distributed.scheduler - INFO - Close client connection: Client-e08760cb-ca5a-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,517 - distributed.scheduler - INFO - Close client connection: Client-4705a5ae-ca5c-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,516 - distributed.scheduler - INFO - Close client connection: Client-ad850035-ca5d-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,516 - distributed.scheduler - INFO - Close client connection: Client-13fae3f0-ca5f-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,516 - distributed.scheduler - INFO - Close client connection: Client-e0ef0b05-ca61-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,516 - distributed.scheduler - INFO - Close client connection: Client-7a747989-ca60-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,516 - distributed.scheduler - INFO - Close client connection: Client-ade3e2e6-ca64-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,516 - distributed.scheduler - INFO - Close client connection: Client-7ae1cc7b-ca67-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,515 - distributed.scheduler - INFO - Close client connection: Client-146ab4d3-ca66-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,515 - distributed.scheduler - INFO - Close client connection: Client-47683eab-ca63-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,515 - distributed.scheduler - INFO - Close client connection: Client-e1504015-ca68-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,515 - distributed.scheduler - INFO - Close client connection: Client-47c80804-ca6a-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,515 - distributed.scheduler - INFO - Close client connection: Client-ae36e7b1-ca6b-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,515 - distributed.scheduler - INFO - Close client connection: Client-14b2b67f-ca6d-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,508 - distributed.scheduler - INFO - Close client connection: Client-7b3038aa-ca6e-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,508 - distributed.scheduler - INFO - Close client connection: Client-e1b97478-ca6f-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,508 - distributed.scheduler - INFO - Close client connection: Client-48277f30-ca71-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,508 - distributed.scheduler - INFO - Close client connection: Client-aea963e1-ca72-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,506 - distributed.scheduler - INFO - Remove client Client-9bc2e27f-ca3d-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,505 - distributed.scheduler - INFO - Remove client Client-0247f151-ca3f-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,505 - distributed.scheduler - INFO - Remove client Client-35c9ff23-ca43-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,505 - distributed.scheduler - INFO - Remove client Client-68ca3d34-ca40-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,504 - distributed.scheduler - INFO - Remove client Client-cf5220f3-ca41-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,504 - distributed.scheduler - INFO - Remove client Client-9c4b525b-ca44-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,504 - distributed.scheduler - INFO - Remove client Client-02c77805-ca46-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,503 - distributed.scheduler - INFO - Remove client Client-69458f89-ca47-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,503 - distributed.scheduler - INFO - Remove client Client-cfc6d6fe-ca48-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,502 - distributed.scheduler - INFO - Remove client Client-36486488-ca4a-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,502 - distributed.scheduler - INFO - Remove client Client-9cc012e0-ca4b-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,502 - distributed.scheduler - INFO - Remove client Client-039427b1-ca4d-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,501 - distributed.scheduler - INFO - Remove client Client-6a03ee2c-ca4e-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,501 - distributed.scheduler - INFO - Remove client Client-13237bc1-ca51-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,501 - distributed.scheduler - INFO - Remove client Client-d07f0524-ca4f-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,500 - distributed.scheduler - INFO - Remove client Client-79a0cead-ca52-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,500 - distributed.scheduler - INFO - Remove client Client-e014195c-ca53-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,500 - distributed.scheduler - INFO - Remove client Client-468b2f4d-ca55-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,499 - distributed.scheduler - INFO - Remove client Client-ad13368f-ca56-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,499 - distributed.scheduler - INFO - Remove client Client-1390ebee-ca58-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,498 - distributed.scheduler - INFO - Remove client Client-7a0c8bdf-ca59-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,498 - distributed.scheduler - INFO - Remove client Client-e08760cb-ca5a-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,498 - distributed.scheduler - INFO - Remove client Client-4705a5ae-ca5c-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,497 - distributed.scheduler - INFO - Remove client Client-ad850035-ca5d-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,497 - distributed.scheduler - INFO - Remove client Client-13fae3f0-ca5f-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,496 - distributed.scheduler - INFO - Remove client Client-e0ef0b05-ca61-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,496 - distributed.scheduler - INFO - Remove client Client-7a747989-ca60-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,495 - distributed.scheduler - INFO - Remove client Client-ade3e2e6-ca64-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,495 - distributed.scheduler - INFO - Remove client Client-7ae1cc7b-ca67-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,494 - distributed.scheduler - INFO - Remove client Client-146ab4d3-ca66-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,494 - distributed.scheduler - INFO - Remove client Client-47683eab-ca63-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,493 - distributed.scheduler - INFO - Remove client Client-e1504015-ca68-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,493 - distributed.scheduler - INFO - Remove client Client-47c80804-ca6a-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,492 - distributed.scheduler - INFO - Remove client Client-ae36e7b1-ca6b-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,491 - distributed.scheduler - INFO - Remove client Client-14b2b67f-ca6d-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,481 - distributed.scheduler - INFO - Remove client Client-7b3038aa-ca6e-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,481 - distributed.scheduler - INFO - Remove client Client-e1b97478-ca6f-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,480 - distributed.scheduler - INFO - Remove client Client-48277f30-ca71-11f0-8001-f2cd832f225c

2025-11-26 03:01:51,479 - distributed.scheduler - INFO - Remove client Client-aea963e1-ca72-11f0-8001-f2cd832f225c

2025-11-26 03:01:50,616 - distributed.scheduler - INFO - Close client connection: Client-152c5d68-ca74-11f0-8001-f2cd832f225c

2025-11-26 03:01:50,615 - distributed.scheduler - INFO - Remove client Client-152c5d68-ca74-11f0-8001-f2cd832f225c

2025-11-26 03:01:50,613 - distributed.scheduler - INFO - Remove client Client-152c5d68-ca74-11f0-8001-f2cd832f225c

2025-11-26 03:00:36,887 - distributed.scheduler - INFO - Receive client connection: Client-152c5d68-ca74-11f0-8001-f2cd832f225c

2025-11-26 02:50:35,415 - distributed.scheduler - INFO - Receive client connection: Client-aea963e1-ca72-11f0-8001-f2cd832f225c

2025-11-26 02:40:33,935 - distributed.scheduler - INFO - Receive client connection: Client-48277f30-ca71-11f0-8001-f2cd832f225c

2025-11-26 02:30:32,589 - distributed.scheduler - INFO - Receive client connection: Client-e1b97478-ca6f-11f0-8001-f2cd832f225c

2025-11-26 02:20:31,061 - distributed.scheduler - INFO - Receive client connection: Client-7b3038aa-ca6e-11f0-8001-f2cd832f225c

2025-11-26 02:10:29,622 - distributed.scheduler - INFO - Receive client connection: Client-14b2b67f-ca6d-11f0-8001-f2cd832f225c

2025-11-26 02:00:28,187 - distributed.scheduler - INFO - Receive client connection: Client-ae36e7b1-ca6b-11f0-8001-f2cd832f225c

2025-11-26 01:50:26,836 - distributed.scheduler - INFO - Receive client connection: Client-47c80804-ca6a-11f0-8001-f2cd832f225c

2025-11-26 01:40:25,459 - distributed.scheduler - INFO - Receive client connection: Client-e1504015-ca68-11f0-8001-f2cd832f225c

2025-11-26 01:30:24,088 - distributed.scheduler - INFO - Receive client connection: Client-7ae1cc7b-ca67-11f0-8001-f2cd832f225c

2025-11-26 01:20:22,668 - distributed.scheduler - INFO - Receive client connection: Client-146ab4d3-ca66-11f0-8001-f2cd832f225c

2025-11-26 01:10:21,163 - distributed.scheduler - INFO - Receive client connection: Client-ade3e2e6-ca64-11f0-8001-f2cd832f225c

2025-11-26 01:00:19,759 - distributed.scheduler - INFO - Receive client connection: Client-47683eab-ca63-11f0-8001-f2cd832f225c

2025-11-26 00:50:18,307 - distributed.scheduler - INFO - Receive client connection: Client-e0ef0b05-ca61-11f0-8001-f2cd832f225c

2025-11-26 00:40:16,883 - distributed.scheduler - INFO - Receive client connection: Client-7a747989-ca60-11f0-8001-f2cd832f225c

2025-11-26 00:30:15,453 - distributed.scheduler - INFO - Receive client connection: Client-13fae3f0-ca5f-11f0-8001-f2cd832f225c

2025-11-26 00:20:14,067 - distributed.scheduler - INFO - Receive client connection: Client-ad850035-ca5d-11f0-8001-f2cd832f225c

2025-11-26 00:10:12,603 - distributed.scheduler - INFO - Receive client connection: Client-4705a5ae-ca5c-11f0-8001-f2cd832f225c

2025-11-26 00:00:11,154 - distributed.scheduler - INFO - Receive client connection: Client-e08760cb-ca5a-11f0-8001-f2cd832f225c

2025-11-25 23:50:09,744 - distributed.scheduler - INFO - Receive client connection: Client-7a0c8bdf-ca59-11f0-8001-f2cd832f225c

2025-11-25 23:40:08,303 - distributed.scheduler - INFO - Receive client connection: Client-1390ebee-ca58-11f0-8001-f2cd832f225c

2025-11-25 23:30:06,840 - distributed.scheduler - INFO - Receive client connection: Client-ad13368f-ca56-11f0-8001-f2cd832f225c

2025-11-25 23:20:05,329 - distributed.scheduler - INFO - Receive client connection: Client-468b2f4d-ca55-11f0-8001-f2cd832f225c

2025-11-25 23:10:03,923 - distributed.scheduler - INFO - Receive client connection: Client-e014195c-ca53-11f0-8001-f2cd832f225c

2025-11-25 23:00:02,541 - distributed.scheduler - INFO - Receive client connection: Client-79a0cead-ca52-11f0-8001-f2cd832f225c

2025-11-25 22:50:01,105 - distributed.scheduler - INFO - Receive client connection: Client-13237bc1-ca51-11f0-8001-f2cd832f225c

2025-11-25 22:40:59,787 - distributed.scheduler - INFO - Receive client connection: Client-d07f0524-ca4f-11f0-8001-f2cd832f225c

2025-11-25 22:30:58,359 - distributed.scheduler - INFO - Receive client connection: Client-6a03ee2c-ca4e-11f0-8001-f2cd832f225c

2025-11-25 22:20:57,005 - distributed.scheduler - INFO - Receive client connection: Client-039427b1-ca4d-11f0-8001-f2cd832f225c

2025-11-25 22:10:54,983 - distributed.scheduler - INFO - Receive client connection: Client-9cc012e0-ca4b-11f0-8001-f2cd832f225c

2025-11-25 22:00:53,579 - distributed.scheduler - INFO - Receive client connection: Client-36486488-ca4a-11f0-8001-f2cd832f225c

2025-11-25 21:50:52,101 - distributed.scheduler - INFO - Receive client connection: Client-cfc6d6fe-ca48-11f0-8001-f2cd832f225c

2025-11-25 21:40:50,662 - distributed.scheduler - INFO - Receive client connection: Client-69458f89-ca47-11f0-8001-f2cd832f225c

2025-11-25 21:30:49,184 - distributed.scheduler - INFO - Receive client connection: Client-02c77805-ca46-11f0-8001-f2cd832f225c

2025-11-25 21:20:47,745 - distributed.scheduler - INFO - Receive client connection: Client-9c4b525b-ca44-11f0-8001-f2cd832f225c

2025-11-25 21:10:46,274 - distributed.scheduler - INFO - Receive client connection: Client-35c9ff23-ca43-11f0-8001-f2cd832f225c

2025-11-25 21:00:44,889 - distributed.scheduler - INFO - Receive client connection: Client-cf5220f3-ca41-11f0-8001-f2cd832f225c

2025-11-25 20:50:43,349 - distributed.scheduler - INFO - Receive client connection: Client-68ca3d34-ca40-11f0-8001-f2cd832f225c

2025-11-25 20:40:41,871 - distributed.scheduler - INFO - Receive client connection: Client-0247f151-ca3f-11f0-8001-f2cd832f225c

2025-11-25 20:30:40,382 - distributed.scheduler - INFO - Receive client connection: Client-9bc2e27f-ca3d-11f0-8001-f2cd832f225c

2025-11-25 20:21:12,277 - distributed.scheduler - INFO - Close client connection: Client-35433c66-ca3c-11f0-8001-f2cd832f225c

2025-11-25 20:21:12,277 - distributed.scheduler - INFO - Remove client Client-35433c66-ca3c-11f0-8001-f2cd832f225c

2025-11-25 20:21:12,275 - distributed.scheduler - INFO - Remove client Client-35433c66-ca3c-11f0-8001-f2cd832f225c

2025-11-25 20:20:38,926 - distributed.scheduler - INFO - Receive client connection: Client-35433c66-ca3c-11f0-8001-f2cd832f225c

2025-11-25 20:11:08,345 - distributed.scheduler - INFO - Close client connection: Client-cec894f5-ca3a-11f0-8001-f2cd832f225c

2025-11-25 20:11:08,344 - distributed.scheduler - INFO - Remove client Client-cec894f5-ca3a-11f0-8001-f2cd832f225c

2025-11-25 20:11:08,342 - distributed.scheduler - INFO - Remove client Client-cec894f5-ca3a-11f0-8001-f2cd832f225c

2025-11-25 20:10:37,491 - distributed.scheduler - INFO - Receive client connection: Client-cec894f5-ca3a-11f0-8001-f2cd832f225c

2025-11-25 20:01:40,813 - distributed.scheduler - INFO - Close client connection: Client-68476fdb-ca39-11f0-8001-f2cd832f225c

2025-11-25 20:01:40,812 - distributed.scheduler - INFO - Remove client Client-68476fdb-ca39-11f0-8001-f2cd832f225c

2025-11-25 20:01:40,810 - distributed.scheduler - INFO - Remove client Client-68476fdb-ca39-11f0-8001-f2cd832f225c

2025-11-25 20:00:36,024 - distributed.scheduler - INFO - Receive client connection: Client-68476fdb-ca39-11f0-8001-f2cd832f225c

2025-11-25 19:51:08,262 - distributed.scheduler - INFO - Close client connection: Client-01c73a05-ca38-11f0-8001-f2cd832f225c

2025-11-25 19:51:08,262 - distributed.scheduler - INFO - Remove client Client-01c73a05-ca38-11f0-8001-f2cd832f225c

2025-11-25 19:51:08,260 - distributed.scheduler - INFO - Remove client Client-01c73a05-ca38-11f0-8001-f2cd832f225c

2025-11-25 19:50:34,563 - distributed.scheduler - INFO - Receive client connection: Client-01c73a05-ca38-11f0-8001-f2cd832f225c

2025-11-25 19:41:13,764 - distributed.scheduler - INFO - Close client connection: Client-9b494ecd-ca36-11f0-8001-f2cd832f225c

2025-11-25 19:41:13,763 - distributed.scheduler - INFO - Remove client Client-9b494ecd-ca36-11f0-8001-f2cd832f225c

2025-11-25 19:41:13,760 - distributed.scheduler - INFO - Remove client Client-9b494ecd-ca36-11f0-8001-f2cd832f225c

2025-11-25 19:40:33,103 - distributed.scheduler - INFO - Receive client connection: Client-9b494ecd-ca36-11f0-8001-f2cd832f225c

2025-11-25 19:31:44,126 - distributed.scheduler - INFO - Close client connection: Client-ce52c066-ca33-11f0-8001-f2cd832f225c

2025-11-25 19:31:44,125 - distributed.scheduler - INFO - Close client connection: Client-67d2a88c-ca32-11f0-8001-f2cd832f225c

2025-11-25 19:31:44,124 - distributed.scheduler - INFO - Remove client Client-ce52c066-ca33-11f0-8001-f2cd832f225c

2025-11-25 19:31:44,124 - distributed.scheduler - INFO - Remove client Client-67d2a88c-ca32-11f0-8001-f2cd832f225c

2025-11-25 19:31:43,264 - distributed.scheduler - INFO - Close client connection: Client-34ca6996-ca35-11f0-8001-f2cd832f225c

2025-11-25 19:31:43,263 - distributed.scheduler - INFO - Remove client Client-34ca6996-ca35-11f0-8001-f2cd832f225c

2025-11-25 19:31:43,262 - distributed.scheduler - INFO - Remove client Client-34ca6996-ca35-11f0-8001-f2cd832f225c

2025-11-25 19:30:31,644 - distributed.scheduler - INFO - Receive client connection: Client-34ca6996-ca35-11f0-8001-f2cd832f225c

2025-11-25 19:20:30,236 - distributed.scheduler - INFO - Receive client connection: Client-ce52c066-ca33-11f0-8001-f2cd832f225c

2025-11-25 19:10:28,796 - distributed.scheduler - INFO - Receive client connection: Client-67d2a88c-ca32-11f0-8001-f2cd832f225c

2025-11-25 19:01:39,145 - distributed.scheduler - INFO - Close client connection: Client-01656430-ca31-11f0-8001-f2cd832f225c

2025-11-25 19:01:39,144 - distributed.scheduler - INFO - Remove client Client-01656430-ca31-11f0-8001-f2cd832f225c

2025-11-25 19:01:39,143 - distributed.scheduler - INFO - Remove client Client-01656430-ca31-11f0-8001-f2cd832f225c

2025-11-25 19:00:27,442 - distributed.scheduler - INFO - Receive client connection: Client-01656430-ca31-11f0-8001-f2cd832f225c

2025-11-25 18:51:02,667 - distributed.scheduler - INFO - Close client connection: Client-9af975ee-ca2f-11f0-8001-f2cd832f225c

2025-11-25 18:51:02,666 - distributed.scheduler - INFO - Remove client Client-9af975ee-ca2f-11f0-8001-f2cd832f225c

2025-11-25 18:51:02,664 - distributed.scheduler - INFO - Remove client Client-9af975ee-ca2f-11f0-8001-f2cd832f225c

2025-11-25 18:50:26,111 - distributed.scheduler - INFO - Receive client connection: Client-9af975ee-ca2f-11f0-8001-f2cd832f225c

2025-11-25 18:41:02,689 - distributed.scheduler - INFO - Close client connection: Client-347c9f0e-ca2e-11f0-8001-f2cd832f225c

2025-11-25 18:41:02,688 - distributed.scheduler - INFO - Remove client Client-347c9f0e-ca2e-11f0-8001-f2cd832f225c

2025-11-25 18:41:02,687 - distributed.scheduler - INFO - Remove client Client-347c9f0e-ca2e-11f0-8001-f2cd832f225c

2025-11-25 18:40:24,699 - distributed.scheduler - INFO - Receive client connection: Client-347c9f0e-ca2e-11f0-8001-f2cd832f225c

2025-11-25 18:31:31,822 - distributed.scheduler - INFO - Close client connection: Client-cdfa6b2c-ca2c-11f0-8001-f2cd832f225c

2025-11-25 18:31:31,821 - distributed.scheduler - INFO - Remove client Client-cdfa6b2c-ca2c-11f0-8001-f2cd832f225c

2025-11-25 18:31:31,820 - distributed.scheduler - INFO - Remove client Client-cdfa6b2c-ca2c-11f0-8001-f2cd832f225c

2025-11-25 18:30:23,191 - distributed.scheduler - INFO - Receive client connection: Client-cdfa6b2c-ca2c-11f0-8001-f2cd832f225c

2025-11-25 18:21:01,426 - distributed.scheduler - INFO - Close client connection: Client-67708551-ca2b-11f0-8001-f2cd832f225c

2025-11-25 18:21:01,424 - distributed.scheduler - INFO - Remove client Client-67708551-ca2b-11f0-8001-f2cd832f225c

2025-11-25 18:21:01,423 - distributed.scheduler - INFO - Remove client Client-67708551-ca2b-11f0-8001-f2cd832f225c

2025-11-25 18:20:21,659 - distributed.scheduler - INFO - Receive client connection: Client-67708551-ca2b-11f0-8001-f2cd832f225c

2025-11-25 18:10:59,429 - distributed.scheduler - INFO - Close client connection: Client-33660f7a-ca20-11f0-8001-f2cd832f225c

2025-11-25 18:10:59,429 - distributed.scheduler - INFO - Close client connection: Client-99ebe10c-ca21-11f0-8001-f2cd832f225c

2025-11-25 18:10:59,428 - distributed.scheduler - INFO - Close client connection: Client-cd6773e9-ca25-11f0-8001-f2cd832f225c

2025-11-25 18:10:59,428 - distributed.scheduler - INFO - Close client connection: Client-006a2106-ca23-11f0-8001-f2cd832f225c

2025-11-25 18:10:59,428 - distributed.scheduler - INFO - Close client connection: Client-66e7aed4-ca24-11f0-8001-f2cd832f225c

2025-11-25 18:10:59,428 - distributed.scheduler - INFO - Close client connection: Client-33f0666d-ca27-11f0-8001-f2cd832f225c

2025-11-25 18:10:59,427 - distributed.scheduler - INFO - Close client connection: Client-9a6c12f5-ca28-11f0-8001-f2cd832f225c

2025-11-25 18:10:59,427 - distributed.scheduler - INFO - Remove client Client-33660f7a-ca20-11f0-8001-f2cd832f225c

2025-11-25 18:10:59,426 - distributed.scheduler - INFO - Remove client Client-99ebe10c-ca21-11f0-8001-f2cd832f225c

2025-11-25 18:10:59,426 - distributed.scheduler - INFO - Remove client Client-cd6773e9-ca25-11f0-8001-f2cd832f225c

2025-11-25 18:10:59,425 - distributed.scheduler - INFO - Remove client Client-006a2106-ca23-11f0-8001-f2cd832f225c

2025-11-25 18:10:59,425 - distributed.scheduler - INFO - Remove client Client-66e7aed4-ca24-11f0-8001-f2cd832f225c

2025-11-25 18:10:59,424 - distributed.scheduler - INFO - Remove client Client-33f0666d-ca27-11f0-8001-f2cd832f225c

2025-11-25 18:10:59,422 - distributed.scheduler - INFO - Remove client Client-9a6c12f5-ca28-11f0-8001-f2cd832f225c

2025-11-25 18:10:58,771 - distributed.scheduler - INFO - Close client connection: Client-00ec0202-ca2a-11f0-8001-f2cd832f225c

2025-11-25 18:10:58,769 - distributed.scheduler - INFO - Remove client Client-00ec0202-ca2a-11f0-8001-f2cd832f225c

2025-11-25 18:10:58,767 - distributed.scheduler - INFO - Remove client Client-00ec0202-ca2a-11f0-8001-f2cd832f225c

2025-11-25 18:10:20,153 - distributed.scheduler - INFO - Receive client connection: Client-00ec0202-ca2a-11f0-8001-f2cd832f225c

2025-11-25 18:00:18,693 - distributed.scheduler - INFO - Receive client connection: Client-9a6c12f5-ca28-11f0-8001-f2cd832f225c

2025-11-25 17:50:17,266 - distributed.scheduler - INFO - Receive client connection: Client-33f0666d-ca27-11f0-8001-f2cd832f225c

2025-11-25 17:40:15,738 - distributed.scheduler - INFO - Receive client connection: Client-cd6773e9-ca25-11f0-8001-f2cd832f225c

2025-11-25 17:30:14,276 - distributed.scheduler - INFO - Receive client connection: Client-66e7aed4-ca24-11f0-8001-f2cd832f225c

2025-11-25 17:20:12,827 - distributed.scheduler - INFO - Receive client connection: Client-006a2106-ca23-11f0-8001-f2cd832f225c

2025-11-25 17:10:11,372 - distributed.scheduler - INFO - Receive client connection: Client-99ebe10c-ca21-11f0-8001-f2cd832f225c

2025-11-25 17:00:09,899 - distributed.scheduler - INFO - Receive client connection: Client-33660f7a-ca20-11f0-8001-f2cd832f225c

2025-11-25 16:50:51,036 - distributed.scheduler - INFO - Close client connection: Client-cce2ee37-ca1e-11f0-8001-f2cd832f225c

2025-11-25 16:50:51,035 - distributed.scheduler - INFO - Remove client Client-cce2ee37-ca1e-11f0-8001-f2cd832f225c

2025-11-25 16:50:51,033 - distributed.scheduler - INFO - Remove client Client-cce2ee37-ca1e-11f0-8001-f2cd832f225c

2025-11-25 16:50:08,399 - distributed.scheduler - INFO - Receive client connection: Client-cce2ee37-ca1e-11f0-8001-f2cd832f225c

2025-11-25 16:40:47,905 - distributed.scheduler - INFO - Close client connection: Client-66614765-ca1d-11f0-8001-f2cd832f225c

2025-11-25 16:40:47,903 - distributed.scheduler - INFO - Remove client Client-66614765-ca1d-11f0-8001-f2cd832f225c

2025-11-25 16:40:47,901 - distributed.scheduler - INFO - Remove client Client-66614765-ca1d-11f0-8001-f2cd832f225c

2025-11-25 16:40:06,928 - distributed.scheduler - INFO - Receive client connection: Client-66614765-ca1d-11f0-8001-f2cd832f225c

2025-11-25 16:31:18,250 - distributed.scheduler - INFO - Close client connection: Client-ffe2ebed-ca1b-11f0-8001-f2cd832f225c

2025-11-25 16:31:18,249 - distributed.scheduler - INFO - Remove client Client-ffe2ebed-ca1b-11f0-8001-f2cd832f225c

2025-11-25 16:31:18,247 - distributed.scheduler - INFO - Remove client Client-ffe2ebed-ca1b-11f0-8001-f2cd832f225c

2025-11-25 16:30:05,478 - distributed.scheduler - INFO - Receive client connection: Client-ffe2ebed-ca1b-11f0-8001-f2cd832f225c

2025-11-25 16:20:43,129 - distributed.scheduler - INFO - Close client connection: Client-996399ef-ca1a-11f0-8001-f2cd832f225c

2025-11-25 16:20:43,128 - distributed.scheduler - INFO - Remove client Client-996399ef-ca1a-11f0-8001-f2cd832f225c

2025-11-25 16:20:43,127 - distributed.scheduler - INFO - Remove client Client-996399ef-ca1a-11f0-8001-f2cd832f225c

2025-11-25 16:20:04,021 - distributed.scheduler - INFO - Receive client connection: Client-996399ef-ca1a-11f0-8001-f2cd832f225c

2025-11-25 16:10:41,542 - distributed.scheduler - INFO - Close client connection: Client-32eb53cf-ca19-11f0-8001-f2cd832f225c

2025-11-25 16:10:41,542 - distributed.scheduler - INFO - Remove client Client-32eb53cf-ca19-11f0-8001-f2cd832f225c

2025-11-25 16:10:41,541 - distributed.scheduler - INFO - Remove client Client-32eb53cf-ca19-11f0-8001-f2cd832f225c

2025-11-25 16:10:02,610 - distributed.scheduler - INFO - Receive client connection: Client-32eb53cf-ca19-11f0-8001-f2cd832f225c

2025-11-25 16:01:13,543 - distributed.scheduler - INFO - Close client connection: Client-89bf770e-ca16-11f0-8001-f2cd832f225c

2025-11-25 16:01:13,542 - distributed.scheduler - INFO - Remove client Client-89bf770e-ca16-11f0-8001-f2cd832f225c

2025-11-25 16:01:12,614 - distributed.scheduler - INFO - Close client connection: Client-cc68a905-ca17-11f0-8001-f2cd832f225c

2025-11-25 16:01:12,612 - distributed.scheduler - INFO - Remove client Client-cc68a905-ca17-11f0-8001-f2cd832f225c

2025-11-25 16:01:12,610 - distributed.scheduler - INFO - Remove client Client-cc68a905-ca17-11f0-8001-f2cd832f225c

2025-11-25 16:00:01,115 - distributed.scheduler - INFO - Receive client connection: Client-cc68a905-ca17-11f0-8001-f2cd832f225c

2025-11-25 15:50:59,779 - distributed.scheduler - INFO - Receive client connection: Client-89bf770e-ca16-11f0-8001-f2cd832f225c

2025-11-25 15:41:39,084 - distributed.scheduler - INFO - Close client connection: Client-233c8416-ca15-11f0-8001-f2cd832f225c

2025-11-25 15:41:39,083 - distributed.scheduler - INFO - Remove client Client-233c8416-ca15-11f0-8001-f2cd832f225c

2025-11-25 15:41:39,081 - distributed.scheduler - INFO - Remove client Client-233c8416-ca15-11f0-8001-f2cd832f225c

2025-11-25 15:40:58,306 - distributed.scheduler - INFO - Receive client connection: Client-233c8416-ca15-11f0-8001-f2cd832f225c

2025-11-25 15:32:16,249 - distributed.scheduler - INFO - Close client connection: Client-bccb17d0-ca13-11f0-8001-f2cd832f225c

2025-11-25 15:32:16,247 - distributed.scheduler - INFO - Remove client Client-bccb17d0-ca13-11f0-8001-f2cd832f225c

2025-11-25 15:32:16,245 - distributed.scheduler - INFO - Remove client Client-bccb17d0-ca13-11f0-8001-f2cd832f225c

2025-11-25 15:31:02,976 - distributed.scheduler - INFO - Receive client connection: Client-worker-c05b3df6-ca13-11f0-83fe-4ec0916b0c7e

2025-11-25 15:31:02,482 - distributed.scheduler - INFO - Receive client connection: Client-worker-c0160ca5-ca13-11f0-82a6-ee8ad16902f8

2025-11-25 15:30:56,936 - distributed.scheduler - INFO - Receive client connection: Client-bccb17d0-ca13-11f0-8001-f2cd832f225c

2025-11-25 15:21:41,478 - distributed.scheduler - INFO - Close client connection: Client-5652bf85-ca12-11f0-8001-f2cd832f225c

2025-11-25 15:21:41,477 - distributed.scheduler - INFO - Remove client Client-5652bf85-ca12-11f0-8001-f2cd832f225c

2025-11-25 15:21:41,475 - distributed.scheduler - INFO - Remove client Client-5652bf85-ca12-11f0-8001-f2cd832f225c

2025-11-25 15:20:55,525 - distributed.scheduler - INFO - Receive client connection: Client-5652bf85-ca12-11f0-8001-f2cd832f225c

2025-11-25 15:11:31,826 - distributed.scheduler - INFO - Close client connection: Client-efdd9b9f-ca10-11f0-8001-f2cd832f225c

2025-11-25 15:11:31,825 - distributed.scheduler - INFO - Remove client Client-efdd9b9f-ca10-11f0-8001-f2cd832f225c

2025-11-25 15:11:31,824 - distributed.scheduler - INFO - Remove client Client-efdd9b9f-ca10-11f0-8001-f2cd832f225c

2025-11-25 15:10:54,131 - distributed.scheduler - INFO - Receive client connection: Client-efdd9b9f-ca10-11f0-8001-f2cd832f225c

2025-11-25 15:03:26,165 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:43927

2025-11-25 15:03:26,163 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:43927', status: closed, memory: 0, processing: 0>

2025-11-25 15:03:25,212 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:41473', status: closed, memory: 0, processing: 0>

2025-11-25 15:03:13,765 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:41473

2025-11-25 15:03:13,763 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:41473', status: closed, memory: 0, processing: 0>

2025-11-25 15:03:12,730 - distributed.scheduler - INFO - Close client connection: Client-worker-d454622c-ca0f-11f0-8283-ee8ad16902f8

2025-11-25 15:03:12,721 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:38431', status: closed, memory: 0, processing: 0>

2025-11-25 15:03:12,720 - distributed.scheduler - INFO - Remove client Client-worker-d454622c-ca0f-11f0-8283-ee8ad16902f8

2025-11-25 15:02:58,434 - distributed.scheduler - INFO - Receive client connection: Client-worker-d454622c-ca0f-11f0-8283-ee8ad16902f8

2025-11-25 15:02:54,233 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:38431

2025-11-25 15:02:54,231 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:38431', status: closed, memory: 0, processing: 0>

2025-11-25 15:02:53,222 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:40421', status: closed, memory: 0, processing: 0>

2025-11-25 15:02:49,730 - distributed.scheduler - INFO - Close client connection: Client-895d16f3-ca0f-11f0-8001-f2cd832f225c

2025-11-25 15:02:49,728 - distributed.scheduler - INFO - Remove client Client-895d16f3-ca0f-11f0-8001-f2cd832f225c

2025-11-25 15:02:49,728 - distributed.scheduler - INFO - Remove client Client-895d16f3-ca0f-11f0-8001-f2cd832f225c

2025-11-25 15:02:25,664 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:37377 failed: OSError: Timed out trying to connect to tcp://172.21.12.165:37377 after 30 s

2025-11-25 15:02:25,536 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:40113 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37377 after 30 s')

2025-11-25 15:02:25,401 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:40623 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37377 after 30 s')

2025-11-25 15:02:24,742 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:34191 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37377 after 30 s')

2025-11-25 15:02:24,697 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:35905 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37377 after 30 s')

2025-11-25 15:02:24,691 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44675 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37377 after 30 s')

2025-11-25 15:02:24,674 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:33359 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.165:37377 after 30 s')

2025-11-25 15:01:55,459 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:40421

2025-11-25 15:01:55,457 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:40421', status: closed, memory: 0, processing: 0>

2025-11-25 15:01:54,408 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:37377', status: closed, memory: 0, processing: 0>

2025-11-25 15:01:32,987 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.165:37377

2025-11-25 15:01:32,985 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.165:37377', status: closed, memory: 0, processing: 0>

2025-11-25 15:01:32,049 - distributed.scheduler - INFO - Close client connection: Client-worker-6fba5cad-c56d-11f0-8073-ee8ad16902f8

2025-11-25 15:01:32,035 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.165:41631', status: closed, memory: 0, processing: 0>

2025-11-25 15:01:32,014 - distributed.scheduler - INFO - Remove client Client-worker-6fba5cad-c56d-11f0-8073-ee8ad16902f8

2025-11-25 15:00:52,671 - distributed.scheduler - INFO - Receive client connection: Client-895d16f3-ca0f-11f0-8001-f2cd832f225c

2025-11-25 14:51:31,185 - distributed.scheduler - INFO - Close client connection: Client-22da42ed-ca0e-11f0-8001-f2cd832f225c

2025-11-25 14:51:31,184 - distributed.scheduler - INFO - Remove client Client-22da42ed-ca0e-11f0-8001-f2cd832f225c

2025-11-25 14:51:31,183 - distributed.scheduler - INFO - Remove client Client-22da42ed-ca0e-11f0-8001-f2cd832f225c

2025-11-25 14:50:51,185 - distributed.scheduler - INFO - Receive client connection: Client-22da42ed-ca0e-11f0-8001-f2cd832f225c

2025-11-25 14:41:27,027 - distributed.scheduler - INFO - Close client connection: Client-bc596791-ca0c-11f0-8001-f2cd832f225c

2025-11-25 14:41:27,027 - distributed.scheduler - INFO - Remove client Client-bc596791-ca0c-11f0-8001-f2cd832f225c

2025-11-25 14:41:27,026 - distributed.scheduler - INFO - Remove client Client-bc596791-ca0c-11f0-8001-f2cd832f225c

2025-11-25 14:40:49,734 - distributed.scheduler - INFO - Receive client connection: Client-bc596791-ca0c-11f0-8001-f2cd832f225c

2025-11-25 14:34:15,766 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:45171 failed: OSError: Timed out trying to connect to tcp://172.21.159.248:45171 after 30 s

2025-11-25 14:34:15,731 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:46147 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:45171 after 30 s')

2025-11-25 14:34:15,684 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:41855 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:45171 after 30 s')

2025-11-25 14:34:15,682 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:35101 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:45171 after 30 s')

2025-11-25 14:34:15,681 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:45171 after 30 s')

2025-11-25 14:34:15,657 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:45171 after 30 s')

2025-11-25 14:34:15,653 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:33359 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:45171 after 30 s')

2025-11-25 14:34:15,645 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44675 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:45171 after 30 s')

2025-11-25 14:34:15,645 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:45171 after 30 s')

2025-11-25 14:34:15,644 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:40623 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:45171 after 30 s')

2025-11-25 14:34:15,643 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:34191 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:45171 after 30 s')

2025-11-25 14:34:15,632 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:43549 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:45171 after 30 s')

2025-11-25 14:33:46,344 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.248:38649

2025-11-25 14:33:46,342 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.248:38649', status: closed, memory: 0, processing: 0>

2025-11-25 14:33:45,503 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.248:45171', status: closed, memory: 0, processing: 0>

2025-11-25 14:33:29,359 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.248:45171

2025-11-25 14:33:29,357 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.248:45171', status: closed, memory: 0, processing: 0>

2025-11-25 14:33:28,514 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.248:35683', status: closed, memory: 0, processing: 0>

2025-11-25 14:33:17,127 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:40323 failed: OSError: Timed out trying to connect to tcp://172.21.159.248:40323 after 30 s

2025-11-25 14:33:17,104 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:46147 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:40323 after 30 s')

2025-11-25 14:33:17,024 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:40323 after 30 s')

2025-11-25 14:33:17,016 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:33359 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:40323 after 30 s')

2025-11-25 14:33:16,986 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:33703 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:40323 after 30 s')

2025-11-25 14:33:16,980 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:40323 after 30 s')

2025-11-25 14:33:16,978 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:35905 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:40323 after 30 s')

2025-11-25 14:33:16,972 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:40113 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:40323 after 30 s')

2025-11-25 14:33:16,961 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:40323 after 30 s')

2025-11-25 14:32:47,649 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.248:35683

2025-11-25 14:32:47,646 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.248:35683', status: closed, memory: 0, processing: 0>

2025-11-25 14:32:46,846 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.248:40323', status: closed, memory: 0, processing: 0>

2025-11-25 14:32:34,950 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:38567 failed: OSError: Timed out trying to connect to tcp://172.21.159.248:38567 after 30 s

2025-11-25 14:32:34,503 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:38567 after 30 s')

2025-11-25 14:32:14,374 - distributed.scheduler - INFO - Close client connection: Client-55ec2be0-ca0b-11f0-8001-f2cd832f225c

2025-11-25 14:32:14,372 - distributed.scheduler - INFO - Remove client Client-55ec2be0-ca0b-11f0-8001-f2cd832f225c

2025-11-25 14:32:14,371 - distributed.scheduler - INFO - Remove client Client-55ec2be0-ca0b-11f0-8001-f2cd832f225c

2025-11-25 14:32:04,976 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.248:40323

2025-11-25 14:32:04,975 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.248:40323', status: closed, memory: 0, processing: 0>

2025-11-25 14:32:04,018 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.248:38567', status: closed, memory: 0, processing: 0>

2025-11-25 14:31:33,958 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.248:38567

2025-11-25 14:31:33,957 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.248:38567', status: closed, memory: 0, processing: 0>

2025-11-25 14:31:33,504 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], run-863e99f3d3fffaf26bc0565c7439f0ab NoneType: None

2025-11-25 14:31:33,503 - distributed.scheduler - ERROR - Couldn't gather keys {'run-863e99f3d3fffaf26bc0565c7439f0ab': []} state: ['processing'] workers: [] NoneType: None

2025-11-25 14:31:33,107 - distributed.scheduler - INFO - Close client connection: Client-worker-58843e17-c83e-11f0-8284-4ec0916b0c7e

2025-11-25 14:31:33,097 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.248:37207', status: closed, memory: 0, processing: 0>

2025-11-25 14:31:33,097 - distributed.scheduler - INFO - Remove client Client-worker-58843e17-c83e-11f0-8284-4ec0916b0c7e

2025-11-25 14:30:48,368 - distributed.scheduler - INFO - Receive client connection: Client-55ec2be0-ca0b-11f0-8001-f2cd832f225c

2025-11-25 14:21:22,519 - distributed.scheduler - INFO - Close client connection: Client-2278b8a4-ca07-11f0-8001-f2cd832f225c

2025-11-25 14:21:22,518 - distributed.scheduler - INFO - Close client connection: Client-bbf8e6f5-ca05-11f0-8001-f2cd832f225c

2025-11-25 14:21:22,518 - distributed.scheduler - INFO - Close client connection: Client-88f6343c-ca08-11f0-8001-f2cd832f225c

2025-11-25 14:21:22,515 - distributed.scheduler - INFO - Remove client Client-2278b8a4-ca07-11f0-8001-f2cd832f225c

2025-11-25 14:21:22,514 - distributed.scheduler - INFO - Remove client Client-bbf8e6f5-ca05-11f0-8001-f2cd832f225c

2025-11-25 14:21:22,513 - distributed.scheduler - INFO - Remove client Client-88f6343c-ca08-11f0-8001-f2cd832f225c

2025-11-25 14:21:21,966 - distributed.scheduler - INFO - Close client connection: Client-ef6f844f-ca09-11f0-8001-f2cd832f225c

2025-11-25 14:21:21,966 - distributed.scheduler - INFO - Remove client Client-ef6f844f-ca09-11f0-8001-f2cd832f225c

2025-11-25 14:21:21,965 - distributed.scheduler - INFO - Remove client Client-ef6f844f-ca09-11f0-8001-f2cd832f225c

2025-11-25 14:20:46,929 - distributed.scheduler - INFO - Receive client connection: Client-ef6f844f-ca09-11f0-8001-f2cd832f225c

2025-11-25 14:10:45,502 - distributed.scheduler - INFO - Receive client connection: Client-88f6343c-ca08-11f0-8001-f2cd832f225c

2025-11-25 14:00:44,060 - distributed.scheduler - INFO - Receive client connection: Client-2278b8a4-ca07-11f0-8001-f2cd832f225c

2025-11-25 13:50:42,607 - distributed.scheduler - INFO - Receive client connection: Client-bbf8e6f5-ca05-11f0-8001-f2cd832f225c

2025-11-25 13:41:22,478 - distributed.scheduler - INFO - Close client connection: Client-5576dca6-ca04-11f0-8001-f2cd832f225c

2025-11-25 13:41:22,475 - distributed.scheduler - INFO - Remove client Client-5576dca6-ca04-11f0-8001-f2cd832f225c

2025-11-25 13:41:22,473 - distributed.scheduler - INFO - Remove client Client-5576dca6-ca04-11f0-8001-f2cd832f225c

2025-11-25 13:40:41,128 - distributed.scheduler - INFO - Receive client connection: Client-5576dca6-ca04-11f0-8001-f2cd832f225c

2025-11-25 13:31:51,843 - distributed.scheduler - INFO - Close client connection: Client-ef048d2d-ca02-11f0-8001-f2cd832f225c

2025-11-25 13:31:51,842 - distributed.scheduler - INFO - Remove client Client-ef048d2d-ca02-11f0-8001-f2cd832f225c

2025-11-25 13:31:51,840 - distributed.scheduler - INFO - Remove client Client-ef048d2d-ca02-11f0-8001-f2cd832f225c

2025-11-25 13:30:39,755 - distributed.scheduler - INFO - Receive client connection: Client-ef048d2d-ca02-11f0-8001-f2cd832f225c

2025-11-25 13:21:19,502 - distributed.scheduler - INFO - Close client connection: Client-8888b3fd-ca01-11f0-8001-f2cd832f225c

2025-11-25 13:21:19,501 - distributed.scheduler - INFO - Remove client Client-8888b3fd-ca01-11f0-8001-f2cd832f225c

2025-11-25 13:21:19,499 - distributed.scheduler - INFO - Remove client Client-8888b3fd-ca01-11f0-8001-f2cd832f225c

2025-11-25 13:20:38,316 - distributed.scheduler - INFO - Receive client connection: Client-8888b3fd-ca01-11f0-8001-f2cd832f225c

2025-11-25 13:11:16,820 - distributed.scheduler - INFO - Close client connection: Client-220cdc4b-ca00-11f0-8001-f2cd832f225c

2025-11-25 13:11:16,818 - distributed.scheduler - INFO - Remove client Client-220cdc4b-ca00-11f0-8001-f2cd832f225c

2025-11-25 13:11:16,816 - distributed.scheduler - INFO - Remove client Client-220cdc4b-ca00-11f0-8001-f2cd832f225c

2025-11-25 13:10:36,892 - distributed.scheduler - INFO - Receive client connection: Client-220cdc4b-ca00-11f0-8001-f2cd832f225c

2025-11-25 13:01:46,876 - distributed.scheduler - INFO - Close client connection: Client-bb8f487a-c9fe-11f0-8001-f2cd832f225c

2025-11-25 13:01:46,874 - distributed.scheduler - INFO - Remove client Client-bb8f487a-c9fe-11f0-8001-f2cd832f225c

2025-11-25 13:01:46,873 - distributed.scheduler - INFO - Remove client Client-bb8f487a-c9fe-11f0-8001-f2cd832f225c

2025-11-25 13:00:35,474 - distributed.scheduler - INFO - Receive client connection: Client-bb8f487a-c9fe-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,065 - distributed.scheduler - INFO - Close client connection: Client-544e80c6-c9ef-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,065 - distributed.scheduler - INFO - Close client connection: Client-bac77b1c-c9f0-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,065 - distributed.scheduler - INFO - Close client connection: Client-21498f12-c9f2-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,065 - distributed.scheduler - INFO - Close client connection: Client-87c75841-c9f3-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,064 - distributed.scheduler - INFO - Close client connection: Client-ee46637f-c9f4-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,064 - distributed.scheduler - INFO - Close client connection: Client-54bcee71-c9f6-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,064 - distributed.scheduler - INFO - Close client connection: Client-bb2d157d-c9f7-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,064 - distributed.scheduler - INFO - Close client connection: Client-21aa1578-c9f9-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,063 - distributed.scheduler - INFO - Close client connection: Client-881c35ca-c9fa-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,062 - distributed.scheduler - INFO - Close client connection: Client-ee9b3567-c9fb-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,061 - distributed.scheduler - INFO - Remove client Client-544e80c6-c9ef-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,060 - distributed.scheduler - INFO - Remove client Client-bac77b1c-c9f0-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,059 - distributed.scheduler - INFO - Remove client Client-21498f12-c9f2-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,059 - distributed.scheduler - INFO - Remove client Client-87c75841-c9f3-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,058 - distributed.scheduler - INFO - Remove client Client-ee46637f-c9f4-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,058 - distributed.scheduler - INFO - Remove client Client-54bcee71-c9f6-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,057 - distributed.scheduler - INFO - Remove client Client-bb2d157d-c9f7-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,056 - distributed.scheduler - INFO - Remove client Client-21aa1578-c9f9-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,052 - distributed.scheduler - INFO - Remove client Client-881c35ca-c9fa-11f0-8001-f2cd832f225c

2025-11-25 12:51:13,052 - distributed.scheduler - INFO - Remove client Client-ee9b3567-c9fb-11f0-8001-f2cd832f225c

2025-11-25 12:51:12,316 - distributed.scheduler - INFO - Close client connection: Client-5515f2db-c9fd-11f0-8001-f2cd832f225c

2025-11-25 12:51:12,312 - distributed.scheduler - INFO - Remove client Client-5515f2db-c9fd-11f0-8001-f2cd832f225c

2025-11-25 12:51:12,310 - distributed.scheduler - INFO - Remove client Client-5515f2db-c9fd-11f0-8001-f2cd832f225c

2025-11-25 12:50:34,014 - distributed.scheduler - INFO - Receive client connection: Client-5515f2db-c9fd-11f0-8001-f2cd832f225c

2025-11-25 12:40:32,579 - distributed.scheduler - INFO - Receive client connection: Client-ee9b3567-c9fb-11f0-8001-f2cd832f225c

2025-11-25 12:30:31,120 - distributed.scheduler - INFO - Receive client connection: Client-881c35ca-c9fa-11f0-8001-f2cd832f225c

2025-11-25 12:20:29,763 - distributed.scheduler - INFO - Receive client connection: Client-21aa1578-c9f9-11f0-8001-f2cd832f225c

2025-11-25 12:10:28,304 - distributed.scheduler - INFO - Receive client connection: Client-bb2d157d-c9f7-11f0-8001-f2cd832f225c

2025-11-25 12:00:26,944 - distributed.scheduler - INFO - Receive client connection: Client-54bcee71-c9f6-11f0-8001-f2cd832f225c

2025-11-25 11:50:25,542 - distributed.scheduler - INFO - Receive client connection: Client-ee46637f-c9f4-11f0-8001-f2cd832f225c

2025-11-25 11:40:24,092 - distributed.scheduler - INFO - Receive client connection: Client-87c75841-c9f3-11f0-8001-f2cd832f225c

2025-11-25 11:30:27,319 - distributed.scheduler - INFO - Receive client connection: Client-worker-240f5864-c9f2-11f0-834e-ba070f366fac

2025-11-25 11:30:22,642 - distributed.scheduler - INFO - Receive client connection: Client-21498f12-c9f2-11f0-8001-f2cd832f225c

2025-11-25 11:20:21,168 - distributed.scheduler - INFO - Receive client connection: Client-bac77b1c-c9f0-11f0-8001-f2cd832f225c

2025-11-25 11:10:19,761 - distributed.scheduler - INFO - Receive client connection: Client-544e80c6-c9ef-11f0-8001-f2cd832f225c

2025-11-25 11:01:30,509 - distributed.scheduler - INFO - Close client connection: Client-edcb1fb2-c9ed-11f0-8001-f2cd832f225c

2025-11-25 11:01:30,507 - distributed.scheduler - INFO - Remove client Client-edcb1fb2-c9ed-11f0-8001-f2cd832f225c

2025-11-25 11:01:30,505 - distributed.scheduler - INFO - Remove client Client-edcb1fb2-c9ed-11f0-8001-f2cd832f225c

2025-11-25 11:00:18,293 - distributed.scheduler - INFO - Receive client connection: Client-edcb1fb2-c9ed-11f0-8001-f2cd832f225c

2025-11-25 10:50:57,024 - distributed.scheduler - INFO - Close client connection: Client-8747d717-c9ec-11f0-8001-f2cd832f225c

2025-11-25 10:50:57,023 - distributed.scheduler - INFO - Remove client Client-8747d717-c9ec-11f0-8001-f2cd832f225c

2025-11-25 10:50:57,021 - distributed.scheduler - INFO - Remove client Client-8747d717-c9ec-11f0-8001-f2cd832f225c

2025-11-25 10:50:16,788 - distributed.scheduler - INFO - Receive client connection: Client-8747d717-c9ec-11f0-8001-f2cd832f225c

2025-11-25 10:40:54,549 - distributed.scheduler - INFO - Close client connection: Client-20c6abaa-c9eb-11f0-8001-f2cd832f225c

2025-11-25 10:40:54,547 - distributed.scheduler - INFO - Remove client Client-20c6abaa-c9eb-11f0-8001-f2cd832f225c

2025-11-25 10:40:54,545 - distributed.scheduler - INFO - Remove client Client-20c6abaa-c9eb-11f0-8001-f2cd832f225c

2025-11-25 10:40:15,318 - distributed.scheduler - INFO - Receive client connection: Client-20c6abaa-c9eb-11f0-8001-f2cd832f225c

2025-11-25 10:31:23,017 - distributed.scheduler - INFO - Close client connection: Client-ba416325-c9e9-11f0-8001-f2cd832f225c

2025-11-25 10:31:23,016 - distributed.scheduler - INFO - Remove client Client-ba416325-c9e9-11f0-8001-f2cd832f225c

2025-11-25 10:31:23,014 - distributed.scheduler - INFO - Remove client Client-ba416325-c9e9-11f0-8001-f2cd832f225c

2025-11-25 10:30:13,817 - distributed.scheduler - INFO - Receive client connection: Client-ba416325-c9e9-11f0-8001-f2cd832f225c

2025-11-25 10:20:48,583 - distributed.scheduler - INFO - Close client connection: Client-53becf1c-c9e8-11f0-8001-f2cd832f225c

2025-11-25 10:20:48,582 - distributed.scheduler - INFO - Remove client Client-53becf1c-c9e8-11f0-8001-f2cd832f225c

2025-11-25 10:20:48,580 - distributed.scheduler - INFO - Remove client Client-53becf1c-c9e8-11f0-8001-f2cd832f225c

2025-11-25 10:20:12,333 - distributed.scheduler - INFO - Receive client connection: Client-53becf1c-c9e8-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,880 - distributed.scheduler - INFO - Close client connection: Client-102d37db-c9d9-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,880 - distributed.scheduler - INFO - Close client connection: Client-769b9676-c9da-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,879 - distributed.scheduler - INFO - Close client connection: Client-dd23a85c-c9db-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,879 - distributed.scheduler - INFO - Close client connection: Client-1fc83fa3-c9dd-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,879 - distributed.scheduler - INFO - Close client connection: Client-86472a10-c9de-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,879 - distributed.scheduler - INFO - Close client connection: Client-ecc81aee-c9df-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,879 - distributed.scheduler - INFO - Close client connection: Client-533c69ae-c9e1-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,878 - distributed.scheduler - INFO - Close client connection: Client-b9b9b9ea-c9e2-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,877 - distributed.scheduler - INFO - Close client connection: Client-203cebac-c9e4-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,877 - distributed.scheduler - INFO - Close client connection: Client-86bc0fbe-c9e5-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,875 - distributed.scheduler - INFO - Remove client Client-102d37db-c9d9-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,874 - distributed.scheduler - INFO - Remove client Client-769b9676-c9da-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,874 - distributed.scheduler - INFO - Remove client Client-dd23a85c-c9db-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,873 - distributed.scheduler - INFO - Remove client Client-1fc83fa3-c9dd-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,872 - distributed.scheduler - INFO - Remove client Client-86472a10-c9de-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,871 - distributed.scheduler - INFO - Remove client Client-ecc81aee-c9df-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,871 - distributed.scheduler - INFO - Remove client Client-533c69ae-c9e1-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,870 - distributed.scheduler - INFO - Remove client Client-b9b9b9ea-c9e2-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,865 - distributed.scheduler - INFO - Remove client Client-203cebac-c9e4-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,864 - distributed.scheduler - INFO - Remove client Client-86bc0fbe-c9e5-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,144 - distributed.scheduler - INFO - Close client connection: Client-ed42db34-c9e6-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,142 - distributed.scheduler - INFO - Remove client Client-ed42db34-c9e6-11f0-8001-f2cd832f225c

2025-11-25 10:10:55,140 - distributed.scheduler - INFO - Remove client Client-ed42db34-c9e6-11f0-8001-f2cd832f225c

2025-11-25 10:10:10,896 - distributed.scheduler - INFO - Receive client connection: Client-ed42db34-c9e6-11f0-8001-f2cd832f225c

2025-11-25 10:00:09,383 - distributed.scheduler - INFO - Receive client connection: Client-86bc0fbe-c9e5-11f0-8001-f2cd832f225c

2025-11-25 09:50:07,930 - distributed.scheduler - INFO - Receive client connection: Client-203cebac-c9e4-11f0-8001-f2cd832f225c

2025-11-25 09:40:06,442 - distributed.scheduler - INFO - Receive client connection: Client-b9b9b9ea-c9e2-11f0-8001-f2cd832f225c

2025-11-25 09:30:04,998 - distributed.scheduler - INFO - Receive client connection: Client-533c69ae-c9e1-11f0-8001-f2cd832f225c

2025-11-25 09:20:03,612 - distributed.scheduler - INFO - Receive client connection: Client-ecc81aee-c9df-11f0-8001-f2cd832f225c

2025-11-25 09:10:02,139 - distributed.scheduler - INFO - Receive client connection: Client-86472a10-c9de-11f0-8001-f2cd832f225c

2025-11-25 09:00:00,683 - distributed.scheduler - INFO - Receive client connection: Client-1fc83fa3-c9dd-11f0-8001-f2cd832f225c

2025-11-25 08:50:59,379 - distributed.scheduler - INFO - Receive client connection: Client-dd23a85c-c9db-11f0-8001-f2cd832f225c

2025-11-25 08:40:57,909 - distributed.scheduler - INFO - Receive client connection: Client-769b9676-c9da-11f0-8001-f2cd832f225c

2025-11-25 08:30:56,531 - distributed.scheduler - INFO - Receive client connection: Client-102d37db-c9d9-11f0-8001-f2cd832f225c

2025-11-25 08:21:36,862 - distributed.scheduler - INFO - Close client connection: Client-a9b11f4c-c9d7-11f0-8001-f2cd832f225c

2025-11-25 08:21:36,861 - distributed.scheduler - INFO - Remove client Client-a9b11f4c-c9d7-11f0-8001-f2cd832f225c

2025-11-25 08:21:36,860 - distributed.scheduler - INFO - Remove client Client-a9b11f4c-c9d7-11f0-8001-f2cd832f225c

2025-11-25 08:20:55,080 - distributed.scheduler - INFO - Receive client connection: Client-a9b11f4c-c9d7-11f0-8001-f2cd832f225c

2025-11-25 08:11:30,741 - distributed.scheduler - INFO - Close client connection: Client-43370932-c9d6-11f0-8001-f2cd832f225c

2025-11-25 08:11:30,740 - distributed.scheduler - INFO - Remove client Client-43370932-c9d6-11f0-8001-f2cd832f225c

2025-11-25 08:11:30,738 - distributed.scheduler - INFO - Remove client Client-43370932-c9d6-11f0-8001-f2cd832f225c

2025-11-25 08:10:53,657 - distributed.scheduler - INFO - Receive client connection: Client-43370932-c9d6-11f0-8001-f2cd832f225c

2025-11-25 08:01:56,836 - distributed.scheduler - INFO - Close client connection: Client-dcb127f2-c9d4-11f0-8001-f2cd832f225c

2025-11-25 08:01:56,833 - distributed.scheduler - INFO - Remove client Client-dcb127f2-c9d4-11f0-8001-f2cd832f225c

2025-11-25 08:01:56,832 - distributed.scheduler - INFO - Remove client Client-dcb127f2-c9d4-11f0-8001-f2cd832f225c

2025-11-25 08:00:52,159 - distributed.scheduler - INFO - Receive client connection: Client-dcb127f2-c9d4-11f0-8001-f2cd832f225c

2025-11-25 07:51:31,584 - distributed.scheduler - INFO - Close client connection: Client-7636e4ad-c9d3-11f0-8001-f2cd832f225c

2025-11-25 07:51:31,583 - distributed.scheduler - INFO - Remove client Client-7636e4ad-c9d3-11f0-8001-f2cd832f225c

2025-11-25 07:51:31,581 - distributed.scheduler - INFO - Remove client Client-7636e4ad-c9d3-11f0-8001-f2cd832f225c

2025-11-25 07:50:50,730 - distributed.scheduler - INFO - Receive client connection: Client-7636e4ad-c9d3-11f0-8001-f2cd832f225c

2025-11-25 07:41:31,010 - distributed.scheduler - INFO - Close client connection: Client-0fc13267-c9d2-11f0-8001-f2cd832f225c

2025-11-25 07:41:31,009 - distributed.scheduler - INFO - Remove client Client-0fc13267-c9d2-11f0-8001-f2cd832f225c

2025-11-25 07:41:31,007 - distributed.scheduler - INFO - Remove client Client-0fc13267-c9d2-11f0-8001-f2cd832f225c

2025-11-25 07:40:49,343 - distributed.scheduler - INFO - Receive client connection: Client-0fc13267-c9d2-11f0-8001-f2cd832f225c

2025-11-25 07:32:03,830 - distributed.scheduler - INFO - Close client connection: Client-a94463fd-c9d0-11f0-8001-f2cd832f225c

2025-11-25 07:32:03,829 - distributed.scheduler - INFO - Remove client Client-a94463fd-c9d0-11f0-8001-f2cd832f225c

2025-11-25 07:32:03,826 - distributed.scheduler - INFO - Remove client Client-a94463fd-c9d0-11f0-8001-f2cd832f225c

2025-11-25 07:30:47,899 - distributed.scheduler - INFO - Receive client connection: Client-a94463fd-c9d0-11f0-8001-f2cd832f225c

2025-11-25 07:21:30,224 - distributed.scheduler - INFO - Close client connection: Client-42c686e1-c9cf-11f0-8001-f2cd832f225c

2025-11-25 07:21:30,223 - distributed.scheduler - INFO - Remove client Client-42c686e1-c9cf-11f0-8001-f2cd832f225c

2025-11-25 07:21:30,221 - distributed.scheduler - INFO - Remove client Client-42c686e1-c9cf-11f0-8001-f2cd832f225c

2025-11-25 07:20:46,449 - distributed.scheduler - INFO - Receive client connection: Client-42c686e1-c9cf-11f0-8001-f2cd832f225c

2025-11-25 07:11:26,263 - distributed.scheduler - INFO - Close client connection: Client-dc488fdf-c9cd-11f0-8001-f2cd832f225c

2025-11-25 07:11:26,262 - distributed.scheduler - INFO - Remove client Client-dc488fdf-c9cd-11f0-8001-f2cd832f225c

2025-11-25 07:11:26,260 - distributed.scheduler - INFO - Remove client Client-dc488fdf-c9cd-11f0-8001-f2cd832f225c

2025-11-25 07:10:45,004 - distributed.scheduler - INFO - Receive client connection: Client-dc488fdf-c9cd-11f0-8001-f2cd832f225c

2025-11-25 07:03:49,644 - distributed.scheduler - INFO - Receive client connection: Client-worker-e4b8ad28-c9cc-11f0-82c1-7e5a6871f9da

2025-11-25 07:03:45,725 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:41855 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33085 after 30 s')

2025-11-25 07:03:45,663 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:35657 failed: OSError: Timed out trying to connect to tcp://172.21.25.104:35657 after 30 s

2025-11-25 07:03:45,662 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:33085 failed: OSError: Timed out trying to connect to tcp://172.21.159.210:33085 after 30 s

2025-11-25 07:03:45,233 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:46147 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.104:35657 after 30 s')

2025-11-25 07:03:45,137 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:33359 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.104:35657 after 30 s')

2025-11-25 07:03:45,136 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:43129 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33085 after 30 s')

2025-11-25 07:03:45,103 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.104:35657 after 30 s')

2025-11-25 07:03:45,096 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33085 after 30 s')

2025-11-25 07:03:45,092 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37207 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33085 after 30 s')

2025-11-25 07:03:45,082 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:43549 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33085 after 30 s')

2025-11-25 07:03:45,081 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:33703 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.104:35657 after 30 s')

2025-11-25 07:03:45,070 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:40623 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:33085 after 30 s')

2025-11-25 07:03:45,068 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:35101 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.104:35657 after 30 s')

2025-11-25 07:03:15,983 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.104:40113

2025-11-25 07:03:15,981 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.104:40113', status: closed, memory: 0, processing: 0>

2025-11-25 07:03:14,946 - distributed.scheduler - INFO - Close client connection: Client-worker-c63b080f-c9cc-11f0-82ac-7e5a6871f9da

2025-11-25 07:03:14,927 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.104:35657', status: closed, memory: 0, processing: 0>

2025-11-25 07:03:14,925 - distributed.scheduler - INFO - Remove client Client-worker-c63b080f-c9cc-11f0-82ac-7e5a6871f9da

2025-11-25 07:03:14,456 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:41855

2025-11-25 07:03:14,453 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:41855', status: closed, memory: 0, processing: 0>

2025-11-25 07:03:13,619 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:33085', status: closed, memory: 0, processing: 0>

2025-11-25 07:02:58,491 - distributed.scheduler - INFO - Receive client connection: Client-worker-c63b080f-c9cc-11f0-82ac-7e5a6871f9da

2025-11-25 07:02:42,962 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:35117 failed: OSError: Timed out trying to connect to tcp://172.21.25.104:35117 after 30 s

2025-11-25 07:02:42,962 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:41965 failed: OSError: Timed out trying to connect to tcp://172.21.159.210:41965 after 30 s

2025-11-25 07:02:42,909 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44675 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.104:35117 after 30 s')

2025-11-25 07:02:42,884 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:33359 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:41965 after 30 s')

2025-11-25 07:02:42,869 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:35101 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.104:35117 after 30 s')

2025-11-25 07:02:42,867 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:43129 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:41965 after 30 s')

2025-11-25 07:02:42,855 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:35905 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:41965 after 30 s')

2025-11-25 07:02:42,853 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37207 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.104:35117 after 30 s')

2025-11-25 07:02:42,822 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.104:35117 after 30 s')

2025-11-25 07:02:42,820 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:41965 after 30 s')

2025-11-25 07:02:42,768 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:40623 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:41965 after 30 s')

2025-11-25 07:02:16,733 - distributed.scheduler - INFO - Close client connection: Client-75c106a6-c9cc-11f0-8001-f2cd832f225c

2025-11-25 07:02:16,732 - distributed.scheduler - INFO - Remove client Client-75c106a6-c9cc-11f0-8001-f2cd832f225c

2025-11-25 07:02:16,730 - distributed.scheduler - INFO - Remove client Client-75c106a6-c9cc-11f0-8001-f2cd832f225c

2025-11-25 07:02:13,784 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.104:35657

2025-11-25 07:02:13,784 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.104:35657', status: closed, memory: 0, processing: 0>

2025-11-25 07:02:13,591 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:33085

2025-11-25 07:02:13,589 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:33085', status: closed, memory: 0, processing: 0>

2025-11-25 07:02:12,653 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.104:35117', status: closed, memory: 0, processing: 0>

2025-11-25 07:02:12,637 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:41965', status: closed, memory: 0, processing: 0>

2025-11-25 07:02:00,833 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36109 after 30 s')

2025-11-25 07:01:59,096 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:36109 failed: OSError: Timed out trying to connect to tcp://172.21.159.210:36109 after 30 s

2025-11-25 07:01:59,064 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:36109 failed: OSError: Timed out trying to connect to tcp://172.21.159.210:36109 after 30 s

2025-11-25 07:01:58,932 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:40623 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36109 after 30 s')

2025-11-25 07:01:58,089 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44675 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36109 after 30 s')

2025-11-25 07:01:57,647 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:34191 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36109 after 30 s')

2025-11-25 07:01:57,061 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.210:36109 after 30 s')

2025-11-25 07:01:31,834 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.104:35117

2025-11-25 07:01:31,833 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.104:35117', status: closed, memory: 0, processing: 0>

2025-11-25 07:01:31,103 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], run-c29c5ace5dee2c6d55257138698971c6 NoneType: None

2025-11-25 07:01:31,103 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], run-51892ecc246ae453d07c744e49c67a88 NoneType: None

2025-11-25 07:01:31,102 - distributed.scheduler - ERROR - Couldn't gather keys {'run-51892ecc246ae453d07c744e49c67a88': [], 'run-c29c5ace5dee2c6d55257138698971c6': []} state: ['processing', 'processing'] workers: [] NoneType: None

2025-11-25 07:01:30,783 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:35391 failed: OSError: Timed out during handshake while connecting to tcp://172.21.25.104:35391 after 30 s

2025-11-25 07:01:30,783 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:35391 failed: OSError: Timed out during handshake while connecting to tcp://172.21.25.104:35391 after 30 s

2025-11-25 07:01:30,779 - distributed.scheduler - INFO - Close client connection: Client-worker-a2f21098-c571-11f0-8074-7e5a6871f9da

2025-11-25 07:01:30,775 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.104:35391', status: closed, memory: 0, processing: 0>

2025-11-25 07:01:30,774 - distributed.scheduler - INFO - Remove client Client-worker-a2f21098-c571-11f0-8074-7e5a6871f9da

2025-11-25 07:01:27,773 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:41965

2025-11-25 07:01:27,773 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:41965', status: closed, memory: 0, processing: 0>

2025-11-25 07:01:26,753 - distributed.scheduler - INFO - Close client connection: Client-worker-dda4e50d-c9bf-11f0-82c3-ba070f366fac

2025-11-25 07:01:26,739 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:36109', status: closed, memory: 0, processing: 0>

2025-11-25 07:01:26,738 - distributed.scheduler - INFO - Remove client Client-worker-dda4e50d-c9bf-11f0-82c3-ba070f366fac

2025-11-25 07:00:43,488 - distributed.scheduler - INFO - Receive client connection: Client-75c106a6-c9cc-11f0-8001-f2cd832f225c

2025-11-25 06:51:25,147 - distributed.scheduler - INFO - Close client connection: Client-424263af-c9c8-11f0-8001-f2cd832f225c

2025-11-25 06:51:25,147 - distributed.scheduler - INFO - Close client connection: Client-dbbd24fe-c9c6-11f0-8001-f2cd832f225c

2025-11-25 06:51:25,146 - distributed.scheduler - INFO - Close client connection: Client-a8c3fac1-c9c9-11f0-8001-f2cd832f225c

2025-11-25 06:51:25,145 - distributed.scheduler - INFO - Remove client Client-424263af-c9c8-11f0-8001-f2cd832f225c

2025-11-25 06:51:25,144 - distributed.scheduler - INFO - Remove client Client-dbbd24fe-c9c6-11f0-8001-f2cd832f225c

2025-11-25 06:51:25,142 - distributed.scheduler - INFO - Remove client Client-a8c3fac1-c9c9-11f0-8001-f2cd832f225c

2025-11-25 06:51:24,402 - distributed.scheduler - INFO - Close client connection: Client-0f4586c4-c9cb-11f0-8001-f2cd832f225c

2025-11-25 06:51:24,401 - distributed.scheduler - INFO - Remove client Client-0f4586c4-c9cb-11f0-8001-f2cd832f225c

2025-11-25 06:51:24,399 - distributed.scheduler - INFO - Remove client Client-0f4586c4-c9cb-11f0-8001-f2cd832f225c

2025-11-25 06:50:42,047 - distributed.scheduler - INFO - Receive client connection: Client-0f4586c4-c9cb-11f0-8001-f2cd832f225c

2025-11-25 06:40:40,572 - distributed.scheduler - INFO - Receive client connection: Client-a8c3fac1-c9c9-11f0-8001-f2cd832f225c

2025-11-25 06:30:39,098 - distributed.scheduler - INFO - Receive client connection: Client-424263af-c9c8-11f0-8001-f2cd832f225c

2025-11-25 06:20:37,607 - distributed.scheduler - INFO - Receive client connection: Client-dbbd24fe-c9c6-11f0-8001-f2cd832f225c

2025-11-25 06:11:15,968 - distributed.scheduler - INFO - Close client connection: Client-754077b0-c9c5-11f0-8001-f2cd832f225c

2025-11-25 06:11:15,968 - distributed.scheduler - INFO - Remove client Client-754077b0-c9c5-11f0-8001-f2cd832f225c

2025-11-25 06:11:15,966 - distributed.scheduler - INFO - Remove client Client-754077b0-c9c5-11f0-8001-f2cd832f225c

2025-11-25 06:10:36,168 - distributed.scheduler - INFO - Receive client connection: Client-754077b0-c9c5-11f0-8001-f2cd832f225c

2025-11-25 06:01:51,147 - distributed.scheduler - INFO - Close client connection: Client-0ebc362d-c9c4-11f0-8001-f2cd832f225c

2025-11-25 06:01:51,146 - distributed.scheduler - INFO - Remove client Client-0ebc362d-c9c4-11f0-8001-f2cd832f225c

2025-11-25 06:01:51,144 - distributed.scheduler - INFO - Remove client Client-0ebc362d-c9c4-11f0-8001-f2cd832f225c

2025-11-25 06:00:34,691 - distributed.scheduler - INFO - Receive client connection: Client-0ebc362d-c9c4-11f0-8001-f2cd832f225c

2025-11-25 05:51:11,012 - distributed.scheduler - INFO - Close client connection: Client-a8460d15-c9c2-11f0-8001-f2cd832f225c

2025-11-25 05:51:11,011 - distributed.scheduler - INFO - Remove client Client-a8460d15-c9c2-11f0-8001-f2cd832f225c

2025-11-25 05:51:11,009 - distributed.scheduler - INFO - Remove client Client-a8460d15-c9c2-11f0-8001-f2cd832f225c

2025-11-25 05:50:33,274 - distributed.scheduler - INFO - Receive client connection: Client-a8460d15-c9c2-11f0-8001-f2cd832f225c

2025-11-25 05:41:10,620 - distributed.scheduler - INFO - Close client connection: Client-41bf1d55-c9c1-11f0-8001-f2cd832f225c

2025-11-25 05:41:10,619 - distributed.scheduler - INFO - Remove client Client-41bf1d55-c9c1-11f0-8001-f2cd832f225c

2025-11-25 05:41:10,618 - distributed.scheduler - INFO - Remove client Client-41bf1d55-c9c1-11f0-8001-f2cd832f225c

2025-11-25 05:40:31,776 - distributed.scheduler - INFO - Receive client connection: Client-41bf1d55-c9c1-11f0-8001-f2cd832f225c

2025-11-25 05:31:42,741 - distributed.scheduler - INFO - Close client connection: Client-db41c367-c9bf-11f0-8001-f2cd832f225c

2025-11-25 05:31:42,740 - distributed.scheduler - INFO - Remove client Client-db41c367-c9bf-11f0-8001-f2cd832f225c

2025-11-25 05:31:42,739 - distributed.scheduler - INFO - Remove client Client-db41c367-c9bf-11f0-8001-f2cd832f225c

2025-11-25 05:30:34,313 - distributed.scheduler - INFO - Receive client connection: Client-worker-dda4e50d-c9bf-11f0-82c3-ba070f366fac

2025-11-25 05:30:30,323 - distributed.scheduler - INFO - Receive client connection: Client-db41c367-c9bf-11f0-8001-f2cd832f225c

2025-11-25 05:21:07,968 - distributed.scheduler - INFO - Close client connection: Client-74c2cd37-c9be-11f0-8001-f2cd832f225c

2025-11-25 05:21:07,967 - distributed.scheduler - INFO - Remove client Client-74c2cd37-c9be-11f0-8001-f2cd832f225c

2025-11-25 05:21:07,965 - distributed.scheduler - INFO - Remove client Client-74c2cd37-c9be-11f0-8001-f2cd832f225c

2025-11-25 05:20:28,865 - distributed.scheduler - INFO - Receive client connection: Client-74c2cd37-c9be-11f0-8001-f2cd832f225c

2025-11-25 05:11:07,008 - distributed.scheduler - INFO - Close client connection: Client-0e4f67ec-c9bd-11f0-8001-f2cd832f225c

2025-11-25 05:11:07,006 - distributed.scheduler - INFO - Remove client Client-0e4f67ec-c9bd-11f0-8001-f2cd832f225c

2025-11-25 05:11:07,005 - distributed.scheduler - INFO - Remove client Client-0e4f67ec-c9bd-11f0-8001-f2cd832f225c

2025-11-25 05:10:27,480 - distributed.scheduler - INFO - Receive client connection: Client-0e4f67ec-c9bd-11f0-8001-f2cd832f225c

2025-11-25 05:01:35,575 - distributed.scheduler - INFO - Close client connection: Client-a7da91fb-c9bb-11f0-8001-f2cd832f225c

2025-11-25 05:01:35,574 - distributed.scheduler - INFO - Remove client Client-a7da91fb-c9bb-11f0-8001-f2cd832f225c

2025-11-25 05:01:35,572 - distributed.scheduler - INFO - Remove client Client-a7da91fb-c9bb-11f0-8001-f2cd832f225c

2025-11-25 05:00:26,090 - distributed.scheduler - INFO - Receive client connection: Client-a7da91fb-c9bb-11f0-8001-f2cd832f225c

2025-11-25 04:51:04,664 - distributed.scheduler - INFO - Close client connection: Client-4158e5bf-c9ba-11f0-8001-f2cd832f225c

2025-11-25 04:51:04,663 - distributed.scheduler - INFO - Remove client Client-4158e5bf-c9ba-11f0-8001-f2cd832f225c

2025-11-25 04:51:04,661 - distributed.scheduler - INFO - Remove client Client-4158e5bf-c9ba-11f0-8001-f2cd832f225c

2025-11-25 04:50:24,611 - distributed.scheduler - INFO - Receive client connection: Client-4158e5bf-c9ba-11f0-8001-f2cd832f225c

2025-11-25 04:41:03,541 - distributed.scheduler - INFO - Close client connection: Client-dae7fae8-c9b8-11f0-8001-f2cd832f225c

2025-11-25 04:41:03,538 - distributed.scheduler - INFO - Remove client Client-dae7fae8-c9b8-11f0-8001-f2cd832f225c

2025-11-25 04:41:03,536 - distributed.scheduler - INFO - Remove client Client-dae7fae8-c9b8-11f0-8001-f2cd832f225c

2025-11-25 04:40:23,257 - distributed.scheduler - INFO - Receive client connection: Client-dae7fae8-c9b8-11f0-8001-f2cd832f225c

2025-11-25 04:31:39,141 - distributed.scheduler - INFO - Close client connection: Client-75a29a2a-c9b7-11f0-8001-f2cd832f225c

2025-11-25 04:31:39,140 - distributed.scheduler - INFO - Remove client Client-75a29a2a-c9b7-11f0-8001-f2cd832f225c

2025-11-25 04:31:39,138 - distributed.scheduler - INFO - Remove client Client-75a29a2a-c9b7-11f0-8001-f2cd832f225c

2025-11-25 04:30:23,849 - distributed.scheduler - INFO - Receive client connection: Client-75a29a2a-c9b7-11f0-8001-f2cd832f225c

2025-11-25 04:21:02,734 - distributed.scheduler - INFO - Close client connection: Client-40f751bf-c9b3-11f0-8001-f2cd832f225c

2025-11-25 04:21:02,733 - distributed.scheduler - INFO - Close client connection: Client-a77b6fe3-c9b4-11f0-8001-f2cd832f225c

2025-11-25 04:21:02,732 - distributed.scheduler - INFO - Remove client Client-40f751bf-c9b3-11f0-8001-f2cd832f225c

2025-11-25 04:21:02,731 - distributed.scheduler - INFO - Remove client Client-a77b6fe3-c9b4-11f0-8001-f2cd832f225c

2025-11-25 04:21:02,066 - distributed.scheduler - INFO - Close client connection: Client-0def7a3b-c9b6-11f0-8001-f2cd832f225c

2025-11-25 04:21:02,065 - distributed.scheduler - INFO - Remove client Client-0def7a3b-c9b6-11f0-8001-f2cd832f225c

2025-11-25 04:21:02,063 - distributed.scheduler - INFO - Remove client Client-0def7a3b-c9b6-11f0-8001-f2cd832f225c

2025-11-25 04:20:20,371 - distributed.scheduler - INFO - Receive client connection: Client-0def7a3b-c9b6-11f0-8001-f2cd832f225c

2025-11-25 04:10:18,982 - distributed.scheduler - INFO - Receive client connection: Client-a77b6fe3-c9b4-11f0-8001-f2cd832f225c

2025-11-25 04:00:17,500 - distributed.scheduler - INFO - Receive client connection: Client-40f751bf-c9b3-11f0-8001-f2cd832f225c

2025-11-25 03:50:59,309 - distributed.scheduler - INFO - Close client connection: Client-da844214-c9b1-11f0-8001-f2cd832f225c

2025-11-25 03:50:59,307 - distributed.scheduler - INFO - Remove client Client-da844214-c9b1-11f0-8001-f2cd832f225c

2025-11-25 03:50:59,306 - distributed.scheduler - INFO - Remove client Client-da844214-c9b1-11f0-8001-f2cd832f225c

2025-11-25 03:50:16,126 - distributed.scheduler - INFO - Receive client connection: Client-da844214-c9b1-11f0-8001-f2cd832f225c

2025-11-25 03:40:53,351 - distributed.scheduler - INFO - Close client connection: Client-74039e6e-c9b0-11f0-8001-f2cd832f225c

2025-11-25 03:40:53,349 - distributed.scheduler - INFO - Remove client Client-74039e6e-c9b0-11f0-8001-f2cd832f225c

2025-11-25 03:40:53,347 - distributed.scheduler - INFO - Remove client Client-74039e6e-c9b0-11f0-8001-f2cd832f225c

2025-11-25 03:40:14,658 - distributed.scheduler - INFO - Receive client connection: Client-74039e6e-c9b0-11f0-8001-f2cd832f225c

2025-11-25 03:31:33,627 - distributed.scheduler - INFO - Close client connection: Client-0d7f050f-c9af-11f0-8001-f2cd832f225c

2025-11-25 03:31:33,627 - distributed.scheduler - INFO - Remove client Client-0d7f050f-c9af-11f0-8001-f2cd832f225c

2025-11-25 03:31:33,626 - distributed.scheduler - INFO - Remove client Client-0d7f050f-c9af-11f0-8001-f2cd832f225c

2025-11-25 03:30:13,157 - distributed.scheduler - INFO - Receive client connection: Client-0d7f050f-c9af-11f0-8001-f2cd832f225c

2025-11-25 03:20:53,988 - distributed.scheduler - INFO - Close client connection: Client-a7022237-c9ad-11f0-8001-f2cd832f225c

2025-11-25 03:20:53,986 - distributed.scheduler - INFO - Remove client Client-a7022237-c9ad-11f0-8001-f2cd832f225c

2025-11-25 03:20:53,984 - distributed.scheduler - INFO - Remove client Client-a7022237-c9ad-11f0-8001-f2cd832f225c

2025-11-25 03:20:11,718 - distributed.scheduler - INFO - Receive client connection: Client-a7022237-c9ad-11f0-8001-f2cd832f225c

2025-11-25 03:10:54,789 - distributed.scheduler - INFO - Close client connection: Client-4083889c-c9ac-11f0-8001-f2cd832f225c

2025-11-25 03:10:54,788 - distributed.scheduler - INFO - Remove client Client-4083889c-c9ac-11f0-8001-f2cd832f225c

2025-11-25 03:10:54,786 - distributed.scheduler - INFO - Remove client Client-4083889c-c9ac-11f0-8001-f2cd832f225c

2025-11-25 03:10:10,277 - distributed.scheduler - INFO - Receive client connection: Client-4083889c-c9ac-11f0-8001-f2cd832f225c

2025-11-25 03:01:23,464 - distributed.scheduler - INFO - Close client connection: Client-d9fd8b68-c9aa-11f0-8001-f2cd832f225c

2025-11-25 03:01:23,463 - distributed.scheduler - INFO - Remove client Client-d9fd8b68-c9aa-11f0-8001-f2cd832f225c

2025-11-25 03:01:23,461 - distributed.scheduler - INFO - Remove client Client-d9fd8b68-c9aa-11f0-8001-f2cd832f225c

2025-11-25 03:00:08,761 - distributed.scheduler - INFO - Receive client connection: Client-d9fd8b68-c9aa-11f0-8001-f2cd832f225c

2025-11-25 02:50:50,705 - distributed.scheduler - INFO - Close client connection: Client-736cf846-c9a9-11f0-8001-f2cd832f225c

2025-11-25 02:50:50,704 - distributed.scheduler - INFO - Remove client Client-736cf846-c9a9-11f0-8001-f2cd832f225c

2025-11-25 02:50:50,703 - distributed.scheduler - INFO - Remove client Client-736cf846-c9a9-11f0-8001-f2cd832f225c

2025-11-25 02:50:07,193 - distributed.scheduler - INFO - Receive client connection: Client-736cf846-c9a9-11f0-8001-f2cd832f225c

2025-11-25 02:40:44,991 - distributed.scheduler - INFO - Close client connection: Client-0ce983a9-c9a8-11f0-8001-f2cd832f225c

2025-11-25 02:40:44,990 - distributed.scheduler - INFO - Remove client Client-0ce983a9-c9a8-11f0-8001-f2cd832f225c

2025-11-25 02:40:44,988 - distributed.scheduler - INFO - Remove client Client-0ce983a9-c9a8-11f0-8001-f2cd832f225c

2025-11-25 02:40:05,708 - distributed.scheduler - INFO - Receive client connection: Client-0ce983a9-c9a8-11f0-8001-f2cd832f225c

2025-11-25 02:31:59,583 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:36109

2025-11-25 02:31:59,581 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:36109', status: closed, memory: 0, processing: 0>

2025-11-25 02:31:58,823 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:39035', status: closed, memory: 0, processing: 0>

2025-11-25 02:31:48,189 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:39035

2025-11-25 02:31:48,186 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:39035', status: closed, memory: 0, processing: 0>

2025-11-25 02:31:47,426 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:43225', status: closed, memory: 0, processing: 0>

2025-11-25 02:31:35,143 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:43225

2025-11-25 02:31:35,141 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:43225', status: closed, memory: 0, processing: 0>

2025-11-25 02:31:34,225 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:42603', status: closed, memory: 0, processing: 0>

2025-11-25 02:31:25,923 - distributed.scheduler - INFO - Close client connection: Client-2f2d004d-c969-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,923 - distributed.scheduler - INFO - Close client connection: Client-ea07f250-c967-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,923 - distributed.scheduler - INFO - Close client connection: Client-933a733c-c96a-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,923 - distributed.scheduler - INFO - Close client connection: Client-60344308-c96d-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,923 - distributed.scheduler - INFO - Close client connection: Client-f9b8c267-c96b-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,922 - distributed.scheduler - INFO - Close client connection: Client-c6bf70da-c96e-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,922 - distributed.scheduler - INFO - Close client connection: Client-2d3c0d94-c970-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,922 - distributed.scheduler - INFO - Close client connection: Client-9412a945-c971-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,922 - distributed.scheduler - INFO - Close client connection: Client-fa3e1844-c972-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,922 - distributed.scheduler - INFO - Close client connection: Client-60c0e6df-c974-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,922 - distributed.scheduler - INFO - Close client connection: Client-c73b191b-c975-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,922 - distributed.scheduler - INFO - Close client connection: Client-2dbe873a-c977-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,922 - distributed.scheduler - INFO - Close client connection: Client-943ce947-c978-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,922 - distributed.scheduler - INFO - Close client connection: Client-fac743ca-c979-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,922 - distributed.scheduler - INFO - Close client connection: Client-613f7a3a-c97b-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,922 - distributed.scheduler - INFO - Close client connection: Client-c7b4f663-c97c-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,921 - distributed.scheduler - INFO - Close client connection: Client-2e349052-c97e-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,921 - distributed.scheduler - INFO - Close client connection: Client-94b432f0-c97f-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,921 - distributed.scheduler - INFO - Close client connection: Client-fb2ca2c4-c980-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,921 - distributed.scheduler - INFO - Close client connection: Client-61ab06f5-c982-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,921 - distributed.scheduler - INFO - Close client connection: Client-c819ab31-c983-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,921 - distributed.scheduler - INFO - Close client connection: Client-2e9b3390-c985-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,921 - distributed.scheduler - INFO - Close client connection: Client-95150b83-c986-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,921 - distributed.scheduler - INFO - Close client connection: Client-fb9820ab-c987-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,921 - distributed.scheduler - INFO - Close client connection: Client-6215249e-c989-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,921 - distributed.scheduler - INFO - Close client connection: Client-2f12ad99-c98c-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,921 - distributed.scheduler - INFO - Close client connection: Client-c8975d84-c98a-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,920 - distributed.scheduler - INFO - Close client connection: Client-95935098-c98d-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,920 - distributed.scheduler - INFO - Close client connection: Client-fc126a64-c98e-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,919 - distributed.scheduler - INFO - Close client connection: Client-c90736be-c991-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,919 - distributed.scheduler - INFO - Close client connection: Client-628ed2a3-c990-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,919 - distributed.scheduler - INFO - Close client connection: Client-2f7cea1f-c993-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,919 - distributed.scheduler - INFO - Close client connection: Client-95ff0aa8-c994-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,919 - distributed.scheduler - INFO - Close client connection: Client-fc7a52db-c995-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,919 - distributed.scheduler - INFO - Close client connection: Client-62f0b284-c997-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,919 - distributed.scheduler - INFO - Close client connection: Client-c9628b60-c998-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,918 - distributed.scheduler - INFO - Close client connection: Client-2fe0bc69-c99a-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,918 - distributed.scheduler - INFO - Close client connection: Client-965ea9d8-c99b-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,918 - distributed.scheduler - INFO - Close client connection: Client-fcd7c480-c99c-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,918 - distributed.scheduler - INFO - Close client connection: Client-30398dd1-c9a1-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,918 - distributed.scheduler - INFO - Close client connection: Client-96bc5700-c9a2-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,918 - distributed.scheduler - INFO - Close client connection: Client-63501cd7-c99e-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,918 - distributed.scheduler - INFO - Close client connection: Client-c9c46671-c99f-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,918 - distributed.scheduler - INFO - Close client connection: Client-da482deb-c9a3-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,918 - distributed.scheduler - INFO - Close client connection: Client-3ff01f94-c9a5-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,914 - distributed.scheduler - INFO - Remove client Client-2f2d004d-c969-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,914 - distributed.scheduler - INFO - Remove client Client-ea07f250-c967-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,914 - distributed.scheduler - INFO - Remove client Client-933a733c-c96a-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,913 - distributed.scheduler - INFO - Remove client Client-60344308-c96d-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,913 - distributed.scheduler - INFO - Remove client Client-f9b8c267-c96b-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,913 - distributed.scheduler - INFO - Remove client Client-c6bf70da-c96e-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,913 - distributed.scheduler - INFO - Remove client Client-2d3c0d94-c970-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,912 - distributed.scheduler - INFO - Remove client Client-9412a945-c971-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,912 - distributed.scheduler - INFO - Remove client Client-fa3e1844-c972-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,912 - distributed.scheduler - INFO - Remove client Client-60c0e6df-c974-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,912 - distributed.scheduler - INFO - Remove client Client-c73b191b-c975-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,911 - distributed.scheduler - INFO - Remove client Client-2dbe873a-c977-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,911 - distributed.scheduler - INFO - Remove client Client-943ce947-c978-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,911 - distributed.scheduler - INFO - Remove client Client-fac743ca-c979-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,910 - distributed.scheduler - INFO - Remove client Client-613f7a3a-c97b-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,910 - distributed.scheduler - INFO - Remove client Client-c7b4f663-c97c-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,910 - distributed.scheduler - INFO - Remove client Client-2e349052-c97e-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,910 - distributed.scheduler - INFO - Remove client Client-94b432f0-c97f-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,909 - distributed.scheduler - INFO - Remove client Client-fb2ca2c4-c980-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,909 - distributed.scheduler - INFO - Remove client Client-61ab06f5-c982-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,909 - distributed.scheduler - INFO - Remove client Client-c819ab31-c983-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,908 - distributed.scheduler - INFO - Remove client Client-2e9b3390-c985-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,908 - distributed.scheduler - INFO - Remove client Client-95150b83-c986-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,908 - distributed.scheduler - INFO - Remove client Client-fb9820ab-c987-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,908 - distributed.scheduler - INFO - Remove client Client-6215249e-c989-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,907 - distributed.scheduler - INFO - Remove client Client-2f12ad99-c98c-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,907 - distributed.scheduler - INFO - Remove client Client-c8975d84-c98a-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,906 - distributed.scheduler - INFO - Remove client Client-95935098-c98d-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,906 - distributed.scheduler - INFO - Remove client Client-fc126a64-c98e-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,898 - distributed.scheduler - INFO - Remove client Client-c90736be-c991-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,898 - distributed.scheduler - INFO - Remove client Client-628ed2a3-c990-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,898 - distributed.scheduler - INFO - Remove client Client-2f7cea1f-c993-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,897 - distributed.scheduler - INFO - Remove client Client-95ff0aa8-c994-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,897 - distributed.scheduler - INFO - Remove client Client-fc7a52db-c995-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,896 - distributed.scheduler - INFO - Remove client Client-62f0b284-c997-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,895 - distributed.scheduler - INFO - Remove client Client-c9628b60-c998-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,895 - distributed.scheduler - INFO - Remove client Client-2fe0bc69-c99a-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,894 - distributed.scheduler - INFO - Remove client Client-965ea9d8-c99b-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,893 - distributed.scheduler - INFO - Remove client Client-fcd7c480-c99c-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,893 - distributed.scheduler - INFO - Remove client Client-30398dd1-c9a1-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,892 - distributed.scheduler - INFO - Remove client Client-96bc5700-c9a2-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,891 - distributed.scheduler - INFO - Remove client Client-63501cd7-c99e-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,891 - distributed.scheduler - INFO - Remove client Client-c9c46671-c99f-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,889 - distributed.scheduler - INFO - Remove client Client-da482deb-c9a3-11f0-8001-f2cd832f225c

2025-11-25 02:31:25,889 - distributed.scheduler - INFO - Remove client Client-3ff01f94-c9a5-11f0-8001-f2cd832f225c

2025-11-25 02:31:24,949 - distributed.scheduler - INFO - Close client connection: Client-a66d088e-c9a6-11f0-8001-f2cd832f225c

2025-11-25 02:31:24,947 - distributed.scheduler - INFO - Remove client Client-a66d088e-c9a6-11f0-8001-f2cd832f225c

2025-11-25 02:31:24,945 - distributed.scheduler - INFO - Remove client Client-a66d088e-c9a6-11f0-8001-f2cd832f225c

2025-11-25 02:31:18,991 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:42603

2025-11-25 02:31:18,990 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:42603', status: closed, memory: 0, processing: 0>

2025-11-25 02:31:18,044 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:33081', status: closed, memory: 0, processing: 0>

2025-11-25 02:30:48,202 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.210:33081

2025-11-25 02:30:48,199 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.210:33081', status: closed, memory: 0, processing: 0>

2025-11-25 02:30:47,339 - distributed.scheduler - INFO - Close client connection: Client-worker-a3aadfd9-c571-11f0-8074-ba070f366fac

2025-11-25 02:30:47,334 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.210:32841', status: closed, memory: 0, processing: 0>

2025-11-25 02:30:47,333 - distributed.scheduler - INFO - Remove client Client-worker-a3aadfd9-c571-11f0-8074-ba070f366fac

2025-11-25 02:30:04,267 - distributed.scheduler - INFO - Receive client connection: Client-a66d088e-c9a6-11f0-8001-f2cd832f225c

2025-11-25 02:20:02,820 - distributed.scheduler - INFO - Receive client connection: Client-3ff01f94-c9a5-11f0-8001-f2cd832f225c

2025-11-25 02:10:02,774 - distributed.scheduler - INFO - Receive client connection: Client-da482deb-c9a3-11f0-8001-f2cd832f225c

2025-11-25 02:00:59,943 - distributed.scheduler - INFO - Receive client connection: Client-96bc5700-c9a2-11f0-8001-f2cd832f225c

2025-11-25 01:50:58,472 - distributed.scheduler - INFO - Receive client connection: Client-30398dd1-c9a1-11f0-8001-f2cd832f225c

2025-11-25 01:40:57,082 - distributed.scheduler - INFO - Receive client connection: Client-c9c46671-c99f-11f0-8001-f2cd832f225c

2025-11-25 01:30:55,699 - distributed.scheduler - INFO - Receive client connection: Client-63501cd7-c99e-11f0-8001-f2cd832f225c

2025-11-25 01:20:54,275 - distributed.scheduler - INFO - Receive client connection: Client-fcd7c480-c99c-11f0-8001-f2cd832f225c

2025-11-25 01:10:52,854 - distributed.scheduler - INFO - Receive client connection: Client-965ea9d8-c99b-11f0-8001-f2cd832f225c

2025-11-25 01:00:51,418 - distributed.scheduler - INFO - Receive client connection: Client-2fe0bc69-c99a-11f0-8001-f2cd832f225c

2025-11-25 00:50:49,956 - distributed.scheduler - INFO - Receive client connection: Client-c9628b60-c998-11f0-8001-f2cd832f225c

2025-11-25 00:40:48,589 - distributed.scheduler - INFO - Receive client connection: Client-62f0b284-c997-11f0-8001-f2cd832f225c

2025-11-25 00:30:47,186 - distributed.scheduler - INFO - Receive client connection: Client-fc7a52db-c995-11f0-8001-f2cd832f225c

2025-11-25 00:20:45,760 - distributed.scheduler - INFO - Receive client connection: Client-95ff0aa8-c994-11f0-8001-f2cd832f225c

2025-11-25 00:10:44,275 - distributed.scheduler - INFO - Receive client connection: Client-2f7cea1f-c993-11f0-8001-f2cd832f225c

2025-11-25 00:00:42,877 - distributed.scheduler - INFO - Receive client connection: Client-c90736be-c991-11f0-8001-f2cd832f225c

2025-11-24 23:50:41,463 - distributed.scheduler - INFO - Receive client connection: Client-628ed2a3-c990-11f0-8001-f2cd832f225c

2025-11-24 23:40:40,029 - distributed.scheduler - INFO - Receive client connection: Client-fc126a64-c98e-11f0-8001-f2cd832f225c

2025-11-24 23:30:38,573 - distributed.scheduler - INFO - Receive client connection: Client-95935098-c98d-11f0-8001-f2cd832f225c

2025-11-24 23:20:37,103 - distributed.scheduler - INFO - Receive client connection: Client-2f12ad99-c98c-11f0-8001-f2cd832f225c

2025-11-24 23:10:35,674 - distributed.scheduler - INFO - Receive client connection: Client-c8975d84-c98a-11f0-8001-f2cd832f225c

2025-11-24 23:00:34,191 - distributed.scheduler - INFO - Receive client connection: Client-6215249e-c989-11f0-8001-f2cd832f225c

2025-11-24 22:50:32,753 - distributed.scheduler - INFO - Receive client connection: Client-fb9820ab-c987-11f0-8001-f2cd832f225c

2025-11-24 22:40:31,268 - distributed.scheduler - INFO - Receive client connection: Client-95150b83-c986-11f0-8001-f2cd832f225c

2025-11-24 22:30:29,843 - distributed.scheduler - INFO - Receive client connection: Client-2e9b3390-c985-11f0-8001-f2cd832f225c

2025-11-24 22:20:28,369 - distributed.scheduler - INFO - Receive client connection: Client-c819ab31-c983-11f0-8001-f2cd832f225c

2025-11-24 22:10:27,017 - distributed.scheduler - INFO - Receive client connection: Client-61ab06f5-c982-11f0-8001-f2cd832f225c

2025-11-24 22:00:25,574 - distributed.scheduler - INFO - Receive client connection: Client-fb2ca2c4-c980-11f0-8001-f2cd832f225c

2025-11-24 21:50:24,152 - distributed.scheduler - INFO - Receive client connection: Client-94b432f0-c97f-11f0-8001-f2cd832f225c

2025-11-24 21:40:22,696 - distributed.scheduler - INFO - Receive client connection: Client-2e349052-c97e-11f0-8001-f2cd832f225c

2025-11-24 21:30:21,235 - distributed.scheduler - INFO - Receive client connection: Client-c7b4f663-c97c-11f0-8001-f2cd832f225c

2025-11-24 21:20:19,835 - distributed.scheduler - INFO - Receive client connection: Client-613f7a3a-c97b-11f0-8001-f2cd832f225c

2025-11-24 21:10:18,428 - distributed.scheduler - INFO - Receive client connection: Client-fac743ca-c979-11f0-8001-f2cd832f225c

2025-11-24 21:00:16,897 - distributed.scheduler - INFO - Receive client connection: Client-943ce947-c978-11f0-8001-f2cd832f225c

2025-11-24 20:50:15,453 - distributed.scheduler - INFO - Receive client connection: Client-2dbe873a-c977-11f0-8001-f2cd832f225c

2025-11-24 20:40:13,957 - distributed.scheduler - INFO - Receive client connection: Client-c73b191b-c975-11f0-8001-f2cd832f225c

2025-11-24 20:30:12,534 - distributed.scheduler - INFO - Receive client connection: Client-60c0e6df-c974-11f0-8001-f2cd832f225c

2025-11-24 20:20:11,053 - distributed.scheduler - INFO - Receive client connection: Client-fa3e1844-c972-11f0-8001-f2cd832f225c

2025-11-24 20:10:10,205 - distributed.scheduler - INFO - Receive client connection: Client-9412a945-c971-11f0-8001-f2cd832f225c

2025-11-24 20:00:08,107 - distributed.scheduler - INFO - Receive client connection: Client-2d3c0d94-c970-11f0-8001-f2cd832f225c

2025-11-24 19:50:06,670 - distributed.scheduler - INFO - Receive client connection: Client-c6bf70da-c96e-11f0-8001-f2cd832f225c

2025-11-24 19:40:05,135 - distributed.scheduler - INFO - Receive client connection: Client-60344308-c96d-11f0-8001-f2cd832f225c

2025-11-24 19:30:03,700 - distributed.scheduler - INFO - Receive client connection: Client-f9b8c267-c96b-11f0-8001-f2cd832f225c

2025-11-24 19:20:02,249 - distributed.scheduler - INFO - Receive client connection: Client-933a733c-c96a-11f0-8001-f2cd832f225c

2025-11-24 19:10:04,894 - distributed.scheduler - INFO - Receive client connection: Client-2f2d004d-c969-11f0-8001-f2cd832f225c

2025-11-24 19:00:59,399 - distributed.scheduler - INFO - Receive client connection: Client-ea07f250-c967-11f0-8001-f2cd832f225c

2025-11-24 18:51:35,004 - distributed.scheduler - INFO - Close client connection: Client-838afffb-c966-11f0-8001-f2cd832f225c

2025-11-24 18:51:35,003 - distributed.scheduler - INFO - Remove client Client-838afffb-c966-11f0-8001-f2cd832f225c

2025-11-24 18:51:35,001 - distributed.scheduler - INFO - Remove client Client-838afffb-c966-11f0-8001-f2cd832f225c

2025-11-24 18:50:57,956 - distributed.scheduler - INFO - Receive client connection: Client-838afffb-c966-11f0-8001-f2cd832f225c

2025-11-24 18:41:35,732 - distributed.scheduler - INFO - Close client connection: Client-1d156b54-c965-11f0-8001-f2cd832f225c

2025-11-24 18:41:35,731 - distributed.scheduler - INFO - Remove client Client-1d156b54-c965-11f0-8001-f2cd832f225c

2025-11-24 18:41:35,729 - distributed.scheduler - INFO - Remove client Client-1d156b54-c965-11f0-8001-f2cd832f225c

2025-11-24 18:40:56,555 - distributed.scheduler - INFO - Receive client connection: Client-1d156b54-c965-11f0-8001-f2cd832f225c

2025-11-24 18:32:07,742 - distributed.scheduler - INFO - Close client connection: Client-b69a4c5f-c963-11f0-8001-f2cd832f225c

2025-11-24 18:32:07,740 - distributed.scheduler - INFO - Remove client Client-b69a4c5f-c963-11f0-8001-f2cd832f225c

2025-11-24 18:32:07,738 - distributed.scheduler - INFO - Remove client Client-b69a4c5f-c963-11f0-8001-f2cd832f225c

2025-11-24 18:30:55,126 - distributed.scheduler - INFO - Receive client connection: Client-b69a4c5f-c963-11f0-8001-f2cd832f225c

2025-11-24 18:21:34,086 - distributed.scheduler - INFO - Close client connection: Client-501fdf9a-c962-11f0-8001-f2cd832f225c

2025-11-24 18:21:34,084 - distributed.scheduler - INFO - Remove client Client-501fdf9a-c962-11f0-8001-f2cd832f225c

2025-11-24 18:21:34,083 - distributed.scheduler - INFO - Remove client Client-501fdf9a-c962-11f0-8001-f2cd832f225c

2025-11-24 18:20:53,698 - distributed.scheduler - INFO - Receive client connection: Client-501fdf9a-c962-11f0-8001-f2cd832f225c

2025-11-24 18:11:31,306 - distributed.scheduler - INFO - Close client connection: Client-e9923f57-c960-11f0-8001-f2cd832f225c

2025-11-24 18:11:31,305 - distributed.scheduler - INFO - Remove client Client-e9923f57-c960-11f0-8001-f2cd832f225c

2025-11-24 18:11:31,303 - distributed.scheduler - INFO - Remove client Client-e9923f57-c960-11f0-8001-f2cd832f225c

2025-11-24 18:10:52,148 - distributed.scheduler - INFO - Receive client connection: Client-e9923f57-c960-11f0-8001-f2cd832f225c

2025-11-24 18:02:04,308 - distributed.scheduler - INFO - Close client connection: Client-83136c0e-c95f-11f0-8001-f2cd832f225c

2025-11-24 18:02:04,307 - distributed.scheduler - INFO - Remove client Client-83136c0e-c95f-11f0-8001-f2cd832f225c

2025-11-24 18:02:04,305 - distributed.scheduler - INFO - Remove client Client-83136c0e-c95f-11f0-8001-f2cd832f225c

2025-11-24 18:00:50,694 - distributed.scheduler - INFO - Receive client connection: Client-83136c0e-c95f-11f0-8001-f2cd832f225c

2025-11-24 17:51:31,314 - distributed.scheduler - INFO - Close client connection: Client-1c8f8a22-c95e-11f0-8001-f2cd832f225c

2025-11-24 17:51:31,313 - distributed.scheduler - INFO - Remove client Client-1c8f8a22-c95e-11f0-8001-f2cd832f225c

2025-11-24 17:51:31,311 - distributed.scheduler - INFO - Remove client Client-1c8f8a22-c95e-11f0-8001-f2cd832f225c

2025-11-24 17:50:49,236 - distributed.scheduler - INFO - Receive client connection: Client-1c8f8a22-c95e-11f0-8001-f2cd832f225c

2025-11-24 17:41:27,240 - distributed.scheduler - INFO - Close client connection: Client-b60b528d-c95c-11f0-8001-f2cd832f225c

2025-11-24 17:41:27,239 - distributed.scheduler - INFO - Remove client Client-b60b528d-c95c-11f0-8001-f2cd832f225c

2025-11-24 17:41:27,237 - distributed.scheduler - INFO - Remove client Client-b60b528d-c95c-11f0-8001-f2cd832f225c

2025-11-24 17:40:47,718 - distributed.scheduler - INFO - Receive client connection: Client-b60b528d-c95c-11f0-8001-f2cd832f225c

2025-11-24 17:31:56,528 - distributed.scheduler - INFO - Close client connection: Client-4f98b5ab-c95b-11f0-8001-f2cd832f225c

2025-11-24 17:31:56,527 - distributed.scheduler - INFO - Remove client Client-4f98b5ab-c95b-11f0-8001-f2cd832f225c

2025-11-24 17:31:56,525 - distributed.scheduler - INFO - Remove client Client-4f98b5ab-c95b-11f0-8001-f2cd832f225c

2025-11-24 17:30:51,486 - distributed.scheduler - INFO - Receive client connection: Client-worker-529dc737-c95b-11f0-82a8-e2572cf160eb

2025-11-24 17:30:46,336 - distributed.scheduler - INFO - Receive client connection: Client-4f98b5ab-c95b-11f0-8001-f2cd832f225c

2025-11-24 17:21:26,842 - distributed.scheduler - INFO - Close client connection: Client-e9239dc4-c959-11f0-8001-f2cd832f225c

2025-11-24 17:21:26,840 - distributed.scheduler - INFO - Remove client Client-e9239dc4-c959-11f0-8001-f2cd832f225c

2025-11-24 17:21:26,838 - distributed.scheduler - INFO - Remove client Client-e9239dc4-c959-11f0-8001-f2cd832f225c

2025-11-24 17:20:44,941 - distributed.scheduler - INFO - Receive client connection: Client-e9239dc4-c959-11f0-8001-f2cd832f225c

2025-11-24 17:11:25,023 - distributed.scheduler - INFO - Close client connection: Client-82a08ba5-c958-11f0-8001-f2cd832f225c

2025-11-24 17:11:25,021 - distributed.scheduler - INFO - Remove client Client-82a08ba5-c958-11f0-8001-f2cd832f225c

2025-11-24 17:11:25,019 - distributed.scheduler - INFO - Remove client Client-82a08ba5-c958-11f0-8001-f2cd832f225c

2025-11-24 17:10:43,464 - distributed.scheduler - INFO - Receive client connection: Client-82a08ba5-c958-11f0-8001-f2cd832f225c

2025-11-24 17:01:57,449 - distributed.scheduler - INFO - Close client connection: Client-1c2e87ea-c957-11f0-8001-f2cd832f225c

2025-11-24 17:01:57,448 - distributed.scheduler - INFO - Remove client Client-1c2e87ea-c957-11f0-8001-f2cd832f225c

2025-11-24 17:01:57,446 - distributed.scheduler - INFO - Remove client Client-1c2e87ea-c957-11f0-8001-f2cd832f225c

2025-11-24 17:00:42,133 - distributed.scheduler - INFO - Receive client connection: Client-1c2e87ea-c957-11f0-8001-f2cd832f225c

2025-11-24 16:51:21,059 - distributed.scheduler - INFO - Close client connection: Client-b59fb86f-c955-11f0-8001-f2cd832f225c

2025-11-24 16:51:21,058 - distributed.scheduler - INFO - Remove client Client-b59fb86f-c955-11f0-8001-f2cd832f225c

2025-11-24 16:51:21,057 - distributed.scheduler - INFO - Remove client Client-b59fb86f-c955-11f0-8001-f2cd832f225c

2025-11-24 16:50:40,534 - distributed.scheduler - INFO - Receive client connection: Client-b59fb86f-c955-11f0-8001-f2cd832f225c

2025-11-24 16:41:19,872 - distributed.scheduler - INFO - Close client connection: Client-4f1b3151-c954-11f0-8001-f2cd832f225c

2025-11-24 16:41:19,871 - distributed.scheduler - INFO - Remove client Client-4f1b3151-c954-11f0-8001-f2cd832f225c

2025-11-24 16:41:19,870 - distributed.scheduler - INFO - Remove client Client-4f1b3151-c954-11f0-8001-f2cd832f225c

2025-11-24 16:40:39,040 - distributed.scheduler - INFO - Receive client connection: Client-4f1b3151-c954-11f0-8001-f2cd832f225c

2025-11-24 16:31:53,431 - distributed.scheduler - INFO - Close client connection: Client-e899b0c1-c952-11f0-8001-f2cd832f225c

2025-11-24 16:31:53,430 - distributed.scheduler - INFO - Remove client Client-e899b0c1-c952-11f0-8001-f2cd832f225c

2025-11-24 16:31:53,428 - distributed.scheduler - INFO - Remove client Client-e899b0c1-c952-11f0-8001-f2cd832f225c

2025-11-24 16:30:37,564 - distributed.scheduler - INFO - Receive client connection: Client-e899b0c1-c952-11f0-8001-f2cd832f225c

2025-11-24 16:21:16,830 - distributed.scheduler - INFO - Close client connection: Client-82126fba-c951-11f0-8001-f2cd832f225c

2025-11-24 16:21:16,827 - distributed.scheduler - INFO - Remove client Client-82126fba-c951-11f0-8001-f2cd832f225c

2025-11-24 16:21:16,825 - distributed.scheduler - INFO - Remove client Client-82126fba-c951-11f0-8001-f2cd832f225c

2025-11-24 16:20:36,055 - distributed.scheduler - INFO - Receive client connection: Client-82126fba-c951-11f0-8001-f2cd832f225c

2025-11-24 16:11:16,815 - distributed.scheduler - INFO - Close client connection: Client-1b903e31-c950-11f0-8001-f2cd832f225c

2025-11-24 16:11:16,813 - distributed.scheduler - INFO - Remove client Client-1b903e31-c950-11f0-8001-f2cd832f225c

2025-11-24 16:11:16,811 - distributed.scheduler - INFO - Remove client Client-1b903e31-c950-11f0-8001-f2cd832f225c

2025-11-24 16:10:34,578 - distributed.scheduler - INFO - Receive client connection: Client-1b903e31-c950-11f0-8001-f2cd832f225c

2025-11-24 16:01:48,953 - distributed.scheduler - INFO - Close client connection: Client-b51237bd-c94e-11f0-8001-f2cd832f225c

2025-11-24 16:01:48,952 - distributed.scheduler - INFO - Remove client Client-b51237bd-c94e-11f0-8001-f2cd832f225c

2025-11-24 16:01:48,950 - distributed.scheduler - INFO - Remove client Client-b51237bd-c94e-11f0-8001-f2cd832f225c

2025-11-24 16:00:33,169 - distributed.scheduler - INFO - Receive client connection: Client-b51237bd-c94e-11f0-8001-f2cd832f225c

2025-11-24 15:51:15,958 - distributed.scheduler - INFO - Close client connection: Client-4e90baf0-c94d-11f0-8001-f2cd832f225c

2025-11-24 15:51:15,957 - distributed.scheduler - INFO - Remove client Client-4e90baf0-c94d-11f0-8001-f2cd832f225c

2025-11-24 15:51:15,956 - distributed.scheduler - INFO - Remove client Client-4e90baf0-c94d-11f0-8001-f2cd832f225c

2025-11-24 15:50:31,655 - distributed.scheduler - INFO - Receive client connection: Client-4e90baf0-c94d-11f0-8001-f2cd832f225c

2025-11-24 15:41:13,622 - distributed.scheduler - INFO - Close client connection: Client-e8123ab4-c94b-11f0-8001-f2cd832f225c

2025-11-24 15:41:13,621 - distributed.scheduler - INFO - Remove client Client-e8123ab4-c94b-11f0-8001-f2cd832f225c

2025-11-24 15:41:13,620 - distributed.scheduler - INFO - Remove client Client-e8123ab4-c94b-11f0-8001-f2cd832f225c

2025-11-24 15:40:30,204 - distributed.scheduler - INFO - Receive client connection: Client-e8123ab4-c94b-11f0-8001-f2cd832f225c

2025-11-24 15:31:46,136 - distributed.scheduler - INFO - Close client connection: Client-81930245-c94a-11f0-8001-f2cd832f225c

2025-11-24 15:31:46,135 - distributed.scheduler - INFO - Remove client Client-81930245-c94a-11f0-8001-f2cd832f225c

2025-11-24 15:31:46,134 - distributed.scheduler - INFO - Remove client Client-81930245-c94a-11f0-8001-f2cd832f225c

2025-11-24 15:30:28,739 - distributed.scheduler - INFO - Receive client connection: Client-81930245-c94a-11f0-8001-f2cd832f225c

2025-11-24 15:21:12,476 - distributed.scheduler - INFO - Close client connection: Client-1b18d88f-c949-11f0-8001-f2cd832f225c

2025-11-24 15:21:12,473 - distributed.scheduler - INFO - Remove client Client-1b18d88f-c949-11f0-8001-f2cd832f225c

2025-11-24 15:21:12,472 - distributed.scheduler - INFO - Remove client Client-1b18d88f-c949-11f0-8001-f2cd832f225c

2025-11-24 15:20:27,308 - distributed.scheduler - INFO - Receive client connection: Client-1b18d88f-c949-11f0-8001-f2cd832f225c

2025-11-24 15:11:16,961 - distributed.scheduler - INFO - Close client connection: Client-b4a52d47-c947-11f0-8001-f2cd832f225c

2025-11-24 15:11:16,960 - distributed.scheduler - INFO - Remove client Client-b4a52d47-c947-11f0-8001-f2cd832f225c

2025-11-24 15:11:16,959 - distributed.scheduler - INFO - Remove client Client-b4a52d47-c947-11f0-8001-f2cd832f225c

2025-11-24 15:10:25,941 - distributed.scheduler - INFO - Receive client connection: Client-b4a52d47-c947-11f0-8001-f2cd832f225c

2025-11-24 15:01:48,615 - distributed.scheduler - INFO - Close client connection: Client-4e25ba47-c946-11f0-8001-f2cd832f225c

2025-11-24 15:01:48,613 - distributed.scheduler - INFO - Remove client Client-4e25ba47-c946-11f0-8001-f2cd832f225c

2025-11-24 15:01:48,611 - distributed.scheduler - INFO - Remove client Client-4e25ba47-c946-11f0-8001-f2cd832f225c

2025-11-24 15:00:24,477 - distributed.scheduler - INFO - Receive client connection: Client-4e25ba47-c946-11f0-8001-f2cd832f225c

2025-11-24 14:51:09,752 - distributed.scheduler - INFO - Close client connection: Client-e7a85157-c944-11f0-8001-f2cd832f225c

2025-11-24 14:51:09,751 - distributed.scheduler - INFO - Remove client Client-e7a85157-c944-11f0-8001-f2cd832f225c

2025-11-24 14:51:09,750 - distributed.scheduler - INFO - Remove client Client-e7a85157-c944-11f0-8001-f2cd832f225c

2025-11-24 14:50:23,027 - distributed.scheduler - INFO - Receive client connection: Client-e7a85157-c944-11f0-8001-f2cd832f225c

2025-11-24 14:41:05,162 - distributed.scheduler - INFO - Close client connection: Client-1a00e1f1-c93b-11f0-8001-f2cd832f225c

2025-11-24 14:41:05,162 - distributed.scheduler - INFO - Close client connection: Client-807d954b-c93c-11f0-8001-f2cd832f225c

2025-11-24 14:41:05,161 - distributed.scheduler - INFO - Close client connection: Client-e70a05fa-c93d-11f0-8001-f2cd832f225c

2025-11-24 14:41:05,161 - distributed.scheduler - INFO - Close client connection: Client-4d9dd45f-c93f-11f0-8001-f2cd832f225c

2025-11-24 14:41:05,160 - distributed.scheduler - INFO - Close client connection: Client-b41be8dd-c940-11f0-8001-f2cd832f225c

2025-11-24 14:41:05,160 - distributed.scheduler - INFO - Close client connection: Client-1a9eff55-c942-11f0-8001-f2cd832f225c

2025-11-24 14:41:05,157 - distributed.scheduler - INFO - Remove client Client-1a00e1f1-c93b-11f0-8001-f2cd832f225c

2025-11-24 14:41:05,156 - distributed.scheduler - INFO - Remove client Client-807d954b-c93c-11f0-8001-f2cd832f225c

2025-11-24 14:41:05,155 - distributed.scheduler - INFO - Remove client Client-e70a05fa-c93d-11f0-8001-f2cd832f225c

2025-11-24 14:41:05,154 - distributed.scheduler - INFO - Remove client Client-4d9dd45f-c93f-11f0-8001-f2cd832f225c

2025-11-24 14:41:05,153 - distributed.scheduler - INFO - Remove client Client-b41be8dd-c940-11f0-8001-f2cd832f225c

2025-11-24 14:41:05,152 - distributed.scheduler - INFO - Remove client Client-1a9eff55-c942-11f0-8001-f2cd832f225c

2025-11-24 14:41:04,558 - distributed.scheduler - INFO - Close client connection: Client-812024a7-c943-11f0-8001-f2cd832f225c

2025-11-24 14:41:04,555 - distributed.scheduler - INFO - Remove client Client-812024a7-c943-11f0-8001-f2cd832f225c

2025-11-24 14:41:04,553 - distributed.scheduler - INFO - Remove client Client-812024a7-c943-11f0-8001-f2cd832f225c

2025-11-24 14:40:21,506 - distributed.scheduler - INFO - Receive client connection: Client-812024a7-c943-11f0-8001-f2cd832f225c

2025-11-24 14:34:41,614 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:38339 failed: OSError: Timed out trying to connect to tcp://172.21.12.150:38339 after 30 s

2025-11-24 14:34:41,470 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:33359 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:38339 after 30 s')

2025-11-24 14:34:41,456 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:35101 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:38339 after 30 s')

2025-11-24 14:34:41,455 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:40623 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:38339 after 30 s')

2025-11-24 14:34:41,455 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:35905 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:38339 after 30 s')

2025-11-24 14:34:41,452 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44675 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:38339 after 30 s')

2025-11-24 14:34:41,429 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:34191 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:38339 after 30 s')

2025-11-24 14:34:41,427 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:38339 after 30 s')

2025-11-24 14:34:41,404 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:33703 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:38339 after 30 s')

2025-11-24 14:34:12,099 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:46147

2025-11-24 14:34:12,096 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:46147', status: closed, memory: 0, processing: 0>

2025-11-24 14:34:11,277 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:38339', status: closed, memory: 0, processing: 0>

2025-11-24 14:33:58,844 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:38209 failed: OSError: Timed out trying to connect to tcp://172.21.12.150:38209 after 30 s

2025-11-24 14:33:58,720 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:32841 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:38209 after 30 s')

2025-11-24 14:33:58,678 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:43129 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:38209 after 30 s')

2025-11-24 14:33:58,667 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:43549 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:38209 after 30 s')

2025-11-24 14:33:29,594 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:38339

2025-11-24 14:33:29,592 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:38339', status: closed, memory: 0, processing: 0>

2025-11-24 14:33:28,546 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:38209', status: closed, memory: 0, processing: 0>

2025-11-24 14:33:16,085 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:40105 failed: OSError: Timed out trying to connect to tcp://172.21.12.150:40105 after 30 s

2025-11-24 14:33:16,072 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:40105 after 30 s')

2025-11-24 14:33:15,994 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:35905 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:40105 after 30 s')

2025-11-24 14:33:15,924 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:40105 after 30 s')

2025-11-24 14:32:46,743 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:38209

2025-11-24 14:32:46,742 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:38209', status: closed, memory: 0, processing: 0>

2025-11-24 14:32:45,838 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:40105', status: closed, memory: 0, processing: 0>

2025-11-24 14:32:20,285 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:34373 failed: OSError: Timed out trying to connect to tcp://172.21.12.150:34373 after 30 s

2025-11-24 14:32:20,183 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:34373 after 30 s')

2025-11-24 14:32:20,172 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:33703 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:34373 after 30 s')

2025-11-24 14:31:50,980 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:40105

2025-11-24 14:31:50,977 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:40105', status: closed, memory: 0, processing: 0>

2025-11-24 14:31:50,056 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:34373', status: closed, memory: 0, processing: 0>

2025-11-24 14:31:29,492 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:39867 failed: OSError: Timed out trying to connect to tcp://172.21.12.150:39867 after 30 s

2025-11-24 14:31:29,481 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44675 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:39867 after 30 s')

2025-11-24 14:31:29,185 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:43549 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:39867 after 30 s')

2025-11-24 14:31:28,783 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:43129 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.150:39867 after 30 s')

2025-11-24 14:30:59,159 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.150:34373

2025-11-24 14:30:59,159 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.150:34373', status: closed, memory: 0, processing: 0>

2025-11-24 14:30:58,240 - distributed.scheduler - INFO - Close client connection: Client-worker-6f9bb612-c56d-11f0-8074-e2572cf160eb

2025-11-24 14:30:58,223 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.150:39867', status: closed, memory: 0, processing: 0>

2025-11-24 14:30:58,223 - distributed.scheduler - INFO - Remove client Client-worker-6f9bb612-c56d-11f0-8074-e2572cf160eb

2025-11-24 14:30:20,030 - distributed.scheduler - INFO - Receive client connection: Client-1a9eff55-c942-11f0-8001-f2cd832f225c

2025-11-24 14:20:18,547 - distributed.scheduler - INFO - Receive client connection: Client-b41be8dd-c940-11f0-8001-f2cd832f225c

2025-11-24 14:10:17,094 - distributed.scheduler - INFO - Receive client connection: Client-4d9dd45f-c93f-11f0-8001-f2cd832f225c

2025-11-24 14:00:15,509 - distributed.scheduler - INFO - Receive client connection: Client-e70a05fa-c93d-11f0-8001-f2cd832f225c

2025-11-24 13:50:13,962 - distributed.scheduler - INFO - Receive client connection: Client-807d954b-c93c-11f0-8001-f2cd832f225c

2025-11-24 13:40:12,529 - distributed.scheduler - INFO - Receive client connection: Client-1a00e1f1-c93b-11f0-8001-f2cd832f225c

2025-11-24 13:31:25,907 - distributed.scheduler - INFO - Close client connection: Client-b383f417-c939-11f0-8001-f2cd832f225c

2025-11-24 13:31:25,907 - distributed.scheduler - INFO - Remove client Client-b383f417-c939-11f0-8001-f2cd832f225c

2025-11-24 13:31:25,905 - distributed.scheduler - INFO - Remove client Client-b383f417-c939-11f0-8001-f2cd832f225c

2025-11-24 13:30:11,085 - distributed.scheduler - INFO - Receive client connection: Client-b383f417-c939-11f0-8001-f2cd832f225c

2025-11-24 13:20:50,495 - distributed.scheduler - INFO - Close client connection: Client-4d009fad-c938-11f0-8001-f2cd832f225c

2025-11-24 13:20:50,493 - distributed.scheduler - INFO - Remove client Client-4d009fad-c938-11f0-8001-f2cd832f225c

2025-11-24 13:20:50,491 - distributed.scheduler - INFO - Remove client Client-4d009fad-c938-11f0-8001-f2cd832f225c

2025-11-24 13:20:09,605 - distributed.scheduler - INFO - Receive client connection: Client-4d009fad-c938-11f0-8001-f2cd832f225c

2025-11-24 13:10:52,761 - distributed.scheduler - INFO - Close client connection: Client-e6778d14-c936-11f0-8001-f2cd832f225c

2025-11-24 13:10:52,760 - distributed.scheduler - INFO - Remove client Client-e6778d14-c936-11f0-8001-f2cd832f225c

2025-11-24 13:10:52,758 - distributed.scheduler - INFO - Remove client Client-e6778d14-c936-11f0-8001-f2cd832f225c

2025-11-24 13:10:08,082 - distributed.scheduler - INFO - Receive client connection: Client-e6778d14-c936-11f0-8001-f2cd832f225c

2025-11-24 13:01:20,682 - distributed.scheduler - INFO - Close client connection: Client-7ded05f5-c935-11f0-8001-f2cd832f225c

2025-11-24 13:01:20,680 - distributed.scheduler - INFO - Remove client Client-7ded05f5-c935-11f0-8001-f2cd832f225c

2025-11-24 13:01:20,678 - distributed.scheduler - INFO - Remove client Client-7ded05f5-c935-11f0-8001-f2cd832f225c

2025-11-24 13:00:03,189 - distributed.scheduler - INFO - Receive client connection: Client-7ded05f5-c935-11f0-8001-f2cd832f225c

2025-11-24 12:50:43,973 - distributed.scheduler - INFO - Close client connection: Client-1771c090-c934-11f0-8001-f2cd832f225c

2025-11-24 12:50:43,972 - distributed.scheduler - INFO - Remove client Client-1771c090-c934-11f0-8001-f2cd832f225c

2025-11-24 12:50:43,970 - distributed.scheduler - INFO - Remove client Client-1771c090-c934-11f0-8001-f2cd832f225c

2025-11-24 12:50:01,762 - distributed.scheduler - INFO - Receive client connection: Client-1771c090-c934-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,945 - distributed.scheduler - INFO - Close client connection: Client-396e6ad1-c918-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,944 - distributed.scheduler - INFO - Close client connection: Client-9fe8b47e-c919-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,944 - distributed.scheduler - INFO - Close client connection: Client-066f8d2c-c91b-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,944 - distributed.scheduler - INFO - Close client connection: Client-6cf170d7-c91c-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,944 - distributed.scheduler - INFO - Close client connection: Client-d380d255-c91d-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,944 - distributed.scheduler - INFO - Close client connection: Client-3a00d57c-c91f-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,944 - distributed.scheduler - INFO - Close client connection: Client-3a7c4254-c926-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,943 - distributed.scheduler - INFO - Close client connection: Client-a087e89a-c920-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,943 - distributed.scheduler - INFO - Close client connection: Client-0702afa5-c922-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,943 - distributed.scheduler - INFO - Close client connection: Client-6d82bddf-c923-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,943 - distributed.scheduler - INFO - Close client connection: Client-d3fc30a2-c924-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,943 - distributed.scheduler - INFO - Close client connection: Client-a0f83898-c927-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,943 - distributed.scheduler - INFO - Close client connection: Client-077402cf-c929-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,943 - distributed.scheduler - INFO - Close client connection: Client-6df3132d-c92a-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,942 - distributed.scheduler - INFO - Close client connection: Client-d47ed14f-c92b-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,942 - distributed.scheduler - INFO - Close client connection: Client-3af7e7c0-c92d-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,942 - distributed.scheduler - INFO - Close client connection: Client-07e855ef-c930-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,942 - distributed.scheduler - INFO - Close client connection: Client-a17f000e-c92e-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,941 - distributed.scheduler - INFO - Close client connection: Client-6e5a0c6d-c931-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,939 - distributed.scheduler - INFO - Remove client Client-396e6ad1-c918-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,939 - distributed.scheduler - INFO - Remove client Client-9fe8b47e-c919-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,939 - distributed.scheduler - INFO - Remove client Client-066f8d2c-c91b-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,938 - distributed.scheduler - INFO - Remove client Client-6cf170d7-c91c-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,938 - distributed.scheduler - INFO - Remove client Client-d380d255-c91d-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,937 - distributed.scheduler - INFO - Remove client Client-3a00d57c-c91f-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,937 - distributed.scheduler - INFO - Remove client Client-3a7c4254-c926-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,936 - distributed.scheduler - INFO - Remove client Client-a087e89a-c920-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,936 - distributed.scheduler - INFO - Remove client Client-0702afa5-c922-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,935 - distributed.scheduler - INFO - Remove client Client-6d82bddf-c923-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,934 - distributed.scheduler - INFO - Remove client Client-d3fc30a2-c924-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,934 - distributed.scheduler - INFO - Remove client Client-a0f83898-c927-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,933 - distributed.scheduler - INFO - Remove client Client-077402cf-c929-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,932 - distributed.scheduler - INFO - Remove client Client-6df3132d-c92a-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,932 - distributed.scheduler - INFO - Remove client Client-d47ed14f-c92b-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,931 - distributed.scheduler - INFO - Remove client Client-3af7e7c0-c92d-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,931 - distributed.scheduler - INFO - Remove client Client-07e855ef-c930-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,930 - distributed.scheduler - INFO - Remove client Client-a17f000e-c92e-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,923 - distributed.scheduler - INFO - Remove client Client-6e5a0c6d-c931-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,024 - distributed.scheduler - INFO - Close client connection: Client-b0f5e94a-c932-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,023 - distributed.scheduler - INFO - Remove client Client-b0f5e94a-c932-11f0-8001-f2cd832f225c

2025-11-24 12:40:38,022 - distributed.scheduler - INFO - Remove client Client-b0f5e94a-c932-11f0-8001-f2cd832f225c

2025-11-24 12:40:00,314 - distributed.scheduler - INFO - Receive client connection: Client-b0f5e94a-c932-11f0-8001-f2cd832f225c

2025-11-24 12:30:59,070 - distributed.scheduler - INFO - Receive client connection: Client-6e5a0c6d-c931-11f0-8001-f2cd832f225c

2025-11-24 12:20:57,695 - distributed.scheduler - INFO - Receive client connection: Client-07e855ef-c930-11f0-8001-f2cd832f225c

2025-11-24 12:10:56,382 - distributed.scheduler - INFO - Receive client connection: Client-a17f000e-c92e-11f0-8001-f2cd832f225c

2025-11-24 12:00:54,869 - distributed.scheduler - INFO - Receive client connection: Client-3af7e7c0-c92d-11f0-8001-f2cd832f225c

2025-11-24 11:50:53,452 - distributed.scheduler - INFO - Receive client connection: Client-d47ed14f-c92b-11f0-8001-f2cd832f225c

2025-11-24 11:40:51,913 - distributed.scheduler - INFO - Receive client connection: Client-6df3132d-c92a-11f0-8001-f2cd832f225c

2025-11-24 11:30:50,460 - distributed.scheduler - INFO - Receive client connection: Client-077402cf-c929-11f0-8001-f2cd832f225c

2025-11-24 11:20:49,023 - distributed.scheduler - INFO - Receive client connection: Client-a0f83898-c927-11f0-8001-f2cd832f225c

2025-11-24 11:10:47,586 - distributed.scheduler - INFO - Receive client connection: Client-3a7c4254-c926-11f0-8001-f2cd832f225c

2025-11-24 11:00:46,124 - distributed.scheduler - INFO - Receive client connection: Client-d3fc30a2-c924-11f0-8001-f2cd832f225c

2025-11-24 10:50:44,703 - distributed.scheduler - INFO - Receive client connection: Client-6d82bddf-c923-11f0-8001-f2cd832f225c

2025-11-24 10:40:43,235 - distributed.scheduler - INFO - Receive client connection: Client-0702afa5-c922-11f0-8001-f2cd832f225c

2025-11-24 10:30:41,809 - distributed.scheduler - INFO - Receive client connection: Client-a087e89a-c920-11f0-8001-f2cd832f225c

2025-11-24 10:20:40,297 - distributed.scheduler - INFO - Receive client connection: Client-3a00d57c-c91f-11f0-8001-f2cd832f225c

2025-11-24 10:10:38,834 - distributed.scheduler - INFO - Receive client connection: Client-d380d255-c91d-11f0-8001-f2cd832f225c

2025-11-24 10:00:43,783 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: [], ('len-agg-b742326930e64998a8e51a7c5cafabdc', 0) NoneType: None

2025-11-24 10:00:43,782 - distributed.scheduler - ERROR - Couldn't gather keys {"('len-agg-b742326930e64998a8e51a7c5cafabdc', 0)": []} state: ['waiting'] workers: [] NoneType: None

2025-11-24 10:00:37,272 - distributed.scheduler - INFO - Receive client connection: Client-6cf170d7-c91c-11f0-8001-f2cd832f225c

2025-11-24 09:50:35,798 - distributed.scheduler - INFO - Receive client connection: Client-066f8d2c-c91b-11f0-8001-f2cd832f225c

2025-11-24 09:40:34,286 - distributed.scheduler - INFO - Receive client connection: Client-9fe8b47e-c919-11f0-8001-f2cd832f225c

2025-11-24 09:30:32,874 - distributed.scheduler - INFO - Receive client connection: Client-396e6ad1-c918-11f0-8001-f2cd832f225c

2025-11-24 09:21:10,386 - distributed.scheduler - INFO - Close client connection: Client-d2f09b03-c916-11f0-8001-f2cd832f225c

2025-11-24 09:21:10,385 - distributed.scheduler - INFO - Remove client Client-d2f09b03-c916-11f0-8001-f2cd832f225c

2025-11-24 09:21:10,383 - distributed.scheduler - INFO - Remove client Client-d2f09b03-c916-11f0-8001-f2cd832f225c

2025-11-24 09:20:31,424 - distributed.scheduler - INFO - Receive client connection: Client-d2f09b03-c916-11f0-8001-f2cd832f225c

2025-11-24 09:11:09,650 - distributed.scheduler - INFO - Close client connection: Client-6c73a81c-c915-11f0-8001-f2cd832f225c

2025-11-24 09:11:09,649 - distributed.scheduler - INFO - Remove client Client-6c73a81c-c915-11f0-8001-f2cd832f225c

2025-11-24 09:11:09,648 - distributed.scheduler - INFO - Remove client Client-6c73a81c-c915-11f0-8001-f2cd832f225c

2025-11-24 09:10:29,984 - distributed.scheduler - INFO - Receive client connection: Client-6c73a81c-c915-11f0-8001-f2cd832f225c

2025-11-24 09:01:47,240 - distributed.scheduler - INFO - Close client connection: Client-05e1df63-c914-11f0-8001-f2cd832f225c

2025-11-24 09:01:47,239 - distributed.scheduler - INFO - Remove client Client-05e1df63-c914-11f0-8001-f2cd832f225c

2025-11-24 09:01:47,238 - distributed.scheduler - INFO - Remove client Client-05e1df63-c914-11f0-8001-f2cd832f225c

2025-11-24 09:00:28,400 - distributed.scheduler - INFO - Receive client connection: Client-05e1df63-c914-11f0-8001-f2cd832f225c

2025-11-24 08:51:05,191 - distributed.scheduler - INFO - Close client connection: Client-9f6f1aca-c912-11f0-8001-f2cd832f225c

2025-11-24 08:51:05,190 - distributed.scheduler - INFO - Remove client Client-9f6f1aca-c912-11f0-8001-f2cd832f225c

2025-11-24 08:51:05,188 - distributed.scheduler - INFO - Remove client Client-9f6f1aca-c912-11f0-8001-f2cd832f225c

2025-11-24 08:50:27,021 - distributed.scheduler - INFO - Receive client connection: Client-9f6f1aca-c912-11f0-8001-f2cd832f225c

2025-11-24 08:41:07,947 - distributed.scheduler - INFO - Close client connection: Client-38ef9deb-c911-11f0-8001-f2cd832f225c

2025-11-24 08:41:07,947 - distributed.scheduler - INFO - Remove client Client-38ef9deb-c911-11f0-8001-f2cd832f225c

2025-11-24 08:41:07,945 - distributed.scheduler - INFO - Remove client Client-38ef9deb-c911-11f0-8001-f2cd832f225c

2025-11-24 08:40:25,563 - distributed.scheduler - INFO - Receive client connection: Client-38ef9deb-c911-11f0-8001-f2cd832f225c

2025-11-24 08:31:42,616 - distributed.scheduler - INFO - Close client connection: Client-d270768e-c90f-11f0-8001-f2cd832f225c

2025-11-24 08:31:42,616 - distributed.scheduler - INFO - Remove client Client-d270768e-c90f-11f0-8001-f2cd832f225c

2025-11-24 08:31:42,615 - distributed.scheduler - INFO - Remove client Client-d270768e-c90f-11f0-8001-f2cd832f225c

2025-11-24 08:30:24,104 - distributed.scheduler - INFO - Receive client connection: Client-d270768e-c90f-11f0-8001-f2cd832f225c

2025-11-24 08:21:03,448 - distributed.scheduler - INFO - Close client connection: Client-9efe450c-c90b-11f0-8001-f2cd832f225c

2025-11-24 08:21:03,447 - distributed.scheduler - INFO - Close client connection: Client-38895d00-c90a-11f0-8001-f2cd832f225c

2025-11-24 08:21:03,447 - distributed.scheduler - INFO - Close client connection: Client-0582d28f-c90d-11f0-8001-f2cd832f225c

2025-11-24 08:21:03,445 - distributed.scheduler - INFO - Remove client Client-9efe450c-c90b-11f0-8001-f2cd832f225c

2025-11-24 08:21:03,444 - distributed.scheduler - INFO - Remove client Client-38895d00-c90a-11f0-8001-f2cd832f225c

2025-11-24 08:21:03,443 - distributed.scheduler - INFO - Remove client Client-0582d28f-c90d-11f0-8001-f2cd832f225c

2025-11-24 08:21:02,846 - distributed.scheduler - INFO - Close client connection: Client-6bf9e6f1-c90e-11f0-8001-f2cd832f225c

2025-11-24 08:21:02,844 - distributed.scheduler - INFO - Remove client Client-6bf9e6f1-c90e-11f0-8001-f2cd832f225c

2025-11-24 08:21:02,842 - distributed.scheduler - INFO - Remove client Client-6bf9e6f1-c90e-11f0-8001-f2cd832f225c

2025-11-24 08:20:22,694 - distributed.scheduler - INFO - Receive client connection: Client-6bf9e6f1-c90e-11f0-8001-f2cd832f225c

2025-11-24 08:10:21,288 - distributed.scheduler - INFO - Receive client connection: Client-0582d28f-c90d-11f0-8001-f2cd832f225c

2025-11-24 08:00:19,792 - distributed.scheduler - INFO - Receive client connection: Client-9efe450c-c90b-11f0-8001-f2cd832f225c

2025-11-24 07:50:18,417 - distributed.scheduler - INFO - Receive client connection: Client-38895d00-c90a-11f0-8001-f2cd832f225c

2025-11-24 07:40:56,275 - distributed.scheduler - INFO - Close client connection: Client-d20a2e6c-c908-11f0-8001-f2cd832f225c

2025-11-24 07:40:56,274 - distributed.scheduler - INFO - Remove client Client-d20a2e6c-c908-11f0-8001-f2cd832f225c

2025-11-24 07:40:56,273 - distributed.scheduler - INFO - Remove client Client-d20a2e6c-c908-11f0-8001-f2cd832f225c

2025-11-24 07:40:16,945 - distributed.scheduler - INFO - Receive client connection: Client-d20a2e6c-c908-11f0-8001-f2cd832f225c

2025-11-24 07:31:28,789 - distributed.scheduler - INFO - Close client connection: Client-6b8a3f42-c907-11f0-8001-f2cd832f225c

2025-11-24 07:31:28,788 - distributed.scheduler - INFO - Remove client Client-6b8a3f42-c907-11f0-8001-f2cd832f225c

2025-11-24 07:31:28,786 - distributed.scheduler - INFO - Remove client Client-6b8a3f42-c907-11f0-8001-f2cd832f225c

2025-11-24 07:30:19,572 - distributed.scheduler - INFO - Receive client connection: Client-worker-6df9d1b6-c907-11f0-83c6-62b326af983b

2025-11-24 07:30:19,390 - distributed.scheduler - INFO - Receive client connection: Client-worker-6dd9ce25-c907-11f0-830d-f23d8172cb06

2025-11-24 07:30:15,505 - distributed.scheduler - INFO - Receive client connection: Client-6b8a3f42-c907-11f0-8001-f2cd832f225c

2025-11-24 07:20:57,107 - distributed.scheduler - INFO - Close client connection: Client-050aaf48-c906-11f0-8001-f2cd832f225c

2025-11-24 07:20:57,106 - distributed.scheduler - INFO - Remove client Client-050aaf48-c906-11f0-8001-f2cd832f225c

2025-11-24 07:20:57,104 - distributed.scheduler - INFO - Remove client Client-050aaf48-c906-11f0-8001-f2cd832f225c

2025-11-24 07:20:14,036 - distributed.scheduler - INFO - Receive client connection: Client-050aaf48-c906-11f0-8001-f2cd832f225c

2025-11-24 07:10:53,247 - distributed.scheduler - INFO - Close client connection: Client-9e89a980-c904-11f0-8001-f2cd832f225c

2025-11-24 07:10:53,246 - distributed.scheduler - INFO - Remove client Client-9e89a980-c904-11f0-8001-f2cd832f225c

2025-11-24 07:10:53,244 - distributed.scheduler - INFO - Remove client Client-9e89a980-c904-11f0-8001-f2cd832f225c

2025-11-24 07:10:12,557 - distributed.scheduler - INFO - Receive client connection: Client-9e89a980-c904-11f0-8001-f2cd832f225c

2025-11-24 07:01:41,707 - distributed.scheduler - INFO - Receive client connection: Client-worker-6e0c1e62-c903-11f0-8258-de470ccc1e79

2025-11-24 07:01:39,092 - distributed.scheduler - INFO - Close client connection: Client-3813eabb-c903-11f0-8001-f2cd832f225c

2025-11-24 07:01:39,091 - distributed.scheduler - INFO - Remove client Client-3813eabb-c903-11f0-8001-f2cd832f225c

2025-11-24 07:01:39,089 - distributed.scheduler - INFO - Remove client Client-3813eabb-c903-11f0-8001-f2cd832f225c

2025-11-24 07:00:50,695 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.191:33703

2025-11-24 07:00:50,692 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.191:33703', status: closed, memory: 0, processing: 0>

2025-11-24 07:00:49,894 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:35905

2025-11-24 07:00:49,893 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:35905', status: closed, memory: 0, processing: 0>

2025-11-24 07:00:49,713 - distributed.scheduler - INFO - Close client connection: Client-worker-6f18ef25-c56d-11f0-8074-de470ccc1e79

2025-11-24 07:00:49,706 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.191:43305', status: closed, memory: 0, processing: 0>

2025-11-24 07:00:49,705 - distributed.scheduler - INFO - Remove client Client-worker-6f18ef25-c56d-11f0-8074-de470ccc1e79

2025-11-24 07:00:49,676 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.233:35101

2025-11-24 07:00:49,676 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.233:35101', status: closed, memory: 0, processing: 0>

2025-11-24 07:00:49,070 - distributed.scheduler - INFO - Close client connection: Client-worker-f75a672d-c8e5-11f0-8375-62b326af983b

2025-11-24 07:00:49,049 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:42469', status: closed, memory: 0, processing: 0>

2025-11-24 07:00:49,048 - distributed.scheduler - INFO - Remove client Client-worker-f75a672d-c8e5-11f0-8375-62b326af983b

2025-11-24 07:00:48,864 - distributed.scheduler - INFO - Close client connection: Client-worker-3099d5b0-c7b8-11f0-821b-f23d8172cb06

2025-11-24 07:00:48,855 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.233:34887', status: closed, memory: 0, processing: 0>

2025-11-24 07:00:48,837 - distributed.scheduler - INFO - Remove client Client-worker-3099d5b0-c7b8-11f0-821b-f23d8172cb06

2025-11-24 07:00:11,166 - distributed.scheduler - INFO - Receive client connection: Client-3813eabb-c903-11f0-8001-f2cd832f225c

2025-11-24 06:50:55,542 - distributed.scheduler - INFO - Close client connection: Client-d1940c81-c901-11f0-8001-f2cd832f225c

2025-11-24 06:50:55,540 - distributed.scheduler - INFO - Remove client Client-d1940c81-c901-11f0-8001-f2cd832f225c

2025-11-24 06:50:55,539 - distributed.scheduler - INFO - Remove client Client-d1940c81-c901-11f0-8001-f2cd832f225c

2025-11-24 06:50:09,705 - distributed.scheduler - INFO - Receive client connection: Client-d1940c81-c901-11f0-8001-f2cd832f225c

2025-11-24 06:40:56,490 - distributed.scheduler - INFO - Close client connection: Client-6b1165b8-c900-11f0-8001-f2cd832f225c

2025-11-24 06:40:56,489 - distributed.scheduler - INFO - Remove client Client-6b1165b8-c900-11f0-8001-f2cd832f225c

2025-11-24 06:40:56,486 - distributed.scheduler - INFO - Remove client Client-6b1165b8-c900-11f0-8001-f2cd832f225c

2025-11-24 06:40:08,226 - distributed.scheduler - INFO - Receive client connection: Client-6b1165b8-c900-11f0-8001-f2cd832f225c

2025-11-24 06:31:25,766 - distributed.scheduler - INFO - Close client connection: Client-048a2d36-c8ff-11f0-8001-f2cd832f225c

2025-11-24 06:31:25,762 - distributed.scheduler - INFO - Remove client Client-048a2d36-c8ff-11f0-8001-f2cd832f225c

2025-11-24 06:31:25,755 - distributed.scheduler - INFO - Remove client Client-048a2d36-c8ff-11f0-8001-f2cd832f225c

2025-11-24 06:30:06,715 - distributed.scheduler - INFO - Receive client connection: Client-048a2d36-c8ff-11f0-8001-f2cd832f225c

2025-11-24 06:20:45,304 - distributed.scheduler - INFO - Close client connection: Client-9e0bf5fa-c8fd-11f0-8001-f2cd832f225c

2025-11-24 06:20:45,304 - distributed.scheduler - INFO - Remove client Client-9e0bf5fa-c8fd-11f0-8001-f2cd832f225c

2025-11-24 06:20:45,303 - distributed.scheduler - INFO - Remove client Client-9e0bf5fa-c8fd-11f0-8001-f2cd832f225c

2025-11-24 06:20:05,273 - distributed.scheduler - INFO - Receive client connection: Client-9e0bf5fa-c8fd-11f0-8001-f2cd832f225c

2025-11-24 06:10:47,088 - distributed.scheduler - INFO - Close client connection: Client-c18d5fa9-c8f6-11f0-8001-f2cd832f225c

2025-11-24 06:10:47,087 - distributed.scheduler - INFO - Close client connection: Client-5b22b239-c8f5-11f0-8001-f2cd832f225c

2025-11-24 06:10:47,087 - distributed.scheduler - INFO - Close client connection: Client-6aa2c4f1-c8f9-11f0-8001-f2cd832f225c

2025-11-24 06:10:47,087 - distributed.scheduler - INFO - Close client connection: Client-27fe7295-c8f8-11f0-8001-f2cd832f225c

2025-11-24 06:10:47,086 - distributed.scheduler - INFO - Close client connection: Client-d11b347f-c8fa-11f0-8001-f2cd832f225c

2025-11-24 06:10:47,085 - distributed.scheduler - INFO - Remove client Client-c18d5fa9-c8f6-11f0-8001-f2cd832f225c

2025-11-24 06:10:47,084 - distributed.scheduler - INFO - Remove client Client-5b22b239-c8f5-11f0-8001-f2cd832f225c

2025-11-24 06:10:47,081 - distributed.scheduler - INFO - Remove client Client-6aa2c4f1-c8f9-11f0-8001-f2cd832f225c

2025-11-24 06:10:47,081 - distributed.scheduler - INFO - Remove client Client-27fe7295-c8f8-11f0-8001-f2cd832f225c

2025-11-24 06:10:47,080 - distributed.scheduler - INFO - Remove client Client-d11b347f-c8fa-11f0-8001-f2cd832f225c

2025-11-24 06:10:46,233 - distributed.scheduler - INFO - Close client connection: Client-3796efd9-c8fc-11f0-8001-f2cd832f225c

2025-11-24 06:10:46,232 - distributed.scheduler - INFO - Remove client Client-3796efd9-c8fc-11f0-8001-f2cd832f225c

2025-11-24 06:10:46,231 - distributed.scheduler - INFO - Remove client Client-3796efd9-c8fc-11f0-8001-f2cd832f225c

2025-11-24 06:10:03,860 - distributed.scheduler - INFO - Receive client connection: Client-3796efd9-c8fc-11f0-8001-f2cd832f225c

2025-11-24 06:00:02,428 - distributed.scheduler - INFO - Receive client connection: Client-d11b347f-c8fa-11f0-8001-f2cd832f225c

2025-11-24 05:50:01,018 - distributed.scheduler - INFO - Receive client connection: Client-6aa2c4f1-c8f9-11f0-8001-f2cd832f225c

2025-11-24 05:40:59,713 - distributed.scheduler - INFO - Receive client connection: Client-27fe7295-c8f8-11f0-8001-f2cd832f225c

2025-11-24 05:30:58,338 - distributed.scheduler - INFO - Receive client connection: Client-c18d5fa9-c8f6-11f0-8001-f2cd832f225c

2025-11-24 05:20:57,030 - distributed.scheduler - INFO - Receive client connection: Client-5b22b239-c8f5-11f0-8001-f2cd832f225c

2025-11-24 05:11:37,929 - distributed.scheduler - INFO - Close client connection: Client-f4b467ad-c8f3-11f0-8001-f2cd832f225c

2025-11-24 05:11:37,928 - distributed.scheduler - INFO - Remove client Client-f4b467ad-c8f3-11f0-8001-f2cd832f225c

2025-11-24 05:11:37,926 - distributed.scheduler - INFO - Remove client Client-f4b467ad-c8f3-11f0-8001-f2cd832f225c

2025-11-24 05:10:55,691 - distributed.scheduler - INFO - Receive client connection: Client-f4b467ad-c8f3-11f0-8001-f2cd832f225c

2025-11-24 05:02:07,455 - distributed.scheduler - INFO - Close client connection: Client-8e4f8095-c8f2-11f0-8001-f2cd832f225c

2025-11-24 05:02:07,454 - distributed.scheduler - INFO - Remove client Client-8e4f8095-c8f2-11f0-8001-f2cd832f225c

2025-11-24 05:02:07,452 - distributed.scheduler - INFO - Remove client Client-8e4f8095-c8f2-11f0-8001-f2cd832f225c

2025-11-24 05:00:54,411 - distributed.scheduler - INFO - Receive client connection: Client-8e4f8095-c8f2-11f0-8001-f2cd832f225c

2025-11-24 04:51:33,693 - distributed.scheduler - INFO - Close client connection: Client-27d77918-c8f1-11f0-8001-f2cd832f225c

2025-11-24 04:51:33,693 - distributed.scheduler - INFO - Remove client Client-27d77918-c8f1-11f0-8001-f2cd832f225c

2025-11-24 04:51:33,691 - distributed.scheduler - INFO - Remove client Client-27d77918-c8f1-11f0-8001-f2cd832f225c

2025-11-24 04:50:52,989 - distributed.scheduler - INFO - Receive client connection: Client-27d77918-c8f1-11f0-8001-f2cd832f225c

2025-11-24 04:41:34,118 - distributed.scheduler - INFO - Close client connection: Client-c158e8ad-c8ef-11f0-8001-f2cd832f225c

2025-11-24 04:41:34,117 - distributed.scheduler - INFO - Remove client Client-c158e8ad-c8ef-11f0-8001-f2cd832f225c

2025-11-24 04:41:34,115 - distributed.scheduler - INFO - Remove client Client-c158e8ad-c8ef-11f0-8001-f2cd832f225c

2025-11-24 04:40:51,532 - distributed.scheduler - INFO - Receive client connection: Client-c158e8ad-c8ef-11f0-8001-f2cd832f225c

2025-11-24 04:32:04,160 - distributed.scheduler - INFO - Close client connection: Client-5add622f-c8ee-11f0-8001-f2cd832f225c

2025-11-24 04:32:04,159 - distributed.scheduler - INFO - Remove client Client-5add622f-c8ee-11f0-8001-f2cd832f225c

2025-11-24 04:32:04,157 - distributed.scheduler - INFO - Remove client Client-5add622f-c8ee-11f0-8001-f2cd832f225c

2025-11-24 04:30:50,102 - distributed.scheduler - INFO - Receive client connection: Client-5add622f-c8ee-11f0-8001-f2cd832f225c

2025-11-24 04:21:34,125 - distributed.scheduler - INFO - Close client connection: Client-f45dd7f1-c8ec-11f0-8001-f2cd832f225c

2025-11-24 04:21:34,123 - distributed.scheduler - INFO - Remove client Client-f45dd7f1-c8ec-11f0-8001-f2cd832f225c

2025-11-24 04:21:34,121 - distributed.scheduler - INFO - Remove client Client-f45dd7f1-c8ec-11f0-8001-f2cd832f225c

2025-11-24 04:20:48,626 - distributed.scheduler - INFO - Receive client connection: Client-f45dd7f1-c8ec-11f0-8001-f2cd832f225c

2025-11-24 04:11:31,609 - distributed.scheduler - INFO - Close client connection: Client-8de183c6-c8eb-11f0-8001-f2cd832f225c

2025-11-24 04:11:31,608 - distributed.scheduler - INFO - Remove client Client-8de183c6-c8eb-11f0-8001-f2cd832f225c

2025-11-24 04:11:31,607 - distributed.scheduler - INFO - Remove client Client-8de183c6-c8eb-11f0-8001-f2cd832f225c

2025-11-24 04:10:47,204 - distributed.scheduler - INFO - Receive client connection: Client-8de183c6-c8eb-11f0-8001-f2cd832f225c

2025-11-24 04:02:04,005 - distributed.scheduler - INFO - Close client connection: Client-276a6990-c8ea-11f0-8001-f2cd832f225c

2025-11-24 04:02:04,004 - distributed.scheduler - INFO - Remove client Client-276a6990-c8ea-11f0-8001-f2cd832f225c

2025-11-24 04:02:04,003 - distributed.scheduler - INFO - Remove client Client-276a6990-c8ea-11f0-8001-f2cd832f225c

2025-11-24 04:00:45,796 - distributed.scheduler - INFO - Receive client connection: Client-276a6990-c8ea-11f0-8001-f2cd832f225c

2025-11-24 03:51:28,222 - distributed.scheduler - INFO - Close client connection: Client-c0ebfc55-c8e8-11f0-8001-f2cd832f225c

2025-11-24 03:51:28,217 - distributed.scheduler - INFO - Remove client Client-c0ebfc55-c8e8-11f0-8001-f2cd832f225c

2025-11-24 03:51:28,216 - distributed.scheduler - INFO - Remove client Client-c0ebfc55-c8e8-11f0-8001-f2cd832f225c

2025-11-24 03:50:44,346 - distributed.scheduler - INFO - Receive client connection: Client-c0ebfc55-c8e8-11f0-8001-f2cd832f225c

2025-11-24 03:41:25,477 - distributed.scheduler - INFO - Close client connection: Client-5a74b69c-c8e7-11f0-8001-f2cd832f225c

2025-11-24 03:41:25,477 - distributed.scheduler - INFO - Remove client Client-5a74b69c-c8e7-11f0-8001-f2cd832f225c

2025-11-24 03:41:25,476 - distributed.scheduler - INFO - Remove client Client-5a74b69c-c8e7-11f0-8001-f2cd832f225c

2025-11-24 03:40:42,931 - distributed.scheduler - INFO - Receive client connection: Client-5a74b69c-c8e7-11f0-8001-f2cd832f225c

2025-11-24 03:31:59,471 - distributed.scheduler - INFO - Close client connection: Client-f3facd7f-c8e5-11f0-8001-f2cd832f225c

2025-11-24 03:31:59,470 - distributed.scheduler - INFO - Remove client Client-f3facd7f-c8e5-11f0-8001-f2cd832f225c

2025-11-24 03:31:59,468 - distributed.scheduler - INFO - Remove client Client-f3facd7f-c8e5-11f0-8001-f2cd832f225c

2025-11-24 03:30:47,182 - distributed.scheduler - INFO - Receive client connection: Client-worker-f75a672d-c8e5-11f0-8375-62b326af983b

2025-11-24 03:30:41,516 - distributed.scheduler - INFO - Receive client connection: Client-f3facd7f-c8e5-11f0-8001-f2cd832f225c

2025-11-24 03:21:26,168 - distributed.scheduler - INFO - Close client connection: Client-8d8b56d6-c8e4-11f0-8001-f2cd832f225c

2025-11-24 03:21:26,167 - distributed.scheduler - INFO - Remove client Client-8d8b56d6-c8e4-11f0-8001-f2cd832f225c

2025-11-24 03:21:26,165 - distributed.scheduler - INFO - Remove client Client-8d8b56d6-c8e4-11f0-8001-f2cd832f225c

2025-11-24 03:20:40,157 - distributed.scheduler - INFO - Receive client connection: Client-8d8b56d6-c8e4-11f0-8001-f2cd832f225c

2025-11-24 03:11:21,895 - distributed.scheduler - INFO - Close client connection: Client-270c56f6-c8e3-11f0-8001-f2cd832f225c

2025-11-24 03:11:21,894 - distributed.scheduler - INFO - Remove client Client-270c56f6-c8e3-11f0-8001-f2cd832f225c

2025-11-24 03:11:21,892 - distributed.scheduler - INFO - Remove client Client-270c56f6-c8e3-11f0-8001-f2cd832f225c

2025-11-24 03:10:38,707 - distributed.scheduler - INFO - Receive client connection: Client-270c56f6-c8e3-11f0-8001-f2cd832f225c

2025-11-24 03:01:53,094 - distributed.scheduler - INFO - Close client connection: Client-c090f641-c8e1-11f0-8001-f2cd832f225c

2025-11-24 03:01:53,093 - distributed.scheduler - INFO - Remove client Client-c090f641-c8e1-11f0-8001-f2cd832f225c

2025-11-24 03:01:53,093 - distributed.scheduler - INFO - Remove client Client-c090f641-c8e1-11f0-8001-f2cd832f225c

2025-11-24 03:00:37,275 - distributed.scheduler - INFO - Receive client connection: Client-c090f641-c8e1-11f0-8001-f2cd832f225c

2025-11-24 02:51:16,457 - distributed.scheduler - INFO - Close client connection: Client-5a0e8de7-c8e0-11f0-8001-f2cd832f225c

2025-11-24 02:51:16,456 - distributed.scheduler - INFO - Remove client Client-5a0e8de7-c8e0-11f0-8001-f2cd832f225c

2025-11-24 02:51:16,454 - distributed.scheduler - INFO - Remove client Client-5a0e8de7-c8e0-11f0-8001-f2cd832f225c

2025-11-24 02:50:35,795 - distributed.scheduler - INFO - Receive client connection: Client-5a0e8de7-c8e0-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,904 - distributed.scheduler - INFO - Close client connection: Client-e090b56f-c8a2-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,904 - distributed.scheduler - INFO - Close client connection: Client-470507aa-c8a4-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,904 - distributed.scheduler - INFO - Close client connection: Client-ad8143c2-c8a5-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,904 - distributed.scheduler - INFO - Close client connection: Client-13fa6081-c8a7-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,904 - distributed.scheduler - INFO - Close client connection: Client-7a7b7519-c8a8-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,904 - distributed.scheduler - INFO - Close client connection: Client-e0f9d8f8-c8a9-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,904 - distributed.scheduler - INFO - Close client connection: Client-476c2781-c8ab-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,904 - distributed.scheduler - INFO - Close client connection: Client-ade3a277-c8ac-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,903 - distributed.scheduler - INFO - Close client connection: Client-14630388-c8ae-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,903 - distributed.scheduler - INFO - Close client connection: Client-7ae8dc1d-c8af-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,903 - distributed.scheduler - INFO - Close client connection: Client-e168ba8e-c8b0-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,903 - distributed.scheduler - INFO - Close client connection: Client-47e65145-c8b2-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,903 - distributed.scheduler - INFO - Close client connection: Client-ae65b995-c8b3-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,903 - distributed.scheduler - INFO - Close client connection: Client-14d062a4-c8b5-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,903 - distributed.scheduler - INFO - Close client connection: Client-7b4cd080-c8b6-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,903 - distributed.scheduler - INFO - Close client connection: Client-e1cb457d-c8b7-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,902 - distributed.scheduler - INFO - Close client connection: Client-48471b0f-c8b9-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,902 - distributed.scheduler - INFO - Close client connection: Client-1534e0e7-c8bc-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,902 - distributed.scheduler - INFO - Close client connection: Client-aebe1c4a-c8ba-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,902 - distributed.scheduler - INFO - Close client connection: Client-7bb60369-c8bd-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,902 - distributed.scheduler - INFO - Close client connection: Client-be55bc9e-c8be-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,902 - distributed.scheduler - INFO - Close client connection: Client-24d2bce8-c8c0-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,902 - distributed.scheduler - INFO - Close client connection: Client-8b4ef24f-c8c1-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,902 - distributed.scheduler - INFO - Close client connection: Client-f1c6b380-c8c2-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,902 - distributed.scheduler - INFO - Close client connection: Client-584964ed-c8c4-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,901 - distributed.scheduler - INFO - Close client connection: Client-bebb4e36-c8c5-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,901 - distributed.scheduler - INFO - Close client connection: Client-253f110a-c8c7-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,901 - distributed.scheduler - INFO - Close client connection: Client-8bc0fbd5-c8c8-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,901 - distributed.scheduler - INFO - Close client connection: Client-f2a9b8f4-c8d0-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,901 - distributed.scheduler - INFO - Close client connection: Client-f23e99c0-c8c9-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,901 - distributed.scheduler - INFO - Close client connection: Client-58bb13c3-c8cb-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,901 - distributed.scheduler - INFO - Close client connection: Client-25b46288-c8ce-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,901 - distributed.scheduler - INFO - Close client connection: Client-bf3a4e96-c8cc-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,901 - distributed.scheduler - INFO - Close client connection: Client-8c2ce6bb-c8cf-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,901 - distributed.scheduler - INFO - Close client connection: Client-592caff4-c8d2-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,901 - distributed.scheduler - INFO - Close client connection: Client-bfadc5be-c8d3-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,900 - distributed.scheduler - INFO - Close client connection: Client-262d6544-c8d5-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,900 - distributed.scheduler - INFO - Close client connection: Client-8cb0f170-c8d6-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,900 - distributed.scheduler - INFO - Close client connection: Client-f320f993-c8d7-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,900 - distributed.scheduler - INFO - Close client connection: Client-599cc48c-c8d9-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,900 - distributed.scheduler - INFO - Close client connection: Client-c0144eef-c8da-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,900 - distributed.scheduler - INFO - Close client connection: Client-26937356-c8dc-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,899 - distributed.scheduler - INFO - Close client connection: Client-8d18556b-c8dd-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,898 - distributed.scheduler - INFO - Remove client Client-e090b56f-c8a2-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,898 - distributed.scheduler - INFO - Remove client Client-470507aa-c8a4-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,898 - distributed.scheduler - INFO - Remove client Client-ad8143c2-c8a5-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,898 - distributed.scheduler - INFO - Remove client Client-13fa6081-c8a7-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,897 - distributed.scheduler - INFO - Remove client Client-7a7b7519-c8a8-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,897 - distributed.scheduler - INFO - Remove client Client-e0f9d8f8-c8a9-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,897 - distributed.scheduler - INFO - Remove client Client-476c2781-c8ab-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,896 - distributed.scheduler - INFO - Remove client Client-ade3a277-c8ac-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,896 - distributed.scheduler - INFO - Remove client Client-14630388-c8ae-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,896 - distributed.scheduler - INFO - Remove client Client-7ae8dc1d-c8af-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,895 - distributed.scheduler - INFO - Remove client Client-e168ba8e-c8b0-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,895 - distributed.scheduler - INFO - Remove client Client-47e65145-c8b2-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,895 - distributed.scheduler - INFO - Remove client Client-ae65b995-c8b3-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,894 - distributed.scheduler - INFO - Remove client Client-14d062a4-c8b5-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,894 - distributed.scheduler - INFO - Remove client Client-7b4cd080-c8b6-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,894 - distributed.scheduler - INFO - Remove client Client-e1cb457d-c8b7-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,889 - distributed.scheduler - INFO - Remove client Client-48471b0f-c8b9-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,889 - distributed.scheduler - INFO - Remove client Client-1534e0e7-c8bc-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,889 - distributed.scheduler - INFO - Remove client Client-aebe1c4a-c8ba-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,888 - distributed.scheduler - INFO - Remove client Client-7bb60369-c8bd-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,888 - distributed.scheduler - INFO - Remove client Client-be55bc9e-c8be-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,888 - distributed.scheduler - INFO - Remove client Client-24d2bce8-c8c0-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,887 - distributed.scheduler - INFO - Remove client Client-8b4ef24f-c8c1-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,887 - distributed.scheduler - INFO - Remove client Client-f1c6b380-c8c2-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,886 - distributed.scheduler - INFO - Remove client Client-584964ed-c8c4-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,886 - distributed.scheduler - INFO - Remove client Client-bebb4e36-c8c5-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,885 - distributed.scheduler - INFO - Remove client Client-253f110a-c8c7-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,885 - distributed.scheduler - INFO - Remove client Client-8bc0fbd5-c8c8-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,884 - distributed.scheduler - INFO - Remove client Client-f2a9b8f4-c8d0-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,884 - distributed.scheduler - INFO - Remove client Client-f23e99c0-c8c9-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,884 - distributed.scheduler - INFO - Remove client Client-58bb13c3-c8cb-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,883 - distributed.scheduler - INFO - Remove client Client-25b46288-c8ce-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,883 - distributed.scheduler - INFO - Remove client Client-bf3a4e96-c8cc-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,882 - distributed.scheduler - INFO - Remove client Client-8c2ce6bb-c8cf-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,881 - distributed.scheduler - INFO - Remove client Client-592caff4-c8d2-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,881 - distributed.scheduler - INFO - Remove client Client-bfadc5be-c8d3-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,880 - distributed.scheduler - INFO - Remove client Client-262d6544-c8d5-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,879 - distributed.scheduler - INFO - Remove client Client-8cb0f170-c8d6-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,878 - distributed.scheduler - INFO - Remove client Client-f320f993-c8d7-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,877 - distributed.scheduler - INFO - Remove client Client-599cc48c-c8d9-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,876 - distributed.scheduler - INFO - Remove client Client-c0144eef-c8da-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,875 - distributed.scheduler - INFO - Remove client Client-26937356-c8dc-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,865 - distributed.scheduler - INFO - Remove client Client-8d18556b-c8dd-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,296 - distributed.scheduler - INFO - Close client connection: Client-f396d236-c8de-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,294 - distributed.scheduler - INFO - Remove client Client-f396d236-c8de-11f0-8001-f2cd832f225c

2025-11-24 02:41:14,292 - distributed.scheduler - INFO - Remove client Client-f396d236-c8de-11f0-8001-f2cd832f225c

2025-11-24 02:40:34,417 - distributed.scheduler - INFO - Receive client connection: Client-f396d236-c8de-11f0-8001-f2cd832f225c

2025-11-24 02:34:49,952 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:34551 failed: OSError: Timed out trying to connect to tcp://172.21.159.199:34551 after 30 s

2025-11-24 02:34:49,926 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:43129 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:34551 after 30 s')

2025-11-24 02:34:49,925 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:34551 after 30 s')

2025-11-24 02:34:49,907 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:43305 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:34551 after 30 s')

2025-11-24 02:34:49,837 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:34887 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:34551 after 30 s')

2025-11-24 02:34:49,836 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44675 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:34551 after 30 s')

2025-11-24 02:34:49,806 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37207 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:34551 after 30 s')

2025-11-24 02:34:49,804 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:34551 after 30 s')

2025-11-24 02:34:49,791 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:34191 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:34551 after 30 s')

2025-11-24 02:34:20,496 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:42469

2025-11-24 02:34:20,493 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:42469', status: closed, memory: 0, processing: 0>

2025-11-24 02:34:19,661 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:34551', status: closed, memory: 0, processing: 0>

2025-11-24 02:34:08,415 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:37529 failed: OSError: Timed out trying to connect to tcp://172.21.159.199:37529 after 30 s

2025-11-24 02:34:08,395 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:37529 after 30 s')

2025-11-24 02:34:07,913 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:39867 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:37529 after 30 s')

2025-11-24 02:34:07,887 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:43305 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:37529 after 30 s')

2025-11-24 02:34:07,824 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:32841 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:37529 after 30 s')

2025-11-24 02:34:07,820 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:33359 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:37529 after 30 s')

2025-11-24 02:34:07,799 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:37529 after 30 s')

2025-11-24 02:34:07,790 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:34191 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:37529 after 30 s')

2025-11-24 02:33:38,629 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:34551

2025-11-24 02:33:38,628 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:34551', status: closed, memory: 0, processing: 0>

2025-11-24 02:33:37,662 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:37529', status: closed, memory: 0, processing: 0>

2025-11-24 02:33:26,282 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:41207 failed: OSError: Timed out trying to connect to tcp://172.21.159.199:41207 after 30 s

2025-11-24 02:33:26,200 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:39867 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41207 after 30 s')

2025-11-24 02:33:26,160 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:43305 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41207 after 30 s')

2025-11-24 02:33:26,143 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37207 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41207 after 30 s')

2025-11-24 02:33:26,139 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:43549 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41207 after 30 s')

2025-11-24 02:33:26,139 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44675 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41207 after 30 s')

2025-11-24 02:33:26,138 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:43129 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41207 after 30 s')

2025-11-24 02:33:26,138 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:34887 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41207 after 30 s')

2025-11-24 02:33:26,137 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:34191 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41207 after 30 s')

2025-11-24 02:33:26,102 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:33359 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41207 after 30 s')

2025-11-24 02:33:26,099 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:32841 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41207 after 30 s')

2025-11-24 02:33:26,080 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41207 after 30 s')

2025-11-24 02:32:57,060 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:37529

2025-11-24 02:32:57,058 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:37529', status: closed, memory: 0, processing: 0>

2025-11-24 02:32:55,961 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:41207', status: closed, memory: 0, processing: 0>

2025-11-24 02:32:30,251 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:33391 failed: OSError: Timed out trying to connect to tcp://172.21.159.199:33391 after 30 s

2025-11-24 02:32:30,201 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:33391 after 30 s')

2025-11-24 02:32:30,148 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:32841 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:33391 after 30 s')

2025-11-24 02:32:30,140 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:43305 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:33391 after 30 s')

2025-11-24 02:32:30,130 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:39867 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:33391 after 30 s')

2025-11-24 02:32:30,109 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:33359 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:33391 after 30 s')

2025-11-24 02:32:30,105 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:33391 after 30 s')

2025-11-24 02:32:30,090 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37207 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:33391 after 30 s')

2025-11-24 02:32:30,078 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:33391 after 30 s')

2025-11-24 02:32:00,875 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:41207

2025-11-24 02:32:00,873 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:41207', status: closed, memory: 0, processing: 0>

2025-11-24 02:31:59,965 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:33391', status: closed, memory: 0, processing: 0>

2025-11-24 02:31:45,376 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:41407 failed: OSError: Timed out trying to connect to tcp://172.21.159.199:41407 after 30 s

2025-11-24 02:31:45,284 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:41407 failed: OSError: Timed out trying to connect to tcp://172.21.159.199:41407 after 30 s

2025-11-24 02:31:45,164 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:32841 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41407 after 30 s')

2025-11-24 02:31:45,105 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:34191 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41407 after 30 s')

2025-11-24 02:31:44,768 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:33391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41407 after 30 s')

2025-11-24 02:31:44,098 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:33359 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41407 after 30 s')

2025-11-24 02:31:43,918 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:43129 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41407 after 30 s')

2025-11-24 02:31:42,990 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:34887 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41407 after 30 s')

2025-11-24 02:31:42,948 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:44675 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:41407 after 30 s')

2025-11-24 02:31:13,335 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:33391

2025-11-24 02:31:13,334 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:33391', status: closed, memory: 0, processing: 0>

2025-11-24 02:31:12,373 - distributed.scheduler - INFO - Close client connection: Client-worker-d04ca99a-c85f-11f0-82cd-62b326af983b

2025-11-24 02:31:12,359 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:41407', status: closed, memory: 0, processing: 0>

2025-11-24 02:31:12,357 - distributed.scheduler - INFO - Remove client Client-worker-d04ca99a-c85f-11f0-82cd-62b326af983b

2025-11-24 02:30:32,927 - distributed.scheduler - INFO - Receive client connection: Client-8d18556b-c8dd-11f0-8001-f2cd832f225c

2025-11-24 02:20:31,423 - distributed.scheduler - INFO - Receive client connection: Client-26937356-c8dc-11f0-8001-f2cd832f225c

2025-11-24 02:10:29,967 - distributed.scheduler - INFO - Receive client connection: Client-c0144eef-c8da-11f0-8001-f2cd832f225c

2025-11-24 02:00:28,561 - distributed.scheduler - INFO - Receive client connection: Client-599cc48c-c8d9-11f0-8001-f2cd832f225c

2025-11-24 01:50:27,120 - distributed.scheduler - INFO - Receive client connection: Client-f320f993-c8d7-11f0-8001-f2cd832f225c

2025-11-24 01:40:25,769 - distributed.scheduler - INFO - Receive client connection: Client-8cb0f170-c8d6-11f0-8001-f2cd832f225c

2025-11-24 01:30:24,281 - distributed.scheduler - INFO - Receive client connection: Client-262d6544-c8d5-11f0-8001-f2cd832f225c

2025-11-24 01:20:22,820 - distributed.scheduler - INFO - Receive client connection: Client-bfadc5be-c8d3-11f0-8001-f2cd832f225c

2025-11-24 01:10:21,353 - distributed.scheduler - INFO - Receive client connection: Client-592caff4-c8d2-11f0-8001-f2cd832f225c

2025-11-24 01:00:19,894 - distributed.scheduler - INFO - Receive client connection: Client-f2a9b8f4-c8d0-11f0-8001-f2cd832f225c

2025-11-24 00:50:18,426 - distributed.scheduler - INFO - Receive client connection: Client-8c2ce6bb-c8cf-11f0-8001-f2cd832f225c

2025-11-24 00:40:17,010 - distributed.scheduler - INFO - Receive client connection: Client-25b46288-c8ce-11f0-8001-f2cd832f225c

2025-11-24 00:30:15,584 - distributed.scheduler - INFO - Receive client connection: Client-bf3a4e96-c8cc-11f0-8001-f2cd832f225c

2025-11-24 00:20:14,123 - distributed.scheduler - INFO - Receive client connection: Client-58bb13c3-c8cb-11f0-8001-f2cd832f225c

2025-11-24 00:10:12,681 - distributed.scheduler - INFO - Receive client connection: Client-f23e99c0-c8c9-11f0-8001-f2cd832f225c

2025-11-24 00:00:11,241 - distributed.scheduler - INFO - Receive client connection: Client-8bc0fbd5-c8c8-11f0-8001-f2cd832f225c

2025-11-23 23:50:09,764 - distributed.scheduler - INFO - Receive client connection: Client-253f110a-c8c7-11f0-8001-f2cd832f225c

2025-11-23 23:40:08,300 - distributed.scheduler - INFO - Receive client connection: Client-bebb4e36-c8c5-11f0-8001-f2cd832f225c

2025-11-23 23:30:06,899 - distributed.scheduler - INFO - Receive client connection: Client-584964ed-c8c4-11f0-8001-f2cd832f225c

2025-11-23 23:20:05,458 - distributed.scheduler - INFO - Receive client connection: Client-f1c6b380-c8c2-11f0-8001-f2cd832f225c

2025-11-23 23:10:04,018 - distributed.scheduler - INFO - Receive client connection: Client-8b4ef24f-c8c1-11f0-8001-f2cd832f225c

2025-11-23 23:00:02,573 - distributed.scheduler - INFO - Receive client connection: Client-24d2bce8-c8c0-11f0-8001-f2cd832f225c

2025-11-23 22:50:01,130 - distributed.scheduler - INFO - Receive client connection: Client-be55bc9e-c8be-11f0-8001-f2cd832f225c

2025-11-23 22:40:59,858 - distributed.scheduler - INFO - Receive client connection: Client-7bb60369-c8bd-11f0-8001-f2cd832f225c

2025-11-23 22:30:58,391 - distributed.scheduler - INFO - Receive client connection: Client-1534e0e7-c8bc-11f0-8001-f2cd832f225c

2025-11-23 22:20:56,986 - distributed.scheduler - INFO - Receive client connection: Client-aebe1c4a-c8ba-11f0-8001-f2cd832f225c

2025-11-23 22:10:55,576 - distributed.scheduler - INFO - Receive client connection: Client-48471b0f-c8b9-11f0-8001-f2cd832f225c

2025-11-23 22:00:54,145 - distributed.scheduler - INFO - Receive client connection: Client-e1cb457d-c8b7-11f0-8001-f2cd832f225c

2025-11-23 21:50:52,698 - distributed.scheduler - INFO - Receive client connection: Client-7b4cd080-c8b6-11f0-8001-f2cd832f225c

2025-11-23 21:40:51,260 - distributed.scheduler - INFO - Receive client connection: Client-14d062a4-c8b5-11f0-8001-f2cd832f225c

2025-11-23 21:30:49,932 - distributed.scheduler - INFO - Receive client connection: Client-ae65b995-c8b3-11f0-8001-f2cd832f225c

2025-11-23 21:20:48,468 - distributed.scheduler - INFO - Receive client connection: Client-47e65145-c8b2-11f0-8001-f2cd832f225c

2025-11-23 21:10:47,021 - distributed.scheduler - INFO - Receive client connection: Client-e168ba8e-c8b0-11f0-8001-f2cd832f225c

2025-11-23 21:00:45,567 - distributed.scheduler - INFO - Receive client connection: Client-7ae8dc1d-c8af-11f0-8001-f2cd832f225c

2025-11-23 20:50:44,058 - distributed.scheduler - INFO - Receive client connection: Client-14630388-c8ae-11f0-8001-f2cd832f225c

2025-11-23 20:40:42,596 - distributed.scheduler - INFO - Receive client connection: Client-ade3a277-c8ac-11f0-8001-f2cd832f225c

2025-11-23 20:30:41,193 - distributed.scheduler - INFO - Receive client connection: Client-476c2781-c8ab-11f0-8001-f2cd832f225c

2025-11-23 20:20:39,818 - distributed.scheduler - INFO - Receive client connection: Client-e0f9d8f8-c8a9-11f0-8001-f2cd832f225c

2025-11-23 20:10:38,359 - distributed.scheduler - INFO - Receive client connection: Client-7a7b7519-c8a8-11f0-8001-f2cd832f225c

2025-11-23 20:00:36,897 - distributed.scheduler - INFO - Receive client connection: Client-13fa6081-c8a7-11f0-8001-f2cd832f225c

2025-11-23 19:50:35,477 - distributed.scheduler - INFO - Receive client connection: Client-ad8143c2-c8a5-11f0-8001-f2cd832f225c

2025-11-23 19:40:34,034 - distributed.scheduler - INFO - Receive client connection: Client-470507aa-c8a4-11f0-8001-f2cd832f225c

2025-11-23 19:30:32,658 - distributed.scheduler - INFO - Receive client connection: Client-e090b56f-c8a2-11f0-8001-f2cd832f225c

2025-11-23 19:21:04,979 - distributed.scheduler - INFO - Close client connection: Client-79934bc3-c89a-11f0-8001-f2cd832f225c

2025-11-23 19:21:04,978 - distributed.scheduler - INFO - Close client connection: Client-469d408a-c89d-11f0-8001-f2cd832f225c

2025-11-23 19:21:04,978 - distributed.scheduler - INFO - Close client connection: Client-e00f085d-c89b-11f0-8001-f2cd832f225c

2025-11-23 19:21:04,978 - distributed.scheduler - INFO - Close client connection: Client-ad0fbee7-c89e-11f0-8001-f2cd832f225c

2025-11-23 19:21:04,978 - distributed.scheduler - INFO - Close client connection: Client-138d0f47-c8a0-11f0-8001-f2cd832f225c

2025-11-23 19:21:04,977 - distributed.scheduler - INFO - Remove client Client-79934bc3-c89a-11f0-8001-f2cd832f225c

2025-11-23 19:21:04,975 - distributed.scheduler - INFO - Remove client Client-469d408a-c89d-11f0-8001-f2cd832f225c

2025-11-23 19:21:04,975 - distributed.scheduler - INFO - Remove client Client-e00f085d-c89b-11f0-8001-f2cd832f225c

2025-11-23 19:21:04,974 - distributed.scheduler - INFO - Remove client Client-ad0fbee7-c89e-11f0-8001-f2cd832f225c

2025-11-23 19:21:04,973 - distributed.scheduler - INFO - Remove client Client-138d0f47-c8a0-11f0-8001-f2cd832f225c

2025-11-23 19:21:04,132 - distributed.scheduler - INFO - Close client connection: Client-7a0b585a-c8a1-11f0-8001-f2cd832f225c

2025-11-23 19:21:04,131 - distributed.scheduler - INFO - Remove client Client-7a0b585a-c8a1-11f0-8001-f2cd832f225c

2025-11-23 19:21:04,130 - distributed.scheduler - INFO - Remove client Client-7a0b585a-c8a1-11f0-8001-f2cd832f225c

2025-11-23 19:20:31,152 - distributed.scheduler - INFO - Receive client connection: Client-7a0b585a-c8a1-11f0-8001-f2cd832f225c

2025-11-23 19:10:29,700 - distributed.scheduler - INFO - Receive client connection: Client-138d0f47-c8a0-11f0-8001-f2cd832f225c

2025-11-23 19:00:28,248 - distributed.scheduler - INFO - Receive client connection: Client-ad0fbee7-c89e-11f0-8001-f2cd832f225c

2025-11-23 18:50:26,882 - distributed.scheduler - INFO - Receive client connection: Client-469d408a-c89d-11f0-8001-f2cd832f225c

2025-11-23 18:40:25,326 - distributed.scheduler - INFO - Receive client connection: Client-e00f085d-c89b-11f0-8001-f2cd832f225c

2025-11-23 18:30:23,895 - distributed.scheduler - INFO - Receive client connection: Client-79934bc3-c89a-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,768 - distributed.scheduler - INFO - Close client connection: Client-7838d130-c885-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,768 - distributed.scheduler - INFO - Close client connection: Client-deadc2fb-c886-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,768 - distributed.scheduler - INFO - Close client connection: Client-452c32d7-c888-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,768 - distributed.scheduler - INFO - Close client connection: Client-aba42a40-c889-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,768 - distributed.scheduler - INFO - Close client connection: Client-1221c576-c88b-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,767 - distributed.scheduler - INFO - Close client connection: Client-78a09922-c88c-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,767 - distributed.scheduler - INFO - Close client connection: Client-df1c0f36-c88d-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,767 - distributed.scheduler - INFO - Close client connection: Client-4599110c-c88f-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,767 - distributed.scheduler - INFO - Close client connection: Client-ac1c1658-c890-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,767 - distributed.scheduler - INFO - Close client connection: Client-129aa209-c892-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,767 - distributed.scheduler - INFO - Close client connection: Client-df9ff1ef-c894-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,766 - distributed.scheduler - INFO - Close client connection: Client-792185b3-c893-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,762 - distributed.scheduler - INFO - Close client connection: Client-ac8d2f2d-c897-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,762 - distributed.scheduler - INFO - Close client connection: Client-46166f8a-c896-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,760 - distributed.scheduler - INFO - Remove client Client-7838d130-c885-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,759 - distributed.scheduler - INFO - Remove client Client-deadc2fb-c886-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,758 - distributed.scheduler - INFO - Remove client Client-452c32d7-c888-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,758 - distributed.scheduler - INFO - Remove client Client-aba42a40-c889-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,757 - distributed.scheduler - INFO - Remove client Client-1221c576-c88b-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,756 - distributed.scheduler - INFO - Remove client Client-78a09922-c88c-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,756 - distributed.scheduler - INFO - Remove client Client-df1c0f36-c88d-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,755 - distributed.scheduler - INFO - Remove client Client-4599110c-c88f-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,755 - distributed.scheduler - INFO - Remove client Client-ac1c1658-c890-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,754 - distributed.scheduler - INFO - Remove client Client-129aa209-c892-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,753 - distributed.scheduler - INFO - Remove client Client-df9ff1ef-c894-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,753 - distributed.scheduler - INFO - Remove client Client-792185b3-c893-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,749 - distributed.scheduler - INFO - Remove client Client-ac8d2f2d-c897-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,748 - distributed.scheduler - INFO - Remove client Client-46166f8a-c896-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,236 - distributed.scheduler - INFO - Close client connection: Client-13094b5b-c899-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,235 - distributed.scheduler - INFO - Remove client Client-13094b5b-c899-11f0-8001-f2cd832f225c

2025-11-23 18:21:01,234 - distributed.scheduler - INFO - Remove client Client-13094b5b-c899-11f0-8001-f2cd832f225c

2025-11-23 18:20:22,360 - distributed.scheduler - INFO - Receive client connection: Client-13094b5b-c899-11f0-8001-f2cd832f225c

2025-11-23 18:10:20,923 - distributed.scheduler - INFO - Receive client connection: Client-ac8d2f2d-c897-11f0-8001-f2cd832f225c

2025-11-23 18:00:19,524 - distributed.scheduler - INFO - Receive client connection: Client-46166f8a-c896-11f0-8001-f2cd832f225c

2025-11-23 17:50:18,117 - distributed.scheduler - INFO - Receive client connection: Client-df9ff1ef-c894-11f0-8001-f2cd832f225c

2025-11-23 17:40:16,668 - distributed.scheduler - INFO - Receive client connection: Client-792185b3-c893-11f0-8001-f2cd832f225c

2025-11-23 17:30:15,156 - distributed.scheduler - INFO - Receive client connection: Client-129aa209-c892-11f0-8001-f2cd832f225c

2025-11-23 17:20:13,706 - distributed.scheduler - INFO - Receive client connection: Client-ac1c1658-c890-11f0-8001-f2cd832f225c

2025-11-23 17:10:12,227 - distributed.scheduler - INFO - Receive client connection: Client-4599110c-c88f-11f0-8001-f2cd832f225c

2025-11-23 17:00:10,780 - distributed.scheduler - INFO - Receive client connection: Client-df1c0f36-c88d-11f0-8001-f2cd832f225c

2025-11-23 16:50:09,342 - distributed.scheduler - INFO - Receive client connection: Client-78a09922-c88c-11f0-8001-f2cd832f225c

2025-11-23 16:40:07,907 - distributed.scheduler - INFO - Receive client connection: Client-1221c576-c88b-11f0-8001-f2cd832f225c

2025-11-23 16:30:06,443 - distributed.scheduler - INFO - Receive client connection: Client-aba42a40-c889-11f0-8001-f2cd832f225c

2025-11-23 16:20:05,033 - distributed.scheduler - INFO - Receive client connection: Client-452c32d7-c888-11f0-8001-f2cd832f225c

2025-11-23 16:10:03,577 - distributed.scheduler - INFO - Receive client connection: Client-deadc2fb-c886-11f0-8001-f2cd832f225c

2025-11-23 16:00:02,195 - distributed.scheduler - INFO - Receive client connection: Client-7838d130-c885-11f0-8001-f2cd832f225c

2025-11-23 15:50:43,653 - distributed.scheduler - INFO - Close client connection: Client-11b5b9b6-c884-11f0-8001-f2cd832f225c

2025-11-23 15:50:43,652 - distributed.scheduler - INFO - Remove client Client-11b5b9b6-c884-11f0-8001-f2cd832f225c

2025-11-23 15:50:43,650 - distributed.scheduler - INFO - Remove client Client-11b5b9b6-c884-11f0-8001-f2cd832f225c

2025-11-23 15:50:00,715 - distributed.scheduler - INFO - Receive client connection: Client-11b5b9b6-c884-11f0-8001-f2cd832f225c

2025-11-23 15:41:41,509 - distributed.scheduler - INFO - Close client connection: Client-cf151bea-c882-11f0-8001-f2cd832f225c

2025-11-23 15:41:41,508 - distributed.scheduler - INFO - Remove client Client-cf151bea-c882-11f0-8001-f2cd832f225c

2025-11-23 15:41:41,506 - distributed.scheduler - INFO - Remove client Client-cf151bea-c882-11f0-8001-f2cd832f225c

2025-11-23 15:40:59,432 - distributed.scheduler - INFO - Receive client connection: Client-cf151bea-c882-11f0-8001-f2cd832f225c

2025-11-23 15:32:13,566 - distributed.scheduler - INFO - Close client connection: Client-68965e50-c881-11f0-8001-f2cd832f225c

2025-11-23 15:32:13,565 - distributed.scheduler - INFO - Remove client Client-68965e50-c881-11f0-8001-f2cd832f225c

2025-11-23 15:32:13,564 - distributed.scheduler - INFO - Remove client Client-68965e50-c881-11f0-8001-f2cd832f225c

2025-11-23 15:31:03,326 - distributed.scheduler - INFO - Receive client connection: Client-worker-6bbc549b-c881-11f0-82b0-2a2d08e6be2e

2025-11-23 15:30:57,982 - distributed.scheduler - INFO - Receive client connection: Client-68965e50-c881-11f0-8001-f2cd832f225c

2025-11-23 15:21:39,044 - distributed.scheduler - INFO - Close client connection: Client-0229db8f-c880-11f0-8001-f2cd832f225c

2025-11-23 15:21:39,043 - distributed.scheduler - INFO - Remove client Client-0229db8f-c880-11f0-8001-f2cd832f225c

2025-11-23 15:21:39,041 - distributed.scheduler - INFO - Remove client Client-0229db8f-c880-11f0-8001-f2cd832f225c

2025-11-23 15:20:56,637 - distributed.scheduler - INFO - Receive client connection: Client-0229db8f-c880-11f0-8001-f2cd832f225c

2025-11-23 15:11:36,949 - distributed.scheduler - INFO - Close client connection: Client-9bae8949-c87e-11f0-8001-f2cd832f225c

2025-11-23 15:11:36,948 - distributed.scheduler - INFO - Remove client Client-9bae8949-c87e-11f0-8001-f2cd832f225c

2025-11-23 15:11:36,947 - distributed.scheduler - INFO - Remove client Client-9bae8949-c87e-11f0-8001-f2cd832f225c

2025-11-23 15:10:55,206 - distributed.scheduler - INFO - Receive client connection: Client-9bae8949-c87e-11f0-8001-f2cd832f225c

2025-11-23 15:04:29,888 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:41829 failed: OSError: Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s

2025-11-23 15:04:29,864 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s')

2025-11-23 15:04:29,023 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:34191 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s')

2025-11-23 15:04:29,004 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:34887 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s')

2025-11-23 15:04:28,994 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:39867 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s')

2025-11-23 15:04:28,993 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:43549 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s')

2025-11-23 15:04:28,977 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:41407 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s')

2025-11-23 15:04:28,970 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:33359 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s')

2025-11-23 15:04:28,969 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s')

2025-11-23 15:04:28,967 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:32841 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s')

2025-11-23 15:04:28,958 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37207 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s')

2025-11-23 15:04:28,936 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:35391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s')

2025-11-23 15:04:28,936 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s')

2025-11-23 15:04:28,931 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:43305 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s')

2025-11-23 15:04:28,920 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:40623 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:41829 after 30 s')

2025-11-23 15:03:59,822 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:44675

2025-11-23 15:03:59,820 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:44675', status: closed, memory: 0, processing: 0>

2025-11-23 15:03:58,801 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:41829', status: closed, memory: 0, processing: 0>

2025-11-23 15:03:47,089 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:42317 failed: OSError: Timed out trying to connect to tcp://172.21.12.199:42317 after 30 s

2025-11-23 15:03:46,996 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:39867 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42317 after 30 s')

2025-11-23 15:03:46,993 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:32841 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42317 after 30 s')

2025-11-23 15:03:46,992 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:34191 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42317 after 30 s')

2025-11-23 15:03:46,973 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42317 after 30 s')

2025-11-23 15:03:46,966 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42317 after 30 s')

2025-11-23 15:03:46,957 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:41407 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42317 after 30 s')

2025-11-23 15:03:46,957 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:43305 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42317 after 30 s')

2025-11-23 15:03:46,957 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:43129 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42317 after 30 s')

2025-11-23 15:03:46,956 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37207 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42317 after 30 s')

2025-11-23 15:03:46,907 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:35391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42317 after 30 s')

2025-11-23 15:03:17,707 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:41829

2025-11-23 15:03:17,704 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:41829', status: closed, memory: 0, processing: 0>

2025-11-23 15:03:16,804 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:42317', status: closed, memory: 0, processing: 0>

2025-11-23 15:03:04,585 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:42241 failed: OSError: Timed out trying to connect to tcp://172.21.12.199:42241 after 30 s

2025-11-23 15:03:04,579 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:39867 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42241 after 30 s')

2025-11-23 15:03:04,502 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:33359 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42241 after 30 s')

2025-11-23 15:03:04,491 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37207 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42241 after 30 s')

2025-11-23 15:03:04,489 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:43305 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42241 after 30 s')

2025-11-23 15:03:04,487 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:41407 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42241 after 30 s')

2025-11-23 15:03:04,487 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:32841 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42241 after 30 s')

2025-11-23 15:03:04,463 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:34887 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42241 after 30 s')

2025-11-23 15:03:04,460 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:35391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42241 after 30 s')

2025-11-23 15:03:04,431 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43581 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:42241 after 30 s')

2025-11-23 15:02:35,267 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:42317

2025-11-23 15:02:35,266 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:42317', status: closed, memory: 0, processing: 0>

2025-11-23 15:02:34,322 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:42241', status: closed, memory: 0, processing: 0>

2025-11-23 15:02:24,065 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:42241

2025-11-23 15:02:24,063 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:42241', status: closed, memory: 0, processing: 0>

2025-11-23 15:02:23,140 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:38829', status: closed, memory: 0, processing: 0>

2025-11-23 15:02:10,160 - distributed.scheduler - INFO - Close client connection: Client-9b4aa4c5-c877-11f0-8001-f2cd832f225c

2025-11-23 15:02:10,159 - distributed.scheduler - INFO - Close client connection: Client-34d0a6d2-c876-11f0-8001-f2cd832f225c

2025-11-23 15:02:10,159 - distributed.scheduler - INFO - Close client connection: Client-01c7ee70-c879-11f0-8001-f2cd832f225c

2025-11-23 15:02:10,159 - distributed.scheduler - INFO - Close client connection: Client-6842e457-c87a-11f0-8001-f2cd832f225c

2025-11-23 15:02:10,158 - distributed.scheduler - INFO - Close client connection: Client-cec81e51-c87b-11f0-8001-f2cd832f225c

2025-11-23 15:02:10,157 - distributed.scheduler - INFO - Remove client Client-9b4aa4c5-c877-11f0-8001-f2cd832f225c

2025-11-23 15:02:10,156 - distributed.scheduler - INFO - Remove client Client-34d0a6d2-c876-11f0-8001-f2cd832f225c

2025-11-23 15:02:10,154 - distributed.scheduler - INFO - Remove client Client-01c7ee70-c879-11f0-8001-f2cd832f225c

2025-11-23 15:02:10,153 - distributed.scheduler - INFO - Remove client Client-6842e457-c87a-11f0-8001-f2cd832f225c

2025-11-23 15:02:10,149 - distributed.scheduler - INFO - Remove client Client-cec81e51-c87b-11f0-8001-f2cd832f225c

2025-11-23 15:02:09,221 - distributed.scheduler - INFO - Close client connection: Client-353f7772-c87d-11f0-8001-f2cd832f225c

2025-11-23 15:02:09,220 - distributed.scheduler - INFO - Remove client Client-353f7772-c87d-11f0-8001-f2cd832f225c

2025-11-23 15:02:09,219 - distributed.scheduler - INFO - Remove client Client-353f7772-c87d-11f0-8001-f2cd832f225c

2025-11-23 15:02:06,997 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:45375 failed: OSError: Timed out trying to connect to tcp://172.21.12.199:45375 after 30 s

2025-11-23 15:02:06,941 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:45375 after 30 s')

2025-11-23 15:02:06,933 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:39867 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:45375 after 30 s')

2025-11-23 15:02:06,897 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:40623 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:45375 after 30 s')

2025-11-23 15:02:06,886 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37207 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.12.199:45375 after 30 s')

2025-11-23 15:01:37,658 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.12.199:38829

2025-11-23 15:01:37,655 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.12.199:38829', status: closed, memory: 0, processing: 0>

2025-11-23 15:01:36,755 - distributed.scheduler - INFO - Close client connection: Client-worker-5b7f2410-c6f7-11f0-8143-2a2d08e6be2e

2025-11-23 15:01:36,738 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.12.199:45375', status: closed, memory: 0, processing: 0>

2025-11-23 15:01:36,737 - distributed.scheduler - INFO - Remove client Client-worker-5b7f2410-c6f7-11f0-8143-2a2d08e6be2e

2025-11-23 15:00:53,841 - distributed.scheduler - INFO - Receive client connection: Client-353f7772-c87d-11f0-8001-f2cd832f225c

2025-11-23 14:50:52,444 - distributed.scheduler - INFO - Receive client connection: Client-cec81e51-c87b-11f0-8001-f2cd832f225c

2025-11-23 14:40:50,954 - distributed.scheduler - INFO - Receive client connection: Client-6842e457-c87a-11f0-8001-f2cd832f225c

2025-11-23 14:30:49,519 - distributed.scheduler - INFO - Receive client connection: Client-01c7ee70-c879-11f0-8001-f2cd832f225c

2025-11-23 14:20:48,069 - distributed.scheduler - INFO - Receive client connection: Client-9b4aa4c5-c877-11f0-8001-f2cd832f225c

2025-11-23 14:10:46,660 - distributed.scheduler - INFO - Receive client connection: Client-34d0a6d2-c876-11f0-8001-f2cd832f225c

2025-11-23 14:01:55,971 - distributed.scheduler - INFO - Close client connection: Client-ce50d6ee-c874-11f0-8001-f2cd832f225c

2025-11-23 14:01:55,969 - distributed.scheduler - INFO - Remove client Client-ce50d6ee-c874-11f0-8001-f2cd832f225c

2025-11-23 14:01:55,967 - distributed.scheduler - INFO - Remove client Client-ce50d6ee-c874-11f0-8001-f2cd832f225c

2025-11-23 14:00:45,188 - distributed.scheduler - INFO - Receive client connection: Client-ce50d6ee-c874-11f0-8001-f2cd832f225c

2025-11-23 13:51:22,879 - distributed.scheduler - INFO - Close client connection: Client-67d64f96-c873-11f0-8001-f2cd832f225c

2025-11-23 13:51:22,877 - distributed.scheduler - INFO - Remove client Client-67d64f96-c873-11f0-8001-f2cd832f225c

2025-11-23 13:51:22,875 - distributed.scheduler - INFO - Remove client Client-67d64f96-c873-11f0-8001-f2cd832f225c

2025-11-23 13:50:43,773 - distributed.scheduler - INFO - Receive client connection: Client-67d64f96-c873-11f0-8001-f2cd832f225c

2025-11-23 13:41:22,358 - distributed.scheduler - INFO - Close client connection: Client-01626966-c872-11f0-8001-f2cd832f225c

2025-11-23 13:41:22,356 - distributed.scheduler - INFO - Remove client Client-01626966-c872-11f0-8001-f2cd832f225c

2025-11-23 13:41:22,353 - distributed.scheduler - INFO - Remove client Client-01626966-c872-11f0-8001-f2cd832f225c

2025-11-23 13:40:42,380 - distributed.scheduler - INFO - Receive client connection: Client-01626966-c872-11f0-8001-f2cd832f225c

2025-11-23 13:31:53,332 - distributed.scheduler - INFO - Close client connection: Client-9aea1266-c870-11f0-8001-f2cd832f225c

2025-11-23 13:31:53,331 - distributed.scheduler - INFO - Remove client Client-9aea1266-c870-11f0-8001-f2cd832f225c

2025-11-23 13:31:53,329 - distributed.scheduler - INFO - Remove client Client-9aea1266-c870-11f0-8001-f2cd832f225c

2025-11-23 13:30:40,971 - distributed.scheduler - INFO - Receive client connection: Client-9aea1266-c870-11f0-8001-f2cd832f225c

2025-11-23 13:21:19,491 - distributed.scheduler - INFO - Close client connection: Client-34755ac2-c86f-11f0-8001-f2cd832f225c

2025-11-23 13:21:19,489 - distributed.scheduler - INFO - Remove client Client-34755ac2-c86f-11f0-8001-f2cd832f225c

2025-11-23 13:21:19,487 - distributed.scheduler - INFO - Remove client Client-34755ac2-c86f-11f0-8001-f2cd832f225c

2025-11-23 13:20:39,573 - distributed.scheduler - INFO - Receive client connection: Client-34755ac2-c86f-11f0-8001-f2cd832f225c

2025-11-23 13:11:20,768 - distributed.scheduler - INFO - Close client connection: Client-67044de6-c865-11f0-8001-f2cd832f225c

2025-11-23 13:11:20,767 - distributed.scheduler - INFO - Close client connection: Client-cd7f61ee-c866-11f0-8001-f2cd832f225c

2025-11-23 13:11:20,767 - distributed.scheduler - INFO - Close client connection: Client-33f920af-c868-11f0-8001-f2cd832f225c

2025-11-23 13:11:20,767 - distributed.scheduler - INFO - Close client connection: Client-9a730901-c869-11f0-8001-f2cd832f225c

2025-11-23 13:11:20,767 - distributed.scheduler - INFO - Close client connection: Client-00f72134-c86b-11f0-8001-f2cd832f225c

2025-11-23 13:11:20,767 - distributed.scheduler - INFO - Close client connection: Client-6774f137-c86c-11f0-8001-f2cd832f225c

2025-11-23 13:11:20,765 - distributed.scheduler - INFO - Remove client Client-67044de6-c865-11f0-8001-f2cd832f225c

2025-11-23 13:11:20,765 - distributed.scheduler - INFO - Remove client Client-cd7f61ee-c866-11f0-8001-f2cd832f225c

2025-11-23 13:11:20,764 - distributed.scheduler - INFO - Remove client Client-33f920af-c868-11f0-8001-f2cd832f225c

2025-11-23 13:11:20,763 - distributed.scheduler - INFO - Remove client Client-9a730901-c869-11f0-8001-f2cd832f225c

2025-11-23 13:11:20,763 - distributed.scheduler - INFO - Remove client Client-00f72134-c86b-11f0-8001-f2cd832f225c

2025-11-23 13:11:20,761 - distributed.scheduler - INFO - Remove client Client-6774f137-c86c-11f0-8001-f2cd832f225c

2025-11-23 13:11:19,848 - distributed.scheduler - INFO - Close client connection: Client-cdf2a1fd-c86d-11f0-8001-f2cd832f225c

2025-11-23 13:11:19,847 - distributed.scheduler - INFO - Remove client Client-cdf2a1fd-c86d-11f0-8001-f2cd832f225c

2025-11-23 13:11:19,846 - distributed.scheduler - INFO - Remove client Client-cdf2a1fd-c86d-11f0-8001-f2cd832f225c

2025-11-23 13:10:38,090 - distributed.scheduler - INFO - Receive client connection: Client-cdf2a1fd-c86d-11f0-8001-f2cd832f225c

2025-11-23 13:00:36,643 - distributed.scheduler - INFO - Receive client connection: Client-6774f137-c86c-11f0-8001-f2cd832f225c

2025-11-23 12:50:35,193 - distributed.scheduler - INFO - Receive client connection: Client-00f72134-c86b-11f0-8001-f2cd832f225c

2025-11-23 12:40:33,706 - distributed.scheduler - INFO - Receive client connection: Client-9a730901-c869-11f0-8001-f2cd832f225c

2025-11-23 12:30:36,725 - distributed.scheduler - INFO - Receive client connection: Client-worker-36a05c5a-c868-11f0-83c0-8253c573df33

2025-11-23 12:30:32,283 - distributed.scheduler - INFO - Receive client connection: Client-33f920af-c868-11f0-8001-f2cd832f225c

2025-11-23 12:20:30,860 - distributed.scheduler - INFO - Receive client connection: Client-cd7f61ee-c866-11f0-8001-f2cd832f225c

2025-11-23 12:10:29,433 - distributed.scheduler - INFO - Receive client connection: Client-67044de6-c865-11f0-8001-f2cd832f225c

2025-11-23 12:01:44,298 - distributed.scheduler - INFO - Close client connection: Client-00859407-c864-11f0-8001-f2cd832f225c

2025-11-23 12:01:44,297 - distributed.scheduler - INFO - Remove client Client-00859407-c864-11f0-8001-f2cd832f225c

2025-11-23 12:01:44,295 - distributed.scheduler - INFO - Remove client Client-00859407-c864-11f0-8001-f2cd832f225c

2025-11-23 12:00:33,752 - distributed.scheduler - INFO - Receive client connection: Client-worker-03ea302d-c864-11f0-8283-32a59f79d364

2025-11-23 12:00:27,979 - distributed.scheduler - INFO - Receive client connection: Client-00859407-c864-11f0-8001-f2cd832f225c

2025-11-23 11:51:07,609 - distributed.scheduler - INFO - Close client connection: Client-9a167162-c862-11f0-8001-f2cd832f225c

2025-11-23 11:51:07,608 - distributed.scheduler - INFO - Remove client Client-9a167162-c862-11f0-8001-f2cd832f225c

2025-11-23 11:51:07,606 - distributed.scheduler - INFO - Remove client Client-9a167162-c862-11f0-8001-f2cd832f225c

2025-11-23 11:50:26,631 - distributed.scheduler - INFO - Receive client connection: Client-9a167162-c862-11f0-8001-f2cd832f225c

2025-11-23 11:41:06,015 - distributed.scheduler - INFO - Close client connection: Client-339c63d5-c861-11f0-8001-f2cd832f225c

2025-11-23 11:41:06,011 - distributed.scheduler - INFO - Remove client Client-339c63d5-c861-11f0-8001-f2cd832f225c

2025-11-23 11:41:06,009 - distributed.scheduler - INFO - Remove client Client-339c63d5-c861-11f0-8001-f2cd832f225c

2025-11-23 11:40:25,201 - distributed.scheduler - INFO - Receive client connection: Client-339c63d5-c861-11f0-8001-f2cd832f225c

2025-11-23 11:31:36,173 - distributed.scheduler - INFO - Close client connection: Client-cd246ee7-c85f-11f0-8001-f2cd832f225c

2025-11-23 11:31:36,173 - distributed.scheduler - INFO - Remove client Client-cd246ee7-c85f-11f0-8001-f2cd832f225c

2025-11-23 11:31:36,172 - distributed.scheduler - INFO - Remove client Client-cd246ee7-c85f-11f0-8001-f2cd832f225c

2025-11-23 11:30:29,129 - distributed.scheduler - INFO - Receive client connection: Client-worker-d04ca99a-c85f-11f0-82cd-62b326af983b

2025-11-23 11:30:23,784 - distributed.scheduler - INFO - Receive client connection: Client-cd246ee7-c85f-11f0-8001-f2cd832f225c

2025-11-23 11:21:04,250 - distributed.scheduler - INFO - Close client connection: Client-66aa520f-c85e-11f0-8001-f2cd832f225c

2025-11-23 11:21:04,247 - distributed.scheduler - INFO - Remove client Client-66aa520f-c85e-11f0-8001-f2cd832f225c

2025-11-23 11:21:04,245 - distributed.scheduler - INFO - Remove client Client-66aa520f-c85e-11f0-8001-f2cd832f225c

2025-11-23 11:20:22,366 - distributed.scheduler - INFO - Receive client connection: Client-66aa520f-c85e-11f0-8001-f2cd832f225c

2025-11-23 11:11:03,385 - distributed.scheduler - INFO - Close client connection: Client-002d95fc-c85d-11f0-8001-f2cd832f225c

2025-11-23 11:11:03,384 - distributed.scheduler - INFO - Remove client Client-002d95fc-c85d-11f0-8001-f2cd832f225c

2025-11-23 11:11:03,382 - distributed.scheduler - INFO - Remove client Client-002d95fc-c85d-11f0-8001-f2cd832f225c

2025-11-23 11:10:20,933 - distributed.scheduler - INFO - Receive client connection: Client-002d95fc-c85d-11f0-8001-f2cd832f225c

2025-11-23 11:02:58,534 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:40623

2025-11-23 11:02:58,533 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:40623', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:58,432 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:33359

2025-11-23 11:02:58,431 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:33359', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:58,270 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:41407

2025-11-23 11:02:58,269 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:41407', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:57,447 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:39145', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:57,389 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:40521', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:57,233 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:40759', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:46,934 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:39145

2025-11-23 11:02:46,933 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:39145', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:46,335 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:40521

2025-11-23 11:02:46,334 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:40521', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:46,035 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:38069', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:45,745 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:40759

2025-11-23 11:02:45,743 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:40759', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:45,208 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:36753', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:44,928 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:40875', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:34,304 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:36753

2025-11-23 11:02:34,303 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:36753', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:34,211 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:38069

2025-11-23 11:02:34,210 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:38069', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:34,005 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:40875

2025-11-23 11:02:34,004 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:40875', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:33,301 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:33301', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:33,255 - distributed.scheduler - INFO - Close client connection: Client-worker-ddad48a8-c85b-11f0-838b-8253c573df33

2025-11-23 11:02:33,253 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:42901', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:33,251 - distributed.scheduler - INFO - Remove client Client-worker-ddad48a8-c85b-11f0-838b-8253c573df33

2025-11-23 11:02:33,130 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:37801', status: closed, memory: 0, processing: 0>

2025-11-23 11:02:13,530 - distributed.scheduler - INFO - Receive client connection: Client-worker-ddad48a8-c85b-11f0-838b-8253c573df33

2025-11-23 11:02:10,703 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:43129 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.199:39663 after 30 s')

2025-11-23 11:02:10,472 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:45375 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:36019 after 30 s')

2025-11-23 11:02:10,394 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:43549 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.252:37333 after 30 s')

2025-11-23 11:02:10,366 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:39663 failed: OSError: Timed out trying to connect to tcp://172.21.159.199:39663 after 30 s

2025-11-23 11:02:10,365 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:37333 failed: OSError: Timed out trying to connect to tcp://172.21.159.252:37333 after 30 s

2025-11-23 11:02:10,364 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:36019 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:36019 after 30 s

2025-11-23 11:02:09,765 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:35391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:36019 after 30 s')

2025-11-23 11:01:46,025 - distributed.scheduler - INFO - Close client connection: Client-99b24ac5-c85b-11f0-8001-f2cd832f225c

2025-11-23 11:01:46,024 - distributed.scheduler - INFO - Remove client Client-99b24ac5-c85b-11f0-8001-f2cd832f225c

2025-11-23 11:01:46,024 - distributed.scheduler - INFO - Remove client Client-99b24ac5-c85b-11f0-8001-f2cd832f225c

2025-11-23 11:01:40,617 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:33301

2025-11-23 11:01:40,616 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:33301', status: closed, memory: 0, processing: 0>

2025-11-23 11:01:40,518 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:42901

2025-11-23 11:01:40,517 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:42901', status: closed, memory: 0, processing: 0>

2025-11-23 11:01:40,410 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:37801

2025-11-23 11:01:40,408 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:37801', status: closed, memory: 0, processing: 0>

2025-11-23 11:01:39,552 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:39663', status: closed, memory: 0, processing: 0>

2025-11-23 11:01:39,545 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:37333', status: closed, memory: 0, processing: 0>

2025-11-23 11:01:39,497 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:36019', status: closed, memory: 0, processing: 0>

2025-11-23 11:00:59,796 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.252:37333

2025-11-23 11:00:59,795 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.252:37333', status: closed, memory: 0, processing: 0>

2025-11-23 11:00:58,988 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: ['tcp://172.21.159.252:45407'], run-bd755283ffdba990de0bd1ef2bc76476 NoneType: None

2025-11-23 11:00:58,988 - distributed.scheduler - ERROR - Shut down workers that don't have promised key: ['tcp://172.21.159.252:45407'], run-50bc0b316d1a522261220d388c4ef740 NoneType: None

2025-11-23 11:00:58,986 - distributed.scheduler - ERROR - Couldn't gather keys {'run-50bc0b316d1a522261220d388c4ef740': ['tcp://172.21.159.252:45407'], 'run-bd755283ffdba990de0bd1ef2bc76476': ['tcp://172.21.159.252:45407']} state: ['processing', 'processing'] workers: ['tcp://172.21.159.252:45407'] NoneType: None

2025-11-23 11:00:58,984 - distributed.scheduler - INFO - Close client connection: Client-worker-efa349ae-c820-11f0-82f7-8253c573df33

2025-11-23 11:00:58,970 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.252:45407', status: closed, memory: 0, processing: 0>

2025-11-23 11:00:58,970 - distributed.scheduler - INFO - Remove client Client-worker-efa349ae-c820-11f0-82f7-8253c573df33

2025-11-23 11:00:58,499 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.199:39663

2025-11-23 11:00:58,498 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.199:39663', status: closed, memory: 0, processing: 0>

2025-11-23 11:00:58,005 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:36019

2025-11-23 11:00:58,003 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:36019', status: closed, memory: 0, processing: 0>

2025-11-23 11:00:57,428 - distributed.scheduler - INFO - Close client connection: Client-worker-4420e102-c62e-11f0-810c-62b326af983b

2025-11-23 11:00:57,421 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.199:33551', status: closed, memory: 0, processing: 0>

2025-11-23 11:00:57,421 - distributed.scheduler - INFO - Remove client Client-worker-4420e102-c62e-11f0-810c-62b326af983b

2025-11-23 11:00:57,189 - distributed.scheduler - INFO - Close client connection: Client-worker-ceb2e01c-c84a-11f0-8204-32a59f79d364

2025-11-23 11:00:57,173 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:45707', status: closed, memory: 0, processing: 0>

2025-11-23 11:00:57,170 - distributed.scheduler - INFO - Remove client Client-worker-ceb2e01c-c84a-11f0-8204-32a59f79d364

2025-11-23 11:00:24,468 - distributed.scheduler - INFO - Receive client connection: Client-worker-9ca411cf-c85b-11f0-833d-f2a719d69604

2025-11-23 11:00:19,493 - distributed.scheduler - INFO - Receive client connection: Client-99b24ac5-c85b-11f0-8001-f2cd832f225c

2025-11-23 10:50:57,060 - distributed.scheduler - INFO - Close client connection: Client-33327d03-c85a-11f0-8001-f2cd832f225c

2025-11-23 10:50:57,059 - distributed.scheduler - INFO - Remove client Client-33327d03-c85a-11f0-8001-f2cd832f225c

2025-11-23 10:50:57,057 - distributed.scheduler - INFO - Remove client Client-33327d03-c85a-11f0-8001-f2cd832f225c

2025-11-23 10:50:18,037 - distributed.scheduler - INFO - Receive client connection: Client-33327d03-c85a-11f0-8001-f2cd832f225c

2025-11-23 10:40:56,683 - distributed.scheduler - INFO - Close client connection: Client-ccc1a43c-c858-11f0-8001-f2cd832f225c

2025-11-23 10:40:56,681 - distributed.scheduler - INFO - Remove client Client-ccc1a43c-c858-11f0-8001-f2cd832f225c

2025-11-23 10:40:56,679 - distributed.scheduler - INFO - Remove client Client-ccc1a43c-c858-11f0-8001-f2cd832f225c

2025-11-23 10:40:16,671 - distributed.scheduler - INFO - Receive client connection: Client-ccc1a43c-c858-11f0-8001-f2cd832f225c

2025-11-23 10:31:26,108 - distributed.scheduler - INFO - Close client connection: Client-6635ceba-c857-11f0-8001-f2cd832f225c

2025-11-23 10:31:26,106 - distributed.scheduler - INFO - Remove client Client-6635ceba-c857-11f0-8001-f2cd832f225c

2025-11-23 10:31:26,104 - distributed.scheduler - INFO - Remove client Client-6635ceba-c857-11f0-8001-f2cd832f225c

2025-11-23 10:30:15,129 - distributed.scheduler - INFO - Receive client connection: Client-6635ceba-c857-11f0-8001-f2cd832f225c

2025-11-23 10:20:52,134 - distributed.scheduler - INFO - Close client connection: Client-ffb5bb61-c855-11f0-8001-f2cd832f225c

2025-11-23 10:20:52,133 - distributed.scheduler - INFO - Remove client Client-ffb5bb61-c855-11f0-8001-f2cd832f225c

2025-11-23 10:20:52,131 - distributed.scheduler - INFO - Remove client Client-ffb5bb61-c855-11f0-8001-f2cd832f225c

2025-11-23 10:20:13,666 - distributed.scheduler - INFO - Receive client connection: Client-ffb5bb61-c855-11f0-8001-f2cd832f225c

2025-11-23 10:10:51,851 - distributed.scheduler - INFO - Close client connection: Client-9933b2d6-c854-11f0-8001-f2cd832f225c

2025-11-23 10:10:51,850 - distributed.scheduler - INFO - Remove client Client-9933b2d6-c854-11f0-8001-f2cd832f225c

2025-11-23 10:10:51,848 - distributed.scheduler - INFO - Remove client Client-9933b2d6-c854-11f0-8001-f2cd832f225c

2025-11-23 10:10:12,187 - distributed.scheduler - INFO - Receive client connection: Client-9933b2d6-c854-11f0-8001-f2cd832f225c

2025-11-23 10:01:22,006 - distributed.scheduler - INFO - Close client connection: Client-32b23205-c853-11f0-8001-f2cd832f225c

2025-11-23 10:01:22,004 - distributed.scheduler - INFO - Remove client Client-32b23205-c853-11f0-8001-f2cd832f225c

2025-11-23 10:01:22,002 - distributed.scheduler - INFO - Remove client Client-32b23205-c853-11f0-8001-f2cd832f225c

2025-11-23 10:00:10,714 - distributed.scheduler - INFO - Receive client connection: Client-32b23205-c853-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,597 - distributed.scheduler - INFO - Close client connection: Client-bc38bbe3-c846-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,596 - distributed.scheduler - INFO - Close client connection: Client-654b1896-c849-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,596 - distributed.scheduler - INFO - Close client connection: Client-22b03dbc-c848-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,596 - distributed.scheduler - INFO - Close client connection: Client-cbc76cc7-c84a-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,596 - distributed.scheduler - INFO - Close client connection: Client-324ec9e0-c84c-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,595 - distributed.scheduler - INFO - Close client connection: Client-98bef158-c84d-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,595 - distributed.scheduler - INFO - Close client connection: Client-ff35d780-c84e-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,595 - distributed.scheduler - INFO - Close client connection: Client-65b6e14b-c850-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,592 - distributed.scheduler - INFO - Remove client Client-bc38bbe3-c846-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,591 - distributed.scheduler - INFO - Remove client Client-654b1896-c849-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,590 - distributed.scheduler - INFO - Remove client Client-22b03dbc-c848-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,588 - distributed.scheduler - INFO - Remove client Client-cbc76cc7-c84a-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,587 - distributed.scheduler - INFO - Remove client Client-324ec9e0-c84c-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,586 - distributed.scheduler - INFO - Remove client Client-98bef158-c84d-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,585 - distributed.scheduler - INFO - Remove client Client-ff35d780-c84e-11f0-8001-f2cd832f225c

2025-11-23 09:50:48,584 - distributed.scheduler - INFO - Remove client Client-65b6e14b-c850-11f0-8001-f2cd832f225c

2025-11-23 09:50:47,786 - distributed.scheduler - INFO - Close client connection: Client-cc346a04-c851-11f0-8001-f2cd832f225c

2025-11-23 09:50:47,785 - distributed.scheduler - INFO - Remove client Client-cc346a04-c851-11f0-8001-f2cd832f225c

2025-11-23 09:50:47,783 - distributed.scheduler - INFO - Remove client Client-cc346a04-c851-11f0-8001-f2cd832f225c

2025-11-23 09:50:09,261 - distributed.scheduler - INFO - Receive client connection: Client-cc346a04-c851-11f0-8001-f2cd832f225c

2025-11-23 09:40:07,810 - distributed.scheduler - INFO - Receive client connection: Client-65b6e14b-c850-11f0-8001-f2cd832f225c

2025-11-23 09:30:06,340 - distributed.scheduler - INFO - Receive client connection: Client-ff35d780-c84e-11f0-8001-f2cd832f225c

2025-11-23 09:20:04,937 - distributed.scheduler - INFO - Receive client connection: Client-98bef158-c84d-11f0-8001-f2cd832f225c

2025-11-23 09:10:03,579 - distributed.scheduler - INFO - Receive client connection: Client-324ec9e0-c84c-11f0-8001-f2cd832f225c

2025-11-23 09:00:06,996 - distributed.scheduler - INFO - Receive client connection: Client-worker-ceb2e01c-c84a-11f0-8204-32a59f79d364

2025-11-23 09:00:02,068 - distributed.scheduler - INFO - Receive client connection: Client-cbc76cc7-c84a-11f0-8001-f2cd832f225c

2025-11-23 08:50:00,654 - distributed.scheduler - INFO - Receive client connection: Client-654b1896-c849-11f0-8001-f2cd832f225c

2025-11-23 08:40:59,384 - distributed.scheduler - INFO - Receive client connection: Client-22b03dbc-c848-11f0-8001-f2cd832f225c

2025-11-23 08:31:02,329 - distributed.scheduler - INFO - Receive client connection: Client-worker-bec57e3c-c846-11f0-8348-b64bf7bee50e

2025-11-23 08:30:57,985 - distributed.scheduler - INFO - Receive client connection: Client-bc38bbe3-c846-11f0-8001-f2cd832f225c

2025-11-23 08:21:35,767 - distributed.scheduler - INFO - Close client connection: Client-55c88454-c845-11f0-8001-f2cd832f225c

2025-11-23 08:21:35,766 - distributed.scheduler - INFO - Remove client Client-55c88454-c845-11f0-8001-f2cd832f225c

2025-11-23 08:21:35,765 - distributed.scheduler - INFO - Remove client Client-55c88454-c845-11f0-8001-f2cd832f225c

2025-11-23 08:20:56,628 - distributed.scheduler - INFO - Receive client connection: Client-55c88454-c845-11f0-8001-f2cd832f225c

2025-11-23 08:11:30,649 - distributed.scheduler - INFO - Close client connection: Client-ef4c649a-c843-11f0-8001-f2cd832f225c

2025-11-23 08:11:30,647 - distributed.scheduler - INFO - Remove client Client-ef4c649a-c843-11f0-8001-f2cd832f225c

2025-11-23 08:11:30,645 - distributed.scheduler - INFO - Remove client Client-ef4c649a-c843-11f0-8001-f2cd832f225c

2025-11-23 08:10:55,185 - distributed.scheduler - INFO - Receive client connection: Client-ef4c649a-c843-11f0-8001-f2cd832f225c

2025-11-23 08:04:19,898 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:43581

2025-11-23 08:04:19,897 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:43581', status: closed, memory: 0, processing: 0>

2025-11-23 08:04:19,650 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:45707

2025-11-23 08:04:19,648 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:45707', status: closed, memory: 0, processing: 0>

2025-11-23 08:04:18,869 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:38821', status: closed, memory: 0, processing: 0>

2025-11-23 08:04:18,801 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:43547', status: closed, memory: 0, processing: 0>

2025-11-23 08:04:08,981 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:38821

2025-11-23 08:04:08,979 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:38821', status: closed, memory: 0, processing: 0>

2025-11-23 08:04:07,970 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:44899', status: closed, memory: 0, processing: 0>

2025-11-23 08:04:04,746 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:43547

2025-11-23 08:04:04,744 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:43547', status: closed, memory: 0, processing: 0>

2025-11-23 08:04:03,624 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:37321', status: closed, memory: 0, processing: 0>

2025-11-23 08:03:52,493 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:38097 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:38097 after 30 s

2025-11-23 08:03:52,493 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:36091 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:36091 after 30 s

2025-11-23 08:03:52,487 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36091 after 30 s')

2025-11-23 08:03:52,422 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.191:43305 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36091 after 30 s')

2025-11-23 08:03:52,410 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:39867 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36091 after 30 s')

2025-11-23 08:03:52,362 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:32841 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:38097 after 30 s')

2025-11-23 08:03:52,350 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:45407 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:38097 after 30 s')

2025-11-23 08:03:52,349 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37207 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36091 after 30 s')

2025-11-23 08:03:52,349 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:33551 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36091 after 30 s')

2025-11-23 08:03:52,348 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:34191 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36091 after 30 s')

2025-11-23 08:03:52,346 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:45375 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36091 after 30 s')

2025-11-23 08:03:52,333 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:43549 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:36091 after 30 s')

2025-11-23 08:03:23,167 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:37321

2025-11-23 08:03:23,166 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:37321', status: closed, memory: 0, processing: 0>

2025-11-23 08:03:23,154 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:44899

2025-11-23 08:03:23,151 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:44899', status: closed, memory: 0, processing: 0>

2025-11-23 08:03:22,202 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:38097', status: closed, memory: 0, processing: 0>

2025-11-23 08:03:22,170 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:36091', status: closed, memory: 0, processing: 0>

2025-11-23 08:02:56,966 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:39725 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:39725 after 30 s

2025-11-23 08:02:56,966 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:37183 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:37183 after 30 s

2025-11-23 08:02:56,964 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:37183 after 30 s')

2025-11-23 08:02:56,929 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:39867 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:39725 after 30 s')

2025-11-23 08:02:56,868 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:45375 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:39725 after 30 s')

2025-11-23 08:02:56,825 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:45407 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:37183 after 30 s')

2025-11-23 08:02:56,824 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:35391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:39725 after 30 s')

2025-11-23 08:02:56,823 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:32841 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:37183 after 30 s')

2025-11-23 08:02:56,810 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:33551 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:37183 after 30 s')

2025-11-23 08:02:56,799 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:34887 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:37183 after 30 s')

2025-11-23 08:02:27,766 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:38097

2025-11-23 08:02:27,765 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:38097', status: closed, memory: 0, processing: 0>

2025-11-23 08:02:27,609 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:36091

2025-11-23 08:02:27,607 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:36091', status: closed, memory: 0, processing: 0>

2025-11-23 08:02:26,722 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:39725', status: closed, memory: 0, processing: 0>

2025-11-23 08:02:26,577 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:37183', status: closed, memory: 0, processing: 0>

2025-11-23 08:02:19,844 - distributed.scheduler - INFO - Close client connection: Client-88d4018f-c842-11f0-8001-f2cd832f225c

2025-11-23 08:02:19,843 - distributed.scheduler - INFO - Remove client Client-88d4018f-c842-11f0-8001-f2cd832f225c

2025-11-23 08:02:19,842 - distributed.scheduler - INFO - Remove client Client-88d4018f-c842-11f0-8001-f2cd832f225c

2025-11-23 08:02:08,598 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:36701 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:36701 after 30 s

2025-11-23 08:02:08,598 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43553 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:43553 after 30 s

2025-11-23 08:02:07,820 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:36701 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:36701 after 30 s

2025-11-23 08:02:07,820 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43553 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:43553 after 30 s

2025-11-23 08:02:07,723 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:43553 after 30 s')

2025-11-23 08:02:07,400 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:32841 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:43553 after 30 s')

2025-11-23 08:02:06,682 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:36701 failed: OSError: Timed out trying to connect to tcp://172.21.25.97:36701 after 30 s

2025-11-23 08:02:06,682 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43553 failed: OSError: Timed out trying to connect to tcp://172.21.25.101:43553 after 30 s

2025-11-23 08:02:06,450 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:39725 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:43553 after 30 s')

2025-11-23 08:02:06,173 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:39725 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:43553 after 30 s')

2025-11-23 08:02:05,998 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:36701 after 30 s')

2025-11-23 08:02:05,943 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:43549 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:43553 after 30 s')

2025-11-23 08:02:05,879 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37207 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:36701 after 30 s')

2025-11-23 08:02:05,444 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:34191 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:43553 after 30 s')

2025-11-23 08:02:05,413 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:43553 after 30 s')

2025-11-23 08:02:05,273 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:33551 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:36701 after 30 s')

2025-11-23 08:02:05,166 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:43549 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:43553 after 30 s')

2025-11-23 08:02:05,031 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37207 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.97:36701 after 30 s')

2025-11-23 08:02:04,957 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:34191 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.101:43553 after 30 s')

2025-11-23 08:01:35,705 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.101:37183

2025-11-23 08:01:35,703 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.101:37183', status: closed, memory: 0, processing: 0>

2025-11-23 08:01:34,663 - distributed.scheduler - INFO - Close client connection: Client-worker-bc4cf27b-c81c-11f0-82aa-f2a719d69604

2025-11-23 08:01:34,641 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.101:43553', status: closed, memory: 0, processing: 0>

2025-11-23 08:01:34,641 - distributed.scheduler - INFO - Remove client Client-worker-bc4cf27b-c81c-11f0-82aa-f2a719d69604

2025-11-23 08:01:33,938 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.97:39725

2025-11-23 08:01:33,937 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.97:39725', status: closed, memory: 0, processing: 0>

2025-11-23 08:01:32,918 - distributed.scheduler - INFO - Close client connection: Client-worker-d7114c62-c575-11f0-8074-32a59f79d364

2025-11-23 08:01:32,915 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.97:36701', status: closed, memory: 0, processing: 0>

2025-11-23 08:01:32,912 - distributed.scheduler - INFO - Remove client Client-worker-d7114c62-c575-11f0-8074-32a59f79d364

2025-11-23 08:00:53,769 - distributed.scheduler - INFO - Receive client connection: Client-88d4018f-c842-11f0-8001-f2cd832f225c

2025-11-23 07:51:34,898 - distributed.scheduler - INFO - Close client connection: Client-225150e7-c841-11f0-8001-f2cd832f225c

2025-11-23 07:51:34,896 - distributed.scheduler - INFO - Remove client Client-225150e7-c841-11f0-8001-f2cd832f225c

2025-11-23 07:51:34,894 - distributed.scheduler - INFO - Remove client Client-225150e7-c841-11f0-8001-f2cd832f225c

2025-11-23 07:50:52,294 - distributed.scheduler - INFO - Receive client connection: Client-225150e7-c841-11f0-8001-f2cd832f225c

2025-11-23 07:41:34,185 - distributed.scheduler - INFO - Close client connection: Client-bbd153fe-c83f-11f0-8001-f2cd832f225c

2025-11-23 07:41:34,184 - distributed.scheduler - INFO - Remove client Client-bbd153fe-c83f-11f0-8001-f2cd832f225c

2025-11-23 07:41:34,183 - distributed.scheduler - INFO - Remove client Client-bbd153fe-c83f-11f0-8001-f2cd832f225c

2025-11-23 07:40:50,825 - distributed.scheduler - INFO - Receive client connection: Client-bbd153fe-c83f-11f0-8001-f2cd832f225c

2025-11-23 07:33:16,805 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:33623 failed: OSError: Timed out trying to connect to tcp://172.21.25.6:33623 after 30 s

2025-11-23 07:33:16,798 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:39867 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.6:33623 after 30 s')

2025-11-23 07:33:16,702 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.24:43549 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.6:33623 after 30 s')

2025-11-23 07:33:16,700 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:43129 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.6:33623 after 30 s')

2025-11-23 07:33:16,696 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:45407 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.6:33623 after 30 s')

2025-11-23 07:33:16,692 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.199:33551 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.6:33623 after 30 s')

2025-11-23 07:33:16,690 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.199:45375 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.6:33623 after 30 s')

2025-11-23 07:33:16,689 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37207 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.6:33623 after 30 s')

2025-11-23 07:33:16,650 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.6:33623 after 30 s')

2025-11-23 07:33:16,639 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:35391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.25.6:33623 after 30 s')

2025-11-23 07:32:47,706 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:34191

2025-11-23 07:32:47,704 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:34191', status: closed, memory: 0, processing: 0>

2025-11-23 07:32:46,533 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:33623', status: closed, memory: 0, processing: 0>

2025-11-23 07:32:36,045 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:33623

2025-11-23 07:32:36,043 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:33623', status: closed, memory: 0, processing: 0>

2025-11-23 07:32:35,153 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:42535', status: closed, memory: 0, processing: 0>

2025-11-23 07:32:25,677 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:42535

2025-11-23 07:32:25,675 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:42535', status: closed, memory: 0, processing: 0>

2025-11-23 07:32:24,544 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:45127', status: closed, memory: 0, processing: 0>

2025-11-23 07:32:18,656 - distributed.scheduler - INFO - Close client connection: Client-555a2b7e-c83e-11f0-8001-f2cd832f225c

2025-11-23 07:32:18,655 - distributed.scheduler - INFO - Remove client Client-555a2b7e-c83e-11f0-8001-f2cd832f225c

2025-11-23 07:32:18,655 - distributed.scheduler - INFO - Remove client Client-555a2b7e-c83e-11f0-8001-f2cd832f225c

2025-11-23 07:31:53,256 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:45127

2025-11-23 07:31:53,255 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:45127', status: closed, memory: 0, processing: 0>

2025-11-23 07:31:52,147 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:44699', status: closed, memory: 0, processing: 0>

2025-11-23 07:31:27,188 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.25.6:44699

2025-11-23 07:31:27,187 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.25.6:44699', status: closed, memory: 0, processing: 0>

2025-11-23 07:31:26,981 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43553 failed: Exception: KeyError('9020dacc266125ea93d46ca3a906a4e9')

2025-11-23 07:31:26,974 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43553 failed: Exception: KeyError('d4670e4b40a88508a4add5be25c5be15')

2025-11-23 07:31:26,900 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:35391 failed: Exception: KeyError('9020dacc266125ea93d46ca3a906a4e9')

2025-11-23 07:31:26,892 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:35391 failed: Exception: KeyError('d4670e4b40a88508a4add5be25c5be15')

2025-11-23 07:31:26,863 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: KeyError('9020dacc266125ea93d46ca3a906a4e9')

2025-11-23 07:31:26,858 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: RuntimeError('Worker tcp://172.21.25.6:35199 left during active shuffle d4670e4b40a88508a4add5be25c5be15')

2025-11-23 07:31:26,190 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:35199 failed: OSError: Timed out during handshake while connecting to tcp://172.21.25.6:35199 after 30 s

2025-11-23 07:31:26,190 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:35199 failed: OSError: Timed out during handshake while connecting to tcp://172.21.25.6:35199 after 30 s

2025-11-23 07:31:26,190 - distributed.scheduler - INFO - Close client connection: Client-worker-30252c97-c7b8-11f0-82b7-b64bf7bee50e

2025-11-23 07:31:26,178 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.25.6:35199', status: closed, memory: 0, processing: 0>

2025-11-23 07:31:26,177 - distributed.scheduler - INFO - Remove client Client-worker-30252c97-c7b8-11f0-82b7-b64bf7bee50e

2025-11-23 07:30:54,758 - distributed.scheduler - INFO - Receive client connection: Client-worker-58843e17-c83e-11f0-8284-4ec0916b0c7e

2025-11-23 07:30:49,431 - distributed.scheduler - INFO - Receive client connection: Client-555a2b7e-c83e-11f0-8001-f2cd832f225c

2025-11-23 07:21:29,373 - distributed.scheduler - INFO - Close client connection: Client-eed5b524-c83c-11f0-8001-f2cd832f225c

2025-11-23 07:21:29,372 - distributed.scheduler - INFO - Remove client Client-eed5b524-c83c-11f0-8001-f2cd832f225c

2025-11-23 07:21:29,370 - distributed.scheduler - INFO - Remove client Client-eed5b524-c83c-11f0-8001-f2cd832f225c

2025-11-23 07:20:47,935 - distributed.scheduler - INFO - Receive client connection: Client-eed5b524-c83c-11f0-8001-f2cd832f225c

2025-11-23 07:11:28,173 - distributed.scheduler - INFO - Close client connection: Client-88557535-c83b-11f0-8001-f2cd832f225c

2025-11-23 07:11:28,172 - distributed.scheduler - INFO - Remove client Client-88557535-c83b-11f0-8001-f2cd832f225c

2025-11-23 07:11:28,170 - distributed.scheduler - INFO - Remove client Client-88557535-c83b-11f0-8001-f2cd832f225c

2025-11-23 07:10:46,475 - distributed.scheduler - INFO - Receive client connection: Client-88557535-c83b-11f0-8001-f2cd832f225c

2025-11-23 07:04:39,544 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:41075 failed: OSError: Timed out trying to connect to tcp://172.21.159.248:41075 after 30 s

2025-11-23 07:04:39,522 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:39867 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:41075 after 30 s')

2025-11-23 07:04:39,464 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.252:45407 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:41075 after 30 s')

2025-11-23 07:04:39,447 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:35199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:41075 after 30 s')

2025-11-23 07:04:39,447 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.233:34887 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:41075 after 30 s')

2025-11-23 07:04:39,446 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.101:43553 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:41075 after 30 s')

2025-11-23 07:04:39,446 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.165:41631 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:41075 after 30 s')

2025-11-23 07:04:39,443 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.210:32841 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:41075 after 30 s')

2025-11-23 07:04:39,412 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.197:43129 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:41075 after 30 s')

2025-11-23 07:04:39,401 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.102:41571 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:41075 after 30 s')

2025-11-23 07:04:39,390 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.104:35391 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:41075 after 30 s')

2025-11-23 07:04:10,158 - distributed.scheduler - INFO - Starting worker compute stream, tcp://172.21.159.248:37207

2025-11-23 07:04:10,155 - distributed.scheduler - INFO - Register worker <WorkerState 'tcp://172.21.159.248:37207', status: closed, memory: 0, processing: 0>

2025-11-23 07:04:09,284 - distributed.scheduler - INFO - Remove worker <WorkerState 'tcp://172.21.159.248:41075', status: closed, memory: 0, processing: 0>

2025-11-23 07:03:58,885 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.97:36701 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:37797 after 30 s')

2025-11-23 07:03:58,164 - distributed.scheduler - ERROR - broadcast to tcp://172.21.159.248:37797 failed: OSError: Timed out trying to connect to tcp://172.21.159.248:37797 after 30 s

2025-11-23 07:03:58,137 - distributed.scheduler - ERROR - broadcast to tcp://172.21.12.150:39867 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:37797 after 30 s')

2025-11-23 07:03:58,046 - distributed.scheduler - ERROR - broadcast to tcp://172.21.25.6:35199 failed: Exception: OSError('Timed out trying to connect to tcp://172.21.159.248:37797 after 30 s')