Yandexはクロール遅延を無視しますか？
私はYandexボットを制御下に置くことを試み、それをブロックするところです。
Robots.txtを2か月調整した後も、サーバーが複数のIPから攻撃を受け続けています。
User-Agent: *
Crawl-delay: 4
User-agent: Yandex
Crawl-delay: 6
1週間の平均遅延は7.6秒（秒/ access_log行の量）です。しかし、それでも、さまざまなIPを使用して数分間ハンマーし続けるピークが毎日あります。
ここと例：276秒（〜1.2秒）で12個のIPからの336ページのリクエスト（ファイル/スクリプトなどは数えません）
5.255.253.3 - - [27/Apr/2019:10:43:14 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:15 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:43:16 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:16 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:17 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:43:18 +0200] "GET …
37.9.113.7 - - [27/Apr/2019:10:43:18 +0200] "GET …
37.9.113.156 - - [27/Apr/2019:10:43:18 +0200] "GET …
141.8.132.26 - - [27/Apr/2019:10:43:19 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:43:19 +0200] "GET …
95.108.181.94 - - [27/Apr/2019:10:43:19 +0200] "GET …
87.250.224.50 - - [27/Apr/2019:10:43:20 +0200] "GET …
87.250.224.55 - - [27/Apr/2019:10:43:21 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:22 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:43:23 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:24 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:43:24 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:25 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:28 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:28 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:43:29 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:31 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:43 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:44 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:43:45 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:46 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:43:47 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:48 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:43:48 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:49 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:43:50 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:51 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:43:52 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:53 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:43:54 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:54 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:43:55 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:56 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:43:56 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:43:57 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:43:58 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:43:59 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:43:59 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:00 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:01 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:44:01 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:02 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:03 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:04 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:04 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:05 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:44:06 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:06 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:07 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:08 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:10 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:10 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:10 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:11 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:12 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:44:12 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:14 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:15 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:16 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:17 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:17 +0200] "GET …
141.8.142.118 - - [27/Apr/2019:10:44:18 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:19 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:19 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:21 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:21 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:21 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:44:22 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:23 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:24 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:24 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:25 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:25 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:44:25 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:26 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:27 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:44:27 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:28 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:29 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:30 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:31 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:32 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:33 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:34 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:35 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:36 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:37 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:37 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:44:37 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:38 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:38 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:44:39 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:39 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:40 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:41 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:42 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:42 +0200] "GET …
141.8.142.118 - - [27/Apr/2019:10:44:42 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:43 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:44:43 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:44 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:45 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:46 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:47 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:48 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:49 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:50 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:51 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:51 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:52 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:53 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:55 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:55 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:56 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:44:57 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:44:59 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:00 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:00 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:02 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:03 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:05 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:05 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:05 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:07 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:07 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:07 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:08 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:45:09 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:10 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:10 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:11 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:12 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:13 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:13 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:14 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:15 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:15 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:16 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:17 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:45:17 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:17 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:19 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:20 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:22 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:23 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:23 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:24 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:25 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:26 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:26 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:27 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:29 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:29 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:30 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:31 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:32 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:32 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:45:33 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:33 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:33 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:45:34 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:35 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:36 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:37 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:37 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:39 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:45:39 +0200] "GET …
87.250.224.119 - - [27/Apr/2019:10:45:39 +0200] "GET …
141.8.132.26 - - [27/Apr/2019:10:45:40 +0200] "GET …
141.8.142.118 - - [27/Apr/2019:10:45:40 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:45:40 +0200] "GET …
87.250.224.119 - - [27/Apr/2019:10:45:41 +0200] "GET …
141.8.132.26 - - [27/Apr/2019:10:45:41 +0200] "GET …
141.8.142.118 - - [27/Apr/2019:10:45:41 +0200] "GET …
213.180.203.40 - - [27/Apr/2019:10:45:42 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:43 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:44 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:44 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:45:45 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:46 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:48 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:49 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:49 +0200] "GET … "Mozilla/5.0 (iPhone; CPU iPhone OS 8_1 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12B411 Safari/600.1.4 (compatible; YandexMobileBot/3.0; +http://yandex.com/bots)"
141.8.132.29 - - [27/Apr/2019:10:45:51 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:45:51 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:51 +0200] "GET …
141.8.132.26 - - [27/Apr/2019:10:45:52 +0200] "GET …
141.8.142.118 - - [27/Apr/2019:10:45:52 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:53 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:53 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:45:54 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:54 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:54 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:56 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:56 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:57 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:45:57 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:45:59 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:45:59 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:02 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:02 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:03 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:05 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:05 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:06 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:46:07 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:46:08 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:08 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:46:08 +0200] "GET …
87.250.224.119 - - [27/Apr/2019:10:46:09 +0200] "GET …
141.8.132.26 - - [27/Apr/2019:10:46:09 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:46:10 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:10 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:10 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:12 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:13 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:14 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:15 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:17 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:17 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:18 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:19 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:20 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:22 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:22 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:24 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:24 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:24 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:25 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:27 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:27 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:28 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:46:28 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:29 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:29 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:46:30 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:32 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:32 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:32 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:33 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:34 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:35 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:35 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:35 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:46:36 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:36 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:46:36 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:46:37 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:38 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:38 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:46:39 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:40 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:41 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:42 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:43 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:44 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:46 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:48 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:48 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:48 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:50 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:50 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:50 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:51 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:52 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:52 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:53 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:53 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:46:54 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:55 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:55 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:46:55 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:56 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:56 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:46:56 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:46:58 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:46:59 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:01 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:02 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:03 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:04 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:05 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:06 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:07 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:08 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:10 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:11 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:11 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:12 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:47:13 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:13 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:14 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:15 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:15 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:16 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:17 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:18 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:19 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:19 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:20 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:20 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:47:21 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:22 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:22 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:47:23 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:23 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:23 +0200] "GET …
87.250.224.119 - - [27/Apr/2019:10:47:24 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:26 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:27 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:27 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:28 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:28 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:47:29 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:29 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:30 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:31 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:33 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:34 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:34 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:35 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:36 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:47:37 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:37 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:37 +0200] "GET …
213.180.203.61 - - [27/Apr/2019:10:47:38 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:39 +0200] "GET …
141.8.132.29 - - [27/Apr/2019:10:47:39 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:48 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:50 +0200] "GET …
5.255.253.3 - - [27/Apr/2019:10:47:50 +0200] "GET …
これは、18個のIP（〜1.57秒）による1547ページのリクエストによる40分の「訪問」の抜粋にすぎません。
GoogleとBingボットが同時にアクティブになると、CPU使用率が高くなります。
Yandexは私のCrawl-delayルールを無視していますか？上記のようにサーバーを傷つけないようにするにはどうすればよいですか？
YandexがCrawl-delayディレクティブに従っていないと思われる場合は、Yandex Webmastersにサインアップして、ドメインを検証してください。検証が完了したら、Indexing-> Crawl rateでクロール速度を手動で設定できます
Yandex Webmastersに関するこの記事を読むことを検討してください Yandex Webmaster Toolsのガイド